Migrating storage from 5.10.0 to 5.14.5

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Migrating storage from 5.10.0 to 5.14.5

christopher
Good Day,

I've been working on migrating servers and upgrading from 5.10.0
to 5.14.5.
We have a number of scheduled tasks and I want to preserve those along
with other items in the queue. I tried moving the files in kahadb to the
new server (after shutting down both servers), but have run into
multiple errors which prevent the new server from starting.
I am wondering what the appropriate way to migrate storage is. Should I
attempt read all of the messages from the old queue and write them into
the new queue? Has anyone else done this before?  It seems like this
would be a useful utility.

Thanks,

Chris
Reply | Threaded
Open this post in threaded view
|

Re: Migrating storage from 5.10.0 to 5.14.5

christopher.l.shannon
What kind of errors are you getting?  If it's related to data marshalling
then it could be because there was an Openwire upgrade (Openwire is the
format the messages are stored in) done in the KahaDB store between those
versions which could be causing your issue. I think there was also some
scheduler work done so not sure off the top of my head if that is the
problem without seeing the stack trace.

If you are simply copying over the journal files and trying to rebuild the
KahaDB index then it will fail as it will pick Openwire version 11 instead
of 6. You set the version manually using the storeOpenWireVersion
property.  See http://activemq.apache.org/kahadb.html.  If the index file
still exists (db.data) then KahaDB should be able to detect that it is an
old store and fall back automatically to the write openwire version.



On Fri, May 19, 2017 at 2:32 PM, <[hidden email]> wrote:

> Good Day,
>
> I've been working on migrating servers and upgrading from 5.10.0
> to 5.14.5.
> We have a number of scheduled tasks and I want to preserve those along
> with other items in the queue. I tried moving the files in kahadb to the
> new server (after shutting down both servers), but have run into
> multiple errors which prevent the new server from starting.
> I am wondering what the appropriate way to migrate storage is. Should I
> attempt read all of the messages from the old queue and write them into
> the new queue? Has anyone else done this before?  It seems like this
> would be a useful utility.
>
> Thanks,
>
> Chris
>
Reply | Threaded
Open this post in threaded view
|

Re: Migrating storage from 5.10.0 to 5.14.5

Tim Bain
What Chris described is how to load the existing messages into your new
broker. That has always seemed clunky to me because there is no way to
store messages in multiple OpenWire formats (which means your new broker's
store is stuck forever on the old broker's OpenWire version), so the option
that always sounded better to me is to stand up the new broker empty and
then attach the old broker in a network of brokers configuration. Clients
would connect to the new broker, and messages would flow from the old
broker to the new one and then to the consumers.

On May 24, 2017 2:45 PM, "Christopher Shannon" <
[hidden email]> wrote:

> What kind of errors are you getting?  If it's related to data marshalling
> then it could be because there was an Openwire upgrade (Openwire is the
> format the messages are stored in) done in the KahaDB store between those
> versions which could be causing your issue. I think there was also some
> scheduler work done so not sure off the top of my head if that is the
> problem without seeing the stack trace.
>
> If you are simply copying over the journal files and trying to rebuild the
> KahaDB index then it will fail as it will pick Openwire version 11 instead
> of 6. You set the version manually using the storeOpenWireVersion
> property.  See http://activemq.apache.org/kahadb.html.  If the index file
> still exists (db.data) then KahaDB should be able to detect that it is an
> old store and fall back automatically to the write openwire version.
>
>
>
> On Fri, May 19, 2017 at 2:32 PM, <[hidden email]> wrote:
>
> > Good Day,
> >
> > I've been working on migrating servers and upgrading from 5.10.0
> > to 5.14.5.
> > We have a number of scheduled tasks and I want to preserve those along
> > with other items in the queue. I tried moving the files in kahadb to the
> > new server (after shutting down both servers), but have run into
> > multiple errors which prevent the new server from starting.
> > I am wondering what the appropriate way to migrate storage is. Should I
> > attempt read all of the messages from the old queue and write them into
> > the new queue? Has anyone else done this before?  It seems like this
> > would be a useful utility.
> >
> > Thanks,
> >
> > Chris
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Migrating storage from 5.10.0 to 5.14.5

christopher
The issue we have is that we have a number of scheduled tasks in the old
queue. What I ended up doing was writing a program to read out of the
old queue and then manually enqueue the tasks in the new queue. It was a
little awkward, but it got the job done.

On Mon, Jun 5, 2017, at 10:04 PM, Tim Bain wrote:

> What Chris described is how to load the existing messages into
> your new> broker. That has always seemed clunky to me because there is no way to> store messages in multiple OpenWire formats (which means your new
> broker's
> store is stuck forever on the old broker's OpenWire version), so the
> option
> that always sounded better to me is to stand up the new broker
> empty and> then attach the old broker in a network of brokers
> configuration. Clients> would connect to the new broker, and messages would flow from the old> broker to the new one and then to the consumers.
>
> On May 24, 2017 2:45 PM, "Christopher Shannon" <
> [hidden email]> wrote:
>
>> What kind of errors are you getting?  If it's related to data
>> marshalling>> then it could be because there was an Openwire upgrade
>> (Openwire is the>> format the messages are stored in) done in the KahaDB store
>> between those>> versions which could be causing your issue. I think there was
>> also some>> scheduler work done so not sure off the top of my head if that is the>> problem without seeing the stack trace.
>>
>> If you are simply copying over the journal files and trying to
>> rebuild the>> KahaDB index then it will fail as it will pick Openwire version 11
>> instead>> of 6. You set the version manually using the storeOpenWireVersion
>> property.  See http://activemq.apache.org/kahadb.html.  If the
>> index file>> still exists (db.data) then KahaDB should be able to detect that
>> it is an>> old store and fall back automatically to the write openwire version.>>
>>
>>
>> On Fri, May 19, 2017 at 2:32 PM, <[hidden email]> wrote:
>>
>>> Good Day,
>>>
>>> I've been working on migrating servers and upgrading from 5.10.0
>>> to 5.14.5.
>>> We have a number of scheduled tasks and I want to preserve
>>> those along>>> with other items in the queue. I tried moving the files in kahadb
>>> to the>>> new server (after shutting down both servers), but have run into
>>> multiple errors which prevent the new server from starting.
>>> I am wondering what the appropriate way to migrate storage is.
>>> Should I>>> attempt read all of the messages from the old queue and write
>>> them into>>> the new queue? Has anyone else done this before?  It seems like this>>> would be a useful utility.
>>>
>>> Thanks,
>>>
>>> Chris
>>>
>>