LevelDB in ActiveMQ 5.9.0

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

LevelDB in ActiveMQ 5.9.0

paulgale
Per Claus Ibsen's blog post:

"Over time level db is planned to be default out of the box persistent
store, instead of the current kahadb store. Though we are not there
yet, so kahadb is still the default store."

What does "we are not there yet" mean? What are the caveats regarding
LevelDB usage?

Copious detail is much appreciated.

Just wondering.

Thanks,
Paul
Reply | Threaded
Open this post in threaded view
|

Re: LevelDB in ActiveMQ 5.9.0

gtully
of the ~3k unit tests - many use the default store and we have not
switched the default store. Until we do, and work through any issues
we won't know for sure. We plan to do that before we make it the
default.
The only known caveat atm is priority support, levelDb does not
respect JMS priority in the same way as kahaDB or JDBC does.

On 22 October 2013 15:25, Paul Gale <[hidden email]> wrote:

> Per Claus Ibsen's blog post:
>
> "Over time level db is planned to be default out of the box persistent
> store, instead of the current kahadb store. Though we are not there
> yet, so kahadb is still the default store."
>
> What does "we are not there yet" mean? What are the caveats regarding
> LevelDB usage?
>
> Copious detail is much appreciated.
>
> Just wondering.
>
> Thanks,
> Paul



--
http://redhat.com
http://blog.garytully.com
Reply | Threaded
Open this post in threaded view
|

Re: LevelDB in ActiveMQ 5.9.0

Claus Ibsen
Does level db have support for that scheduled persistent store?
http://activemq.apache.org/delay-and-schedule-message-delivery.html

Or maybe I am mistaking or remembering wrong that the scheduler has a
separate store from the regular store for its scheduled messages.



On Tue, Oct 22, 2013 at 5:00 PM, Gary Tully <[hidden email]> wrote:

> of the ~3k unit tests - many use the default store and we have not
> switched the default store. Until we do, and work through any issues
> we won't know for sure. We plan to do that before we make it the
> default.
> The only known caveat atm is priority support, levelDb does not
> respect JMS priority in the same way as kahaDB or JDBC does.
>
> On 22 October 2013 15:25, Paul Gale <[hidden email]> wrote:
>> Per Claus Ibsen's blog post:
>>
>> "Over time level db is planned to be default out of the box persistent
>> store, instead of the current kahadb store. Though we are not there
>> yet, so kahadb is still the default store."
>>
>> What does "we are not there yet" mean? What are the caveats regarding
>> LevelDB usage?
>>
>> Copious detail is much appreciated.
>>
>> Just wondering.
>>
>> Thanks,
>> Paul
>
>
>
> --
> http://redhat.com
> http://blog.garytully.com



--
Claus Ibsen
-----------------
Red Hat, Inc.
Email: [hidden email]
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Reply | Threaded
Open this post in threaded view
|

Re: LevelDB in ActiveMQ 5.9.0

ceposta
There is a separate store for scheduled messages.
KahaDB is implemented.

There is no impl for leveldb yet.




On Tue, Oct 22, 2013 at 9:09 AM, Claus Ibsen <[hidden email]> wrote:

> Does level db have support for that scheduled persistent store?
> http://activemq.apache.org/delay-and-schedule-message-delivery.html
>
> Or maybe I am mistaking or remembering wrong that the scheduler has a
> separate store from the regular store for its scheduled messages.
>
>
>
> On Tue, Oct 22, 2013 at 5:00 PM, Gary Tully <[hidden email]> wrote:
> > of the ~3k unit tests - many use the default store and we have not
> > switched the default store. Until we do, and work through any issues
> > we won't know for sure. We plan to do that before we make it the
> > default.
> > The only known caveat atm is priority support, levelDb does not
> > respect JMS priority in the same way as kahaDB or JDBC does.
> >
> > On 22 October 2013 15:25, Paul Gale <[hidden email]> wrote:
> >> Per Claus Ibsen's blog post:
> >>
> >> "Over time level db is planned to be default out of the box persistent
> >> store, instead of the current kahadb store. Though we are not there
> >> yet, so kahadb is still the default store."
> >>
> >> What does "we are not there yet" mean? What are the caveats regarding
> >> LevelDB usage?
> >>
> >> Copious detail is much appreciated.
> >>
> >> Just wondering.
> >>
> >> Thanks,
> >> Paul
> >
> >
> >
> > --
> > http://redhat.com
> > http://blog.garytully.com
>
>
>
> --
> Claus Ibsen
> -----------------
> Red Hat, Inc.
> Email: [hidden email]
> Twitter: davsclaus
> Blog: http://davsclaus.com
> Author of Camel in Action: http://www.manning.com/ibsen
>



--
*Christian Posta*
http://www.christianposta.com/blog
twitter: @christianposta
Reply | Threaded
Open this post in threaded view
|

Re: LevelDB in ActiveMQ 5.9.0

gtully
true but in fact, the scheduler has its own 'home grown' kahadb like
store - and really needs to be layered over the existing persistence
adapters - https://issues.apache.org/jira/browse/AMQ-3758 - on the
todo list for 5.10

On 22 October 2013 17:30, Christian Posta <[hidden email]> wrote:

> There is a separate store for scheduled messages.
> KahaDB is implemented.
>
> There is no impl for leveldb yet.
>
>
>
>
> On Tue, Oct 22, 2013 at 9:09 AM, Claus Ibsen <[hidden email]> wrote:
>
>> Does level db have support for that scheduled persistent store?
>> http://activemq.apache.org/delay-and-schedule-message-delivery.html
>>
>> Or maybe I am mistaking or remembering wrong that the scheduler has a
>> separate store from the regular store for its scheduled messages.
>>
>>
>>
>> On Tue, Oct 22, 2013 at 5:00 PM, Gary Tully <[hidden email]> wrote:
>> > of the ~3k unit tests - many use the default store and we have not
>> > switched the default store. Until we do, and work through any issues
>> > we won't know for sure. We plan to do that before we make it the
>> > default.
>> > The only known caveat atm is priority support, levelDb does not
>> > respect JMS priority in the same way as kahaDB or JDBC does.
>> >
>> > On 22 October 2013 15:25, Paul Gale <[hidden email]> wrote:
>> >> Per Claus Ibsen's blog post:
>> >>
>> >> "Over time level db is planned to be default out of the box persistent
>> >> store, instead of the current kahadb store. Though we are not there
>> >> yet, so kahadb is still the default store."
>> >>
>> >> What does "we are not there yet" mean? What are the caveats regarding
>> >> LevelDB usage?
>> >>
>> >> Copious detail is much appreciated.
>> >>
>> >> Just wondering.
>> >>
>> >> Thanks,
>> >> Paul
>> >
>> >
>> >
>> > --
>> > http://redhat.com
>> > http://blog.garytully.com
>>
>>
>>
>> --
>> Claus Ibsen
>> -----------------
>> Red Hat, Inc.
>> Email: [hidden email]
>> Twitter: davsclaus
>> Blog: http://davsclaus.com
>> Author of Camel in Action: http://www.manning.com/ibsen
>>
>
>
>
> --
> *Christian Posta*
> http://www.christianposta.com/blog
> twitter: @christianposta



--
http://redhat.com
http://blog.garytully.com
Reply | Threaded
Open this post in threaded view
|

Re: LevelDB in ActiveMQ 5.9.0

khandelwalanuj
leveDB also does not support multiple kahaDB instance (persistent adapters) similar to kahaDB.

With the Mulitple KahaDB persistence adapter, destination partitioning across journals is possible. but there is no support in levelDB for this.

Please correct me if I am wrong !!  


Thanks,
Anuj
Reply | Threaded
Open this post in threaded view
|

Re: LevelDB in ActiveMQ 5.9.0

Oleg Dulin
In reply to this post by gtully
Gary:

I just tried to use 5.9 with replicated levelDB and my test failed epically...

The specific problem I have is that after about 1700 messages the whole
thing slows down to a crawl. It doesn't seem to be the case if I am
using plain leveldb, but when using replication I come across problems.

Is there some configuration I am missing ? This is all there is to it:

 94         <persistenceAdapter>
 95             <replicatedLevelDB
 96                     directory="${activemq.data}"
 97                     replicas="3"
 98                     bind="tcp://192.168.7.107:3201"
 99                     zkAddress="127.0.0.1:2181"
100                     zkPassword="password"
101                     zkPath="/activemq/leveldb-stores"
102                     sync="quorum_mem"
103                     />
104         </persistenceAdapter>


On 2013-10-22 19:51:43 +0000, Gary Tully said:

> true but in fact, the scheduler has its own 'home grown' kahadb like
> store - and really needs to be layered over the existing persistence
> adapters - https://issues.apache.org/jira/browse/AMQ-3758 - on the
> todo list for 5.10
>
> On 22 October 2013 17:30, Christian Posta
> <[hidden email]> wrote:
>> There is a separate store for scheduled messages.
>> KahaDB is implemented.
>>
>> There is no impl for leveldb yet.
>>
>>
>>
>>
>> On Tue, Oct 22, 2013 at 9:09 AM, Claus Ibsen
>> <[hidden email]> wrote:
>>
>>> Does level db have support for that scheduled persistent store?
>>> http://activemq.apache.org/delay-and-schedule-message-delivery.html
>>>
>>> Or maybe I am mistaking or remembering wrong that the scheduler has a
>>> separate store from the regular store for its scheduled messages.
>>>
>>>
>>>
>>> On Tue, Oct 22, 2013 at 5:00 PM, Gary Tully
>>> <[hidden email]> wrote:
>>>> of the ~3k unit tests - many use the default store and we have not
>>>> switched the default store. Until we do, and work through any issues
>>>> we won't know for sure. We plan to do that before we make it the
>>>> default.
>>>> The only known caveat atm is priority support, levelDb does not
>>>> respect JMS priority in the same way as kahaDB or JDBC does.
>>>>
>>>> On 22 October 2013 15:25, Paul Gale
>>>> <[hidden email]> wrote:
>>>>> Per Claus Ibsen's blog post:
>>>>>
>>>>> "Over time level db is planned to be default out of the box persistent
>>>>> store, instead of the current kahadb store. Though we are not there
>>>>> yet, so kahadb is still the default store."
>>>>>
>>>>> What does "we are not there yet" mean? What are the caveats regarding
>>>>> LevelDB usage?
>>>>>
>>>>> Copious detail is much appreciated.
>>>>>
>>>>> Just wondering.
>>>>>
>>>>> Thanks,
>>>>> Paul
>>>>
>>>>
>>>>
>>>> --
>>>> http://redhat.com
>>>> http://blog.garytully.com
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> -----------------
>>> Red Hat, Inc.
>>> Email: [hidden email]
>>> Twitter: davsclaus
>>> Blog: http://davsclaus.com
>>> Author of Camel in Action: http://www.manning.com/ibsen
>>>
>>
>>
>>
>> --
>> *Christian Posta*
>> http://www.christianposta.com/blog
>> twitter: @christianposta


--
Regards,
Oleg Dulin
http://www.olegdulin.com


Reply | Threaded
Open this post in threaded view
|

Re: LevelDB in ActiveMQ 5.9.0

gtully
there has been a bunch of work in that area for 5.10 so my first
suggestion is to give a 5.10-SNAPSHOT a whirl. If that shows the same
behaviour we can get on the job of sorting out what the problem is.
A fresh snapshot was minted today -
http://repository.apache.org/content/repositories/snapshots/org/apache/activemq/apache-activemq/5.10-SNAPSHOT/apache-activemq-5.10-20140325.181704-38-bin.tar.gz

On 25 March 2014 17:57, Oleg Dulin <[hidden email]> wrote:

> Gary:
>
> I just tried to use 5.9 with replicated levelDB and my test failed
> epically...
>
> The specific problem I have is that after about 1700 messages the whole
> thing slows down to a crawl. It doesn't seem to be the case if I am
> using plain leveldb, but when using replication I come across problems.
>
> Is there some configuration I am missing ? This is all there is to it:
>
> 94         <persistenceAdapter>
> 95             <replicatedLevelDB
> 96                     directory="${activemq.data}"
> 97                     replicas="3"
> 98                     bind="tcp://192.168.7.107:3201"
> 99                     zkAddress="127.0.0.1:2181"
> 100                     zkPassword="password"
> 101                     zkPath="/activemq/leveldb-stores"
> 102                     sync="quorum_mem"
> 103                     />
> 104         </persistenceAdapter>
>
>
>
> On 2013-10-22 19:51:43 +0000, Gary Tully said:
>
>> true but in fact, the scheduler has its own 'home grown' kahadb like
>> store - and really needs to be layered over the existing persistence
>> adapters - https://issues.apache.org/jira/browse/AMQ-3758 - on the
>> todo list for 5.10
>>
>> On 22 October 2013 17:30, Christian Posta <[hidden email]>
>> wrote:
>>>
>>> There is a separate store for scheduled messages.
>>> KahaDB is implemented.
>>>
>>> There is no impl for leveldb yet.
>>>
>>>
>>>
>>>
>>> On Tue, Oct 22, 2013 at 9:09 AM, Claus Ibsen <[hidden email]>
>>> wrote:
>>>
>>>> Does level db have support for that scheduled persistent store?
>>>> http://activemq.apache.org/delay-and-schedule-message-delivery.html
>>>>
>>>> Or maybe I am mistaking or remembering wrong that the scheduler has a
>>>> separate store from the regular store for its scheduled messages.
>>>>
>>>>
>>>>
>>>> On Tue, Oct 22, 2013 at 5:00 PM, Gary Tully <[hidden email]>
>>>> wrote:
>>>>>
>>>>> of the ~3k unit tests - many use the default store and we have not
>>>>> switched the default store. Until we do, and work through any issues
>>>>> we won't know for sure. We plan to do that before we make it the
>>>>> default.
>>>>> The only known caveat atm is priority support, levelDb does not
>>>>> respect JMS priority in the same way as kahaDB or JDBC does.
>>>>>
>>>>> On 22 October 2013 15:25, Paul Gale <[hidden email]> wrote:
>>>>>>
>>>>>> Per Claus Ibsen's blog post:
>>>>>>
>>>>>> "Over time level db is planned to be default out of the box persistent
>>>>>> store, instead of the current kahadb store. Though we are not there
>>>>>> yet, so kahadb is still the default store."
>>>>>>
>>>>>> What does "we are not there yet" mean? What are the caveats regarding
>>>>>> LevelDB usage?
>>>>>>
>>>>>> Copious detail is much appreciated.
>>>>>>
>>>>>> Just wondering.
>>>>>>
>>>>>> Thanks,
>>>>>> Paul
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> http://redhat.com
>>>>> http://blog.garytully.com
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> -----------------
>>>> Red Hat, Inc.
>>>> Email: [hidden email]
>>>> Twitter: davsclaus
>>>> Blog: http://davsclaus.com
>>>> Author of Camel in Action: http://www.manning.com/ibsen
>>>>
>>>
>>>
>>>
>>> --
>>> *Christian Posta*
>>> http://www.christianposta.com/blog
>>> twitter: @christianposta
>
>
>
> --
> Regards,
> Oleg Dulin
> http://www.olegdulin.com
>
>



--
http://redhat.com
http://blog.garytully.com
Reply | Threaded
Open this post in threaded view
|

Re: LevelDB in ActiveMQ 5.9.0

Tom_Z
In reply to this post by gtully
gtully wrote
The only known caveat atm is priority support, levelDb does not
respect JMS priority in the same way as kahaDB or JDBC does.
Hi Gary,

can you please expain a little more? Not the same is not "not at all", so what are the results when using message priorities? Does it work in most cases or rarely?

I'm using ActiveMQ 5.8 und have LevelDB activated. Should I go back to KahaDB?

Thanks,
Tom
Reply | Threaded
Open this post in threaded view
|

Re: LevelDB in ActiveMQ 5.9.0

ceposta
On Mon, Mar 31, 2014 at 12:06 PM, Tom_Z <[hidden email]> wrote:
> gtully wrote
>> The only known caveat atm is priority support, levelDb does not
>> respect JMS priority in the same way as kahaDB or JDBC does.
>
> Hi Gary,
>
> can you please expain a little more? Not the same is not "not at all", so
> what are the results when using message priorities? Does it work in most
> cases or rarely?
I suppose it would work in most cases because priority support is also
implemented at the cursors and in the consumers. However, KahaDB has
explicit support for priority levels at the persistence level and
messages can be pulled off disk with some notion of priority. This is
the part missing in LevelDB AFAIK

>
> I'm using ActiveMQ 5.8 und have LevelDB activated. Should I go back to
> KahaDB?
>
> Thanks,
> Tom
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/LevelDB-in-ActiveMQ-5-9-0-tp4673129p4679750.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.



--
Christian Posta
http://www.christianposta.com/blog
twitter: @christianposta