[Kahadb vs Leveldb vs Replciated Leveldb] Performance Results

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

[Kahadb vs Leveldb vs Replciated Leveldb] Performance Results

khandelwalanuj
This post was updated on .
I have done performance testing for kahadb, leveldb and replication leveldb. Details:

Scenario:
> ActiveMQ : 5.10
> Machine : Unix
> I tested performance by sending messages to the broker continuously with any sleep or wait on producer side and measure the throughput by messages sent per seconds. The throughput increase initially but after some some it becomes constant. Similarly consumer is also running and consuming the messages. I also measure messages received per seconds. I can see that after some time producer and consumer throughput becomes equal.  And that is what I consider as throughput.
> I run each test mentioned below for ~2 hours.
> I did all tests by sending persistence messages.

Results:

K = 1000

LevelDB:

1 producer 1 consumer: 0.48K/sec(Queue) || 0.64K/sec (Topic)
2 producer 1 consumer: 0.88K/sec(Queue) || 0.79K/sec (Topic)
1 producer 2 consumer: 0.48K/sec(Queue) || 0.64K/sec (Topic)

kahaDB:

1 producer 1 consumer: 0.58K/sec(Queue) || 0.6K/sec (Topic)
2 producer 1 consumer: 0.6K/sec(Queue) || 0.6K/sec (Topic)

Replicated Leveldb:

1 producer 1 consumer: 0.06K/sec(Queue) || 0.06K/sec (Topic)
2 producer 1 consumer: 0.07K/sec(Queue) || 0.06K/sec (Topic)
1 producer 2 consumer: 0.06K/sec(Queue) || 0.05K/sec (Topic)


Questions:

> Performance is same with kahadb and leveldb which is strange. Why kahadb performance is now higher? Any particular configuration I need to tuneup ?
> Does kahadb provides better performance with multiple producers and single consumers ?(multiple writes)
> Replicated leveldb performance is too low. Almost 10 times going down. Is this expected ? Or I should tune some configurations to get better performance?

Also If there is any testing done by ActiveMQ developers, please point me to that.

Thanks,
Anuj





Reply | Threaded
Open this post in threaded view
|

Re: [Kahadb vs Leveldb vs Replciated Leveldb] Performance Results

gtully
the difference between kahadb and leveldb stores is really only visible
when there are multiple competing consumers and producers.
Both need to persist messages to a journal so they will both be disk bound
for writes/sends.
kahadb has a single index lock that becomes a bottleneck for adds and acks.
leveldb allows concurrent reads and more parallel updates.
There needs to be contention (mulitple completing consumer connections) to
make this visible.

On 3 December 2014 at 06:19, khandelwalanuj <[hidden email]>
wrote:

> I have done performance testing for kahadb, leveldb and replication
> leveldb.
> Details:
>
> *Scenario: *
> > ActiveMQ : 5.10
> > Machine : Unix
> > I tested performance by sending messages to the broker continuously with
> > any sleep or wait on producer side and measure the throughput by messages
> > sent per seconds. The throughput increase initially but after some some
> it
> > becomes constant. Similarly consumer is also running and consuming the
> > messages. I also measure messages received per seconds. I can see that
> > after some time producer and consumer throughput becomes equal.  And that
> > is what I consider as throughput.
> > I run each test mentioned below for ~2 hours.
> > I did all tests by sending persistence messages.
>
> *Results:
> *
>
> *LevelDB:
> *
> 1 producer 1 consumer: 0.48K/sec(Queue) || 0.64K/sec (Topic)
> 2 producer 1 consumer: 0.88K/sec(Queue) || 0.79K/sec (Topic)
> 1 producer 2 consumer: 0.48K/sec(Queue) || 0.64K/sec (Topic)
>
> *kahaDB:
> *
> 1 producer 1 consumer: 0.58K/sec(Queue) || 0.6K/sec (Topic)
> 2 producer 1 consumer: 0.6K/sec(Queue) || 0.6K/sec (Topic)
>
> *Replicated Leveldb:
> *
> 1 producer 1 consumer: 0.06K/sec(Queue) || 0.06K/sec (Topic)
> 2 producer 1 consumer: 0.07K/sec(Queue) || 0.06K/sec (Topic)
> 1 producer 2 consumer: 0.06K/sec(Queue) || 0.05K/sec (Topic)
>
>
> Questions:
>
> > Performance is same with kahadb and leveldb which is strange. Why kahadb
> > performance is now higher? Any particular configuration I need to tuneup
> ?
> > Does kahadb provides better performance with multiple producers and
> single
> > consumers ?(multiple writes)
> > Replicated leveldb performance is too low. Almost 10 times going down. Is
> > this expected ? Or I should tune some configurations to get better
> > performance?
>
> Also If there is any testing done by ActiveMQ developers, please point me
> to
> that.
>
> Thanks,
> Anuj
>
>
>
>
>
>
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Kahadb-vs-Leveldb-vs-Replciated-Leveldb-Performance-Results-tp4688306.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: [Kahadb vs Leveldb vs Replciated Leveldb] Performance Results

Kevin Burton
In reply to this post by khandelwalanuj
god. the performance of replicated levelDB seems horrible here.

On Tue, Dec 2, 2014 at 10:19 PM, khandelwalanuj <[hidden email]

> wrote:
>
> I have done performance testing for kahadb, leveldb and replication
> leveldb.
> Details:
>
> *Scenario: *
> > ActiveMQ : 5.10
> > Machine : Unix
> > I tested performance by sending messages to the broker continuously with
> > any sleep or wait on producer side and measure the throughput by messages
> > sent per seconds. The throughput increase initially but after some some
> it
> > becomes constant. Similarly consumer is also running and consuming the
> > messages. I also measure messages received per seconds. I can see that
> > after some time producer and consumer throughput becomes equal.  And that
> > is what I consider as throughput.
> > I run each test mentioned below for ~2 hours.
> > I did all tests by sending persistence messages.
>
> *Results:
> *
>
> *LevelDB:
> *
> 1 producer 1 consumer: 0.48K/sec(Queue) || 0.64K/sec (Topic)
> 2 producer 1 consumer: 0.88K/sec(Queue) || 0.79K/sec (Topic)
> 1 producer 2 consumer: 0.48K/sec(Queue) || 0.64K/sec (Topic)
>
> *kahaDB:
> *
> 1 producer 1 consumer: 0.58K/sec(Queue) || 0.6K/sec (Topic)
> 2 producer 1 consumer: 0.6K/sec(Queue) || 0.6K/sec (Topic)
>
> *Replicated Leveldb:
> *
> 1 producer 1 consumer: 0.06K/sec(Queue) || 0.06K/sec (Topic)
> 2 producer 1 consumer: 0.07K/sec(Queue) || 0.06K/sec (Topic)
> 1 producer 2 consumer: 0.06K/sec(Queue) || 0.05K/sec (Topic)
>
>
> Questions:
>
> > Performance is same with kahadb and leveldb which is strange. Why kahadb
> > performance is now higher? Any particular configuration I need to tuneup
> ?
> > Does kahadb provides better performance with multiple producers and
> single
> > consumers ?(multiple writes)
> > Replicated leveldb performance is too low. Almost 10 times going down. Is
> > this expected ? Or I should tune some configurations to get better
> > performance?
>
> Also If there is any testing done by ActiveMQ developers, please point me
> to
> that.
>
> Thanks,
> Anuj
>
>
>
>
>
>
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Kahadb-vs-Leveldb-vs-Replciated-Leveldb-Performance-Results-tp4688306.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>


--

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Reply | Threaded
Open this post in threaded view
|

Re: [Kahadb vs Leveldb vs Replciated Leveldb] Performance Results

louisphilip91
This post has NOT been accepted by the mailing list yet.
In reply to this post by khandelwalanuj
Oh that's greats post and informative for me and everyone. Thanks for sharing this post with us.
Thanks and regards.
Viagra Generic
Reply | Threaded
Open this post in threaded view
|

Re: [Kahadb vs Leveldb vs Replciated Leveldb] Performance Results

jessica121turner
This post has NOT been accepted by the mailing list yet.
In reply to this post by khandelwalanuj
Hello,
   Very Nice post... Thank you so much for sharing with us...


Thanks & Regards,
Buy Edegra
Reply | Threaded
Open this post in threaded view
|

Re: [Kahadb vs Leveldb vs Replciated Leveldb] Performance Results

neoanderson352
This post has NOT been accepted by the mailing list yet.
In reply to this post by khandelwalanuj
Wonderful information about Performance Results between Kahadb vs Leveldb vs Replciated Leveldb. Very soon i will contact to your developers and consult about everything. I just want to say thanks for everyone who have given their contribution to this post.

Thanks & Best Regards.
Generic Viagra Online.
Reply | Threaded
Open this post in threaded view
|

Re: [Kahadb vs Leveldb vs Replciated Leveldb] Performance Results

elliethomas
In reply to this post by gtully
I would love to collect some more details about this matter. It is a very nice post and i am highly thankful to you for sharing these details with us. Keep it up. See more at: http://www.angelmeds.com/
Reply | Threaded
Open this post in threaded view
|

Re: [Kahadb vs Leveldb vs Replciated Leveldb] Performance Results

tisasmith
In reply to this post by khandelwalanuj
Hello,

Informative article. Thanks a lot for sharing such nice information here.

Regards
Buy Generic Levitra
Reply | Threaded
Open this post in threaded view
|

Re: [Kahadb vs Leveldb vs Replciated Leveldb] Performance Results

buygenericrx
This post has NOT been accepted by the mailing list yet.
In reply to this post by khandelwalanuj
The Information you have given for levelDB, KahaDB, replicated levelDB is really nice. I would like to read more information.
Thanks & best Regards
Kamagra oral jelly online
Reply | Threaded
Open this post in threaded view
|

Re: [Kahadb vs Leveldb vs Replciated Leveldb] Performance Results

danielbiggs
In reply to this post by khandelwalanuj


The two stores are similar in that they both utilize an index to track the
location and state of messages written into a Journal. The differences lay
in the way in which the index is implemented among other things. LevelDB is
a slightly faster index than KahaDB and can provide somewhat better
performance numbers. In later releases of ActiveMQ the LevelDB store also
supports replication.

Given that you are on an older release where the LevelDB store
implementation was still very young I'd recommend sticking with KahaDB as it
is much more mature and has many of the early bugs worked out. The LevelDB
store in 5.8 is not really production-ready and will probably cause you some
trouble. You can move to the newest release v5.10 and LevelDB will be much
more robust.

Buy etinax <https://www.buyetilaam.com/product/etinax/ >  



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: [Kahadb vs Leveldb vs Replciated Leveldb] Performance Results

Tim Bain
That message appears to have been delayed by several years. Lest anyone get
confused by the recency of the timestamp, LevelDB is no longer supported
within ActiveMQ, and anyone who chooses to use it as the persistent store
should be prepared to debug and fix problems without assistance from the
mailing list.

Tim

On Sat, Nov 23, 2019, 5:54 PM danielbiggs <[hidden email]> wrote:

>
>
> The two stores are similar in that they both utilize an index to track the
> location and state of messages written into a Journal. The differences lay
> in the way in which the index is implemented among other things. LevelDB is
> a slightly faster index than KahaDB and can provide somewhat better
> performance numbers. In later releases of ActiveMQ the LevelDB store also
> supports replication.
>
> Given that you are on an older release where the LevelDB store
> implementation was still very young I'd recommend sticking with KahaDB as
> it
> is much more mature and has many of the early bugs worked out. The LevelDB
> store in 5.8 is not really production-ready and will probably cause you
> some
> trouble. You can move to the newest release v5.10 and LevelDB will be much
> more robust.
>
> Buy etinax <https://www.buyetilaam.com/product/etinax/ >
>
>
>
> --
> Sent from:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
>