Multiple Producers not scaling well

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Multiple Producers not scaling well

phill.tomlinson
Hi,

I have a single queue and decided to run some basic performance tests. Its a durable queue with manual acknowledgements.

I started with a single producer thread, that gave a message rate of 96msg/s. Then I tried 20 producer threads, that still gave 96msg/s. Same again for 50 producer threads.

I am wondering if I there is something wrong here or I have missed some vital config setting? In the web console I can see for example 20 separate connections so all producers have their own session and connection.

Thanks,
Phill
Reply | Threaded
Open this post in threaded view
|

Re: Multiple Producers not scaling well

Tim Bain
What's your message store?  If its throughput maxes out at 96 msg/sec, that
could certainly explain what you're seeing.  You could test that theory by
disabling persistence and re-running your test to see if throughput goes up.
On Mar 23, 2015 3:54 AM, "phill.tomlinson" <[hidden email]> wrote:

> Hi,
>
> I have a single queue and decided to run some basic performance tests. Its
> a
> durable queue with manual acknowledgements.
>
> I started with a single producer thread, that gave a message rate of
> 96msg/s. Then I tried 20 producer threads, that still gave 96msg/s. Same
> again for 50 producer threads.
>
> I am wondering if I there is something wrong here or I have missed some
> vital config setting? In the web console I can see for example 20 separate
> connections so all producers have their own session and connection.
>
> Thanks,
> Phill
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Multiple-Producers-not-scaling-well-tp4693596.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: Multiple Producers not scaling well

phill.tomlinson
Thanks Tim. I tried without message persistence and although I did get higher throughput it still didn't scale too well.

1 producer -13K msg/s
5 producers - 16K msg/s
10 producers - 17K msg/s

When compared with RabbitMQ and Kafka, it doesn't appear to scale too well when increasing the number of producers. It may be something in my environment but I'm pretty much using a vanilla ActiveMQ installation.

Phill
Reply | Threaded
Open this post in threaded view
|

Re: Multiple Producers not scaling well

James A. Robinson-2
What OS are you running this under?  Are you seeing a lot of
threads in your JVM?  Are you seeing a lot of GC activity?

On Mon, Mar 23, 2015 at 9:43 AM, phill.tomlinson <[hidden email]>
wrote:

> Thanks Tim. I tried without message persistence and although I did get
> higher
> throughput it still didn't scale too well.
>
> 1 producer -13K msg/s
> 5 producers - 16K msg/s
> 10 producers - 17K msg/s
>
> When compared with RabbitMQ and Kafka, it doesn't appear to scale too well
> when increasing the number of producers. It may be something in my
> environment but I'm pretty much using a vanilla ActiveMQ installation.
>
> Phill
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Multiple-Producers-not-scaling-well-tp4693596p4693626.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: Multiple Producers not scaling well

ceposta
In reply to this post by phill.tomlinson
What's your use case? What numbers are you trying to get?

On Mon, Mar 23, 2015 at 9:43 AM, phill.tomlinson <[hidden email]>
wrote:

> Thanks Tim. I tried without message persistence and although I did get
> higher
> throughput it still didn't scale too well.
>
> 1 producer -13K msg/s
> 5 producers - 16K msg/s
> 10 producers - 17K msg/s
>
> When compared with RabbitMQ and Kafka, it doesn't appear to scale too well
> when increasing the number of producers. It may be something in my
> environment but I'm pretty much using a vanilla ActiveMQ installation.
>
> Phill
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Multiple-Producers-not-scaling-well-tp4693596p4693626.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>



--
*Christian Posta*
twitter: @christianposta
http://www.christianposta.com/blog
http://fabric8.io
Reply | Threaded
Open this post in threaded view
|

Re: Multiple Producers not scaling well

phill.tomlinson
I am running the ActiveMQ server in an Ubuntu Trusty Docker container. I am then running my producers on my Windows developer environment.

I am not really targeting numbers, I am just trying to understand how much you can increase message throughput when you add more producers to a queue.
Reply | Threaded
Open this post in threaded view
|

Re: Multiple Producers not scaling well

phill.tomlinson
As a quick test I used the LevelDB persistence adapter and it actually started to scale as expected.

Does KahaDB have problems scaling with multiple producer writes when they are very fast?