Measures to improve the throughput

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Measures to improve the throughput

Frizz
In an atempt to improve the throughput of my system I did various tests. I
started with a simple setup:
- 1 Topic, non persistent
- 1 Producer
- 1 Consumer
- Messge size: 4k
- Active MQ 5.12.1

My system is reasonably fast (8 cores, 32GB memory, SSD) - still I only
manage to send & receive about 45.000 messages per second.

Basically my producer looks like this:
    ...
    ConnectionFactory factory = new
ActiveMQConnectionFactory("tcp://loclhost:61616");
    Connection connection = factory.createConnection();
    session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
    producer = session.createProducer(null);
    connection.start();

    for (long i = 0; i < 1000000; i++) {
        final Topic topic = session.createTopic(topicName);
        producer.setDeliveryMode(DeliveryMode.NON_PERSISTENT);

        BytesMessage message = session.createBytesMessage();
        message.writeBytes(someByteArray); // size 4k
        producer.send(topic, message);
    }
    ...

I measured the time that the consumer spends in its onMessage() method and
it's only about 1% of its total runtime (so consuming the messages is not a
bottleneck (no surprise, because of BytesMessages)).

45.000 messages/second does not sound *that* much to me - is this the
maximum I can expect to get from Active MQ?

I did some further tests increasing the number of topic consumers - and
noticed that my msg/sec drop. But the total amount of mes/sec that pass
through my system stays roughly the same (OK, with some shrinkage/friction):

producer: 45.000
consumer: 45.000
---------        -> 90.000 total
producer: 28.700
consumer: 28.700
consumer: 28.700
---------        -> 86.100 total
producer: 20.400
consumer: 20.400
consumer: 20.400
consumer: 20.400
---------        -> 81.600 total
producer: 15.700
consumer: 15.700
consumer: 15.700
consumer: 15.700
consumer: 15.700
---------        -> 78.500 total


What I had expected would look more like this:
producer: 43.000
consumer: 43.000
consumer: 43.000
consumer: 43.000
consumer: 43.000

Where does this "global total msg/sec limit" come from? Where is the
bottleneck?
Reply | Threaded
Open this post in threaded view
|

Re: Measures to improve the throughput

hqstevenson
That seems very slow for a simple throughput test - my simple Camel tests have usually been in the 1000’s msgs/sec range.

Can you share the full test?

> On Mar 31, 2016, at 12:01 AM, Frizz <[hidden email]> wrote:
>
> In an atempt to improve the throughput of my system I did various tests. I
> started with a simple setup:
> - 1 Topic, non persistent
> - 1 Producer
> - 1 Consumer
> - Messge size: 4k
> - Active MQ 5.12.1
>
> My system is reasonably fast (8 cores, 32GB memory, SSD) - still I only
> manage to send & receive about 45.000 messages per second.
>
> Basically my producer looks like this:
>    ...
>    ConnectionFactory factory = new
> ActiveMQConnectionFactory("tcp://loclhost:61616");
>    Connection connection = factory.createConnection();
>    session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
>    producer = session.createProducer(null);
>    connection.start();
>
>    for (long i = 0; i < 1000000; i++) {
>        final Topic topic = session.createTopic(topicName);
>        producer.setDeliveryMode(DeliveryMode.NON_PERSISTENT);
>
>        BytesMessage message = session.createBytesMessage();
>        message.writeBytes(someByteArray); // size 4k
>        producer.send(topic, message);
>    }
>    ...
>
> I measured the time that the consumer spends in its onMessage() method and
> it's only about 1% of its total runtime (so consuming the messages is not a
> bottleneck (no surprise, because of BytesMessages)).
>
> 45.000 messages/second does not sound *that* much to me - is this the
> maximum I can expect to get from Active MQ?
>
> I did some further tests increasing the number of topic consumers - and
> noticed that my msg/sec drop. But the total amount of mes/sec that pass
> through my system stays roughly the same (OK, with some shrinkage/friction):
>
> producer: 45.000
> consumer: 45.000
> ---------        -> 90.000 total
> producer: 28.700
> consumer: 28.700
> consumer: 28.700
> ---------        -> 86.100 total
> producer: 20.400
> consumer: 20.400
> consumer: 20.400
> consumer: 20.400
> ---------        -> 81.600 total
> producer: 15.700
> consumer: 15.700
> consumer: 15.700
> consumer: 15.700
> consumer: 15.700
> ---------        -> 78.500 total
>
>
> What I had expected would look more like this:
> producer: 43.000
> consumer: 43.000
> consumer: 43.000
> consumer: 43.000
> consumer: 43.000
>
> Where does this "global total msg/sec limit" come from? Where is the
> bottleneck?

Reply | Threaded
Open this post in threaded view
|

Re: Measures to improve the throughput

Tim Bain
I think he's using European-style formatting, and that those numbers are
forty-five thousand, ninety thousand, etc.

Those numbers sound great to me; I was always getting numbers in the order
of magnitude of 1000 like Quinn said, even though I expected to go faster
since we only did non-persistent messages. But my consumer code was never a
no-op so I'm sure that was part of it.

Tim
On Mar 31, 2016 8:34 AM, "Quinn Stevenson" <[hidden email]>
wrote:

> That seems very slow for a simple throughput test - my simple Camel tests
> have usually been in the 1000’s msgs/sec range.
>
> Can you share the full test?
>
> > On Mar 31, 2016, at 12:01 AM, Frizz <[hidden email]> wrote:
> >
> > In an atempt to improve the throughput of my system I did various tests.
> I
> > started with a simple setup:
> > - 1 Topic, non persistent
> > - 1 Producer
> > - 1 Consumer
> > - Messge size: 4k
> > - Active MQ 5.12.1
> >
> > My system is reasonably fast (8 cores, 32GB memory, SSD) - still I only
> > manage to send & receive about 45.000 messages per second.
> >
> > Basically my producer looks like this:
> >    ...
> >    ConnectionFactory factory = new
> > ActiveMQConnectionFactory("tcp://loclhost:61616");
> >    Connection connection = factory.createConnection();
> >    session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
> >    producer = session.createProducer(null);
> >    connection.start();
> >
> >    for (long i = 0; i < 1000000; i++) {
> >        final Topic topic = session.createTopic(topicName);
> >        producer.setDeliveryMode(DeliveryMode.NON_PERSISTENT);
> >
> >        BytesMessage message = session.createBytesMessage();
> >        message.writeBytes(someByteArray); // size 4k
> >        producer.send(topic, message);
> >    }
> >    ...
> >
> > I measured the time that the consumer spends in its onMessage() method
> and
> > it's only about 1% of its total runtime (so consuming the messages is
> not a
> > bottleneck (no surprise, because of BytesMessages)).
> >
> > 45.000 messages/second does not sound *that* much to me - is this the
> > maximum I can expect to get from Active MQ?
> >
> > I did some further tests increasing the number of topic consumers - and
> > noticed that my msg/sec drop. But the total amount of mes/sec that pass
> > through my system stays roughly the same (OK, with some
> shrinkage/friction):
> >
> > producer: 45.000
> > consumer: 45.000
> > ---------        -> 90.000 total
> > producer: 28.700
> > consumer: 28.700
> > consumer: 28.700
> > ---------        -> 86.100 total
> > producer: 20.400
> > consumer: 20.400
> > consumer: 20.400
> > consumer: 20.400
> > ---------        -> 81.600 total
> > producer: 15.700
> > consumer: 15.700
> > consumer: 15.700
> > consumer: 15.700
> > consumer: 15.700
> > ---------        -> 78.500 total
> >
> >
> > What I had expected would look more like this:
> > producer: 43.000
> > consumer: 43.000
> > consumer: 43.000
> > consumer: 43.000
> > consumer: 43.000
> >
> > Where does this "global total msg/sec limit" come from? Where is the
> > bottleneck?
>
>