non persistent messages and maxPageSize queue performance

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

non persistent messages and maxPageSize queue performance

Alex Colomb
I've been tracking down some performance issues on one of our queues. The queue is hosted on an embedded broker. There is one producer and one consumer. All messages sent to the queue are non-persistent. The producer initially will have a burst of ~30k messages and then slow down to 100 messages a second. During the initial burst of messages, I see the consumer slowly consuming messages and eventually a large number of messages expire off the queue before the consumer can get to them. I've tracked the issue down to the maxPageSize setting. If I leave it at the default, messages are consumed very quickly and non are expired. If I set it to 100k, I see the behavior described above.

The documentation (http://activemq.apache.org/per-destination-policies.html) describes maxPageSize as "maximum number of persistent messages to page from store at a time." Why is changing this setting affecting the delivery/consumption of non persistent messages? Why would a large value slow things down?

Here is the programmatic broker configuration:

BrokerService embeddedBroker = new BrokerService();

PolicyEntry policyEntry = new PolicyEntry();
policyEntry.setQueue( ">" );
policyEntry.setProducerFlowControl( true );
policyEntry.setAdvisoryForSlowConsumers( true );
policyEntry.setMaxPageSize( 100000 );
policyEntry.setExpireMessagesPeriod( 15000 );

PolicyMap policyMap = new PolicyMap();
policyMap.setDefaultEntry( policyEntry );
embeddedBroker.setDestinationPolicy( policyMap );
embeddedBroker.addConnector( "tcp://0.0.0.0:51515" );

I'm using Active MQ 5.10.0.

Thank you.
Reply | Threaded
Open this post in threaded view
|

Re: non persistent messages and maxPageSize queue performance

Tim Bain
Since no one responded with an explanation, I think you should submit this
as a bug in the ActiveMQ JIRA, ideally with a unit test.
On Dec 31, 2014 8:58 AM, "Alex Colomb" <[hidden email]> wrote:

> I've been tracking down some performance issues on one of our queues. The
> queue is hosted on an embedded broker. There is one producer and one
> consumer. All messages sent to the queue are non-persistent. The producer
> initially will have a burst of ~30k messages and then slow down to 100
> messages a second. During the initial burst of messages, I see the consumer
> slowly consuming messages and eventually a large number of messages expire
> off the queue before the consumer can get to them. I've tracked the issue
> down to the maxPageSize setting. If I leave it at the default, messages are
> consumed very quickly and non are expired. If I set it to 100k, I see the
> behavior described above.
>
> The documentation (
> http://activemq.apache.org/per-destination-policies.html) describes
> maxPageSize as "maximum number of persistent messages to page from store at
> a time." Why is changing this setting affecting the delivery/consumption of
> non persistent messages? Why would a large value slow things down?
>
> Here is the programmatic broker configuration:
>
> BrokerService embeddedBroker = new BrokerService();
>
> PolicyEntry policyEntry = new PolicyEntry();
> policyEntry.setQueue( ">" );
> policyEntry.setProducerFlowControl( true );
> policyEntry.setAdvisoryForSlowConsumers( true );
> policyEntry.setMaxPageSize( 100000 );
> policyEntry.setExpireMessagesPeriod( 15000 );
>
> PolicyMap policyMap = new PolicyMap();
> policyMap.setDefaultEntry( policyEntry );
> embeddedBroker.setDestinationPolicy( policyMap );
> embeddedBroker.addConnector( "tcp://0.0.0.0:51515" );
>
> I'm using Active MQ 5.10.0.
>
> Thank you.
>
Reply | Threaded
Open this post in threaded view
|

Re: non persistent messages and maxPageSize queue performance

artnaseef
In reply to this post by Alex Colomb
That is very odd.  Can you create a minimal stand-alone test program that shows the problem?

I would like to see assertions failing on the slow-consumption case and passing on the fast-consumption case.  That will make it much easier to diagnose.