Quantcast

Consumer is not able to pick messages from queue

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Consumer is not able to pick messages from queue

manua
Hi All,

I am working on request-response system, where all the requests are posted on a single queue with different selectors. There are multiple consumers listening on the queue.

When the load is low, this works fine. Each consumer is able to process the respective message. But if the load is high and there are pending messages of one type in the queue, other consumers timed out without picking the message from the queue. If I didn't allow the consumers to time out, then they will wait till all the previous messages are processed and then process their respective messages.

To give little more idea,

There are around 10000 messages in the queue with selector type A and four consumers with selector A are running.Consumers for type A message took around 2-3 seconds to process each message. When there are 10000 messages pending in the queue, another 5 messages with selector B and selector C are added to the queue.There are two consumers running for type B and one for type C, each with a timeout of 60 seconds.

But, consumers B and C never got the messages and timed out.

Is it possible that due to the other messages in the queue, B and C are not processed. Or there is some other setting that needs to be set.

Please let me know your inputs.

Thanks,
Manu
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Consumer is not able to pick messages from queue

gtully
yea, this is a known issue with sparse selectors. increasing the
maxPageSize via a destination policy entry for that queue will help as
it will ensure that more messages are held in memory for dispatch.

see: the comments on
https://issues.apache.org/activemq/browse/AMQ-2217 and
http://trenaman.blogspot.com/2009/01/message-selectors-and-activemq.html

On 8 September 2010 18:44, manua <[hidden email]> wrote:

>
> Hi All,
>
> I am working on request-response system, where all the requests are posted
> on a single queue with different selectors. There are multiple consumers
> listening on the queue.
>
> When the load is low, this works fine. Each consumer is able to process the
> respective message. But if the load is high and there are pending messages
> of one type in the queue, other consumers timed out without picking the
> message from the queue. If I didn't allow the consumers to time out, then
> they will wait till all the previous messages are processed and then process
> their respective messages.
>
> To give little more idea,
>
> There are around 10000 messages in the queue with selector type A and four
> consumers with selector A are running.Consumers for type A message took
> around 2-3 seconds to process each message. When there are 10000 messages
> pending in the queue, another 5 messages with selector B and selector C are
> added to the queue.There are two consumers running for type B and one for
> type C, each with a timeout of 60 seconds.
>
> But, consumers B and C never got the messages and timed out.
>
> Is it possible that due to the other messages in the queue, B and C are not
> processed. Or there is some other setting that needs to be set.
>
> Please let me know your inputs.
>
> Thanks,
> Manu
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/Consumer-is-not-able-to-pick-messages-from-queue-tp2531722p2531722.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>



--
http://blog.garytully.com

Open Source Integration
http://fusesource.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Consumer is not able to pick messages from queue

Fredrik Jonson-2
Gary Tully wrote:

>  yea, this is a known issue with sparse selectors. increasing the
>  maxPageSize via a destination policy entry for that queue will help as
>  it will ensure that more messages are held in memory for dispatch.

Well, yes it helps, but only as long as the message your selector wants
can be found within the configured page size.

>  see: the comments on
>  http://trenaman.blogspot.com/2009/01/message-selectors-and-activemq.html

The last sentence on the above post offer a very good alternative. If you need
to adress messages by type and must be able to handle a backlog bigger than
what is possible to contain in the maxPageSize, replace your selectors with
separate named queues, each specific for whatever you selected for before.

This assumes that you do not have too many selectors though, and obviously
that your selector queries can be mapped to named queues.

--
Fredrik Jonson

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Consumer is not able to pick messages from queue

gtully
The named queue concept can be realized in configuration using
filtered destination that use selectors. see;
http://activemq.apache.org/virtual-destinations.html#VirtualDestinations-Usingfiltereddestinations

As the selectors are evaluated at send time there is no limit imposed
by sparse matching.

On 9 September 2010 21:19, Fredrik Jonson <[hidden email]> wrote:

> Gary Tully wrote:
>
>>  yea, this is a known issue with sparse selectors. increasing the
>>  maxPageSize via a destination policy entry for that queue will help as
>>  it will ensure that more messages are held in memory for dispatch.
>
> Well, yes it helps, but only as long as the message your selector wants
> can be found within the configured page size.
>
>>  see: the comments on
>>  http://trenaman.blogspot.com/2009/01/message-selectors-and-activemq.html
>
> The last sentence on the above post offer a very good alternative. If you need
> to adress messages by type and must be able to handle a backlog bigger than
> what is possible to contain in the maxPageSize, replace your selectors with
> separate named queues, each specific for whatever you selected for before.
>
> This assumes that you do not have too many selectors though, and obviously
> that your selector queries can be mapped to named queues.
>
> --
> Fredrik Jonson
>
>



--
http://blog.garytully.com

Open Source Integration
http://fusesource.com
Loading...