Activating consumer for each message groups

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Activating consumer for each message groups

UcayaliFish
Hi,
I need some advice with message groups and I've failed to find a solution in the net.
What I'm doing is accepting RTP audio from several sources and converting each RTP frame to JMS BytesMessage and sending it to ActiveMQ queue setting JMSXGroupID property according to audio origin. I use Spring MessageListenerContainer and MessageListener at the consumer side. I set concurrentConsumers of the container to the supposed number of audio sources. I expected that a separate consumer would be created/used for each message group thus allowing to accumulate the audio data into larger chunks, process it and send further. Internal structure required for processing would be bound to MessageListener instance.
Unfortunately, the container starts a single consumer that receives all messages from all groups. I.e. the promise is kept - all messages in a group arrive to the same consumer instance. Actually, I just implement direct mapping of incoming messages with the same JMSXGroupID to a separate internal processor (through vm: queues) in a single MessageListener and it works as expected. It just consumes quite a lot of CPU/memory and I'm not sure it is an optimal solution whatsoever.
I'm afraid that there is a misunderstanding from my part. Is it a viable idea to implement this kind of message routing with message groups? What would community suggest for this type of problem?

TIA

Sergey
Reply | Threaded
Open this post in threaded view
|

Re: Activating consumer for each message groups

ceposta
Can you verify (JMX) that your consumer (are you using Camel? or straight
DMLC from spring?) is actually creating the requisite number of consumers
you're expecting?


On Thu, Jul 10, 2014 at 4:00 AM, UcayaliFish <[hidden email]> wrote:

> Hi,
> I need some advice with message groups and I've failed to find a solution
> in
> the net.
> What I'm doing is accepting RTP audio from several sources and converting
> each RTP frame to JMS BytesMessage and sending it to ActiveMQ queue setting
> JMSXGroupID property according to audio origin. I use Spring
> MessageListenerContainer and MessageListener at the consumer side. I set
> concurrentConsumers of the container to the supposed number of audio
> sources. I expected that a separate consumer would be created/used for each
> message group thus allowing to accumulate the audio data into larger
> chunks,
> process it and send further. Internal structure required for processing
> would be bound to MessageListener instance.
> Unfortunately, the container starts a single consumer that receives all
> messages from all groups. I.e. the promise is kept - all messages in a
> group
> arrive to the same consumer instance. Actually, I just implement direct
> mapping of incoming messages with the same JMSXGroupID to a separate
> internal processor (through vm: queues) in a single MessageListener and it
> works as expected. It just consumes quite a lot of CPU/memory and I'm not
> sure it is an optimal solution whatsoever.
> I'm afraid that there is a misunderstanding from my part. Is it a viable
> idea to implement this kind of message routing with message groups? What
> would community suggest for this type of problem?
>
> TIA
>
> Sergey
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Activating-consumer-for-each-message-groups-tp4683066.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>



--
*Christian Posta*
http://www.christianposta.com/blog
http://fabric8.io
twitter: @christianposta
Reply | Threaded
Open this post in threaded view
|

Re: Activating consumer for each message groups

UcayaliFish
ceposta wrote
Can you verify (JMX) that your consumer (are you using Camel? or straight
DMLC from spring?) is actually creating the requisite number of consumers
you're expecting?

--SKIPPED--
I'm using DMLC. Is web console to broker sufficient? I believe it shows 3 consumers of the queue but I'll check as soon as I can.
I created another prototype with embedded broker and three consumers that I create directly in code and the problem still exists.
One more question. It is unclear in documentation whether the same connection and consumers should be used for all producers in order for message groups to work?

TIA
Sergey
Reply | Threaded
Open this post in threaded view
|

Re: Activating consumer for each message groups

UcayaliFish
In reply to this post by ceposta
Hi,

I've checked it once again and broker's web console shows 3 active consumers.

TIA.

Sergey
Reply | Threaded
Open this post in threaded view
|

Re: Activating consumer for each message groups

SuoNayi
If you have existing messages in the broker and add consumers at a later
stage, it is a good idea to delay message dispatch start until all
consumers are present (or at least to give enough time for them to
subscribe). If you don't do that the first consumer will probably acquire
all message groups and all messages will be dispatched to it. You can
achieve this by usingconsumersBeforeDispatchStarts and
timeBeforeDispatchStarts destination policies
<http://activemq.apache.org/per-destination-policies.html>.
take a look at http://activemq.apache.org/message-groups.html