Message Group

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Message Group

Mehra
Hi,
We are using activemq to handle our JMS messages, the messages come from different devices with unique device_id and we like to make sure that messages from a specific device , process in order.
also we have thousands of devices.
I am new on AMQ but we have performance issue and I came across message group concept so I think it can help us but I am just worry that, if we use multiple consumers and use that device_id as our JMSXGroupID, basically for example all messages from device_id 205 and 105 will all go to the consumer1 , and all messaged from 102 and 202 go to consumer2 and so on,  now think about this scenario that a message form 105 go into the consumer1 and it take long to get processed, then a message from 205 come , we dont want it to wait because although they are belong to same consumer but they are for different devices and we dont care if they are not in a same order!
so is there any mechanism that we can check and see which device_id  make consumer busy , so if new message comes in, we first check see if it has the same device_id or not! if not we send this message to a another consumer that is not too busy but if it has a same device_id then it has to wait in the queue of that consumer because of the order issue.

Thank you for any help and suggestion in advnace
Reply | Threaded
Open this post in threaded view
|

Re: Message Group

artnaseef
Message groups force processing of a single destination to only one consumer at a time - not really the processing model you've described.

If there were separate queues for each device, message groups would work.  However, if there are large numbers of devices, that could be a real concern since it would mean large numbers of destinations.

Using camel, it would be possible to route to specific instances of the consumer application (engine) by having each engine listen on a queue dedicated to it, then having a camel component that decides which engine will process which device and route to that engine.  On top of that, it would be possible to have 2 instances of the engine listening on each queue, using JMSXGroup's, to ensure high availability and redundancy.  Just a thought...
Reply | Threaded
Open this post in threaded view
|

Re: Message Group

Mehra
Thank you for your reply, so what this means:
"Since there could be a massive number of individual message groups we use hash buckets rather than the actual JMSXGroupID string"
so I mean if we can use hash then we dont really need to have one consumer per each id! so there is no need to have large number of destinations. but as I said the concern is how we can make sure that we dont delay the messages that are not really have exactly same id but they are in a same group! they hash is the same basically!
I want to make sure there is no other way before I use the Camel
Thank you again