Consuming all messages from a queue

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Consuming all messages from a queue

garfen

Hello,

I'm using ActiveMQ 5.5.1 with ActiveMQ-CPP.

I've pre-loaded a queue with 30 000 messages and have a single consumer (INDIVIDUAL_ACKNOWLEDGE) that sits in a tight loop receiving messages from the queue using aConsumer->receive(1000).

I was hoping of being able to consume all the messages from the queue in a somewhat continuous manner. Instead I may get 200 messages initially, followed by ~30 sec. of no messages. Then 4000 messages, followed by ~30 sec. of no messages. Another 4000 messages, waiting for ~30 sec., and so on.

Running the test once more will consume the message continuously. Re-starting the AMQ broker, and I'm back in the situation explained above.

I've tried both 0 and 30000 for the prefetch limit in the connection uri without luck (tcp://my.host:61616?jms.prefetchPolicy.all=30000&wireFormat=openwire).

In the end I would like to have several consumers consuming messages from the queue. Each message would represent a job to process, which could take several hours to complete and all the consumers would together hold, say 500 000, such jobs. Now, if some consumer goes down, it is important that the jobs will be distributed and continued as soon as possible by other consumers.

It would be much appreciated if someone could give me some hints regarding this.

Thanks in advance.