Messages consumed multiple times without rollback?
we encounter the following somewhat strange behaviour with ActiveMQ 5.8.0. I had no luck finding anything similar on the net about it though.
We run an embedded broker with multiple queues. Non-persistent messages are put into Queue 1 by the clients. Consumers pull them from there (prefetch 0), process them and send a response into Queue 2. This is done within a transaction, so removal from Q1 and and adding response to Q2 are somewhat atomic. This setup usually works fine but at times it seems that messages which were already removed from Q1 and processed in the above manner are consumed again (We log the arrival of a message and the commit of a response so I'm pretty sure about that). We have no indication that a rollback was triggered on the session (log that too). At this point there are a messages sitting in Q1 which are not consumed even by idle consumers. If I put another message into that queue though, the oldest message is consumed. Via JMX I see a queue size of say 66 which is the difference between Enqueue Count and Dispatch/Dequeue Count but if I browse the queue I only get to see 30 messages. MemoryUsageByteCount is > 0 too. We are far from hitting any memory bounds.
What could possibly cause this behaviour? Is it possible that some internal cursor is somewhat offset?
We are not able to reproduce this but observe it after having processed several thousand to million messages.
I'm happy to provide additional information if needed. :)