Queue - doBrowse - infinite loop (AMQ 5.10.0)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Queue - doBrowse - infinite loop (AMQ 5.10.0)

rkraus
Hello,
I have AMQ 5.10.0 (java 1.6) and I face to problem with infinite loop, org.apache.activemq.broker.region.Queue.doBrowse(...) method:

while (shouldPageInMoreForBrowse(max)) {
  pageInMessages(!memoryUsage.isFull(110));
};
I have thread dump and heap dump, but till now, I can't to write simple example, which can simulate this problem.

I read following resources:
 - http://activemq.2283324.n4.nabble.com/org-apache-activemq-broker-region-Queue-doBrowse-never-returns-td4680530.html
 - https://issues.apache.org/jira/browse/AMQ-4930

I have one problem with "non persistent" queue, where I guess that destinationStatistics is out of sync with "store" (following logs are repeated):
2017-03-02 17:32:27.964 TRACE {ActiveMQ Broker[JmsEngineActivemqBroker] Scheduler} [N/A] [N/A   ] [Queue] max 400, alreadyPagedIn 0, messagesCount 4925, memoryUsage 0%
2017-03-02 17:32:27.964 DEBUG {ActiveMQ Broker[JmsEngineActivemqBroker] Scheduler} [N/A] [N/A   ] [Queue] non-persistent-queue toPageIn: 0, Inflight: 0, pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 3284134, dequeueCount: 3279209, memUsage:0

I have second problem with "persistent" queue, where I guess that cursor is out of sync with "store" (following logs are repeated):
2017-01-20 13:05:18.024 TRACE {ActiveMQ Broker[JmsEngineActivemqBroker] Scheduler} [N/A] [N/A   ] [Queue] max 400, alreadyPagedIn 0, messagesCount 6914, memoryUsage 0%
2017-01-20 13:05:18.024 DEBUG {ActiveMQ Broker[JmsEngineActivemqBroker] Scheduler} [N/A] [N/A   ] [Queue] persistence-queue toPageIn: 200, Inflight: 0, pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 30316, dequeueCount: 23402, memUsage:0

I registered commits (which was done in context AMQ-4930 issue in 5.11.x and above):
 1. ensure we page in messages for browse/expire when destination stats are disabled via config (replacing destinationStatistics by message cursor)
   - https://fisheye.apache.org/changelog/activemq-6?cs=41659725f4c4fa027386148077aa76c31d8853af
   - https://fisheye.apache.org/changelog/activemq-6?cs=f158e7da6752e4a48b12547ad09f1ae90a0d452d
   - I guess that this patch should solve my problem with "non persistent" queue even when destinationStatistics are enabled (due destinationStatistics aren't used)
 2. limit browse page in iterations in case cursor and store are out of sync - avoid a spin when store does not return messages
   - https://fisheye.apache.org/changelog/activemq-6?cs=8216e7f4d5e275cc41a3d21dea8d7ffaa430719e
   - I guess that this patch should solve my problem with "persistent" queue (cursor and store are out of sync)

But I don't understand the last patch at all. Yes, it probably causes, that infinite loop is finished (in some iterations). But I guess that JMX monitoring
still display wrong information and cursor and store are still out of sync or I miss something?

Could you explain me the "cursor and store are out of sync" commit? I didn't find the test for it.
Any idea how to create example, which should simulate this "endless loop" of doBrowse(...) method
invoked from org.apache.activemq.broker.region.Queue.expireMessages() method initiated by "expireMessagesTask".

Thank you.
Radek Kraus