QueueSize == 0 && CursorMemoryUsage > 0 - is that a problem?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

QueueSize == 0 && CursorMemoryUsage > 0 - is that a problem?

david.sitsky
Hi,

I have an application using ActiveMQ 5.13.5 which uses non-persistent
messages.  There is a queue which has several producers and a single
consumer.

At times I see the CursorMemoryUsage not being 0 despite the queue being
empty.  When this application runs for a very long time,
CursorMemoryUsage grows and this can become an issue with system memory
usage.  My understanding might be wrong but is this a problem?

At first this sounded like
https://issues.apache.org/jira/browse/AMQ-6094 but this has apparently
been fixed for this release.

I've attached a jconsole view showing the issue.  When I invoked via JMX
these operations they return:

doesCursorHaveMessagesBuffered -> false
cursorSize -> 0
doesCursorHaveSpace -> 0

On a different run, I had DEBUG enabled and I've seen these sort of
lines confirming memory usage despite the queue being empty:

2017-03-15 21:47:55,624 [ActiveMQ
BrokerService[cb01e0612bb1468d88828716182f7f39] Task-5] 214837 DEBUG
org.apache.activemq.broker.region.Queue - queue://master-items-7.0,
subscriptions=1, memory=0%, size=0, pending=0 toPageIn: 0, Inflight: 0,
pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount:
5518, dequeueCount: 5518, memUsage:28538

Is this a bug?  I know I'll be asked for a test case, but I wanted to
confirm it first before proceeding.



--
Cheers,
David
Reply | Threaded
Open this post in threaded view
|

Re: QueueSize == 0 && CursorMemoryUsage > 0 - is that a problem?

christopher.l.shannon
Yes, memory usage should be 0 for a destination if there are no messages so
this seems like a bug.  There was another memory bug fixed more recently
but that only had to do with persistent messages when using concurrent
store and dispatch.  As you pointed out the best thing to do to help would
be to create a test case that can reproduce the issue but make sure you use
the latest version (5.14.4).

On Wed, Mar 15, 2017 at 7:01 AM, <[hidden email]> wrote:

> Hi,
>
> I have an application using ActiveMQ 5.13.5 which uses non-persistent
> messages.  There is a queue which has several producers and a single
> consumer.
>
> At times I see the CursorMemoryUsage not being 0 despite the queue being
> empty.  When this application runs for a very long time,
> CursorMemoryUsage grows and this can become an issue with system memory
> usage.  My understanding might be wrong but is this a problem?
>
> At first this sounded like
> https://issues.apache.org/jira/browse/AMQ-6094 but this has apparently
> been fixed for this release.
>
> I've attached a jconsole view showing the issue.  When I invoked via JMX
> these operations they return:
>
> doesCursorHaveMessagesBuffered -> false
> cursorSize -> 0
> doesCursorHaveSpace -> 0
>
> On a different run, I had DEBUG enabled and I've seen these sort of
> lines confirming memory usage despite the queue being empty:
>
> 2017-03-15 21:47:55,624 [ActiveMQ
> BrokerService[cb01e0612bb1468d88828716182f7f39] Task-5] 214837 DEBUG
> org.apache.activemq.broker.region.Queue - queue://master-items-7.0,
> subscriptions=1, memory=0%, size=0, pending=0 toPageIn: 0, Inflight: 0,
> pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount:
> 5518, dequeueCount: 5518, memUsage:28538
>
> Is this a bug?  I know I'll be asked for a test case, but I wanted to
> confirm it first before proceeding.
>
>
>
> --
> Cheers,
> David
>
>
Reply | Threaded
Open this post in threaded view
|

Re: QueueSize == 0 && CursorMemoryUsage > 0 - is that a problem?

Tim Bain
In reply to this post by david.sitsky
It does sound like a bug, and it certainly could be the same root cause as
the bug you linked to. Does the workaround in that bug also work
(temporarily) for you?

A shot in the dark: is there any chance that you might have messages
expiring before they can be consumed? I've got a vague recollection of a
bug being fixed recently where we still counted a message as pending when
it expired while in the consumer's dispatch queue; maybe there's a similar
bug for memory usage?

Tim

On Mar 15, 2017 5:01 AM, <[hidden email]> wrote:

Hi,

I have an application using ActiveMQ 5.13.5 which uses non-persistent
messages.  There is a queue which has several producers and a single
consumer.

At times I see the CursorMemoryUsage not being 0 despite the queue being
empty.  When this application runs for a very long time,
CursorMemoryUsage grows and this can become an issue with system memory
usage.  My understanding might be wrong but is this a problem?

At first this sounded like
https://issues.apache.org/jira/browse/AMQ-6094 but this has apparently
been fixed for this release.

I've attached a jconsole view showing the issue.  When I invoked via JMX
these operations they return:

doesCursorHaveMessagesBuffered -> false
cursorSize -> 0
doesCursorHaveSpace -> 0

On a different run, I had DEBUG enabled and I've seen these sort of
lines confirming memory usage despite the queue being empty:

2017-03-15 21:47:55,624 [ActiveMQ
BrokerService[cb01e0612bb1468d88828716182f7f39] Task-5] 214837 DEBUG
org.apache.activemq.broker.region.Queue - queue://master-items-7.0,
subscriptions=1, memory=0%, size=0, pending=0 toPageIn: 0, Inflight: 0,
pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount:
5518, dequeueCount: 5518, memUsage:28538

Is this a bug?  I know I'll be asked for a test case, but I wanted to
confirm it first before proceeding.



--
Cheers,
David
Reply | Threaded
Open this post in threaded view
|

Re: QueueSize == 0 && CursorMemoryUsage > 0 - is that a problem?

david.sitsky
Hi Tim and Christopher,

> It does sound like a bug, and it certainly could be the same root
> cause as the bug you linked to. Does the workaround in that bug also
> work (temporarily) for you?

I can't really apply that workaround to our application (deleting the
queue and re-creating it) I'm afraid.

> A shot in the dark: is there any chance that you might have messages
> expiring before they can be consumed? I've got a vague recollection of a
> bug being fixed recently where we still counted a message as pending when
> it expired while in the consumer's dispatch queue; maybe there's a similar
> bug for memory usage?

We don't have any message expiry set so I don't think that is it.  FWIW
I tried 5.14.4 and the same problem seems to be present.

When I get some free time in the weeks ahead I'll see if I can reproduce
this in a unit test.

--
Cheers,
David