memory leak

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

memory leak

vox_de_hg
Hi,

I'm facing a strange memory leak on one of my production environment (others
environments are ok)

The Environment
 1) AIX 7.1
 2) IBM JVM 1.7
 3) ActiveMQ broker 5.12.2 embedded in our java monitoring application
 3) Client are either java client (embedded inside a front office war
application) or C++ client based on CMS 3.8 (Back office standalone
application).

the operating mode is straightforward : front office application invoke
business services (request - reply) to back office applications.
We are sure that all requests are correctly queued and  de-queued. We also
didn't notice slow consumer effects.

We got a memory leak, as reported by memory leak analysis (see attached
images)
<http://activemq.2283324.n4.nabble.com/file/t379265/clup-ml0.png>
<http://activemq.2283324.n4.nabble.com/file/t379265/clup-ml1.png>
<http://activemq.2283324.n4.nabble.com/file/t379265/clup-ml2.png>
<http://activemq.2283324.n4.nabble.com/file/t379265/clup-ml3.png>

Our memory analysis tools points 2 suspects and one hint:

suspect 1 : 113 206 instance of ActiveMQMessage
suspect 2 : 157 instances of
"org.apache.activemq.broker.region.TopicSubscription
hint : all suspects have a common chain : o.a.a.broker.BrokerService =>
o.a.a.broker.jmx.managedregionbroker

can that mean that the problem is related to jmx monitoring ?



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: memory leak

art.licis
Hi,

From the side, it does look like pending messages accumulated in memory
because of a slow or blocked subscriber. Since you have heap dump, you
should look at this specific TopicSubscription object just to make sure you
don't have actually a consumer subscribed to some of your topics, but not
accepting messages (e.g., a consumer created but no listener attached).
Within TopicSubscription you'll have destination info, consumer info, and
some other useful data that could indicate the cause of your issue.

On a side note, it looks like you are using VM Cursor, which was the main
cursor type prior to version 5. It holds pending messages in memory, thus
can cause OOM. By default, 5.x would use Store cursor avoiding memory
issues. You can read more about it here:
http://activemq.apache.org/message-cursors.html
.. and I would advise not using VM Cursor.

- Art



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html