Another memory blow out

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Another memory blow out

Lionel van den Berg
Hi all,

We had another site issue of memory blow out today in our operations system.

AMQ had been running for a month and had been stable at 1GB of memory use.
Then at 3am approximately it started rising by about 50mb every 5 minutes
until we had to restart it 4 hours later to prevent an uncontrolled outage.

I managed to get activemq.log, not much information in there, there are a
bunch of EOFException and a few of already connected messages for one
client. I also got the output of running the query argument to the startup
script. I'm not sure how to read this file completely, it's massive, 68MB.
What I do see is there are around 33K topics, a large number of those with
no subscribers, but I don't think this is necessarily any indication of an
issue, my understanding of our system is that the topic number didn't climb

I noticed on interesting topic which as TotalConsumerCount = 1334, this is
way higher than I expected, could this be an indication of any issue?

What else can I look for in that output? I thought it would describe
everything I would need to understand where the memory is used.