"java.lang.OutOfMemoryError" in consumer application

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

"java.lang.OutOfMemoryError" in consumer application

khandelwalanuj
Hi,

We sometimes face :

Exception in thread "ActiveMQ Transport: tcp://mqbroker/*.*.*.*:61616@35637" java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:714)
at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
at org.apache.activemq.thread.TaskRunnerFactory.execute(TaskRunnerFactory.java:152)
at org.apache.activemq.thread.TaskRunnerFactory.execute(TaskRunnerFactory.java:145)
at org.apache.activemq.transport.tcp.TcpTransport.doStop(TcpTransport.java:538)
at org.apache.activemq.util.ServiceSupport.stop(ServiceSupport.java:71)
at org.apache.activemq.transport.tcp.TcpTransport.stop(TcpTransport.java:582)
at org.apache.activemq.transport.AbstractInactivityMonitor.stop(AbstractInactivityMonitor.java:145)
at org.apache.activemq.transport.TransportFilter.stop(TransportFilter.java:65)
at org.apache.activemq.transport.WireFormatNegotiator.stop(WireFormatNegotiator.java:91)
at org.apache.activemq.util.ServiceSupport.dispose(ServiceSupport.java:43)
at org.apache.activemq.transport.failover.FailoverTransport.disposeTransport(FailoverTransport.java:240)
at org.apache.activemq.transport.failover.FailoverTransport.handleTransportFailure(FailoverTransport.java:253)
at org.apache.activemq.transport.failover.FailoverTransport$3.onException(FailoverTransport.java:215)
at org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:101)
at org.apache.activemq.transport.WireFormatNegotiator.onException(WireFormatNegotiator.java:160)
at org.apache.activemq.transport.AbstractInactivityMonitor.onException(AbstractInactivityMonitor.java:314)
at org.apache.activemq.transport.TransportSupport.onException(TransportSupport.java:96)
at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:205)
at java.lang.Thread.run(Thread.java:745)


I have checked http://activemq.apache.org/javalangoutofmemory.html but non of the mentioned condition is getting triggered. Any idea what can cause this on consumer side ?

Thanks,
Anuj

Reply | Threaded
Open this post in threaded view
|

Re: "java.lang.OutOfMemoryError" in consumer application

Tim Bain
This is caused by hitting the native thread limit for the user that
ActiveMQ runs as, causing the JVM to be "unable to create a new native
thread" as stated in the stack trace you quoted.  Increase that user's
maxproc setting in the Linux OS (I assume you're running Linux, since I
don't think this occurs under Windows) and you should be good; we use 8192
and haven't seen any issues.

Tim
On Feb 19, 2015 5:38 AM, "khandelwalanuj" <[hidden email]>
wrote:

> Hi,
>
> We sometimes face :
>
> Exception in thread "ActiveMQ Transport: tcp://mqbroker/*.*.*.*:61616@35637
> "
> java.lang.OutOfMemoryError: unable to create new native thread
> at java.lang.Thread.start0(Native Method)
> at java.lang.Thread.start(Thread.java:714)
> at
>
> java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950)
> at
>
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
> at
>
> org.apache.activemq.thread.TaskRunnerFactory.execute(TaskRunnerFactory.java:152)
> at
>
> org.apache.activemq.thread.TaskRunnerFactory.execute(TaskRunnerFactory.java:145)
> at
>
> org.apache.activemq.transport.tcp.TcpTransport.doStop(TcpTransport.java:538)
> at org.apache.activemq.util.ServiceSupport.stop(ServiceSupport.java:71)
> at
> org.apache.activemq.transport.tcp.TcpTransport.stop(TcpTransport.java:582)
> at
>
> org.apache.activemq.transport.AbstractInactivityMonitor.stop(AbstractInactivityMonitor.java:145)
> at
> org.apache.activemq.transport.TransportFilter.stop(TransportFilter.java:65)
> at
>
> org.apache.activemq.transport.WireFormatNegotiator.stop(WireFormatNegotiator.java:91)
> at org.apache.activemq.util.ServiceSupport.dispose(ServiceSupport.java:43)
> at
>
> org.apache.activemq.transport.failover.FailoverTransport.disposeTransport(FailoverTransport.java:240)
> at
>
> org.apache.activemq.transport.failover.FailoverTransport.handleTransportFailure(FailoverTransport.java:253)
> at
>
> org.apache.activemq.transport.failover.FailoverTransport$3.onException(FailoverTransport.java:215)
> at
>
> org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:101)
> at
>
> org.apache.activemq.transport.WireFormatNegotiator.onException(WireFormatNegotiator.java:160)
> at
>
> org.apache.activemq.transport.AbstractInactivityMonitor.onException(AbstractInactivityMonitor.java:314)
> at
>
> org.apache.activemq.transport.TransportSupport.onException(TransportSupport.java:96)
> at
> org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:205)
> at java.lang.Thread.run(Thread.java:745)
>
>
> I have checked http://activemq.apache.org/javalangoutofmemory.html but non
> of the mentioned condition is getting triggered. Any idea what can cause
> this on consumer side ?
>
> Thanks,
> Anuj
>
>
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/java-lang-OutOfMemoryError-in-consumer-application-tp4691751.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: "java.lang.OutOfMemoryError" in consumer application

artnaseef
Try taking a stack dump of the process once in a while and look at the list of threads.  There may be a leak, such as a connection leak, that needs to be addressed.  Using the stack dump, it should be easy to identify which threads are being created and sticking around for a long time.
Reply | Threaded
Open this post in threaded view
|

Re: "java.lang.OutOfMemoryError" in consumer application

Tim Bain
Keep in mind too that the native thread limit is for the user, counting all
processes running as that user on the box, so if you're doing other things
on that box as that user you can contribute to this problem.  A co-worker
ran one of our brokers out of threads after accidentally firing up a couple
copies of Eclipse as the ActiveMQ user instead of his own user, so if you
did anything like that at the time this happened, that might be the
explanation.
On Feb 19, 2015 6:44 PM, "artnaseef" <[hidden email]> wrote:

> Try taking a stack dump of the process once in a while and look at the list
> of threads.  There may be a leak, such as a connection leak, that needs to
> be addressed.  Using the stack dump, it should be easy to identify which
> threads are being created and sticking around for a long time.
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/java-lang-OutOfMemoryError-in-consumer-application-tp4691751p4691790.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: "java.lang.OutOfMemoryError" in consumer application

khandelwalanuj
Hi,

It checked it and it was because of the client application was running more processes then the ulimit configured.

Thanks,
Anuj