Artemis: Memory leak while shutting down the tomcat server with embedded JMS server

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Artemis: Memory leak while shutting down the tomcat server with embedded JMS server

vishal3007
This post was updated on .
I am using Artemis server as an embedded server in my webapp.
The Embedded server is started as spring-bean as explained in the Spring Integration examples provided by Artemis.

Here is the sample code: SpringMaventest.zip

When tomcat server is shut down following warning comes in console:

Mar 05, 2017 5:41:09 PM org.apache.catalina.core.StandardServer await
INFO: A valid shutdown command was received via the shutdown port. Stopping the Server instance.
Mar 05, 2017 5:41:09 PM org.apache.coyote.AbstractProtocol pause
INFO: Pausing ProtocolHandler ["http-nio-8080"]
Mar 05, 2017 5:41:09 PM org.apache.coyote.AbstractProtocol pause
INFO: Pausing ProtocolHandler ["ajp-nio-8009"]
Mar 05, 2017 5:41:09 PM org.apache.catalina.core.StandardService stopInternal
INFO: Stopping service Catalina
Mar 05, 2017 5:41:10 PM org.springframework.context.support.ClassPathXmlApplicationContext doClose
INFO: Closing org.springframework.context.support.ClassPathXmlApplicationContext@7feef2a0: startup date [Sun Mar 05 17:40:40 IST 2017]; root of context hierarchy
Mar 05, 2017 5:41:10 PM org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl stop
INFO: AMQ221002: Apache ActiveMQ Artemis Message Broker version 1.5.3 [59915ee6-019a-11e7-a49a-005056c00008] stopped, uptime 29.048 seconds
Mar 05, 2017 5:41:10 PM org.apache.catalina.loader.WebappClassLoaderBase clearReferencesThreads
WARNING: The web application [Spring4MVCHelloWorldRestServiceDemo] appears to have started a thread named [threadDeathWatcher-1-1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Thread.sleep(Native Method)
 io.netty.util.ThreadDeathWatcher$Watcher.run(ThreadDeathWatcher.java:150)
 io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
 java.lang.Thread.run(Thread.java:745)
Mar 05, 2017 5:41:10 PM org.apache.catalina.loader.WebappClassLoaderBase clearReferencesThreads
WARNING: The web application [Spring4MVCHelloWorldRestServiceDemo] appears to have started a thread named [Thread-0 (ActiveMQ-client-global-threads-1227208364)] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 sun.misc.Unsafe.park(Native Method)
 java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
 java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
 java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
 java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
 java.lang.Thread.run(Thread.java:745)
Mar 05, 2017 5:41:10 PM org.apache.catalina.loader.WebappClassLoaderBase checkThreadLocalMapForLeaks
SEVERE: The web application [Spring4MVCHelloWorldRestServiceDemo] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@198498b3]) and a value of type [io.netty.util.internal.InternalThreadLocalMap] (value [io.netty.util.internal.InternalThreadLocalMap@357b809a]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
Mar 05, 2017 5:41:10 PM org.apache.catalina.loader.WebappClassLoaderBase checkThreadLocalMapForLeaks
SEVERE: The web application [Spring4MVCHelloWorldRestServiceDemo] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@198498b3]) and a value of type [io.netty.util.internal.InternalThreadLocalMap] (value [io.netty.util.internal.InternalThreadLocalMap@26e9b16a]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
Mar 05, 2017 5:41:10 PM org.apache.coyote.AbstractProtocol stop
INFO: Stopping ProtocolHandler ["http-nio-8080"]
Mar 05, 2017 5:41:10 PM org.apache.coyote.AbstractProtocol stop
INFO: Stopping ProtocolHandler ["ajp-nio-8009"]
Mar 05, 2017 5:41:10 PM org.apache.coyote.AbstractProtocol destroy
INFO: Destroying ProtocolHandler ["http-nio-8080"]
Mar 05, 2017 5:41:10 PM org.apache.coyote.AbstractProtocol destroy
INFO: Destroying ProtocolHandler ["ajp-nio-8009"]
Reply | Threaded
Open this post in threaded view
|

Re: Artemis: Memory leak while shutting down the tomcat server with embedded JMS server

clebertsuconic
On Sun, Mar 5, 2017 at 7:08 AM, vishal3007 <[hidden email]> wrote:
> I am using Artemis server as an embedded server in my webapp.
> The Embedded server is started as spring-bean as explained in the Spring
> Integration examples provided by Artemis.

What integration example provided by Artemis? I don't recall anything
like that..


You're doing this integration yourself AFAIK? If you provide me some
more information on where you took that from, I can try helping out.
Reply | Threaded
Open this post in threaded view
|

Re: Artemis: Memory leak while shutting down the tomcat server with embedded JMS server

jbertram
There is an example called "spring-integration" shipped with Artemis that demonstrates how to boostrap an instance via Spring.  Of course, it doesn't use Tomcat.


Justin

----- Original Message -----
From: "Clebert Suconic" <[hidden email]>
To: [hidden email]
Sent: Sunday, March 5, 2017 3:16:50 PM
Subject: Re: Artemis: Memory leak while shutting down the tomcat server with embedded JMS server

On Sun, Mar 5, 2017 at 7:08 AM, vishal3007 <[hidden email]> wrote:
> I am using Artemis server as an embedded server in my webapp.
> The Embedded server is started as spring-bean as explained in the Spring
> Integration examples provided by Artemis.

What integration example provided by Artemis? I don't recall anything
like that..


You're doing this integration yourself AFAIK? If you provide me some
more information on where you took that from, I can try helping out.
Reply | Threaded
Open this post in threaded view
|

Re: Artemis: Memory leak while shutting down the tomcat server with embedded JMS server

jbertram
In reply to this post by vishal3007
The thread-pool from which the "ActiveMQ-client-global-threads-1227208364" thread comes is a static pool created for the instances of org.apache.activemq.artemis.api.core.client.ServerLocator in the JVM.  This pool is designed to hang around until the JVM is shutdown.  If you don't want this behavior and/or don't want to share a static pool between all your ServerLocator instances then you should set useGlobalPools to false either on the object itself or in the URL you provide it.  Aside from that you could just ignore the warning because it's a false positive IMO.


Justin

----- Original Message -----
From: "vishal3007" <[hidden email]>
To: [hidden email]
Sent: Sunday, March 5, 2017 6:08:29 AM
Subject: Artemis: Memory leak while shutting down the tomcat server with embedded JMS server

I am using Artemis server as an embedded server in my webapp.
The Embedded server is started as spring-bean as explained in the Spring
Integration examples provided by Artemis.
When tomcat server is shut down following warning comes in console:

Mar 05, 2017 5:41:09 PM org.apache.catalina.core.StandardServer await
INFO: A valid shutdown command was received via the shutdown port. Stopping
the Server instance.
Mar 05, 2017 5:41:09 PM org.apache.coyote.AbstractProtocol pause
INFO: Pausing ProtocolHandler ["http-nio-8080"]
Mar 05, 2017 5:41:09 PM org.apache.coyote.AbstractProtocol pause
INFO: Pausing ProtocolHandler ["ajp-nio-8009"]
Mar 05, 2017 5:41:09 PM org.apache.catalina.core.StandardService
stopInternal
INFO: Stopping service Catalina
Mar 05, 2017 5:41:10 PM
org.springframework.context.support.ClassPathXmlApplicationContext doClose
INFO: Closing
org.springframework.context.support.ClassPathXmlApplicationContext@7feef2a0:
startup date [Sun Mar 05 17:40:40 IST 2017]; root of context hierarchy
Mar 05, 2017 5:41:10 PM
org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl stop
INFO: AMQ221002: Apache ActiveMQ Artemis Message Broker version 1.5.3
[59915ee6-019a-11e7-a49a-005056c00008] stopped, uptime 29.048 seconds
Mar 05, 2017 5:41:10 PM org.apache.catalina.loader.WebappClassLoaderBase
clearReferencesThreads
WARNING: The web application [Spring4MVCHelloWorldRestServiceDemo] appears
to have started a thread named [threadDeathWatcher-1-1] but has failed to
stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Thread.sleep(Native Method)
 io.netty.util.ThreadDeathWatcher$Watcher.run(ThreadDeathWatcher.java:150)

io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
 java.lang.Thread.run(Thread.java:745)
Mar 05, 2017 5:41:10 PM org.apache.catalina.loader.WebappClassLoaderBase
clearReferencesThreads
WARNING: The web application [Spring4MVCHelloWorldRestServiceDemo] appears
to have started a thread named [Thread-0
(ActiveMQ-client-global-threads-1227208364)] but has failed to stop it. This
is very likely to create a memory leak. Stack trace of thread:
 sun.misc.Unsafe.park(Native Method)
 java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)

java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
 java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)

java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)

java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)

java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
 java.lang.Thread.run(Thread.java:745)
Mar 05, 2017 5:41:10 PM org.apache.catalina.loader.WebappClassLoaderBase
checkThreadLocalMapForLeaks
SEVERE: The web application [Spring4MVCHelloWorldRestServiceDemo] created a
ThreadLocal with key of type [java.lang.ThreadLocal] (value
[java.lang.ThreadLocal@198498b3]) and a value of type
[io.netty.util.internal.InternalThreadLocalMap] (value
[io.netty.util.internal.InternalThreadLocalMap@357b809a]) but failed to
remove it when the web application was stopped. Threads are going to be
renewed over time to try and avoid a probable memory leak.
Mar 05, 2017 5:41:10 PM org.apache.catalina.loader.WebappClassLoaderBase
checkThreadLocalMapForLeaks
SEVERE: The web application [Spring4MVCHelloWorldRestServiceDemo] created a
ThreadLocal with key of type [java.lang.ThreadLocal] (value
[java.lang.ThreadLocal@198498b3]) and a value of type
[io.netty.util.internal.InternalThreadLocalMap] (value
[io.netty.util.internal.InternalThreadLocalMap@26e9b16a]) but failed to
remove it when the web application was stopped. Threads are going to be
renewed over time to try and avoid a probable memory leak.
Mar 05, 2017 5:41:10 PM org.apache.coyote.AbstractProtocol stop
INFO: Stopping ProtocolHandler ["http-nio-8080"]
Mar 05, 2017 5:41:10 PM org.apache.coyote.AbstractProtocol stop
INFO: Stopping ProtocolHandler ["ajp-nio-8009"]
Mar 05, 2017 5:41:10 PM org.apache.coyote.AbstractProtocol destroy
INFO: Destroying ProtocolHandler ["http-nio-8080"]
Mar 05, 2017 5:41:10 PM org.apache.coyote.AbstractProtocol destroy
INFO: Destroying ProtocolHandler ["ajp-nio-8009"]




--
View this message in context: http://activemq.2283324.n4.nabble.com/Artemis-Memory-leak-while-shutting-down-the-tomcat-server-with-embedded-JMS-server-tp4722906.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

Re: Artemis: Memory leak while shutting down the tomcat server with embedded JMS server

vishal3007
Thanks for the reply...If there is no memory leak in case of Tomcat shutdown/restart then I am ok with it...BTW can you please point out the sample code\documentation to disable the global thread pools?

Note: I am using broker.xml to setup the server and looked at the https://activemq.apache.org/artemis/docs/1.5.3/thread-pooling.html but this does not explain how to configure this in case of broker.xml
Reply | Threaded
Open this post in threaded view
|

Re: Artemis: Memory leak while shutting down the tomcat server with embedded JMS server

nigro_franz
Hi!!

Agreed with Justin about not being a real issue..
Check the last comment here: https://github.com/netty/netty/issues/2522
You could try to put a FastThreadLocal.destroy() on the application exit.
Anyway, there are good chances that the memory leak protection (https://wiki.apache.org/tomcat/MemoryLeakProtection) of Tomcat will recognize it as a memory leak because you'll need a complete GC cycle to get rid of the thead local instances.