Do you know when ActiveMQ Session threads die ?

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Do you know when ActiveMQ Session threads die ?

Eric-AWL
This post was updated on .
Hi

in 5.3-snapshot and 5.2 release, I think I correctly close consumers/producers/session and the threads like this one :

ActiveMQ Session: ID:td0sib01s-32880-1245239727292-2:0:25

don't seem to die.

Is there something to do ?


Note : I noticed that the number of subscriptions on ActiveMQ.advisory.Consumer.Queue.XXXX, for the 2 queues I use, increases as consumers are created and closed. But they don't decrease when consumers and sessions are closed. It is the broker in which these consumers are created that seems to be the client for these advisory queues. (I am in a network of broker topology).

Number of subscriptions for these queues are OK.


Eric-AWL
Reply | Threaded
Open this post in threaded view
|

Re: Do you know when ActiveMQ Session threads dies ?

gtully
I wonder are you experiencing
https://issues.apache.org/activemq/browse/AMQ-2169

2009/6/17 Eric-AWL <[hidden email]>

>
> Hi
>
> in 5.3-snapshot and 5.2 release, I think I correctly close
> consumers/producers/session and the threads like this one :
>
> ActiveMQ Session: ID:td0sib01s-32880-1245239727292-2:0:25
>
> doesn't seem to die.
>
> Is there something to do ?
>
> Eric-AWL
> --
> View this message in context:
> http://www.nabble.com/Do-you-know-when-ActiveMQ-Session-threads-dies---tp24071802p24071802.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>


--
http://blog.garytully.com

Open Source Integration
http://fusesource.com
Reply | Threaded
Open this post in threaded view
|

Re: Do you know when ActiveMQ Session threads dies ?

Eric-AWL
Hi !

No, I don't use Web console at all. But I use JMX console.

Do you know the conditions required for a thread session to terminate ? Should session.close() be enough or have we to wait for some kind of timeout (Inactivity) before the thread die ? ... I extensively use network of brokers too (in relatively complex configurations), and it is possible that some related consumers/producers continue to be active on other nodes.

I am in a full AUTO_ACKNOWLEDGEMENT configuration.


If you explain me when a session thread is supposed to die, I will try to find where is located the error.



Gary Tully wrote
I wonder are you experiencing
https://issues.apache.org/activemq/browse/AMQ-2169

2009/6/17 Eric-AWL <eric.vincent@atosorigin.com>

>
> Hi
>
> in 5.3-snapshot and 5.2 release, I think I correctly close
> consumers/producers/session and the threads like this one :
>
> ActiveMQ Session: ID:td0sib01s-32880-1245239727292-2:0:25
>
> doesn't seem to die.
>
> Is there something to do ?
>
> Eric-AWL
> --
> View this message in context:
> http://www.nabble.com/Do-you-know-when-ActiveMQ-Session-threads-dies---tp24071802p24071802.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>


--
http://blog.garytully.com

Open Source Integration
http://fusesource.com
Reply | Threaded
Open this post in threaded view
|

Re: Do you know when ActiveMQ Session threads dies ?

gtully
session.close should do it!

2009/6/19 Eric-AWL <[hidden email]>

>
> Hi !
>
> No, I don't use Web console at all. But I use JMX console.
>
> Do you know the conditions required for a thread session to terminate ?
> Should session.close() be enough or have we to wait for some kind of
> timeout
> (Inactivity) before the thread die ? ... I extensively use network of
> brokers too (in relatively complex configurations), and it is possible that
> some related consumers/producers continue to be active on other nodes.
>
> I am in a full AUTO_ACKNOWLEDGEMENT configuration.
>
>
> If you explain me when a session thread is supposed to die, I will try to
> find where is located the error.
>
>
>
>
> Gary Tully wrote:
> >
> > I wonder are you experiencing
> > https://issues.apache.org/activemq/browse/AMQ-2169
> >
> > 2009/6/17 Eric-AWL <[hidden email]>
> >
> >>
> >> Hi
> >>
> >> in 5.3-snapshot and 5.2 release, I think I correctly close
> >> consumers/producers/session and the threads like this one :
> >>
> >> ActiveMQ Session: ID:td0sib01s-32880-1245239727292-2:0:25
> >>
> >> doesn't seem to die.
> >>
> >> Is there something to do ?
> >>
> >> Eric-AWL
> >> --
> >> View this message in context:
> >>
> http://www.nabble.com/Do-you-know-when-ActiveMQ-Session-threads-dies---tp24071802p24071802.html
> >> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> >>
> >>
> >
> >
> > --
> > http://blog.garytully.com
> >
> > Open Source Integration
> > http://fusesource.com
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/Do-you-know-when-ActiveMQ-Session-threads-die---tp24071802p24109839.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>


--
http://blog.garytully.com

Open Source Integration
http://fusesource.com
Reply | Threaded
Open this post in threaded view
|

Re: Do you know when ActiveMQ Session threads dies ?

Eric-AWL
I think that I have some news.

The problem is more complex than I wrote initially (I wanted to be clear).

In fact, I have two ActiveMQ connections initialized in the same "main" JVM. One with an embedded broker, the second with a distant broker embedded in an "other" JVM. Some threads didn't die when I want to shutdown my "main" JVM in this condition :
- I stop the "other" JVM gracefully.
- OnException is called on the "main JVM", but I don't close sessions nor close this connection on the main JVM, since I think that this is done automatically.
- I want to stop the "main" JVM by closing session on the embedded connection, closing connection, and shutdown the embedded broker.

It is in this configuration that some sessions are not closed and my process doesn't want to stop gracefully.

I think that threads that don't die are finally affected to the broken connection.
I can see that "ActiveMQ Connection Worker" thread associated with my broken connection is always active.

What is the best way to deal with this problem ?

Thank you
Eric-AWL



Gary Tully wrote
session.close should do it!

2009/6/19 Eric-AWL <eric.vincent@atosorigin.com>

>
> Hi !
>
> No, I don't use Web console at all. But I use JMX console.
>
> Do you know the conditions required for a thread session to terminate ?
> Should session.close() be enough or have we to wait for some kind of
> timeout
> (Inactivity) before the thread die ? ... I extensively use network of
> brokers too (in relatively complex configurations), and it is possible that
> some related consumers/producers continue to be active on other nodes.
>
> I am in a full AUTO_ACKNOWLEDGEMENT configuration.
>
>
> If you explain me when a session thread is supposed to die, I will try to
> find where is located the error.
>
>
>
>
> Gary Tully wrote:
> >
> > I wonder are you experiencing
> > https://issues.apache.org/activemq/browse/AMQ-2169
> >
> > 2009/6/17 Eric-AWL <eric.vincent@atosorigin.com>
> >
> >>
> >> Hi
> >>
> >> in 5.3-snapshot and 5.2 release, I think I correctly close
> >> consumers/producers/session and the threads like this one :
> >>
> >> ActiveMQ Session: ID:td0sib01s-32880-1245239727292-2:0:25
> >>
> >> doesn't seem to die.
> >>
> >> Is there something to do ?
> >>
> >> Eric-AWL
> >> --
> >> View this message in context:
> >>
> http://www.nabble.com/Do-you-know-when-ActiveMQ-Session-threads-dies---tp24071802p24071802.html
> >> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> >>
> >>
> >
> >
> > --
> > http://blog.garytully.com
> >
> > Open Source Integration
> > http://fusesource.com
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/Do-you-know-when-ActiveMQ-Session-threads-die---tp24071802p24109839.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>


--
http://blog.garytully.com

Open Source Integration
http://fusesource.com
Reply | Threaded
Open this post in threaded view
|

connection.close(), on a brokenConnection resolves my problem

Eric-AWL
NEW

If I force a brokenConnection.close() into the onException callback, (called when the "other" connection is broken), my process can now terminate gracefully and is not blocked waiting threads die.

However there is a JMSException thrown by this "close" on a brokenConnection. But it is not a big problem.

I think it is not the way ActiveMQ should work.


Eric-AWL

Eric-AWL wrote
I think that I have some news.

The problem is more complex than I wrote initially (I wanted to be clear).

In fact, I have two ActiveMQ connections initialized in the same "main" JVM. One with an embedded broker, the second with a distant broker embedded in an "other" JVM. Some threads didn't die when I want to shutdown my "main" JVM in this condition :
- I stop the "other" JVM gracefully.
- OnException is called on the "main JVM", but I don't close sessions nor close this connection on the main JVM, since I think that this is done automatically.
- I want to stop the "main" JVM by closing session on the embedded connection, closing connection, and shutdown the embedded broker.

It is in this configuration that some sessions are not closed and my process doesn't want to stop gracefully.

I think that threads that don't die are finally affected to the broken connection.
I can see that "ActiveMQ Connection Worker" thread associated with my broken connection is always active.

What is the best way to deal with this problem ?

Thank you
Eric-AWL



Gary Tully wrote
session.close should do it!

2009/6/19 Eric-AWL <eric.vincent@atosorigin.com>

>
> Hi !
>
> No, I don't use Web console at all. But I use JMX console.
>
> Do you know the conditions required for a thread session to terminate ?
> Should session.close() be enough or have we to wait for some kind of
> timeout
> (Inactivity) before the thread die ? ... I extensively use network of
> brokers too (in relatively complex configurations), and it is possible that
> some related consumers/producers continue to be active on other nodes.
>
> I am in a full AUTO_ACKNOWLEDGEMENT configuration.
>
>
> If you explain me when a session thread is supposed to die, I will try to
> find where is located the error.
>
>
>
>
> Gary Tully wrote:
> >
> > I wonder are you experiencing
> > https://issues.apache.org/activemq/browse/AMQ-2169
> >
> > 2009/6/17 Eric-AWL <eric.vincent@atosorigin.com>
> >
> >>
> >> Hi
> >>
> >> in 5.3-snapshot and 5.2 release, I think I correctly close
> >> consumers/producers/session and the threads like this one :
> >>
> >> ActiveMQ Session: ID:td0sib01s-32880-1245239727292-2:0:25
> >>
> >> doesn't seem to die.
> >>
> >> Is there something to do ?
> >>
> >> Eric-AWL
> >> --
> >> View this message in context:
> >>
> http://www.nabble.com/Do-you-know-when-ActiveMQ-Session-threads-dies---tp24071802p24071802.html
> >> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> >>
> >>
> >
> >
> > --
> > http://blog.garytully.com
> >
> > Open Source Integration
> > http://fusesource.com
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/Do-you-know-when-ActiveMQ-Session-threads-die---tp24071802p24109839.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>


--
http://blog.garytully.com

Open Source Integration
http://fusesource.com
Reply | Threaded
Open this post in threaded view
|

Re: connection.close(), on a brokenConnection resolves my problem

Eric-AWL
Unfortunately, all my problems are not resolved ... I tried with two equivalent processes on two different servers (and the third which is abruptly closed). One terminated gracefully, the other had always 2 active threads

"ActiveMQ Session: ID:td0sib01s.priv.atos.fr-46825-1246630520929-0:2:3" prio=10 tid=0x000000000a52d000 nid=0x6618 in Object.wait() [0x00000000468bf000..0x00000000468bfc10]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00002aaab3e73ae8> (a java.lang.Object)
        at java.lang.Object.wait(Object.java:502)
        at org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:105)
        - locked <0x00002aaab3e73ae8> (a java.lang.Object)
        at org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:36)

"Multicast Discovery Agent Notifier" daemon prio=10 tid=0x000000000a82dc00 nid=0x6586 waiting on condition [0x0000000043188000..0x0000000043188b90]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00002aaab3cf20a8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1978)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:386)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:636)

..... :-(


Eric-AWL wrote
NEW

If I force a brokenConnection.close() into the onException callback, (called when the "other" connection is broken), my process can now terminate gracefully and is not blocked waiting threads die.

However there is a JMSException thrown by this "close" on a brokenConnection. But it is not a big problem.

I think it is not the way ActiveMQ should work.


Eric-AWL
Reply | Threaded
Open this post in threaded view
|

Re: connection.close(), on a brokenConnection resolves my problem

Eric-AWL
I think this session is associated to the network of broker topology between (multicast) my two "equivalent processes".

I think I have several problems.

However Close() seems to let the process in a cleaner situation. Here it is an other problem where "multicast" network of brokers are broken when several processes in the same network of brokers try to terminate simultaneously. I will try not to stop them simultaneously and see if the problem appears when I want the second process to die gracefully.

And the number of consumers which increases is perhaps the AMQ-2109 which is resolved in 5.3.
Reply | Threaded
Open this post in threaded view
|

Re: connection.close(), on a brokenConnection resolves my problem

Eric-AWL
"Multicast Discovery Agent Notifier" daemon prio=10 tid=0x000000000a82dc00 nid=0x6586 waiting on condition [0x0000000043188000..0x0000000043188b90]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00002aaab3cf20a8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1978)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:386)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:636)


It seems there is not any executor.shutdown() call in MulticastDiscoveryAgent Class .... Perhaps, it could be useful to add such a call in the stop() method ...