deadlock condition caused by ExceptionListener?

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

deadlock condition caused by ExceptionListener?

danfoygel
We seem to be running into a deadlock condition caused by an ExceptionListener

We use a SingleConnectionFactory with reconnectOnException set to true.  When an exception happens, the thread that's created to deal with it appears to get deadlocked, like so:

          at org.apache.activemq.MessageDispatchChannel.stop(MessageDispatchChannel.java:115)
          at org.apache.activemq.ActiveMQMessageConsumer.stop(ActiveMQMessageConsumer.java:1073)
          at org.apache.activemq.ActiveMQSession.stop(ActiveMQSession.java:1550)
          at org.apache.activemq.ActiveMQConnection.stop(ActiveMQConnection.java:534)
          at org.springframework.jms.connection.SingleConnectionFactory.closeConnection(SingleConnectionFactory.java:423)
          at org.springframework.jms.connection.SingleConnectionFactory.resetConnection(SingleConnectionFactory.java:322)
          at org.springframework.jms.connection.SingleConnectionFactory.onException(SingleConnectionFactory.java:303)
          at org.springframework.jms.connection.ChainedExceptionListener.onException(ChainedExceptionListener.java:60)
          at org.apache.activemq.ActiveMQConnection$4.run(ActiveMQConnection.java:1779)
          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
          at java.lang.Thread.run(Thread.java:619)

It appears to be waiting on a mutex for the MessageDispatchChannel - however, since it's a different thread, I don't understand why it should be assumed that it can get that mutex, since the consumer associated with that channel may be holding it.

The ExceptionListener thread appears to lock up the entire connection by synchronizing on the connectionMonitor in SingleConnectionFactory, which basically causes all other JMS threads to block when trying to send or receive.  In the application, this condition appears to be unrecoverable.

I suspect that something is misconfigured on our side, but I can't figure out what - can anyone help?

We're using ActiveMQ 5.2.0 and Java 1.6.0_16.  Not sure if any other configuration factors matter, but happy to provide.

Thanks!