[activemq-user] [4.0-M2] Lots race condition exceptions during shutdown

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[activemq-user] [4.0-M2] Lots race condition exceptions during shutdown

memelet
I'm getting gobs of exceptions when shutting down the broker.

The context: The broker and some lingo beans are being created in the
same spring context. The broker is not embedded, but rather explicitly
created. The lingo beans dispose first, and close cleanly. But when the
broker goes to shutdown, the tcp-transport is quite upset because its
socket is closed.

The first SocketException exception below occurs even if there are not
clients (AMQ-425).

But the remaining exception only occur when the lingo beans are
created/connected and then disposed/closed.

Note, that the system behaves correctly prior to shutdown. It appears
that its race conditions during shutdown that are causing these exceptions.

I will file an issue, but first want to see if anyone can help to
isolate the issue a bit further.

-barry

----

20:35:20,878 INFO-org.activemq.broker.BrokerService - ActiveMQ Message
Broker (localhost) is shutting down [][ActiveMQ
ShutdownHook][org.activemq.broker.BrokerService]
20:35:20,878 DEBUG-org.activemq.broker.BrokerService - Caught exception,
must be shutting down: java.lang.IllegalStateException: Shutdown in
progress [][ActiveMQ ShutdownHook][org.activemq.broker.BrokerService]
20:35:20,894 ERROR-org.activemq.broker.TransportConnector - Could not
accept connection: java.net.SocketException: socket closed
[][TcpTransportServer@tcp://nano:61710][org.activemq.broker.TransportConnector]
java.net.SocketException: socket closed
    at java.net.PlainSocketImpl.socketAccept(Native Method)
    at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
    at java.net.ServerSocket.implAccept(ServerSocket.java:450)
    at java.net.ServerSocket.accept(ServerSocket.java:421)
    at
org.activemq.transport.tcp.TcpTransportServer.run(TcpTransportServer.java:95)
    at java.lang.Thread.run(Thread.java:595)
20:35:20,894 WARN-org.activemq.ra.ActiveMQManagedConnection - Connection
failed: javax.jms.JMSException: java.io.EOFException [][TcpTransport:
Socket[addr=localhost/127.0.0.1,port=61710,localport=1093]][org.activemq.ra.ActiveMQManagedConnection]
20:35:20,894 DEBUG-org.activemq.ra.ActiveMQManagedConnection - Cause:  
[][TcpTransport:
Socket[addr=localhost/127.0.0.1,port=61710,localport=1093]][org.activemq.ra.ActiveMQManagedConnection]
javax.jms.JMSException: java.io.EOFException
    at
org.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:44)
    at
org.activemq.ActiveMQConnection.onAsyncException(ActiveMQConnection.java:1289)
    at
org.activemq.ActiveMQConnection.onException(ActiveMQConnection.java:1298)
    at
org.activemq.transport.TransportFilter.onException(TransportFilter.java:104)
    at
org.activemq.transport.TransportFilter.onException(TransportFilter.java:104)
    at
org.activemq.transport.TransportFilter.onException(TransportFilter.java:104)
    at
org.activemq.transport.TransportSupport.onException(TransportSupport.java:91)
    at org.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:148)
    at java.lang.Thread.run(Thread.java:595)
Caused by: java.io.EOFException
    at java.io.DataInputStream.readInt(DataInputStream.java:358)
    at
org.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:182)
    at org.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:134)
    ... 1 more
20:35:20,894 DEBUG-org.activemq.broker.AbstractConnection - Transport
failed: java.net.SocketException: Socket closed [][TcpTransport:
Socket[addr=/127.0.0.1,port=1093,localport=61710]][org.activemq.broker.AbstractConnection]
java.net.SocketException: Socket closed
    at java.net.SocketInputStream.read(SocketInputStream.java:162)
    at
org.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:47)
    at
org.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:54)
    at java.io.DataInputStream.readInt(DataInputStream.java:353)
    at
org.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:182)
    at org.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:134)
    at java.lang.Thread.run(Thread.java:595)
20:35:20,894 WARN-org.activemq.ra.SimpleConnectionManager - Managed
connection experiened an error:  [][TcpTransport:
Socket[addr=localhost/127.0.0.1,port=61710,localport=1093]][org.activemq.ra.SimpleConnectionManager]
javax.jms.JMSException: java.io.EOFException
    at
org.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:44)
    at
org.activemq.ActiveMQConnection.onAsyncException(ActiveMQConnection.java:1289)
    at
org.activemq.ActiveMQConnection.onException(ActiveMQConnection.java:1298)
    at
org.activemq.transport.TransportFilter.onException(TransportFilter.java:104)
    at
org.activemq.transport.TransportFilter.onException(TransportFilter.java:104)
    at
org.activemq.transport.TransportFilter.onException(TransportFilter.java:104)
    at
org.activemq.transport.TransportSupport.onException(TransportSupport.java:91)
    at org.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:148)
    at java.lang.Thread.run(Thread.java:595)
Caused by: java.io.EOFException
    at java.io.DataInputStream.readInt(DataInputStream.java:358)
    at
org.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:182)
    at org.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:134)
    ... 1 more
20:35:20,909 DEBUG-org.activemq.broker.AbstractConnection - Transport
failed: java.net.SocketException: Socket closed [][TcpTransport:
Socket[addr=/127.0.0.1,port=1093,localport=61710]][org.activemq.broker.AbstractConnection]
java.net.SocketException: Socket closed
    at java.net.SocketInputStream.read(SocketInputStream.java:162)
    at
org.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:47)
    at
org.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:54)
    at java.io.DataInputStream.readInt(DataInputStream.java:353)
    at
org.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:182)
    at org.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:134)
    at java.lang.Thread.run(Thread.java:595)
20:35:20,909 WARN-org.activemq.ra.SimpleConnectionManager - Error
occured during the cleanup of a managed connection:  [][TcpTransport:
Socket[addr=localhost/127.0.0.1,port=61710,localport=1093]][org.activemq.ra.SimpleConnectionManager]
Could cleanup the ActiveMQ connection: javax.jms.JMSException: Cannot
write to the stream any more it has already been closed
    at
org.activemq.ra.ActiveMQManagedConnection.cleanup(ActiveMQManagedConnection.java:257)
    at
org.activemq.ra.SimpleConnectionManager.connectionErrorOccurred(SimpleConnectionManager.java:98)
    at
org.activemq.ra.ActiveMQManagedConnection.fireErrorOccurredEvent(ActiveMQManagedConnection.java:193)
    at
org.activemq.ra.ActiveMQManagedConnection.onException(ActiveMQManagedConnection.java:411)
    at
org.activemq.ActiveMQConnection.onAsyncException(ActiveMQConnection.java:1290)
    at
org.activemq.ActiveMQConnection.onException(ActiveMQConnection.java:1298)
    at
org.activemq.transport.TransportFilter.onException(TransportFilter.java:104)
    at
org.activemq.transport.TransportFilter.onException(TransportFilter.java:104)
    at
org.activemq.transport.TransportFilter.onException(TransportFilter.java:104)
    at
org.activemq.transport.TransportSupport.onException(TransportSupport.java:91)
    at org.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:148)
    at java.lang.Thread.run(Thread.java:595)
Caused by: javax.jms.JMSException: Cannot write to the stream any more
it has already been closed
    at
org.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:53)
    at
org.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1036)
    at org.activemq.ActiveMQConnection.cleanup(ActiveMQConnection.java:1171)
    at
org.activemq.ra.ActiveMQManagedConnection.cleanup(ActiveMQManagedConnection.java:255)
    ... 11 more
Caused by: java.io.EOFException: Cannot write to the stream any more it
has already been closed
    at
org.activemq.transport.tcp.TcpBufferedOutputStream.checkClosed(TcpBufferedOutputStream.java:130)
    at
org.activemq.transport.tcp.TcpBufferedOutputStream.write(TcpBufferedOutputStream.java:68)
    at java.io.DataOutputStream.writeInt(DataOutputStream.java:180)
    at org.activemq.openwire.OpenWireFormat.marshal(OpenWireFormat.java:171)
    at org.activemq.transport.tcp.TcpTransport.oneway(TcpTransport.java:116)
    at
org.activemq.transport.TransportFilter.oneway(TransportFilter.java:92)
    at
org.activemq.transport.WireFormatNegotiator.oneway(WireFormatNegotiator.java:67)
    at org.activemq.transport.MutexTransport.oneway(MutexTransport.java:46)
    at
org.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:56)
    at
org.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1034)
    ... 13 more
20:35:20,909 WARN-org.activemq.broker.jmx.ManagedTransportConnector -
Failed to unregister mbean:
org.activemq:Type=Connection,BrokerName=localhost,Connection=2
[][TcpTransport:
Socket[addr=/127.0.0.1,port=1093,localport=61710]][org.activemq.broker.jmx.ManagedTransportConnector]
20:35:20,909 WARN-org.activemq.ra.SimpleConnectionManager - Error
occured during the destruction of a managed connection:  
[][TcpTransport:
Socket[addr=localhost/127.0.0.1,port=61710,localport=1093]][org.activemq.ra.SimpleConnectionManager]
Could cleanup the ActiveMQ connection: javax.jms.JMSException: Cannot
write to the stream any more it has already been closed
    at
org.activemq.ra.ActiveMQManagedConnection.cleanup(ActiveMQManagedConnection.java:257)
    at
org.activemq.ra.ActiveMQManagedConnection.destroy(ActiveMQManagedConnection.java:223)
    at
org.activemq.ra.SimpleConnectionManager.connectionErrorOccurred(SimpleConnectionManager.java:104)
    at
org.activemq.ra.ActiveMQManagedConnection.fireErrorOccurredEvent(ActiveMQManagedConnection.java:193)
    at
org.activemq.ra.ActiveMQManagedConnection.onException(ActiveMQManagedConnection.java:411)
    at
org.activemq.ActiveMQConnection.onAsyncException(ActiveMQConnection.java:1290)
    at
org.activemq.ActiveMQConnection.onException(ActiveMQConnection.java:1298)
    at
org.activemq.transport.TransportFilter.onException(TransportFilter.java:104)
    at
org.activemq.transport.TransportFilter.onException(TransportFilter.java:104)
    at
org.activemq.transport.TransportFilter.onException(TransportFilter.java:104)
    at
org.activemq.transport.TransportSupport.onException(TransportSupport.java:91)
    at org.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:148)
    at java.lang.Thread.run(Thread.java:595)
Caused by: javax.jms.JMSException: Cannot write to the stream any more
it has already been closed
    at
org.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:53)
    at
org.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1036)
    at org.activemq.ActiveMQConnection.cleanup(ActiveMQConnection.java:1171)
    at
org.activemq.ra.ActiveMQManagedConnection.cleanup(ActiveMQManagedConnection.java:255)
    ... 12 more
Caused by: java.io.EOFException: Cannot write to the stream any more it
has already been closed
    at
org.activemq.transport.tcp.TcpBufferedOutputStream.checkClosed(TcpBufferedOutputStream.java:130)
    at
org.activemq.transport.tcp.TcpBufferedOutputStream.write(TcpBufferedOutputStream.java:68)
    at java.io.DataOutputStream.writeInt(DataOutputStream.java:180)
    at org.activemq.openwire.OpenWireFormat.marshal(OpenWireFormat.java:171)
    at org.activemq.transport.tcp.TcpTransport.oneway(TcpTransport.java:116)
    at
org.activemq.transport.TransportFilter.oneway(TransportFilter.java:92)
    at
org.activemq.transport.WireFormatNegotiator.oneway(WireFormatNegotiator.java:67)
    at org.activemq.transport.MutexTransport.oneway(MutexTransport.java:46)
    at
org.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:56)
    at
org.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1034)
    ... 14 more
20:35:20,925 INFO-org.activemq.broker.BrokerService - ActiveMQ JMS
Message Broker (localhost) stopped [][ActiveMQ
ShutdownHook][org.activemq.broker.BrokerService]

--

barry kaplan
[hidden email]