Either Producer or consumer hanging after a while

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Either Producer or consumer hanging after a while

RAJIV
Hi Team,

I am using a application where in producer , consumer both uses ActiveMQ broker..
Problem is after a while either producer or consumer is hanging not processing any messages,... SO i had to restart broker, producer program and consumer program as well , then only it is working...

Because of this issue , i have been doing multiple restart very often..

It is exactly hanging in the line where session is being created ... Below is the line ..
session = connection.createQueueSession( false, Session.AUTO_ACKNOWLEDGE );


In my Log these below are the DEBUG messages comes in,..
 I could not figure out anything.. Any have any idea ,Plese help.


DEBUG | Queue created: dynamicQueues/archive1mq?consumer.prefetchSize=2000
DEBUG | Initialized TaskRunnerFactory[ActiveMQ Task] using ExecutorService: java.util.concurrent.ThreadPoolExecutor@4e5f150f[Running, pool siz
e = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
DEBUG | Reconnect was triggered but transport is not started yet. Wait for start to connect the transport.
DEBUG | Started unconnected
DEBUG | Waking up reconnect task
DEBUG | Connection created in ApexServiceAgentArchiveJMSThread
DEBUG | urlList connectionList:[tcp://ACTIVEMQHOST2:5001, tcp://ACTIVEMQHOST1:5001], from: [tcp://ACTIVEMQHOST1:5001, tcp://ACTIVEMQHOST2:5001
]
DEBUG | Attempting 0th connect to: tcp://ACTIVEMQHOST2:5001
DEBUG | Connect fail to: tcp://ACTIVEMQHOST2:5001, reason: java.net.ConnectException: Connection refused
DEBUG | Stopping transport tcp://ACTIVEMQHOST2:5001
DEBUG | Initialized TaskRunnerFactory[ActiveMQ Task] using ExecutorService: java.util.concurrent.ThreadPoolExecutor@37de9bd8[Running, pool siz
e = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
DEBUG | Closed socket Socket[unconnected]
DEBUG | Forcing shutdown of ExecutorService: java.util.concurrent.ThreadPoolExecutor@37de9bd8[Running, pool size = 1, active threads = 0, queu
ed tasks = 0, completed tasks = 1]
DEBUG | Attempting 0th connect to: tcp://ACTIVEMQHOST1:5001
DEBUG | Sending: WireFormatInfo { version=11, properties={TcpNoDelayEnabled=true, SizePrefixDisabled=false, CacheSize=1024, StackTraceEnabled=
true, CacheEnabled=true, TightEncodingEnabled=true, MaxFrameSize=9223372036854775807, MaxInactivityDuration=30000, MaxInactivityDurationInital
Delay=10000}, magic=[A,c,t,i,v,e,M,Q]}
DEBUG | Connection established
 INFO | Successfully connected to tcp://ACTIVEMQHOST1:5001
DEBUG | Stopping transport tcp://ACTIVEMQHOST1/10.129.66.129:5001@60323
DEBUG | Initialized TaskRunnerFactory[ActiveMQ Task] using ExecutorService: java.util.concurrent.ThreadPoolExecutor@12eb3770[Running, pool siz
e = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
DEBUG | Closed socket Socket[addr=ACTIVEMQHOST1/10.129.66.129,port=5001,localport=60323]
DEBUG | Forcing shutdown of ExecutorService: java.util.concurrent.ThreadPoolExecutor@12eb3770[Running, pool size = 1, active threads = 0, queu
ed tasks = 0, completed tasks = 1]
DEBUG | Send oneway attempt: 0 failed for command: ConnectionInfo {commandId = 1, responseRequired = true, connectionId = ID:sjc3-vg4-apexv02a-
60312-1486017164856-1:1, clientId = ID:sjc3-vg4-apexv02a-60312-1486017164856-0:1, clientIp = null, userName = fdc, password = *****, brokerPath
 = null, brokerMasterConnector = false, manageable = true, clientMaster = true, faultTolerant = true, failoverReconnect = false}
 WARN | Transport (tcp://ACTIVEMQHOST1:5001) failed , attempting to automatically reconnect: java.io.EOFException
DEBUG | urlList connectionList:[tcp://ACTIVEMQHOST2:5001, tcp://ACTIVEMQHOST1:5001], from: [tcp://ACTIVEMQHOST1:5001, tcp://ACTIVEMQHOST2:5001
]
DEBUG | Waiting 10 ms before attempting connection
DEBUG | Attempting 0th connect to: tcp://ACTIVEMQHOST2:5001
DEBUG | Connect fail to: tcp://ACTIVEMQHOST2:5001, reason: java.net.ConnectException: Connection refused
DEBUG | Stopping transport tcp://ACTIVEMQHOST2:5001
DEBUG | Initialized TaskRunnerFactory[ActiveMQ Task] using ExecutorService: java.util.concurrent.ThreadPoolExecutor@63f2789e[Running, pool siz
e = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
DEBUG | Closed socket Socket[unconnected]
DEBUG | Forcing shutdown of ExecutorService: java.util.concurrent.ThreadPoolExecutor@63f2789e[Running, pool size = 1, active threads = 1, queu
ed tasks = 0, completed tasks = 0]
DEBUG | Attempting 0th connect to: tcp://ACTIVEMQHOST1:5001
DEBUG | Sending: WireFormatInfo { version=11, properties={TcpNoDelayEnabled=true, SizePrefixDisabled=false, CacheSize=1024, StackTraceEnabled=
true, CacheEnabled=true, TightEncodingEnabled=true, MaxFrameSize=9223372036854775807, MaxInactivityDuration=30000, MaxInactivityDurationInital
Delay=10000}, magic=[A,c,t,i,v,e,M,Q]}
DEBUG | Connect fail to: tcp://ACTIVEMQHOST1:5001, reason: org.apache.activemq.transport.InactivityIOException: Cannot send, channel has alrea
dy failed: tcp://10.129.66.129:5001
DEBUG | Stopping transport tcp://ACTIVEMQHOST1/10.129.66.129:5001@60329
DEBUG | Initialized TaskRunnerFactory[ActiveMQ Task] using ExecutorService: java.util.concurrent.ThreadPoolExecutor@5cce2b3c[Running, pool siz
e = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
DEBUG | Closed socket Socket[addr=ACTIVEMQHOST1/10.129.66.129,port=5001,localport=60329]
DEBUG | Forcing shutdown of ExecutorService: java.util.concurrent.ThreadPoolExecutor@5cce2b3c[Running, pool size = 1, active threads = 0, queu
ed tasks = 0, completed tasks = 1]
DEBUG | Waiting 20 ms before attempting connection
DEBUG | urlList connectionList:[tcp://ACTIVEMQHOST2:5001, tcp://ACTIVEMQHOST1:5001], from: [tcp://ACTIVEMQHOST1:5001, tcp://ACTIVEMQHOST2:5001
]
Reply | Threaded
Open this post in threaded view
|

Re: Either Producer or consumer hanging after a while

Tim Bain
Those client logs look like neither broker is accepting a connection (I see
"connection refused" and EOFException).  Can you share the broker logs from
the same time period?

Also, your description didn't say anything about multiple brokers; can you
describe what state each is in at the time, how they're configured with
respect to each other (e.g. master-slave pair, network of brokers,
standalone/unconnected), and what URI your clients use to connect?

Tim

On Feb 2, 2017 3:33 AM, "RAJIV" <[hidden email]> wrote:

Hi Team,

I am using a application where in producer , consumer both uses ActiveMQ
broker..
Problem is after a while either producer or consumer is hanging not
processing any messages,... SO i had to restart broker, producer program and
consumer program as well , then only it is working...

Because of this issue , i have been doing multiple restart very often..

It is exactly hanging in the line where session is being created ... Below
is the line ..
session = connection.createQueueSession( false, Session.AUTO_ACKNOWLEDGE );


In my Log these below are the DEBUG messages comes in,..
 I could not figure out anything.. Any have any idea ,Plese help.


DEBUG | Queue created: dynamicQueues/archive1mq?consumer.prefetchSize=2000
DEBUG | Initialized TaskRunnerFactory[ActiveMQ Task] using ExecutorService:
java.util.concurrent.ThreadPoolExecutor@4e5f150f[Running, pool siz
e = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
DEBUG | Reconnect was triggered but transport is not started yet. Wait for
start to connect the transport.
DEBUG | Started unconnected
DEBUG | Waking up reconnect task
DEBUG | Connection created in ApexServiceAgentArchiveJMSThread
DEBUG | urlList connectionList:[tcp://ACTIVEMQHOST2:5001,
tcp://ACTIVEMQHOST1:5001], from: [tcp://ACTIVEMQHOST1:5001,
tcp://ACTIVEMQHOST2:5001
]
DEBUG | Attempting 0th connect to: tcp://ACTIVEMQHOST2:5001
DEBUG | Connect fail to: tcp://ACTIVEMQHOST2:5001, reason:
java.net.ConnectException: Connection refused
DEBUG | Stopping transport tcp://ACTIVEMQHOST2:5001
DEBUG | Initialized TaskRunnerFactory[ActiveMQ Task] using ExecutorService:
java.util.concurrent.ThreadPoolExecutor@37de9bd8[Running, pool siz
e = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
DEBUG | Closed socket Socket[unconnected]
DEBUG | Forcing shutdown of ExecutorService:
java.util.concurrent.ThreadPoolExecutor@37de9bd8[Running, pool size = 1,
active threads = 0, queu
ed tasks = 0, completed tasks = 1]
DEBUG | Attempting 0th connect to: tcp://ACTIVEMQHOST1:5001
DEBUG | Sending: WireFormatInfo { version=11,
properties={TcpNoDelayEnabled=true, SizePrefixDisabled=false,
CacheSize=1024, StackTraceEnabled=
true, CacheEnabled=true, TightEncodingEnabled=true,
MaxFrameSize=9223372036854775807, MaxInactivityDuration=30000,
MaxInactivityDurationInital
Delay=10000}, magic=[A,c,t,i,v,e,M,Q]}
DEBUG | Connection established
 INFO | Successfully connected to tcp://ACTIVEMQHOST1:5001
DEBUG | Stopping transport tcp://ACTIVEMQHOST1/10.129.66.129:5001@60323
DEBUG | Initialized TaskRunnerFactory[ActiveMQ Task] using ExecutorService:
java.util.concurrent.ThreadPoolExecutor@12eb3770[Running, pool siz
e = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
DEBUG | Closed socket
Socket[addr=ACTIVEMQHOST1/10.129.66.129,port=5001,localport=60323]
DEBUG | Forcing shutdown of ExecutorService:
java.util.concurrent.ThreadPoolExecutor@12eb3770[Running, pool size = 1,
active threads = 0, queu
ed tasks = 0, completed tasks = 1]
DEBUG | Send oneway attempt: 0 failed for command: ConnectionInfo {commandId
= 1, responseRequired = true, connectionId = ID:sjc3-vg4-apexv02a-
60312-1486017164856-1:1, clientId =
ID:sjc3-vg4-apexv02a-60312-1486017164856-0:1, clientIp = null, userName =
fdc, password = *****, brokerPath
 = null, brokerMasterConnector = false, manageable = true, clientMaster =
true, faultTolerant = true, failoverReconnect = false}
 WARN | Transport (tcp://ACTIVEMQHOST1:5001) failed , attempting to
automatically reconnect: java.io.EOFException
DEBUG | urlList connectionList:[tcp://ACTIVEMQHOST2:5001,
tcp://ACTIVEMQHOST1:5001], from: [tcp://ACTIVEMQHOST1:5001,
tcp://ACTIVEMQHOST2:5001
]
DEBUG | Waiting 10 ms before attempting connection
DEBUG | Attempting 0th connect to: tcp://ACTIVEMQHOST2:5001
DEBUG | Connect fail to: tcp://ACTIVEMQHOST2:5001, reason:
java.net.ConnectException: Connection refused
DEBUG | Stopping transport tcp://ACTIVEMQHOST2:5001
DEBUG | Initialized TaskRunnerFactory[ActiveMQ Task] using ExecutorService:
java.util.concurrent.ThreadPoolExecutor@63f2789e[Running, pool siz
e = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
DEBUG | Closed socket Socket[unconnected]
DEBUG | Forcing shutdown of ExecutorService:
java.util.concurrent.ThreadPoolExecutor@63f2789e[Running, pool size = 1,
active threads = 1, queu
ed tasks = 0, completed tasks = 0]
DEBUG | Attempting 0th connect to: tcp://ACTIVEMQHOST1:5001
DEBUG | Sending: WireFormatInfo { version=11,
properties={TcpNoDelayEnabled=true, SizePrefixDisabled=false,
CacheSize=1024, StackTraceEnabled=
true, CacheEnabled=true, TightEncodingEnabled=true,
MaxFrameSize=9223372036854775807, MaxInactivityDuration=30000,
MaxInactivityDurationInital
Delay=10000}, magic=[A,c,t,i,v,e,M,Q]}
DEBUG | Connect fail to: tcp://ACTIVEMQHOST1:5001, reason:
org.apache.activemq.transport.InactivityIOException: Cannot send, channel
has alrea
dy failed: tcp://10.129.66.129:5001
DEBUG | Stopping transport tcp://ACTIVEMQHOST1/10.129.66.129:5001@60329
DEBUG | Initialized TaskRunnerFactory[ActiveMQ Task] using ExecutorService:
java.util.concurrent.ThreadPoolExecutor@5cce2b3c[Running, pool siz
e = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
DEBUG | Closed socket
Socket[addr=ACTIVEMQHOST1/10.129.66.129,port=5001,localport=60329]
DEBUG | Forcing shutdown of ExecutorService:
java.util.concurrent.ThreadPoolExecutor@5cce2b3c[Running, pool size = 1,
active threads = 0, queu
ed tasks = 0, completed tasks = 1]
DEBUG | Waiting 20 ms before attempting connection
DEBUG | urlList connectionList:[tcp://ACTIVEMQHOST2:5001,
tcp://ACTIVEMQHOST1:5001], from: [tcp://ACTIVEMQHOST1:5001,
tcp://ACTIVEMQHOST2:5001
]




--
View this message in context: http://activemq.2283324.n4.
nabble.com/Either-Producer-or-consumer-hanging-after-a-while-tp4721496.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.