ActiveMQ client app hold unexpectedly!!!

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

ActiveMQ client app hold unexpectedly!!!

DepestBlue
Dear all,


I found a one of the ActiveMQ thread got stuck unexpectedly for several days. Customer was very angry to the ActiveMQ for the bad behavior. Did anybody experienced this issue? It would be much appreciated if any clue!!!
My application is normal Java JMS application like the following:
new ActiveMQConnectionFactory.
create Connection;
conn.start.
create session.
create destination, producer and message
send messages
commit;
conn.close.


The follows is the stack when I looked into it with debug tools:

sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.park(LockSupport.java:169)

java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1787)

java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:356)

org.apache.activemq.transport.FutureResponse.getResult(FutureResponse.java:40)

org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:80)

org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1233)

org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1339)

org.apache.activemq.ActiveMQConnection.start(ActiveMQConnection.java:488)

nc.bs.mqv2.mmgp.mq.UFMQProcessor.getConnection(UFMQProcessor.java:487)

nc.bs.mqv2.mmgp.mq.UFMQProcessor.getConnection(UFMQProcessor.java:462)

nc.bs.mqv2.mmgp.mq.UFMQProcessor.sendSerializedObjMsg(UFMQProcessor.java:128)

nc.bs.mqv2.meam.mq.sendbasedatas.AbstractMeamBDSyncService.sendBaseData(AbstractMeamBDSyncService.java:64)

nc.bs.mqv2.mes.bd.mq.basedoc.AbstractBDServerSendService.dosendDataProcess(AbstractBDServerSendService.java:213)

nc.bs.mqv2.mes.bd.mq.basedoc.AbstractBDServerSendService.sendDatas(AbstractBDServerSendService.java:62)

nc.impl.mqv2.mes.bd.mq.SyncSendServiceImpl.ncserverSendDatas(SyncSendServiceImpl.java:58)

nc.gejb.meam.cmt.MeamEjbBean.ncserverSendDatas(MeamEjbBean.java:8741)

nc.gejb.meam.cmt.EJSLocalStatelessnc_gejb_meam_cmt_Meam_b44269e7.ncserverSendDatas(Unknown Source)

sun.reflect.GeneratedMethodAccessor447.invoke(Unknown Source)

sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

java.lang.reflect.Method.invoke(Method.java:615)

nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:56)

nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:34)

$Proxy58.ncserverSendDatas(Unknown Source)

nc.bs.mqv2.meam.mq.task.DoSendBaseDataTaskUtil.runTask(DoSendBaseDataTaskUtil.java:63)

nc.bs.mqv2.meam.mq.task.NCServerBaseDatasSendTask.executeTask(NCServerBaseDatasSendTask.java:26)

nc.bs.pub.taskcenter.BgBizsPluginProcess.callBusinessPlugin(BgBizsPluginProcess.java:43)

nc.bs.pub.pa.read.TaskBodyBase.callBizsPlugin(TaskBodyBase.java:188)

nc.bs.pub.pa.read.TaskBodyBase.callBusinessPlugin(TaskBodyBase.java:174)

nc.bs.pub.pa.read.TaskBodyBase.createMetaMsg(TaskBodyBase.java:95)

nc.bs.pub.pa.read.TaskBodyBase.createAndSendMessage(TaskBodyBase.java:41)

nc.bs.pub.taskcenter.BackgroundTaskBody.execute(BackgroundTaskBody.java:47)

nc.impl.uap.scheduler.ExecutorServiceImpl.executeTask_RequiresNew(ExecutorServiceImpl.java:27)

nc.itf.uap.mw.MwBizEJBEjbBean.executeTask_RequiresNew(MwBizEJBEjbBean.java:1682)

nc.itf.uap.mw.EJSLocalStatelessnc_itf_uap_mw_MwBizEJB_f98e92a8.executeTask_RequiresNew(Unknown Source)

sun.reflect.GeneratedMethodAccessor1036.invoke(Unknown Source)

sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

java.lang.reflect.Method.invoke(Method.java:615)

nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:56)

nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:34)

$Proxy39.executeTask_RequiresNew(Unknown Source)

nc.bs.uap.scheduler.impl.TaskExecutor.executeTask(TaskExecutor.java:110)

nc.bs.uap.scheduler.impl.TaskExecutor.run(TaskExecutor.java:75)

nc.bs.framework.execute.Executor$1.run(Executor.java:80)

nc.bs.framework.execute.commonj.impl.WorkAdaptor.run(WorkAdaptor.java:29)

com.ibm.ws.asynchbeans.J2EEContext.run(J2EEContext.java:1114)

com.ibm.ws.asynchbeans.WorkWithExecutionContextImpl.go(WorkWithExecutionContextImpl.java:195)

com.ibm.ws.asynchbeans.CJWorkItemImpl.run(CJWorkItemImpl.java:187)

com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1469)




Best regards,

Zhuran Li
Reply | Threaded
Open this post in threaded view
|

Re: ActiveMQ client app hold unexpectedly!!!

ceposta
This means the client didn't' receive a response from the broker.
Did the broker producer flow control this client? ie, was the destination
to which the client sending full?


On Tue, Jul 16, 2013 at 7:23 AM, lzr <[hidden email]> wrote:

> Dear all,
>
>
> I found a one of the ActiveMQ thread got stuck unexpectedly for several
> days. Customer was very angry to the ActiveMQ for the bad behavior. Did
> anybody experienced this issue? It would be much appreciated if any clue!!!
> My application is normal Java JMS application like the following:
> new ActiveMQConnectionFactory.
> create Connection;
> conn.start.
> create session.
> create destination, producer and message
> send messages
> commit;
> conn.close.
>
>
> The follows is the stack when I looked into it with debug tools:
>
> sun.misc.Unsafe.park(Native Method)
>
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:169)
>
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1787)
>
> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:356)
>
>
> org.apache.activemq.transport.FutureResponse.getResult(FutureResponse.java:40)
>
>
> org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:80)
>
>
> org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1233)
>
>
> org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1339)
>
> org.apache.activemq.ActiveMQConnection.start(ActiveMQConnection.java:488)
>
> nc.bs.mqv2.mmgp.mq.UFMQProcessor.getConnection(UFMQProcessor.java:487)
>
> nc.bs.mqv2.mmgp.mq.UFMQProcessor.getConnection(UFMQProcessor.java:462)
>
>
> nc.bs.mqv2.mmgp.mq.UFMQProcessor.sendSerializedObjMsg(UFMQProcessor.java:128)
>
>
> nc.bs.mqv2.meam.mq.sendbasedatas.AbstractMeamBDSyncService.sendBaseData(AbstractMeamBDSyncService.java:64)
>
>
> nc.bs.mqv2.mes.bd.mq.basedoc.AbstractBDServerSendService.dosendDataProcess(AbstractBDServerSendService.java:213)
>
>
> nc.bs.mqv2.mes.bd.mq.basedoc.AbstractBDServerSendService.sendDatas(AbstractBDServerSendService.java:62)
>
>
> nc.impl.mqv2.mes.bd.mq.SyncSendServiceImpl.ncserverSendDatas(SyncSendServiceImpl.java:58)
>
> nc.gejb.meam.cmt.MeamEjbBean.ncserverSendDatas(MeamEjbBean.java:8741)
>
> nc.gejb.meam.cmt.EJSLocalStatelessnc_gejb_meam_cmt_Meam_b44269e7.ncserverSendDatas(Unknown
> Source)
>
> sun.reflect.GeneratedMethodAccessor447.invoke(Unknown Source)
>
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> java.lang.reflect.Method.invoke(Method.java:615)
>
>
> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:56)
>
>
> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:34)
>
> $Proxy58.ncserverSendDatas(Unknown Source)
>
>
> nc.bs.mqv2.meam.mq.task.DoSendBaseDataTaskUtil.runTask(DoSendBaseDataTaskUtil.java:63)
>
>
> nc.bs.mqv2.meam.mq.task.NCServerBaseDatasSendTask.executeTask(NCServerBaseDatasSendTask.java:26)
>
>
> nc.bs.pub.taskcenter.BgBizsPluginProcess.callBusinessPlugin(BgBizsPluginProcess.java:43)
>
> nc.bs.pub.pa.read.TaskBodyBase.callBizsPlugin(TaskBodyBase.java:188)
>
> nc.bs.pub.pa.read.TaskBodyBase.callBusinessPlugin(TaskBodyBase.java:174)
>
> nc.bs.pub.pa.read.TaskBodyBase.createMetaMsg(TaskBodyBase.java:95)
>
> nc.bs.pub.pa.read.TaskBodyBase.createAndSendMessage(TaskBodyBase.java:41)
>
> nc.bs.pub.taskcenter.BackgroundTaskBody.execute(BackgroundTaskBody.java:47)
>
>
> nc.impl.uap.scheduler.ExecutorServiceImpl.executeTask_RequiresNew(ExecutorServiceImpl.java:27)
>
>
> nc.itf.uap.mw.MwBizEJBEjbBean.executeTask_RequiresNew(MwBizEJBEjbBean.java:1682)
>
> nc.itf.uap.mw.EJSLocalStatelessnc_itf_uap_mw_MwBizEJB_f98e92a8.executeTask_RequiresNew(Unknown
> Source)
>
> sun.reflect.GeneratedMethodAccessor1036.invoke(Unknown Source)
>
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> java.lang.reflect.Method.invoke(Method.java:615)
>
>
> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:56)
>
>
> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:34)
>
> $Proxy39.executeTask_RequiresNew(Unknown Source)
>
> nc.bs.uap.scheduler.impl.TaskExecutor.executeTask(TaskExecutor.java:110)
>
> nc.bs.uap.scheduler.impl.TaskExecutor.run(TaskExecutor.java:75)
>
> nc.bs.framework.execute.Executor$1.run(Executor.java:80)
>
> nc.bs.framework.execute.commonj.impl.WorkAdaptor.run(WorkAdaptor.java:29)
>
> com.ibm.ws.asynchbeans.J2EEContext.run(J2EEContext.java:1114)
>
>
> com.ibm.ws.asynchbeans.WorkWithExecutionContextImpl.go(WorkWithExecutionContextImpl.java:195)
>
> com.ibm.ws.asynchbeans.CJWorkItemImpl.run(CJWorkItemImpl.java:187)
>
> com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1469)
>
>
>
>
> Best regards,
>
> Zhuran Li




--
*Christian Posta*
http://www.christianposta.com/blog
twitter: @christianposta
Reply | Threaded
Open this post in threaded view
|

Re:Re: ActiveMQ client app hold unexpectedly!!!

DepestBlue
Based on my experiences if the issue caused by the producer flow control configuration, the logs will have some warning messages, right? In fact I didn't found any warning or errors in the activemq.log.
Anyway, I'll check the configuration files right now.


thanks lot,
Zhuran.


At 2013-07-16 19:36:44,"Christian Posta" <[hidden email]> wrote:

>This means the client didn't' receive a response from the broker.
>Did the broker producer flow control this client? ie, was the destination
>to which the client sending full?
>
>
>On Tue, Jul 16, 2013 at 7:23 AM, lzr <[hidden email]> wrote:
>
>> Dear all,
>>
>>
>> I found a one of the ActiveMQ thread got stuck unexpectedly for several
>> days. Customer was very angry to the ActiveMQ for the bad behavior. Did
>> anybody experienced this issue? It would be much appreciated if any clue!!!
>> My application is normal Java JMS application like the following:
>> new ActiveMQConnectionFactory.
>> create Connection;
>> conn.start.
>> create session.
>> create destination, producer and message
>> send messages
>> commit;
>> conn.close.
>>
>>
>> The follows is the stack when I looked into it with debug tools:
>>
>> sun.misc.Unsafe.park(Native Method)
>>
>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:169)
>>
>>
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1787)
>>
>> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:356)
>>
>>
>> org.apache.activemq.transport.FutureResponse.getResult(FutureResponse.java:40)
>>
>>
>> org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:80)
>>
>>
>> org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1233)
>>
>>
>> org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1339)
>>
>> org.apache.activemq.ActiveMQConnection.start(ActiveMQConnection.java:488)
>>
>> nc.bs.mqv2.mmgp.mq.UFMQProcessor.getConnection(UFMQProcessor.java:487)
>>
>> nc.bs.mqv2.mmgp.mq.UFMQProcessor.getConnection(UFMQProcessor.java:462)
>>
>>
>> nc.bs.mqv2.mmgp.mq.UFMQProcessor.sendSerializedObjMsg(UFMQProcessor.java:128)
>>
>>
>> nc.bs.mqv2.meam.mq.sendbasedatas.AbstractMeamBDSyncService.sendBaseData(AbstractMeamBDSyncService.java:64)
>>
>>
>> nc.bs.mqv2.mes.bd.mq.basedoc.AbstractBDServerSendService.dosendDataProcess(AbstractBDServerSendService.java:213)
>>
>>
>> nc.bs.mqv2.mes.bd.mq.basedoc.AbstractBDServerSendService.sendDatas(AbstractBDServerSendService.java:62)
>>
>>
>> nc.impl.mqv2.mes.bd.mq.SyncSendServiceImpl.ncserverSendDatas(SyncSendServiceImpl.java:58)
>>
>> nc.gejb.meam.cmt.MeamEjbBean.ncserverSendDatas(MeamEjbBean.java:8741)
>>
>> nc.gejb.meam.cmt.EJSLocalStatelessnc_gejb_meam_cmt_Meam_b44269e7.ncserverSendDatas(Unknown
>> Source)
>>
>> sun.reflect.GeneratedMethodAccessor447.invoke(Unknown Source)
>>
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>
>> java.lang.reflect.Method.invoke(Method.java:615)
>>
>>
>> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:56)
>>
>>
>> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:34)
>>
>> $Proxy58.ncserverSendDatas(Unknown Source)
>>
>>
>> nc.bs.mqv2.meam.mq.task.DoSendBaseDataTaskUtil.runTask(DoSendBaseDataTaskUtil.java:63)
>>
>>
>> nc.bs.mqv2.meam.mq.task.NCServerBaseDatasSendTask.executeTask(NCServerBaseDatasSendTask.java:26)
>>
>>
>> nc.bs.pub.taskcenter.BgBizsPluginProcess.callBusinessPlugin(BgBizsPluginProcess.java:43)
>>
>> nc.bs.pub.pa.read.TaskBodyBase.callBizsPlugin(TaskBodyBase.java:188)
>>
>> nc.bs.pub.pa.read.TaskBodyBase.callBusinessPlugin(TaskBodyBase.java:174)
>>
>> nc.bs.pub.pa.read.TaskBodyBase.createMetaMsg(TaskBodyBase.java:95)
>>
>> nc.bs.pub.pa.read.TaskBodyBase.createAndSendMessage(TaskBodyBase.java:41)
>>
>> nc.bs.pub.taskcenter.BackgroundTaskBody.execute(BackgroundTaskBody.java:47)
>>
>>
>> nc.impl.uap.scheduler.ExecutorServiceImpl.executeTask_RequiresNew(ExecutorServiceImpl.java:27)
>>
>>
>> nc.itf.uap.mw.MwBizEJBEjbBean.executeTask_RequiresNew(MwBizEJBEjbBean.java:1682)
>>
>> nc.itf.uap.mw.EJSLocalStatelessnc_itf_uap_mw_MwBizEJB_f98e92a8.executeTask_RequiresNew(Unknown
>> Source)
>>
>> sun.reflect.GeneratedMethodAccessor1036.invoke(Unknown Source)
>>
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>
>> java.lang.reflect.Method.invoke(Method.java:615)
>>
>>
>> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:56)
>>
>>
>> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:34)
>>
>> $Proxy39.executeTask_RequiresNew(Unknown Source)
>>
>> nc.bs.uap.scheduler.impl.TaskExecutor.executeTask(TaskExecutor.java:110)
>>
>> nc.bs.uap.scheduler.impl.TaskExecutor.run(TaskExecutor.java:75)
>>
>> nc.bs.framework.execute.Executor$1.run(Executor.java:80)
>>
>> nc.bs.framework.execute.commonj.impl.WorkAdaptor.run(WorkAdaptor.java:29)
>>
>> com.ibm.ws.asynchbeans.J2EEContext.run(J2EEContext.java:1114)
>>
>>
>> com.ibm.ws.asynchbeans.WorkWithExecutionContextImpl.go(WorkWithExecutionContextImpl.java:195)
>>
>> com.ibm.ws.asynchbeans.CJWorkItemImpl.run(CJWorkItemImpl.java:187)
>>
>> com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1469)
>>
>>
>>
>>
>> Best regards,
>>
>> Zhuran Li
>
>
>
>
>--
>*Christian Posta*
>http://www.christianposta.com/blog
>twitter: @christianposta
Reply | Threaded
Open this post in threaded view
|

Re: Re: ActiveMQ client app hold unexpectedly!!!

ceposta
Yes, on the first time PFC kicked in you would see the log messages. Not on
subsequent times.


On Tue, Jul 16, 2013 at 8:01 AM, lzr <[hidden email]> wrote:

> Based on my experiences if the issue caused by the producer flow control
> configuration, the logs will have some warning messages, right? In fact I
> didn't found any warning or errors in the activemq.log.
> Anyway, I'll check the configuration files right now.
>
>
> thanks lot,
> Zhuran.
>
>
> At 2013-07-16 19:36:44,"Christian Posta" <[hidden email]>
> wrote:
>
> >This means the client didn't' receive a response from the broker.
> >Did the broker producer flow control this client? ie, was the destination
> >to which the client sending full?
> >
> >
> >On Tue, Jul 16, 2013 at 7:23 AM, lzr <[hidden email]> wrote:
> >
> >> Dear all,
> >>
> >>
> >> I found a one of the ActiveMQ thread got stuck unexpectedly for several
> >> days. Customer was very angry to the ActiveMQ for the bad behavior. Did
> >> anybody experienced this issue? It would be much appreciated if any
> clue!!!
> >> My application is normal Java JMS application like the following:
> >> new ActiveMQConnectionFactory.
> >> create Connection;
> >> conn.start.
> >> create session.
> >> create destination, producer and message
> >> send messages
> >> commit;
> >> conn.close.
> >>
> >>
> >> The follows is the stack when I looked into it with debug tools:
> >>
> >> sun.misc.Unsafe.park(Native Method)
> >>
> >> java.util.concurrent.locks.LockSupport.park(LockSupport.java:169)
> >>
> >>
> >>
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1787)
> >>
> >>
> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:356)
> >>
> >>
> >>
> org.apache.activemq.transport.FutureResponse.getResult(FutureResponse.java:40)
> >>
> >>
> >>
> org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:80)
> >>
> >>
> >>
> org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1233)
> >>
> >>
> >>
> org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1339)
> >>
> >>
> org.apache.activemq.ActiveMQConnection.start(ActiveMQConnection.java:488)
> >>
> >> nc.bs.mqv2.mmgp.mq.UFMQProcessor.getConnection(UFMQProcessor.java:487)
> >>
> >> nc.bs.mqv2.mmgp.mq.UFMQProcessor.getConnection(UFMQProcessor.java:462)
> >>
> >>
> >>
> nc.bs.mqv2.mmgp.mq.UFMQProcessor.sendSerializedObjMsg(UFMQProcessor.java:128)
> >>
> >>
> >>
> nc.bs.mqv2.meam.mq.sendbasedatas.AbstractMeamBDSyncService.sendBaseData(AbstractMeamBDSyncService.java:64)
> >>
> >>
> >>
> nc.bs.mqv2.mes.bd.mq.basedoc.AbstractBDServerSendService.dosendDataProcess(AbstractBDServerSendService.java:213)
> >>
> >>
> >>
> nc.bs.mqv2.mes.bd.mq.basedoc.AbstractBDServerSendService.sendDatas(AbstractBDServerSendService.java:62)
> >>
> >>
> >>
> nc.impl.mqv2.mes.bd.mq.SyncSendServiceImpl.ncserverSendDatas(SyncSendServiceImpl.java:58)
> >>
> >> nc.gejb.meam.cmt.MeamEjbBean.ncserverSendDatas(MeamEjbBean.java:8741)
> >>
> >>
> nc.gejb.meam.cmt.EJSLocalStatelessnc_gejb_meam_cmt_Meam_b44269e7.ncserverSendDatas(Unknown
> >> Source)
> >>
> >> sun.reflect.GeneratedMethodAccessor447.invoke(Unknown Source)
> >>
> >>
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >>
> >> java.lang.reflect.Method.invoke(Method.java:615)
> >>
> >>
> >>
> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:56)
> >>
> >>
> >>
> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:34)
> >>
> >> $Proxy58.ncserverSendDatas(Unknown Source)
> >>
> >>
> >>
> nc.bs.mqv2.meam.mq.task.DoSendBaseDataTaskUtil.runTask(DoSendBaseDataTaskUtil.java:63)
> >>
> >>
> >>
> nc.bs.mqv2.meam.mq.task.NCServerBaseDatasSendTask.executeTask(NCServerBaseDatasSendTask.java:26)
> >>
> >>
> >>
> nc.bs.pub.taskcenter.BgBizsPluginProcess.callBusinessPlugin(BgBizsPluginProcess.java:43)
> >>
> >> nc.bs.pub.pa.read.TaskBodyBase.callBizsPlugin(TaskBodyBase.java:188)
> >>
> >> nc.bs.pub.pa.read.TaskBodyBase.callBusinessPlugin(TaskBodyBase.java:174)
> >>
> >> nc.bs.pub.pa.read.TaskBodyBase.createMetaMsg(TaskBodyBase.java:95)
> >>
> >>
> nc.bs.pub.pa.read.TaskBodyBase.createAndSendMessage(TaskBodyBase.java:41)
> >>
> >>
> nc.bs.pub.taskcenter.BackgroundTaskBody.execute(BackgroundTaskBody.java:47)
> >>
> >>
> >>
> nc.impl.uap.scheduler.ExecutorServiceImpl.executeTask_RequiresNew(ExecutorServiceImpl.java:27)
> >>
> >>
> >>
> nc.itf.uap.mw.MwBizEJBEjbBean.executeTask_RequiresNew(MwBizEJBEjbBean.java:1682)
> >>
> >>
> nc.itf.uap.mw.EJSLocalStatelessnc_itf_uap_mw_MwBizEJB_f98e92a8.executeTask_RequiresNew(Unknown
> >> Source)
> >>
> >> sun.reflect.GeneratedMethodAccessor1036.invoke(Unknown Source)
> >>
> >>
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >>
> >> java.lang.reflect.Method.invoke(Method.java:615)
> >>
> >>
> >>
> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:56)
> >>
> >>
> >>
> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:34)
> >>
> >> $Proxy39.executeTask_RequiresNew(Unknown Source)
> >>
> >> nc.bs.uap.scheduler.impl.TaskExecutor.executeTask(TaskExecutor.java:110)
> >>
> >> nc.bs.uap.scheduler.impl.TaskExecutor.run(TaskExecutor.java:75)
> >>
> >> nc.bs.framework.execute.Executor$1.run(Executor.java:80)
> >>
> >>
> nc.bs.framework.execute.commonj.impl.WorkAdaptor.run(WorkAdaptor.java:29)
> >>
> >> com.ibm.ws.asynchbeans.J2EEContext.run(J2EEContext.java:1114)
> >>
> >>
> >>
> com.ibm.ws.asynchbeans.WorkWithExecutionContextImpl.go(WorkWithExecutionContextImpl.java:195)
> >>
> >> com.ibm.ws.asynchbeans.CJWorkItemImpl.run(CJWorkItemImpl.java:187)
> >>
> >> com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1469)
> >>
> >>
> >>
> >>
> >> Best regards,
> >>
> >> Zhuran Li
> >
> >
> >
> >
> >--
> >*Christian Posta*
> >http://www.christianposta.com/blog
> >twitter: @christianposta
>



--
*Christian Posta*
http://www.christianposta.com/blog
twitter: @christianposta
Reply | Threaded
Open this post in threaded view
|

Re:Re: Re: ActiveMQ client app hold unexpectedly!!!

DepestBlue
I just check the configuration file, actually I've disabled the flow control:


        <destinationPolicy>
            <policyMap>
              <policyEntries>
                <policyEntry topic=">" producerFlowControl="false" memoryLimit="100mb">
                  <pendingSubscriberPolicy>
                    <vmCursor />
                  </pendingSubscriberPolicy>
                </policyEntry>
                <policyEntry queue=">" producerFlowControl="false" memoryLimit="100mb">
                  <!-- Use VM cursor for better latency
                       For more information, see:


                       http://activemq.apache.org/message-cursors.html


                  <pendingQueuePolicy>
                    <vmQueueCursor/>
                  </pendingQueuePolicy>
                  -->
                </policyEntry>
              </policyEntries>
            </policyMap>
        </destinationPolicy>










At 2013-07-16 20:15:11,"Christian Posta" <[hidden email]> wrote:

>Yes, on the first time PFC kicked in you would see the log messages. Not on
>subsequent times.
>
>
>On Tue, Jul 16, 2013 at 8:01 AM, lzr <[hidden email]> wrote:
>
>> Based on my experiences if the issue caused by the producer flow control
>> configuration, the logs will have some warning messages, right? In fact I
>> didn't found any warning or errors in the activemq.log.
>> Anyway, I'll check the configuration files right now.
>>
>>
>> thanks lot,
>> Zhuran.
>>
>>
>> At 2013-07-16 19:36:44,"Christian Posta" <[hidden email]>
>> wrote:
>>
>> >This means the client didn't' receive a response from the broker.
>> >Did the broker producer flow control this client? ie, was the destination
>> >to which the client sending full?
>> >
>> >
>> >On Tue, Jul 16, 2013 at 7:23 AM, lzr <[hidden email]> wrote:
>> >
>> >> Dear all,
>> >>
>> >>
>> >> I found a one of the ActiveMQ thread got stuck unexpectedly for several
>> >> days. Customer was very angry to the ActiveMQ for the bad behavior. Did
>> >> anybody experienced this issue? It would be much appreciated if any
>> clue!!!
>> >> My application is normal Java JMS application like the following:
>> >> new ActiveMQConnectionFactory.
>> >> create Connection;
>> >> conn.start.
>> >> create session.
>> >> create destination, producer and message
>> >> send messages
>> >> commit;
>> >> conn.close.
>> >>
>> >>
>> >> The follows is the stack when I looked into it with debug tools:
>> >>
>> >> sun.misc.Unsafe.park(Native Method)
>> >>
>> >> java.util.concurrent.locks.LockSupport.park(LockSupport.java:169)
>> >>
>> >>
>> >>
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1787)
>> >>
>> >>
>> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:356)
>> >>
>> >>
>> >>
>> org.apache.activemq.transport.FutureResponse.getResult(FutureResponse.java:40)
>> >>
>> >>
>> >>
>> org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:80)
>> >>
>> >>
>> >>
>> org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1233)
>> >>
>> >>
>> >>
>> org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1339)
>> >>
>> >>
>> org.apache.activemq.ActiveMQConnection.start(ActiveMQConnection.java:488)
>> >>
>> >> nc.bs.mqv2.mmgp.mq.UFMQProcessor.getConnection(UFMQProcessor.java:487)
>> >>
>> >> nc.bs.mqv2.mmgp.mq.UFMQProcessor.getConnection(UFMQProcessor.java:462)
>> >>
>> >>
>> >>
>> nc.bs.mqv2.mmgp.mq.UFMQProcessor.sendSerializedObjMsg(UFMQProcessor.java:128)
>> >>
>> >>
>> >>
>> nc.bs.mqv2.meam.mq.sendbasedatas.AbstractMeamBDSyncService.sendBaseData(AbstractMeamBDSyncService.java:64)
>> >>
>> >>
>> >>
>> nc.bs.mqv2.mes.bd.mq.basedoc.AbstractBDServerSendService.dosendDataProcess(AbstractBDServerSendService.java:213)
>> >>
>> >>
>> >>
>> nc.bs.mqv2.mes.bd.mq.basedoc.AbstractBDServerSendService.sendDatas(AbstractBDServerSendService.java:62)
>> >>
>> >>
>> >>
>> nc.impl.mqv2.mes.bd.mq.SyncSendServiceImpl.ncserverSendDatas(SyncSendServiceImpl.java:58)
>> >>
>> >> nc.gejb.meam.cmt.MeamEjbBean.ncserverSendDatas(MeamEjbBean.java:8741)
>> >>
>> >>
>> nc.gejb.meam.cmt.EJSLocalStatelessnc_gejb_meam_cmt_Meam_b44269e7.ncserverSendDatas(Unknown
>> >> Source)
>> >>
>> >> sun.reflect.GeneratedMethodAccessor447.invoke(Unknown Source)
>> >>
>> >>
>> >>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> >>
>> >> java.lang.reflect.Method.invoke(Method.java:615)
>> >>
>> >>
>> >>
>> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:56)
>> >>
>> >>
>> >>
>> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:34)
>> >>
>> >> $Proxy58.ncserverSendDatas(Unknown Source)
>> >>
>> >>
>> >>
>> nc.bs.mqv2.meam.mq.task.DoSendBaseDataTaskUtil.runTask(DoSendBaseDataTaskUtil.java:63)
>> >>
>> >>
>> >>
>> nc.bs.mqv2.meam.mq.task.NCServerBaseDatasSendTask.executeTask(NCServerBaseDatasSendTask.java:26)
>> >>
>> >>
>> >>
>> nc.bs.pub.taskcenter.BgBizsPluginProcess.callBusinessPlugin(BgBizsPluginProcess.java:43)
>> >>
>> >> nc.bs.pub.pa.read.TaskBodyBase.callBizsPlugin(TaskBodyBase.java:188)
>> >>
>> >> nc.bs.pub.pa.read.TaskBodyBase.callBusinessPlugin(TaskBodyBase.java:174)
>> >>
>> >> nc.bs.pub.pa.read.TaskBodyBase.createMetaMsg(TaskBodyBase.java:95)
>> >>
>> >>
>> nc.bs.pub.pa.read.TaskBodyBase.createAndSendMessage(TaskBodyBase.java:41)
>> >>
>> >>
>> nc.bs.pub.taskcenter.BackgroundTaskBody.execute(BackgroundTaskBody.java:47)
>> >>
>> >>
>> >>
>> nc.impl.uap.scheduler.ExecutorServiceImpl.executeTask_RequiresNew(ExecutorServiceImpl.java:27)
>> >>
>> >>
>> >>
>> nc.itf.uap.mw.MwBizEJBEjbBean.executeTask_RequiresNew(MwBizEJBEjbBean.java:1682)
>> >>
>> >>
>> nc.itf.uap.mw.EJSLocalStatelessnc_itf_uap_mw_MwBizEJB_f98e92a8.executeTask_RequiresNew(Unknown
>> >> Source)
>> >>
>> >> sun.reflect.GeneratedMethodAccessor1036.invoke(Unknown Source)
>> >>
>> >>
>> >>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> >>
>> >> java.lang.reflect.Method.invoke(Method.java:615)
>> >>
>> >>
>> >>
>> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:56)
>> >>
>> >>
>> >>
>> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:34)
>> >>
>> >> $Proxy39.executeTask_RequiresNew(Unknown Source)
>> >>
>> >> nc.bs.uap.scheduler.impl.TaskExecutor.executeTask(TaskExecutor.java:110)
>> >>
>> >> nc.bs.uap.scheduler.impl.TaskExecutor.run(TaskExecutor.java:75)
>> >>
>> >> nc.bs.framework.execute.Executor$1.run(Executor.java:80)
>> >>
>> >>
>> nc.bs.framework.execute.commonj.impl.WorkAdaptor.run(WorkAdaptor.java:29)
>> >>
>> >> com.ibm.ws.asynchbeans.J2EEContext.run(J2EEContext.java:1114)
>> >>
>> >>
>> >>
>> com.ibm.ws.asynchbeans.WorkWithExecutionContextImpl.go(WorkWithExecutionContextImpl.java:195)
>> >>
>> >> com.ibm.ws.asynchbeans.CJWorkItemImpl.run(CJWorkItemImpl.java:187)
>> >>
>> >> com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1469)
>> >>
>> >>
>> >>
>> >>
>> >> Best regards,
>> >>
>> >> Zhuran Li
>> >
>> >
>> >
>> >
>> >--
>> >*Christian Posta*
>> >http://www.christianposta.com/blog
>> >twitter: @christianposta
>>
>
>
>
>--
>*Christian Posta*
>http://www.christianposta.com/blog
>twitter: @christianposta
Reply | Threaded
Open this post in threaded view
|

Re: Re: Re: ActiveMQ client app hold unexpectedly!!!

ceposta
Is this something that happens all the time? Maybe your network had a
glitch and didn't deliver the response back to the client.


On Tue, Jul 16, 2013 at 8:18 AM, lzr <[hidden email]> wrote:

> I just check the configuration file, actually I've disabled the flow
> control:
>
>
>         <destinationPolicy>
>             <policyMap>
>               <policyEntries>
>                 <policyEntry topic=">" producerFlowControl="false"
> memoryLimit="100mb">
>                   <pendingSubscriberPolicy>
>                     <vmCursor />
>                   </pendingSubscriberPolicy>
>                 </policyEntry>
>                 <policyEntry queue=">" producerFlowControl="false"
> memoryLimit="100mb">
>                   <!-- Use VM cursor for better latency
>                        For more information, see:
>
>
>                        http://activemq.apache.org/message-cursors.html
>
>
>                   <pendingQueuePolicy>
>                     <vmQueueCursor/>
>                   </pendingQueuePolicy>
>                   -->
>                 </policyEntry>
>               </policyEntries>
>             </policyMap>
>         </destinationPolicy>
>
>
>
>
>
>
>
>
>
>
> At 2013-07-16 20:15:11,"Christian Posta" <[hidden email]>
> wrote:
> >Yes, on the first time PFC kicked in you would see the log messages. Not
> on
> >subsequent times.
> >
> >
> >On Tue, Jul 16, 2013 at 8:01 AM, lzr <[hidden email]> wrote:
> >
> >> Based on my experiences if the issue caused by the producer flow control
> >> configuration, the logs will have some warning messages, right? In fact
> I
> >> didn't found any warning or errors in the activemq.log.
> >> Anyway, I'll check the configuration files right now.
> >>
> >>
> >> thanks lot,
> >> Zhuran.
> >>
> >>
> >> At 2013-07-16 19:36:44,"Christian Posta" <[hidden email]>
> >> wrote:
> >>
> >> >This means the client didn't' receive a response from the broker.
> >> >Did the broker producer flow control this client? ie, was the
> destination
> >> >to which the client sending full?
> >> >
> >> >
> >> >On Tue, Jul 16, 2013 at 7:23 AM, lzr <[hidden email]> wrote:
> >> >
> >> >> Dear all,
> >> >>
> >> >>
> >> >> I found a one of the ActiveMQ thread got stuck unexpectedly for
> several
> >> >> days. Customer was very angry to the ActiveMQ for the bad behavior.
> Did
> >> >> anybody experienced this issue? It would be much appreciated if any
> >> clue!!!
> >> >> My application is normal Java JMS application like the following:
> >> >> new ActiveMQConnectionFactory.
> >> >> create Connection;
> >> >> conn.start.
> >> >> create session.
> >> >> create destination, producer and message
> >> >> send messages
> >> >> commit;
> >> >> conn.close.
> >> >>
> >> >>
> >> >> The follows is the stack when I looked into it with debug tools:
> >> >>
> >> >> sun.misc.Unsafe.park(Native Method)
> >> >>
> >> >> java.util.concurrent.locks.LockSupport.park(LockSupport.java:169)
> >> >>
> >> >>
> >> >>
> >>
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1787)
> >> >>
> >> >>
> >>
> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:356)
> >> >>
> >> >>
> >> >>
> >>
> org.apache.activemq.transport.FutureResponse.getResult(FutureResponse.java:40)
> >> >>
> >> >>
> >> >>
> >>
> org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:80)
> >> >>
> >> >>
> >> >>
> >>
> org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1233)
> >> >>
> >> >>
> >> >>
> >>
> org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1339)
> >> >>
> >> >>
> >>
> org.apache.activemq.ActiveMQConnection.start(ActiveMQConnection.java:488)
> >> >>
> >> >>
> nc.bs.mqv2.mmgp.mq.UFMQProcessor.getConnection(UFMQProcessor.java:487)
> >> >>
> >> >>
> nc.bs.mqv2.mmgp.mq.UFMQProcessor.getConnection(UFMQProcessor.java:462)
> >> >>
> >> >>
> >> >>
> >>
> nc.bs.mqv2.mmgp.mq.UFMQProcessor.sendSerializedObjMsg(UFMQProcessor.java:128)
> >> >>
> >> >>
> >> >>
> >>
> nc.bs.mqv2.meam.mq.sendbasedatas.AbstractMeamBDSyncService.sendBaseData(AbstractMeamBDSyncService.java:64)
> >> >>
> >> >>
> >> >>
> >>
> nc.bs.mqv2.mes.bd.mq.basedoc.AbstractBDServerSendService.dosendDataProcess(AbstractBDServerSendService.java:213)
> >> >>
> >> >>
> >> >>
> >>
> nc.bs.mqv2.mes.bd.mq.basedoc.AbstractBDServerSendService.sendDatas(AbstractBDServerSendService.java:62)
> >> >>
> >> >>
> >> >>
> >>
> nc.impl.mqv2.mes.bd.mq.SyncSendServiceImpl.ncserverSendDatas(SyncSendServiceImpl.java:58)
> >> >>
> >> >> nc.gejb.meam.cmt.MeamEjbBean.ncserverSendDatas(MeamEjbBean.java:8741)
> >> >>
> >> >>
> >>
> nc.gejb.meam.cmt.EJSLocalStatelessnc_gejb_meam_cmt_Meam_b44269e7.ncserverSendDatas(Unknown
> >> >> Source)
> >> >>
> >> >> sun.reflect.GeneratedMethodAccessor447.invoke(Unknown Source)
> >> >>
> >> >>
> >> >>
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >> >>
> >> >> java.lang.reflect.Method.invoke(Method.java:615)
> >> >>
> >> >>
> >> >>
> >>
> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:56)
> >> >>
> >> >>
> >> >>
> >>
> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:34)
> >> >>
> >> >> $Proxy58.ncserverSendDatas(Unknown Source)
> >> >>
> >> >>
> >> >>
> >>
> nc.bs.mqv2.meam.mq.task.DoSendBaseDataTaskUtil.runTask(DoSendBaseDataTaskUtil.java:63)
> >> >>
> >> >>
> >> >>
> >>
> nc.bs.mqv2.meam.mq.task.NCServerBaseDatasSendTask.executeTask(NCServerBaseDatasSendTask.java:26)
> >> >>
> >> >>
> >> >>
> >>
> nc.bs.pub.taskcenter.BgBizsPluginProcess.callBusinessPlugin(BgBizsPluginProcess.java:43)
> >> >>
> >> >> nc.bs.pub.pa.read.TaskBodyBase.callBizsPlugin(TaskBodyBase.java:188)
> >> >>
> >> >>
> nc.bs.pub.pa.read.TaskBodyBase.callBusinessPlugin(TaskBodyBase.java:174)
> >> >>
> >> >> nc.bs.pub.pa.read.TaskBodyBase.createMetaMsg(TaskBodyBase.java:95)
> >> >>
> >> >>
> >>
> nc.bs.pub.pa.read.TaskBodyBase.createAndSendMessage(TaskBodyBase.java:41)
> >> >>
> >> >>
> >>
> nc.bs.pub.taskcenter.BackgroundTaskBody.execute(BackgroundTaskBody.java:47)
> >> >>
> >> >>
> >> >>
> >>
> nc.impl.uap.scheduler.ExecutorServiceImpl.executeTask_RequiresNew(ExecutorServiceImpl.java:27)
> >> >>
> >> >>
> >> >>
> >>
> nc.itf.uap.mw.MwBizEJBEjbBean.executeTask_RequiresNew(MwBizEJBEjbBean.java:1682)
> >> >>
> >> >>
> >>
> nc.itf.uap.mw.EJSLocalStatelessnc_itf_uap_mw_MwBizEJB_f98e92a8.executeTask_RequiresNew(Unknown
> >> >> Source)
> >> >>
> >> >> sun.reflect.GeneratedMethodAccessor1036.invoke(Unknown Source)
> >> >>
> >> >>
> >> >>
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >> >>
> >> >> java.lang.reflect.Method.invoke(Method.java:615)
> >> >>
> >> >>
> >> >>
> >>
> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:56)
> >> >>
> >> >>
> >> >>
> >>
> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:34)
> >> >>
> >> >> $Proxy39.executeTask_RequiresNew(Unknown Source)
> >> >>
> >> >>
> nc.bs.uap.scheduler.impl.TaskExecutor.executeTask(TaskExecutor.java:110)
> >> >>
> >> >> nc.bs.uap.scheduler.impl.TaskExecutor.run(TaskExecutor.java:75)
> >> >>
> >> >> nc.bs.framework.execute.Executor$1.run(Executor.java:80)
> >> >>
> >> >>
> >>
> nc.bs.framework.execute.commonj.impl.WorkAdaptor.run(WorkAdaptor.java:29)
> >> >>
> >> >> com.ibm.ws.asynchbeans.J2EEContext.run(J2EEContext.java:1114)
> >> >>
> >> >>
> >> >>
> >>
> com.ibm.ws.asynchbeans.WorkWithExecutionContextImpl.go(WorkWithExecutionContextImpl.java:195)
> >> >>
> >> >> com.ibm.ws.asynchbeans.CJWorkItemImpl.run(CJWorkItemImpl.java:187)
> >> >>
> >> >> com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1469)
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> Best regards,
> >> >>
> >> >> Zhuran Li
> >> >
> >> >
> >> >
> >> >
> >> >--
> >> >*Christian Posta*
> >> >http://www.christianposta.com/blog
> >> >twitter: @christianposta
> >>
> >
> >
> >
> >--
> >*Christian Posta*
> >http://www.christianposta.com/blog
> >twitter: @christianposta
>



--
*Christian Posta*
http://www.christianposta.com/blog
twitter: @christianposta
Reply | Threaded
Open this post in threaded view
|

Re:Re: Re: Re: ActiveMQ client app hold unexpectedly!!!

DepestBlue
I found the issue just this time. It worked fine for about two month with the ActiveMQ M/S cluster.
I've thought of the network glitch; What I feel strange most is there is no any error/exception on both the ActiveMQ client side and server side.
I went through the related codes and found there is a FutureResponse.getResult method with parameter 'timeout'. Maybe this method is more suitable for getting the result. Not very sure, hehe.








At 2013-07-16 20:23:01,"Christian Posta" <[hidden email]> wrote:

>Is this something that happens all the time? Maybe your network had a
>glitch and didn't deliver the response back to the client.
>
>
>On Tue, Jul 16, 2013 at 8:18 AM, lzr <[hidden email]> wrote:
>
>> I just check the configuration file, actually I've disabled the flow
>> control:
>>
>>
>>         <destinationPolicy>
>>             <policyMap>
>>               <policyEntries>
>>                 <policyEntry topic=">" producerFlowControl="false"
>> memoryLimit="100mb">
>>                   <pendingSubscriberPolicy>
>>                     <vmCursor />
>>                   </pendingSubscriberPolicy>
>>                 </policyEntry>
>>                 <policyEntry queue=">" producerFlowControl="false"
>> memoryLimit="100mb">
>>                   <!-- Use VM cursor for better latency
>>                        For more information, see:
>>
>>
>>                        http://activemq.apache.org/message-cursors.html
>>
>>
>>                   <pendingQueuePolicy>
>>                     <vmQueueCursor/>
>>                   </pendingQueuePolicy>
>>                   -->
>>                 </policyEntry>
>>               </policyEntries>
>>             </policyMap>
>>         </destinationPolicy>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> At 2013-07-16 20:15:11,"Christian Posta" <[hidden email]>
>> wrote:
>> >Yes, on the first time PFC kicked in you would see the log messages. Not
>> on
>> >subsequent times.
>> >
>> >
>> >On Tue, Jul 16, 2013 at 8:01 AM, lzr <[hidden email]> wrote:
>> >
>> >> Based on my experiences if the issue caused by the producer flow control
>> >> configuration, the logs will have some warning messages, right? In fact
>> I
>> >> didn't found any warning or errors in the activemq.log.
>> >> Anyway, I'll check the configuration files right now.
>> >>
>> >>
>> >> thanks lot,
>> >> Zhuran.
>> >>
>> >>
>> >> At 2013-07-16 19:36:44,"Christian Posta" <[hidden email]>
>> >> wrote:
>> >>
>> >> >This means the client didn't' receive a response from the broker.
>> >> >Did the broker producer flow control this client? ie, was the
>> destination
>> >> >to which the client sending full?
>> >> >
>> >> >
>> >> >On Tue, Jul 16, 2013 at 7:23 AM, lzr <[hidden email]> wrote:
>> >> >
>> >> >> Dear all,
>> >> >>
>> >> >>
>> >> >> I found a one of the ActiveMQ thread got stuck unexpectedly for
>> several
>> >> >> days. Customer was very angry to the ActiveMQ for the bad behavior.
>> Did
>> >> >> anybody experienced this issue? It would be much appreciated if any
>> >> clue!!!
>> >> >> My application is normal Java JMS application like the following:
>> >> >> new ActiveMQConnectionFactory.
>> >> >> create Connection;
>> >> >> conn.start.
>> >> >> create session.
>> >> >> create destination, producer and message
>> >> >> send messages
>> >> >> commit;
>> >> >> conn.close.
>> >> >>
>> >> >>
>> >> >> The follows is the stack when I looked into it with debug tools:
>> >> >>
>> >> >> sun.misc.Unsafe.park(Native Method)
>> >> >>
>> >> >> java.util.concurrent.locks.LockSupport.park(LockSupport.java:169)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1787)
>> >> >>
>> >> >>
>> >>
>> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:356)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> org.apache.activemq.transport.FutureResponse.getResult(FutureResponse.java:40)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:80)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1233)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1339)
>> >> >>
>> >> >>
>> >>
>> org.apache.activemq.ActiveMQConnection.start(ActiveMQConnection.java:488)
>> >> >>
>> >> >>
>> nc.bs.mqv2.mmgp.mq.UFMQProcessor.getConnection(UFMQProcessor.java:487)
>> >> >>
>> >> >>
>> nc.bs.mqv2.mmgp.mq.UFMQProcessor.getConnection(UFMQProcessor.java:462)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> nc.bs.mqv2.mmgp.mq.UFMQProcessor.sendSerializedObjMsg(UFMQProcessor.java:128)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> nc.bs.mqv2.meam.mq.sendbasedatas.AbstractMeamBDSyncService.sendBaseData(AbstractMeamBDSyncService.java:64)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> nc.bs.mqv2.mes.bd.mq.basedoc.AbstractBDServerSendService.dosendDataProcess(AbstractBDServerSendService.java:213)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> nc.bs.mqv2.mes.bd.mq.basedoc.AbstractBDServerSendService.sendDatas(AbstractBDServerSendService.java:62)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> nc.impl.mqv2.mes.bd.mq.SyncSendServiceImpl.ncserverSendDatas(SyncSendServiceImpl.java:58)
>> >> >>
>> >> >> nc.gejb.meam.cmt.MeamEjbBean.ncserverSendDatas(MeamEjbBean.java:8741)
>> >> >>
>> >> >>
>> >>
>> nc.gejb.meam.cmt.EJSLocalStatelessnc_gejb_meam_cmt_Meam_b44269e7.ncserverSendDatas(Unknown
>> >> >> Source)
>> >> >>
>> >> >> sun.reflect.GeneratedMethodAccessor447.invoke(Unknown Source)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> >> >>
>> >> >> java.lang.reflect.Method.invoke(Method.java:615)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:56)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:34)
>> >> >>
>> >> >> $Proxy58.ncserverSendDatas(Unknown Source)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> nc.bs.mqv2.meam.mq.task.DoSendBaseDataTaskUtil.runTask(DoSendBaseDataTaskUtil.java:63)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> nc.bs.mqv2.meam.mq.task.NCServerBaseDatasSendTask.executeTask(NCServerBaseDatasSendTask.java:26)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> nc.bs.pub.taskcenter.BgBizsPluginProcess.callBusinessPlugin(BgBizsPluginProcess.java:43)
>> >> >>
>> >> >> nc.bs.pub.pa.read.TaskBodyBase.callBizsPlugin(TaskBodyBase.java:188)
>> >> >>
>> >> >>
>> nc.bs.pub.pa.read.TaskBodyBase.callBusinessPlugin(TaskBodyBase.java:174)
>> >> >>
>> >> >> nc.bs.pub.pa.read.TaskBodyBase.createMetaMsg(TaskBodyBase.java:95)
>> >> >>
>> >> >>
>> >>
>> nc.bs.pub.pa.read.TaskBodyBase.createAndSendMessage(TaskBodyBase.java:41)
>> >> >>
>> >> >>
>> >>
>> nc.bs.pub.taskcenter.BackgroundTaskBody.execute(BackgroundTaskBody.java:47)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> nc.impl.uap.scheduler.ExecutorServiceImpl.executeTask_RequiresNew(ExecutorServiceImpl.java:27)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> nc.itf.uap.mw.MwBizEJBEjbBean.executeTask_RequiresNew(MwBizEJBEjbBean.java:1682)
>> >> >>
>> >> >>
>> >>
>> nc.itf.uap.mw.EJSLocalStatelessnc_itf_uap_mw_MwBizEJB_f98e92a8.executeTask_RequiresNew(Unknown
>> >> >> Source)
>> >> >>
>> >> >> sun.reflect.GeneratedMethodAccessor1036.invoke(Unknown Source)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> >> >>
>> >> >> java.lang.reflect.Method.invoke(Method.java:615)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:56)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:34)
>> >> >>
>> >> >> $Proxy39.executeTask_RequiresNew(Unknown Source)
>> >> >>
>> >> >>
>> nc.bs.uap.scheduler.impl.TaskExecutor.executeTask(TaskExecutor.java:110)
>> >> >>
>> >> >> nc.bs.uap.scheduler.impl.TaskExecutor.run(TaskExecutor.java:75)
>> >> >>
>> >> >> nc.bs.framework.execute.Executor$1.run(Executor.java:80)
>> >> >>
>> >> >>
>> >>
>> nc.bs.framework.execute.commonj.impl.WorkAdaptor.run(WorkAdaptor.java:29)
>> >> >>
>> >> >> com.ibm.ws.asynchbeans.J2EEContext.run(J2EEContext.java:1114)
>> >> >>
>> >> >>
>> >> >>
>> >>
>> com.ibm.ws.asynchbeans.WorkWithExecutionContextImpl.go(WorkWithExecutionContextImpl.java:195)
>> >> >>
>> >> >> com.ibm.ws.asynchbeans.CJWorkItemImpl.run(CJWorkItemImpl.java:187)
>> >> >>
>> >> >> com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1469)
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> Best regards,
>> >> >>
>> >> >> Zhuran Li
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >--
>> >> >*Christian Posta*
>> >> >http://www.christianposta.com/blog
>> >> >twitter: @christianposta
>> >>
>> >
>> >
>> >
>> >--
>> >*Christian Posta*
>> >http://www.christianposta.com/blog
>> >twitter: @christianposta
>>
>
>
>
>--
>*Christian Posta*
>http://www.christianposta.com/blog
>twitter: @christianposta
Reply | Threaded
Open this post in threaded view
|

Re:Re:Re: Re: Re: ActiveMQ client app hold unexpectedly!!!

SuoNayi-2
CONTENTS DELETED
The author has deleted this message.