Stuck messages - Dispatch issues

classic Classic list List threaded Threaded
27 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Stuck messages - Dispatch issues

chrajanirao
We are seeing issues with ActiveMQ 5.1 and 5.2 RC2 with message dispatching from queues. It is easily reproducible even using the out of the box activemq configuration.

Send messages to the a queue (QueueA) using multiple threads (10 or 20 thread in a loop of 10) using JMeter or your custom code. Have the consumer setup using spring DMLC (DefaultMessageListenerContainer) or with regular JMS API sync or async consumption with multiple consumers. Consumer part can be configured using Camel to consumer from QueueA and put the messages in QueueB within ActiveMQ configuration as well.

After receiving some messages (the number is different each time), the consumers stop receiving any messages even though there are some left on the queue. Basically, the broker don't dispatch the messages  and they are stuck until restart of the broker. Any new messages sent to the queue after this are sometimes dispatched and other times they are stuck too.

This certainly seems like a major bug in the dispatch mechanism. I have found below posts that state the exact problem.

http://www.nabble.com/Pending-Messages-are-shown-in-ActiveMQ-td20241332.html

http://www.nabble.com/Consumer-Listener-stop-receving-message-until-ActiveMQ-restart-td20355247.html

This is a very basic use case and I wonder how the version 5.1 is currently used in production, if anyone is using at all.

We tried with prefetch limit as 1, asyncDispatch as true and false, session transacted as true and false. In all cases, the dispatch problem still exists starting after 100 messages until 1000 messages.

I hope any of the active commiters looks into this issue seriously. Would really appreciate the help.

Thanks,
Rajani.
Reply | Threaded
Open this post in threaded view
|

Re: Stuck messages - Dispatch issues

joe_fernandez
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|

Re: Stuck messages - Dispatch issues

gtully
In reply to this post by chrajanirao
Once such problem[1] was resolved for 5.2.0 RC3, would it be possible
to validate RC3[2]?

What would really help here is a JUnit test case that demonstrates the problem.

[1] https://issues.apache.org/activemq/browse/AMQ-1984
[2] http://people.apache.org/~gtully/staging-repos/activemq-5.2.0/org/apache/activemq/apache-activemq/5.2.0

2008/11/12 chrajanirao <[hidden email]>:

>
> We are seeing issues with ActiveMQ 5.1 and 5.2 RC2 with message dispatching
> from queues. It is easily reproducible even using the out of the box
> activemq configuration.
>
> Send messages to the a queue (QueueA) using multiple threads (10 or 20
> thread in a loop of 10) using JMeter or your custom code. Have the consumer
> setup using spring DMLC (DefaultMessageListenerContainer) or with regular
> JMS API sync or async consumption with multiple consumers. Consumer part can
> be configured using Camel to consumer from QueueA and put the messages in
> QueueB within ActiveMQ configuration as well.
>
> After receiving some messages (the number is different each time), the
> consumers stop receiving any messages even though there are some left on the
> queue. Basically, the broker don't dispatch the messages  and they are stuck
> until restart of the broker. Any new messages sent to the queue after this
> are sometimes dispatched and other times they are stuck too.
>
> This certainly seems like a major bug in the dispatch mechanism. I have
> found below posts that state the exact problem.
>
> http://www.nabble.com/Pending-Messages-are-shown-in-ActiveMQ-td20241332.html
>
> http://www.nabble.com/Consumer-Listener-stop-receving-message-until-ActiveMQ-restart-td20355247.html
>
> This is a very basic use case and I wonder how the version 5.1 is currently
> used in production, if anyone is using at all.
>
> We tried with prefetch limit as 1, asyncDispatch as true and false, session
> transacted as true and false. In all cases, the dispatch problem still
> exists starting after 100 messages until 1000 messages.
>
> I hope any of the active commiters looks into this issue seriously. Would
> really appreciate the help.
>
> Thanks,
> Rajani.
> --
> View this message in context: http://www.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p20467949.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Stuck messages - Dispatch issues

chrajanirao
I tried 5.2 RC3 and it seem to have new bugs. It dispatching duplicates. One of my test run resulted below:

Messages sent: 1000 (using 50 threads 20 times)
Consumers: 2 trasacted

Queue Attributes in JConsole:
DequeCount: 1000
DipatchCount: 4849
EnqueueCount:1000
InFlightCount: 3849
QueueSize: 49

Consumer end:
Received: 2025

Its acting weird. when I tried with Non-transacted+Auto Ack, it sent duplicates and also the broker was still holding on to all of 1000 messages.

I will try to come up with a JUnit test case. But this is faily staright forward case when messages are sent faster than they can be consumed!!

Thanks.
Reply | Threaded
Open this post in threaded view
|

Re: Stuck messages - Dispatch issues

gtully
A test case would be great because it would be good to get this issue
fully understood. This is a potential blocker for the 5.2.0 release
IMHO. Can you raise a jira issue to track this?
thanks.

2008/11/14 chrajanirao <[hidden email]>:

>
> I tried 5.2 RC3 and it seem to have new bugs. It dispatching duplicates. One
> of my test run resulted below:
>
> Messages sent: 1000 (using 50 threads 20 times)
> Consumers: 2 trasacted
>
> Queue Attributes in JConsole:
> DequeCount: 1000
> DipatchCount: 4849
> EnqueueCount:1000
> InFlightCount: 3849
> QueueSize: 49
>
> Consumer end:
> Received: 2025
>
> Its acting weird. when I tried with Non-transacted+Auto Ack, it sent
> duplicates and also the broker was still holding on to all of 1000 messages.
>
> I will try to come up with a JUnit test case. But this is faily staright
> forward case when messages are sent faster than they can be consumed!!
>
> Thanks.
>
> --
> View this message in context: http://www.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p20492251.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Stuck messages - Dispatch issues

chrajanirao
Here is the issue that I created. Although the test case may not be of much help as I could not get it to fail with embedded broker, hope you would look into the issue.

https://issues.apache.org/activemq/browse/AMQ-2009

Thanks,
Rajani.


Gary Tully wrote
A test case would be great because it would be good to get this issue
fully understood. This is a potential blocker for the 5.2.0 release
IMHO. Can you raise a jira issue to track this?
thanks.

2008/11/14 chrajanirao <rajanicm@gmail.com>:
>
> I tried 5.2 RC3 and it seem to have new bugs. It dispatching duplicates. One
> of my test run resulted below:
>
> Messages sent: 1000 (using 50 threads 20 times)
> Consumers: 2 trasacted
>
> Queue Attributes in JConsole:
> DequeCount: 1000
> DipatchCount: 4849
> EnqueueCount:1000
> InFlightCount: 3849
> QueueSize: 49
>
> Consumer end:
> Received: 2025
>
> Its acting weird. when I tried with Non-transacted+Auto Ack, it sent
> duplicates and also the broker was still holding on to all of 1000 messages.
>
> I will try to come up with a JUnit test case. But this is faily staright
> forward case when messages are sent faster than they can be consumed!!
>
> Thanks.
>
> --
> View this message in context: http://www.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p20492251.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Stuck messages - Dispatch issues

couzteau
In reply to this post by chrajanirao
Has this been solved yet? I'm seeing it with 5.2.0.

I'm seeing an issue that occurs on some machines (where machines are identical regarding OS, Java version and hardware).

Consumers that are on the same machine as the producer usually work fine.

It's a major blocker for us - any comments highly appreciated.


TIA  Jacques



chrajanirao wrote
We are seeing issues with ActiveMQ 5.1 and 5.2 RC2 with message dispatching from queues. It is easily reproducible even using the out of the box activemq configuration.

Send messages to the a queue (QueueA) using multiple threads (10 or 20 thread in a loop of 10) using JMeter or your custom code. Have the consumer setup using spring DMLC (DefaultMessageListenerContainer) or with regular JMS API sync or async consumption with multiple consumers. Consumer part can be configured using Camel to consumer from QueueA and put the messages in QueueB within ActiveMQ configuration as well.

After receiving some messages (the number is different each time), the consumers stop receiving any messages even though there are some left on the queue. Basically, the broker don't dispatch the messages  and they are stuck until restart of the broker. Any new messages sent to the queue after this are sometimes dispatched and other times they are stuck too.

This certainly seems like a major bug in the dispatch mechanism. I have found below posts that state the exact problem.

http://www.nabble.com/Pending-Messages-are-shown-in-ActiveMQ-td20241332.html

http://www.nabble.com/Consumer-Listener-stop-receving-message-until-ActiveMQ-restart-td20355247.html

This is a very basic use case and I wonder how the version 5.1 is currently used in production, if anyone is using at all.

We tried with prefetch limit as 1, asyncDispatch as true and false, session transacted as true and false. In all cases, the dispatch problem still exists starting after 100 messages until 1000 messages.

I hope any of the active commiters looks into this issue seriously. Would really appreciate the help.

Thanks,
Rajani.
Reply | Threaded
Open this post in threaded view
|

Re: Stuck messages - Dispatch issues

gtully
in the absence of a test case, the best approach is to try out a
5.3-SNAPSHOT

2009/7/30 couzteau <[hidden email]>

>
> Has this been solved yet? I'm seeing it with 5.2.0.
>
> I'm seeing an issue that occurs on some machines (where machines are
> identical regarding OS, Java version and hardware).
>
> Consumers that are on the same machine as the producer usually work fine.
>
> It's a major blocker for us - any comments highly appreciated.
>
>
> TIA  Jacques
>
>
>
>
> chrajanirao wrote:
> >
> > We are seeing issues with ActiveMQ 5.1 and 5.2 RC2 with message
> > dispatching from queues. It is easily reproducible even using the out of
> > the box activemq configuration.
> >
> > Send messages to the a queue (QueueA) using multiple threads (10 or 20
> > thread in a loop of 10) using JMeter or your custom code. Have the
> > consumer setup using spring DMLC (DefaultMessageListenerContainer) or
> with
> > regular JMS API sync or async consumption with multiple consumers.
> > Consumer part can be configured using Camel to consumer from QueueA and
> > put the messages in QueueB within ActiveMQ configuration as well.
> >
> > After receiving some messages (the number is different each time), the
> > consumers stop receiving any messages even though there are some left on
> > the queue. Basically, the broker don't dispatch the messages  and they
> are
> > stuck until restart of the broker. Any new messages sent to the queue
> > after this are sometimes dispatched and other times they are stuck too.
> >
> > This certainly seems like a major bug in the dispatch mechanism. I have
> > found below posts that state the exact problem.
> >
> >
> http://www.nabble.com/Pending-Messages-are-shown-in-ActiveMQ-td20241332.html
> >
> >
> http://www.nabble.com/Consumer-Listener-stop-receving-message-until-ActiveMQ-restart-td20355247.html
> >
> > This is a very basic use case and I wonder how the version 5.1 is
> > currently used in production, if anyone is using at all.
> >
> > We tried with prefetch limit as 1, asyncDispatch as true and false,
> > session transacted as true and false. In all cases, the dispatch problem
> > still exists starting after 100 messages until 1000 messages.
> >
> > I hope any of the active commiters looks into this issue seriously. Would
> > really appreciate the help.
> >
> > Thanks,
> > Rajani.
> >
>
> --
> View this message in context:
> http://www.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p24733104.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>


--
http://blog.garytully.com

Open Source Integration
http://fusesource.com
Reply | Threaded
Open this post in threaded view
|

RE: Stuck messages - Dispatch issues

MatsH
In reply to this post by couzteau
Hi Jacques,

I've just recently logged the following issue:

https://issues.apache.org/activemq/browse/AMQ-2332

It sounds exactly the same as the original email in this thread, but it
is however different from the other threads the original poster linked
to.

Basically it's possible to deadlock queues, and they will then not
deliver or accept any more messages until the broker is restarted.

Regards,

Mats




> -----Original Message-----
> From: couzteau [mailto:[hidden email]]
> Sent: Thursday, July 30, 2009 2:57 AM
> To: [hidden email]
> Subject: Re: Stuck messages - Dispatch issues
>
>
> Has this been solved yet? I'm seeing it with 5.2.0.
>
> I'm seeing an issue that occurs on some machines (where machines are
> identical regarding OS, Java version and hardware).
>
> Consumers that are on the same machine as the producer usually work
fine.

>
> It's a major blocker for us - any comments highly appreciated.
>
>
> TIA  Jacques
>
>
>
>
> chrajanirao wrote:
> >
> > We are seeing issues with ActiveMQ 5.1 and 5.2 RC2 with message
> > dispatching from queues. It is easily reproducible even using the
out of
> > the box activemq configuration.
> >
> > Send messages to the a queue (QueueA) using multiple threads (10 or
20
> > thread in a loop of 10) using JMeter or your custom code. Have the
> > consumer setup using spring DMLC (DefaultMessageListenerContainer)
or
> with
> > regular JMS API sync or async consumption with multiple consumers.
> > Consumer part can be configured using Camel to consumer from QueueA
and
> > put the messages in QueueB within ActiveMQ configuration as well.
> >
> > After receiving some messages (the number is different each time),
the
> > consumers stop receiving any messages even though there are some
left on
> > the queue. Basically, the broker don't dispatch the messages  and
they
> are
> > stuck until restart of the broker. Any new messages sent to the
queue
> > after this are sometimes dispatched and other times they are stuck
too.
> >
> > This certainly seems like a major bug in the dispatch mechanism. I
have

> > found below posts that state the exact problem.
> >
> > http://www.nabble.com/Pending-Messages-are-shown-in-ActiveMQ-
> td20241332.html
> >
> > http://www.nabble.com/Consumer-Listener-stop-receving-message-until-
> ActiveMQ-restart-td20355247.html
> >
> > This is a very basic use case and I wonder how the version 5.1 is
> > currently used in production, if anyone is using at all.
> >
> > We tried with prefetch limit as 1, asyncDispatch as true and false,
> > session transacted as true and false. In all cases, the dispatch
problem

> > still exists starting after 100 messages until 1000 messages.
> >
> > I hope any of the active commiters looks into this issue seriously.
> Would
> > really appreciate the help.
> >
> > Thanks,
> > Rajani.
> >
>
> --
> View this message in context: http://www.nabble.com/Stuck-messages---
> Dispatch-issues-tp20467949p24733104.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>


Reply | Threaded
Open this post in threaded view
|

Re: Stuck messages - Dispatch issues

couzteau
In reply to this post by gtully
Thanks for the info!

I integrated 5.3-snapshot but ran into compile issues with the latest jar - So I abandoned that.


Gary Tully wrote
in the absence of a test case, the best approach is to try out a
5.3-SNAPSHOT

2009/7/30 couzteau <couzteau@bitfaeule.net>

>
> Has this been solved yet? I'm seeing it with 5.2.0.
>
> I'm seeing an issue that occurs on some machines (where machines are
> identical regarding OS, Java version and hardware).
>
> Consumers that are on the same machine as the producer usually work fine.
>
> It's a major blocker for us - any comments highly appreciated.
>
>
> TIA  Jacques
>
>
>
>
> chrajanirao wrote:
> >
> > We are seeing issues with ActiveMQ 5.1 and 5.2 RC2 with message
> > dispatching from queues. It is easily reproducible even using the out of
> > the box activemq configuration.
> >
> > Send messages to the a queue (QueueA) using multiple threads (10 or 20
> > thread in a loop of 10) using JMeter or your custom code. Have the
> > consumer setup using spring DMLC (DefaultMessageListenerContainer) or
> with
> > regular JMS API sync or async consumption with multiple consumers.
> > Consumer part can be configured using Camel to consumer from QueueA and
> > put the messages in QueueB within ActiveMQ configuration as well.
> >
> > After receiving some messages (the number is different each time), the
> > consumers stop receiving any messages even though there are some left on
> > the queue. Basically, the broker don't dispatch the messages  and they
> are
> > stuck until restart of the broker. Any new messages sent to the queue
> > after this are sometimes dispatched and other times they are stuck too.
> >
> > This certainly seems like a major bug in the dispatch mechanism. I have
> > found below posts that state the exact problem.
> >
> >
> http://www.nabble.com/Pending-Messages-are-shown-in-ActiveMQ-td20241332.html
> >
> >
> http://www.nabble.com/Consumer-Listener-stop-receving-message-until-ActiveMQ-restart-td20355247.html
> >
> > This is a very basic use case and I wonder how the version 5.1 is
> > currently used in production, if anyone is using at all.
> >
> > We tried with prefetch limit as 1, asyncDispatch as true and false,
> > session transacted as true and false. In all cases, the dispatch problem
> > still exists starting after 100 messages until 1000 messages.
> >
> > I hope any of the active commiters looks into this issue seriously. Would
> > really appreciate the help.
> >
> > Thanks,
> > Rajani.
> >
>
> --
> View this message in context:
> http://www.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p24733104.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>


--
http://blog.garytully.com

Open Source Integration
http://fusesource.com
Reply | Threaded
Open this post in threaded view
|

Re: Stuck messages - Dispatch issues

gtully
can you share the details of the compilation issues so we can help diagnose?

2009/7/30 couzteau <[hidden email]>

>
> Thanks for the info!
>
> I integrated 5.3-snapshot but ran into compile issues with the latest jar -
> So I abandoned that.
>
>
>
> Gary Tully wrote:
> >
> > in the absence of a test case, the best approach is to try out a
> > 5.3-SNAPSHOT
> >
> > 2009/7/30 couzteau <[hidden email]>
> >
> >>
> >> Has this been solved yet? I'm seeing it with 5.2.0.
> >>
> >> I'm seeing an issue that occurs on some machines (where machines are
> >> identical regarding OS, Java version and hardware).
> >>
> >> Consumers that are on the same machine as the producer usually work
> fine.
> >>
> >> It's a major blocker for us - any comments highly appreciated.
> >>
> >>
> >> TIA  Jacques
> >>
> >>
> >>
> >>
> >> chrajanirao wrote:
> >> >
> >> > We are seeing issues with ActiveMQ 5.1 and 5.2 RC2 with message
> >> > dispatching from queues. It is easily reproducible even using the out
> >> of
> >> > the box activemq configuration.
> >> >
> >> > Send messages to the a queue (QueueA) using multiple threads (10 or 20
> >> > thread in a loop of 10) using JMeter or your custom code. Have the
> >> > consumer setup using spring DMLC (DefaultMessageListenerContainer) or
> >> with
> >> > regular JMS API sync or async consumption with multiple consumers.
> >> > Consumer part can be configured using Camel to consumer from QueueA
> and
> >> > put the messages in QueueB within ActiveMQ configuration as well.
> >> >
> >> > After receiving some messages (the number is different each time), the
> >> > consumers stop receiving any messages even though there are some left
> >> on
> >> > the queue. Basically, the broker don't dispatch the messages  and they
> >> are
> >> > stuck until restart of the broker. Any new messages sent to the queue
> >> > after this are sometimes dispatched and other times they are stuck
> too.
> >> >
> >> > This certainly seems like a major bug in the dispatch mechanism. I
> have
> >> > found below posts that state the exact problem.
> >> >
> >> >
> >>
> http://www.nabble.com/Pending-Messages-are-shown-in-ActiveMQ-td20241332.html
> >> >
> >> >
> >>
> http://www.nabble.com/Consumer-Listener-stop-receving-message-until-ActiveMQ-restart-td20355247.html
> >> >
> >> > This is a very basic use case and I wonder how the version 5.1 is
> >> > currently used in production, if anyone is using at all.
> >> >
> >> > We tried with prefetch limit as 1, asyncDispatch as true and false,
> >> > session transacted as true and false. In all cases, the dispatch
> >> problem
> >> > still exists starting after 100 messages until 1000 messages.
> >> >
> >> > I hope any of the active commiters looks into this issue seriously.
> >> Would
> >> > really appreciate the help.
> >> >
> >> > Thanks,
> >> > Rajani.
> >> >
> >>
> >> --
> >> View this message in context:
> >>
> http://www.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p24733104.html
> >> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> >>
> >>
> >
> >
> > --
> > http://blog.garytully.com
> >
> > Open Source Integration
> > http://fusesource.com
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p24744031.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>


--
http://blog.garytully.com

Open Source Integration
http://fusesource.com
Reply | Threaded
Open this post in threaded view
|

Re: Stuck messages - Dispatch issues

Elliot Barlas
In reply to this post by chrajanirao
Was this issue ever resolved?  I am seeing this as well with a large number of concurrent consumers.  Same symptoms.  Lost messages until broker restart.

Thanks,
Elliot


chrajanirao wrote
We are seeing issues with ActiveMQ 5.1 and 5.2 RC2 with message dispatching from queues. It is easily reproducible even using the out of the box activemq configuration.

Send messages to the a queue (QueueA) using multiple threads (10 or 20 thread in a loop of 10) using JMeter or your custom code. Have the consumer setup using spring DMLC (DefaultMessageListenerContainer) or with regular JMS API sync or async consumption with multiple consumers. Consumer part can be configured using Camel to consumer from QueueA and put the messages in QueueB within ActiveMQ configuration as well.

After receiving some messages (the number is different each time), the consumers stop receiving any messages even though there are some left on the queue. Basically, the broker don't dispatch the messages  and they are stuck until restart of the broker. Any new messages sent to the queue after this are sometimes dispatched and other times they are stuck too.

This certainly seems like a major bug in the dispatch mechanism. I have found below posts that state the exact problem.

http://www.nabble.com/Pending-Messages-are-shown-in-ActiveMQ-td20241332.html

http://www.nabble.com/Consumer-Listener-stop-receving-message-until-ActiveMQ-restart-td20355247.html

This is a very basic use case and I wonder how the version 5.1 is currently used in production, if anyone is using at all.

We tried with prefetch limit as 1, asyncDispatch as true and false, session transacted as true and false. In all cases, the dispatch problem still exists starting after 100 messages until 1000 messages.

I hope any of the active commiters looks into this issue seriously. Would really appreciate the help.

Thanks,
Rajani.
Reply | Threaded
Open this post in threaded view
|

Re: Stuck messages - Dispatch issues

rajdavies
This should be resolved in 5.3
On 20 Feb 2010, at 07:12, Elliot Barlas wrote:

>
> Was this issue ever resolved?  I am seeing this as well with a large  
> number
> of concurrent consumers.  Same symptoms.  Lost messages until broker
> restart.
>
> Thanks,
> Elliot
>
>
>
> chrajanirao wrote:
>>
>> We are seeing issues with ActiveMQ 5.1 and 5.2 RC2 with message
>> dispatching from queues. It is easily reproducible even using the  
>> out of
>> the box activemq configuration.
>>
>> Send messages to the a queue (QueueA) using multiple threads (10 or  
>> 20
>> thread in a loop of 10) using JMeter or your custom code. Have the
>> consumer setup using spring DMLC (DefaultMessageListenerContainer)  
>> or with
>> regular JMS API sync or async consumption with multiple consumers.
>> Consumer part can be configured using Camel to consumer from QueueA  
>> and
>> put the messages in QueueB within ActiveMQ configuration as well.
>>
>> After receiving some messages (the number is different each time),  
>> the
>> consumers stop receiving any messages even though there are some  
>> left on
>> the queue. Basically, the broker don't dispatch the messages  and  
>> they are
>> stuck until restart of the broker. Any new messages sent to the queue
>> after this are sometimes dispatched and other times they are stuck  
>> too.
>>
>> This certainly seems like a major bug in the dispatch mechanism. I  
>> have
>> found below posts that state the exact problem.
>>
>> http://www.nabble.com/Pending-Messages-are-shown-in-ActiveMQ-td20241332.html
>>
>> http://www.nabble.com/Consumer-Listener-stop-receving-message-until-ActiveMQ-restart-td20355247.html
>>
>> This is a very basic use case and I wonder how the version 5.1 is
>> currently used in production, if anyone is using at all.
>>
>> We tried with prefetch limit as 1, asyncDispatch as true and false,
>> session transacted as true and false. In all cases, the dispatch  
>> problem
>> still exists starting after 100 messages until 1000 messages.
>>
>> I hope any of the active commiters looks into this issue seriously.  
>> Would
>> really appreciate the help.
>>
>> Thanks,
>> Rajani.
>>
>
> --
> View this message in context: http://old.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p27664135.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>

Rob Davies
http://twitter.com/rajdavies
I work here: http://fusesource.com
My Blog: http://rajdavies.blogspot.com/
I'm writing this: http://www.manning.com/snyder/





Reply | Threaded
Open this post in threaded view
|

Re: Stuck messages - Dispatch issues

Elliot Barlas
Unfortunately I am seeing it in 5.3.  I just posted on this more current thread as well: http://old.nabble.com/50k-%2B-messages-stuck-in-queue-with-all-consumers-blocking-on-receive-td27162095.html

Thanks,
Elliot


rajdavies wrote
This should be resolved in 5.3
On 20 Feb 2010, at 07:12, Elliot Barlas wrote:

>
> Was this issue ever resolved?  I am seeing this as well with a large  
> number
> of concurrent consumers.  Same symptoms.  Lost messages until broker
> restart.
>
> Thanks,
> Elliot
>
>
>
> chrajanirao wrote:
>>
>> We are seeing issues with ActiveMQ 5.1 and 5.2 RC2 with message
>> dispatching from queues. It is easily reproducible even using the  
>> out of
>> the box activemq configuration.
>>
>> Send messages to the a queue (QueueA) using multiple threads (10 or  
>> 20
>> thread in a loop of 10) using JMeter or your custom code. Have the
>> consumer setup using spring DMLC (DefaultMessageListenerContainer)  
>> or with
>> regular JMS API sync or async consumption with multiple consumers.
>> Consumer part can be configured using Camel to consumer from QueueA  
>> and
>> put the messages in QueueB within ActiveMQ configuration as well.
>>
>> After receiving some messages (the number is different each time),  
>> the
>> consumers stop receiving any messages even though there are some  
>> left on
>> the queue. Basically, the broker don't dispatch the messages  and  
>> they are
>> stuck until restart of the broker. Any new messages sent to the queue
>> after this are sometimes dispatched and other times they are stuck  
>> too.
>>
>> This certainly seems like a major bug in the dispatch mechanism. I  
>> have
>> found below posts that state the exact problem.
>>
>> http://www.nabble.com/Pending-Messages-are-shown-in-ActiveMQ-td20241332.html
>>
>> http://www.nabble.com/Consumer-Listener-stop-receving-message-until-ActiveMQ-restart-td20355247.html
>>
>> This is a very basic use case and I wonder how the version 5.1 is
>> currently used in production, if anyone is using at all.
>>
>> We tried with prefetch limit as 1, asyncDispatch as true and false,
>> session transacted as true and false. In all cases, the dispatch  
>> problem
>> still exists starting after 100 messages until 1000 messages.
>>
>> I hope any of the active commiters looks into this issue seriously.  
>> Would
>> really appreciate the help.
>>
>> Thanks,
>> Rajani.
>>
>
> --
> View this message in context: http://old.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p27664135.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>

Rob Davies
http://twitter.com/rajdavies
I work here: http://fusesource.com
My Blog: http://rajdavies.blogspot.com/
I'm writing this: http://www.manning.com/snyder/




Reply | Threaded
Open this post in threaded view
|

Re: Stuck messages - Dispatch issues

Pavel-3
Sounds scary to me, as system I'm working on uses activemq extensively, and
is not production-proven yet.

Do you have/can you build a junit test case that reproduces the issue under
5.3? If so, that looks like a good start for Jira defect.

Disclaimer - I'm not an AMQ committer.

Thanks,
Pavel

On Sat, Feb 20, 2010 at 9:23 AM, Elliot Barlas <[hidden email]>wrote:

>
> Unfortunately I am seeing it in 5.3.  I just posted on this more current
> thread as well:
>
> http://old.nabble.com/50k-%2B-messages-stuck-in-queue-with-all-consumers-blocking-on-receive-td27162095.html
>
> Thanks,
> Elliot
>
>
>
> rajdavies wrote:
> >
> > This should be resolved in 5.3
> > On 20 Feb 2010, at 07:12, Elliot Barlas wrote:
> >
> >>
> >> Was this issue ever resolved?  I am seeing this as well with a large
> >> number
> >> of concurrent consumers.  Same symptoms.  Lost messages until broker
> >> restart.
> >>
> >> Thanks,
> >> Elliot
> >>
> >>
> >>
> >> chrajanirao wrote:
> >>>
> >>> We are seeing issues with ActiveMQ 5.1 and 5.2 RC2 with message
> >>> dispatching from queues. It is easily reproducible even using the
> >>> out of
> >>> the box activemq configuration.
> >>>
> >>> Send messages to the a queue (QueueA) using multiple threads (10 or
> >>> 20
> >>> thread in a loop of 10) using JMeter or your custom code. Have the
> >>> consumer setup using spring DMLC (DefaultMessageListenerContainer)
> >>> or with
> >>> regular JMS API sync or async consumption with multiple consumers.
> >>> Consumer part can be configured using Camel to consumer from QueueA
> >>> and
> >>> put the messages in QueueB within ActiveMQ configuration as well.
> >>>
> >>> After receiving some messages (the number is different each time),
> >>> the
> >>> consumers stop receiving any messages even though there are some
> >>> left on
> >>> the queue. Basically, the broker don't dispatch the messages  and
> >>> they are
> >>> stuck until restart of the broker. Any new messages sent to the queue
> >>> after this are sometimes dispatched and other times they are stuck
> >>> too.
> >>>
> >>> This certainly seems like a major bug in the dispatch mechanism. I
> >>> have
> >>> found below posts that state the exact problem.
> >>>
> >>>
> http://www.nabble.com/Pending-Messages-are-shown-in-ActiveMQ-td20241332.html
> >>>
> >>>
> http://www.nabble.com/Consumer-Listener-stop-receving-message-until-ActiveMQ-restart-td20355247.html
> >>>
> >>> This is a very basic use case and I wonder how the version 5.1 is
> >>> currently used in production, if anyone is using at all.
> >>>
> >>> We tried with prefetch limit as 1, asyncDispatch as true and false,
> >>> session transacted as true and false. In all cases, the dispatch
> >>> problem
> >>> still exists starting after 100 messages until 1000 messages.
> >>>
> >>> I hope any of the active commiters looks into this issue seriously.
> >>> Would
> >>> really appreciate the help.
> >>>
> >>> Thanks,
> >>> Rajani.
> >>>
> >>
> >> --
> >> View this message in context:
> >>
> http://old.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p27664135.html
> >> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> >>
> >
> > Rob Davies
> > http://twitter.com/rajdavies
> > I work here: http://fusesource.com
> > My Blog: http://rajdavies.blogspot.com/
> > I'm writing this: http://www.manning.com/snyder/
> >
> >
> >
> >
> >
> >
> >
>
> --
> View this message in context:
> http://old.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p27664182.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Stuck messages - Dispatch issues

rajdavies
In reply to this post by Elliot Barlas
Do you still get problems if not using Spring ?

On 20 Feb 2010, at 07:12, Elliot Barlas wrote:

>
> Was this issue ever resolved?  I am seeing this as well with a large  
> number
> of concurrent consumers.  Same symptoms.  Lost messages until broker
> restart.
>
> Thanks,
> Elliot
>
>
>
> chrajanirao wrote:
>>
>> We are seeing issues with ActiveMQ 5.1 and 5.2 RC2 with message
>> dispatching from queues. It is easily reproducible even using the  
>> out of
>> the box activemq configuration.
>>
>> Send messages to the a queue (QueueA) using multiple threads (10 or  
>> 20
>> thread in a loop of 10) using JMeter or your custom code. Have the
>> consumer setup using spring DMLC (DefaultMessageListenerContainer)  
>> or with
>> regular JMS API sync or async consumption with multiple consumers.
>> Consumer part can be configured using Camel to consumer from QueueA  
>> and
>> put the messages in QueueB within ActiveMQ configuration as well.
>>
>> After receiving some messages (the number is different each time),  
>> the
>> consumers stop receiving any messages even though there are some  
>> left on
>> the queue. Basically, the broker don't dispatch the messages  and  
>> they are
>> stuck until restart of the broker. Any new messages sent to the queue
>> after this are sometimes dispatched and other times they are stuck  
>> too.
>>
>> This certainly seems like a major bug in the dispatch mechanism. I  
>> have
>> found below posts that state the exact problem.
>>
>> http://www.nabble.com/Pending-Messages-are-shown-in-ActiveMQ-td20241332.html
>>
>> http://www.nabble.com/Consumer-Listener-stop-receving-message-until-ActiveMQ-restart-td20355247.html
>>
>> This is a very basic use case and I wonder how the version 5.1 is
>> currently used in production, if anyone is using at all.
>>
>> We tried with prefetch limit as 1, asyncDispatch as true and false,
>> session transacted as true and false. In all cases, the dispatch  
>> problem
>> still exists starting after 100 messages until 1000 messages.
>>
>> I hope any of the active commiters looks into this issue seriously.  
>> Would
>> really appreciate the help.
>>
>> Thanks,
>> Rajani.
>>
>
> --
> View this message in context: http://old.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p27664135.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>

Rob Davies
http://twitter.com/rajdavies
I work here: http://fusesource.com
My Blog: http://rajdavies.blogspot.com/
I'm writing this: http://www.manning.com/snyder/





Reply | Threaded
Open this post in threaded view
|

Re: Stuck messages - Dispatch issues

Elliot Barlas
Yes.  I am using spring 3.0, but not for JMS.  I manage my own connections, sessions, producers, and consumers.  I have tried reproducing the issue in a test case, but I cannot.  It only occurs in my application, when I have roughly 20 or more concurrent consumers, each associated with a different session.  I am using the VM transport exclusively, as all communication takes place with an embedded AMQ message broker.  

Thanks,
Elliot


rajdavies wrote
Do you still get problems if not using Spring ?

On 20 Feb 2010, at 07:12, Elliot Barlas wrote:

>
> Was this issue ever resolved?  I am seeing this as well with a large  
> number
> of concurrent consumers.  Same symptoms.  Lost messages until broker
> restart.
>
> Thanks,
> Elliot
>
>
>
> chrajanirao wrote:
>>
>> We are seeing issues with ActiveMQ 5.1 and 5.2 RC2 with message
>> dispatching from queues. It is easily reproducible even using the  
>> out of
>> the box activemq configuration.
>>
>> Send messages to the a queue (QueueA) using multiple threads (10 or  
>> 20
>> thread in a loop of 10) using JMeter or your custom code. Have the
>> consumer setup using spring DMLC (DefaultMessageListenerContainer)  
>> or with
>> regular JMS API sync or async consumption with multiple consumers.
>> Consumer part can be configured using Camel to consumer from QueueA  
>> and
>> put the messages in QueueB within ActiveMQ configuration as well.
>>
>> After receiving some messages (the number is different each time),  
>> the
>> consumers stop receiving any messages even though there are some  
>> left on
>> the queue. Basically, the broker don't dispatch the messages  and  
>> they are
>> stuck until restart of the broker. Any new messages sent to the queue
>> after this are sometimes dispatched and other times they are stuck  
>> too.
>>
>> This certainly seems like a major bug in the dispatch mechanism. I  
>> have
>> found below posts that state the exact problem.
>>
>> http://www.nabble.com/Pending-Messages-are-shown-in-ActiveMQ-td20241332.html
>>
>> http://www.nabble.com/Consumer-Listener-stop-receving-message-until-ActiveMQ-restart-td20355247.html
>>
>> This is a very basic use case and I wonder how the version 5.1 is
>> currently used in production, if anyone is using at all.
>>
>> We tried with prefetch limit as 1, asyncDispatch as true and false,
>> session transacted as true and false. In all cases, the dispatch  
>> problem
>> still exists starting after 100 messages until 1000 messages.
>>
>> I hope any of the active commiters looks into this issue seriously.  
>> Would
>> really appreciate the help.
>>
>> Thanks,
>> Rajani.
>>
>
> --
> View this message in context: http://old.nabble.com/Stuck-messages---Dispatch-issues-tp20467949p27664135.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>

Rob Davies
http://twitter.com/rajdavies
I work here: http://fusesource.com
My Blog: http://rajdavies.blogspot.com/
I'm writing this: http://www.manning.com/snyder/




Reply | Threaded
Open this post in threaded view
|

Re: Stuck messages - Dispatch issues

timi.blossom
This post has NOT been accepted by the mailing list yet.
Hi everyone,

Did we resolve this issue?  My company is planning to roll out production a project which depends on ActiveMQ.  Should we continue with ActiveMQ or abort it? Thanks
Reply | Threaded
Open this post in threaded view
|

Re: Stuck messages - Dispatch issues

njain
This post has NOT been accepted by the mailing list yet.
I am seeing the same issue in our environment, with activemq 5.3.0.4-FUSE. It happens when there are more than one queue consumers. After some time, the consumers stop consuming messages from some queues. Restarting activemq fixes the issue.
Reply | Threaded
Open this post in threaded view
|

Re: Stuck messages - Dispatch issues

lernen.2007
In reply to this post by timi.blossom
Hi,

the problem we have too. Sometimes the messages stuck in queue and after a restart it deliver to the consumer.
12