Quantcast

Stuck messages - Dispatch issues

classic Classic list List threaded Threaded
27 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Stuck messages - Dispatch issues

nnprasad
 We have been experiencing the same Issue and using 5.2.0.

Recently upgraded to 5.4.2 because of this, But I am Not Sure this has been fixed.

Responding to Rob Davis Post..

I think nothing to suspect with Spring here, because when this is happening I tried hitting the same queue with a new simple consumer(AMQ connections and NO Spring JMS). This Simple consumer is also waiting on the newly sent message. created a new queue and subscribed to it while this is happening.

New Producer - new Queue - New Single Consumer = No Receive Happened.

Useful Links:
                   https://jira.springsource.org/browse/SPR-5110
                   https://issues.apache.org/jira/browse/AMQ-2009

I created an Issue with good links
                   https://issues.apache.org/jira/browse/AMQ-3225


I am really afraid of 5.4.2 because Once I saw in it...but Some one also reported the same in 5.4.2

http://activemq.2283324.n4.nabble.com/Message-stuck-on-queue-td3338786.html#a3354797


We are planning to go to production with 5.4.2, Its a public project related to claims, what if they were not processed because of activeMQ not delivering messages ?????????????????? :-(

Please take this as serious.

Thank You,
Nag. P
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Stuck messages - Dispatch issues

lernen.2007
We tested with activemq version 5.4.2. The message stuck with this version too. I think a jms provider in which the messages stuck is not useable. The activemq team should stop to fix another problems and begin to analyse what the reason for this problem is and deliver a patch as soon as possible.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Stuck messages - Dispatch issues

swplotner
Hi,

Using 5.7.0 I find that the problem exists in the following way:

2 brokers connected via network.

Broker A
        has consumer connected to a /queue/foo.test

Broker B
        has a producer connected to /queue/foo.test

the producer is producing message for /queue/foo.test and the consumer will receive them just fine. The producer is configured to stuff messages into the queue one after the other, the consumer on purpose consumes messages slowly.

Once we have several hundred messages in /queue/foo.test we abort the producer and consumer.

Looking at queue sizes on both brokers, broker A has a large number of messages and broker B has some number of messages or none.

Take the consumer that was previously talking via broker A and connect him to broker B. That consumer will now only see the messages that are in broker B and will NOT receive ANY of the messages stuck in broker A.

The only way I can get out of this problem is to restart both brokers and all messages are then process correctly.

I have spent quite a bit of time on this problem and if there is only one broker this is not a problem - the moment you have a network of 2 brokers this problem exists.

Here are my network connectors, one for topics and one for queues:
       
        <networkConnectors>
            <networkConnector name="net-node1-node2-queues"
                uri="static:(ssl://activemq-node2.amherst.edu:61615)"
                networkTTL="3"
                dynamicOnly="true"
                prefetchSize="1"
                duplex="false"
                userName="${netlink.username}"
                password="${netlink.password}"
                conduitSubscriptions="false"
                >
                <dynamicallyIncludedDestinations>
                   <queue physicalName=">"/>
                </dynamicallyIncludedDestinations>
            </networkConnector>

            <networkConnector name="net-node1-node2-topics"
                uri="static:(ssl://activemq-node2.amherst.edu:61615)"
                networkTTL="3"
                dynamicOnly="true"
                prefetchSize="1"
                duplex="false"
                userName="${netlink.username}"
                password="${netlink.password}"
                conduitSubscriptions="false"
                >
                <dynamicallyIncludedDestinations>
                    <topic physicalName=">"/>
                </dynamicallyIncludedDestinations>
            </networkConnector>
        </networkConnectors>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Stuck messages - Dispatch issues

ceposta
Looks like your network connectors are in one direction only, ie, node1 -->
node2
What this means is messages will only flow across the network FROM node1 TO
node2 based on the demand (consumers) on node2. They will not flow the
other way unless you set up a bridge in the other direction (node2 -->
node1) or use duplex="true" on your existing network connectors. Not sure
what you mean when you say restarting the brokers fixes what you were
seeing.


On Fri, Jan 11, 2013 at 8:43 PM, swplotner <[hidden email]> wrote:

> Hi,
>
> Using 5.7.0 I find that the problem exists in the following way:
>
> 2 brokers connected via network.
>
> Broker A
>         has consumer connected to a /queue/foo.test
>
> Broker B
>         has a producer connected to /queue/foo.test
>
> the producer is producing message for /queue/foo.test and the consumer will
> receive them just fine. The producer is configured to stuff messages into
> the queue one after the other, the consumer on purpose consumes messages
> slowly.
>
> Once we have several hundred messages in /queue/foo.test we abort the
> producer and consumer.
>
> Looking at queue sizes on both brokers, broker A has a large number of
> messages and broker B has some number of messages or none.
>
> Take the consumer that was previously talking via broker A and connect him
> to broker B. That consumer will now only see the messages that are in
> broker
> B and will NOT receive ANY of the messages stuck in broker A.
>
> The only way I can get out of this problem is to restart both brokers and
> all messages are then process correctly.
>
> I have spent quite a bit of time on this problem and if there is only one
> broker this is not a problem - the moment you have a network of 2 brokers
> this problem exists.
>
> Here are my network connectors, one for topics and one for queues:
>
>         <networkConnectors>
>             <networkConnector name="net-node1-node2-queues"
>                 uri="static:(ssl://activemq-node2.amherst.edu:61615)"
>                 networkTTL="3"
>                 dynamicOnly="true"
>                 prefetchSize="1"
>                 duplex="false"
>                 userName="${netlink.username}"
>                 password="${netlink.password}"
>                 conduitSubscriptions="false"
>                 >
>                 <dynamicallyIncludedDestinations>
>                    <queue physicalName=">"/>
>                 </dynamicallyIncludedDestinations>
>             </networkConnector>
>
>             <networkConnector name="net-node1-node2-topics"
>                 uri="static:(ssl://activemq-node2.amherst.edu:61615)"
>                 networkTTL="3"
>                 dynamicOnly="true"
>                 prefetchSize="1"
>                 duplex="false"
>                 userName="${netlink.username}"
>                 password="${netlink.password}"
>                 conduitSubscriptions="false"
>                 >
>                 <dynamicallyIncludedDestinations>
>                     <topic physicalName=">"/>
>                 </dynamicallyIncludedDestinations>
>             </networkConnector>
>         </networkConnectors>
>
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Stuck-messages-Dispatch-issues-tp2367852p4661679.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>



--
*Christian Posta*
http://www.christianposta.com/blog
twitter: @christianposta
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: Stuck messages - Dispatch issues

swplotner
Hi

Good point, I actually do have the same network config on the other node2 pointing to node1. However, I have now full duplex enabled and still have the same problem.

Steffen


From: ceposta [via ActiveMQ] [ml-node+[hidden email]]
Sent: Saturday, January 12, 2013 09:57
To: Steffen Plotner
Subject: Re: Stuck messages - Dispatch issues

Looks like your network connectors are in one direction only, ie, node1 -->
node2
What this means is messages will only flow across the network FROM node1 TO
node2 based on the demand (consumers) on node2. They will not flow the
other way unless you set up a bridge in the other direction (node2 -->
node1) or use duplex="true" on your existing network connectors. Not sure
what you mean when you say restarting the brokers fixes what you were
seeing.


On Fri, Jan 11, 2013 at 8:43 PM, swplotner <[hidden email]> wrote:

> Hi,
>
> Using 5.7.0 I find that the problem exists in the following way:
>
> 2 brokers connected via network.
>
> Broker A
>         has consumer connected to a /queue/foo.test
>
> Broker B
>         has a producer connected to /queue/foo.test
>
> the producer is producing message for /queue/foo.test and the consumer will
> receive them just fine. The producer is configured to stuff messages into
> the queue one after the other, the consumer on purpose consumes messages
> slowly.
>
> Once we have several hundred messages in /queue/foo.test we abort the
> producer and consumer.
>
> Looking at queue sizes on both brokers, broker A has a large number of
> messages and broker B has some number of messages or none.
>
> Take the consumer that was previously talking via broker A and connect him
> to broker B. That consumer will now only see the messages that are in
> broker
> B and will NOT receive ANY of the messages stuck in broker A.
>
> The only way I can get out of this problem is to restart both brokers and
> all messages are then process correctly.
>
> I have spent quite a bit of time on this problem and if there is only one
> broker this is not a problem - the moment you have a network of 2 brokers
> this problem exists.
>
> Here are my network connectors, one for topics and one for queues:
>
>         <networkConnectors>
>             <networkConnector name="net-node1-node2-queues"
>                 uri="static:(ssl://activemq-node2.amherst.edu:61615)"
>                 networkTTL="3"
>                 dynamicOnly="true"
>                 prefetchSize="1"
>                 duplex="false"
>                 userName="${netlink.username}"
>                 password="${netlink.password}"
>                 conduitSubscriptions="false"
>                 >
>                 <dynamicallyIncludedDestinations>
>                    <queue physicalName=">"/>
>                 </dynamicallyIncludedDestinations>
>             </networkConnector>
>
>             <networkConnector name="net-node1-node2-topics"
>                 uri="static:(ssl://activemq-node2.amherst.edu:61615)"
>                 networkTTL="3"
>                 dynamicOnly="true"
>                 prefetchSize="1"
>                 duplex="false"
>                 userName="${netlink.username}"
>                 password="${netlink.password}"
>                 conduitSubscriptions="false"
>                 >
>                 <dynamicallyIncludedDestinations>
>                     <topic physicalName=">"/>
>                 </dynamicallyIncludedDestinations>
>             </networkConnector>
>         </networkConnectors>
>
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Stuck-messages-Dispatch-issues-tp2367852p4661679.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>



--
*Christian Posta*
http://www.christianposta.com/blog
twitter: @christianposta



If you reply to this email, your message will be added to the discussion below:
http://activemq.2283324.n4.nabble.com/Stuck-messages-Dispatch-issues-tp2367852p4661693.html
To unsubscribe from Stuck messages - Dispatch issues, click here.
NAML
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Reply:RE: Stuck messages - Dispatch issues

SuoNayi-2
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: Reply:RE: Stuck messages - Dispatch issues

swplotner

Hi – thank you for pointing this out – that was it. I was able to follow the blog’s description to the dot and it solved the problem – yeah – we can go into production with this . Thank you all for listening and helping out.

 

Steffen

 

_______________________________________________________________________________________________

Steffen Plotner                            Amherst College            Tel (413) 542-2348

Systems/Network Administrator/Programmer   PO BOX 5000                Fax (413) 542-2626

Systems & Networking                       Amherst, MA 01002-5000     [hidden email]

 

From: SuoNayi [via ActiveMQ] [mailto:ml-node+[hidden email]]
Sent: Sunday, January 13, 2013 8:33 PM
To: Steffen Plotner
Subject: Reply:RE: Stuck messages - Dispatch issues

 

Hi,there is a well-known reason about stuck messages when brokers are networked.
You may take a look at http://activemq.apache.org/networks-of-brokers.html


At 2013-01-13 00:29:01,swplotner <[hidden email]> wrote:


>Hi
>
>Good point, I actually do have the same network config on the other node2 pointing to node1. However, I have now full duplex enabled and still have the same problem.
>
>Steffen
>
>________________________________
>From: ceposta [via ActiveMQ] [[hidden email]]
>Sent: Saturday, January 12, 2013 09:57
>To: Steffen Plotner
>Subject: Re: Stuck messages - Dispatch issues
>
>Looks like your network connectors are in one direction only, ie, node1 -->
>node2
>What this means is messages will only flow across the network FROM node1 TO
>node2 based on the demand (consumers) on node2. They will not flow the
>other way unless you set up a bridge in the other direction (node2 -->
>node1) or use duplex="true" on your existing network connectors. Not sure
>what you mean when you say restarting the brokers fixes what you were
>seeing.
>
>
>On Fri, Jan 11, 2013 at 8:43 PM, swplotner <[hidden email]<UrlBlockedError.aspx>> wrote:
>
>> Hi,
>>
>> Using 5.7.0 I find that the problem exists in the following way:
>>
>> 2 brokers connected via network.
>>
>> Broker A
>>         has consumer connected to a /queue/foo.test
>>
>> Broker B
>>         has a producer connected to /queue/foo.test
>>
>> the producer is producing message for /queue/foo.test and the consumer will
>> receive them just fine. The producer is configured to stuff messages into
>> the queue one after the other, the consumer on purpose consumes messages
>> slowly.
>>
>> Once we have several hundred messages in /queue/foo.test we abort the
>> producer and consumer.
>>
>> Looking at queue sizes on both brokers, broker A has a large number of
>> messages and broker B has some number of messages or none.
>>
>> Take the consumer that was previously talking via broker A and connect him
>> to broker B. That consumer will now only see the messages that are in
>> broker
>> B and will NOT receive ANY of the messages stuck in broker A.
>>
>> The only way I can get out of this problem is to restart both brokers and
>> all messages are then process correctly.
>>
>> I have spent quite a bit of time on this problem and if there is only one
>> broker this is not a problem - the moment you have a network of 2 brokers
>> this problem exists.
>>
>> Here are my network connectors, one for topics and one for queues:
>>
>>         <networkConnectors>
>>             <networkConnector name="net-node1-node2-queues"
>>                 uri="static:(ssl://activemq-node2.amherst.edu:61615)"
>>                 networkTTL="3"
>>                 dynamicOnly="true"
>>                 prefetchSize="1"
>>                 duplex="false"
>>                 userName="${netlink.username}"
>>                 password="${netlink.password}"
>>                 conduitSubscriptions="false"
>>                 >
>>                 <dynamicallyIncludedDestinations>
>>                    <queue physicalName=">"/>
>>                 </dynamicallyIncludedDestinations>
>>             </networkConnector>
>>
>>             <networkConnector name="net-node1-node2-topics"
>>                 uri="static:(ssl://activemq-node2.amherst.edu:61615)"
>>                 networkTTL="3"
>>                 dynamicOnly="true"
>>                 prefetchSize="1"
>>                 duplex="false"
>>                 userName="${netlink.username}"
>>                 password="${netlink.password}"
>>                 conduitSubscriptions="false"
>>                 >
>>                 <dynamicallyIncludedDestinations>
>>                     <topic physicalName=">"/>
>>                 </dynamicallyIncludedDestinations>
>>             </networkConnector>
>>         </networkConnectors>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://activemq.2283324.n4.nabble.com/Stuck-messages-Dispatch-issues-tp2367852p4661679.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>
>
>
>--
>*Christian Posta*
>http://www.christianposta.com/blog
>twitter: @christianposta
>http://www.christianposta.com/blog
>
>
>________________________________
>If you reply to this email, your message will be added to the discussion below:
>
>NAML<
http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
>
>
>
>--
>View this message in context: http://activemq.2283324.n4.nabble.com/Stuck-messages-Dispatch-issues-tp2367852p4661694.html
>Sent from the ActiveMQ - User mailing list archive at Nabble.com.

 


If you reply to this email, your message will be added to the discussion below:

http://activemq.2283324.n4.nabble.com/Stuck-messages-Dispatch-issues-tp2367852p4661704.html

To unsubscribe from Stuck messages - Dispatch issues, click here.
NAML

12
Loading...