Need Help With A JMS Exception

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Need Help With A JMS Exception

Michael.CTR.Tarullo
We had a rather unique situation some time ago where our message broker log was recording the JMSException for an Invalid acknowledgement.

We traced it to the fact that the consumer of the message was using a more recent version of ActiveMQ than our message broker was running (i.e. message broker was v5.5.1 and the consumer was >v5.5.1).

We would like to recreate this error in one of our test environments so that we can learn more about it.

It appears that the exception was related to a consumer acknowledging a message.  We determined this from observing that the specific exception that was thrown appears to originate in only one place, that is the acknowledge() function of the TopicSubscription class.

I have configured an environment with a v5.5.1 message broker and a consumer and producer compiled with v5.11.0 of ActiveMQ.  The producer sends messages to a topic and the consumer receives these messages and then acknowledges those messages.  The session was configured for client acknowledgement.  Under these conditions I could not reproduce the exception described above.

I tried looking through the source code for the acknowledge() function of the ActiveMQMessage class to see if I could determine how my consumer call to acknowkedge() landed on the TopicSubscription acknowledge(), but have been unable to make a connection between the acknowledge() function in ActiveMQMessage and TopicSubscription even after following the calls starting with the call to ActiveMQMessage acknowledge().

Can you offer any explanation as to why I am unable to trace the functions calls to TopicSubscription acknowledge?  And more importantly, can you offer any advice on how I might recreate the exception condition described here?

Thanks,
Mike

Michael Tarullo
Contractor (Engility Corp)
Software Engineer
FAA WJH Technical Center
(609)485-5294

Reply | Threaded
Open this post in threaded view
|

Re: Need Help With A JMS Exception

Tim Bain
Since you have the 5.5.1 source code, I recommend you attach a debugger to
the broker process and set a breakpoint on the
TopicSubscription.acknowledge() method; the stack will tell you how you got
there.  Setting breakpoints on the broker can lead to instability and may
force you to restart it (plus you have to restart it to enable the debug
port and again to disable it, and it's a security vulnerability while it's
open), so do this in a dev/test environment, not a production one.

You should not expect that there would be a call chain between the two
methods you referenced.  One is executed on the client and one is executed
on the broker, so between the two there is the creation, transmission, and
handling of the acknowledgement message.

Tim

On Aug 31, 2016 4:15 PM, <[hidden email]> wrote:

> We had a rather unique situation some time ago where our message broker
> log was recording the JMSException for an Invalid acknowledgement.
>
> We traced it to the fact that the consumer of the message was using a more
> recent version of ActiveMQ than our message broker was running (i.e.
> message broker was v5.5.1 and the consumer was >v5.5.1).
>
> We would like to recreate this error in one of our test environments so
> that we can learn more about it.
>
> It appears that the exception was related to a consumer acknowledging a
> message.  We determined this from observing that the specific exception
> that was thrown appears to originate in only one place, that is the
> acknowledge() function of the TopicSubscription class.
>
> I have configured an environment with a v5.5.1 message broker and a
> consumer and producer compiled with v5.11.0 of ActiveMQ.  The producer
> sends messages to a topic and the consumer receives these messages and then
> acknowledges those messages.  The session was configured for client
> acknowledgement.  Under these conditions I could not reproduce the
> exception described above.
>
> I tried looking through the source code for the acknowledge() function of
> the ActiveMQMessage class to see if I could determine how my consumer call
> to acknowkedge() landed on the TopicSubscription acknowledge(), but have
> been unable to make a connection between the acknowledge() function in
> ActiveMQMessage and TopicSubscription even after following the calls
> starting with the call to ActiveMQMessage acknowledge().
>
> Can you offer any explanation as to why I am unable to trace the functions
> calls to TopicSubscription acknowledge?  And more importantly, can you
> offer any advice on how I might recreate the exception condition described
> here?
>
> Thanks,
> Mike
>
> Michael Tarullo
> Contractor (Engility Corp)
> Software Engineer
> FAA WJH Technical Center
> (609)485-5294
>
>
Reply | Threaded
Open this post in threaded view
|

RE: Need Help With A JMS Exception

Michael.CTR.Tarullo
Thank you Tim.

Michael Tarullo
Contractor (Engility Corp)
Software Engineer
FAA WJH Technical Center
(609)485-5294

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Tim Bain
Sent: Thursday, September 01, 2016 8:47 AM
To: ActiveMQ Users
Subject: Re: Need Help With A JMS Exception

Since you have the 5.5.1 source code, I recommend you attach a debugger to the broker process and set a breakpoint on the
TopicSubscription.acknowledge() method; the stack will tell you how you got there.  Setting breakpoints on the broker can lead to instability and may force you to restart it (plus you have to restart it to enable the debug port and again to disable it, and it's a security vulnerability while it's open), so do this in a dev/test environment, not a production one.

You should not expect that there would be a call chain between the two methods you referenced.  One is executed on the client and one is executed on the broker, so between the two there is the creation, transmission, and handling of the acknowledgement message.

Tim

On Aug 31, 2016 4:15 PM, <[hidden email]> wrote:

> We had a rather unique situation some time ago where our message
> broker log was recording the JMSException for an Invalid acknowledgement.
>
> We traced it to the fact that the consumer of the message was using a
> more recent version of ActiveMQ than our message broker was running (i.e.
> message broker was v5.5.1 and the consumer was >v5.5.1).
>
> We would like to recreate this error in one of our test environments
> so that we can learn more about it.
>
> It appears that the exception was related to a consumer acknowledging
> a message.  We determined this from observing that the specific
> exception that was thrown appears to originate in only one place, that
> is the
> acknowledge() function of the TopicSubscription class.
>
> I have configured an environment with a v5.5.1 message broker and a
> consumer and producer compiled with v5.11.0 of ActiveMQ.  The producer
> sends messages to a topic and the consumer receives these messages and
> then acknowledges those messages.  The session was configured for
> client acknowledgement.  Under these conditions I could not reproduce
> the exception described above.
>
> I tried looking through the source code for the acknowledge() function
> of the ActiveMQMessage class to see if I could determine how my
> consumer call to acknowkedge() landed on the TopicSubscription
> acknowledge(), but have been unable to make a connection between the
> acknowledge() function in ActiveMQMessage and TopicSubscription even
> after following the calls starting with the call to ActiveMQMessage acknowledge().
>
> Can you offer any explanation as to why I am unable to trace the
> functions calls to TopicSubscription acknowledge?  And more
> importantly, can you offer any advice on how I might recreate the
> exception condition described here?
>
> Thanks,
> Mike
>
> Michael Tarullo
> Contractor (Engility Corp)
> Software Engineer
> FAA WJH Technical Center
> (609)485-5294
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Need Help With A JMS Exception

Matt Pavlovich-2
In reply to this post by Tim Bain
+1 Agree that debugging it is the only surefire way to understand what
is going on.

On a side note-- CLIENT_ACKNOWLEDGE is kludgy of sorts. Be sure to
understand that it acknowledges the current message and _all_ previous
messages. It isn't an individual message acknowledgement as most people
are hoping. To get that, you need to use ActiveMQ's extension
ActiveMQSession.INDIVIDUAL_ACKNOWLEDGE or go full blown TRANSACTED.

All JMS providers have this "extended" ack mode and there is a JMS feat
request to make INDIVIDUAL_ACKNOWLEDGE a standardized thing:

Vote here: https://java.net/jira/browse/JMS_SPEC-95

On 9/1/16 7:47 AM, Tim Bain wrote:

> Since you have the 5.5.1 source code, I recommend you attach a debugger to
> the broker process and set a breakpoint on the
> TopicSubscription.acknowledge() method; the stack will tell you how you got
> there.  Setting breakpoints on the broker can lead to instability and may
> force you to restart it (plus you have to restart it to enable the debug
> port and again to disable it, and it's a security vulnerability while it's
> open), so do this in a dev/test environment, not a production one.
>
> You should not expect that there would be a call chain between the two
> methods you referenced.  One is executed on the client and one is executed
> on the broker, so between the two there is the creation, transmission, and
> handling of the acknowledgement message.
>
> Tim
>
> On Aug 31, 2016 4:15 PM, <[hidden email]> wrote:
>
>> We had a rather unique situation some time ago where our message broker
>> log was recording the JMSException for an Invalid acknowledgement.
>>
>> We traced it to the fact that the consumer of the message was using a more
>> recent version of ActiveMQ than our message broker was running (i.e.
>> message broker was v5.5.1 and the consumer was >v5.5.1).
>>
>> We would like to recreate this error in one of our test environments so
>> that we can learn more about it.
>>
>> It appears that the exception was related to a consumer acknowledging a
>> message.  We determined this from observing that the specific exception
>> that was thrown appears to originate in only one place, that is the
>> acknowledge() function of the TopicSubscription class.
>>
>> I have configured an environment with a v5.5.1 message broker and a
>> consumer and producer compiled with v5.11.0 of ActiveMQ.  The producer
>> sends messages to a topic and the consumer receives these messages and then
>> acknowledges those messages.  The session was configured for client
>> acknowledgement.  Under these conditions I could not reproduce the
>> exception described above.
>>
>> I tried looking through the source code for the acknowledge() function of
>> the ActiveMQMessage class to see if I could determine how my consumer call
>> to acknowkedge() landed on the TopicSubscription acknowledge(), but have
>> been unable to make a connection between the acknowledge() function in
>> ActiveMQMessage and TopicSubscription even after following the calls
>> starting with the call to ActiveMQMessage acknowledge().
>>
>> Can you offer any explanation as to why I am unable to trace the functions
>> calls to TopicSubscription acknowledge?  And more importantly, can you
>> offer any advice on how I might recreate the exception condition described
>> here?
>>
>> Thanks,
>> Mike
>>
>> Michael Tarullo
>> Contractor (Engility Corp)
>> Software Engineer
>> FAA WJH Technical Center
>> (609)485-5294
>>
>>

Reply | Threaded
Open this post in threaded view
|

RE: Need Help With A JMS Exception

Michael.CTR.Tarullo
Thanks Matt.  And yes I was aware of the behavior of CLIENT_ACK, although when I started this research I was not.  Before this I was under the impression that it was a single message ack.

Michael Tarullo
Contractor (Engility Corp)
Software Engineer
FAA WJH Technical Center
(609)485-5294


-----Original Message-----
From: Matt Pavlovich [mailto:[hidden email]]
Sent: Thursday, September 01, 2016 10:58 AM
To: [hidden email]
Subject: Re: Need Help With A JMS Exception

+1 Agree that debugging it is the only surefire way to understand what
is going on.

On a side note-- CLIENT_ACKNOWLEDGE is kludgy of sorts. Be sure to
understand that it acknowledges the current message and _all_ previous
messages. It isn't an individual message acknowledgement as most people
are hoping. To get that, you need to use ActiveMQ's extension
ActiveMQSession.INDIVIDUAL_ACKNOWLEDGE or go full blown TRANSACTED.

All JMS providers have this "extended" ack mode and there is a JMS feat
request to make INDIVIDUAL_ACKNOWLEDGE a standardized thing:

Vote here: https://java.net/jira/browse/JMS_SPEC-95

On 9/1/16 7:47 AM, Tim Bain wrote:

> Since you have the 5.5.1 source code, I recommend you attach a debugger to
> the broker process and set a breakpoint on the
> TopicSubscription.acknowledge() method; the stack will tell you how you got
> there.  Setting breakpoints on the broker can lead to instability and may
> force you to restart it (plus you have to restart it to enable the debug
> port and again to disable it, and it's a security vulnerability while it's
> open), so do this in a dev/test environment, not a production one.
>
> You should not expect that there would be a call chain between the two
> methods you referenced.  One is executed on the client and one is executed
> on the broker, so between the two there is the creation, transmission, and
> handling of the acknowledgement message.
>
> Tim
>
> On Aug 31, 2016 4:15 PM, <[hidden email]> wrote:
>
>> We had a rather unique situation some time ago where our message broker
>> log was recording the JMSException for an Invalid acknowledgement.
>>
>> We traced it to the fact that the consumer of the message was using a more
>> recent version of ActiveMQ than our message broker was running (i.e.
>> message broker was v5.5.1 and the consumer was >v5.5.1).
>>
>> We would like to recreate this error in one of our test environments so
>> that we can learn more about it.
>>
>> It appears that the exception was related to a consumer acknowledging a
>> message.  We determined this from observing that the specific exception
>> that was thrown appears to originate in only one place, that is the
>> acknowledge() function of the TopicSubscription class.
>>
>> I have configured an environment with a v5.5.1 message broker and a
>> consumer and producer compiled with v5.11.0 of ActiveMQ.  The producer
>> sends messages to a topic and the consumer receives these messages and then
>> acknowledges those messages.  The session was configured for client
>> acknowledgement.  Under these conditions I could not reproduce the
>> exception described above.
>>
>> I tried looking through the source code for the acknowledge() function of
>> the ActiveMQMessage class to see if I could determine how my consumer call
>> to acknowkedge() landed on the TopicSubscription acknowledge(), but have
>> been unable to make a connection between the acknowledge() function in
>> ActiveMQMessage and TopicSubscription even after following the calls
>> starting with the call to ActiveMQMessage acknowledge().
>>
>> Can you offer any explanation as to why I am unable to trace the functions
>> calls to TopicSubscription acknowledge?  And more importantly, can you
>> offer any advice on how I might recreate the exception condition described
>> here?
>>
>> Thanks,
>> Mike
>>
>> Michael Tarullo
>> Contractor (Engility Corp)
>> Software Engineer
>> FAA WJH Technical Center
>> (609)485-5294
>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: Need Help With A JMS Exception

artnaseef
A couple of thoughts on this front.

First, how was the conclusion of different client versions made?  I'm curious because I can't think of any way that should be an issue.

Second, is the consumer using the failover transport?

This JMS Exception can happen normally without indicating a problem, especially with the failover transport, as the consumer may then send an acknowledgement to the broker after it has already received another acknowledgement for the same message.  Or, with Topics, the rebuilt subscription may not have the message - I'm not sure how the failover transport handles non-durable Topic subscriptions.  My suspicion is that the new subscription, after failover, will start fresh, with none of the old messages.  So, if the consumer then proceeds to send an acknowledgement for the message, the broker won't be able to match the message.



On Thu, Sep 1, 2016 at 11:13 AM, Michael.CTR.Tarullo [via ActiveMQ] <[hidden email]> wrote:
Thanks Matt.  And yes I was aware of the behavior of CLIENT_ACK, although when I started this research I was not.  Before this I was under the impression that it was a single message ack.

Michael Tarullo
Contractor (Engility Corp)
Software Engineer
FAA WJH Technical Center
<a href="tel:%28609%29485-5294" value="+16094855294" target="_blank">(609)485-5294


-----Original Message-----
From: Matt Pavlovich [mailto:[hidden email]]
Sent: Thursday, September 01, 2016 10:58 AM
To: [hidden email]
Subject: Re: Need Help With A JMS Exception

+1 Agree that debugging it is the only surefire way to understand what
is going on.

On a side note-- CLIENT_ACKNOWLEDGE is kludgy of sorts. Be sure to
understand that it acknowledges the current message and _all_ previous
messages. It isn't an individual message acknowledgement as most people
are hoping. To get that, you need to use ActiveMQ's extension
ActiveMQSession.INDIVIDUAL_ACKNOWLEDGE or go full blown TRANSACTED.

All JMS providers have this "extended" ack mode and there is a JMS feat
request to make INDIVIDUAL_ACKNOWLEDGE a standardized thing:

Vote here: https://java.net/jira/browse/JMS_SPEC-95

On 9/1/16 7:47 AM, Tim Bain wrote:

> Since you have the 5.5.1 source code, I recommend you attach a debugger to
> the broker process and set a breakpoint on the
> TopicSubscription.acknowledge() method; the stack will tell you how you got
> there.  Setting breakpoints on the broker can lead to instability and may
> force you to restart it (plus you have to restart it to enable the debug
> port and again to disable it, and it's a security vulnerability while it's
> open), so do this in a dev/test environment, not a production one.
>
> You should not expect that there would be a call chain between the two
> methods you referenced.  One is executed on the client and one is executed
> on the broker, so between the two there is the creation, transmission, and
> handling of the acknowledgement message.
>
> Tim
>
> On Aug 31, 2016 4:15 PM, <[hidden email]> wrote:
>
>> We had a rather unique situation some time ago where our message broker
>> log was recording the JMSException for an Invalid acknowledgement.
>>
>> We traced it to the fact that the consumer of the message was using a more
>> recent version of ActiveMQ than our message broker was running (i.e.
>> message broker was v5.5.1 and the consumer was >v5.5.1).
>>
>> We would like to recreate this error in one of our test environments so
>> that we can learn more about it.
>>
>> It appears that the exception was related to a consumer acknowledging a
>> message.  We determined this from observing that the specific exception
>> that was thrown appears to originate in only one place, that is the
>> acknowledge() function of the TopicSubscription class.
>>
>> I have configured an environment with a v5.5.1 message broker and a
>> consumer and producer compiled with v5.11.0 of ActiveMQ.  The producer
>> sends messages to a topic and the consumer receives these messages and then
>> acknowledges those messages.  The session was configured for client
>> acknowledgement.  Under these conditions I could not reproduce the
>> exception described above.
>>
>> I tried looking through the source code for the acknowledge() function of
>> the ActiveMQMessage class to see if I could determine how my consumer call
>> to acknowkedge() landed on the TopicSubscription acknowledge(), but have
>> been unable to make a connection between the acknowledge() function in
>> ActiveMQMessage and TopicSubscription even after following the calls
>> starting with the call to ActiveMQMessage acknowledge().
>>
>> Can you offer any explanation as to why I am unable to trace the functions
>> calls to TopicSubscription acknowledge?  And more importantly, can you
>> offer any advice on how I might recreate the exception condition described
>> here?
>>
>> Thanks,
>> Mike
>>
>> Michael Tarullo
>> Contractor (Engility Corp)
>> Software Engineer
>> FAA WJH Technical Center
>> <a href="tel:%28609%29485-5294" value="+16094855294" target="_blank">(609)485-5294
>>
>>



If you reply to this email, your message will be added to the discussion below:
http://activemq.2283324.n4.nabble.com/Need-Help-With-A-JMS-Exception-tp4716045p4716062.html
To start a new topic under ActiveMQ - User, email [hidden email]
To unsubscribe from ActiveMQ - User, click here.
NAML

Reply | Threaded
Open this post in threaded view
|

RE: Need Help With A JMS Exception

Michael.CTR.Tarullo
I am not certain about this but I think the version mismatch was discovered because the ackType in the exception was = 6, and someone (not me  because at that time I was not involved with the project) noticed that v5.5.1 did not support ackType = 6, i.e. the max ackType for v5.5.1 is 5.

I don't know if the client was using failover transport.  I'm not sure if this was ever investigated but I can try to find out.

Thanks for your thoughts.

Michael Tarullo
Contractor (Engility Corp)
Software Engineer
FAA WJH Technical Center
(609)485-5294


-----Original Message-----
From: artnaseef [mailto:[hidden email]]
Sent: Thursday, September 01, 2016 12:05 PM
To: [hidden email]
Subject: Re: Need Help With A JMS Exception

A couple of thoughts on this front.

First, how was the conclusion of different client versions made?  I'm curious because I can't think of any way that should be an issue.

Second, is the consumer using the failover transport?

This JMS Exception can happen normally without indicating a problem, especially with the failover transport, as the consumer may then send an acknowledgement to the broker after it has already received another acknowledgement for the same message.  Or, with Topics, the rebuilt subscription may not have the message - I'm not sure how the failover transport handles non-durable Topic subscriptions.  My suspicion is that the new subscription, after failover, will start fresh, with none of the old messages.  So, if the consumer then proceeds to send an acknowledgement for the message, the broker won't be able to match the message.



On Thu, Sep 1, 2016 at 11:13 AM, Michael.CTR.Tarullo [via ActiveMQ] <
[hidden email]> wrote:

> Thanks Matt.  And yes I was aware of the behavior of CLIENT_ACK,
> although when I started this research I was not.  Before this I was
> under the impression that it was a single message ack.
>
> Michael Tarullo
> Contractor (Engility Corp)
> Software Engineer
> FAA WJH Technical Center
> (609)485-5294
>
>
> -----Original Message-----
> From: Matt Pavlovich [mailto:[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=4716062&i=0>]
> Sent: Thursday, September 01, 2016 10:58 AM
> To: [hidden email]
> <http:///user/SendEmail.jtp?type=node&node=4716062&i=1>
> Subject: Re: Need Help With A JMS Exception
>
> +1 Agree that debugging it is the only surefire way to understand what
> is going on.
>
> On a side note-- CLIENT_ACKNOWLEDGE is kludgy of sorts. Be sure to
> understand that it acknowledges the current message and _all_ previous
> messages. It isn't an individual message acknowledgement as most
> people are hoping. To get that, you need to use ActiveMQ's extension
> ActiveMQSession.INDIVIDUAL_ACKNOWLEDGE or go full blown TRANSACTED.
>
> All JMS providers have this "extended" ack mode and there is a JMS
> feat request to make INDIVIDUAL_ACKNOWLEDGE a standardized thing:
>
> Vote here: https://java.net/jira/browse/JMS_SPEC-95
>
> On 9/1/16 7:47 AM, Tim Bain wrote:
>
> > Since you have the 5.5.1 source code, I recommend you attach a
> > debugger
> to
> > the broker process and set a breakpoint on the
> > TopicSubscription.acknowledge() method; the stack will tell you how
> > you
> got
> > there.  Setting breakpoints on the broker can lead to instability
> > and
> may
> > force you to restart it (plus you have to restart it to enable the
> > debug port and again to disable it, and it's a security
> > vulnerability while
> it's
> > open), so do this in a dev/test environment, not a production one.
> >
> > You should not expect that there would be a call chain between the
> > two methods you referenced.  One is executed on the client and one
> > is
> executed
> > on the broker, so between the two there is the creation,
> > transmission,
> and
> > handling of the acknowledgement message.
> >
> > Tim
> >
> > On Aug 31, 2016 4:15 PM, <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=4716062&i=2>> wrote:
> >
> >> We had a rather unique situation some time ago where our message
> >> broker log was recording the JMSException for an Invalid acknowledgement.
> >>
> >> We traced it to the fact that the consumer of the message was using
> >> a
> more
> >> recent version of ActiveMQ than our message broker was running (i.e.
> >> message broker was v5.5.1 and the consumer was >v5.5.1).
> >>
> >> We would like to recreate this error in one of our test
> >> environments so that we can learn more about it.
> >>
> >> It appears that the exception was related to a consumer
> >> acknowledging a message.  We determined this from observing that
> >> the specific exception that was thrown appears to originate in only
> >> one place, that is the
> >> acknowledge() function of the TopicSubscription class.
> >>
> >> I have configured an environment with a v5.5.1 message broker and a
> >> consumer and producer compiled with v5.11.0 of ActiveMQ.  The
> >> producer sends messages to a topic and the consumer receives these
> >> messages and
> then
> >> acknowledges those messages.  The session was configured for client
> >> acknowledgement.  Under these conditions I could not reproduce the
> >> exception described above.
> >>
> >> I tried looking through the source code for the acknowledge()
> >> function
> of
> >> the ActiveMQMessage class to see if I could determine how my
> >> consumer
> call
> >> to acknowkedge() landed on the TopicSubscription acknowledge(), but
> have
> >> been unable to make a connection between the acknowledge() function
> >> in ActiveMQMessage and TopicSubscription even after following the
> >> calls starting with the call to ActiveMQMessage acknowledge().
> >>
> >> Can you offer any explanation as to why I am unable to trace the
> functions
> >> calls to TopicSubscription acknowledge?  And more importantly, can
> >> you offer any advice on how I might recreate the exception
> >> condition
> described
> >> here?
> >>
> >> Thanks,
> >> Mike
> >>
> >> Michael Tarullo
> >> Contractor (Engility Corp)
> >> Software Engineer
> >> FAA WJH Technical Center
> >> (609)485-5294
> >>
> >>
>
>
>
> ------------------------------
> If you reply to this email, your message will be added to the
> discussion
> below:
> http://activemq.2283324.n4.nabble.com/Need-Help-With-A-JMS-Exception-
> tp4716045p4716062.html
> To start a new topic under ActiveMQ - User, email
> [hidden email]
> To unsubscribe from ActiveMQ - User, click here
> <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=
> unsubscribe_by_code&node=2341805&code=YXJ0QGFtbGludi5jb218MjM0MTgwNXwy
> MDc3NjQwODU5>
> .
> NAML
> <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=
> macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.na
> mespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabbl
> e.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nab
> ble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_ema
> il%21nabble%3Aemail.naml>
>




--
View this message in context: http://activemq.2283324.n4.nabble.com/Need-Help-With-A-JMS-Exception-tp4716045p4716067.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

Re: Need Help With A JMS Exception

artnaseef
The broker and client negotiate the version of the OpenWire protocol when the client connects.  If the client sends a version of an ack that the broker doesn't support, something is really wrong.

Are all the brokers running the same version of ActiveMQ?


On Thu, Sep 1, 2016 at 12:52 PM, Michael.CTR.Tarullo [via ActiveMQ] <[hidden email]> wrote:
I am not certain about this but I think the version mismatch was discovered because the ackType in the exception was = 6, and someone (not me  because at that time I was not involved with the project) noticed that v5.5.1 did not support ackType = 6, i.e. the max ackType for v5.5.1 is 5.

I don't know if the client was using failover transport.  I'm not sure if this was ever investigated but I can try to find out.

Thanks for your thoughts.

Michael Tarullo
Contractor (Engility Corp)
Software Engineer
FAA WJH Technical Center
<a href="tel:%28609%29485-5294" value="+16094855294" target="_blank">(609)485-5294


-----Original Message-----
From: artnaseef [mailto:[hidden email]]
Sent: Thursday, September 01, 2016 12:05 PM
To: [hidden email]
Subject: Re: Need Help With A JMS Exception

A couple of thoughts on this front.

First, how was the conclusion of different client versions made?  I'm curious because I can't think of any way that should be an issue.

Second, is the consumer using the failover transport?

This JMS Exception can happen normally without indicating a problem, especially with the failover transport, as the consumer may then send an acknowledgement to the broker after it has already received another acknowledgement for the same message.  Or, with Topics, the rebuilt subscription may not have the message - I'm not sure how the failover transport handles non-durable Topic subscriptions.  My suspicion is that the new subscription, after failover, will start fresh, with none of the old messages.  So, if the consumer then proceeds to send an acknowledgement for the message, the broker won't be able to match the message.



On Thu, Sep 1, 2016 at 11:13 AM, Michael.CTR.Tarullo [via ActiveMQ] <
[hidden email]> wrote:

> Thanks Matt.  And yes I was aware of the behavior of CLIENT_ACK,
> although when I started this research I was not.  Before this I was
> under the impression that it was a single message ack.
>
> Michael Tarullo
> Contractor (Engility Corp)
> Software Engineer
> FAA WJH Technical Center
> <a href="tel:%28609%29485-5294" value="+16094855294" target="_blank">(609)485-5294
>
>
> -----Original Message-----
> From: Matt Pavlovich [mailto:[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=4716062&i=0>]
> Sent: Thursday, September 01, 2016 10:58 AM
> To: [hidden email]
> <http:///user/SendEmail.jtp?type=node&node=4716062&i=1>

> Subject: Re: Need Help With A JMS Exception
>
> +1 Agree that debugging it is the only surefire way to understand what
> is going on.
>
> On a side note-- CLIENT_ACKNOWLEDGE is kludgy of sorts. Be sure to
> understand that it acknowledges the current message and _all_ previous
> messages. It isn't an individual message acknowledgement as most
> people are hoping. To get that, you need to use ActiveMQ's extension
> ActiveMQSession.INDIVIDUAL_ACKNOWLEDGE or go full blown TRANSACTED.
>
> All JMS providers have this "extended" ack mode and there is a JMS
> feat request to make INDIVIDUAL_ACKNOWLEDGE a standardized thing:
>
> Vote here: https://java.net/jira/browse/JMS_SPEC-95
>
> On 9/1/16 7:47 AM, Tim Bain wrote:
>
> > Since you have the 5.5.1 source code, I recommend you attach a
> > debugger
> to
> > the broker process and set a breakpoint on the
> > TopicSubscription.acknowledge() method; the stack will tell you how
> > you
> got
> > there.  Setting breakpoints on the broker can lead to instability
> > and
> may
> > force you to restart it (plus you have to restart it to enable the
> > debug port and again to disable it, and it's a security
> > vulnerability while
> it's
> > open), so do this in a dev/test environment, not a production one.
> >
> > You should not expect that there would be a call chain between the
> > two methods you referenced.  One is executed on the client and one
> > is
> executed
> > on the broker, so between the two there is the creation,
> > transmission,
> and
> > handling of the acknowledgement message.
> >
> > Tim
> >
> > On Aug 31, 2016 4:15 PM, <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=4716062&i=2>> wrote:

> >
> >> We had a rather unique situation some time ago where our message
> >> broker log was recording the JMSException for an Invalid acknowledgement.
> >>
> >> We traced it to the fact that the consumer of the message was using
> >> a
> more
> >> recent version of ActiveMQ than our message broker was running (i.e.
> >> message broker was v5.5.1 and the consumer was >v5.5.1).
> >>
> >> We would like to recreate this error in one of our test
> >> environments so that we can learn more about it.
> >>
> >> It appears that the exception was related to a consumer
> >> acknowledging a message.  We determined this from observing that
> >> the specific exception that was thrown appears to originate in only
> >> one place, that is the
> >> acknowledge() function of the TopicSubscription class.
> >>
> >> I have configured an environment with a v5.5.1 message broker and a
> >> consumer and producer compiled with v5.11.0 of ActiveMQ.  The
> >> producer sends messages to a topic and the consumer receives these
> >> messages and
> then
> >> acknowledges those messages.  The session was configured for client
> >> acknowledgement.  Under these conditions I could not reproduce the
> >> exception described above.
> >>
> >> I tried looking through the source code for the acknowledge()
> >> function
> of
> >> the ActiveMQMessage class to see if I could determine how my
> >> consumer
> call
> >> to acknowkedge() landed on the TopicSubscription acknowledge(), but
> have
> >> been unable to make a connection between the acknowledge() function
> >> in ActiveMQMessage and TopicSubscription even after following the
> >> calls starting with the call to ActiveMQMessage acknowledge().
> >>
> >> Can you offer any explanation as to why I am unable to trace the
> functions
> >> calls to TopicSubscription acknowledge?  And more importantly, can
> >> you offer any advice on how I might recreate the exception
> >> condition
> described
> >> here?
> >>
> >> Thanks,
> >> Mike
> >>
> >> Michael Tarullo
> >> Contractor (Engility Corp)
> >> Software Engineer
> >> FAA WJH Technical Center
> >> <a href="tel:%28609%29485-5294" value="+16094855294" target="_blank">(609)485-5294
> >>
> >>
>
>
>
> ------------------------------
> If you reply to this email, your message will be added to the
> discussion
> below:
> http://activemq.2283324.n4.nabble.com/Need-Help-With-A-JMS-Exception-
> tp4716045p4716062.html
> To start a new topic under ActiveMQ - User, email
> [hidden email]
> To unsubscribe from ActiveMQ - User, click here
> <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=

> unsubscribe_by_code&node=2341805&code=YXJ0QGFtbGludi5jb218MjM0MTgwNXwy
> MDc3NjQwODU5>
> .
> NAML
> <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=
> macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.na
> mespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabbl
> e.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nab
> ble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_ema
> il%21nabble%3Aemail.naml>
>



--
View this message in context: http://activemq.2283324.n4.nabble.com/Need-Help-With-A-JMS-Exception-tp4716045p4716067.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.



If you reply to this email, your message will be added to the discussion below:
http://activemq.2283324.n4.nabble.com/Need-Help-With-A-JMS-Exception-tp4716045p4716079.html
To start a new topic under ActiveMQ - User, email [hidden email]
To unsubscribe from ActiveMQ - User, click here.
NAML

Reply | Threaded
Open this post in threaded view
|

RE: Need Help With A JMS Exception

Michael.CTR.Tarullo
I'm not sure I understand your question.  We have the simplest (and I thought the only) case here, one broker and two clients.  One of the clients is a message producer the other a message consumer.

To reiterate from my original post, the message broker is ActiveMQ v5.5.1.  The clients were a more current release, I am not sure of the exact release when the exception originally occurred over a year ago.  However, in trying to reproduce this exception the test environment I am using is a message broker with ActiveMQ v5.5.1 and clients (producer and consumer) built with ActiveMQ v5.11.0.

Hope you find this helpful, and thanks for trying to assist with this.

The more information I have the better I will understand the problem.

Michael Tarullo
Contractor (Engility Corp)
Software Engineer
FAA WJH Technical Center
(609)485-5294

-----Original Message-----
From: artnaseef [mailto:[hidden email]]
Sent: Friday, September 02, 2016 12:36 AM
To: [hidden email]
Subject: Re: Need Help With A JMS Exception

The broker and client negotiate the version of the OpenWire protocol when the client connects.  If the client sends a version of an ack that the broker doesn't support, something is really wrong.

Are all the brokers running the same version of ActiveMQ?


On Thu, Sep 1, 2016 at 12:52 PM, Michael.CTR.Tarullo [via ActiveMQ] <
[hidden email]> wrote:

> I am not certain about this but I think the version mismatch was
> discovered because the ackType in the exception was = 6, and someone
> (not me  because at that time I was not involved with the project)
> noticed that
> v5.5.1 did not support ackType = 6, i.e. the max ackType for v5.5.1 is 5.
>
> I don't know if the client was using failover transport.  I'm not sure
> if this was ever investigated but I can try to find out.
>
> Thanks for your thoughts.
>
> Michael Tarullo
> Contractor (Engility Corp)
> Software Engineer
> FAA WJH Technical Center
> (609)485-5294
>
>
> -----Original Message-----
> From: artnaseef [mailto:[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=4716079&i=0>]
> Sent: Thursday, September 01, 2016 12:05 PM
> To: [hidden email]
> <http:///user/SendEmail.jtp?type=node&node=4716079&i=1>
> Subject: Re: Need Help With A JMS Exception
>
> A couple of thoughts on this front.
>
> First, how was the conclusion of different client versions made?  I'm
> curious because I can't think of any way that should be an issue.
>
> Second, is the consumer using the failover transport?
>
> This JMS Exception can happen normally without indicating a problem,
> especially with the failover transport, as the consumer may then send
> an acknowledgement to the broker after it has already received another
> acknowledgement for the same message.  Or, with Topics, the rebuilt
> subscription may not have the message - I'm not sure how the failover
> transport handles non-durable Topic subscriptions.  My suspicion is
> that the new subscription, after failover, will start fresh, with none
> of the old messages.  So, if the consumer then proceeds to send an
> acknowledgement for the message, the broker won't be able to match the message.
>
>
>
> On Thu, Sep 1, 2016 at 11:13 AM, Michael.CTR.Tarullo [via ActiveMQ] <
> [hidden email]
> <http:///user/SendEmail.jtp?type=node&node=4716079&i=2>>
> wrote:
>
> > Thanks Matt.  And yes I was aware of the behavior of CLIENT_ACK,
> > although when I started this research I was not.  Before this I was
> > under the impression that it was a single message ack.
> >
> > Michael Tarullo
> > Contractor (Engility Corp)
> > Software Engineer
> > FAA WJH Technical Center
> > (609)485-5294
> >
> >
> > -----Original Message-----
> > From: Matt Pavlovich [mailto:[hidden email]
> > <http:///user/SendEmail.jtp?type=node&node=4716062&i=0>]
> > Sent: Thursday, September 01, 2016 10:58 AM
> > To: [hidden email]
> > <http:///user/SendEmail.jtp?type=node&node=4716062&i=1>
> > Subject: Re: Need Help With A JMS Exception
> >
> > +1 Agree that debugging it is the only surefire way to understand
> > +what
> > is going on.
> >
> > On a side note-- CLIENT_ACKNOWLEDGE is kludgy of sorts. Be sure to
> > understand that it acknowledges the current message and _all_
> > previous messages. It isn't an individual message acknowledgement as
> > most people are hoping. To get that, you need to use ActiveMQ's
> > extension ActiveMQSession.INDIVIDUAL_ACKNOWLEDGE or go full blown TRANSACTED.
> >
> > All JMS providers have this "extended" ack mode and there is a JMS
> > feat request to make INDIVIDUAL_ACKNOWLEDGE a standardized thing:
> >
> > Vote here: https://java.net/jira/browse/JMS_SPEC-95
> >
> > On 9/1/16 7:47 AM, Tim Bain wrote:
> >
> > > Since you have the 5.5.1 source code, I recommend you attach a
> > > debugger
> > to
> > > the broker process and set a breakpoint on the
> > > TopicSubscription.acknowledge() method; the stack will tell you
> > > how you
> > got
> > > there.  Setting breakpoints on the broker can lead to instability
> > > and
> > may
> > > force you to restart it (plus you have to restart it to enable the
> > > debug port and again to disable it, and it's a security
> > > vulnerability while
> > it's
> > > open), so do this in a dev/test environment, not a production one.
> > >
> > > You should not expect that there would be a call chain between the
> > > two methods you referenced.  One is executed on the client and one
> > > is
> > executed
> > > on the broker, so between the two there is the creation,
> > > transmission,
> > and
> > > handling of the acknowledgement message.
> > >
> > > Tim
> > >
> > > On Aug 31, 2016 4:15 PM, <[hidden email]
> > <http:///user/SendEmail.jtp?type=node&node=4716062&i=2>> wrote:
> > >
> > >> We had a rather unique situation some time ago where our message
> > >> broker log was recording the JMSException for an Invalid
> acknowledgement.
> > >>
> > >> We traced it to the fact that the consumer of the message was
> > >> using a
> > more
> > >> recent version of ActiveMQ than our message broker was running (i.e.
> > >> message broker was v5.5.1 and the consumer was >v5.5.1).
> > >>
> > >> We would like to recreate this error in one of our test
> > >> environments so that we can learn more about it.
> > >>
> > >> It appears that the exception was related to a consumer
> > >> acknowledging a message.  We determined this from observing that
> > >> the specific exception that was thrown appears to originate in
> > >> only one place, that is the
> > >> acknowledge() function of the TopicSubscription class.
> > >>
> > >> I have configured an environment with a v5.5.1 message broker and
> > >> a consumer and producer compiled with v5.11.0 of ActiveMQ.  The
> > >> producer sends messages to a topic and the consumer receives
> > >> these messages and
> > then
> > >> acknowledges those messages.  The session was configured for
> > >> client acknowledgement.  Under these conditions I could not
> > >> reproduce the exception described above.
> > >>
> > >> I tried looking through the source code for the acknowledge()
> > >> function
> > of
> > >> the ActiveMQMessage class to see if I could determine how my
> > >> consumer
> > call
> > >> to acknowkedge() landed on the TopicSubscription acknowledge(),
> > >> but
> > have
> > >> been unable to make a connection between the acknowledge()
> > >> function in ActiveMQMessage and TopicSubscription even after
> > >> following the calls starting with the call to ActiveMQMessage acknowledge().
> > >>
> > >> Can you offer any explanation as to why I am unable to trace the
> > functions
> > >> calls to TopicSubscription acknowledge?  And more importantly,
> > >> can you offer any advice on how I might recreate the exception
> > >> condition
> > described
> > >> here?
> > >>
> > >> Thanks,
> > >> Mike
> > >>
> > >> Michael Tarullo
> > >> Contractor (Engility Corp)
> > >> Software Engineer
> > >> FAA WJH Technical Center
> > >> (609)485-5294
> > >>
> > >>
> >
> >
> >
> > ------------------------------
> > If you reply to this email, your message will be added to the
> > discussion
> > below:
> > http://activemq.2283324.n4.nabble.com/Need-Help-With-A-JMS-Exception
> > -
> > tp4716045p4716062.html
> > To start a new topic under ActiveMQ - User, email [hidden email]
> > <http:///user/SendEmail.jtp?type=node&node=4716079&i=3>
> > To unsubscribe from ActiveMQ - User, click here
> > <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macr
> > o=
> > unsubscribe_by_code&node=2341805&code=YXJ0QGFtbGludi5jb218MjM0MTgwNX
> > wy
> > MDc3NjQwODU5>
> > .
> > NAML
> > <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macr
> > o=
> > macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.
> > na
> > mespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nab
> > bl
> > e.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21n
> > ab
> > ble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_e
> > ma
> > il%21nabble%3Aemail.naml>
> >
>
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.
> nabble.com/Need-Help-With-A-JMS-Exception-tp4716045p4716067.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>
> ------------------------------
> If you reply to this email, your message will be added to the
> discussion
> below:
> http://activemq.2283324.n4.nabble.com/Need-Help-With-A-JMS-Exception-
> tp4716045p4716079.html
> To start a new topic under ActiveMQ - User, email
> [hidden email]
> To unsubscribe from ActiveMQ - User, click here
> <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=
> unsubscribe_by_code&node=2341805&code=YXJ0QGFtbGludi5jb218MjM0MTgwNXwy
> MDc3NjQwODU5>
> .
> NAML
> <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=
> macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.na
> mespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabbl
> e.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nab
> ble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_ema
> il%21nabble%3Aemail.naml>
>




--
View this message in context: http://activemq.2283324.n4.nabble.com/Need-Help-With-A-JMS-Exception-tp4716045p4716096.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.