How to ensure reliability of publish/subscribe against occassional network hiccup?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

How to ensure reliability of publish/subscribe against occassional network hiccup?

Naresh Bhatia
I would like to use ActiveMQ in an application where subcribers must receive EVERY message published to their topic. If a subscriber looses a message, its state would be out-of-sync with the publisher - and that would be a bad thing! Note however, that there is no need to receive messages when the subscriber is down or inactive, in other words durability of messages is not required.

1) What is the best way to use ActiveMQ in this scenario to protect against occassional network hiccups? Is it enough to have a transacted session or a non-transacted session with AUTO_ACKNOWLEDGE or CLIENT_ACKNOWLEDGE? Does this guarantee the receipt of all messages?

2) How does publish-subscribe actually work? Does the publisher send a seperate message to each subscriber and receive an explicit acknowledgement from each or is the message sent out as a broadcast (multicast?) and then explicit acknowledgements coming in from each subscriber?

Thanks.
Naresh
Reply | Threaded
Open this post in threaded view
|

Re: How to ensure reliability of publish/subscribe against occassional network hiccup?

Tom Samplonius-2

----- "Naresh Bhatia" <[hidden email]> wrote:

> I would like to use ActiveMQ in an application where subcribers must
> receive
> EVERY message published to their topic. If a subscriber looses a
> message,
> its state would be out-of-sync with the publisher - and that would be
> a bad
> thing! Note however, that there is no need to receive messages when
> the
> subscriber is down or inactive, in other words durability of messages
> is not
> required.

  I don't understand the distinction between, "subscriber can not lose a message" vs. "subscriber can drop messages when it is disconnected".  Subscribers are only going to lose messages unless they disconnect, or are disconnected.  So I think you want durable messages.


> 1) What is the best way to use ActiveMQ in this scenario to protect
> against
> occassional network hiccups? Is it enough to have a transacted session
> or a
> non-transacted session with AUTO_ACKNOWLEDGE or CLIENT_ACKNOWLEDGE?
> Does
> this guarantee the receipt of all messages?

  I suppose so, and persistent subscriptions probably.

> 2) How does publish-subscribe actually work? Does the publisher send
> a
> seperate message to each subscriber and receive an explicit
> acknowledgement
> from each or is the message sent out as a broadcast (multicast?) and
> then
> explicit acknowledgements coming in from each subscriber?

  Well, the publisher sends one message to the broker (ActiveMQ), and the broker distributes to the subscribers.  Subscribers ack to the broker.  Normally, the publisher has no idea who received the message, and in most applications, does not care.  If subscribers need reliable messages, they have to ask for it.


> Thanks.
> Naresh
> --
> View this message in context:
> http://www.nabble.com/How-to-ensure-reliability-of-publish-subscribe-against-occassional-network-hiccup--tf4068985s2354.html#a11562991
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.


Tom
Reply | Threaded
Open this post in threaded view
|

Re: How to ensure reliability of publish/subscribe against occassional network hiccup?

Naresh Bhatia
Tom Samplonius-2 wrote
  I don't understand the distinction between, "subscriber can not lose a message" vs. "subscriber can drop messages when it is disconnected".  Subscribers are only going to lose messages unless they disconnect, or are disconnected.  So I think you want durable messages.
Let me try to clarify my use case this way: I have a server that maintains the master copy of all business (domain) objects. I have multiple thick clients (Swing based) that replicate these objects and show them in various views. I am using JMS publish/subscribe to keep the client objects in sync with the server objects. This synchronization is only necessary when the Swing client is running - when the user closes the app, all client state is gone and no synchronization is necessary. That's what I meant by "there is no need to receive messages when the subscriber is down or inactive". Given this context and the assumption that I will use a transacted session or acked messages at the client end, is durability necessary? In other words, if an active JMS subscriber does not successfully ack a message (because of a network hiccup), will ActiveMQ retry sending the message?

Thanks.
Naresh
Reply | Threaded
Open this post in threaded view
|

Re: How to ensure reliability of publish/subscribe against occassional network hiccup?

Naresh Bhatia
I tried a little experiment to understand ActiveMQ's behavior when the network is disrupted. It appears that there is some sort of connection monitoring going on between the client and the ActiveMQ broker (either at the broker level or at the socket level). ActiveMQ keeps retrying the message until it gets an acknowledgement from the subscriber. However during these retries, if the connection times out then the client's ExceptionListener simply receives a JMSException (Connection reset). For shorter network disruptions, even non-durable messages seem to go through pretty reliably. Does this conclusion make sense?

Thanks.
Naresh
Reply | Threaded
Open this post in threaded view
|

Re: How to ensure reliability of publish/subscribe against occassional network hiccup?

rajdavies
Hi Naresh,

your conclusion does make sense - ActiveMQ is designed to handle  
interruptions

cheers,

Rob
On Jul 16, 2007, at 5:38 PM, Naresh Bhatia wrote:

>
> I tried a little experiment to understand ActiveMQ's behavior when the
> network is disrupted. It appears that there is some sort of connection
> monitoring going on between the client and the ActiveMQ broker  
> (either at
> the broker level or at the socket level). ActiveMQ keeps retrying the
> message until it gets an acknowledgement from the subscriber.  
> However during
> these retries, if the connection times out then the client's
> ExceptionListener simply receives a JMSException (Connection  
> reset). For
> shorter network disruptions, even non-durable messages seem to go  
> through
> pretty reliably. Does this conclusion make sense?
>
> Thanks.
> Naresh
> --
> View this message in context: http://www.nabble.com/How-to-ensure- 
> reliability-of-publish-subscribe-against-occassional-network-
> hiccup--tf4068985s2354.html#a11621573
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>

Reply | Threaded
Open this post in threaded view
|

Re: How to ensure reliability of publish/subscribe against occassional network hiccup?

pragmaticjdev
In reply to this post by Naresh Bhatia
I would like to revive this email thread since I have a exact similar use
case for which I am planning to use activemq - to keep the java objects in
sync between multiple nodes. This thread discusses a couple of approaches -
transacted session (does this refer to transactions?) or acknowledgements
(AUTO_ACKNOWLEDGE or CLIENT_ACKNOWLEDGE). There is no conclusion on this
discussion though, hence I wanted to bring it back up to get expert
opinions. Kindly suggest. Thanks!



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: How to ensure reliability of publish/subscribe against occassional network hiccup?

Tim Bain
On Mon, Apr 16, 2018 at 6:08 AM, pragmaticjdev <[hidden email]> wrote:

> I would like to revive this email thread since I have a exact similar use
> case for which I am planning to use activemq - to keep the java objects in
> sync between multiple nodes. This thread discusses a couple of approaches -
> transacted session (does this refer to transactions?)



Yes, this refers to transactions, either JMS transactions or XA
transactions. A transaction allows you to roll back the work that you've
done if some portion of the transaction fails. JMS transactions cover only
interactions with the ActiveMQ broker (acking and publishing messages),
whereas XA transactions can cover interactions with RDBMSes as well.
http://activemq.apache.org/how-do-transactions-work.html has some
additional details.



> or acknowledgements
> (AUTO_ACKNOWLEDGE or CLIENT_ACKNOWLEDGE).



Using some form of explicit acknowledgement (CLIENT_ACKNOWLEDGE,
INDIVIDUAL_ACKNOWLEDGE) instead of AUTO_ACKNOWLEDGE allows you to avoid
acknowledging the message until you've actually consumed it, but doesn't
allow you to roll back other operations if something fails.



> There is no conclusion on this
> discussion though, hence I wanted to bring it back up to get expert
> opinions. Kindly suggest. Thanks!
>


I suspect that there is no conclusion in part because different options are
best for different situations.
http://activemq.apache.org/should-i-use-transactions.html recommends using
transactions over explicit acknowledgement, but both can work depending on
your needs, so it really comes down to what your needs are. If you need to
roll back database transactions, use XA transactions. If you need the
ability to roll back one or more JMS operation if another JMS operation
fails but you don't need the ability to roll back database transactions,
use JMS transactions to avoid the overhead of XA transactions. If you just
need to ensure that messages are delivered at least once but don't need the
ability to roll back any processing if an error occurs, use explicit
acknowledgement.

Tim