How to SYNC messages in Cluster without using persistence on the message broker

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

How to SYNC messages in Cluster without using persistence on the message broker

senthil2community@gmail.com
Hi There,

I'm wondering if there is any way to sync/replicate the message without
persisting on the message broker.

I have THREE node ODL cluster and ActiveMQ runs on all the nodes (without
knowing each other). There are few topics created and message producer on
one of the nodes and would like to check if there is any way to SYNC the
topics and the messages on other two nodes without using persistence.

Thanks,
Senthil Kumar Jayakumar
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How to SYNC messages in Cluster without using persistence on the message broker

Tim Bain
This is ActiveMQ 5.x, not Artemis, right?

What do you mean by "SYNC"? If you mean "replicate the message so a copy
exists on all three brokers and can be consumed from any of them, and
delete it from all three when it's consumed from one", ActiveMQ 5.x doesn't
do that.

Artemis, however, does support clustered operation, so you might want to
consider switching to it.

If you mean something else, please describe the behavior you're hoping to
achieve.

Tim

On Jul 17, 2017 6:15 AM, "Senthil Jayakumar" <[hidden email]>
wrote:

> Hi There,
>
> I'm wondering if there is any way to sync/replicate the message without
> persisting on the message broker.
>
> I have THREE node ODL cluster and ActiveMQ runs on all the nodes (without
> knowing each other). There are few topics created and message producer on
> one of the nodes and would like to check if there is any way to SYNC the
> topics and the messages on other two nodes without using persistence.
>
> Thanks,
> Senthil Kumar Jayakumar
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How to SYNC messages in Cluster without using persistence on the message broker

senthil2community@gmail.com
Hi Tim,
We are using ActiveMQ 5.9.0 release.

Yes, I was exactly referring to what you have specified, but, WITHOUT persisting any of the messages on the message broker.

i.e. "replicate the message so a copy
exists on all three brokers and can be consumed from any of them, and
delete it from all three when it's consumed from one"

Please let me know if this can be achieved through Artemis.

Thank you
Senthil Kumar Jayakumar

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How to SYNC messages in Cluster without using persistence on the message broker

Justin Bertram
> Please let me know if this can be achieved through Artemis.

In Artemis the only data replicated between a live and a backup is
persistent data.  You can't have one without the other.

That said, I'm not sure what benefit you'd receive by replicating
non-persistent data.  Firstly, non-persistent data is, by definition,
volatile (i.e. will not survive a crash).  Second, the non-blocking design
of Artemis means that the data is sent to the disk and to the network
concurrently.  I highly doubt the network will be faster than local disk
which means you'll be getting a copy persisted to disk essentially for
"free" from a performance perspective.  What's your use-case that you'd
want non-persistent data to be replicated?


Justin

On Fri, Jul 21, 2017 at 8:00 AM, [hidden email] <
[hidden email]> wrote:

> Hi Tim,
> We are using ActiveMQ 5.9.0 release.
>
> Yes, I was exactly referring to what you have specified, but, WITHOUT
> persisting any of the messages on the message broker.
>
> i.e. "replicate the message so a copy
> exists on all three brokers and can be consumed from any of them, and
> delete it from all three when it's consumed from one"
>
> Please let me know if this can be achieved through Artemis.
>
> Thank you
> Senthil Kumar Jayakumar
>
>
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.
> nabble.com/How-to-SYNC-messages-in-Cluster-without-
> using-persistence-on-the-message-broker-tp4728592p4728765.html
> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How to SYNC messages in Cluster without using persistence on the message broker

senthil2community@gmail.com
Hi Justin,

Thanks for the detailed reply.

The reason for not persisting is because the same data is persisted or available in the different data store (ODL MDSAL).

The use case was "My application (Controller) receives events asynchronously from the south bound devices which are persisted in the local data store (ODL MDSAL) and at the same time put in the topic for the north bound applications to consume asynchronously."

Since, it already persisted locally do not want to persist again in the message broker.

Regards
Senthil Jayakumar
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How to SYNC messages in Cluster without using persistence on the message broker

Justin Bertram
If the data for the messages you're sending is already persisted somewhere
then why not simply send the messages non-persistently?


Justin

On Fri, Jul 21, 2017 at 9:53 AM, [hidden email] <
[hidden email]> wrote:

> Hi Justin,
>
> Thanks for the detailed reply.
>
> The reason for not persisting is because the same data is persisted or
> available in the different data store (ODL MDSAL).
>
> The use case was "My application (Controller) receives events
> asynchronously
> from the south bound devices which are persisted in the local data store
> (ODL MDSAL) and at the same time put in the topic for the north bound
> applications to consume asynchronously."
>
> Since, it already persisted locally do not want to persist again in the
> message broker.
>
> Regards
> Senthil Jayakumar
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.
> nabble.com/How-to-SYNC-messages-in-Cluster-without-
> using-persistence-on-the-message-broker-tp4728592p4728771.html
> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How to SYNC messages in Cluster without using persistence on the message broker

senthil2community@gmail.com
Yes, that is why I was trying to check if there is any way to replicate messages WITHOUT persistence and looks like it is not possible.

Thanks for the quick reply and support.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How to SYNC messages in Cluster without using persistence on the message broker

Justin Bertram
My point is that replication is meant for the high availability of message
data, but if the message data is already persisted to disk in a reliable
environment then there's no need to use replication at the broker level.
If there's a broker failure then the message data can be recovered from the
other environment where it was persisted.


Justin

On Thu, Jul 27, 2017 at 6:49 AM, [hidden email] <
[hidden email]> wrote:

> Yes, that is why I was trying to check if there is any way to replicate
> messages WITHOUT persistence and looks like it is not possible.
>
> Thanks for the quick reply and support.
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.
> nabble.com/How-to-SYNC-messages-in-Cluster-without-
> using-persistence-on-the-message-broker-tp4728592p4728955.html
> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
>
Loading...