Broker Federation with an Active-Active Broker Cluster

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Broker Federation with an Active-Active Broker Cluster

andi welchlin
Hello,

I would like to setup a active-active cluster of brokers using ActiveMq.

But I also would like to connect single ActiveMq satellite brokers to this
cluster while they should share some queues and exchanges. So this will be
pretty much like a federation between some satellite brokers and a central
broker cluster. Clients will connect to these satellite brokers using AMQP.

Do you think this would be possible using ActiveMq?

Kind Regards,
Andreas
Reply | Threaded
Open this post in threaded view
|

Re: Broker Federation with an Active-Active Broker Cluster

Tim Bain
Not with ActiveMQ 5.x, since it doesn't have a capacity to do an
active-active cluster, but ActiveMQ Artemis can. Have you looked at it?

Tim

On Dec 5, 2017 7:28 AM, "andi welchlin" <[hidden email]> wrote:

> Hello,
>
> I would like to setup a active-active cluster of brokers using ActiveMq.
>
> But I also would like to connect single ActiveMq satellite brokers to this
> cluster while they should share some queues and exchanges. So this will be
> pretty much like a federation between some satellite brokers and a central
> broker cluster. Clients will connect to these satellite brokers using AMQP.
>
> Do you think this would be possible using ActiveMq?
>
> Kind Regards,
> Andreas
>
Reply | Threaded
Open this post in threaded view
|

Re: Broker Federation with an Active-Active Broker Cluster

andi welchlin
Hello Tim,

Thank you for the hint.

Yes, I looked at Artemis and have a little test installation on a
test-machine. I think it is sure that active - active clustering will work.

Anyway, I never used ActiveMQ or Artemis, before.


The second point is that I would like to connect several satellite brokers
to the cluster which share queues and exchanges. The point here is that I
want the clients to be able to work offline and as soon as the connection
is up the messages from persistent queues in the satellite broker are sent
into the cluster and vice versa. Also it would be better to have a single
connection at the satellites because of the firewalls.

I think that you can realize the satellites with Apache QPID brokers using
routes but they do not support active active clustering. So I was asking
myself if Artemis would be able to handle satellites.

If it is not possible, then I will think about using the QPID Dispatch
Router but this makes the setup more complicated, I think.

Kind Regards,
Andreas





On Tue, Dec 5, 2017 at 11:21 PM, Tim Bain <[hidden email]> wrote:

> Not with ActiveMQ 5.x, since it doesn't have a capacity to do an
> active-active cluster, but ActiveMQ Artemis can. Have you looked at it?
>
> Tim
>
> On Dec 5, 2017 7:28 AM, "andi welchlin" <[hidden email]> wrote:
>
> > Hello,
> >
> > I would like to setup a active-active cluster of brokers using ActiveMq.
> >
> > But I also would like to connect single ActiveMq satellite brokers to
> this
> > cluster while they should share some queues and exchanges. So this will
> be
> > pretty much like a federation between some satellite brokers and a
> central
> > broker cluster. Clients will connect to these satellite brokers using
> AMQP.
> >
> > Do you think this would be possible using ActiveMq?
> >
> > Kind Regards,
> > Andreas
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Broker Federation with an Active-Active Broker Cluster

Alec Henninger
In reply to this post by Tim Bain
Isn't network of brokers an active-active cluster?

On Tue, Dec 5, 2017, 5:21 PM Tim Bain <[hidden email]> wrote:

> Not with ActiveMQ 5.x, since it doesn't have a capacity to do an
> active-active cluster, but ActiveMQ Artemis can. Have you looked at it?
>
> Tim
>
> On Dec 5, 2017 7:28 AM, "andi welchlin" <[hidden email]> wrote:
>
> > Hello,
> >
> > I would like to setup a active-active cluster of brokers using ActiveMq.
> >
> > But I also would like to connect single ActiveMq satellite brokers to
> this
> > cluster while they should share some queues and exchanges. So this will
> be
> > pretty much like a federation between some satellite brokers and a
> central
> > broker cluster. Clients will connect to these satellite brokers using
> AMQP.
> >
> > Do you think this would be possible using ActiveMq?
> >
> > Kind Regards,
> > Andreas
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Broker Federation with an Active-Active Broker Cluster

andi welchlin
Hello Alec,

yes, I think your are right. As far as I understood the documentation this
is also active/active (if you use <networkConnector uri="static:...">)

Kind Regards,
Andreas

On Wed, Dec 6, 2017 at 1:35 PM, Alec Henninger <[hidden email]>
wrote:

> Isn't network of brokers an active-active cluster?
>
> On Tue, Dec 5, 2017, 5:21 PM Tim Bain <[hidden email]> wrote:
>
> > Not with ActiveMQ 5.x, since it doesn't have a capacity to do an
> > active-active cluster, but ActiveMQ Artemis can. Have you looked at it?
> >
> > Tim
> >
> > On Dec 5, 2017 7:28 AM, "andi welchlin" <[hidden email]> wrote:
> >
> > > Hello,
> > >
> > > I would like to setup a active-active cluster of brokers using
> ActiveMq.
> > >
> > > But I also would like to connect single ActiveMq satellite brokers to
> > this
> > > cluster while they should share some queues and exchanges. So this will
> > be
> > > pretty much like a federation between some satellite brokers and a
> > central
> > > broker cluster. Clients will connect to these satellite brokers using
> > AMQP.
> > >
> > > Do you think this would be possible using ActiveMq?
> > >
> > > Kind Regards,
> > > Andreas
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Broker Federation with an Active-Active Broker Cluster

Tim Bain
In reply to this post by Alec Henninger
My definition of a cluster is that a given message is available (without
forwarding) from all nodes in the cluster and will remain available when a
single node in the cluster is lost. Master-slave pairs are clusters (but
not active-active clusters) under that definition, while a network of
brokers is not.

So Andi, is that the definition you were using when you wrote this question?

Tim

On Dec 6, 2017 5:35 AM, "Alec Henninger" <[hidden email]> wrote:

> Isn't network of brokers an active-active cluster?
>
> On Tue, Dec 5, 2017, 5:21 PM Tim Bain <[hidden email]> wrote:
>
> > Not with ActiveMQ 5.x, since it doesn't have a capacity to do an
> > active-active cluster, but ActiveMQ Artemis can. Have you looked at it?
> >
> > Tim
> >
> > On Dec 5, 2017 7:28 AM, "andi welchlin" <[hidden email]> wrote:
> >
> > > Hello,
> > >
> > > I would like to setup a active-active cluster of brokers using
> ActiveMq.
> > >
> > > But I also would like to connect single ActiveMq satellite brokers to
> > this
> > > cluster while they should share some queues and exchanges. So this will
> > be
> > > pretty much like a federation between some satellite brokers and a
> > central
> > > broker cluster. Clients will connect to these satellite brokers using
> > AMQP.
> > >
> > > Do you think this would be possible using ActiveMq?
> > >
> > > Kind Regards,
> > > Andreas
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Broker Federation with an Active-Active Broker Cluster

andi welchlin
Hello Tim,

yes, that was exactly my definition.

Maybe I misunderstood the documentation of ActiveMQ "network of brokers".

Kind Regards,
Andreas

On Wed, Dec 6, 2017 at 2:33 PM, Tim Bain <[hidden email]> wrote:

> My definition of a cluster is that a given message is available (without
> forwarding) from all nodes in the cluster and will remain available when a
> single node in the cluster is lost. Master-slave pairs are clusters (but
> not active-active clusters) under that definition, while a network of
> brokers is not.
>
> So Andi, is that the definition you were using when you wrote this
> question?
>
> Tim
>
> On Dec 6, 2017 5:35 AM, "Alec Henninger" <[hidden email]> wrote:
>
> > Isn't network of brokers an active-active cluster?
> >
> > On Tue, Dec 5, 2017, 5:21 PM Tim Bain <[hidden email]> wrote:
> >
> > > Not with ActiveMQ 5.x, since it doesn't have a capacity to do an
> > > active-active cluster, but ActiveMQ Artemis can. Have you looked at it?
> > >
> > > Tim
> > >
> > > On Dec 5, 2017 7:28 AM, "andi welchlin" <[hidden email]>
> wrote:
> > >
> > > > Hello,
> > > >
> > > > I would like to setup a active-active cluster of brokers using
> > ActiveMq.
> > > >
> > > > But I also would like to connect single ActiveMq satellite brokers to
> > > this
> > > > cluster while they should share some queues and exchanges. So this
> will
> > > be
> > > > pretty much like a federation between some satellite brokers and a
> > > central
> > > > broker cluster. Clients will connect to these satellite brokers using
> > > AMQP.
> > > >
> > > > Do you think this would be possible using ActiveMq?
> > > >
> > > > Kind Regards,
> > > > Andreas
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Broker Federation with an Active-Active Broker Cluster

Tim Bain
In reply to this post by andi welchlin
Andreas,

I'm not familiar enough with either of those QPID products to comment on
the ways you're considering using them, but hopefully someone else on here
can.

Tim

On Dec 6, 2017 1:18 AM, "andi welchlin" <[hidden email]> wrote:

> Hello Tim,
>
> Thank you for the hint.
>
> Yes, I looked at Artemis and have a little test installation on a
> test-machine. I think it is sure that active - active clustering will work.
>
> Anyway, I never used ActiveMQ or Artemis, before.
>
>
> The second point is that I would like to connect several satellite brokers
> to the cluster which share queues and exchanges. The point here is that I
> want the clients to be able to work offline and as soon as the connection
> is up the messages from persistent queues in the satellite broker are sent
> into the cluster and vice versa. Also it would be better to have a single
> connection at the satellites because of the firewalls.
>
> I think that you can realize the satellites with Apache QPID brokers using
> routes but they do not support active active clustering. So I was asking
> myself if Artemis would be able to handle satellites.
>
> If it is not possible, then I will think about using the QPID Dispatch
> Router but this makes the setup more complicated, I think.
>
> Kind Regards,
> Andreas
>
>
>
>
>
> On Tue, Dec 5, 2017 at 11:21 PM, Tim Bain <[hidden email]> wrote:
>
> > Not with ActiveMQ 5.x, since it doesn't have a capacity to do an
> > active-active cluster, but ActiveMQ Artemis can. Have you looked at it?
> >
> > Tim
> >
> > On Dec 5, 2017 7:28 AM, "andi welchlin" <[hidden email]> wrote:
> >
> > > Hello,
> > >
> > > I would like to setup a active-active cluster of brokers using
> ActiveMq.
> > >
> > > But I also would like to connect single ActiveMq satellite brokers to
> > this
> > > cluster while they should share some queues and exchanges. So this will
> > be
> > > pretty much like a federation between some satellite brokers and a
> > central
> > > broker cluster. Clients will connect to these satellite brokers using
> > AMQP.
> > >
> > > Do you think this would be possible using ActiveMq?
> > >
> > > Kind Regards,
> > > Andreas
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Broker Federation with an Active-Active Broker Cluster

Tim Bain
In reply to this post by andi welchlin
Networks of brokers use store-and-forward to move messages between the
brokers, and each message is on only one broker at a time so it is lost (at
least temporarily) if that broker goes offline. It's not a cluster under
the definition we just laid out.

Tim

On Dec 6, 2017 6:36 AM, "andi welchlin" <[hidden email]> wrote:

> Hello Tim,
>
> yes, that was exactly my definition.
>
> Maybe I misunderstood the documentation of ActiveMQ "network of brokers".
>
> Kind Regards,
> Andreas
>
> On Wed, Dec 6, 2017 at 2:33 PM, Tim Bain <[hidden email]> wrote:
>
> > My definition of a cluster is that a given message is available (without
> > forwarding) from all nodes in the cluster and will remain available when
> a
> > single node in the cluster is lost. Master-slave pairs are clusters (but
> > not active-active clusters) under that definition, while a network of
> > brokers is not.
> >
> > So Andi, is that the definition you were using when you wrote this
> > question?
> >
> > Tim
> >
> > On Dec 6, 2017 5:35 AM, "Alec Henninger" <[hidden email]>
> wrote:
> >
> > > Isn't network of brokers an active-active cluster?
> > >
> > > On Tue, Dec 5, 2017, 5:21 PM Tim Bain <[hidden email]> wrote:
> > >
> > > > Not with ActiveMQ 5.x, since it doesn't have a capacity to do an
> > > > active-active cluster, but ActiveMQ Artemis can. Have you looked at
> it?
> > > >
> > > > Tim
> > > >
> > > > On Dec 5, 2017 7:28 AM, "andi welchlin" <[hidden email]>
> > wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > > I would like to setup a active-active cluster of brokers using
> > > ActiveMq.
> > > > >
> > > > > But I also would like to connect single ActiveMq satellite brokers
> to
> > > > this
> > > > > cluster while they should share some queues and exchanges. So this
> > will
> > > > be
> > > > > pretty much like a federation between some satellite brokers and a
> > > > central
> > > > > broker cluster. Clients will connect to these satellite brokers
> using
> > > > AMQP.
> > > > >
> > > > > Do you think this would be possible using ActiveMq?
> > > > >
> > > > > Kind Regards,
> > > > > Andreas
> > > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Broker Federation with an Active-Active Broker Cluster

andi welchlin
Hello Tim,

thank you. Now I got the difference.

As far as I understood the Artemis documentation it is possible to
configure a cluster and also connect single satellite brokers to this
cluster.

The satellite brokers can be connected using a bi-directional bridge so I
would use the core bridge and would use a network-connector where duplex is
set to true.

Is this a way I could go?

Kind Regards,
Andreas

On Wed, Dec 6, 2017 at 2:39 PM, Tim Bain <[hidden email]> wrote:

> Networks of brokers use store-and-forward to move messages between the
> brokers, and each message is on only one broker at a time so it is lost (at
> least temporarily) if that broker goes offline. It's not a cluster under
> the definition we just laid out.
>
> Tim
>
> On Dec 6, 2017 6:36 AM, "andi welchlin" <[hidden email]> wrote:
>
> > Hello Tim,
> >
> > yes, that was exactly my definition.
> >
> > Maybe I misunderstood the documentation of ActiveMQ "network of brokers".
> >
> > Kind Regards,
> > Andreas
> >
> > On Wed, Dec 6, 2017 at 2:33 PM, Tim Bain <[hidden email]> wrote:
> >
> > > My definition of a cluster is that a given message is available
> (without
> > > forwarding) from all nodes in the cluster and will remain available
> when
> > a
> > > single node in the cluster is lost. Master-slave pairs are clusters
> (but
> > > not active-active clusters) under that definition, while a network of
> > > brokers is not.
> > >
> > > So Andi, is that the definition you were using when you wrote this
> > > question?
> > >
> > > Tim
> > >
> > > On Dec 6, 2017 5:35 AM, "Alec Henninger" <[hidden email]>
> > wrote:
> > >
> > > > Isn't network of brokers an active-active cluster?
> > > >
> > > > On Tue, Dec 5, 2017, 5:21 PM Tim Bain <[hidden email]> wrote:
> > > >
> > > > > Not with ActiveMQ 5.x, since it doesn't have a capacity to do an
> > > > > active-active cluster, but ActiveMQ Artemis can. Have you looked at
> > it?
> > > > >
> > > > > Tim
> > > > >
> > > > > On Dec 5, 2017 7:28 AM, "andi welchlin" <[hidden email]>
> > > wrote:
> > > > >
> > > > > > Hello,
> > > > > >
> > > > > > I would like to setup a active-active cluster of brokers using
> > > > ActiveMq.
> > > > > >
> > > > > > But I also would like to connect single ActiveMq satellite
> brokers
> > to
> > > > > this
> > > > > > cluster while they should share some queues and exchanges. So
> this
> > > will
> > > > > be
> > > > > > pretty much like a federation between some satellite brokers and
> a
> > > > > central
> > > > > > broker cluster. Clients will connect to these satellite brokers
> > using
> > > > > AMQP.
> > > > > >
> > > > > > Do you think this would be possible using ActiveMq?
> > > > > >
> > > > > > Kind Regards,
> > > > > > Andreas
> > > > > >
> > > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Broker Federation with an Active-Active Broker Cluster

Tim Bain
Andreas,

I don't have enough experience with Artemis to be able to answer your
question, so I've been hoping that one of the Artemis folks on this list
will jump in to answer. Since that's not happening, you may want to start a
fresh message thread asking your question specifically in the context of
Artemis, in the hopes that people who might be ignoring this "5.x" thread
might see and respond to a new Artemis thread.

Best,
Tim

On Thu, Dec 7, 2017 at 9:37 AM, andi welchlin <[hidden email]>
wrote:

> Hello Tim,
>
> thank you. Now I got the difference.
>
> As far as I understood the Artemis documentation it is possible to
> configure a cluster and also connect single satellite brokers to this
> cluster.
>
> The satellite brokers can be connected using a bi-directional bridge so I
> would use the core bridge and would use a network-connector where duplex is
> set to true.
>
> Is this a way I could go?
>
> Kind Regards,
> Andreas
>
> On Wed, Dec 6, 2017 at 2:39 PM, Tim Bain <[hidden email]> wrote:
>
> > Networks of brokers use store-and-forward to move messages between the
> > brokers, and each message is on only one broker at a time so it is lost
> (at
> > least temporarily) if that broker goes offline. It's not a cluster under
> > the definition we just laid out.
> >
> > Tim
> >
> > On Dec 6, 2017 6:36 AM, "andi welchlin" <[hidden email]> wrote:
> >
> > > Hello Tim,
> > >
> > > yes, that was exactly my definition.
> > >
> > > Maybe I misunderstood the documentation of ActiveMQ "network of
> brokers".
> > >
> > > Kind Regards,
> > > Andreas
> > >
> > > On Wed, Dec 6, 2017 at 2:33 PM, Tim Bain <[hidden email]>
> wrote:
> > >
> > > > My definition of a cluster is that a given message is available
> > (without
> > > > forwarding) from all nodes in the cluster and will remain available
> > when
> > > a
> > > > single node in the cluster is lost. Master-slave pairs are clusters
> > (but
> > > > not active-active clusters) under that definition, while a network of
> > > > brokers is not.
> > > >
> > > > So Andi, is that the definition you were using when you wrote this
> > > > question?
> > > >
> > > > Tim
> > > >
> > > > On Dec 6, 2017 5:35 AM, "Alec Henninger" <[hidden email]>
> > > wrote:
> > > >
> > > > > Isn't network of brokers an active-active cluster?
> > > > >
> > > > > On Tue, Dec 5, 2017, 5:21 PM Tim Bain <[hidden email]>
> wrote:
> > > > >
> > > > > > Not with ActiveMQ 5.x, since it doesn't have a capacity to do an
> > > > > > active-active cluster, but ActiveMQ Artemis can. Have you looked
> at
> > > it?
> > > > > >
> > > > > > Tim
> > > > > >
> > > > > > On Dec 5, 2017 7:28 AM, "andi welchlin" <[hidden email]
> >
> > > > wrote:
> > > > > >
> > > > > > > Hello,
> > > > > > >
> > > > > > > I would like to setup a active-active cluster of brokers using
> > > > > ActiveMq.
> > > > > > >
> > > > > > > But I also would like to connect single ActiveMq satellite
> > brokers
> > > to
> > > > > > this
> > > > > > > cluster while they should share some queues and exchanges. So
> > this
> > > > will
> > > > > > be
> > > > > > > pretty much like a federation between some satellite brokers
> and
> > a
> > > > > > central
> > > > > > > broker cluster. Clients will connect to these satellite brokers
> > > using
> > > > > > AMQP.
> > > > > > >
> > > > > > > Do you think this would be possible using ActiveMq?
> > > > > > >
> > > > > > > Kind Regards,
> > > > > > > Andreas
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Broker Federation with an Active-Active Broker Cluster

Martyn Taylor
Hi Andreas,

If I understand correctly you're wanting to create a single cluster, with
active-active style availability, with some satellite brokers?

To set up an active/active style HA cluster in Artemis, you can use using
co-located pairs[1].  Essentially what this is doing is creating a live and
a backup broker in a single JVM.  Where the backup here is acting as the
backup for node on a separate machine.  You can then add satellite brokers
that get a view into the HA cluster.  You can control exactly which
addresses are shared between your HA cluster and individual brokers by
setting the address <address> on the cluster connection[2].

[1] https://activemq.apache.org/artemis/docs/latest/ha.html
[2] https://activemq.apache.org/artemis/docs/latest/clusters.html

Thanks
Martyn






On Tue, Dec 12, 2017 at 7:23 AM, Tim Bain <[hidden email]> wrote:

> Andreas,
>
> I don't have enough experience with Artemis to be able to answer your
> question, so I've been hoping that one of the Artemis folks on this list
> will jump in to answer. Since that's not happening, you may want to start a
> fresh message thread asking your question specifically in the context of
> Artemis, in the hopes that people who might be ignoring this "5.x" thread
> might see and respond to a new Artemis thread.
>
> Best,
> Tim
>
> On Thu, Dec 7, 2017 at 9:37 AM, andi welchlin <[hidden email]>
> wrote:
>
> > Hello Tim,
> >
> > thank you. Now I got the difference.
> >
> > As far as I understood the Artemis documentation it is possible to
> > configure a cluster and also connect single satellite brokers to this
> > cluster.
> >
> > The satellite brokers can be connected using a bi-directional bridge so I
> > would use the core bridge and would use a network-connector where duplex
> is
> > set to true.
> >
> > Is this a way I could go?
> >
> > Kind Regards,
> > Andreas
> >
> > On Wed, Dec 6, 2017 at 2:39 PM, Tim Bain <[hidden email]> wrote:
> >
> > > Networks of brokers use store-and-forward to move messages between the
> > > brokers, and each message is on only one broker at a time so it is lost
> > (at
> > > least temporarily) if that broker goes offline. It's not a cluster
> under
> > > the definition we just laid out.
> > >
> > > Tim
> > >
> > > On Dec 6, 2017 6:36 AM, "andi welchlin" <[hidden email]>
> wrote:
> > >
> > > > Hello Tim,
> > > >
> > > > yes, that was exactly my definition.
> > > >
> > > > Maybe I misunderstood the documentation of ActiveMQ "network of
> > brokers".
> > > >
> > > > Kind Regards,
> > > > Andreas
> > > >
> > > > On Wed, Dec 6, 2017 at 2:33 PM, Tim Bain <[hidden email]>
> > wrote:
> > > >
> > > > > My definition of a cluster is that a given message is available
> > > (without
> > > > > forwarding) from all nodes in the cluster and will remain available
> > > when
> > > > a
> > > > > single node in the cluster is lost. Master-slave pairs are clusters
> > > (but
> > > > > not active-active clusters) under that definition, while a network
> of
> > > > > brokers is not.
> > > > >
> > > > > So Andi, is that the definition you were using when you wrote this
> > > > > question?
> > > > >
> > > > > Tim
> > > > >
> > > > > On Dec 6, 2017 5:35 AM, "Alec Henninger" <[hidden email]>
> > > > wrote:
> > > > >
> > > > > > Isn't network of brokers an active-active cluster?
> > > > > >
> > > > > > On Tue, Dec 5, 2017, 5:21 PM Tim Bain <[hidden email]>
> > wrote:
> > > > > >
> > > > > > > Not with ActiveMQ 5.x, since it doesn't have a capacity to do
> an
> > > > > > > active-active cluster, but ActiveMQ Artemis can. Have you
> looked
> > at
> > > > it?
> > > > > > >
> > > > > > > Tim
> > > > > > >
> > > > > > > On Dec 5, 2017 7:28 AM, "andi welchlin" <
> [hidden email]
> > >
> > > > > wrote:
> > > > > > >
> > > > > > > > Hello,
> > > > > > > >
> > > > > > > > I would like to setup a active-active cluster of brokers
> using
> > > > > > ActiveMq.
> > > > > > > >
> > > > > > > > But I also would like to connect single ActiveMq satellite
> > > brokers
> > > > to
> > > > > > > this
> > > > > > > > cluster while they should share some queues and exchanges. So
> > > this
> > > > > will
> > > > > > > be
> > > > > > > > pretty much like a federation between some satellite brokers
> > and
> > > a
> > > > > > > central
> > > > > > > > broker cluster. Clients will connect to these satellite
> brokers
> > > > using
> > > > > > > AMQP.
> > > > > > > >
> > > > > > > > Do you think this would be possible using ActiveMq?
> > > > > > > >
> > > > > > > > Kind Regards,
> > > > > > > > Andreas
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Broker Federation with an Active-Active Broker Cluster

andi welchlin
Hello Martyn,

thank you for your answer. It was helpful.

Yes, what you described is what I want to achieve ... but the cluster
should not run within the JVM it should be spread around different
locations worldwide.

So basically what I want to do in the cluster is what I read in one of your
links:

*Apache ActiveMQ Artemis cluster connections can be configured to only
distribute to other nodes if they have matching consumers.*
And to this cluster I would like to connect the satellite brokers.

Kind Regards,
Andreas


On Tue, Dec 12, 2017 at 11:29 AM, Martyn Taylor <[hidden email]> wrote:

> Hi Andreas,
>
> If I understand correctly you're wanting to create a single cluster, with
> active-active style availability, with some satellite brokers?
>
> To set up an active/active style HA cluster in Artemis, you can use using
> co-located pairs[1].  Essentially what this is doing is creating a live and
> a backup broker in a single JVM.  Where the backup here is acting as the
> backup for node on a separate machine.  You can then add satellite brokers
> that get a view into the HA cluster.  You can control exactly which
> addresses are shared between your HA cluster and individual brokers by
> setting the address <address> on the cluster connection[2].
>
> [1] https://activemq.apache.org/artemis/docs/latest/ha.html
> [2] https://activemq.apache.org/artemis/docs/latest/clusters.html
>
> Thanks
> Martyn
>
>
>
>
>
>
> On Tue, Dec 12, 2017 at 7:23 AM, Tim Bain <[hidden email]> wrote:
>
> > Andreas,
> >
> > I don't have enough experience with Artemis to be able to answer your
> > question, so I've been hoping that one of the Artemis folks on this list
> > will jump in to answer. Since that's not happening, you may want to
> start a
> > fresh message thread asking your question specifically in the context of
> > Artemis, in the hopes that people who might be ignoring this "5.x" thread
> > might see and respond to a new Artemis thread.
> >
> > Best,
> > Tim
> >
> > On Thu, Dec 7, 2017 at 9:37 AM, andi welchlin <[hidden email]>
> > wrote:
> >
> > > Hello Tim,
> > >
> > > thank you. Now I got the difference.
> > >
> > > As far as I understood the Artemis documentation it is possible to
> > > configure a cluster and also connect single satellite brokers to this
> > > cluster.
> > >
> > > The satellite brokers can be connected using a bi-directional bridge
> so I
> > > would use the core bridge and would use a network-connector where
> duplex
> > is
> > > set to true.
> > >
> > > Is this a way I could go?
> > >
> > > Kind Regards,
> > > Andreas
> > >
> > > On Wed, Dec 6, 2017 at 2:39 PM, Tim Bain <[hidden email]>
> wrote:
> > >
> > > > Networks of brokers use store-and-forward to move messages between
> the
> > > > brokers, and each message is on only one broker at a time so it is
> lost
> > > (at
> > > > least temporarily) if that broker goes offline. It's not a cluster
> > under
> > > > the definition we just laid out.
> > > >
> > > > Tim
> > > >
> > > > On Dec 6, 2017 6:36 AM, "andi welchlin" <[hidden email]>
> > wrote:
> > > >
> > > > > Hello Tim,
> > > > >
> > > > > yes, that was exactly my definition.
> > > > >
> > > > > Maybe I misunderstood the documentation of ActiveMQ "network of
> > > brokers".
> > > > >
> > > > > Kind Regards,
> > > > > Andreas
> > > > >
> > > > > On Wed, Dec 6, 2017 at 2:33 PM, Tim Bain <[hidden email]>
> > > wrote:
> > > > >
> > > > > > My definition of a cluster is that a given message is available
> > > > (without
> > > > > > forwarding) from all nodes in the cluster and will remain
> available
> > > > when
> > > > > a
> > > > > > single node in the cluster is lost. Master-slave pairs are
> clusters
> > > > (but
> > > > > > not active-active clusters) under that definition, while a
> network
> > of
> > > > > > brokers is not.
> > > > > >
> > > > > > So Andi, is that the definition you were using when you wrote
> this
> > > > > > question?
> > > > > >
> > > > > > Tim
> > > > > >
> > > > > > On Dec 6, 2017 5:35 AM, "Alec Henninger" <
> [hidden email]>
> > > > > wrote:
> > > > > >
> > > > > > > Isn't network of brokers an active-active cluster?
> > > > > > >
> > > > > > > On Tue, Dec 5, 2017, 5:21 PM Tim Bain <[hidden email]>
> > > wrote:
> > > > > > >
> > > > > > > > Not with ActiveMQ 5.x, since it doesn't have a capacity to do
> > an
> > > > > > > > active-active cluster, but ActiveMQ Artemis can. Have you
> > looked
> > > at
> > > > > it?
> > > > > > > >
> > > > > > > > Tim
> > > > > > > >
> > > > > > > > On Dec 5, 2017 7:28 AM, "andi welchlin" <
> > [hidden email]
> > > >
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hello,
> > > > > > > > >
> > > > > > > > > I would like to setup a active-active cluster of brokers
> > using
> > > > > > > ActiveMq.
> > > > > > > > >
> > > > > > > > > But I also would like to connect single ActiveMq satellite
> > > > brokers
> > > > > to
> > > > > > > > this
> > > > > > > > > cluster while they should share some queues and exchanges.
> So
> > > > this
> > > > > > will
> > > > > > > > be
> > > > > > > > > pretty much like a federation between some satellite
> brokers
> > > and
> > > > a
> > > > > > > > central
> > > > > > > > > broker cluster. Clients will connect to these satellite
> > brokers
> > > > > using
> > > > > > > > AMQP.
> > > > > > > > >
> > > > > > > > > Do you think this would be possible using ActiveMq?
> > > > > > > > >
> > > > > > > > > Kind Regards,
> > > > > > > > > Andreas
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Broker Federation with an Active-Active Broker Cluster

Martyn Taylor
No  problem.  My point about the JVM was not that the cluster will run
within the JVM.  It's that the way an active active pair works, is that a
live and a backup server is started within the same JVM.  So a 3 machine
cluster might look like this:

Machine 1) Live A, Backup C
Machine 2) Live B, Backup A,
Machine 3) Live C, Backup B

You have 3 physical machines each acting as a live and a backup for another
machine in the cluster.  The statement you quoted sounds like a
distribution policy configuration option.  ON_DEMAND will ensure that
messages are only distributed to other nodes in the cluster providing they
have consumers present.

Cheers



On Tue, Dec 12, 2017 at 2:56 PM, andi welchlin <[hidden email]>
wrote:

> Hello Martyn,
>
> thank you for your answer. It was helpful.
>
> Yes, what you described is what I want to achieve ... but the cluster
> should not run within the JVM it should be spread around different
> locations worldwide.
>
> So basically what I want to do in the cluster is what I read in one of your
> links:
>
> *Apache ActiveMQ Artemis cluster connections can be configured to only
> distribute to other nodes if they have matching consumers.*
> And to this cluster I would like to connect the satellite brokers.
>
> Kind Regards,
> Andreas
>
>
> On Tue, Dec 12, 2017 at 11:29 AM, Martyn Taylor <[hidden email]>
> wrote:
>
> > Hi Andreas,
> >
> > If I understand correctly you're wanting to create a single cluster, with
> > active-active style availability, with some satellite brokers?
> >
> > To set up an active/active style HA cluster in Artemis, you can use using
> > co-located pairs[1].  Essentially what this is doing is creating a live
> and
> > a backup broker in a single JVM.  Where the backup here is acting as the
> > backup for node on a separate machine.  You can then add satellite
> brokers
> > that get a view into the HA cluster.  You can control exactly which
> > addresses are shared between your HA cluster and individual brokers by
> > setting the address <address> on the cluster connection[2].
> >
> > [1] https://activemq.apache.org/artemis/docs/latest/ha.html
> > [2] https://activemq.apache.org/artemis/docs/latest/clusters.html
> >
> > Thanks
> > Martyn
> >
> >
> >
> >
> >
> >
> > On Tue, Dec 12, 2017 at 7:23 AM, Tim Bain <[hidden email]> wrote:
> >
> > > Andreas,
> > >
> > > I don't have enough experience with Artemis to be able to answer your
> > > question, so I've been hoping that one of the Artemis folks on this
> list
> > > will jump in to answer. Since that's not happening, you may want to
> > start a
> > > fresh message thread asking your question specifically in the context
> of
> > > Artemis, in the hopes that people who might be ignoring this "5.x"
> thread
> > > might see and respond to a new Artemis thread.
> > >
> > > Best,
> > > Tim
> > >
> > > On Thu, Dec 7, 2017 at 9:37 AM, andi welchlin <[hidden email]
> >
> > > wrote:
> > >
> > > > Hello Tim,
> > > >
> > > > thank you. Now I got the difference.
> > > >
> > > > As far as I understood the Artemis documentation it is possible to
> > > > configure a cluster and also connect single satellite brokers to this
> > > > cluster.
> > > >
> > > > The satellite brokers can be connected using a bi-directional bridge
> > so I
> > > > would use the core bridge and would use a network-connector where
> > duplex
> > > is
> > > > set to true.
> > > >
> > > > Is this a way I could go?
> > > >
> > > > Kind Regards,
> > > > Andreas
> > > >
> > > > On Wed, Dec 6, 2017 at 2:39 PM, Tim Bain <[hidden email]>
> > wrote:
> > > >
> > > > > Networks of brokers use store-and-forward to move messages between
> > the
> > > > > brokers, and each message is on only one broker at a time so it is
> > lost
> > > > (at
> > > > > least temporarily) if that broker goes offline. It's not a cluster
> > > under
> > > > > the definition we just laid out.
> > > > >
> > > > > Tim
> > > > >
> > > > > On Dec 6, 2017 6:36 AM, "andi welchlin" <[hidden email]>
> > > wrote:
> > > > >
> > > > > > Hello Tim,
> > > > > >
> > > > > > yes, that was exactly my definition.
> > > > > >
> > > > > > Maybe I misunderstood the documentation of ActiveMQ "network of
> > > > brokers".
> > > > > >
> > > > > > Kind Regards,
> > > > > > Andreas
> > > > > >
> > > > > > On Wed, Dec 6, 2017 at 2:33 PM, Tim Bain <[hidden email]>
> > > > wrote:
> > > > > >
> > > > > > > My definition of a cluster is that a given message is available
> > > > > (without
> > > > > > > forwarding) from all nodes in the cluster and will remain
> > available
> > > > > when
> > > > > > a
> > > > > > > single node in the cluster is lost. Master-slave pairs are
> > clusters
> > > > > (but
> > > > > > > not active-active clusters) under that definition, while a
> > network
> > > of
> > > > > > > brokers is not.
> > > > > > >
> > > > > > > So Andi, is that the definition you were using when you wrote
> > this
> > > > > > > question?
> > > > > > >
> > > > > > > Tim
> > > > > > >
> > > > > > > On Dec 6, 2017 5:35 AM, "Alec Henninger" <
> > [hidden email]>
> > > > > > wrote:
> > > > > > >
> > > > > > > > Isn't network of brokers an active-active cluster?
> > > > > > > >
> > > > > > > > On Tue, Dec 5, 2017, 5:21 PM Tim Bain <[hidden email]
> >
> > > > wrote:
> > > > > > > >
> > > > > > > > > Not with ActiveMQ 5.x, since it doesn't have a capacity to
> do
> > > an
> > > > > > > > > active-active cluster, but ActiveMQ Artemis can. Have you
> > > looked
> > > > at
> > > > > > it?
> > > > > > > > >
> > > > > > > > > Tim
> > > > > > > > >
> > > > > > > > > On Dec 5, 2017 7:28 AM, "andi welchlin" <
> > > [hidden email]
> > > > >
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hello,
> > > > > > > > > >
> > > > > > > > > > I would like to setup a active-active cluster of brokers
> > > using
> > > > > > > > ActiveMq.
> > > > > > > > > >
> > > > > > > > > > But I also would like to connect single ActiveMq
> satellite
> > > > > brokers
> > > > > > to
> > > > > > > > > this
> > > > > > > > > > cluster while they should share some queues and
> exchanges.
> > So
> > > > > this
> > > > > > > will
> > > > > > > > > be
> > > > > > > > > > pretty much like a federation between some satellite
> > brokers
> > > > and
> > > > > a
> > > > > > > > > central
> > > > > > > > > > broker cluster. Clients will connect to these satellite
> > > brokers
> > > > > > using
> > > > > > > > > AMQP.
> > > > > > > > > >
> > > > > > > > > > Do you think this would be possible using ActiveMq?
> > > > > > > > > >
> > > > > > > > > > Kind Regards,
> > > > > > > > > > Andreas
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Broker Federation with an Active-Active Broker Cluster

andi welchlin
Cool, thank you, Martyn!


On Tue, Dec 12, 2017 at 4:01 PM, Martyn Taylor <[hidden email]> wrote:

> No  problem.  My point about the JVM was not that the cluster will run
> within the JVM.  It's that the way an active active pair works, is that a
> live and a backup server is started within the same JVM.  So a 3 machine
> cluster might look like this:
>
> Machine 1) Live A, Backup C
> Machine 2) Live B, Backup A,
> Machine 3) Live C, Backup B
>
> You have 3 physical machines each acting as a live and a backup for another
> machine in the cluster.  The statement you quoted sounds like a
> distribution policy configuration option.  ON_DEMAND will ensure that
> messages are only distributed to other nodes in the cluster providing they
> have consumers present.
>
> Cheers
>
>
>
> On Tue, Dec 12, 2017 at 2:56 PM, andi welchlin <[hidden email]>
> wrote:
>
> > Hello Martyn,
> >
> > thank you for your answer. It was helpful.
> >
> > Yes, what you described is what I want to achieve ... but the cluster
> > should not run within the JVM it should be spread around different
> > locations worldwide.
> >
> > So basically what I want to do in the cluster is what I read in one of
> your
> > links:
> >
> > *Apache ActiveMQ Artemis cluster connections can be configured to only
> > distribute to other nodes if they have matching consumers.*
> > And to this cluster I would like to connect the satellite brokers.
> >
> > Kind Regards,
> > Andreas
> >
> >
> > On Tue, Dec 12, 2017 at 11:29 AM, Martyn Taylor <[hidden email]>
> > wrote:
> >
> > > Hi Andreas,
> > >
> > > If I understand correctly you're wanting to create a single cluster,
> with
> > > active-active style availability, with some satellite brokers?
> > >
> > > To set up an active/active style HA cluster in Artemis, you can use
> using
> > > co-located pairs[1].  Essentially what this is doing is creating a live
> > and
> > > a backup broker in a single JVM.  Where the backup here is acting as
> the
> > > backup for node on a separate machine.  You can then add satellite
> > brokers
> > > that get a view into the HA cluster.  You can control exactly which
> > > addresses are shared between your HA cluster and individual brokers by
> > > setting the address <address> on the cluster connection[2].
> > >
> > > [1] https://activemq.apache.org/artemis/docs/latest/ha.html
> > > [2] https://activemq.apache.org/artemis/docs/latest/clusters.html
> > >
> > > Thanks
> > > Martyn
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Tue, Dec 12, 2017 at 7:23 AM, Tim Bain <[hidden email]>
> wrote:
> > >
> > > > Andreas,
> > > >
> > > > I don't have enough experience with Artemis to be able to answer your
> > > > question, so I've been hoping that one of the Artemis folks on this
> > list
> > > > will jump in to answer. Since that's not happening, you may want to
> > > start a
> > > > fresh message thread asking your question specifically in the context
> > of
> > > > Artemis, in the hopes that people who might be ignoring this "5.x"
> > thread
> > > > might see and respond to a new Artemis thread.
> > > >
> > > > Best,
> > > > Tim
> > > >
> > > > On Thu, Dec 7, 2017 at 9:37 AM, andi welchlin <
> [hidden email]
> > >
> > > > wrote:
> > > >
> > > > > Hello Tim,
> > > > >
> > > > > thank you. Now I got the difference.
> > > > >
> > > > > As far as I understood the Artemis documentation it is possible to
> > > > > configure a cluster and also connect single satellite brokers to
> this
> > > > > cluster.
> > > > >
> > > > > The satellite brokers can be connected using a bi-directional
> bridge
> > > so I
> > > > > would use the core bridge and would use a network-connector where
> > > duplex
> > > > is
> > > > > set to true.
> > > > >
> > > > > Is this a way I could go?
> > > > >
> > > > > Kind Regards,
> > > > > Andreas
> > > > >
> > > > > On Wed, Dec 6, 2017 at 2:39 PM, Tim Bain <[hidden email]>
> > > wrote:
> > > > >
> > > > > > Networks of brokers use store-and-forward to move messages
> between
> > > the
> > > > > > brokers, and each message is on only one broker at a time so it
> is
> > > lost
> > > > > (at
> > > > > > least temporarily) if that broker goes offline. It's not a
> cluster
> > > > under
> > > > > > the definition we just laid out.
> > > > > >
> > > > > > Tim
> > > > > >
> > > > > > On Dec 6, 2017 6:36 AM, "andi welchlin" <[hidden email]
> >
> > > > wrote:
> > > > > >
> > > > > > > Hello Tim,
> > > > > > >
> > > > > > > yes, that was exactly my definition.
> > > > > > >
> > > > > > > Maybe I misunderstood the documentation of ActiveMQ "network of
> > > > > brokers".
> > > > > > >
> > > > > > > Kind Regards,
> > > > > > > Andreas
> > > > > > >
> > > > > > > On Wed, Dec 6, 2017 at 2:33 PM, Tim Bain <
> [hidden email]>
> > > > > wrote:
> > > > > > >
> > > > > > > > My definition of a cluster is that a given message is
> available
> > > > > > (without
> > > > > > > > forwarding) from all nodes in the cluster and will remain
> > > available
> > > > > > when
> > > > > > > a
> > > > > > > > single node in the cluster is lost. Master-slave pairs are
> > > clusters
> > > > > > (but
> > > > > > > > not active-active clusters) under that definition, while a
> > > network
> > > > of
> > > > > > > > brokers is not.
> > > > > > > >
> > > > > > > > So Andi, is that the definition you were using when you wrote
> > > this
> > > > > > > > question?
> > > > > > > >
> > > > > > > > Tim
> > > > > > > >
> > > > > > > > On Dec 6, 2017 5:35 AM, "Alec Henninger" <
> > > [hidden email]>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Isn't network of brokers an active-active cluster?
> > > > > > > > >
> > > > > > > > > On Tue, Dec 5, 2017, 5:21 PM Tim Bain <
> [hidden email]
> > >
> > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Not with ActiveMQ 5.x, since it doesn't have a capacity
> to
> > do
> > > > an
> > > > > > > > > > active-active cluster, but ActiveMQ Artemis can. Have you
> > > > looked
> > > > > at
> > > > > > > it?
> > > > > > > > > >
> > > > > > > > > > Tim
> > > > > > > > > >
> > > > > > > > > > On Dec 5, 2017 7:28 AM, "andi welchlin" <
> > > > [hidden email]
> > > > > >
> > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hello,
> > > > > > > > > > >
> > > > > > > > > > > I would like to setup a active-active cluster of
> brokers
> > > > using
> > > > > > > > > ActiveMq.
> > > > > > > > > > >
> > > > > > > > > > > But I also would like to connect single ActiveMq
> > satellite
> > > > > > brokers
> > > > > > > to
> > > > > > > > > > this
> > > > > > > > > > > cluster while they should share some queues and
> > exchanges.
> > > So
> > > > > > this
> > > > > > > > will
> > > > > > > > > > be
> > > > > > > > > > > pretty much like a federation between some satellite
> > > brokers
> > > > > and
> > > > > > a
> > > > > > > > > > central
> > > > > > > > > > > broker cluster. Clients will connect to these satellite
> > > > brokers
> > > > > > > using
> > > > > > > > > > AMQP.
> > > > > > > > > > >
> > > > > > > > > > > Do you think this would be possible using ActiveMq?
> > > > > > > > > > >
> > > > > > > > > > > Kind Regards,
> > > > > > > > > > > Andreas
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>