Hub and spoke network topology

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Hub and spoke network topology

Jako
Hi,
 we have a lot of brokers all attached to a central H broker in a hub and spoke topology.
Clients will connect to the external brokers A, B, C, D, E and F.


I have a working setup like this with more or less 350 external brokers.
Through logs and jmx I see that all the queues are 'present' on all the clients.
Actually the clients of different brokers have to speak to each other in rare occasions.

So, is there a way to lighten the clients and remove the existence of the whole network queues from their awareness? In exclusion of the queues of the same broker clients.
You could say, and how could clients of different nodes communicate?
I hope that there is a mean to achieve routing as the ip protocol: when the broker A receives a message for a non local queue, it should forward it to the H broker; H will have all the queues in the network, that's expected.

Is this possible, or am I talking nonsense?

Thank you for your help
Simone

Reply | Threaded
Open this post in threaded view
|

Re: Hub and spoke network topology

Tim Bain
If you know in advance which destinations are local-only and which ones
might need to be routed across the network of brokers, it's possible to
dynamically include or exclude destinations whose names match certain
patterns.  See activemq.apache.org/networks-of-brokers.html for more
details.

You can also go to a fully statically routed network of brokers to
eliminate the overhead of advisory messages; details are at the same link.

But are you sure you need to?  Are you sure this isn't a premature
optimization that will have a cost (more complicated broker configs) but no
measurable benefit?  Do you have an indication that either of those is
necessary from a performance standpoint?  What's driving your original
question?
On Oct 28, 2015 3:32 AM, "Jako" <[hidden email]> wrote:

> Hi,
>  we have a lot of brokers all attached to a central H broker in a hub and
> spoke topology.
> Clients will connect to the external brokers A, B, C, D, E and F.
>
>
> I have a working setup like this with more or less 350 external brokers.
> Through logs and jmx I see that all the queues are 'present' on all the
> clients.
> Actually the clients of different brokers have to speak to each other in
> rare occasions.
>
> So, is there a way to lighten the clients and remove the existence of the
> whole network queues from their awareness? In exclusion of the queues of
> the
> same broker clients.
> You could say, and how could clients of different nodes communicate?
> I hope that there is a mean to achieve routing as the ip protocol: when the
> broker A receives a message for a non local queue, it should forward it to
> the H broker; H will have all the queues in the network, that's expected.
>
> Is this possible, or am I talking nonsense?
>
> Thank you for your help
> Simone
>
> <
> http://activemq.2283324.n4.nabble.com/file/n4703434/broker_networks_09.gif
> >
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Hub-and-spoke-network-topology-tp4703434.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: Hub and spoke network topology

Jako
Hi Tim,
  thank you for your replay and your very thoughtful questions.

>But are you sure you need to? Are you sure this isn't a premature [...]

I think so. The central broker is on our server farm and it is very resourceful machine; anyway it often uses a lot of cpu. The broker uses thousands of threads.
Beside this, the external brokers are on cheap hardware and I think the advisory messages about hundreds queues are too much.
In any case those information doesn't really belong there; those computers are not fully trusted.
And sometimes the network don't behave very well: some messages are lost.

I'm not sure the issue of the lost messages arises from the ActiveMQ network, but I want to clean up things and have a more appropriate structure.

Yesterday I tried a lot of combinations with some success.
But I really didn't land to a final and satisfactory configuration.

The best I could hope for is a sort of 'default gateway', making an analogy to the ip protocol.

I tried to emulate that using a staticallyIncludedDestinations(">") queue and using decreaseNetworkConsumerPriority(true). But this had side effects.
And many others configurations...

I wonder one thing, among others: the network-connector of the external brokers are in full-duplex mode and I'm able to setup many parameters: consumerTTL, staticallyIncludedDestinations, staticBridge.
But, I can't do it in the reverse direction.
In other words, to have the same options I think I should make the network-connector from the H central broker to the external brokers. It would be nice to able to specify in the H center broker that it had to prefer local consumer (I did it with decreaseNetworkConsumerPriority) but I don't know how to set that in the central broker.

Maybe I'm rambling a bit, because I tried a lot of different configurations, avoiding the static routing; I couldn't use it because it's not possible to know in advance the network structure.

Thank you again

Simone
Reply | Threaded
Open this post in threaded view
|

Re: Hub and spoke network topology

Tim Bain
Do you have any evidence that advisory messages about hundreds of empty
queues actually has a measurable impact on either your central broker or
your edge brokers?  It's definitely possible that the overhead of advisory
messages really is too high, but if you haven't measured it somehow and
you're just going with your gut on this, there's a very good chance you're
doing premature optimization.  The path you're going down has pain points
(some of which you've found), so if you don't need to go down it, you
should save yourself the trouble and let your brokers worry about message
routing instead worrying about it for them.

Statically including all destinations will send messages for all
destinations to the central broker when all consumers are offline, even if
the consumers will only ever be on the local broker.  That doesn't sound
like what you want, particularly because the default configuration doesn't
allow messages to go back to a broker they've already been to.

I thought you'd have just dynamically included those destinations that can
have non-local consumers; is there a reason that's not the path you're
going down?

Setting the dynamically included destinations from the local broker side of
the duplex connections should be fine; you're trying to limit the ability
for certain messages to escape the local broker, not prevent the central
broker from passing on messages (which couldn't happen anyway if all local
brokers are properly configured).

Tim
On Oct 29, 2015 11:28 AM, "Jako" <[hidden email]> wrote:

> Hi Tim,
>   thank you for your replay and your very thoughtful questions.
>
> >But are you sure you need to? Are you sure this isn't a premature [...]
>
> I think so. The central broker is on our server farm and it is very
> resourceful machine; anyway it often uses a lot of cpu. The broker uses
> thousands of threads.
> Beside this, the external brokers are on cheap hardware and I think the
> advisory messages about hundreds queues are too much.
> In any case those information doesn't really belong there; those computers
> are not fully trusted.
> And sometimes the network don't behave very well: some messages are lost.
>
> I'm not sure the issue of the lost messages arises from the ActiveMQ
> network, but I want to clean up things and have a more appropriate
> structure.
>
> Yesterday I tried a lot of combinations with some success.
> But I really didn't land to a final and satisfactory configuration.
>
> The best I could hope for is a sort of 'default gateway', making an analogy
> to the ip protocol.
>
> I tried to emulate that using a staticallyIncludedDestinations(">") queue
> and using decreaseNetworkConsumerPriority(true). But this had side effects.
> And many others configurations...
>
> I wonder one thing, among others: the network-connector of the external
> brokers are in full-duplex mode and I'm able to setup many parameters:
> consumerTTL, staticallyIncludedDestinations, staticBridge.
> But, I can't do it in the reverse direction.
> In other words, to have the same options I think I should make the
> network-connector from the H central broker to the external brokers. It
> would be nice to able to specify in the H center broker that it had to
> prefer local consumer (I did it with decreaseNetworkConsumerPriority) but I
> don't know how to set that in the central broker.
>
> Maybe I'm rambling a bit, because I tried a lot of different
> configurations,
> avoiding the static routing; I couldn't use it because it's not possible to
> know in advance the network structure.
>
> Thank you again
>
> Simone
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Hub-and-spoke-network-topology-tp4703434p4703491.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: Hub and spoke network topology

Jako
Is there a way to trace all messages passing through a broker?
In that way I could verify what load the server is under.

>Statically including all destinations will send messages for all
>destinations to the central broker when all consumers are offline, even if
>the consumers will only ever be on the local broker.  That doesn't sound
>like what you want [...]

Actually this would be quite acceptable: it is an unlikely situation.

Thanks
Reply | Threaded
Open this post in threaded view
|

Re: Hub and spoke network topology

Tim Bain
That depends what you mean by "trace".  If you mean "get counts per
destination at a point in time, so you can subtract the values at two times
to find the number of messages in that time period", you can use the web
console or JMX to get that information.  If you mean something else, please
explain what you mean.

Keep in mind that advisory messages are only sent when clients connect and
disconnect (and when messages expire), so your expected message rates for
advisory messages is very low.  I don't think you're going to see the
performance problems that it seems you think you'll see.
On Oct 30, 2015 8:25 AM, "Jako" <[hidden email]> wrote:

> Is there a way to trace all messages passing through a broker?
> In that way I could verify what load the server is under.
>
> >Statically including all destinations will send messages for all
> >destinations to the central broker when all consumers are offline, even if
> >the consumers will only ever be on the local broker.  That doesn't sound
> >like what you want [...]
>
> Actually this would be quite acceptable: it is an unlikely situation.
>
> Thanks
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Hub-and-spoke-network-topology-tp4703434p4703519.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>