Using topics for publishing broker metrics

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Using topics for publishing broker metrics

Raffi
Wanted to run this by the community for some advice.

We have 20 brokers in total (10 master/slave pairs). Operational tooling is an issue, we need something simple to address short term needs. So our goal is to create a web page showing unified operational view for the entire cluster. Simple metrics for each broker (master/slave role, process load, memory, connections, store/temp usage), is all we need for now. Each broker

Our first thought was to create a JMX spring service configured to run inside the broker. The service opens a local JMX connection, polls the desired metrics, and publishes to "topic/metrics" on the local broker as a simple json payload. The problem with this approach is it doesn't work when the broker is a slave.

An alternative option is along the same lines, a JMX service running inside the broker, but the difference is that, on startup, the services creates an embedded broker with single network connector configured to talk to all other brokers in the cluster. Now, regardless of the broker's role, "topic/metrics" is always a valid topic for publishing and consuming metrics info for the entire cluster.

Another option is using advisory messages for carrying metrics info, although we're not sure if flooding this channel with such data will negatively impact openwire clients receiving cluster updates with no interest in metrics.
Reply | Threaded
Open this post in threaded view
|

Re: Using topics for publishing broker metrics

Tim Bain
I wouldn't try to make your own advisory topics; just use normal topics as
you've proposed.

Wouldn't it be simpler to publish to a failover URL containing all the
brokers rather than starting an embedded broker whose sole purpose is to
publish stats?  It seems like your proposed approach adds unnecessary
complexity.

Tim
On Dec 16, 2015 3:59 PM, "Raffi" <[hidden email]> wrote:

> Wanted to run this by the community for some advice.
>
> We have 20 brokers in total (10 master/slave pairs). Operational tooling is
> an issue, we need something simple to address short term needs. So our goal
> is to create a web page showing unified operational view for the entire
> cluster. Simple metrics for each broker (master/slave role, process load,
> memory, connections, store/temp usage), is all we need for now. Each broker
>
> Our first thought was to create a JMX spring service configured to run
> inside the broker. The service opens a local JMX connection, polls the
> desired metrics, and publishes to "topic/metrics" on the local broker as a
> simple json payload. The problem with this approach is it doesn't work when
> the broker is a slave.
>
> An alternative option is along the same lines, a JMX service running inside
> the broker, but the difference is that, on startup, the services creates an
> embedded broker with single network connector configured to talk to all
> other brokers in the cluster. Now, regardless of the broker's role,
> "topic/metrics" is always a valid topic for publishing and consuming
> metrics
> info for the entire cluster.
>
> Another option is using advisory messages for carrying metrics info,
> although we're not sure if flooding this channel with such data will
> negatively impact openwire clients receiving cluster updates with no
> interest in metrics.
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Using-topics-for-publishing-broker-metrics-tp4705077.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

RE: Using topics for publishing broker metrics

Devlin
Hi Tim,

We assumed if the broker was a slave it would be unable to publish to itself. But you're saying each broker should publish to "failover:(all brokers in cluster)" instead of "tcp://localhost:61616"?



-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Tim Bain
Sent: Thursday, December 17, 2015 8:52 AM
To: ActiveMQ Users
Subject: Re: Using topics for publishing broker metrics [ EXTERNAL ]

I wouldn't try to make your own advisory topics; just use normal topics as you've proposed.

Wouldn't it be simpler to publish to a failover URL containing all the brokers rather than starting an embedded broker whose sole purpose is to publish stats?  It seems like your proposed approach adds unnecessary complexity.

Tim
On Dec 16, 2015 3:59 PM, "Raffi" <[hidden email]> wrote:

> Wanted to run this by the community for some advice.
>
> We have 20 brokers in total (10 master/slave pairs). Operational
> tooling is an issue, we need something simple to address short term
> needs. So our goal is to create a web page showing unified operational
> view for the entire cluster. Simple metrics for each broker
> (master/slave role, process load, memory, connections, store/temp
> usage), is all we need for now. Each broker
>
> Our first thought was to create a JMX spring service configured to run
> inside the broker. The service opens a local JMX connection, polls the
> desired metrics, and publishes to "topic/metrics" on the local broker
> as a simple json payload. The problem with this approach is it doesn't
> work when the broker is a slave.
>
> An alternative option is along the same lines, a JMX service running
> inside the broker, but the difference is that, on startup, the
> services creates an embedded broker with single network connector
> configured to talk to all other brokers in the cluster. Now,
> regardless of the broker's role, "topic/metrics" is always a valid
> topic for publishing and consuming metrics info for the entire
> cluster.
>
> Another option is using advisory messages for carrying metrics info,
> although we're not sure if flooding this channel with such data will
> negatively impact openwire clients receiving cluster updates with no
> interest in metrics.
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Using-topics-for-publishing-brok
> er-metrics-tp4705077.html Sent from the ActiveMQ - User mailing list
> archive at Nabble.com.
>

This e-mail transmission may contain information that is proprietary, privileged and/or confidential and is intended exclusively for the person(s) to whom it is addressed. Any use, copying, retention or disclosure by any person other than the intended recipient or the intended recipient's designees is strictly prohibited. If you are not the intended recipient or their designee, please notify the sender immediately by return e-mail and delete all copies. OppenheimerFunds may, at its sole discretion, monitor, review, retain and/or disclose the content of all email communications.
Reply | Threaded
Open this post in threaded view
|

RE: Using topics for publishing broker metrics

Tim Bain
Your assumption is valid, and having both the master and the slave (times N
pairs) in the failover URI guarantees that you'll be able to reach a broker
no matter who's active.
On Dec 17, 2015 7:28 AM, "Basmajian, Raffi" <[hidden email]>
wrote:

> Hi Tim,
>
> We assumed if the broker was a slave it would be unable to publish to
> itself. But you're saying each broker should publish to "failover:(all
> brokers in cluster)" instead of "tcp://localhost:61616"?
>
>
>
> -----Original Message-----
> From: [hidden email] [mailto:[hidden email]] On Behalf Of Tim Bain
> Sent: Thursday, December 17, 2015 8:52 AM
> To: ActiveMQ Users
> Subject: Re: Using topics for publishing broker metrics [ EXTERNAL ]
>
> I wouldn't try to make your own advisory topics; just use normal topics as
> you've proposed.
>
> Wouldn't it be simpler to publish to a failover URL containing all the
> brokers rather than starting an embedded broker whose sole purpose is to
> publish stats?  It seems like your proposed approach adds unnecessary
> complexity.
>
> Tim
> On Dec 16, 2015 3:59 PM, "Raffi" <[hidden email]> wrote:
>
> > Wanted to run this by the community for some advice.
> >
> > We have 20 brokers in total (10 master/slave pairs). Operational
> > tooling is an issue, we need something simple to address short term
> > needs. So our goal is to create a web page showing unified operational
> > view for the entire cluster. Simple metrics for each broker
> > (master/slave role, process load, memory, connections, store/temp
> > usage), is all we need for now. Each broker
> >
> > Our first thought was to create a JMX spring service configured to run
> > inside the broker. The service opens a local JMX connection, polls the
> > desired metrics, and publishes to "topic/metrics" on the local broker
> > as a simple json payload. The problem with this approach is it doesn't
> > work when the broker is a slave.
> >
> > An alternative option is along the same lines, a JMX service running
> > inside the broker, but the difference is that, on startup, the
> > services creates an embedded broker with single network connector
> > configured to talk to all other brokers in the cluster. Now,
> > regardless of the broker's role, "topic/metrics" is always a valid
> > topic for publishing and consuming metrics info for the entire
> > cluster.
> >
> > Another option is using advisory messages for carrying metrics info,
> > although we're not sure if flooding this channel with such data will
> > negatively impact openwire clients receiving cluster updates with no
> > interest in metrics.
> >
> >
> >
> > --
> > View this message in context:
> > http://activemq.2283324.n4.nabble.com/Using-topics-for-publishing-brok
> > er-metrics-tp4705077.html Sent from the ActiveMQ - User mailing list
> > archive at Nabble.com.
> >
>
> This e-mail transmission may contain information that is proprietary,
> privileged and/or confidential and is intended exclusively for the
> person(s) to whom it is addressed. Any use, copying, retention or
> disclosure by any person other than the intended recipient or the intended
> recipient's designees is strictly prohibited. If you are not the intended
> recipient or their designee, please notify the sender immediately by return
> e-mail and delete all copies. OppenheimerFunds may, at its sole discretion,
> monitor, review, retain and/or disclose the content of all email
> communications.
>