browsing an entire queue?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

browsing an entire queue?

Kevin Burton
This seems like a question that is probably answered somewhere but heck if
I can find it.

I want to browse a significant amount of messages in a queue.  Some of my
queues can have thousands and thousands of messages and I’d like to figure
out why they are backing up.

So I want to write some code to compute some stats via a sample and issue a
report.

Which I could do easily by just browsing the queue.

But I’m limited to 400.  Which apparently is hard coded here in the broker.

http://activemq.apache.org/per-destination-policies.html

… but I’d like to be able to browse more.  On the order of 1-5k.

So how does one tell the client to browse more than 400 messages?  Or do I
need to reconfigure the broker as there is no way of doing this otherwise.

--

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Reply | Threaded
Open this post in threaded view
|

Re: browsing an entire queue?

Tim Bain
I don't know if there's a way to do this without changing the broker config
(I doubt it, but I don't have any direct knowledge either way), but it
seems like a simple enough change to the broker config.  Other than the
need for a broker restart, is there a reason not to just do that?

Alternatively, you might be able to dump all messages to a log file by
adding a Camel route to the broker and using a wiretap within that route
that logs out each message as it passes through.  I've never used Camel
embedded on the broker, so I'm not sure if this route could be added
without a broker restart...

But more generally, do you need to browse more than 400?  Is that number of
messages really not enough to figure out why messages are backing up, and
yet seeing a few thousand is expected to show exactly what's going on?  And
even if you do need a larger sample size, can't you just browse multiple
times over a period of time and report on the aggregate numbers?

On Tue, Nov 4, 2014 at 7:51 PM, Kevin Burton <[hidden email]> wrote:

> This seems like a question that is probably answered somewhere but heck if
> I can find it.
>
> I want to browse a significant amount of messages in a queue.  Some of my
> queues can have thousands and thousands of messages and I’d like to figure
> out why they are backing up.
>
> So I want to write some code to compute some stats via a sample and issue a
> report.
>
> Which I could do easily by just browsing the queue.
>
> But I’m limited to 400.  Which apparently is hard coded here in the broker.
>
> http://activemq.apache.org/per-destination-policies.html
>
> … but I’d like to be able to browse more.  On the order of 1-5k.
>
> So how does one tell the client to browse more than 400 messages?  Or do I
> need to reconfigure the broker as there is no way of doing this otherwise.
>
> --
>
> Founder/CEO Spinn3r.com
> Location: *San Francisco, CA*
> blog: http://burtonator.wordpress.com
> … or check out my Google+ profile
> <https://plus.google.com/102718274791889610666/posts>
> <http://spinn3r.com>
>
Reply | Threaded
Open this post in threaded view
|

Re: browsing an entire queue?

seijoed
What Tim said and this sounds like a bad dequeue scenario where you either don't have enough consumer or they are way too slow, it also sounds a bit like using Amq as data storage.

On Wed, Nov 5, 2014 at 9:07 AM, Tim Bain <[hidden email]> wrote:

> I don't know if there's a way to do this without changing the broker config
> (I doubt it, but I don't have any direct knowledge either way), but it
> seems like a simple enough change to the broker config.  Other than the
> need for a broker restart, is there a reason not to just do that?
> Alternatively, you might be able to dump all messages to a log file by
> adding a Camel route to the broker and using a wiretap within that route
> that logs out each message as it passes through.  I've never used Camel
> embedded on the broker, so I'm not sure if this route could be added
> without a broker restart...
> But more generally, do you need to browse more than 400?  Is that number of
> messages really not enough to figure out why messages are backing up, and
> yet seeing a few thousand is expected to show exactly what's going on?  And
> even if you do need a larger sample size, can't you just browse multiple
> times over a period of time and report on the aggregate numbers?
> On Tue, Nov 4, 2014 at 7:51 PM, Kevin Burton <[hidden email]> wrote:
>> This seems like a question that is probably answered somewhere but heck if
>> I can find it.
>>
>> I want to browse a significant amount of messages in a queue.  Some of my
>> queues can have thousands and thousands of messages and I’d like to figure
>> out why they are backing up.
>>
>> So I want to write some code to compute some stats via a sample and issue a
>> report.
>>
>> Which I could do easily by just browsing the queue.
>>
>> But I’m limited to 400.  Which apparently is hard coded here in the broker.
>>
>> http://activemq.apache.org/per-destination-policies.html
>>
>> … but I’d like to be able to browse more.  On the order of 1-5k.
>>
>> So how does one tell the client to browse more than 400 messages?  Or do I
>> need to reconfigure the broker as there is no way of doing this otherwise.
>>
>> --
>>
>> Founder/CEO Spinn3r.com
>> Location: *San Francisco, CA*
>> blog: http://burtonator.wordpress.com
>> … or check out my Google+ profile
>> <https://plus.google.com/102718274791889610666/posts>
>> <http://spinn3r.com>
>>
Reply | Threaded
Open this post in threaded view
|

Re: browsing an entire queue?

gtully
There is no way to browse all the messages in very large queue - only what
can fit in memory.

There is a destination policy entry  - maxBrowsePageSize - that controls
the max messages to page in for a browse. If that is very large it will be
limited by the destination memory limit.


On 5 November 2014 16:19, <[hidden email]> wrote:

> What Tim said and this sounds like a bad dequeue scenario where you either
> don't have enough consumer or they are way too slow, it also sounds a bit
> like using Amq as data storage.
>
> On Wed, Nov 5, 2014 at 9:07 AM, Tim Bain <[hidden email]> wrote:
>
> > I don't know if there's a way to do this without changing the broker
> config
> > (I doubt it, but I don't have any direct knowledge either way), but it
> > seems like a simple enough change to the broker config.  Other than the
> > need for a broker restart, is there a reason not to just do that?
> > Alternatively, you might be able to dump all messages to a log file by
> > adding a Camel route to the broker and using a wiretap within that route
> > that logs out each message as it passes through.  I've never used Camel
> > embedded on the broker, so I'm not sure if this route could be added
> > without a broker restart...
> > But more generally, do you need to browse more than 400?  Is that number
> of
> > messages really not enough to figure out why messages are backing up, and
> > yet seeing a few thousand is expected to show exactly what's going on?
> And
> > even if you do need a larger sample size, can't you just browse multiple
> > times over a period of time and report on the aggregate numbers?
> > On Tue, Nov 4, 2014 at 7:51 PM, Kevin Burton <[hidden email]> wrote:
> >> This seems like a question that is probably answered somewhere but heck
> if
> >> I can find it.
> >>
> >> I want to browse a significant amount of messages in a queue.  Some of
> my
> >> queues can have thousands and thousands of messages and I’d like to
> figure
> >> out why they are backing up.
> >>
> >> So I want to write some code to compute some stats via a sample and
> issue a
> >> report.
> >>
> >> Which I could do easily by just browsing the queue.
> >>
> >> But I’m limited to 400.  Which apparently is hard coded here in the
> broker.
> >>
> >> http://activemq.apache.org/per-destination-policies.html
> >>
> >> … but I’d like to be able to browse more.  On the order of 1-5k.
> >>
> >> So how does one tell the client to browse more than 400 messages?  Or
> do I
> >> need to reconfigure the broker as there is no way of doing this
> otherwise.
> >>
> >> --
> >>
> >> Founder/CEO Spinn3r.com
> >> Location: *San Francisco, CA*
> >> blog: http://burtonator.wordpress.com
> >> … or check out my Google+ profile
> >> <https://plus.google.com/102718274791889610666/posts>
> >> <http://spinn3r.com>
> >>
>