Slow consumer on Queue ?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Slow consumer on Queue ?

Jérôme Barotin
Hi,

I just went to read the page in documentation about slow consumer
handling : http://activemq.apache.org/slow-consumer-handling.html and I
have two questions :

  * Is "slow consumer" feature exists about queue ? in the doc there's
    only topic mentioned.
  * what is the default pending message limit strategy ?

Best,

--

*Jérome*
Reply | Threaded
Open this post in threaded view
|

Re: Slow consumer on Queue ?

Tim Bain
Slow consumers can be identified for both queues and topics. But whereas
one of the primary responses (terminate the subscription and force the
consumer to re-subscribe) will protect the broker in the case of a
non-durable topic subscription (because the broker no longer needs to keep
those unconsumed messages for that consumer, since there is no longer a
subscription for the consumer), forcing a disconnect of a slow consumer for
a queue or a durable topic subscription typically just make the consumer
even slower by interfering with its ability to consume the messages. So
although it's definitely possible to identify slow consumers for both
queues and topics, people tend to focus more on topics than on queues.

The pending message limit strategy will discard older unconsumed messages
for a particular subscription, to keep the total number of unconsumed
messages below a particular maximum value. There are two flavors:
ConstantPendingMessageLimitStrategy, which has a fixed limit that's applied
to all consumers, and PrefetchRatePendingMessageLimitStrategy, which
specifies the value as a multiplier on each consumer's prefetch queue size
(and therefore allows the possibility of differing limits for different
consumers).

Tim

On Thu, Nov 21, 2019 at 2:50 AM Jérôme Barotin <[hidden email]> wrote:

> Hi,
>
> I just went to read the page in documentation about slow consumer
> handling : http://activemq.apache.org/slow-consumer-handling.html and I
> have two questions :
>
>   * Is "slow consumer" feature exists about queue ? in the doc there's
>     only topic mentioned.
>   * what is the default pending message limit strategy ?
>
> Best,
>
> --
>
> *Jérome*
>
Reply | Threaded
Open this post in threaded view
|

Re: Slow consumer on Queue ?

Tim Bain
A few things.

1) Slow consumers themselves can't be disabled. What can be enabled or
disabled is configuration for what the broker should do in response to
them. And you should definitely configure limits to protect the broker in
the event that a consumer is consistently slower than the producer it's
consuming from (or the even that the consumer just disappears), because you
don't want the broker to run out of memory or fill its message store and be
unable to accept more messages if one of those things were to happen.

2) Persistent vs. non-persistent messaging is irrelevant to the question of
how to respond to slow consumers. A slow consumer can cause the same types
of problems for non-persistent messages as it can for persistent ones.

3) As shown in the example near the bottom of the page you linked to, you
enable these strategies in your activemq.xml file, by choosing wildcard
patterns for the topic/queue names to match and then specifying the slow
consumer handling strategy (if any) you want to use for destinations
matching that pattern.

Tim

On Mon, Nov 25, 2019 at 5:58 AM Jérôme Barotin <[hidden email]> wrote:

> Thanks Tim for your answer.
>
> So from what I understand, slow consumer should be disabled when
> persistence is enabled ?
>
> I'm using kahadb so how to make sure it's the case ?
>
> Jérôme
>
> Le 25/11/2019 à 07:14, Tim Bain a écrit :
> > Slow consumers can be identified for both queues and topics. But whereas
> > one of the primary responses (terminate the subscription and force the
> > consumer to re-subscribe) will protect the broker in the case of a
> > non-durable topic subscription (because the broker no longer needs to
> keep
> > those unconsumed messages for that consumer, since there is no longer a
> > subscription for the consumer), forcing a disconnect of a slow consumer
> for
> > a queue or a durable topic subscription typically just make the consumer
> > even slower by interfering with its ability to consume the messages. So
> > although it's definitely possible to identify slow consumers for both
> > queues and topics, people tend to focus more on topics than on queues.
> >
> > The pending message limit strategy will discard older unconsumed messages
> > for a particular subscription, to keep the total number of unconsumed
> > messages below a particular maximum value. There are two flavors:
> > ConstantPendingMessageLimitStrategy, which has a fixed limit that's
> applied
> > to all consumers, and PrefetchRatePendingMessageLimitStrategy, which
> > specifies the value as a multiplier on each consumer's prefetch queue
> size
> > (and therefore allows the possibility of differing limits for different
> > consumers).
> >
> > Tim
> >
> > On Thu, Nov 21, 2019 at 2:50 AM Jérôme Barotin <[hidden email]> wrote:
> >
> >> Hi,
> >>
> >> I just went to read the page in documentation about slow consumer
> >> handling : http://activemq.apache.org/slow-consumer-handling.html and I
> >> have two questions :
> >>
> >>    * Is "slow consumer" feature exists about queue ? in the doc there's
> >>      only topic mentioned.
> >>    * what is the default pending message limit strategy ?
> >>
> >> Best,
> >>
> >> --
> >>
> >> *Jérome*
> >>
>
>