Messages expiration not always working

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Messages expiration not always working

ncasaux
This post was updated on .
Hello,

I have a configuration with one virtualtopic and three queues.
I send multiples messages on the virtualtopic and they all have a expiration
date.

Sometime, when the queue has no consumer, the expired messages on the queue
are not removed properly and remain on the queue, but this does not happen
to all the queues...
I don't know how to reproduce this:

<http://activemq.2283324.n4.nabble.com/file/t379235/Capture.png

As you can see on the picture, the 2 queues with no consumer should have the
same size, but it's not the case...

The problem is that is uses a huge amount of diskspace:
<http://activemq.2283324.n4.nabble.com/file/t379235/Capture2.png

As soon as I browse the queue from the webconsole, I can see that the
expiration mechanism starts, and after a couple of minutes, all the expired
messages are gone.

There is no error message or weird message in the log file.

Could anyone explain why this happens ?
I'm using ActiveMQ 5.15.4

Thanks in advance.







--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: Messages expiration not always working

Tim Bain
It's my understanding (from what others have written on this mailing list,
not from checking the code) that message expiration is checked twice: once
broker-side when the message is dispatched to the consumer (or queue
browser), and once client-side when the message is taken from the front of
the prefetch buffer to be consumed. Both of those mechanisms rely on having
a consumer, and I believe that the design decision was made to not create
an additional code path that would require runtime resources and code
maintenance simply to expire messages in the absence of a consumer.

Tim

On Oct 22, 2018 2:11 PM, "ncasaux" <[hidden email]> wrote:

Hello,

I have a configuration with one virtualtopic and three queues.
I send multiples messages on the virtualtopic and they all have a expiration
date.

Sometime, when the queue has no consumer, the expired messages on the queue
are not removed properly and remain on the queue, but this does not happen
to all the queues...
I don't know how to reproduce this:

<http://activemq.2283324.n4.nabble.com/file/t379235/Capture.png>

As you can see on the picture, the 2 queues with no consumer should have the
same size, but it's not the case...

The problem is that is uses a huge amount of diskspace:
<http://activemq.2283324.n4.nabble.com/file/t379235/Capture2.png>

As soon as I browse the queue from the webconsole, I can see that the
expiration mechanism starts, and after a couple of minutes, all the expired
messages are gone.

Could anyone explain why this happens ?
I'm using ActiveMQ 5.15.4

Thanks in advance.







--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: Messages expiration not always working

ncasaux
Problem occurred again on a queue 3 days after it was created and no consumer
at all. ActiveMQ finally shut down due to not enough space on the machine.



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: Messages expiration not always working

jbertram
FWIW, ActiveMQ Artemis supports message expiration on the broker without a
consumer via the "expiry reaper thread" [1].  I realize you may not be able
to update your ActiveMQ broker, but I thought it was worth presenting the
option.


Justin

[1] https://activemq.apache.org/artemis/docs/latest/message-expiry.html



On Thu, Nov 15, 2018 at 3:47 PM ncasaux <[hidden email]> wrote:

> Problem occurred again on a queue 3 days after it was created and no
> consumer
> at all. ActiveMQ finally shut down due to not enough space on the machine.
>
>
>
> --
> Sent from:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
>
Reply | Threaded
Open this post in threaded view
|

Re: Messages expiration not always working

Tim Bain
In reply to this post by ncasaux
You're welcome to submit an enhancement request in JIRA for this. I think
it would be a useful feature.

Tim

On Thu, Nov 15, 2018, 2:47 PM ncasaux <[hidden email] wrote:

> Problem occurred again on a queue 3 days after it was created and no
> consumer
> at all. ActiveMQ finally shut down due to not enough space on the machine.
>
>
>
> --
> Sent from:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
>