Unexpected DLQ behaviour with non-persistent messages

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Unexpected DLQ behaviour with non-persistent messages

Martin Lichtin
Can somone help me understand why a non-persistent message is "converted" into a persistent message on entering DLQ?
It happens in RegionBroker, around line 733:

                             if (!message.isPersistent()) {
                                 message.setPersistent(true);
message.setProperty("originalDeliveryMode", "NON_PERSISTENT");
                             }

I'm puzzled as to why this would make sense.

A non-persistent message, even if configured to use DLQ, should stay non-persistent, i.e. not survive broker restarts.

- Martin

Reply | Threaded
Open this post in threaded view
|

Re: Unexpected DLQ behaviour with non-persistent messages

jbonofre
Hi,

If you enable DLQ to non-persistent message (it's disabled by default),
then, IMHO, any message stored in DLQ should survive to broker restart
(else it's not really a DLQ).
Again, as non-persistent messages are not supposed to go in DLQ. To send
the non-persistent messages to DLQ, you have to use for instance:

 <sharedDeadLetterStrategy processNonPersistent="true" />

In that case, it makes sense to "store" the message persistently in the DLQ.

Regards
JB

On 30/01/2020 17:35, Martin Lichtin wrote:

> Can somone help me understand why a non-persistent message is
> "converted" into a persistent message on entering DLQ?
> It happens in RegionBroker, around line 733:
>
>                             if (!message.isPersistent()) {
>                                 message.setPersistent(true);
> message.setProperty("originalDeliveryMode", "NON_PERSISTENT");
>                             }
>
> I'm puzzled as to why this would make sense.
>
> A non-persistent message, even if configured to use DLQ, should stay
> non-persistent, i.e. not survive broker restarts.
>
> - Martin
>

--
Jean-Baptiste Onofré
[hidden email]
http://blog.nanthrax.net
Talend - http://www.talend.com
Reply | Threaded
Open this post in threaded view
|

Re: Re: Unexpected DLQ behaviour with non-persistent messages

Martin Lichtin
Hi JB, I have to disagree with this view.

The JMS delivery mode should be honoured by ActiveMQ, it should not 'improve' or change the delivery mode in any way. There's no reason such a message should survive a broker re-start, in fact I'm pretty sure most everyone will expect to loose such a message.

In my case it is perfectly reasonable for a non-persistent message to go to DLQ, having a server-side retry configured, to pass the message to another consumer that can perhaps handle the message. There's no reason for DLQ to have 'stronger' persistence requirements.

If the current behaviour makes sense for anyone, then please at least can we make it configurable? I'm happy to help.

- Martin


On 30.01.2020 17:41, Jean-Baptiste Onofré wrote:

> Hi,
>
> If you enable DLQ to non-persistent message (it's disabled by default),
> then, IMHO, any message stored in DLQ should survive to broker restart
> (else it's not really a DLQ).
> Again, as non-persistent messages are not supposed to go in DLQ. To send
> the non-persistent messages to DLQ, you have to use for instance:
>
>   <sharedDeadLetterStrategy processNonPersistent="true" />
>
> In that case, it makes sense to "store" the message persistently in the DLQ.
>
> Regards
> JB
>
> On 30/01/2020 17:35, Martin Lichtin wrote:
>> Can somone help me understand why a non-persistent message is
>> "converted" into a persistent message on entering DLQ?
>> It happens in RegionBroker, around line 733:
>>
>>                              if (!message.isPersistent()) {
>>                                  message.setPersistent(true);
>> message.setProperty("originalDeliveryMode", "NON_PERSISTENT");
>>                              }
>>
>> I'm puzzled as to why this would make sense.
>>
>> A non-persistent message, even if configured to use DLQ, should stay
>> non-persistent, i.e. not survive broker restarts.
>>
>> - Martin
>>
Reply | Threaded
Open this post in threaded view
|

Re: Unexpected DLQ behaviour with non-persistent messages

jbonofre
Hi Martin,

Just to be sure: did you enable DLQ for non persistent message ?

I'm splitted about that: as a message in DLQ never expired, I would
expect that it would stay forever (as DLQ requires a "manual" action).
However, I see your point: as the original message is non-persistent,
what would it change in DLQ.

I think it makes sense to have this configuration (on the DLQ strategy)
allowing to match the two use cases.

Can you please create a Jira and assign it to me ? I will do the
improvement.

Regards
JB

On 03/02/2020 20:50, Martin Lichtin wrote:

> Hi JB, I have to disagree with this view.
>
> The JMS delivery mode should be honoured by ActiveMQ, it should not
> 'improve' or change the delivery mode in any way. There's no reason such
> a message should survive a broker re-start, in fact I'm pretty sure most
> everyone will expect to loose such a message.
>
> In my case it is perfectly reasonable for a non-persistent message to go
> to DLQ, having a server-side retry configured, to pass the message to
> another consumer that can perhaps handle the message. There's no reason
> for DLQ to have 'stronger' persistence requirements.
>
> If the current behaviour makes sense for anyone, then please at least
> can we make it configurable? I'm happy to help.
>
> - Martin
>
>
> On 30.01.2020 17:41, Jean-Baptiste Onofré wrote:
>> Hi,
>>
>> If you enable DLQ to non-persistent message (it's disabled by default),
>> then, IMHO, any message stored in DLQ should survive to broker restart
>> (else it's not really a DLQ).
>> Again, as non-persistent messages are not supposed to go in DLQ. To send
>> the non-persistent messages to DLQ, you have to use for instance:
>>
>>   <sharedDeadLetterStrategy processNonPersistent="true" />
>>
>> In that case, it makes sense to "store" the message persistently in
>> the DLQ.
>>
>> Regards
>> JB
>>
>> On 30/01/2020 17:35, Martin Lichtin wrote:
>>> Can somone help me understand why a non-persistent message is
>>> "converted" into a persistent message on entering DLQ?
>>> It happens in RegionBroker, around line 733:
>>>
>>>                              if (!message.isPersistent()) {
>>>                                  message.setPersistent(true);
>>> message.setProperty("originalDeliveryMode", "NON_PERSISTENT");
>>>                              }
>>>
>>> I'm puzzled as to why this would make sense.
>>>
>>> A non-persistent message, even if configured to use DLQ, should stay
>>> non-persistent, i.e. not survive broker restarts.
>>>
>>> - Martin
>>>

--
Jean-Baptiste Onofré
[hidden email]
http://blog.nanthrax.net
Talend - http://www.talend.com
Reply | Threaded
Open this post in threaded view
|

Re: Re: Unexpected DLQ behaviour with non-persistent messages

Martin Lichtin
Hi JB

Yes I've enabled DLQ to accept non-persistent messages. Noone likes to loose a message for no apparent reason, i.e. simply because a consumer cannot currently handle it :-) Not persisting a message does not mean to not care about it.

I've created AMQ-7397, glad to see we can make this behaviour configurable.

Thanks much
- Martin


On 04.02.2020 05:33, Jean-Baptiste Onofré wrote:

> Hi Martin,
>
> Just to be sure: did you enable DLQ for non persistent message ?
>
> I'm splitted about that: as a message in DLQ never expired, I would
> expect that it would stay forever (as DLQ requires a "manual" action).
> However, I see your point: as the original message is non-persistent,
> what would it change in DLQ.
>
> I think it makes sense to have this configuration (on the DLQ strategy)
> allowing to match the two use cases.
>
> Can you please create a Jira and assign it to me ? I will do the
> improvement.
>
> Regards
> JB
>
> On 03/02/2020 20:50, Martin Lichtin wrote:
>> Hi JB, I have to disagree with this view.
>>
>> The JMS delivery mode should be honoured by ActiveMQ, it should not
>> 'improve' or change the delivery mode in any way. There's no reason such
>> a message should survive a broker re-start, in fact I'm pretty sure most
>> everyone will expect to loose such a message.
>>
>> In my case it is perfectly reasonable for a non-persistent message to go
>> to DLQ, having a server-side retry configured, to pass the message to
>> another consumer that can perhaps handle the message. There's no reason
>> for DLQ to have 'stronger' persistence requirements.
>>
>> If the current behaviour makes sense for anyone, then please at least
>> can we make it configurable? I'm happy to help.
>>
>> - Martin
>>
>>
>> On 30.01.2020 17:41, Jean-Baptiste Onofré wrote:
>>> Hi,
>>>
>>> If you enable DLQ to non-persistent message (it's disabled by default),
>>> then, IMHO, any message stored in DLQ should survive to broker restart
>>> (else it's not really a DLQ).
>>> Again, as non-persistent messages are not supposed to go in DLQ. To send
>>> the non-persistent messages to DLQ, you have to use for instance:
>>>
>>>    <sharedDeadLetterStrategy processNonPersistent="true" />
>>>
>>> In that case, it makes sense to "store" the message persistently in
>>> the DLQ.
>>>
>>> Regards
>>> JB
>>>
>>> On 30/01/2020 17:35, Martin Lichtin wrote:
>>>> Can somone help me understand why a non-persistent message is
>>>> "converted" into a persistent message on entering DLQ?
>>>> It happens in RegionBroker, around line 733:
>>>>
>>>>                               if (!message.isPersistent()) {
>>>>                                   message.setPersistent(true);
>>>> message.setProperty("originalDeliveryMode", "NON_PERSISTENT");
>>>>                               }
>>>>
>>>> I'm puzzled as to why this would make sense.
>>>>
>>>> A non-persistent message, even if configured to use DLQ, should stay
>>>> non-persistent, i.e. not survive broker restarts.
>>>>
>>>> - Martin
>>>>