ActiveMQ broker discards messages for topic durable subscriber if they were sent to topic while client was offline due to unexpecteable lost connection

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

ActiveMQ broker discards messages for topic durable subscriber if they were sent to topic while client was offline due to unexpecteable lost connection

HappyMan
Hello, everyone.

This is my first post in internet. First I mean really first for all the
time :)
So let's try to explain. We use broker activeMQ version 5.15.4 and MQTT
3.1.1 client (mobile application).
I adjusted broker and wrote MQTT client. I have only one problem now with
durable subscriptions.
MQTT client make successfull connect to Topic (let it be ABC) with specified
parameters and subscribe. So durable subscription created. I see this
durable subscription in ActiveMQ brkoer in Active durable subscriptions.
When I manually disconnect active durable subscription moves to offline
durable subscriptiion.
After that I send messages to this Topic ABC and they keeps enqueued. When I
connect I use subscribe method and Offline durable subscription becomes
Active durable subscription and I get all these messages from Topic ABC.
But if I connect to Topic ABC and not disconnect manually and connection
lost (mobile internet turned off or wi-fi turned off, no matter) this
subscription stays Active durable subscription (but in fact client is
offline).  And if there were sent messages to this Topic ABC while client
was offline due to connection problem, all messages are dequeued
automatically but nowhere. So logically, when client becomes offline he can
not inform broker that he is offline, but is it real or maybe there
something on broker side, that could tell broker not to dequeue message. I
mean maybe broker can send some kind of ping request to client before
dequeueing this message and if no ping response move this active durable
subscription to offline durable subscription.

For now due to this problem we have to connect, send all messages, that were
manually stored before and recieve all messages in subscription topic and
then disconnect to make it offline, so in that case messages won't be lost
(of course if for this period connect-disconnect there was not unexpectable
disconnect).

Sorry for my English and if there were same posts already. Could you please
help me if it can be solved.



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: ActiveMQ broker discards messages for topic durable subscriber if they were sent to topic while client was offline due to unexpecteable lost connection

HappyMan
By the way forgot to wright. When it happens (unexpectable lost connection)
broker keeps this connection alive and logically it is ok because client did
not send any request for disconnect. And probably this subscription keeps
alive because broker still has infromation that client is connected.



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: ActiveMQ broker discards messages for topic durable subscriber if they were sent to topic while client was offline due to unexpecteable lost connection

Justin Reock-3
Hi HappyMan,

So, architecturally speaking, this is really something that should be handled at the client/application layer using your exception handling and try/catch/finally logic.  Your code should never be allowed to crash without releasing the connection, even with a hard kill of the process.  Be sure then that your client is doing adequate cleanup first.  

However, things like firewalls, etc, at the TCP layer can interfere here as well, so, once you are sure your client is solid, if you are still having the same issue then try turning off TCP keepalive by adding the parameter transport.useKeepAlive=false to the transport connector for MQTT in your broker.xml file, as described here:  https://activemq.apache.org/tcp-transport-reference

That should force the connection to disconnect after no data on the line for a specified amount of time.  Your clients should be using the failover: prefix to ensure that they attempt to reconnect to make sure that any falsely disconnected clients are able to come back up.  

Hope that helps!!

-Justin

On 10/21/19, 1:38 AM, "HappyMan" <[hidden email]> wrote:

    By the way forgot to wright. When it happens (unexpectable lost connection)
    broker keeps this connection alive and logically it is ok because client did
    not send any request for disconnect. And probably this subscription keeps
    alive because broker still has infromation that client is connected.
   
   
   
    --
    Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
   
   
    CAUTION: This email originated from outside of the organization. Do not click on links or open attachments unless you recognize the sender and know the content is safe.
   

Reply | Threaded
Open this post in threaded view
|

Re: ActiveMQ broker discards messages for topic durable subscriber if they were sent to topic while client was offline due to unexpecteable lost connection

HappyMan
This post was updated on .
Hi Justin,

Thank you very much for your reply. It helps me to think in a right way (I
guess).
Not only I adjusted ActiveMQ broker. I checked broker file ActiveMQ.xml
again and find out that for mqtt transport connector was set parameter
"transport.useInactivityMonitor=false". It means that checking if connection
active or not was disabled...
Tommorow I wil turn it on and I will add keepAlive parameter for MQTT client
and I will set it 10 seconds. I hope it will work. Probably I think broker
will close connection in 10 seconds when pingreq/pingresp won't be passed
due to mobile internet will be off and connection on MQTT client will be
lost. In that case I really hope that Active durable subscription will
become Offline durable subscription.

All in all, thank you, will try tommorow.



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: ActiveMQ broker discards messages for topic durable subscriber if they were sent to topic while client was offline due to unexpecteable lost connection

Justin Reock-3
Hi HappyMan,

You're welcome!

That's good to know -- I figured the InactivityMonitor was enabled, good to know that it was disabled.  Enabling it will help I'm sure, you might need to tweak some of the timeout values, etc, but, good luck and keep us posted!

-Justin

On 10/21/19, 3:30 PM, "HappyMan" <[hidden email]> wrote:

    Hi Justin,
   
    Thank you ver much for your reply. It helps me to think in a right way (I
    guess).
    Not only I adjusted ActiveMQ broker. I checked broker file ActiveMQ.xml
    again and find out that for mqtt transport connector was set parameter
    "transport.useInactivityMonitor=false". It means that checking if connection
    active or not was disabled...
    Tommorow I wil turn it on and I will add keepAlive parameter for MQTT client
    and I will set it 10 seconds. I hope it will work. Probably I think broker
    will close connection in 10 seconds when pingreq/pingresp won't be passed
    due to mobile internet will be off and connection on MQTT client will be
    lost. In that case I really hope that Active durable subscription will
    become Offline durable subscription.
   
    All in all, thank you, will try tommorow.
   
   
   
    --
    Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
   
   
    CAUTION: This email originated from outside of the organization. Do not click on links or open attachments unless you recognize the sender and know the content is safe.
   

Reply | Threaded
Open this post in threaded view
|

Re: ActiveMQ broker discards messages for topic durable subscriber if they were sent to topic while client was offline due to unexpecteable lost connection

HappyMan
This post was updated on .
Hi, Justin.

I turned on InactivityMonitor and now everything works great. I set
keepAlive interval in MQTT client for 10 seconds and
defaultKeepAliveInterval on MQTT transport connectot in ActiveMQ.xml 60
seconds (if somehow keepAliveInterval from mqtt client will be 0, probably
it won't but for sure). If mqtt client unexpectedly disconnect and there is
no ping req/resp for 10 seconds connection closed on server side and Active
durable subscription becomes Offline and there are no more connection leaks and
message loss is very very low (only if message will be sent to topic
subscriber between unexpected disconnect and close connection on server
side, but quantity of incoming messages is very low and such probability is
very low).

Thank you for moving me in a right way thinking :)



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: ActiveMQ broker discards messages for topic durable subscriber if they were sent to topic while client was offline due to unexpecteable lost connection

Justin Reock-3
Nice work, HappyMan -- glad to help!

-Justin

On 10/22/19, 2:38 AM, "HappyMan" <[hidden email]> wrote:

    Hi, Justin.
   
    I turned on InactivityMonitor and now everything works great. I set
    keepAlive interval in MQTT client for 10 seconds and
    defaultKeepAliveInterval on MQTT transport connectot in ActiveMQ.xml 60
    seconds (if somehow keepAliveInterval from mqtt client will be 0, probably
    it won't but for sure). If mqtt client unexpectedly disconnect and there is
    no ping req/resp for 10 seconds connection closed on server side and Active
    durable subscription becomes Offline and there are connection leaks and
    message loss is very very low (only if message will be sent to topic
    subscriber between unexpected disconnect and close connection on server
    side, but quantity of incoming messages is very low and such probability is
    very low).
   
    Thank you for moving me in a right way thinking :)
   
   
   
    --
    Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
   
   
    CAUTION: This email originated from outside of the organization. Do not click on links or open attachments unless you recognize the sender and know the content is safe.