ActiveMQ HA on failover topic mode

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

ActiveMQ HA on failover topic mode

PedroRP
Hi!

Does ActiveMQ cluster (master-slave) offer high availability on failover
mode?

I am publishing messages on a topic but when I kill active node and my
consumer connect to the other node It loses several messages.

I need only topic mode, because I need to have different pods of the same
openshift service to consume the messages at the same time.

If ActiveMQ doesn't support this functionality, is there other broker that
could do this?

Thanks in advance!



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: ActiveMQ HA on failover topic mode

jbertram
ActiveMQ 5.x does support high availability via a master-slave
configuration. See the documentation [1] for more details.


Justin

[1] http://activemq.apache.org/masterslave.html

On Thu, Nov 29, 2018 at 11:09 AM PedroRP <[hidden email]>
wrote:

> Hi!
>
> Does ActiveMQ cluster (master-slave) offer high availability on failover
> mode?
>
> I am publishing messages on a topic but when I kill active node and my
> consumer connect to the other node It loses several messages.
>
> I need only topic mode, because I need to have different pods of the same
> openshift service to consume the messages at the same time.
>
> If ActiveMQ doesn't support this functionality, is there other broker that
> could do this?
>
> Thanks in advance!
>
>
>
> --
> Sent from:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
>
Reply | Threaded
Open this post in threaded view
|

Re: ActiveMQ HA on failover topic mode

PedroRP
We have used Spring boot to develop sender and consumer apps.

*SENDER:*

To send messages we used and API REST that sends 5000 messages to a topic.
If some convertAndSend fails (ActiveMQ node 1 killed) we retry it until the
failover.

This is my sender class:

@RestController
public class RestApiController {

    @Autowired
    private JmsTemplate jmsTemplate;

    @RequestMapping(value = "/produce")
    public String produce() {
        String result = "Done";
        for (int i = 0; i < 5000; i++) {
            boolean repetir = true;
            while (repetir) {
                try {
                    jmsTemplate.convertAndSend("TestTopic", "Message " + i);
                    repetir = false;
                    result = "Done";
                } catch (JmsException e) {
                    e.printStackTrace();
                    result = "ERROR";
                }
            }
        }
        return result;
    }
}

This is de sender's application.properties:

spring.activemq.broker-url=failover:(tcp://172.18.13.45:61616,tcp://172.18.13.45:61626)?initialReconnectDelay=1&backup=true
spring.activemq.user=admin
spring.activemq.password=admin
spring.jms.pub-sub-domain=true
server.port=8081

CONSUMER:

This is my listener class:

@Service
public class ContactTransactionReceiver {

    @JmsListener(destination = "TestTopic")
    public void receiveMessageSendMessage(Message message) throws Exception
{
        System.out.println(((TextMessage) message).getText());
    }
}

This is the consumer's application.properties:

spring.activemq.broker-url=failover:(tcp://172.18.13.45:61616,tcp://172.18.13.45:61626)?initialReconnectDelay=1&backup=true
spring.activemq.user=admin
spring.activemq.password=admin
spring.jms.pub-sub-domain=true
server.port=8082

*ACTIVEMQ NODE 1*

We have included this configuration in activemq.xml for HA, that refers to
node2:

    <persistenceAdapter>
        <kahaDB directory="${activemq.data}/kahadb"/>
    </persistenceAdapter>
    <networkConnectors>
        <networkConnector
uri="static:(tcp://172.18.13.45:61616,tcp://172.18.13.45:61626)" />
    </networkConnectors>
We have proved too master-slave:

    <persistenceAdapter>
        <kahaDB directory="${activemq.data}/kahadb"/>
    </persistenceAdapter>
    <networkConnectors>
        <networkConnector
uri="masterslave:(tcp://172.18.13.45:61616,tcp://172.18.13.45:61626)" />
    </networkConnectors>

*ACTIVEMQ NODE 2*

We have included this configuration in activemq.xml for HA, that refers to
node2:

    <persistenceAdapter>
        <kahaDB directory="${activemq.data}/kahadb"/>
    </persistenceAdapter>
    <networkConnectors>
        <networkConnector
uri="static:(tcp://172.18.13.45:61626,tcp://172.18.13.45:61616)" />
    </networkConnectors>

We have proved too master-slave:

    <persistenceAdapter>
        <kahaDB directory="${activemq.data}/kahadb"/>
    </persistenceAdapter>
    <networkConnectors>
        <networkConnector
uri="masterslave:(tcp://172.18.13.45:61626,tcp://172.18.13.45:61616)" />
    </networkConnectors>

We have made test with durable messages and subscriptions wit we got the
same problem. Moreover, we also proved including networkTTL="2"
messageTTL="2" consumerTTL="2" and alwaysSyncSend="true" in both
networkConnectors but the same result.

You can find the full code and ActiveMQ configuration files in:

https://github.com/PedroRamirezTOR/ActiveMQ-HA-Sender-Consumer.git
<https://github.com/PedroRamirezTOR/ActiveMQ-HA-Sender-Consumer.git>  

Thanks in advance!



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: ActiveMQ HA on failover topic mode

Tim Bain
You're mixing your paradigms.

The persistenceAdapter portion of your config is all you need for a
master/slave cluster, *as long as* your activemq.data variable resolves to
the same value in both processes. (Both processes run on the same machine,
right?) If so, the second broker started will pause in its initialization
to wait for the filesystem lock to become available, and will not accept
incoming connections from clients on its OpenWire ports. If the second
broker accepts incoming connections, that's the problem, and you need to
figure out why.

All the other broker settings you referenced (networkConnectors, TTL
settings, the difference between static and masterslave transports, etc.)
are related to networks of brokers, and should not be in use for a
master/slave cluster (unless you're networking the cluster with other
brokers, but you haven't described that), so take them out.

For the message loss you're seeing, are you losing a few messages from
right before the failover, or are you losing *all* of the messages from
before the failover, or something else? If it's the former, you probably
need to set alwaysSyncSend=true on your producer's connection factory. I'm
not very familiar with JmsTemplate, so I'm not sure if you can set up a
custom connection factory when using it, but if not then you can use the
JMS API directly.

Tim

On Fri, Nov 30, 2018, 6:30 AM PedroRP <[hidden email] wrote:

> We have used Spring boot to develop sender and consumer apps.
>
> *SENDER:*
>
> To send messages we used and API REST that sends 5000 messages to a topic.
> If some convertAndSend fails (ActiveMQ node 1 killed) we retry it until the
> failover.
>
> This is my sender class:
>
> @RestController
> public class RestApiController {
>
>     @Autowired
>     private JmsTemplate jmsTemplate;
>
>     @RequestMapping(value = "/produce")
>     public String produce() {
>         String result = "Done";
>         for (int i = 0; i < 5000; i++) {
>             boolean repetir = true;
>             while (repetir) {
>                 try {
>                     jmsTemplate.convertAndSend("TestTopic", "Message " +
> i);
>                     repetir = false;
>                     result = "Done";
>                 } catch (JmsException e) {
>                     e.printStackTrace();
>                     result = "ERROR";
>                 }
>             }
>         }
>         return result;
>     }
> }
>
> This is de sender's application.properties:
>
> spring.activemq.broker-url=failover:(tcp://172.18.13.45:61616,tcp://
> 172.18.13.45:61626)?initialReconnectDelay=1&backup=true
> spring.activemq.user=admin
> spring.activemq.password=admin
> spring.jms.pub-sub-domain=true
> server.port=8081
>
> CONSUMER:
>
> This is my listener class:
>
> @Service
> public class ContactTransactionReceiver {
>
>     @JmsListener(destination = "TestTopic")
>     public void receiveMessageSendMessage(Message message) throws Exception
> {
>         System.out.println(((TextMessage) message).getText());
>     }
> }
>
> This is the consumer's application.properties:
>
> spring.activemq.broker-url=failover:(tcp://172.18.13.45:61616,tcp://
> 172.18.13.45:61626)?initialReconnectDelay=1&backup=true
> spring.activemq.user=admin
> spring.activemq.password=admin
> spring.jms.pub-sub-domain=true
> server.port=8082
>
> *ACTIVEMQ NODE 1*
>
> We have included this configuration in activemq.xml for HA, that refers to
> node2:
>
>     <persistenceAdapter>
>         <kahaDB directory="${activemq.data}/kahadb"/>
>     </persistenceAdapter>
>     <networkConnectors>
>         <networkConnector
> uri="static:(tcp://172.18.13.45:61616,tcp://172.18.13.45:61626)" />
>     </networkConnectors>
> We have proved too master-slave:
>
>     <persistenceAdapter>
>         <kahaDB directory="${activemq.data}/kahadb"/>
>     </persistenceAdapter>
>     <networkConnectors>
>         <networkConnector
> uri="masterslave:(tcp://172.18.13.45:61616,tcp://172.18.13.45:61626)" />
>     </networkConnectors>
>
> *ACTIVEMQ NODE 2*
>
> We have included this configuration in activemq.xml for HA, that refers to
> node2:
>
>     <persistenceAdapter>
>         <kahaDB directory="${activemq.data}/kahadb"/>
>     </persistenceAdapter>
>     <networkConnectors>
>         <networkConnector
> uri="static:(tcp://172.18.13.45:61626,tcp://172.18.13.45:61616)" />
>     </networkConnectors>
>
> We have proved too master-slave:
>
>     <persistenceAdapter>
>         <kahaDB directory="${activemq.data}/kahadb"/>
>     </persistenceAdapter>
>     <networkConnectors>
>         <networkConnector
> uri="masterslave:(tcp://172.18.13.45:61626,tcp://172.18.13.45:61616)" />
>     </networkConnectors>
>
> We have made test with durable messages and subscriptions wit we got the
> same problem. Moreover, we also proved including networkTTL="2"
> messageTTL="2" consumerTTL="2" and alwaysSyncSend="true" in both
> networkConnectors but the same result.
>
> You can find the full code and ActiveMQ configuration files in:
>
> https://github.com/PedroRamirezTOR/ActiveMQ-HA-Sender-Consumer.git
> <https://github.com/PedroRamirezTOR/ActiveMQ-HA-Sender-Consumer.git>
>
> Thanks in advance!
>
>
>
> --
> Sent from:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
>
Reply | Threaded
Open this post in threaded view
|

Re: ActiveMQ HA on failover topic mode

Tim Bain
Actually, you should be able to set the alwaysSyncSend via the URI by
adding &jms.alwaysSyncSend=true to the end of your producer's URI, so you
should be able to continue using JmsTemplate.

Tim

On Fri, Nov 30, 2018, 7:30 AM Tim Bain <[hidden email] wrote:

> You're mixing your paradigms.
>
> The persistenceAdapter portion of your config is all you need for a
> master/slave cluster, *as long as* your activemq.data variable resolves to
> the same value in both processes. (Both processes run on the same machine,
> right?) If so, the second broker started will pause in its initialization
> to wait for the filesystem lock to become available, and will not accept
> incoming connections from clients on its OpenWire ports. If the second
> broker accepts incoming connections, that's the problem, and you need to
> figure out why.
>
> All the other broker settings you referenced (networkConnectors, TTL
> settings, the difference between static and masterslave transports, etc.)
> are related to networks of brokers, and should not be in use for a
> master/slave cluster (unless you're networking the cluster with other
> brokers, but you haven't described that), so take them out.
>
> For the message loss you're seeing, are you losing a few messages from
> right before the failover, or are you losing *all* of the messages from
> before the failover, or something else? If it's the former, you probably
> need to set alwaysSyncSend=true on your producer's connection factory. I'm
> not very familiar with JmsTemplate, so I'm not sure if you can set up a
> custom connection factory when using it, but if not then you can use the
> JMS API directly.
>
> Tim
>
> On Fri, Nov 30, 2018, 6:30 AM PedroRP <[hidden email]
> wrote:
>
>> We have used Spring boot to develop sender and consumer apps.
>>
>> *SENDER:*
>>
>> To send messages we used and API REST that sends 5000 messages to a topic.
>> If some convertAndSend fails (ActiveMQ node 1 killed) we retry it until
>> the
>> failover.
>>
>> This is my sender class:
>>
>> @RestController
>> public class RestApiController {
>>
>>     @Autowired
>>     private JmsTemplate jmsTemplate;
>>
>>     @RequestMapping(value = "/produce")
>>     public String produce() {
>>         String result = "Done";
>>         for (int i = 0; i < 5000; i++) {
>>             boolean repetir = true;
>>             while (repetir) {
>>                 try {
>>                     jmsTemplate.convertAndSend("TestTopic", "Message " +
>> i);
>>                     repetir = false;
>>                     result = "Done";
>>                 } catch (JmsException e) {
>>                     e.printStackTrace();
>>                     result = "ERROR";
>>                 }
>>             }
>>         }
>>         return result;
>>     }
>> }
>>
>> This is de sender's application.properties:
>>
>> spring.activemq.broker-url=failover:(tcp://172.18.13.45:61616,tcp://
>> 172.18.13.45:61626)?initialReconnectDelay=1&backup=true
>> spring.activemq.user=admin
>> spring.activemq.password=admin
>> spring.jms.pub-sub-domain=true
>> server.port=8081
>>
>> CONSUMER:
>>
>> This is my listener class:
>>
>> @Service
>> public class ContactTransactionReceiver {
>>
>>     @JmsListener(destination = "TestTopic")
>>     public void receiveMessageSendMessage(Message message) throws
>> Exception
>> {
>>         System.out.println(((TextMessage) message).getText());
>>     }
>> }
>>
>> This is the consumer's application.properties:
>>
>> spring.activemq.broker-url=failover:(tcp://172.18.13.45:61616,tcp://
>> 172.18.13.45:61626)?initialReconnectDelay=1&backup=true
>> spring.activemq.user=admin
>> spring.activemq.password=admin
>> spring.jms.pub-sub-domain=true
>> server.port=8082
>>
>> *ACTIVEMQ NODE 1*
>>
>> We have included this configuration in activemq.xml for HA, that refers to
>> node2:
>>
>>     <persistenceAdapter>
>>         <kahaDB directory="${activemq.data}/kahadb"/>
>>     </persistenceAdapter>
>>     <networkConnectors>
>>         <networkConnector
>> uri="static:(tcp://172.18.13.45:61616,tcp://172.18.13.45:61626)" />
>>     </networkConnectors>
>> We have proved too master-slave:
>>
>>     <persistenceAdapter>
>>         <kahaDB directory="${activemq.data}/kahadb"/>
>>     </persistenceAdapter>
>>     <networkConnectors>
>>         <networkConnector
>> uri="masterslave:(tcp://172.18.13.45:61616,tcp://172.18.13.45:61626)" />
>>     </networkConnectors>
>>
>> *ACTIVEMQ NODE 2*
>>
>> We have included this configuration in activemq.xml for HA, that refers to
>> node2:
>>
>>     <persistenceAdapter>
>>         <kahaDB directory="${activemq.data}/kahadb"/>
>>     </persistenceAdapter>
>>     <networkConnectors>
>>         <networkConnector
>> uri="static:(tcp://172.18.13.45:61626,tcp://172.18.13.45:61616)" />
>>     </networkConnectors>
>>
>> We have proved too master-slave:
>>
>>     <persistenceAdapter>
>>         <kahaDB directory="${activemq.data}/kahadb"/>
>>     </persistenceAdapter>
>>     <networkConnectors>
>>         <networkConnector
>> uri="masterslave:(tcp://172.18.13.45:61626,tcp://172.18.13.45:61616)" />
>>     </networkConnectors>
>>
>> We have made test with durable messages and subscriptions wit we got the
>> same problem. Moreover, we also proved including networkTTL="2"
>> messageTTL="2" consumerTTL="2" and alwaysSyncSend="true" in both
>> networkConnectors but the same result.
>>
>> You can find the full code and ActiveMQ configuration files in:
>>
>> https://github.com/PedroRamirezTOR/ActiveMQ-HA-Sender-Consumer.git
>> <https://github.com/PedroRamirezTOR/ActiveMQ-HA-Sender-Consumer.git>
>>
>> Thanks in advance!
>>
>>
>>
>> --
>> Sent from:
>> http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: ActiveMQ HA on failover topic mode

PedroRP
Hi Tim!

Ok, so for a HA ActiveMQ master-slave is not required any network of
brokers, isn't it?
For the messages we are losing. We see that all messages are sent but when
failover happens consumer stops listening messages. Does It makes sense with
a wrong configuration?

Thanks for your answer!



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: ActiveMQ HA on failover topic mode

Tim Bain
You would use masterslave for connections to a master/slave cluster from
another broker. So if broker C was connecting to the master/slave pair A/B,
then C would use a masterslave networkConnector to A/B. But A and B would
have no networkConnector to each other, and no masterslave
networkConnectors.

Your description of the consumer behavior sounds like the consumer isn't
failing over, rather than that the messages are being lost. Can you try to
confirm that theory via a JMX viewer such as JConsole?

Tim

On Fri, Nov 30, 2018, 9:35 AM PedroRP <[hidden email] wrote:

> Hi Tim!
>
> Ok, so for a HA ActiveMQ master-slave is not required any network of
> brokers, isn't it?
> For the messages we are losing. We see that all messages are sent but when
> failover happens consumer stops listening messages. Does It makes sense
> with
> a wrong configuration?
>
> Thanks for your answer!
>
>
>
> --
> Sent from:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
>
Reply | Threaded
Open this post in threaded view
|

Re: ActiveMQ HA on failover topic mode

PedroRP
Hi Tim!

Sorry about the late answer. I fixed the configuration and now I have a
master/slave that work fine. But I still lose messages. I checked the
consumer connection and It is equal than the producer. Moreover when I
disconnect master broker consumer and producer reconnect to the slave
broker. To make the failover I only introduced in url of connection factory
failover:(tcp://broker1:61616,tcp://broker2:61616)
Is it necessary any additional configuratio?
For more information, I tried inserting 10000 messages in a topic.

Thanks in advance!



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: ActiveMQ HA on failover topic mode

Tim Bain
Can you please characterize the messages that are lost? Do you lose the
last few messages sent before the failover, or one here and there, or...?

Also, are you killing the broker gracefully (sending it a stop command) or
ungracefully (kill -9, etc.)? Might the messages that are lost simply be
the result of asynchronously published messages that haven't yet hit the
disk when you ungracefully kill the broker process? Have you tried using
synchronous sends?

Tim

On Wed, Dec 5, 2018, 10:31 AM PedroRP <[hidden email] wrote:

> Hi Tim!
>
> Sorry about the late answer. I fixed the configuration and now I have a
> master/slave that work fine. But I still lose messages. I checked the
> consumer connection and It is equal than the producer. Moreover when I
> disconnect master broker consumer and producer reconnect to the slave
> broker. To make the failover I only introduced in url of connection factory
> failover:(tcp://broker1:61616,tcp://broker2:61616)
> Is it necessary any additional configuratio?
> For more information, I tried inserting 10000 messages in a topic.
>
> Thanks in advance!
>
>
>
> --
> Sent from:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
>
Reply | Threaded
Open this post in threaded view
|

Re: ActiveMQ HA on failover topic mode

PedroRP
Hi Tim!

The lost messages are last few messages before the failover.

I'm trying to simulate energy failure, so I kill the broker ungracefully
(kill -9).

Due to my requirements, I need to use async sends. Could this be a problem?

I realized that If I reduce timeout to 1 ms no messages are lost. Does that
make sense? But I would like to have more timeout.

Thanks!



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: ActiveMQ HA on failover topic mode

PedroRP
In reply to this post by Tim Bain
Hi Tim!

My sender app is sending text messages with the text: "Message (sequence)".
The sender's log is showing the message after send it. This is the log:

Message 6291
Message 6292
Message 6293
Message 6294
Message 6295
2018-12-10 10:37:43.175  WARN 8732 --- [.45:61626@55241]
o.a.a.t.failover.FailoverTransport       : Transport
(tcp://172.18.13.45:61626) failed , attempting to automatically reconnect:
{}

java.io.EOFException: null
        at java.io.DataInputStream.readInt(DataInputStream.java:392)
~[na:1.8.0_181]
        at
org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:268)
~[activemq-all-5.15.0.jar:5.15.0]
        at
org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:240)
~[activemq-all-5.15.0.jar:5.15.0]
        at
org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:232)
~[activemq-all-5.15.0.jar:5.15.0]
        at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:215)
~[activemq-all-5.15.0.jar:5.15.0]
        at java.lang.Thread.run(Thread.java:748) [na:1.8.0_181]

2018-12-10 10:37:50.567  INFO 8732 --- [ActiveMQ Task-1]
o.a.a.t.failover.FailoverTransport       : Successfully reconnected to
tcp://172.18.13.45:61616
Message 6296
Message 6297
Message 6298
Message 6299

The consumer's log is the following:

TopicMessageConsumer Received message [ Message 6292 ]
TopicMessageConsumer Received message [ Message 6293 ]
TopicMessageConsumer Received message [ Message 6294 ]
TopicMessageConsumer Received message [ Message 6295 ]
2018-12-10 10:37:43.175  WARN 6360 --- [.45:61626@55240]
o.a.a.t.failover.FailoverTransport       : Transport
(tcp://172.18.13.45:61626) failed , attempting to automatically reconnect:
{}

java.io.EOFException: null
        at java.io.DataInputStream.readInt(DataInputStream.java:392)
~[na:1.8.0_181]
        at
org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:268)
~[activemq-all-5.15.0.jar:5.15.0]
        at
org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:240)
~[activemq-all-5.15.0.jar:5.15.0]
        at
org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:232)
~[activemq-all-5.15.0.jar:5.15.0]
        at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:215)
~[activemq-all-5.15.0.jar:5.15.0]
        at java.lang.Thread.run(Thread.java:748) [na:1.8.0_181]

2018-12-10 10:37:43.175  WARN 6360 --- [.45:61626@55242]
o.a.a.t.failover.FailoverTransport       : Transport
(tcp://172.18.13.45:61626) failed , attempting to automatically reconnect:
{}

java.io.EOFException: null
        at java.io.DataInputStream.readInt(DataInputStream.java:392)
~[na:1.8.0_181]
        at
org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:268)
~[activemq-all-5.15.0.jar:5.15.0]
        at
org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:240)
~[activemq-all-5.15.0.jar:5.15.0]
        at
org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:232)
~[activemq-all-5.15.0.jar:5.15.0]
        at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:215)
~[activemq-all-5.15.0.jar:5.15.0]
        at java.lang.Thread.run(Thread.java:748) [na:1.8.0_181]

2018-12-10 10:37:50.533  INFO 6360 --- [ActiveMQ Task-6]
o.a.a.t.failover.FailoverTransport       : Successfully reconnected to
tcp://172.18.13.45:61616
2018-12-10 10:37:51.586  INFO 6360 --- [ActiveMQ Task-6]
o.a.a.t.failover.FailoverTransport       : Successfully reconnected to
tcp://172.18.13.45:61616
TopicMessageConsumer Received message [ Message 7290 ]
TopicMessageConsumer Received message [ Message 7291 ]
TopicMessageConsumer Received message [ Message 7292 ]
TopicMessageConsumer Received message [ Message 7293 ]

I'm killing the broker ungracefully (kill -9).

I test the app with async and sync sending. Both cases with the same result.

The above logs are using the following url to connect to the brokers:

failover:(tcp://broker1:61616,tcp://broker2:61626)?jms.useAsyncSend=false&trackMessages=true&timeout=1

Thanks!



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: ActiveMQ HA on failover topic mode

Tim Bain
You're using a non-durable subscription on your topic, aren't you? That's
your (primary) problem: you're losing messages around the time of the
failover because your consumer isn't subscribed during that period of time.

It's possible that the broker is actually losing a few messages, especially
if you're using async sends as your primary configuration, but you're
losing 1000 messages because you're using a configuration where that's the
expected behavior. Once you get that fixed, if you're still seeing message
loss, we can try to help dig deeper into it.

Also, you referenced reducing a timeout to 1 ms. Which timeout was that?

Tim

On Mon, Dec 10, 2018, 2:51 AM PedroRP <[hidden email] wrote:

> Hi Tim!
>
> My sender app is sending text messages with the text: "Message (sequence)".
> The sender's log is showing the message after send it. This is the log:
>
> Message 6291
> Message 6292
> Message 6293
> Message 6294
> Message 6295
> 2018-12-10 10:37:43.175  WARN 8732 --- [.45:61626@55241]
> o.a.a.t.failover.FailoverTransport       : Transport
> (tcp://172.18.13.45:61626) failed , attempting to automatically reconnect:
> {}
>
> java.io.EOFException: null
>         at java.io.DataInputStream.readInt(DataInputStream.java:392)
> ~[na:1.8.0_181]
>         at
>
> org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:268)
> ~[activemq-all-5.15.0.jar:5.15.0]
>         at
>
> org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:240)
> ~[activemq-all-5.15.0.jar:5.15.0]
>         at
> org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:232)
> ~[activemq-all-5.15.0.jar:5.15.0]
>         at
> org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:215)
> ~[activemq-all-5.15.0.jar:5.15.0]
>         at java.lang.Thread.run(Thread.java:748) [na:1.8.0_181]
>
> 2018-12-10 10:37:50.567  INFO 8732 --- [ActiveMQ Task-1]
> o.a.a.t.failover.FailoverTransport       : Successfully reconnected to
> tcp://172.18.13.45:61616
> Message 6296
> Message 6297
> Message 6298
> Message 6299
>
> The consumer's log is the following:
>
> TopicMessageConsumer Received message [ Message 6292 ]
> TopicMessageConsumer Received message [ Message 6293 ]
> TopicMessageConsumer Received message [ Message 6294 ]
> TopicMessageConsumer Received message [ Message 6295 ]
> 2018-12-10 10:37:43.175  WARN 6360 --- [.45:61626@55240]
> o.a.a.t.failover.FailoverTransport       : Transport
> (tcp://172.18.13.45:61626) failed , attempting to automatically reconnect:
> {}
>
> java.io.EOFException: null
>         at java.io.DataInputStream.readInt(DataInputStream.java:392)
> ~[na:1.8.0_181]
>         at
>
> org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:268)
> ~[activemq-all-5.15.0.jar:5.15.0]
>         at
>
> org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:240)
> ~[activemq-all-5.15.0.jar:5.15.0]
>         at
> org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:232)
> ~[activemq-all-5.15.0.jar:5.15.0]
>         at
> org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:215)
> ~[activemq-all-5.15.0.jar:5.15.0]
>         at java.lang.Thread.run(Thread.java:748) [na:1.8.0_181]
>
> 2018-12-10 10:37:43.175  WARN 6360 --- [.45:61626@55242]
> o.a.a.t.failover.FailoverTransport       : Transport
> (tcp://172.18.13.45:61626) failed , attempting to automatically reconnect:
> {}
>
> java.io.EOFException: null
>         at java.io.DataInputStream.readInt(DataInputStream.java:392)
> ~[na:1.8.0_181]
>         at
>
> org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:268)
> ~[activemq-all-5.15.0.jar:5.15.0]
>         at
>
> org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:240)
> ~[activemq-all-5.15.0.jar:5.15.0]
>         at
> org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:232)
> ~[activemq-all-5.15.0.jar:5.15.0]
>         at
> org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:215)
> ~[activemq-all-5.15.0.jar:5.15.0]
>         at java.lang.Thread.run(Thread.java:748) [na:1.8.0_181]
>
> 2018-12-10 10:37:50.533  INFO 6360 --- [ActiveMQ Task-6]
> o.a.a.t.failover.FailoverTransport       : Successfully reconnected to
> tcp://172.18.13.45:61616
> 2018-12-10 10:37:51.586  INFO 6360 --- [ActiveMQ Task-6]
> o.a.a.t.failover.FailoverTransport       : Successfully reconnected to
> tcp://172.18.13.45:61616
> TopicMessageConsumer Received message [ Message 7290 ]
> TopicMessageConsumer Received message [ Message 7291 ]
> TopicMessageConsumer Received message [ Message 7292 ]
> TopicMessageConsumer Received message [ Message 7293 ]
>
> I'm killing the broker ungracefully (kill -9).
>
> I test the app with async and sync sending. Both cases with the same
> result.
>
> The above logs are using the following url to connect to the brokers:
>
>
> failover:(tcp://broker1:61616,tcp://broker2:61626)?jms.useAsyncSend=false&trackMessages=true&timeout=1
>
> Thanks!
>
>
>
> --
> Sent from:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
>
Reply | Threaded
Open this post in threaded view
|

Re: ActiveMQ HA on failover topic mode

PedroRP
Hi Tim!

Thank you very much for your answer!

Yes, I was using non-durable subscription on my topic. When I changed to
durable subscription, It didn't lose messages if I use sync sends. The main
problem now is the sending response time. It spends too much time sending
messages. Is there any way to accelerate it without async sends?

Thanks in advance.



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html