How to make scheduled messages highly available?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

How to make scheduled messages highly available?

Willem van Doesburg
EAE_Logo_FC_rgb xxs

Hi,

 

We are using ActiveMQ in several of our production systems. It works very well, but we would like to make the broker fail-over, for when the broker application (which runs seperately of the consumers on other machines) crashes.

 

To do this, we are already able to create a JDBC-based or a filesystem-based high-availability store, so that our queue is always kept in the air by the two brokers. However, that is not what we want, we want the high-availability as well for all scheduled messages. We rely heavily on scheduled messages for quite some functionalities (sending e-mail, doing some tasks just before other hardware performs tasks). We noticed that the scheduled messages are not placed in our queue, but always stored together with the broker, which makes it impossible to make the broker fail-over. Currently we really need to restart that crashed broker because it holds scheduled messages, so we rely on brokers specifically instead of a queue which is managed by some restartable independent brokers.

 

How can we best resolve this?

 

Best regards,

-- Willem van Doesburg



Met vriendelijke groet, / Kind regards,

Willem van Doesburg
Information Architect, Project Leader


Eijkelkamp Agrisearch Equipment

T +31 (0)313 800 981

I www.eijkelkamp.com


Please consider the environment before printing this e-mail.

Help supply 100.000 people with safe drinking water. How? Support The Water Entrepreneur!

Reply | Threaded
Open this post in threaded view
|

Re: How to make scheduled messages highly available?

Matt Pavlovich-2
I suggest looking at replacing the broker for generating the messages with simple Camel routes using the timer or quartz endpoints.  Clustering scheduled jobs is always a challenge. If you run Camel in a Karaf-based container (ServiceMix, JBoss Fuse, etc) you can setup the containers to be highly available using the same type of shared lock as you do ActiveMQ (JDBC or file-based).  If the primary container goes down, the secondary starts up and then the Camel routes in that container would be generating the scheduled events.

-Matt

On Sep 2, 2014, at 4:37 AM, Willem van Doesburg <[hidden email]> wrote:

>
> Hi,
>  
> We are using ActiveMQ in several of our production systems. It works very well, but we would like to make the broker fail-over, for when the broker application (which runs seperately of the consumers on other machines) crashes.
>  
> To do this, we are already able to create a JDBC-based or a filesystem-based high-availability store, so that our queue is always kept in the air by the two brokers. However, that is not what we want, we want the high-availability as well for all scheduled messages. We rely heavily on scheduled messages for quite some functionalities (sending e-mail, doing some tasks just before other hardware performs tasks). We noticed that the scheduled messages are not placed in our queue, but always stored together with the broker, which makes it impossible to make the broker fail-over. Currently we really need to restart that crashed broker because it holds scheduled messages, so we rely on brokers specifically instead of a queue which is managed by some restartable independent brokers.
>  
> How can we best resolve this?
>  
> Best regards,
> -- Willem van Doesburg
>
>
> Met vriendelijke groet, / Kind regards,
>
> Willem van Doesburg
> Information Architect, Project Leader
>
>
> Eijkelkamp Agrisearch Equipment
>
> T +31 (0)313 800 981
>
> I www.eijkelkamp.com
>
>
> Please consider the environment before printing this e-mail.
>
> Help supply 100.000 people with safe drinking water. How? Support The Water Entrepreneur!
>
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: How to make scheduled messages highly available?

Claus Ibsen
Hi

I think there is a JIRA ticket about making the scheduler support
persistent JDBC store.
https://issues.apache.org/jira/browse/AMQ-5238

Use the JIRA voting system to vote for tickets you want.



On Thu, Sep 4, 2014 at 3:07 AM, Matt Pavlovich <[hidden email]> wrote:

> I suggest looking at replacing the broker for generating the messages with simple Camel routes using the timer or quartz endpoints.  Clustering scheduled jobs is always a challenge. If you run Camel in a Karaf-based container (ServiceMix, JBoss Fuse, etc) you can setup the containers to be highly available using the same type of shared lock as you do ActiveMQ (JDBC or file-based).  If the primary container goes down, the secondary starts up and then the Camel routes in that container would be generating the scheduled events.
>
> -Matt
>
> On Sep 2, 2014, at 4:37 AM, Willem van Doesburg <[hidden email]> wrote:
>
>>
>> Hi,
>>
>> We are using ActiveMQ in several of our production systems. It works very well, but we would like to make the broker fail-over, for when the broker application (which runs seperately of the consumers on other machines) crashes.
>>
>> To do this, we are already able to create a JDBC-based or a filesystem-based high-availability store, so that our queue is always kept in the air by the two brokers. However, that is not what we want, we want the high-availability as well for all scheduled messages. We rely heavily on scheduled messages for quite some functionalities (sending e-mail, doing some tasks just before other hardware performs tasks). We noticed that the scheduled messages are not placed in our queue, but always stored together with the broker, which makes it impossible to make the broker fail-over. Currently we really need to restart that crashed broker because it holds scheduled messages, so we rely on brokers specifically instead of a queue which is managed by some restartable independent brokers.
>>
>> How can we best resolve this?
>>
>> Best regards,
>> -- Willem van Doesburg
>>
>>
>> Met vriendelijke groet, / Kind regards,
>>
>> Willem van Doesburg
>> Information Architect, Project Leader
>>
>>
>> Eijkelkamp Agrisearch Equipment
>>
>> T +31 (0)313 800 981
>>
>> I www.eijkelkamp.com
>>
>>
>> Please consider the environment before printing this e-mail.
>>
>> Help supply 100.000 people with safe drinking water. How? Support The Water Entrepreneur!
>>
>>
>>
>>
>



--
Claus Ibsen
-----------------
Red Hat, Inc.
Email: [hidden email]
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
hawtio: http://hawt.io/
fabric8: http://fabric8.io/