Message redelivery when producer broker killed (without persistence)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

Message redelivery when producer broker killed (without persistence)

pypen
Hi,
I am trying to solve a problem and I am wondering if Virtual Topics provide
a solution for the problem.
I have 2 brokers in a network of brokers.
Broker A has a producer to VirtualTopic.Test and a local consumer
Consumer.A.VirtualTopic.Test (lets call it consumer A).
Broker B has a local consumer Consumer.A.VirtualTopic.Test (lets call it
consumer B).
(All clients [producers and consumers] are connected to the local brokers
only)

Broker A and Broker B are both running, consumer A is running, consumer B is
not running.

I want to be able to kill consumer A and broker A entirely (pull the plug),
start consumer B and redeliver/process all messages that were not
acknowledged by consumer A on consumer B.

What I thought would happen (a very naive assumption) is that when a message
is produced on the VirtualTopic.Test topic, both brokers will receive the
message and when it is acknowledged by consumer A consumer B will not
receive it (basically that the topics are treated as separate topics, but
the queues as one). But the way it seems is that the message is handed from
broker A to consumer A, if consumer A does not acknowledge the message is
handed back to the broker A, then handled over to broker B and then to
consumer B. The problem is that when broker A is killed, that message is
lost.

I tried also to use a statically included destination (VirtualTopic.Test),
which seems to send copies of the messages to consumer A and consumer B so
that once consumer B starts, it will receive all messages (that did not
expire), even the ones that were acknowledged already by consumer A. (when
both consumers are running all messages are delivered to both consumers)

So, am I just missing something or are virtual topics not a solution?





--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: Message redelivery when producer broker killed (without persistence)

jbertram
> So, am I just missing something or are virtual topics not a solution?

I don't think virtual topics are a solution to your problem.


Justin

On Thu, Oct 12, 2017 at 6:08 PM, pypen <[hidden email]> wrote:

> Hi,
> I am trying to solve a problem and I am wondering if Virtual Topics provide
> a solution for the problem.
> I have 2 brokers in a network of brokers.
> Broker A has a producer to VirtualTopic.Test and a local consumer
> Consumer.A.VirtualTopic.Test (lets call it consumer A).
> Broker B has a local consumer Consumer.A.VirtualTopic.Test (lets call it
> consumer B).
> (All clients [producers and consumers] are connected to the local brokers
> only)
>
> Broker A and Broker B are both running, consumer A is running, consumer B
> is
> not running.
>
> I want to be able to kill consumer A and broker A entirely (pull the plug),
> start consumer B and redeliver/process all messages that were not
> acknowledged by consumer A on consumer B.
>
> What I thought would happen (a very naive assumption) is that when a
> message
> is produced on the VirtualTopic.Test topic, both brokers will receive the
> message and when it is acknowledged by consumer A consumer B will not
> receive it (basically that the topics are treated as separate topics, but
> the queues as one). But the way it seems is that the message is handed from
> broker A to consumer A, if consumer A does not acknowledge the message is
> handed back to the broker A, then handled over to broker B and then to
> consumer B. The problem is that when broker A is killed, that message is
> lost.
>
> I tried also to use a statically included destination (VirtualTopic.Test),
> which seems to send copies of the messages to consumer A and consumer B so
> that once consumer B starts, it will receive all messages (that did not
> expire), even the ones that were acknowledged already by consumer A. (when
> both consumers are running all messages are delivered to both consumers)
>
> So, am I just missing something or are virtual topics not a solution?
>
>
>
>
>
> --
> Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-
> f2341805.html
>
Reply | Threaded
Open this post in threaded view
|

Re: Message redelivery when producer broker killed (without persistence)

Tim Bain
I agree.

The fundamental problem you're hitting is that in ActiveMQ 5.x, a message
can exist in only one active broker, so networks of brokers move messages
around via store-and-forward, as you described. For a message to be
available to multiple brokers at the same time (such that it would still be
available if a broker went down) you would need active-active clustering,
which ActiveMQ 5.x doesn't support.

However, ActiveMQ Artemis does support that feature, so you might want to
consider switching to it to better support your use case.

Tim

On Oct 12, 2017 8:01 PM, "Justin Bertram" <[hidden email]> wrote:

> > So, am I just missing something or are virtual topics not a solution?
>
> I don't think virtual topics are a solution to your problem.
>
>
> Justin
>
> On Thu, Oct 12, 2017 at 6:08 PM, pypen <[hidden email]> wrote:
>
> > Hi,
> > I am trying to solve a problem and I am wondering if Virtual Topics
> provide
> > a solution for the problem.
> > I have 2 brokers in a network of brokers.
> > Broker A has a producer to VirtualTopic.Test and a local consumer
> > Consumer.A.VirtualTopic.Test (lets call it consumer A).
> > Broker B has a local consumer Consumer.A.VirtualTopic.Test (lets call it
> > consumer B).
> > (All clients [producers and consumers] are connected to the local brokers
> > only)
> >
> > Broker A and Broker B are both running, consumer A is running, consumer B
> > is
> > not running.
> >
> > I want to be able to kill consumer A and broker A entirely (pull the
> plug),
> > start consumer B and redeliver/process all messages that were not
> > acknowledged by consumer A on consumer B.
> >
> > What I thought would happen (a very naive assumption) is that when a
> > message
> > is produced on the VirtualTopic.Test topic, both brokers will receive the
> > message and when it is acknowledged by consumer A consumer B will not
> > receive it (basically that the topics are treated as separate topics, but
> > the queues as one). But the way it seems is that the message is handed
> from
> > broker A to consumer A, if consumer A does not acknowledge the message is
> > handed back to the broker A, then handled over to broker B and then to
> > consumer B. The problem is that when broker A is killed, that message is
> > lost.
> >
> > I tried also to use a statically included destination
> (VirtualTopic.Test),
> > which seems to send copies of the messages to consumer A and consumer B
> so
> > that once consumer B starts, it will receive all messages (that did not
> > expire), even the ones that were acknowledged already by consumer A.
> (when
> > both consumers are running all messages are delivered to both consumers)
> >
> > So, am I just missing something or are virtual topics not a solution?
> >
> >
> >
> >
> >
> > --
> > Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-
> > f2341805.html
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Message redelivery when producer broker killed (without persistence)

jbertram
Artemis does support this in a way via colocated lives and backups.
However, even then this is only valid for *persisted* messages.  The title
of the original email indicates this is needed for *non-persistent*
messages.  At least that's the way I understood it.


Justin

On Fri, Oct 13, 2017 at 7:44 AM, Tim Bain <[hidden email]> wrote:

> I agree.
>
> The fundamental problem you're hitting is that in ActiveMQ 5.x, a message
> can exist in only one active broker, so networks of brokers move messages
> around via store-and-forward, as you described. For a message to be
> available to multiple brokers at the same time (such that it would still be
> available if a broker went down) you would need active-active clustering,
> which ActiveMQ 5.x doesn't support.
>
> However, ActiveMQ Artemis does support that feature, so you might want to
> consider switching to it to better support your use case.
>
> Tim
>
> On Oct 12, 2017 8:01 PM, "Justin Bertram" <[hidden email]> wrote:
>
> > > So, am I just missing something or are virtual topics not a solution?
> >
> > I don't think virtual topics are a solution to your problem.
> >
> >
> > Justin
> >
> > On Thu, Oct 12, 2017 at 6:08 PM, pypen <[hidden email]> wrote:
> >
> > > Hi,
> > > I am trying to solve a problem and I am wondering if Virtual Topics
> > provide
> > > a solution for the problem.
> > > I have 2 brokers in a network of brokers.
> > > Broker A has a producer to VirtualTopic.Test and a local consumer
> > > Consumer.A.VirtualTopic.Test (lets call it consumer A).
> > > Broker B has a local consumer Consumer.A.VirtualTopic.Test (lets call
> it
> > > consumer B).
> > > (All clients [producers and consumers] are connected to the local
> brokers
> > > only)
> > >
> > > Broker A and Broker B are both running, consumer A is running,
> consumer B
> > > is
> > > not running.
> > >
> > > I want to be able to kill consumer A and broker A entirely (pull the
> > plug),
> > > start consumer B and redeliver/process all messages that were not
> > > acknowledged by consumer A on consumer B.
> > >
> > > What I thought would happen (a very naive assumption) is that when a
> > > message
> > > is produced on the VirtualTopic.Test topic, both brokers will receive
> the
> > > message and when it is acknowledged by consumer A consumer B will not
> > > receive it (basically that the topics are treated as separate topics,
> but
> > > the queues as one). But the way it seems is that the message is handed
> > from
> > > broker A to consumer A, if consumer A does not acknowledge the message
> is
> > > handed back to the broker A, then handled over to broker B and then to
> > > consumer B. The problem is that when broker A is killed, that message
> is
> > > lost.
> > >
> > > I tried also to use a statically included destination
> > (VirtualTopic.Test),
> > > which seems to send copies of the messages to consumer A and consumer B
> > so
> > > that once consumer B starts, it will receive all messages (that did not
> > > expire), even the ones that were acknowledged already by consumer A.
> > (when
> > > both consumers are running all messages are delivered to both
> consumers)
> > >
> > > So, am I just missing something or are virtual topics not a solution?
> > >
> > >
> > >
> > >
> > >
> > > --
> > > Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-
> > > f2341805.html
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Message redelivery when producer broker killed (without persistence)

pypen
Thank you both for your answers.
Yes, Justin, that's what I meant (non-persistent messaging).

I guess, I will have to look into different solutions.
Is it for example possible to force messages to be always sent to a local
consumer, if one exists, otherwise to a remote consumer (if one exists and
no local consumer is running)? And if the local consumer starts again to
send all messages to the local consumer? It looks like the JMSXGroupID seems
to work for the case that the local consumer receives the first message and
all following messages will be delivered to that same consumer, until this
consumer is stopped. The problem is that once another (remote) consumer is
"pinned", the local consumer will not receive messages until that remote
consumer is stopped.



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: Message redelivery when producer broker killed (without persistence)

jbertram
> Is it for example possible to force messages to be always sent to a local
consumer, if one exists, otherwise to a remote consumer (if one exists and
no local consumer is running)? And if the local consumer starts again to
send all messages to the local consumer?

I'm not familiar with the intricacies of the 5.x broker in this use-case,
but from what I recall Artemis does all the things you want here (and
there's no need to use JMSXGroupID).


Justin

On Fri, Oct 13, 2017 at 2:32 PM, pypen <[hidden email]> wrote:

> Thank you both for your answers.
> Yes, Justin, that's what I meant (non-persistent messaging).
>
> I guess, I will have to look into different solutions.
> Is it for example possible to force messages to be always sent to a local
> consumer, if one exists, otherwise to a remote consumer (if one exists and
> no local consumer is running)? And if the local consumer starts again to
> send all messages to the local consumer? It looks like the JMSXGroupID
> seems
> to work for the case that the local consumer receives the first message and
> all following messages will be delivered to that same consumer, until this
> consumer is stopped. The problem is that once another (remote) consumer is
> "pinned", the local consumer will not receive messages until that remote
> consumer is stopped.
>
>
>
> --
> Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-
> f2341805.html
>
Reply | Threaded
Open this post in threaded view
|

Re: Message redelivery when producer broker killed (without persistence)

clebertsuconic
In reply to this post by pypen
Persistent messages in artemis are pretty fast. You should consider
persistent.

The fact you need them reliable means you need them durable.

On Fri, Oct 13, 2017 at 3:32 PM pypen <[hidden email]> wrote:

> Thank you both for your answers.
> Yes, Justin, that's what I meant (non-persistent messaging).
>
> I guess, I will have to look into different solutions.
> Is it for example possible to force messages to be always sent to a local
> consumer, if one exists, otherwise to a remote consumer (if one exists and
> no local consumer is running)? And if the local consumer starts again to
> send all messages to the local consumer? It looks like the JMSXGroupID
> seems
> to work for the case that the local consumer receives the first message and
> all following messages will be delivered to that same consumer, until this
> consumer is stopped. The problem is that once another (remote) consumer is
> "pinned", the local consumer will not receive messages until that remote
> consumer is stopped.
>
>
>
> --
> Sent from:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
>
--
Clebert Suconic
Reply | Threaded
Open this post in threaded view
|

Re: Message redelivery when producer broker killed (without persistence)

pypen
Thanks guys.
I am a little hesitant since I heard that artemis is not really prime-time
ready yet.
But I might give it a shot.



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: Message redelivery when producer broker killed (without persistence)

jbertram
I don't think of Artemis as "not really prime-time ready yet."  The Artemis
code-base was based on the HornetQ message broker which has served
commercially as the JMS implementation in Red Hat's JBoss Enterprise
Application Platform (EAP) 6.x since 2012.  Artemis itself has served in
the same capacity in EAP 7.x since May of last year.  Artemis is also the
message broker at the heart of Red Hat's JBoss AMQ 7.x product released in
May of this year.  All three of these are used in production situations of
all kinds where performance and stability are critical.  That is, of
course, not to mention all the users who have deployed into production
using the bits straight from Apache without commercial support.

I'm curious.  Where did you hear that Artemis wasn't really ready for prime
time?


Justin

On Tue, Oct 17, 2017 at 3:36 PM, pypen <[hidden email]> wrote:

> Thanks guys.
> I am a little hesitant since I heard that artemis is not really prime-time
> ready yet.
> But I might give it a shot.
>
>
>
> --
> Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-
> f2341805.html
>
Reply | Threaded
Open this post in threaded view
|

Re: Message redelivery when producer broker killed (without persistence)

Tim Bain
In reply to this post by pypen
ActiveMQ 5.x allows messages to be delivered to local consumers over remote
ones via the decreaseNetworkConsumerPriority property as described at
http://activemq.apache.org/networks-of-brokers.html. JMSXGroupID will not
achieve this goal for the reasons you described.

Also, I dispute your claim that what you want is non-persistent messaging.
Non-persistent messages are ones that you don't mind losing in the event of
a broker restart, where durability does not matter. In this email thread,
you're looking for a way to not lose your messages when one broker fails,
which means you're not looking for non-persistent messages. You want
persistent messages, you just didn't know it.

Tim



On Fri, Oct 13, 2017 at 1:32 PM, pypen <[hidden email]> wrote:

> Thank you both for your answers.
> Yes, Justin, that's what I meant (non-persistent messaging).
>
> I guess, I will have to look into different solutions.
> Is it for example possible to force messages to be always sent to a local
> consumer, if one exists, otherwise to a remote consumer (if one exists and
> no local consumer is running)? And if the local consumer starts again to
> send all messages to the local consumer? It looks like the JMSXGroupID
> seems
> to work for the case that the local consumer receives the first message and
> all following messages will be delivered to that same consumer, until this
> consumer is stopped. The problem is that once another (remote) consumer is
> "pinned", the local consumer will not receive messages until that remote
> consumer is stopped.
>
>
>
> --
> Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-
> f2341805.html
>
Reply | Threaded
Open this post in threaded view
|

Re: Message redelivery when producer broker killed (without persistence)

pypen
:)
You are right. And you are wrong.
I DON'T want persistent messaging via KhahaDB (or jdbc). I don't want to
introduce a database, a NAS or shared folders (I'm on windows).
I DO want in memory behavior similar to it, where two brokers share their
messages (in memory) so that if both brokers die, messages can be lost.






--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: Message redelivery when producer broker killed (without persistence)

jbertram
> I DON'T want persistent messaging via KhahaDB (or jdbc). I don't want to
introduce a database, a NAS or shared folders (I'm on windows).

You could move to Artemis and use the network replication functionality
which doesn't use a database, NAS, etc.  The only concern here is with
split brain which is hard to mitigate with just 2 nodes.


> I DO want in memory behavior similar to it, where two brokers share their
messages (in memory) so that if both brokers die, messages can be lost.

In-memory, non-durable messages are, by definition, volatile which means
even with Artemis' network replication those messages won't be replicated.
However, the client and broker can be tuned so that you can achieve nearly
the same performance with durable messages (assuming that's your concern).
See more about that here [1].


Justin

P.S. You didn't answer my previous question about "prime-time" Artemis

[1] https://blogs.apache.org/activemq/entry/fast_messaging_with_artemis


On Mon, Oct 23, 2017 at 11:06 AM, pypen <[hidden email]> wrote:

> :)
> You are right. And you are wrong.
> I DON'T want persistent messaging via KhahaDB (or jdbc). I don't want to
> introduce a database, a NAS or shared folders (I'm on windows).
> I DO want in memory behavior similar to it, where two brokers share their
> messages (in memory) so that if both brokers die, messages can be lost.
>
>
>
>
>
>
> --
> Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-
> f2341805.html
>
Reply | Threaded
Open this post in threaded view
|

Re: Message redelivery when producer broker killed (without persistence)

pypen
We actually use more than 2 nodes, but still an even amount.
About the prime-time ready: A consultant told me that some functionality was
missing.

Maybe I will look into Artemis.



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: Message redelivery when producer broker killed (without persistence)

Justin Bertram
Can you elaborate on what functionality was identified as missing?  We're
always looking to improve things so if there's a legitimate need/gap it's
worth exploring.


Justin

On Mon, Oct 23, 2017 at 1:00 PM, pypen <[hidden email]> wrote:

> We actually use more than 2 nodes, but still an even amount.
> About the prime-time ready: A consultant told me that some functionality
> was
> missing.
>
> Maybe I will look into Artemis.
>
>
>
> --
> Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-
> f2341805.html
>