activeMQ topics via rest - missing messages

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

activeMQ topics via rest - missing messages

rrazey82
I'm trying to set up a framework for our AMQ testing and running into a roadblock.  Say I make a POST to http://localhost:8161/api/message/TESTTHISTOPIC?type=Topic&clientId=testClient with body=foo1

Then I make a DELETE (or GET, neither work) to the same url with no body param.  The call returns a 204 (no content).  So I did some more digging.  It appears, if I do a couple of POST's in a row, so that I have 4 items in the topic for example.  

body = foo1
body = foo2
body = foo3
body = foo4

So then I do 4 DELETE/GET's... to attempt to return the messages I just sent.  Here are the results

foo2
foo3
foo4
HTTP 204 (empty)

In other words, the foo1 message can't be pulled via REST.  It appears the message is actually there in the admin console but I can't access it.  Has anyone had any experience with this and/or can give some guidance?
Reply | Threaded
Open this post in threaded view
|

Re: activeMQ topics via rest - missing messages

artnaseef
First off, let me warn that this isn't a "recipe" that I would use;
actually any REST interface for JMS is a nice feature for prototyping, but
not something I would use for production-quality code.  Add Topics to that
mix, and it's really concerning.

With that said, your problem may be easy to explain.  In this test, is the
first GET performed after the first POST?  If so, the first message is not
available for the GET because there is no consumer registered on the
Topic, and Topics do not store messages for consumers that are neither
Durable nor Active (i.e. connected and consuming).

Try doing a GET first, then POST the first message.

The reason that will work is that the broker is maintaining an internal
consumer on behalf of the REST client.  That consumer is only created
on-demand.


>
>
> I'm trying to set up a framework for our AMQ testing and running into a
> roadblock.  Say I make a POST to
> http://localhost:8161/api/message/TESTTHISTOPIC?type=Topic&clientId=testClient
> with body=foo1
>
> Then I make a DELETE (or GET, neither work) to the same url with no body
> param.  The call returns a 204 (no content).  So I did some more digging.
> It appears, if I do a couple of POST's in a row, so that I have 4 items in
> the topic for example.
>
> body = foo1
> body = foo2
> body = foo3
> body = foo4
>
> So then I do 4 DELETE/GET's... to attempt to return the messages I just
> sent.  Here are the results
>
> foo2
> foo3
> foo4
> HTTP 204 (empty)
>
> In other words, the foo1 message can't be pulled via REST.  It appears the
> message is actually there in the admin console but I can't access it.  Has
> anyone had any experience with this and/or can give some guidance?
>
>
>
> _______________________________________________
> If you reply to this email, your message will be added to the discussion
> below:
> http://activemq.2283324.n4.nabble.com/activeMQ-topics-via-rest-missing-messages-tp4677698.html
> To start a new topic under ActiveMQ - User, email
> [hidden email]
> To unsubscribe from ActiveMQ - User, visit
>
Reply | Threaded
Open this post in threaded view
|

RE: activeMQ topics via rest - missing messages

Steve.V.
When I read "http://activemq.apache.org/rest.html", I don't see anything saying the REST/HTTP interface should not be used in production code.  Please elaborate, send me, or direct me to a list of your concerns.  I understand that Java clients would want to use the JMS API, but what is so wrong with C clients using POST and GET over HTTP?

Thanks,
Stephen Vincent

________________________________________
>From: artnaseef [[hidden email]]
>Sent: Monday, February 10, 2014 5:13 PM
>
>First off, let me warn that this isn't a "recipe" that I would use;
>actually any REST interface for JMS is a nice feature for prototyping, but
>not something I would use for production-quality code.  Add Topics to that
>mix, and it's really concerning...
...
snip
...
Stephen Vincent
Reply | Threaded
Open this post in threaded view
|

RE: activeMQ topics via rest - missing messages

artnaseef
Allow me to clarify.  First, perhaps my statement is a little stronger than it should be - I wouldn't use it, but I can see it being used.

Here's my reasoning:

* REST and JMS protocols are notably different; REST is one-message-per-connection while JMS is many-messages-per-connnection
* JMS subscriptions are dropped when the connection is dropped.  For Topics, this means missed messages.  For queues, this means less efficiency (for both client and broker) and less fairness of message delivery if both JMS and REST consumers are used.
* REST can lead to dropped messages; if the message is consumed by the activemq internal consumer and then the REST connection drops before the client receives that message, it will be lost.  So, JMS message delivery guarantees are weakened.
* The REST implementation in the broker caches a consumer.  Any messages prefetched to that consumer will remain stuck indefinitely until a REST client requests it.  If a REST client uses the built-in session management, and the session that creates the internal consumer is removed, that internal consumer will be abandoned, leading to stuck messages.
* With Topics, regardless of the consumer prefetch, messages will be stored in the broker's memory for the subscription.  If the REST client is lost or away for a long time, those messages could starve the broker for memory.

There may be more issues.  From my perspective, it would be best to create a real REST-based web service that uses the message bus than to use a pure REST/JMS adapter so that handling of things like possible lost messages can be defined by the web service (e.g. using CLIENT_ACKNOWLEDGE and only acknowledging the message after completing the REST-based request; perhaps caching responses in case the same request is duplicated).
Reply | Threaded
Open this post in threaded view
|

RE: activeMQ topics via rest - missing messages

pfreed
I am curious about your recommendation not use use REST in production, and your comment "REST can lead to dropped messages."  ActiveMQ has seen a lot of growth in the last few years; I'm wondering if the situation has changed since then.

Here's some background about my use case:
I represent a vendor that has a working REST interface with our client.  It's not high volume; we have thousands of messages a day -- most are a smaller than 1 KB.

Since their software is Camel-based, I recommended that they communicate with our software via ActiveMQ (for improved reliability).  They have advised me that REST won't work (or won't work well) for this, and that we need to add JMS capabilities to our software so that I can subscribe directly to their queues.

I have a basic understanding of MQ, though I have not used it.  But it seems to me that ActiveMQ is a lot less flexible than I had thought if this is indeed the case.

I'd appreciate any thoughts you have on this.  Is there some way to reliably use ActiveMQ via REST?

Thanks much.