[activemq-user] Message Selectors on a durable topic subscription

Previous Topic Next Topic
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view

[activemq-user] Message Selectors on a durable topic subscription

Graham Uphill-2

with thanks to all the help I have received so far on this list, I have integrated ActiveMQ (3.1 M6) into our own product. However, I do have an issue with how durable subscriptions on a topic seem to work with message selectors.

I currently have 2 durable subscriptions. One that receives all messages, and a second that only is interested in a very small subset, which is controlled by specifying a message selector when the subscription is made. As expected, the second subscription is only sent the messages that match the message selector filter.

However, because the messages it receives are a very small subset of all the messages sent within the system, the ACTIVEMQ_MSGS table very rarely gets cleared down. Because the second subscription doesn't actually acknowledge the messages (as it doesn't receive them), the cleardown task within ActiveMQ keeps the messages around, even though the other agent has acknowledged them. (It seems to work by only clearing messages that all subscriptions have explicitly acknowledged, no matter what message selector they have). This means the ACTIVEMQ_MSGS table is filling up quite considerably, as it only gets cleared down when a message arrives at the second subscription and is acknowledged. This sets the subscriptions last_acked_id to the message id it has received, allowing the clear down task to delete all the other messages. I suppose ideally here we would be hoping that messages that are not required by subscriptions would be included by the cleardown task as long as they had been acknowledged by those subscriptions that were interested in them, or maybe subscriptions are automatically set to have acknowledged messages they are not going to receive.

Secondly, and even worse for us, is subscription recovery. If ActiveMQ is stopped and restarted, as part of the subscription recovery, all these messages are loaded and then sent to the second subscription. The recovery operation doesn't seem to take into account the message selector. So as our system starts up, we have a long period where conceivably thousands of messages are sent to a subscription which has no interest in receiving them (and has explicitly set it's message selector so that it wont).

As a workaround for this, we have taken out the message selector, and put something into our own code which checks the message and ignores those that we are not interested in. This means that as far as ActiveMQ is concerned, both subscriptions receive and acknowledge every message, so they are cleared down and not resent as part of subscription recovery. This of course uses up more CPU time, and defeats the purpose of the message selector.

Has anyone any ideas about this? Is this a bug, or have I misunderstood something about subscription recovery?

Any feedback would be appreciated. Thanks again,

Reply | Threaded
Open this post in threaded view

Re: [activemq-user] Message Selectors on a durable topic subscription

On Tuesday 16 August 2005 06:07 am, Graham Uphill wrote:
> ...
> Has anyone any ideas about this? Is this a bug, or have I misunderstood
> something about subscription recovery?

I'm interested in this as well, since we are planning on using durable
subscribers soon.  It would seem to me that when ActiveMQ first attempts
delivery and a subscriber's selector "rejects" a message, that ActiveMQ
should mark that message acknowledged by that subscriber?  If this were true,
it would probably solve both issues (just guessing).  (Note, I know nothing
about how ActiveMQ handles durable subscribers internally, so I'm making a
lot of assumptions here).

  - Andy