Time based filter?

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Time based filter?

joseph.gagnon
I'm looking for strategies/techniques that could be used to implement time-based filtering of received messages. The scenario is that a message producer produces messages at a fast rate (e.g. 100 Hz) and the consumer needs to receive and handle them at a slower rate (e.g. 10 Hz). If messages are lost between reads, that is OK.  When it's time to do the next read, it gets whatever is currently available from the sender.

Thanks.
Reply | Threaded
Open this post in threaded view
|

Re: Time based filter?

Kevin Burton
you want to rate limit the clients?

On Wed, Feb 25, 2015 at 9:37 AM, joseph.gagnon <[hidden email]>
wrote:

> I'm looking for strategies/techniques that could be used to implement
> time-based filtering of received messages. The scenario is that a message
> producer produces messages at a fast rate (e.g. 100 Hz) and the consumer
> needs to receive and handle them at a slower rate (e.g. 10 Hz). If messages
> are lost between reads, that is OK.  When it's time to do the next read, it
> gets whatever is currently available from the sender.
>
> Thanks.
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Time-based-filter-tp4692048.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>



--

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Reply | Threaded
Open this post in threaded view
|

RE: Time based filter?

joseph.gagnon

Yes. I want the receiver to only have to worry about dealing with receiving messages every tenth of a second (using my example) even though the producer is sending them out much faster.  The receiver will get only every tenth message, which is fine.

 

From: Kevin Burton [via ActiveMQ] [mailto:[hidden email]]
Sent: Wednesday, February 25, 2015 3:33 PM
To: Gagnon, Joseph - 0558 - MITLL
Subject: Re: Time based filter?

 

you want to rate limit the clients?

On Wed, Feb 25, 2015 at 9:37 AM, joseph.gagnon <[hidden email]>
wrote:


> I'm looking for strategies/techniques that could be used to implement
> time-based filtering of received messages. The scenario is that a message
> producer produces messages at a fast rate (e.g. 100 Hz) and the consumer
> needs to receive and handle them at a slower rate (e.g. 10 Hz). If messages
> are lost between reads, that is OK.  When it's time to do the next read, it
> gets whatever is currently available from the sender.
>
> Thanks.
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Time-based-filter-tp4692048.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>



--

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>


If you reply to this email, your message will be added to the discussion below:

http://activemq.2283324.n4.nabble.com/Time-based-filter-tp4692048p4692058.html

To unsubscribe from Time based filter?, click here.
NAML


smime.p7s (7K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Time based filter?

Kevin Burton
Read a bit about slow consumers ..  you need to make sure you avoid issues
here.

You’re also going to have to write some throttle code.  ActiveMQ itself
doesn’t really have support for this.  There are Delay messages but that’s
really not what you want I think.

One solution is to have the client do the throttling.  Basically opening a
new consumer, receiving a message, processing it, then determining how long
to sleep for, then repeat.

On Wed, Feb 25, 2015 at 12:38 PM, joseph.gagnon <[hidden email]>
wrote:

> Yes. I want the receiver to only have to worry about dealing with
> receiving messages every tenth of a second (using my example) even though
> the producer is sending them out much faster.  The receiver will get only
> every tenth message, which is fine.
>
>
--

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Reply | Threaded
Open this post in threaded view
|

Re: Time based filter?

Tim Bain
You're not looking to throttle your clients, you're looking to have them
periodically read all messages, discard all but the most recent, and
process that last one.  That doesn't sound too hard; sleep for 100ms, then
read all messages till you find the most recent one; ack all but the last
one, then process the last one, then ack it and go back to sleep.

On Wed, Feb 25, 2015 at 2:07 PM, Kevin Burton <[hidden email]> wrote:

> Read a bit about slow consumers ..  you need to make sure you avoid issues
> here.
>
> You’re also going to have to write some throttle code.  ActiveMQ itself
> doesn’t really have support for this.  There are Delay messages but that’s
> really not what you want I think.
>
> One solution is to have the client do the throttling.  Basically opening a
> new consumer, receiving a message, processing it, then determining how long
> to sleep for, then repeat.
>
> On Wed, Feb 25, 2015 at 12:38 PM, joseph.gagnon <[hidden email]>
> wrote:
>
> > Yes. I want the receiver to only have to worry about dealing with
> > receiving messages every tenth of a second (using my example) even though
> > the producer is sending them out much faster.  The receiver will get only
> > every tenth message, which is fine.
> >
> >
> --
>
> Founder/CEO Spinn3r.com
> Location: *San Francisco, CA*
> blog: http://burtonator.wordpress.com
> … or check out my Google+ profile
> <https://plus.google.com/102718274791889610666/posts>
> <http://spinn3r.com>
>