creating a new message once all previous messages have been executed?

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

creating a new message once all previous messages have been executed?

Kevin Burton
I’m curious if there is a good pattern for creating a new messages once all
previous tasks have been properly executed.

So say I have 100 A messages which are consumed by a bunch of tasks in your
cluster.

When all the A tasks have executed, you want to run 1 B task to compute
their results.

So it’s sort of like map-reduce in a way.

I think what I could do is use JMS transactions, and then read the value of
an incrementing pointer, when it hits 100, I can create the B task, and
then commit() the entire session.


--

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Reply | Threaded
Open this post in threaded view
|

Re: creating a new message once all previous messages have been executed?

Tim Bain
Can you use Camel to pull the messages, and insert a processor in your A
route that will just count how many A messages you've gotten, and when
(count % 100) == 0 you can create a message for your B route to tell it to
do the clean-up processing?  That would all be pretty simple to do in Camel.

You might also be able to get what you want from the Aggregator EIP (
http://camel.apache.org/aggregator2.html), though I haven't used it so I
can't say for sure that it will do what you need.

On Mon, Nov 3, 2014 at 2:40 PM, Kevin Burton <[hidden email]> wrote:

> I’m curious if there is a good pattern for creating a new messages once all
> previous tasks have been properly executed.
>
> So say I have 100 A messages which are consumed by a bunch of tasks in your
> cluster.
>
> When all the A tasks have executed, you want to run 1 B task to compute
> their results.
>
> So it’s sort of like map-reduce in a way.
>
> I think what I could do is use JMS transactions, and then read the value of
> an incrementing pointer, when it hits 100, I can create the B task, and
> then commit() the entire session.
>
>
> --
>
> Founder/CEO Spinn3r.com
> Location: *San Francisco, CA*
> blog: http://burtonator.wordpress.com
> … or check out my Google+ profile
> <https://plus.google.com/102718274791889610666/posts>
> <http://spinn3r.com>
>