Does MessageStore.getMessage need to be fast?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Does MessageStore.getMessage need to be fast?

Kevin Burton
So I’m torn between two implementations of a new MemoryStore using JMS
priority.

I could do it the FAST way, and use just a regular queue object, which
would include support for priorities, BUT would be slower for certain
operations like remove, get, and maybe size.

It would just use a PriorityBlockingQueue

OR

I could do it the SLOW (and somewhat ugly) way but make it more compatible
with what works now. Basically I would use a ReentrantReadWriteLock around
both a map and a queue.  The queue would be used or (obvious) queue
operations but the map would be used for getMessage and remove() … It would
also work with topics.

The FAST way above wouldn’t work well with getMessage… because I would have
to iterate over the whole queue and then compare each message.

I guess maybe the best way to go is to be conservative/pragmatic here and
move to a ReentrantReadWrite lock model and use both a map and a queue.
This would prevent any weird behavior that I don’t anticipate in exchange
for speed.  It would also mean it would work well for topics too.



--

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Reply | Threaded
Open this post in threaded view
|

Re: Does MessageStore.getMessage need to be fast?

tabish121@gmail.com
On 04/06/2015 05:05 PM, Kevin Burton wrote:

> So I’m torn between two implementations of a new MemoryStore using JMS
> priority.
>
> I could do it the FAST way, and use just a regular queue object, which
> would include support for priorities, BUT would be slower for certain
> operations like remove, get, and maybe size.
>
> It would just use a PriorityBlockingQueue
>
> OR
>
> I could do it the SLOW (and somewhat ugly) way but make it more compatible
> with what works now. Basically I would use a ReentrantReadWriteLock around
> both a map and a queue.  The queue would be used or (obvious) queue
> operations but the map would be used for getMessage and remove() … It would
> also work with topics.
>
> The FAST way above wouldn’t work well with getMessage… because I would have
> to iterate over the whole queue and then compare each message.
>
> I guess maybe the best way to go is to be conservative/pragmatic here and
> move to a ReentrantReadWrite lock model and use both a map and a queue.
> This would prevent any weird behavior that I don’t anticipate in exchange
> for speed.  It would also mean it would work well for topics too.
I'd go with the second option as you still want to management operations
to perform in a reasonable fashion even with the memory store.

--
Tim Bish
Sr Software Engineer | RedHat Inc.
[hidden email] | www.redhat.com
twitter: @tabish121
blog: http://timbish.blogspot.com/

Reply | Threaded
Open this post in threaded view
|

Re: Does MessageStore.getMessage need to be fast?

Kevin Burton
How are messages served now.. I *think* they’re served based on message ID
and sequence and served with the lowest ID first.  Which is why a map
works..but it means that I can’t re-implement this using just a queue
sorted by priorities and a comparator.

I spent about 5 minutes looking at how KahaDB implements it and it looks
like it’s just using the btree and that the cursor implements the priority
support.




On Mon, Apr 6, 2015 at 2:38 PM, Timothy Bish <[hidden email]> wrote:

> On 04/06/2015 05:05 PM, Kevin Burton wrote:
> > So I’m torn between two implementations of a new MemoryStore using JMS
> > priority.
> >
> > I could do it the FAST way, and use just a regular queue object, which
> > would include support for priorities, BUT would be slower for certain
> > operations like remove, get, and maybe size.
> >
> > It would just use a PriorityBlockingQueue
> >
> > OR
> >
> > I could do it the SLOW (and somewhat ugly) way but make it more
> compatible
> > with what works now. Basically I would use a ReentrantReadWriteLock
> around
> > both a map and a queue.  The queue would be used or (obvious) queue
> > operations but the map would be used for getMessage and remove() … It
> would
> > also work with topics.
> >
> > The FAST way above wouldn’t work well with getMessage… because I would
> have
> > to iterate over the whole queue and then compare each message.
> >
> > I guess maybe the best way to go is to be conservative/pragmatic here and
> > move to a ReentrantReadWrite lock model and use both a map and a queue.
> > This would prevent any weird behavior that I don’t anticipate in exchange
> > for speed.  It would also mean it would work well for topics too.
> I'd go with the second option as you still want to management operations
> to perform in a reasonable fashion even with the memory store.
>
> --
> Tim Bish
> Sr Software Engineer | RedHat Inc.
> [hidden email] | www.redhat.com
> twitter: @tabish121
> blog: http://timbish.blogspot.com/
>
>


--

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Reply | Threaded
Open this post in threaded view
|

Re: Does MessageStore.getMessage need to be fast?

gtully
peek at org.apache.activemq.broker.region.cursors.VMPendingMessageCursor
and the use of PrioritizedPendingList


On 6 April 2015 at 22:57, Kevin Burton <[hidden email]> wrote:

> How are messages served now.. I *think* they’re served based on message ID
> and sequence and served with the lowest ID first.  Which is why a map
> works..but it means that I can’t re-implement this using just a queue
> sorted by priorities and a comparator.
>
> I spent about 5 minutes looking at how KahaDB implements it and it looks
> like it’s just using the btree and that the cursor implements the priority
> support.
>
>
>
>
> On Mon, Apr 6, 2015 at 2:38 PM, Timothy Bish <[hidden email]> wrote:
>
>> On 04/06/2015 05:05 PM, Kevin Burton wrote:
>> > So I’m torn between two implementations of a new MemoryStore using JMS
>> > priority.
>> >
>> > I could do it the FAST way, and use just a regular queue object, which
>> > would include support for priorities, BUT would be slower for certain
>> > operations like remove, get, and maybe size.
>> >
>> > It would just use a PriorityBlockingQueue
>> >
>> > OR
>> >
>> > I could do it the SLOW (and somewhat ugly) way but make it more
>> compatible
>> > with what works now. Basically I would use a ReentrantReadWriteLock
>> around
>> > both a map and a queue.  The queue would be used or (obvious) queue
>> > operations but the map would be used for getMessage and remove() … It
>> would
>> > also work with topics.
>> >
>> > The FAST way above wouldn’t work well with getMessage… because I would
>> have
>> > to iterate over the whole queue and then compare each message.
>> >
>> > I guess maybe the best way to go is to be conservative/pragmatic here and
>> > move to a ReentrantReadWrite lock model and use both a map and a queue.
>> > This would prevent any weird behavior that I don’t anticipate in exchange
>> > for speed.  It would also mean it would work well for topics too.
>> I'd go with the second option as you still want to management operations
>> to perform in a reasonable fashion even with the memory store.
>>
>> --
>> Tim Bish
>> Sr Software Engineer | RedHat Inc.
>> [hidden email] | www.redhat.com
>> twitter: @tabish121
>> blog: http://timbish.blogspot.com/
>>
>>
>
>
> --
>
> Founder/CEO Spinn3r.com
> Location: *San Francisco, CA*
> blog: http://burtonator.wordpress.com
> … or check out my Google+ profile
> <https://plus.google.com/102718274791889610666/posts>
> <http://spinn3r.com>