500ooo+ queues?

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

500ooo+ queues?

Igor Katkov
Hi,

Is it a good idea to create 500ooo+ queues and send messages to these destinations? Load is a few messages per _minute_ into each queue.
Producers and consumers will be on different hosts. Network of brokers topology.

Does anyone have experience pushing ActiveMQ that far, or even further with 2M queues and several messages a second?

Will it scale to that extent? I can add boxes to host additional producers and consumers, but not sure if pushing metadata of that mighty number of queues around will be handled OK
Reply | Threaded
Open this post in threaded view
|

Re: 500ooo+ queues?

gtully
With that many destinations, a single JVM will have a lot of context
switching to do. It may be best to partition your destinations across a
cluster.
Having said that, to scale up the number of destinations,  memory will be
your main requirement. Follow the advice from
http://activemq.apache.org/javalangoutofmemory.html , and use the KahaDB
store to reduce fd usage. How many concurrent connections do you expect?
Scaling connections will require the use of NIO.
There is an activemq-scaleability.xml configuration file in 5.3.0 that can
provide a good starting point configuration.

2009/11/18 Igor Katkov <[hidden email]>

>
> Hi,
>
> Is it a good idea to create 500ooo+ queues and send messages to these
> destinations? Load is a few messages per _minute_ into each queue.
> Producers and consumers will be on different hosts. Network of brokers
> topology.
>
> Does anyone have experience pushing ActiveMQ that far, or even further with
> 2M queues and several messages a second?
>
> Will it scale to that extent? I can add boxes to host additional producers
> and consumers, but not sure if pushing metadata of that mighty number of
> queues around will be handled OK
> --
> View this message in context:
> http://old.nabble.com/500ooo%2B-queues--tp26417356p26417356.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>


--
http://blog.garytully.com

Open Source Integration
http://fusesource.com
Reply | Threaded
Open this post in threaded view
|

Re: 500ooo+ queues?

Igor Katkov
Thank you, this is the kind of insight I was looking for.

Gary Tully wrote
It may be best to partition your destinations across a cluster.
How do you mean it?
What I have in mind is a network of brokers, X producers generate messages on one end of the network, Y consumers process them on the other end. ActiveMQ takes care of storing messages in a different queues and delivering them to appropriate consumers.

Reply | Threaded
Open this post in threaded view
|

Re: 500ooo+ queues?

ammulder
In reply to this post by Igor Katkov
It was my experience that ActiveMQ does not handle that well
(destinations are not particularly "lightweight") and you'd do better
to have a smaller number of destinations with selectors.  RAM is not
the main/only issue -- file handles and startup time also caused
problems.

Thanks,
    Aaron

On Wed, Nov 18, 2009 at 5:58 PM, Igor Katkov <[hidden email]> wrote:

>
> Hi,
>
> Is it a good idea to create 500ooo+ queues and send messages to these
> destinations? Load is a few messages per _minute_ into each queue.
> Producers and consumers will be on different hosts. Network of brokers
> topology.
>
> Does anyone have experience pushing ActiveMQ that far, or even further with
> 2M queues and several messages a second?
>
> Will it scale to that extent? I can add boxes to host additional producers
> and consumers, but not sure if pushing metadata of that mighty number of
> queues around will be handled OK
> --
> View this message in context: http://old.nabble.com/500ooo%2B-queues--tp26417356p26417356.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>
Reply | Threaded
Open this post in threaded view
|

Re: 500ooo+ queues?

gtully
In reply to this post by Igor Katkov
w.r.t to partitioning, I imagine a scheme where destinations 1-X exist on
broker cluster A and X-N on broker cluster B. The client needs to randomly
choose a destination or use some lookup/location mechanism.
But as per Aaron's advice, do consider selectors as a way to reduce the
destination count.

Also, be aware that there is a cost in dynamic network creation, all the
advisories tend to slow things down if consumers come and go with any
frequency. You may want to streamline routes though your network so that
every node does not need to know about every destination. Filters will help
here.

Let us know how you get on.

2009/11/20 Igor Katkov <[hidden email]>

>
> Thank you, this is the kind of insight I was looking for.
>
>
> Gary Tully wrote:
> >
> > It may be best to partition your destinations across a cluster.
> >
>
> How do you mean it?
> What I have in mind is a network of brokers, X producers generate messages
> on one end of the network, Y consumers process them on the other end.
> ActiveMQ takes care of storing messages in a different queues and
> delivering
> them to appropriate consumers.
>
>
> --
> View this message in context:
> http://old.nabble.com/500ooo%2B-queues--tp26417356p26438475.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>


--
http://blog.garytully.com

Open Source Integration
http://fusesource.com
Reply | Threaded
Open this post in threaded view
|

Re: 500ooo+ queues?

rajdavies
In reply to this post by ammulder
Hi Aaron,

we've spent a lot time addressing file handles/startup time - ActiveMQ  
scales well in 5.3

cheers,

Rob
On 20 Nov 2009, at 11:40, Aaron Mulder wrote:

> It was my experience that ActiveMQ does not handle that well
> (destinations are not particularly "lightweight") and you'd do better
> to have a smaller number of destinations with selectors.  RAM is not
> the main/only issue -- file handles and startup time also caused
> problems.
>
> Thanks,
>    Aaron
>
> On Wed, Nov 18, 2009 at 5:58 PM, Igor Katkov <[hidden email]>  
> wrote:
>>
>> Hi,
>>
>> Is it a good idea to create 500ooo+ queues and send messages to these
>> destinations? Load is a few messages per _minute_ into each queue.
>> Producers and consumers will be on different hosts. Network of  
>> brokers
>> topology.
>>
>> Does anyone have experience pushing ActiveMQ that far, or even  
>> further with
>> 2M queues and several messages a second?
>>
>> Will it scale to that extent? I can add boxes to host additional  
>> producers
>> and consumers, but not sure if pushing metadata of that mighty  
>> number of
>> queues around will be handled OK
>> --
>> View this message in context: http://old.nabble.com/500ooo%2B-queues--tp26417356p26417356.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>>

Rob Davies
http://twitter.com/rajdavies
I work here: http://fusesource.com
My Blog: http://rajdavies.blogspot.com/
I'm writing this: http://www.manning.com/snyder/





Reply | Threaded
Open this post in threaded view
|

Re: 500ooo+ queues?

ammulder
OK, I stand (well, sit) corrected.  :)

Thanks,
      Aaron

On Sat, Nov 21, 2009 at 1:34 AM, Rob Davies <[hidden email]> wrote:

> Hi Aaron,
>
> we've spent a lot time addressing file handles/startup time - ActiveMQ
> scales well in 5.3
>
> cheers,
>
> Rob
> On 20 Nov 2009, at 11:40, Aaron Mulder wrote:
>
>> It was my experience that ActiveMQ does not handle that well
>> (destinations are not particularly "lightweight") and you'd do better
>> to have a smaller number of destinations with selectors.  RAM is not
>> the main/only issue -- file handles and startup time also caused
>> problems.
>>
>> Thanks,
>>   Aaron
>>
>> On Wed, Nov 18, 2009 at 5:58 PM, Igor Katkov <[hidden email]> wrote:
>>>
>>> Hi,
>>>
>>> Is it a good idea to create 500ooo+ queues and send messages to these
>>> destinations? Load is a few messages per _minute_ into each queue.
>>> Producers and consumers will be on different hosts. Network of brokers
>>> topology.
>>>
>>> Does anyone have experience pushing ActiveMQ that far, or even further
>>> with
>>> 2M queues and several messages a second?
>>>
>>> Will it scale to that extent? I can add boxes to host additional
>>> producers
>>> and consumers, but not sure if pushing metadata of that mighty number of
>>> queues around will be handled OK
>>> --
>>> View this message in context:
>>> http://old.nabble.com/500ooo%2B-queues--tp26417356p26417356.html
>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>
>>>
>
> Rob Davies
> http://twitter.com/rajdavies
> I work here: http://fusesource.com
> My Blog: http://rajdavies.blogspot.com/
> I'm writing this: http://www.manning.com/snyder/
>
>
>
>
>
>