We have an application that, when run in Hadoop, may cause a very large
number (up to 1000) clients to attempt connections to the message broker
simultaneously. Currently we use a custom TCP protocol, and I find that when
there are that many inbound connections, the OS starts to refuse them, and I
need to back off and retry a few seconds later. Will I need to do anything
similar with an ActiveMQ client, or will it handle that situation
automatically? Can you recommend client or broker settings that will work
better in that scenario?
Doing a bit more reading I can see there is a "backlog" setting in the server
URL. I also see there is a failover
which can be configured client-side. Refining my question to be more
specific, what combination of backlog and failover transport is recommended
to ensure that there are no failures when some large number (say 1000) of
new clients connect more or less simultaneously?