ActiveMQ shared file Shared File System hangs server startups

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

ActiveMQ shared file Shared File System hangs server startups

julienColin
This post has NOT been accepted by the mailing list yet.
Hi everyone,
I have a question about AMQ Shared File System solution with Kahadb for HA (seems to be very common to use this solution).
I have two tomcat servers running the same webapp, each embedding an ActiveMQ broker and a Camel client.
I'm using the failover protocol on the Camel client side to connect to activemq like this :
failover:(tcp://server1:61616,tcp://server2:61616)?randomize=false

It works fine when i start the first server, but then when i start the second server,  i logically have the following message :
[INFO] | 2014-10-21 10:45:19,584 | SharedFileLocker | Database /lock is locked... waiting 10 seconds for the database to be unlocked. Reason: java.io.IOException: File '/lock' could not be locked.

And the machine keeps trying to get the lock over and over...
The problem is that, while doing so , the second server hangs and never finishes its startup, and so my second camel client is never reachable.

Is there any way to run this routine of getting the lock in background, so that it doesnt make my tomcat stuck ?

Thanks anyone who can help me on this topic.

Regards,
Julien.
Reply | Threaded
Open this post in threaded view
|

Re: ActiveMQ shared file Shared File System hangs server startups

uromahn
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|

Re: ActiveMQ shared file Shared File System hangs server startups

julienColin
This post has NOT been accepted by the mailing list yet.
Hi uromahn, and thank you for your answer.
I know this is the expected behaviour. My problem is not to see the second ActiveMQ trying to obtain the lock, my problem is that the thread which takes care of getting that lock is blocking the rest of my app to start ( my webapp doesnt contains only an ActiveMQ, it has a Camel bus, Rest APIs etc.. and there is no reason why those wouldnt run even if the ActiveMQ broker is not started ).

I was wondering if there was any way  to make that thread non-blocking ?


Thanks,
Julien.


Reply | Threaded
Open this post in threaded view
|

Re: ActiveMQ shared file Shared File System hangs server startups

artnaseef
It is possible to start ActiveMQ asynchronously - there's a "startAsync" setting on the broker.

With that said, I recommend running separate activemq servers instead of embedding the brokers.  ActiveMQ is not a light-weight tool.  Embedding it inside tomcat will limit scaling of both ActiveMQ and Tomcat, and can lead to a lot of confusion when problems arise.
Reply | Threaded
Open this post in threaded view
|

Re: ActiveMQ shared file Shared File System hangs server startups

julienColin
This post has NOT been accepted by the mailing list yet.
Hello artnaseef,
thank you very much , this is exactly what i was looking for :)
i'm taking good note of your point concerning the separation between my webapp and brokers. I dont have an extensive usage of activeMQ so i have no issues of scalability for now ... I might have to think about separating those later and install a network of brokers.

Thanks again,

Julien.
Reply | Threaded
Open this post in threaded view
|

Re: ActiveMQ shared file Shared File System hangs server startups

artnaseef
You're welcome.  Glad I could help.

Note, btw, that with async startup, some things might look funny/incomplete on the passive broker.  Such as the webconsole.  If something looks wrong, I recommend checking first if the broker is active as that could explain it.  With sync startup, those issues don't occur because the related services aren't started until the broker becomes active.