ActiveMQ: Exception while executing listener (ignored) and broker dies

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

ActiveMQ: Exception while executing listener (ignored) and broker dies

ctalluri
Hi
We have 3 brokers connected to Zookeeper using replicated leveldb. Some times we get the following INFO message on Master

Client session timed out, have not heard from server in 2xxxms for sessionid xxxxx 
we do have  zkSessionTimeout="3s"
and the broker recycles and starts as slave, most of the times it becomes slave successfully. but some times while restarting gives the following warning and broker dies

2017-02-01 17:24:33,062 | WARN  | Exception while executing listener (ignored) | org.apache.activemq.leveldb.replicated.groups.ZKClient | ZooKeeper state change dispatcher thread
java.lang.NullPointerException
        at org.linkedin.zookeeper.tracker.ZooKeeperTreeTracker.raiseEvents(ZooKeeperTreeTracker.java:402)[org.linkedin.zookeeper-impl-1.4.0.jar:]
        at org.linkedin.zookeeper.tracker.ZooKeeperTreeTracker.track(ZooKeeperTreeTracker.java:240)[org.linkedin.zookeeper-impl-1.4.0.jar:]
        at org.linkedin.zookeeper.tracker.ZooKeeperTreeTracker.track(ZooKeeperTreeTracker.java:228)[org.linkedin.zookeeper-impl-1.4.0.jar:]
        at org.apache.activemq.leveldb.replicated.groups.ZooKeeperGroup.onConnected(ZooKeeperGroup.scala:124)[activemq-leveldb-store-5.13.3.jar:5.13.3]
        at org.apache.activemq.leveldb.replicated.groups.ZKClient.callListeners(ZKClient.java:385)[activemq-leveldb-store-5.13.3.jar:5.13.3]
        at org.apache.activemq.leveldb.replicated.groups.ZKClient$StateChangeDispatcher.run(ZKClient.java:354)[activemq-leveldb-store-5.13.3.jar:5.13.3]
2017-02-01 17:24:33,065 | ERROR | Failed to start Apache ActiveMQ ([broker-aamqwafiu1-amq, null], java.lang.NullPointerException) | org.apache.activemq.broker.BrokerService | main
2017-02-01 17:24:33,066 | INFO  | Apache ActiveMQ 5.13.3 (broker-aamqwafiu1-amq, null) is shutting down | org.apache.activemq.broker.BrokerService | main
2017-02-01 17:24:33,067 | INFO  | Connector openwire stopped | org.apache.activemq.broker.TransportConnector | main
2017-02-01 17:24:33,068 | INFO  | StateChangeDispatcher terminated. | org.apache.activemq.leveldb.replicated.groups.ZKClient | ZooKeeper state change dispatcher thread
2017-02-01 17:24:33,071 | INFO  | Session: 0x458b45c4bc3001e closed | org.apache.zookeeper.ZooKeeper | main
2017-02-01 17:24:33,072 | INFO  | EventThread shut down | org.apache.zookeeper.ClientCnxn | main-EventThread
2017-02-01 17:24:33,174 | INFO  | Apache ActiveMQ 5.13.3 (broker-aamqwafiu1-amq, null) uptime 0.178 seconds | org.apache.activemq.broker.BrokerService | main
2017-02-01 17:24:33,175 | INFO  | Apache ActiveMQ 5.13.3 (broker-aamqwafiu1-amq, null) is shutdown | org.apache.activemq.broker.BrokerService | main
2017-02-01 17:24:33,176 | INFO  | Closing org.apache.activemq.xbean.XBeanBrokerFactory$1@717ee24a: startup date [Wed Feb 01 17:24:32 EST 2017]; root of context hierarchy | org.apache.activemq.xbean.XBeanBrokerFactory$1 | main
2017-02-01 17:24:33,177 | WARN  | Exception thrown from LifecycleProcessor on context close | org.apache.activemq.xbean.XBeanBrokerFactory$1 | main
java.lang.IllegalStateException: LifecycleProcessor not initialized - call 'refresh' before invoking lifecycle methods via the context: org.apache.activemq.xbean.XBeanBrokerFactory$1@717ee24a: startup date [Wed Feb 01 17:24:32 EST 2017]; root of context hierarchy
        at org.springframework.context.support.AbstractApplicationContext.getLifecycleProcessor(AbstractApplicationContext.java:357)[spring-context-4.1.9.RELEASE.jar:4.1.9.RELEASE]
        at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:884)[spring-context-4.1.9.RELEASE.jar:4.1.9.RELEASE]
        at org.springframework.context.support.AbstractApplicationContext.close(AbstractApplicationContext.java:843)[spring-context-4.1.9.RELEASE.jar:4.1.9.RELEASE]
        at org.apache.activemq.hooks.SpringContextHook.run(SpringContextHook.java:30)[activemq-spring-5.13.3.jar:5.13.3]
        at org.apache.activemq.broker.BrokerService.stop(BrokerService.java:873)[activemq-broker-5.13.3.jar:5.13.3]
        at org.apache.activemq.xbean.XBeanBrokerService.stop(XBeanBrokerService.java:122)[activemq-spring-5.13.3.jar:5.13.3]
        at org.apache.activemq.broker.BrokerService.start(BrokerService.java:627)[activemq-broker-5.13.3.jar:5.13.3]
        at org.apache.activemq.xbean.XBeanBrokerService.afterPropertiesSet(XBeanBrokerService.java:73)[activemq-spring-5.13.3.jar:5.13.3]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)[:1.7.0_60]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)[:1.7.0_60]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:1.7.0_60]
        at java.lang.reflect.Method.invoke(Method.java:606)[:1.7.0_60]
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1700)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1639)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1568)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:539)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
        at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:303)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:755)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]


We are using
Zookeeper 3.4.8
activemq 5.13.3

Any insight is appreciated