mkahadb: Failed to start per destination persistence adapter for destination

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

mkahadb: Failed to start per destination persistence adapter for destination

boday
This post was updated on .
after bouncing our ActiveMQ 5.9.0 broker, I see these in the logs

 INFO | Recovering from the journal ...
 INFO | Recovery replayed 6125 operations from the journal in 0.254 seconds.
 INFO | KahaDB is version 5
 INFO | Recovering from the journal ...
 INFO | Recovery replayed 1486 operations from the journal in 0.069 seconds.
 INFO | Apache ActiveMQ 5.9.0 (localhost, ID:machine-53296-1394580902989-0:1) is starting
INFO | pending local transactions: []
 INFO | Corrupt journal records found in '/usr/local/eda/AMQ_GW/data/kaha/queue#3a#2f#2f/db-1.log' between offsets: 3123034..3123105
 INFO | Corrupt journal records found in '/usr/local/eda/AMQ_GW/data/kaha/queue#3a#2f#2f/db-1.log' between offsets: 3125439..3125510
 INFO | Corrupt journal records found in '/usr/local/eda/AMQ_GW/data/kaha/queue#3a#2f#2f/db-1.log' between offsets: 3127844..3127915
 INFO | Corrupt journal records found in '/usr/local/eda/AMQ_GW/data/kaha/queue#3a#2f#2f/db-1.log' between offsets: 3130249..3130320
 INFO | KahaDB is version 5
 INFO | Recovering from the journal ...
 INFO | Recovery replayed 18364 operations from the journal in 0.699 seconds.
ERROR | java.lang.RuntimeException: Failed to start per destination persistence adapter for destination: queue://, options:[KahaDBPersistenceAdapter[/usr/local/eda/AMQ_GW/data/kaha/queue#3a#2f#2fMyQueue...[all queue names listed]...]

followed by this...

javax.management.InstanceAlreadyExistsException: org.apache.activemq:type=Broker,brokerName=localhost,Service=PersistenceAdapter,InstanceName=KahaDBPersistenceAdapter[/usr/local/eda/AMQ_GW/data/kaha/queue#3a#2f#2f]
        at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
        at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
        at org.apache.activemq.broker.jmx.ManagementContext.registerMBean(ManagementContext.java:380)
        at org.apache.activemq.broker.jmx.AnnotatedMBean.registerMBean(AnnotatedMBean.java:72)
        at org.apache.activemq.store.kahadb.KahaDBPersistenceAdapter.doStart(KahaDBPersistenceAdapter.java:231)
        at org.apache.activemq.util.ServiceSupport.start(ServiceSupport.java:55)
        at org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter.startAdapter(MultiKahaDBPersistenceAdapter.java:179)
        at org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter.getMatchingPersistenceAdapter(MultiKahaDBPersistenceAdapter.java:173)
        at org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter.createQueueMessageStore(MultiKahaDBPersistenceAdapter.java:157)
        at org.apache.activemq.broker.region.DestinationFactoryImpl.createDestination(DestinationFactoryImpl.java:84)
        at org.apache.activemq.broker.region.AbstractRegion.createDestination(AbstractRegion.java:512)
        at org.apache.activemq.broker.jmx.ManagedQueueRegion.createDestination(ManagedQueueRegion.java:56)
        at org.apache.activemq.broker.region.AbstractRegion.addDestination(AbstractRegion.java:134)
        at org.apache.activemq.broker.region.RegionBroker.addDestination(RegionBroker.java:322)
        at org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:167)
        at org.apache.activemq.advisory.AdvisoryBroker.addDestination(AdvisoryBroker.java:185)
        at org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:167)
        at org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:167)
        at org.apache.activemq.broker.MutableBrokerFilter.addDestination(MutableBrokerFilter.java:172)
        at org.apache.activemq.broker.region.AbstractRegion.start(AbstractRegion.java:98)
        at org.apache.activemq.broker.region.RegionBroker.start(RegionBroker.java:197)
        at org.apache.activemq.broker.jmx.ManagedRegionBroker.start(ManagedRegionBroker.java:120)
        at org.apache.activemq.broker.BrokerFilter.start(BrokerFilter.java:182)
        at org.apache.activemq.broker.BrokerFilter.start(BrokerFilter.java:182)
        at org.apache.activemq.broker.TransactionBroker.start(TransactionBroker.java:120)
        at org.apache.activemq.broker.BrokerService$5.start(BrokerService.java:2144)
        at org.apache.activemq.broker.BrokerService.doStartBroker(BrokerService.java:648)
        at org.apache.activemq.broker.BrokerService.startBroker(BrokerService.java:632)
        at org.apache.activemq.broker.BrokerService.start(BrokerService.java:568)
        at org.apache.activemq.xbean.XBeanBrokerService.afterPropertiesSet(XBeanBrokerService.java:73)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1608)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1549)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1479)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:521)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:458)
        at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:295)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:223)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:292)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:628)
        at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:932)
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:479)
        at org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:64)
        at org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:52)
        at org.apache.activemq.xbean.XBeanBrokerFactory$1.<init>(XBeanBrokerFactory.java:104)
        at org.apache.activemq.xbean.XBeanBrokerFactory.createApplicationContext(XBeanBrokerFactory.java:104)
        at org.apache.activemq.xbean.XBeanBrokerFactory.createBroker(XBeanBrokerFactory.java:67)
        at org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:71)
        at org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:54)
        at org.apache.activemq.console.command.StartCommand.runTask(StartCommand.java:87)
        at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57)
        at org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:150)
        at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57)
        at org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:104)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.activemq.console.Main.runTaskClass(Main.java:262)
        at org.apache.activemq.console.Main.main(Main.java:115)
 INFO | Apache ActiveMQ 5.9.0 (localhost, ID:machine-53406-1394581237814-0:1) is shutting down
 INFO | Connector openwire stopped
 INFO | Connector vm stopped
 INFO | PListStore:[/usr/local/eda/AMQ_GW/data/localhost/tmp_storage] stopped
 INFO | Stopping async queue tasks
 INFO | Stopping async topic tasks
 INFO | Stopped KahaDB

I tried removing the directory in questions to get it to start, but it just got hung up on another queue.  I eventually just archived the entire /data directory to get it to start cleanly...

here is my mkahadb config...

       <persistenceAdapter>
            <mKahaDB directory="${activemq.base}/data/kaha" >
                <filteredPersistenceAdapters>
                    <filteredKahaDB perDestination="true">
                        <persistenceAdapter>
                            <kahaDB journalMaxFileLength="8mb"/>
                        </persistenceAdapter>
                    </filteredKahaDB>
                </filteredPersistenceAdapters>
            </mKahaDB>
        </persistenceAdapter>

question, is this a known issue?  are there any other workaround?  any idea what cause this?

thanks...
Reply | Threaded
Open this post in threaded view
|

Re: mkahadb: Failed to start per destination persistence adapter for destination

gtully
I wonder if it is related to truncation. See
https://issues.apache.org/jira/browse/AMQ-4183
On 11 Mar 2014 23:50, "boday" <[hidden email]> wrote:

> after bouncing our ActiveMQ 5.9.0 broker, I see these in the logs
>
> INFO | pending local transactions: []
>  INFO | Corrupt journal records found in
> '/usr/local/eda/AMQ_GW/data/kaha/queue#3a#2f#2f/db-1.log' between offsets:
> 3123034..3123105
>  INFO | Corrupt journal records found in
> '/usr/local/eda/AMQ_GW/data/kaha/queue#3a#2f#2f/db-1.log' between offsets:
> 3125439..3125510
>  INFO | Corrupt journal records found in
> '/usr/local/eda/AMQ_GW/data/kaha/queue#3a#2f#2f/db-1.log' between offsets:
> 3127844..3127915
>  INFO | Corrupt journal records found in
> '/usr/local/eda/AMQ_GW/data/kaha/queue#3a#2f#2f/db-1.log' between offsets:
> 3130249..3130320
>  INFO | KahaDB is version 5
>  INFO | Recovering from the journal ...
>  INFO | Recovery replayed 18364 operations from the journal in 0.699
> seconds.
> ERROR | java.lang.RuntimeException: Failed to start per destination
> persistence adapter for destination: queue://
>
> followed by this...
>
> javax.management.InstanceAlreadyExistsException:
>
> org.apache.activemq:type=Broker,brokerName=localhost,Service=PersistenceAdapter,InstanceName=KahaDBPersistenceAdapter[/usr/local/eda/AMQ_GW/data/kaha/queue#3a#2f#2f]
>         at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
>         at
>
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
>         at
>
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
>         at
>
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
>         at
>
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
>         at
>
> com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
>         at
>
> org.apache.activemq.broker.jmx.ManagementContext.registerMBean(ManagementContext.java:380)
>         at
>
> org.apache.activemq.broker.jmx.AnnotatedMBean.registerMBean(AnnotatedMBean.java:72)
>         at
>
> org.apache.activemq.store.kahadb.KahaDBPersistenceAdapter.doStart(KahaDBPersistenceAdapter.java:231)
>         at
> org.apache.activemq.util.ServiceSupport.start(ServiceSupport.java:55)
>         at
>
> org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter.startAdapter(MultiKahaDBPersistenceAdapter.java:179)
>         at
>
> org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter.getMatchingPersistenceAdapter(MultiKahaDBPersistenceAdapter.java:173)
>         at
>
> org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter.createQueueMessageStore(MultiKahaDBPersistenceAdapter.java:157)
>         at
>
> org.apache.activemq.broker.region.DestinationFactoryImpl.createDestination(DestinationFactoryImpl.java:84)
>         at
>
> org.apache.activemq.broker.region.AbstractRegion.createDestination(AbstractRegion.java:512)
>         at
>
> org.apache.activemq.broker.jmx.ManagedQueueRegion.createDestination(ManagedQueueRegion.java:56)
>         at
>
> org.apache.activemq.broker.region.AbstractRegion.addDestination(AbstractRegion.java:134)
>         at
>
> org.apache.activemq.broker.region.RegionBroker.addDestination(RegionBroker.java:322)
>         at
>
> org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:167)
>         at
>
> org.apache.activemq.advisory.AdvisoryBroker.addDestination(AdvisoryBroker.java:185)
>         at
>
> org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:167)
>         at
>
> org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:167)
>         at
>
> org.apache.activemq.broker.MutableBrokerFilter.addDestination(MutableBrokerFilter.java:172)
>         at
>
> org.apache.activemq.broker.region.AbstractRegion.start(AbstractRegion.java:98)
>         at
> org.apache.activemq.broker.region.RegionBroker.start(RegionBroker.java:197)
>         at
>
> org.apache.activemq.broker.jmx.ManagedRegionBroker.start(ManagedRegionBroker.java:120)
>         at
> org.apache.activemq.broker.BrokerFilter.start(BrokerFilter.java:182)
>         at
> org.apache.activemq.broker.BrokerFilter.start(BrokerFilter.java:182)
>         at
>
> org.apache.activemq.broker.TransactionBroker.start(TransactionBroker.java:120)
>         at
> org.apache.activemq.broker.BrokerService$5.start(BrokerService.java:2144)
>         at
>
> org.apache.activemq.broker.BrokerService.doStartBroker(BrokerService.java:648)
>         at
>
> org.apache.activemq.broker.BrokerService.startBroker(BrokerService.java:632)
>         at
> org.apache.activemq.broker.BrokerService.start(BrokerService.java:568)
>         at
>
> org.apache.activemq.xbean.XBeanBrokerService.afterPropertiesSet(XBeanBrokerService.java:73)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at
>
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1608)
>         at
>
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1549)
>         at
>
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1479)
>         at
>
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:521)
>         at
>
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:458)
>         at
>
> org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:295)
>         at
>
> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:223)
>         at
>
> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:292)
>         at
>
> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
>         at
>
> org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:628)
>         at
>
> org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:932)
>         at
>
> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:479)
>         at
>
> org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:64)
>         at
>
> org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:52)
>         at
>
> org.apache.activemq.xbean.XBeanBrokerFactory$1.<init>(XBeanBrokerFactory.java:104)
>         at
>
> org.apache.activemq.xbean.XBeanBrokerFactory.createApplicationContext(XBeanBrokerFactory.java:104)
>         at
>
> org.apache.activemq.xbean.XBeanBrokerFactory.createBroker(XBeanBrokerFactory.java:67)
>         at
>
> org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:71)
>         at
>
> org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:54)
>         at
>
> org.apache.activemq.console.command.StartCommand.runTask(StartCommand.java:87)
>         at
>
> org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57)
>         at
>
> org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:150)
>         at
>
> org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57)
>         at
>
> org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:104)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.activemq.console.Main.runTaskClass(Main.java:262)
>         at org.apache.activemq.console.Main.main(Main.java:115)
>  INFO | Apache ActiveMQ 5.9.0 (localhost,
> ID:machine-53406-1394581237814-0:1) is shutting down
>  INFO | Connector openwire stopped
>  INFO | Connector vm stopped
>  INFO | PListStore:[/usr/local/eda/AMQ_GW/data/localhost/tmp_storage]
> stopped
>  INFO | Stopping async queue tasks
>  INFO | Stopping async topic tasks
>  INFO | Stopped KahaDB
>
> I tried removing the directory in questions to get it to start, but it just
> got hung up on another queue.  I eventually just archived the entire /data
> directory to get it to start cleanly...
>
> here is my mkahadb config...
>
>        <persistenceAdapter>
>             <mKahaDB directory="${activemq.base}/data/kaha" >
>                 <filteredPersistenceAdapters>
>                     <filteredKahaDB perDestination="true">
>                         <persistenceAdapter>
>                             <kahaDB journalMaxFileLength="8mb"/>
>                         </persistenceAdapter>
>                     </filteredKahaDB>
>                 </filteredPersistenceAdapters>
>             </mKahaDB>
>         </persistenceAdapter>
>
> question, is this a known issue?  are there any other workaround?  any idea
> what cause this?
>
> thanks...
>
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/mkahadb-Failed-to-start-per-destination-persistence-adapter-for-destination-tp4678754.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: mkahadb: Failed to start per destination persistence adapter for destination

boday
thanks, I took a look at that issue, but I don't see any truncated directory names or warning about this in the logs...any other thoughts?

gtully wrote
I wonder if it is related to truncation. See
https://issues.apache.org/jira/browse/AMQ-4183
On 11 Mar 2014 23:50, "boday" <[hidden email]> wrote:

> after bouncing our ActiveMQ 5.9.0 broker, I see these in the logs
>
> INFO | pending local transactions: []
>  INFO | Corrupt journal records found in
> '/usr/local/eda/AMQ_GW/data/kaha/queue#3a#2f#2f/db-1.log' between offsets:
> 3123034..3123105
>  INFO | Corrupt journal records found in
> '/usr/local/eda/AMQ_GW/data/kaha/queue#3a#2f#2f/db-1.log' between offsets:
> 3125439..3125510
>  INFO | Corrupt journal records found in
> '/usr/local/eda/AMQ_GW/data/kaha/queue#3a#2f#2f/db-1.log' between offsets:
> 3127844..3127915
>  INFO | Corrupt journal records found in
> '/usr/local/eda/AMQ_GW/data/kaha/queue#3a#2f#2f/db-1.log' between offsets:
> 3130249..3130320
>  INFO | KahaDB is version 5
>  INFO | Recovering from the journal ...
>  INFO | Recovery replayed 18364 operations from the journal in 0.699
> seconds.
> ERROR | java.lang.RuntimeException: Failed to start per destination
> persistence adapter for destination: queue://
>
> followed by this...
>
> javax.management.InstanceAlreadyExistsException:
>
> org.apache.activemq:type=Broker,brokerName=localhost,Service=PersistenceAdapter,InstanceName=KahaDBPersistenceAdapter[/usr/local/eda/AMQ_GW/data/kaha/queue#3a#2f#2f]
>         at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
>         at
>
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
>         at
>
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
>         at
>
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
>         at
>
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
>         at
>
> com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
>         at
>
> org.apache.activemq.broker.jmx.ManagementContext.registerMBean(ManagementContext.java:380)
>         at
>
> org.apache.activemq.broker.jmx.AnnotatedMBean.registerMBean(AnnotatedMBean.java:72)
>         at
>
> org.apache.activemq.store.kahadb.KahaDBPersistenceAdapter.doStart(KahaDBPersistenceAdapter.java:231)
>         at
> org.apache.activemq.util.ServiceSupport.start(ServiceSupport.java:55)
>         at
>
> org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter.startAdapter(MultiKahaDBPersistenceAdapter.java:179)
>         at
>
> org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter.getMatchingPersistenceAdapter(MultiKahaDBPersistenceAdapter.java:173)
>         at
>
> org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter.createQueueMessageStore(MultiKahaDBPersistenceAdapter.java:157)
>         at
>
> org.apache.activemq.broker.region.DestinationFactoryImpl.createDestination(DestinationFactoryImpl.java:84)
>         at
>
> org.apache.activemq.broker.region.AbstractRegion.createDestination(AbstractRegion.java:512)
>         at
>
> org.apache.activemq.broker.jmx.ManagedQueueRegion.createDestination(ManagedQueueRegion.java:56)
>         at
>
> org.apache.activemq.broker.region.AbstractRegion.addDestination(AbstractRegion.java:134)
>         at
>
> org.apache.activemq.broker.region.RegionBroker.addDestination(RegionBroker.java:322)
>         at
>
> org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:167)
>         at
>
> org.apache.activemq.advisory.AdvisoryBroker.addDestination(AdvisoryBroker.java:185)
>         at
>
> org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:167)
>         at
>
> org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:167)
>         at
>
> org.apache.activemq.broker.MutableBrokerFilter.addDestination(MutableBrokerFilter.java:172)
>         at
>
> org.apache.activemq.broker.region.AbstractRegion.start(AbstractRegion.java:98)
>         at
> org.apache.activemq.broker.region.RegionBroker.start(RegionBroker.java:197)
>         at
>
> org.apache.activemq.broker.jmx.ManagedRegionBroker.start(ManagedRegionBroker.java:120)
>         at
> org.apache.activemq.broker.BrokerFilter.start(BrokerFilter.java:182)
>         at
> org.apache.activemq.broker.BrokerFilter.start(BrokerFilter.java:182)
>         at
>
> org.apache.activemq.broker.TransactionBroker.start(TransactionBroker.java:120)
>         at
> org.apache.activemq.broker.BrokerService$5.start(BrokerService.java:2144)
>         at
>
> org.apache.activemq.broker.BrokerService.doStartBroker(BrokerService.java:648)
>         at
>
> org.apache.activemq.broker.BrokerService.startBroker(BrokerService.java:632)
>         at
> org.apache.activemq.broker.BrokerService.start(BrokerService.java:568)
>         at
>
> org.apache.activemq.xbean.XBeanBrokerService.afterPropertiesSet(XBeanBrokerService.java:73)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at
>
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1608)
>         at
>
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1549)
>         at
>
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1479)
>         at
>
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:521)
>         at
>
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:458)
>         at
>
> org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:295)
>         at
>
> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:223)
>         at
>
> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:292)
>         at
>
> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
>         at
>
> org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:628)
>         at
>
> org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:932)
>         at
>
> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:479)
>         at
>
> org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:64)
>         at
>
> org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:52)
>         at
>
> org.apache.activemq.xbean.XBeanBrokerFactory$1.<init>(XBeanBrokerFactory.java:104)
>         at
>
> org.apache.activemq.xbean.XBeanBrokerFactory.createApplicationContext(XBeanBrokerFactory.java:104)
>         at
>
> org.apache.activemq.xbean.XBeanBrokerFactory.createBroker(XBeanBrokerFactory.java:67)
>         at
>
> org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:71)
>         at
>
> org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:54)
>         at
>
> org.apache.activemq.console.command.StartCommand.runTask(StartCommand.java:87)
>         at
>
> org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57)
>         at
>
> org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:150)
>         at
>
> org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57)
>         at
>
> org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:104)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.activemq.console.Main.runTaskClass(Main.java:262)
>         at org.apache.activemq.console.Main.main(Main.java:115)
>  INFO | Apache ActiveMQ 5.9.0 (localhost,
> ID:machine-53406-1394581237814-0:1) is shutting down
>  INFO | Connector openwire stopped
>  INFO | Connector vm stopped
>  INFO | PListStore:[/usr/local/eda/AMQ_GW/data/localhost/tmp_storage]
> stopped
>  INFO | Stopping async queue tasks
>  INFO | Stopping async topic tasks
>  INFO | Stopped KahaDB
>
> I tried removing the directory in questions to get it to start, but it just
> got hung up on another queue.  I eventually just archived the entire /data
> directory to get it to start cleanly...
>
> here is my mkahadb config...
>
>        <persistenceAdapter>
>             <mKahaDB directory="${activemq.base}/data/kaha" >
>                 <filteredPersistenceAdapters>
>                     <filteredKahaDB perDestination="true">
>                         <persistenceAdapter>
>                             <kahaDB journalMaxFileLength="8mb"/>
>                         </persistenceAdapter>
>                     </filteredKahaDB>
>                 </filteredPersistenceAdapters>
>             </mKahaDB>
>         </persistenceAdapter>
>
> question, is this a known issue?  are there any other workaround?  any idea
> what cause this?
>
> thanks...
>
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/mkahadb-Failed-to-start-per-destination-persistence-adapter-for-destination-tp4678754.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: mkahadb: Failed to start per destination persistence adapter for destination

gtully
is there a stack trace with the:
ERROR | java.lang.RuntimeException: Failed to start per destination
persistence adapter for destination: queue://

odd that there is no queue name in there.

On 12 March 2014 14:52, boday <[hidden email]> wrote:

> thanks, I took a look at that issue, but I don't see any truncated directory
> names or warning about this in the logs...any other thoughts?
>
>
> gtully wrote
>> I wonder if it is related to truncation. See
>> https://issues.apache.org/jira/browse/AMQ-4183
>> On 11 Mar 2014 23:50, "boday" &lt;
>
>> ben.oday@
>
>> &gt; wrote:
>>
>>> after bouncing our ActiveMQ 5.9.0 broker, I see these in the logs
>>>
>>> INFO | pending local transactions: []
>>>  INFO | Corrupt journal records found in
>>> '/usr/local/eda/AMQ_GW/data/kaha/queue#3a#2f#2f/db-1.log' between
>>> offsets:
>>> 3123034..3123105
>>>  INFO | Corrupt journal records found in
>>> '/usr/local/eda/AMQ_GW/data/kaha/queue#3a#2f#2f/db-1.log' between
>>> offsets:
>>> 3125439..3125510
>>>  INFO | Corrupt journal records found in
>>> '/usr/local/eda/AMQ_GW/data/kaha/queue#3a#2f#2f/db-1.log' between
>>> offsets:
>>> 3127844..3127915
>>>  INFO | Corrupt journal records found in
>>> '/usr/local/eda/AMQ_GW/data/kaha/queue#3a#2f#2f/db-1.log' between
>>> offsets:
>>> 3130249..3130320
>>>  INFO | KahaDB is version 5
>>>  INFO | Recovering from the journal ...
>>>  INFO | Recovery replayed 18364 operations from the journal in 0.699
>>> seconds.
>>> ERROR | java.lang.RuntimeException: Failed to start per destination
>>> persistence adapter for destination: queue://
>>>
>>> followed by this...
>>>
>>> javax.management.InstanceAlreadyExistsException:
>>>
>>> org.apache.activemq:type=Broker,brokerName=localhost,Service=PersistenceAdapter,InstanceName=KahaDBPersistenceAdapter[/usr/local/eda/AMQ_GW/data/kaha/queue#3a#2f#2f]
>>>         at
>>> com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
>>>         at
>>>
>>> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
>>>         at
>>>
>>> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
>>>         at
>>>
>>> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
>>>         at
>>>
>>> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
>>>         at
>>>
>>> com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
>>>         at
>>>
>>> org.apache.activemq.broker.jmx.ManagementContext.registerMBean(ManagementContext.java:380)
>>>         at
>>>
>>> org.apache.activemq.broker.jmx.AnnotatedMBean.registerMBean(AnnotatedMBean.java:72)
>>>         at
>>>
>>> org.apache.activemq.store.kahadb.KahaDBPersistenceAdapter.doStart(KahaDBPersistenceAdapter.java:231)
>>>         at
>>> org.apache.activemq.util.ServiceSupport.start(ServiceSupport.java:55)
>>>         at
>>>
>>> org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter.startAdapter(MultiKahaDBPersistenceAdapter.java:179)
>>>         at
>>>
>>> org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter.getMatchingPersistenceAdapter(MultiKahaDBPersistenceAdapter.java:173)
>>>         at
>>>
>>> org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter.createQueueMessageStore(MultiKahaDBPersistenceAdapter.java:157)
>>>         at
>>>
>>> org.apache.activemq.broker.region.DestinationFactoryImpl.createDestination(DestinationFactoryImpl.java:84)
>>>         at
>>>
>>> org.apache.activemq.broker.region.AbstractRegion.createDestination(AbstractRegion.java:512)
>>>         at
>>>
>>> org.apache.activemq.broker.jmx.ManagedQueueRegion.createDestination(ManagedQueueRegion.java:56)
>>>         at
>>>
>>> org.apache.activemq.broker.region.AbstractRegion.addDestination(AbstractRegion.java:134)
>>>         at
>>>
>>> org.apache.activemq.broker.region.RegionBroker.addDestination(RegionBroker.java:322)
>>>         at
>>>
>>> org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:167)
>>>         at
>>>
>>> org.apache.activemq.advisory.AdvisoryBroker.addDestination(AdvisoryBroker.java:185)
>>>         at
>>>
>>> org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:167)
>>>         at
>>>
>>> org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:167)
>>>         at
>>>
>>> org.apache.activemq.broker.MutableBrokerFilter.addDestination(MutableBrokerFilter.java:172)
>>>         at
>>>
>>> org.apache.activemq.broker.region.AbstractRegion.start(AbstractRegion.java:98)
>>>         at
>>> org.apache.activemq.broker.region.RegionBroker.start(RegionBroker.java:197)
>>>         at
>>>
>>> org.apache.activemq.broker.jmx.ManagedRegionBroker.start(ManagedRegionBroker.java:120)
>>>         at
>>> org.apache.activemq.broker.BrokerFilter.start(BrokerFilter.java:182)
>>>         at
>>> org.apache.activemq.broker.BrokerFilter.start(BrokerFilter.java:182)
>>>         at
>>>
>>> org.apache.activemq.broker.TransactionBroker.start(TransactionBroker.java:120)
>>>         at
>>> org.apache.activemq.broker.BrokerService$5.start(BrokerService.java:2144)
>>>         at
>>>
>>> org.apache.activemq.broker.BrokerService.doStartBroker(BrokerService.java:648)
>>>         at
>>>
>>> org.apache.activemq.broker.BrokerService.startBroker(BrokerService.java:632)
>>>         at
>>> org.apache.activemq.broker.BrokerService.start(BrokerService.java:568)
>>>         at
>>>
>>> org.apache.activemq.xbean.XBeanBrokerService.afterPropertiesSet(XBeanBrokerService.java:73)
>>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>         at
>>>
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>         at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>         at java.lang.reflect.Method.invoke(Method.java:606)
>>>         at
>>>
>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1608)
>>>         at
>>>
>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1549)
>>>         at
>>>
>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1479)
>>>         at
>>>
>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:521)
>>>         at
>>>
>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:458)
>>>         at
>>>
>>> org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:295)
>>>         at
>>>
>>> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:223)
>>>         at
>>>
>>> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:292)
>>>         at
>>>
>>> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
>>>         at
>>>
>>> org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:628)
>>>         at
>>>
>>> org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:932)
>>>         at
>>>
>>> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:479)
>>>         at
>>>
>>> org.apache.xbean.spring.context.ResourceXmlApplicationContext.
>> <init>
>> (ResourceXmlApplicationContext.java:64)
>>>         at
>>>
>>> org.apache.xbean.spring.context.ResourceXmlApplicationContext.
>> <init>
>> (ResourceXmlApplicationContext.java:52)
>>>         at
>>>
>>> org.apache.activemq.xbean.XBeanBrokerFactory$1.
>> <init>
>> (XBeanBrokerFactory.java:104)
>>>         at
>>>
>>> org.apache.activemq.xbean.XBeanBrokerFactory.createApplicationContext(XBeanBrokerFactory.java:104)
>>>         at
>>>
>>> org.apache.activemq.xbean.XBeanBrokerFactory.createBroker(XBeanBrokerFactory.java:67)
>>>         at
>>>
>>> org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:71)
>>>         at
>>>
>>> org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:54)
>>>         at
>>>
>>> org.apache.activemq.console.command.StartCommand.runTask(StartCommand.java:87)
>>>         at
>>>
>>> org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57)
>>>         at
>>>
>>> org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:150)
>>>         at
>>>
>>> org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57)
>>>         at
>>>
>>> org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:104)
>>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>         at
>>>
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>         at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>         at java.lang.reflect.Method.invoke(Method.java:606)
>>>         at org.apache.activemq.console.Main.runTaskClass(Main.java:262)
>>>         at org.apache.activemq.console.Main.main(Main.java:115)
>>>  INFO | Apache ActiveMQ 5.9.0 (localhost,
>>> ID:machine-53406-1394581237814-0:1) is shutting down
>>>  INFO | Connector openwire stopped
>>>  INFO | Connector vm stopped
>>>  INFO | PListStore:[/usr/local/eda/AMQ_GW/data/localhost/tmp_storage]
>>> stopped
>>>  INFO | Stopping async queue tasks
>>>  INFO | Stopping async topic tasks
>>>  INFO | Stopped KahaDB
>>>
>>> I tried removing the directory in questions to get it to start, but it
>>> just
>>> got hung up on another queue.  I eventually just archived the entire
>>> /data
>>> directory to get it to start cleanly...
>>>
>>> here is my mkahadb config...
>>>
>>>
>> <persistenceAdapter>
>>>
>> <mKahaDB directory="${activemq.base}/data/kaha" >
>>>
>> <filteredPersistenceAdapters>
>>>
>> <filteredKahaDB perDestination="true">
>>>
>> <persistenceAdapter>
>>>
>> <kahaDB journalMaxFileLength="8mb"/>
>>>
>> </persistenceAdapter>
>>>
>> </filteredKahaDB>
>>>
>> </filteredPersistenceAdapters>
>>>
>> </mKahaDB>
>>>
>> </persistenceAdapter>
>>>
>>> question, is this a known issue?  are there any other workaround?  any
>>> idea
>>> what cause this?
>>>
>>> thanks...
>>>
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://activemq.2283324.n4.nabble.com/mkahadb-Failed-to-start-per-destination-persistence-adapter-for-destination-tp4678754.html
>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>
>
>
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/mkahadb-Failed-to-start-per-destination-persistence-adapter-for-destination-tp4678754p4678855.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.



--
http://redhat.com
http://blog.garytully.com
Reply | Threaded
Open this post in threaded view
|

Re: mkahadb: Failed to start per destination persistence adapter for destination

boday
This post was updated on .
I scrubbed out the queue names because it was just a long list of all my queues

INFO | Recovery replayed 18364 operations from the journal in 0.699 seconds.
ERROR | java.lang.RuntimeException: Failed to start per destination persistence adapter for destination: queue://, options:[KahaDBPersistenceAdapter[/usr/local/eda/AMQ_GW/data/kaha/queue#3a#2f#2fMyQueue...[all queues listed]...]

and the stacktrace immediately follows this list...

javax.management.InstanceAlreadyExistsException: org.apache.activemq:type=Broker,brokerName=localhost,Service=PersistenceAdapter,InstanceName=KahaDBPersistenceAdapter[/usr/local/eda/AMQ_GW/data/kaha/queue#3a#2f#2f]
        at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
        at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
        at org.apache.activemq.broker.jmx.ManagementContext.registerMBean(ManagementContext.java:380)
        at org.apache.activemq.broker.jmx.AnnotatedMBean.registerMBean(AnnotatedMBean.java:72)
        at org.apache.activemq.store.kahadb.KahaDBPersistenceAdapter.doStart(KahaDBPersistenceAdapter.java:231)
        at org.apache.activemq.util.ServiceSupport.start(ServiceSupport.java:55)
        at org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter.startAdapter(MultiKahaDBPersistenceAdapter.java:179)
        at org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter.getMatchingPersistenceAdapter(MultiKahaDBPersistenceAdapter.java:173)
        at org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter.createQueueMessageStore(MultiKahaDBPersistenceAdapter.java:157)


gtully wrote
is there a stack trace with the:
ERROR | java.lang.RuntimeException: Failed to start per destination
persistence adapter for destination: queue://

odd that there is no queue name in there.

On 12 March 2014 14:52, boday <[hidden email]> wrote:
> thanks, I took a look at that issue, but I don't see any truncated directory
> names or warning about this in the logs...any other thoughts?
>
>
> gtully wrote
>> I wonder if it is related to truncation. See
>> https://issues.apache.org/jira/browse/AMQ-4183
>> On 11 Mar 2014 23:50, "boday" <
>
>> ben.oday@
>
>> > wrote:
>>
>>> after bouncing our ActiveMQ 5.9.0 broker, I see these in the logs
>>>
>>> INFO | pending local transactions: []
>>>  INFO | Corrupt journal records found in
>>> '/usr/local/eda/AMQ_GW/data/kaha/queue#3a#2f#2f/db-1.log' between
>>> offsets:
>>> 3123034..3123105
>>>  INFO | Corrupt journal records found in
>>> '/usr/local/eda/AMQ_GW/data/kaha/queue#3a#2f#2f/db-1.log' between
>>> offsets:
>>> 3125439..3125510
>>>  INFO | Corrupt journal records found in
>>> '/usr/local/eda/AMQ_GW/data/kaha/queue#3a#2f#2f/db-1.log' between
>>> offsets:
>>> 3127844..3127915
>>>  INFO | Corrupt journal records found in
>>> '/usr/local/eda/AMQ_GW/data/kaha/queue#3a#2f#2f/db-1.log' between
>>> offsets:
>>> 3130249..3130320
>>>  INFO | KahaDB is version 5
>>>  INFO | Recovering from the journal ...
>>>  INFO | Recovery replayed 18364 operations from the journal in 0.699
>>> seconds.
>>> ERROR | java.lang.RuntimeException: Failed to start per destination
>>> persistence adapter for destination: queue://
>>>
>>> followed by this...
>>>
>>> javax.management.InstanceAlreadyExistsException:
>>>
>>> org.apache.activemq:type=Broker,brokerName=localhost,Service=PersistenceAdapter,InstanceName=KahaDBPersistenceAdapter[/usr/local/eda/AMQ_GW/data/kaha/queue#3a#2f#2f]
>>>         at
>>> com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
>>>         at
>>>
>>> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
>>>         at
>>>
>>> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
>>>         at
>>>
>>> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
>>>         at
>>>
>>> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
>>>         at
>>>
>>> com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
>>>         at
>>>
>>> org.apache.activemq.broker.jmx.ManagementContext.registerMBean(ManagementContext.java:380)
>>>         at
>>>
>>> org.apache.activemq.broker.jmx.AnnotatedMBean.registerMBean(AnnotatedMBean.java:72)
>>>         at
>>>
>>> org.apache.activemq.store.kahadb.KahaDBPersistenceAdapter.doStart(KahaDBPersistenceAdapter.java:231)
>>>         at
>>> org.apache.activemq.util.ServiceSupport.start(ServiceSupport.java:55)
>>>         at
>>>
>>> org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter.startAdapter(MultiKahaDBPersistenceAdapter.java:179)
>>>         at
>>>
>>> org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter.getMatchingPersistenceAdapter(MultiKahaDBPersistenceAdapter.java:173)
>>>         at
>>>
>>> org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter.createQueueMessageStore(MultiKahaDBPersistenceAdapter.java:157)
>>>         at
>>>
>>> org.apache.activemq.broker.region.DestinationFactoryImpl.createDestination(DestinationFactoryImpl.java:84)
>>>         at
>>>
>>> org.apache.activemq.broker.region.AbstractRegion.createDestination(AbstractRegion.java:512)
>>>         at
>>>
>>> org.apache.activemq.broker.jmx.ManagedQueueRegion.createDestination(ManagedQueueRegion.java:56)
>>>         at
>>>
>>> org.apache.activemq.broker.region.AbstractRegion.addDestination(AbstractRegion.java:134)
>>>         at
>>>
>>> org.apache.activemq.broker.region.RegionBroker.addDestination(RegionBroker.java:322)
>>>         at
>>>
>>> org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:167)
>>>         at
>>>
>>> org.apache.activemq.advisory.AdvisoryBroker.addDestination(AdvisoryBroker.java:185)
>>>         at
>>>
>>> org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:167)
>>>         at
>>>
>>> org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:167)
>>>         at
>>>
>>> org.apache.activemq.broker.MutableBrokerFilter.addDestination(MutableBrokerFilter.java:172)
>>>         at
>>>
>>> org.apache.activemq.broker.region.AbstractRegion.start(AbstractRegion.java:98)
>>>         at
>>> org.apache.activemq.broker.region.RegionBroker.start(RegionBroker.java:197)
>>>         at
>>>
>>> org.apache.activemq.broker.jmx.ManagedRegionBroker.start(ManagedRegionBroker.java:120)
>>>         at
>>> org.apache.activemq.broker.BrokerFilter.start(BrokerFilter.java:182)
>>>         at
>>> org.apache.activemq.broker.BrokerFilter.start(BrokerFilter.java:182)
>>>         at
>>>
>>> org.apache.activemq.broker.TransactionBroker.start(TransactionBroker.java:120)
>>>         at
>>> org.apache.activemq.broker.BrokerService$5.start(BrokerService.java:2144)
>>>         at
>>>
>>> org.apache.activemq.broker.BrokerService.doStartBroker(BrokerService.java:648)
>>>         at
>>>
>>> org.apache.activemq.broker.BrokerService.startBroker(BrokerService.java:632)
>>>         at
>>> org.apache.activemq.broker.BrokerService.start(BrokerService.java:568)
>>>         at
>>>
>>> org.apache.activemq.xbean.XBeanBrokerService.afterPropertiesSet(XBeanBrokerService.java:73)
>>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>         at
>>>
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>         at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>         at java.lang.reflect.Method.invoke(Method.java:606)
>>>         at
>>>
>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1608)
>>>         at
>>>
>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1549)
>>>         at
>>>
>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1479)
>>>         at
>>>
>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:521)
>>>         at
>>>
>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:458)
>>>         at
>>>
>>> org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:295)
>>>         at
>>>
>>> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:223)
>>>         at
>>>
>>> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:292)
>>>         at
>>>
>>> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
>>>         at
>>>
>>> org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:628)
>>>         at
>>>
>>> org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:932)
>>>         at
>>>
>>> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:479)
>>>         at
>>>
>>> org.apache.xbean.spring.context.ResourceXmlApplicationContext.
>> <init>
>> (ResourceXmlApplicationContext.java:64)
>>>         at
>>>
>>> org.apache.xbean.spring.context.ResourceXmlApplicationContext.
>> <init>
>> (ResourceXmlApplicationContext.java:52)
>>>         at
>>>
>>> org.apache.activemq.xbean.XBeanBrokerFactory$1.
>> <init>
>> (XBeanBrokerFactory.java:104)
>>>         at
>>>
>>> org.apache.activemq.xbean.XBeanBrokerFactory.createApplicationContext(XBeanBrokerFactory.java:104)
>>>         at
>>>
>>> org.apache.activemq.xbean.XBeanBrokerFactory.createBroker(XBeanBrokerFactory.java:67)
>>>         at
>>>
>>> org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:71)
>>>         at
>>>
>>> org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:54)
>>>         at
>>>
>>> org.apache.activemq.console.command.StartCommand.runTask(StartCommand.java:87)
>>>         at
>>>
>>> org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57)
>>>         at
>>>
>>> org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:150)
>>>         at
>>>
>>> org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57)
>>>         at
>>>
>>> org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:104)
>>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>         at
>>>
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>         at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>         at java.lang.reflect.Method.invoke(Method.java:606)
>>>         at org.apache.activemq.console.Main.runTaskClass(Main.java:262)
>>>         at org.apache.activemq.console.Main.main(Main.java:115)
>>>  INFO | Apache ActiveMQ 5.9.0 (localhost,
>>> ID:machine-53406-1394581237814-0:1) is shutting down
>>>  INFO | Connector openwire stopped
>>>  INFO | Connector vm stopped
>>>  INFO | PListStore:[/usr/local/eda/AMQ_GW/data/localhost/tmp_storage]
>>> stopped
>>>  INFO | Stopping async queue tasks
>>>  INFO | Stopping async topic tasks
>>>  INFO | Stopped KahaDB
>>>
>>> I tried removing the directory in questions to get it to start, but it
>>> just
>>> got hung up on another queue.  I eventually just archived the entire
>>> /data
>>> directory to get it to start cleanly...
>>>
>>> here is my mkahadb config...
>>>
>>>
>> <persistenceAdapter>
>>>
>> <mKahaDB directory="${activemq.base}/data/kaha" >
>>>
>> <filteredPersistenceAdapters>
>>>
>> <filteredKahaDB perDestination="true">
>>>
>> <persistenceAdapter>
>>>
>> <kahaDB journalMaxFileLength="8mb"/>
>>>
>> </persistenceAdapter>
>>>
>> </filteredKahaDB>
>>>
>> </filteredPersistenceAdapters>
>>>
>> </mKahaDB>
>>>
>> </persistenceAdapter>
>>>
>>> question, is this a known issue?  are there any other workaround?  any
>>> idea
>>> what cause this?
>>>
>>> thanks...
>>>
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://activemq.2283324.n4.nabble.com/mkahadb-Failed-to-start-per-destination-persistence-adapter-for-destination-tp4678754.html
>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>
>
>
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/mkahadb-Failed-to-start-per-destination-persistence-adapter-for-destination-tp4678754p4678855.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.



--
http://redhat.com
http://blog.garytully.com
Reply | Threaded
Open this post in threaded view
|

Re: mkahadb: Failed to start per destination persistence adapter for destination

boday
This post was updated on .
any other thoughts on this?  we have seen this several times now when we restart AMQ (using mkahadb) with messages in the queues.  we have had to archive the entire data directory to restart...any other way to recover this data (manually moving files back in or something)? otherwise, these queues/messages are orphaned/lost
Reply | Threaded
Open this post in threaded view
|

Re: mkahadb: Failed to start per destination persistence adapter for destination

boday
This post was updated on .
FYI, I was able to get the broker to start by disabling JMX and could see more of the error...

Corrupt journal records found in '/usr/local/AMQ/data/kaha/queue#3a#2f#2f/db-7.log

So, I removed the /kaha/queue#3a#2f#2f directory and restarted w/o error...other queues were there as expected...

here is the full stacktrace...

2014-03-27 09:20:28,588 | INFO  | Corrupt journal records found in '/usr/local/AMQ/data/kaha/queue#3a#2f#2f/db-7.log' between offsets: 7406256..7406302 | org.apache.activemq.store.kahadb.disk.journal.Journal | main
2014-03-27 09:20:28,593 | INFO  | KahaDB is version 5 | org.apache.activemq.store.kahadb.MessageDatabase | main
2014-03-27 09:20:28,595 | WARN  | Cannot recover ackMessageFileMap | org.apache.activemq.store.kahadb.MessageDatabase | main
java.io.StreamCorruptedException: invalid type code: 00
        at java.io.ObjectInputStream$BlockDataInputStream.readBlockHeader(ObjectInputStream.java:2486)
        at java.io.ObjectInputStream$BlockDataInputStream.refill(ObjectInputStream.java:2521)
        at java.io.ObjectInputStream$BlockDataInputStream.skipBlockData(ObjectInputStream.java:2423)
        at java.io.ObjectInputStream.skipCustomData(ObjectInputStream.java:1919)
        at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1597)
        at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1494)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1748)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1327)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:349)
        at org.apache.activemq.store.kahadb.MessageDatabase.recoverAckMessageFileMap(MessageDatabase.java:685)
        at org.apache.activemq.store.kahadb.MessageDatabase.recover(MessageDatabase.java:573)
        at org.apache.activemq.store.kahadb.MessageDatabase.open(MessageDatabase.java:389)
        at org.apache.activemq.store.kahadb.MessageDatabase.load(MessageDatabase.java:407)
        at org.apache.activemq.store.kahadb.MessageDatabase.doStart(MessageDatabase.java:260)
        at org.apache.activemq.store.kahadb.KahaDBStore.doStart(KahaDBStore.java:194)
        at org.apache.activemq.util.ServiceSupport.start(ServiceSupport.java:55)
        at org.apache.activemq.store.kahadb.KahaDBPersistenceAdapter.doStart(KahaDBPersistenceAdapter.java:215)
        at org.apache.activemq.util.ServiceSupport.start(ServiceSupport.java:55)
        at org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter.startAdapter(MultiKahaDBPersistenceAdapter.java:179)
        at org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter.registerExistingAdapter(MultiKahaDBPersistenceAdapter.java:351)
        at org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter.findAndRegisterExistingAdapters(MultiKahaDBPersistenceAdapter.java:344)
        at org.apache.activemq.store.kahadb.MultiKahaDBPersistenceAdapter.doStart(MultiKahaDBPersistenceAdapter.java:326)
        at org.apache.activemq.util.ServiceSupport.start(ServiceSupport.java:55)
        at org.apache.activemq.broker.BrokerService.doStartPersistenceAdapter(BrokerService.java:613)
        at org.apache.activemq.broker.BrokerService.startPersistenceAdapter(BrokerService.java:602)
        at org.apache.activemq.broker.BrokerService.start(BrokerService.java:567)
        at org.apache.activemq.xbean.XBeanBrokerService.afterPropertiesSet(XBeanBrokerService.java:73)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1608)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1549)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1479)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:521)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:458)
        at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:295)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:223)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:292)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:628)
        at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:932)
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:479)
        at org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:64)
        at org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:52)
        at org.apache.activemq.xbean.XBeanBrokerFactory$1.<init>(XBeanBrokerFactory.java:104)
        at org.apache.activemq.xbean.XBeanBrokerFactory.createApplicationContext(XBeanBrokerFactory.java:104)
        at org.apache.activemq.xbean.XBeanBrokerFactory.createBroker(XBeanBrokerFactory.java:67)
        at org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:71)
        at org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:54)
        at org.apache.activemq.console.command.StartCommand.runTask(StartCommand.java:87)
        at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57)
        at org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:150)
        at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57)
        at org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:104)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.activemq.console.Main.runTaskClass(Main.java:262)
        at org.apache.activemq.console.Main.main(Main.java:115)



boday wrote
any other thoughts on this?  we have seen this several times now when we restart AMQ (using mkahadb) with messages in the queues.  we have had to archive the entire data directory to restart...any other way to recover this data (manually moving files back in or something)? otherwise, these queues/messages are orphaned/lost
Reply | Threaded
Open this post in threaded view
|

Re: mkahadb: Failed to start per destination persistence adapter for destination

boday
FYI, I was able to reproduce this issue…its because a queue with no name was created when using JMX to send messages in between queues...I'll log a ticket with more info...
Reply | Threaded
Open this post in threaded view
|

Re: mkahadb: Failed to start per destination persistence adapter for destination

gtully
Great news ;-)
On 2 Apr 2014 22:20, "boday" <[hidden email]> wrote:

> FYI, I was able to reproduce this issue...its because a queue with no name
> was
> created when using JMX to send messages in between queues...I'll log a
> ticket with more info...
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/mkahadb-Failed-to-start-per-destination-persistence-adapter-for-destination-tp4678754p4679848.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: mkahadb: Failed to start per destination persistence adapter for destination

boday
In reply to this post by boday
logged with a possible patch: https://issues.apache.org/jira/browse/AMQ-5211

boday wrote
FYI, I was able to reproduce this issue…its because a queue with no name was created when using JMX to send messages in between queues...I'll log a ticket with more info...