How to access KahaDB db.data in activeMQ-5.15.0 ?

classic Classic list List threaded Threaded
29 messages Options
12
Reply | Threaded
Open this post in threaded view
|

How to access KahaDB db.data in activeMQ-5.15.0 ?

skao
Hi All,

I would like to view the KahaDB db.data to check the detail of enqueue
message body.
But it is binary file format.

Is there any tool to open the KahaDB db.data file or how to convert it to
txt file ?

Please kindly help.

Sincerely,
Samuel




--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: How to access KahaDB db.data in activeMQ-5.15.0 ?

Francois Papon
Hi,

You can use the Webconsole to show the messages.

regards,

François
[hidden email]

Le 20/05/2020 à 07:39, skao a écrit :

> Hi All,
>
> I would like to view the KahaDB db.data to check the detail of enqueue
> message body.
> But it is binary file format.
>
> Is there any tool to open the KahaDB db.data file or how to convert it to
> txt file ?
>
> Please kindly help.
>
> Sincerely,
> Samuel
>
>
>
>
> --
> Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: How to access KahaDB db.data in activeMQ-5.15.0 ?

jbonofre
In reply to this post by skao
Hi,

You can:

1. Use JMX to browse the queue (directly or via REST thanks to Jolokia)
2. Use admin console
3. Use any client (including REST) to browse
4. Use jms:* command in Karaf
5. Dump kahadb or browse content using bin/activemq (browse, …) script

Generally speaking, we never manipulate kahadb directly as it’s a specific database we did for ActiveMQ (so not "human readable").

Regards
JB

> Le 20 mai 2020 à 07:39, skao <[hidden email]> a écrit :
>
> Hi All,
>
> I would like to view the KahaDB db.data to check the detail of enqueue
> message body.
> But it is binary file format.
>
> Is there any tool to open the KahaDB db.data file or how to convert it to
> txt file ?
>
> Please kindly help.
>
> Sincerely,
> Samuel
>
>
>
>
> --
> Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html

Reply | Threaded
Open this post in threaded view
|

Re: How to access KahaDB db.data in activeMQ-5.15.0 ?

gtully
there is: https://github.com/apache/activemq-cli-tools/tree/master/activemq-kahadb-exporter

On Wed, 20 May 2020 at 10:16, Jean-Baptiste Onofre <[hidden email]> wrote:

>
> Hi,
>
> You can:
>
> 1. Use JMX to browse the queue (directly or via REST thanks to Jolokia)
> 2. Use admin console
> 3. Use any client (including REST) to browse
> 4. Use jms:* command in Karaf
> 5. Dump kahadb or browse content using bin/activemq (browse, …) script
>
> Generally speaking, we never manipulate kahadb directly as it’s a specific database we did for ActiveMQ (so not "human readable").
>
> Regards
> JB
>
> > Le 20 mai 2020 à 07:39, skao <[hidden email]> a écrit :
> >
> > Hi All,
> >
> > I would like to view the KahaDB db.data to check the detail of enqueue
> > message body.
> > But it is binary file format.
> >
> > Is there any tool to open the KahaDB db.data file or how to convert it to
> > txt file ?
> >
> > Please kindly help.
> >
> > Sincerely,
> > Samuel
> >
> >
> >
> >
> > --
> > Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
>
Reply | Threaded
Open this post in threaded view
|

Re: How to access KahaDB db.data in activeMQ-5.15.0 ?

skao
In reply to this post by jbonofre
Hi JB,

Thanks a lot for your help.
Could you be more specific on these steps below ?
For instance, how to use the bin/activemq script ?

Thansk,
Samuel

1. Use JMX to browse the queue (directly or via REST thanks to Jolokia)
2. Use admin console
3. Use any client (including REST) to browse
4. Use jms:* command in Karaf
5. Dump kahadb or browse content using bin/activemq (browse, …) script



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: How to access KahaDB db.data in activeMQ-5.15.0 ?

jbonofre
Hi,

It’s not really steps, it’s more "alternative".

For instance, you can use bin/activemq browse.

Regards
JB

> Le 20 mai 2020 à 16:36, skao <[hidden email]> a écrit :
>
> Hi JB,
>
> Thanks a lot for your help.
> Could you be more specific on these steps below ?
> For instance, how to use the bin/activemq script ?
>
> Thansk,
> Samuel
>
> 1. Use JMX to browse the queue (directly or via REST thanks to Jolokia)
> 2. Use admin console
> 3. Use any client (including REST) to browse
> 4. Use jms:* command in Karaf
> 5. Dump kahadb or browse content using bin/activemq (browse, …) script
>
>
>
> --
> Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html

Reply | Threaded
Open this post in threaded view
|

Re: How to access KahaDB db.data in activeMQ-5.15.0 ?

skao
Hi JB,

I never used bin/activemq browse before but I could Google it a little bit.

Thanks a lot.

Regards,
Samuel



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: How to access KahaDB db.data in activeMQ-5.15.0 ?

jbonofre
Hi,

Basically you can do bin/activemq browse FOO

Where FOO is the destination name.

Regards
JB

> Le 20 mai 2020 à 18:16, skao <[hidden email]> a écrit :
>
> Hi JB,
>
> I never used bin/activemq browse before but I could Google it a little bit.
>
> Thanks a lot.
>
> Regards,
> Samuel
>
>
>
> --
> Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html

Reply | Threaded
Open this post in threaded view
|

Re: How to access KahaDB db.data in activeMQ-5.15.0 ?

skao
Hi JB,

This is what I did in the Linux.
Did I do something wrong to view the db.data file ?

Please kindly advise.

Regards,
Samuel

cd /home/jboss/apache-activemq-5.15.0/bin

$ *./activemq browse /hom/jboss/apache-activemq-5.15.0/data/kahadb/db.data*
INFO: Loading '/home/jboss/apache-activemq-5.15.0//bin/env'
INFO: Using java '/usr/java/jdk1.8.0_144/bin/java'
Java Runtime: Oracle Corporation 1.8.0_144 /usr/java/jdk1.8.0_144/jre
  Heap sizes: current=62976k  free=61992k  max=932352k
    JVM args: -Xms64M -Xmx1G
-Djava.util.logging.config.file=logging.properties
-Djava.security.auth.login.config=/home/jboss/apache-activemq-5.15.0//conf/login.config
-Dactivemq.classpath=/home/jboss/apache-activemq-5.15.0//conf:/home/jboss/apache-activemq-5.15.0//../lib/:
-Dactivemq.home=/home/jboss/apache-activemq-5.15.0/
-Dactivemq.base=/home/jboss/apache-activemq-5.15.0/
-Dactivemq.conf=/home/jboss/apache-activemq-5.15.0//conf
-Dactivemq.data=/home/jboss/apache-activemq-5.15.0//data
Extensions classpath:
 
[/home/jboss/apache-activemq-5.15.0/lib,/home/jboss/apache-activemq-5.15.0/lib/camel,/home/jboss/apache-activemq-5.15.0/lib/optional,/home/jboss/apache-activemq-5.15.0/lib/web,/home/jboss/apache-activemq-5.15.0/lib/extra]
ACTIVEMQ_HOME: /home/jboss/apache-activemq-5.15.0
ACTIVEMQ_BASE: /home/jboss/apache-activemq-5.15.0
ACTIVEMQ_CONF: /home/jboss/apache-activemq-5.15.0/conf
ACTIVEMQ_DATA: /home/jboss/apache-activemq-5.15.0/data




--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: How to access KahaDB db.data in activeMQ-5.15.0 ?

skao
Got error when I tried to browse the queue in activeMQ.

activemq browse --amqurl tcp://xxxxxxxxxxxxx:61616 MESF_TO_MII_CFR

INFO: Loading '/home/jboss/apache-activemq-5.15.0//bin/env'
INFO: Using java '/usr/java/jdk1.8.0_144/bin/java'
Java Runtime: Oracle Corporation 1.8.0_144 /usr/java/jdk1.8.0_144/jre
  Heap sizes: current=62976k  free=61992k  max=932352k
    JVM args: -Xms64M -Xmx1G
-Djava.util.logging.config.file=logging.properties
-Djava.security.auth.login.config=/home/jboss/apache-activemq-5.15.0//conf/login.config
-Dactivemq.classpath=/home/jboss/apache-activemq-5.15.0//conf:/home/jboss/apache-activemq-5.15.0//../lib/:
-Dactivemq.home=/home/jboss/apache-activemq-5.15.0/
-Dactivemq.base=/home/jboss/apache-activemq-5.15.0/
-Dactivemq.conf=/home/jboss/apache-activemq-5.15.0//conf
-Dactivemq.data=/home/jboss/apache-activemq-5.15.0//data
Extensions classpath:
 
[/home/jboss/apache-activemq-5.15.0/lib,/home/jboss/apache-activemq-5.15.0/lib/camel,/home/jboss/apache-activemq-5.15.0/lib/optional,/home/jboss/apache-activemq-5.15.0/lib/web,/home/jboss/apache-activemq-5.15.0/lib/extra]
ACTIVEMQ_HOME: /home/jboss/apache-activemq-5.15.0
ACTIVEMQ_BASE: /home/jboss/apache-activemq-5.15.0
ACTIVEMQ_CONF: /home/jboss/apache-activemq-5.15.0/conf
ACTIVEMQ_DATA: /home/jboss/apache-activemq-5.15.0/data
INFO: Failed to execute browse task.
ERROR: java.lang.reflect.InvocationTargetException
java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at
org.apache.activemq.console.filter.MapTransformFilter.transformElement(MapTransformFilter.java:68)
        at
org.apache.activemq.console.filter.ResultTransformFilter.transformList(ResultTransformFilter.java:67)
        at
org.apache.activemq.console.filter.ResultTransformFilter.query(ResultTransformFilter.java:43)
        at
org.apache.activemq.console.filter.PropertiesViewFilter.query(PropertiesViewFilter.java:52)
        at
org.apache.activemq.console.filter.PropertiesViewFilter.query(PropertiesViewFilter.java:52)
        at
org.apache.activemq.console.util.AmqMessagesUtil.filterMessagesView(AmqMessagesUtil.java:64)
        at
org.apache.activemq.console.command.AmqBrowseCommand.runTask(AmqBrowseCommand.java:151)
        at
org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:63)
        at
org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:154)
        at
org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:63)
        at
org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:104)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.activemq.console.Main.runTaskClass(Main.java:262)
        at org.apache.activemq.console.Main.main(Main.java:115)
Caused by: javax.jms.JMSException: Failed to build body from content.
Serializable class not available to broker. Reason:
java.lang.ClassNotFoundException:
com.rockwell.integration.messaging.MessageEnvelope
        at
org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:36)
        at
org.apache.activemq.command.ActiveMQObjectMessage.getObject(ActiveMQObjectMessage.java:208)
        at
org.apache.activemq.console.filter.MapTransformFilter.transformToMap(MapTransformFilter.java:188)
        ... 21 more
Caused by: java.lang.ClassNotFoundException:
com.rockwell.integration.messaging.MessageEnvelope
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:348)
        at
org.apache.activemq.util.ClassLoadingAwareObjectInputStream.load(ClassLoadingAwareObjectInputStream.java:142)
        at
org.apache.activemq.util.ClassLoadingAwareObjectInputStream.resolveClass(ClassLoadingAwareObjectInputStream.java:56)
        at
java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1826)
        at
java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713)
        at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2000)
        at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
        at
org.apache.activemq.command.ActiveMQObjectMessage.getObject(ActiveMQObjectMessage.java:206)
        ... 22 more



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: How to access KahaDB db.data in activeMQ-5.15.0 ?

skao
In reply to this post by Francois Papon
Hi François,

Are you talking about the ActiveMQ console ?
If not, could you please explain the detail of how to use Webconsole t open
the KahaDB db.data file ?

Thansk,
Simon



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: How to access KahaDB db.data in activeMQ-5.15.0 ?

jbonofre
In reply to this post by skao
ActiveMQ browse doesn’t take kahadb location as argument but directly the destination:

./bin/activemq browse FOO

If you want to extract/dump kahadb, use the kahadb exporter:

./bin/export kahadb --source /some/directory/kahadb/ --target ~/some/directory/output.xml

What’s your use case exactly ?

Regards
JB

> Le 21 mai 2020 à 00:14, skao <[hidden email]> a écrit :
>
> Hi JB,
>
> This is what I did in the Linux.
> Did I do something wrong to view the db.data file ?
>
> Please kindly advise.
>
> Regards,
> Samuel
>
> cd /home/jboss/apache-activemq-5.15.0/bin
>
> $ *./activemq browse /hom/jboss/apache-activemq-5.15.0/data/kahadb/db.data*
> INFO: Loading '/home/jboss/apache-activemq-5.15.0//bin/env'
> INFO: Using java '/usr/java/jdk1.8.0_144/bin/java'
> Java Runtime: Oracle Corporation 1.8.0_144 /usr/java/jdk1.8.0_144/jre
>  Heap sizes: current=62976k  free=61992k  max=932352k
>    JVM args: -Xms64M -Xmx1G
> -Djava.util.logging.config.file=logging.properties
> -Djava.security.auth.login.config=/home/jboss/apache-activemq-5.15.0//conf/login.config
> -Dactivemq.classpath=/home/jboss/apache-activemq-5.15.0//conf:/home/jboss/apache-activemq-5.15.0//../lib/:
> -Dactivemq.home=/home/jboss/apache-activemq-5.15.0/
> -Dactivemq.base=/home/jboss/apache-activemq-5.15.0/
> -Dactivemq.conf=/home/jboss/apache-activemq-5.15.0//conf
> -Dactivemq.data=/home/jboss/apache-activemq-5.15.0//data
> Extensions classpath:
>
> [/home/jboss/apache-activemq-5.15.0/lib,/home/jboss/apache-activemq-5.15.0/lib/camel,/home/jboss/apache-activemq-5.15.0/lib/optional,/home/jboss/apache-activemq-5.15.0/lib/web,/home/jboss/apache-activemq-5.15.0/lib/extra]
> ACTIVEMQ_HOME: /home/jboss/apache-activemq-5.15.0
> ACTIVEMQ_BASE: /home/jboss/apache-activemq-5.15.0
> ACTIVEMQ_CONF: /home/jboss/apache-activemq-5.15.0/conf
> ACTIVEMQ_DATA: /home/jboss/apache-activemq-5.15.0/data
>
>
>
>
> --
> Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html

Reply | Threaded
Open this post in threaded view
|

Re: How to access KahaDB db.data in activeMQ-5.15.0 ?

skao
Hi JB,

Thanks a lot for your help
.
Let me try this one below.
./bin/export kahadb --source /some/directory/kahadb/ --target
~/some/directory/output.xml

Basically I would like to see the message body from activeMQ in kahadb
folder and see the detail Info of each message but they are in binary file
format so hope this output.xml file can show me the detail.


Thanks,
Samuel







--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: How to access KahaDB db.data in activeMQ-5.15.0 ?

jbonofre
OK, so, regarding your use case, I don’t think it’s a good idea or require to use the exporter.

Just use browse or the web console (http://host:8161/admin <http://host:8161/admin> by default).

You can see all details about messages per destination, if they are persistent or not (so in kahadb or not), etc.

That’s a better approach IMHO.

Regards
JB

> Le 21 mai 2020 à 06:37, skao <[hidden email]> a écrit :
>
> Hi JB,
>
> Thanks a lot for your help
> .
> Let me try this one below.
> ./bin/export kahadb --source /some/directory/kahadb/ --target
> ~/some/directory/output.xml
>
> Basically I would like to see the message body from activeMQ in kahadb
> folder and see the detail Info of each message but they are in binary file
> format so hope this output.xml file can show me the detail.
>
>
> Thanks,
> Samuel
>
>
>
>
>
>
>
> --
> Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html

Reply | Threaded
Open this post in threaded view
|

Re: How to access KahaDB db.data in activeMQ-5.15.0 ?

skao
Hi JB,

I would like to see all messages in activeMQ even though they have been
de-queued by consumers already.
Our users kept saying that they received duplicate messaged when they
de-queued the messages.
I would like to see all messages to confirm it.

Will the activeMQ console still kept those de-queued messages after the
messages have been de-queued ?

Thanks a lot for your outstanding help.

Regards,
Samuel



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: How to access KahaDB db.data in activeMQ-5.15.0 ?

jbonofre
Hi,

You should understand better how ActiveMQ and JMS works.

Once a consumer consumed a message, the message is not in KahaDB anymore.

Regards
JB

> Le 21 mai 2020 à 07:17, skao <[hidden email]> a écrit :
>
> Hi JB,
>
> I would like to see all messages in activeMQ even though they have been
> de-queued by consumers already.
> Our users kept saying that they received duplicate messaged when they
> de-queued the messages.
> I would like to see all messages to confirm it.
>
> Will the activeMQ console still kept those de-queued messages after the
> messages have been de-queued ?
>
> Thanks a lot for your outstanding help.
>
> Regards,
> Samuel
>
>
>
> --
> Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html

Reply | Threaded
Open this post in threaded view
|

Re: How to access KahaDB db.data in activeMQ-5.15.0 ?

skao
Hi JB,

That is bad news for me.

The only thing that we can do maybe is the mirror queued in activeMQ.

Let me see if I can make it working.

Anyway, thanks a lot for your help.

Regards,
Samuel



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: How to access KahaDB db.data in activeMQ-5.15.0 ?

jbonofre
In reply to this post by skao
By the way, if you are using queue, a message is dispatched to one and only one consumer (that’s different for topic).

So, your setup is probably not what you want. I would review your use case and setup.

Regards
JB

> Le 21 mai 2020 à 07:17, skao <[hidden email]> a écrit :
>
> Hi JB,
>
> I would like to see all messages in activeMQ even though they have been
> de-queued by consumers already.
> Our users kept saying that they received duplicate messaged when they
> de-queued the messages.
> I would like to see all messages to confirm it.
>
> Will the activeMQ console still kept those de-queued messages after the
> messages have been de-queued ?
>
> Thanks a lot for your outstanding help.
>
> Regards,
> Samuel
>
>
>
> --
> Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html

Reply | Threaded
Open this post in threaded view
|

Re: How to access KahaDB db.data in activeMQ-5.15.0 ?

Francois Papon
In reply to this post by skao
Hi,

Yes I was talking about the Webconsole.

You just have to configure your ActiveMQ instance to the kahadb location
in the conf/activemq.xml

regards,

François
[hidden email]

Le 21/05/2020 à 04:47, skao a écrit :

> Hi François,
>
> Are you talking about the ActiveMQ console ?
> If not, could you please explain the detail of how to use Webconsole t open
> the KahaDB db.data file ?
>
> Thansk,
> Simon
>
>
>
> --
> Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: How to access KahaDB db.data in activeMQ-5.15.0 ?

skao
Hi François,

Thanks a lot for your reply.

Like JB mention earlier, if the consumer de-queue the message and then it
won't show up in kahaDB.
Then there is no way for me to see the hahadb file to review the en-queue
messages.

I would like to see all messages even though they have been de-queued by
consumers.

Is that possible ?

Thanks,
Samuel



>>Hi,

>>You should understand better how ActiveMQ and JMS works.

>>Once a consumer consumed a message, the message is not in KahaDB anymore.

>>Regards
>>JB



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
12