[activemq-user] Reusing ActiveMQ filter

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

[activemq-user] Reusing ActiveMQ filter

colincrist

Hi,

I've added selector support to Hermes, not only on destinations but on
the message store too. This means you can search for messages using a
filter on the entire store or on messages originating from a particular
queue or topic.

For the message store I have two implementations, one is based on a
thing called JAMSEL which is a soon to be released standalone opensource
JMS selector and the other implementation uses org.activemq.filter.

It *seems* to work OK, I've only done a few simple tests with Tibco EMS
as the message factory so far - can anyone confirm that this is good to
work with messages created from other JMS providers?

I think it will be quite nice to be able to use the Hermes GUI or ant
tasks to dump messages from a load of queues or topics into a database
and then run a selector accross the whole store of messages searching
not only for header properties but XPath expressions or MapMessage
properties. Of course you'll also be able to replay the messages that
match this filter too giving some interestng options.

Regards,

Colin.

Reply | Threaded
Open this post in threaded view
|

[activemq-user] Wonder, why would two clients cannot exchange messages in the following cluster configuration?

gklyuzner
Wonder, why would two clients cannot exchange messages in the following cluster configuration?
Multicast should not be an issue, since it uses hard coded URL.

Client 1 connected to nyotc030:
BrokerURL reliable:tcp://nyotc030:61616,tcp://nyotc029:61616

INFO 2005-08-31 15:10:22,873 - channel status changed: Channel: ReliableTransportChannel: TcpTransportChannel: Socket[addr=nyotc030/170.198.95.219,port=61616,localport=2643] has connected
I
Client 2 connected to nyotc029:
BrokerURL reliable:tcp://nyotc030:61616,tcp://nyotc029:61616

NFO 2005-08-31 15:09:44,779 - channel status changed: Channel: ReliableTransportChannel: TcpTransportChannel: Socket[addr=nyotc029/170.198.95.212,port=61616,localport=2640] has connected
I

Broker on nyotc029

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE beans PUBLIC "-//ACTIVEMQ//DTD//EN" "http://activemq.org/dtd/activemq.dtd">
<beans>
        <!-- ==================================================================== -->
        <!-- ActiveMQ Broker Configuration -->
        <!-- ==================================================================== -->
        <broker name="nyotc030-broker-1-M6" maxMem="41943040">
                <connector>
                        <tcpServerTransport uri="tcp://nyotc030:61616" backlog="1000" useAsyncSend="true" maxOutstandingMessages="50"/>
                </connector>
    <networkConnector>
      <networkChannel uri="reliable:tcp://nyotc029:61616" remoteBrokerName="nyotc029-broker-1-M6" maximumRetries="1000" reconnectSleepTime="3000"/>
      <localPrefetchPolicy queuePrefetch="1" queueBrowserPrefetch="2"/>
      <remotePrefetchPolicy topicPrefetch="3" durableTopicPrefetch="4"/>
    </networkConnector>

    <persistence>
      <vmPersistence/>
    </persistence>

</broker>
Broker on nyotc029

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE beans PUBLIC "-//ACTIVEMQ//DTD//EN" "http://activemq.org/dtd/activemq.dtd">
<beans>
        <!-- ==================================================================== -->
        <!-- ActiveMQ Broker Configuration -->
        <!-- ==================================================================== -->
        <broker name="nyotc029-broker-1-M6" maxMem="41943040">
        <connector>
        <tcpServerTransport uri="tcp://nyotc029:61616" backlog="1000" useAsyncSend="true" maxOutstandingMessages="50"/>
        </connector>
<!-- Cluster of TWO Brokers, each runs on own host -->
    <networkConnector>
      <networkChannel uri="reliable:tcp://nyotc030:61616" remoteBrokerName="nyotc030-broker-1-M6" maximumRetries="1000" reconnectSleepTime="5000"/>

      <localPrefetchPolicy queuePrefetch="1" queueBrowserPrefetch="2"/>
      <remotePrefetchPolicy topicPrefetch="3" durableTopicPrefetch="4"/>
    </networkConnector>

    <persistence>
      <vmPersistence/>
    </persistence>
</broker>
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
The information contained in this email is confidential and may also contain privileged information. Sender does not waive confidentiality or legal privilege. If you are not the intended recipient please notify the sender immediately; you should not retain this message or disclose its content to anyone.
Internet communications are not secure or error free and the sender does not accept any liability for the content of the email. Although emails are routinely screened for viruses, the sender does not accept responsibility for any damage caused. Replies to this email may be monitored.
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Reply | Threaded
Open this post in threaded view
|

RE: [activemq-user] Wonder, why would two clients cannot exchange messages in the following cluster configuration?

Joe Barefoot
Very strange.  I don't have any ideas as to the cause, but I would try
removing all non-required attributes, options, etc., to see if any of
those are causing the problem.  If the bare-bones config works you can
add back options one at a time to see what breaks it.  i.e. start like
this:


Broker on nyotc029

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE beans PUBLIC "-//ACTIVEMQ//DTD//EN"
"http://activemq.org/dtd/activemq.dtd">
<beans>
        <!--
==================================================================== -->
        <!-- ActiveMQ Broker Configuration -->
        <!--
==================================================================== -->
        <broker>
                <connector>
                        <tcpServerTransport uri="tcp://nyotc030:61616"
/>
                </connector>
    <networkConnector>
      <networkChannel uri="reliable:tcp://nyotc029:61616" />    
    </networkConnector>

    <persistence>
      <vmPersistence/>
    </persistence>

</broker>

Broker on nyotc029

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE beans PUBLIC "-//ACTIVEMQ//DTD//EN"
"http://activemq.org/dtd/activemq.dtd">
<beans>
        <!--
==================================================================== -->
        <!-- ActiveMQ Broker Configuration -->
        <!--
==================================================================== -->
        <broker>
        <connector>
        <tcpServerTransport uri="tcp://nyotc029:61616"/>
        </connector>
<!-- Cluster of TWO Brokers, each runs on own host -->
    <networkConnector>
      <networkChannel uri="reliable:tcp://nyotc030:61616" />  
    </networkConnector>

    <persistence>
      <vmPersistence/>
    </persistence>
</broker>

-----Original Message-----
From: Klyuzner, Glen [mailto:[hidden email]]
Sent: Wednesday, August 31, 2005 12:48 PM
To: [hidden email]
Subject: [activemq-user] Wonder, why would two clients cannot exchange
messages in the following cluster configuration?

Wonder, why would two clients cannot exchange messages in the following
cluster configuration?
Multicast should not be an issue, since it uses hard coded URL.

Client 1 connected to nyotc030:
BrokerURL reliable:tcp://nyotc030:61616,tcp://nyotc029:61616

INFO 2005-08-31 15:10:22,873 - channel status changed: Channel:
ReliableTransportChannel: TcpTransportChannel:
Socket[addr=nyotc030/170.198.95.219,port=61616,localport=2643] has
connected I Client 2 connected to nyotc029:
BrokerURL reliable:tcp://nyotc030:61616,tcp://nyotc029:61616

NFO 2005-08-31 15:09:44,779 - channel status changed: Channel:
ReliableTransportChannel: TcpTransportChannel:
Socket[addr=nyotc029/170.198.95.212,port=61616,localport=2640] has
connected I

Broker on nyotc029

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE beans PUBLIC "-//ACTIVEMQ//DTD//EN"
"http://activemq.org/dtd/activemq.dtd">
<beans>
        <!--
==================================================================== -->
        <!-- ActiveMQ Broker Configuration -->
        <!--
==================================================================== -->
        <broker name="nyotc030-broker-1-M6" maxMem="41943040">
                <connector>
                        <tcpServerTransport uri="tcp://nyotc030:61616"
backlog="1000" useAsyncSend="true" maxOutstandingMessages="50"/>
                </connector>
    <networkConnector>
      <networkChannel uri="reliable:tcp://nyotc029:61616"
remoteBrokerName="nyotc029-broker-1-M6" maximumRetries="1000"
reconnectSleepTime="3000"/>
      <localPrefetchPolicy queuePrefetch="1" queueBrowserPrefetch="2"/>
      <remotePrefetchPolicy topicPrefetch="3" durableTopicPrefetch="4"/>
    </networkConnector>

    <persistence>
      <vmPersistence/>
    </persistence>

</broker>
Broker on nyotc029

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE beans PUBLIC "-//ACTIVEMQ//DTD//EN"
"http://activemq.org/dtd/activemq.dtd">
<beans>
        <!--
==================================================================== -->
        <!-- ActiveMQ Broker Configuration -->
        <!--
==================================================================== -->
        <broker name="nyotc029-broker-1-M6" maxMem="41943040">
        <connector>
        <tcpServerTransport uri="tcp://nyotc029:61616" backlog="1000"
useAsyncSend="true" maxOutstandingMessages="50"/>
        </connector>
<!-- Cluster of TWO Brokers, each runs on own host -->
    <networkConnector>
      <networkChannel uri="reliable:tcp://nyotc030:61616"
remoteBrokerName="nyotc030-broker-1-M6" maximumRetries="1000"
reconnectSleepTime="5000"/>

      <localPrefetchPolicy queuePrefetch="1" queueBrowserPrefetch="2"/>
      <remotePrefetchPolicy topicPrefetch="3" durableTopicPrefetch="4"/>
    </networkConnector>

    <persistence>
      <vmPersistence/>
    </persistence>
</broker>
------------------------------------------------------------------------
------------------------------------------------------------------------
---------------------------------
The information contained in this email is confidential and may also
contain privileged information. Sender does not waive confidentiality or
legal privilege. If you are not the intended recipient please notify the
sender immediately; you should not retain this message or disclose its
content to anyone.
Internet communications are not secure or error free and the sender does
not accept any liability for the content of the email. Although emails
are routinely screened for viruses, the sender does not accept
responsibility for any damage caused. Replies to this email may be
monitored.
------------------------------------------------------------------------
------------------------------------------------------------------------
---------------------------------
Reply | Threaded
Open this post in threaded view
|

RE: [activemq-user] Wonder, why would two clients cannot exchange messages in the following cluster configuration?

Mark Lewis
I've got a similar problem (well, started with a 4-broker solution, but
that didn't work so I tried a simple 2-broker solution).  Just used a
default config xml, the only thing added was each broker had a reliable:
network channel to the other.  They still weren't passing messages back
and forth if the producer connected to one broker and the consumer
connected to the other.  It seems like persistent messages just don't
work across network of brokers.

Is there ANYBODY out there who has gotten persistent messaging working
across a network of brokers?  If so, could you share your configs?

-- Mark

On Wed, 2005-08-31 at 18:32 -0400, Joe Barefoot wrote:

> Very strange.  I don't have any ideas as to the cause, but I would try
> removing all non-required attributes, options, etc., to see if any of
> those are causing the problem.  If the bare-bones config works you can
> add back options one at a time to see what breaks it.  i.e. start like
> this:
>
>
> Broker on nyotc029
>
> <?xml version="1.0" encoding="UTF-8"?>
> <!DOCTYPE beans PUBLIC "-//ACTIVEMQ//DTD//EN"
> "http://activemq.org/dtd/activemq.dtd">
> <beans>
> <!--
> ==================================================================== -->
> <!-- ActiveMQ Broker Configuration -->
> <!--
> ==================================================================== -->
> <broker>
> <connector>
> <tcpServerTransport uri="tcp://nyotc030:61616"
> />
> </connector>
>     <networkConnector>
>       <networkChannel uri="reliable:tcp://nyotc029:61616" />    
>     </networkConnector>
>
>     <persistence>
>       <vmPersistence/>
>     </persistence>
>
> </broker>
>
> Broker on nyotc029
>
> <?xml version="1.0" encoding="UTF-8"?>
> <!DOCTYPE beans PUBLIC "-//ACTIVEMQ//DTD//EN"
> "http://activemq.org/dtd/activemq.dtd">
> <beans>
> <!--
> ==================================================================== -->
> <!-- ActiveMQ Broker Configuration -->
> <!--
> ==================================================================== -->
> <broker>
> <connector>
> <tcpServerTransport uri="tcp://nyotc029:61616"/>
> </connector>
> <!-- Cluster of TWO Brokers, each runs on own host -->
>     <networkConnector>
>       <networkChannel uri="reliable:tcp://nyotc030:61616" />  
>     </networkConnector>
>
>     <persistence>
>       <vmPersistence/>
>     </persistence>
> </broker>
>
> -----Original Message-----
> From: Klyuzner, Glen [mailto:[hidden email]]
> Sent: Wednesday, August 31, 2005 12:48 PM
> To: [hidden email]
> Subject: [activemq-user] Wonder, why would two clients cannot exchange
> messages in the following cluster configuration?
>
> Wonder, why would two clients cannot exchange messages in the following
> cluster configuration?
> Multicast should not be an issue, since it uses hard coded URL.
>
> Client 1 connected to nyotc030:
> BrokerURL reliable:tcp://nyotc030:61616,tcp://nyotc029:61616
>
> INFO 2005-08-31 15:10:22,873 - channel status changed: Channel:
> ReliableTransportChannel: TcpTransportChannel:
> Socket[addr=nyotc030/170.198.95.219,port=61616,localport=2643] has
> connected I Client 2 connected to nyotc029:
> BrokerURL reliable:tcp://nyotc030:61616,tcp://nyotc029:61616
>
> NFO 2005-08-31 15:09:44,779 - channel status changed: Channel:
> ReliableTransportChannel: TcpTransportChannel:
> Socket[addr=nyotc029/170.198.95.212,port=61616,localport=2640] has
> connected I
>
> Broker on nyotc029
>
> <?xml version="1.0" encoding="UTF-8"?>
> <!DOCTYPE beans PUBLIC "-//ACTIVEMQ//DTD//EN"
> "http://activemq.org/dtd/activemq.dtd">
> <beans>
> <!--
> ==================================================================== -->
> <!-- ActiveMQ Broker Configuration -->
> <!--
> ==================================================================== -->
> <broker name="nyotc030-broker-1-M6" maxMem="41943040">
> <connector>
> <tcpServerTransport uri="tcp://nyotc030:61616"
> backlog="1000" useAsyncSend="true" maxOutstandingMessages="50"/>
> </connector>
>     <networkConnector>
>       <networkChannel uri="reliable:tcp://nyotc029:61616"
> remoteBrokerName="nyotc029-broker-1-M6" maximumRetries="1000"
> reconnectSleepTime="3000"/>
>       <localPrefetchPolicy queuePrefetch="1" queueBrowserPrefetch="2"/>
>       <remotePrefetchPolicy topicPrefetch="3" durableTopicPrefetch="4"/>
>     </networkConnector>
>
>     <persistence>
>       <vmPersistence/>
>     </persistence>
>
> </broker>
> Broker on nyotc029
>
> <?xml version="1.0" encoding="UTF-8"?>
> <!DOCTYPE beans PUBLIC "-//ACTIVEMQ//DTD//EN"
> "http://activemq.org/dtd/activemq.dtd">
> <beans>
> <!--
> ==================================================================== -->
> <!-- ActiveMQ Broker Configuration -->
> <!--
> ==================================================================== -->
> <broker name="nyotc029-broker-1-M6" maxMem="41943040">
> <connector>
> <tcpServerTransport uri="tcp://nyotc029:61616" backlog="1000"
> useAsyncSend="true" maxOutstandingMessages="50"/>
> </connector>
> <!-- Cluster of TWO Brokers, each runs on own host -->
>     <networkConnector>
>       <networkChannel uri="reliable:tcp://nyotc030:61616"
> remoteBrokerName="nyotc030-broker-1-M6" maximumRetries="1000"
> reconnectSleepTime="5000"/>
>
>       <localPrefetchPolicy queuePrefetch="1" queueBrowserPrefetch="2"/>
>       <remotePrefetchPolicy topicPrefetch="3" durableTopicPrefetch="4"/>
>     </networkConnector>
>
>     <persistence>
>       <vmPersistence/>
>     </persistence>
> </broker>
> ------------------------------------------------------------------------
> ------------------------------------------------------------------------
> ---------------------------------
> The information contained in this email is confidential and may also
> contain privileged information. Sender does not waive confidentiality or
> legal privilege. If you are not the intended recipient please notify the
> sender immediately; you should not retain this message or disclose its
> content to anyone.
> Internet communications are not secure or error free and the sender does
> not accept any liability for the content of the email. Although emails
> are routinely screened for viruses, the sender does not accept
> responsibility for any damage caused. Replies to this email may be
> monitored.
> ------------------------------------------------------------------------
> ------------------------------------------------------------------------
> ---------------------------------

Reply | Threaded
Open this post in threaded view
|

[activemq-user] Re: Reusing ActiveMQ filter

James Strachan-2
In reply to this post by colincrist
Colin Crist wrote:
>
> Hi,
>
> I've added selector support to Hermes, not only on destinations but on
> the message store too. This means you can search for messages using a
> filter on the entire store or on messages originating from a particular
> queue or topic.

Cool!

> For the message store I have two implementations, one is based on a
> thing called JAMSEL which is a soon to be released standalone opensource
> JMS selector and the other implementation uses org.activemq.filter.
>
> It *seems* to work OK, I've only done a few simple tests with Tibco EMS
> as the message factory so far - can anyone confirm that this is good to
> work with messages created from other JMS providers?

I see no reason why org.activemq.filter can't work on any JMS provider.


> I think it will be quite nice to be able to use the Hermes GUI or ant
> tasks to dump messages from a load of queues or topics into a database
> and then run a selector accross the whole store of messages searching
> not only for header properties but XPath expressions or MapMessage
> properties. Of course you'll also be able to replay the messages that
> match this filter too giving some interestng options.

Sounds great. Let us know if there's anything we need to do to
org.activemq.filter to help out.

BTW did you manage to reuse the JDBC message store from ActiveMQ in
Hermes or was it a bit too 'implementation focussed' to be easily reusable?

--

James
-------
http://radio.weblogs.com/0112098/

Reply | Threaded
Open this post in threaded view
|

[activemq-user] Re: Reusing ActiveMQ filter

colincrist

Hey James,

> I see no reason why org.activemq.filter can't work on any JMS provider.

Excellent.

> Sounds great. Let us know if there's anything we need to do to
> org.activemq.filter to help out.

Here is my problem. Hermes loads providers up in separate class loaders
meaning the system classloader is clear of any JMS providers. I don't
want to have a version of ActiveMQ in the main classloader as it may
interfere with other versions loaded in different class loaders. I may
be able to bodge it by loading my selector wrapper code and activemq in
a completely different class loader but its just nasty and confusing as
hell. I've actually had about 8 different JMS providers accessed
concurrently from Hermes, including multiple version of JBoss and WMQ
which is handy, albeit for a small number of users. Its also good for
different versions of serialised objects (i.e. for object messages)
being on different message flows - e.g. dev/test/prod.

Suggestions on how I could get round this welcome. Putting just the
filter code in a new jar does not help either as it could affect
ActiveMQ loaded in other loaders too. Argghhh...

By get out is using JamSel but I'd like to support both as ActiveMQ
filters have some stuff not in JamSel selectors and vice-versa.

>
> BTW did you manage to reuse the JDBC message store from ActiveMQ in
> Hermes or was it a bit too 'implementation focussed' to be easily reusable?
>

I had to roll my own, the ActiveMQ store is too ActiveMQ focused so I
could not store other providers messages in it. I store each message as
an XML blob so it can be re-created with a different JMS provider to
where it came from originally - hence you can capture messages from say
ActiveMQ and them replay them to EMS or WMQ.

Cheers,

Colin.
http://www.hermesjms.com

Reply | Threaded
Open this post in threaded view
|

Re: [activemq-user] Re: Reusing ActiveMQ filter

James Strachan-2
On 1 Sep 2005, at 18:57, Colin Crist wrote:
> Hey James,
>
>
>> I see no reason why org.activemq.filter can't work on any JMS  
>> provider.
>>
>
> Excellent.

BTW we've not tested heavily on non-ActiveMQ messages but it should  
be trivial to patch if you hit an issue.


>> Sounds great. Let us know if there's anything we need to do to  
>> org.activemq.filter to help out.
>>
>
> Here is my problem. Hermes loads providers up in separate class  
> loaders meaning the system classloader is clear of any JMS  
> providers. I don't want to have a version of ActiveMQ in the main  
> classloader as it may interfere with other versions loaded in  
> different class loaders. I may be able to bodge it by loading my  
> selector wrapper code and activemq in a completely different class  
> loader but its just nasty and confusing as hell. I've actually had  
> about 8 different JMS providers accessed concurrently from Hermes,  
> including multiple version of JBoss and WMQ which is handy, albeit  
> for a small number of users. Its also good for different versions  
> of serialised objects (i.e. for object messages) being on different  
> message flows - e.g. dev/test/prod.
> Suggestions on how I could get round this welcome. Putting just the  
> filter code in a new jar does not help either as it could affect  
> ActiveMQ loaded in other loaders too. Argghhh...


Another option is that jarjar tool which refactors the bytecode to  
use another package. So you can take the activemq-core.jar and  
swizzle it to com.hermesjms.activemq.whatnot

http://tonicsystems.com/products/jarjar/

There's a jarjar plugin for Maven I think; e.g. Groovy uses jarjar to  
rename the ASM classes to avoid classpath hell when using different  
tools dependent on different ASM versions.


>> BTW did you manage to reuse the JDBC message store from ActiveMQ  
>> in Hermes or was it a bit too 'implementation focussed' to be  
>> easily reusable?
>>
>
> I had to roll my own, the ActiveMQ store is too ActiveMQ focused so  
> I could not store other providers messages in it. I store each  
> message as an XML blob so it can be re-created with a different JMS  
> provider to where it came from originally - hence you can capture  
> messages from say ActiveMQ and them replay them to EMS or WMQ.

No worries.

BTW it'd be nice to reuse your message store in ServiceMix (http://
servicemix.org) so that we could route artbirary JMS messages to the  
store so that they can be kept around for a while and maybe re-
ordered, filtered, queried or whatever. Some folks might just want a  
repository of messages as a history that they can then replay  
whenever they like.

Come to think of it, since its an XML blob, it should work for SOAP  
messages too right? Maybe its time to make the message store part of  
Hermes a reusable component?

James
-------
http://radio.weblogs.com/0112098/

Reply | Threaded
Open this post in threaded view
|

[activemq-user] Re: Reusing ActiveMQ filter

colincrist
James Strachan wrote:

> Another option is that jarjar tool which refactors the bytecode to  use
> another package. So you can take the activemq-core.jar and  swizzle it
> to com.hermesjms.activemq.whatnot
>
> http://tonicsystems.com/products/jarjar/
>
> There's a jarjar plugin for Maven I think; e.g. Groovy uses jarjar to  
> rename the ASM classes to avoid classpath hell when using different  
> tools dependent on different ASM versions.

Interesting, I'll take a look. It would certainly work. Thanks for the
pointer.

> BTW it'd be nice to reuse your message store in ServiceMix (http://
> servicemix.org) so that we could route artbirary JMS messages to the  
> store so that they can be kept around for a while and maybe re- ordered,
> filtered, queried or whatever. Some folks might just want a  repository
> of messages as a history that they can then replay  whenever they like.
>
> Come to think of it, since its an XML blob, it should work for SOAP  
> messages too right? Maybe its time to make the message store part of  
> Hermes a reusable component?

Very plausable. The schema for each message has a storeId, destination,
messageId and a CLOB for the XML. There are some other tables to
organise the stores as each db can hold many stores. This could easily
become storeId, bucketKey and bucketPrimaryKey plus the CLOB (or other
more abstract names). I'd then have a CLOBStore that my JMS message
store uses.... For want of a new design, the implementation has a
JDBCAdapter, StatementProvider etc, surprisingly familiar to any
ActiveMQ developer :-)

Heck, then Hermes could look into ServiceMix message stores, interesting...

I'm working on a 1.10 Hermes release right now, maybe we can chat after
that about any refactoring and you can give me some requirements.

Cheers,

Colin.

ps. any thoughts on my comments about remote access to a broker message
store?


Reply | Threaded
Open this post in threaded view
|

RE: [activemq-user] Wonder, why would two clients cannot exchange messages in the following cluster configuration?

gklyuzner
In reply to this post by gklyuzner
Mark,
Probably in case of persistent messages you have to use clustered datastore, but we have to get some input from the developers.
Unfortunately for unknown reason they do ignore all questions regards the cluster issues :(

Anyway you gave me very good clue about the problem. I will try to change to persistent messages and see if it helps.

Thanks, Glen


-----Original Message-----
From: Mark Lewis [mailto:[hidden email]]
Sent: Wednesday, August 31, 2005 6:51 PM
To: [hidden email]
Subject: RE: [activemq-user] Wonder, why would two clients cannot
exchangemessages in the following cluster configuration?


I've got a similar problem (well, started with a 4-broker solution, but
that didn't work so I tried a simple 2-broker solution).  Just used a
default config xml, the only thing added was each broker had a reliable:
network channel to the other.  They still weren't passing messages back
and forth if the producer connected to one broker and the consumer
connected to the other.  It seems like persistent messages just don't
work across network of brokers.

Is there ANYBODY out there who has gotten persistent messaging working
across a network of brokers?  If so, could you share your configs?

-- Mark

On Wed, 2005-08-31 at 18:32 -0400, Joe Barefoot wrote:

> Very strange.  I don't have any ideas as to the cause, but I would try
> removing all non-required attributes, options, etc., to see if any of
> those are causing the problem.  If the bare-bones config works you can
> add back options one at a time to see what breaks it.  i.e. start like
> this:
>
>
> Broker on nyotc029
>
> <?xml version="1.0" encoding="UTF-8"?>
> <!DOCTYPE beans PUBLIC "-//ACTIVEMQ//DTD//EN"
> "http://activemq.org/dtd/activemq.dtd">
> <beans>
> <!--
> ==================================================================== -->
> <!-- ActiveMQ Broker Configuration -->
> <!--
> ==================================================================== -->
> <broker>
> <connector>
> <tcpServerTransport uri="tcp://nyotc030:61616"
> />
> </connector>
>     <networkConnector>
>       <networkChannel uri="reliable:tcp://nyotc029:61616" />    
>     </networkConnector>
>
>     <persistence>
>       <vmPersistence/>
>     </persistence>
>
> </broker>
>
> Broker on nyotc029
>
> <?xml version="1.0" encoding="UTF-8"?>
> <!DOCTYPE beans PUBLIC "-//ACTIVEMQ//DTD//EN"
> "http://activemq.org/dtd/activemq.dtd">
> <beans>
> <!--
> ==================================================================== -->
> <!-- ActiveMQ Broker Configuration -->
> <!--
> ==================================================================== -->
> <broker>
> <connector>
> <tcpServerTransport uri="tcp://nyotc029:61616"/>
> </connector>
> <!-- Cluster of TWO Brokers, each runs on own host -->
>     <networkConnector>
>       <networkChannel uri="reliable:tcp://nyotc030:61616" />  
>     </networkConnector>
>
>     <persistence>
>       <vmPersistence/>
>     </persistence>
> </broker>
>
> -----Original Message-----
> From: Klyuzner, Glen [mailto:[hidden email]]
> Sent: Wednesday, August 31, 2005 12:48 PM
> To: [hidden email]
> Subject: [activemq-user] Wonder, why would two clients cannot exchange
> messages in the following cluster configuration?
>
> Wonder, why would two clients cannot exchange messages in the following
> cluster configuration?
> Multicast should not be an issue, since it uses hard coded URL.
>
> Client 1 connected to nyotc030:
> BrokerURL reliable:tcp://nyotc030:61616,tcp://nyotc029:61616
>
> INFO 2005-08-31 15:10:22,873 - channel status changed: Channel:
> ReliableTransportChannel: TcpTransportChannel:
> Socket[addr=nyotc030/170.198.95.219,port=61616,localport=2643] has
> connected I Client 2 connected to nyotc029:
> BrokerURL reliable:tcp://nyotc030:61616,tcp://nyotc029:61616
>
> NFO 2005-08-31 15:09:44,779 - channel status changed: Channel:
> ReliableTransportChannel: TcpTransportChannel:
> Socket[addr=nyotc029/170.198.95.212,port=61616,localport=2640] has
> connected I
>
> Broker on nyotc029
>
> <?xml version="1.0" encoding="UTF-8"?>
> <!DOCTYPE beans PUBLIC "-//ACTIVEMQ//DTD//EN"
> "http://activemq.org/dtd/activemq.dtd">
> <beans>
> <!--
> ==================================================================== -->
> <!-- ActiveMQ Broker Configuration -->
> <!--
> ==================================================================== -->
> <broker name="nyotc030-broker-1-M6" maxMem="41943040">
> <connector>
> <tcpServerTransport uri="tcp://nyotc030:61616"
> backlog="1000" useAsyncSend="true" maxOutstandingMessages="50"/>
> </connector>
>     <networkConnector>
>       <networkChannel uri="reliable:tcp://nyotc029:61616"
> remoteBrokerName="nyotc029-broker-1-M6" maximumRetries="1000"
> reconnectSleepTime="3000"/>
>       <localPrefetchPolicy queuePrefetch="1" queueBrowserPrefetch="2"/>
>       <remotePrefetchPolicy topicPrefetch="3" durableTopicPrefetch="4"/>
>     </networkConnector>
>
>     <persistence>
>       <vmPersistence/>
>     </persistence>
>
> </broker>
> Broker on nyotc029
>
> <?xml version="1.0" encoding="UTF-8"?>
> <!DOCTYPE beans PUBLIC "-//ACTIVEMQ//DTD//EN"
> "http://activemq.org/dtd/activemq.dtd">
> <beans>
> <!--
> ==================================================================== -->
> <!-- ActiveMQ Broker Configuration -->
> <!--
> ==================================================================== -->
> <broker name="nyotc029-broker-1-M6" maxMem="41943040">
> <connector>
> <tcpServerTransport uri="tcp://nyotc029:61616" backlog="1000"
> useAsyncSend="true" maxOutstandingMessages="50"/>
> </connector>
> <!-- Cluster of TWO Brokers, each runs on own host -->
>     <networkConnector>
>       <networkChannel uri="reliable:tcp://nyotc030:61616"
> remoteBrokerName="nyotc030-broker-1-M6" maximumRetries="1000"
> reconnectSleepTime="5000"/>
>
>       <localPrefetchPolicy queuePrefetch="1" queueBrowserPrefetch="2"/>
>       <remotePrefetchPolicy topicPrefetch="3" durableTopicPrefetch="4"/>
>     </networkConnector>
>
>     <persistence>
>       <vmPersistence/>
>     </persistence>
> </broker>
> ------------------------------------------------------------------------
> ------------------------------------------------------------------------
> ---------------------------------
> The information contained in this email is confidential and may also
> contain privileged information. Sender does not waive confidentiality or
> legal privilege. If you are not the intended recipient please notify the
> sender immediately; you should not retain this message or disclose its
> content to anyone.
> Internet communications are not secure or error free and the sender does
> not accept any liability for the content of the email. Although emails
> are routinely screened for viruses, the sender does not accept
> responsibility for any damage caused. Replies to this email may be
> monitored.
> ------------------------------------------------------------------------
> ------------------------------------------------------------------------
> ---------------------------------
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
The information contained in this email is confidential and may also contain privileged information. Sender does not waive confidentiality or legal privilege. If you are not the intended recipient please notify the sender immediately; you should not retain this message or disclose its content to anyone.
Internet communications are not secure or error free and the sender does not accept any liability for the content of the email. Although emails are routinely screened for viruses, the sender does not accept responsibility for any damage caused. Replies to this email may be monitored.
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Reply | Threaded
Open this post in threaded view
|

RE: [activemq-user] Re: Reusing ActiveMQ filter

DAVIES, GARETH I [AG-Contractor/1000]
In reply to this post by colincrist
Colin, James et al:

Just to add my $0.02...

We use our own database tables for JDBC persistence of ActiveMQ messages and
(separately) their XML payloads that use BLOBs instead of CLOBs so that we
can compress the payload using, say, GZIP. Might this be a strategy to
include with any reusable message store?

Gareth Davies

-----Original Message-----
From: Colin Crist [mailto:[hidden email]]
Sent: Thursday, September 01, 2005 2:34 PM
To: [hidden email]
Subject: [activemq-user] Re: Reusing ActiveMQ filter


James Strachan wrote:

> Another option is that jarjar tool which refactors the bytecode to  
> use
> another package. So you can take the activemq-core.jar and  swizzle it
> to com.hermesjms.activemq.whatnot
>
> http://tonicsystems.com/products/jarjar/
>
> There's a jarjar plugin for Maven I think; e.g. Groovy uses jarjar to
> rename the ASM classes to avoid classpath hell when using different  
> tools dependent on different ASM versions.

Interesting, I'll take a look. It would certainly work. Thanks for the
pointer.

> BTW it'd be nice to reuse your message store in ServiceMix (http://
> servicemix.org) so that we could route artbirary JMS messages to the  
> store so that they can be kept around for a while and maybe re- ordered,
> filtered, queried or whatever. Some folks might just want a  repository
> of messages as a history that they can then replay  whenever they like.
>
> Come to think of it, since its an XML blob, it should work for SOAP
> messages too right? Maybe its time to make the message store part of  
> Hermes a reusable component?

Very plausable. The schema for each message has a storeId, destination,
messageId and a CLOB for the XML. There are some other tables to
organise the stores as each db can hold many stores. This could easily
become storeId, bucketKey and bucketPrimaryKey plus the CLOB (or other
more abstract names). I'd then have a CLOBStore that my JMS message
store uses.... For want of a new design, the implementation has a
JDBCAdapter, StatementProvider etc, surprisingly familiar to any
ActiveMQ developer :-)

Heck, then Hermes could look into ServiceMix message stores, interesting...

I'm working on a 1.10 Hermes release right now, maybe we can chat after
that about any refactoring and you can give me some requirements.

Cheers,

Colin.

ps. any thoughts on my comments about remote access to a broker message
store?



---------------------------------------------------------------------------------------------------------
This e-mail message may contain privileged and/or confidential information, and is intended to be received only by persons entitled to receive such information. If you have received this e-mail in error, please notify the sender immediately. Please delete it and all attachments from any servers, hard drives or any other media. Other use of this e-mail by you is strictly prohibited.


All e-mails and attachments sent and received are subject to monitoring, reading and archival by Monsanto. The recipient of this e-mail is solely responsible for checking for the presence of "Viruses" or other "Malware". Monsanto accepts no liability for any damage caused by any such code transmitted by or accompanying this e-mail or any attachment.
---------------------------------------------------------------------------------------------------------

Reply | Threaded
Open this post in threaded view
|

[activemq-user] Re: Reusing ActiveMQ filter

colincrist

Agreed. Good suggestion.

Colin.

DAVIES, GARETH I [AG-Contractor/1000] wrote:

> Colin, James et al:
>
> Just to add my $0.02...
>
> We use our own database tables for JDBC persistence of ActiveMQ messages and
> (separately) their XML payloads that use BLOBs instead of CLOBs so that we
> can compress the payload using, say, GZIP. Might this be a strategy to
> include with any reusable message store?
>
> Gareth Davies
>
> -----Original Message-----
> From: Colin Crist [mailto:[hidden email]]
> Sent: Thursday, September 01, 2005 2:34 PM
> To: [hidden email]
> Subject: [activemq-user] Re: Reusing ActiveMQ filter
>
>
> James Strachan wrote:
>
>
>>Another option is that jarjar tool which refactors the bytecode to  
>>use
>>another package. So you can take the activemq-core.jar and  swizzle it
>>to com.hermesjms.activemq.whatnot
>>
>>http://tonicsystems.com/products/jarjar/
>>
>>There's a jarjar plugin for Maven I think; e.g. Groovy uses jarjar to
>>rename the ASM classes to avoid classpath hell when using different  
>>tools dependent on different ASM versions.
>
>
> Interesting, I'll take a look. It would certainly work. Thanks for the
> pointer.
>
>
>>BTW it'd be nice to reuse your message store in ServiceMix (http://
>>servicemix.org) so that we could route artbirary JMS messages to the  
>>store so that they can be kept around for a while and maybe re- ordered,
>>filtered, queried or whatever. Some folks might just want a  repository
>>of messages as a history that they can then replay  whenever they like.
>>
>>Come to think of it, since its an XML blob, it should work for SOAP
>>messages too right? Maybe its time to make the message store part of  
>>Hermes a reusable component?
>
>
> Very plausable. The schema for each message has a storeId, destination,
> messageId and a CLOB for the XML. There are some other tables to
> organise the stores as each db can hold many stores. This could easily
> become storeId, bucketKey and bucketPrimaryKey plus the CLOB (or other
> more abstract names). I'd then have a CLOBStore that my JMS message
> store uses.... For want of a new design, the implementation has a
> JDBCAdapter, StatementProvider etc, surprisingly familiar to any
> ActiveMQ developer :-)
>
> Heck, then Hermes could look into ServiceMix message stores, interesting...
>
> I'm working on a 1.10 Hermes release right now, maybe we can chat after
> that about any refactoring and you can give me some requirements.
>
> Cheers,
>
> Colin.
>
> ps. any thoughts on my comments about remote access to a broker message
> store?
>
>
>
> ---------------------------------------------------------------------------------------------------------
> This e-mail message may contain privileged and/or confidential information, and is intended to be received only by persons entitled to receive such information. If you have received this e-mail in error, please notify the sender immediately. Please delete it and all attachments from any servers, hard drives or any other media. Other use of this e-mail by you is strictly prohibited.
>
>
> All e-mails and attachments sent and received are subject to monitoring, reading and archival by Monsanto. The recipient of this e-mail is solely responsible for checking for the presence of "Viruses" or other "Malware". Monsanto accepts no liability for any damage caused by any such code transmitted by or accompanying this e-mail or any attachment.
> ---------------------------------------------------------------------------------------------------------
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Reusing ActiveMQ filter

mihaela.ion
In reply to this post by colincrist
Colin Crist <colincrist@...> writes:

>
>
> Hi,
>
> I've added selector support to Hermes, not only on destinations but on
> the message store too. This means you can search for messages using a
> filter on the entire store or on messages originating from a particular
> queue or topic.
>
> For the message store I have two implementations, one is based on a
> thing called JAMSEL which is a soon to be released standalone opensource
> JMS selector and the other implementation uses org.activemq.filter.

Hi Colin,

I am trying to write a custom filter for activemq. I came across the
org.activemq.filter package. I wrote a class that implements the Filter
interface and overrides the matches() method. I am having trouble integrating
this with activemq. Do I need to specify the filter class in the activemq.xml
file? How? What would you recommend? Thank you.

Regards,
Mihaela

>
> It *seems* to work OK, I've only done a few simple tests with Tibco EMS
> as the message factory so far - can anyone confirm that this is good to
> work with messages created from other JMS providers?
>
> I think it will be quite nice to be able to use the Hermes GUI or ant
> tasks to dump messages from a load of queues or topics into a database
> and then run a selector accross the whole store of messages searching
> not only for header properties but XPath expressions or MapMessage
> properties. Of course you'll also be able to replay the messages that
> match this filter too giving some interestng options.
>
> Regards,
>
> Colin.
>
>