Camel routing poor performance vs VirtualDestinations and Topics

classic Classic list List threaded Threaded
5 messages Options
drp
Reply | Threaded
Open this post in threaded view
|

Camel routing poor performance vs VirtualDestinations and Topics

drp
Hi.


Im getting very slow performance using camel based routing.

I have 1 producer sending messages to QUEUE.COLOR and then ActiveMQ duplicate the messages to QUEUE.BLUE and QUEUE.RED (except for topics...).


Is this normal?


My tests:

1. Camel Route defined in XML
2. Camel Route defined in RouteBuilder Plugin
3. VirtualDestination Routing
4. Topics

                 Test N°      1        2          3           4
Body Size
128B                         703     763     6377     10590
512B                         670     692     5872     8143
1KB                          628     635     5037     4749
2KB                          566     557     2376     2641

*messages/second


Camel Route defined in XML Code

        <camelContext id="camel" xmlns="http://activemq.apache.org/camel/schema/spring">
                <route>
                        <from uri="activemq:QUEUE.COLOR" />
                        <to uri="activemq:QUEUE.RED" />
                        <to uri="activemq:QUEUE.BLUE" />
                </route>
        </camelContext>


Camel Route defined in RouteBuilder Plugin Code


class MyRouteBuilder extends RouteBuilder {
   
    @Override
    public void configure() throws Exception {
       
        from("activemq:queue:QUEUE.COLOR").to("activemq:queue:QUEUE.RED").to("activemq:queue:QUEUE.BLUE");
    }
 
}

VirtualDestination Code

                <destinationInterceptors>
                        <virtualDestinationInterceptor>
                                <virtualDestinations>
                                        <compositeQueue name="QUEUE.COLOR">
                                                <forwardTo>
                                                        <queue physicalName="QUEUE.RED" />
                                                        <queue physicalName="QUEUE.BLUE" />
                                                </forwardTo>
                                        </compositeQueue>
                                </virtualDestinations>
                        </virtualDestinationInterceptor>
                </destinationInterceptors>
         



Graphic with results attached.


--
Regards,
Daniel Romero P.

activemq-routing.jpg
Reply | Threaded
Open this post in threaded view
|

Re: Camel routing poor performance vs VirtualDestinations and Topics

James Strachan-2
Which version of ActiveMQ/Camel/Spring are you using? Is the camel
code inside the broker? Do you have any specific spring configuration
for the ActiveMQ component?

BTW the out of the box won't be using transactions, so you'll be doing
a request/reply with the broker (I presume using TCP unless you've
some custom config) waiting for the send to hit disk for each
destination before committing the consume. So using a JMS transaction
should speed that up hugely.

However fast we get the camel code; topics and virtual queues are
always gonna be faster; since on the broker side its just moving an in
memory message to multiple destinations in a single thread without the
overhead of going through the JMS client API.

Having said that - if you are testing like for like (e.g. using VM
transport, using transactions & using async sends) it should get
close.


2009/3/31 DanielR <[hidden email]>:

>
> Hi.
>
>
> Im getting very slow performance using camel based routing.
>
> I have 1 producer sending messages to QUEUE.COLOR and then ActiveMQ
> duplicate the messages to QUEUE.BLUE and QUEUE.RED (except for topics...).
>
>
> Is this normal?
>
>
> My tests:
>
> 1. Camel Route defined in XML
> 2. Camel Route defined in RouteBuilder Plugin
> 3. VirtualDestination Routing
> 4. Topics
>
>                 Test N°      1        2          3           4
> Body Size
> 128B                         703     763     6377     10590
> 512B                         670     692     5872     8143
> 1KB                          628     635     5037     4749
> 2KB                          566     557     2376     2641
>
> *messages/second
>
>
> Camel Route defined in XML Code
>
>        <camelContext id="camel"
> xmlns="http://activemq.apache.org/camel/schema/spring">
>                <route>
>                        <from uri="activemq:QUEUE.COLOR" />
>                        <to uri="activemq:QUEUE.RED" />
>                        <to uri="activemq:QUEUE.BLUE" />
>                </route>
>        </camelContext>
>
>
> Camel Route defined in RouteBuilder Plugin Code
>
> class MyRouteBuilder extends RouteBuilder {
>
>    @Override
>    public void configure() throws Exception {
>
>
> from("activemq:queue:QUEUE.COLOR").to("activemq:queue:QUEUE.RED").to("activemq:queue:QUEUE.BLUE");
>    }
>
> }
>
> VirtualDestination Code
>
>                <destinationInterceptors>
>                        <virtualDestinationInterceptor>
>                                <virtualDestinations>
>                                        <compositeQueue name="QUEUE.COLOR">
>                                                <forwardTo>
>                                                        <queue
> physicalName="QUEUE.RED" />
>                                                        <queue
> physicalName="QUEUE.BLUE" />
>                                                </forwardTo>
>                                        </compositeQueue>
>                                </virtualDestinations>
>                        </virtualDestinationInterceptor>
>                </destinationInterceptors>
>
>
>
>
> Graphic with results attached.
>
>
> --
> Regards,
> Daniel Romero P.
>
> http://www.nabble.com/file/p22812467/activemq-routing.jpg
> activemq-routing.jpg
> --
> View this message in context: http://www.nabble.com/Camel-routing-poor-performance-vs-VirtualDestinations-and-Topics-tp22812467p22812467.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>



--
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/
drp
Reply | Threaded
Open this post in threaded view
|

Re: Camel routing poor performance vs VirtualDestinations and Topics

drp
Hi.

I'm using AMQ 5.2 (apache-activemq-5.2.0-bin.tar).
The camel code is in a java class of my own package (cl.niclabs.activemq.camel), loaded at the startup in the AMQ's xml config with this code:

<camelContext id="camel" xmlns="http://activemq.apache.org/camel/schema/spring">
                <package>cl.niclabs.activemq.camel</package>
</camelContext>

You can download my configs here: activemq-tests.tar.gz


So, you say that using JMS transactions should rise the camel routing performance near the VirtualDestination one, that's all?

Thanks for your time.



James.Strachan wrote
Which version of ActiveMQ/Camel/Spring are you using? Is the camel
code inside the broker? Do you have any specific spring configuration
for the ActiveMQ component?

BTW the out of the box won't be using transactions, so you'll be doing
a request/reply with the broker (I presume using TCP unless you've
some custom config) waiting for the send to hit disk for each
destination before committing the consume. So using a JMS transaction
should speed that up hugely.

However fast we get the camel code; topics and virtual queues are
always gonna be faster; since on the broker side its just moving an in
memory message to multiple destinations in a single thread without the
overhead of going through the JMS client API.

Having said that - if you are testing like for like (e.g. using VM
transport, using transactions & using async sends) it should get
close.


2009/3/31 DanielR <romero.cl@gmail.com>:
>
> Hi.
>
>
> Im getting very slow performance using camel based routing.
>
> I have 1 producer sending messages to QUEUE.COLOR and then ActiveMQ
> duplicate the messages to QUEUE.BLUE and QUEUE.RED (except for topics...).
>
>
> Is this normal?
>
>
> My tests:
>
> 1. Camel Route defined in XML
> 2. Camel Route defined in RouteBuilder Plugin
> 3. VirtualDestination Routing
> 4. Topics
>
>                 Test N°      1        2          3           4
> Body Size
> 128B                         703     763     6377     10590
> 512B                         670     692     5872     8143
> 1KB                          628     635     5037     4749
> 2KB                          566     557     2376     2641
>
> *messages/second
>
>
> Camel Route defined in XML Code
>
>        <camelContext id="camel"
> xmlns="http://activemq.apache.org/camel/schema/spring">
>                <route>
>                        <from uri="activemq:QUEUE.COLOR" />
>                        <to uri="activemq:QUEUE.RED" />
>                        <to uri="activemq:QUEUE.BLUE" />
>                </route>
>        </camelContext>
>
>
> Camel Route defined in RouteBuilder Plugin Code
>
> class MyRouteBuilder extends RouteBuilder {
>
>    @Override
>    public void configure() throws Exception {
>
>
> from("activemq:queue:QUEUE.COLOR").to("activemq:queue:QUEUE.RED").to("activemq:queue:QUEUE.BLUE");
>    }
>
> }
>
> VirtualDestination Code
>
>                <destinationInterceptors>
>                        <virtualDestinationInterceptor>
>                                <virtualDestinations>
>                                        <compositeQueue name="QUEUE.COLOR">
>                                                <forwardTo>
>                                                        <queue
> physicalName="QUEUE.RED" />
>                                                        <queue
> physicalName="QUEUE.BLUE" />
>                                                </forwardTo>
>                                        </compositeQueue>
>                                </virtualDestinations>
>                        </virtualDestinationInterceptor>
>                </destinationInterceptors>
>
>
>
>
> Graphic with results attached.
>
>
> --
> Regards,
> Daniel Romero P.
>
> http://www.nabble.com/file/p22812467/activemq-routing.jpg
> activemq-routing.jpg
> --
> View this message in context: http://www.nabble.com/Camel-routing-poor-performance-vs-VirtualDestinations-and-Topics-tp22812467p22812467.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>



--
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/
Reply | Threaded
Open this post in threaded view
|

Re: Camel routing poor performance vs VirtualDestinations and Topics

James Strachan-2
Also configuring the activemq component to use vm transport (as it
defaults to tcp://locahost:61616) would probably help too

2009/3/31 DanielR <[hidden email]>:

>
> Hi.
>
> I'm using AMQ 5.2 (apache-activemq-5.2.0-bin.tar).
> The camel code is in a java class of my own package
> (cl.niclabs.activemq.camel), loaded at the startup in the AMQ's xml config
> with this code:
>
> <camelContext id="camel"
> xmlns="http://activemq.apache.org/camel/schema/spring">
>                <package>cl.niclabs.activemq.camel</package>
> </camelContext>
>
> You can download my configs here:
> http://www.nabble.com/file/p22815567/activemq-tests.tar.gz
> activemq-tests.tar.gz
>
>
> So, you say that using JMS transactions should rise the camel routing
> performance near the VirtualDestination one, that's all?
>
> Thanks for your time.
>
>
>
>
> James.Strachan wrote:
>>
>> Which version of ActiveMQ/Camel/Spring are you using? Is the camel
>> code inside the broker? Do you have any specific spring configuration
>> for the ActiveMQ component?
>>
>> BTW the out of the box won't be using transactions, so you'll be doing
>> a request/reply with the broker (I presume using TCP unless you've
>> some custom config) waiting for the send to hit disk for each
>> destination before committing the consume. So using a JMS transaction
>> should speed that up hugely.
>>
>> However fast we get the camel code; topics and virtual queues are
>> always gonna be faster; since on the broker side its just moving an in
>> memory message to multiple destinations in a single thread without the
>> overhead of going through the JMS client API.
>>
>> Having said that - if you are testing like for like (e.g. using VM
>> transport, using transactions & using async sends) it should get
>> close.
>>
>>
>> 2009/3/31 DanielR <[hidden email]>:
>>>
>>> Hi.
>>>
>>>
>>> Im getting very slow performance using camel based routing.
>>>
>>> I have 1 producer sending messages to QUEUE.COLOR and then ActiveMQ
>>> duplicate the messages to QUEUE.BLUE and QUEUE.RED (except for
>>> topics...).
>>>
>>>
>>> Is this normal?
>>>
>>>
>>> My tests:
>>>
>>> 1. Camel Route defined in XML
>>> 2. Camel Route defined in RouteBuilder Plugin
>>> 3. VirtualDestination Routing
>>> 4. Topics
>>>
>>>                 Test N°      1        2          3           4
>>> Body Size
>>> 128B                         703     763     6377     10590
>>> 512B                         670     692     5872     8143
>>> 1KB                          628     635     5037     4749
>>> 2KB                          566     557     2376     2641
>>>
>>> *messages/second
>>>
>>>
>>> Camel Route defined in XML Code
>>>
>>>        <camelContext id="camel"
>>> xmlns="http://activemq.apache.org/camel/schema/spring">
>>>                <route>
>>>                        <from uri="activemq:QUEUE.COLOR" />
>>>                        <to uri="activemq:QUEUE.RED" />
>>>                        <to uri="activemq:QUEUE.BLUE" />
>>>                </route>
>>>        </camelContext>
>>>
>>>
>>> Camel Route defined in RouteBuilder Plugin Code
>>>
>>> class MyRouteBuilder extends RouteBuilder {
>>>
>>>    @Override
>>>    public void configure() throws Exception {
>>>
>>>
>>> from("activemq:queue:QUEUE.COLOR").to("activemq:queue:QUEUE.RED").to("activemq:queue:QUEUE.BLUE");
>>>    }
>>>
>>> }
>>>
>>> VirtualDestination Code
>>>
>>>                <destinationInterceptors>
>>>                        <virtualDestinationInterceptor>
>>>                                <virtualDestinations>
>>>                                        <compositeQueue
>>> name="QUEUE.COLOR">
>>>                                                <forwardTo>
>>>                                                        <queue
>>> physicalName="QUEUE.RED" />
>>>                                                        <queue
>>> physicalName="QUEUE.BLUE" />
>>>                                                </forwardTo>
>>>                                        </compositeQueue>
>>>                                </virtualDestinations>
>>>                        </virtualDestinationInterceptor>
>>>                </destinationInterceptors>
>>>
>>>
>>>
>>>
>>> Graphic with results attached.
>>>
>>>
>>> --
>>> Regards,
>>> Daniel Romero P.
>>>
>>> http://www.nabble.com/file/p22812467/activemq-routing.jpg
>>> activemq-routing.jpg
>>> --
>>> View this message in context:
>>> http://www.nabble.com/Camel-routing-poor-performance-vs-VirtualDestinations-and-Topics-tp22812467p22812467.html
>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>>
>> --
>> James
>> -------
>> http://macstrac.blogspot.com/
>>
>> Open Source Integration
>> http://fusesource.com/
>>
>>
>
> --
> View this message in context: http://www.nabble.com/Camel-routing-poor-performance-vs-VirtualDestinations-and-Topics-tp22812467p22815567.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>



--
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/
Reply | Threaded
Open this post in threaded view
|

Re: Camel routing poor performance vs VirtualDestinations and Topics

James Strachan-2
Oh and async sending...

2009/3/31 James Strachan <[hidden email]>:

> Also configuring the activemq component to use vm transport (as it
> defaults to tcp://locahost:61616) would probably help too
>
> 2009/3/31 DanielR <[hidden email]>:
>>
>> Hi.
>>
>> I'm using AMQ 5.2 (apache-activemq-5.2.0-bin.tar).
>> The camel code is in a java class of my own package
>> (cl.niclabs.activemq.camel), loaded at the startup in the AMQ's xml config
>> with this code:
>>
>> <camelContext id="camel"
>> xmlns="http://activemq.apache.org/camel/schema/spring">
>>                <package>cl.niclabs.activemq.camel</package>
>> </camelContext>
>>
>> You can download my configs here:
>> http://www.nabble.com/file/p22815567/activemq-tests.tar.gz
>> activemq-tests.tar.gz
>>
>>
>> So, you say that using JMS transactions should rise the camel routing
>> performance near the VirtualDestination one, that's all?
>>
>> Thanks for your time.
>>
>>
>>
>>
>> James.Strachan wrote:
>>>
>>> Which version of ActiveMQ/Camel/Spring are you using? Is the camel
>>> code inside the broker? Do you have any specific spring configuration
>>> for the ActiveMQ component?
>>>
>>> BTW the out of the box won't be using transactions, so you'll be doing
>>> a request/reply with the broker (I presume using TCP unless you've
>>> some custom config) waiting for the send to hit disk for each
>>> destination before committing the consume. So using a JMS transaction
>>> should speed that up hugely.
>>>
>>> However fast we get the camel code; topics and virtual queues are
>>> always gonna be faster; since on the broker side its just moving an in
>>> memory message to multiple destinations in a single thread without the
>>> overhead of going through the JMS client API.
>>>
>>> Having said that - if you are testing like for like (e.g. using VM
>>> transport, using transactions & using async sends) it should get
>>> close.
>>>
>>>
>>> 2009/3/31 DanielR <[hidden email]>:
>>>>
>>>> Hi.
>>>>
>>>>
>>>> Im getting very slow performance using camel based routing.
>>>>
>>>> I have 1 producer sending messages to QUEUE.COLOR and then ActiveMQ
>>>> duplicate the messages to QUEUE.BLUE and QUEUE.RED (except for
>>>> topics...).
>>>>
>>>>
>>>> Is this normal?
>>>>
>>>>
>>>> My tests:
>>>>
>>>> 1. Camel Route defined in XML
>>>> 2. Camel Route defined in RouteBuilder Plugin
>>>> 3. VirtualDestination Routing
>>>> 4. Topics
>>>>
>>>>                 Test N°      1        2          3           4
>>>> Body Size
>>>> 128B                         703     763     6377     10590
>>>> 512B                         670     692     5872     8143
>>>> 1KB                          628     635     5037     4749
>>>> 2KB                          566     557     2376     2641
>>>>
>>>> *messages/second
>>>>
>>>>
>>>> Camel Route defined in XML Code
>>>>
>>>>        <camelContext id="camel"
>>>> xmlns="http://activemq.apache.org/camel/schema/spring">
>>>>                <route>
>>>>                        <from uri="activemq:QUEUE.COLOR" />
>>>>                        <to uri="activemq:QUEUE.RED" />
>>>>                        <to uri="activemq:QUEUE.BLUE" />
>>>>                </route>
>>>>        </camelContext>
>>>>
>>>>
>>>> Camel Route defined in RouteBuilder Plugin Code
>>>>
>>>> class MyRouteBuilder extends RouteBuilder {
>>>>
>>>>    @Override
>>>>    public void configure() throws Exception {
>>>>
>>>>
>>>> from("activemq:queue:QUEUE.COLOR").to("activemq:queue:QUEUE.RED").to("activemq:queue:QUEUE.BLUE");
>>>>    }
>>>>
>>>> }
>>>>
>>>> VirtualDestination Code
>>>>
>>>>                <destinationInterceptors>
>>>>                        <virtualDestinationInterceptor>
>>>>                                <virtualDestinations>
>>>>                                        <compositeQueue
>>>> name="QUEUE.COLOR">
>>>>                                                <forwardTo>
>>>>                                                        <queue
>>>> physicalName="QUEUE.RED" />
>>>>                                                        <queue
>>>> physicalName="QUEUE.BLUE" />
>>>>                                                </forwardTo>
>>>>                                        </compositeQueue>
>>>>                                </virtualDestinations>
>>>>                        </virtualDestinationInterceptor>
>>>>                </destinationInterceptors>
>>>>
>>>>
>>>>
>>>>
>>>> Graphic with results attached.
>>>>
>>>>
>>>> --
>>>> Regards,
>>>> Daniel Romero P.
>>>>
>>>> http://www.nabble.com/file/p22812467/activemq-routing.jpg
>>>> activemq-routing.jpg
>>>> --
>>>> View this message in context:
>>>> http://www.nabble.com/Camel-routing-poor-performance-vs-VirtualDestinations-and-Topics-tp22812467p22812467.html
>>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>>
>>> Open Source Integration
>>> http://fusesource.com/
>>>
>>>
>>
>> --
>> View this message in context: http://www.nabble.com/Camel-routing-poor-performance-vs-VirtualDestinations-and-Topics-tp22812467p22815567.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>>
>
>
>
> --
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://fusesource.com/
>



--
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/