Artemis 2.1.0 compatibility with 1.5.x

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Artemis 2.1.0 compatibility with 1.5.x

hcrobison
We have been using Artemis 1.5.1. Recently we upgrade the server to 2.1.0. While it can communicate with 2.1.0 client successfully, it does not seem to work properly with 1.5.1 client.

We are passing messages in protobuf format. The end result is the receiving end can no longer deserialize the protobuf messages due to the message body length problem. (com.google.protobuf.InvalidProtocolBufferException: While parsing a protocol message, the input ended unexpectedly in the middle of a field.  This could mean either than the input has been truncated or that an embedded message misreported its own length.)

Our investigation shows that the message sent and received have a length difference of 4 bytes. What is causing this? Could the new message encoding has anything to do with this? Can a 2.1.0 server work with 1.5.1 client?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Artemis 2.1.0 compatibility with 1.5.x

nigro_franz
This post was updated on .
Hi,

Yep, I'm testing what could be the cause right now.
My fix is here: https://github.com/franz1981/activemq-artemis/tree/old_client_jms_temp_fix if you want to try it
If the tests will be successfull I'll make a PR with it :)

Regards,
Franz
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Artemis 2.1.0 compatibility with 1.5.x

Justin Bertram
In reply to this post by hcrobison
A 1.5.1 client should be able to interact with a 2.1.0 broker.  I don't
know why there would be a difference in the size of the messages.

To be clear, the new encoding applies to AMQP messages which you don't
appear to be using.

Can you work up a simple, reproducible test-case to demonstrate the issue
you're seeing?


Justin

On Thu, Jun 22, 2017 at 7:33 AM, hcrobison <[hidden email]>
wrote:

> We have been using Artemis 1.5.1. Recently we upgrade the server to 2.1.0.
> While it can communicate with 2.1.0 client successfully, it does not seem
> to
> work properly with 1.5.1 client.
>
> We are passing messages in protobuf format. The end result is the receiving
> end can no longer deserialize the protobuf messages due to the message body
> length problem. (com.google.protobuf.InvalidProtocolBufferException: While
> parsing a protocol message, the input ended unexpectedly in the middle of a
> field.  This could mean either than the input has been truncated or that an
> embedded message misreported its own length.)
>
> Our investigation shows that the message sent and received have a length
> difference of 4 bytes. What is causing this? Could the new message encoding
> has anything to do with this? Can a 2.1.0 server work with 1.5.1 client?
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.
> nabble.com/Artemis-2-1-0-compatibility-with-1-5-x-tp4727741.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Artemis 2.1.0 compatibility with 1.5.x

nigro_franz
Thanks to your comment on the branch I've noticed I've posted the wrong branch, now I've fixed the branch name in the comment :P thanks!
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Artemis 2.1.0 compatibility with 1.5.x

hcrobison
This post was updated on .
nigro_franz, we have downloaded the branch you provided, compiled and tested but the same issue still exists.

When messages are sent from 2.1.0 to 1.5.1, the payload is truncated by 4 bytes. When messages are sent from 1.5.1 to 2.1.0, the payload is extended by 4 bytes.

For example, on the sending side (2.1.0), in org.apache.activemq.artemis.jms.client.ActiveMQMessageProducer.doSendx, jmsMessage.bodyLength=178; On the receiving side (1.5.1), in org.apache.activemq.artemis.jms.client.ActiveMQMessage.getJMSDestination, this.bodyLength=174.

We are working on a simpler example to demonstrate the problem.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Artemis 2.1.0 compatibility with 1.5.x

nigro_franz
This post was updated on .
thanks for the update, It would be great to have a test/reproducer for it!
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Artemis 2.1.0 compatibility with 1.5.x

hcrobison
We managed to reproduce our message size change problem with simple java programs (see QueueReader.java and QueueWriter.java) based on your QueueExample.

One of them sends a binary message (array of bytes), another one reads this message.
The pom.xml files were changed so that the Artemis server is not started.
We ran a standalone Artemis 2.1 server.

The test programs were built with different Artemis versions, 1.5.5 and 2.1.

When the sender is 1.5.5 and the receiver is 2.1, the message read back is always 4 bytes longer, garbage bytes are added to it.

When the sender is 2.1 and the receiver is 1.5.5, the message read back is always 4 bytes shorter, the last 4 bytes are lost.


When both programs are built with the same version of Artemis, does not matter which, the read back message is correct.

Besides, if we send a text message, it is read back correctly.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Artemis 2.1.0 compatibility with 1.5.x

Justin Bertram
You wouldn't by chance have a git repo where you've Mavenized this stuff to
it's easy to run against a local broker would you?


Justin

On Wed, Jun 28, 2017 at 1:08 PM, hcrobison <[hidden email]>
wrote:

> We managed to reproduce our message size change problem with simple java
> programs (see  QueueReader.java
> <http://activemq.2283324.n4.nabble.com/file/n4728024/QueueReader.java>
>  and
> QueueWriter.java
> <http://activemq.2283324.n4.nabble.com/file/n4728024/QueueWriter.java>  )
> based on your QueueExample.
>
> One of them sends a binary message (array of bytes), another one reads this
> message.
> The pom.xml files were changed so that the Artemis server is not started.
> We ran a standalone Artemis 2.1 server.
>
> The test programs were built with different Artemis versions, 1.5.5 and
> 2.1.
>
> When the sender is 1.5.5 and the receiver is 2.1, the message read back is
> always 4 bytes longer, garbage bytes are added to it.
>
> When the sender is 2.1 and the receiver is 1.5.5, the message read back is
> always 4 bytes shorter, the last 4 bytes are lost.
>
>
> When both programs are built with the same version of Artemis, does not
> matter which, the read back message is correct.
>
> Besides, if we send a text message, it is read back correctly.
>
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.
> nabble.com/Artemis-2-1-0-compatibility-with-1-5-x-tp4727741p4728024.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Artemis 2.1.0 compatibility with 1.5.x

hcrobison
Sorry, we don't.
Any specific file/setting you need, let us know.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Artemis 2.1.0 compatibility with 1.5.x

Justin Bertram
In reply to this post by hcrobison
> Besides, if we send a text message, it is read back correctly.

Can you confirm this?  Here's my testing procedure:

  - Broker running from the head of the "master" branch
  - Modified "queue" example also from the "master" branch which just sends
a text message (removed the consuming code)
  - Modified "queue" example from 1.5.5 which just consumes a message
(removed the sending code)

I receive this when receive() is invoked:

  java.lang.IndexOutOfBoundsException: readerIndex(22) + length(44) exceeds
writerIndex(62): UnpooledDuplicatedByteBuf(ridx: 22, widx: 62, cap: 315,
unwrapped: UnpooledUnsafeHeapByteBuf(ridx: 303, widx: 315, cap: 315))


Justin

On Wed, Jun 28, 2017 at 1:08 PM, hcrobison <[hidden email]>
wrote:

> We managed to reproduce our message size change problem with simple java
> programs (see  QueueReader.java
> <http://activemq.2283324.n4.nabble.com/file/n4728024/QueueReader.java>
>  and
> QueueWriter.java
> <http://activemq.2283324.n4.nabble.com/file/n4728024/QueueWriter.java>  )
> based on your QueueExample.
>
> One of them sends a binary message (array of bytes), another one reads this
> message.
> The pom.xml files were changed so that the Artemis server is not started.
> We ran a standalone Artemis 2.1 server.
>
> The test programs were built with different Artemis versions, 1.5.5 and
> 2.1.
>
> When the sender is 1.5.5 and the receiver is 2.1, the message read back is
> always 4 bytes longer, garbage bytes are added to it.
>
> When the sender is 2.1 and the receiver is 1.5.5, the message read back is
> always 4 bytes shorter, the last 4 bytes are lost.
>
>
> When both programs are built with the same version of Artemis, does not
> matter which, the read back message is correct.
>
> Besides, if we send a text message, it is read back correctly.
>
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.
> nabble.com/Artemis-2-1-0-compatibility-with-1-5-x-tp4727741p4728024.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Artemis 2.1.0 compatibility with 1.5.x

clebertsuconic
Make sure you don't mix the jms with the core versions... 1.5 for both
jms and core in one side..


When you use 1.5, all your libraries on that client need to be 1.5..
.when 2.1.. all of them need to be 2.1...

On Wed, Jun 28, 2017 at 4:15 PM, Justin Bertram <[hidden email]> wrote:

>> Besides, if we send a text message, it is read back correctly.
>
> Can you confirm this?  Here's my testing procedure:
>
>   - Broker running from the head of the "master" branch
>   - Modified "queue" example also from the "master" branch which just sends
> a text message (removed the consuming code)
>   - Modified "queue" example from 1.5.5 which just consumes a message
> (removed the sending code)
>
> I receive this when receive() is invoked:
>
>   java.lang.IndexOutOfBoundsException: readerIndex(22) + length(44) exceeds
> writerIndex(62): UnpooledDuplicatedByteBuf(ridx: 22, widx: 62, cap: 315,
> unwrapped: UnpooledUnsafeHeapByteBuf(ridx: 303, widx: 315, cap: 315))
>
>
> Justin
>
> On Wed, Jun 28, 2017 at 1:08 PM, hcrobison <[hidden email]>
> wrote:
>
>> We managed to reproduce our message size change problem with simple java
>> programs (see  QueueReader.java
>> <http://activemq.2283324.n4.nabble.com/file/n4728024/QueueReader.java>
>>  and
>> QueueWriter.java
>> <http://activemq.2283324.n4.nabble.com/file/n4728024/QueueWriter.java>  )
>> based on your QueueExample.
>>
>> One of them sends a binary message (array of bytes), another one reads this
>> message.
>> The pom.xml files were changed so that the Artemis server is not started.
>> We ran a standalone Artemis 2.1 server.
>>
>> The test programs were built with different Artemis versions, 1.5.5 and
>> 2.1.
>>
>> When the sender is 1.5.5 and the receiver is 2.1, the message read back is
>> always 4 bytes longer, garbage bytes are added to it.
>>
>> When the sender is 2.1 and the receiver is 1.5.5, the message read back is
>> always 4 bytes shorter, the last 4 bytes are lost.
>>
>>
>> When both programs are built with the same version of Artemis, does not
>> matter which, the read back message is correct.
>>
>> Besides, if we send a text message, it is read back correctly.
>>
>>
>>
>>
>> --
>> View this message in context: http://activemq.2283324.n4.
>> nabble.com/Artemis-2-1-0-compatibility-with-1-5-x-tp4727741p4728024.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>



--
Clebert Suconic
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Artemis 2.1.0 compatibility with 1.5.x

hcrobison
In reply to this post by Justin Bertram
Justin Bertram,
Yes, we can confirm we received the same error message when sending a text message from 2.1 and receiving at 1.5.5.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Artemis 2.1.0 compatibility with 1.5.x

hcrobison
In reply to this post by Justin Bertram
It appears that you were able to reproduce the issue.
Is there a plan to fix the problem? If so, what is your estimated time of completion?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Artemis 2.1.0 compatibility with 1.5.x

hcrobison
In reply to this post by clebertsuconic
We are not mixing core version and jms version.
Loading...