Artemis 2.1.0 compatibility with 1.5.x

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

Artemis 2.1.0 compatibility with 1.5.x

hcrobison
We have been using Artemis 1.5.1. Recently we upgrade the server to 2.1.0. While it can communicate with 2.1.0 client successfully, it does not seem to work properly with 1.5.1 client.

We are passing messages in protobuf format. The end result is the receiving end can no longer deserialize the protobuf messages due to the message body length problem. (com.google.protobuf.InvalidProtocolBufferException: While parsing a protocol message, the input ended unexpectedly in the middle of a field.  This could mean either than the input has been truncated or that an embedded message misreported its own length.)

Our investigation shows that the message sent and received have a length difference of 4 bytes. What is causing this? Could the new message encoding has anything to do with this? Can a 2.1.0 server work with 1.5.1 client?
Reply | Threaded
Open this post in threaded view
|

Re: Artemis 2.1.0 compatibility with 1.5.x

nigro_franz
This post was updated on .
Hi,

Yep, I'm testing what could be the cause right now.
My fix is here: https://github.com/franz1981/activemq-artemis/tree/old_client_jms_temp_fix if you want to try it
If the tests will be successfull I'll make a PR with it :)

Regards,
Franz
Reply | Threaded
Open this post in threaded view
|

Re: Artemis 2.1.0 compatibility with 1.5.x

Justin Bertram
In reply to this post by hcrobison
A 1.5.1 client should be able to interact with a 2.1.0 broker.  I don't
know why there would be a difference in the size of the messages.

To be clear, the new encoding applies to AMQP messages which you don't
appear to be using.

Can you work up a simple, reproducible test-case to demonstrate the issue
you're seeing?


Justin

On Thu, Jun 22, 2017 at 7:33 AM, hcrobison <[hidden email]>
wrote:

> We have been using Artemis 1.5.1. Recently we upgrade the server to 2.1.0.
> While it can communicate with 2.1.0 client successfully, it does not seem
> to
> work properly with 1.5.1 client.
>
> We are passing messages in protobuf format. The end result is the receiving
> end can no longer deserialize the protobuf messages due to the message body
> length problem. (com.google.protobuf.InvalidProtocolBufferException: While
> parsing a protocol message, the input ended unexpectedly in the middle of a
> field.  This could mean either than the input has been truncated or that an
> embedded message misreported its own length.)
>
> Our investigation shows that the message sent and received have a length
> difference of 4 bytes. What is causing this? Could the new message encoding
> has anything to do with this? Can a 2.1.0 server work with 1.5.1 client?
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.
> nabble.com/Artemis-2-1-0-compatibility-with-1-5-x-tp4727741.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: Artemis 2.1.0 compatibility with 1.5.x

nigro_franz
Thanks to your comment on the branch I've noticed I've posted the wrong branch, now I've fixed the branch name in the comment :P thanks!
Reply | Threaded
Open this post in threaded view
|

Re: Artemis 2.1.0 compatibility with 1.5.x

hcrobison
This post was updated on .
nigro_franz, we have downloaded the branch you provided, compiled and tested but the same issue still exists.

When messages are sent from 2.1.0 to 1.5.1, the payload is truncated by 4 bytes. When messages are sent from 1.5.1 to 2.1.0, the payload is extended by 4 bytes.

For example, on the sending side (2.1.0), in org.apache.activemq.artemis.jms.client.ActiveMQMessageProducer.doSendx, jmsMessage.bodyLength=178; On the receiving side (1.5.1), in org.apache.activemq.artemis.jms.client.ActiveMQMessage.getJMSDestination, this.bodyLength=174.

We are working on a simpler example to demonstrate the problem.
Reply | Threaded
Open this post in threaded view
|

Re: Artemis 2.1.0 compatibility with 1.5.x

nigro_franz
This post was updated on .
thanks for the update, It would be great to have a test/reproducer for it!
Reply | Threaded
Open this post in threaded view
|

Re: Artemis 2.1.0 compatibility with 1.5.x

hcrobison
We managed to reproduce our message size change problem with simple java programs (see QueueReader.java and QueueWriter.java) based on your QueueExample.

One of them sends a binary message (array of bytes), another one reads this message.
The pom.xml files were changed so that the Artemis server is not started.
We ran a standalone Artemis 2.1 server.

The test programs were built with different Artemis versions, 1.5.5 and 2.1.

When the sender is 1.5.5 and the receiver is 2.1, the message read back is always 4 bytes longer, garbage bytes are added to it.

When the sender is 2.1 and the receiver is 1.5.5, the message read back is always 4 bytes shorter, the last 4 bytes are lost.


When both programs are built with the same version of Artemis, does not matter which, the read back message is correct.

Besides, if we send a text message, it is read back correctly.
Reply | Threaded
Open this post in threaded view
|

Re: Artemis 2.1.0 compatibility with 1.5.x

Justin Bertram
You wouldn't by chance have a git repo where you've Mavenized this stuff to
it's easy to run against a local broker would you?


Justin

On Wed, Jun 28, 2017 at 1:08 PM, hcrobison <[hidden email]>
wrote:

> We managed to reproduce our message size change problem with simple java
> programs (see  QueueReader.java
> <http://activemq.2283324.n4.nabble.com/file/n4728024/QueueReader.java>
>  and
> QueueWriter.java
> <http://activemq.2283324.n4.nabble.com/file/n4728024/QueueWriter.java>  )
> based on your QueueExample.
>
> One of them sends a binary message (array of bytes), another one reads this
> message.
> The pom.xml files were changed so that the Artemis server is not started.
> We ran a standalone Artemis 2.1 server.
>
> The test programs were built with different Artemis versions, 1.5.5 and
> 2.1.
>
> When the sender is 1.5.5 and the receiver is 2.1, the message read back is
> always 4 bytes longer, garbage bytes are added to it.
>
> When the sender is 2.1 and the receiver is 1.5.5, the message read back is
> always 4 bytes shorter, the last 4 bytes are lost.
>
>
> When both programs are built with the same version of Artemis, does not
> matter which, the read back message is correct.
>
> Besides, if we send a text message, it is read back correctly.
>
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.
> nabble.com/Artemis-2-1-0-compatibility-with-1-5-x-tp4727741p4728024.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: Artemis 2.1.0 compatibility with 1.5.x

hcrobison
Sorry, we don't.
Any specific file/setting you need, let us know.
Reply | Threaded
Open this post in threaded view
|

Re: Artemis 2.1.0 compatibility with 1.5.x

Justin Bertram
In reply to this post by hcrobison
> Besides, if we send a text message, it is read back correctly.

Can you confirm this?  Here's my testing procedure:

  - Broker running from the head of the "master" branch
  - Modified "queue" example also from the "master" branch which just sends
a text message (removed the consuming code)
  - Modified "queue" example from 1.5.5 which just consumes a message
(removed the sending code)

I receive this when receive() is invoked:

  java.lang.IndexOutOfBoundsException: readerIndex(22) + length(44) exceeds
writerIndex(62): UnpooledDuplicatedByteBuf(ridx: 22, widx: 62, cap: 315,
unwrapped: UnpooledUnsafeHeapByteBuf(ridx: 303, widx: 315, cap: 315))


Justin

On Wed, Jun 28, 2017 at 1:08 PM, hcrobison <[hidden email]>
wrote:

> We managed to reproduce our message size change problem with simple java
> programs (see  QueueReader.java
> <http://activemq.2283324.n4.nabble.com/file/n4728024/QueueReader.java>
>  and
> QueueWriter.java
> <http://activemq.2283324.n4.nabble.com/file/n4728024/QueueWriter.java>  )
> based on your QueueExample.
>
> One of them sends a binary message (array of bytes), another one reads this
> message.
> The pom.xml files were changed so that the Artemis server is not started.
> We ran a standalone Artemis 2.1 server.
>
> The test programs were built with different Artemis versions, 1.5.5 and
> 2.1.
>
> When the sender is 1.5.5 and the receiver is 2.1, the message read back is
> always 4 bytes longer, garbage bytes are added to it.
>
> When the sender is 2.1 and the receiver is 1.5.5, the message read back is
> always 4 bytes shorter, the last 4 bytes are lost.
>
>
> When both programs are built with the same version of Artemis, does not
> matter which, the read back message is correct.
>
> Besides, if we send a text message, it is read back correctly.
>
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.
> nabble.com/Artemis-2-1-0-compatibility-with-1-5-x-tp4727741p4728024.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: Artemis 2.1.0 compatibility with 1.5.x

clebertsuconic
Make sure you don't mix the jms with the core versions... 1.5 for both
jms and core in one side..


When you use 1.5, all your libraries on that client need to be 1.5..
.when 2.1.. all of them need to be 2.1...

On Wed, Jun 28, 2017 at 4:15 PM, Justin Bertram <[hidden email]> wrote:

>> Besides, if we send a text message, it is read back correctly.
>
> Can you confirm this?  Here's my testing procedure:
>
>   - Broker running from the head of the "master" branch
>   - Modified "queue" example also from the "master" branch which just sends
> a text message (removed the consuming code)
>   - Modified "queue" example from 1.5.5 which just consumes a message
> (removed the sending code)
>
> I receive this when receive() is invoked:
>
>   java.lang.IndexOutOfBoundsException: readerIndex(22) + length(44) exceeds
> writerIndex(62): UnpooledDuplicatedByteBuf(ridx: 22, widx: 62, cap: 315,
> unwrapped: UnpooledUnsafeHeapByteBuf(ridx: 303, widx: 315, cap: 315))
>
>
> Justin
>
> On Wed, Jun 28, 2017 at 1:08 PM, hcrobison <[hidden email]>
> wrote:
>
>> We managed to reproduce our message size change problem with simple java
>> programs (see  QueueReader.java
>> <http://activemq.2283324.n4.nabble.com/file/n4728024/QueueReader.java>
>>  and
>> QueueWriter.java
>> <http://activemq.2283324.n4.nabble.com/file/n4728024/QueueWriter.java>  )
>> based on your QueueExample.
>>
>> One of them sends a binary message (array of bytes), another one reads this
>> message.
>> The pom.xml files were changed so that the Artemis server is not started.
>> We ran a standalone Artemis 2.1 server.
>>
>> The test programs were built with different Artemis versions, 1.5.5 and
>> 2.1.
>>
>> When the sender is 1.5.5 and the receiver is 2.1, the message read back is
>> always 4 bytes longer, garbage bytes are added to it.
>>
>> When the sender is 2.1 and the receiver is 1.5.5, the message read back is
>> always 4 bytes shorter, the last 4 bytes are lost.
>>
>>
>> When both programs are built with the same version of Artemis, does not
>> matter which, the read back message is correct.
>>
>> Besides, if we send a text message, it is read back correctly.
>>
>>
>>
>>
>> --
>> View this message in context: http://activemq.2283324.n4.
>> nabble.com/Artemis-2-1-0-compatibility-with-1-5-x-tp4727741p4728024.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>



--
Clebert Suconic
Reply | Threaded
Open this post in threaded view
|

Re: Artemis 2.1.0 compatibility with 1.5.x

hcrobison
In reply to this post by Justin Bertram
Justin Bertram,
Yes, we can confirm we received the same error message when sending a text message from 2.1 and receiving at 1.5.5.
Reply | Threaded
Open this post in threaded view
|

Re: Artemis 2.1.0 compatibility with 1.5.x

hcrobison
In reply to this post by Justin Bertram
It appears that you were able to reproduce the issue.
Is there a plan to fix the problem? If so, what is your estimated time of completion?
Reply | Threaded
Open this post in threaded view
|

Re: Artemis 2.1.0 compatibility with 1.5.x

hcrobison
In reply to this post by clebertsuconic
We are not mixing core version and jms version.
Reply | Threaded
Open this post in threaded view
|

Re: Artemis 2.1.0 compatibility with 1.5.x

hcrobison
In reply to this post by Justin Bertram
We see that Artemis 2.4.0 has been released. Has the compatibility issue with
1.5.5 resolved?



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: Artemis 2.1.0 compatibility with 1.5.x

Justin Bertram
I don't personally recall fixing any compatibility issues.  Try it out and
see.

I managed to reproduce this awhile back but it fell off my radar.  If the
issue isn't fixed then it would probably be good to create a JIRA and
attach a reproducible test-case.


Justin

On Mon, Nov 13, 2017 at 11:56 AM, hcrobison <[hidden email]
> wrote:

> We see that Artemis 2.4.0 has been released. Has the compatibility issue
> with
> 1.5.5 resolved?
>
>
>
> --
> Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-
> f2341805.html
>
Reply | Threaded
Open this post in threaded view
|

Re: Artemis 2.1.0 compatibility with 1.5.x

Justin Bertram
Just for future reference if there is an issue that you think should block
the release I'd recommend checking the release when the VOTE goes out to
ensure it has the fix you want and voting accordingly.


Justin

On Mon, Nov 13, 2017 at 12:09 PM, Justin Bertram <[hidden email]>
wrote:

> I don't personally recall fixing any compatibility issues.  Try it out and
> see.
>
> I managed to reproduce this awhile back but it fell off my radar.  If the
> issue isn't fixed then it would probably be good to create a JIRA and
> attach a reproducible test-case.
>
>
> Justin
>
> On Mon, Nov 13, 2017 at 11:56 AM, hcrobison <hrobison@industrialdefender.
> com> wrote:
>
>> We see that Artemis 2.4.0 has been released. Has the compatibility issue
>> with
>> 1.5.5 resolved?
>>
>>
>>
>> --
>> Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805
>> .html
>>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Artemis 2.1.0 compatibility with 1.5.x

hcrobison
Thanks but these all sound pretty foreign to me.
First, are you saying, as a user, we should create a JIRA when we find a
bug?
Second, what is this VOTE process you are speaking of?



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: Artemis 2.1.0 compatibility with 1.5.x

Justin Bertram
This "users" list is the appropriate place to raise initial questions about
issues which might be bugs.  Once the issue is confirmed on the list to be
a bug it's a good idea to create a JIRA so it doesn't get lost in the
shuffle.

Regarding the VOTE process, that's a standard Apache practice when new
releases are proposed.  However, in ActiveMQ this is (usually) done on the
"developer" list [1].  If you're interested in participating in this
process (e.g. if you have a specific issue you're concerned about) then I
recommend you subscribe to that list.  Here's the VOTE email [2] I sent
about the 2.4.0 release.

In general the best way to ensure an issue you care about gets fixed is to
submit a fix for review (typically done via a pull-request on GitHub).  If
the fix is good it will be merged into the code-base.


Justin

[1] http://activemq.apache.org/mailing-lists.html
[2]
http://mail-archives.apache.org/mod_mbox/activemq-dev/201711.mbox/browser

On Mon, Nov 13, 2017 at 12:22 PM, hcrobison <[hidden email]
> wrote:

> Thanks but these all sound pretty foreign to me.
> First, are you saying, as a user, we should create a JIRA when we find a
> bug?
> Second, what is this VOTE process you are speaking of?
>
>
>
> --
> Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-
> f2341805.html
>