BytesMessage vs. already compressed byte[] payloads - fixed in 4.x?

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

BytesMessage vs. already compressed byte[] payloads - fixed in 4.x?

Holger Hoffstätte-2

Hi,

ActiveMQ 3.x unconditionally uncompresses BytesMessages whose input byte[]
was already compressed with the JDK-builtin GZip stuff. This is obviously
wrong since the compressed original byte[] should come out on the other
end, not the huge uncompressed payload. Is this fixed in 4.x? I figured I
ask before I forward-port. This bug makes ActiveMQ susceptible to DOS
attacks, even unintentionally if someone sends a meager 10 MB of
compressed XML over the wire that is exploded to >1GB, taking the VM with
it.
A simple ActiveMQ-specific prepended tag indicating transport-level
compression (or not) would help to distinguish between the two. If this
warrants a JIRA please yell.

thanks
Holger

PS: http://en.wikipedia.org/wiki/Zip_of_death, s/zip/gzip/r ;)


Reply | Threaded
Open this post in threaded view
|

Re: BytesMessage vs. already compressed byte[] payloads - fixed in 4.x?

chirino
Amq 4 should not be uncompressing message contents (unless it's sending to a
STOMP client I think).  Each message has flag indicating if the message
content is compressed or not.

On 10/18/06, Holger Hoffstaette <[hidden email]> wrote:

>
>
> Hi,
>
> ActiveMQ 3.x unconditionally uncompresses BytesMessages whose input byte[]
> was already compressed with the JDK-builtin GZip stuff. This is obviously
> wrong since the compressed original byte[] should come out on the other
> end, not the huge uncompressed payload. Is this fixed in 4.x? I figured I
> ask before I forward-port. This bug makes ActiveMQ susceptible to DOS
> attacks, even unintentionally if someone sends a meager 10 MB of
> compressed XML over the wire that is exploded to >1GB, taking the VM with
> it.
> A simple ActiveMQ-specific prepended tag indicating transport-level
> compression (or not) would help to distinguish between the two. If this
> warrants a JIRA please yell.



You want to distinguish between compressed and uncompressed messages?  This
can be done on a per message basis.  I don't think it has anything to do
with the transport.

thanks
> Holger
>
> PS: http://en.wikipedia.org/wiki/Zip_of_death, s/zip/gzip/r ;)
>
>
>


--
Regards,
Hiram

Blog: http://hiramchirino.com
Reply | Threaded
Open this post in threaded view
|

Re: BytesMessage vs. already compressed byte[] payloads - fixed in 4.x?

sileshi

Hiram Chirino wrote
>Amq 4 should not be uncompressing message contents (unless it's sending to a
>STOMP client I think).  Each message has flag indicating if the message
>content is compressed or not.
Well, even for sending to Stomp, AMQ should not uncompress the bytes message.
The contract between Java JMS client and Message Broker is to transmit the
byte streams to consumers with no alterations. This is true of irrespective of
the consumer is Java JMS and Stomp client.

The thinking is RPC  can be built over Java JMS and Stomp. Thus, RPC implmentation
may have one or more protocol (one protocol envolpes another) encoding, and data
that is compressed using standard or non-standard techniques.

The way to deal with this is by not interpreting/altering the byte streams.

Regards,
Sileshi

On 10/18/06, Holger Hoffstaette <holger@wizards.de> wrote:
>
>
> Hi,
>
> ActiveMQ 3.x unconditionally uncompresses BytesMessages whose input byte[]
> was already compressed with the JDK-builtin GZip stuff. This is obviously
> wrong since the compressed original byte[] should come out on the other
> end, not the huge uncompressed payload. Is this fixed in 4.x? I figured I
> ask before I forward-port. This bug makes ActiveMQ susceptible to DOS
> attacks, even unintentionally if someone sends a meager 10 MB of
> compressed XML over the wire that is exploded to >1GB, taking the VM with
> it.
> A simple ActiveMQ-specific prepended tag indicating transport-level
> compression (or not) would help to distinguish between the two. If this
> warrants a JIRA please yell.



You want to distinguish between compressed and uncompressed messages?  This
can be done on a per message basis.  I don't think it has anything to do
with the transport.

thanks
> Holger
>
> PS: http://en.wikipedia.org/wiki/Zip_of_death, s/zip/gzip/r ;)
>
>
>


--
Regards,
Hiram

Blog: http://hiramchirino.com
Reply | Threaded
Open this post in threaded view
|

Re: BytesMessage vs. already compressed byte[] payloads - fixed in 4.x?

chirino
well the jms client the put the byte[] into the message uncompressed.  It's
ActiveMQ under the covers that's doing the compression.  So if it had to
deliver the message to a stomp client, I think it's normal if ActiveMQ
uncompresses that data.

On 10/24/06, sileshi <[hidden email]> wrote:

>
>
>
>
> Hiram Chirino wrote:
> >
> >>Amq 4 should not be uncompressing message contents (unless it's sending
> to
> a
> >>STOMP client I think).  Each message has flag indicating if the message
> >>content is compressed or not.
> > Well, even for sending to Stomp, AMQ should not uncompress the bytes
> > message.
> > The contract between Java JMS client and Message Broker is to transmit
> the
> > byte streams to consumers with no alterations. This is true of
> > irrespective of
> > the consumer is Java JMS and Stomp client.
> >
> > The thinking is RPC  can be built over Java JMS and Stomp. Thus, RPC
> > implmentation
> > may have one or more protocol (one protocol envolpes another) encoding,
> > and data
> > that is compressed using standard or non-standard techniques.
> >
> > The way to deal with this is by not interpreting/altering the byte
> > streams.
> >
> > Regards,
> > Sileshi
> >
> > On 10/18/06, Holger Hoffstaette <[hidden email]> wrote:
> >>
> >>
> >> Hi,
> >>
> >> ActiveMQ 3.x unconditionally uncompresses BytesMessages whose input
> >> byte[]
> >> was already compressed with the JDK-builtin GZip stuff. This is
> obviously
> >> wrong since the compressed original byte[] should come out on the other
> >> end, not the huge uncompressed payload. Is this fixed in 4.x? I figured
> I
> >> ask before I forward-port. This bug makes ActiveMQ susceptible to DOS
> >> attacks, even unintentionally if someone sends a meager 10 MB of
> >> compressed XML over the wire that is exploded to >1GB, taking the VM
> with
> >> it.
> >> A simple ActiveMQ-specific prepended tag indicating transport-level
> >> compression (or not) would help to distinguish between the two. If this
> >> warrants a JIRA please yell.
> >
> >
> >
> > You want to distinguish between compressed and uncompressed messages?
> > This
> > can be done on a per message basis.  I don't think it has anything to do
> > with the transport.
> >
> > thanks
> >> Holger
> >>
> >> PS: http://en.wikipedia.org/wiki/Zip_of_death, s/zip/gzip/r ;)
> >>
> >>
> >>
> >
> >
> > --
> > Regards,
> > Hiram
> >
> > Blog: http://hiramchirino.com
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/BytesMessage-vs.-already-compressed-byte---payloads---fixed-in-4.x--tf2469871.html#a6977787
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>


--
Regards,
Hiram

Blog: http://hiramchirino.com
Reply | Threaded
Open this post in threaded view
|

Re: BytesMessage vs. already compressed byte[] payloads - fixed in 4.x?

sileshi

Hiram Chirino wrote
>well the jms client the put the byte[] into the message uncompressed.  It's
>ActiveMQ under the covers that's doing the compression.  So if it had to
>deliver the message to a stomp client, I think it's normal if ActiveMQ
>uncompresses that data.

AMQ doing compress/uncompress during routing to Stomp is fine.
I was talking about already compressed data in byte[] from
the client.


On 10/24/06, sileshi <sileshikassa@yahoo.com> wrote:
>
>
>
>
> Hiram Chirino wrote:
> >
> >>Amq 4 should not be uncompressing message contents (unless it's sending
> to
> a
> >>STOMP client I think).  Each message has flag indicating if the message
> >>content is compressed or not.
> > Well, even for sending to Stomp, AMQ should not uncompress the bytes
> > message.
> > The contract between Java JMS client and Message Broker is to transmit
> the
> > byte streams to consumers with no alterations. This is true of
> > irrespective of
> > the consumer is Java JMS and Stomp client.
> >
> > The thinking is RPC  can be built over Java JMS and Stomp. Thus, RPC
> > implmentation
> > may have one or more protocol (one protocol envolpes another) encoding,
> > and data
> > that is compressed using standard or non-standard techniques.
> >
> > The way to deal with this is by not interpreting/altering the byte
> > streams.
> >
> > Regards,
> > Sileshi
> >
> > On 10/18/06, Holger Hoffstaette <holger@wizards.de> wrote:
> >>
> >>
> >> Hi,
> >>
> >> ActiveMQ 3.x unconditionally uncompresses BytesMessages whose input
> >> byte[]
> >> was already compressed with the JDK-builtin GZip stuff. This is
> obviously
> >> wrong since the compressed original byte[] should come out on the other
> >> end, not the huge uncompressed payload. Is this fixed in 4.x? I figured
> I
> >> ask before I forward-port. This bug makes ActiveMQ susceptible to DOS
> >> attacks, even unintentionally if someone sends a meager 10 MB of
> >> compressed XML over the wire that is exploded to >1GB, taking the VM
> with
> >> it.
> >> A simple ActiveMQ-specific prepended tag indicating transport-level
> >> compression (or not) would help to distinguish between the two. If this
> >> warrants a JIRA please yell.
> >
> >
> >
> > You want to distinguish between compressed and uncompressed messages?
> > This
> > can be done on a per message basis.  I don't think it has anything to do
> > with the transport.
> >
> > thanks
> >> Holger
> >>
> >> PS: http://en.wikipedia.org/wiki/Zip_of_death, s/zip/gzip/r ;)
> >>
> >>
> >>
> >
> >
> > --
> > Regards,
> > Hiram
> >
> > Blog: http://hiramchirino.com
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/BytesMessage-vs.-already-compressed-byte---payloads---fixed-in-4.x--tf2469871.html#a6977787
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>


--
Regards,
Hiram

Blog: http://hiramchirino.com