[activemq-dev] help understanding prefetch please

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[activemq-dev] help understanding prefetch please

Dennis Cook
Our application makes use of a queue but the processing of each queue
message by a client is in the time period of ?minutes? not ?seconds?.
My goal is to have every message published should be readily available
to ANY possible client.

However if I understand pre-fetching correctly each client subscription
has a buffer (actually I think there are two buffers) pre-filled with
messages from the main store.  The buffers reside at each end of a
client?s connection; one at client side and one at the broker side of
the link. Messages are removed from the store and first put into the
broke side buffer.  Another thread then sends the messages across the
link to the client side buffer.  The client subscriber then extract the
message from the local buffer, and ACKs its acceptance.  When the ACK is
received, the message it removed from the broker side buffer.

I hope this is an accurate description of the how pre-fetching works
because I am setting my policy for queue pre-fetch to a value of one to
obtain my goal stated above.  If I were off track here would someone
jump in steer me in the right direction?

Okay so all looks good in theory if all clients are connected to a
single broker.  But the use of networked brokers look like it could
cause problems for me.  Network channels are still a little grey for me,
but it looks like they are basically implemented in the same manner as a
client to broker connection.  While the broker that creates the
connection, knows that there is something going on, the broker at the
remote end of the connection treats it the same as any other client
connection.  Assuming I am correct so far, then this connection should
also have a pre-fetch policy.  But in fact there seems to be two pre
fetch policies for a network channel (remote and local).  The remote
policy seems to be the equivalent to the one described above, but use of
the local is still opaque to me.

For further discussion lets assume two networked brokers A and B with a
single producer of messages on broker A.  Client subscribers are
attached to broker A and one client subscriber is attached to broker B.
  In this case the subscription of the client on broker B will establish
pre-fetch buffer on its connection and also pre-fetch buffers across the
network bridge.  This is where thing get very grey for me.  Just how do
the policies on the network channel (remote and local) effect the
availability of message to other clients (on broker A).  To keep things
consistent, lets assume that the network channels remote and local
policy is also set to 1.

Without even considering the network channels local pre-fetch policy and
applying my assumptions made above.  It seems to me that the client
subscribe will always have one additional message (total of two)
awaiting processing.  The first is in the pre-fetch buffer between the
client and broker B and the other is in the pre-fetch buffer on the
bridge between broker B and broker A.

Again my subscribing client application can take minutes to process a
single message.  It had been written to use another JMS implementation
that did not use pre-fetching and would ACK the message, then perform
the processing.  But if the application follows that same technique now,
even without a subscription across the network channel, there will still
be one additional message in the client?s buffer while the first message
is being processed.

I guess I would like some clarification on a couple of issues.

- Are my assumptions above correct? And if not where are they wrong.

- What is the effect of the local pre-fetch policy of a network channel?

- Is it possible to use a pre-fetch value of 0 and allow the client to
initiate retrieval on its own terms?

- How will the adding more networked brokers effect the buffering of
additional messages for a single client?  Let me elaborate.  Increase
the total broker count to 4 (A, B, C and D) and create all possible
cross connections (each broker has 3 network channels).  Now, with a
producer on broker A and a subscribing client on broker D, I think there
are 6 messages in the following buffers:

client/broker D - 1 message
broker D/broker A - 1 message
broker D/broker B - 1 message
broker B/broker A - 1 message
broker D/broker C - 1 message
broker C/broker A - 1 message

I sure hope my deduction on for this one is way off, because removing
that many message from play for other possible clients is totally
unacceptable.

Reply | Threaded
Open this post in threaded view
|

[activemq-dev] Re: help understanding prefetch please

Dennis Cook
Can anyone help with some addition information on prefetch.  No
responses yet.  I have to assume someone understands this better than I
do. ;-)


Dennis Cook wrote:

> Our application makes use of a queue but the processing of each queue
> message by a client is in the time period of ?minutes? not ?seconds?. My
> goal is to have every message published should be readily available to
> ANY possible client.
>
> However if I understand pre-fetching correctly each client subscription
> has a buffer (actually I think there are two buffers) pre-filled with
> messages from the main store.  The buffers reside at each end of a
> client?s connection; one at client side and one at the broker side of
> the link. Messages are removed from the store and first put into the
> broke side buffer.  Another thread then sends the messages across the
> link to the client side buffer.  The client subscriber then extract the
> message from the local buffer, and ACKs its acceptance.  When the ACK is
> received, the message it removed from the broker side buffer.
>
> I hope this is an accurate description of the how pre-fetching works
> because I am setting my policy for queue pre-fetch to a value of one to
> obtain my goal stated above.  If I were off track here would someone
> jump in steer me in the right direction?
>
> Okay so all looks good in theory if all clients are connected to a
> single broker.  But the use of networked brokers look like it could
> cause problems for me.  Network channels are still a little grey for me,
> but it looks like they are basically implemented in the same manner as a
> client to broker connection.  While the broker that creates the
> connection, knows that there is something going on, the broker at the
> remote end of the connection treats it the same as any other client
> connection.  Assuming I am correct so far, then this connection should
> also have a pre-fetch policy.  But in fact there seems to be two pre
> fetch policies for a network channel (remote and local).  The remote
> policy seems to be the equivalent to the one described above, but use of
> the local is still opaque to me.
>
> For further discussion lets assume two networked brokers A and B with a
> single producer of messages on broker A.  Client subscribers are
> attached to broker A and one client subscriber is attached to broker B.
>  In this case the subscription of the client on broker B will establish
> pre-fetch buffer on its connection and also pre-fetch buffers across the
> network bridge.  This is where thing get very grey for me.  Just how do
> the policies on the network channel (remote and local) effect the
> availability of message to other clients (on broker A).  To keep things
> consistent, lets assume that the network channels remote and local
> policy is also set to 1.
>
> Without even considering the network channels local pre-fetch policy and
> applying my assumptions made above.  It seems to me that the client
> subscribe will always have one additional message (total of two)
> awaiting processing.  The first is in the pre-fetch buffer between the
> client and broker B and the other is in the pre-fetch buffer on the
> bridge between broker B and broker A.
>
> Again my subscribing client application can take minutes to process a
> single message.  It had been written to use another JMS implementation
> that did not use pre-fetching and would ACK the message, then perform
> the processing.  But if the application follows that same technique now,
> even without a subscription across the network channel, there will still
> be one additional message in the client?s buffer while the first message
> is being processed.
>
> I guess I would like some clarification on a couple of issues.
>
> - Are my assumptions above correct? And if not where are they wrong.
>
> - What is the effect of the local pre-fetch policy of a network channel?
>
> - Is it possible to use a pre-fetch value of 0 and allow the client to
> initiate retrieval on its own terms?
>
> - How will the adding more networked brokers effect the buffering of
> additional messages for a single client?  Let me elaborate.  Increase
> the total broker count to 4 (A, B, C and D) and create all possible
> cross connections (each broker has 3 network channels).  Now, with a
> producer on broker A and a subscribing client on broker D, I think there
> are 6 messages in the following buffers:
>
> client/broker D - 1 message
> broker D/broker A - 1 message
> broker D/broker B - 1 message
> broker B/broker A - 1 message
> broker D/broker C - 1 message
> broker C/broker A - 1 message
>
> I sure hope my deduction on for this one is way off, because removing
> that many message from play for other possible clients is totally
> unacceptable.
>
>

Reply | Threaded
Open this post in threaded view
|

Re: [activemq-dev] help understanding prefetch please

chirino
In reply to this post by Dennis Cook

On Aug 4, 2005, at 12:32 PM, Dennis Cook wrote:

> Our application makes use of a queue but the processing of each  
> queue message by a client is in the time period of “minutes” not  
> “seconds”. My goal is to have every message published should be  
> readily available to ANY possible client.
>
> However if I understand pre-fetching correctly each client  
> subscription has a buffer (actually I think there are two buffers)  
> pre-filled with messages from the main store.  The buffers reside  
> at each end of a client’s connection; one at client side and one at  
> the broker side of the link. Messages are removed from the store  
> and first put into the broke side buffer.  Another thread then  
> sends the messages across the link to the client side buffer.  The  
> client subscriber then extract the message from the local buffer,  
> and ACKs its acceptance.  When the ACK is received, the message it  
> removed from the broker side buffer.
>
> I hope this is an accurate description of the how pre-fetching  
> works because I am setting my policy for queue pre-fetch to a value  
> of one to obtain my goal stated above.  If I were off track here  
> would someone jump in steer me in the right direction?
>

This is accurate.

> Okay so all looks good in theory if all clients are connected to a  
> single broker.  But the use of networked brokers look like it could  
> cause problems for me.  Network channels are still a little grey  
> for me, but it looks like they are basically implemented in the  
> same manner as a client to broker connection.  While the broker  
> that creates the connection, knows that there is something going  
> on, the broker at the remote end of the connection treats it the  
> same as any other client connection.  Assuming I am correct so far,  
> then this connection should also have a pre-fetch policy.

This is correct.

> But in fact there seems to be two pre fetch policies for a network  
> channel (remote and local).  The remote policy seems to be the  
> equivalent to the one described above, but use of the local is  
> still opaque to me.
>

The reason for the 2 policies is that 2 connections, the local and  
remote connections, are established to bridge messages between the 2  
brokers.  But only the connection consuming messages really cares  
about the pre-fetch policy.

> For further discussion lets assume two networked brokers A and B  
> with a single producer of messages on broker A.  Client subscribers  
> are attached to broker A and one client subscriber is attached to  
> broker B.  In this case the subscription of the client on broker B  
> will establish pre-fetch buffer on its connection and also pre-
> fetch buffers across the network bridge.  This is where thing get  
> very grey for me.  Just how do the policies on the network channel  
> (remote and local) effect the availability of message to other  
> clients (on broker A).  To keep things consistent, lets assume that  
> the network channels remote and local policy is also set to 1.
>

Ok, so there is a pre-fetch policy on the network bridge that slows  
down the delivery of messages across the bridge until the bridge has  
processed the message.  But in the case of network bridge, processing  
the message is very fast, just sends it down to the other broker.  So  
cranking down the pre-fetch policy, only slows moving messages from  
the producer broker to a consumer broker a little.  I guess the  
effect of this is that on the producer broker, where you have a slow  
consumer that is doing slow processing of the message, all the  
messages that are queueing up get sent down to the network bridge  
consumer since he is doing fast processing in comparatively.

So then the other broker starts queueing up messages, since he also  
has a slow consumer.  So the question is, does the other broker start  
sending messages back to originating broker?  This is a little grey  
for me right now.  I think we have loop detection in our networks so  
that messages don't get sent back to the originator so I think they  
just stay in the second broker.  But I need to double check.

> Without even considering the network channels local pre-fetch  
> policy and applying my assumptions made above.  It seems to me that  
> the client subscribe will always have one additional message (total  
> of two) awaiting processing.  The first is in the pre-fetch buffer  
> between the client and broker B and the other is in the pre-fetch  
> buffer on the bridge between broker B and broker A.

I don't think there should be any messages in the pre-fetch buffer  
between the brokers since a message is only there for the length of  
time it takes  to send a message to the next broker which is fast.

>
> Again my subscribing client application can take minutes to process  
> a single message.  It had been written to use another JMS  
> implementation that did not use pre-fetching and would ACK the  
> message, then perform the processing.  But if the application  
> follows that same technique now, even without a subscription across  
> the network channel, there will still be one additional message in  
> the client’s buffer while the first message is being processed.
>
> I guess I would like some clarification on a couple of issues.
>
> - Are my assumptions above correct? And if not where are they wrong.
>
> - What is the effect of the local pre-fetch policy of a network  
> channel?
>

It usually has none.

> - Is it possible to use a pre-fetch value of 0 and allow the client  
> to initiate retrieval on its own terms?
>

I think this would be a great feature that should be implemented.

> - How will the adding more networked brokers effect the buffering  
> of additional messages for a single client?  Let me elaborate.  
> Increase the total broker count to 4 (A, B, C and D) and create all  
> possible cross connections (each broker has 3 network channels).  
> Now, with a producer on broker A and a subscribing client on broker  
> D, I think there are 6 messages in the following buffers:
>
> client/broker D - 1 message
> broker D/broker A - 1 message
> broker D/broker B - 1 message
> broker B/broker A - 1 message
> broker D/broker C - 1 message
> broker C/broker A - 1 message
>
> I sure hope my deduction on for this one is way off, because  
> removing that many message from play for other possible clients is  
> totally unacceptable.
>

I don't see why this scenario would hold.  Since there are no  
consumers on the other brokers, those brokers would not get any of  
the messages.


Regards,
Hiram
Reply | Threaded
Open this post in threaded view
|

[activemq-dev] Re: help understanding prefetch please

Dennis Cook
Hiram Chirino wrote:

>
> On Aug 4, 2005, at 12:32 PM, Dennis Cook wrote:
>
>> Our application makes use of a queue but the processing of each  queue
>> message by a client is in the time period of ?minutes? not  ?seconds?.
>> My goal is to have every message published should be  readily
>> available to ANY possible client.
>>
>> However if I understand pre-fetching correctly each client  
>> subscription has a buffer (actually I think there are two buffers)  
>> pre-filled with messages from the main store.  The buffers reside  at
>> each end of a client?s connection; one at client side and one at  the
>> broker side of the link. Messages are removed from the store  and
>> first put into the broke side buffer.  Another thread then  sends the
>> messages across the link to the client side buffer.  The  client
>> subscriber then extract the message from the local buffer,  and ACKs
>> its acceptance.  When the ACK is received, the message it  removed
>> from the broker side buffer.
>>
>> I hope this is an accurate description of the how pre-fetching  works
>> because I am setting my policy for queue pre-fetch to a value  of one
>> to obtain my goal stated above.  If I were off track here  would
>> someone jump in steer me in the right direction?
>>
>
> This is accurate.
>
>> Okay so all looks good in theory if all clients are connected to a  
>> single broker.  But the use of networked brokers look like it could  
>> cause problems for me.  Network channels are still a little grey  for
>> me, but it looks like they are basically implemented in the  same
>> manner as a client to broker connection.  While the broker  that
>> creates the connection, knows that there is something going  on, the
>> broker at the remote end of the connection treats it the  same as any
>> other client connection.  Assuming I am correct so far,  then this
>> connection should also have a pre-fetch policy.
>
>
> This is correct.
>
>> But in fact there seems to be two pre fetch policies for a network  
>> channel (remote and local).  The remote policy seems to be the  
>> equivalent to the one described above, but use of the local is  still
>> opaque to me.
>>
>
> The reason for the 2 policies is that 2 connections, the local and  
> remote connections, are established to bridge messages between the 2  
> brokers.  But only the connection consuming messages really cares  about
> the pre-fetch policy.

Hmmm, so the remote connection is the equivalenet of to that of a an
external client attaching to a broker.  What purpose is the local
connection? Is it providing a concept of an intermediate message store
from which messages are doled out to clients connected to this broker?

>> For further discussion lets assume two networked brokers A and B  with
>> a single producer of messages on broker A.  Client subscribers  are
>> attached to broker A and one client subscriber is attached to  broker
>> B.  In this case the subscription of the client on broker B  will
>> establish pre-fetch buffer on its connection and also pre- fetch
>> buffers across the network bridge.  This is where thing get  very grey
>> for me.  Just how do the policies on the network channel  (remote and
>> local) effect the availability of message to other  clients (on broker
>> A).  To keep things consistent, lets assume that  the network channels
>> remote and local policy is also set to 1.
>>
>
> Ok, so there is a pre-fetch policy on the network bridge that slows  
> down the delivery of messages across the bridge until the bridge has  
> processed the message.  But in the case of network bridge, processing  
> the message is very fast, just sends it down to the other broker.  So  
> cranking down the pre-fetch policy, only slows moving messages from  the
> producer broker to a consumer broker a little.  I guess the  effect of
> this is that on the producer broker, where you have a slow  consumer
> that is doing slow processing of the message, all the  messages that are
> queueing up get sent down to the network bridge  consumer since he is
> doing fast processing in comparatively.

I am hoping that the network bridge is only sent enough messages to fill
the pre-fetch buffer at the far end of the bridge.

>
> So then the other broker starts queueing up messages, since he also  has
> a slow consumer.  So the question is, does the other broker start  
> sending messages back to originating broker?  This is a little grey  for
> me right now.  I think we have loop detection in our networks so  that
> messages don't get sent back to the originator so I think they  just
> stay in the second broker.  But I need to double check.

Lets not get into the issue of loop detection yet, I have already posted
  something in AMQ-316 about messages echoed back.  But the fact that
there is a slow client on the far end of the bridge does lead me to
believe that the message would be unavailable to consumers on the
producing broker (when no other local message were available) unless
there was some sort of give back/loop back from the far end broker.

>
>> Without even considering the network channels local pre-fetch  policy
>> and applying my assumptions made above.  It seems to me that  the
>> client subscribe will always have one additional message (total  of
>> two) awaiting processing.  The first is in the pre-fetch buffer  
>> between the client and broker B and the other is in the pre-fetch  
>> buffer on the bridge between broker B and broker A.
>
>
> I don't think there should be any messages in the pre-fetch buffer  
> between the brokers since a message is only there for the length of  
> time it takes  to send a message to the next broker which is fast.

If there is some sort of loop back/give back then I would agree that the
re should be none in the bridge buffers.  But that I think that the
buffer on the broker that is mirroring the connecting client still could
be holding the number of messages specified by the client pre-fetch policy.

>
>>
>> Again my subscribing client application can take minutes to process  a
>> single message.  It had been written to use another JMS  
>> implementation that did not use pre-fetching and would ACK the  
>> message, then perform the processing.  But if the application  follows
>> that same technique now, even without a subscription across  the
>> network channel, there will still be one additional message in  the
>> client?s buffer while the first message is being processed.
>>
>> I guess I would like some clarification on a couple of issues.
>>
>> - Are my assumptions above correct? And if not where are they wrong.
>>
>> - What is the effect of the local pre-fetch policy of a network  channel?
>>
>
> It usually has none.
>
>> - Is it possible to use a pre-fetch value of 0 and allow the client  
>> to initiate retrieval on its own terms?
>>
>
> I think this would be a great feature that should be implemented.
>
>> - How will the adding more networked brokers effect the buffering  of
>> additional messages for a single client?  Let me elaborate.   Increase
>> the total broker count to 4 (A, B, C and D) and create all  possible
>> cross connections (each broker has 3 network channels).   Now, with a
>> producer on broker A and a subscribing client on broker  D, I think
>> there are 6 messages in the following buffers:
>>
>> client/broker D - 1 message
>> broker D/broker A - 1 message
>> broker D/broker B - 1 message
>> broker B/broker A - 1 message
>> broker D/broker C - 1 message
>> broker C/broker A - 1 message
>>
>> I sure hope my deduction on for this one is way off, because  removing
>> that many message from play for other possible clients is  totally
>> unacceptable.
>>
>
> I don't see why this scenario would hold.  Since there are no  consumers
> on the other brokers, those brokers would not get any of  the messages.

If all the brokers are cross connected, and producer client is connected
to broker A, I thought the following subscription paths would be
established when the client subsciber on broker D was created.

client->broker D->broker A
         broker D->broker B->broker->A
         broker D->broker C->broker->A

I gathered this info when I noticed that consumer advisories were
forward across network the network bridge as soon as the client creates
the subscription.

So if broker A see three consumers connections (from broker D, broker B
and broker C0 requesting messages. Would following buffers contain a
message:

subscriber mirror buffer on broker D
local connection bridgeD->A
                  bridgeD->B
                  bridgeB->A
                  bridgeD->C
                  bridgeC->A

Dennis Cook