Selectors with virtual topics in broker networks

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Selectors with virtual topics in broker networks

Devlin
This post was updated on .
5.11.0

Trying to validate if our current setup is configured properly for virtual
topics with message selectors in a broker network. Based on preliminary
observations, virtual topics appear to works as expected; producers send on
topics, consumers receive on queues, regardless of where they reside in the
network. We did, however, observe the following problem:

Virtual topic producer sends 10 messages, consumer gets 7 messages, then we
kill the consumer to force a reconnect. When consumer connects back, the
remaining 3 messages are not forwarded to the consumer, they appear stuck on
the queue. If the producer sends new messages, the consumer, oddly, receives
the new messages, but messages 8, 9, 10 remain on the queue.

I found this post [1], it suggests disabling conduit subs on queue connectors,
but enabling it on topic connectors (currently we disable it for both).

Wanted to share our current setup before attempting config changes based on post above,

## Virtual topic interceptor
## We deviated from standard virtual topic conventions due to our security
model, but it works
<virtualDestinationInterceptor>
        <virtualDestinations>
            <virtualTopic   name="*.*.*.vtopic.*"   prefix=""   postfix=".*"
selectorAware="true" />


## network connector
         
          <networkConnector
                name="queue-connector1"
                networkTTL="12"
                conduitSubscriptions="false"
                suppressDuplicateQueueSubscriptions="true">
                <dynamicallyIncludedDestinations>
                    <queue physicalName=">"/>
                </dynamicallyIncludedDestinations>
            </networkConnector>
                     
            <networkConnector
                name="topic-connector-1"
                networkTTL="12"  
                conduitSubscriptions="false">
                <dynamicallyIncludedDestinations>
                    <topic physicalName=">"/>
                </dynamicallyIncludedDestinations>
            </networkConnector> 


[1] - http://activemq.2283324.n4.nabble.com/conduit-subscriptions-td2365115.html#a2365117 



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: Selectors with virtual topics in broker networks

Tim Bain
Which hosts are the producer and consumer on? Could this be the problem
described at the bottom of
http://activemq.apache.org/how-do-i-use-durable-subscribers-in-a-network-of-brokers.html
?

On Dec 4, 2017 8:56 AM, "Devlin" <[hidden email]> wrote:

> 5.11.0
>
> Trying to validate if our current setup is configured properly for virtual
> topics with message selectors in a broker network. Based on preliminary
> observations, virtual topics appear to works as expected; producers send on
> topics, consumers receive on queues, regardless of where they reside in the
> network. We did, however, observe the following problem:
>
> Virtual topic producer sends 10 messages, consumer gets 7 messages, then we
> kill the consumer to force a reconnect. When consumer connects back, the
> remaining 3 messages are not forwarded to the consumer, they appear stuck
> on
> the queue. If the producer sends new messages, the consumer, oddly,
> receives
> the new messages.
>
> I found this post, it suggests disabling conduit subs on queue connectors,
> but enabling it on topic connectors (currently we disable it for both)
> http://activemq.2283324.n4.nabble.com/conduit-
> subscriptions-td2365115.html#a2365117
>
> ## Virtual topic interceptor
> ## We deviated from standard virtual topic conventions due to our security
> model, but it works
> <virtualDestinationInterceptor>
>         <virtualDestinations>
>             <virtualTopic   name="*.*.*.vtopic.*"   prefix=""
>  postfix=".*"
> selectorAware="true" />
>
>
> ## network connector
>
>           <networkConnector
>                 name="queue-connector1"
>                 networkTTL="12"
>                 conduitSubscriptions="false"
>                 suppressDuplicateQueueSubscriptions="true">
>                 <dynamicallyIncludedDestinations>
>                     <queue physicalName=">"/>
>                 </dynamicallyIncludedDestinations>
>             </networkConnector>
>
>             <networkConnector
>                 name="topic-connector-1"
>                 networkTTL="12"
>                 conduitSubscriptions="false">
>                 <dynamicallyIncludedDestinations>
>                     <topic physicalName=">"/>
>                 </dynamicallyIncludedDestinations>
>             </networkConnector>
>
>
>
>
>
> --
> Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-
> f2341805.html
>
Reply | Threaded
Open this post in threaded view
|

Re: Selectors with virtual topics in broker networks

Devlin
Hi Tim,

Different hosts.

We used conditionalNetworkBridgeFilterFactory in the past, but removed it
months ago thinking it was causing a different issue, turned out to be
false, so we are considering enabling it again. What I don't like about this
filter is that it only kicks in on broker startup when no consumers are
active. If we set replayWhenNoConsumers="false", does the filter constantly
replay messages, irrespective of whether consumers exist or not?

<destinationPolicy>
  <policyMap>
    <policyEntries>
      <policyEntry queue="TEST.>" enableAudit="false">
        <conditionalNetworkBridgeFilterFactory
replayWhenNoConsumers="true"/>
      </policyEntry>
    </policyEntries>
  </policyMap>
</destinationPolicy>



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: Selectors with virtual topics in broker networks

Tim Bain
The name is a little misleading, but it doesn't do what you're afraid it
will. A better name would be
replayToRemoteBrokerWithAtLeastOneConsumerWhenNoLocalConsumers. But most
people don't like long names, so instead it's the ambiguous
replayWhenNoConsumers.

Messages are still forwarded only on demand (unless you have
staticallyIncludedDestinations set), but when no local consumers exist, the
broker is allowed to disregard the normal rule that a message may not be
sent back to a broker it's already been to. Nothing more.

Tim

On Dec 7, 2017 6:51 AM, "Devlin" <[hidden email]> wrote:

> Hi Tim,
>
> Different hosts.
>
> We used conditionalNetworkBridgeFilterFactory in the past, but removed it
> months ago thinking it was causing a different issue, turned out to be
> false, so we are considering enabling it again. What I don't like about
> this
> filter is that it only kicks in on broker startup when no consumers are
> active. If we set replayWhenNoConsumers="false", does the filter constantly
> replay messages, irrespective of whether consumers exist or not?
>
> <destinationPolicy>
>   <policyMap>
>     <policyEntries>
>       <policyEntry queue="TEST.>" enableAudit="false">
>         <conditionalNetworkBridgeFilterFactory
> replayWhenNoConsumers="true"/>
>       </policyEntry>
>     </policyEntries>
>   </policyMap>
> </destinationPolicy>
>
>
>
> --
> Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-
> f2341805.html
>