Federation using Bi-Directional/Duplex between Artemis brokers

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Federation using Bi-Directional/Duplex between Artemis brokers

sateesh kumar kapu
We are trying to implement a hub and spoke topology using the Federation
feature by configuring both upstream and downstream connections.  In our
case, we wanted to configure everything that is needed for the federation on
the broker which is acting as spoke.  This is mainly done this way as our
main requirement was hub should not be aware of any spoke as spokes can be
behind the firewall and are added or removed dynamically.

     In order to achieve this, In the downstream configuration, we set a
share-connection flag that is supposed to use the same upstream connection
to use for downstream communication. However, it seems we also required to
configure the upstream-connector-ref which will be used by the hub to create
a new connection to communicate. This is making Hub to aware of spoke which
is violating our intend. Ideally, we want to avoid spoke's hostname the
connector that is given in the upstream-connector-ref configuration.

    Is there any way to make the same connection Bi-directional/Duplex?        
    In addition to that as spokes can be behind the firewall where only HTTP
connections are allowed, can federation done via HTTP?


Broker.xml on Hub

<configuration xmlns="urn:activemq"
               xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
               xmlns:xi="http://www.w3.org/2001/XInclude"
               xsi:schemaLocation="urn:activemq
/schema/artemis-configuration.xsd">
   <core xmlns="urn:activemq:core"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="urn:activemq:core ">
      <name>0.0.0.0</name>
       …
       <connectors>
               <connector
name="netty-connector">tcp://0.0.0.0:61618</connector>
      </connectors>
       <acceptors>
               <acceptor
name="netty-acceptor">tcp://0.0.0.0:61618</acceptor>
       </acceptors>
       <addresses>
               <address name="exampleTopic">
                               <multicast>
                                              <queue
name="exampleSubscription"/>
                               </multicast>
               </address>
      </addresses>
  </core>
</configuration>


Broker.xml on Spoke.
<configuration xmlns="urn:activemq"
               xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
               xmlns:xi="http://www.w3.org/2001/XInclude"
               xsi:schemaLocation="urn:activemq
/schema/artemis-configuration.xsd">
   <core xmlns="urn:activemq:core"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="urn:activemq:core ">
      <name>0.0.0.0</name>
       …
       <connectors>
               <connector
name="netty-connector">tcp://spokeHostName:61618</connector>
              <connector
name="us-central-1-connector">tcp://hubHostName:61618</connector>
      </connectors>
     <acceptors>
               <acceptor
name="netty-acceptor">tcp://0.0.0.0:61618</acceptor>
       </acceptors>
       <federations>
               <federation name="eu-west-1-federation">
                               <upstream name="us-central-1-upstream">
                                             
<circuit-breaker-timeout>1000</circuit-breaker-timeout>
                                             
<share-connection>true</share-connection>
                                              <static-connectors>
                                                             
<connector-ref>us-central-1-connector</connector-ref>
                                              </static-connectors>
                                              <policy ref="policySetA"/>
                               </upstream>
                               <downstream name="us-central-1-downstream">
                                             
<circuit-breaker-timeout>1000</circuit-breaker-timeout>
                                             
<share-connection>true</share-connection>
                                              <static-connectors>
                                                             
<connector-ref>us-central-1-connector</connector-ref>
                                              </static-connectors>
                                              <policy ref="policySetA"/>
                                             
<upstream-connector-ref>netty-connector</upstream-connector-ref>
                               </downstream>
                               <policy-set name="policySetA">
                                              <policy
ref="address-federation" />
                               </policy-set>
                               <address-policy name="address-federation">
                                              <include
address-match="exampleTopic" />
                               </address-policy>
               </federation>
      </federations>
       <addresses>
               <address name="exampleTopic">
                               <multicast>
                                              <queue
name="exampleSubscription"/>
                               </multicast>
               </address>
      </addresses>
  </core>
</configuration>

Thanks in advance.





--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: Federation using Bi-Directional/Duplex between Artemis brokers

christopher.l.shannon
There is currently no duplex option but it is on the list of things in the
future. Originally there was only upstream connections but I recently added
downstream configuration but it still requires creating a connection in
each direction.

The main issue is there is no current support for duplex connections in
Artemis to make it easy to implement (ServerLocator, etc create a new
connection and can't use existing).  So it will take a little work to get
there but ultimately having real duplex connection is nice so the remote
broker doesn't have to know how to create a new connection.

On Wed, Feb 26, 2020 at 5:00 PM satees kumar kapu <[hidden email]>
wrote:

> We are trying to implement a hub and spoke topology using the Federation
> feature by configuring both upstream and downstream connections.  In our
> case, we wanted to configure everything that is needed for the federation
> on
> the broker which is acting as spoke.  This is mainly done this way as our
> main requirement was hub should not be aware of any spoke as spokes can be
> behind the firewall and are added or removed dynamically.
>
>      In order to achieve this, In the downstream configuration, we set a
> share-connection flag that is supposed to use the same upstream connection
> to use for downstream communication. However, it seems we also required to
> configure the upstream-connector-ref which will be used by the hub to
> create
> a new connection to communicate. This is making Hub to aware of spoke which
> is violating our intend. Ideally, we want to avoid spoke's hostname the
> connector that is given in the upstream-connector-ref configuration.
>
>     Is there any way to make the same connection Bi-directional/Duplex?
>
>     In addition to that as spokes can be behind the firewall where only
> HTTP
> connections are allowed, can federation done via HTTP?
>
>
> Broker.xml on Hub
>
> <configuration xmlns="urn:activemq"
>                xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>                xmlns:xi="http://www.w3.org/2001/XInclude"
>                xsi:schemaLocation="urn:activemq
> /schema/artemis-configuration.xsd">
>    <core xmlns="urn:activemq:core"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>          xsi:schemaLocation="urn:activemq:core ">
>       <name>0.0.0.0</name>
>        …
>        <connectors>
>                <connector
> name="netty-connector">tcp://0.0.0.0:61618</connector>
>       </connectors>
>        <acceptors>
>                <acceptor
> name="netty-acceptor">tcp://0.0.0.0:61618</acceptor>
>        </acceptors>
>        <addresses>
>                <address name="exampleTopic">
>                                <multicast>
>                                               <queue
> name="exampleSubscription"/>
>                                </multicast>
>                </address>
>       </addresses>
>   </core>
> </configuration>
>
>
> Broker.xml on Spoke.
> <configuration xmlns="urn:activemq"
>                xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>                xmlns:xi="http://www.w3.org/2001/XInclude"
>                xsi:schemaLocation="urn:activemq
> /schema/artemis-configuration.xsd">
>    <core xmlns="urn:activemq:core"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>          xsi:schemaLocation="urn:activemq:core ">
>       <name>0.0.0.0</name>
>        …
>        <connectors>
>                <connector
> name="netty-connector">tcp://spokeHostName:61618</connector>
>               <connector
> name="us-central-1-connector">tcp://hubHostName:61618</connector>
>       </connectors>
>      <acceptors>
>                <acceptor
> name="netty-acceptor">tcp://0.0.0.0:61618</acceptor>
>        </acceptors>
>        <federations>
>                <federation name="eu-west-1-federation">
>                                <upstream name="us-central-1-upstream">
>
> <circuit-breaker-timeout>1000</circuit-breaker-timeout>
>
> <share-connection>true</share-connection>
>                                               <static-connectors>
>
> <connector-ref>us-central-1-connector</connector-ref>
>                                               </static-connectors>
>                                               <policy ref="policySetA"/>
>                                </upstream>
>                                <downstream name="us-central-1-downstream">
>
> <circuit-breaker-timeout>1000</circuit-breaker-timeout>
>
> <share-connection>true</share-connection>
>                                               <static-connectors>
>
> <connector-ref>us-central-1-connector</connector-ref>
>                                               </static-connectors>
>                                               <policy ref="policySetA"/>
>
> <upstream-connector-ref>netty-connector</upstream-connector-ref>
>                                </downstream>
>                                <policy-set name="policySetA">
>                                               <policy
> ref="address-federation" />
>                                </policy-set>
>                                <address-policy name="address-federation">
>                                               <include
> address-match="exampleTopic" />
>                                </address-policy>
>                </federation>
>       </federations>
>        <addresses>
>                <address name="exampleTopic">
>                                <multicast>
>                                               <queue
> name="exampleSubscription"/>
>                                </multicast>
>                </address>
>       </addresses>
>   </core>
> </configuration>
>
> Thanks in advance.
>
>
>
>
>
> --
> Sent from:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
>
Reply | Threaded
Open this post in threaded view
|

Re: Federation using Bi-Directional/Duplex between Artemis brokers

sateesh kumar kapu
Thanks Chris,
Currently we are using ActiveMQ5.x and planning to switch to Artemis  to
leverage its benefits but it appears Duplex feature is a deal breaker for
now to make this switch.

Even it is required two connections to federate, if they initiated by one
broker then it would have solved our problem.

Anyways we stay tune to get updates on this.




--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: Federation using Bi-Directional/Duplex between Artemis brokers

christopher.l.shannon
I am coming from a 5.x background as well with heavy usage of duplex
network connectors for the same reason as you (such as firewall
configurations) so I definitely understand it being a deal breaker.  I'm
not sure when I will get time to work on it more but I'm hoping to soon as
it will make Federation much more useful in a lot of cases.

On Thu, Feb 27, 2020 at 11:17 AM sateesh kumar kapu <[hidden email]>
wrote:

> Thanks Chris,
> Currently we are using ActiveMQ5.x and planning to switch to Artemis  to
> leverage its benefits but it appears Duplex feature is a deal breaker for
> now to make this switch.
>
> Even it is required two connections to federate, if they initiated by one
> broker then it would have solved our problem.
>
> Anyways we stay tune to get updates on this.
>
>
>
>
> --
> Sent from:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
>