[activemq-dev] Re: [activemq-user] Network of Brokers issues

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[activemq-dev] Re: [activemq-user] Network of Brokers issues

Ramzi Saba
I believe the problem is due to a bug in
ActiveMQConnection.asyncSendPacket(Packet packet, boolean
doSendWhileReconnecting):

    public void asyncSendPacket(Packet packet, boolean
doSendWhileReconnecting)
            throws JMSException {
        if (!closed
                && (doSendWhileReconnecting || transportChannel
                        .isTransportConnected())) {
            packet.setId(packetIdGenerator.getNextShortSequence());
            packet.setReceiptRequired(false);
            if (packet.isJMSMessage() && flowControlSleepTime > 0) {
                try {
                    Thread.sleep(flowControlSleepTime);
                } catch (InterruptedException e) {
                }
            }
            this.transportChannel.asyncSend(packet);
        }
    }

Adding this check prevents the enormous error logging:

    public void asyncSendPacket(Packet packet, boolean
doSendWhileReconnecting)
            throws JMSException {
        if (isTransportOK   // making sure transport is still OK
                && !closed
                && (doSendWhileReconnecting || transportChannel
                        .isTransportConnected())) {
            packet.setId(packetIdGenerator.getNextShortSequence());
            packet.setReceiptRequired(false);
            if (packet.isJMSMessage() && flowControlSleepTime > 0) {
                try {
                    Thread.sleep(flowControlSleepTime);
                } catch (InterruptedException e) {
                }
            }
            this.transportChannel.asyncSend(packet);
        }
    }

I've created a tracker item in JIRA and checked in this code fix into SVN.

I think this will fix your issue, but I would ask you to verify.  You
could either get the latest from SVN and rebuild, or wait until for the
next release.

I'd be also interested in what command you're using to kill the broker?

Thanks.
-ramzi

Christopher Kingsbury wrote:

> Attached is beta's activemq.xml file as well.
>
> Basically I've installed the two brokers in seperate directories,
> using the following structure:
>
> $ find . -type d -print
> .
> ./activemq-3.1-M3
> ./activemq-3.1-M3/bin
> ./activemq-3.1-M3/conf
> ./activemq-3.1-M3/docs
> ./activemq-3.1-M3/example
> ./activemq-3.1-M3/example/conf
> ./activemq-3.1-M3/example/src
> ./activemq-3.1-M3/lib
> ./activemq-3.1-M3/lib/optional
> ./activemq-3.1-M3/var
> ./alpha_activemq-3.1-M3
> ./alpha_activemq-3.1-M3/bin
> ./alpha_activemq-3.1-M3/conf
> ./alpha_activemq-3.1-M3/docs
> ./alpha_activemq-3.1-M3/example
> ./alpha_activemq-3.1-M3/example/conf
> ./alpha_activemq-3.1-M3/example/src
> ./alpha_activemq-3.1-M3/example/src/ddl
> ./alpha_activemq-3.1-M3/example/target
> ./alpha_activemq-3.1-M3/example/target/classes
> ./alpha_activemq-3.1-M3/lib
> ./alpha_activemq-3.1-M3/lib/optional
> ./alpha_activemq-3.1-M3/var
> ./alpha_activemq-3.1-M3/var/derbydb
> ./alpha_activemq-3.1-M3/var/derbydb/log
> ./alpha_activemq-3.1-M3/var/derbydb/seg0
> ./alpha_activemq-3.1-M3/var/derbydb/tmp
> ./alpha_activemq-3.1-M3/var/journal
> ./beta_activemq-3.1-M3
> ./beta_activemq-3.1-M3/bin
> ./beta_activemq-3.1-M3/conf
> ./beta_activemq-3.1-M3/docs
> ./beta_activemq-3.1-M3/example
> ./beta_activemq-3.1-M3/example/conf
> ./beta_activemq-3.1-M3/example/src
> ./beta_activemq-3.1-M3/lib
> ./beta_activemq-3.1-M3/lib/optional
> ./beta_activemq-3.1-M3/var
> ./beta_activemq-3.1-M3/var/derbydb
> ./beta_activemq-3.1-M3/var/derbydb/log
> ./beta_activemq-3.1-M3/var/derbydb/seg0
> ./beta_activemq-3.1-M3/var/derbydb/tmp
> ./beta_activemq-3.1-M3/var/journal
>
> I ran them both on a single windows XP box using cygwin
>
> Thanks,
> Christopher
>
>
> On 6/30/05, *Ramzi Saba* <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Hi Christopher, could you please submit beta's activemq.xml as
>     well?  I
>     presume you are running standalone brokers.  Thanks.
>
>     Christopher Kingsbury wrote:
>
>     > I've just downloaded activemq-3.1-M3 and configured clustering
>     between
>     > 2 brokers (I called them alpha and beta).  To configure clustering I
>     > added the following into the activemq.xml file:
>     >
>     >     <discoveryAgent>
>     >       <zeroconfDiscovery type="_activemq.broker.development."/>
>     >     </discoveryAgent>
>     >     <discoveryNetworkConnector/>
>     >
>     > When I start the brokers up, clustering seems fine, as both brokers
>     > successfully join the cluster, and I'm able to use the example
>     > producer/consumer to send/receive from either of the brokers.
>     >
>     > However, when I shutdown one of the brokers via kill (in this
>     test run
>     > I killed beta), the remaining broker gets into a bad state, and
>     ends
>     > up logging approximately 80 megs worth of stack traces. I've
>     attached
>     > the file activemq_clustering_problems.log.gz which contains the
>     first
>     > 1000 lines + the last 5000 or so lines of the alpha logfile (as I
>     > didn't think anybody would want to see all 80 megs since its really
>     > the same message over and over again, just a bit deeper in a
>     recursive
>     > call)
>     >
>     > After restarting the beta broker I just killed, it seems to join
>     the
>     > cluster properly, but as soon as I connect the consumer example
>     to the
>     > alpha broker, the alpha proceeds to log another 80 megs worth of the
>     > same exceptions.
>     >
>     > I've attached alpha's activemq.xml file as well.
>     >
>     > Have I done something wrong with the configuration?  Seems dead easy
>     > to reproduce, and existed in the 3.1-M2 distribution as well (I have
>     > not tried earlier versions).
>     >
>     > I appreciate any help you can provide,
>     > Christopher Kingsbury
>     >
>     >------------------------------------------------------------------------
>     >
>     ><?xml version="1.0" encoding="UTF-8"?>
>     ><!DOCTYPE beans PUBLIC  "-//ACTIVEMQ//DTD//EN"
>     "http://activemq.org/dtd/activemq.dtd"
>     <http://activemq.org/dtd/activemq.dtd%22>>
>     ><beans>
>     >
>     >  <!--
>     ====================================================================
>     -->
>     >  <!-- ActiveMQ Broker Configuration -->
>     >  <!--
>     ====================================================================
>     -->
>     >  <broker name="alpha">
>     >    <connector>
>     >      <tcpServerTransport uri="tcp://localhost:61616"
>     backlog="1000" useAsyncSend="true" maxOutstandingMessages="50"/>
>     >    </connector>
>     >
>     >    <discoveryAgent>
>     >      <zeroconfDiscovery type="_activemq.broker.development."/>
>     >    </discoveryAgent>
>     >    <discoveryNetworkConnector/>
>     >
>     >    <persistence>
>     >      <cachePersistence>
>     >        <journalPersistence directory="../var/journal">
>     >          <jdbcPersistence dataSourceRef="derby-ds"/>
>     >
>     >          <!-- Choose one of the following three configurationss
>     to work with mysql-ds-->
>     >          <!--
>     >          <jdbcPersistence dataSourceRef="mysql-ds"
>     >                        
>     adapterClass="org.activemq.store.jdbc.adapter.BytesJDBCAdaptor"/>
>     >          <jdbcPersistence dataSourceRef="mysql-ds"
>     >                        
>     adapterClass="org.activemq.store.jdbc.adapter.BlobJDBCAdaptor"/>
>     >          <jdbcPersistence dataSourceRef="mysql-ds"
>     >                        
>     adapterClass="org.activemq.store.jdbc.adapter.StreamJDBCAdaptor"/>
>     >          -->
>     >        </journalPersistence>
>     >      </cachePersistence>
>     >    </persistence>
>     >  </broker>
>     >
>     >  <!--
>     ====================================================================
>     -->
>     >  <!-- JDBC DataSource Configurations -->
>     >  <!--
>     ====================================================================
>     -->
>     >
>     >  <!-- The Derby Datasource that will be used by the Broker -->
>     >  <bean id="derby-ds"
>     class="org.apache.commons.dbcp.BasicDataSource"
>     destroy-method="close">
>     >    <property name="driverClassName">
>     >      <value>org.apache.derby.jdbc.EmbeddedDriver</value>
>     >    </property>
>     >    <property name="url">
>     >      <!-- Use a URL like 'jdbc:hsqldb:hsql://localhost:9001' if
>     you want to connect to a remote hsqldb -->
>     >      <value>jdbc:derby:derbydb;create=true</value>
>     >    </property>
>     >    <property name="username">
>     >      <value></value>
>     >    </property>
>     >    <property name="password">
>     >      <value></value>
>     >    </property>
>     >    <property name="poolPreparedStatements">
>     >      <value>true</value>
>     >    </property>
>     >  </bean>
>     >
>     >  <!-- The MYSQL Datasource that will be used by the Broker -->
>     >  <bean id="mysql-ds"
>     >    class="org.apache.commons.dbcp.BasicDataSource"
>     >    destroy-method="close">
>     >    <property name="driverClassName">
>     >      <value>com.mysql.jdbc.Driver </value>
>     >    </property>
>     >    <property name="url">
>     >      <value>jdbc:mysql://localhost/activemq</value>
>     >    </property>
>     >    <property name="username">
>     >      <value>myname</value>
>     >    </property>
>     >    <property name="password">
>     >      <value>mypassword</value>
>     >    </property>
>     >    <property name="poolPreparedStatements">
>     >      <value>true</value>
>     >    </property>
>     >  </bean>
>     >
>     >
>     ></beans>
>     >
>     >
>     >
>
>     --
>     Ramzi Saba
>     Optaros, Inc - www.optaros.com <http://www.optaros.com>
>
>
>
>------------------------------------------------------------------------
>
><?xml version="1.0" encoding="UTF-8"?>
><!DOCTYPE beans PUBLIC  "-//ACTIVEMQ//DTD//EN" "http://activemq.org/dtd/activemq.dtd">
><beans>
>  
>  <!-- ==================================================================== -->
>  <!-- ActiveMQ Broker Configuration -->
>  <!-- ==================================================================== -->
>  <broker name="beta">
>    <connector>
>      <tcpServerTransport uri="tcp://localhost:61617" backlog="1000" useAsyncSend="true" maxOutstandingMessages="50"/>
>    </connector>
>
>    <discoveryAgent>
>      <zeroconfDiscovery type="_activemq.broker.development."/>
>    </discoveryAgent>
>    <discoveryNetworkConnector/>
>
>    <persistence>
>      <cachePersistence>
>        <journalPersistence directory="../var/journal">
>          <jdbcPersistence dataSourceRef="derby-ds"/>
>
>          <!-- Choose one of the following three configurationss to work with mysql-ds-->
>          <!--
>          <jdbcPersistence dataSourceRef="mysql-ds"
>                         adapterClass="org.activemq.store.jdbc.adapter.BytesJDBCAdaptor"/>
>          <jdbcPersistence dataSourceRef="mysql-ds"
>                         adapterClass="org.activemq.store.jdbc.adapter.BlobJDBCAdaptor"/>
>          <jdbcPersistence dataSourceRef="mysql-ds"
>                         adapterClass="org.activemq.store.jdbc.adapter.StreamJDBCAdaptor"/>
>          -->
>        </journalPersistence>
>      </cachePersistence>
>    </persistence>
>  </broker>
>
>  <!-- ==================================================================== -->
>  <!-- JDBC DataSource Configurations -->
>  <!-- ==================================================================== -->
>
>  <!-- The Derby Datasource that will be used by the Broker -->
>  <bean id="derby-ds" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
>    <property name="driverClassName">
>      <value>org.apache.derby.jdbc.EmbeddedDriver</value>
>    </property>
>    <property name="url">
>      <!-- Use a URL like 'jdbc:hsqldb:hsql://localhost:9001' if you want to connect to a remote hsqldb -->
>      <value>jdbc:derby:derbydb;create=true</value>
>    </property>
>    <property name="username">
>      <value></value>
>    </property>
>    <property name="password">
>      <value></value>
>    </property>
>    <property name="poolPreparedStatements">
>      <value>true</value>
>    </property>
>  </bean>
>
>  <!-- The MYSQL Datasource that will be used by the Broker -->
>  <bean id="mysql-ds"
>    class="org.apache.commons.dbcp.BasicDataSource"
>    destroy-method="close">
>    <property name="driverClassName">
>      <value>com.mysql.jdbc.Driver</value>
>    </property>
>    <property name="url">
>      <value>jdbc:mysql://localhost/activemq</value>
>    </property>
>    <property name="username">
>      <value>myname</value>
>    </property>
>    <property name="password">
>      <value>mypassword</value>
>    </property>
>    <property name="poolPreparedStatements">
>      <value>true</value>
>    </property>
>  </bean>
>
>
></beans>
>
>  
>

--
Ramzi Saba
Optaros, Inc - www.optaros.com