Artemis scheduled thread pool max size

classic Classic list List threaded Threaded
1 message Options
slx
Reply | Threaded
Open this post in threaded view
|

Artemis scheduled thread pool max size

slx
Hello,

I have a question about the effects of the the scheduled pool max size
in artemis (2.10.1).
I had a recurring issue with an embedded artemis instance which I had
significant trouble to track down.
The application stack uses a small number (3) of internal consumers do
process messages, connected using the vm transport using pooledJms (and
spring DefaultMessageListenerContainer) and a a few JmsTemplate
instances for sending responses (also using pooledJMS ; removing
pooledJMS did not help)

The failure occured while sending a message inside the server
environment; from "client" side it looks like that :

Caused by: javax.jms.JMSException

    at
org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:464)
    at
org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:358)
    at
org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager.createSessionContext(ActiveMQClientProtocolManager.java:300)
    at
org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager.createSessionContext(ActiveMQClientProtocolManager.java:249)
    at
org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.createSessionChannel(ClientSessionFactoryImpl.java:1348)
    at
org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.createSessionInternal(ClientSessionFactoryImpl.java:673)
    at
org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.createSession(ClientSessionFactoryImpl.java:298)
    at
org.apache.activemq.artemis.jms.client.ActiveMQConnection.authorize(ActiveMQConnection.java:684)
    at
org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createConnectionInternal(ActiveMQConnectionFactory.java:920)
    at
org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:299)
    at
org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:294)
    at
org.messaginghub.pooled.jms.JmsPoolConnectionFactory.createProviderConnection(JmsPoolConnectionFactory.java:650)
    at
org.messaginghub.pooled.jms.JmsPoolConnectionFactory$1.makeObject(JmsPoolConnectionFactory.java:107)
    at
org.messaginghub.pooled.jms.JmsPoolConnectionFactory$1.makeObject(JmsPoolConnectionFactory.java:104)
    at
org.apache.commons.pool2.impl.GenericKeyedObjectPool.create(GenericKeyedObjectPool.java:1041)
    at
org.apache.commons.pool2.impl.GenericKeyedObjectPool.addObject(GenericKeyedObjectPool.java:1233)
    at
org.messaginghub.pooled.jms.JmsPoolConnectionFactory.createJmsPoolConnection(JmsPoolConnectionFactory.java:704)
    at
org.messaginghub.pooled.jms.JmsPoolConnectionFactory.createConnection(JmsPoolConnectionFactory.java:236)
    at
org.messaginghub.pooled.jms.JmsPoolConnectionFactory.createConnection(JmsPoolConnectionFactory.java:231)
    at
org.springframework.jms.support.JmsAccessor.createConnection(JmsAccessor.java:196)
    at
org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:494)
Caused by: ActiveMQInternalErrorException[errorType=INTERNAL_ERROR
message=null] ... 28 more

On "server" (artemis-side), it produces a debug trace like that

2019-10-17 10:44:07,853 DEBUG {Thread-8
(ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@66c08833)}
[o.a.a.artemis.core.server.impl.QueueImpl] :
QueueImpl[name=FullEvent.2.In, postOffice=PostOfficeImpl
[server=ActiveMQServerImpl::serverUUID=c7fa0fe7-6688-11e9-9127-005056aaab0e],
temp=false]@6c22dc9a::All the consumers were busy, giving up now

Note : the failing send occurs in the context of a thread of a scheduled
timer task created by the application (end of batch notification emitted
every 30s).
At some point, if the real client (i.e. client side, netty connector
distinct from server-side clients using the vm transport) emit new
messages, the consumer processes, another send is triggered and the
previous "stuck" send would appear in its destination queue.
After tinkering with quite a lot the artemis parameters, the issue
finally went away when I increased <scheduled-thread-pool-max-size> from
10 to 20.

According to the documentation here :
https://activemq.apache.org/components/artemis/documentation/latest/thread-pooling.html
the default value is 5 and "usually sufficient". Is there any ressources
available with deeper explanation of this parameter and guidance on the
sizing policy by use cases or is this caused by an issue with the
implementation ?

Regards,

Sébastien