Ignoring sub command log messages

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Ignoring sub command log messages

xabhi
Hi,

I saw below log messages in my ActiveMQ instance's logs. When does this happen and what effects can it have?

[20170108 15:29:19.584 EST (main) org.apache.activemq.store.kahadb.MessageDatabase#updateIndex 1405 DEBUG] - no message sequence exists for id: ID:geneva13.nyc.43322-1482156880537-3:13196:1:1:3 and sub: gear-munshi-geneva.gear.sync.listener.GelbaseGenevaMessageListener
[20170108 15:29:19.584 EST (main) org.apache.activemq.store.kahadb.MessageDatabase#updateIndex 1405 DEBUG] - no message sequence exists for id: ID:geneva13.nyc.43322-1482156880537-3:13196:1:1:4 and sub: gear-munshi-geneva.gear.sync.listener.GelbaseGenevaMessageListener
[20170108 15:29:19.584 EST (main) org.apache.activemq.store.kahadb.MessageDatabase#updateIndex 1405 DEBUG] - no message sequence exists for id: ID:geneva13.nyc.43322-1482156880537-3:1:2:1:5946 and sub: gear-munshi-listener:geneva.gear.sync.listener.GelbaseGenevaMessageListener
[20170108 15:29:19.585 EST (main) org.apache.activemq.store.kahadb.MessageDatabase#process 1047 DEBUG] - ignoring add sub command during recovery replay:destination {
  type: TOPIC
  name: geneva.loader.messageTopic
}
subscriptionKey: gear-munshi-listener:geneva.gear.sync.listener.GelbaseGenevaMessageListener
retroactive: false
subscriptionInfo: org.apache.activemq.protobuf.Buffer@ffffffc5
 
[20170108 15:29:19.585 EST (main) org.apache.activemq.store.kahadb.MessageDatabase#process 1047 DEBUG] - ignoring add sub command during recovery replay:destination {
  type: TOPIC
  name: gear.tdpBundleCompletionTopic
}
subscriptionKey: post-tdp-bundle-listener:geneva.gear.sync.listener.PostTdpMessageListener
retroactive: false
subscriptionInfo: org.apache.activemq.protobuf.Buffer@ffffffea


Thanks,
Abhi
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Ignoring sub command log messages

christopher.l.shannon
Did this happen while replaying the journal?  This message means that
an acknowledgement was processed in the journal that doesn't have a
matching message.  This log message would typically print out if you
were replaying the journal from the beginning to rebuild the index
because it's possible that the original journal files that contained
those messages have already been garbage collected but the journal
file that contains the acks hasn't been garbage collected yet.  If
this is the case then this is normal and nothing to worry about (which
is why it's a debug statement only)

On Tue, Jan 10, 2017 at 2:25 AM, xabhi <[hidden email]> wrote:

> Hi,
>
> I saw below log messages in my ActiveMQ instance's logs. When does this
> happen and what effects can it have?
>
> [20170108 15:29:19.584 EST (main)
> org.apache.activemq.store.kahadb.MessageDatabase#updateIndex 1405 DEBUG] -
> no message sequence exists for id:
> ID:geneva13.nyc.43322-1482156880537-3:13196:1:1:3 and sub:
> gear-munshi-geneva.gear.sync.listener.GelbaseGenevaMessageListener
> [20170108 15:29:19.584 EST (main)
> org.apache.activemq.store.kahadb.MessageDatabase#updateIndex 1405 DEBUG] -
> no message sequence exists for id:
> ID:geneva13.nyc.43322-1482156880537-3:13196:1:1:4 and sub:
> gear-munshi-geneva.gear.sync.listener.GelbaseGenevaMessageListener
> [20170108 15:29:19.584 EST (main)
> org.apache.activemq.store.kahadb.MessageDatabase#updateIndex 1405 DEBUG] -
> no message sequence exists for id:
> ID:geneva13.nyc.43322-1482156880537-3:1:2:1:5946 and sub:
> gear-munshi-listener:geneva.gear.sync.listener.GelbaseGenevaMessageListener
> [20170108 15:29:19.585 EST (main)
> org.apache.activemq.store.kahadb.MessageDatabase#process 1047 DEBUG] -
> ignoring add sub command during recovery replay:destination {
>   type: TOPIC
>   name: geneva.loader.messageTopic
> }
> subscriptionKey:
> gear-munshi-listener:geneva.gear.sync.listener.GelbaseGenevaMessageListener
> retroactive: false
> subscriptionInfo: org.apache.activemq.protobuf.Buffer@ffffffc5
>
> [20170108 15:29:19.585 EST (main)
> org.apache.activemq.store.kahadb.MessageDatabase#process 1047 DEBUG] -
> ignoring add sub command during recovery replay:destination {
>   type: TOPIC
>   name: gear.tdpBundleCompletionTopic
> }
> subscriptionKey:
> post-tdp-bundle-listener:geneva.gear.sync.listener.PostTdpMessageListener
> retroactive: false
> subscriptionInfo: org.apache.activemq.protobuf.Buffer@ffffffea
>
>
> Thanks,
> Abhi
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/Ignoring-sub-command-log-messages-tp4720928.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Ignoring sub command log messages

xabhi
Hi,

Yes these log messages appeared at the startup only. I had faced an issue where activemq dumped  7gb file - tx-14693130-1483907222896.tmp in one of the mkahadb directories when i tried to destroy a durable topic consumer which was inactive for quite some time and had hit the memory threshold of 70% (this was causing activemq to be unresponsive). I removed this file manually and when failover switch happened I saw above log messages in log file.

Is there a clean way to discard all the messages in durable topic subscription without dumping a 7gb file?

Thanks,
Abhishek
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Ignoring sub command log messages

christopher.l.shannon
Yes if you have a ton of data on a durable subscription and delete the
durable it tends to write a large temp file because of the kahadb
index.  Even though the temp file should be deleted automatically
after deletion this is certainly not ideal.

Right now there isn't really a good way to discard all of the messages
other than the current deletion.  I've been working on a prototype for
purging a durable subscription that works similar to how purge works
for a Queue in that it iterates over the messages and acks them one by
one so it doesn't blow memory trying to delete them all at once.  I
haven't added it to the broker yet but it seems to work ok in my local
testing and is something I can probably add to the 5.15.0 release.

Also in regards to the inactive durable sub using up all of the
memory, that's something else I want to improve in 5.15.0.  If you
look at the discussion from
https://issues.apache.org/jira/browse/AMQ-4467 from the last couple of
days this issue came up.  One solution discussed is purging an
inactive destination from the in memory cache (but not from the store)
to save memory for other destinations.

On Wed, Jan 11, 2017 at 8:19 AM, xabhi <[hidden email]> wrote:

> Hi,
>
> Yes these log messages appeared at the startup only. I had faced an issue
> where activemq dumped  7gb file - tx-14693130-1483907222896.tmp in one of
> the mkahadb directories when i tried to destroy a durable topic consumer
> which was inactive for quite some time and had hit the memory threshold of
> 70% (this was causing activemq to be unresponsive). I removed this file
> manually and when failover switch happened I saw above log messages in log
> file.
>
> Is there a clean way to discard all the messages in durable topic
> subscription without dumping a 7gb file?
>
> Thanks,
> Abhishek
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/Ignoring-sub-command-log-messages-tp4720928p4720974.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Ignoring sub command log messages

xabhi
Thanks christopher for the explanation. This makes sense. What are the timelines for ActiveMQ v5.15.0 release - mid year? I am asking because I plan to upgrade to 5.14.3 and if this issue will be handled well in 5.15.0 i would love to upgrade directly to 5.15.0.

Thanks,
Abhi

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Ignoring sub command log messages

christopher.l.shannon
It's hard to say an exact time frame.  Finished new features would be
the driving force behind a 5.15.0 release as bug fixes are already
backported but right now there hasn't been much done to justify a
5.15.0 release.  Most work has been focused on Artemis but I would say
that mid year sounds about right to me.

On Wed, Jan 11, 2017 at 9:24 AM, xabhi <[hidden email]> wrote:

> Thanks christopher for the explanation. This makes sense. What are the
> timelines for ActiveMQ v5.15.0 release - mid year? I am asking because I
> plan to upgrade to 5.14.3 and if this issue will be handled well in 5.15.0 i
> would love to upgrade directly to 5.15.0.
>
> Thanks,
> Abhi
>
>
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/Ignoring-sub-command-log-messages-tp4720928p4720977.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Loading...