Meaning of log "not removing data file"

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Meaning of log "not removing data file"

itelleria
Hi,

We are having some issues with kahadb. We see that some journal files, named
"db-", are not removed and this makes ActiveMQ to use a lot of disk space.

We've activated ActiveMQ logs and there are some logs with this information:

 2018-04-23 11:32:52,218 [eckpoint Worker] TRACE MessageDatabase              
- not removing data file: 348 as contained ack(s) refer to referenced file:
[348, 234]
 2018-04-23 11:32:52,218 [eckpoint Worker] TRACE MessageDatabase              
- not removing data file: 357 as contained ack(s) refer to referenced file:
[356, 357]
 2018-04-23 11:32:52,218 [eckpoint Worker] TRACE MessageDatabase              
- not removing data file: 369 as contained ack(s) refer to referenced file:
[368, 369]
 2018-04-23 11:32:52,218 [eckpoint Worker] TRACE MessageDatabase              
- not removing data file: 370 as contained ack(s) refer to referenced file:
[369, 370]
 2018-04-23 11:32:52,218 [eckpoint Worker] TRACE MessageDatabase              
- not removing data file: 371 as contained ack(s) refer to referenced file:
[370, 371]
 2018-04-23 11:32:52,218 [eckpoint Worker] TRACE MessageDatabase              
- not removing data file: 372 as contained ack(s) refer to referenced file:
[372, 371]
 2018-04-23 11:32:52,218 [eckpoint Worker] TRACE MessageDatabase              
- not removing data file: 396 as contained ack(s) refer to referenced file:
[396, 395]

I know that a journal file is not removed until all its messaged are ACKed.
However, I don't understand the meaning of this message:  *not removing data
file: X as contained ack(s) refer to referenced file: [X, Y]* Apparently the
journal file with X id is not removed because it contains messages with
references to another journal file. I don't understand how this can be
possible.

I've searched information about this message but I haven't found a clear
answer. Could you point me to a resource with information about this?

Thanks in advance.

Iñigo



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: Meaning of log "not removing data file"

Tim Bain-2
It means that either the file contains an unconsumed message, or it
contains acks for one or more messages in a data file that's being kept, or
both.

http://activemq.apache.org/why-do-kahadb-log-files-remain-after-cleanup.html
has additional explanation.

Tim


On Apr 23, 2018 4:41 AM, "itelleria" <[hidden email]> wrote:

Hi,

We are having some issues with kahadb. We see that some journal files, named
"db-", are not removed and this makes ActiveMQ to use a lot of disk space.

We've activated ActiveMQ logs and there are some logs with this information:

 2018-04-23 11:32:52,218 [eckpoint Worker] TRACE MessageDatabase

- not removing data file: 348 as contained ack(s) refer to referenced file:
[348, 234]
 2018-04-23 11:32:52,218 [eckpoint Worker] TRACE MessageDatabase

- not removing data file: 357 as contained ack(s) refer to referenced file:
[356, 357]
 2018-04-23 11:32:52,218 [eckpoint Worker] TRACE MessageDatabase

- not removing data file: 369 as contained ack(s) refer to referenced file:
[368, 369]
 2018-04-23 11:32:52,218 [eckpoint Worker] TRACE MessageDatabase

- not removing data file: 370 as contained ack(s) refer to referenced file:
[369, 370]
 2018-04-23 11:32:52,218 [eckpoint Worker] TRACE MessageDatabase

- not removing data file: 371 as contained ack(s) refer to referenced file:
[370, 371]
 2018-04-23 11:32:52,218 [eckpoint Worker] TRACE MessageDatabase

- not removing data file: 372 as contained ack(s) refer to referenced file:
[372, 371]
 2018-04-23 11:32:52,218 [eckpoint Worker] TRACE MessageDatabase

- not removing data file: 396 as contained ack(s) refer to referenced file:
[396, 395]

I know that a journal file is not removed until all its messaged are ACKed.
However, I don't understand the meaning of this message:  *not removing data
file: X as contained ack(s) refer to referenced file: [X, Y]* Apparently the
journal file with X id is not removed because it contains messages with
references to another journal file. I don't understand how this can be
possible.

I've searched information about this message but I haven't found a clear
answer. Could you point me to a resource with information about this?

Thanks in advance.

Iñigo



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Reply | Threaded
Open this post in threaded view
|

Re: Meaning of log "not removing data file"

itelleria
Thanks Tim.

I didn't know that ack messages are stored in the journal files as well. Now
it makes sense the logs and why the number of journal files are increasing.



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html