Quantcast

The size of KahaDB doesn't seem to decrease

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

The size of KahaDB doesn't seem to decrease

Hidekazu
I use ActiveMQ ver 5.14.3.

I thought the size of kahaDB wouldn't get bigger if ActiveMQ could sent data to subscriber.

However, the size of kahaDB continues increasing even it can send all the data to subscriber.
There is no pending queue in GUI management console.

Is there any necessary parameter to keep making kahaDB small?
I would appreciate it if someone tells me how I should do.


------------------------<kahaDB.log>------------------------
[apl01 kahadb]$ ll
合計 3746764
-rw-r--r-- 1 root root  33554432  2月 28 16:09 2017 db-105.log
-rw-r--r-- 1 root root  33554432  2月 19 01:29 2017 db-11.log
-rw-r--r-- 1 root root  33554432  3月  1 00:31 2017 db-110.log
-rw-r--r-- 1 root root  33554432  2月 19 02:43 2017 db-12.log
-rw-r--r-- 1 root root  33554432  3月  1 13:06 2017 db-123.log
-rw-r--r-- 1 root root  33554432  3月  1 17:11 2017 db-124.log
-rw-r--r-- 1 root root  33554432  2月 21 00:09 2017 db-16.log
-rw-r--r-- 1 root root  33554432  3月  6 23:42 2017 db-177.log
-rw-r--r-- 1 root root  33554432  3月  9 19:10 2017 db-235.log
-rw-r--r-- 1 root root  33554430  3月 10 01:00 2017 db-247.log
-rw-r--r-- 1 root root  33554432  3月 10 02:01 2017 db-268.log
-rw-r--r-- 1 root root  33554432  3月 10 03:03 2017 db-269.log
-rw-r--r-- 1 root root  33554432  3月 10 03:08 2017 db-270.log
-rw-r--r-- 1 root root  33554432  3月 10 04:12 2017 db-292.log
-rw-r--r-- 1 root root  33554432  3月 10 04:22 2017 db-294.log
-rw-r--r-- 1 root root  33554432  3月 10 04:32 2017 db-296.log
-rw-r--r-- 1 root root  33554432  3月 10 04:42 2017 db-298.log
-rw-r--r-- 1 root root  33554432  3月 10 09:08 2017 db-338.log
-rw-r--r-- 1 root root  11158203  3月 10 16:02 2017 db-366.log
-rw-r--r-- 1 root root  11244566  3月 10 16:08 2017 db-367.log
-rw-r--r-- 1 root root  11764385  3月 10 16:13 2017 db-368.log
-rw-r--r-- 1 root root  21460669  3月 10 16:19 2017 db-369.log
-rw-r--r-- 1 root root  23776551  3月 10 16:25 2017 db-370.log
-rw-r--r-- 1 root root  23069634  3月 10 16:30 2017 db-371.log
-rw-r--r-- 1 root root  23219525  3月 10 16:36 2017 db-372.log
-rw-r--r-- 1 root root  33554371  3月 10 16:53 2017 db-373.log
-rw-r--r-- 1 root root  22192884  3月 10 16:48 2017 db-374.log
-rw-r--r-- 1 root root  33554462  3月 10 17:41 2017 db-375.log
-rw-r--r-- 1 root root  18633461  3月 10 16:54 2017 db-376.log
-rw-r--r-- 1 root root  17423968  3月 10 16:59 2017 db-377.log
-rw-r--r-- 1 root root  17114570  3月 10 17:05 2017 db-378.log
-rw-r--r-- 1 root root  16535790  3月 10 17:10 2017 db-379.log
-rw-r--r-- 1 root root  16265110  3月 10 17:16 2017 db-380.log
-rw-r--r-- 1 root root  16283027  3月 10 17:21 2017 db-381.log
-rw-r--r-- 1 root root  16230848  3月 10 17:27 2017 db-382.log
-rw-r--r-- 1 root root  16099668  3月 10 17:32 2017 db-383.log
-rw-r--r-- 1 root root   4124591  3月 10 17:38 2017 db-384.log
-rw-r--r-- 1 root root  33554432  3月 10 17:46 2017 db-385.log
-rw-r--r-- 1 root root  15641628  3月 10 17:43 2017 db-386.log
-rw-r--r-- 1 root root  10966867  3月 10 17:49 2017 db-388.log
-rw-r--r-- 1 root root  11988658  3月 10 17:55 2017 db-390.log
-rw-r--r-- 1 root root   1713321  3月 10 18:00 2017 db-392.log
-rw-r--r-- 1 root root  11952512  3月 10 18:06 2017 db-394.log
-rw-r--r-- 1 root root  13227031  3月 10 18:11 2017 db-396.log
-rw-r--r-- 1 root root    698330  3月 10 18:17 2017 db-398.log
-rw-r--r-- 1 root root  33554432  3月 10 18:27 2017 db-399.log
-rw-r--r-- 1 root root  20457202  3月 10 18:22 2017 db-400.log
-rw-r--r-- 1 root root  33554432  3月 10 18:32 2017 db-401.log
-rw-r--r-- 1 root root  23959259  3月 10 18:28 2017 db-402.log
-rw-r--r-- 1 root root  33554432  3月 10 18:38 2017 db-403.log
-rw-r--r-- 1 root root   5412549  3月 10 18:33 2017 db-404.log
-rw-r--r-- 1 root root  33554448  3月 10 18:44 2017 db-405.log
-rw-r--r-- 1 root root      2580  3月 10 18:39 2017 db-406.log
-rw-r--r-- 1 root root  33554441  3月 10 18:50 2017 db-407.log
-rw-r--r-- 1 root root  11947455  3月 10 18:49 2017 db-408.log
-rw-r--r-- 1 root root  33554432  3月 10 18:55 2017 db-409.log
-rw-r--r-- 1 root root  11983421  3月 10 18:55 2017 db-410.log
-rw-r--r-- 1 root root  33554432  3月 10 19:01 2017 db-411.log
-rw-r--r-- 1 root root   9228757  3月 10 19:01 2017 db-412.log
-rw-r--r-- 1 root root  33554432  3月 10 19:07 2017 db-413.log
-rw-r--r-- 1 root root  13031238  3月 10 19:06 2017 db-414.log
-rw-r--r-- 1 root root  33554432  3月 10 19:13 2017 db-415.log
-rw-r--r-- 1 root root  13249263  3月 10 19:12 2017 db-416.log
-rw-r--r-- 1 root root  33554432  3月 10 19:19 2017 db-417.log
-rw-r--r-- 1 root root   8052175  3月 10 19:17 2017 db-418.log
-rw-r--r-- 1 root root  33554432  3月 10 19:24 2017 db-419.log
-rw-r--r-- 1 root root  11440497  3月 10 19:23 2017 db-420.log
-rw-r--r-- 1 root root  33554432  3月 10 19:30 2017 db-421.log
-rw-r--r-- 1 root root  11458860  3月 10 19:28 2017 db-422.log
-rw-r--r-- 1 root root  33554432  3月 10 19:36 2017 db-423.log
-rw-r--r-- 1 root root  11453650  3月 10 19:34 2017 db-424.log
-rw-r--r-- 1 root root  33554432  3月 10 19:42 2017 db-425.log
-rw-r--r-- 1 root root  11558436  3月 10 19:39 2017 db-426.log
-rw-r--r-- 1 root root  33554432  3月 10 19:48 2017 db-427.log
-rw-r--r-- 1 root root  11512783  3月 10 19:45 2017 db-428.log
-rw-r--r-- 1 root root  33554432  3月 10 19:53 2017 db-429.log
-rw-r--r-- 1 root root  11427258  3月 10 19:50 2017 db-430.log
-rw-r--r-- 1 root root  33554432  3月 10 19:59 2017 db-431.log
-rw-r--r-- 1 root root  11426117  3月 10 19:56 2017 db-432.log
-rw-r--r-- 1 root root  33554451  3月 10 20:04 2017 db-433.log
-rw-r--r-- 1 root root  11752543  3月 10 20:02 2017 db-434.log
-rw-r--r-- 1 root root  33554432  3月 10 20:13 2017 db-435.log
-rw-r--r-- 1 root root  17178983  3月 10 20:10 2017 db-436.log
-rw-r--r-- 1 root root  33554432  3月 10 20:23 2017 db-437.log
-rw-r--r-- 1 root root  13064742  3月 10 20:19 2017 db-438.log
-rw-r--r-- 1 root root  33554432  3月 10 20:34 2017 db-439.log
-rw-r--r-- 1 root root  11496506  3月 10 20:28 2017 db-440.log
-rw-r--r-- 1 root root  33554432  3月 10 20:44 2017 db-441.log
-rw-r--r-- 1 root root  11613996  3月 10 20:37 2017 db-442.log
-rw-r--r-- 1 root root  33554432  3月 10 20:54 2017 db-443.log
-rw-r--r-- 1 root root  11572453  3月 10 20:48 2017 db-444.log
-rw-r--r-- 1 root root  33554432  3月 10 21:04 2017 db-445.log
-rw-r--r-- 1 root root  11657084  3月 10 20:56 2017 db-446.log
-rw-r--r-- 1 root root  11736317  3月 10 21:03 2017 db-447.log
-rw-r--r-- 1 root root  33554432  3月 10 21:14 2017 db-448.log
-rw-r--r-- 1 root root  11655340  3月 10 21:11 2017 db-449.log
-rw-r--r-- 1 root root  33554432  3月 10 21:24 2017 db-450.log
-rw-r--r-- 1 root root  11599701  3月 10 21:17 2017 db-451.log
-rw-r--r-- 1 root root  11569231  3月 10 21:24 2017 db-452.log
-rw-r--r-- 1 root root  33554432  3月 10 21:34 2017 db-453.log
-rw-r--r-- 1 root root  11557135  3月 10 21:31 2017 db-454.log
-rw-r--r-- 1 root root  33554432  3月 10 21:44 2017 db-455.log
-rw-r--r-- 1 root root  11484629  3月 10 21:38 2017 db-456.log
-rw-r--r-- 1 root root  11465856  3月 10 21:44 2017 db-457.log
-rw-r--r-- 1 root root  33554432  3月 10 21:53 2017 db-458.log
-rw-r--r-- 1 root root  11509188  3月 10 21:52 2017 db-459.log
-rw-r--r-- 1 root root  33554432  3月 10 22:03 2017 db-460.log
-rw-r--r-- 1 root root  11535369  3月 10 21:58 2017 db-461.log
-rw-r--r-- 1 root root  33554432  3月 10 22:13 2017 db-462.log
-rw-r--r-- 1 root root  11493312  3月 10 22:05 2017 db-463.log
-rw-r--r-- 1 root root  11418328  3月 10 22:12 2017 db-464.log
-rw-r--r-- 1 root root  33554432  3月 10 22:24 2017 db-465.log
-rw-r--r-- 1 root root  11354254  3月 10 22:19 2017 db-466.log
-rw-r--r-- 1 root root  33554432  3月 10 22:37 2017 db-467.log
-rw-r--r-- 1 root root  11424887  3月 10 22:25 2017 db-468.log
-rw-r--r-- 1 root root  11461242  3月 10 22:32 2017 db-469.log
-rw-r--r-- 1 root root  11466414  3月 10 22:37 2017 db-470.log
-rw-r--r-- 1 root root  33554432  3月 10 23:34 2017 db-471.log
-rw-r--r-- 1 root root  11445462  3月 10 22:42 2017 db-472.log
-rw-r--r-- 1 root root  11366383  3月 10 22:47 2017 db-473.log
-rw-r--r-- 1 root root  11523021  3月 10 22:52 2017 db-474.log
-rw-r--r-- 1 root root  11050675  3月 10 22:57 2017 db-475.log
-rw-r--r-- 1 root root  11044206  3月 10 23:02 2017 db-476.log
-rw-r--r-- 1 root root  12162160  3月 10 23:07 2017 db-477.log
-rw-r--r-- 1 root root  21747941  3月 10 23:12 2017 db-478.log
-rw-r--r-- 1 root root  23387783  3月 10 23:18 2017 db-479.log
-rw-r--r-- 1 root root  23885560  3月 10 23:23 2017 db-480.log
-rw-r--r-- 1 root root  24294349  3月 10 23:28 2017 db-481.log
-rw-r--r-- 1 root root  20831859  3月 10 23:33 2017 db-482.log
-rw-r--r-- 1 root root  33554432  3月 11 00:03 2017 db-483.log
-rw-r--r-- 1 root root  17501114  3月 10 23:38 2017 db-484.log
-rw-r--r-- 1 root root  16312158  3月 10 23:43 2017 db-485.log
-rw-r--r-- 1 root root  15078670  3月 10 23:48 2017 db-486.log
-rw-r--r-- 1 root root  14234846  3月 10 23:53 2017 db-487.log
-rw-r--r-- 1 root root  13205763  3月 10 23:58 2017 db-488.log
-rw-r--r-- 1 root root  11803947  3月 11 00:03 2017 db-489.log
-rw-r--r-- 1 root root  33554432  3月 11 00:08 2017 db-490.log
-rw-r--r-- 1 root root  33554432  3月 11 00:12 2017 db-491.log
-rw-r--r-- 1 root root  33554432  3月 11 00:17 2017 db-492.log
-rw-r--r-- 1 root root  10925899  3月 11 00:13 2017 db-493.log
-rw-r--r-- 1 root root  33554432  3月 11 00:22 2017 db-494.log
-rw-r--r-- 1 root root   9561631  3月 11 00:18 2017 db-495.log
-rw-r--r-- 1 root root  33554432  3月 13 16:07 2017 db-496.log
-rw-r--r-- 1 root root   8318697  3月 11 00:25 2017 db-497.log
-rw-r--r-- 1 root root  33554432  2月  3 09:09 2017 db-7.log
-rw-r--r-- 1 root root  33554432  2月  5 06:37 2017 db-8.log
-rw-r--r-- 1 root root  33554435  2月 19 00:05 2017 db-9.log
-rw-r--r-- 1 root root 589115392  3月 11 11:05 2017 db.data
-rw-r--r-- 1 root root  38602216  3月 11 11:05 2017 db.redo
-rw-r--r-- 1 root root         8  3月 10 16:43 2017 lock

------------------------<part of ActiveMQ config>------------------------
    <broker xmlns="http://activemq.apache.org/schema/core" brokerName="localhost" persistent="true" deleteAllMessagesOnStartup="false" dataDirectory="${activemq.data}">
                <plugins>
                 
                        <jaasAuthenticationPlugin configuration="LdapConfiguration" />
                </plugins>

        <destinationPolicy>
            <policyMap>
              <policyEntries>
                <policyEntry topic=">" enableAudit="false">
                   
                  <pendingMessageLimitStrategy>
                    <constantPendingMessageLimitStrategy limit="1000"/>
                  </pendingMessageLimitStrategy>
                </policyEntry>
              </policyEntries>
            </policyMap>
        </destinationPolicy>


       
        <managementContext>
            <managementContext createConnector="false"/>
        </managementContext>

       
        <persistenceAdapter>
            <kahaDB directory="/data/ems/apache-activemq/data/kahadb"/>
        </persistenceAdapter>


         
          <systemUsage>
            <systemUsage>
                <memoryUsage>
                    <memoryUsage percentOfJvmHeap="70" />
                </memoryUsage>
                <storeUsage>
                    <storeUsage limit="20 gb"/>
                </storeUsage>
                <tempUsage>
                    <tempUsage limit="10 gb"/>
                </tempUsage>
            </systemUsage>
        </systemUsage>
------------------------------------------------------------------------
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: The size of KahaDB doesn't seem to decrease

Andreas Gies
Hi there,


we had the same issue when we were using topics and had durable subscribers
which wouldn't connect for up to 4 days.

The kahadb files would only be cleaned up when the last durable
subscriber has
received the message. In our case the topic in in question didn't have a
lot of traffic,
but it still had the effect you are seeing.

We have solved the issue with configuring a mulit kaha db store and
giving the topic
it's own space.

AFAIR Gary Tully has written a pretty good blog article about what's
going on and why
using a multi kahadb works in that case.

http://blog.garytully.com/2011/11/activemq-multiple-kahadb-instances.html


Hope that helps

Andreas


On 13/03/17 11:39, Hidekazu wrote:

> I use ActiveMQ ver 5.14.3.
>
> I thought the size of kahaDB wouldn't get bigger if ActiveMQ could sent data
> to subscriber.
>
> However, the size of kahaDB continues increasing even it can send all the
> data to subscriber.
> There is no pending queue in GUI management console.
>
> Is there any necessary parameter to keep making kahaDB small?
> I would appreciate it if someone tells me how I should do.
>
>
> ------------------------<kahaDB.log>------------------------
> [apl01 kahadb]$ ll
> 合計 3746764
> -rw-r--r-- 1 root root  33554432  2月 28 16:09 2017 db-105.log
> -rw-r--r-- 1 root root  33554432  2月 19 01:29 2017 db-11.log
> -rw-r--r-- 1 root root  33554432  3月  1 00:31 2017 db-110.log
> -rw-r--r-- 1 root root  33554432  2月 19 02:43 2017 db-12.log
> -rw-r--r-- 1 root root  33554432  3月  1 13:06 2017 db-123.log
> -rw-r--r-- 1 root root  33554432  3月  1 17:11 2017 db-124.log
> -rw-r--r-- 1 root root  33554432  2月 21 00:09 2017 db-16.log
> -rw-r--r-- 1 root root  33554432  3月  6 23:42 2017 db-177.log
> -rw-r--r-- 1 root root  33554432  3月  9 19:10 2017 db-235.log
> -rw-r--r-- 1 root root  33554430  3月 10 01:00 2017 db-247.log
> -rw-r--r-- 1 root root  33554432  3月 10 02:01 2017 db-268.log
> -rw-r--r-- 1 root root  33554432  3月 10 03:03 2017 db-269.log
> -rw-r--r-- 1 root root  33554432  3月 10 03:08 2017 db-270.log
> -rw-r--r-- 1 root root  33554432  3月 10 04:12 2017 db-292.log
> -rw-r--r-- 1 root root  33554432  3月 10 04:22 2017 db-294.log
> -rw-r--r-- 1 root root  33554432  3月 10 04:32 2017 db-296.log
> -rw-r--r-- 1 root root  33554432  3月 10 04:42 2017 db-298.log
> -rw-r--r-- 1 root root  33554432  3月 10 09:08 2017 db-338.log
> -rw-r--r-- 1 root root  11158203  3月 10 16:02 2017 db-366.log
> -rw-r--r-- 1 root root  11244566  3月 10 16:08 2017 db-367.log
> -rw-r--r-- 1 root root  11764385  3月 10 16:13 2017 db-368.log
> -rw-r--r-- 1 root root  21460669  3月 10 16:19 2017 db-369.log
> -rw-r--r-- 1 root root  23776551  3月 10 16:25 2017 db-370.log
> -rw-r--r-- 1 root root  23069634  3月 10 16:30 2017 db-371.log
> -rw-r--r-- 1 root root  23219525  3月 10 16:36 2017 db-372.log
> -rw-r--r-- 1 root root  33554371  3月 10 16:53 2017 db-373.log
> -rw-r--r-- 1 root root  22192884  3月 10 16:48 2017 db-374.log
> -rw-r--r-- 1 root root  33554462  3月 10 17:41 2017 db-375.log
> -rw-r--r-- 1 root root  18633461  3月 10 16:54 2017 db-376.log
> -rw-r--r-- 1 root root  17423968  3月 10 16:59 2017 db-377.log
> -rw-r--r-- 1 root root  17114570  3月 10 17:05 2017 db-378.log
> -rw-r--r-- 1 root root  16535790  3月 10 17:10 2017 db-379.log
> -rw-r--r-- 1 root root  16265110  3月 10 17:16 2017 db-380.log
> -rw-r--r-- 1 root root  16283027  3月 10 17:21 2017 db-381.log
> -rw-r--r-- 1 root root  16230848  3月 10 17:27 2017 db-382.log
> -rw-r--r-- 1 root root  16099668  3月 10 17:32 2017 db-383.log
> -rw-r--r-- 1 root root   4124591  3月 10 17:38 2017 db-384.log
> -rw-r--r-- 1 root root  33554432  3月 10 17:46 2017 db-385.log
> -rw-r--r-- 1 root root  15641628  3月 10 17:43 2017 db-386.log
> -rw-r--r-- 1 root root  10966867  3月 10 17:49 2017 db-388.log
> -rw-r--r-- 1 root root  11988658  3月 10 17:55 2017 db-390.log
> -rw-r--r-- 1 root root   1713321  3月 10 18:00 2017 db-392.log
> -rw-r--r-- 1 root root  11952512  3月 10 18:06 2017 db-394.log
> -rw-r--r-- 1 root root  13227031  3月 10 18:11 2017 db-396.log
> -rw-r--r-- 1 root root    698330  3月 10 18:17 2017 db-398.log
> -rw-r--r-- 1 root root  33554432  3月 10 18:27 2017 db-399.log
> -rw-r--r-- 1 root root  20457202  3月 10 18:22 2017 db-400.log
> -rw-r--r-- 1 root root  33554432  3月 10 18:32 2017 db-401.log
> -rw-r--r-- 1 root root  23959259  3月 10 18:28 2017 db-402.log
> -rw-r--r-- 1 root root  33554432  3月 10 18:38 2017 db-403.log
> -rw-r--r-- 1 root root   5412549  3月 10 18:33 2017 db-404.log
> -rw-r--r-- 1 root root  33554448  3月 10 18:44 2017 db-405.log
> -rw-r--r-- 1 root root      2580  3月 10 18:39 2017 db-406.log
> -rw-r--r-- 1 root root  33554441  3月 10 18:50 2017 db-407.log
> -rw-r--r-- 1 root root  11947455  3月 10 18:49 2017 db-408.log
> -rw-r--r-- 1 root root  33554432  3月 10 18:55 2017 db-409.log
> -rw-r--r-- 1 root root  11983421  3月 10 18:55 2017 db-410.log
> -rw-r--r-- 1 root root  33554432  3月 10 19:01 2017 db-411.log
> -rw-r--r-- 1 root root   9228757  3月 10 19:01 2017 db-412.log
> -rw-r--r-- 1 root root  33554432  3月 10 19:07 2017 db-413.log
> -rw-r--r-- 1 root root  13031238  3月 10 19:06 2017 db-414.log
> -rw-r--r-- 1 root root  33554432  3月 10 19:13 2017 db-415.log
> -rw-r--r-- 1 root root  13249263  3月 10 19:12 2017 db-416.log
> -rw-r--r-- 1 root root  33554432  3月 10 19:19 2017 db-417.log
> -rw-r--r-- 1 root root   8052175  3月 10 19:17 2017 db-418.log
> -rw-r--r-- 1 root root  33554432  3月 10 19:24 2017 db-419.log
> -rw-r--r-- 1 root root  11440497  3月 10 19:23 2017 db-420.log
> -rw-r--r-- 1 root root  33554432  3月 10 19:30 2017 db-421.log
> -rw-r--r-- 1 root root  11458860  3月 10 19:28 2017 db-422.log
> -rw-r--r-- 1 root root  33554432  3月 10 19:36 2017 db-423.log
> -rw-r--r-- 1 root root  11453650  3月 10 19:34 2017 db-424.log
> -rw-r--r-- 1 root root  33554432  3月 10 19:42 2017 db-425.log
> -rw-r--r-- 1 root root  11558436  3月 10 19:39 2017 db-426.log
> -rw-r--r-- 1 root root  33554432  3月 10 19:48 2017 db-427.log
> -rw-r--r-- 1 root root  11512783  3月 10 19:45 2017 db-428.log
> -rw-r--r-- 1 root root  33554432  3月 10 19:53 2017 db-429.log
> -rw-r--r-- 1 root root  11427258  3月 10 19:50 2017 db-430.log
> -rw-r--r-- 1 root root  33554432  3月 10 19:59 2017 db-431.log
> -rw-r--r-- 1 root root  11426117  3月 10 19:56 2017 db-432.log
> -rw-r--r-- 1 root root  33554451  3月 10 20:04 2017 db-433.log
> -rw-r--r-- 1 root root  11752543  3月 10 20:02 2017 db-434.log
> -rw-r--r-- 1 root root  33554432  3月 10 20:13 2017 db-435.log
> -rw-r--r-- 1 root root  17178983  3月 10 20:10 2017 db-436.log
> -rw-r--r-- 1 root root  33554432  3月 10 20:23 2017 db-437.log
> -rw-r--r-- 1 root root  13064742  3月 10 20:19 2017 db-438.log
> -rw-r--r-- 1 root root  33554432  3月 10 20:34 2017 db-439.log
> -rw-r--r-- 1 root root  11496506  3月 10 20:28 2017 db-440.log
> -rw-r--r-- 1 root root  33554432  3月 10 20:44 2017 db-441.log
> -rw-r--r-- 1 root root  11613996  3月 10 20:37 2017 db-442.log
> -rw-r--r-- 1 root root  33554432  3月 10 20:54 2017 db-443.log
> -rw-r--r-- 1 root root  11572453  3月 10 20:48 2017 db-444.log
> -rw-r--r-- 1 root root  33554432  3月 10 21:04 2017 db-445.log
> -rw-r--r-- 1 root root  11657084  3月 10 20:56 2017 db-446.log
> -rw-r--r-- 1 root root  11736317  3月 10 21:03 2017 db-447.log
> -rw-r--r-- 1 root root  33554432  3月 10 21:14 2017 db-448.log
> -rw-r--r-- 1 root root  11655340  3月 10 21:11 2017 db-449.log
> -rw-r--r-- 1 root root  33554432  3月 10 21:24 2017 db-450.log
> -rw-r--r-- 1 root root  11599701  3月 10 21:17 2017 db-451.log
> -rw-r--r-- 1 root root  11569231  3月 10 21:24 2017 db-452.log
> -rw-r--r-- 1 root root  33554432  3月 10 21:34 2017 db-453.log
> -rw-r--r-- 1 root root  11557135  3月 10 21:31 2017 db-454.log
> -rw-r--r-- 1 root root  33554432  3月 10 21:44 2017 db-455.log
> -rw-r--r-- 1 root root  11484629  3月 10 21:38 2017 db-456.log
> -rw-r--r-- 1 root root  11465856  3月 10 21:44 2017 db-457.log
> -rw-r--r-- 1 root root  33554432  3月 10 21:53 2017 db-458.log
> -rw-r--r-- 1 root root  11509188  3月 10 21:52 2017 db-459.log
> -rw-r--r-- 1 root root  33554432  3月 10 22:03 2017 db-460.log
> -rw-r--r-- 1 root root  11535369  3月 10 21:58 2017 db-461.log
> -rw-r--r-- 1 root root  33554432  3月 10 22:13 2017 db-462.log
> -rw-r--r-- 1 root root  11493312  3月 10 22:05 2017 db-463.log
> -rw-r--r-- 1 root root  11418328  3月 10 22:12 2017 db-464.log
> -rw-r--r-- 1 root root  33554432  3月 10 22:24 2017 db-465.log
> -rw-r--r-- 1 root root  11354254  3月 10 22:19 2017 db-466.log
> -rw-r--r-- 1 root root  33554432  3月 10 22:37 2017 db-467.log
> -rw-r--r-- 1 root root  11424887  3月 10 22:25 2017 db-468.log
> -rw-r--r-- 1 root root  11461242  3月 10 22:32 2017 db-469.log
> -rw-r--r-- 1 root root  11466414  3月 10 22:37 2017 db-470.log
> -rw-r--r-- 1 root root  33554432  3月 10 23:34 2017 db-471.log
> -rw-r--r-- 1 root root  11445462  3月 10 22:42 2017 db-472.log
> -rw-r--r-- 1 root root  11366383  3月 10 22:47 2017 db-473.log
> -rw-r--r-- 1 root root  11523021  3月 10 22:52 2017 db-474.log
> -rw-r--r-- 1 root root  11050675  3月 10 22:57 2017 db-475.log
> -rw-r--r-- 1 root root  11044206  3月 10 23:02 2017 db-476.log
> -rw-r--r-- 1 root root  12162160  3月 10 23:07 2017 db-477.log
> -rw-r--r-- 1 root root  21747941  3月 10 23:12 2017 db-478.log
> -rw-r--r-- 1 root root  23387783  3月 10 23:18 2017 db-479.log
> -rw-r--r-- 1 root root  23885560  3月 10 23:23 2017 db-480.log
> -rw-r--r-- 1 root root  24294349  3月 10 23:28 2017 db-481.log
> -rw-r--r-- 1 root root  20831859  3月 10 23:33 2017 db-482.log
> -rw-r--r-- 1 root root  33554432  3月 11 00:03 2017 db-483.log
> -rw-r--r-- 1 root root  17501114  3月 10 23:38 2017 db-484.log
> -rw-r--r-- 1 root root  16312158  3月 10 23:43 2017 db-485.log
> -rw-r--r-- 1 root root  15078670  3月 10 23:48 2017 db-486.log
> -rw-r--r-- 1 root root  14234846  3月 10 23:53 2017 db-487.log
> -rw-r--r-- 1 root root  13205763  3月 10 23:58 2017 db-488.log
> -rw-r--r-- 1 root root  11803947  3月 11 00:03 2017 db-489.log
> -rw-r--r-- 1 root root  33554432  3月 11 00:08 2017 db-490.log
> -rw-r--r-- 1 root root  33554432  3月 11 00:12 2017 db-491.log
> -rw-r--r-- 1 root root  33554432  3月 11 00:17 2017 db-492.log
> -rw-r--r-- 1 root root  10925899  3月 11 00:13 2017 db-493.log
> -rw-r--r-- 1 root root  33554432  3月 11 00:22 2017 db-494.log
> -rw-r--r-- 1 root root   9561631  3月 11 00:18 2017 db-495.log
> -rw-r--r-- 1 root root  33554432  3月 13 16:07 2017 db-496.log
> -rw-r--r-- 1 root root   8318697  3月 11 00:25 2017 db-497.log
> -rw-r--r-- 1 root root  33554432  2月  3 09:09 2017 db-7.log
> -rw-r--r-- 1 root root  33554432  2月  5 06:37 2017 db-8.log
> -rw-r--r-- 1 root root  33554435  2月 19 00:05 2017 db-9.log
> -rw-r--r-- 1 root root 589115392  3月 11 11:05 2017 db.data
> -rw-r--r-- 1 root root  38602216  3月 11 11:05 2017 db.redo
> -rw-r--r-- 1 root root         8  3月 10 16:43 2017 lock
>
> ------------------------<part of ActiveMQ config>------------------------
>      <broker xmlns="http://activemq.apache.org/schema/core"
> brokerName="localhost" persistent="true" deleteAllMessagesOnStartup="false"
> dataDirectory="${activemq.data}">
>                  <plugins>
>                    
>                          <jaasAuthenticationPlugin
> configuration="LdapConfiguration" />
>                  </plugins>
>
>          <destinationPolicy>
>              <policyMap>
>                <policyEntries>
>                  <policyEntry topic=">" enableAudit="false">
>                      
>                    <pendingMessageLimitStrategy>
>                      <constantPendingMessageLimitStrategy limit="1000"/>
>                    </pendingMessageLimitStrategy>
>                  </policyEntry>
>                </policyEntries>
>              </policyMap>
>          </destinationPolicy>
>
>
>          
>          <managementContext>
>              <managementContext createConnector="false"/>
>          </managementContext>
>
>          
>          <persistenceAdapter>
>              <kahaDB directory="/data/ems/apache-activemq/data/kahadb"/>
>          </persistenceAdapter>
>
>
>            
>            <systemUsage>
>              <systemUsage>
>                  <memoryUsage>
>                      <memoryUsage percentOfJvmHeap="70" />
>                  </memoryUsage>
>                  <storeUsage>
>                      <storeUsage limit="20 gb"/>
>                  </storeUsage>
>                  <tempUsage>
>                      <tempUsage limit="10 gb"/>
>                  </tempUsage>
>              </systemUsage>
>          </systemUsage>
> ------------------------------------------------------------------------
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/The-size-of-KahaDB-doesn-t-seem-to-decrease-tp4723669.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: The size of KahaDB doesn't seem to decrease

tabish121@gmail.com
In reply to this post by Hidekazu

Best place to start is in the docs, see why the files are being retained.
http://activemq.apache.org/why-do-kahadb-log-files-remain-after-cleanup.html


On 03/13/2017 06:39 AM, Hidekazu wrote:

> I use ActiveMQ ver 5.14.3.
>
> I thought the size of kahaDB wouldn't get bigger if ActiveMQ could sent data
> to subscriber.
>
> However, the size of kahaDB continues increasing even it can send all the
> data to subscriber.
> There is no pending queue in GUI management console.
>
> Is there any necessary parameter to keep making kahaDB small?
> I would appreciate it if someone tells me how I should do.
>
>
> ------------------------<kahaDB.log>------------------------
> [apl01 kahadb]$ ll
> 合計 3746764
> -rw-r--r-- 1 root root  33554432  2月 28 16:09 2017 db-105.log
> -rw-r--r-- 1 root root  33554432  2月 19 01:29 2017 db-11.log
> -rw-r--r-- 1 root root  33554432  3月  1 00:31 2017 db-110.log
> -rw-r--r-- 1 root root  33554432  2月 19 02:43 2017 db-12.log
> -rw-r--r-- 1 root root  33554432  3月  1 13:06 2017 db-123.log
> -rw-r--r-- 1 root root  33554432  3月  1 17:11 2017 db-124.log
> -rw-r--r-- 1 root root  33554432  2月 21 00:09 2017 db-16.log
> -rw-r--r-- 1 root root  33554432  3月  6 23:42 2017 db-177.log
> -rw-r--r-- 1 root root  33554432  3月  9 19:10 2017 db-235.log
> -rw-r--r-- 1 root root  33554430  3月 10 01:00 2017 db-247.log
> -rw-r--r-- 1 root root  33554432  3月 10 02:01 2017 db-268.log
> -rw-r--r-- 1 root root  33554432  3月 10 03:03 2017 db-269.log
> -rw-r--r-- 1 root root  33554432  3月 10 03:08 2017 db-270.log
> -rw-r--r-- 1 root root  33554432  3月 10 04:12 2017 db-292.log
> -rw-r--r-- 1 root root  33554432  3月 10 04:22 2017 db-294.log
> -rw-r--r-- 1 root root  33554432  3月 10 04:32 2017 db-296.log
> -rw-r--r-- 1 root root  33554432  3月 10 04:42 2017 db-298.log
> -rw-r--r-- 1 root root  33554432  3月 10 09:08 2017 db-338.log
> -rw-r--r-- 1 root root  11158203  3月 10 16:02 2017 db-366.log
> -rw-r--r-- 1 root root  11244566  3月 10 16:08 2017 db-367.log
> -rw-r--r-- 1 root root  11764385  3月 10 16:13 2017 db-368.log
> -rw-r--r-- 1 root root  21460669  3月 10 16:19 2017 db-369.log
> -rw-r--r-- 1 root root  23776551  3月 10 16:25 2017 db-370.log
> -rw-r--r-- 1 root root  23069634  3月 10 16:30 2017 db-371.log
> -rw-r--r-- 1 root root  23219525  3月 10 16:36 2017 db-372.log
> -rw-r--r-- 1 root root  33554371  3月 10 16:53 2017 db-373.log
> -rw-r--r-- 1 root root  22192884  3月 10 16:48 2017 db-374.log
> -rw-r--r-- 1 root root  33554462  3月 10 17:41 2017 db-375.log
> -rw-r--r-- 1 root root  18633461  3月 10 16:54 2017 db-376.log
> -rw-r--r-- 1 root root  17423968  3月 10 16:59 2017 db-377.log
> -rw-r--r-- 1 root root  17114570  3月 10 17:05 2017 db-378.log
> -rw-r--r-- 1 root root  16535790  3月 10 17:10 2017 db-379.log
> -rw-r--r-- 1 root root  16265110  3月 10 17:16 2017 db-380.log
> -rw-r--r-- 1 root root  16283027  3月 10 17:21 2017 db-381.log
> -rw-r--r-- 1 root root  16230848  3月 10 17:27 2017 db-382.log
> -rw-r--r-- 1 root root  16099668  3月 10 17:32 2017 db-383.log
> -rw-r--r-- 1 root root   4124591  3月 10 17:38 2017 db-384.log
> -rw-r--r-- 1 root root  33554432  3月 10 17:46 2017 db-385.log
> -rw-r--r-- 1 root root  15641628  3月 10 17:43 2017 db-386.log
> -rw-r--r-- 1 root root  10966867  3月 10 17:49 2017 db-388.log
> -rw-r--r-- 1 root root  11988658  3月 10 17:55 2017 db-390.log
> -rw-r--r-- 1 root root   1713321  3月 10 18:00 2017 db-392.log
> -rw-r--r-- 1 root root  11952512  3月 10 18:06 2017 db-394.log
> -rw-r--r-- 1 root root  13227031  3月 10 18:11 2017 db-396.log
> -rw-r--r-- 1 root root    698330  3月 10 18:17 2017 db-398.log
> -rw-r--r-- 1 root root  33554432  3月 10 18:27 2017 db-399.log
> -rw-r--r-- 1 root root  20457202  3月 10 18:22 2017 db-400.log
> -rw-r--r-- 1 root root  33554432  3月 10 18:32 2017 db-401.log
> -rw-r--r-- 1 root root  23959259  3月 10 18:28 2017 db-402.log
> -rw-r--r-- 1 root root  33554432  3月 10 18:38 2017 db-403.log
> -rw-r--r-- 1 root root   5412549  3月 10 18:33 2017 db-404.log
> -rw-r--r-- 1 root root  33554448  3月 10 18:44 2017 db-405.log
> -rw-r--r-- 1 root root      2580  3月 10 18:39 2017 db-406.log
> -rw-r--r-- 1 root root  33554441  3月 10 18:50 2017 db-407.log
> -rw-r--r-- 1 root root  11947455  3月 10 18:49 2017 db-408.log
> -rw-r--r-- 1 root root  33554432  3月 10 18:55 2017 db-409.log
> -rw-r--r-- 1 root root  11983421  3月 10 18:55 2017 db-410.log
> -rw-r--r-- 1 root root  33554432  3月 10 19:01 2017 db-411.log
> -rw-r--r-- 1 root root   9228757  3月 10 19:01 2017 db-412.log
> -rw-r--r-- 1 root root  33554432  3月 10 19:07 2017 db-413.log
> -rw-r--r-- 1 root root  13031238  3月 10 19:06 2017 db-414.log
> -rw-r--r-- 1 root root  33554432  3月 10 19:13 2017 db-415.log
> -rw-r--r-- 1 root root  13249263  3月 10 19:12 2017 db-416.log
> -rw-r--r-- 1 root root  33554432  3月 10 19:19 2017 db-417.log
> -rw-r--r-- 1 root root   8052175  3月 10 19:17 2017 db-418.log
> -rw-r--r-- 1 root root  33554432  3月 10 19:24 2017 db-419.log
> -rw-r--r-- 1 root root  11440497  3月 10 19:23 2017 db-420.log
> -rw-r--r-- 1 root root  33554432  3月 10 19:30 2017 db-421.log
> -rw-r--r-- 1 root root  11458860  3月 10 19:28 2017 db-422.log
> -rw-r--r-- 1 root root  33554432  3月 10 19:36 2017 db-423.log
> -rw-r--r-- 1 root root  11453650  3月 10 19:34 2017 db-424.log
> -rw-r--r-- 1 root root  33554432  3月 10 19:42 2017 db-425.log
> -rw-r--r-- 1 root root  11558436  3月 10 19:39 2017 db-426.log
> -rw-r--r-- 1 root root  33554432  3月 10 19:48 2017 db-427.log
> -rw-r--r-- 1 root root  11512783  3月 10 19:45 2017 db-428.log
> -rw-r--r-- 1 root root  33554432  3月 10 19:53 2017 db-429.log
> -rw-r--r-- 1 root root  11427258  3月 10 19:50 2017 db-430.log
> -rw-r--r-- 1 root root  33554432  3月 10 19:59 2017 db-431.log
> -rw-r--r-- 1 root root  11426117  3月 10 19:56 2017 db-432.log
> -rw-r--r-- 1 root root  33554451  3月 10 20:04 2017 db-433.log
> -rw-r--r-- 1 root root  11752543  3月 10 20:02 2017 db-434.log
> -rw-r--r-- 1 root root  33554432  3月 10 20:13 2017 db-435.log
> -rw-r--r-- 1 root root  17178983  3月 10 20:10 2017 db-436.log
> -rw-r--r-- 1 root root  33554432  3月 10 20:23 2017 db-437.log
> -rw-r--r-- 1 root root  13064742  3月 10 20:19 2017 db-438.log
> -rw-r--r-- 1 root root  33554432  3月 10 20:34 2017 db-439.log
> -rw-r--r-- 1 root root  11496506  3月 10 20:28 2017 db-440.log
> -rw-r--r-- 1 root root  33554432  3月 10 20:44 2017 db-441.log
> -rw-r--r-- 1 root root  11613996  3月 10 20:37 2017 db-442.log
> -rw-r--r-- 1 root root  33554432  3月 10 20:54 2017 db-443.log
> -rw-r--r-- 1 root root  11572453  3月 10 20:48 2017 db-444.log
> -rw-r--r-- 1 root root  33554432  3月 10 21:04 2017 db-445.log
> -rw-r--r-- 1 root root  11657084  3月 10 20:56 2017 db-446.log
> -rw-r--r-- 1 root root  11736317  3月 10 21:03 2017 db-447.log
> -rw-r--r-- 1 root root  33554432  3月 10 21:14 2017 db-448.log
> -rw-r--r-- 1 root root  11655340  3月 10 21:11 2017 db-449.log
> -rw-r--r-- 1 root root  33554432  3月 10 21:24 2017 db-450.log
> -rw-r--r-- 1 root root  11599701  3月 10 21:17 2017 db-451.log
> -rw-r--r-- 1 root root  11569231  3月 10 21:24 2017 db-452.log
> -rw-r--r-- 1 root root  33554432  3月 10 21:34 2017 db-453.log
> -rw-r--r-- 1 root root  11557135  3月 10 21:31 2017 db-454.log
> -rw-r--r-- 1 root root  33554432  3月 10 21:44 2017 db-455.log
> -rw-r--r-- 1 root root  11484629  3月 10 21:38 2017 db-456.log
> -rw-r--r-- 1 root root  11465856  3月 10 21:44 2017 db-457.log
> -rw-r--r-- 1 root root  33554432  3月 10 21:53 2017 db-458.log
> -rw-r--r-- 1 root root  11509188  3月 10 21:52 2017 db-459.log
> -rw-r--r-- 1 root root  33554432  3月 10 22:03 2017 db-460.log
> -rw-r--r-- 1 root root  11535369  3月 10 21:58 2017 db-461.log
> -rw-r--r-- 1 root root  33554432  3月 10 22:13 2017 db-462.log
> -rw-r--r-- 1 root root  11493312  3月 10 22:05 2017 db-463.log
> -rw-r--r-- 1 root root  11418328  3月 10 22:12 2017 db-464.log
> -rw-r--r-- 1 root root  33554432  3月 10 22:24 2017 db-465.log
> -rw-r--r-- 1 root root  11354254  3月 10 22:19 2017 db-466.log
> -rw-r--r-- 1 root root  33554432  3月 10 22:37 2017 db-467.log
> -rw-r--r-- 1 root root  11424887  3月 10 22:25 2017 db-468.log
> -rw-r--r-- 1 root root  11461242  3月 10 22:32 2017 db-469.log
> -rw-r--r-- 1 root root  11466414  3月 10 22:37 2017 db-470.log
> -rw-r--r-- 1 root root  33554432  3月 10 23:34 2017 db-471.log
> -rw-r--r-- 1 root root  11445462  3月 10 22:42 2017 db-472.log
> -rw-r--r-- 1 root root  11366383  3月 10 22:47 2017 db-473.log
> -rw-r--r-- 1 root root  11523021  3月 10 22:52 2017 db-474.log
> -rw-r--r-- 1 root root  11050675  3月 10 22:57 2017 db-475.log
> -rw-r--r-- 1 root root  11044206  3月 10 23:02 2017 db-476.log
> -rw-r--r-- 1 root root  12162160  3月 10 23:07 2017 db-477.log
> -rw-r--r-- 1 root root  21747941  3月 10 23:12 2017 db-478.log
> -rw-r--r-- 1 root root  23387783  3月 10 23:18 2017 db-479.log
> -rw-r--r-- 1 root root  23885560  3月 10 23:23 2017 db-480.log
> -rw-r--r-- 1 root root  24294349  3月 10 23:28 2017 db-481.log
> -rw-r--r-- 1 root root  20831859  3月 10 23:33 2017 db-482.log
> -rw-r--r-- 1 root root  33554432  3月 11 00:03 2017 db-483.log
> -rw-r--r-- 1 root root  17501114  3月 10 23:38 2017 db-484.log
> -rw-r--r-- 1 root root  16312158  3月 10 23:43 2017 db-485.log
> -rw-r--r-- 1 root root  15078670  3月 10 23:48 2017 db-486.log
> -rw-r--r-- 1 root root  14234846  3月 10 23:53 2017 db-487.log
> -rw-r--r-- 1 root root  13205763  3月 10 23:58 2017 db-488.log
> -rw-r--r-- 1 root root  11803947  3月 11 00:03 2017 db-489.log
> -rw-r--r-- 1 root root  33554432  3月 11 00:08 2017 db-490.log
> -rw-r--r-- 1 root root  33554432  3月 11 00:12 2017 db-491.log
> -rw-r--r-- 1 root root  33554432  3月 11 00:17 2017 db-492.log
> -rw-r--r-- 1 root root  10925899  3月 11 00:13 2017 db-493.log
> -rw-r--r-- 1 root root  33554432  3月 11 00:22 2017 db-494.log
> -rw-r--r-- 1 root root   9561631  3月 11 00:18 2017 db-495.log
> -rw-r--r-- 1 root root  33554432  3月 13 16:07 2017 db-496.log
> -rw-r--r-- 1 root root   8318697  3月 11 00:25 2017 db-497.log
> -rw-r--r-- 1 root root  33554432  2月  3 09:09 2017 db-7.log
> -rw-r--r-- 1 root root  33554432  2月  5 06:37 2017 db-8.log
> -rw-r--r-- 1 root root  33554435  2月 19 00:05 2017 db-9.log
> -rw-r--r-- 1 root root 589115392  3月 11 11:05 2017 db.data
> -rw-r--r-- 1 root root  38602216  3月 11 11:05 2017 db.redo
> -rw-r--r-- 1 root root         8  3月 10 16:43 2017 lock
>
> ------------------------<part of ActiveMQ config>------------------------
>      <broker xmlns="http://activemq.apache.org/schema/core"
> brokerName="localhost" persistent="true" deleteAllMessagesOnStartup="false"
> dataDirectory="${activemq.data}">
>                  <plugins>
>                    
>                          <jaasAuthenticationPlugin
> configuration="LdapConfiguration" />
>                  </plugins>
>
>          <destinationPolicy>
>              <policyMap>
>                <policyEntries>
>                  <policyEntry topic=">" enableAudit="false">
>                      
>                    <pendingMessageLimitStrategy>
>                      <constantPendingMessageLimitStrategy limit="1000"/>
>                    </pendingMessageLimitStrategy>
>                  </policyEntry>
>                </policyEntries>
>              </policyMap>
>          </destinationPolicy>
>
>
>          
>          <managementContext>
>              <managementContext createConnector="false"/>
>          </managementContext>
>
>          
>          <persistenceAdapter>
>              <kahaDB directory="/data/ems/apache-activemq/data/kahadb"/>
>          </persistenceAdapter>
>
>
>            
>            <systemUsage>
>              <systemUsage>
>                  <memoryUsage>
>                      <memoryUsage percentOfJvmHeap="70" />
>                  </memoryUsage>
>                  <storeUsage>
>                      <storeUsage limit="20 gb"/>
>                  </storeUsage>
>                  <tempUsage>
>                      <tempUsage limit="10 gb"/>
>                  </tempUsage>
>              </systemUsage>
>          </systemUsage>
> ------------------------------------------------------------------------
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/The-size-of-KahaDB-doesn-t-seem-to-decrease-tp4723669.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>


--
Tim Bish
twitter: @tabish121
blog: http://timbish.blogspot.com/

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: The size of KahaDB doesn't seem to decrease

Hidekazu
In reply to this post by Andreas Gies
Thanks Andreas

The article you introduced me was very helpful to understand the specification of kahaDB.

I have to say there was a miss-communication among our developer team.
I heard that there was no pending queue in ActiveMQ, however,
There were a lot of pending queue in ActiveMQ and that was the reason kahaDB journals were never deleted.
I'm sorry.. (but thanks to you, I understood the it well.)
After deleting pending queue in ActiveMQ GUI console, kahaDB journals were deleted.

Thanks a lot.
Kawai
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: The size of KahaDB doesn't seem to decrease

Hidekazu
In reply to this post by tabish121@gmail.com
Dear Tabish121

As you introduced the helpful and fundamental specification of kahaDB to me,
the reason of the size of kahaDB kept increasing was really elementary.
(I'm sorry)
there were a lot of pending queues in ActiveMQ and that was the reason.

anyway, your advice was really helpful.
thanks a lot.

Kawai
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: The size of KahaDB doesn't seem to decrease

cschneider
In reply to this post by Hidekazu
We recently had a case at a customer where some dead letter queues
retained messages for a long time while messages in all other queues
were short lived. The long lived messages were messages from short lived
queues that were moved to the dead letter queues. This created very
sparsely populated kahadb journals of quite extensive size (~50GB).

In that case it helps to switch to multi kahadb:
See chapter "Automatic Per Destination Persistence Adapter" in
http://activemq.apache.org/kahadb.html.

This creates one kahadb per destination. So you can keep the kahadb size
small without manually deleting the queues regularly. The dead letter
queues of course remain but now are very densely populated so they just
use minimal space.

Christian

On 15.03.2017 08:18, Hidekazu wrote:

> Thanks Andreas
>
> The article you introduced me was very helpful to understand the
> specification of kahaDB.
>
> I have to say there was a miss-communication among our developer team.
> I heard that there was no pending queue in ActiveMQ, however,
> There were a lot of pending queue in ActiveMQ and that was the reason kahaDB
> journals were never deleted.
> I'm sorry.. (but thanks to you, I understood the it well.)
> After deleting pending queue in ActiveMQ GUI console, kahaDB journals were
> deleted.
>
> Thanks a lot.
> Kawai
>
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/The-size-of-KahaDB-doesn-t-seem-to-decrease-tp4723669p4723771.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.


--
Christian Schneider
http://www.liquid-reality.de

Open Source Architect
http://www.talend.com

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: The size of KahaDB doesn't seem to decrease

Hidekazu
Dear cschneider

Thanks for teaching me a good way to avoid increasing kahaDB.
I came to know that dead letter queues are one of keys.

By the way, I have two questions. (I'm sorry, I'm ActiveMQ newcomer...)

1. How do I see dead letter queues?
   I want to make sure whether dead letter queue make kahadb bigger.
   command? filename for dead letter queue?

2.I was able to keep KahaDB small by deleteing pending queues and
   delete journal files.
   however, I noticed that db.data kept increasing.
   I think this is a different problem from managing kahaDB journals.
   Do you know how you keep db.data small?

Thanks
Kawai
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: The size of KahaDB doesn't seem to decrease

cschneider
On 15.03.2017 13:01, Hidekazu wrote:

> Dear cschneider
>
> Thanks for teaching me a good way to avoid increasing kahaDB.
> I came to know that dead letter queues are one of keys.
>
> By the way, I have two questions. (I'm sorry, I'm ActiveMQ newcomer...)
>
> 1. How do I see dead letter queues?
>     I want to make sure whether dead letter queue make kahadb bigger.
>     command? filename for dead letter queue?
You can see dead letter queues like any other queues in the web console.
There is no specific filename as in the default kahadb setup all queues
are handled in the same set of kahadb journal files.

>
> 2.I was able to keep KahaDB small by deleteing pending queues and
>     delete journal files.
>     however, I noticed that db.data kept increasing.
>     I think this is a different problem from managing kahaDB journals.
>     Do you know how you keep db.data small?
Be careful about deleting journal files. This can easily damage the
integrity and you can loose messages.
A journal file is automatically deleted when activemq finds that all
entries in there are fully handled.
So simply manage your queues and activemq takes care of the journals.

If activemq does not seem to delete old journal files then one thing you
can do is write a small script that reads the messages of long term
queues like dead letter queues and just writes them to another queue.
This way the old message is done for activemq and the new message will
be in the newest journal.

db.data is just a kind of cache of the meta data from the journals. So
when activemq is stopped you can safely delete it.
See e.g. https://access.redhat.com/solutions/276323

Christian

>
> Thanks
> Kawai
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/The-size-of-KahaDB-doesn-t-seem-to-decrease-tp4723669p4723778.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.


--
Christian Schneider
http://www.liquid-reality.de

Open Source Architect
http://www.talend.com

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: The size of KahaDB doesn't seem to decrease

Tim Bain
In reply to this post by Hidekazu
1a) The default name for the dead letter queue is DLQ. You can use the web
console (http://activemq.apache.org/web-console.html) to view the queues
the broker has. Note that ActiveMQ will automatically create a destination
when the first message is published into it, so if it doesn't exist then it
just means that there hasn't yet been a message moved to the DLQ.

1b) The link Tim Bish provided contains instructions for telling which
destinations are causing your KahaDB log files to be held. If the DLQ had
old messages, it's likely that you would see that queue in the output of
that command, telling you that those old DLQ messages were keeping log
files from being deleted.

2) db.data is the index. It won't shrink when messages are deleted, but it
should stop growing once the broker reaches its max size. So it may not
stay small per se (depending on your definition of small), but it should
not grow unbounded. The comments on
https://issues.apache.org/jira/plugins/servlet/mobile#issue/AMQ-3736 may be
useful for understanding the expected behavior of the index file.

Tim

On Mar 15, 2017 6:08 AM, "Hidekazu" <[hidden email]> wrote:

Dear cschneider

Thanks for teaching me a good way to avoid increasing kahaDB.
I came to know that dead letter queues are one of keys.

By the way, I have two questions. (I'm sorry, I'm ActiveMQ newcomer...)

1. How do I see dead letter queues?
   I want to make sure whether dead letter queue make kahadb bigger.
   command? filename for dead letter queue?

2.I was able to keep KahaDB small by deleteing pending queues and
   delete journal files.
   however, I noticed that db.data kept increasing.
   I think this is a different problem from managing kahaDB journals.
   Do you know how you keep db.data small?

Thanks
Kawai



--
View this message in context: http://activemq.2283324.n4.nab
ble.com/The-size-of-KahaDB-doesn-t-seem-to-decrease-tp4723669p4723778.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: The size of KahaDB doesn't seem to decrease

Hidekazu
In reply to this post by cschneider
Dear cschneider

Thanks to your advice, we could delete data.db safely.

I found that activemq.xml had a parameter which limits the cache size.
Is the parameter below perhaps for db.data?
Or is it different from the limit for db.data because it explains that "cached in memory."

------------
http://activemq.apache.org/kahadb.html
  indexCacheSize
  10000
  Number of index pages cached in memory.
------------

I would appreciate if you know activemq has a parameter to limit the size of data.db and tell me it.

Anyway, you are my ActiveMQ teacher.
I understand ActiveMQ more than before thanks to you.

Kawai
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: The size of KahaDB doesn't seem to decrease

Hidekazu
In reply to this post by Tim Bain
Dear Tim

Thanks for helping me understand the specification of ActiveMQ.
Thanks to you, I understand ActiveMQ more than before.

I could delete data.db safely.
(Thank you very much!!)

By the way, I use ActiveMQ with TOPIC (not queue.)
Will I see DLQ in queue tab in ActiveMQ console when my ActiveMQ with Topic has a DLQ?

Anyway, I come to know what ActiveMQ is like thanks to you.

kawai
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: The size of KahaDB doesn't seem to decrease

Tim Bain
Yes, the DLQ is a queue, irrespective of what destination type the messages
were originally sent to.

Tim


On Mar 17, 2017 1:49 AM, "Hidekazu" <[hidden email]> wrote:

Dear Tim

Thanks for helping me understand the specification of ActiveMQ.
Thanks to you, I understand ActiveMQ more than before.

I could delete data.db safely.
(Thank you very much!!)

By the way, I use ActiveMQ with TOPIC (not queue.)
Will I see DLQ in queue tab in ActiveMQ console when my ActiveMQ with Topic
has a DLQ?

Anyway, I come to know what ActiveMQ is like thanks to you.

kawai



--
View this message in context: http://activemq.2283324.n4.
nabble.com/The-size-of-KahaDB-doesn-t-seem-to-decrease-
tp4723669p4723836.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: The size of KahaDB doesn't seem to decrease

Tim Bain
In reply to this post by Hidekazu
No, indexCacheSize is not a limit on the size of the index. It specifies
how much of the index should be stored in memory (which is fast); if that's
not enough to hold the full index, the remaining content will be paged in
from db.data on disk (which is slow).

As I understand it, db.data grows only based on the max number of live
messages and the number of live destinations (topics, for you), so the only
way to limit it is to ensure that messages are consumed promptly, and to
promptly GC any destinations you no longer need (this only applies if you
continually create and discard short-lived destinations, which may not be
how you use ActiveMQ).

But let's step back for a second: why are you trying to limit the size of
db.data in the first place? Why is its current size a problem?

Tim

On Mar 17, 2017 1:32 AM, "Hidekazu" <[hidden email]> wrote:

Dear cschneider

Thanks to your advice, we could delete data.db safely.

I found that activemq.xml had a parameter which limits the cache size.
Is the parameter below perhaps for db.data?
Or is it different from the limit for db.data because it explains that
"cached in memory."

------------
http://activemq.apache.org/kahadb.html
  indexCacheSize
  10000
  Number of index pages cached in memory.
------------

I would appreciate if you know activemq has a parameter to limit the size of
data.db and tell me it.

Anyway, you are my ActiveMQ teacher.
I understand ActiveMQ more than before thanks to you.

Kawai



--
View this message in context: http://activemq.2283324.n4.
nabble.com/The-size-of-KahaDB-doesn-t-seem-to-decrease-
tp4723669p4723835.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: The size of KahaDB doesn't seem to decrease

Hidekazu
Dear Tim

Thanks for your always help to understand ActiveMQ.
I didn't know ActiveMQ first at all but thanks to you ,
now I became to be confident about it a little.

> But let's step back for a second: why are you trying to limit the size of
> db.data in the first place? Why is its current size a problem?

We've managed ActiveMQ even thought didn't know how to configure/manage it.
Our ActiveMQ db size limit (it was LevelDB at that time) used to be just 1GB. lol!
I thought it was legit because we didn't know the default size limit of DB in activemq.conf. (it's 100GB)
We came across the trouble because of surpassing the size limit (1GB) of DB.
We wouldn't have been afraid of DB size if had set the db size limit properly.
The reason, i was afraid that data.db could grow to the point where it surpass a disk size.

But even if not having this problem, I think every files should be small not to surpass the disk size.

Thanks.
Kawai
Loading...