Replicated LevelDB : Manually copying data from one leveldb store to another

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Replicated LevelDB : Manually copying data from one leveldb store to another

khandelwalanuj
Hi,

Is it possible to copy data from leveldb store of one broker to another broker while using replicated leveldb.

For ex. : Let's say 3 brokers are running fine and replicating the data. I suddenly stop one slave broker. After some time before starting the slave Can I manually copy leveldb directory of master and then start the slave ? (I am copying each and every file other than the nodeid.txt)

In my test setup it is working fine but not sure if it will fail in some corner cases. Let me know if some issues are there with this.


Thanks,
Anuj
Reply | Threaded
Open this post in threaded view
|

Re: Replicated LevelDB : Manually copying data from one leveldb store to another

Tim Bain
I've never used LevelDB, but I'd be concerned about what you're proposing (copying the files that back a database while that database is running) no matter what the database technology was.  If you're copying from the master while it's active, then the master could be writing to its files while you're doing your copy, and I doubt that anyone will tell you it's 100% guaranteed that you won't get an inconsistent copy of the files.  Most of the time you'll probably be fine, but I'd bet that there are race conditions where you wouldn't be, so you're probably rolling the dice each time you do it.  So for the process you've described, the answer is almost certainly "don't do it, because you can't guarantee atomicity of your copy operation and/or synchronization with the database's write operations".

I would expect better odds if you shut down the master and then copy its files, since then you know it's not writing to them as you copy them over, but that may or may not accomplish your goal.  (You haven't said what that goal is, or why you can't just restart the slave with its existing LevelDB database, so I can't speculate on what you're actually trying to do.)

But again, I'm not a LevelDB expert and it doesn't seem like there are many of them on this mailing list, so since you want answers about the inner workings of LevelDB, you might want to post those questions on the LevelDB mailing list instead.


khandelwalanuj wrote
Hi,

Is it possible to copy data from leveldb store of one broker to another broker while using replicated leveldb.

For ex. : Let's say 3 brokers are running fine and replicating the data. I suddenly stop one slave broker. After some time before starting the slave Can I manually copy leveldb directory of master and then start the slave ? (I am copying each and every file other than the nodeid.txt)

In my test setup it is working fine but not sure if it will fail in some corner cases. Let me know if some issues are there with this.


Thanks,
Anuj
Reply | Threaded
Open this post in threaded view
|

Re: Replicated LevelDB : Manually copying data from one leveldb store to another

khandelwalanuj
This post was updated on .
I tested it again very carefully and found some interesting aspects :

> 3 brokers were running fine where one is master and other two are slaves.
> I stopped one of the slaves and deleted all leveldb files.
> I copied all master broker leveldb files into the slaves leveldb directory. (removed nodeid.txt)
> Now when i start primary, I am seeing that
  > One tmp folder is created which has some .log files.
  > Now slave will download data log files from master and as soon as it copies all the log files, it is deleting the stash folder and i guess replaying in indexes from logs.
 > After that everything works well./

Snapshot of stash  directory:
14:59]khandela@agni05.hyd:/u/khandela/activemq/amq3>du -sh leveldb/*
101M leveldb/0000000000000000.log
952K leveldb/00000000064004c6.log
4.0K leveldb/dirty.index
0 leveldb/lock
0 leveldb/nodeid.txt
13M leveldb/stash
0 leveldb/store-version.txt

and
[14:59]khandela@agni05.hyd:/u/khandela/activemq/amq3>du -sh leveldb/stash/*
101M leveldb/stash/0000000000000000.log
0 leveldb/stash/0000000000000000.log.crc32
5.1M leveldb/stash/00000000064004c6.index
7.2M leveldb/stash/00000000064004c6.log


I want to know what is "stash.tmp" and "stash" and ".crc32" files and why they are created ? I didn't find any documentation about these files.  Please respond.


Thanks,
Anuj