Mongodb incremental backups

巧了我就是萌 提交于 2020-07-22 03:06:14

问题


I was given the task to setup incremental backups for mongodb replicaset, as start point of course I googled about it and could not find anything on mongodb docs, I did find however this question which encouraged to develop my own solution as didn't find Tayra very active.

I read about oplog and realized it was very easy to develop something to replay the log, but it turns out that I didn't have to as mongorestore does that for me.

Now I have a working solution with bash scripts and it was quite easy, that's the reason I am asking here if there is any flaw in my logic, or maybe something that will bite me in the future.

Below how I implemented that:

Full backup procedure:

  1. lock writes on a secondary member db.fsyncLock()
  2. Take snapshot
  3. Record last position from oplog

db.oplog.rs.find().sort({$natural:-1}).limit(1).next().ts

  1. Unlock writes db.fsyncUnlock()

Incremental backup procedure:

  1. lock writes on a secondary member
  2. Dump oplog from the recorded oplog position on full (or latest incremental ) backup: mongodump --host <secondary> -d local -c oplog.rs -o /mnt/mongo-test_backup/1 --query '{ "ts" : { $gt : Timestamp(1437725201, 50) } }'
  3. Record latest oplog position (same way as for full backups)
  4. Unlock writes

Full backup restore procedure:

  1. stop all instances of mongod
  2. copy snapshot to data dir of the box which will be the primary, but make sure to exclude all local* and mongod.lock this restore technique is called reconfigure by breaking mirror
  3. Start primary
  4. reconfigure replicaset
  5. start secondaries without any data, let them perform initial sync. Or copy the data from the new primary with fresh local database

Restore incremental backup:

When we created incremental backup it stored it like this:

/mnt/mongo-test_backup/1/local/oplog.rs.bson
/mnt/mongo-test_backup/1/local/oplog.rs.metadata.json

We're instered on oplog.rs.bson but we will have to rename it, so here are the steps:

  1. change directory to the backup: cd /mnt/mongo-test_backup/1/local
  2. delete the json file rm *.json
  3. rename the bson file mv oplog.rs.bson oplog.bson
  4. restore it : mongorestore -h <primary> --port <port> --oplogReplay /mnt/mongo-test_backup/1/local

I have it all scripted, I may commit it on github later.

Question is if there is any flaw in the logic. I am bit suspicious as the procedure is quite straight forward and still I couldn't find it documented anywhere.

来源:https://stackoverflow.com/questions/31605963/mongodb-incremental-backups

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!