Tayra performs selective restore to a point in time

Narrative

IncyWincyWebApp stores some of its persistent data on MongoDB. Adam, MongoDB admin, performs backup of a replica set on an ongoing basis. On a not so great day, Dave, the developer approaches Adam and requests:

Dave: Hey Adam, looks like something has gone wrong with the data since 10 minutes.
Adam offering a solution: Do you want me to restore only the collections which you want ?
Dave: Sure Adam, that'll be great if you could restore only transactions database and ips collection in users database.


Given source replica set and target node are running
with configuration source {unsecureSrcNode} on port {unsecureSrcPort} , target {unsecureTgtNode} on port {unsecureTgtPort}

1. Do Some CUD Operations

run mongo command on
source
Comments
use database users
run
db.createCollection("profile")
run
db.profile.insert({name:"One"})
run
db.profile.save({name:"Two"})
run
db.profile.save({name:"Three"})
run
db.profile.save({name:"Five"})
run
db.profile.save({name:"Six"})
run
db.profile.save({name:"Seven"})
run
db.profile.save({name:"Eight"})
run
db.profile.remove({name:"One"})
run
db.profile.update({name:"Three"}, {$set : {name:"Four"}})
run
db.sessions.insert({sessionId:"One"})
run
db.ips.insert({ip:"1.2.3.4"})
use database transactions
run
db.orders.insert({order:"One"})
run
db.orders.save({order:"Two"})

2. Run Backup Utility

Open terminal
and run
backup -s {unsecureSrcNode} --port={unsecureSrcPort} -f {file}
and show
stdout
and ensure
stdout
contains Process started

3. Get Time in Timestamp format 10 mins before from the backed up file.

Note: As it is technically not possible to -
  • Get handle to the backedup file while the process is holding it, we will query the Source MongoDB oplog, the master source.
  • Know the timestamp 10 mins before as different machines have different execution speeds and sleep is a bad strategy.
    Instead, we will simulate this by going back 10 records in Oplog and get timestamp of the 10th record.

Totally we dumped 15 documents, we travel back 10 records, get the timestamp and we expect 5 documents in target.

Open oplog for node
source
and travel 10 documents back in time and save timestamp in {sUntil}

4. Run Restore Utility to a Point in time.

Open terminal
and run
restore -d {unsecureTgtNode} --port={unsecureTgtPort} -f {file} --sUntil={sUntil}
and show
stdout
and ensure
stdout
contains Process started

4. Ensure Expected Results

Checking documents of database users
Ensure source and target has
run in database
users
query
db.profile.count()
and cleanup databases false
source valuedestination value
64
Ensure source and target has
find documents for collection
profile
in database
users
where
null
and cleanup databases false
source valuedestination value
{ "name" : "Two"}
{ "name" : "One"}
{ "name" : "Four"}
{ "name" : "Two"}
{ "name" : "Five"}
{ "name" : "Three"}
{ "name" : "Six"}
{ "name" : "Five"}
{ "name" : "Seven"}
{ "name" : "Eight"}

Ensure source and target has
run in database
users
query
db.sessions.count()
and cleanup databases false
source valuedestination value
10

Ensure source and target has
run in database
users
query
db.ips.count()
and cleanup databases false
source valuedestination value
10

Ensure source and target has
find documents for collection
ips
in database
users
where
null
and cleanup databases true
source valuedestination value
{ "ip" : "1.2.3.4"}


Checking documents of database transactions
Ensure source and target has
run in database
transactions
query
db.orders.count()
and cleanup databases true
source valuedestination value
20