Narrative
IncyWincyWebApp stores some of its persistent data on MongoDB.
Adam, MongoDB admin, performs backup of a replica set on an ongoing
basis. On a not so great day, Dave, the developer approaches Adam and requests:
Dave: Hey Adam, looks like something has gone wrong with the data since 10 minutes.
Adam offering a solution: Do you want me to restore only the collections
which you want ?
Dave: Sure Adam, that'll be great if you could restore only
transactions database and
ips collection in
users database.
Given source replica set and target node are running |
with configuration |
source {unsecureSrcNode} on port {unsecureSrcPort} , target {unsecureTgtNode} on port {unsecureTgtPort} |
1. Do Some CUD Operations
run mongo command on |
source |
Comments |
use database |
users |
|
run |
db.createCollection("profile") |
|
run |
db.profile.insert({name:"One"}) |
|
run |
db.profile.save({name:"Two"}) |
|
run |
db.profile.save({name:"Three"}) |
|
run |
db.profile.save({name:"Five"}) |
|
run |
db.profile.save({name:"Six"}) |
|
run |
db.profile.save({name:"Seven"}) |
|
run |
db.profile.save({name:"Eight"}) |
|
run |
db.profile.remove({name:"One"}) |
|
run |
db.profile.update({name:"Three"}, {$set : {name:"Four"}}) |
|
run |
db.sessions.insert({sessionId:"One"}) |
|
run |
db.ips.insert({ip:"1.2.3.4"}) |
|
use database |
transactions |
|
run |
db.orders.insert({order:"One"}) |
|
run |
db.orders.save({order:"Two"}) |
|
2. Run Backup Utility
Open terminal |
and run |
backup -s {unsecureSrcNode} --port={unsecureSrcPort} -f {file} |
and show |
stdout |
and ensure |
stdout |
contains |
Process started |
3. Get Time in Timestamp format 10 mins before from the backed up file.
Note: As it is technically not possible to -
- Get handle to the backedup file while the process is holding it, we will
query the Source MongoDB oplog, the master source.
- Know the timestamp 10 mins before as different machines have
different execution speeds and sleep is a bad strategy.
Instead, we
will simulate this by going back 10 records in Oplog and get timestamp
of the 10th record.
Totally we dumped 15 documents, we travel back 10 records, get the timestamp
and we expect 5 documents in target.
Open oplog for node |
source |
and travel |
10 |
documents back in time and save timestamp in |
{sUntil} |
4. Run Restore Utility to a Point in time.
Open terminal |
and run |
restore -d {unsecureTgtNode} --port={unsecureTgtPort} -f {file} --sUntil={sUntil} |
and show |
stdout |
and ensure |
stdout |
contains |
Process started |
4. Ensure Expected Results
Checking documents of database users
Ensure source and target has
|
run in database |
users |
query |
db.profile.count() |
and cleanup databases |
false |
source value | destination value |
6 | 4 |
Ensure source and target has
|
find documents for collection |
profile |
in database |
users |
where |
null |
and cleanup databases |
false |
source value | destination value |
{ "name" : "Two"} | { "name" : "One"} |
{ "name" : "Four"} | { "name" : "Two"} |
{ "name" : "Five"} | { "name" : "Three"} |
{ "name" : "Six"} | { "name" : "Five"} |
{ "name" : "Seven"} | |
{ "name" : "Eight"} | |
Ensure source and target has
|
run in database |
users |
query |
db.sessions.count() |
and cleanup databases |
false |
source value | destination value |
1 | 0 |
Ensure source and target has
|
run in database |
users |
query |
db.ips.count() |
and cleanup databases |
false |
source value | destination value |
1 | 0 |
Ensure source and target has
|
find documents for collection |
ips |
in database |
users |
where |
null |
and cleanup databases |
true |
source value | destination value |
{ "ip" : "1.2.3.4"} | |
Checking documents of database transactions
Ensure source and target has
|
run in database |
transactions |
query |
db.orders.count() |
and cleanup databases |
true |
source value | destination value |
2 | 0 |