How to test backup AWS AMI?

Management and Administration
1

On AWS, I have created a backup AMI (with appropriate snapshot EBS volumes) from a running production Express Plus instance.

I would like to test that backup without affecting the production instance at all.

What steps do I have to take, either before or after clicking the scary “Launch instance from image” button?

Do I need to create a new VPC?

Mess with KMS?

IAM roles?

Shut down the running production instance?

Elastic IP / Lets Encrypt?

New S3 bucket?

Anything else?

I noticed that the Standard package includes an Automated Recovery script. Any chance that one of those could be created for Express Plus?

2

Couldn’t you create another Express Plus and then detach and reattach the volumes from the snapshot?

edit: How about this?

  1. What’s important are the values in sqlconf.php of the openemr (root) volume.
  2. Practice a restore of a database dump of the production database volume to the backup database volume.
  3. Use the test database credentials that has the imported backup db for the test openemr volume.
3

Ugh. Good question.

For Standard, it looks like we basically…

  • Ask what KMS key is currently protecting the recovery snapshot and buckets.
  • Spin up the snapshot we’re pointed at.
  • Jot down the bucket we’re pointed at. (Note: This will use that bucket going forward for automated backups, so it needs to be duplicated if you’re just testing. I wonder if I wrote that down anywhere?)
  • Spin up a VPC and subnets as directed, then launch a new instance.
  • Skip all of the normal setup steps (including messing with sqlconf) and instead run an immediate restore of the filesystem from S3. With the database restored from snapshot and the files restored from S3, we’re once more operational.

This Standard restoration process doesn’t use a backup AMI, though – it’s essentially a clean install that has the old state recovered and grafted onto it.

So for an Express Plus AMI… all of the state is already on the instance, really. You’d want to make sure it didn’t overwrite production backups on S3, but that’s the only trap I can really see coming. As long as you inhibit its ability to write to the bucket you shouldn’t be able to damage your data by launching the instance.

Options to stop the instance from writing to the bucket include:

  • Disabling backups before you snapshot and turning them back on after. (Delete etc/cron.daily/duplicity-backups to accomplish this.)
  • Deleting that file /after/ the snapshot is taken and restored into a volume by mounting the volume on another host as a second drive and editing the volume directly before any instance tries to boot from it.
  • Duplicate the IAM instance role granted to the instance and then remove the S3 permissions from that duplicate, then apply that role instead of the original to the instance before launch.

As for writing a recovery script… doable but might be rough. I’ll note the request though.