Backup Integrity Checks in the Cloud

Moving into the cloud is a big paradigm shift. Its hard to imagine what such a big change might look like, so start small. Don’t try to offload your whole system at once. Offload the small stuff that doesn’t fit anywhere else.

One interesting use case for using the cloud is backup integrity checks for databases. Here is the scenario. You are doing backups regularly of your databases. You need to restore them every once and a while and verify they aren’t corrupt. There are a few options:

  • Run this on your production server - Asking for trouble

  • Run this on a dedicated server for integrity checks - $$$ Expensive

  • Run this on a shared machine - Sharing is for adolescents, your developers should share as little as possible

But there is one more option, infrastructure in the cloud. One of my teams recently started using cloud servers to perform the actual verification checks. So you continue to do the backups as normal. But now, your integrity checks spin up a new server in a few minutes, upload the data and verification code, and run the tests. Once your done, you spin down the server and your done. (Cloud servers doesn’t have an API yet, so we just leave a cheap instance running currently, but once the API is released it will be fully automated)

cloud-server-screen

This is very useful for a few reasons. You don’t mess with your production environment. You only need a small amount of computing power to accomplish this, so you only pay for a small amount of computing power. You don’t share servers, so one team can’t hurt other teams.

Once you understand the new paradigm, super simple solutions to problems start appearing everywhere. Don’t try to rewrite everything at once, just start making small iterative changes that make sense.