How to create and rotate Automatic Snapshot?

Question

On Compute Engine we can do Snapshots, which are basically backups. Could you try to figure out how we could create a script to do automated snapshots every day and keep lik

Answer 1

Also, at the time of writing this, Windows instances support Volume Shadow Copy Service (VSS), but Linux instances do not.

Hence, you can safely snapshot Windows drives whilst the instance is running using the --guest-flush switch, but not so for Linux drives.

Prior to snapshotting Linux drives, you will need some other mechanism to prepare it for snapshotting i.e. freeze the drive, detach the drive or poweroff the instance.

Answer 2

The following is a very rude ruby script to accomplish this task. Please, consider it just as an example from which to take inspiration.

Any feedback to improve it is welcome ;-)

require 'date'

ARCHIVE = 30 # Days
DISKS   = [] # The names of the disks to snapshot
FORMAT  = '%y%m%d'

today = Date.today
date = today.strftime(FORMAT).to_i
limit = (today - ARCHIVE).strftime(FORMAT).to_i

# Backup
`gcloud compute disks snapshot #{DISKS.join(' ')} --snapshot-names #{DISKS.join("-#{date},")}-#{date}`

# Rotate
snapshots = []
`gcloud compute snapshots list`.split("\n").each do |row|
  name = date
  row.split("\s").each do |cell|
    name = cell
    break
  end
  next if name == 'NAME'
  snapshots << name if name[-6, 6].to_i < limit
end
`yes | gcloud compute snapshots delete #{snapshots.join(' ')}` if snapshots.length > 0

Answer 3

Update Google Cloud has scheduled backups that can be configured per disk. See Creating scheduled snapshots for persistent disk in the Google Cloud documentation.

Original Answer

Documentation is pretty clear about how to do it:

gcloud compute disks snapshot DISK

Note, that

Snapshots are always created based on the last successful snapshot taken

And before you will remove any of your snapshots -- take a look on that diagram:

More information about API.

Answer 4

UPDATE:

The script has changed a lot since I first gave this answer - please see Github repo for latest code: https://github.com/jacksegal/google-compute-snapshot

ORIGINAL ANSWER:

I had the same problem, so I created a simple shell script to take a daily snapshot and delete all snapshots over 7 days: https://github.com/Forward-Action/google-compute-snapshot

#!/usr/bin/env bash
export PATH=$PATH:/usr/local/bin/:/usr/bin

#
# CREATE DAILY SNAPSHOT
#

# get the device name for this vm
DEVICE_NAME="$(curl -s "http://metadata.google.internal/computeMetadata/v1/instance/disks/0/device-name" -H "Metadata-Flavor: Google")"

# get the device id for this vm
DEVICE_ID="$(curl -s "http://metadata.google.internal/computeMetadata/v1/instance/id" -H "Metadata-Flavor: Google")"

# get the zone that this vm is in
INSTANCE_ZONE="$(curl -s "http://metadata.google.internal/computeMetadata/v1/instance/zone" -H "Metadata-Flavor: Google")"

# strip out the zone from the full URI that google returns
INSTANCE_ZONE="${INSTANCE_ZONE##*/}"

# create a datetime stamp for filename
DATE_TIME="$(date "+%s")"

# create the snapshot
echo "$(gcloud compute disks snapshot ${DEVICE_NAME} --snapshot-names gcs-${DEVICE_NAME}-${DEVICE_ID}-${DATE_TIME} --zone ${INSTANCE_ZONE})"


#
# DELETE OLD SNAPSHOTS (OLDER THAN 7 DAYS)
#

# get a list of existing snapshots, that were created by this process (gcs-), for this vm disk (DEVICE_ID)
SNAPSHOT_LIST="$(gcloud compute snapshots list --regexp "(.*gcs-.*)|(.*-${DEVICE_ID}-.*)" --uri)"

# loop through the snapshots
echo "${SNAPSHOT_LIST}" | while read line ; do

   # get the snapshot name from full URL that google returns
   SNAPSHOT_NAME="${line##*/}"

   # get the date that the snapshot was created
   SNAPSHOT_DATETIME="$(gcloud compute snapshots describe ${SNAPSHOT_NAME} | grep "creationTimestamp" | cut -d " " -f 2 | tr -d \')"

   # format the date
   SNAPSHOT_DATETIME="$(date -d ${SNAPSHOT_DATETIME} +%Y%m%d)"

   # get the expiry date for snapshot deletion (currently 7 days)
   SNAPSHOT_EXPIRY="$(date -d "-7 days" +"%Y%m%d")"

   # check if the snapshot is older than expiry date
if [ $SNAPSHOT_EXPIRY -ge $SNAPSHOT_DATETIME ];
        then
     # delete the snapshot
         echo "$(gcloud compute snapshots delete ${SNAPSHOT_NAME} --quiet)"
   fi
done