Clayton's Tech Bits

Home

Contact

Resumé / C.V.

Links

Search this site:
Custom Search

Categories:

/ (224)
  Admin/ (86)
    Apache/ (7)
      HTTPS-SSL/ (4)
    Cherokee/ (1)
    LAN/ (4)
    LVM/ (3)
    Monitoring/ (2)
      munin/ (2)
    OpenVPN/ (1)
    SSH-Proxy/ (3)
    SSH-SSL/ (6)
    backups/ (16)
      SpiderOak/ (1)
      backuppc/ (5)
      dirvish/ (1)
      misc/ (6)
      rdiff-backup/ (1)
      rsync/ (1)
      unison/ (1)
    commandLine/ (11)
    crontab/ (1)
    databases/ (8)
      MSSQL/ (2)
      MySQL/ (5)
      PostgreSQL/ (1)
    dynamicDNS/ (2)
    email/ (9)
      Dovecot/ (1)
      deliverability/ (1)
      misc/ (1)
      postfix/ (6)
    iptables/ (2)
    virtualization/ (8)
      VMware/ (1)
      virtualBox/ (7)
  Coding/ (11)
    bash/ (1)
    gdb/ (1)
    git/ (2)
    php/ (4)
    python/ (3)
      Django/ (1)
  Education/ (1)
  Hosting/ (23)
    Amazon/ (14)
      EBS/ (3)
      EC2/ (11)
    Godaddy/ (2)
    NearlyFreeSpeech/ (3)
    Rackspace/ (1)
    vpslink/ (3)
  Linux/ (20)
    Awesome/ (3)
    CPUfreq/ (1)
    Chinese/ (1)
    Debian/ (5)
      WPA/ (1)
    audio/ (1)
    encryption/ (2)
    fonts/ (1)
    misc/ (4)
    router-bridge/ (2)
  SW/ (39)
    browser/ (2)
      Chrome/ (1)
      Firefox/ (1)
    business/ (25)
      Drupal/ (8)
      KnowledgeTree/ (6)
      Redmine/ (2)
      SugarCRM/ (6)
      WebERP/ (2)
      eGroupware/ (1)
    email/ (1)
    fileSharing/ (1)
      mldonkey/ (1)
    graphics/ (2)
    research/ (2)
    website/ (6)
      blog/ (6)
        blosxom/ (3)
        rss2email/ (1)
        webgen/ (1)
  Security/ (12)
    IMchat/ (1)
    circumvention/ (2)
    e-mail/ (4)
    greatFirewall/ (1)
    hacking/ (1)
    password/ (1)
    privacy/ (1)
    skype/ (1)
  Services/ (1)
    fileSharing/ (1)
  TechWriting/ (1)
  xHW/ (13)
    Lenovo/ (1)
    Motorola_A1200/ (2)
    Thinkpad_600e/ (1)
    Thinkpad_a21m/ (3)
    Thinkpad_i1300/ (1)
    Thinkpad_x24/ (1)
    USB_audio/ (1)
    scanner/ (1)
    wirelessCards/ (2)
  xLife/ (17)
    China/ (9)
      Beijing/ (5)
        OpenSource/ (3)
    Expatriation/ (1)
    Vietnam/ (7)

Archives:

  • 2012/03
  • 2012/01
  • 2011/12
  • 2011/11
  • 2011/10
  • 2011/09
  • 2011/08
  • 2011/07
  • 2011/06
  • 2011/05
  • 2011/04
  • 2011/02
  • 2010/12
  • 2010/11
  • 2010/10
  • 2010/09
  • 2010/08
  • 2010/07
  • 2010/06
  • 2010/05
  • 2010/04
  • 2010/03
  • 2010/02
  • 2010/01
  • 2009/12
  • 2009/11
  • 2009/10
  • 2009/09
  • 2009/08
  • 2009/07
  • 2009/06
  • 2009/05
  • 2009/04
  • 2009/03
  • 2009/02
  • 2009/01
  • 2008/12
  • 2008/11
  • 2008/10
  • 2008/09
  • Subscribe XML RSS Feed

    Fri, 25 Jun 2010


    /Hosting/Amazon/EC2: How to Backup an Amazon EBS-Boot Server

    This is a script I wrote to backup several Amazon AWS EBS-boot servers, using the Python boto library[1]. Basically it takes a snapshot of the server's root EBS volume (defined in volumeIDs list) and then builds a new bootable Amazon Machine Image (AMI) for the server based upon that snapshot.

    #!/usr/bin/env python
    
    ##########################################################
    # Installation:
    # This script requires boto ('pip install boto') to communicate with Amazon AWS API,
    # and at least version 1.9a to handle an EBS boot instance
    # Script inspired by http://www.elastician.com/2009/12/creating-ebs-backed-ami-from-s3-backed.html
    #
    # As of early June, we need svn version of boto:
    #   pip install -e svn+http://boto.googlecode.com/svn/trunk/@1428#egg=boto
    # until next release, which should be soon per this forum post:
    #   http://groups.google.com/group/boto-users/browse_thread/thread/21dc3482ed7e49da
    ##########################################################
    
    ##########################################################
    # This script is for the backup of Amazon AWS EBS Boot servers, and will perform one backup
    # of one server, per script invocation. Note that there is nothing machine environment-specific
    # in this script at the moment, so it can be run on any machine with the correct environment,
    # to backup any other machine.
    # Usage:
    # * adjust the "Constants" section to reflect you current account & server environment.
    # * Choose which server you would like to backup in the "which server" section.
    # * Install the ElasticFox plugin in Firefox to observer the results of the backup process.
    # * Run the script to perform a backup of the chosen server.
    ##########################################################
    
    ##########################################################
    # Constants
    ##########################################################
    instances = {"master":"i-xxxxxxxx",   "staging":"i-xxxxxxxx",   "production":"i-xxxxxxxx"}
    volumeIDs = {"master":"vol-xxxxxxxx", "staging":"vol-xxxxxxxx", "production":"vol-xxxxxxxx"}
    ownerID = 'xxxxxxxxxxxx' # I got this from ElasticFox, should be AWS account specific
    waitSnapshot = 10 # wait increment (in seconds) while waiting for snapshot to complete
    
    ##########################################################
    # Which server is this script backing up?
    ##########################################################
    # thisServer = "master"
    thisServer = "staging"
    # thisServer = "production"
    
    thisInstance = instances[thisServer]
    thisVolume = volumeIDs[thisServer]
    
    ##########################################################
    print ''
    print '##########################################################'
    print 'Backup of ' + thisServer + ' server:'
    print 'Logging into Amazon AWS....'
    from boto.ec2.connection import EC2Connection
    conn = EC2Connection('xxxxxxxxxxxxxxxxxxxx', 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx')
    
    ##########################################################
    from datetime import datetime
    timeStamp = str(datetime.today())
    
    print ''
    print timeStamp
    print 'Snapshotting the ' + thisServer + ' servers boot volume....'
    comment = thisServer + ' ' + timeStamp
    name = comment.replace(":",".") # AWS does not like ":" in AMI names
    name = name.replace(" ","_") # make life easier when deleting stale images
    snapshot = conn.create_snapshot(thisVolume, comment)
    newSnapshot = snapshot.id
    print 'Created new volume snapshot:', newSnapshot
    
    ##########################################################
    import time
    waitSnapshotTotal = waitSnapshot
    snapshot = conn.get_all_snapshots(str(newSnapshot))
    print ''
    while snapshot[0].status != 'completed':
        print 'Snapshot status is ' + snapshot[0].status + ', ' \
              'wait ', waitSnapshotTotal, ' secs for the snapshot to complete before building the AMI.'
        time.sleep(waitSnapshot)
        waitSnapshotTotal = waitSnapshotTotal + waitSnapshot
        snapshot = conn.get_all_snapshots(str(newSnapshot))
    
    ##########################################################
    print ''
    print 'Building a bootable AMI based up this snapshot....'
    
    # setup for building an EBS boot snapshot"
    from boto.ec2.blockdevicemapping import EBSBlockDeviceType, BlockDeviceMapping
    ebs = EBSBlockDeviceType()
    ebs.snapshot_id = newSnapshot
    block_map = BlockDeviceMapping()
    block_map['/dev/sda1'] = ebs
    
    # use the same kernel & ramdisk from running server in the new AMI:
    attribute = conn.get_instance_attribute(thisInstance, 'kernel')
    kernelID = attribute['kernel']
    print 'kernel ID = ', kernelID
    attribute = conn.get_instance_attribute(thisInstance, 'ramdisk')
    ramdiskID = attribute['ramdisk']
    print 'ramdisk ID = ', ramdiskID
    
    # create the new AMI:
    result = conn.register_image(name=name,
        description=timeStamp,
        architecture='i386',
        kernel_id=kernelID,
        ramdisk_id=ramdiskID,
        root_device_name='/dev/sda1',
        block_device_map=block_map)
    print 'The new AMI ID = ', result
    

    [1] http://code.google.com/p/boto/

    posted at: 08:12 | path: /Hosting/Amazon/EC2 | permanent link to this entry