Friday, August 14, 2009

Amazon Adds Data Portability With New Import/Export Service

Amazon Web Service today announced a new AWS Import/Export feature. A potentially huge step forward for data portabilty when using the Amazon Cloud computing infrastructure.

In a recent post by AWS Evangelist, Jeff Barr outlined the new functionality stating that "using a workflow similar to the one you'd use to import data, you prepare a MANIFEST file, email it to us, receive a job identifier in return, and then send us one or more specially prepared storage devices. We'll take the devices, verify them against your manifest file, copy the data from one or more S3 buckets to your device(s) and ship them back to you."

A "specially prepared" storage device contains a SIGNATURE file. The file uniquely identifies the Import/Export job and also authenticates your request.

You can use the new CREATE EXPORT PLAN email command to simplify the process of exporting a data set that won't fit on a single storage device. Given the block size, and the device capacity (either formatted or unformatted), the command returns a link to a zip file with a set of MANIFEST files inside.

You will be charged a fixed fee of $80.00 per device and $2.49 per hour for the time spent copying the data to your device. Normal charges for S3 requests also apply. There is no charge for bandwidth.

Barr also outlines several uses for this new feature including.

  • Disaster Recovery - If your local storage fails or is destroyed, use the Export feature to retrieve your precious data.
  • Data Retrieval - After creating a large data set (either by gathering it up or by computing it) in the cloud, use the Export feature to get a local copy.
  • Data Distribution - Take a large data set, sell copies, and use the Export feature to take care of the distribution.
  • Data Processing - Use the Import feature to load a large data set (yours or a customers') into the cloud, do some computationally intensive processing (e.g. de-duplication), and then get the data back using the Export feature.
Amazon goes futher outlining the theoretical transfer time if you're using the internet to transfer your data.

Available Internet Connection Theoretical Min. Number of Days to Transfer 1TB at 80% Network Utilization When to Consider AWS Import/Export?
82 days 100GB or more
10Mbps 13 days 600GB or more
3 days 2TB or more
100Mbps 1 to 2 days 5TB or more
1000Mbps Less than 1 day 60TB or more

Sign up here to get started with AWS Import/Export.

As always, nice work Amazon!
Reblog this post [with Zemanta]

#DigitalNibbles Podcast Sponsored by Intel

If you would like to be a guest on the show, please get in touch.