 |
AWS Import Export with Snowball
Originally a single device used for migrating data into AWS, Snowball is now part of the AWS Snow family of services used for both speeding up data import and export to and from AWS, and for secure edge computing.
The AWS Snowball family includes:
- AWS Snowball Edge Storage Optimized
- AWS Snowball Edge Compute Optimized
- AWS Snowmobile
The AWS Snow family can import and export exabytes of data into and out of AWS. Snow family devices are owned and operated by AWS, and integrate with other AWS services including compute, storage and monitoring. So let’s take a closer look at each member of the Snow family and explore use cases, and as the title of the post suggests, the costs of using each device.
AWS Snowball Appliance
The AWS Snowball Edge appliance is a ruggedized data migration and edge computing device that now comes in 2 options – Storage Optimized and Compute Optimized. The AWS Snowball can move terabytes of data in around a week – a useful alternative to saturating your Internet or WAN bandwidth.
AWS Snowball Edge Storage Optimized.
The Storage Optimized Snowball Edge has an 80 TB useable storage capacity with 40 vCPUS and 80 GB RAM. It is capable of running both AWS IoT Greengrass and EC2 AMIs (Amazon Machine Images). It can be clustered together with 15 nodes, and is typically used for data migration, content distribution and content aggregation. Data can be transferred to the appliance via NFS or the S3 API.
Generally it takes around 1 week to transfer data with a Snowball Edge appliance – this includes shipping time to and from your premises, and upload into S3.
AWS Snowball Edge Compute Optimized.
The Compute Optimized Snowball Edge has a 42TB usable storage capacity with 52 vCPUS and 208 GB RAM. It is also capable of running both AWS IoT Greengrass and EC2 AMIs (Amazon Machine Images). It can also be clustered together with 15 nodes, and is typically used for data analytics, machine learning inference, tactical edge computing and industrial IoT. Data can be transferred to the appliance via NFS or the S3 API.
Both the Storage Optimized and Compute Optimized Snowball Edge appliances have 2×10 Gb RJ45 , 1 x 25 Gb SFP+ and 1 x 100 Gb QSFP28 network interfaces. A 256-bit encryption algorithm is used on both devices to encrypt data at rest.
Both appliances are capable of being HIPAA compliant if the data requires it.
The appliances are useful for both compute and storage workloads in remote locations such as military or maritime operations where connectivity is scarce or intermittent. Example use cases in these locations including storing and processing data from drones, IoT sensors and CCTV cameras.
How do you order a Snowball Edge?
AWS Snowball Edge appliances can be ordered via the AWS Snowball console. You create one or more Snowball jobs, depending on how many appliances you require, selecting your preferred device. When creating the jobs you need to associate them with an S3 bucket and set up tracking with SNS (Simple Notification Service). If you want any EC2 AMIs on the appliance, you’ll need to add these to the job.
Once the job is set up, AWS will prepare and ship the device to you. On arrival you’ll need to power it up, connect it to your LAN and unlock it with AWS OpsHub. OpsHub is then used to manage the appliance – setting up data transfer or launching EC2 instances from your AMIs. When you’ve finished your local tasks, the appliance is shut down and returned to AWS. What’s really cool is that the shipping label automatically appears on the e-ink screen which is like a Kindle device built in to the Snowball appliance.
AWS Snowball Cost
Pricing for AWS Snowball consists of 4 main components:
- A ‘per job’ service fee.
- Amazon S3 Data Transfer Fees (Transfer in is free, Transfer out priced by region and varies between $0.03 and $0.05 per GB).
- Shipping Fees for the Snowball appliance to and from your location.
- Number of days the Snowball appliance is at your location (the first 10 days are included, and then there is a per day charge after that). The price per day varies by region and also by appliance type, with Storage Optimized being the cheapest, and Compute Optimized with GPU the most expensive.
AWS Snowball Pricing Example
Let’s look at an example based on importing 10TB of data into S3 in the London (eu-west-2) region. The service fee for a Snowball Edge Storage Optimized appliance is $300 for the first 10 days, then $30 per day thereafter. As long as you complete the job within 10 days, you would only be billed $300 plus shipping. You would not be charged for data transfer in to S3, but you would be charged for S3 requests (eg PUT requests) and S3 Storage charges. The request charges will depend on the number of objects being imported. You should note that there are some pretty hefty fees should the Snowball device be lost or irreparably damaged while in your possession, so you may want to check that your insurance covers this:
- Snowball Edge Storage Optimized: $12,500
- Snowball Edge Compute Optimized: $20,000
- Snowball Edge Compute Optimized + GPU – $30,000
AWS Snowmobile
AWS Snowmobile enables Exabyte scale data transfer from on premises datacenters into AWS. Each AWS Snowmobile can transfer up to 100 Petabytes of data into AWS via a ruggedized shipping container towed behind an articulated truck, and multiple Snowmobiles can be used in parallel. That’s equivalent to using 1,250 Snowball Edge devices. Data that would take years or decades to transfer over an Internet connection can be moved to the cloud in a matter of weeks.
Snowmobile can transfer data at up to 1Tbps, which means the 100PB capacity can be filled in less than 10 days if your network capacity can keep up!
In addition to strong 256-bit encryption managed through AWS KMS (Key Management Service) , the AWS Snowmobile has a number of physical security measures including:
- GPS Tracking
- Alarm Monitoring
- 24×7 CCTV Surveillance
- Optional escort security vehicle
AWS will conduct an initial Site Survey which covers your migration objectives, datacenter environment and network infrastructure to feed into a migration plan. Before driving a Snowmobile to your datacenter, AWS will work with you on site preparation to ensure that you have sufficient parking space, power and network connectivity. AWS staff will then configure the Snowmobile to be accessible as a network storage target from your network, and connect it up to your network via a removable connector cabinet which slots in to one of your datacenter racks. The connector cabinet has multiple 40Gbps interfaces which can be aggregated up to 1Tbps throughput.
Once your data is loaded, the Snowmobile will be driven back to the AWS datacenter (with a security escort if required), where your data will be imported into S3.
How do you order a Snowmobile?
AWS Snowmobile can only be ordered by contacting AWS Sales directly – it cannot be ordered via the AWS console.
AWS Snowmobile Cost
AWS Snowmobile pricing is pretty simple. AWS Snowmobile is charges at $0.005 per GB per month, based on the amount of PROVISIONED storage capacity of the Snowmobile – not the actual amount of data that you upload. This is chargeable from the date that the Snowmobile leaves the AWS datacenter facility, right up to the point that the data import into AWS is finished.
As with Snowball, you’ll also be charged for Storage in S3, and any S3 requests, in line with the standard S3 pricing.
Snowball or Snowmobile?
As a rule of thumb, if you are transferring large datasets of 10PB or more from a single location, Snowmobile is the most appropriate choice. If your dataset is less than 10PB, or spread across multiple locations, then you should opt for Snowmobile. You should also consider the availability of bandwidth on your backbone network – Snowmobile will need 100s of Gb/s of spare throughput – if you don’t have this network capacity available, then multiple Snowballs distributed throughout your network may be a better option.
You will also need a spare 350KW of power available to power the Snowmobile, or AWS can provide a generator which takes up around another shipping container’s worth of space. Think 2 articulated lorries parked next to one another…
Snowmobile only supports data import – for export, you have to use Snowball.
AWS Snow Family Feature Comparison
| AWS Snowball Edge Storage Optimized | AWS Snowball Edge Compute Optimized | AWS Snowmobile |
|
|---|---|---|---|
| Usage Scenario | Data Migration, content distribution, content aggregation | data analytics, machine learning inference, tactical edge computing, industrial IoT | Data Migration |
| Storage Capacity | 80 TB usable | 42 TB usable | 100 PB |
| Onboard Computing Options | AWS IoT Greengrass Amazon EC2 AMIs | AWS IoT Greengrass Amazon EC2 AMIs | N/A |
| Encryption | Yes, 256-bit | Yes, 256-bit | Yes, 256-bit |
| Transfers via NFS | Yes | Yes | Yes |
| Transfers via S3 API | Yes | Yes | No |
| Portability | No | No | No |
| Wireless | No | No | No |
| Clustering | Yes, up to 15 Nodes | Yes, up to 15 Nodes | N/A |
| Usable vCPU# | 40 vCPU | 52 vCPU | N/A |
| Available Memory | 80 GB | 208 GB | N/A |
| Network Interfaces | 2x 10 Gbit – RJ45 1x 25 Gbit – SFP+ 1x 100 Gbit – QSFP28 | 2x 10 Gbit – RJ45 1x 25 Gbit – SFP+ 1x 100 Gbit – QSFP28 | 6x 40 Gbit |
| DataSync Agent Pre-installed | No | No | No |
| HIPAA Compliant | Yes, eligible | Yes, eligible | No |
| Typical Job Lifetime | Offline Data Transfer: Days-Weeks | Edge Compute: Weeks-Months | Data Migration: Months |
| Max Job Length | On-going Data Transfer: up to 1,095 Days | Edge Compute: up to 1,095 Days | 120 to 360 Days |
So there you have it – the AWS Snow family. If you’ve got Terabytes, Petabytes or Exabytes of data to move to AWS, then a member of the Snow family is likely to be your quickest and most cost effective option. If you’re daunted by the prospect of moving such huge volumes of data to the AWS cloud, check out Logicata’s AWS Migration Service which can help you to plan and execute your data migration.
