19. AWS Snow Family

AWS Snowcone

• AWS Snowcone is a portable, rugged, and secure device for edge computing and data transfer.
• You can use Snowcone to collect, process, and move data to AWS, either offline by shipping the device to AWS, or online by using AWS DataSync.
• With two CPUs and 8 TB of storage, Snowcone can run edge computing workloads that use Amazon Elastic Compute Cloud (Amazon EC2) instances, and store data securely.
• The Snowcone device is small (227 mm x 148.6 mm x 82.65 mm), so it can be placed next to machinery in a factory to collect, format, and transport data back to AWS for storage and analysis.
• Snowcone supports a file interface with NFS support. 
•  Snowcone is designed for data migration needs up to dozens of terabytes. It can be used in space-constrained environments where Snowball Edge devices don't fit.
• AWS Snowcone is low power, portable, lightweight, and vibration resistant, so you can use it in a wide variety of remote and austere locations.
• AWS Snowcone is a region-specific service, so make sure that the service is available in your region before you plan the job. 

Use Cases

You can use AWS Snowcone for the following use cases:

• For edge computing applications, to collect data, process the data to gain immediate insight, and then transfer the data online to AWS.
• To transfer data that is continuously generated by sensors or machines online to AWS in a factory or at other edge locations.
• To distribute media, scientific, or other content from AWS storage services to your partners and customers.
• To aggregate content by transferring media, scientific, or other content from your edge locations to AWS.
• For one-time data migration scenarios where your data is ready to be transferred, where Snowcone offers a simple, quick, and low-cost way to transfer up to 8 TB of data into AWS by shipping the device back to AWS.

How AWS Snowcone Works

AWS Snowcone Workflow

• Create the job – you request one or more Snowcone devices in the AWS Management Console based on how much data you need to transfer and the compute performance required.
• A device is prepared for your job – The Amazon Simple Storage Service (Amazon S3) buckets, data, and Amazon Elastic Compute Cloud (Amazon EC2) AMIs that you choose are automatically configured, encrypted, and pre-installed on your devices. The AWS DataSync agent is also pre-installed before your devices are shipped to you.
• A device is shipped to you by your region's carrier
• Get a Snowcone power supply
• Receive the device
• Get your credentials and download the AWS OpsHub or Snowball Edge client for Snow family application
• Position the hardware
• Power on the device
• Get the IP address for the device 
• Use AWS OpsHub to unlock the device 
• Use the device 
• Don't unplug the Ethernet or power supply cables
• Prepare the device for its return trip 
• Your region's carrier returns the device to AWS

Data Transfer

• AWS DataSync is an online data transfer service that simplifies, automates, and accelerates copying large amounts of data to and from AWS storage services over the internet or AWS Direct Connect.
• For offline data import jobs, you connect the Snowcone device to your on-premises network and copy data from on-premises storage devices to Snowcone through the NFS file interface.

Job Storage

• For Import Job Storage——Internally, Snowcone contains 8 TiB of disk storage that can be used with the internal NFS service or local Amazon EC2 instances through a local Amazon EBS volume presentation. You can use 8 TB for NFS storage volume and 150 GB for (sbg1) Amazon EBS storage volume.
• For Compute Job Storage——If the job type is local compute, you might create a total of 8 TiB local (sbg1) Amazon EBS volumes and attach them to Amazon EC2 instances. This allows local EC2 instances to access more local capacity than the root volume alone.
  • Note that this is local storage only, so data written to the EBS volumes is lost when the device is returned to AWS because it can't be imported into Amazon S3.
  • The NFS server is not available for compute jobs. If you need to import or export data to or from the AWS Cloud, don't choose the local compute job type when you place your order.

Using AWS Services on AWS Snowcone

• Using Amazon EC2 Compute Instances
• Using AWS DataSync to Transfer Files——It can transfer data between the device and Amazon S3 buckets, Amazon EFS, and Amazon FSx for Windows File Server.
• Using NFS for Offline Data Transfer——transfer data offline to Amazon S3 

Performance

The following table outlines how your network's transfer rate impacts how long it takes to fill an AWS Snowcone with data.

Rate (MB/s)	8 TB Transfer Time	4 TB Transfer Time
100	21.17 hours	10.59 hours
60	36.57 hours	18.29 hours
30	68.57 hours	34.29 hours
10	210.29 hours	105.15 hours

AWS Snowball 

• The AWS Snowball service uses physical storage devices to transfer large amounts of data between Amazon Simple Storage Service (Amazon S3) and your onsite data storage location at faster-than-internet speeds. 
• 80 TB and 50 TB models are available in US Regions; 50 TB model available in all other AWS Regions.
• In the process of being deprecated: The first-generation 80 TB Snowball device is no longer available. Use the Snowball Edge storage optimized devices for all data transfer jobs.

AWS Snowball Edge

• AWS Snowball Edge is a type of Snowball device with on-board storage and compute power for select AWS capabilities. 
• Snowball Edge can do local processing and edge-computing workloads in addition to transferring data between your local environment and the AWS Cloud.
• Snowball Edge devices have three options for device configurations—Storage Optimized, Compute Optimized, and Compute Optimized with GPU.
• AWS Snowball Edge is a region-specific service.
• A cluster job is a special kind of job for local storage and compute only.
• Clusters have 5–10 AWS Snowball Edge devices, called nodes. 
• A cluster offers increased durability and storage capacity.
• Clusters must be either compute-optimized or storage-optimized. A cluster can't be made of a combination of compute-optimized nodes and storage-optimized nodes.
• Using the Amazon S3 Adapter for Snowball, you can programmatically transfer data to and from a Snowball Edge with Amazon S3 API actions. 

Snowball Edge Device Options

Snowball Edge devices have the following options for device configurations:

• Snowball Edge Storage Optimized (for data transfer) – This Snowball Edge device option has a 100 TB (80 TB usable) storage capacity.
• Snowball Edge Storage Optimized (with EC2 compute functionality) – This Snowball Edge device option has up to 80 TB of usable storage space, 24 vCPUs, and 32 GiB of memory for compute functionality. It also comes with 1 TB of additional SSD storage space for block volumes attached to Amazon EC2 AMIs.
• Snowball Edge Compute Optimized – This Snowball Edge device option has the most compute functionality, with 52 vCPUs, 208 GiB of memory, and 42 TB (39.5 usable) plus 7.68 TB of dedicated NVMe SSD for compute instances for block storage volumes for EC2 compute instances, and 42 TB of HDD capacity for either object storage or block storage volumes.
• Snowball Edge Compute Optimized with GPU – This Snowball Edge device option is identical to the Compute Optimized option, except for an installed GPU, equivalent to the one available in the P3 Amazon EC2 instance type. It has a storage capacity of 42 TB (39.5 TB of HDD storage that can be used for a combination of Amazon S3 compatible object storage and Amazon EBS compatible block storage volumes) plus 7.68 TB of dedicated NVMe SSD for compute instances.

AWS Snowball Edge Jobs

• Importing Jobs into Amazon S3 – The transfer of 80 TB or less of your local data copied onto a single device, and then moved into Amazon S3. 
  • With an import job, your data is copied to the AWS Snowball Edge device with the built-in Amazon S3 Adapter for Snowball or NFS mount point. 
  • The import job type is also capable of local storage and compute functionality. This functionality uses the file interface or Amazon S3 Adapter for Snowball to read and write data, and triggers Lambda functions based off of Amazon S3 PUT object API actions running locally on the AWS Snowball Edge device.
• Exporting Jobs from Amazon S3 – The transfer of any amount of data (located in Amazon S3), copied onto any number of Snowball Edge devices, and then moved one AWS Snowball Edge device at a time into your on-premises data destination.
  • Your data source for an export job is one or more Amazon S3 buckets. 
  • The export job type is also capable of local storage and compute functionality. This functionality uses the file interface or Amazon S3 Adapter for Snowball to read and write data, and triggers Lambda functions based off of Amazon S3 PUT object API actions running locally on the AWS Snowball Edge device.
• Local Compute and Storage Only Jobs – These jobs involve one AWS Snowball Edge device, or multiple devices used in a cluster.
  • Local compute and storage jobs enable you to use Amazon S3 and AWS Lambda powered by AWS IoT Greengrass locally, without an internet connection.
  • You can't export data from Amazon S3 onto the device or import data into Amazon S3 when the device is returned.
  • one Lambda function can be associated with one bucket when the job is created.

Speeding Up Data Transfer

• Perform multiple write operations at one time –
• Transfer small files in batches 
• Don't perform other operations on files during transfer 
• Reduce local network use 
• Eliminate unnecessary hops 

AWS Snow Family Use Case Differences

The following table shows the different use cases for the different AWS Snow Family devices.

Use case	Snowball Edge	AWS Snowcone
Import data into Amazon S3	✓	✓
Export from Amazon S3	✓
Durable local storage	✓
Local compute with AWS Lambda	✓
Local compute instances	✓	✓
Durable Amazon S3 storage in a cluster of devices	✓
Use with AWS IoT Greengrass (IoT)	✓
Transfer files through NFS with a GUI	✓	✓
GPU workloads	✓

Reference

https://docs.aws.amazon.com/snowball/latest/snowcone-guide/snowcone-what-is-snowcone.html

https://docs.aws.amazon.com/snowball/latest/developer-guide/notifications.html

https://docs.aws.amazon.com/snowball/latest/ug/whatissnowball.html