17. Amazon Elastic Block Store

Overview

• Amazon Elastic Block Store (Amazon EBS) provides block level storage volumes for use with EC2 instances
• EBS volumes that are attached to an instance are exposed as storage volumes that persist independently from the life of the instance
• We recommend Amazon EBS for data that must be quickly accessible and requires long-term persistence. 
• EBS volumes are particularly well-suited for use as the primary storage for file systems, databases, or for any applications that require fine granular updates and access to raw, unformatted, block-level storage.

Features of Amazon EBS

• You create an EBS volume in a specific Availability Zone, and then attach it to an instance in that same Availability Zone.
• To make a volume available outside of the Availability Zone, you can create a snapshot and restore that snapshot to a new volume anywhere in that Region.
• Amazon EBS provides the following volume types: General Purpose SSD, Provisioned IOPS SSD, Throughput Optimized HDD, and Cold HDD. 
• You can create your EBS volumes as encrypted volumes
• You can create point-in-time snapshots of EBS volumes, which are persisted to Amazon S3

Benefits of using EBS volumes

• Data availability
  • When you create an EBS volume, it is automatically replicated within its Availability Zone to prevent data loss due to failure of any single hardware component.
  • If you attach multiple volumes to a device that you have named, you can stripe data across the volumes for increased I/O and throughput performance.
  • You can attach io1 and io2 EBS volumes to up to 16 Nitro-based instances. Amazon EBS Multi-Attach enables you to attach a single Provisioned IOPS SSD (io1 or io2) volume to multiple instances that are in the same Availability Zone.
• Data persistence
  • An EBS volume is off-instance storage that can persist independently from the life of an instance. 
  • whether a volume is deleted or persisted depends on the DeleteOnTermination flag
• Data encryption
  • Amazon EBS encryption uses AWS Key Management Service (AWS KMS) master keys when creating encrypted volumes and any snapshots created from your encrypted volumes.
• Amazon EBS provides the ability to create snapshots (backups) of any EBS volume 
• Flexibility: EBS volumes support live configuration changes while in production. You can modify volume type, volume size, and IOPS capacity without service interruptions. 

Amazon EBS volume types

• General Purpose SSD volumes (gp2 and gp3) balance price and performance for a wide variety of transactional workloads. These volumes are ideal for use cases such as boot volumes, medium-size single instance databases, and development and test environments.
• Provisioned IOPS SSD volumes (io1 and io2) are designed to meet the needs of I/O-intensive workloads that are sensitive to storage performance and consistency.
• Throughput Optimized HDD volumes (st1) provide low-cost magnetic storage that defines performance in terms of throughput rather than IOPS. These volumes are ideal for large, sequential workloads such as Amazon EMR, ETL, data warehouses, and log processing.
• Cold HDD volumes (sc1) provide low-cost magnetic storage that defines performance in terms of throughput rather than IOPS. These volumes are ideal for large, sequential, cold-data workloads. If you require infrequent access to your data and are looking to save costs, these volumes provides inexpensive block storage.

Solid state drives (SSD)

	General Purpose SSD		Provisioned IOPS SSD
Volume type	gp3	gp2	io2 Block Express ‡	io2	io1
Durability	99.8% - 99.9% durability (0.1% - 0.2% annual failure rate)	99.8% - 99.9% durability (0.1% - 0.2% annual failure rate)	99.999% durability (0.001% annual failure rate)	99.999% durability (0.001% annual failure rate)	99.8% - 99.9% durability (0.1% - 0.2% annual failure rate)
Use cases	Low-latency interactive apps Development and test environments		Workloads that require: Sub-millisecond latency Sustained IOPS performance More than 64,000 IOPS or 1,000 MiB/s of throughput	Workloads that require sustained IOPS performance or more than 16,000 IOPS I/O-intensive database workloads
Volume size	1 GiB - 16 TiB		4 GiB - 64 TiB	4 GiB - 16 TiB
Max IOPS per volume (16 KiB I/O)	16,000		256,000	64,000 †
Max throughput per volume	1,000 MiB/s	250 MiB/s *	4,000 MiB/s	1,000 MiB/s †
Amazon EBS Multi-attach	Not supported		Supported
Boot volume	Supported

Hard disk drives (HDD)

	Throughput Optimized HDD	Cold HDD
Volume type	st1	sc1
Durability	99.8% - 99.9% durability (0.1% - 0.2% annual failure rate)	99.8% - 99.9% durability (0.1% - 0.2% annual failure rate)
Use cases	Big data Data warehouses Log processing	Throughput-oriented storage for data that is infrequently accessed Scenarios where the lowest storage cost is important
Volume size	125 GiB - 16 TiB	125 GiB - 16 TiB
Max IOPS per volume (1 MiB I/O)	500	250
Max throughput per volume	500 MiB/s	250 MiB/s
Amazon EBS Multi-attach	Not supported	Not supported
Boot volume	Not supported	Not supported

General Purpose SSD volumes (gp3)

• These volumes deliver a consistent baseline rate of 3,000 IOPS and 125 MiB/s, included with the price of storage. You can provision additional IOPS (up to 16,000) and throughput (up to 1,000 MiB/s) for an additional cost.
• The maximum ratio of provisioned IOPS to provisioned volume size is 500 IOPS per GiB. The maximum ratio of provisioned throughput to provisioned IOPS is .25 MiB/s per IOPS. 
• 32 GiB or larger: 500 IOPS/GiB x 32 GiB = 16,000 IOPS
• 8 GiB or larger and 4,000 IOPS or higher: 4,000 IOPS x 0.25 MiB/s/IOPS = 1,000 MiB/s
•  A gp3 volume can range in size from 1 GiB to 16 TiB.

General Purpose SSD volumes (gp2)

• These volumes deliver single-digit millisecond latencies and the ability to burst to 3,000 IOPS for extended periods of time.
• Between a minimum of 100 IOPS (at 33.33 GiB and below) and a maximum of 16,000 IOPS (at 5,334 GiB and above), baseline performance scales linearly at 3 IOPS per GiB of volume size. For example, a 100 GiB gp2 volume has a baseline performance of 300 IOPS.
•  A gp2 volume can range in size from 1 GiB to 16 TiB.
• The performance of gp2 volumes is tied to volume size, which determines the baseline performance level of the volume and how quickly it accumulates I/O credits; larger volumes have higher baseline performance levels and accumulate I/O credits faster. 
• Each volume receives an initial I/O credit balance of 5.4 million I/O credits, which is enough to sustain the maximum burst performance of 3,000 IOPS for at least 30 minutes.
• The maximum I/O credit balance for a volume is equal to the initial credit balance (5.4 million I/O credits).
• Burst duration = (Credit balance) /((Burst IOPS) - 3(Volume size in GiB))

• Volume size (GiB)	Baseline performance (IOPS)	Burst duration when driving sustained 3,000 IOPS (second)	Seconds to fill empty credit balance when driving no IO
  1	100	1,802	54,000
  100	300	2,000	18,000
  250	750	2,400	7,200
  334 (Min. size for max throughput)	1,002	2,703	5,389
  500	1,500	3,600	3,600
  750	2,250	7,200	2,400
  1,000	3,000	N/A*	N/A*
  5,334 (Min. size for max IOPS)	16,000	N/A*	N/A*
  16,384 (16 TiB, max volume size)	16,000	N/A*	N/A*

• Throughput in MiB/s = ((Volume size in GiB) × (IOPS per GiB) × (I/O size in KiB))

Provisioned IOPS SSD volumes

• Provisioned IOPS SSD volumes can range in size from 4 GiB to 16 TiB and you can provision from 100 IOPS up to 64,000 IOPS per volume.
• You can achieve up to 64,000 IOPS only on Instances built on the Nitro System. On other instance families you can achieve performance up to 32,000 IOPS. 
• The maximum ratio of provisioned IOPS to requested volume size (in GiB) is 50:1 for io1 volumes, and 500:1 for io2 volumes.
• io1 volume 1,280 GiB in size or greater (50 × 1,280 GiB = 64,000 IOPS)
• io2 volume 128 GiB in size or greater (500 × 128 GiB = 64,000 IOPS)
• io2 Block Express volumes is the next generation of Amazon EBS storage server architecture
• With io2 Block Express volumes, you can provision volumes with:
  • Sub-millisecond average latency
  • Storage capacity up to 64 TiB (65,536 GiB)
  • Provisioned IOPS up to 256,000, with an IOPS:GiB ratio of 1,000:1. Maximum IOPS can be provisioned with volumes 256 GiB in size and larger (1,000 IOPS x 256 GiB = 256,000 IOPS).
  • Volume throughput up to 4,000 MiB/s.

Throughput Optimized HDD volumes

• Volume size determines the baseline throughput of your volume, which is the rate at which the volume accumulates throughput credits.
• Volume size also determines the burst throughput of your volume, which is the rate at which you can spend credits when they are available. 

Volume size (TiB)	ST1 base throughput (MiB/s)	ST1 burst throughput (MiB/s)
0.125	5	31
0.5	20	125
1	40	250
2	80	500
3	120	500
4	160	500
5	200	500
6	240	500
7	280	500
8	320	500
9	360	500
10	400	500
11	440	500
12	480	500
12.5	500	500
13	500	500
14	500	500
15	500	500
16	500	500

Cold HDD volumes

• Volume size determines the baseline throughput of your volume, which is the rate at which the volume accumulates throughput credits.
• Volume size also determines the burst throughput of your volume, which is the rate at which you can spend credits when they are available. 

Volume Size (TiB)	SC1 Base Throughput (MiB/s)	SC1 Burst Throughput (MiB/s)
0.125	1.5	10
0.5	6	40
1	12	80
2	24	160
3	36	240
3.125	37.5	250
4	48	250
5	60	250
6	72	250
7	84	250
8	96	250
9	108	250
10	120	250
11	132	250
12	144	250
13	156	250
14	168	250
15	180	250
16	192	250

Amazon EBS Multi-Attach

• Multi-Attach enabled volumes can be attached to up to 16 Linux instances built on the Nitro System that are in the same Availability Zone. 
• Using Multi-Attach with a standard file system can result in data corruption or loss, so this not safe for production workloads. You can use a clustered file system to ensure data resiliency and reliability for production workload
• Multi-Attach enabled volumes do not support I/O fencing.
• Multi-Attach enabled volumes can't be created as boot volumes.
• You can enable Multi-Attach for io1 and io2 volumes during creation.
  • You can also enable Multi-Attach for io2 volumes after they have been created.
  • You can not enable/disable Multi-Attach for io1 volumes after they have been created.
• Multi-Attach enabled volumes are deleted on instance termination if the last attached instance is terminated and if that instance is configured to delete the volume on termination.

Amazon EBS snapshots

• You can back up the data on your Amazon EBS volumes to Amazon S3 by taking point-in-time snapshots.
• Snapshots are incremental backups
• When you create an EBS volume based on a snapshot,The replicated volume loads data in the background so that you can begin using it immediately
• Multi-volume snapshots allow you to take exact point-in-time, data coordinated, and crash-consistent snapshots across multiple EBS volumes attached to an EC2 instance.
• A snapshot is constrained to the AWS Region where it was created. After you create a snapshot of an EBS volume, you can use it to create new volumes in the same Region.
• You can also copy snapshots across Regions, making it possible to use multiple Regions for geographical expansion, data center migration, and disaster recovery. 
• EBS snapshots fully support EBS encryption.
  • Snapshots of encrypted volumes are automatically encrypted.
  • Volumes that you create from encrypted snapshots are automatically encrypted.
  • Volumes that you create from an unencrypted snapshot that you own or have access to can be encrypted on-the-fly.
  • When you copy an unencrypted snapshot that you own, you can encrypt it during the copy process.
  • When you copy an encrypted snapshot that you own or have access to, you can reencrypt it with a different key during the copy process.
  • The first snapshot you take of an encrypted volume that has been created from an unencrypted snapshot is always a full snapshot.
  • The first snapshot you take of a reencrypted volume, which has a different CMK compared to the source snapshot, is always a full snapshot.
• By default, snapshots of EBS volumes on an Outpost are stored in Amazon S3 in the Region of the Outpost. You can also use Amazon EBS local snapshots on Outposts to store snapshots of volumes on an Outpost locally in Amazon S3 on the Outpost itself.

Create Amazon EBS snapshots

Relations among multiple snapshots of the same volume 

Relations among incremental snapshots of different volumes

Delete an Amazon EBS snapshot

• To delete multi-volume snapshots, retrieve all of the snapshots for your multi-volume snapshot set using the tag you applied to the set when you created the snapshots. Then, delete the snapshots individually.

Deleting a snapshot with some of its data referenced by another snapshot

Amazon Data Lifecycle Manager

• You can use Amazon Data Lifecycle Manager to automate the creation, retention, and deletion of EBS snapshots and EBS-backed AMIs.
• An Amazon Data Lifecycle Manager policy (described later) targets an instance or volume for backup using a single tag.
• You can create up to 100 lifecycle policies per Region.
• You can add up to 45 tags per resource.

Lifecycle policies

• Policy type—Defines the type of resources that the policy can manage. Amazon Data Lifecycle Manager supports two types of lifecycle policies:
  • Snapshot lifecycle policy—Used to automate the lifecycle of EBS snapshots. 
  • EBS-backed AMI lifecycle policy—Used to automate the lifecycle of EBS-backed AMIs. 
  • Cross-account copy event policy—Used to automate the copying of snapshots across accounts. This policy type should be used in conjunction with an EBS snapshot policy that shares snapshots across accounts.
• Resource type—Defines the type of resources that are targeted by the policy.
  • Use VOLUME to create snapshots of individual volumes, or use INSTANCE to create multi-volume snapshots of all of the volumes that are attached to an instance. 
  • AMI lifecycle policies can target instances only.
  • Snapshot lifecycle policies can target instances or volumes. 
• Target tags—Specifies the tags that must be assigned to an EBS volume or an Amazon EC2 instance for it to be targeted by the policy.
• Schedules—The start times and intervals for creating snapshots or AMIs. 
• Retention—Specifies how snapshots or AMIs are to be retained.

Policy schedules

• Policy schedules define when snapshots or AMIs are created by the policy. 
• Policies can have up to four schedules—one mandatory schedule, and up to three optional schedules.
• Adding multiple schedules to a single policy lets you create snapshots or AMIs at different frequencies using the same policy
•  If multiple schedules are initiated at the same time, Amazon Data Lifecycle Manager creates only one snapshot or AMI and applies the retention settings of the schedule that has the highest retention period. 

Amazon EBS data services

Amazon EBS Elastic Volumes

• With Amazon EBS Elastic Volumes, you can increase the volume size, change the volume type, or adjust the performance of your EBS volumes.
• If your instance supports Elastic Volumes, you can do so without detaching the volume or restarting the instance. 
• Linux AMIs require a GUID partition table (GPT) and GRUB 2 for boot volumes that are 2 TiB (2,048 GiB) or larger. 
• You can't decrease the size of an EBS volume. 
• If you cannot use Elastic Volumes but you need to modify the root (boot) volume, you must stop the instance, modify the volume, and then restart the instance.
• Elastic Volumes are supported on the following instances:
  • All current-generation instances
  • The following previous-generation instances: C1, C3, CC2, CR1, G2, I2, M1, M3, and R3
• After you increase the size of an EBS volume, you must use file system–specific commands to extend the file system to the larger size.

Amazon EBS encryption

• Amazon EBS encryption uses AWS KMS keys when creating encrypted volumes and snapshots.
• How EBS encryption works when the snapshot is encrypted
  • Amazon EC2 sends a GenerateDataKeyWithoutPlaintext request to AWS KMS, specifying the KMS key that you chose for volume encryption.
  • AWS KMS generates a new data key, encrypts it under the KMS key that you chose for volume encryption, and sends the encrypted data key to Amazon EBS to be stored with the volume metadata.
  • When you attach the encrypted volume to an instance, Amazon EC2 sends a CreateGrant request to AWS KMS so that it can decrypt the data key.
  • AWS KMS decrypts the encrypted data key and sends the decrypted data key to Amazon EC2.
  • Amazon EC2 uses the plaintext data key in hypervisor memory to encrypt disk I/O to the volume. The plaintext data key persists in memory as long as the volume is attached to the instance.
• Your data key never appears on disk in plaintext.
• When you create a new, empty EBS volume, you can encrypt it by enabling encryption for the specific volume creation operation.
• Although there is no direct way to encrypt an existing unencrypted volume or snapshot, you can encrypt them by creating either a volume or a snapshot.
• Restore an unencrypted volume (encryption by default not enabled)
  • Without encryption by default enabled, a volume restored from an unencrypted snapshot is unencrypted by default. However, you can encrypt the resulting volume by setting the Encrypted parameter and, optionally, the KmsKeyId parameter.
  • 
• Restore an unencrypted volume (encryption by default enabled)
  • When you have enabled encryption by default, encryption is mandatory for volumes restored from unencrypted snapshots, and no encryption parameters are required for your default KMS key to be used.
  • 
• Copy an unencrypted snapshot (encryption by default not enabled)
  • Without encryption by default enabled, a copy of an unencrypted snapshot is unencrypted by default. However, you can encrypt the resulting snapshot by setting the Encrypted parameter and, optionally, the KmsKeyId parameter.
  • 
• Copy an unencrypted snapshot (encryption by default enabled)
  • When you have enabled encryption by default, encryption is mandatory for copies of unencrypted snapshots, and no encryption parameters are required if your default KMS key is used. 
  • 
• Re-encrypt an encrypted volume
  • When the CreateVolume action operates on an encrypted snapshot, you have the option of re-encrypting it with a different KMS key. 
  • 
• Re-encrypt an encrypted snapshot
  • The ability to encrypt a snapshot during copying allows you to apply a new symmetric KMS key to an already-encrypted snapshot that you own.
  • 

Amazon EBS fast snapshot restore

• Amazon EBS fast snapshot restore enables you to create a volume from a snapshot that is fully initialized at creation. 
• Fast snapshot restore must be explicitly enabled on a per-snapshot basis. 
• You can enable up to 50 snapshots for fast snapshot restore per Region.

Amazon EBS–optimized instances

• EBS–optimized instances deliver dedicated bandwidth to Amazon EBS.

RAID configuration

• With Amazon EBS, you can use any of the standard RAID configurations that you can use with a traditional bare metal server, as long as that particular RAID configuration is supported by the operating system for your instance. This is because all RAID is accomplished at the software level.
• Amazon EBS volume data is replicated across multiple servers in an Availability Zone to prevent the loss of data from the failure of any single component. 
• Creating a RAID 0 array allows you to achieve a higher level of performance for a file system than you can provision on a single Amazon EBS volume.
• Use RAID 0 when I/O performance is of the utmost importance.
• The resulting size of a RAID 0 array is the sum of the sizes of the volumes within it, and the bandwidth is the sum of the available bandwidth of the volumes within it.
• To create a consistent set of snapshots for your RAID array, use Multi-volume snapshots 

Reference

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AmazonEBS.html