AWS学习笔记——Chapter4 Introduction to Amazon Elastic Compute Cloud

前注：
学习书籍

Introduction to Amazon Elastic Compute Cloud

1. Benefits of Amazon EC2

Time to market

Scalability

Control

Reliable

Secure

Multiple instance type

Integration

Cost effective

2. Amazon EC2 Instance Types and Features

(1) Types

Instance Type: determines the hardware of the host computer used for your instance.

Two types of instances in EC2 ecosystem: current-generation instances, previous-generation instances.

Current-generation: contain the latest of everything (eg. the latest version of chipsets, memory, processor…)

Previous-generation: consist of the machines that are one or two generations older than the current one.

Instance types can be broadly divided into the following categories: general purpose, compute optimized, memory optimized, storage optimized, advanced computing.

General Purpose (T2, M5, M4, and M3)

Provide a balance of computer memory and network resources and are a pretty good choice for many applications.

T2: provide burst performance, suitable for workloads that do not use full CPU utilization, such as web server, development environments …

M5, M4 and M3: don’t provide burstable performance, can be used for building websites, development environments, build servers, code repositories, microservices, test and staging environments and so on.

Compute Optimized (C5, C4 and C3)

Used for the application or workload that is heavy on compute, such as media transcoding, applications supporting a large number of concurrent users, long-running batch jobs, high-performance computing, gaming servers and so on.

Memory Optimized (X1E, X1, R4, and R3)

Used for the workload that has a lot of memory requirements like memory databased, such as a SAP HANA or Oracle database in-memory, NoSQL databases like MongoDB and Cassandra, big data processing engines like Presto or Apache Spark, high-performance computing (HPC) and Electronic Design Automation (EDA) applications, Genome assembly and analysis, and so on.

Storage Optimized (H1, I3 and D2)

Used for the workloads that require high sequential read and write access to very large data sets on local storage, such as running a relational database that is I/O bound, running an I/O-bound application, NoSQL databases, data warehouse applications, MapReduce and Hadoop distributed caches for in-memory databased like Redis and so on.

Advanced Computing (P3, P2, G3 and F1)

Used for high-processing computing requirements, such as machine learning algorithms, molecular modeling, genomics, computation of fluid dynamics, computational finance and so on.

(2) Features

Processor Features

EC2 instances use an Intel processor, so they use all the processor features that
Intel provides.

Network Features

EC2-Classic: the original release of EC2, instances run in a single, flat network that is shared with other customers.

Now all accounts are automatically enabled with the default Amazon VPC.

A placement group / cluster networking: a logical grouping of instances within a single AZ, can provide the benefit of low-latency or high-network throughput.

R4, X1, M5, M4, C5, C4, C3, I2, P3, P2, G3 and D2 instances support cluster networking.

A placement group cannot span multiple AZs.

Recommend:

· Use the same type of instance in a placement group.

· To get the maximum benefit for the placement group, should choose an instance type that supports “enhanced networking”.

Storage Features

The block storage that you attach along with the EC2 instance is known as Elastic Block Storage (EBS).

Some EC2 instances include a local disk in the physical hardware, which is known as an instance store.

Three types of volumes:

A general-purpose EBS volume: backed up by solid-state drives.

A provisioned IOPS-based EBS volume: maximize the I/O throughput.

A magnetic hard drive: provide the lowest cost per gigabyte.

3. Steps for Using Amazon EC2

(1) Select a preconfigured Amazon Machine Image. You can also create your custom AMI and later use that to launch an instance.

(2) Configure the networking and security (VPC, public/private subnet …)

(3) Choose the instance type

(4) Choose the AZ, attach EBS and optionally choose static EIP

(5) Start the instance and you are all set

4. Pricing for Amazon EC2

The instances are divided into three categories from a pricing perspective: On-demand instance, Reserved instance, Spot instance.

(1) On-demand Instance

The most popular pricing model.

Pay just for the usage on a flat hourly rate or per-second billing.

No up-front costs or hidden costs or anything else.

(2) Reserved Instance

A reserved instance is ideal when you know your application has a pretty steady state or is predictable in terms of performance. (how many resources your workload is going to take and for how long)

Compared to an on-demand instance, a reserved instance provides up to a 75 percent discount.

Time options: either one-year or three-year commitment.

Payment options: all up-front costs, partial up-front costs, no up-front costs.

Two subcategories: standard reserved instance, convertible reserved instance.

The convertible reserved instance provides better flexibility if your compute requirement changes over the given period of time. It is only for three-year commitment.

Standard and convertible reserved instances can be purchased to apply to instances in a specific availability zone or to instances in a region.

(3) Spot Instance

Bid for the unused capacity and create the instance.

The spot price fluctuates based on supply and demand, and if someone overbids, you then lose the instance at a very short notice.

Spot instances are great for running non-mission-critical projects. Often customers add a few spot instances along with on-demand instances to provide additional horsepower.

5. Shared Tenancy, Dedicated Hosts, and Dedicated instances

EC2 runs on a virtualized environment, therefore, it is possible that on the same physical machine another customer might be running a different EC2 instance.

Sometimes, because of some compliance requirement, you may have to segregate your instances even at the physical level. Dedicated hosts and dedicated instances solve this problem.

(1) Shared Tenancy

Default behavior when you launch an EC2 instance. You run the EC2 instances on multitenant hardware.

(2) Dedicated Host

A physical server exclusively assigned to you.

Dedicated hosts can help you reduce costs by allowing you to use your existing server-bound software licenses, including Windows Server, SQL Server, and SUSE Linux Enterprise Server.

(3) Dedicated Instance

The EC2 instances run on single-tenant hardware.

Dedicated instances are Amazon EC2 instances that run in a virtual private cloud on hardware that’s dedicated to a single customer.

6. Instance and AMIs

An AMI is a blueprint that has all the details of the software configuration of the server that you are going to launch in the Amazon Cloud.

The AMI contains a blueprint about the root volume for the instance, and the root volume contains information about the operating system and various other software running on top of the operating system.

Launch permission information that controls what AWS account can use the AMI to launch the instance.

Public: grant launch permissions to all AWS accounts.

Explicit: grant launch permissions to specific AWS accounts.

Implicit: have implicit launch permission for an AMI.

(1) Instance Root Volume

Instance store-backed AMI

When EC2 is launched initially, all the root devices used to get launched from S3 since the instance root used to be backed up at S3.

If you launch an instance that is backed up by instance store, then when the instance is launched, the image that is used to boot the instance is copied to the root volume.

As long as the instance is running, all the data in the instance store volume persists, but whenever the instance is terminated, all the data is gone. The instance store-backed instances do not support the stop action, so you can’t stop the instance. The only exception is when the instance is rebooted, data in the instance store persists.

It is important that you should back up your data to a persistent storage regularly and/or on your instance stores across multiple AZs.

Also, you can’t detach an instance store volume from one instance and re-attach it to a different instance.

Amazon EBS-backed AMI

When the instance is launched, the root device for the instance is launched from the EBS volume, which is created from an EBS snapshot.

If you launch an instance that is backed up any Amazon EBS-backed instance, the =data always persists. Even if the instance terminates or fails, you don’t lose data. These instances also support the stop action.

Since the instance root volume runs out of an EBS volume, you can even attach the root volume of your instance to a different running instance for debugging or any other purpose, such as changing the size of instance or modifying the properties of the instance.

(2) Obtaining an AMI

From AWS publisher, community, AWS Marketplace …

An AMI is a regional resource. If you want to share an AMI to a different region, you need to copy the AMI to a different region and then share it.

(3) Virtualization in AMI

Linux AMI use one of two types of virtualization: Hardware Virtual Machine (HVM), Paravirtual (PV) (半虚拟化).

The difference between them is how they boot and how they take advantage of the hardware extensions in the context of CPU, memory and storage to provide better performance.

HVM AMI

HVM AMI executes the master boot records of the root block device and then presents a fully virtualized set of hardware to the operating system. As a result, the operating system runs directly on top of the VM as it is without any modification similar to the way it runs on a bare-metal hardware.

HVM AMI执行根块设备的主引导记录，然后将完整的虚拟化硬件集提供给操作系统。因此，操作系统可以直接在VM上运行而无需任何修改，就像其在裸机硬件上运行一样。

The performance becomes really fast since HVM guests can take full advantage of all the hardware extensions that provide fast access to the underlying hardware on the host system.

All current-generation instance types support HVM AMIs. The CC2, CR1, HI1, and HS1 previous-generation instance types support HVM AMIs.

PV AMI

PV AMIs boot with a boot loaded called PV-GRUB. It starts the boot cycle and loads the kernel specified in the menu.lst file on your image. Paravirtual guests can run on host hardware that does not have explicit support for virtualization. But unfortunately, they can’t really take advantage of special hardware extensions that HVM can take such as enhanced networking or GPU processing and so on.

PV AMIs在启动时加载了称为PV-GRUB的启动程序，其开始启动周期，并在映像上加载menu.lst文件中指定的内核。半虚拟化客户机可以在不显式支持虚拟化的主机硬件上运行。但不幸的是，它们无法真正利用HVM可以利用的特殊硬件扩展，如增强的网络或GPU处理等。

The C3 and M3 current-generation instance types support PV AMIs. The C1, HI1, HS1, M1, M2, and T1 previous-generation instance types support PV AMIs.

Amazon recommends you use an HMV image to get the maximum performance when you launch
your instance.

7. Instance Life Cycle

(1) Launch

When you launch an instance, immediately it enters into the “pending” state.

Before starting the instance, a few health checks are performed to make sure that there are no issues with the hardware and the instance can come online without any issues.

Once the instance is up and running and is ready for you, it enters into the “running” state.

As soon as it is in the running state, the instance is ready for all practical purposes, and you can connect to it and start using it. At this moment, the billing starts.

(2) Start and Stop

If health check fails, the instance does not get started. → You can either start a new instance or try to fix the issue.

If there are no issues with the health check, the instance starts normally. → You can start using it thereafter.

You can stop an instance only if it is backed up by an EBS-backup instance. You can’t stop an instance backed by an instance store.

When you stop your instance, it enters the “stopping” state and then the “stopped” state.

(3) Reboot

You can reboot an instance that is either backed up by instance store or backed by EBS.

All the data is saved after the reboot. The IP address, machine type, and DNS name all remain the same after the reboot.

You can reboot the instance either via the Amazon console or via CLI and API calls.

(4) Termination

If you do not need the instance anymore, you can terminate it.

As soon as you terminate the instance, you will see that the status changes to “shutting down” or “terminated”.

If the instance has termination protection enabled, you may have to perform an additional step, or you may have to disable termination protection to terminate the environment.

The Amazon EBS volume supports the “DeleteOnTermination” attribute, which controls whether the volume is deleted or preserved when you terminate the instance. The default behavior is to delete the root device volume and preserve any other EBS volumes.

(5) Retirement

When AWS determines there is an irreparable hardware failure that is hosting the instance, then the instance is either retired or scheduled to be retired.

If the instance is a backed EBS volume, you have all the data stored in the EBS volume.

If the instance’s root volume is backed up by the instance store, you must take the backup of all the files stored in the instance store before it gets terminated or you will lose all the data.

8. Connecting to an Instance

(1) Launch an instance in a public subnet

It will be assigned a public IP address and public DNS name via which you can reach the instance from the Internet and a private IP address and private DNS. If you choose the IPv6 address, it will be allocated a public IPv6 IP address as well.

Public DNS Name: automatically create; stay with the instance for its tenure; cannot change.

Public IP Address: automatically create; unique; cannot modify; persist for the life span of the instance; cannot associate the same public IP address in any other server even if you terminate an instance; if you want to associate an IP address from one server to another, you can do it via an elastic IP address.

(2) Launch an instance in a private subnet

It will be allocated a private IP address and a private DNS.

(3) Connect to an instance

To connect to an instance, the console prompts you to download a private key in your local machine and then change the permission in it.

Amazon EC2 uses the public-private key concept used in cryptography to encrypt and decrypt the login information.

9. Security Group

A security group acts as a virtual firewall that controls the traffic for one or more instances.

(1) The characteristics of security group rules

By default, security groups allow all outbound traffic;

Can’t change the outbound rules for an EC2-Classic security group;

Security group rules are always permissive, can’t create deny access rules;

Security group are stateful. (If you send a request from your instance, the response traffic for that request is allowed to flow in regardless of the inbound security group rules.)

Can add and remove rules at any time, the changes are automatically applied.

When you associate multiple security groups with an instance, the rules from each security group are effectively aggregated to create one set of rules.

(2) For each rule, you specify the following

Protocol: the most common protocols are 6 (TCP), 17 (UDP) and 1 (ICMP).

Port range: For TCP, UDP or a custom protocol, you can specify a single port number or a range of port numbers.

ICMP type and code

Source or destination: the source(inbound rules) or destination (outbound rules) for the traffic. Specify one of these options: an individual IPv4 address, (VPC only) an individual IPv6 address, a range of IPv4 addresses, (VPC only) a range of IPv6 addressed, another security group, the current security group.

EC2-Classic

EC2-Classic

EC2-VPC

10.Amazon Elastic Container Service (ECS)

Amazon Elastic Container Service (ECS) is a container management service that allows you to manage Docker containers on a cluster of Amazon EC2.

Containers are similar to hardware virtualization (like EC2), but instead of partitioning a machine, containers isolate the processes running on a single operating system. (容器类似于硬件虚拟化（如EC2），但是容器不对计算机进行分区，而是隔离在单个操作系统上运行的进程。)

You can use the OS kernel to create multiple isolated user space processes that can have constraints on them like CPU and memory. (可以使用OS内核创建多个隔离的用户空间进程，这些进程可能会对它们产生约束，如CPU和内存。)

These isolated user space processes are called containers.

Containers enable the concept of microservices.

Benefits of running containers on Amazon ECS:

· Eliminates cluster management software. (No need to install any cluster management software).

· Easily manage clusters for any scale.

· Can design fault-tolerant cluster architecture.

· Can manage cluster state using Amazon ECS.

· Can easily control and monitor the containers seamlessly.

· Can scale from one to tens of thousands of containers almost instantly.

· ECS gives you the ability to make good placement decisions about where to place your containers.

· ECS gives you the intel about the availability of resources (CPU, memory).

· At any time, you can add new resources to the cluster with EC2 Auto Scaling.

· It is integrated with other services such as Amazon Elastic Container Registry, ELB, EBS, Elastic Network Interfaces, VPC, IAM and CloudTrail.