Amazon EC2 CPU steal

[总结]CPU steal time就是虚拟机调度器把同一个CPU分配给其他VM的时间，  Amazon small EC2 VM 正常情况下只能分配到40%左右的CPU资源(Intel(R) Xeon(R) CPU           E5430  @ 2.66GHz).   2.66 * 40% 正好在Amazon 承诺的1.0-1.2 GHzXeon processor区间。

Steal Time（ST）通常还被称作“Stolen CPU”，存在于虚拟计算环境 —— CPU使用内部虚拟机运行任务的时间，由Hypervisor将CPU周期分配给其他“外部任务”产生；而这些外部任务很可能就是你吵闹的邻居（云端共享同一片资源的用户）递交的。

由于虚拟化层在底层的基础设施上，Linux实例不会报告CPU的正确使用率。为了得到EC2基础设施上正确的CPU利用率，云用户只能使用CloudWatch来测量。

It mean the total CPU utilization taking up for my processes are %user + %steal


This is expected for m1.small instances. They're half a physical core, so you'll never get more than 50% utilization.
Your m1.small instance does not get a whole CPU to itself and any CPU time not allocated to you looks like it is being "stolen" by the hypervisor.


'Steal Time' is the CPU time that the hypervisor allocated to something else other than your VM, when your VM wanted to run. 


The steal time means you're trying to go over your allocated resources. AWS doesn't give you a dedicated amount of resources, it gives you access to more, so that when the host machine isn't under heavy load you can use more resources, which would otherwise be wasted.


"Steal time is the time that (1) the CPU had something runnable, but (2) the hypervisor chose to run something else instead."
(from http://3spoken.wordpress.com/2006/12/10/cpu-steal-time-the-new-statistic/)


The m1.small is stated to have "1 EC2 Compute Unit (1 virtual core with 1 EC2 Compute Unit)" and "One EC2 Compute Unit provides the equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor." (both from http://aws.amazon.com/ec2/instance-types/)


If you look at /proc/cpuinfo, you will see that the real processor in use has much more capacity than the processor in the description. So Amazon is sharing that processor between multiple instances. The steal is when it is servicing other instances.


I'm assuming that in the worst case scenario, the following (taken from the EC2 description) is still true:


"Each instance predictably provides the equivalent of a system with a 1.7Ghz x86 processor, 1.75GB of RAM, 160GB of local disk, and 250Mb/s of network bandwidth"


just for general information:
 %steal
              Show  the  percentage of time spent in involuntary wait by the virtual CPU
              or CPUs while the hypervisor was servicing another virtual processor.


%steal is the amount of CPU that the hypervisor allocated to another VM instead of giving it to your guest VM.
              
I understand that if I have a CPU heavy application (e.g. tight infinite loop) the steal time goes up. 
This is because I'm using up all the ticks alotted to me (my share of CPU). I assume hypervisor is giving everybody a fair share.


Possible workarounds: 
- relaunch your instance and try your luck again. With any luck you'll get a host whose other guests are less demanding.
- move to a larger instance type. The .small instances use the smallest fraction of the host's hardware, and the .medium and .large consume progressively higher shares. The .xlarge instances have the hardware all to themselves (still within the hypervisor, though).




How does Netflix handle CPU "steal time"?
http://blog.sciencelogic.com/netflix-steals-time-in-the-cloud-and-from-users/03/2011
"One performance statistic that is very valuable to Netflix is stolen time. (...)
So how does Netflix handle this problem when using Amazon’s Cloud? Adrian admits that they tracked this statistic so closely that when an instance crossed a stolen time threshold 
the standard operating procedure at Netflix was to kill the VM and start it up on a different hypervisor.What Netflix realized over time was that once a VM was performing poorly because another VM was crashing the party, usually due to a poorly written or compute intensive application hogging the machine, it never really got any better and their best learned approach was to get off that machine."