CPU亲合力(CPU Affinity)

翻译 2006年06月03日 18:09:00

CPU Affinity

作者:Robert Love   翻译:normalnotebook
Bind specific processes to specific processors with a new system call.
使用新的系统调用,可以将一个进程绑定到一个特定的CPU上运行。

The ability in Linux to bind one or more processes to one or more processors, called CPU affinity, is a long-requested feature. The idea is to say ``always run this process on processor one'' or ``run these processes on all processors but processor zero''. The scheduler then obeys the order, and the process runs only on the allowed processors.

 

       CPU亲合力就是指在Linux中能够将一个或多个进程绑定到一个或多个处理器上运行,这是期待已久的特性。也就是说:“在1号处理器上一直运行该程序”或者是“在所有的处理器上运行这些程序,而不是在0号处理器上运行”。然后,调度器将遵循该规则,程序仅仅运行在允许的处理器上。

Other operating systems, such as Windows NT, have long provided a system call to set the CPU affinity for a process. Consequently, demand for such a system call in Linux has been high. Finally, the 2.5 kernel introduced a set of system calls for setting and retrieving the CPU affinity of a process.

 

        其他类型的操作系统,比如Windows NT,很早以前就开始提供系统调用,用于设置应用程序的CPU亲合力。因此,要求Linux也提供这样的系统调用的呼声日益高涨。最终在2.5内核中,引入了一套系统调用用于设置和获取某个进程的CPU亲合力特性。

In this article, I look at the reasons for introducing a CPU affinity interface to Linux. I then cover how to use the interface in your programs. If you are not a programmer or if you have an existing program you are unable to modify, I cover a simple utility for changing the affinity of a given process using its PID. Finally, we look at the actual implementation of the system call.

 

       本文将探讨CPU亲合力被引入到Linux中的动机,以及在应用程序中如何使用亲合力接口。假如你不是一个程序员或者你拥有一个程序,但又不想修改它,那么我将运用一套简单的工具,通过PID改变给定进程的亲合力。最后,我们将探讨亲合力系统调用的具体实现原理。

Soft vs. Hard CPU Affinity

软亲合力 VS. 硬亲合力

There are two types of CPU affinity. The first, soft affinity, also called natural affinity, is the tendency of a scheduler to try to keep processes on the same CPU as long as possible. It is merely an attempt; if it is ever infeasible, the processes certainly will migrate to another processor. The new O(1) scheduler in 2.5 exhibits excellent natural affinity. On the opposite end, however, is the 2.4 scheduler, which has poor CPU affinity. This behavior results in the ping-pong effect. The scheduler bounces processes between multiple processors each time they are scheduled and rescheduled. Table 1 is an example of poor natural affinity; Table 2 shows what good natural affinity looks like.

  time 1 time 2 time 3 time 4
Process A CPU 0 CPU 1 CPU 0 CPU 1

Table 1. The Ping-Pong Effect

  time 1 time 2 time 3 time 4
Process A CPU 0 CPU 0 CPU 0 CPU 0

Table 2. Good Affinitiy

Hard affinity, on the other hand, is what a CPU affinity system call provides. It is a requirement, and processes must adhere to a specified hard affinity. If a processor is bound to CPU zero, for example, then it can run only on CPU zero.

Why One Needs CPU Affinity

Before we cover the new system calls, let's discuss why anyone would need such a feature. The first benefit of CPU affinity is optimizing cache performance. I said the O(1) scheduler tries hard to keep tasks on the same processor, and it does. But in some performance-critical situations--perhaps a large database or a highly threaded Java server--it makes sense to enforce the affinity as a hard requirement. Multiprocessing computers go through a lot of trouble to keep the processor caches valid. Data can be kept in only one processor's cache at a time. Otherwise, the processor's cache may grow out of sync, leading to the question, who has the data that is the most up-to-date copy of the main memory? Consequently, whenever a processor adds a line of data to its local cache, all the other processors in the system also caching it must invalidate that data. This invalidation is costly and unpleasant. But the real problem comes into play when processes bounce between processors: they constantly cause cache invalidations, and the data they want is never in the cache when they need it. Thus, cache miss rates grow very large. CPU affinity protects against this and improves cache performance.

A second benefit of CPU affinity is a corollary to the first. If multiple threads are accessing the same data, it might make sense to bind them all to the same processor. Doing so guarantees that the threads do not contend over data and cause cache misses. This does diminish the performance gained from multithreading on SMP. If the threads are inherently serialized, however, the improved cache hit rate may be worth it.

The third and final benefit is found in real-time or otherwise time-sensitive applications. In this approach, all the system processes are bound to a subset of the processors on the system. The specialized application then is bound to the remaining processors. Commonly, in a dual-processor system, the specialized application is bound to one processor, and all other processes are bound to the other. This ensures that the specialized application receives the full attention of the processor.

Getting the New System Calls

The system calls are new, so they are not available yet in all systems. You need at least kernel 2.5.8-pre3 and glibc 2.3.1; glibc 2.3.0 supports the system calls, but it has a bug. The system calls are not yet in 2.4, but patches are available at www.kernel.org/pub/linux/kernel/people/rml/cpu-affinity.

Many distribution kernels also support the new system calls. In particular, Red Hat 9 is shipping with both kernel and glibc support for the new calls. Real-time solutions, such as MontaVista Linux, also fully support the new interface.

Affinity Masks

On most systems, Linux included, the interface for setting CPU affinity uses a bitmask. A bitmask is a series of n bits, where each bit individually corresponds to the status of some other object. For example, CPU affinity (on 32-bit machines) is represented by a 32-bit bitmask. Each bit represents whether the given task is bound to the corresponding processor. Count the bits from right to left, bit 0 to bit 31 and, thus, processor zero to processor 31. For example:

11111111111111111111111111111111 = 4,294,967,295

is the default CPU affinity mask for all processes. Because all bits are set, the process can run on any processor. Conversely:

00000000000000000000000000000001 = 1

is much more restrictive. Only bit 0 is set, so the process may run only on processor zero. That is, this affinity mask binds a process to processor zero.

Get it? What do the next two masks equal in decimal? What is the result of using them as the affinity mask of a process?

10000000000000000000000000000000
00000000000000000000000000000011

The first is equal to 2,147,483,648 and, because bit 31 is set, binds the process to processor number 31. The second is equal to 3, and it binds the process in question to processor zero and processor one.

The Linux CPU affinity interface uses a bitmask like that shown above. Unfortunately, C does not support binary constants, so you always have to use the decimal or hexadecimal equivalent. You may get a compiler warning for very large decimal constants that set bit 31, but they will work.

Using the New System Calls

With the correct kernel and glibc in hand, using the system calls is easy:

#define _GNU_SOURCE
#include <sched.h>
long
sched_setaffinity(pid_t pid, unsigned int len, 
                  unsigned long *user_mask_ptr);
long
sched_getaffinity(pid_t pid, unsigned int len,
                  unsigned long *user_mask_ptr);

The first system call is used to set the affinity of a process, and the second system call retrieves it.

In either system call, the PID argument is the PID of the process whose mask you wish to set or retrieve. If the PID is set to zero, the PID of the current task is used.

The second argument is the length in bytes of the CPU affinity bitmask, currently four bytes (32 bits). This number is included in case the kernel ever changes the size of the CPU affinity mask and allows the system calls to be forward-compatible with any changes; breaking syscalls is bad form, after all. The third argument is a pointer to the bitmask itself.

Let us look at retrieving the CPU affinity of a task:

unsigned long mask;
unsigned int len = sizeof(mask);
if (sched_getaffinity(0, len, &mask) < 0) {
    perror("sched_getaffinity");
    return -1;
    }
printf("my affinity mask is: %08lx/n", mask);

As a convenience, the returned mask is binary ANDed against the mask of all processors in the system. Thus, processors in your system that are not on-line have corresponding bits that are not set. For example, a uniprocessor system always returns 1 for the above call (bit 0 is set and no others).

Setting the mask is equally easy:

unsigned long mask = 7; /* processors 0, 1, and 2 */
unsigned int len = sizeof(mask);
if (sched_setaffinity(0, len, &mask) < 0) {
    perror("sched_setaffinity");
}

This example binds the current process to the first three processors in the system.

You then can call sched_getaffinity() to ensure the change took effect. What does sched_getaffinity() return for the above setup if you have only two processors? What if you have only one? The system call fails unless at least one processor in the bitmask exists. Using a mask of zero always fails. Likewise, binding to processor seven if you do not have a processor seven will fail.

It is possible to retrieve the CPU affinity mask of any process on the system. You can set the affinity of only the processes you own, however. Of course, root can set any process' affinity.

I Want a Tool!

If you are not a programmer, or if you cannot modify the source for whatever reason, you still can bind processes. Listing 1 is the source code for a simple command-line utility to set the CPU affinity mask of any process, given its PID. As we discussed above, you must own the process or be root to do this.

Listing 1. bind

Usage is simple; once you learn the decimal equivalent of the CPU mask, you need:

usage: bind pid cpu_mask

As an example, assume we have a dual computer and want to bind our Quake process (with PID 1600) to processor two. We would enter the following:

bind 1600 2



Getting Really Crafty

In the previous example, we bound Quake to one of the two processors in our system. To ensure top-notch frame rates, we need to bind all the other processes on the system to the other processor. You can do this by hand or by writing a crafty script, but neither is efficient. Instead, make use of the fact that CPU affinity is inherited across a fork(). All of a process' children receive the same CPU affinity mask as their parent.

Then, all we need to do is have init bind itself to one processor. All other processes, by nature of init being the root of the process tree and thus the superparent of all processes, are then likewise bound to the one processor.

The cleanest way to do this type of bind is to hack this feature into init itself and pass in the desired CPU affinity mask using the kernel command line. We can accomplish our goal with a simpler solution, though, without having to modify and recompile init. Instead, we can edit the system startup script. On most systems this is /etc/rc.d/rc.sysinit or /etc/rc.sysinit, the first script run by init. Place the sample bind program in /bin, and add these lines to the start of rc.sysinit:

/bin/bind 1 1
/bin/bind $ 1

These lines bind init (whose PID is one) and the current process to processor zero. All future processes will fork from one of these two processes and thus inherit the CPU affinity mask. You then can bind your process (whether it be a real-time nuclear control system or Quake) to processor one. All processes will run on processor zero except our special process (and any children), which will run on processor one. This ensures that the entire processor is available for our special process.

Kernel Implementation of CPU Affinity

Long before Linus merged the CPU affinity system calls, the kernel supported and respected a CPU affinity mask. There was no interface by which user space could set the mask.

Each process' mask is stored in its task_struct as an unsigned long, cpus_allowed. The task_struct structure is called the process descriptor. It stores all the information about a process. The CPU affinity interface merely reads and writes cpus_allowed.

Whenever the kernel attempts to migrate a process from one processor to another, it first checks to see if the destination processor's bit is set in cpus_allowed. If the bit is not set, the kernel does not migrate the process. Further, whenever the CPU affinity mask is changed, if the process is no longer on an allowed processor it is migrated to one that is allowed. This ensures the process begins on a legal processor and can migrate only to a legal processor. Of course, if it is bound to only a single processor, it does not migrate anywhere.

Conclusion

The CPU affinity interface introduced in 2.5 and back-ported elsewhere provides a simple yet powerful mechanism for controlling which processes are scheduled onto which processors. Users with more than one processor may find the system calls useful in squeezing another drop of performance out of their systems or for ensuring that processor time is available for even the most demanding real-time task. Of course, users with only one processor need not feel left out. They also can use the system calls, but they aren't going to be too useful.

用Packstack 安装 Openstack,无法启动实例,cpu affinity is not supported

用Packstack 安装 Openstack N版本,无法启动实例 ,nova日志报错:Couldn't obtain the vpu count from domain. Requested op...

Java线程CPU亲和性工具

Thread Affinity为什么需要线程的CPU亲和性应用通过多线程的方式执行,多数情况下线程能够被合理的调度。但在某些情况下某个重要的线程被暂停,而时间片被分配给了一个无关重要的线程。当一个线程...
  • liu251
  • liu251
  • 2015年05月03日 22:30
  • 4701

CPU Affinity (CPU亲合力)

在Linux下进行多核编程的时候,通常要设置CPU的亲和力(Affinity)。下面通过讲解sched_setaffinity()和sched_getaffinity()函数以及CPU_ZERO()、...

关于CPU affinity的几篇文章

cpu绑定和cpu亲和性 原文地址:http://blog.csdn.net/joker0910/article/details/7484371 将进程/线程与cpu绑定,最直观的好处就是提高...

设备中断绑定到特定CPU(SMP IRQ AFFINITY)

因为单机运行多个实例,必须对网络进行优化,我们通过多个的IP的方式,将多个MySQL实例绑定在不同的网卡上,从而提高整体的网络能力。还有一种更高级的做法是,将不同网卡的中断与CPU绑定,这样可以大幅度...

多核优化,使用linux affinity 将进程,线程,中断指定到对应的cpu运行,用ftrace查看消耗时间

cpu的affinity简介 使用cpu的affinity机制可以将对应的进程,线程,以及中断指定代对应的cpu上运行,如果合理配置,减少某个cpu负担,提高其他cpu的使用率,从而到达提高系统...
  • welljrj
  • welljrj
  • 2017年06月05日 10:26
  • 230

python 绑定进程在某个cpu上执行 affinity

因为最近在看multiprocessing 所以想看看多进程在多cpu机器上的利用率.然后早上写了个脚本在i5(4核),起了4个进程,发现只有一颗cpu跑的比较高,然后觉得不是很舒服.  开始还...

【Linux】 CPU亲和性(affinity)及与亲和性有关的两个函数 sched_setaffinity()和 sched_getaffinity()

转自: http://www.ibm.com/developerworks/cn/linux/l-affinity.html 简单地说,CPU 亲和性(affinity) 就是进程要在某个给定的 C...

【Linux】 CPU亲和性(affinity)及与亲和性有关的两个函数 sched_setaffinity()和 sched_getaffinity()

简单地说,CPU 亲和性(affinity) 就是进程要在某个给定的 CPU 上尽量长时间地运行而不被迁移到其他处理器的倾向性。Linux 内核进程调度器天生就具有被称为 软 CPU 亲和性(affi...
  • pugu12
  • pugu12
  • 2015年07月27日 17:13
  • 250

【Linux】 CPU亲和性(affinity)及与亲和性有关的两个函数 sched_setaffinity()和 sched_getaffinity()

【Linux】 CPU亲和性(affinity)及与亲和性有关的两个函数 sched_setaffinity()和 sched_getaffinity() 转自: http://www.ibm....
内容举报
返回顶部
收藏助手
不良信息举报
您举报文章:CPU亲合力(CPU Affinity)
举报原因:
原因补充:

(最多只允许输入30个字)