sched_setaffinity如何在Linux内核中工作

Abstract
Sometimes, we may want to migrate one process/thread to one specific CPU for some specific purpose. In the Unix/Linux systems, you may choose sched_setaffinity to finish this job. This article will help you to understand how sched_setaffinity (or other APIs like pthread_setaffinity_np in user-space) works internal Linux kernel.

抽象
有时，出于某些特定目的，我们可能希望将一个进程/线程迁移到一个特定的CPU。 在Unix / Linux 系统中 ，您可以选择sched_setaffinity完成此工作。 本文将帮助您了解sched_setaffinity（或用户空间中的其他API，如pthread_setaffinity_np）如何在内部Linux内核中工作。

Details

细节

SYSCALL_DEFINE3(sched_setaffinity, pid_t, pid, unsigned int, len, unsigned long __user *, user_mask_ptr)
-- sched_setaffinity(pid_t pid, const struct cpumask *in_mask)
--- __set_cpus_allowed_ptr(struct task_struct *p, const struct cpumask *new_mask, bool check)
---- stop_one_cpu(unsigned int cpu, cpu_stop_fn_t fn, void *arg)
----- migration_cpu_stop(void *data)
------ __migrate_task(struct rq *rq, struct task_struct *p, int dest_cpu)
------- move_queued_task(struct rq *rq, struct task_struct *p, int new_cpu)
-------- enqueue_task(struct rq *rq, struct task_struct *p, int flags)
--------- returns the new run queue of destination CPU

Above character steps give a workflow of how sched_setaffinity works (how it migrates one process/thread from the run queue of source CPU to the run queue of destination CPU). Let’s analyze them in details (Note that this article is discussing about how sched_setaffinity works inside the Linux Kernel 4.7.4 and other versions may have a little differences).

上面的字符步骤提供了sched_setaffinity如何工作的工作流程（它如何将一个进程/线程从源CPU的运行队列迁移到目标CPU的运行队列）。 让我们对其进行详细分析（请注意，本文正在讨论sched_setaffinity在Linux Kernel 4.7.4内的工作方式，其他版本可能会有一些不同）。

Obviously, we can find that sched_setaffinity is a system call in Linux System. Sched_setaffinity gets the process ID we want to migrate and the destination CPU mask bits we want to migrate to. Then, it calls __set_cpus_allowed_ptr to do some checking works before migration. In __set_cpus_allowed_ptr, it changes the affinity of the process/thread and then calls the most important function "stop_one_cpu" to do the migration. However, before this, it checks whether the process/thread is running (or is going to run, TASK_WAKING). If true, the stop_one_cpu triggers. If not true, check whether the process/thread is on the run queue of source CPU, if true, the CPU which executes sched_setaffinity just migrates the process/thread from the run queue of source CPU to the run queue of destination CPU directly.

显然，我们可以发现sched_setaffinity是Linux系统中的系统调用。 Sched_setaffinity获取我们要迁移的进程ID和我们要迁移到的目标CPU掩码位。 然后，它在迁移之前调用__set_cpus_allowed_ptr进行一些检查。 在__set_cpus_allowed_ptr ，它更改进程/线程的亲和力，然后调用最重要的函数“ stop_one_cpu”进行迁移。 但是，在此之前，它将检查进程/线程是否正在运行（或将要运行，TASK_WAKING）。 如果为true，则stop_one_cpu触发。 如果不是，则检查进程/线程是否在源CPU的运行队列中；如果为true，则执行sched_setaffinity的CPU会将进程/线程从源CPU的运行队列直接迁移到目标CPU的运行队列。

In stop_one_cpu, it invokes migration_cpu_stop on the CPU of the process/thread we want to migrate with high priority. In migration_cpu_stop, it calls __migrate_task to test whether the affinity of the process/thread has been changed correctly previously and then, it calls move_queued_task to move the process/thread from the old run queue. At last, in move_queued_task function, it calls enqueue_task to move the process/thread to the new CPU’s run queue. Up to here, stop_one_cpu function returns and the migration is done.

在stop_one_cpu中，它在我们要高优先级迁移的进程/线程的CPU上调用migration_cpu_stop。 在migration_cpu_stop中，它调用__migrate_task来测试进程/线程的亲和性先前是否已正确更改，然后，它调用move_queued_task将进程/线程从旧的运行队列中移出。 最后，在move_queued_task函数中，它调用enqueue_task将进程/线程移至新CPU的运行队列。 到此为止，stop_one_cpu函数将返回并完成迁移。

Conclusion
In a word, the sched_setaffinity does following jobs internal Linux Kernel.
1, Check the status of migrated process.
2, If it is in the running/task_waking status, let the source CPU of this process/thread to do migration.
3, If it is in the run queue of source CPU, let the CPU (executes sched_setaffinity system call) to do migration.
4, If it is in the waiting queue, only change the affinity of the source CPU.
5, The migration is to move the process/thread from old run queue to the new run queue of destionation CPU.

结论
简而言之，sched_setaffinity会执行以下内部Linux内核作业。
1，检查迁移过程的状态。
2，如果它处于运行中/任务唤醒状态，则让该进程/线程的源CPU进行迁移。
3，如果它在源CPU的运行队列中，请让CPU（执行sched_setaffinity系统调用）进行迁移。
4，如果它在等待队列中，则仅更改源CPU的亲和力。
5，迁移是将进程/线程从目标CPU的旧运行队列移动到新运行队列。

References:
1, Linux Kernel source codes – https://www.kernel.org/
2, http://lxr.free-electrons.com/source/kernel/sched/core.c?v=4.7#L4706

参考文献：
1，Linux Kernel源代码– https://www.kernel.org/
2， http://lxr.free-electrons.com/source/kernel/sched/core.c?v=4.7#L4706

翻译自: https://www.systutorials.com/sched_setaffinity-works-inside-of-linux-kernel/