linux 线程迁移_迁移线程如何在Linux内核内部工作

linux 线程迁移

Abstract
In computer systems, resources have to be balanced so that the performance will be better based on the same hardware. In Linux Kernel system, we will see some migration kernel threads running as daemons to do this kind of jobs as follows. In this article, we will discuss how Linux Kernel balances its hardware/software resources (NOTE: this article is based on Linux Kernel 4.7 and other versions may have some differences).

抽象
在计算机系统中 ，必须平衡资源，以便基于相同的硬件可以提高性能。 在Linux内核系统中，我们将看到一些作为后台程序运行的迁移内核线程可以完成以下工作。 在本文中，我们将讨论Linux Kernel如何平衡其硬件/软件资源（注意：本文基于Linux Kernel 4.7，其他版本可能有所不同）。

$ ps aux | grep migration
root         9  0.0  0.0      0     0 ?        S    Nov17   0:00 [migration/0]
root        15  0.0  0.0      0     0 ?        S    Nov17   0:00 [migration/1]
root        21  0.0  0.0      0     0 ?        S    Nov17   0:00 [migration/2]
root        27  0.0  0.0      0     0 ?        S    Nov17   0:00 [migration/3]
root        33  0.0  0.0      0     0 ?        S    Nov17   0:00 [migration/4]

Details
In Linux Kernel, each CPU has one migration daemon to do migration jobs. We will not discuss the algorithms/mechanisms about how Linux Kernel balance resources and we will only discuss how Linux Kernel implement migration thread and how to migrate threads among CPUs. In How set_affinity works inside of Linux Kernel, we have discussed when the thread is running/TASK_WAKING, it will trigger stop_one_cpu to do migration jobs but we didn’t talk about who/when will do these kind of jobs inside of Linux Kernel. Actually, when you look inside of stop_one_cpu, you will find some clues as follows.

细节
在Linux内核中，每个CPU都有一个迁移守护程序来执行迁移作业。 我们将不会讨论有关Linux Kernel如何平衡资源的算法/机制，而只会讨论Linux Kernel如何实现迁移线程以及如何在CPU之间迁移线程。 在set_affinity在Linux内核内部的工作方式中 ，我们讨论了线程何时运行/ TASK_WAKING，它将触发stop_one_cpu进行迁移作业，但是我们没有讨论谁 /何时在Linux内核中进行此类作业。 实际上，当您查看stop_one_cpu的内部时，会发现一些线索，如下所示。

stop_one_cpu
--cpu_stop_queue_work
---__cpu_stop_queue_work
----list_add_tail and wake_up_process

In stop_one_cpu, the migration job will be packed as a work and call cpu_stop_queue_work to get the stopper thread of the CPU. And then, call __cpu_stop_queue_work to put this work into the work queue of stopper thread upon this CPU. At last, wake up the stopper thread to do migration. The stopper thread is named migration thread in the Linux Kernel (see following source codes).

在stop_one_cpu中，迁移作业将打包为工作，并调用cpu_stop_queue_work以获取CPU的停止线程。 然后，调用__cpu_stop_queue_work将此工作放入此CPU上的停止线程的工作队列中。 最后，唤醒停止线程进行迁移。 停止器线程在Linux内核中称为迁移线程（请参阅以下源代码）。

504 static struct smp_hotplug_thread cpu_stop_threads = {
505         .store                  = &cpu_stopper.thread,
506         .thread_should_run      = cpu_stop_should_run,
507         .thread_fn              = cpu_stopper_thread,
508         .thread_comm            = "migration/%u",
509         .create                 = cpu_stop_create,
510         .park                   = cpu_stop_park,
511         .selfparking            = true,
512 };
513 
514 static int __init cpu_stop_init(void)
515 {
516         unsigned int cpu;
517 
518         for_each_possible_cpu(cpu) {
519                 struct cpu_stopper *stopper = &per_cpu(cpu_stopper, cpu);
520 
521                 spin_lock_init(&stopper->lock);
522                 INIT_LIST_HEAD(&stopper->works);
523         }
524 
525         BUG_ON(smpboot_register_percpu_thread(&cpu_stop_threads));
526         stop_machine_unpark(raw_smp_processor_id());
527         stop_machine_initialized = true;
528         return 0;
529 }
530 early_initcall(cpu_stop_init);

We find that Linux Kernel migration thread upon each CPU is initialized by early_initcall which is done when the kernel is started as follows.

我们发现每个CPU上的Linux Kernel迁移线程都是通过early_initcall初始化的，这是在启动内核时完成的，如下所示。

start_kernel
--rest_init
---kernel_init
----kernel_init_freeable
-----do_pre_smp_initcalls
------do_one_initcall

Linux Kernel migration thread daemon is one of the early_initcall to be initialized. There are lots of init calls for Linux Kernel to do, which are like following.

Linux Kernel迁移线程守护程序是要初始化的early_initcall之一。 Linux内核有许多初始化调用，如下所示。

814 static char *initcall_level_names[] __initdata = {
815         "early",
816         "core",
817         "postcore",
818         "arch",
819         "subsys",
820         "fs",
821         "device",
822         "late",
823 };

Up to here, I think everything is clear, migration daemon thread in Linux Kernel will be initialized when Kernel is started. And it will be woken up to do migration jobs once there is a migration work to do (NOTE: Here, I only consider the case for sched_setaffinity and, maybe, there are some other ways in Linux Kernel to wake up migration thread daemon). In addition, for other kernel thread daemon (like softirqd, kworker, etc), they have the similar way (like migration thread daemon) created by Linux Kernel.

到目前为止，我认为一切都已经清楚了，启动内核后，Linux内核中的迁移守护进程线程将被初始化。 一旦完成迁移工作，它就会被唤醒以进行迁移工作（注意：在这里，我只考虑sched_setaffinity的情况，也许Linux内核中还有其他方法可以唤醒迁移线程守护程序）。 此外，对于其他内核线程守护程序（例如softirqd，kworker等），它们具有由Linux Kernel创建的类似方式（例如迁移线程守护程序）。

Conclusion
In Linux Kernel, each CPU will have one migration thread daemon to do resource balance jobs. If we call sched_setaffinity system call to migrate one thread from source CPU to destination CPU and if the thread is running or at the TASK_WAKING status, the migration jobs will be packed into the work queue of source CPU’s migration thread, then, the migration thread will be waken up to finish migration job.

结论
在Linux内核中，每个CPU都有一个迁移线程守护程序来执行资源平衡作业。 如果我们调用sched_setaffinity系统调用将一个线程从源CPU迁移到目标CPU，并且该线程正在运行或处于TASK_WAKING状态，则迁移作业将打包到源CPU的迁移线程的工作队列中，然后，迁移线程将被唤醒以完成迁移工作。

References
1, Linux Kernel Source Codes – https://www.kernel.org/
2, Linux Cross Reference – http://lxr.free-electrons.com/

参考资料
1，Linux内核源代码– https://www.kernel.org/
2，Linux交叉参考-http://lxr.free-electrons.com/

翻译自: https://www.systutorials.com/migration-thread-works-inside-linux-kernel/

linux 线程迁移