RTlinux ubuntu 移植

原文：https://blog.csdn.net/21cnbao/article/details/8038279

Vanilla kernel的问题

Linux kernel在spinlock、irq上下文方面无法抢占，因此高优先级任务被唤醒到得以执行的时间并不能完全确定。同时，Linux kernel本身也不处理优先级反转。RT-Preempt Patch是在Linux社区kernel的基础上，加上相关的补丁，以使得Linux满足硬实时的需求。本文描述了该patch在PC上的实践。我们的测试环境为Ubuntu 10.10，默认情况下使用Ubuntu 10.10自带的kernel：

1. barry@barry-VirtualBox:/lib/modules$ uname -a
2. 2.6.35-32-generic #67-Ubuntu SMP Mon Mar 5 19:35:26 UTC 2012 i686 GNU/Linux

在Ubuntu 10.10，apt-get install rt-tests安装rt测试工具集，运行其中的cyclictest测试工具，默认创建5个SCHED_FIFO策略的realtime线程，优先级76-80，运行周期是1000,1500,2000,2500,3000微秒：

1. barry@barry-VirtualBox:~/development/panda/android$ sudo cyclictest -p 80 -t5 -n
2. [sudo] password for barry:
3. policy: fifo: loadavg: 9.22 8.57 6.75 11/374 21385
4.  
5. T: 0 (20606) P:80 I:1000 C: 18973 Min: 26 Act: 76 Avg: 428 Max: 12637
6. T: 1 (20607) P:79 I:1500 C: 12648 Min: 31 Act: 68 Avg: 447 Max: 10320
7. T: 2 (20608) P:78 I:2000 C: 9494 Min: 28 Act: 151 Avg: 383 Max: 9481
8. T: 3 (20609) P:77 I:2500 C: 7589 Min: 29 Act: 889 Avg: 393 Max: 12670
9. T: 4 (20610) P:76 I:3000 C: 6325 Min: 37 Act: 167 Avg: 553 Max: 13673

由此可见在标准Linux内，rt线程投入运行的jitter非常不稳定，最小值在26-37微秒，平均值为68-889微秒，而最大值则分布在9481-13673微秒之间。

我们还是运行这个测试，但是在运行这个测试的过程中引入更多干扰，如mount /dev/sdb1 ~/development，则结果变为：

1. barry@barry-VirtualBox:~$ sudo cyclictest -p 80 -t5 -n
2. policy: fifo: loadavg: 0.14 0.29 0.13 2/308 1908
3.  
4. T: 0 ( 1874) P:80 I:1000 C: 28521 Min: 0 Act: 440 Avg: 2095 Max: 331482
5. T: 1 ( 1875) P:79 I:1500 C: 19014 Min: 2 Act: 988 Avg: 2099 Max: 330503
6. T: 2 ( 1876) P:78 I:2000 C: 14261 Min: 7 Act: 534 Avg: 2096 Max: 329989
7. T: 3 ( 1877) P:77 I:2500 C: 11409 Min: 4 Act: 554 Avg: 2073 Max: 328490
8. T: 4 ( 1878) P:76 I:3000 C: 9507 Min: 12 Act: 100 Avg: 2081 Max: 328991

 

mount过程中引入的irq、softirq和spinlock导致最大jitter明显地加大甚至达到了331482us，充分显示出了标准Linux内核中RT线程投入运行时间的不可预期性(硬实时要求意味着可预期)。

如果我们编译一份kernel，选择的是“Voluntary Kernel Preemption (Desktop)“，这类似于2.4不支持kernel抢占的情况，我们运行同样的case，时间的不确定性大地几乎让我们无法接受：

1. barry@barry-VirtualBox:~$ sudo /usr/local/bin/cyclictest -p 80 -t5 -n
2. # /dev/cpu_dma_latency set to 0us
3. policy: fifo: loadavg: 0.23 0.30 0.15 3/247 5086
4.  
5. T: 0 ( 5082) P:80 I:1000 C: 5637 Min: 60 Act:15108679 Avg:11195196 Max:15108679
6. T: 1 ( 5083) P:80 I:1500 C: 5723 Min: 48 Act:12364955 Avg:6389691 Max:12364955
7. T: 2 ( 5084) P:80 I:2000 C: 4821 Min: 32 Act:11119979 Avg:8061814 Max:11661123
8. T: 3 ( 5085) P:80 I:2500 C: 3909 Min: 27 Act:11176854 Avg:4563549 Max:11176854
9. T: 4 ( 5086) P:80 I:3000 C: 3598 Min: 37 Act:9951432 Avg:8761137 Max:116026155

RT-Preempt Patch使能

RT-Preempt Patch对Linux kernel的主要改造包括：

• Making in-kernel locking-primitives (using spinlocks) preemptible though reimplementation with rtmutexes:
• Critical sections protected by i.e. spinlock_t and rwlock_t are now preemptible. The creation of non-preemptible sections (in kernel) is still possible with raw_spinlock_t (same APIs like spinlock_t)
• Implementing priority inheritance for in-kernel spinlocks and semaphores. For more information on priority inversion and priority inheritance please consultIntroduction to Priority Inversion
• Converting interrupt handlers into preemptible kernel threads: The RT-Preempt patch treats soft interrupt handlers in kernel thread context, which is represented by a task_struct like a common userspace process. However it is also possible to register an IRQ in kernel context.
• Converting the old Linux timer API into separate infrastructures for high resolution kernel timers plus one for timeouts, leading to userspace POSIX timers with high resolution.

在本试验中，我们取的带RT-Preempt Patch的kernel tree是git://git.kernel.org/pub/scm/linux/kernel/git/rt/linux-stable- rt.git，使用其v3.4-rt-rebase branch，编译kernel时选中了"Fully Preemptible Kernel"抢占模型:

───────────────────────── Preemption Model ─────────────────────────┐

 

│ │          ( ) No Forced Preemption (Server)                 

│ │          ( ) Voluntary Kernel Preemption (Desktop)       

│ │          ( ) Preemptible Kernel (Low-Latency Desktop)    

│ │          ( ) Preemptible Kernel (Basic RT)                

│ │          (X) Fully Preemptible Kernel (RT)      

 

另外，kernel中需支持tickless和高精度timer：

┌───────────────────Processor type and features ─────────────────────────┐

│ │                                      [*] Tickless System (Dynamic Ticks)                                                               

│ │                                      [*] High Resolution Timer Support       

 

make modules_install、make install、mkintramfs后，我们得到一个可以在Ubuntu中启动的RT kernel。具体编译方法可详见http://www.linuxidc.com/Linux/2012-01/50749.htm，根据该文修改版本号等信息即可，我们运行的命令包括：

安装模块

1. barry@barry-VirtualBox:~/development/linux-2.6$ sudo make modules_install
2. ....
3.   INSTALL /lib/firmware/whiteheat_loader.fw
4.   INSTALL /lib/firmware/whiteheat.fw
5.   INSTALL /lib/firmware/keyspan_pda/keyspan_pda.fw
6.   INSTALL /lib/firmware/keyspan_pda/xircom_pgs.fw
7.   INSTALL /lib/firmware/cpia2/stv0672_vp4.bin
8.   INSTALL /lib/firmware/yam/1200.bin
9.   INSTALL /lib/firmware/yam/9600.bin
10.   DEPMOD  3.4.11-rt19

安装kernel

1. barry@barry-VirtualBox:~/development/linux-2.6$ sudo make install
2. sh /home/barry/development/linux-2.6/arch/x86/boot/install.sh 3.4.11-rt19 arch/x86/boot/bzImage \
3. System.map "/boot"

制作initrd

barry@barry-VirtualBox:~/development/linux-2.6$ sudo mkinitramfs 3.4.11-rt19 -o /boot/initrd.img-3.4.11-rt19

修改grub配置

在grub.conf中增加新的启动entry，仿照现有的menuentry，增加一个新的，把其中的相关版本号都变更为3.4.11-rt19，我们的修改如下：

1. menuentry 'Ubuntu, with Linux 3.4.11-rt19' --class ubuntu --class gnu-linux --class gnu --class os {
2. recordfail
3. insmod part_msdos
4. insmod ext2
5. set root='(hd0,msdos1)'
6. search --no-floppy --fs-uuid --set a0db5cf0-6ce3-404f-9808-88ce18f0177a
7. linux /boot/vmlinuz-3.4.11-rt19 root=UUID=a0db5cf0-6ce3-404f-9808-88ce18f0177a ro quiet splash
8. initrd /boot/initrd.img-3.4.11-rt19
9. }

开机时选择3.4.11-rt19启动：

RT-Preempt Patch试用

运行同样的测试cyclictest benchmark工具，结果迥异：

1. barry@barry-VirtualBox:~$ sudo cyclictest -p 80 -t5 -n
2. WARNING: Most functions require kernel 2.6
3. policy: fifo: loadavg: 0.71 0.42 0.17 1/289 1926
4.  
5. T: 0 ( 1921) P:80 I:1000 C: 7294 Min: 7 Act: 89 Avg: 197 Max: 3177
6. T: 1 ( 1922) P:79 I:1500 C: 4863 Min: 10 Act: 85 Avg: 186 Max: 2681
7. T: 2 ( 1923) P:78 I:2000 C: 3647 Min: 15 Act: 93 Avg: 160 Max: 2504
8. T: 3 ( 1924) P:77 I:2500 C: 2918 Min: 23 Act: 67 Avg: 171 Max: 2114
9. T: 4 ( 1925) P:76 I:3000 C: 2432 Min: 19 Act: 134 Avg: 339 Max: 3129

我们还是运行这个测试，但是在运行这个测试的过程中引入更多干扰，如mount /dev/sdb1 ~/development，则结果变为：

1. barry@barry-VirtualBox:~$ sudo cyclictest -p 80 -t5 -n
2. # /dev/cpu_dma_latency set to 0us
3. policy: fifo: loadavg: 0.11 0.12 0.13 1/263 2860
4.  
5. T: 0 ( 2843) P:80 I:1000 C: 28135 Min: 5 Act: 198 Avg: 200 Max: 7387
6. T: 1 ( 2844) P:80 I:1500 C: 18756 Min: 22 Act: 169 Avg: 188 Max: 6875
7. T: 2 ( 2845) P:80 I:2000 C: 14067 Min: 7 Act: 91 Avg: 149 Max: 7288
8. T: 3 ( 2846) P:80 I:2500 C: 11254 Min: 19 Act: 131 Avg: 155 Max: 6287
9. T: 4 ( 2847) P:80 I:3000 C: 9378 Min: 25 Act: 58 Avg: 172 Max: 6121

时间在可预期的范围内，没有出现标准kernel里面jitter达到331482的情况。需要说明的是，这个jitter大到超过了我们的预期，达到了10ms量级，相信是受到了我们的测试都是在Virtualbox虚拟机进行的影响。按照其他文档显示，这个jitter应该在数十us左右。

我们在这个kernel里面运行ps aux命令，可以看出线程化了的irq：

1. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
2. root 1 0.8 0.1 2880 1788 ? Ss 18:39 0:03 init
3. root 2 0.0 0.0 0 0 ? S 18:39 0:00 kthreadd
4. ...
5.  
6. root 45 0.0 0.0 0 0 ? S 18:39 0:00 irq/14-ata_piix
7. root 46 0.0 0.0 0 0 ? S 18:39 0:00 irq/15-ata_piix
8. root 50 0.0 0.0 0 0 ? S 18:39 0:00 irq/19-ehci_hcd
9. root 51 0.0 0.0 0 0 ? S 18:39 0:00 irq/22-ohci_hcd
10. root 55 0.0 0.0 0 0 ? S 18:39 0:00 irq/12-i8042
11. root 56 0.0 0.0 0 0 ? S 18:39 0:00 irq/1-i8042
12. root 57 0.0 0.0 0 0 ? S 18:39 0:00 irq/8-rtc0
13. root 863 0.0 0.0 0 0 ? S 18:39 0:00 irq/19-eth0
14. root 864 0.0 0.0 0 0 ? S 18:39 0:00 irq/16-eth1
15. root 1002 0.5 0.0 0 0 ? S 18:39 0:01 irq/21-snd_inte
16. ...

在其中编写一个RT 线程的应用程序，通常需要如下步骤：

• Setting a real time scheduling policy and priority.
• Locking memory so that page faults caused by virtual memory will not undermine deterministic behavior
• Pre-faulting the stack, so that a future stack fault will not undermine deterministic behavior

例子test_rt.c，其中的mlockall是为了防止进程的虚拟地址空间对应的物理页面被swap出去，而stack_prefault()则故意提前导致stack往下增长8KB，因此其后的函数调用和局部变量的使用将不再导致栈增长（依赖于page fault和内存申请）：

1. #include <stdlib.h>
2. #include <stdio.h>
3. #include <time.h>
4. #include <sched.h>
5. #include <sys/mman.h>
6. #include <string.h>
7.  
8. #define MY_PRIORITY (49) /* we use 49 as the PRREMPT_RT use 50
9. as the priority of kernel tasklets
10. and interrupt handler by default */
11.  
12. #define MAX_SAFE_STACK (8*1024) /* The maximum stack size which is
13. guaranteed safe to access without
14. faulting */
15.  
16. #define NSEC_PER_SEC (1000000000) /* The number of nsecs per sec. */
17.  
18. void stack_prefault(void) {
19.  
20. unsigned char dummy[MAX_SAFE_STACK];
21.  
22. memset(dummy, 0, MAX_SAFE_STACK);
23. return;
24. }
25.  
26. int main(int argc, char* argv[])
27. {
28. struct timespec t;
29. struct sched_param param;
30. int interval = 50000; /* 50us*/
31.  
32. /* Declare ourself as a real time task */
33.  
34. param.sched_priority = MY_PRIORITY;
35. if(sched_setscheduler(0, SCHED_FIFO, ¶m) == -1) {
36. perror("sched_setscheduler failed");
37. exit(-1);
38. }
39.  
40. /* Lock memory */
41.  
42. if(mlockall(MCL_CURRENT|MCL_FUTURE) == -1) {
43. perror("mlockall failed");
44. exit(-2);
45. }
46.  
47. /* Pre-fault our stack */
48.  
49. stack_prefault();
50.  
51. clock_gettime(CLOCK_MONOTONIC ,&t);
52. /* start after one second */
53. t.tv_sec++;
54.  
55. while(1) {
56. /* wait until next shot */
57. clock_nanosleep(CLOCK_MONOTONIC, TIMER_ABSTIME, &t, NULL);
58.  
59. /* do the stuff */
60.  
61. /* calculate next shot */
62. t.tv_nsec += interval;
63.  
64. while (t.tv_nsec >= NSEC_PER_SEC) {
65. t.tv_nsec -= NSEC_PER_SEC;
66. t.tv_sec++;
67. }
68. }
69. }

编译之：gcc -o test_rt test_rt.c -lrt。本节就到这里，后续我们会有一系列博文来描述RT-Preempt Patch对kernel的主要改动，以及其工作原理。