编译 linux kernel + xenomai 2/3(在orocos中需要注意资源配置)

尊重原作，从github拷贝的，以免翻墙或者找不到。
原作链接：https://github.com/kuka-isir/rtt_lwr/blob/rtt_lwr-2.0/docs/source/rtpc/xenomai.rst

Xenomai 2.6.5 on Ubuntu 14.04/16.04

Nvidia Drivers are NOT supported (creates a lot of interruptions that breaks the real-time).
Please consider removing the dedicated graphic card and use the integrated graphics (Intel HD graphics).

Download Xenomai 2.6.5

wget http://xenomai.org/downloads/xenomai/stable/xenomai-2.6.5.tar.bz2
tar xfvj xenomai-2.6.5.tar.bz2

Download Linux kernel 3.18.20

wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.18.20.tar.gz
tar xfv linux-3.18.20.tar.gz

Configuration

Prepare the kernel

sudo apt install kernel-package

Patch the Linux kernel with Xenomai ipipe patch

cd linux-3.18.20
../xenomai-2.6.5/scripts/prepare-kernel.sh

Press Enter to use the default options.

Configure the kernel

Now it’s time to configure :

Gui version :

make xconfig

Or without gui :

sudo apt install libncurses5-dev
make menuconfig

Some guidelines to configure the linux kernel:

Recommended options:

    * General setup
      --> Local version - append to kernel release: -xenomai-2.6.5
      --> Timers subsystem
          --> High Resolution Timer Support (Enable)
    * Real-time sub-system
      --> Xenomai (Enable)
      --> Nucleus (Enable)
    * Power management and ACPI options
      --> Run-time PM core functionality (Disable)
      --> ACPI (Advanced Configuration and Power Interface) Support
          --> Processor (Disable)
      --> CPU Frequency scaling
          --> CPU Frequency scaling (Disable)
      --> CPU idle
          --> CPU idle PM support (Disable)
    * Pocessor type and features
      --> Processor family
          --> Core 2/newer Xeon (if \"cat /proc/cpuinfo | grep family\" returns 6, set as Generic otherwise)
    * Power management and ACPI options
      --> Memory power savings
          --> Intel chipset idle memory power saving driver

warning:

For OROCOS, we need to increase the amount of ressources available for Xenomai tasks, otherwise we might hit the limits quickly as we add multiples components/ports etc. http://www.orocos.org/forum/orocos/orocos-users/orocos-limits-under-xenomai

* Real-time sub-system
  --> Number of registry slots
      --> 4096
  --> Size of the system heap
      --> 2048 Kb
  --> Size of the private stack pool
      --> 1024 Kb
  --> Size of private semaphores heap
      --> 48 Kb
  --> Size of global semaphores heap
      --> 48 Kb

Save the config and close the gui.

Compile the kernel (通过 make-kpkg 工具生成 .deb 安装包)

Now it’s time to compile.

CONCURRENCY_LEVEL=$(nproc) make-kpkg --rootcmd fakeroot --initrd kernel_image kernel_headers

Take a coffee and come back in 20min.
如果没有安装 make-kpkg 工具，这里也可以通过如下命令编译安装新的kernel:

make
sudo make modules_install  // 必须 sudo 运行，将模块安装到 /lib/`uname -r`/ 目录下
sudo make install  // 这一步会将编译好的内核复制到 /boot 目录下
sudo update-grub  // 更新grub后重启即可看到新的内核了
// 这样也就不需要下面的 Install the kernel 步骤了

Install the kernel

cd ..
sudo dpkg -i linux-headers-3.18.20-xenomai-2.6.5_3.18.20-xenomai-2.6.5-10.00.Custom_amd64.deb linux-image-3.18.20-xenomai-2.6.5_3.18.20-xenomai-2.6.5-10.00.Custom_amd64.deb

Allow non-root users

sudo addgroup xenomai --gid 1234
sudo addgroup root xenomai
sudo usermod -a -G xenomai $USER

tip:

If the addgroup command fails (ex: GID ``xenomai`` is already in use), change it to a different random value, and report it in the next section.

Configure GRUB

Edit the grub config :

sudo nano /etc/default/grub

GRUB_DEFAULT=saved
GRUB_SAVEDEFAULT=true
#GRUB_HIDDEN_TIMEOUT=0
#GRUB_HIDDEN_TIMEOUT_QUIET=true
GRUB_TIMEOUT=5
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash xeno_nucleus.xenomai_gid=1234 xenomai.allowed_group=1234"
GRUB_CMDLINE_LINUX=""

note:

Please note the xenomai group (here 1234) should match what you set above (allow non-root users).

tip:

noapic option might be added if the screen goes black at startup and you can’t boot.

If you have an Intel HD Graphics integrated GPU (any type) :

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash i915.enable_rc6=0 i915.powersave=0 noapic xeno_nucleus.xenomai_gid=1234 xenomai.allowed_group=1234"
# This removes powersavings from the graphics, that creates disturbing interruptions.

If you have an Intel Skylake (2015 processors), you need to add nosmap to fix the latency hang (https://xenomai.org/pipermail/xenomai/2016-October/036787.html) :

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash i915.enable_rc6=0 i915.powersave=0 xeno_nucleus.xenomai_gid=1234 nosmap"

Update GRUB and reboot

sudo update-grub
sudo reboot

Install Xenomai libraries

cd xenomai-2.6.5/
./configure
make -j$(nproc)
sudo make install  // 默认安装到 /usr/xenomai 下

Update your bashrc

echo '
#### Xenomai
export XENOMAI_ROOT_DIR=/usr/xenomai
export XENOMAI_PATH=/usr/xenomai
export PATH=$PATH:$XENOMAI_PATH/bin
export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:$XENOMAI_PATH/lib/pkgconfig
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$XENOMAI_PATH/lib
export OROCOS_TARGET=xenomai
' >> ~/.bashrc

Test your installation，运行下述命令：

xeno latency  // 如果出现权限问题，说明添加用户组 xenomai 失败了，重新添加组和用户即可

This loop will allow you to monitor a xenomai latency. Here’s the output for a i7 4Ghz :

== Sampling period: 100 us
== Test mode: periodic user-mode task
== All results in microseconds
warming up...
RTT|  00:00:01  (periodic user-mode task, 100 us period, priority 99)
RTH|----lat min|----lat avg|----lat max|-overrun|---msw|---lat best|--lat worst
RTD|      0.174|      0.464|      1.780|       0|     0|      0.174|      1.780
RTD|      0.088|      0.464|      1.357|       0|     0|      0.088|      1.780
RTD|      0.336|      0.464|      1.822|       0|     0|      0.088|      1.822
RTD|      0.342|      0.464|      1.360|       0|     0|      0.088|      1.822
RTD|      0.327|      0.462|      2.297|       0|     0|      0.088|      2.297
RTD|      0.347|      0.463|      1.313|       0|     0|      0.088|      2.297
RTD|      0.314|      0.464|      1.465|       0|     0|      0.088|      2.297
RTD|      0.190|      0.464|      1.311|       0|     0|      0.088|      2.297

tip:

To get pertinent results, you need to stress your system.
to do so, you can use stress or dohell from the apt.

# Using stress
stress -v -c 8 -i 10 -d 8

Negative latency issues

You need to be in root sudo -s, then you can set values to the latency calibration variable in nanoseconds:

$ echo 0 > /proc/xenomai/latency
# Now run the latency test

# If the minimum latency value is positive, 
# then get the lowest value from the latency test (ex: 0.088 us)
# and write it to the calibration file ( here you have to write 88 ns) : 
$ echo my_super_value_in_ns > /proc/xenomai/latency

Source : https://xenomai.org/pipermail/xenomai/2007-May/009063.html

注意:

如果运行 xeno latency 发现有 周期性高延时 或者
dmesg | grep -i xenomai 发现有如下信息：

[    0.584381] Xenomai: disabling automatic C1E state promotion on Intel processor
[    0.584388] Xenomai: SMI-enabled chipset found, but SMI workaround disabled

那么你可能遇到了 SMI 配置问题： google 如何禁用 SMI， 有个工具叫 smictrl：
see link: https://github.com/zultron/smictrl-deb

安装完后可按如下方法运行 smictrl 禁用 SMI:

smictrl -s 0

see ref link:
https://rt.wiki.kernel.org/index.php/HOWTO:_Build_an_RT-application
http://wiki.linuxcnc.org/cgi-bin/wiki.pl?FixingSMIIssues

---

xenomai3 cross compile:

打了preempt-rt补丁之后：

ln -s arm-linux-gnueabihf-gcc-4.x arm-linux-gnueabihf-gcc

./configure --help

./configure CFLAGS="-march=armv7-a -mfpu=vfp3" LDFLAGS="-march=armv7-a" --build=i686-pc-linux-gnu --host=arm-gnueabihf- --with-core=mercury --enable-smp --enable-pshared

make DESTDIR=/absolute/path/build install

arm 版 xenomai3 native compile:

./configure CFLAGS="-march=armv7-a -mfpu=vfp3" LDFLAGS="-march=armv7-a" --with-core=mercury --enable-smp --enable-pshared

full details:

http://rtt-lwr.readthedocs.io/en/latest/rtpc/xenomai.html