ARM64 vanilla kernel in QEMU

最新推荐文章于 2023-02-19 21:22:42 发布

maimang09

最新推荐文章于 2023-02-19 21:22:42 发布

阅读量404

点赞数

分类专栏： LInux kernel 文章标签： linux 服务器

原文链接：https://kasiviswanathanblog.wordpress.com/2020/05/03/arm64-vanilla-kernel-in-qemu/

版权

LInux kernel 专栏收录该内容

7 篇文章 1 订阅

订阅专栏

ARM64 vanilla kernel in QEMU – SimpleBlogs

It has been quite some time since i have experimented on any thing and the COVID-19 pandemic lock down provided the needed time to experiment few things other than official work.

This blog is about compiling and running ARM64 linux kernel in a qemu environment with the rootfs compiled using buildroot.

With buildroot this can be done easily with just 3 steps.

Clone the latest buildroot.
Find the proper config file and set it up.
Start the build using make command.

In short, if the below commands are used then we have a complete build of ARM64 embedded linux environment.

git clone git://git.buildroot.net/buildroot
make qemu_aarch64_virt_defconfig;
time make

When the build gets completed the linux kernel & filesystem images can be loaded and things should work out of the box. From my personal experience buildroot is the best tool for bringing up linux based embedded systems without much fuss.

The intention of this blog is to use the latest vanilla kernel from kernel.org and to use the filesystem alone created using buildroot.

Things which are going to be done, in short the steps :

Compile a minimal filesystem using buildroot.
Using linaro toolchain for ARM64 compile vanilla kernel after linking it with the filesystem which is created using buildroot.
Load it using qemu and confirm it is working.
Execute a simple C program compiled for ARM64.

Even though low cost development boards based on ARM64bit cores are is available from different vendors, using qemu helps in quicker turnaround time for application developers.

Building filesystem using buildroot

As said above first step is to clone the buildroot repo using the command “git clone git://git.buildroot.net/buildroot“

In the cloned repo, the list of available configs files are in the directory by the name “configs“. The config file of interest is “configs/qemu_aarch64_virt_defconfig“. By executing “make qemu_aarch64_virt_defconfig” .config file which will be used by the buildroot for the build will be created. For this custom build lot more options can be selected using “make menuconfig” command other than the ones which are already provided by the config file (complete menu of the available options will be presented when this command is executed). Since few options needs to be enabled execute make menuconfig command, enter inside the Filesystem images submenu and then select “cpio the root filesystem (for use as an initial RAM filesystem)” option. Below is a screenshot:

Setup Cross compiler

To cross compile the kernel for ARM64, toolchain provided by Linaro can be used. Below is the link to download the toolchain:

Linaro Releases

This file can be downloaded using the command “wget -c https://releases.linaro.org/components/toolchain/binaries/latest-7/aarch64-linux-gnu/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu.tar.xz” under the desired directory of choice. Untar this file so that the cross compile environment can be setup. Using the below commands the cross compiler path is exported and the toolchain is setup for cross compilation.

export PATH=<DIRECTORY WHERE THE TAR FILE IS UNTARRED>/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin:$PATH
export CROSS_COMPILE=aarch64-linux-gnu-
export ARCH=arm64

If the cross compiler is not setup properly then things may not work as expected. So double check and make sure it is setup properly.

Download vanilla kernel and start the build

At the time of writing this material the latest stable vanilla kernel is 5.6.8. Again using wget command “wget -c https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.6.8.tar.xz” command kernel source code can be downloaded at the desired directory location. After untarring this file, the default ARM64 config file is available at the location “arch/arm64/configs/” inside the kernel source code.

kasi@Vostro linux-5.6.8]$ ls arch/arm64/configs/ -l
total 24
-rw-rw-r– 1 kasi kasi 20822 Apr 29 20:04 defconfig

The already available config file in the linux kernel can be used as base and then changes can be made on top of that.

kasi@Vostro linux-5.6.8]$ make defconfig
HOSTCC scripts/basic/fixdep
HOSTCC scripts/kconfig/conf.o
HOSTCC scripts/kconfig/confdata.o
HOSTCC scripts/kconfig/expr.o
LEX scripts/kconfig/lexer.lex.c
YACC scripts/kconfig/parser.tab.[ch]
HOSTCC scripts/kconfig/lexer.lex.o
HOSTCC scripts/kconfig/parser.tab.o
HOSTCC scripts/kconfig/preprocess.o
HOSTCC scripts/kconfig/symbol.o
HOSTCC scripts/kconfig/util.o
HOSTLD scripts/kconfig/conf
*** Default configuration is based on ‘defconfig’
#
# configuration written to .config
#

Kernel compilation can be started with the .config file created as the toolchain has also been setup but before doing that few changes needs to be done to the .config file. Inside the .config file search for the config value CONFIG_CMDLINE= and change it like this CONFIG_CMDLINE=”console=ttyAMA0″. Next option which needs to be modified is to add the path of cpio archive to be used as initramfs by the kernel. Modify the CONFIG_INITRAMFS_SOURCE=”” option with the location of the cpio archive created using buildroot. For example, CONFIG_INITRAMFS_SOURCE=”<BUILDROOT BUILD DIRECTORY/output/images/rootfs.cpio”. These two are the changes which needs to be done to the config file. Sample of the changes made to the .config file in the kernel source code.

kasi@Vostro linux-5.6.8]$ grep CONFIG_CMDLINE .config
CONFIG_CMDLINE=”console=ttyAMA0″
kasi@Vostro linux-5.6.8]$ grep CONFIG_INITRAMFS .config
CONFIG_INITRAMFS_SOURCE=”/Sourcefiles/armv8/buildroot/output/images/rootfs.cpio”

Run “make menuconfig” once and save the config file. Kernel build can now be started using make command. Since the kernel provided defconfig config file has lot of options enabled by default it takes quite some time to compile the kernel. When kernel gets compiled qemu session can be started with the newly compiled image for ARM64. When the build has completed (it took a solid 90 mins in a quad core machine) the kernel image is available at “arch/arm64/boot“.

kasi@Vostro linux-5.6.8]$ ls arch/arm64/boot/Image -lh
-rw-rw-r– 1 kasi kasi 28M May 1 15:10 arch/arm64/boot/Image

The image is around 28 Megabytes since it has both kernel and filesystem packed into it. Time to validate what has been done till now. For this “qemu-system-aarch64” command needs to be used. This command is part of the package “qemu-system-arm” and can be installed easily on any distros (For example, on debian/ubuntu clones apt-get install qemu-system-arm).

Start qemu with vanilla kernel

With the below command vanilla kernel which has been compiled can be started in a qemu environment.

qemu-system-aarch64 -machine virt -cpu cortex-a57 -machine type=virt -nographic -smp 2 -m 4096 -kernel arch/arm64/boot/Image

Most of the options are self explanatory but for understanding purposes let us go one by obe. Both machine and machine type are populated as virt(ual) as the environment is based on qemu. cpu parameter is the ARM cpu core which will be emulated by qemu. Cortex-a57 core was the greatest ARM64bit core supported in the qemu which was used. no-graphic option means no fancy graphic stuff and only do cli. smp parameter is used for specifying how many cpu cores that needs to be emulated by the qemu. The more the cpu core count the more time it takes for the qemu to boot up. Fair value would be two since whatever processing which is going to be done in qemu will be done by the native hardware which will be mostly based on a x86-64 intel machine. m parameter mentions the amount RAM which qemu can use and -kernel parameter is passed with the location of the image.

Few lines from the boot up log

kasi@Vostro linux-5.6.8]$ qemu-system-aarch64 -machine virt -cpu cortex-a57 -machine type=virt -nographic -smp 2 -m 4096 -kernel arch/arm64/boot/Image

[ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x411fd070]
[ 0.000000] Linux version 5.6.8 (kasi@Vostro) (gcc version 7.5.0 (Linaro GCC 7.5-2019.12)) #1 SMP PREEMPT Fri May 1 13:29:30 IST 2020
[ 0.000000] Machine model: linux,dummy-virt

Of the first three lines, second one shows the linux kernel version and time/date when this image was compiled. The third line shows the machine model as virtual which is correct. If the kernel image was compiled properly then login prompt will be presented and by entering the just the username “root” login should happen.

Welcome to Buildroot
buildroot login: root

After entering into the bash the complete ARM64 environment is available and can be used for any purpose. Few things which can be checked are the cpu information and the memory available for use.

# cat /proc/cpuinfo
processor : 0
BogoMIPS : 125.00
Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x1
CPU part : 0xd07
CPU revision : 0

processor : 1
BogoMIPS : 125.00
Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x1
CPU part : 0xd07
CPU revision : 0

# free -m
total used free shared buff/cache available
Mem: 3931 14 3915 1 1 3801
Swap: 0 0 0

The above information from the qemu matches with the parameters which was passed to it. To come out of the qemu environment at any time just press CTRL+A and then press x.

Saving few more seconds

What has been done till now is vanilla kernel has been compiled and loaded with the filesystem which contains tools of our choice. One issue with this approach is that it takes too much time to compile the kernel image as the config file being used has lot of unwanted options selected and most of them is not going to be used in the qemu environment. This also means that it takes quite sometime (in secs) to load the image. Buildroot while creating the filesystem image will also compile the kernel but mostly it will not compile the latest one. So the config file available in the buildroot’s kernel build can be used in the kernel source code which has been downloaded and built. Take a backup of the current .config file and copy the kernel config file from buildroot’s build.

cp <BUILDROOT BUILD DIRECTORY>/output/build/linux-5.4.35/.config <KERNEL_BUILD_DIR>/.config

After copying the config file, setup the two kernel options CONFIG_CMDLINE and CONFIG_INITRAMFS_SOURCE as done previously. Do make menuconfig, save and then exit. Again start the kernel compilation using make command and wait.

The build with the config file from buildroot got compiled within 15 mins whereas the build using the default configuration took around 90 mins which is six times more. Let us check if the new kernel is also working.

kasi@Vostro linux-5.6.8]$ qemu-system-aarch64 -machine virt -cpu cortex-a57 -machine type=virt -nographic -smp 2 -m 4096 -kernel arch/arm64/boot/Image

Booting Linux on physical CPU 0x0000000000 [0x411fd070]
Linux version 5.6.8 (kasi@Vostro) (gcc version 7.5.0 (Linaro GCC 7.5-2019.12)) #1 SMP Fri May 1 19:51:35 IST 2020
Machine model: linux,dummy-virt

The second line shows at what time this image was compiled (which is different from the first compilation). This image actually took around 15 to 20 seconds to load as it was trying to load eth0 interface and it failed but was still able to login. This image should have loaded in 3 to 5 seconds (max) which is few seconds lesser than the previous build. Let us try to check if this can be solved and boot time can be reduced.

To remove unwanted loading up of eth0 interface from the image do like as shown below. Inside the buildroot’s build directory execute make menuconfig command and enter inside Target packages > Networking applications menu. Here deselect the option “ifupdown scripts“.

Save this config and then exit. Next step is to do a “make clean” to clean up the old stuff and execute make do a fresh build to create a new filesystem image. When the buildroot build gets over trigger the kernel build one more time to make sure the kernel picks up the new image. In the new kernel build printk is enabled to add timing information to the boot up logs so that the timing is displayed in the logs which will be useful in the future. This change has been done in the kernel .config file as this option was missing in the kernel config file which was copied from buildroot’s kernel build. The newly built image can be tested again to confirm whether it works fine or not.

Few lines from the boot up logs

[ 0.000000] Linux version 5.6.8 (kasi@Vostro) (gcc version 7.5.0 (Linaro GCC 7.5-2019.12)) #5 SMP Fri May 1 23:03:17 IST 2020

<Cut here>

[ 1.359785] Run /init as init process
Starting syslogd: OK
Starting klogd: OK
Running sysctl: OK
Saving random seed: [ 2.585752] random: dd: uninitialized urandom read (512 bytes read)
OK

Welcome to Buildroot
buildroot login: root
#

The kernel has booted up in 3 seconds which is quite fast when compared to the old build. Timing saved will add up exponentially with the number of times the qemu is executed

Execute a simple C program in the qemu

With the execution environment setup it is time to write a simple C program, compile it for ARM64, add it to the filesystem image and execute it inside the qemu environment. Before starting to write the program create a directory by the name of your choice and then start coding there. The reason for creating a separate directory is this directory location will be added to the buildroot’s config file and build will be started. Once this is done only the final image will have the C program which has written and compiled.

Contents of the C program:

A directory by the name pgms has been created under <BUILDROOT BUILD DIRECTORY>. C program with the name hello.c is created and the contents are shown below.

kasi@Vostro pgms]$ pwd
/Sourcefiles/armv8/buildroot/pgms
kasi@Vostro pgms]$ cat hello.c
#include <stdio.h>
main()
{
printf(“Inside QEMU and ARM64\n”);
}

This program can be compiled using the cross compiler available in the buildroot itself. The buildroot cross compiler is available at <BUILDROOT BUILD DIRECTORY>/output/host/bin.

kasi@Vostro pgms]$ pwd
/Sourcefiles/armv8/buildroot/pgms
kasi@Vostro pgms]$ ../output/host/bin/aarch64-linux-gcc -o hello hello.c
hello.c:2:1: warning: return type defaults to ‘int’ [-Wimplicit-int]
main()
^~~~
kasi@Vostro pgms]$ ls -l
total 16
-rwxrwxr-x 1 kasi kasi 8672 May 2 20:04 hello
-rw-rw-r– 1 kasi kasi 66 May 2 11:28 hello.c
kasi@Vostro pgms]$

Now the C program has been compiled and it is now time to make this part of the buildroot’s filesystem. For this in the buildroot’s .config file one entry needs to be updated with the location of the C program. The parameter is BR2_ROOTFS_OVERLAY and it needs to be updated with the directory location of the C program. For example, it is updated as shown below:

BR2_ROOTFS_OVERLAY=”/Sourcefiles/armv8/buildroot/pgms”

Do a “make menuconfig“ (whenever the config file is changed it is better to do make menuconfig), save this config and exit. Time to start build again using “make“. It will take only few seconds to compile as it is just including only the specified files. When the buildroot build completes, build the kernel image once again to make sure the C program source and binaries are included in the final image. Kernel build should also take only few seconds as nothing has been changed from kernel’s perspective and it is going to just repackage the final image alone.

Welcome to Buildroot
buildroot login: root
#
#
# ls /
bin hello lib media proc sbin usr
dev hello.c lib64 mnt root sys var
etc init linuxrc opt run tmp
# cd /
# ./hello
Inside QEMU and ARM64
#

For any further changes done to the C program, it needs to be cross compiled first and then buildroot needs to be built and then finally the kernel needs to be built. This procedure needs to be applied if any new file is created also.