Run Linux for ARMv8 on qemu

Run Linux for ARMv8 on qemu

对于很多热爱ARM Linux的Geeker来说，如果随时随地打开电脑就能快速搭建一个ARM Linux开发环境，并进行各种hack，将是一件大快人心的事情。但是现实总是不那么尽如人意：你的ARM开发板总不能随时都处于你身边唾手可得的位置 ，它可能躺在你办公室的工位上，也有可能你手边刚好缺一个能够给你的开发板供电的适配器。幸运的是，我们有Qemu，借助这个完善的开源模拟器，我们就有了一块随时随地能hack的开发板。只要你不是为了调试那些需要严格依赖硬件的外设，比如某一颗HDMI转换芯片，或者某个基于I2C接口的Audio codec，而是用它来研究一些软件框架相关的模块，比如Linux kernel的内存管理，比如ARM的启动代码，比如Linux 内核的进程调度，这些对具体硬件依赖程度不高的模块，它都能胜任。

下面我会做一个简单的例子，描述我是如何利用upstream linux kernel + buildroot 制作根文件系统，构建一个可以在qemu上运行的ARM64 最小的Linux系统的。

如果以位数（32/64）区分，目前的ARM处理器有两类：ARMv7及其以下的版本为32位处理器，ARMv8及其以上的版本为64位处理器。针对这两类ARM CPU，Qemu对应的模拟器可执行程序分别是： qemu-system-arm和qemu-system-aarch64.

在Hack的过程中，我发现Ubuntu 16.04自带的qemu-system-aarch64（版本比较低）在运行Linux 4.5及其以上版本的内核的时候，一启动就会报错，然后我尝试自己编译了最新版本的qemu-system-aarch64（v2.10.1），能够正常运行，然后我就直接用自己编译的版本把Ubuntu系统自带的版本给覆盖了 。暂时还没有去研究为什么低版本的Qemu启动新版本的内核会出错的原因，也许是低版本的qemu不支持某些ARMv8的新特性，哈哈，这个以后再说，暂时现在先聚焦主题，来boot一个ARMv8 Linux最小系统。

一般要启动一个Linux最小系统，需要两个部分： Linux kernel + 根文件系统。最近发现buildroot用来构建根文件系统非常方便：

buildroot源码下载：

git clone git://git.buildroot.net/buildroot

编译： 一个最小的根文件系统只需要通过make menuconfig打开如下配置即可

BR2_aarch64=y
BR2_cortex_a57=y
BR2_TOOLCHAIN_BUILDROOT_GLIBC=y               /* 使用buildroot编译的toolchain */
BR2_KERNEL_HEADERS_4_4=y                      /* kernel 头文件 */
BR2_TARGET_GENERIC_GETTY_PORT="ttyAMA0"       /* ARM virt 平台的串口名称 */
BR2_PACKAGE_BUSYBOX_SHOW_OTHERS=y
BR2_TARGET_ROOTFS_CPIO=y                      /* 这里编译成压缩的文件系统，做initramfs */

该配置参考了一位Linaro工程师的博文: Running Linux in QEMU’s aarch64 system emulation mode

执行make 命令进行编译，最终得到的根文件镜像为：output/images/rootfs.cpio

Linux kernel直接使用upstream的最新代码：

下载：

git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git

编译：

make ARCH=arm64 defconfig
make ARCH=arm64 menuconfig 指定上面buildroot编译的rootfs做为initramfs
CONFIG_INITRAMFS_SOURCE="../opensource/buildroot/output/images/rootfs.cpio "
make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu-

编译完成后，生成内核镜像：arch/arm64/boot/Image

然后就可以利用qemu启动该镜像进入Linux最小系统了：

qemu-system-aarch64 -machine virt -cpu cortex-a53 -machine type=virt -nographic -smp 4 -m 512 -kernel arch/arm64/boot/Image  --append "earlycon console=ttyAMA0"

启动log：

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034]
[    0.000000] Linux version 4.16.0-00002-ga378398 (andy@Xeon) (gcc version 6.3.1 20170404 (Linaro GCC 6.3-2017.05)) #46 SMP PREEMPT Sun Apr 22 11:32:47 CST 2018
[    0.000000] Machine model: linux,dummy-virt
[    0.000000] earlycon: pl11 at MMIO 0x0000000009000000 (options '')
[    0.000000] bootconsole [pl11] enabled
[    0.000000] efi: Getting EFI parameters from FDT:
[    0.000000] efi: UEFI not found.
[    0.000000] NUMA: No NUMA configuration found
[    0.000000] NUMA: Faking a node at [mem 0x0000000000000000-0x000000005fffffff]
[    0.000000] NUMA: NODE_DATA [mem 0x5ffeab00-0x5ffec2ff]
[    0.000000] Zone ranges:
[    0.000000]   DMA32    [mem 0x0000000040000000-0x000000005fffffff]
[    0.000000]   Normal   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000040000000-0x000000005fffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000005fffffff]
[    0.000000] psci: probing for conduit method from DT.
[    0.000000] psci: PSCIv0.2 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: Trusted OS migration not required
[    0.000000] random: get_random_bytes called from start_kernel+0xa4/0x3fc with crng_init=0
[    0.000000] percpu: Embedded 21 pages/cpu @        (ptrval) s54936 r0 d31080 u86016
[    0.000000] Detected VIPT I-cache on CPU0
[    0.000000] CPU features: enabling workaround for ARM erratum 845719
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 129024
[    0.000000] Policy zone: DMA32
[    0.000000] Kernel command line: earlycon console=ttyAMA0
[    0.000000] Memory: 496772K/524288K available (5308K kernel code, 430K rwdata, 1296K rodata, 3200K init, 313K bss, 27516K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] Preemptible hierarchical RCU implementation.
[    0.000000]  Tasks RCU enabled.
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] GICv2m: range[mem 0x08020000-0x08020fff], SPI[80:143]
[    0.000000] arch_timer: cp15 timer(s) running at 62.50MHz (virt).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x1cd42e208c, max_idle_ns: 881590405314 ns
[    0.000296] sched_clock: 56 bits at 62MHz, resolution 16ns, wraps every 4398046511096ns
[    0.012764] Console: colour dummy device 80x25
[    0.014503] Calibrating delay loop (skipped), value calculated using timer frequency.. 125.00 BogoMIPS (lpj=250000)
[    0.015136] pid_max: default: 32768 minimum: 301
[    0.018630] Dentry cache hash table entries: 65536 (order: 7, 524288 bytes)
[    0.019588] Inode-cache hash table entries: 32768 (order: 6, 262144 bytes)
[    0.020227] Mount-cache hash table entries: 1024 (order: 1, 8192 bytes)
[    0.020598] Mountpoint-cache hash table entries: 1024 (order: 1, 8192 bytes)
[    0.081167] ASID allocator initialised with 32768 entries
[    0.088726] Hierarchical SRCU implementation.
[    0.103012] EFI services will not be available.
[    0.111318] smp: Bringing up secondary CPUs ...
[    0.145152] Detected VIPT I-cache on CPU1
[    0.146719] CPU1: Booted secondary processor 0x0000000001 [0x410fd034]
[    0.180304] Detected VIPT I-cache on CPU2
[    0.180646] CPU2: Booted secondary processor 0x0000000002 [0x410fd034]
[    0.210690] Detected VIPT I-cache on CPU3
[    0.211055] CPU3: Booted secondary processor 0x0000000003 [0x410fd034]
[    0.212162] smp: Brought up 1 node, 4 CPUs
[    0.215012] SMP: Total of 4 processors activated.
[    0.215551] CPU features: detected feature: 32-bit EL0 Support
[    0.216023] CPU features: detected feature: Kernel page table isolation (KPTI)
[    0.293793] CPU: All CPU(s) started at EL1
[    0.294664] alternatives: patching kernel code
[    0.324397] devtmpfs: initialized
[    0.350758] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.351812] futex hash table entries: 1024 (order: 5, 131072 bytes)
[    0.357964] pinctrl core: initialized pinctrl subsystem
[    0.380976] random: fast init done
[    0.395311] DMI not present or invalid.
[    0.403080] NET: Registered protocol family 16
[    0.408010] audit: initializing netlink subsys (disabled)
[    0.412454] audit: type=2000 audit(0.372:1): state=initialized audit_enabled=0 res=1
[    0.421406] cpuidle: using governor menu
[    0.422241] vdso: 2 pages (1 code @         (ptrval), 1 data @         (ptrval))
[    0.423053] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    0.425936] DMA: preallocated 256 KiB pool for atomic allocations
[    0.428492] Serial: AMBA PL011 UART driver
[    0.472014] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 39, base_baud = 0) is a PL011 rev1
[    0.474227] console [ttyAMA0] enabled
[    0.474227] console [ttyAMA0] enabled
[    0.474996] bootconsole [pl11] disabled
[    0.474996] bootconsole [pl11] disabled
[    0.541652] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    0.546979] cryptd: max_cpu_qlen set to 1000
[    0.555611] vgaarb: loaded
[    0.558169] SCSI subsystem initialized
[    0.560435] pps_core: LinuxPPS API ver. 1 registered
[    0.560798] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.561537] PTP clock support registered
[    0.577997] clocksource: Switched to clocksource arch_sys_counter
[    0.585547] VFS: Disk quotas dquot_6.6.0
[    0.586966] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.667685] NET: Registered protocol family 2
[    0.679278] tcp_listen_portaddr_hash hash table entries: 256 (order: 0, 4096 bytes)
[    0.680122] TCP established hash table entries: 4096 (order: 3, 32768 bytes)
[    0.681037] TCP bind hash table entries: 4096 (order: 4, 65536 bytes)
[    0.682116] TCP: Hash tables configured (established 4096 bind 4096)
[    0.692515] UDP hash table entries: 256 (order: 1, 8192 bytes)
[    0.693391] UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
[    0.697140] NET: Registered protocol family 1
[    1.091659] hw perfevents: enabled with armv8_pmuv3 PMU driver, 1 counters available
[    1.158568] workingset: timestamp_bits=44 max_order=17 bucket_order=0
[    1.196143] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    1.238625] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)
[    1.239695] io scheduler noop registered (default)
[    1.249668] pl061_gpio 9030000.pl061: PL061 GPIO chip @0x0000000009030000 registered
[    1.254717] PCI: OF: host bridge /pcie@10000000 ranges:
[    1.255990] PCI: OF:    IO 0x3eff0000..0x3effffff -> 0x00000000
[    1.256914] PCI: OF:   MEM 0x10000000..0x3efeffff -> 0x10000000
[    1.257382] PCI: OF:   MEM 0x8000000000..0xffffffffff -> 0x8000000000
[    1.259121] pci-host-generic 3f000000.pcie: ECAM at [mem 0x3f000000-0x3fffffff] for [bus 00-0f]
[    1.261471] pci-host-generic 3f000000.pcie: PCI host bridge to bus 0000:00
[    1.262833] pci_bus 0000:00: root bus resource [bus 00-0f]
[    1.263400] pci_bus 0000:00: root bus resource [io  0x0000-0xffff]
[    1.263807] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]
[    1.264227] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]
[    1.278742] pci 0000:00:01.0: BAR 6: assigned [mem 0x10000000-0x1003ffff pref]
[    1.279628] pci 0000:00:01.0: BAR 4: assigned [mem 0x8000000000-0x8000003fff 64bit pref]
[    1.280333] pci 0000:00:01.0: BAR 1: assigned [mem 0x10040000-0x10040fff]
[    1.280726] pci 0000:00:01.0: BAR 0: assigned [io  0x1000-0x101f]
[    1.300393] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)
[    1.342393] cacheinfo: Unable to detect cache hierarchy for CPU 0
[    1.376846] loop: module loaded
[    1.452650] rtc-pl031 9010000.pl031: rtc core: registered pl031 as rtc0
[    1.454729] i2c /dev entries driver
[    1.480609] NET: Registered protocol family 17
[    1.482712] Bridge firewalling registered
[    1.483086] 8021q: 802.1Q VLAN Support v1.8
[    1.488440] registered taskstats version 1
[    1.503669] input: gpio-keys as /devices/platform/gpio-keys/input/input0
[    1.509552] rtc-pl031 9010000.pl031: setting system clock to 2018-04-22 09:53:06 UTC (1524390786)
[    1.517606] uart-pl011 9000000.pl011: no DMA platform data
[    1.619787] Freeing unused kernel memory: 3200K
Starting logging: OK
Initializing random number generator... done.
Starting network: OK

Welcome to Buildroot
buildroot login: root
# cd /
# ls
bin      init     linuxrc  opt      run      tmp
dev      lib      media    proc     sbin     usr
etc      lib64    mnt      root     sys      var
# mount
rootfs on / type rootfs (rw,size=248384k,nr_inodes=62096)
devtmpfs on /dev type devtmpfs (rw,relatime,size=248384k,nr_inodes=62096,mode=755)
proc on /proc type proc (rw,relatime)
devpts on /dev/pts type devpts (rw,relatime,gid=5,mode=620,ptmxmode=666)
tmpfs on /dev/shm type tmpfs (rw,relatime,mode=777)
tmpfs on /tmp type tmpfs (rw,relatime)
tmpfs on /run type tmpfs (rw,nosuid,nodev,relatime,mode=755)
sysfs on /sys type sysfs (rw,relatime)
# 

按Ctrl-a x组合键可以退出qemu。

本文为博主原创，转载请著名出处。欢迎关注博主的公众号：HackforFun