Lately, I came up with the idea to do some development on Aarch64. However, I couldn't get my hands on real hardware easily so I started to look for alternatives (i.e., emulators). The ARMv8 Foundation Model seemed to be the trivial solution but I've heard that QEMU is somewhat faster so I gave it a try. My goal was to set up the VM as quick as possible: reuse whatever is already "out there" and rebuild only what's utterly necessary. In the end it turned out that it's quite easy to get such a VM working ... once you know what you need. To figure that out, well, that required some googling and emailing around. Below are my notes (I cleaned them up a bit and tried to make them somewhat coherent), so that others won't need to compile all these information again. I hope that someone will find it useful.
(Note: My host machine, where everything described below was done/tested on, is an x86_64 desktop box running a (more or less) fresh Ubuntu 14.04.1.)
Build your own QEMU/Aarch64
Okay, my goal was not to rebuild, and I failed at the very first step. For Aarch64, it's best to build QEMU from source. Fortunately, building QEMU is really simple:
sudo apt-get build-dep qemu # install the dependencies git clone git://git.qemu.org/qemu.git # get the source (at the time of writing, HEAD was at revision 69f87f713069f1f70f86cb65883f7d43e3aa21de) cd qemu ./configure --target-list=aarch64-softmmu --enable-virtfs # configure for aarch64 (virtfs only needed if you'd like to mount up a dir of the host in the guest OS) make # build qemu (make install is not necessary)
That's it, QEMU is ready to run. The questions are "what" (images) and "how" (command line options)?
Create a clean Aarch64 Ubuntu Core 14.04.1 image
First, I wanted a well-prepared and well-maintained root filesystem. And the good news is that Ubuntu has 14.04 out for Aarch64 as well. So, it only takes a download from the official website, and either some manual work or the help of a useful script to turn the downloaded tarball into a QEMU-compatible image. The script I ended up using is attached to this blogpost.
wget http://cdimage.ubuntu.com/ubuntu-core/releases/14.04/release/ubuntu-core... ./arm64-prepare-image-qemu.sh ubuntu-core-14.04.1-core-arm64.tar.gz # the script will sudo!
(Note: The script creates a 32 GB file! See at the end of the post how to change that.)
Get your hands on suitable kernel & initrd (for those who are lazy to build their own)
Beside the root filesystem, I also needed the images of an already built kernel and an initrd. Again, Ubuntu has those, even if a bit hidden away: they can be taken from inside a QCOW2-format Ubuntu 14.04 cloud image. This needs a bit more steps than creating the rootfs image before but it's still not rocket science (and needs to be done only once).
Preparation steps:
sudo apt-get install qemu-utils # install tools required to deal with the QCOW2 format wget http://cloud-images.ubuntu.com/releases/14.04/release/ubuntu-14.04-serve... # download the cloud image
Mount the cloud image:
sudo modprobe nbd max_part=63 sudo qemu-nbd -c /dev/nbd0 ubuntu-14.04-server-cloudimg-arm64-disk1.img mkdir mnt sudo mount /dev/nbd0p1 mnt
Copy out the kernel & initrd files from the cloud image to the file system of the host:
sudo cp mnt/boot/vmlinuz-3.13.0-32-generic . sudo cp mnt/boot/initrd.img-3.13.0-32-generic .
(Note: Some chown-ing is needed here here, since the files are owned by root.)
Finally, clean up (unmount the image):
sudo umount mnt sudo qemu-nbd -d /dev/nbd0 rmdir mnt
Run your brand new Aarch64 Ubuntu 14.04.1 system
QEMU and the images are all ready for use by now, so the only issue that remains is to get all the command line options right. Here is what worked for me the best:
./aarch64-softmmu/qemu-system-aarch64 -machine virt -cpu cortex-a57 -nographic -smp 1 -m 4096 \ -global virtio-blk-device.scsi=off -device virtio-scsi-device,id=scsi \ -drive file=../qemu-images/ubuntu-core-14.04.1-core-arm64.img,id=coreimg,cache=unsafe,if=none -device scsi-hd,drive=coreimg \ -kernel ../qemu-images/vmlinuz-3.13.0-32-generic \ -initrd ../qemu-images/initrd.img-3.13.0-32-generic \ -netdev user,id=unet -device virtio-net-device,netdev=unet \ --append "console=ttyAMA0 root=/dev/sda"
This will start up the system, so ... Enjoy!
Some notes/advices:
- The first login prompt will already be lost somewhere amidst the startup log messages by the time they stop rolling. Just press enter to get a new one.
- The initial password for root is root. (Don't forget to change it after the first login!)
- The current command line sets up user mode networking for the VM. It has its limitations (e.g., no ping) but it should be fine for simple web access for the start.
- Do apt-get update && apt-get upgrade at least at the first startup.
- Don't forget to properly shut down your VM if you're done with it (shutdown -h now). Just killing the qemu process is as rude as plugging out the power cord from your PC! Once you see "reboot: System halted" on the console, you can switch to the QEMU monitor by pressing CTRL+A C and then type quit.
Customization
So, all the above is OK, so-so, but not exactly what you want. Then, you may want to:
- change the size of the disk image holding the root fs: change the count=32768 argument of dd in the arm64-prepare-image-qemu script (specify size in MB).
- change the memory allocated for the VM: change the -m 4096 command line parameter of qemu-system-aarch64 (specify size in MB).
- replace the default user networking (slirp) with a more powerful networking setup: experiment with the -netdev command line options.
- build your own kernel & initrd: good luck with that :)
Acknowledgements / Sources of information
I received a lot of help from Alex Bennée, for which I'm really thankful, and collected info/inspiration from the sources below:
- http://www.bennee.com/~alex/blog/2014/05/09/running-linux-in-qemus-aarch...
- https://wiki.ubuntu.com/ARM64/FoundationModel
- http://alexeytorkhov.blogspot.hu/2009/09/mounting-raw-and-qcow2-vm-disk-...
- https://lists.gnu.org/archive/html/qemu-devel/2014-03/msg04775.html
- http://www.linux-kvm.org/page/Virtio
http://webkit.sed.hu/blog/20140816/quickndirty-set-aarch64-ubuntu-1404-vm-qemu