基于 QEMU进行 arm 仿真开发 （以 vexpress-a9 为例）

背景

基于 QEMU 的仿真可以节省 硬件成本。

参考：《qemu-system-arm仿真vexpress-a9踩坑记》、《在Ubuntu下使用QEMU搭建arm开发环境（一）搭建基本开发环境》

我们在这一讲主要以搭建仿真环境为主。

host平台　　 ：Ubuntu 16.04
qemu 3.0.0
arm-linux-gcc : v7.4.1 20181213 [linaro-7.4-2019.02 revision 56ec6f6b99cc167ff0c2f8e1a2eed33b1edc85d4]

QEMU 介绍

Qemu 是纯软件实现的虚拟化模拟器，几乎可以模拟任何硬件设备，我们最熟悉的就是能够模拟一台能够独立运行操作系统的虚拟机，虚拟机认为自己和硬件打交道，但其实是和 Qemu 模拟出来的硬件打交道，Qemu 将这些指令转译给真正的硬件。

正因为 Qemu 是纯软件实现的，所有的指令都要经 Qemu 过一手，性能非常低，所以，在生产环境中，大多数的做法都是配合 KVM 来完成虚拟化工作，因为 KVM 是硬件辅助的虚拟化技术，主要负责 比较繁琐的 CPU 和内存虚拟化，而 Qemu 则负责 I/O 虚拟化，两者合作各自发挥自身的优势，相得益彰。

QEMU 同时也是一个非常简单的虚拟机，给它一个硬盘镜像就可以启动一个虚拟机，如果想定制这个虚拟机的配置，比如用什么样的 CPU 啊、什么样的显卡啊、什么样的网络配置啊，指定相应的命令行参数就可以了。它支持许多格式的磁盘镜像，包括 VirtualBox 创建的磁盘镜像文件。它同时也提供一个创建和管理磁盘镜像的工具 qemu-img。QEMU 及其工具所使用的命令行参数，直接查看其文档即可。

QEMU 安装

由于包管理器中的qemu太老了，所以我们采取基于源码的安装。

在此之前，需要先准备好有关的环境

sudo apt-get install build-essential pkg-config zlib1g-dev libglib2.0-0 libglib2.0-dev  libsdl1.2-dev libpixman-1-dev libfdt-dev autoconf automake libtool librbd-dev libaio-dev flex bison -y

下载有关的源码

wget https://download.qemu.org/qemu-3.0.0.tar.bz2

编译有关的源码

tar xf qemu-3.0.0.tar.xz
cd qemu-3.0.0
./configure --prefix=/usr/local/qemu --target-list=arm-softmmu --audio-drv-list=
sudo make && sudo make install
sudo ln -s /usr/local/qemu/bin/* /usr/local/bin/

# --target-list：选择目标机器的架构。默认是将所有的架构都编译，但为了更快的完成编译，指定需要的架构即可。

测试

qemu-img -V
qemu-img version 3.0.0
Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers

查看 QEMU 支持的板子

$ qemu-system-arm -M help
Supported machines are:
akita                Sharp SL-C1000 (Akita) PDA (PXA270)
ast2500-evb          Aspeed AST2500 EVB (ARM1176)
borzoi               Sharp SL-C3100 (Borzoi) PDA (PXA270)
canon-a1100          Canon PowerShot A1100 IS
cheetah              Palm Tungsten|E aka. Cheetah PDA (OMAP310)
collie               Sharp SL-5500 (Collie) PDA (SA-1110)
connex               Gumstix Connex (PXA255)
cubieboard           cubietech cubieboard
emcraft-sf2          SmartFusion2 SOM kit from Emcraft (M2S010)
highbank             Calxeda Highbank (ECX-1000)
imx25-pdk            ARM i.MX25 PDK board (ARM926)
integratorcp         ARM Integrator/CP (ARM926EJ-S)
kzm                  ARM KZM Emulation Baseboard (ARM1136)
lm3s6965evb          Stellaris LM3S6965EVB
lm3s811evb           Stellaris LM3S811EVB
mainstone            Mainstone II (PXA27x)
mcimx7d-sabre        Freescale i.MX7 DUAL SABRE (Cortex A7)
midway               Calxeda Midway (ECX-2000)
mps2-an385           ARM MPS2 with AN385 FPGA image for Cortex-M3
mps2-an505           ARM MPS2 with AN505 FPGA image for Cortex-M33
mps2-an511           ARM MPS2 with AN511 DesignStart FPGA image for Cortex-M3
musicpal             Marvell 88w8618 / MusicPal (ARM926EJ-S)
n800                 Nokia N800 tablet aka. RX-34 (OMAP2420)
n810                 Nokia N810 tablet aka. RX-44 (OMAP2420)
netduino2            Netduino 2 Machine
none                 empty machine
nuri                 Samsung NURI board (Exynos4210)
palmetto-bmc         OpenPOWER Palmetto BMC (ARM926EJ-S)
raspi2               Raspberry Pi 2
realview-eb          ARM RealView Emulation Baseboard (ARM926EJ-S)
realview-eb-mpcore   ARM RealView Emulation Baseboard (ARM11MPCore)
realview-pb-a8       ARM RealView Platform Baseboard for Cortex-A8
realview-pbx-a9      ARM RealView Platform Baseboard Explore for Cortex-A9
romulus-bmc          OpenPOWER Romulus BMC (ARM1176)
sabrelite            Freescale i.MX6 Quad SABRE Lite Board (Cortex A9)
smdkc210             Samsung SMDKC210 board (Exynos4210)
spitz                Sharp SL-C3000 (Spitz) PDA (PXA270)
sx1                  Siemens SX1 (OMAP310) V2
sx1-v1               Siemens SX1 (OMAP310) V1
terrier              Sharp SL-C3200 (Terrier) PDA (PXA270)
tosa                 Sharp SL-6000 (Tosa) PDA (PXA255)
verdex               Gumstix Verdex (PXA270)
versatileab          ARM Versatile/AB (ARM926EJ-S)
versatilepb          ARM Versatile/PB (ARM926EJ-S)
vexpress-a15         ARM Versatile Express for Cortex-A15
vexpress-a9          ARM Versatile Express for Cortex-A9
virt-2.10            QEMU 2.10 ARM Virtual Machine
virt-2.11            QEMU 2.11 ARM Virtual Machine
virt-2.12            QEMU 2.12 ARM Virtual Machine
virt-2.6             QEMU 2.6 ARM Virtual Machine
virt-2.7             QEMU 2.7 ARM Virtual Machine
virt-2.8             QEMU 2.8 ARM Virtual Machine
virt-2.9             QEMU 2.9 ARM Virtual Machine
virt                 QEMU 3.0 ARM Virtual Machine (alias of virt-3.0)
virt-3.0             QEMU 3.0 ARM Virtual Machine
witherspoon-bmc      OpenPOWER Witherspoon BMC (ARM1176)
xilinx-zynq-a9       Xilinx Zynq Platform Baseboard for Cortex-A9
z2                   Zipit Z2 (PXA27x)

仿真文件的准备

仿真很简单，只需要类似这样的：

qemu-system-arm -M vexpress-a9 
-m 512M 
-kernel ./uImage 
-dtb ./vexpress-v2p-ca9.dtb 
-nographic 
-append "console=ttyAMA0"

或者是这样的

qemu-system-arm -M vexpress-a9 -m 512M -kernel arch/arm/boot/zImage -dtb arch/arm/boot/dts/vexpress-v2p-ca9.dtb -nographic -append "console=ttyAMA0"

其中，有关参数的解释如下：

-M vexpress-a9 模拟vexpress-a9单板，你能够使用-M ?參数来获取该qemu版本号支持的全部单板
-m 512M 单板执行物理内存512M
-kernel arch/arm/boot/zImage   告诉qemu单板执行内核镜像路径
-dtb arch/arm/boot/dts/vexpress-v2p-ca9.dtb  告诉qemu单板的设备树（必须加入）
-nographic 不使用图形化界面，仅仅使用串口
-append "console=ttyAMA0" 内核启动參数。这里告诉内核vexpress单板执行。串口设备是哪个tty。

我们唯一缺少的就是有关的启动文件，查阅有关资料才发现，QEMU的仿真需要有 编译好以后的文件，因为根据QEMU的原理来看，需要二进制文件。

QEMU是一个主机上的VMM（virtual machine monitor）,通过动态二进制转换来模拟CPU，并提供一系列的硬件模型，使guest os认为自己和硬件直接打交道，其实是同QEMU模拟出来的硬件打交道，QEMU再将这些指令翻译给真正硬件进行操作。通过这种模式，guest os可以和主机上的硬盘，网卡，CPU，CD-ROM，音频设备和USB设备进行交互。

编译有关的 Uboot、kernel、文件系统

安装有关环境：sudo apt-get install flex bison flex -y

编译 uboot 并仿真

编译

git clone git://git.denx.de/u-boot.git

cd u-boot
export ARCH=arm 
export CROSS_COMPILE=arm-linux-gnueabihf-
make clean && make vexpress_ca9x4_defconfig
make -j4

仿真

qemu-system-arm -M vexpress-a9 -m 256 -kernel u-boot -nographic

结果

因为下载实在是太慢了，我使用了一个之前下载好的早期uboot版本。

U-Boot 2018.01 (Feb 22 2020 - 17:34:52 +0800)

DRAM:  256 MiB
WARNING: Caches not enabled
Flash: 128 MiB
MMC:   MMC: 0
*** Warning - bad CRC, using default environment

In:    serial
Out:   serial
Err:   serial
Net:   smc911x-0
Hit any key to stop autoboot:  0
MMC Device 1 not found
no mmc device at slot 1
Card did not respond to voltage select!
mmc_init: -95, time 23
smc911x: MAC 52:54:00:12:34:56
smc911x: detected LAN9118 controller
smc911x: phy initialized
smc911x: MAC 52:54:00:12:34:56
BOOTP broadcast 1
DHCP client bound to address 10.0.2.15 (3 ms)
*** Warning: no boot file name; using '0A00020F.img'
Using smc911x-0 device
TFTP from server 10.0.2.2; our IP address is 10.0.2.15
...

输入ctrl + A 后按 X退出 QEMU

编译 kernel 并仿真

编译

# git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git kernel && cd kernel 
wget https://mirrors.aliyun.com/linux-kernel/v4.x/linux-4.14.14.tar.xz && tar -xf linux-4.14.14.tar.xz && cd linux-4.14.14


export ARCH=arm
export CROSS_COMPILE=arm-linux-gnueabi-
make clean
make vexpress_defconfig
make menuconfig 
# 在配置中找到并去掉 Enable the L2x0 outer cache controller 取消，会导致仿真失败起不来
#    System Type-> Enable the L2x0 outer cache controller ，或搜索 CACHE_L2X0

make -j8

# schips @ ubuntu in ~/arm/sc/linux-4.1.11 [18:19:19]
$ make -j4
  HOSTCC  scripts/kallsyms
In file included from include/linux/compiler.h:54:0,
                 from include/uapi/linux/stddef.h:1,
                 from include/linux/stddef.h:4,
                 from ./include/uapi/linux/posix_types.h:4,
                 from include/uapi/linux/types.h:13,
                 from include/linux/types.h:5,
                 from include/linux/mod_devicetable.h:11,
                 from scripts/mod/devicetable-offsets.c:2:
include/linux/compiler-gcc.h:121:1: fatal error: linux/compiler-gcc7.h: No such file or directory
 #include gcc_header(__GNUC__)

如果遇到上面的情况，就说明是工具链太新了，要么换旧的工具链来匹配这个低版本的内核；要么就再找一个高版本的内核

仿真

qemu-system-arm -M vexpress-a9 -m 512M -kernel arch/arm/boot/zImage -dtb arch/arm/boot/dts/vexpress-v2p-ca9.dtb -nographic -append "console=ttyAMA0"
# 指定设备树，否则会失败

结果
不出意外的话会在 文件系统那里停下来。

Booting Linux on physical CPU 0x0
Linux version 4.14.14 (schips@ubuntu) (gcc version 7.4.1 20181213 [linaro-7.4-2019.02 revision 56ec6f6b99cc167ff0c2f8e1a2eed33b1edc85d4] (Linaro GCC 7.4-2019.02)) #1 SMP Sat Feb 22 21:14:14 CST 2020
CPU: ARMv7 Processor [410fc090] revision 0 (ARMv7), cr=10c5387d
CPU: PIPT / VIPT nonaliasing data cache, VIPT nonaliasing instruction cache
OF: fdt: Machine model: V2P-CA9
Memory policy: Data cache writeback
CPU: All CPU(s) started in SVC mode.
percpu: Embedded 16 pages/cpu @9fbae000 s36428 r8192 d20916 u65536
Built 1 zonelists, mobility grouping on.  Total pages: 130048
Kernel command line: console=ttyAMA0
log_buf_len individual max cpu contribution: 4096 bytes
log_buf_len total cpu_extra contributions: 12288 bytes
log_buf_len min size: 16384 bytes
log_buf_len: 32768 bytes
early log buf free: 14964(91%)
PID hash table entries: 2048 (order: 1, 8192 bytes)
Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
Memory: 509556K/524288K available (6144K kernel code, 402K rwdata, 1364K rodata, 1024K init, 169K bss, 14732K reserved, 0K cma-reserved)
Virtual kernel memory layout:
    vector  : 0xffff0000 - 0xffff1000   (   4 kB)
    fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
    vmalloc : 0xa0800000 - 0xff800000   (1520 MB)
    lowmem  : 0x80000000 - 0xa0000000   ( 512 MB)
    modules : 0x7f000000 - 0x80000000   (  16 MB)
      .text : 0x80008000 - 0x80700000   (7136 kB)
      .init : 0x80900000 - 0x80a00000   (1024 kB)
      .data : 0x80a00000 - 0x80a649b8   ( 403 kB)
       .bss : 0x80a6bd30 - 0x80a964b4   ( 170 kB)
SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
Hierarchical RCU implementation.
        RCU event tracing is enabled.
        RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=4.
RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
NR_IRQS: 16, nr_irqs: 16, preallocated irqs: 16
GIC CPU mask not found - kernel will fail to boot.

... (省略很多行)

input: ImExPS/2 Generic Explorer Mouse as /devices/platform/smb@4000000/smb@4000000:motherboard/smb@4000000:motherboard:iofpga@7,00000000/10007000.kmi/serio1/input/input2
VFS: Cannot open root device "(null)" or unknown-block(0,0): error -6
Please append a correct "root=" boot option; here are the available partitions:
1f00          131072 mtdblock0
 (driver?)
1f01           32768 mtdblock1
 (driver?)
Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.14.14 #1
Hardware name: ARM-Versatile Express
[<8010faac>] (unwind_backtrace) from [<8010bdd4>] (show_stack+0x10/0x14)
[<8010bdd4>] (show_stack) from [<8065a728>] (dump_stack+0x88/0x9c)
[<8065a728>] (dump_stack) from [<8011db88>] (panic+0xdc/0x248)
[<8011db88>] (panic) from [<809012b0>] (mount_block_root+0x1d4/0x2a8)
[<809012b0>] (mount_block_root) from [<809014a4>] (mount_root+0x120/0x128)
[<809014a4>] (mount_root) from [<809015fc>] (prepare_namespace+0x150/0x194)
[<809015fc>] (prepare_namespace) from [<80900eb4>] (kernel_init_freeable+0x248/0x258)
[<80900eb4>] (kernel_init_freeable) from [<8066d548>] (kernel_init+0x8/0x108)
[<8066d548>] (kernel_init) from [<80107968>] (ret_from_fork+0x14/0x2c)
---[ end Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)

基于 busybox 制作rootfs，并让 kernel 带有 rootfs 仿真

编译以及制作

git clone git://busybox.net/busybox.git && cd busybox

export ARCH=arm
export CROSS_COMPILE=arm-linux-gnueabi-

make defconfig
make menuconfig
#   勾选 Busybox Setting-> Build Options-> [*] Build static binary (no shared libs)    或者 搜索 CONFIG_STATIC

make -j4
make install

制作

##
#    Copyright By Schips, All Rights Reserved
#    https://gitee.com/schips/

#    File Name:  make_rootfs.sh
#    Created  :  Sat 22 Feb 2020 09:39:26 PM CST

##
#!/bin/sh
base=`pwd`
tmpfs=/_tmpfs

sudo rm -rf rootfs
sudo rm -rf ${tmpfs}
sudo rm -f a9rootfs.ext3
sudo mkdir rootfs
sudo cp _install/*  rootfs/ -raf

#sudo mkdir -p rootfs/{lib,proc,sys,tmp,root,var,mnt}
cd rootfs && sudo mkdir -p lib proc sys tmp root var mnt && cd ${base}

# 根据自己的实际情况, 找到并 拷贝 arm-gcc 中的 libc中的所有.so 库
sudo cp -arf /usr/cross/gcc-linaro-7.4.1-2019.02-i686_arm-linux-gnueabi/arm-linux-gnueabi/libc/lib/*so*  rootfs/lib

sudo cp app rootfs
sudo cp examples/bootfloppy/etc rootfs/ -arf
sudo sed -r  "/askfirst/ s/.*/::respawn:-/bin/sh/" rootfs/etc/inittab -i
sudo mkdir -p rootfs/dev/
sudo mknod rootfs/dev/tty1 c 4 1
sudo mknod rootfs/dev/tty2 c 4 2pro
sudo mknod rootfs/dev/tty3 c 4 3
sudo mknod rootfs/dev/tty4 c 4 4
sudo mknod rootfs/dev/console c 5 1
sudo mknod rootfs/dev/null c 1 3
sudo dd if=/dev/zero of=a9rootfs.ext3 bs=1M count=64
# 如果提示 "No space left on device" 证明 dd 命令中 count 的大小不够

sudo mkfs.ext3 a9rootfs.ext3
sudo mkdir -p ${tmpfs}
sudo chmod 777 ${tmpfs}
sudo mount -t ext3 a9rootfs.ext3 ${tmpfs}/ -o loop
sudo cp -r rootfs/*  ${tmpfs}/
sudo umount ${tmpfs}

仿真

qemu-system-arm -M vexpress-a9 -m 512M -kernel arch/arm/boot/zImage -dtb arch/arm/boot/dts/vexpress-v2p-ca9.dtb -nographic -append "root=/dev/mmcblk0 rw console=ttyAMA0" -sd a9rootfs.ext3
# 同样地，只要是 仿真kernle 就需要 指定设备树

结果（节选）

usbhid: USB HID core driver
mmcblk0: mmc0:4567 QEMU! 32.0 MiB
aaci-pl041 10004000.aaci: ARM AC'97 Interface PL041 rev0 at 0x10004000, irq 33
aaci-pl041 10004000.aaci: FIFO 512 entries
oprofile: using arm/armv7-ca9
NET: Registered protocol family 17
9pnet: Installing 9P2000 support
Registering SWP/SWPB emulation handler
rtc-pl031 10017000.rtc: setting system clock to 2020-02-22 14:26:16 UTC (1582381576)
ALSA device list:
  #0: ARM AC'97 Interface PL041 rev0 at 0x10004000, irq 33
input: ImExPS/2 Generic Explorer Mouse as /devices/platform/smb@4000000/smb@4000000:motherboard/smb@4000000:motherboard:iofpga@7,00000000/10007000.kmi/serio1/input/input2
EXT4-fs (mmcblk0): mounting ext3 file system using the ext4 subsystem
EXT4-fs (mmcblk0): mounted filesystem with ordered data mode. Opts: (null)
VFS: Mounted root (ext3 filesystem) on device 179:0.
Freeing unused kernel memory: 1024K
random: crng init done

Processing /etc/profile... Done

/ # ls
bin         lib         mnt         sbin        usr
dev         linuxrc     proc        sys         var
etc         lost+found  root        tmp

由于篇幅有限，我们在下一讲介绍更加高级的：《qemu运行uboot，在仿真中动态加载内核与文件系统》