在MPSoC单板上运行Docker
目录
作者
Hank FU 付汉杰 hankf@xilinx.com
测试环境
Xilinx ZCU106 单板
Xilinx VCU TRD2020.1
Linux 内核配置
根据文档Docker on Zynq Ultrascale+ (Xilinx Yocto Flow)，在PetaLinux工程的文件project-spec/meta-user/recipes-kernel/linux/linux-xlnx/user.cfg里添加下列配置项。
CONFIG_NAMESPACES=y
CONFIG_NET_NS=y
CONFIG_PID_NS=y
CONFIG_IPC_NS=y
CONFIG_UTS_NS=y
CONFIG_CGROUP_CPUACCT=y
CONFIG_CGROUP_DEVICE=y
CONFIG_CGROUP_FREEZER=y
CONFIG_CGROUP_SCHED=y
CONFIG_CPUSETS=y
CONFIG_MEMCG=y
CONFIG_VETH=y
CONFIG_IP_NF_TARGET_MASQUERADE=y
CONFIG_NETFILTER_XT_MATCH_ADDRTYPE=y
CONFIG_NETFILTER_XT_MATCH_IPVS=y
CONFIG_IP_NF_NAT=y
CONFIG_USER_NS=y
CONFIG_SECCOMP=y
CONFIG_CGROUP_PIDS=y
CONFIG_MEMCG_SWAP=y
CONFIG_MEMCG_SWAP_ENABLED=y
CONFIG_BLK_CGROUP=y
CONFIG_BLK_DEV_THROTTLING=y
CONFIG_CFQ_GROUP_IOSCHED=y
CONFIG_CGROUP_PERF=y
CONFIG_CGROUP_HUGETLB=y
CONFIG_NET_CLS_CGROUP=y
CONFIG_CGROUP_NET_PRIO=y
CONFIG_CFS_BANDWIDTH=y
CONFIG_FAIR_GROUP_SCHED=y
CONFIG_RT_GROUP_SCHED=y
CONFIG_IP_NF_TARGET_REDIRECT=y
CONFIG_IP_VS=y
CONFIG_IP_VS_NFCT=y
CONFIG_IP_VS_PROTO_TCP=y
CONFIG_IP_VS_PROTO_UDP=y
CONFIG_IP_VS_RR=y
CONFIG_EXT3_FS_XATTR=y
CONFIG_EXT3_FS_POSIX_ACL=y
CONFIG_EXT3_FS_SECURITY=y
CONFIG_VXLAN=y
CONFIG_INET_ESP=y
CONFIG_IPVLAN=y
CONFIG_MACVLAN=y
CONFIG_DUMMY=y
CONFIG_NF_NAT_FTP=y
CONFIG_NF_CONNTRACK_FTP=y
CONFIG_NF_NAT_TFTP=y
CONFIG_NF_CONNTRACK_TFTP=y
CONFIG_AUFS_FS=y
CONFIG_BTRFS_FS_POSIX_ACL=y
CONFIG_BLK_DEV_DM=y
CONFIG_DM_THIN_PROVISIONING=y
CONFIG_OVERLAY_FS=y
CONFIG_MD=y
CONFIG_NET_SCHED=y
CONFIG_NET_L3_MASTER_DEV=y
CONFIG_BRIDGE=m
# CONFIG_EXT2_FS is not set
CONFIG_EXT4_USE_FOR_EXT2=y
Linux 文件系统配置
在PetaLinux工程里选择Packagegroup-petalinux-ocicontainers。
启动命令
使用命令“dockerd &”可以启动Docker。运行之前，要先导出变量DOCKER_RAMDISK。
export DOCKER_RAMDISK=true
测试命令
使用命令“docker run --rm hello-world”可以运行一个简单容器。
常见问题
网络权限
单板要有访问网络权限，要能下载Docker的镜像。
root@vcu_trd:~# docker run --rm hello-world
Unable to find image 'hello-world:latest' locally
WARN[2020-11-11T09:47:14.257031091Z] Could not get operating system name: Error opening /usr/lib/os-release: open /usr/lib/os-release: no such file or directory
WARN[2020-11-11T09:47:14.286697565Z] Error getting v2 registry: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on [::1]:53: read udp [::1]:41143->[::1]:53: read: connection refused
INFO[2020-11-11T09:47:14.286851881Z] Attempting next endpoint for pull after error: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on [::1]:53: read udp [::1]:41143->[::1]:53: read: connection refused
ERRO[2020-11-11T09:47:14.287032029Z] Handler for POST /v1.40/images/create returned error: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on [::1]:53: read udp [::1]:41143->[::1]:53: read: connection refused
docker: Error response from daemon: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on [::1]:53: read udp [::1]:41143->[::1]:53: read: connection refused.
See 'docker run --help'.
存储空间
单板上可能缺乏足够存储空间。可以使用dockerd的选项data-root指定一个有足够空间的目录给dockerd使用。在ZCU106单板上，使用ramdisk时，最好进入/run/目录运行docker.
#ERROR Download failed: write /var/lib/docker/tmp/GetImageBlob091922966: no space left on device
docker启动失败
另外，docker启动过程中，可能没能启动containerd，出现下列错误。
root@vcu_trd:/run/test# docker run --rm hello-world
docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.
See 'docker run --help'.
root@vcu_trd:/run/test# docker-compose pull
ERROR: Couldn't connect to Docker daemon at http+docker://localhost - is it running?
If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable.
这时候可以杀死dockerd进程，再次启动dockerd。可以借助下列命令来快速杀死dockerd进程和启动dockerd。
export DOCKER_RAMDISK=true
echo -e "
DOCKER_RAMDISK flag: $DOCKER_RAMDISK
" | tee -a $docker_log_file
CONTAINERD_PID=$(ps -A | grep -v grep | grep containerd | grep -v .sh |awk '{printf $1}')
DOCKERD_PID=$(ps -A | grep -v grep | grep dockerd | grep -v .sh |awk '{printf $1}')
echo -e "containerd PID: $CONTAINERD_PID
" | tee -a $docker_log_file
echo -e "dockerd PID: $DOCKERD_PID
" | tee -a $docker_log_file
if [ ! "$CONTAINERD_PID" = "" ]; then
echo -e "kill containerd PID: $CONTAINERD_PID
" | tee -a $docker_log_file
kill $CONTAINERD_PID
fi
if [ ! "$DOCKERD_PID" = "" ]; then
echo -e "kill dockerd PID: $DOCKERD_PID
" | tee -a $docker_log_file
kill $DOCKERD_PID
# Error starting daemon: pid file found, ensure docker is not running or delete /var/run/docker.pid
rm -f /var/run/docker.pid | tee -a $docker_log_file
fi
sleep 1
echo -e "
Start Docker...
" | tee -a $docker_log_file
ls -l /run/docker | tee -a $docker_log_file
mkdir -p /run/docker/data | tee -a $docker_log_file
dockerd --data-root /run/docker/data | tee -a $docker_log_file &
sleep 12
ps -A | grep docker | tee -a $docker_log_file
ps -A | grep containerd | tee -a $docker_log_file
ls -l /var/run/docker.sock | tee -a $docker_log_file
文件/var/run/docker.pid已经存在
启动启动dockerd时，可能报错，说文件/var/run/docker.pid已经存在。重启动dockerd时，需要先删除文件/var/run/docker.pid。
# Error starting daemon: pid file found, ensure docker is not running or delete /var/run/docker.pid
docker: Error response from daemon: OCI runtime create failed
PetaLinux编译的Linux启动后，已经启动dockerd。直接运行docker的hello-world ，出现下列错误。
root@vcu_trd:~# ps -A | grep -v grep | grep containerd
937 ? 00:00:00 containerd
root@vcu_trd:~# ps -A | grep -v grep | grep dockerd
926 ? 00:00:00 dockerd
root@vcu_trd:/run/test# docker run --rm hello-world
docker: Error response from daemon: OCI runtime create failed: container_linux.go:346: starting container process caused "process_linux.go:449: container init caused "rootfs_linux.go:109: jailing process inside rootfs caused \"pivot_root invalid argument\""": unknown.
ERRO[0001] error waiting for container: context canceled
借助上面描述的脚本设置DOCKER_RAMDISK为true，并重启动dockerd，运行docker的hello-world成功。
运行记录如下：
Run Docker hello-world...
Unable to find image 'hello-world:latest' locally
WARN[2020-11-12T01:47:00.103866590Z] Could not get operating system name: Error opening /usr/lib/os-release: open /usr/lib/os-release: no such file or directory
latest: Pulling from library/hello-world
256ab8fe8778: Pulling fs layer
256ab8fe8778: Download complete
256ab8fe8778: Pull complete
Digest: sha256:8c5aeeb6a5f3ba4883347d3747a7249f491766ca1caa47e5da5dfcf6b9b717c0
Status: Downloaded newer image for hello-world:latest
INFO[2020-11-12T01:47:10.600246198Z] shim containerd-shim started address="/containerd-shim/moby/b506434ba15b4ba4896d909c1b0bd24fa9b788c6a75f26ea53da7b9679694fd8/shim.sock" debug=false pid=1618
Hello from Docker!
This message shows that your installation appears to be working correctly.
To generate this message, Docker took the following steps:
1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
(arm64v8)
3. The Docker daemon created a new container from that image which runs the
executable that produces the output you are currently reading.
4. The Docker daemon streamed that output to the Docker client, which sent it
to your terminal.
To try something more ambitious, you can run an Ubuntu container with:
$ docker run -it ubuntu bash
Share images, automate workflows, and more with a free Docker ID:
https://hub.docker.com/
For more examples and ideas, visit:
https://docs.docker.com/get-started/
INFO[2020-11-12T01:47:11.352948396Z] shim reaped id=b506434ba15b4ba4896d909c1b0bd24fa9b788c6a75f26ea53da7b9679694fd8
INFO[2020-11-12T01:47:11.362202321Z] ignoring event module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
WARN[2020-11-12T01:47:11.453657929Z] b506434ba15b4ba4896d909c1b0bd24fa9b788c6a75f26ea53da7b9679694fd8 cleanup: failed to unmount IPC: umount /run/docker/data/containers/b506434ba15b4ba4896d909c1b0bd24fa9b788c6a75f26ea53da7b9679694fd8/mounts/shm, flags: 0x2: no such file or directory
pivot_root invalid argument: unknown
PetaLinux编译的Linux启动后，运行docker的hello-world ，出现下列错误“pivot_root invalid argument: unknown”。借助上面描述的脚本设置DOCKER_RAMDISK为true，并重启动dockerd，运行docker的hello-world成功。
root@vcu_trd:/run/test# export DOCKER_RAMDISK=true
root@vcu_trd:/run/test# echo -e "
DOCKER_RAMDISK flag: $DOCKER_RAMDISK
" | tee -a $docker_log_file
DOCKER_RAMDISK flag: true
root@vcu_trd:/run/test# docker run --rm hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
256ab8fe8778: Pull complete
Digest: sha256:8c5aeeb6a5f3ba4883347d3747a7249f491766ca1caa47e5da5dfcf6b9b717c0
Status: Downloaded newer image for hello-world:latest
docker: Error response from daemon: OCI runtime create failed: container_linux.go:346: starting container process caused "process_linux.go:449: container init caused "rootfs_linux.go:109: jailing process inside rootfs caused \"pivot_root invalid argument\""": unknown.