Running Kata Containers with Docker on IBM Power Systems

Containers have taken off for good reason — they’re light, they’re performant, and they’re easy to integrate. The problem is, the traditional containers architecture involves a shared kernel between the host operating system and the guest containers, leaving the other container workloads in a cluster vulnerable if one container is comprised. This issue is one of the big drivers behind Kata Containers.

Kata Containers is a new open source project building extremely lightweight virtual machines that seamlessly plug into the containers ecosystem. It provides the speed of containers and the security of VMs. It is designed to be architecture agnostic, run on multiple hypervisors and be compatible with the OCI specification for Docker containers and CRI for Kubernetes. Kata Containers combines technology from Intel® Clear Containers and Hyper runV. The code is hosted on Github under the Apache 2 license and the project is managed by the OpenStack Foundation.

In Kata Containers, each container has its own lightweight virtual machine with a mini-kernel, providing container isolation via hardware virtualization. This hardens the security layer, and also provides the possibility of containers-as-a-service and software-as-a-service models since mutually untrusting tenants can be put on the same cluster.

Source: http://superuser.openstack.org/wp-content/uploads/2018/05/katacontainers-image2.jpg

The Kata Containers project has six components: Agent, Runtime, Proxy, Shim, Kernel and Qemu.

Kata-runtime: kata-runtime creates a QEMU/KVM virtual machine for each container or pod, the Docker engine or Kubernetes’ kubelet creates respectively.

Kata-agent: kata-agent is a process running in the guest as a supervisor for managing containers and processes running within those containers.

Kata-proxy: kata-proxy is a process offering access to the VM kata-agent to multiple kata-shim and kata-runtime clients associated with the VM. Its main role is to route the I/O streams and signals between each kata-shim instance and the kata-agent. kata-proxy connects to kata-agent on a unix domain socket that kata-runtime provides while spawning kata-proxy. kata-proxy uses yamux to multiplex gRPC requests on its connection to the kata-agent

Kata-Shim: A container process reaper, such as Docker’s containerd-shim or CRI-O’s conmon, is designed around the assumption that it can monitor and reap the actual container process.

Guest-Kernel: The guest kernel is passed to the hypervisor and used to boot the virtual machine. The default kernel provided in Kata Containers is highly optimized for kernel boot time and minimal memory footprint, providing only those services required by a container workload. This is based on a very current upstream Linux kernel.

All the Kata Container components communicate with each other using gRPC protocol.

Source: https://katacontainers.io/

---

In this article, you will learn how to use Kata Containers on IBM Power systems.

#Prerequisites:

You need to install golang version 1.8.3 or newer, make, gcc, qemu.

#Steps:

1.Build and install the Kata Containers runtime

$ go get -d -u github.com/kata-containers/runtime$ cd $GOPATH/src/github.com/kata-containers/runtime$ make && sudo -E PATH=$PATH make install

The build will create the following:

runtime binary: /usr/local/bin/kata-runtime

configuration file: /usr/share/defaults/kata-containers/configuration.toml

2. Check if your system is capable of creating a Kata Container:

$ sudo kata-runtime kata-check

If your system is not able to run Kata Containers, the previous command will error out and explain why.

3. Configure to use initrd image

$ sudo sed -i ‘s/^(image =.*)/# 1/g’ /usr/share/defaults/kata-containers/configuration.toml

4. Enable full debug

$ sudo sed -i -e ‘s/^# *(enable_debug).*=.*$/1 = true/g’ /usr/share/defaults/kata-containers/configuration.toml$ sudo sed -i -e ‘s/^kernel_params = “(.*)”/kernel_params = “1 agent.log=debug”/g’ /usr/share/defaults/kata-containers/configuration.toml

5. Build and install Kata proxy

$ go get -d -u github.com/kata-containers/proxy$ cd $GOPATH/src/github.com/kata-containers/proxy && make && sudo make install

6. Build and install Kata shim

$ go get -d -u github.com/kata-containers/shim$ cd $GOPATH/src/github.com/kata-containers/shim && make && sudo make install

7. Get the osbuilder

$ go get -d -u github.com/kata-containers/osbuilder

8. Build a custom Kata agent — OPTIONAL

$ go get -d -u github.com/kata-containers/agent$ cd $GOPATH/src/github.com/kata-containers/agent && make

9. Create an initrd image

$ export ROOTFS_DIR=”${GOPATH}/src/github.com/kata-containers/osbuilder/rootfs-builder/rootfs"$ sudo rm -rf ${ROOTFS_DIR}$ cd $GOPATH/src/github.com/kata-containers/osbuilder/rootfs-builder$ script -fec ‘sudo -E GOPATH=$GOPATH AGENT_INIT=yes USE_DOCKER=true ./rootfs.sh ${distro}’

AGENT_INIT controls if the guest image uses kata agent as the guest init process. When you create an initrd image, always set AGENT_INIT to yes.

You MUST choose one of alpine, centos and fedora for ${distro}.

Optionally, add your custom agent binary to the rootfs with the following:

$ sudo install -o root -g root -m 0550 -T ../../agent/kata-agent ${ROOTFS_DIR}/sbin/init

10. Build an initrd image

$ cd $GOPATH/src/github.com/kata-containers/osbuilder/initrd-builder$ script -fec ‘sudo -E AGENT_INIT=yes USE_DOCKER=true ./initrd_builder.sh ${ROOTFS_DIR}’

11. Install the initrd image

$ commit=$(git log — format=%h -1 HEAD)$ date=$(date +%Y-%m-%d-%T.%N%z)$ image=”kata-containers-initrd-${date}-${commit}”$ sudo install -o root -g root -m 0640 -D kata-containers-initrd.img “/usr/share/kata-containers/${image}”$ (cd /usr/share/kata-containers && sudo ln -sf “$image” kata-containers-initrd.img)

12. Install guest kernel images

As a prerequisite, you need to install libelf-dev and bc. Otherwise, you will not be able to build the kernel from sources.

$ go get github.com/kata-containers/tests$ cd $GOPATH/src/github.com/kata-containers/tests/.ci$ kernel_arch=”$(./kata-arch.sh)”$ kernel_dir=”$(./kata-arch.sh — kernel)”$ tmpdir=”$(mktemp -d)”$ pushd “$tmpdir”$ curl -L https://raw.githubusercontent.com/kata-containers/packaging/master/kernel/configs/${kernel_arch}_kata_kvm_4.14.x -o .config$ kernel_version=$(grep “Linux/[${kernel_arch}]*” .config | cut -d’ ‘ -f3 | tail -1)$ kernel_tar_file=”linux-${kernel_version}.tar.xz”$ kernel_url=”https://cdn.kernel.org/pub/linux/kernel/v$(echo $kernel_version | cut -f1 -d.).x/${kernel_tar_file}”$ curl -LOk ${kernel_url}$ tar -xf ${kernel_tar_file}$ mv .config “linux-${kernel_version}”$ pushd “linux-${kernel_version}”$ make ARCH=${kernel_dir} -j$(nproc)$ kata_kernel_dir=”/usr/share/kata-containers”$ kata_vmlinuz=”${kata_kernel_dir}/kata-vmlinuz-${kernel_version}.container”$ [ $kernel_arch = ppc64le ] && kernel_file=”$(realpath ./vmlinux)” || kernel_file=”$(realpath arch/${kernel_arch}/boot/bzImage)”$ sudo install -o root -g root -m 0755 -D “${kernel_file}” “${kata_vmlinuz}”$ sudo ln -sf “${kata_vmlinuz}” “${kata_kernel_dir}/vmlinuz.container”$ kata_vmlinux=”${kata_kernel_dir}/kata-vmlinux-${kernel_version}”$ sudo install -o root -g root -m 0755 -D “$(realpath vmlinux)” “${kata_vmlinux}”$ sudo ln -sf “${kata_vmlinux}” “${kata_kernel_dir}/vmlinux.container”$ popd$ popd$ rm -rf “${tmpdir}”

13. Run Kata Containers with Docker

Update Docker configuration

$ dir=/etc/systemd/system/docker.service.d$ file=”$dir/kata-containers.conf”$ sudo mkdir -p “$dir”$ sudo test -e “$file” || echo -e “[Service]
Type=simple
ExecStart=
ExecStart=/usr/bin/dockerd -D — default-runtime runc” | sudo tee “$file”$ sudo grep -q “kata-runtime=” $file || sudo sed -i ‘s!^(ExecStart=[^$].*$)!1 — add-runtime kata-runtime=/usr/local/bin/kata-runtime!g’ “$file”$ sudo systemctl daemon-reload$ sudo systemctl restart docker

14. Create a Kata Container using Docker

$ sudo docker run -ti — runtime kata-runtime busybox sh

15. Check the logs in-case of failure

$ journalctl -q -o cat -a -t kata-runtime
$ journalctl -q -o cat -a -t kata-proxy