Creating Custom rootfs and kernel Images

Creating Custom rootfs and kernel Images

Creating a kernel Image

Currently, Firecracker supports only uncompressed, ELF kernel images. You can build an uncompressed Linux kernel image with:

make vmlinux

Here's a quick step-by-step guide to building your own kernel that Firecracker can boot:

1. Get the Linux source code:

   git clone https://github.com/torvalds/linux.git linux.git
   cd linux.git

2. Check out the Linux version you want to build (e.g. we'll be using v4.20 here):

   git checkout v4.20

3. You will need to configure your Linux build. You can start from our recommended config - just copy it to .config (under the Linux sources dir). You can make interactive config adjustments using:

   make menuconfig

   Note: there are many ways of building a kernel config file, other than menuconfig. You are free to use whichever one you choose.

4. Build the uncompressed kernel image:

   make vmlinux

5. Upon a successful build, you can find the uncompressed kernel image under ./vmlinux.

Creating a rootfs Image

A rootfs image is just a file system image, that hosts at least an init system. For instance, our getting started guide uses an EXT4 FS image with OpenRC as an init system. Note that, whichever file system you choose to use, support for it will have to be compiled into the kernel, so it can be mounted at boot time.

To build an EXT4 image:

1. Prepare a properly-sized file. We'll use 50MiB here, but this depends on how much data you'll want to fit inside:

   dd if=/dev/zero of=rootfs.ext4 bs=1M count=50

2. Create an empty file system on the file you created:

   mkfs.ext4 rootfs.ext4

You now have an empty EXT4 image in rootfs.ext4, so let's prepare to populate it. First, you'll need to mount this new file system, so you can easily access its contents:

mkdir /tmp/my-rootfs
sudo mount rootfs.ext4 /tmp/my-rootfs

The minimal init system would be just an ELF binary, placed at /sbin/init. The final step in the Linux boot process executes /sbin/init and expects it to never exit. More complex init systems build on top of this, providing service configuration files, startup / shutdown scripts for various services, and many other features.

For the sake of simplicity, let's set up an Alpine-based rootfs, with OpenRC as an init system. To that end, we'll use the official Docker image for Alpine Linux:

1. First, let's start the Alpine container, bind-mounting the EXT4 image created earlier, to /my-rootfs:

   docker run -it --rm -v /tmp/my-rootfs:/my-rootfs alpine

2. Then, inside the container, install the OpenRC init system, and some basic tools:

   apk add openrc
   apk add util-linux

3. And set up userspace init (still inside the container shell):

   # Set up a login terminal on the serial console (ttyS0):
   ln -s agetty /etc/init.d/agetty.ttyS0
   echo ttyS0 > /etc/securetty
   rc-update add agetty.ttyS0 default
   
   # Make sure special file systems are mounted on boot:
   rc-update add devfs boot
   rc-update add procfs boot
   rc-update add sysfs boot
   
   # Then, copy the newly configured system to the rootfs image:
   for d in bin etc lib root sbin usr; do tar c "/$d" | tar x -C /my-rootfs; done
   for dir in dev proc run sys var; do mkdir /my-rootfs/${dir}; done
   
   # All done, exit docker shell
   exit

4. Finally, unmount your rootfs image:

   sudo umount /tmp/my-rootfs

You should now have a kernel image (vmlinux) and a rootfs image (rootfs.ext4), that you can boot with Firecracker.

root@ubuntu:~/rootfs/my# dd if=/dev/zero of=rootfs.ext4 bs=1M count=50
50+0 records in
50+0 records out
52428800 bytes (52 MB, 50 MiB) copied, 0.080313 s, 653 MB/s
root@ubuntu:~/rootfs/my# ls
rootfs.ext4
root@ubuntu:~/rootfs/my# mkfs.ext4 rootfs.ext4
mke2fs 1.44.1 (24-Mar-2018)
Discarding device blocks: done                            
Creating filesystem with 51200 1k blocks and 12824 inodes
Filesystem UUID: 06211353-f960-45df-a7a4-f0b2c2e3bebf
Superblock backups stored on blocks: 
        8193, 24577, 40961

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: done

root@ubuntu:~/rootfs/my# mkdir /tmp/my-rootfs
root@ubuntu:~/rootfs/my#  mount rootfs.ext4 /tmp/my-rootfs
root@ubuntu:~/rootfs/my# docker run -it --rm -v /tmp/my-rootfs:/my-rootfs alpine

root@ubuntu:~/rootfs/my# docker run -it --rm -v /tmp/my-rootfs:/my-rootfs alpine
/ # apk add openrc
fetch http://dl-cdn.alpinelinux.org/alpine/v3.12/main/aarch64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.12/community/aarch64/APKINDEX.tar.gz
(1/1) Installing openrc (0.42.1-r11)
Executing openrc-0.42.1-r11.post-install
Executing busybox-1.31.1-r16.trigger
OK: 8 MiB in 15 packages
/ # apk add util-linux
(1/17) Installing libblkid (2.35.2-r0)
(2/17) Installing blkid (2.35.2-r0)
(3/17) Installing libcap-ng (0.7.10-r1)
(4/17) Installing setpriv (2.35.2-r0)
(5/17) Installing libmount (2.35.2-r0)
(6/17) Installing libsmartcols (2.35.2-r0)
(7/17) Installing findmnt (2.35.2-r0)
(8/17) Installing mcookie (2.35.2-r0)
(9/17) Installing ncurses-terminfo-base (6.2_p20200523-r0)
(10/17) Installing ncurses-libs (6.2_p20200523-r0)
(11/17) Installing hexdump (2.35.2-r0)
(12/17) Installing lsblk (2.35.2-r0)
(13/17) Installing libuuid (2.35.2-r0)
(14/17) Installing libfdisk (2.35.2-r0)
(15/17) Installing sfdisk (2.35.2-r0)
(16/17) Installing cfdisk (2.35.2-r0)
(17/17) Installing util-linux (2.35.2-r0)
Executing busybox-1.31.1-r16.trigger
OK: 14 MiB in 32 packages
/ # 

OK: 14 MiB in 32 packages
/ # ln -s agetty /etc/init.d/agetty.ttyS0
/ # echo ttyS0 > /etc/securetty
/ # rc-update add agetty.ttyS0 default
 * service agetty.ttyS0 added to runlevel default
/ # rc-update add devfs boot
 * service devfs added to runlevel boot
/ # rc-update add procfs boot
 * service procfs added to runlevel boot
/ # rc-update add sysfs boot
 * service sysfs added to runlevel boot
/ # for d in bin etc lib root sbin usr; do tar c "/$d" | tar x -C /my-rootfs; done
tar: removing leading '/' from member names
tar: removing leading '/' from member names
tar: removing leading '/' from member names
tar: removing leading '/' from member names
tar: removing leading '/' from member names
tar: removing leading '/' from member names
/ # ls
bin        etc        lib        mnt        opt        root       sbin       sys        usr
dev        home       media      my-rootfs  proc       run        srv        tmp        var
/ # ls my-rootfs/
bin         etc         lib         lost+found  root        sbin        usr
/ # ls my-rootfs/bin/
arch           df             getopt         ln             mv             rm             touch
ash            dmesg          grep           login          netstat        rmdir          true
base64         dnsdomainname  gunzip         ls             nice           run-parts      umount
bbconfig       dumpkmap       gzip           lsblk          pidof          sed            uname
busybox        echo           hostname       lzop           ping           setpriv        usleep
cat            ed             ionice         makemime       ping6          setserial      watch
chgrp          egrep          iostat         mkdir          pipe_progress  sh             wdctl
chmod          false          ipcalc         mknod          printenv       sleep          zcat
chown          fatattr        kbd_mode       mktemp         ps             stat
conspy         fdflush        kill           more           pwd            stty
cp             fgrep          link           mount          rc-status      su
date           findmnt        linux32        mountpoint     reformime      sync
dd             fsync          linux64        mpstat         rev            tar
/ # 

root@ubuntu:~# chroot /mnt /bin/bash
chroot /mnt /bin/bash
chroot: failed to run command ‘/bin/bash’: No such file or directory

修改 密码

cp -av  /lib/aarch64-linux-gnu/ld-2.27.so      /tmp/my-rootfs/lib/aarch64-linux-gnu

root@ubuntu:~# ldd /tmp/my-rootfs/bin/bash
ldd /tmp/my-rootfs/bin/bash
        linux-vdso.so.1 (0x0000ffffb51d5000)
        libtinfo.so.5 => /lib/aarch64-linux-gnu/libtinfo.so.5 (0x0000ffffb5059000)
        libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x0000ffffb5044000)
        libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000ffffb4eea000)
        /lib/ld-linux-aarch64.so.1 (0x0000ffffb51aa000)

root@ubuntu:~# ls /lib/ld-linux-aarch64.so.1 -al
ls /lib/ld-linux-aarch64.so.1 -al
lrwxrwxrwx 1 root root 28 Jun  5 01:25 /lib/ld-linux-aarch64.so.1 -> aarch64-linux-gnu/ld-2.27.so

root@ubuntu:/home/ubuntu# for i in `ldd /bin/bash`;do echo $i;done |grep -v = |grep -v 0x |grep /|xargs ls -l
lrwxrwxrwx 1 root root 12 Jun  5 01:25 /lib/aarch64-linux-gnu/libc.so.6 -> libc-2.27.so
lrwxrwxrwx 1 root root 13 Jun  5 01:25 /lib/aarch64-linux-gnu/libdl.so.2 -> libdl-2.27.so
lrwxrwxrwx 1 root root 15 Sep 24 17:53 /lib/aarch64-linux-gnu/libtinfo.so.5 -> libtinfo.so.5.9
lrwxrwxrwx 1 root root 28 Jun  5 01:25 /lib/ld-linux-aarch64.so.1 -> aarch64-linux-gnu/ld-2.27.so
root@ubuntu:/home/ubuntu# 

root@ubuntu:~# chroot /tmp/my-rootfs
chroot /tmp/my-rootfs
bash-4.4#