kas-container and QEMU

KAS

KAS [1] is a setup tool for bitbake based projects such as Yocto. There are many similiar alternatives out there and I've tried most of them, but my absolute favorite is KAS.

In order to use KAS, you have to setup a YAML file to contain information about your machine, distribution, meta layers and local configuration. Here is a small example configuration copied from the KAS documentation:

# Every file needs to contain a header, that provides kas with information
# about the context of this file.
header:
  # The `version` entry in the header describes for which configuration
  # format version this file was created for. It's used by kas to figure
  # out if it's compatible with this file. The version is an integer that
  # is increased on every format change.
  version: x
# The machine as it's written into the `local.conf` of bitbake.
machine: qemux86-64
# The distro name as it's written into the `local.conf` of bitbake.
distro: poky
repos:
  # This entry includes the repository where the config file is located
  # to the bblayers.conf:
  meta-custom:
  # Here we include a list of layers from the poky repository to the
  # bblayers.conf:
  poky:
    url: "https://git.yoctoproject.org/git/poky"
    commit: 89e6c98d92887913cadf06b2adb97f26cde4849b
    layers:
      meta:
      meta-poky:
      meta-yocto-bsp:

bblayers_conf_header:
  meta-custom: |
    POKY_BBLAYERS_CONF_VERSION = "2"
    BBPATH = "${TOPDIR}"
    BBFILES ?= ""    
local_conf_header:
  meta-custom: |
    PATCHRESOLVE = "noop"
    CONF_VERSION = "1"
    IMAGE_FSTYPES = "tar"    

That is all you need to start to build your distribution:

kas build kas-project.yml

Yocto and QEMU

Yocto [3] let you emulate and virtualize your images you have built using the Yocto Project. It makes use of the runqemu help script to find the build artifacts and setup everything to start the emulator.

To add support for qemu you have to include the qemuboot image class into your local.conf:

IMAGE_CLASSES += "qemuboot"

When you include the image class to your project it will generate an *.qemuboot.conf file among your artifacts and contains the configuration for runqemu.

The configuration file [4] has many variables that can be overridden in your local.conf. Here are some of them:

• QB_SYSTEM_NAME - qemu name, e.g., "qemu-system-i386"
• QB_OPT_APPEND - options to append to qemu, e.g., "-device usb-mouse"
• QB_DEFAULT_KERNEL - default kernel to boot
• QB_DEFAULT_FSTYPE - default FSTYPE to boot
• QB_MEM - memory
• QB_MACHINE - qemu machine
• QB_CPU - qemu cpu
• QB_SMP - amount of CPU cores inside qemu guest, each mapped to a thread on the host
• QB_KERNEL_CMDLINE_APPEND - options to append to kernel's -append option
• QB_DTB - qemu dtb name
• QB_AUDIO_DRV - qemu audio driver
• QB_AUDIO_OPT - qemu audio option
• QB_RNG - Pass-through for host random number generator
• QB_KERNEL_ROOT - kernel's root gets passed to the kernel.
• QB_NETWORK_DEVICE - network device
• QB_TAP_OPT - network option for 'tap' mode
• QB_SLIRP_OPT - network option for SLIRP mode
• QB_CMDLINE_IP_SLIRP - If QB_NETWORK_DEVICE adds more than one network interface to qemu
• QB_ROOTFS_OPT - used as rootfs
• QB_SERIAL_OPT - serial port
• QB_TCPSERIAL_OPT - tcp serial port option
• QB_ROOTFS_EXTRA_OPT - extra options to be appended to the rootfs device in case there is none specified by QB_ROOTFS_OPT.
• QB_GRAPHICS - QEMU video card type
• QB_NFSROOTFS_EXTRA_OPT - extra options to be appended to the nfs rootfs options in kernel boot arg

QB_MEM is set to -m 256 as default. I had to increase it a lot as I'm running Azure in my setup.

kas-container and QEMU

Running qemu in kas-container is pretty straight forward, there is just a few things to keep in mind if you need a network connection.

By default, qemu is mapping a TAP [5] interface to the emulated environment to be able to route traffic to your network. First, it requires you to have the tap module loaded on the host:

sudo modprobe tap

runqemu then uses iptables to setup NAT routing for the TAP interface. Unfortunately, iptables is not included in the docker image so we have to add it. You find the Dockerfile in the KAS repository [6].

diff --git a/Dockerfile b/Dockerfile
index 0e79cb5..c331e45 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -33,7 +33,7 @@ RUN apt-get update && \
         python3-pip python3-setuptools python3-wheel python3-yaml python3-distro python3-jsonschema \
         python3-newt python3-colorlog python3-kconfiglib \
         gosu lsb-release file vim less procps tree tar bzip2 zstd pigz lz4 unzip tmux libncurses-dev \
-        git-lfs mercurial iproute2 ssh-client telnet curl rsync gnupg awscli sudo \
+        git-lfs mercurial iproute2 ssh-client telnet curl rsync gnupg awscli sudo iptables \
         socat bash-completion && \
     apt-get clean && \
     rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
diff --git a/kas-container b/kas-container
index 8fa2d16..2cd88e1 100755
--- a/kas-container
+++ b/kas-container
@@ -135,7 +135,7 @@ run_clean() {
        fi
 }

Now we have to expose the tap device into the container and give it permission to create network rules. This is done by send --device /dev/net/tun:/dev/net/tun and --cap-add=NET_ADMIN as arguments to docker.

BE AWARE OF THAT YOU NOW ARE GIVING THE CONTAINER NET_ADMIN CAPABILITIES!

Now we are ready to start a shell:

kas-container --docker-args "--device /dev/net/tun:/dev/net/tun --cap-add=NET_ADMIN"  shell ./kas-project.yml

As we do not export a framebuffer device to the container, we start runqemu with the nographic parameter.

runqemu nographic

[  OK  ] Finished IPv6 Packet Filtering Framework.
[  OK  ] Finished IPv4 Packet Filtering Framework.
[  OK  ] Reached target Preparation for Network.
         Starting Network Configuration...
[  OK  ] Finished OpenSSH Key Generation.
[  OK  ] Started D-Bus System Message Bus.
[  OK  ] Started User Login Management.
[  OK  ] Started Network Configuration.
         Starting Wait for Network to be Configured...
         Starting Network Name Resolution...
[  OK  ] Started Network Name Resolution.
[  OK  ] Reached target Network.
[  OK  ] Reached target Host and Network Name Lookups.
         Starting containerd container runtime...
         Starting DNS forwarder and DHCP server...
         Starting Hostapd IEEE 802.…A2/EAP/RADIUS Authenticator...
[  OK  ] Started DNS forwarder and DHCP server.
[  OK  ] Started containerd container runtime.

Test Distro v1 qemuarm64 ttyAMA0

qemuarm64 login:

Here we go.

References