Install b4

First we have to install b4 on our system.

B4 is propbably already available in your distribution:

Archlinux:

$ pacman -Sy b4

Ubuntu:

$ apt-get install b4

Fedora:

$ dnf install b4

Or whatever package manager you use.

It is also possible to install it with pip:

$ python3 -m pip install --user b4

And of course, run it directly from the git repository [3] :

$ git clone https://git.kernel.org/pub/scm/utils/b4/b4.git
$ cd b4
$ git submodule update --init
$ pip install --user -r requirements.txt

The workflow

These steps is more or less copied directly from the documentation [1].

1. Prepare your patch series by using b4 prep and queueing your commits. Use git rebase -i to arrange the commits in the right order and to write good commit messages.
2. Prepare your cover letter using b4 prep --edit-cover. You should provide a good overview of what your series does and why you think it will improve the current code.
3. When you are almost ready to send, use b4 prep --auto-to-cc to collect the relevant addresses from your commits. If your project uses a MAINTAINERS file, this will also perform the required query to figure out who should be included on your patch series submission.
4. Review the list of addresses that were added to the cover letter and, if you know what you're doing, remove any that you think are unnecessary.
5. Send your series using b4 send. This will automatically reroll your series to the next version and add changelog entries to the cover letter.
6. Await code review and feedback from maintainers.
7. Apply any received code-review trailers using b4 trailers -u.
8. Use git rebase -i to make any changes to the code based on the feedback you receive. Remember to record these changes in the cover letter's changelog.
9. Unless series is accepted upstream, GOTO 3.
10. Clean up obsolete prep-managed branches using b4 prep --cleanup

Compare between versions

As I only have one version of the patch (and there will probably not be a version 2), I have no use for the cool compare feature. However, b4 let you compare your versions by simply :

b4 prep --compare-to v1

Trailers

Going through all mail threads and collect all trailer tags could be a painstaking work. B4 will do this for you and put the tag into the right patch magically.

Unfortunately, I forgot to mention my friend in the original patch, so I sent another mail [7] with the Suggested-by: Mark Olsson <mark@markolsson.se> tag.

Fetch the tag with b4 trailers -S -u (-S because the tag was sent by me and not Mark):

$ b4 trailers -S -u
Calculating patch-ids from commits, this may take a moment...
Checking change-id "20230824-pxrc-doc-1addbaa2250f"
Grabbing search results from lore.kernel.org
---
  input: docs: pxrc: remove reference to phoenix-sim
    + Suggested-by: Mark Olsson <mark@markolsson.se>
---
Invoking git-filter-repo to update trailers.
New history written in 0.03 seconds...
Completely finished after 0.17 seconds.
Trailers updated.

As we can see the Suggested-by tag is now applied to the patch in the right place:

commit 650264be66f0e589cf67a49f769ddc7d51e076cb (HEAD -> b4/pxrc)
Author: Marcus Folkesson <marcus.folkesson@gmail.com>
Date:   Thu Aug 24 15:47:24 2023 +0200

    input: docs: pxrc: remove reference to phoenix-sim
    
    The reference undeniably points to something unrelated nowadays.
    Remove it.
    
    Suggested-by: Mark Olsson <mark@markolsson.se>
    Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com>

Magic.

Principles

There are a few principles [1] that the Linux kernel follows in order to fulfull the requirements for on use of the radio spectrum:

• It should be reasonably impossible for a user to fail to comply with local regulations either unwittingly or by accident.
• Default configurations should err in favor of more restrictive behavior in order to protect unwitting users.
• Configurations that have no known compliant usage should not be part of the 'official' kernel tree.
• Configurations that are compliant only under special circumstances (e.g. with special licenses) should not be part of the 'official' kernel tree. Any exceptions should have their legal requirements clearly marked and those options should never be configured by default.
• Configurations that disable regulatory enforcement mechanisms should not be part of the 'official' kernel tree.
• The kernel should rely on userland components to determine regulatory policy. Consequently, the kernel's regulatory enforcement mechanisms should be flexible enough to cover known or reasonably anticipated regulatory policies.
• It is the moral duty of responsible distribution vendors, software developers, and community members to make every good faith effort to ensure proper compliance with applicable regulations regarding wireless communications.

The overall approach is "better safe than sorry" with respect to radio regulations. In other words, if no local configuration is setup, the system will fall back to the more restrictive world regulatory domain.

An example on such behaviour could be that the system is not allowed to initiate radio communication on certain radio frequencies that is not globally allowed.

CRDA

(Used pre Linux v4.15.)

CRDA [3], Central Regulatory Domain Agent, is a userspace agent responsible to read and intepret the regulatory.bin file and update the regulatory domains.

CRDA is intended to be trigged on uevents from the kernel (via udev) upon changes in the regulatory domain and setup the new regulations .

The udev rule to do this my look like this:

KERNEL=="regulatory*", ACTION=="change", SUBSYSTEM=="platform", RUN+="/sbin/crda"

Nowadays, CRDA is no longer needed as of kernel v.4.15 ( commit [2], "cfg80211: support loading regulatory database as firmware file"), the regulatory database is read by the Linux kernel directly as a firmware file during boot.

wireless-regdb

wireless-regdb [4] is the regulatory database used by Linux. The db.txt file in the repository contains regulatory information for each domain.

The output from this project is regulatory.db, which is loaded by the kernel as a firmware. The integrity of regulatory.db is typically ensured by the regulatory daemon by verifying the built-in RSA signature against a list of public keys in a preconfigured directory.

Although it is possible to build regulatory.db without any RSA key signature checking, it is highly recommended not to do so, as if the regulatory database is compromised in some way we could end up with a product that violates product radio compatibility.

IMAGE_INSTALL:append = " wireless-regdb-static "

World regulatory domain

# iw reg get
global
country 00: DFS-UNSET
        (755 - 928 @ 2), (N/A, 20), (N/A), PASSIVE-SCAN
        (2402 - 2472 @ 40), (N/A, 20), (N/A)
        (2457 - 2482 @ 20), (N/A, 20), (N/A), AUTO-BW, PASSIVE-SCAN
        (2474 - 2494 @ 20), (N/A, 20), (N/A), NO-OFDM, PASSIVE-SCAN
        (5170 - 5250 @ 80), (N/A, 20), (N/A), AUTO-BW, PASSIVE-SCAN
        (5250 - 5330 @ 80), (N/A, 20), (0 ms), DFS, AUTO-BW, PASSIVE-SCAN
        (5490 - 5730 @ 160), (N/A, 20), (0 ms), DFS, PASSIVE-SCAN
        (5735 - 5835 @ 80), (N/A, 20), (N/A), PASSIVE-SCAN
        (57240 - 63720 @ 2160), (N/A, 0), (N/A)

Country 00 is the world regulatory domain. This could be a result of a system that failed to load the regulatory database.

Look into the output from dmesg for verify:

$ dmesg | grep cfg80211
[    3.268852] cfg80211: Loading compiled-in X.509 certificates for regulatory database
[    3.269107] cfg80211: failed to load regulatory.db

As a result, this is the restrictions we have on the 5GHz band:

# iw list
[...]
                Frequencies:
                        * 5040 MHz [8] (disabled)
                        * 5060 MHz [12] (disabled)
                        * 5080 MHz [16] (disabled)
                        * 5170 MHz [34] (disabled)
                        * 5190 MHz [38] (20.0 dBm) (no IR)
                        * 5210 MHz [42] (20.0 dBm) (no IR)
                        * 5230 MHz [46] (20.0 dBm) (no IR)
                        * 5180 MHz [36] (20.0 dBm) (no IR)
                        * 5200 MHz [40] (20.0 dBm) (no IR)
                        * 5220 MHz [44] (20.0 dBm) (no IR)
                        * 5240 MHz [48] (20.0 dBm) (no IR)
                        * 5260 MHz [52] (20.0 dBm) (no IR, radar detection)
                        * 5280 MHz [56] (20.0 dBm) (no IR, radar detection)
                        * 5300 MHz [60] (20.0 dBm) (no IR, radar detection)
                        * 5320 MHz [64] (20.0 dBm) (no IR, radar detection)
                        * 5500 MHz [100] (20.0 dBm) (no IR, radar detection)
                        * 5520 MHz [104] (20.0 dBm) (no IR, radar detection)
                        * 5540 MHz [108] (20.0 dBm) (no IR, radar detection)
                        * 5560 MHz [112] (20.0 dBm) (no IR, radar detection)
                        * 5580 MHz [116] (20.0 dBm) (no IR, radar detection)
                        * 5600 MHz [120] (20.0 dBm) (no IR, radar detection)
                        * 5620 MHz [124] (20.0 dBm) (no IR, radar detection)
                        * 5640 MHz [128] (20.0 dBm) (no IR, radar detection)
                        * 5660 MHz [132] (20.0 dBm) (no IR, radar detection)
                        * 5680 MHz [136] (20.0 dBm) (no IR, radar detection)
                        * 5700 MHz [140] (20.0 dBm) (no IR, radar detection)
                        * 5745 MHz [149] (20.0 dBm) (no IR)
                        * 5765 MHz [153] (20.0 dBm) (no IR)
                        * 5785 MHz [157] (20.0 dBm) (no IR)
                        * 5805 MHz [161] (20.0 dBm) (no IR)
                        * 5825 MHz [165] (20.0 dBm) (no IR)
[...]

We can see the no IR flag is set for almost all frequencies on the 5GHz band. Please note that NO-IR is not the same as disabled, it simply means that we cannot initiate radiation on those frequencies.

Initiate radiation simply includes all modes of operations that require us to initiate radiation first. Think of acting as an Access Point, IBSS, Mesh or P2P master.

We can still use the frequency though, there is no problem to connect to an Access Point (we are not the one who initiate the radiation) on these frequencies.

Local regulatory domain

When a proper regulatory database is loaded into the system, we can setup the local regulatory domain instead of the globally one.

Set Swedish (SE) as our local regulatory domain:

# iw reg set SE
# iw reg get
global
country SE: DFS-ETSI
        (2400 - 2483 @ 40), (N/A, 20), (N/A)
        (5150 - 5250 @ 80), (N/A, 23), (N/A), NO-OUTDOOR, AUTO-BW
        (5250 - 5350 @ 80), (N/A, 20), (0 ms), NO-OUTDOOR, DFS, AUTO-BW
        (5470 - 5725 @ 160), (N/A, 26), (0 ms), DFS
        (5725 - 5875 @ 80), (N/A, 13), (N/A)
        (5945 - 6425 @ 160), (N/A, 23), (N/A), NO-OUTDOOR
        (57000 - 71000 @ 2160), (N/A, 40), (N/A)

And we are now allowed to use the 5GHz band with other restrictions:

#iw list
[...]
                Frequencies:
                        * 5040 MHz [8] (disabled)
                        * 5060 MHz [12] (disabled)
                        * 5080 MHz [16] (disabled)
                        * 5170 MHz [34] (23.0 dBm)
                        * 5190 MHz [38] (23.0 dBm)
                        * 5210 MHz [42] (23.0 dBm)
                        * 5230 MHz [46] (23.0 dBm)
                        * 5180 MHz [36] (23.0 dBm)
                        * 5200 MHz [40] (23.0 dBm)
                        * 5220 MHz [44] (23.0 dBm)
                        * 5240 MHz [48] (23.0 dBm)
                        * 5260 MHz [52] (20.0 dBm) (no IR, radar detection)
                        * 5280 MHz [56] (20.0 dBm) (no IR, radar detection)
                        * 5300 MHz [60] (20.0 dBm) (no IR, radar detection)
                        * 5320 MHz [64] (20.0 dBm) (no IR, radar detection)
                        * 5500 MHz [100] (26.0 dBm) (no IR, radar detection)
                        * 5520 MHz [104] (26.0 dBm) (no IR, radar detection)
                        * 5540 MHz [108] (26.0 dBm) (no IR, radar detection)
                        * 5560 MHz [112] (26.0 dBm) (no IR, radar detection)
                        * 5580 MHz [116] (26.0 dBm) (no IR, radar detection)
                        * 5600 MHz [120] (26.0 dBm) (no IR, radar detection)
                        * 5620 MHz [124] (26.0 dBm) (no IR, radar detection)
                        * 5640 MHz [128] (26.0 dBm) (no IR, radar detection)
                        * 5660 MHz [132] (26.0 dBm) (no IR, radar detection)
                        * 5680 MHz [136] (26.0 dBm) (no IR, radar detection)
                        * 5700 MHz [140] (26.0 dBm) (no IR, radar detection)
                        * 5745 MHz [149] (13.0 dBm)
                        * 5765 MHz [153] (13.0 dBm)
                        * 5785 MHz [157] (13.0 dBm)
                        * 5805 MHz [161] (13.0 dBm)
                        * 5825 MHz [165] (13.0 dBm)
[...]

Regulatory flags

Some of the flags reported by iw may not be obvious at a first glance. Here is an explaination for some of them:

• DFS: stands for Dynamic Frequency Selection and is a channel allocation scheme used to prevent electromagnetic interference with systems that predates Wi-Fi.
• TPC: stands for Transmit Power Control which is a mechanism to automatically reduce the used transmission output power when other networks are within range.
• SRD: stands for Short-Range Device and cover devices that are low-power transmitters typically limited to the range of 24-100mW ERP.

Add compatible strings

The Linux driver infrastructure binds a certain device to a driver by a string (or other uniq identifies such as VID/PID for USB). When, for example, the compatible property of a device tree node matches a device driver, a device is instantiated and the probe function is called.

As a single device driver could handle multiple similiar IC:s where some of their properites may differ, we have to differentiate those somehow. This is done by provide device specific data to each instance of the device. This data is called "driver data" or "private data" and is part of the device lookup table.

E.g. the driver_data field of the struct spi_device_id:

struct spi_device_id {
	char name[SPI_NAME_SIZE];
	kernel_ulong_t driver_data;	/* Data private to the driver */
};

Or the data field of the struct of_device_id:

/*
 * Struct used for matching a device
 */
struct of_device_id {
	char	name[32];
	char	type[32];
	char	compatible[128];
	const void *data;
};

For this driver, the driver data to these ID tables looks as follows:

static const struct of_device_id mcp3911_dt_ids[] = {
-       { .compatible = "microchip,mcp3911" },
+       { .compatible = "microchip,mcp3910", .data = &mcp3911_chip_info[MCP3910] },
+       { .compatible = "microchip,mcp3911", .data = &mcp3911_chip_info[MCP3911] },
+       { .compatible = "microchip,mcp3912", .data = &mcp3911_chip_info[MCP3912] },
+       { .compatible = "microchip,mcp3913", .data = &mcp3911_chip_info[MCP3913] },
+       { .compatible = "microchip,mcp3914", .data = &mcp3911_chip_info[MCP3914] },
+       { .compatible = "microchip,mcp3918", .data = &mcp3911_chip_info[MCP3918] },
+       { .compatible = "microchip,mcp3919", .data = &mcp3911_chip_info[MCP3919] },
    { }
};
MODULE_DEVICE_TABLE(of, mcp3911_dt_ids);

static const struct spi_device_id mcp3911_id[] = {
-       { "mcp3911", 0 },
+       { "mcp3910", (kernel_ulong_t)&mcp3911_chip_info[MCP3910] },
+       { "mcp3911", (kernel_ulong_t)&mcp3911_chip_info[MCP3911] },
+       { "mcp3912", (kernel_ulong_t)&mcp3911_chip_info[MCP3912] },
+       { "mcp3913", (kernel_ulong_t)&mcp3911_chip_info[MCP3913] },
+       { "mcp3914", (kernel_ulong_t)&mcp3911_chip_info[MCP3914] },
+       { "mcp3918", (kernel_ulong_t)&mcp3911_chip_info[MCP3918] },
+       { "mcp3919", (kernel_ulong_t)&mcp3911_chip_info[MCP3919] },
    { }

The driver data is then reachable in the probe function via spi_get_device_match_data():

    adc->chip = spi_get_device_match_data(spi);

Driver data

The driver data is used to distinguish between different devices and provide enough information to make it possible for the driver to handle all differencies between the IC:s in a common way.

The driver data for these devices looks as follows:

+struct mcp3911_chip_info {
+       const struct iio_chan_spec *channels;
+       unsigned int num_channels;
+
+       int (*config)(struct mcp3911 *adc);
+       int (*get_osr)(struct mcp3911 *adc, int *val);
+       int (*set_osr)(struct mcp3911 *adc, int val);
+       int (*get_offset)(struct mcp3911 *adc, int channel, int *val);
+       int (*set_offset)(struct mcp3911 *adc, int channel, int val);
+       int (*set_scale)(struct mcp3911 *adc, int channel, int val);
+};
+

Description of the structure members:

• .channels is a a pointer to struct iio_chan_spec where all ADC and timestamp channels are specified.
• .num_channels is the number of channels
• .config is a function pointer to configure the device
• .get_* and .set_* is function pointers used to get/set certain registers

A struct mcp3911_chip_info is created for each type of supported IC:

+static const struct mcp3911_chip_info mcp3911_chip_info[] = {
+       [MCP3910] = {
+               .channels = mcp3910_channels,
+               .num_channels = ARRAY_SIZE(mcp3910_channels),
+               .config = mcp3910_config,
+               .get_osr = mcp3910_get_osr,
+               .set_osr = mcp3910_set_osr,
+               .get_offset = mcp3910_get_offset,
+               .set_offset = mcp3910_set_offset,
+               .set_scale = mcp3910_set_scale,
+       },
+       [MCP3911] = {
+               .channels = mcp3911_channels,
+               .num_channels = ARRAY_SIZE(mcp3911_channels),
+               .config = mcp3911_config,
+               .get_osr = mcp3911_get_osr,
+               .set_osr = mcp3911_set_osr,
+               .get_offset = mcp3911_get_offset,
+               .set_offset = mcp3911_set_offset,
+               .set_scale = mcp3911_set_scale,
+       },
+       [MCP3912] = {
+               .channels = mcp3912_channels,
+               .num_channels = ARRAY_SIZE(mcp3912_channels),
+               .config = mcp3910_config,
+               .get_osr = mcp3910_get_osr,
+               .set_osr = mcp3910_set_osr,
+               .get_offset = mcp3910_get_offset,
+               .set_offset = mcp3910_set_offset,
+               .set_scale = mcp3910_set_scale,
+       },
+       [MCP3913] = {
+               .channels = mcp3913_channels,
+               .num_channels = ARRAY_SIZE(mcp3913_channels),
+               .config = mcp3910_config,
+               .get_osr = mcp3910_get_osr,
+               .set_osr = mcp3910_set_osr,
+               .get_offset = mcp3910_get_offset,
+               .set_offset = mcp3910_set_offset,
+               .set_scale = mcp3910_set_scale,
+       },
+       [MCP3914] = {
+               .channels = mcp3914_channels,
+               .num_channels = ARRAY_SIZE(mcp3914_channels),
+               .config = mcp3910_config,
+               .get_osr = mcp3910_get_osr,
+               .set_osr = mcp3910_set_osr,
+               .get_offset = mcp3910_get_offset,
+               .set_offset = mcp3910_set_offset,
+               .set_scale = mcp3910_set_scale,
+       },
+       [MCP3918] = {
+               .channels = mcp3918_channels,
+               .num_channels = ARRAY_SIZE(mcp3918_channels),
+               .config = mcp3910_config,
+               .get_osr = mcp3910_get_osr,
+               .set_osr = mcp3910_set_osr,
+               .get_offset = mcp3910_get_offset,
+               .set_offset = mcp3910_set_offset,
+               .set_scale = mcp3910_set_scale,
+       },
+       [MCP3919] = {
+               .channels = mcp3919_channels,
+               .num_channels = ARRAY_SIZE(mcp3919_channels),
+               .config = mcp3910_config,
+               .get_osr = mcp3910_get_osr,
+               .set_osr = mcp3910_set_osr,
+               .get_offset = mcp3910_get_offset,
+               .set_offset = mcp3910_set_offset,
+               .set_scale = mcp3910_set_scale,
+       },
+};

Thanks to this, all differences between the IC:s is in one place and the driver code is common for all devices. See the code below how oversampling ration is set. The differences between IC:s is handled by the callback function:

        case IIO_CHAN_INFO_OVERSAMPLING_RATIO:
                for (int i = 0; i < ARRAY_SIZE(mcp3911_osr_table); i++) {
                        if (val == mcp3911_osr_table[i]) {
-                               val = FIELD_PREP(MCP3911_CONFIG_OSR, i);
-                               ret = mcp3911_update(adc, MCP3911_REG_CONFIG, MCP3911_CONFIG_OSR,
-                                               val, 2);
+                               ret = adc->chip->set_osr(adc, i);
                                break;
                        }
                }

Usage

CRIU has many possible use-cases. Some of those are:

• Container live migration
• Slow-boot services speed up
• Seamless kernel upgrade
• Seamless kernel upgrade
• "Save" ability in apps (games), that don't have such
• Snapshots of apps

My use case or now is just to save a snapshot of an application and poweroff the CPU module to later be able to power on and restore it.

Summary

I do not know if CRIU is applicable for what I want to achieve right now, but it is a cool project that I will probably find usage for in the future, so it is a welcome tool to my toolbag.

References

Adding the readonly-rootfs-overlay layer to your build

In order to use this layer, you need to make the build system aware of it.

Assuming the readonly-rootfs-overlay layer exists at the top-level of your OpenEmbedded source tree, you can add it to the build system by adding the location of the readonly-rootfs-overlay layer to bblayers.conf, along with any other layers needed. e.g.:

BBLAYERS ?= " \
  /path/to/layers/meta \
  /path/to/layers/meta-poky \
  /path/to/layers/meta-yocto-bsp \
  /path/to/layers/meta-readonly-rootfs-overlay \
  "

To add the script to your image, just add:

IMAGE_INSTALL:append = " initscripts-readonly-rootfs-overlay"

to your local.conf or image recipe. Or use core-image-rorootfs-overlay-initramfs as initrd.

Read-only root filesystem

If you use this layer you do not need to set read-only-rootfs in the IMAGE_FEATURES or EXTRA_IMAGE_FEATURES variable.

Kernel command line parameters

These examples are not meant to be complete. They just contain parameters that are used by the initscript of this repository. Some additional paramters might be necessary.

root=/dev/sda1 rootrw=/dev/sda2

root=/dev/sda1 rootrw=/dev/sda2 init=/bin/sh

root=/dev/sda1 rootrw=/dev/sda2 init=/init

root=/dev/sda1 rootrw=/dev/sda2 init=/init rootinit=/bin/sh

References

The conference

(I got stuck on a picture. Don't know which talk, but it seems like I enjoyed it)

References

iproute2

The ip command could be handy to manually create or change the metric value for a certain route:

$ ip route replace default via {IP} dev {DEVICE} metric {METRIC}

ifmetric

ifmetric is a tool for setting the metric value for IPv4 routes attached to a given network interface. Compared to the raw ip command above, ifmetric works on interfaces rather than routes.

$ ifmetric INTERFACE [METRIC]

dhcpcd

Metric values could be set in /etc/dhcpcd.conf according to the manual [1]:

metric metric
Metrics are used to prefer an interface over another one, lowest wins.

e.g.:

interface wlan0
metric 200

If no metric value is given, the default metric is calculated by 200 + if_nametoindex(3). An extra 100 will be added for wireless interfaces.

NetworkManager

Add ipv4.route-metric METRIC to your /etc/NetworkManager/system-connections/<connection>.nmconnection file.

You could also use the command line tool:

    $ nmcli connection edit tuxnet

    ===| nmcli interactive connection editor | ===

    Editing existing '802-11-wireless' connection: 'tuxnet'

    Type 'help' or '?' for available commands.
    Type 'print' to show all the connection properties.
    Type 'describe [<setting>.<prop>]' for detailed property description.

    You may edit the following settings: connection, 802-11-wireless (wifi), 802-11-wireless-security (wifi-sec), 802-1x, ethtool, match, ipv4, ipv6, tc, proxy
    nmcli> set ipv4.route-metric 600
    nmcli> save
    nmcli> quit

PPPD

PPP is a protocol used for establishing internet links over dial-up modems. These links is usually not the preferred link when the device has other more reliable and/or cheaper connections.

The pppd daemon has a few options as specified in the manual [2] for creating a default route and set the metric value:

defaultroute
       Add a default route to the system routing tables, using
       the peer as the gateway, when IPCP negotiation is
       successfully completed.  This entry is removed when the
       PPP connection is broken.  This option is privileged if
       the nodefaultroute option has been specified.

defaultroute-metric
       Define the metric of the defaultroute and only add it if
       there is no other default route with the same metric.
       With the default value of -1, the route is only added if
       there is no default route at all.

replacedefaultroute
       This option is a flag to the defaultroute option. If
       defaultroute is set and this flag is also set, pppd
       replaces an existing default route with the new default
       route.  This option is privileged.

E.g.

replacedefaultroute
defaultroute-metric 900

The RISC-V Linux port; past/current/next

Björn Töpel talked about the current status of RISC-V architecture in the Linux kernel.

For those who don't know - RISC-V is a open and royalty free Instruction Set Architecture. In practice, this means for example that whenever you want to implement your own CPU core in your FPGA, you are free to do so using the RISC-V ISA. Compare that to ARM that you are strictly not allowed to even think about it without pay royalties and other fees.

RISC-V is a rather new port, the first proposol was sent out to the mailing list in 2016. It makes it a pretty good target to get involved into if you want to get to know the kernel in-depth as the implementation is still quite small in lines of code, which makes it easier to overview.

Björn told us that kernel support for RISC-V has made huge progress in the embedded area, but still lack some important functionality to be useful on the server side. Parts that are missing is e.g. support for ACPI, EUFI, AP-TEE, hotplugs and an advanced interrupt controller.

The architecture gets more support for each kernel release though. Some of the news for RISC-V in linux v6.4 are:

• Support for Kernel Adress Space Layout Randomization (KASLR)
• Relocatable kernel
• HWprobe syscall

Vector support is on its way, but it currently break the ABI, so there are a few things left that needs to be addressed before we can expect any merge.

One giant leap for security: leveraging capabilities in Linux

Kevin Brodsky talked about self aware pointers, which I found interresting. That we can use address bits for other purposes than addresses is nothing new. In a 64bit ARM kernel we do often use only 52bits anyway (4PiB of addressable memory is more than enough for all people(phun intended )).

What Kevin and his team has done is to extend the address to 129bits to even include meta data for boundaries, capabilities and validity tags. The 129bits reservaton has of course a huge impact on the system as it use more than double the size compared to a normal 64-bit system, but it also gives us much in return.

These 129 bits is by the way already a compressed version of the 256 bit variant they started with..

Unfortunately, the implementation is for userpace only, which is a little sad because we already have tons of tools to run application in a protected and constrained environment, but it proves that the concept works and maybe we will see something like this for kernel space in the future.

The implementation requires changes is several parts of the system. The memory allocator and unwind code is most affected, but even the kernel itself and glibc has to be modified. Most of the applications and libraries is not affected at all though.

There is a working meta-layer for Yocto called Morello that can be used to test it out. It contains a usage guide and even a little tutorial on howto build and run Doom :-)

Supporting zoned storage in Ublk

Andreas Hindborg has been working with support for zoned storage [2] in the ublk driver. Zoned storage is basically about to spit the address space into regions called zones that can only be written sequentially. This leads to higher throughput and increased capacity. It also eliminates the need for a Flash Translation Layer (FTL) for e.g. SSD devices.

ublk make use of io_uring internally, which by the way is a cool feature. The io_uring let you queue system calls into a ring buffer, which makes it possible to do more work every time you enter the kernel space. This has impact on the performance as you do not need to context switch back and forth to userspace between each system call.

It is quite easy to add support for io_uring operations to normal character devices, as the struct file_operation now has a uring_cmd callback function that could be populated. This makes it to a high performance alternative to the IOCTL we are used to.

ublk is used to create a block device driver in userspace. It works as all requests and results to/from the block device is redirected to a userspace daemon. The userspace daemon used for this is called ublk-rs, which is entirely written i Rust (of course..). Unfortunately, the source code is not yet available due to legal reasons, but is on its way.

His work was to add support for zoned storage (basically split the address space into regions called zones)

Rust

Then there was a couple of talks about the most hip programming language for now; Rust.

Linus Walleij gave us a history lecture in programming languages in his talk "Rust: Abstraction and Productivity" and his thoughts aout why Rust could be something good for kernel. Andreas Hindborg continued and showed how he implemented a null_blk driver completely in Rust.

But why should we even consider Rust for the kernel? In fact, the language is guaranteed to have a few properties C does not, and we basic Rust support was introduced in Linux v6.1.

We say that Rust is safe, and when we state that, we think of that Rust does have:

• No buffer overflows
• No use after free
• No dereferencing null or invalid pointers
• No double free
• No pointer aliasing
• No type errors
• No data races
• ... and more

What was new to me is that a Rust application does not even compile if you try something of the above.

This together makes Rust both memory safe, type safe and thread safe. Consider that 20-60% of the bug fixes in the kernel are for memory safety bugs. These memory bugs takes a lot of productivity away as it often takes long time to find and fix them. Maybe Rust is not that bad after all.

Many cool projects are going on in Rust, example on those are:

• TLS handshake in the kernel
• Ethernet-SPI drivers
• M1&M3 GPU drivers.

The goal with Andreas null_blk driver is to first write a solid Rust API for the blk-mq implementation and then use it in the null_blk driver to provide a reference implementation for linux kernel developers to get started with.

Install Zephyr

Zephyr does use west [10] for pretty much everything. West is a meta tool used for repository management, building, debugging, deploying.. you name it. It has many similarities with bitbake that you will find in Yocto. I'm more of a "do one thing and do it well"-guy, so these tools (nor west or bitbake) makes a huge impression on me.

West is written in Python, as the nature of Python is as it is, you have to make a virtual environment to make sure that your setup will work for more than a week. Otherwise you will end up in incompatibilities as soon you upgrading some of the python dependencies.

The documentation [9] is actually really good nowadays. Most of these commands are just copy&paste from there.

Create a new virtual environment:

python -m venv ~/zephyrproject/.venv

Activate the virtual environment:

source ~/zephyrproject/.venv/bin/activate

Install west:

pip install west

Get the Zephyr source code:

west init ~/zephyrproject
cd ~/zephyrproject
west update

Export a Zephyr CMake package to allow CMake to automatically load boilerplate code required for building Zephyr applications:

west zephyr-export

The Zephyr project does contain a file with additional Python dependencies, install them:

pip install -r ~/zephyrproject/zephyr/scripts/requirements.txt

Install Zephyr SDK

The Zephyr Software Development Kit (SDK) contain toolchains for all architectures that is supported by Zephyr.

Download the latest SDK bundle:

cd ~
wget https://github.com/zephyrproject-rtos/sdk-ng/releases/download/v0.16.0/zephyr-sdk-0.16.0_linux-x86_64.tar.xz
wget -O - https://github.com/zephyrproject-rtos/sdk-ng/releases/download/v0.16.0/sha256.sum | shasum --check --ignore-missing

Extract the archive:

tar xvf zephyr-sdk-0.16.0_linux-x86_64.tar.xz

Run the setup script:

cd zephyr-sdk-0.16.0
./setup.sh

Build OpenOCD

The Raspberry Pi Pico has an SWD interface that can be used to program and debug the on board RP2040 MCU.

This interface can be utilized by OpenOCD. Support for RP2040 is not mainlined though, so we have to go for a rpi fork [11].

Clone repository:

git clone https://github.com/raspberrypi/openocd.git
cd openocd

Build:

./bootstrap
./configure
make

And install:

make install

Build sample application

The Raspberry Pi Pico does have a LED on board. So blinky, an application that will flash the LED with 1Hz, is a good test to prove that at least something is alive. Build it:

cd ~/zephyrproject/zephyr
west build -b rpi_pico samples/basic/blinky -- -DOPENOCD=/usr/local/bin/openocd -DOPENOCD_DEFAULT_PATH=/usr/local/share/openocd/scripts -DRPI_PICO_DEBUG_ADAPTER=jlink

Note that we specify the board (-b) to rpi_pico.

OPENOCD and OPENOCD_DEFAULT_PATH should point to where OpenOCD is installed in the previous step.

Flash the application

To flash our Raspberry Pi Pico, we just run:

west flash

As we have set the RPI_PICO_DEBUG_ADAPTER during the build stage, it is cached so it can be omitted from the west flash and west debug commands. Otherwise we had to provide the --runner option. E.g. :

west flash --runner jlink

You don't have to use a J-link to flash the Raspberry Pi Pico, you could also copy the UF2 file to target. If you power up the Pico with the BOOTSEL button pressed, it will appear on the host as a mass storage device where you could simply copy the UF2 file to. You loose the possibility to debug with GDB though.

Debug the application

The most straight forward way is to use west to start a GDB session (--runner is still cached from the build stage):

west debug

I prefer to use the Text User Interface (TUI) as it is easier to follow the code, both in C and assembler. Enter TUI mode by press CTRL+X+A or enter "tui enable" on the command line.

If you do not want to use west, you could start openocd by yourself:

openocd -f interface/jlink.cfg -c 'transport select swd' -f target/rp2040.cfg -c "adapter speed 2000" -c 'targets rp2040.core0'

And manually connect with GDB:

gdb-multiarch -tui
(gdb) target external :3333
(gdb) file ./build/zephyr/zephyr.elf

The result is the same.

Devicetrees

A Devicetree [2] is a data structure that describe the static hardware configuration in a standard manner. One of the motivations behind devicetree is that it should not be specific for any kernel. In the best of the worlds, you should be able to boot a Linux kernel, BSD kernel or Zephyr (well..) with the same devicetree. I've never heard about a working example IRL though, but the idea is good.

In the same way, you should be able to boot the same kernel on different board by only swap the devicetree. In Zephyr, the devicetree is integrated to the binary blob, so this idea does not fully apply to Zephyr though.

There are two types of files related to device trees in Zephyr:

• Devicetree sources - the devicetree itself (including dts, interface files and overlays).
• Devicetree bindings - description of its content. E.g. data types and which properties that is required or optional.

Zephyr does make use of both of these type of files during the build process. It allows the build process to make a build-time validation of the devicetree sources against the bindings, generate KConfig macros and a whole bunch of other macros that is to be used by the application and by Zephyr itself. We will see example of these macros later on.

Here is a simplified picture of the build process with respect to devicetrees:

Driver

All drivers is located in the ./driver directory. It is C-files that contains the actual implementation of the driver.

KConfig

Like the Linux kernel (and U-boot, busybox, Barebox, Buildroot...), Zephyr uses the KConfig system to select what subsystem, libraries and drivers to be included in the build.

Remember when we did build the blinky application in the part1? We did provide -b rpi_pico to the build command to specify board:

west build -b rpi_pico ....

This will load ./boards/arm/rpi_pico/rpi_pico_defconfig as the default configuration and store it into ./build/zephyr/.config, which is the actual configuration the build system will use.

The .config file contains all configuration options selected by e.g. menuconfig AND the generated configuration options from the devicetree.

Unit tests

Zephyr makes use of Twister [1] for unit tests. By default it will build the majority of all tests on a defined set of boards. All these tests is part of the automatic test procedure for every pull request.

Driver

Create an empty file for now:

touch drivers/dac/dac_ltc166x.c

The driver will support both ltc1660 (10-bit, 8 channels) and ltc1665 (8-bit, 8 channels) DAC. I do not prefer to name drivers with an x as there actually are chips out there with an x in their name, so it could be a little fraudulent. That is at least something we try to avoid it in the Linux kernel.

A better name would be just dac_ltc1660.c and support all ICs that are compatible with dac_ltc1660. However, the Zephyr project has choosen to make use of the x in names to indicate that multiple chips are supported. When in Rome, do as the Romans do.

Add the file to the CMake build system:

diff --git a/drivers/dac/CMakeLists.txt b/drivers/dac/CMakeLists.txt
index b0e86e3bd4..800bc895fd 100644
--- a/drivers/dac/CMakeLists.txt
+++ b/drivers/dac/CMakeLists.txt
@@ -9,6 +9,7 @@ zephyr_library_sources_ifdef(CONFIG_DAC_SAM             dac_sam.c)
 zephyr_library_sources_ifdef(CONFIG_DAC_SAM0           dac_sam0.c)
 zephyr_library_sources_ifdef(CONFIG_DAC_DACX0508       dac_dacx0508.c)
 zephyr_library_sources_ifdef(CONFIG_DAC_DACX3608       dac_dacx3608.c)
+zephyr_library_sources_ifdef(CONFIG_DAC_LTC166X     dac_ltc166x.c)
 zephyr_library_sources_ifdef(CONFIG_DAC_SHELL          dac_shell.c)
 zephyr_library_sources_ifdef(CONFIG_DAC_MCP4725                dac_mcp4725.c)
 zephyr_library_sources_ifdef(CONFIG_DAC_MCP4728                dac_mcp4728.c)

CONFIG_DAC_LTC166X comes from the Kconfig system and could be either 'y' or 'n' dependig on if it is selected or not.

Kconfig

Create two new Kconfig configuration options. One for the driver itself and one for its init priority:

diff --git a/drivers/dac/Kconfig.ltc166x b/drivers/dac/Kconfig.ltc166x
new file mode 100644
index 0000000000..6053bc39bf
--- /dev/null
+++ b/drivers/dac/Kconfig.ltc166x
@@ -0,0 +1,22 @@
+# DAC configuration options
+
+# Copyright (C) 2023 Marcus Folkesson <marcus.folkesson@gmail.com>
+#
+# SPDX-License-Identifier: Apache-2.0
+
+config DAC_LTC166X
+       bool "Linear Technology LTC166X DAC"
+       default y
+       select SPI
    +       depends on DT_HAS_LLTC_LTC1660_ENABLED  || \
+               DT_HAS_LLTC_LTC1665_ENABLED
+       help
+         Enable the driver for the Linear Technology LTC166X DAC
+
+if DAC_LTC166X
+
+config DAC_LTC166X_INIT_PRIORITY
+       int "Init priority"
+       default 80
+       help
+         Linear Technology LTC166X DAC device driver initialization priority.
+
+endif # DAC_LTC166X

DT_HAS_LLTC_LTC1660_ENABLED and DT_HAS_LLTC_LTC1660_ENABLED is configuration options that is generated from the seleted devicetree. By depend on it, the DAC_LTC166X option will only show up if there are such a node specified. I really like this feature.

Also add it into the build stucture:

diff --git a/drivers/dac/Kconfig b/drivers/dac/Kconfig
index 7b54572146..77b0db902b 100644
--- a/drivers/dac/Kconfig
+++ b/drivers/dac/Kconfig
@@ -42,6 +42,8 @@ source "drivers/dac/Kconfig.dacx0508"

 source "drivers/dac/Kconfig.dacx3608"

+source "drivers/dac/Kconfig.ltc166x"
+
 source "drivers/dac/Kconfig.mcp4725"

 source "drivers/dac/Kconfig.mcp4728"

Device tree

The bindings for all devices has to be described in the YAML format. These bindings is verified during compile time in order to make sure that the device tree node fulfills all required properties and not tries to invent some new ones. This protects us against typos, which also is a really good feature. The Linux kernel does not have this...

We have to create such a binding, one for each chip:

diff --git a/dts/bindings/dac/lltc,ltc1660.yaml b/dts/bindings/dac/lltc,ltc1660.yaml
new file mode 100644
index 0000000000..196204236a
--- /dev/null
+++ b/dts/bindings/dac/lltc,ltc1660.yaml
@@ -0,0 +1,8 @@
+# Copyright (C) 2023 Marcus Folkesson <marcus.folkesson@gmail.com>
+# SPDX-License-Identifier: Apache-2.0
+
+include: [dac-controller.yaml, spi-device.yaml]
+
+description: Linear Technology Micropower octal 10-Bit DAC
+
+compatible: "lltc,ltc1660"
diff --git a/dts/bindings/dac/lltc,ltc1665.yaml b/dts/bindings/dac/lltc,ltc1665.yaml
new file mode 100644
index 0000000000..2c789ecc56
--- /dev/null
+++ b/dts/bindings/dac/lltc,ltc1665.yaml
@@ -0,0 +1,8 @@
+# Copyright (C) 2023 Marcus Folkesson <marcus.folkesson@gmail.com>
+# SPDX-License-Identifier: Apache-2.0
+
+include: [dac-controller.yaml, spi-device.yaml]
+
+description: Linear Technology Micropower octal 8-Bit DAC
+
+compatible: "lltc,ltc1665"

dac-controller.yaml and spi-device.yaml is included to inherit some of the required properties (such as spi-max-speed) for this of device.

Unit tests

Add the driver to the test framework and allow the test to be executed on the native_posix platform:

diff --git a/tests/drivers/build_all/dac/testcase.yaml b/tests/drivers/build_all/dac/testcase.yaml
index fa2eb5ac7a..1c7fa521d0 100644
--- a/tests/drivers/build_all/dac/testcase.yaml
+++ b/tests/drivers/build_all/dac/testcase.yaml
@@ -5,7 +5,7 @@ tests:
   drivers.dac.build:
     # will cover I2C, SPI based drivers
     platform_allow: native_posix
-    tags: dac_dacx0508 dac_dacx3608 dac_mcp4725 dac_mcp4728
+    tags: dac_dacx0508 dac_dacx3608 dac_mcp4725 dac_mcp4728 dac_ltc1660 dac_ltc1665
     extra_args: "CONFIG_GPIO=y"
   drivers.dac.mcux.build:
     platform_allow: frdm_k22f

Also add nodes in app.overlay to make it possible for the unit tests to instantiate the DAC:

diff --git a/tests/drivers/build_all/dac/app.overlay b/tests/drivers/build_all/dac/app.overlay
index 471bfae6e8..c1e9146974 100644
--- a/tests/drivers/build_all/dac/app.overlay
+++ b/tests/drivers/build_all/dac/app.overlay
@@ -68,6 +68,8 @@

                        /* one entry for every devices at spi.dtsi */
                        cs-gpios = <&test_gpio 0 0>,
+                                  <&test_gpio 0 0>,
+                                  <&test_gpio 0 0>,
                                   <&test_gpio 0 0>,
                                   <&test_gpio 0 0>;

@@ -118,6 +120,20 @@
                                channel6-gain = <0>;
                                channel7-gain = <0>;
                        };
+
+                       test_spi_ltc1660: ltc1660@3 {
+                               compatible = "lltc,ltc1660";
+                               reg = <0x3>;
+                               spi-max-frequency = <0>;
+                               #io-channel-cells = <1>;
+                       };
+
+                       test_spi_ltc1665: ltc1665@4 {
+                               compatible = "lltc,ltc1665";
+                               reg = <0x4>;
+                               spi-max-frequency = <0>;
+                               #io-channel-cells = <1>;
+                       };
                };
        };
 };