TIL - Reuse your SSH connection

TIL - Reuse your SSH connection

TIL, Today I Learned, is more of a "I just figured this out: here are my notes, you may find them useful too" rather than a full blog post

OpenSSH has the feature to reuse an existing SSH connection for multiple subsequent connections to the same host. It will improve your workflow because it reduces the time it takes to establish a new connection and in cases where you have to type a password, the time saving is even more.

The feature is ControlMaster and is documented in ssh_config(5) [1]:

ControlMaster
Enables the sharing of multiple sessions over a single network connection.
When set to ''yes'', ssh(1) will listen for connections on a control
socket specified using the ControlPath argument.  Additional sessions
can connect to this socket using the same ControlPath with ControlMaster
set to ''no'' (the default).  These sessions will try to reuse the
master instance's network connection rather than initiating new ones,
but will fall back to connecting normally if the control socket does not
exist, or is not listening.

To make use of this feature add the following line to your ~./ssh/config file:

Host *
        ControlMaster auto
        ControlPath ~/.ssh/master-%r@%h:%p
        ControlPersist 4h

It will keep your connection for 4 hours.

TIL - xxd color support

TIL - xxd color support

TIL, Today I Learned, is more of a "I just figured this out: here are my notes, you may find them useful too" rather than a full blog post

Today (2 January 2024), version 9.1 of the Vim editor has been released [1]!

xxd [2] is a utility that is distributed with Vim and is my go-to-tool when it comes to view or manipulating binary files.

A feature I've been missing for a long time is finally here - color support!

To enable colors, use the -R flag (same as for less and hexdump). E.g.:

$xxd -R always $FILE | less -R
/media/xxd.png

The hex-value and value are both colored with the same color depending on the hex-value:

  • 0x00 = white
  • 0xff = blue
  • printable = green
  • non-printable = red
  • tabs and linebreaks = yellow

TIL - git credential storage

TIL - git credential storage

TIL, Today I Learned, is more of a "I just figured this out: here are my notes, you may find them useful too" rather than a full blog post

When using SSH as transport protocol for connecting to remotes you could use a key without need to type a username nor password. Unfortunately, this is not possible when the underlaying protocol is HTTPS as it requires a username and password for every connection made.

This is where git credential store gets a chance to shine.

Credential storage

Git has a credential system. All you need to do is to set the credential.helper in your .gitconfig.

Possible values are:

  • none, this is the default behavior. You will be promted for username and password for every connection made.
  • cache, this option caches the credentials in memory for a certain period of time. Default is 15 minutes. The passwords are purged after this period.
  • store, saves the credentials in a (plain-text) file.

For example:

$ git config credential.helper store

Will create this entry in your .gitconfig:

[credential]
        helper = store

The default storage file is ~/.git-credentials, you may change it by including the --file option. E.g.:

$ git config credential.helper 'store --file ~/.my-secrets'

For more information, see the manpage for gitcredentials [1].

KAS

KAS [2] is my favorite tool to setup bitbake based projects. I always build my images using kas-container, as the state of my Archlinux setup is not always... comptible. (I love rolling distributions though).

However, many tools, including kas-container, makes use of git credential files for authentication.

kas-container has the --git-credential-store <file> option to specify the credential file.

TIL - docker scratch image

TIL - Docker scratch image

TIL, Today I Learned, is more of a "I just figured this out: here are my notes, you may find them useful too" rather than a full blog post

The scratch image is the smallest possible image for docker. It does not contain any libraries nor other executables. It is simply a new, fresh and empty setup of namespaces.

The FROM scratch line is even a no-op [1] in the Dockerfile, which results in that it will not create an extra layer in you image. Example of a Dockerfile:

FROM scratch
ADD hello /
CMD ["/hello"]

As the Docker image does not contains any libraries, the hello application in the example above has to be compiled statically.

One use I see is to setup a Docker image based on a completely custom made root filesystem, e.g. the output from Buildroot.

See Docker documentation [2] for further reading.

TIL - sort in vim

TIL - Sort in VIM

TIL, Today I Learned, is more of a "I just figured this out: here are my notes, you may find them useful too" rather than a full blog post

After 15+ years of daily VIM usage I just discovered the :sort function. Quite embarassing.

/media/vimsort.gif

It is also possible to sort in revese ( :sort!) and remove duplicate lines ( :sort u).

TIL - notmuch-lore

TIL - notmuch-lore

TIL, Today I Learned, is more of a "I just figured this out: here are my notes, you may find them useful too" rather than a full blog post

Notmuch [1] is an mail indexer and is a great tool to search in high-volume mailboxes (e.g. mailing lists). Being subscribed to all these mailing lists and retreiving all mails over IMAP daily could be quite annoying and harm your mail quota.

Therefor, public-inbox [2] makes archives of many of the public maling lists out there and makes it available in a git repository.

That is good, but you want to read your emails in your email-client, right?

notmuch-lore

notmuch-lore is a script written by Tobias Waldekranz [3] which is intended to be used as a pre-new-hook together with notmuch. Tobias is actually an old colleague of mine, we worked together over 10 years ago before he moved to another city.

notmuch-lore connects the public-inbox repositories and convert them to Maildir format so that notmuch can read them. Many tools could then be used as a front-end to notmuch, including vim and mutt :-)

How to setup and use notmuch-lore is well described in the README file of the project [3], so I'll leave it to that.

I encourage you to take a look if you also spend a lot of time on mailing lists!

TIL - git man-pages

TIL - git man-pages

TIL, Today I Learned, is more of a "I just figured this out: here are my notes, you may find them useful too" rather than a full blog post

I'm a big user of man-pages.

Today I found a set of pages that I've not noticed before when I was reading man git:

SEE ALSO
        gittutorial(7), gittutorial-2(7), giteveryday(7), gitcvs-migration(7), gitglossary(7), gitcore-tutorial(7),
       gitcli(7), The Git User’s Manual[1], gitworkflows(7)

Especially

  • man gitcore-tutorial
  • man giteveryday
  • man gitglossary
  • man gittutorial
  • man gitworkflows

They are worth a look.

TIL - Git --color-moved

TIL - Git --color-moved

TIL, Today I Learned, is more of a "I just figured this out: here are my notes, you may find them useful too" rather than a full blog post

Did you know that Git is able to detect moved blocks and use different colors from the usual added/removed lines? Me neither until now.

The paremeter is --color-moved and has been around since v0.4.0, so there is no new feature.

This is from the git manual page [1] :

--color-moved[=<mode>]
Moved lines of code are colored differently. The <mode> defaults to no if the option is not given and to zebra if the option with no mode is given. The mode must be one of:

no
Moved lines are not highlighted.

default
Is a synonym for zebra. This may change to a more sensible mode in the future.

plain
Any line that is added in one location and was removed in another location will be colored with color.diff.newMoved. Similarly color.diff.oldMoved will be used for removed lines that are added somewhere else in the diff. This mode picks up any moved line, but it is not very useful in a review to determine if a block of code was moved without permutation.

blocks
Blocks of moved text of at least 20 alphanumeric characters are detected greedily. The detected blocks are painted using either the color.diff.{old,new}Moved color. Adjacent blocks cannot be told apart.

zebra
Blocks of moved text are detected as in blocks mode. The blocks are painted using either the color.diff.{old,new}Moved color or color.diff.{old,new}MovedAlternative. The change between the two colors indicates that a new block was detected.

dimmed-zebra
Similar to zebra, but additional dimming of uninteresting parts of moved code is performed. The bordering lines of two adjacent blocks are considered interesting, the rest is uninteresting. dimmed_zebra is a deprecated synonym.

The --color-moved argument can be used for any commands that generates a diff, such as git diff, git log -p, git stash show and more.

Example

This is an example of git diff where I have moved a function and made som small changes:

/media/git-block-diff.png

That the block is moved is clear, but it is not obvious what have changed in the block itself.

If we instead use --color-moved, we get a different style on the output:

/media/git-block-diff-color.png

The magenta and cyan is the moved block, and in addition, we get the red/green colors for removed/added lines.

Make it default

Set diff.colormoved to default (or any other mode) in your .gitconfig.

For example:

$git config --global diff.colormoved default

TIL - Buildroot and LIBFOO_LINUX_CONFIG_FIXUPS

TIL - Buildroot and LIBFOO_LINUX_CONFIG_FIXUPS

TIL, Today I Learned, is more of a "I just figured this out: here are my notes, you may find them useful too" rather than a full blog post

Some applications in a Linux system depends on certain kernel features to work properly. I'm currently working on adding support for CRIU [1] in Buildroot [2] which has such requirements.

That's when I stumble upon the LIBFOO_LINUX_CONFIG_FIXUP variable.

LIBFOO_LINUX_CONFIG_FIXUPS

As the documentation [3] says:

* +LIBFOO_LINUX_CONFIG_FIXUPS+ lists the Linux kernel configuration
  options that are needed to build and use this package, and without
  which the package is fundamentally broken. This shall be a set of
  calls to one of the kconfig tweaking option: `KCONFIG_ENABLE_OPT`,
  `KCONFIG_DISABLE_OPT`, or `KCONFIG_SET_OPT`.
  This is seldom used, as package usually have no strict requirements on
  the kernel options.

This is a variable to tweak the kernel configuration from the package itself. In my case I had to:

define CRIU_LINUX_CONFIG_FIXUPS
         $(call KCONFIG_ENABLE_OPT,CONFIG_CHECKPOINT_RESTORE)
endef

To enable CONFIG_CHECKPOINT_RESTORE upon package selection.

TIL - ignore_routes_with_linkdown

TIL - ignore_routes_with_linkdown

TIL, Today I Learned, is more of a "I just figured this out: here are my notes, you may find them useful too" rather than a full blog post

Usually, a high level network managers such as NetworkManager or Connman, removes routes for unaccessible ("linkdown" or "unplugged") links states.

In some cases, these links are not managed eiter by a network manager but are managed in other ways or in somehow misconfigured, then you can end up with a default route still hangs around even if the link is down.

It doesn't sound too bad, if it weren't for the fact that it is still active and if its metric value is lower than some other default route, it will be the chosen route.

Consider the following routes:

# ip route
default via 10.42.0.1 dev rndis proto static metric 100 linkdown
default via 192.168.30.1 dev br0 proto static metric 425
10.42.0.0/24 dev rndis proto kernel scope link src 10.42.0.20 metric 100 linkdown
192.168.30.0/23 dev br0 proto static scope link metric 425
192.168.30.0/23 dev br0 proto kernel scope link src 192.168.30.97 metric 425

Even if the rndis interface has the "linkdown" flag, Linux will use it as default route regardless (lowest metric value) for any FIB (Forward Information Base) lookup.

/media/ignore_routes_with_linkdown.png

The kernel provide a feature to ignore routes that use links with the down state; ignore_routes_with_linkdown.

This is a rather old feature [1] introduced in v4.2 and documented [3] as follows:

ignore_routes_with_linkdown - BOOLEAN
        Ignore routes whose link is down when performing a FIB lookup.

When set, the route will not only be set as linkdown but dead linkdown to indicate that the route is currently ignored during a FIB lookup.

The flag could be set with some granuality, both for IPv4/IPv6, all interfaces or just on a single interface. For example:

$sysctl -a | grep ignore_routes_with_link
net.ipv4.conf.all.ignore_routes_with_linkdown = 0
net.ipv4.conf.default.ignore_routes_with_linkdown = 0
net.ipv4.conf.docker0.ignore_routes_with_linkdown = 0
net.ipv4.conf.enp0s13f0u2u4.ignore_routes_with_linkdown = 0
net.ipv4.conf.enp0s31f6.ignore_routes_with_linkdown = 0
net.ipv4.conf.eth0.ignore_routes_with_linkdown = 0
net.ipv4.conf.lo.ignore_routes_with_linkdown = 0
net.ipv4.conf.tun0.ignore_routes_with_linkdown = 0
net.ipv4.conf.wlp0s20f3.ignore_routes_with_linkdown = 0
net.ipv6.conf.all.ignore_routes_with_linkdown = 0
net.ipv6.conf.default.ignore_routes_with_linkdown = 0
net.ipv6.conf.docker0.ignore_routes_with_linkdown = 0
net.ipv6.conf.enp0s13f0u2u4.ignore_routes_with_linkdown = 0
net.ipv6.conf.enp0s31f6.ignore_routes_with_linkdown = 0
net.ipv6.conf.eth0.ignore_routes_with_linkdown = 0
net.ipv6.conf.lo.ignore_routes_with_linkdown = 0
net.ipv6.conf.tun0.ignore_routes_with_linkdown = 0
net.ipv6.conf.wlp0s20f3.ignore_routes_with_linkdown = 0

Set the flag

The flag could be set in a few different ways:

  • Via sysctl [4] on command line:

    sudo sysctl -w net.ipv4.conf.all.ignore_routes_with_linkdown=1
    
  • Via sysctl configuration file:

    echo "net.ipv4.conf.all.ignore_routes_with_linkdown = 1" | sudo tee /etc/sysctl.conf
    sudo /sbin/sysctl -p
    
  • Via procfs:

    echo 1 | sudo tee /proc/sys/net/ipv4/conf/all/ignore_routes_with_linkdown