TIL - git credential storage

TIL - git credential storage

TIL, Today I Learned, is more of a "I just figured this out: here are my notes, you may find them useful too" rather than a full blog post

When using SSH as transport protocol for connecting to remotes you could use a key without need to type a username nor password. Unfortunately, this is not possible when the underlaying protocol is HTTPS as it requires a username and password for every connection made.

This is where git credential store gets a chance to shine.

Credential storage

Git has a credential system. All you need to do is to set the credential.helper in your .gitconfig.

Possible values are:

  • none, this is the default behavior. You will be promted for username and password for every connection made.
  • cache, this option caches the credentials in memory for a certain period of time. Default is 15 minutes. The passwords are purged after this period.
  • store, saves the credentials in a (plain-text) file.

For example:

$ git config credential.helper store

Will create this entry in your .gitconfig:

[credential]
        helper = store

The default storage file is ~/.git-credentials, you may change it by including the --file option. E.g.:

$ git config credential.helper 'store --file ~/.my-secrets'

For more information, see the manpage for gitcredentials [1].

KAS

KAS [2] is my favorite tool to setup bitbake based projects. I do always build my images using kas-container, as the state of my Archlinux setup is not always... compatible. (I love rolling distributions though).

However, many tools, including kas-container, makes use of git credential files for authentication.

kas-container has the --git-credential-store <file> option to specify the credential file.

Git version in cmake

Git version in CMake

All applications have versions. The version should somehow be exposed in the application to make it possible to determine which application we are actually running.

I've seen a plenty of variants on how this is achieved, some are good and some are really bad. Since it's such a common thing, I thought I'd show how I usually do it.

I use to let CMake determine the version based on git describe and tags, the benefit's that it is part of the build process (i.e no extra step on the build server - I've seen it all...), and you will get right version information also for local builds.

git describe also gives you useful information which allows you to derive the build to the actual commit. Incredible useful.

The version information will be stored as a define in a header file which should be included into the project.

Hands on

The version file we intend to include in the project is generated from Version.h.in:

#pragma once
#define SOFTWARE_VERSION "@FOO_VERSION@"

@FOO_VERSION@ is a placeholder that will be replaced by configure_file() later on.

The "core" functionality is in GenerateVersion.cmake:

find_package(Git)

if(GIT_EXECUTABLE)
  get_filename_component(WORKING_DIR ${SRC} DIRECTORY)
  execute_process(
    COMMAND ${GIT_EXECUTABLE} describe --tags --dirty
    WORKING_DIRECTORY ${WORKING_DIR}
    OUTPUT_VARIABLE FOO_VERSION
    RESULT_VARIABLE ERROR_CODE
    OUTPUT_STRIP_TRAILING_WHITESPACE
    )
endif()

if(FOO_VERSION STREQUAL "")
  set(FOO_VERSION 0.0.0-unknown)
  message(WARNING "Failed to determine version from Git tags. Using default version \"${FOO_VERSION}\".")
endif()

configure_file(${SRC} ${DST} @ONLY)

It will look for the git package, receive the tag information with git describe and store the version in the FOO_VERSION variable.

configure_file() is then used to create a version.h out of version.h.in where the FOO_VERSION placeholder is replaced by the actual version.

To use GenerateVersion.cmake, we have to create a custom target in the CMakeLists.txt file and provide the source and destination path for the version file:

add_custom_target(version
  ${CMAKE_COMMAND} -D SRC=${CMAKE_SOURCE_DIR}/Version.h.in
                   -D DST=${CMAKE_SOURCE_DIR}/Version.h
                   -P ${CMAKE_SOURCE_DIR}/GenerateVersion.cmake
  )

Finally, make sure that the target (${PROJECT_NAME} in this case) depends on the version target:

add_dependencies(${PROJECT_NAME} version)

That's basically it.

TIL - git man-pages

TIL - git man-pages

TIL, Today I Learned, is more of a "I just figured this out: here are my notes, you may find them useful too" rather than a full blog post

I'm a big user of man-pages.

Today I found a set of pages that I've not noticed before when I was reading man git:

SEE ALSO
        gittutorial(7), gittutorial-2(7), giteveryday(7), gitcvs-migration(7), gitglossary(7), gitcore-tutorial(7),
       gitcli(7), The Git User’s Manual[1], gitworkflows(7)

Especially

  • man gitcore-tutorial
  • man giteveryday
  • man gitglossary
  • man gittutorial
  • man gitworkflows

They are worth a look.

TIL - Git --color-moved

TIL - Git --color-moved

TIL, Today I Learned, is more of a "I just figured this out: here are my notes, you may find them useful too" rather than a full blog post

Did you know that Git is able to detect moved blocks and use different colors from the usual added/removed lines? Me neither until now.

The paremeter is --color-moved and has been around since v0.4.0, so there is no new feature.

This is from the git manual page [1] :

--color-moved[=<mode>]
Moved lines of code are colored differently. The <mode> defaults to no if the option is not given and to zebra if the option with no mode is given. The mode must be one of:

no
Moved lines are not highlighted.

default
Is a synonym for zebra. This may change to a more sensible mode in the future.

plain
Any line that is added in one location and was removed in another location will be colored with color.diff.newMoved. Similarly color.diff.oldMoved will be used for removed lines that are added somewhere else in the diff. This mode picks up any moved line, but it is not very useful in a review to determine if a block of code was moved without permutation.

blocks
Blocks of moved text of at least 20 alphanumeric characters are detected greedily. The detected blocks are painted using either the color.diff.{old,new}Moved color. Adjacent blocks cannot be told apart.

zebra
Blocks of moved text are detected as in blocks mode. The blocks are painted using either the color.diff.{old,new}Moved color or color.diff.{old,new}MovedAlternative. The change between the two colors indicates that a new block was detected.

dimmed-zebra
Similar to zebra, but additional dimming of uninteresting parts of moved code is performed. The bordering lines of two adjacent blocks are considered interesting, the rest is uninteresting. dimmed_zebra is a deprecated synonym.

The --color-moved argument can be used for any commands that generates a diff, such as git diff, git log -p, git stash show and more.

Example

This is an example of git diff where I have moved a function and made som small changes:

/media/git-block-diff.png

That the block is moved is clear, but it is not obvious what have changed in the block itself.

If we instead use --color-moved, we get a different style on the output:

/media/git-block-diff-color.png

The magenta and cyan is the moved block, and in addition, we get the red/green colors for removed/added lines.

Make it default

Set diff.colormoved to default (or any other mode) in your .gitconfig.

For example:

$git config --global diff.colormoved default

TIL - Git jump

TIL - Git jump

TIL, Today I Learned, is more of a "I just figured this out: here are my notes, you may find them useful too" rather than a full blog post

The Git v2.40.0 was released [1] yesterday (2023-03-13) and one of the release notes that caught my eyes was this one:

* "git jump" (in contrib/) learned to present the "quickfix list" to
  its standard output (instead of letting it consumed by the editor
  it invokes), and learned to also drive emacs/emacsclient.

I'm familiar with the "quickfix list" in Vim as I have been using it for many years, I even wrote a small post [2] about it looong time ago.

But what about git jump thing?

What is "quickfix list"

First a few words about quickfix. Quickfix is a list of "interesting" locations in a project such as compilation errors, lint warnings or whatever you want to feed the list with.

Vim has several commands for navigating the quickfix list, here is the most common ones:

  • :copen - Open the quickfix list window.
  • :ccl or :cclose - Close the quickfix list window.
  • :cnext or :cn - Go to the next item on the list.
  • :cprev or :cp - Go to the previous item on the list.
  • :cfirst - Go to the first item on the list.
  • :clast - Go to the last item on the list.

I almost exclusively use the :copen in my daily work. The quickfix list window let you iterate through all items and easily navigate to selected position.

Example on a quickfix window generated with git jump grep compatible:

/media/vim-quickfix.jpg

Git-jump

Git-jump [3] is a contribution script (has it's location in the ./contrib directory of the git repository) and has actually been around since 2011 without me noticing.

Git-jump can help you generate "interesting" quickfix lists and feed it to vim directly. It currently support these lists:

  • The beginning of any diff hunks.

    git jump diff
    
  • The beginning of any merge conflict markers.

    git jump merge
    
  • Any grep matches, including the column of the first match on a line.

    git jump grep compatible
    
  • Any whitespace errors detected by git diff --check.

The script does also supports Emacs, for those who like that kind of thing.

Conclusion

I have only been aware of this tool for a day and it is already totally integrated in my daily workflow.

The commands that I use most are git jump diff and git jump grep. Those are just awesome.

Patch changelogs with git-notes

Patch changelogs with git-notes

Git notes [1] is a neat function that has been around since v1.6.6. Notes is a kind of metadata that belongs to a certain commit but is stored separately (different git object) from the commit itself. The fact that it's a separate git object is important, it will therefor keep the commit hash intact, and as a bonus - it has its own diff log.

Basic usage

git-notes has support for add, append, copy edit, list, prune, remove and show subcommands. These subcommands is rather self-explanatory, so I will not describe them any further.

add and edit fits almost all my needs, so I will stick to them.

First, create a git repository

mkdir gitnotes
cd gitnotes/
git init
echo testfile > test
git add test
git commit --signoff -m "Initial commit"
echo add-more-stuff >> test
git commit --signoff -a -m "Feature X"

We now have two commits:

commit bce22657cb5fd8353d43537e6b64422a8ce1107d (HEAD -> master)
Author: Marcus Folkesson <marcus.folkesson@gmail.com>
Date:   Sun Feb 11 21:56:17 2018 +0100

    Feature X

    Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com>

commit 93726c3f6bf5c95faf969f040966b15d58d6fd0b
Author: Marcus Folkesson <marcus.folkesson@gmail.com>
Date:   Sun Feb 11 21:55:21 2018 +0100

    Initial commit

    Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com>

Add a note

Add a note is simple:

git notes add -m "v2: update locking handling for Feature X"

We now have a note appended to our git log HEAD -1

commit bce22657cb5fd8353d43537e6b64422a8ce1107d (HEAD -> master)
Author: Marcus Folkesson <marcus.folkesson@gmail.com>
Date:   Sun Feb 11 21:56:17 2018 +0100

    Feature X

    Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com>

Notes:
    v2: update locking handling for Feature X

Note that the SHA, bce22657cb5fd8353d43537e6b64422a8ce1107d, hasn't changed.

Edit a note

Edit a note is as simple as adding one:

git notes edit HEAD

When to use notes?

The first field of application is obvious, keep notes about individual commits. The kind of notes I usually use is comments during log reviews.

For the last year I have been using notes for a more practical use; patch change log history, which is an incredible powerful but undocumented (try to google it yourself) area of use for notes.

Patch format and changelogs

When talking about patches, the --- marker line serves the essential purpose of marking for patch handling tools where the commit message ends. One good use for the additional comments after the --- marker is for patch changelogs, which describe what has changed between the v1 and v2 version of the patch.

Rebase and amend

When working with patches, it's not unusual that it will take a few iterations until the patch set is in a ready-to-merge state. The way to rework patches is with an interactive rebase and amending.

A note is linked to a certain git object and this will be a problem with rebase and amend since it will overwrite that git object. With that said; if you amend your commit, the notes will disappear... unless... you configure git to rewrite the note upon amend and rebase.

From git-notes(1):

notes.rewrite.<command>

When rewriting commits with <command> (currently amend or rebase) and this variable is set to true, Git automatically copies your notes from the original to the rewritten commit. Defaults to true, but see notes.rewriteRef below.

notes.rewriteRef

When copying notes during a rewrite, specifies the (fully qualified) ref whose notes should be copied. The ref may be a glob, in which case notes in all matching refs will be copied. You may also specify this configuration several times.

Does not have a default value; you must configure this variable to enable note rewriting. Set it to refs/notes/commits to enable rewriting for the default commit notes.

This setting can be overridden with the GIT_NOTES_REWRITE_REF environment variable, which must be a colon separated list of refs or globs.

The manpage simply tells you to add these lines to your ~/.gitconfig or .git/config

[notes "rewrite"]
    amend = true
    rebase = true
[notes]
    rewriteRef = refs/notes/commits

To make amend and rebase rewrite the notes.

Generate patches

The best part is that git format-patch --notes will generate patches with appended notes after the --- marker.

For example: git format-patch --notes -1 -v2 will generate v2-0001-Feature-X.patch:

From bce22657cb5fd8353d43537e6b64422a8ce1107d Mon Sep 17 00:00:00 2001
From: Marcus Folkesson <marcus.folkesson@gmail.com>
Date: Mon, 19 Feb 2018 21:42:45 +0100
Subject: [PATCH v2] Feature X

Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com>
---

Notes:
    v2: update locking handling for Feature X

 test | 1 +
 1 file changed, 1 insertion(+)

diff --git a/test b/test
index 2691857..066c37d 100644
--- a/test
+++ b/test
@@ -1 +1,2 @@
 testfile
+add-more-stuff
--
2.15.1

This makes the patch workflow real smooth instead of manually track and update your patch changelogs.

My typical workflow

/media/git-patch-workflow.png

Other tricks

Note logs

Even notes has a log (all objects in refs/XXX has), and it could be useful to see how your notes has evolved over time:

git log -p notes/commits

Push and fetch notes

The notes itself is stored as git object in .git/refs/notes. A thing to keep in mind is that refs/notes is not pushed by default, but that is not a problem - my opinion is that notes should be local only anyway.

If you really want to push your notes, you can to that with

git push <remote> refs/notes/*

and fetch them with

git fetch origin refs/notes/*:refs/notes/*

get_maintainers and git send-email

get_maintainers and git send-email

Many with me prefer email as communication channel, especially for patches. Github, Gerrit and all other "nice" and "user friendly" tools that tries to "help" you to manage your submissions does not simply fit my workflow.

As you may already know, all patches to the Linux kernel is by email. scripts/get_maintainer.pl (see [1] for more info about the process) is a handy tool that takes a patch as input and gives back a bunch of emails addresses. These email addresses is usually passed to git send-email [2] for submission.

I have used various scripts to make the output from get_maintainer.pl to fit git send-email, but was not completely satisfied until I found the --to-cmd and --cc-cmd parameters to git send-email:

--to-cmd=<command>
 Specify a command to execute once per patch file which should generate patch file specific "To:" entries. Output of this command must be single email address per line. Default is the value of sendemail.tocmd configuration value.
--cc-cmd=<command>
 Specify a command to execute once per patch file which should generate patch file specific "Cc:" entries. Output of this command must be single email address per line. Default is the value of sendemail.ccCmd configuration value.

I'm very pleased with these parameters. All I have to to is to put these extra lines into my ~/.gitconfig (or use git config):

[sendemail.linux]
    tocmd ="`pwd`/scripts/get_maintainer.pl --nogit --nogit-fallback --norolestats --nol"
    cccmd ="`pwd`/scripts/get_maintainer.pl --nogit --nogit-fallback --norolestats --nom"

To submit a patch, I just type:

git send-email --identity=linux ./0001-my-fancy-patch.patch

and let --to and --cc to be populated automatically.