Managing profiles declaratively

When we use the guix command line tool we add and remove applications interactively in a profile: often this is the implicit default profile ~/.guix-profile. Managing multiple profiles using this imperative approach is difficult, and if you want the same profile on multiple hosts it becomes almost impossible to stay in sync. The alternative is to use a declarative approach with a manifest for each profile that's created.

The manifest for each profile is kept in Git so its contents can be synchronised across multiple hosts. All my different profile manifests are managed by a dot file manager: it deploys the latest version of each manifest onto all my hosts. I've been using DotBot to manage my configuration files for a long time, it's definitely worth checking out. Guix also provides a native capability for this - called Guix Home - that I'd like to explore in future.

It's important to remember that using a manifest is different to how you add packages manually. A manifest creates a new generation of a profile exactly as defined in the manifest file: it will remove any applications that were previously installed in the profile if they are not listed in the manifest file. It is fully declarative. If a package is added to a profile using the guix command line tool then when you next use the manifest this will remove the package that was installed imperatively. Imperatively using the command line tooling, and declaratively using manifests do not mix.

To keep to a declarative approach every time I want to make a change I edit the manifests in my configuration repository and then run DotBot to deploy them into $HOME/.config/guix/manifests. After they are on the host I can activate them whenever I need them with one of my scripts.

Activate profiles

With lots of profiles the first problem is how to activate them. This script, originally from David Wilson, makes it easy to activate one of the available profiles. It can also activate multiple, or all profiles.

Using our configuration manager we place a set of manifests in $HOME/.config/guix/manifests/. David's script accepts a space separated list of manifest file names (without extensions):

# list the available profiles
$ ls $HOME/.config/guix/manifests
core-shell.scm dev-apps.scm Xwindows-apps.scm

# activates all profiles
$ activate-profiles

# activates a specific profile
$ activate-profiles core-shell

The script processes the manifest file, creating the profile under ~/guix-extra-profiles and installing the specified applications. Finally, it sources the profile, altering $PATH so that the new commands are available.

To use it add the following to .bashrc:

alias activate-profiles='source activate-profiles'

Create a file somewhere on your $PATH called activate-profiles. The script is:

GREEN='\033[1;32m'
RED='\033[1;31m'
NC='\033[0m'
GUIX_EXTRA_PROFILES=$HOME/.guix-extra-profiles
GUIX=$HOME/.config/guix/current/bin/guix

if ! test -e $GUIX ; then
    echo -e "${RED}Guix command doesn't exist${NC}"
    exit 0
fi

profiles=$*
if [[ $# -eq 0 ]]; then
    profiles="$HOME/.config/guix/manifests/*.scm";
fi

for profile in $profiles; do
  # Remove the path and file extension, if any
  profileName=$(basename $profile)
  profileName="${profileName%.*}"
  profilePath="$GUIX_EXTRA_PROFILES/$profileName"
  manifestPath=$HOME/.config/guix/manifests/$profileName.scm

  if [ -f $manifestPath ]; then
    echo
    echo -e "${GREEN}Activating profile:" $manifestPath "${NC}"
    echo

    mkdir -p $profilePath
    $GUIX package --manifest=$manifestPath --profile="$profilePath/$profileName"

    # Source the new profile
    GUIX_PROFILE="$profilePath/$profileName"
    if [ -f $GUIX_PROFILE/etc/profile ]; then
        . "$GUIX_PROFILE"/etc/profile
        echo -e "${GREEN} Sourced the profile $GUIX_PROFILE ${NC} \n"
        #echo "PATH in activate-profiles is ${PATH}"
    else
        echo -e "${RED}Couldn't find profile:" $GUIX_PROFILE/etc/profile "${NC} \n"
    fi
  else
    echo -e "${RED}No profile found at path:" $profilePath "${NC}"
  fi
done

I'm not much of a shell scripter, so it took me a while to realise that there's no #!/bin/bash line at the start of the script because it's designed to be directly sourced as an alias.

List multiple profiles

Now that we have multiple profiles created we need to be able to list them. All our active profiles are in ~/.guix-extra-profiles . This script, again originally from David Wilson, goes through that directory and lists all the profiles that we have.

To use it add the following to .bashrc:

alias list-profiles = 'source list-profiles'

Then we use it like so:

$ list-profiles
Profiles are: all-shell all-Xwin base-shell dev-shell dev-Xwin util-shell util-Xwin

Create a file called list-profiles in a location on your $PATH: for me I keep local scripts like this in ~/.bin:

GREEN='\033[1;32m'
RED='\033[1;31m'
NC='\033[0m'
GUIX_EXTRA_PROFILES=$HOME/.guix-extra-profiles

if [ -n "$(ls -A $GUIX_EXTRA_PROFILES 2>/dev/null)" ]; then
    echo -ne "${GREEN}Profiles are:"
    for profile in $GUIX_EXTRA_PROFILES/*; do
        profileName=$(basename $profile)
        echo -ne " $profileName"
    done
    echo -e " ${NC}"
else
    echo -e "${RED}No profiles in $GUIX_EXTRA_PROFILES ${NC}"
fi

Run a Guix command in a profile

With a lot of profiles it's useful to be able to run a Guix command in a specific profile. For example, list-generations or delete generations. Pavel Korytov has a nice script that I've adapted.

To use it we do something like this:

guix-prfctl [profile name] [command]

guix-prfctl base-shell --list-generations
guix-prfctl base-shell --delete-generations

The script is:

#!/bin/bash
GREEN='\033[1;32m'
RED='\033[1;31m'
#RED='\033[1;30m'
NC='\033[0m'
GUIX_EXTRA_PROFILES=$HOME/.guix-extra-profiles
GUIX=$HOME/.config/guix/current/bin/guix
profileName=$(basename $1)
profileName="${profileName%.*}"
profilePath="$GUIX_EXTRA_PROFILES/$profileName"

if ! test -e $GUIX ; then
    echo -e "${RED}Guix command doesn't exist${NC}"
    exit 0
fi

if [ -d $profilePath ]; then
    $GUIX package --profile="$profilePath/$profileName" ${@:2}
    if [ $? -eq 0 ]; then
        echo -e "${GREEN}Success running ${@:2} for ${profileName}${NC}"
    else
        echo -e "${RED}Failed running ${@:2} for ${profileName}${NC}"
    fi
else
    echo -e "${RED}No profile found at path: ${profilePath} ${NC}"
fi

Manually update all profiles

See the next section for an automated approach to updating Guix and reinstallng all profiles daily. However, sometimes it's useful to manually upgrade all the profiles that you have in ~/.guix-extra-profiles. Again, David Wilsons scripts were a great starting point.

I add this to my .bashrc as:

alias update-profiles='source update-profiles'

It's worth recalling that technically we are not "upgrading" in the way you might think if you're used to Debian. What we're doing is updating the list of available packages, and then installing them all - if there are any new versions this will install a newer version.

GREEN='\033[1;32m'
RED='\033[1;31m'
NC='\033[0m'
GUIX_EXTRA_PROFILES=$HOME/.guix-extra-profiles
GUIX=$HOME/.config/guix/current/bin/guix

profiles=$*
if [[ $# -eq 0 ]]; then
    profiles="$GUIX_EXTRA_PROFILES/*";
fi

for profile in $profiles; do
  profileName=$(basename $profile)
  profilePath=$GUIX_EXTRA_PROFILES/$profileName

  echo
  echo -e "${GREEN}Updating profile:" $profilePath "${NC}"
  echo

  $GUIX package --profile="$profilePath/$profileName" --manifest="$HOME/.config/guix/manifests/$profileName.scm"
done

Automating updates

Updating each profile manually is quite cumbersome so I wrote a script that updates Guix's repository of packages and then upgrades (reinstalls) the manifest for each profile. I run it from crontab:

23 23 * * * /bin/bash /home/steve/bin/guix-upgrade.sh 2>&1 > /dev/null

The script itself is:

#!/bin/bash

GREEN='\033[1;32m'
#RED='\033[1;31m'
RED='\033[1;30m'
NC='\033[0m'
GUIX_EXTRA_PROFILES=$HOME/.guix-extra-profiles
GUIX=$HOME/.config/guix/current/bin/guix
LOG=/tmp/guix-upgrade.log
manifestPath=$HOME/.config/guix/manifests

if ! test -e $GUIX ; then
    echo "FAILED: Guix command doesn't exist" > $LOG 2>&1
    echo -e "${RED}Guix command doesn't exist${NC}"
    exit 0
fi

# called without anything it does a guix pull for the user
f_guix_pull() {
    PROFILE=${1:-~/.guix-profile}

    echo "SCRIPT-> Pulling down package updates for ${PROFILE}" >> $LOG 2>&1
    echo -e "${GREEN}SCRIPT-> Pulling down packages updates for ${PROFILE} ${NC}"

    # the return value will be the from the tee unless you have pipefail
    # set +o pipefail
    #$GUIX pull --profile=${PROFILE} 2>&1 | tee /tmp/guix-upgrade.log
    $GUIX pull >> $LOG 2>&1

    if [ $? -eq 0 ]; then
        echo "SCRIPT-> SUCCESS: updating package listing for ${PROFILE}" >> $LOG 2>&1
        echo -e "${GREEN}SCRIPT-> Success updating package listing for ${PROFILE}${NC}"
    else
        echo "SCRIPT-> FAILED: updating package listing for ${PROFILE}" >> $LOG 2>&1
        echo -e "${RED}SCRIPT-> Failed updating package listing for ${PROFILE}${NC}"
    fi

    return 0
}

# lists all the profiles and upgrades each one
# if there are no extra profiles upgrades the default profile
f_profile_list_and_upgrade() {
    if [ -n "$(ls -A $GUIX_EXTRA_PROFILES 2>/dev/null)" ]; then
        for profile in $GUIX_EXTRA_PROFILES/*; do
            profileName=$(basename $profile)
            profilePath=$GUIX_EXTRA_PROFILES/$profileName
            echo "SCRIPT-> Profile to upgrade is $profileName" >> $LOG 2>&1
            echo -e "${GREEN}SCRIPT-> Profile to upgrade is $profileName${NC}"
            f_guix_manifest_install "$manifestPath/$profileName.scm" "$profilePath/$profileName"
        done
    else
        echo "SCRIPT-> No additional profiles in $GUIX_EXTRA_PROFILES" >> $LOG 2>&1
        echo -e "${RED}SCRIPT-> No additional profiles in $GUIX_EXTRA_PROFILES ${NC}"
    fi

    return 0
}

# Receives the manifest that a profile was based on. Installs the manifest
# using the latest versions that were updated from guix pull.
# This upgrades a profile by using the manifest ensuring it is reproducible
f_guix_manifest_install() {
    MANIFEST=${1}
    PROFILE=${2}

    echo "SCRIPT-> Installing packages for ${MANIFEST}" into ${PROFILE} >> $LOG 2>&1
    echo -e "${GREEN}SCRIPT-> Installing packages for ${MANIFEST} into ${PROFILE} ${NC}"

    $GUIX package --manifest=${MANIFEST} --profile=${PROFILE} --install >> $LOG 2>&1

    if [ $? -eq 0 ]; then
        echo "SCRIPT-> SUCCESS: installing packages for ${MANIFEST} into ${PROFILE}" >> $LOG 2>&1
        echo -e "${GREEN}SCRIPT-> Success installing packages for ${MANIFEST} into ${PROFILE} ${NC}"
    else
        echo "SCRIPT-> FAILED: Failed installing packages for ${MANIFEST} into ${PROFILE}" >> $LOG 2>&1
        echo -e "${RED}SCRIPT-> Failed installing packages for ${MANIFEST} into ${PROFILE} ${NC}"

    fi

    return 0
}

# Receives a specific profile or acts on the default profile. Upgrades
# packages to the latest versions we're aware of. It's preferable to
# use manifests and f_guix_manifest_install as the upgrade path as it's
# more certain and reproducible
f_guix_upgrade() {
    PROFILE=${1:-~/.guix-profile}

    echo "SCRIPT-> Upgrading packages for ${PROFILE}" >> $LOG 2>&1
    echo -e "${GREEN}SCRIPT-> Upgrading packages for ${PROFILE} ${NC}"

    $GUIX upgrade --profile=${PROFILE} >> $LOG 2>&1

    if [ $? -eq 0 ]; then
        echo "SCRIPT-> SUCCESS: upgrading ${PROFILE}" >> $LOG 2>&1
        echo -e "${GREEN}SCRIPT-> Success upgrading ${PROFILE} ${NC}"
    else
        echo "SCRIPT-> FAILED: upgrading ${PROFILE}" >> $LOG 2>&1
        echo -e "${RED}SCRIPT-> Failed upgrading ${PROFILE} ${NC}"
    fi

    return 0
}

echo "SCRIPT-> START: " `date` > $LOG 2>&1
echo -e "${GREEN}SCRIPT-> START: " `date` "${NC}"

# Do Guix pull, then for each profile upgrade
f_guix_pull
f_guix_upgrade
f_profile_list_and_upgrade

echo "SCRIPT-> END:" `date` >> $LOG 2>&1
echo -e "${GREEN}SCRIPT-> END: " `date` "${NC}"
exit 0

The f_guix_pull function is called initially to do a 'guix pull' using the default (implicit) profile. It then does a guix upgrade on the default profile in the f_guix_upgrade function. Finally, the f_profile_list_and_upgrade function goes through each profile under .guix-extra-profiles and (re)installs the appropriate manifest for the profile.

Summary

Guix's concept of a profile that is created by a manifest is really useful for creating set-ups that are specific to a particular project, or other aspect. But, before you know it there's a proliferation of profiles - managing them with manifests and a configuration management system is the best method for keeping a clean system.

I keep saying 'aspects' which is very vague because I personally like to divide my applications into sets - core shell applications in one manifest, X Windows applications in a different manifest. In the next post we'll look at this layered manifest approach.