Automating provisioning Arch continued - dotfiles

How I do all the user level configuration for my system, including setting up my python environment.
configuration
linux
arch
python
Published

November 25, 2020

This is part 3 of a 4 part series describing how I provision my systems. Links to each part are below:

Introduction

In this post I’ll go through setting up configuration files (also called dotfiles or rcs).

As with the earlier posts in this series I’ll be copying liberally from Brennan Fee. Since I’m building off his guide I will also use RCM to manage my dotfiles. At the time of this writing I don’t think I need a lot of the tag or host specific tools it offers, and I could probably stick with using my old approach of a bare git repository right in my home. This seems a bit cleaner though, and maybe I’ll want to extend it eventually. The idea of being able to extend things to WSL or MacOS is cool and I could see using host or tag specific features to be able to use these on work machines as well.

I have a hard time remembering why I structure my dotfiles the way I do, or what some features do, so hopefully by writing all this out I will make it easier to update them in the future. As a bonus maybe this will be useful to others looking to configure their own environments. I’m not going to dissect the files in this document, it’s too easy to let things get out of sync between this guide and my actual dotfiles. Instead this will focus on explaining the directory structure and what files do what, since that’s not easily captured by comments. I’ll try to heavily comment the actual files in the repository. That will have a slightly better chance of staying relevant.

Repository setup

Following Brennan’s example, I have two repositories to contain my configs. The first is for configuration I don’t mind sharing and it’s available here. In addition to that I have a second private repository for configurations that contain personal information, which I will not be sharing a link to for obvious reasons.

Just for reference in this guide the public dotfiles repository is cloned to ~/.dotfiles, which is the default location for RCM. The private ones are cloned to ~/.private_dotfiles. That one I’ll have to manually specify, and if I want to add a file to it I’ll have to move it over manually.

Repository root files

setup.sh

This file either links in or generates the config file for RCM (rcrc). This is the file that identifies which tags are applicable for the machine so it has to be configured properly before the rest of the dotfiles can be brought up. The base implementation checks what Operating System you’re running and adds tags for that. At work I have it generate additional tags for which user is running it so I can create user specific tagged files (for things like email addresses).

After running this script, if we didn’t link in an already existing .rcrc file then you’ll have a host specific one generated, but it won’t be saved in the repository, it will just be a reglar file. If as prompted you run mkrc -o ~/.rcrc it will add a host specific rcrc file to the repository.

base-rcrc

This file is used by setup.sh to generate the host specific ~/.rcrc. The script adds tags to this file based on the operating system you’re running. You can add additional tags if you’d like.

other root files

The other files in the root of the repository are generic repository management files. README.md will show on the base of the page on GitHub and should point back to this blog post for more details. I picked GPL V3 for the license somewhat arbitrarily. I think I used the GitHub license picker helper for it. .gitignore and .gitattributes handle files for git to ignore and enforce consistent line break characters. .editorconfig tells a variety of text editors things like whether to use tabs or spaces for indentation.

bash

This folder contains all the stuff that gets loaded into my profile at login. It’s where things like custom functions and the layout of my command prompt are defined.

completions

These scripts let you tab complete commands for certain applications. At the time of this writing I have completions for git, pipx and poetry installed.

distros/manjaro/aliases

I don’t actually use manjaro, but I wanted to keep this in as an example for myself of how to set distribution specific functionality.

linux

This has a few commands to set start or open to run xdg-open in linux. Makes the syntax compatible against platforms. That would be for opening a file in a gui rather than with a command line app.

macos

I don’t have any mac machines to test this stuff out on right now. It’s got a few files that presumably help make behaviour consistent on macs.

windows-wsl

Similar to the mac and linux entries above. Lets you use the same commands regardless of your specific platform.

nerdfonts

This maps a bunch of nerd fonts to environment variables so they can be included in shell scripts. It lets you do things like put a check mark in your command prompt. Very important stuff.

shared

This is where the bulk of the content is in the bash directory. All of these files are cross platform and should work the same on linux, mac or WSL.

aliases

Basically these are all the command shortcuts. For example alias grep="grep --color" means you can just type grep but get nicely coloured results.

exports

This is where environment variables are set. For example EDITOR=vim is set here.

functions

This is where user defined functions/tools live. For example extract is defined here to call the appropriate underlying app to extract a file based on its extension.

options

Sets a bunch of shell options. Things like turning on vi mode for the command line.

other

A catch all. Code to set up conda, manage the file path, and actually set the appearance of my command prompt all live here.

third party

A place to dump cool code snippets you found on the internet that you want to be able to manage in your shell.

bin

As opposed to the functions in the bash folder that get added to your environment, these are scripts that are supposed to be called directly, and are therefore on the path but not parsed until they’re called. At least I think that’s the distinction. I’m not super good at bash namespaces yet so this might need to be edited.

files/mac and iterm2

I don’t have a mac, not totally sure what this stuff does. But maybe some day I will! Then it’ll be super nice to have this stuff enabled… I assume.

rcs

This is where the actual config files live

bash_logout

Clear the screen when you log out. I’m not sure if I actually need this, doesn’t seem to hurt

bash_profile

I’m sure in theory there’s a difference between this file and .bashrc but in practice they seem to be the same. Just map this one to load ~/.bashrc so whichever one your terminal expects you get the same result.

bashrc

bashrc configures your shell on login. Brennan has a nice modular design that I’m going to emulate. Basically nothing goes in bashrc itself, rather it walks through all the folders in the previously described bash folder and adds them in (at least those relevant to your Operating System). A snippet of what that looks like is below.

# We want to walk "outside" in... which is to say run all options files first, then all
# exports, then all functions, etc.
for folder in "options" "exports" "functions" "third-party" "other" "aliases"; do
  for base in "shared" "$OS_PRIMARY" "distros/$OS_SECONDARY"; do
    for root in "$DOTFILES/bash" "$DOTFILES_PRIVATE/bash"; do
      if [[ -d "$root/$base/$folder" ]]; then
        for file in $root/$base/$folder/*.bash; do
          # shellcheck source=/dev/null
          source "$file"
        done
      fi
    done
  done
done

All the actual functionality lives in the bash folders of the dotfiles repositories and only this file itself needs to be linked in by RCM. Distribution and OS specific functionality can be managed by just placing the script in the appropriate folder. Because of the order of execution the more granular files will overwrite more general settings if there’s a conflict.

other files

  • dircolors: make ls show pretty colours.
  • gitignore: files and patterns to ignore in all git repositories
  • inputrc: manage basic keyboard mappings for the shell (home to go to the beginning of the line for example)
  • prettierrc: configurations for the code formatter prettier. Kind of like black for other languages
  • tmux.conf: configuration for tmux. I don’t use tmux enough to have strong opinions about these commands so the commenting is pretty sparse at the time of this writing

config folder

Polite applications store their configuration files here rather than your home directory. The Arch Wiki has a good list of polite applications and how to override some of the impolite ones. The folders all correspond to the name of the application they configure (e.g. git) so they layout is pretty self explanatory.

host-* folders

Host specific configs. Everything within here will have the same layout as the rcs folder above it, but will have machine specific configs. For my setup that’s just the ~/.rcrc file that sets the tags for everything else on the machine.

tag-* folders

The same idea as hosts, except each host can have multiple tags. In general this is used for OS specific configurations. At work I also add tags for each user on the system for things like configuring e-mail addresses.

vifm and vim

These folder should really be under config. They’re just the settings for vim and vifm. Rude of them to demand their own space in ~.

hooks

These just live in the rcs folder, but they’re special so I want to give them their own place. Some configurations need to have setup steps run, either before they’re installed or after. For instance, you can specify which plugins you want vim to use, but they won’t actually be installed until you run vim -N -u "$HOME/.vim/vimrc.bundles" +PlugUpdate +PlugClean! +qa -. You can put a script that does that in hooks/post-up and it will automatically run that after loading in your configuration files. I’m pushing this feature a little beyond what it’s intended to install some user level things like miniconda and pyenv. Doing that with this tool doesn’t quite fit its intended use, but it seems to work so I’ll stick with it.

conclusion

This guide gave an overview of the structure of my dotfiles. For more details on the tool used to set them up check out RCM and for the specifics of the configurations check the files themselves in my repository.