Yet Another SysAdmin Website - Blog

How to check that an SSL certificate and its private key match

2026-03-09

Introduction

Though it is less common nowadays, SSL certificate provisioning can still be full of surprises when dealing with old school setups in client environments.

Some colleagues did not know how to easily check if a certificate and private key match in a clean one liner, and their LLM research came up with “interesting” suggestions with at least 6 pipes and too many checksum operations.

This prompted me to document my way of doing this.

Extracting the public key

The public key can be extracted with openssl using two commands. Here is the first one for the certificate:

openssl x509 -in mycertificate.crt -noout -pubkey

Here is the second one for the private key:

openssl pkey -in mykey.key -pubout

The diff command and process substitutions

Bash process substitutions are very useful, in particular in this context. They allow a command to read the output of other commands as pseudo files. For our purpose it looks like this:

diff -quw <(openssl x509 -in mycertificate.crt -noout -pubkey) \
 <(openssl pkey -in mykey.key -pubout)

The -quw flags are not important here, just a habit of mine. -q is to ask diff to be quiet, -u is to ask for a unified output while -w is to ignore whitespace differences.

If running the diff command from a script, know that it also exits 0 when the inputs are the same, 1 if they are different and >1 if another error occurred.

Conclusion

This provides a quick way to verify that a certificate and its private key match, while also demonstrating Bash process substitution. I encourage anyone to use these process substitutions more, they have many practical uses and can simplify command line workflows.

Updating dependencies

2026-02-04

Introduction

I find most of the popular dependency update automation tools like Dependabot or Renovate cumbersome to integrate, first and foremost because they cannot be run locally easily. Secondly, they cover everything and the kitchen sink and you cannot easily pick some languages or ecosystems to support. But if I could somehow ignore that, sadly they are clunky even in a forge setting as soon as you have to deal with private repositories and need to tightly manage access control.

I understand how valuable these popular tools can be though, especially in ecosystems that churn out new versions every other day. This makes these dependency update automation tools a must have at work where you need to reliably manage security updates across dozens of repositories I often do not own.

Since I try to avoid dependencies as much as I can in my personal development and infrastructure management, I believe I can get 90% of the value of these tools for 10% of the complexity thanks to a few lighweight scripts.

Updating dependencies

Docker Hub

The Docker Hub offers the an API endpoint to check the latest tags on a specific image. The trick is to write the right grep filter that will select only the release tags you care about:

local new_version
if ! new_version=$(curl -sSfL "https://hub.docker.com/v2/repositories/valkey/valkey/tags?page_size=100&ordering=last_updated" | \
 jq -r '.results|.[]|.name' | \
 grep -E '^[0-9](\.[0-9]){2}$'| \
 sort -V | \
 tail -n1); then
 echo "failed to get new version from docker hub tags" >&2; exit 1
fi
if [[ -z "$new_version" ]]; then
 echo "failed to get a non empty version from docker hub tags" >&2; exit 2
fi
if [[ "$new_version" != "$version" ]]; then
 echo "updating the repository to $project version $new_version"
 version="${version//./\\.}"
 sed -e "s!^version='$version'\$!version='$new_version'!" -i make.sh
fi

Git tags

The Git CLI can be used quite efficiently to get the latest tags on a specific repository. The trick is again to write the right filter the select only the tags you care about:

local new_seaweedfs_version
if ! new_seaweedfs_version=$(git ls-remote --tags --refs https://github.com/seaweedfs/seaweedfs | \
 perl -lane 'print $1 if /refs\/tags\/(\d.*)/'| \
 sort -V |
 tail -n1); then
 echo "failed to get new seaweedfs version from git RELEASE tags" >&2; exit 1
fi
if [[ -z "$new_seaweedfs_version" ]]; then
 echo "failed to get a non empty seaweedfs git RELEASE tag" >&2; exit 2
fi
if [[ "$new_seaweedfs_version" != "$seaweedfs_version" ]]; then
 echo "Updating the repository for $project version $new_seaweedfs_version"
 sed -e "s!^seaweedfs_version='$seaweedfs_version'\$!seaweedfs_version='$new_seaweedfs_version'!" -i make.sh
fi

Go

The official Go website conveniently offers a simple API endpoint to check the latest releases. I also update all module dependencies in this step for personal convenience:

local new_golang_version
if ! new_golang_version=$(curl -fsSL 'https://go.dev/dl/?mode=json' | \
 jq -r '[.[] | select(.stable == true)][0].version' | \
 sed -e 's/^go//'); then
 echo "failed to get new golang version from go.dev" >&2; exit 1
fi
if [[ -z "$new_golang_version" ]]; then
 echo "failed to get a non empty golang version number" >&2; exit 2
fi
sed -e "s/^go [0-9\\.]\+\$/go $new_golang_version/" -i go.mod
go get -t -u ./... && go mod tidy

OpenTofu/Terraform

Fetching the latest versions of OpenTofu/Terraform providers is trickier because proper parsing is needed to handle the files correctly.

I work around this by enforcing conventions. First I keep all provider definitions and configurations separate in a providers.tf file. Then I keep the source = and version = statements always sorted in that order (not a problem for me because I sort everything alphabetically anywhere I can, but critical for the script to work). I also do not use trailing comments for the source and version statements.

Note that this will pick the latest versions of each provider, not necessarily the latest compatible with your infrastructure code. One could update the script to filter on the same major versions the code currently relies on, but I do not bother in my personal circumstances and prefer to know and upgrade my code.

If I find any changes, I regenerate the Opentofu/Terraform lock file.

With this considered, I can forego proper parsing for the simpler:

get_latest_version() {
 local source=$1 latest
 if ! latest=$(curl -fsSL "https://registry.terraform.io/v1/providers/${source}/versions" | \
 jq -r '.versions[].version' | \
 sort -V | tail -n1); then
 echo "failed to get $source tofu provider version from the registry" >&2; exit 1
 fi
 if [[ -z "$latest" ]]; then
 echo "failed to get a non empty $source tofu provider version number" >&2
 exit 2
 fi
 printf '%s' "$latest"
}

process_tf_file() {
 local line source
 while IFS= read -r line <&3; do
 if [[ "$line" =~ ^[[:space:]]+source[[:space:]]+=[[:space:]]\"([A-Za-z0-9/]+)\"$ ]]; then
 printf '%s\n' "$line" >&4
 source="${BASH_REMATCH[1]}"
 elif [[ "$line" =~ ^([[:space:]]+version[[:space:]]+=[[:space:]])\"[0-9\.]+\"$ ]]; then
 printf '%s"%s"\n' "${BASH_REMATCH[1]}" "$(get_latest_version "$source")" >&4
 else
 printf '%s\n' "$line" >&4
 fi
 done
}
exec 3<providers.tf
exec 4>|providers_new.tf
process_tf_file
exec 3<&-
exec 4>&-
mv providers_new.tf providers.tf
if ! git diff --quiet providers.tf; then
 tofu providers lock -platform=linux_amd64
fi

Conclusion

Dependency management and updates are tricky subjects, and I am a firm believer that reducing the amount of dependencies is the best thing you can do in the long term. Dependency management software is no exception in my view, as they are another rather big dependency themselves.

Last but not least: this scripting approach makes me happy than I can discover new updates and bump pinned versions with a few curl, jq commands and regexes tailored to my stack.

Using Forgejo Actions without nodejs

2026-01-15

Introduction

I have been using Forgejo Actions to run CI tasks for about a year. I have been mostly content with them because they are well integrated with Forgejo itself, but I always thoroughly disliked relying on a stack of nodejs Actions that do way too much and that need to be updated way too often.

With all the supply chain attacks that make the headlines on hacker news, I started experimenting with simple handwritten shell actions that do only the minimal subset of what I need.

Checkout

The most commonly used Action must be actions/checkout which is the first step of almost every CI job. I suspect most Action users must have never looked at the source code of this action, but I encourage you to do so. Excluding tests, there are about 3k lines of typescript code, and it in turn has dependencies on other actions and libraries.

After years of GitHub Actions and now Forgejo Actions, I can count on one hand the number of times when I needed more than the simplest shallow clone. Therefore I replaced this action with:

 - name: 'checkout'
 run: |
 set -eu
 git init --quiet --initial-branch main
 git remote add origin "https://git.adyxax.org/${GITHUB_REPOSITORY}.git"
 git fetch --quiet --no-tags --depth=1 origin "$GITHUB_SHA"
 git checkout --quiet "$GITHUB_SHA"

When I need to support authentication, submodules, git LFS, or to checkout only a few sparse files, this small boilerplate can easily be taylored.

Complete workflow example

Using the same principles of minimal shell-based steps all the way, here is what this website’s main workflow looks like now:

---
on:
 push:
 branches:
 - 'main'
 workflow_dispatch:

jobs:
 all:
 runs-on: 'self-hosted'
 steps:
 - name: 'checkout'
 run: |
 set -eu
 git init --quiet --initial-branch main
 git remote add origin "https://git.adyxax.org/${GITHUB_REPOSITORY}.git"
 git fetch --quiet --no-tags --depth=1 origin "$GITHUB_SHA"
 git checkout --quiet "$GITHUB_SHA"
 - name: 'fmt'
 run: './make.sh tidy --fail-if-dirty=true'
 - name: 'build'
 run: './make.sh build'
 # we need the build step done before running check because the `search`
 # server embeds the `index.html` file, which gets generated by hugo during
 # the build step.
 - name: 'check'
 run: './make.sh check'
 - name: 'deploy'
 run: './make.sh deploy'
 env:
 SSH_PRIVATE_KEY: '${{ secrets.SSH_PRIVATE_KEY }}'

The make.sh script from this example allows for both local development and CI Actions which is how I prefer to think about CI: as a thin wrapper around the same commands you run locally.

I use a shell script because I am comfortable with shell scripting, but it could be a Perl or Raku script, a GNUmakefile or another tool as long as it can drive the full workflow locally using the same entrypoints as the CI.

Conclusion

I do not mind a little boilerplate, especially when I know I will rarely need to update it. And if I ever do need to update it, it is so simple that I fully understand and control it.

I did not mention a nice bonus: my CI is four times faster! The time spent building my blog went from about 24 seconds all the way down to about 6 seconds!

The true test on this journey will be to migrate my Terraform providers Actions workflows, more about this in another article.

Advent of code 2025 in Haskell

2026-01-06

Introduction

I participated in Advent of Code 2025 in Haskell. It was a fun experience as always! Why write about this now? Because I finished the last puzzle last Saturday!

I did all the puzzles each day on time last December except for day 10: part 2 was harder than all the rest combined! Life happened around Christmas, and I took the usual long break away from the puzzles before finishing.

Haskell for puzzles

The puzzles were all interesting without overstaying their welcome. Well except for day 10 part 2, but you need one of those for things to stay interesting! In this section I will present some of the days I enjoyed the most.

One of my favourite puzzles was day 3 where you need to find number patterns in the input. I found my solution elegant:

type Input = [String]

maxWithIndex :: String -> (Int, Char)
maxWithIndex (x:xs) = let (_, i, m) = L.foldl' step (1, 0, x) xs in (i, m)
 where
 step :: (Int, Int, Char) -> Char -> (Int, Int, Char)
 step (index, maxIndex, max) c | max < c = (index + 1, index, c)
 | otherwise = (index + 1, maxIndex, max)

compute :: Input -> Int
compute = sum . map compute'
 where
 compute' :: String -> Int
 compute' n = read . fst $ L.foldl' compute'' ("", 0) $ drop (length n - 11) $ L.inits n
 compute'' :: (String, Int) -> String -> (String, Int)
 compute'' (acc, i) s = let (ai, a) = maxWithIndex (drop i s)
 in (acc ++ [a], i + ai + 1)

I found Day 5 particularly satisfying to write. The puzzle is about merging intervals:

type Interval = (Int, Int)
data Input = Input [Interval] [Int]

compute :: Input -> Int
compute (Input intervals _) = sum $ map ilen intervals'
 where
 ilen :: Interval -> Int
 ilen (a, b) = b - a + 1
 (i:is) = L.sortOn fst intervals
 intervals' = L.foldl' step [i] is
 step :: [Interval] -> Interval -> [Interval]
 step acc@((c, d):xs) i@(a, b) | a > d = i:acc
 | otherwise = (c, max b d):xs

Day 6 was all about parsing operations with a twist (we transpose the input to parse column wise), and I always enjoy these problems:

data Op = Add | Mul deriving (Eq, Show)
type Input = [Int]
type Parser = Parsec Void String

parseNumber :: Parser Int
parseNumber = read <$> (some digitChar <* optional hspace)

parseOp' :: Parser Op
parseOp' = char '+' $> Add
 <|> char '*' $> Mul

parseOp :: Parser Int
parseOp = do
 n <- optional hspace *> parseNumber
 op <- parseOp' <* eol
 ns <- some (optional hspace *> parseNumber <* eol <* optional hspace)
 pure $ case op of
 Add -> sum $ n:ns
 Mul -> product $ n:ns

parseInput' :: Parser Input
parseInput' = some (parseOp <* optional eol) <* eof

parseInput :: String -> IO Input
parseInput filename = do
 input <- lines <$> readFile filename
 let len = maximum $ map length input
 input' = map complete input
 complete s = s ++ take (len - length s) (repeat ' ')
 input'' = unlines $ L.transpose input'
 case runParser parseInput' filename input'' of
 Left bundle -> error $ errorBundlePretty bundle
 Right input' -> return input'

compute :: Input -> Int
compute = sum

Day 7 was a fun set-management puzzle, at least that’s how I approached it. I also had fun using Parsec’s getSourcePos escape hatch to get the abscissa of the element being parsed:

type Input = [S.Set Int]
type Parser = Parsec Void String

parseTile :: Parser (Maybe Int)
parseTile = do
 SourcePos _ _ x <- getSourcePos
 char '.' $> Nothing <|> (char '^' <|> char 'S') $> Just (unPos x)

parseLine :: Parser (S.Set Int)
parseLine = S.fromList . catMaybes <$> some parseTile

parseInput' :: Parser Input
parseInput' = some (parseLine <* eol) <* eof

parseInput :: String -> IO Input
parseInput filename = do
 input <- readFile filename
 case runParser parseInput' filename input of
 Left bundle -> error $ errorBundlePretty bundle
 Right input' -> return input'

compute :: Input -> Int
compute (s:ls) = total $ foldl' compute' (M.fromList $ zip (S.toList s) [1]) ls
 where
 total :: M.Map Int Int -> Int
 total = M.foldl' (+) 0
 compute' :: M.Map Int Int -> S.Set Int -> M.Map Int Int
 compute' acc splitters = let splits = S.intersection (S.fromList $ M.keys acc) splitters
 continuingBeams = M.difference acc $ M.fromList $ zip (S.toList splits) $ repeat 0
 in S.foldl' (split acc) continuingBeams splits
 split :: M.Map Int Int -> M.Map Int Int -> Int -> M.Map Int Int
 split origin acc i = let v = origin M.! i
 acc' = case M.lookup (i+1) acc of
 Just n -> M.insert (i+1) (n+v) acc
 Nothing -> M.insert (i+1) v acc
 in case M.lookup (i-1) acc of
 Just n -> M.insert (i-1) (n+v) acc'
 Nothing -> M.insert (i-1) v acc'

I solved day 10 by very stubbornly writing a solver for underconstrained systems of linear equations, and it was a lot of fun!

Day 11 might have been my favourite this year. It is about memoization and graph exploration:

type Label = String
type Device = (Label, [Label])
type Input = M.Map Label [Label]

type Memo = M.Map (Bool, Bool, Label) Int

compute :: Input -> Int
compute input = (\(a, b) -> trace (show a) b) $ L.foldl' step (M.empty, 0) [(False, False, "svr")]
 where
 step :: (Memo, Int) -> (Bool, Bool, Label) -> (Memo, Int)
 step (m, a) (True, True, "out") = (m, a+1)
 step acc (_, _, "out") = acc
 step acc@(m, a) e@(d, f, l) = case M.lookup e m of
 Just v -> (m, a+v)
 Nothing -> let (m', a') = L.foldl' step (m, 0) $ zip3 (repeat $ if l == "dac" then True else d) (repeat $ if l == "fft" then True else f) (input M.! l)
 in (M.insert e a' m', a + a')

Some Befunge fun

I only did day 1 in befunge, but it was fun. I would love to do more puzzles in befunge but I really lack the time commitment.

Part1 is about simple parsing and keeping track of counters and modulos:

000p5a*>6j@.g000~&~$\'L-|
^ _v#:%d'<-<
^p00+1g00< ^+<

Part2 is a bit more complex because you need some math, but not too bad:

p5a*10p>6j@.g000~&~$\'L-|
v >:10g\-'d%'d+'d%\10g\:0w
;^p01p00+g00/d'+< <;# < >\:!#;_aa*\-
^ >:10g+'d%\10g^ <

Conclusion

I will always recommend tackling this kind of challenge: it is good to maintain or develop proficiency in a programming language. Also I love Haskell! I wish I could use it daily and not just for seasonal puzzles.

Wireguard jail on FreeBSD

2025-11-25

Introduction

One of my readers contacted me with an interesting question regarding Wireguard on FreeBSD: how to run Wireguard inside a jail instead of on the host. I only ever ran Wireguard on the host since that’s where I centralize all my routing and firewalling but was curious to explore the question.

Requirements

Nowadays Wireguard is easier to configure than ever on FreeBSD! But running it inside a jail needs two things that might not be straightforward: adding the if_wg module to /boot/loader.conf and using a vnet jail that exposes the right devfs rule set.

I did not take for granted that the if_wg kernel module handled jails properly, so I checked the sources. Good news is that the module seems to have enough jail specific code to properly handle the network segregation.

Loader.conf

When running Wireguard on the host, the kernel module is loaded automatically by an ifconfig wg0 create command. While running inside a jail though, this command will produce the following error message:

SIOCIFCREATE2 (wg0): Invalid argument

This is because ifconfig inside the jail does not have permissions to load kernel modules. To work around that, ensure that the module is loaded on boot by adding the following to /boot/loader.conf:

if_wg_load="YES"

Vnet jail

A standard vnet jail without special permissions is enough to run Wireguard. Just make sure to use the same devfs statements that I have in my /etc/jail.conf.d/wireguard.conf file:

wireguard {
 exec.consolelog = "/var/log/jail_console_${name}.log";

 # PERMISSIONS
 exec.clean;
 mount.devfs;
 devfs_ruleset = 5;

 # HOSTNAME/PATH
 host.hostname = "${name}";
 path = "/usr/local/jails/containers/${name}";

 # NETWORK
 vnet;
 vnet.interface = "${epair}b";
 $id = "154";
 $ip = "10.0.2.${id}/24";
 $gateway = "10.0.2.2";
 $bridge = "bridge0";
 $epair = "epair${id}";
 # ADD TO bridge INTERFACE
 exec.prestart = "/sbin/ifconfig ${epair} create up";
 exec.prestart += "/sbin/ifconfig ${epair}a up descr jail:${name}";
 exec.prestart += "/sbin/ifconfig ${bridge} addm ${epair}a up";
 exec.start = "/sbin/ifconfig ${epair}b ${ip} up";
 exec.start += "/sbin/route add default ${gateway}";
 exec.start += "/bin/sh /etc/rc";
 exec.stop = "/bin/sh /etc/rc.shutdown";
 exec.poststop = "/sbin/ifconfig ${bridge} deletem ${epair}a";
 exec.poststop += "/sbin/ifconfig ${epair}a destroy";
}

Make sure to change the jail Id, IP network, gateway and bridge interface name to match your setup.

Wireguard

Create the following files inside your jail. An easy way to do that is to start the jail and exec into it:

service jail start wireguard
jexec wireguard sh

Wireguard works out of the box on an up to date FreeBSD 14.3 without any additional packages like wireguard-tools. Just set the interface name and IP address in /etc/rc.conf:

network_interfaces="wg0"
ifconfig_wg0="inet 10.1.2.18/24"

Then add the following to /etc/start_if.wg0:

ifconfig wg0 create
wg syncconf wg0 /etc/wg0.conf

And finally your Wireguard configuration at /etc/wg0.conf:

[Interface]
PrivateKey = XXXXXX
ListenPort = 342

[Peer]
PublicKey = R4A01RXXqRJSY9TiKQrZGR85HsFNSXxhRKKEu/bEdTQ=
Endpoint = 168.119.114.183:342
AllowedIPs = 10.1.2.9/32
PersistentKeepalive = 60

Start or restart your jail, then confirm that Wireguard is running properly with:

jexec wireguard ifconfig wg0
jexec wireguard wg

Conclusion

All this made me think that an interesting next step to take would be to configure a Wireguard interface on the host, then pass it as the only interface for a jail instead of a bridge. The ramifications of this sound pretty neat!

Scripting the download of a private repository's GitHub release asset

2025-11-13

Introduction

Last week I needed to script the download of a specific asset from a release of a private GitHub repository. It turns out that there is no direct way to do this as you first need to resolve the asset name into an ID. Here is a little script that does just that without any big dependency like the gh CLI.

The script

#!/usr/bin/env bash
set -euCo pipefail

github_pat="XXXXXX"
owner="adyxax"
repository="private-repository"
regex=".*-linux-x86_64.tar.gz"
tag="v2025.11.06.0402"

if ! releases=$(curl -fsSL \
 "https://api.github.com/repos/$owner/$repository/releases/tags/$tag" \
 -H "Accept: application/vnd.github+json" \
 -H "Authorization: token $github_pat" \
 -H "X-GitHub-Api-Version: 2022-11-28"); then
 echo "failed to get latest release assets from GitHub" >&2; exit 1
fi

value=$(printf '%s' "$releases" \
 | jq -r --arg regex "$regex" \
 'first(.assets
 |to_entries[]
 |select(.value.name
 |test($regex))
 |.value)')

id=$(printf '%s' "$value" | jq -r '.id')
filename=$(printf '%s' "$value" | jq -r '.name')

if ! curl -fsSL "https://api.github.com/repos/$owner/$repository/releases/assets/$id" \
 -H "Accept: application/octet-stream" \
 -H "Authorization: token $github_pat" \
 -H "X-GitHub-Api-Version: 2022-11-28" \
 -o "$filename"; then
 echo "failed to download asset from GitHub" >&2; exit 2
fi

Conclusion

This is simple and works well. Scripting is fun!

OpenTofu/Terraform state locking with S3 object locking

2025-10-20

Introduction

Today’s AWS outage is a good time to remind sysadmins that AWS DynamoDB is no longer required to handle state file locking when storing state files on AWS S3.

Since last year, the S3 state backend has supported state locking via S3 object locks. This locking method is simpler, faster and removes a dependency on an AWS service that we no longer need. DynamoDB was the default state backend locking mechanism for many years and has served us well, but today’s AWS outage is a good time to move on if you have not already done so.

Migration steps

Make sure bucket versioning is enabled

I cannot imaging why anyone would store OpenTofu/Terraform state files without versioning, but if for some reason you did not already then know that versioning is a requirement for activating S3 object locks. A simple configuration like the following is enough:

resource "aws_s3_bucket_versioning" "tofu_states" {
 bucket = aws_s3_bucket.tofu_states.id
 versioning_configuration {
 status = "Enabled"
 }
}

resource "aws_s3_bucket_lifecycle_configuration" "tofu_states" {
 bucket = aws_s3_bucket.tofu_states.id
 rule {
 filter {}
 id = "main"
 noncurrent_version_expiration {
 noncurrent_days = 90
 }
 noncurrent_version_transition {
 noncurrent_days = 2
 storage_class = "STANDARD_IA"
 }
 noncurrent_version_transition {
 noncurrent_days = 14
 storage_class = "GLACIER"
 }
 status = "Enabled"
 }

 depends_on = [aws_s3_bucket_versioning.tofu_states]
}

Activating S3 object locks

The first step is to activate object locks on the OpenTofu/Terraform states S3 bucket. Sadly, simply adding the attribute to the aws_s3_bucket resource won’t be sufficient as OpenTofu/Terraform will then try to recreate the bucket.

aws --profile core \
 s3api put-object-lock-configuration \
 --bucket adyxax-tofu-states \
 --object-lock-configuration='{"ObjectLockEnabled":"Enabled"}'

After typing this AWS CLI command, you will be able to update your aws_s3_bucket resource. Applying the changes will refresh the state and confirm the change is correctly implemented.

resource "aws_s3_bucket" "tofu_states" {
 bucket = "adyxax-tofu-states"
 object_lock_enabled = true
}

Reconfiguring your terraform backend

Update your OpenTofu/Terraform S3 backend configuration blocks to add the use_lockfile = true attribute:

terraform {
 backend "s3" {
 bucket = "adyxax-tofu-states"
 dynamodb_table = "tofu-states"
 key = "repositories/${local.name}"
 profile = "core"
 region = "eu-west-3"
 use_lockfile = true
 }
}

By setting both dynamodb_table and use_lockfile, applying changes will lock both systems at the same time. This is important if you need to roll out your change gradually as to not disrupt your colleagues work or your CI jobs.

If you can afford the disruption, it will be faster to just delete the DynamoDB state locks table and remove the dynamodb_table attribute.

As with all backend configuration changes, you will need to init your OpenTofu/Terraform folders again:

tofu init -reconfigure

Once the change has been applied in all your OpenTofu/Terraform folders and repositories, all your pull requests merged or rebased and you are sure everybody and everything is on the new locking system, you can safely remove the dynamodb_table from your backend configuration blocks:

terraform {
 backend "s3" {
 bucket = "adyxax-tofu-states"
 key = "repositories/${local.name}"
 profile = "core"
 region = "eu-west-3"
 use_lockfile = true
 }

You will need to init -reconfigure your folders again.

Deleting the DynamoDB state locks table

The last step is to destroy the DynamoDB table that used to hold your locks. If you have deletion protection activated, remove it with:

aws --profile core --region eu-west-3 \
 dynamodb update-table \
 --table-name tofu-states \
 --no-deletion-protection-enabled

With this done, destroy your DynamoDB table and remove any IAM policies controlling DynamoDB access to the table and any cloudwatch alerts used to monitor it. I personally do this with a tofu apply because everything is managed from there.

Conclusion

S3 object locks have served me well since they were made available as a backend locking mechanism. If you are still relying on DynamoDB, now is a good time to migrate away from this service.

A Forgejo action for OpenTofu module testing on AWS

2025-09-30

Introduction

I have been using a Forgejo action (compatible with GitHub actions) for testing my OpenTofu/Terraform modules on AWS. I planned to blog this earlier, but since it worked without quirks I completely forgot to publish it.

Usage example

The action relies on having an AWS IAM access key provisioned in your CI’s secrets. An astute reader will notice I am naming the secret AWS_ACCESS_KEY_SECRET instead of the standard AWS_SECRET_ACCESS_KEY, mind this if trying to reproduce in your own environment:

- uses: "https://git.adyxax.org/adyxax/action-tofu-aws-test@2.0.0"
 with:
 aws-access-key-id: "${{ vars.AWS_ACCESS_KEY_ID }}"
 aws-access-key-secret: "${{ secrets.AWS_ACCESS_KEY_SECRET }}"

Action steps

This action initializes the AWS credentials, then runs formatting and linting checks as well as tofu tests on the repository of an OpenTofu/Terraform module.

Boilerplate

Forgejo actions need some light boilerplate to define how they are to be used:

name: "tofu-aws-test"
description: "Test a tofu module on AWS."

inputs:
 aws-access-key-id:
 description: "AWS access key id."
 required: true
 aws-access-key-secret:
 description: "AWS access key secret."
 required: true

runs:
 using: "composite"
 steps:

Formatting and linting

These are two simple steps. The only caveat is that I need to unset the GITHUB_TOKEN environment variable before running tflint. This is because Forgejo strives for compatibility with GitHub actions, but sadly tflint will try to use said token for cloning its plugins. Since the token is generated on my Forgejo instance, it is only valid there and not on GitHub. This would cause the step to fail.

 - name: "fmt"
 shell: "bash"
 run: |
 tofu fmt -check -recursive
 - name: "lint"
 shell: "bash"
 run: |
 unset GITHUB_TOKEN
 tflint --init
 tflint --recursive

AWS credentials initialization

If you followed my past articles you will know that I am using multiple AWS accounts to separate everything. I strive to have all my test resources created in a tests AWS account, but some things like DNS records will occasionally require access to my core AWS account where DNS zones are provisioned.

I am using AWS policies to tightly control what tests can provision in this core account, but I will not detail it in this article.

 - name: "configure AWS profiles"
 shell: "bash"
 run: |
 ROLE_NAME="repository_${GITHUB_REPOSITORY/\//_}"
 cat >aws_config <<EOF
 [profile core]
 role_arn = arn:aws:iam::123456789012:role/${ROLE_NAME}
 source_profile = root

 [profile root]
 aws_access_key_id = ${{ inputs.aws-access-key-id }}
 aws_secret_access_key = ${{ inputs.aws-access-key-secret }}
 region = eu-west-3

 [profile tests]
 role_arn = arn:aws:iam::345678901234:role/${ROLE_NAME}
 source_profile = root
 EOF

One should of course use their own AWS account IDs here!

OpenTofu/Terraform lock files

It is important to make sure your lock files match the provider versions pinned in your configuration. This step does exactly this!

This step has some code to recurse in subfolders because I love to use a pattern where some OpenTofu code specific to a repository lives alongside a module or some other code:

 - name: "check tofu providers lock files"
 shell: "bash"
 run: |
 unset GITHUB_TOKEN
 export AWS_CONFIG_FILE="$(pwd)/aws_config"
 shopt -s globstar nullglob
 for lockfile in **/.terraform.lock.hcl; do
 (cd "$(dirname "$lockfile")"; tofu init; tofu providers lock -platform=linux_amd64)
 done
 git diff --exit-code

OpenTofu/Terraform tests

I am running module tests with:

 - name: "tofu test"
 shell: "bash"
 run: |
 export AWS_CONFIG_FILE="$(pwd)/aws_config"
 tofu init
 tofu test

Cleanup

We clean up to be sure that the next steps in the workflow are not affected:

 - name: "clean"
 shell: "bash"
 run: |
 rm aws_config

Conclusion

Here is the link to the repository. Writing composite actions like this one is a good exercise and can be used to keep your workflows tidy and DRY!

How to set the VNC resolution of a QEMU virtual machine

2025-09-19

Introduction

I last blogged about how to run simple QEMU virtual machines in 2021 in this article. I often use these commands to spawn virtual machines that I only access via SSH, but it was never for their GUI! Now the need arose and the necessary bits were harder to piece together than I expected.

Installation

I updated the VNC and drive sections for modern QEMU flags compared to the last article:

qemu-img create -f raw alpine.raw 16G
qemu-system-x86_64 -drive if=none,id=disk,file=/alpine.raw,format=raw,cache=writeback \  (base)
 -device virtio-blk-pci,drive=disk \
 -cdrom Downloads/alpine.iso \
 -boot d -machine type=q35,accel=kvm \
 -cpu host -smp 2 -m 2048 \
 -display vnc 127.0.0.1:0 \
 -device virtio-net,netdev=vmnic \
 -netdev user,id=vmnic,hostfwd=tcp::10022-:22

Connect to the console with a vncviewer :0, or if an ssh server is running, use ssh -p10022 root@localhost.

Post-Installation

We boot from the installed disk and drop the options to mount the ISO image:

qemu-system-x86_64 -drive if=none,id=disk,file=/alpine.raw,format=raw,cache=writeback \  (base)
 -device virtio-blk-pci,drive=disk \
 -boot c -machine type=q35,accel=kvm \
 -cpu host -smp 2 -m 2048 \
 -display vnc 127.0.0.1:0 \
 -device virtio-net,netdev=vmnic \
 -netdev user,id=vmnic,hostfwd=tcp::10022-:22

How to customize the VNC resolution

You only needs to add an extra line of flags. Just substitute your native resolution (use xrandr on your host machine if you are in doubt about which values to use) in xres=...,yres=...:

qemu-system-x86_64 -drive if=none,id=disk,file=/alpine.raw,format=raw,cache=writeback \  (base)
 -device virtio-blk-pci,drive=disk \
 -boot c -machine type=q35,accel=kvm \
 -cpu host -smp 2 -m 2048 \
 -display vnc 127.0.0.1:0 \
 -device virtio-net,netdev=vmnic \
 -netdev user,id=vmnic,hostfwd=tcp::10022-:22 \
 -vga none -device virtio-vga,edid=on,xres=2560,yres=1440

Conclusion

It is good to refresh one’s memory for something so simple and useful: no virt-manager or other tools required, no complicated networking… QEMU is fun and easy!

The Raku programming language

2025-08-31

Introduction

I gave Raku several tries over the last 15 years. I was initially a bit disappointed when it was first released as Perl6 as it was very slow to start and a resource hog. I was disappointed again sometime in the late 2010s I tried the freshly renamed Raku. Still very slow to start up and consumming too much memory for what I was attempting, but much better.

I tried again this summer and I am glad to see that Raku has become much more usable! The language and its ecosystem have made strides of progress. Startup time (critical for a scripting language imho) is now good. CPU and memory usage remain on the high side, but have become more reasonable.

The language

The language is clearly Perlish, with sigils and unusual syntax that seem alien compared to other programming languages. Having liked Perl, I must say that I very much like the subtle changes Raku brought to the overall syntax. One such change is the array indexing: where you would have written $a[0] in Perl, you now write @a[0] in Raku.

I love all the convenience features and all the syntax sugar Raku brings to the table. For example, you can declare a list of string literals with my @l = <one two three>; instead of the heavier my @l = ("one", "two", "three");. Note that the parentheses are also optional here. Another related example is about hash indexing when keys are string literals: %h{'one'} is equivalent to %h<one>.

Raku has gradual typing, and its type system is really powerful. Two of my favorite features are junctions and subsets. Junctions in particular are wild:

my $j = 0|1|2|3; # $j is a junction, a composite value
if 3 == $j + 1 { # this matches because 2 is a possible value for $j
 say 'yes';
}

Note that junctions are not sets, they are only meant for boolean evaluation. But they are very neat! Subsets are also fun, allowing you to write things like:

subset MyBiggerInts of Int where * >= 42;
subset Three-letter of Str where .chars == 3;
subset Foo of List where (Int,Str); # Only a List where the first element is an
 # Int and the second a Str will pass the type check.

Many scripts are very concise in Raku. One of the neatest things is the amount of features, classes and functions that are built-in. This might seem trivial, but not having 10 lines of imports at the top of every script is refreshing! This is of course true for all the regex stuff, but also for some things like argument parsing on the command line. Here is how you write it in Raku:

sub MAIN(
 Str $file where *.IO.f = 'file.dat', #= an existing file to frobnicate
 Int :$length = 24, #= length needed for frobnication
 Bool :$verbose, #= required verbosity
) {
 say $length if $length.defined;
 say $file if $file.defined;
 say 'Verbosity ', ($verbose ?? 'on' !! 'off');
}

Calling a program with --help or invalid flags will produce the following for you, free of charge:

Usage:
 frobnicate.raku [--length=<Int>] [--verbose] [<file>]

 [<file>] an existing file to frobnicate
 --length=<Int> length needed for frobnication
 --verbose required verbosity

With all that I did not say anything yet about the object system which is very well thought out, the regexes and grammar which are a lot of fun, or all the convenience added to the standard classes. For example you can open a file and read its contents with:

"some/file.txt".IO.lines

The documentation and the language reference seem complete and are full of useful examples. Numeric maths are treated carefully: arithmetic is exact by default (big integrers when values exceed 64 bits, and rational numbers for fractions). How neat is that?

Conclusion

Learning Raku has been a lot of fun and I recommend giving it a try. Installing it is easy thanks to a tool called rakubrew. Packages are managed with zef and finding libraries is easy by browsing raku.land.

Of course wanted to solve some puzzles in Raku, as I do when learning new languages: https://git.adyxax.org/adyxax/advent-of-code/src/branch/master/2019

Advent of code 2024 in Haskell

2025-07-24

Introduction

I participated in advent of code 2024 in Haskell: it was a fun experience as always! Why writing about this now? Because I just finished the last puzzle!

I did the first 12 puzzles each day last December but then life happened and I could no longer complete one puzzle per day. I only finished the first 19 puzzles by Christmas then took the usual long break. I picked up this challenge again about a month ago while waiting at the airport and have now completed the last one.

Haskell for puzzles

Usually this kind of article is an opportunity to explain some of the patterns I used and things I learned. Solving these puzzles was a lot of fun as always.

The puzzles were interesting, my favorite one being day 24 in which you need to debug a binary adder. It is made of logic gates with a few wires that have been inverted and need to be fixed. It was the hardest challenge for me by far and I only solved it when I realized that instead of needing to simulate wire permutations I could try to build the adder from the pool of logic gates that were available. When I could not find a gate I needed, it meant I had to look for a wire to swap.

I also thoroughly enjoyed day 21 where you need to input codes via proxy robots and need to compute a very convoluted shortest path algorithm where the key was careful usage of memoization.

I will also make a special mention for day 13 where the problem can be reduced to a system of linear equations. I love an excuse to whip out a matrix triangularization to achieve this, but I doubly loved that I could use the Rational type to deal with exact ratio numbers computations. Haskell really shines in these situations!

Conclusion

I recommend tackling this kind of challenge, it is good to maintain or develop proficiency in a programming language. I love Haskell, I wish I could use it daily and not just for seasonal puzzles.

Enforcing AWS Secret version with OpenTofu/Terraform

2025-07-08

Introduction

Managing secrets in AWS is a common task. It is therefore surprising that the default aws_secretsmanager_secret_version usage does not properly enforce a secret value.

At first glance, it appears to enforce secret versions properly because updating the secret’s value results in an updated AWS secret version accordingly. Furthermore, if the secret is deleted then OpenTofu/Terraform will recreate it with the proper value as well! However, the unexpected behavior occurs when the value of the secret is manually changed: in that case, OpenTofu/Terraform will do nothing to reconcile or restore the value.

Properly enforcing a secret value

To solve this issue, the stage of the managed secret version needs to be enforced. Given the following basic resources that generate a random password and a secret:

resource "random_password" "main" {
 length = 64
}

resource "aws_secretsmanager_secret" "main" {
 name = "secret"
}

A secret version stage can be enforced with:

resource "aws_secretsmanager_secret_version" "main" {
 secret_id = aws_secretsmanager_secret.main.id
 secret_string = random_password.main.result
 version_stages = ["AWSCURRENT"]
}

The important attribute in the context of this article is version_stages. Though optional and not mentioned in the example usages of the resource’s documentation, it is what properly enforces this secret’s value as the current version.

Conclusion

I am in awe that I managed to go on for so many years without encountering this particular issue! Systematically specifying the version_stages attribute in all secret version resources is a boilerplate that I could have lived without, but necessary to ensure reliability. I find solace knowing that any manual changes to a secret value performed outside of OpenTofu/Terraform are now properly detected and corrected.

AWS Identity Center with OpenTofu/Terraform

2025-07-01

Introduction

Many people make mistakes and create immediate tech debt when they get started with AWS. They rightfully fear that everything is going to be expensive and try to keep it under control by cramming everything in a single VPC in a single AWS Account.

This is a big mistake though, because creating multiple AWS accounts (which are administrative objects isolating everything) is free, and using IAM Identity Center (successor to AWS Single Sign-On) is a great way to centralize access control to a multitude of AWS accounts. Separating each environment or project into its own sub-account is a great way to control security boundaries as well as keep an eye on costs per AWS sub-account.

I recently took the time to advise a former colleague and friend about getting started on AWS: here are the resulting notes.

Bootstrapping AWS Identity Center

Sadly not everything can be automated with OpenTofu/Terraform. The initial bootstrap must be performed by clicking through the AWS web console:

Login to your root admin account.
Select the primary AWS region you will operate in.
Use the Search Bar to navigate to the IAM Identity Center console.
Click on the orange Enable button to activate Identity Center for your organization.
Configure your Identity Source. You can use the default AWS Identity Center directory like I do, or connect an external identity provider.
Configure the AWS access portal URL as well as your Instance name.
Configure Multi Factor Authentication.

With these settings out of the way, everything else can be automated with OpenTofu/Terraform.

Creating sub-accounts

I use something close to the following input variable in order to manage my additional AWS accounts:

variable "aws_accounts" {
 description = "AWS accounts to manage."
 nullable = false
 type = map(object({
 email = string
 ou = optional(string, null)
 }))
}

Here is an example terraform.tfvars file provisioning this data structure:

aws_accounts = {
 core = {
 email = "julien.dessaux+aws-core@adyxax.eu"
 ou = "core-engineering"
 }
 root = {
 email = "julien.dessaux+aws-root@adyxax.eu"
 }
 tests = {
 email = "julien.dessaux+aws-tests@adyxax.eu"
 ou = "core-engineering"
 }
}

You might ask yourselves why I use an email attribute that looks pretty easy to derive consistently from the account name. I do this because creating and deleting AWS accounts is easy! Though the account names can be reused, the email address cannot.

After a few years of projects creations and deletions, you will happen to reuse an account name and will need a different email address. This data structure helps me remember that and I keep a list of former accounts email addresses in a comment above this structure.

I manage the Organization Units (OUs) with:

locals {
 ous = toset([for name, info in var.aws_accounts :
 info.ou if info.ou != null
 ])
}

data "aws_organizations_organization" "org" {}

resource "aws_organizations_organizational_unit" "ou" {
 for_each = local.ous

 name = each.key
 parent_id = data.aws_organizations_organization.org.roots[0].id
}

And I manage the AWS accounts using the following configuration:

data "aws_ssoadmin_instances" "root" {}

locals {
 identity_center_arn = data.aws_ssoadmin_instances.root.arns[0]
 identity_center_store_id = data.aws_ssoadmin_instances.root.identity_store_ids[0]
}

resource "aws_organizations_account" "main" {
 for_each = var.aws_accounts

 close_on_deletion = true
 email = each.value.email
 name = each.key
 parent_id = each.value.ou != null ? aws_organizations_organizational_unit.ou[each.value.ou].id : null

 lifecycle {
 ignore_changes = [role_name]
 }
}

The ignore_changes lifecycle entry allows importing existing accounts into this automation, which I do for the root account itself.

Managing user accounts

I use something close to the following input variable in order to manage Identity Center user accounts:

variable "users" {
 description = "Users to manage accounts for."
 nullable = false
 type = map(object({
 admin = object({
 aws = bool
 })
 display_name = optional(string, null)
 email = string
 family_name = optional(string, null)
 given_name = optional(string, null)
 }))
}

Here is an example terraform.tfvars file provisioning this data structure:

users = {
 julien-dessaux = {
 admin = { aws = true }
 email = "julien.dessaux@adyxax.org"
 }
}

The following local variable augments the user accounts by setting defaults for the optional fields. I use the convention that all usernames follow the <firstname>-<lastname> format and this handles cases where a user’s display name, family name or given name do not exactly fit this scheme:

locals {
 users = { for username, info in var.users :
 username => merge(
 info,
 info.display_name == null ? { display_name = title(replace(username, "-", " ")) } : {},
 info.family_name == null ? { family_name = split(" ", title(replace(username, "-", " ")))[1] } : {},
 info.given_name == null ? { given_name = split(" ", title(replace(username, "-", " ")))[0] } : {},
 )
 }
}

Creating the actual IAM Identity Center user is done with:

resource "aws_identitystore_user" "main" {
 for_each = local.users

 display_name = each.value.display_name
 emails {
 primary = true
 value = each.value.email
 }
 identity_store_id = local.identity_center_store_id
 name {
 family_name = each.value.family_name
 given_name = each.value.given_name
 }
 user_name = each.key
}

Now that we have users provisioned, let’s grant them permissions on our infrastructure.

Granting admin access

An IAM Identity Center group can be created with:

resource "aws_identitystore_group" "admin" {
 display_name = "admin"
 identity_store_id = local.identity_center_store_id
}

Assigning members to this group is a matter of:

resource "aws_identitystore_group_membership" "admin" {
 for_each = { for username, info in var.users :
 username => info if info.admin.aws
 }

 group_id = aws_identitystore_group.admin.group_id
 identity_store_id = local.identity_center_store_id
 member_id = aws_identitystore_user.main[each.key].user_id
}

Permissions are granted through permission sets of attached policies:

resource "aws_ssoadmin_permission_set" "admin" {
 instance_arn = local.identity_center_arn
 name = "admin"
 session_duration = "PT12H"
}

resource "aws_ssoadmin_managed_policy_attachment" "admin" {
 instance_arn = local.identity_center_arn
 managed_policy_arn = "arn:aws:iam::aws:policy/AdministratorAccess"
 permission_set_arn = aws_ssoadmin_permission_set.admin.arn
}

resource "aws_ssoadmin_account_assignment" "admin" {
 for_each = aws_organizations_account.main

 instance_arn = local.identity_center_arn
 permission_set_arn = aws_ssoadmin_permission_set.admin.arn
 principal_id = aws_identitystore_group.admin.group_id
 principal_type = "GROUP"
 target_id = each.value.id
 target_type = "AWS_ACCOUNT"
}

Generating your AWS CLI configuration file

I rely on the following OpenTofu/Terraform template to generate my ~/.aws/config file:

[default]
region = eu-west-3
sso_session = adyxax

[sso-session adyxax]
sso_start_url = https://adyxax.awsapps.com/start
sso_region = eu-west-3
sso_registration_scopes = sso:account:access

%{~for name, id in accounts}
[profile ${name}]
sso_account_id = ${id}
sso_role_name = admin
sso_session = adyxax
%{endfor~}

Using this template, I output my configuration with:

output "aws_config" {
 value = templatefile("./aws_config", {
 accounts = { for name, info in aws_organizations_account.main :
 name => info.id
 }
 })
}

Each morning, I log in with:

aws sso login

To access a specific account, I use the --profile CLI flag:

aws --profile core s3 ls

Conclusion

Starting your AWS journey with multiple accounts and centralized access management as shown in this article will help you avoid quite a few pitfalls. Though there are a few clicks to perform for the initial setup, everything important can be automated quite well.

I recommend everyone to make the effort to commit to this approach from the beginning in order to have a scalable, secure and cost-effective AWS environment at your disposal.

Writing an OpenTofu/Terraform provider for Forgejo

2025-05-28

Introduction

Last month I started writing an OpenTofu/Terraform provider for Forgejo. I wanted to automate this forge in the same way I already automate GitHub: create and manage repositories, provision actions secrets and variables, configure issue labels, manage mirrors, etc.

A community provider already existed but it is really barebones and missing almost all the resources I need. I could have tried to contribute to it, but after looking at the code I decided to start from scratch.

I wrote providers before so I could get right to it.

Writing a Terraform provider

My notes from writing a Terraform provider for eventline still apply. With a few previous experiences writing providers, this has become straightforward work.

The only significant difficulties and frustrations arose from the Forgejo API which is pretty inconsistent to say it nicely. A community SDK forked from Gitea exists but after looking at its code I decided to write my own small API client.

Conclusion

Writing a Terraform provider is a lot of fun, I recommend it! If you have a piece of software that you wish had a Terraform provider, know that it is not hard to make it a reality. It is also a great learning project, being well-bounded in scope and immediately useful.

Here is the repository of my Forgejo provider for reference and here is the documentation.

Hugo RSS feed generation issue when running in CI

2025-04-28

Introduction

I just fixed an issue with RSS feed generation that began a month ago after migrating the CI job that builds my blog from a git hook in Gitolite to a workflow in Forgejo Actions.

The issue

I only realized this issue after not seeing yesterday’s article appear in my miniflux. I was puzzled when I noticed that the XML index file did not contain any articles on the hosting server. I immediately checked the Forgejo Actions workflow logs, but there were no errors or warnings. Whatever was failing just failed silently.

Solving the issue

I worked around the issue quickly by building the website locally on my workstation and seeing the XML index file properly populated. I redeployed this version of the website and refreshed my miniflux feed: it worked.

I first theorized that I might be missing a build dependency on my Forgejo runners. It was disproved when connecting over SSH to the runner that ran the last build: cloning and building the website there produced a valid XML index file once again.

I then added a sleep statement before the deployment step in the workflow file and pushed a commit so that I could inspect a CI run in progress. I managed to SSH on the runner, navigate to the temporary build directory and reproduce the issue there: whatever was happening was not intermittent.

I therefore examined the workflow closely and saw the actions/checkout step that innocuously starts all GitHub or Forgejo Actions workflow. Having been bitten by this in the past, I knew it performed a shallow git clone by default. Therefore I followed my instinct to try a deep clone instead: this fixed the issue.

With this new information, I checked Hugo’s documentation and figured I had to set enableGitInfo = false in my config.toml file. When enabled, Hugo uses git to figure out the last modification date of a file and this breaks the RSS feeds.

Though I did not use git information anywhere in my templates, this still affected the logic that filters which articles show up in the feed. This particular configuration flag was a remnant from before I wrote my own Hugo theme: The theme I used once upon a time required this flag.

Conclusion

It was frustrating to encounter a silent failure in Hugo over something seemingly trivial. Despite the silent failure, I was relieved my past experiences made it straightforward to resolve.

OpenTofu/Terraform module testing

2025-04-26

Introduction

For the last two months, I finally got around to playing with OpenTofu tests. Infrastructure tests have been teasing me for such a long time! Having mostly dealt with terraform mono-repos in my career, this was never a big deal because it is easy to test on just part of an existing environment and therefore never became a priority.

In the last six months I started building a multi repository system and thoroughly experimented with testing modules. Here are my thoughts on testing infrastructure code and a few examples.

Static checks

Static checks are cheap and useful. Before running terraform tests, my CI action runs the following:

a tofu fmt check to ensure files are properly formatted.
a tflint check to lint the tofu code.
a tofu providers lock to check that the provider locks files all have the required platform signatures.

Only after all these steps succeed do I run tofu test.

Testing OpenTofu/Terraform code

The OpenTofu/Terraform test command lets you test a configuration by creating real infrastructure. This means one or more instantiations of the module in need of testing, as well as support infrastructure or resources.

Once the test infrastructure is created, assertions are performed to check that all the conditions relevant to the module’s behavior are met. This involves checking the module’s outputs but one usually also use the outputs of various data-sources.

Once the test is complete, OpenTofu destroys the resources it created. Know already that this can fail if the test is not written correctly. All terraform failures are properly handled, but a common failure case I met was when using the external data-source to run some shell commands and capture their outputs.

Basic example

The simplest form of tofu test is to create a main.tftest.hcl file inside a module. Here is an example for a module that creates an AWS IAM role:

provider "aws" {
 profile = "tests"
 region = "eu-west-3"
}

run "main" {
 assert {
 condition = output.arn != null
 error_message = "invalid IAM role ARN"
 }
}

variables {
 name = "tftest-role"
}

The module is quite simple: its only purpose being to add a bunch of policy entries. Testing the correct provisioning of the role would be way more code than the role itself, so this simple test only does a dummy check to confirm that all the module’s resources instantiate properly.

More elaborate example

A more complete example that actually tests the correct behavior is what I do with a module that creates an AWS IAM user. Here the test is to log in with the user’s access key and check its identity.

The main.tftest.hcl is simpler because it relies on a support module:

provider "aws" {
 profile = "tests"
 region = "eu-west-3"
}

run "main" {
 assert {
 condition = data.external.main.result.Arn == local.expected_arn
 error_message = "user ARN mismatch"
 }
 module {
 source = "./test"
 }
}

The support module contains multiple files. The most important one is main.tf:

module "main" {
 source = "../"

 name = "tftest-user"
}

data "aws_caller_identity" "current" {}

# tflint-ignore: terraform_unused_declarations
data "external" "main" {
 program = ["${path.module}/test.sh"]

 depends_on = [local_file.aws_config]
}

locals {
 # tflint-ignore: terraform_unused_declarations
 expected_arn = format(
 "arn:aws:iam::%s:user/tftest-user",
 data.aws_caller_identity.current.account_id,
 )
}

resource "local_file" "aws_config" {
 filename = "${path.module}/aws_config"
 file_permission = "0600"
 content = templatefile("${path.module}/aws_config.tftpl", {
 aws_access_key_id = module.main.access_key_id
 aws_access_key_secret = module.main.access_key_secret
 })
}

The module main is the instantiation of the module we are testing. The other resources are here to allow a login via the AWS CLI in order to test the access. Note the tflint-ignore directives: They are annoying but needed since tflint does not know how to reconcile that these are used in of main.tftest.hcl file.

The test relies on a aws_config.tftpl file containing:

[default]
aws_access_key_id = ${aws_access_key_id}
aws_secret_access_key = ${aws_access_key_secret}
region = eu-west-3

It also relies on this script:

#!/usr/bin/env bash
set -euo pipefail

# Wait a bit for the ACCESS KEY to be usable on AWS
sleep 10

export AWS_CONFIG_FILE="${PWD}/test/aws_config"
aws sts get-caller-identity

Note that the external data-source works with scripts that take JSON input and JSON output. Luckily this is what the AWS CLI outputs by default, but if you changed it you will have to tweak this.

Conclusion

Writing module tests is worthwhile, even if just to validate the proper instantiation of a module in its most used configurations. I am now validating that I can properly spawn VPCs, databases, load balancers, generate certificates… I have never felt more confident in my OpenTofu/Terraform code!

Though I have found that writing the tofu testing code was not the hardest part: making it all work in CI was. In a next article I will present the test infrastructure I use to run all this.