The tagline for my blog is “Always keep a spare for redundancy,” and nowhere is that philosophy more thoroughly tested—and relied upon—than in my homelab.

Earlier this year, I wrote about how I automate volume backups using Kyverno sidecars to protect my Longhorn persistent data. But storage backups are only one gear in the clockwork. Over the past couple of years, my homelab infrastructure has evolved from a collection of ad-hoc Docker containers and bespoke scripts into a fully declarative, multi-node K3s cluster managed strictly through GitOps.

I have been cleaning up my music collection, which I have ripped from CDs over the years, finally getting around to tagging them properly. I used MusicBrainz Picard to tag my music, but what I didn’t know is that it was adding a bunch of tags which contain ‘PMEDIA’ (even in places like the composer field, which makes no sense).

Seeing this as annoying, I needed a way to clean up my tags to remove this unwanted metadata. Since I have imported the music via beets.io, I was able to use that to help fix up the metadata. However, I ran into several albums in my library where the “PMEDIA” watermark kept reappearing, even after Beets reported that the fields had been successfully cleaned and synced.

In my homelab, I’ve been using Longhorn as my primary Kubernetes storage solution. It’s fantastic for high-availability block storage, but I still wanted a way to keep my data synchronized with my central NAS for long-term backups and to make it easier to recover from total cluster failures.

Initially, I was manually adding a pair of initContainers as sidecars to every single service YAML. This worked, but it was a maintenance nightmare—I had nearly 1,500 lines of redundant boilerplate across my repository.

My family has a lot of books collected over time and so I wanted to track all the physical books we have. Although there are a few good self-hosted physical book management out there, but none are quite as user-friendly/nice to look at as some of the ebook management applications. In particular I found Booklore which was close to perfect - but it couldn’t handle bulk importing of physical books. There were some some references online to using placeholder epubs to represent physical books - but nothing which described how to do this (or a way to do it in bulk).

Recently, I migrated all my homelab Kubernetes storage to Longhorn. While Longhorn provides excellent features, monitoring its performance is crucial for identifying potential issues and optimizing resource utilization.

Longhorn offers Prometheus as the recommended monitoring solution. However, since I use InfluxDB for my time-series data needs, I needed to find a way to integrate Longhorn metrics with my existing setup.

Longhorn exposes metrics on each of its longhorn-manager pods. However, there’s no dedicated service to access these metrics directly. Furthermore, the names of these pods can change dynamically, making it impossible to create a static endpoint for InfluxDB.

I run a variety of systems including a couple which are running Debian stable. This means that packages (although stable) are often not as current as I’d like. To help get around this I started using prebuilt binaries directly from the upstream projects when they are hosted on GitHub. But this created a new problem - how to keep them up to date.
This article introduces a shell script designed to automate the process of downloading and updating binary files from GitHub repositories.

Recently I was thinking about how my websites are handled and realised that they are extremely static and so using WordPress as the CMS was overkill. This led me down the path of trying to find a replacement.

After looking at some of the various static-site generator options I settled on Hugo for a couple of reasons.

• The first reason was that I’ve spent a lot of time over the past few years working with YAML (thanks k8s) so I appreciate that I can do all the configuration in YAML
• The second was that the site contents are all written using Markdown (used for internal documentation where I work)
• The third was that it all runs from a single binary which I can grab pre-compiled from github (so no need to install any packages/recompile manually)
• The fourth I also like the what content can be laid out in a structured and navigable manner
• The fifth is that it is themeable in a reproducible way
• Finally I really appreciate that the way it works fits nicely into version control (so I can put it all in git)

The end result is a nicely organised and tracked website which can be found in GitHub. The entire site is tracked in there with change logging etc meaning I can quickly rebuild if needed. I am using the theme Mainroad and to enable searching it is using Pagefind. The only modification needed to the theme was to add ‘data-pagefind-body’ to single.html so that only the standalone pages would be indexed (not the list pages).

Now that I have xLights being built fine within docker (see Automatic testing of xLights builds via Travis-CI/Docker/Github) the next step is to package up the application as an appimage binary so that it can easily be run on varied linux systems.

It is being built based on Ubuntu Trusty and has been tested to work on Ubuntu 16.04, 16.10, Fedora 25 and OpenSuse 42.2 (and should hopefully work pretty much anywhere else as well)