Skip to content

ADR 012

What to self-host

One operator runs the whole lab, so every self-hosted service has to earn its upgrade, backup and restore work.

Status
Accepted
Rollout
0 of 6 gates
Related
009 011

Accepted, not yet implemented. Git hosting and the registry still run in the lab, and the Oracle VM isn't in Ansible yet.

Context

One operator maintains the homelab. Each self-hosted service is another thing to upgrade, back up and restore, and some of them sit in the recovery path of the platform they run on. The lab needs a rule for what it runs itself, and a rule for how much to spend on bootstrap and recovery.

Decision

Self-host rule

Self-host what teaches something, saves money, or keeps personal data private. Use a SaaS when it is free or cheap, holds no personal data, and running it teaches nothing.

On SaaS Why
GitHub for Git hosting GitHub Free covers private repos. Pro is optional, only for the PR gate in ADR 011. The repos are private, and gitleaks checks the full history before anything leaves the LAN.
GHCR for container images Comes with GitHub. Images stay private.
Bitwarden cloud for copies of the bootstrap and recovery credentials Must work when the lab is down. Used only then. Every rotation updates both copies.
The Oracle Cloud VM, running ntfy and Uptime Kuma Alerting and the dead-man's switch must work when the lab is down.

The rule keeps secrets, personal apps, observability, CI runners and the public docs site in the lab:

  • secrets, in OpenBao (ADR 009), and self-hosted Bitwarden as the primary password manager;
  • personal apps;
  • observability;
  • CI runners;
  • the curated public docs site. CI builds it into the homelab-docs image on GHCR, and it runs on staging Talos, later production, behind the public ingress at docs.dev.igresc.com. GitHub Pages is not used.

ADR 011 covers how GitHub, GHCR and the runners are used.

Bootstrap and recovery rule

The chicken-and-egg problem can't be removed, only made cheaper.

  • Keep a measure if it costs nothing day to day: a one-time setup, or a job that runs on its own and alerts when it breaks.
  • Cut a measure that adds a step to every change or a second thing to keep in sync, unless it covers a failure that happens about once a year: a failed upgrade, OpenBao sealed after a power cut, or a bad Talos config.
  • Protect data hard. Accept a slow, manual rebuild for rare cases.
  • Every recovery path gets a first test and a place in the yearly drill rotation, or it goes. A path nobody has run in a year is probably broken.

Consequences

The lab runs fewer services, and the services it runs are the ones worth learning from or the ones holding private data. A GitHub outage stops merges and CI. The clusters keep running, but new image pulls from GHCR and image builds stop. A rebuild can point Flux at the Docker host's NFS-backed Git copy.

Losing the site and the GitHub account together is an accepted risk. The rebuild is then slow and manual, from the B2 copies of the NFS-backed Git backup and the recovery credentials in Bitwarden cloud.

The Oracle VM is on the free tier, and Oracle reclaims idle free-tier VMs. The lab checks that the VM is alive.

References

Rollout gates

  • The repos in use are on GitHub, and Gitea is shut down. open
  • Images pull from GHCR. open
  • The curated docs site builds from its allowlist into the homelab-docs image on GHCR and serves at docs.dev.igresc.com from Talos. open
  • Bitwarden cloud holds the bootstrap and recovery credentials. Rotating a bootstrap or recovery credential updates both copies. open
  • The Oracle VM is in the Ansible inventory and the docs. ntfy delivers to the phone, Uptime Kuma receives its push pings, and the lab checks that the VM is alive. open
  • The Docker host Git backup runs daily, and one bundle has been restored. open

docs/decisions/012-what-to-self-host.md