Homelab Ansible

This repository runs Ansible playbooks against hosts from NetBox.

Set up the environment

cd ansible
source .venv/bin/activate
ansible-galaxy collection install -r requirements.yml
bao login -method=oidc
direnv allow
ansible-inventory --graph

.envrc loads NetBox credentials from OpenBao into NETBOX_API and NETBOX_TOKEN.

Ansible source, inventories, playbooks, and roles remain under ansible/.

Run a playbook

ansible-playbook playbooks/maintenance.yml

For hosts that use sudo instead of a root connection, prompt for the sudo password:

ansible-playbook playbooks/docker.yml --ask-become-pass

To run against one host or group, use:

ansible-playbook playbooks/maintenance.yml -e target_hosts=my-host
ansible-playbook playbooks/maintenance.yml -e target_hosts=proxmox_hosts

NetBox inventory fallback

NetBox is the source of truth through inventory/netbox.yml. If NetBox is down or its API token is unavailable, use the checked-in static fallback inventory.

ansible-inventory -i inventory/fallback.yml --graph
ansible-playbook -i inventory/fallback.yml playbooks/ping.yml
ansible-playbook -i inventory/fallback.yml playbooks/maintenance.yml -e target_hosts=proxmox_hosts

inventory/fallback.yml uses the same NetBox group names: linux_hosts, proxmox_hosts, k3s_cluster, grafana_alloy_hosts, opentelemetry_collector_hosts, docker_hosts, and hermes_agent_hosts. The playbooks need no changes during an incident. Keep the fallback small, and update host membership in NetBox after it recovers. Refresh the fallback from checked-in Terraform or DNS records only while it remains a useful break-glass inventory.

Playbooks

The OpenTelemetry playbook installs a pinned otelcol-contrib package, reads host metrics and new journald entries, and forwards them over OTLP gRPC. Add the opentelemetry or otel-collector tag to selected NetBox hosts. Set the backend endpoint in inventory before running the playbook:

otelcol_exporter_endpoint: otel-gateway.example.net:4317

TLS verification is on by default. Keep authentication headers and TLS keys in the existing secret store, not in the checked-in role defaults. Run the baseline on selected hosts rather than linux_hosts; Kubernetes node telemetry remains owned by the GitOps-managed collector from ADR 004.

Use the host OpenTelemetry Collector runbook to verify a rollout or remove the collector from a host.

Use common playbooks

To update hosts and reboot only if needed, run:

ansible-playbook playbooks/maintenance.yml -e common_maintenance_reboot_if_required=true

To install the QEMU guest agent, run:

ansible-playbook playbooks/maintenance.yml \
  -e target_hosts=my-vm \
  -e common_maintenance_install_qemu_guest_agent=true \
  -e common_maintenance_apt_upgrade=false

Upgrade the Docker VM from Ubuntu 20.04 to the latest LTS supported by the Ubuntu LTS upgrader, 24.04 by default. Run the playbook against one host at a time. It performs each required LTS hop and reboots between hops. It then reruns the Docker role so the APT repository and packages match the new Ubuntu codename.

ansible-playbook playbooks/ubuntu-lts-upgrade.yml --ask-become-pass -e target_hosts=docker

Only override the target after Ubuntu marks a newer LTS as supported in meta-release-lts.

ansible-playbook playbooks/ubuntu-lts-upgrade.yml \
  --ask-become-pass \
  -e target_hosts=docker \
  -e ubuntu_lts_upgrade_target_version=26.04

To prepare one Proxmox node for the PVE 8→9 apt upgrade without changing VMs or CTs, run:

ansible-playbook playbooks/proxmox-pve9-prereqs.yml -e target_hosts=pve.local.isabelsoler.es

The playbook follows the Proxmox VE 8→9 repository migration in the upgrade guide. It changes existing Debian apt sources from Bookworm to Trixie and points the Proxmox VE repository at PVE 9. It removes legacy list-style Proxmox entries, including entries in /etc/apt/sources.list, to avoid duplicate targets. It also updates an existing Ceph repository to Ceph Squid for Trixie, installs the CPU microcode package for the host, and refreshes the apt cache.

The operator must still run apt policy, pve8to9 --full, and apt dist-upgrade. The playbook does not run apt modernize-sources. That optional post-upgrade command is unavailable on Debian Bookworm with PVE 8 apt. Set proxmox_pve9_write_debian_deb822_sources=true to rewrite the Debian sources as deb822.

Proxmox network contract

All nodes provide the same guest-facing bridges, even though their physical NIC names differ.

VMs and containers attach directly to their network bridge. A VLAN 20 guest uses bridge=vmbr20 without a Proxmox VLAN tag. pfSense uses one NIC on each routed bridge, matching VM 103 on pve2. Proxmox SDN packages remain installed, but the cluster has no SDN zones or VNets that duplicate these physical bridges.

Physical interface mappings and management addresses are in inventory/host_vars/<node>.yml. To review a node without changing it, run:

ansible-playbook playbooks/proxmox-network.yml \
  -e target_hosts=pve2 \
  --check --diff

After you review the diff and confirm console access, write one node's persistent configuration:

ansible-playbook playbooks/proxmox-network.yml \
  -e target_hosts=pve2 \
  -e proxmox_network_confirm=true \
  --diff

The playbook checks that the mapped NICs exist and that ifupdown2 can parse the generated file. It does not reload networking. Run ifreload -a from the node's console during a maintenance window. Verify pfSense and management connectivity before continuing with the next node.

To create a DNS record, run:

DNS_UPDATE_KEY_SECRET=... ansible-playbook playbooks/k8s-dns-record.yml \
  -e name=app \
  -e ip=192.168.1.50

Where to change things

docs/platform/ansible.md