home-services/.github/copilot-instructions.md

Copilot instructions for home-services

Repository overview

This repository manages a home lab through two delivery paths:

• Host services via Docker/Podman Compose under compose-services/
• A Talos Kubernetes cluster bootstrapped with Terraform and reconciled by Argo CD

The core directories are:

• compose-services/ — host-specific compose stacks (media_nas1, media_rhel1)
• bootstrap/applicationsets/ — Argo CD ApplicationSets for infrastructure and app workloads
• infrastructure/ — cluster-level Helm apps such as Cilium and Longhorn
• services/ — application Helm charts such as Vaultwarden
• terraform/ — Talos bootstrap, cluster provisioning, and Argo CD installation
• .gitea/workflows/deploy.yaml — deployment automation for the host stacks

Build, test, and validation commands

There is no repo-wide lint step in this project. Validation is a combination of the Python unit tests for composectl.py and Terraform validation for the bootstrap path.

Run the unit tests:

python3 -m unittest tests.test_composectl

Run a single test by name:

python3 -m unittest tests.test_composectl.ComposeCtlTests.test_build_compose_command_uses_selected_runtime

Inspect the CLI and supported actions:

python3 composectl.py --help

Dry-run a stack refresh before changing live workloads:

python3 composectl.py --dry-run --stack media_nas1 refresh
python3 composectl.py --dry-run --stack media_rhel1 refresh

Terraform bootstrap validation (from terraform/):

cd terraform
terraform init
terraform fmt -check -diff
terraform validate

Architecture and operational flow

1) Compose deployment path

composectl.py is the entry point for runtime-agnostic stack operations. It detects Docker or Podman at runtime and runs the same docker compose/podman compose flow against a stack under compose-services/<stack>/compose.yaml.

The default maintenance flow for a stack is:

1. compose config
2. pull
3. up -d --remove-orphans
4. optional image prune -f

This is the path used by the Gitea workflow in .gitea/workflows/deploy.yaml for media_nas1 and media_rhel1.

2) Talos + GitOps path

terraform/main.tf bootstraps the Talos cluster and installs Argo CD. It creates:

• Talos machine secrets and client configuration
• per-role machine configs for control-plane and worker nodes
• bootstrap of the selected control-plane node
• a kubeconfig for the cluster
• a Helm release for Argo CD
• the gitea-repository secret used by Argo CD to pull from this repo

Once the cluster is ready, Argo CD discovers the application manifests via bootstrap/applicationsets/, which fan out into infrastructure/* and services/* chart definitions.

3) Secret handling and environment boundaries

Sensitive values are intentionally not committed to the repo. The project expects:

• .sops.yaml as the repository secret policy
• age-based decryption for encrypted secrets
• local-only Terraform .tfvars files and environment variables such as TF_VAR_gitea_username / TF_VAR_gitea_password
• non-committed kubeconfig and talosconfig outputs saved to local paths

MCP server

VS Code loads the portable workspace MCP configuration from .mcp.json. It runs:

• The pinned HashiCorp Terraform MCP Server image through Podman, exposing only read-only public Registry lookups for providers and modules. Podman must be installed; the image is pulled on first use. The configuration does not provide HCP Terraform credentials or enable workspace operations.
• KSail via ksail open mcp, which requires the KSail CLI to be installed and available on PATH. KSail executes commands from the workspace directory with the permissions of the current user; its MCP tools can include operations that mutate clusters or local files, so review operations before approving them.

Conventions specific to this repo

• Treat compose-services/<host>/compose.yaml as the source of truth for host workloads; keep stack naming aligned with the folder name.
• Prefer composectl.py --dry-run before any live deployment or refresh.
• Keep the runtime abstraction intact: avoid writing Docker-only or Podman-only commands into repo scripts when composectl.py can handle both.
• Use the existing stack naming pattern: media_nas1 maps to media_nas1_services, and similarly for media_rhel1.
• Keep Terraform inputs site-specific and local; terraform.tfvars.example is a template, not a checked-in secret file.
• Do not commit credentials, tokens, kubeconfigs, or local state artifacts. If a file is sensitive, keep it outside the repo or encrypted under the SOPS policy.
• The Gitea workflow is keyed to the main branch and the first Friday of the month schedule; do not change that deployment cadence without understanding the automated rollback behavior.
• When editing cluster bootstrapping or Argo CD setup, keep the GitOps boundary clear: Terraform provisions the platform, while chart directories under infrastructure/ and services/ hold the workloads.

Practical guidance for Copilot sessions

When making changes here, favor the repository's two-layer model:

• Compose changes belong in compose-services/*/compose.yaml or composectl.py
• Cluster or app delivery changes belong in terraform/, bootstrap/applicationsets/, infrastructure/, or services/

Before proposing deploy commands, check whether the change targets the host Compose path or the Kubernetes/GitOps path. The repository intentionally keeps those flows separate.

Code quality and maintainability

• Read the relevant implementation, tests, and configuration before changing behavior. Follow existing patterns and keep each change focused on the requested outcome.
• Preserve the boundaries between host Compose deployment, Terraform bootstrap, and Argo CD-managed workloads. Keep settings in their existing source of truth instead of duplicating them across scripts, charts, and workflows.
• Prefer small, cohesive functions and straightforward control flow. Reuse existing helpers where they fit, but do not add abstractions for one-off operations or speculative future needs.
• Preserve public interfaces unless the task requires a change: CLI actions and flags, stack naming, Terraform input/output names, chart values, and workflow triggers can all be relied on by operators or automation. If an interface must change, update its callers and documentation together.
• In Python, use clear names, type annotations where they improve clarity, and argument lists rather than shell-built command strings. Keep Docker/Podman differences behind the runtime selection and command-building helpers.
• In Terraform, make site-specific choices explicit inputs, validate important input constraints, and keep provider and module versions consistent with the existing pinning approach.
• In Helm charts, follow existing naming helpers and values conventions; keep resources for the same app consistent in labels and selectors.
• Before editing a chart's values.yaml, check the schema for the exact chart/dependency version declared in its Chart.yaml. Prefer a schema shipped with that exact chart package; never use a schema from main, latest, or a different version as if it were authoritative.
• Current chart schema/reference locations:
  • Cilium: infrastructure/cilium/Chart.yaml pins the Cilium dependency. Its matching upstream schema is values.schema.json for Cilium v1.17.6. Keep this reference aligned with the dependency version when upgrading.
  • cert-manager: infrastructure/cert-manager/Chart.yaml pins the Jetstack cert-manager chart. Its matching upstream schema is values.schema.json for cert-manager v1.21.2. Keep this reference aligned with the dependency version when upgrading; cert-manager's CRDs are enabled and retained by this chart.
  • Longhorn: infrastructure/longhorn/Chart.yaml pins the Longhorn dependency. The matching v1.8.1 chart directory does not include values.schema.json; use that release's values.yaml and chart documentation alongside this repository's values and templates. Do not substitute a schema from another Longhorn release.
  • Vaultwarden: this is a repository-maintained chart under services/vaultwarden/, not an upstream chart dependency, and it currently has no values.schema.json. Use the Gabe565 Vaultwarden chart values as a reference for upstream configuration options, but treat this repository's services/vaultwarden/values.yaml and templates as the actual contract. The upstream chart has its own structure and common-library dependency, so do not copy its values or assume they work here without checking the local templates. If changing or adding local chart values, add or update services/vaultwarden/values.schema.json to validate the local contract.
• Pin versions for deployed software and build/deployment dependencies. Avoid latest, floating tags, and unversioned references; for container images, use an explicit version tag and prefer a digest where practical for immutable deployments. If a floating reference is necessary, explain why and its update risk.
• Keep each version in its configuration source of truth (for example, image tags in Compose or chart values, chart dependencies in Chart.yaml, and Terraform provider selections in constraints and the lock file). Do not duplicate current version numbers in general documentation; explain the pinning or upgrade process there instead.
• Treat version changes as deliberate upgrades: check compatibility with dependent components, update the appropriate lock file, and run the relevant validation. Update operational docs when upgrade steps or compatibility constraints change.
• Make failures visible and actionable. Preserve useful error context and nonzero exit codes; do not silently ignore failed commands or substitute success-shaped defaults.
• Add or update focused tests for changed behavior, including relevant failure cases. Prefer tests that do not require a running container runtime, cluster, or external service.
• Update the directly related operator documentation when commands, configuration, behavior, or operational risks change. Keep examples aligned with the implementation.
• Avoid unrelated formatting or cleanup in files touched by the change. Add comments only when they explain a non-obvious constraint or reason.

Change validation and operational safety

• Run the smallest relevant validation and report exactly what was run and any limitations:
  • python3 -m unittest tests.test_composectl for composectl.py or its tests.
  • terraform fmt -check -diff and terraform validate from terraform/ for Terraform changes; initialize Terraform only when required for validation.
  • helm lint and helm template for chart changes when Helm is available.
• For Compose changes, run the applicable composectl.py --dry-run command and inspect the generated commands. Do not deploy to a host unless explicitly asked.
• For Terraform or cluster changes, review a plan when suitable inputs are available. Do not run terraform apply, mutate a cluster, or otherwise make live infrastructure changes unless explicitly asked.
• Treat workflow cadence, runner targeting, deployment order, and rollback behavior as operational interfaces; preserve them unless the request explicitly requires a change.