whentimeslows/home-services
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Configure Talos nodes for Longhorn storage
Deploy home services / deploy-nas1 (push) Has been cancelled
Details
Deploy home services / deploy-rhel1 (push) Has been cancelled
Details

Browse Source 
Add the Talos extension image and dedicated user volume configuration, point Longhorn at the mounted disk, and document rebuild and rolling-upgrade steps.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
Scott Sorell
2026-10-09 11:22:06 -06:00
parent c351a1c3d6
commit e3c51bb3ae
9 changed files with 97 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files
GITOPS.md
+18 -5
View File
@@ -6,28 +6,41 @@ Terraform owns Talos machine configuration, cluster bootstrap, and the Argo CD H
## First bootstrap
1. Confirm Talos `v1.14.0`, a supported Kubernetes version, the control-plane endpoint, the VM install disk, and the `nodes` list. Mark exactly one control-plane node as `bootstrap = true`.
1. Confirm Talos `v1.14.0`, the matching `talos_installer_image` (including `iscsi-tools` and `util-linux-tools`), a supported Kubernetes version, the control-plane endpoint, the VM install disk, a dedicated `/dev/vdb` Longhorn data disk on every storage node, and the `nodes` list. Mark exactly one control-plane node as `bootstrap = true`.
2. Confirm the Gitea URL is reachable from the cluster and its certificate is trusted.
3. Run Terraform from `terraform/` using a local variable file that is not committed.
4. Verify the `argocd` namespace and Argo CD pods.
5. Verify the `gitea-repository` Secret has the Argo CD repository label and does not expose credentials in Git.
6. Apply `bootstrap/applicationsets/` once.
6. Apply `terraform/longhorn-user-volume.yaml` to each storage node with `talosctl patch machineconfig`, then reboot nodes one at a time (worker first) and verify they rejoin with the volume mounted at `/var/mnt/longhorn`:
```bash
talosctl patch machineconfig --endpoints 10.0.0.246 --nodes 10.0.0.246 --patch @terraform/longhorn-user-volume.yaml
talosctl patch machineconfig --endpoints 10.0.0.99 --nodes 10.0.0.99 --patch @terraform/longhorn-user-volume.yaml
talosctl reboot --endpoints 10.0.0.246 --nodes 10.0.0.246 --wait
talosctl health --endpoints 10.0.0.99 --nodes 10.0.0.99,10.0.0.246
talosctl reboot --endpoints 10.0.0.99 --nodes 10.0.0.99 --wait
talosctl health --endpoints 10.0.0.99 --nodes 10.0.0.99,10.0.0.246
```
7. Apply `bootstrap/applicationsets/` once.
Terraform generates one machine-secrets bundle, renders one machine configuration per role, applies the matching configuration to every node, and bootstraps only the selected control-plane node. For more than one control-plane node, `cluster_endpoint` must resolve to a stable VIP or load balancer rather than an individual node address.
The Talos installer image is generated from `terraform/talos-image-schematic.yaml`; keep its Image Factory schematic ID and Talos version aligned in the Terraform default and this runbook. The extra Longhorn data disk is provisioned separately using the checked-in UserVolumeConfig because it is a Talos resource, not a Kubernetes workload.
## Talos 1.13 to 1.14 upgrade
Talos 1.14 uses the Image Factory installer image. The old image `ghcr.io/siderolabs/installer:v1.14.0` is not published and returns `NotFound`. For these amd64 KVM nodes, use the configured schematic:
```bash
export TALOSCONFIG="$HOME/.talos/home-services/talosconfig"
export TALOS_INSTALLER_IMAGE="factory.talos.dev/metal-installer/376567988ad370138ad8b2698212367b8edcb69b5fd68c80be1f2ec7d603b4ba:v1.14.0"
export TALOS_INSTALLER_IMAGE="factory.talos.dev/metal-installer/613e1592b2da41ae5e265e8789429f22e121aab91cb4deb6bc3c0b6262961245:v1.14.0"
```
Upgrade one node at a time, worker first:
```bash
talosctl upgrade --endpoints 10.0.0.246 --nodes 10.0.0.246 --image "$TALOS_INSTALLER_IMAGE" --wait
talosctl upgrade --endpoints 10.0.0.246 --nodes 10.0.0.246 --image "$TALOS_INSTALLER_IMAGE" --drain=false --wait
talosctl version --endpoints 10.0.0.246 --nodes 10.0.0.246
talosctl upgrade --endpoints 10.0.0.99 --nodes 10.0.0.99 --image "$TALOS_INSTALLER_IMAGE" --drain=false --wait
@@ -35,7 +48,7 @@ talosctl version --endpoints 10.0.0.99 --nodes 10.0.0.99
talosctl health --endpoints 10.0.0.99 --nodes 10.0.0.99,10.0.0.246
```
Set `--endpoints` to the same address as `--nodes` for each upgrade. This prevents Talosctl from attempting request forwarding through the other node during the version-compatibility check. The control-plane upgrade uses `--drain=false` because this single-control-plane cluster cannot safely drain through a Kubernetes kubeconfig during the Talos transition.
Set `--endpoints` to the same address as `--nodes` for each upgrade. This prevents Talosctl from attempting request forwarding through the other node during the version-compatibility check. Both upgrades use `--drain=false`: there is only one schedulable worker for Argo CD, and the single-control-plane cluster cannot safely drain through a Kubernetes kubeconfig during the Talos transition.
After both nodes report 1.14, set `talos_version = "v1.14.0"` in `terraform.tfvars`, then run `terraform plan` and `terraform apply` to reconcile the generated machine configurations and GitOps workloads. Do not reset or recreate the VMs during an in-place upgrade.
README.md
+21 -4
View File
@@ -135,7 +135,7 @@ The Kubernetes path accepts a `nodes` list containing any number of control-plan
The token is stored in Terraform state and in the Kubernetes Secret, so protect the state backend and shell history. Use an environment variable or a protected variable-file mechanism rather than committing the token.
4. Run `terraform init`, `terraform fmt`, and `terraform validate` from `terraform/`.
5. Boot every KVM VM from the Talos ISO with its install disk attached. Terraform applies the role-specific configuration to every node, installs Talos to `install_disk`, bootstraps the selected control-plane node, and installs Argo CD.
5. Boot every KVM VM from the Talos ISO with its install disk and dedicated 10 GiB Longhorn data disk attached. The data disk must appear as `/dev/vdb`. Terraform applies the role-specific configuration to every node, installs the pinned Talos Image Factory image (including the Longhorn-required extensions) to `install_disk`, bootstraps the selected control-plane node, and installs Argo CD.
6. Export the kubeconfig produced by Terraform before running `kubectl`. Terraform cannot safely infer your shell's active kubeconfig or make a later interactive `kubectl` command part of the apply, but it does expose the generated config as the sensitive `kubeconfig` output:
```bash
@@ -157,8 +157,25 @@ The Kubernetes path accepts a `nodes` list containing any number of control-plan
talosctl --nodes "$(terraform output -json talos_nodes | jq -r 'join(",")')" version
```
8. Apply both ApplicationSets with `kubectl apply -f bootstrap/applicationsets/`.
9. Connect the Argo CD CLI through the internal service:
8. Provision the dedicated Longhorn disk on each node using the checked-in Talos volume configuration. Apply it to both nodes before rebooting either node:
```bash
talosctl patch machineconfig --endpoints 10.0.0.246 --nodes 10.0.0.246 --patch @terraform/longhorn-user-volume.yaml
talosctl patch machineconfig --endpoints 10.0.0.99 --nodes 10.0.0.99 --patch @terraform/longhorn-user-volume.yaml
```
For a fresh bootstrap, reboot one node at a time, worker first, and wait for it to rejoin before restarting the control-plane node:
```bash
talosctl reboot --endpoints 10.0.0.246 --nodes 10.0.0.246 --wait
talosctl health --endpoints 10.0.0.99 --nodes 10.0.0.99,10.0.0.246
talosctl reboot --endpoints 10.0.0.99 --nodes 10.0.0.99 --wait
talosctl health --endpoints 10.0.0.99 --nodes 10.0.0.99,10.0.0.246
```
For an existing cluster, use the rolling extension upgrade procedure in [TALOS.md](./TALOS.md) instead of a plain reboot. The Talos volume is mounted at `/var/mnt/longhorn`, which is the Longhorn data path. The selector only targets `/dev/vdb`; verify the added disk appears there on every VM before applying it.
9. Apply both ApplicationSets with `kubectl apply -f bootstrap/applicationsets/`.
10. Connect the Argo CD CLI through the internal service:
```bash
argocd login --port-forward --port-forward-namespace argocd argocd-server
@@ -171,7 +188,7 @@ The Kubernetes path accepts a `nodes` list containing any number of control-plan
kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath='{.data.password}' | base64 -d
```
10. Confirm Argo CD discovers `infrastructure/cilium`, `infrastructure/longhorn`, and `services/vaultwarden`.
11. Confirm Argo CD discovers `infrastructure/cilium`, `infrastructure/longhorn`, and `services/vaultwarden`, and that the Longhorn StorageClass is available before relying on Vaultwarden persistence.
Talos uses its default CNI during bootstrap so Argo CD can start. Cilium is then reconciled as the first infrastructure application. Confirm the LAN interface and address pool before enabling production LoadBalancer services. Longhorn also requires a validated disk or mount on the storage node.
TALOS.md
+35 -3
View File
@@ -31,26 +31,36 @@ Talos 1.14 no longer publishes the release installer image at `ghcr.io/siderolab
```bash
export TALOSCONFIG="$HOME/.talos/home-services/talosconfig"
export TALOS_INSTALLER_IMAGE="factory.talos.dev/metal-installer/376567988ad370138ad8b2698212367b8edcb69b5fd68c80be1f2ec7d603b4ba:v1.14.0"
export TALOS_INSTALLER_IMAGE="factory.talos.dev/metal-installer/613e1592b2da41ae5e265e8789429f22e121aab91cb4deb6bc3c0b6262961245:v1.14.0"
```
Upgrade the worker first:
This repository's schematic is defined in `terraform/talos-image-schematic.yaml` and includes `siderolabs/iscsi-tools` and `siderolabs/util-linux-tools`, which Longhorn requires. Rebuild the schematic if the Talos version or extension list changes, then update both `talos_installer_image` in Terraform and `TALOS_INSTALLER_IMAGE` here. To regenerate the schematic ID:
```bash
curl -fsS -X POST https://factory.talos.dev/schematics \
-H 'Content-Type: application/yaml' \
--data-binary @terraform/talos-image-schematic.yaml | jq -r .id
```
For an existing cluster, upgrade one node at a time, worker first:
```bash
talosctl upgrade \
--endpoints 10.0.0.246 \
--nodes 10.0.0.246 \
--image "$TALOS_INSTALLER_IMAGE" \
--drain=false \
--wait
```
After the worker is healthy, upgrade the control-plane node:
Use `--drain=false` for the worker because it is the only schedulable worker and Argo CD has no second worker on which to reschedule. After the worker is healthy, upgrade the single control-plane node without draining:
```bash
talosctl upgrade \
--endpoints 10.0.0.99 \
--nodes 10.0.0.99 \
--image "$TALOS_INSTALLER_IMAGE" \
--drain=false \
--wait
```
@@ -70,3 +80,25 @@ terraform apply
```
Do not use `talosctl reset` or recreate the VMs for this upgrade. The installer preserves the Talos state and Kubernetes data.
## Longhorn node prerequisites
Every node providing Longhorn storage must have the Image Factory extensions above and a dedicated data disk attached as `/dev/vdb`. The current VMs use a 10 GiB disk for Longhorn and a separate `/dev/vda` install disk. Do not select or format `/dev/vda`.
Talos provisions the extra disk through `terraform/longhorn-user-volume.yaml` as a `UserVolumeConfig`, mounting it at `/var/mnt/longhorn`. After Terraform configures the nodes and before applying the Argo CD ApplicationSets, apply the volume config to each node:
```bash
talosctl patch machineconfig \
--endpoints 10.0.0.246 --nodes 10.0.0.246 \
--patch @terraform/longhorn-user-volume.yaml
talosctl patch machineconfig \
--endpoints 10.0.0.99 --nodes 10.0.0.99 \
--patch @terraform/longhorn-user-volume.yaml
```
Reboot nodes one at a time, worker first. For an existing cluster, the extension upgrade above also reboots each node and can be performed after applying the volume configuration. Wait for the worker to return Ready before upgrading or rebooting the control-plane node. Verify the Talos volume is available on both nodes before allowing Longhorn to use it. Longhorn's chart sets `defaultDataPath` to `/var/mnt/longhorn`.
## Cilium Install
Cilium is installed using the recommended method via helm using ArgoCD.
https://docs.siderolabs.com/kubernetes-guides/cni/deploying-cilium#deploy-cilium-cni
infrastructure/longhorn/values.yaml
+1 -1
View File
@@ -6,7 +6,7 @@ longhorn:
defaultSettings:
defaultReplicaCount: 1
createDefaultDiskLabeledNodes: false
defaultDataPath: /var/lib/longhorn
defaultDataPath: /var/mnt/longhorn
persistence:
defaultClass: true
terraform/longhorn-user-volume.yaml
+8
View File
@@ -0,0 +1,8 @@
apiVersion: v1alpha1
kind: UserVolumeConfig
name: longhorn
provisioning:
diskSelector:
match: disk.dev_path == '/dev/vdb'
maxSize: 10GiB
grow: false
terraform/main.tf
+2 -1
View File
@@ -54,7 +54,8 @@ data "talos_machine_configuration" "role" {
yamlencode({
machine = {
install = {
disk = var.install_disk
disk = var.install_disk
image = var.talos_installer_image
}
}
cluster = {
terraform/talos-image-schematic.yaml
+5
View File
@@ -0,0 +1,5 @@
customization:
systemExtensions:
officialExtensions:
- siderolabs/iscsi-tools
- siderolabs/util-linux-tools
terraform/terraform.tfvars.example
+1
View File
@@ -1,5 +1,6 @@
cluster_name = "home-services"
talos_version = "v1.14.0"
talos_installer_image = "factory.talos.dev/metal-installer/613e1592b2da41ae5e265e8789429f22e121aab91cb4deb6bc3c0b6262961245:v1.14.0"
kubernetes_version = "<supported-kubernetes-version>"
cluster_endpoint = "https://<control-plane-vip>:6443"
install_disk = "/dev/vda"
terraform/variables.tf
+6
View File
@@ -10,6 +10,12 @@ variable "talos_version" {
default = "v1.14.0"
}
variable "talos_installer_image" {
type = string
description = "Talos Image Factory installer image with iscsi-tools and util-linux-tools extensions. Keep its Talos version aligned with talos_version."
default = "factory.talos.dev/metal-installer/613e1592b2da41ae5e265e8789429f22e121aab91cb4deb6bc3c0b6262961245:v1.14.0"
}
variable "kubernetes_version" {
type = string
description = "Kubernetes version supported by the selected Talos version."