fixed terraform talos build

GITOPS.md

@@ -6,13 +6,21 @@ Terraform owns Talos machine configuration, cluster bootstrap, and the Argo CD H
 
 ## First bootstrap
 
-1. Confirm Talos `v1.14.0`, a supported Kubernetes version, the control-plane endpoint, and both node addresses.
+1. Confirm Talos `v1.14.0`, a supported Kubernetes version, the control-plane endpoint, the VM install disk, and the `nodes` list. Mark exactly one control-plane node as `bootstrap = true`.
 2. Confirm the Gitea URL is reachable from the cluster and its certificate is trusted.
 3. Run Terraform from `terraform/` using a local variable file that is not committed.
 4. Verify the `argocd` namespace and Argo CD pods.
 5. Verify the `gitea-repository` Secret has the Argo CD repository label and does not expose credentials in Git.
 6. Apply `bootstrap/applicationsets/` once.
 
+Terraform generates one machine-secrets bundle, renders one machine configuration per role, applies the matching configuration to every node, and bootstraps only the selected control-plane node. For more than one control-plane node, `cluster_endpoint` must resolve to a stable VIP or load balancer rather than an individual node address.
+
+## Talos certificate mismatch
+
+`talos_machine_secrets.cluster` generates the Talos CA and client certificate used by Terraform. The nodes must have been configured from that same secrets bundle. An error such as `x509: certificate signed by unknown authority` means the node at the configured IP was initialized with a different Talos CA, or the Terraform state was replaced.
+
+Do not disable TLS verification and do not regenerate `talos_machine_secrets` when preserving the cluster. Recover the original machine-secrets bundle and its Terraform state, then retry. If the cluster is disposable, remove the old Talos installation and Terraform state together and bootstrap the nodes again from the newly generated secrets. Verify the target IPs before either action; destroying state alone does not change the CA installed on a node.
+
 ## Reconciliation order
 
 Cilium must become healthy before workloads depending on its networking features. Longhorn must have a usable disk before PVC-backed workloads are considered ready. ApplicationSets use automated sync, pruning, self-healing, and namespace creation, so deleting a chart directory intentionally removes its generated Application resources.

README.md

@@ -68,17 +68,21 @@ The `up` operation avoids an unnecessary `down`, reducing downtime during normal
 
 ## GitOps bootstrap
 
-The Kubernetes path targets the two-node Talos cluster with control plane `10.0.0.99` and worker `10.0.0.246`. The endpoint, Kubernetes version, storage device, Cilium interface, and load-balancer address pool remain explicit inputs because they are site-specific.
+The Kubernetes path accepts a `nodes` list containing any number of control-plane and worker nodes. Each node has a unique name, IP address, and role; exactly one control-plane node is marked `bootstrap = true`. The endpoint, Kubernetes version, storage device, Cilium interface, and load-balancer address pool remain explicit inputs because they are site-specific.
 
-1. Copy `terraform/terraform.tfvars.example` to a local `.tfvars` file and fill in the supported Kubernetes version and stable control-plane endpoint.
+1. Copy `terraform/terraform.tfvars.example` to a local `.tfvars` file and fill in the supported Kubernetes version, stable control-plane endpoint, and `nodes` list.
 2. Run `terraform init`, `terraform fmt`, and `terraform validate` from `terraform/`.
-3. Apply Terraform to generate Talos configuration, bootstrap the control plane, join the worker, and install Argo CD.
+3. Boot every KVM VM from the Talos ISO with its install disk attached. Terraform applies the role-specific configuration to every node, installs Talos to `install_disk`, bootstraps the selected control-plane node, and installs Argo CD.
 4. Supply the Gitea credential through Terraform variables or create the Argo CD repository Secret out-of-band. Never commit tokens or private keys.
 5. Apply both manifests with `kubectl apply -f bootstrap/applicationsets/`.
 6. Confirm Argo CD discovers `infrastructure/cilium`, `infrastructure/longhorn`, and `services/vaultwarden`.
 
 Talos uses its default CNI during bootstrap so Argo CD can start. Cilium is then reconciled as the first infrastructure application. Confirm the LAN interface and address pool before enabling production LoadBalancer services. Longhorn also requires a validated disk or mount on the storage node.
 
+The Terraform Talos provider uses one generated machine-secrets bundle for the whole node list. If the nodes were previously configured with another Talos CA, reset/reinstall them or restore the matching Terraform state before applying. Terraform state contains the generated Talos credentials and must remain private.
+
+Terraform's Talos client certificates must match the CA already installed on the nodes. If apply reports `x509: certificate signed by unknown authority`, stop and verify the node IPs and recover the original machine-secrets bundle/state. Recreating Terraform secrets without rebuilding the nodes will not repair the trust relationship.
+
 ## Secrets
 
 SOPS with age is the intended secret boundary. The repository policy is in `.sops.yaml`; age private keys and plaintext secret files are ignored. Provision the age private key to the cluster through a protected bootstrap process, then commit only files matching the repository's encrypted-file policy. Vaultwarden expects an existing Secret named `vaultwarden-secrets`.

TALOS.md

@@ -0,0 +1,25 @@
+# TALOS Configuration
+
+Prior and separate to this build chain, Talos needs to be configured. Here are the steps that were completed. 
+
+Reference: https://docs.siderolabs.com/talos/v1.14/platform-specific-installations/virtualized-platforms/kvm
+
+Note that kubectl, talosctl, and terraform were installed using homebrew prior.
+
+```Shell
+
+# Setting up VMs
+mkdir -p ~/vm/talos/configs
+cd ~/vm/talos/configs
+vim my-talos-net.xml
+sudo virsh net-define my-talos-net.xml
+sudo virsh net-start my-talos-net
+sudo virsh net-autostart my-talos-net
+sudo mv ~/downloads/metal-arm64.iso /var/lib/libvirt/images/
+sudo chown root: /var/lib/libvirt/images/metal-amd64.iso
+sudo virt-install --virt-type kvm --name control-plane-node-1 --ram 2048 --vcpus 2 --disk path=control-plane-node-1-disk.qcow2,bus=virtio,size=40,format=qcow2 --cdrom metal-amd64.iso --os-variant=linux2022 --network network=my-talos-net --boot hd,cdrom --noautoconsole
+sudo virt-install --virt-type kvm --name worker-node-1 --ram 4096 --vcpus 4 --disk path=control-plane-node-1-disk.qcow2,bus=virtio,size=40,format=qcow2 --cdrom metal-amd64.iso --os-variant=linux2022 --network network=my-talos-net --boot hd,cdrom --noautoconsole
+
+```
+
+Note that the VMs are booted into the ISO and terraform initiates the install and cluster build.

terraform/.terraform.lock.hcl

@@ -0,0 +1,64 @@
+# This file is maintained automatically by "terraform init".
+# Manual edits may be lost in future updates.
+
+provider "registry.terraform.io/hashicorp/helm" {
+  version     = "2.17.0"
+  constraints = "~> 2.16"
+  hashes = [
+    "h1:K5FEjxvDnxb1JF1kG1xr8J3pNGxoaR3Z0IBG9Csm/Is=",
+    "zh:06fb4e9932f0afc1904d2279e6e99353c2ddac0d765305ce90519af410706bd4",
+    "zh:104eccfc781fc868da3c7fec4385ad14ed183eb985c96331a1a937ac79c2d1a7",
+    "zh:129345c82359837bb3f0070ce4891ec232697052f7d5ccf61d43d818912cf5f3",
+    "zh:3956187ec239f4045975b35e8c30741f701aa494c386aaa04ebabffe7749f81c",
+    "zh:66a9686d92a6b3ec43de3ca3fde60ef3d89fb76259ed3313ca4eb9bb8c13b7dd",
+    "zh:88644260090aa621e7e8083585c468c8dd5e09a3c01a432fb05da5c4623af940",
+    "zh:a248f650d174a883b32c5b94f9e725f4057e623b00f171936dcdcc840fad0b3e",
+    "zh:aa498c1f1ab93be5c8fbf6d48af51dc6ef0f10b2ea88d67bcb9f02d1d80d3930",
+    "zh:bf01e0f2ec2468c53596e027d376532a2d30feb72b0b5b810334d043109ae32f",
+    "zh:c46fa84cc8388e5ca87eb575a534ebcf68819c5a5724142998b487cb11246654",
+    "zh:d0c0f15ffc115c0965cbfe5c81f18c2e114113e7a1e6829f6bfd879ce5744fbb",
+    "zh:f569b65999264a9416862bca5cd2a6177d94ccb0424f3a4ef424428912b9cb3c",
+  ]
+}
+
+provider "registry.terraform.io/hashicorp/kubernetes" {
+  version     = "2.38.0"
+  constraints = "~> 2.35"
+  hashes = [
+    "h1:5CkveFo5ynsLdzKk+Kv+r7+U9rMrNjfZPT3a0N/fhgE=",
+    "zh:0af928d776eb269b192dc0ea0f8a3f0f5ec117224cd644bdacdc682300f84ba0",
+    "zh:1be998e67206f7cfc4ffe77c01a09ac91ce725de0abaec9030b22c0a832af44f",
+    "zh:326803fe5946023687d603f6f1bab24de7af3d426b01d20e51d4e6fbe4e7ec1b",
+    "zh:4a99ec8d91193af961de1abb1f824be73df07489301d62e6141a656b3ebfff12",
+    "zh:5136e51765d6a0b9e4dbcc3b38821e9736bd2136cf15e9aac11668f22db117d2",
+    "zh:63fab47349852d7802fb032e4f2b6a101ee1ce34b62557a9ad0f0f0f5b6ecfdc",
+    "zh:924fb0257e2d03e03e2bfe9c7b99aa73c195b1f19412ca09960001bee3c50d15",
+    "zh:b63a0be5e233f8f6727c56bed3b61eb9456ca7a8bb29539fba0837f1badf1396",
+    "zh:d39861aa21077f1bc899bc53e7233262e530ba8a3a2d737449b100daeb303e4d",
+    "zh:de0805e10ebe4c83ce3b728a67f6b0f9d18be32b25146aa89116634df5145ad4",
+    "zh:f569b65999264a9416862bca5cd2a6177d94ccb0424f3a4ef424428912b9cb3c",
+    "zh:faf23e45f0090eef8ba28a8aac7ec5d4fdf11a36c40a8d286304567d71c1e7db",
+  ]
+}
+
+provider "registry.terraform.io/siderolabs/talos" {
+  version     = "0.11.0"
+  constraints = "~> 0.7"
+  hashes = [
+    "h1:dBKhGGL1o8WSJdn+fZaZxiTFEkk0x9JNZLsOW+emNNo=",
+    "zh:0fa82a384b25a58b65523e0ea4768fa1212b1f5cfc0c9379d31162454fedcc9d",
+    "zh:12e0f007b86240b6d06d497fcc5a2092056683a7bb2dac56571d9f0cab0d99a6",
+    "zh:1468add8a3bc8d2d12af9671431b0881639c81b5c04c89f7cbfd34f7eed4e860",
+    "zh:194fca4484d2dbb55351235cfb8c174471925b3390f99ce148d8908a7ae42c20",
+    "zh:1a0c53146625e6e13b6794e823079e696dae53719ffb421bfa77c9e625cf64bb",
+    "zh:245987cdc2cf922a72814ebc502ddd7eec68fbb6d1a64b11a7d2b3f2a514f3a6",
+    "zh:2491690b2881c2670eec72fcfd02a2956865464b6e962110b9317f04ca5d8857",
+    "zh:28d4f6ea1b3ffffaf70823d8ccd05a05e5092b462ad2f66735915f532b71157f",
+    "zh:362cfb8929712dc2fec16d3245bac955f3a99c8b511eee7b9dde13f9ada49dd5",
+    "zh:3c807429a02213e36d044549ec03cb883d141084e1f113553088aa88497de543",
+    "zh:50755afc9c8921e264ba8bf8680e498970ddd4b726f3f41e3ad37d973ba9204c",
+    "zh:585582cf4e629de2b044ce2cf634ee0772f25937edea1c22a3a9b5102d8fc7bf",
+    "zh:696e4d43139cfd55dce7d91da92318839bc9ba553d89885ba202ce61cd0bb729",
+    "zh:d218bab0f67a2a8b15add9b51df3d30f514b57e9a7c1d733ebe97966ea132acb",
+  ]
+}

terraform/main.tf

@@ -1,66 +1,80 @@
 provider "talos" {}
 
 provider "kubernetes" {
-  host                   = data.talos_cluster_kubeconfig.cluster.kubernetes_client_configuration.host
+  host                   = talos_cluster_kubeconfig.cluster.kubernetes_client_configuration.host
-  client_certificate     = base64decode(data.talos_cluster_kubeconfig.cluster.kubernetes_client_configuration.client_certificate)
+  client_certificate     = base64decode(talos_cluster_kubeconfig.cluster.kubernetes_client_configuration.client_certificate)
-  client_key             = base64decode(data.talos_cluster_kubeconfig.cluster.kubernetes_client_configuration.client_key)
+  client_key             = base64decode(talos_cluster_kubeconfig.cluster.kubernetes_client_configuration.client_key)
-  cluster_ca_certificate = base64decode(data.talos_cluster_kubeconfig.cluster.kubernetes_client_configuration.ca_certificate)
+  cluster_ca_certificate = base64decode(talos_cluster_kubeconfig.cluster.kubernetes_client_configuration.ca_certificate)
 }
 
 provider "helm" {
   kubernetes {
-    host                   = data.talos_cluster_kubeconfig.cluster.kubernetes_client_configuration.host
+    host                   = talos_cluster_kubeconfig.cluster.kubernetes_client_configuration.host
-    client_certificate     = base64decode(data.talos_cluster_kubeconfig.cluster.kubernetes_client_configuration.client_certificate)
+    client_certificate     = base64decode(talos_cluster_kubeconfig.cluster.kubernetes_client_configuration.client_certificate)
-    client_key             = base64decode(data.talos_cluster_kubeconfig.cluster.kubernetes_client_configuration.client_key)
+    client_key             = base64decode(talos_cluster_kubeconfig.cluster.kubernetes_client_configuration.client_key)
-    cluster_ca_certificate = base64decode(data.talos_cluster_kubeconfig.cluster.kubernetes_client_configuration.ca_certificate)
+    cluster_ca_certificate = base64decode(talos_cluster_kubeconfig.cluster.kubernetes_client_configuration.ca_certificate)
   }
 }
 
+locals {
+  nodes = {
+    for node in var.nodes : node.name => node
+  }
+
+  bootstrap_nodes = [
+    for node in var.nodes : node
+    if node.bootstrap
+  ]
+
+  bootstrap_node = one(local.bootstrap_nodes)
+}
+
 resource "talos_machine_secrets" "cluster" {
   talos_version = var.talos_version
 }
 
-data "talos_machine_configuration" "control_plane" {
+data "talos_machine_configuration" "role" {
-  cluster_name     = var.cluster_name
+  for_each = toset(["controlplane", "worker"])
-  cluster_endpoint = var.cluster_endpoint
-  machine_type     = "controlplane"
-  talos_version    = var.talos_version
-  kubernetes_version = var.kubernetes_version
-  machine_secrets  = talos_machine_secrets.cluster.machine_secrets
-}
 
-data "talos_machine_configuration" "worker" {
   cluster_name       = var.cluster_name
   cluster_endpoint   = var.cluster_endpoint
-  machine_type       = "worker"
+  machine_type       = each.key
   talos_version      = var.talos_version
   kubernetes_version = var.kubernetes_version
   machine_secrets    = talos_machine_secrets.cluster.machine_secrets
+
+  config_patches = [
+    yamlencode({
+      machine = {
+        install = {
+          disk = var.install_disk
+        }
+      }
+    })
+  ]
 }
 
-resource "talos_machine_configuration_apply" "control_plane" {
+resource "talos_machine_configuration_apply" "node" {
-  client_configuration        = talos_machine_secrets.cluster.client_configuration
+  for_each = local.nodes
-  machine_configuration_input = data.talos_machine_configuration.control_plane.machine_configuration
-  node                        = var.control_plane_ip
-}
 
-resource "talos_machine_configuration_apply" "worker" {
   client_configuration        = talos_machine_secrets.cluster.client_configuration
-  machine_configuration_input = data.talos_machine_configuration.worker.machine_configuration
+  machine_configuration_input = data.talos_machine_configuration.role[each.value.role].machine_configuration
-  node                        = var.worker_ip
+  endpoint                    = each.value.ip
+  node                        = each.value.ip
 }
 
 resource "talos_machine_bootstrap" "cluster" {
-  node                 = var.control_plane_ip
+  node                 = local.bootstrap_node.ip
+  endpoint             = local.bootstrap_node.ip
   client_configuration = talos_machine_secrets.cluster.client_configuration
-  depends_on           = [talos_machine_configuration_apply.control_plane]
+  depends_on           = [talos_machine_configuration_apply.node]
 }
 
-data "talos_cluster_kubeconfig" "cluster" {
+resource "talos_cluster_kubeconfig" "cluster" {
   client_configuration = talos_machine_secrets.cluster.client_configuration
-  node                 = var.control_plane_ip
+  node                 = local.bootstrap_node.ip
-  endpoints            = [var.control_plane_ip]
+  endpoint             = local.bootstrap_node.ip
-  depends_on           = [talos_machine_bootstrap.cluster, talos_machine_configuration_apply.worker]
+  depends_on           = [talos_machine_bootstrap.cluster]
 }
 
 resource "helm_release" "argocd" {
@@ -88,7 +102,7 @@ resource "helm_release" "argocd" {
     }
   })]
 
-  depends_on = [data.talos_cluster_kubeconfig.cluster]
+  depends_on = [talos_cluster_kubeconfig.cluster]
 }
 
 resource "kubernetes_secret_v1" "gitea" {

terraform/outputs.tf

@@ -1,6 +1,12 @@
 output "kubeconfig" {
   description = "Kubeconfig for the bootstrapped Talos cluster."
-  value       = data.talos_cluster_kubeconfig.cluster.kubeconfig_raw
+  value       = talos_cluster_kubeconfig.cluster.kubeconfig_raw
+  sensitive   = true
+}
+
+output "talos_client_configuration" {
+  description = "Talos client configuration generated for this cluster."
+  value       = talos_machine_secrets.cluster.client_configuration
   sensitive   = true
 }
 

terraform/terraform.tfvars

@@ -0,0 +1,21 @@
+cluster_name       = "home-services"
+talos_version      = "v1.14.0"
+kubernetes_version = "v1.37.0"
+cluster_endpoint   = "https://10.0.0.99:6443"
+install_disk       = "/dev/vda"
+nodes = [
+  {
+    name      = "control-plane-node-1"
+    ip        = "10.0.0.99"
+    role      = "controlplane"
+    bootstrap = true
+  },
+  {
+    name = "worker-node-1"
+    ip   = "10.0.0.246"
+    role = "worker"
+  }
+]
+gitea_repository_url = "https://git.sorells.me/whentimeslows/home-services.git"
+gitea_username       = "<gitea-user>"
+gitea_password       = "<gitea-token>"

terraform/terraform.tfvars.example

@@ -1,9 +1,23 @@
 cluster_name       = "home-services"
 talos_version      = "v1.14.0"
 kubernetes_version = "<supported-kubernetes-version>"
-control_plane_ip   = "10.0.0.99"
-worker_ip          = "10.0.0.246"
 cluster_endpoint   = "https://<control-plane-vip>:6443"
+install_disk       = "/dev/vda"
+
+nodes = [
+	{
+		name      = "control-plane-node-1"
+		ip        = "10.0.0.99"
+		role      = "controlplane"
+		bootstrap = true
+	},
+	{
+		name = "worker-node-1"
+		ip   = "10.0.0.246"
+		role = "worker"
+	}
+]
+
 gitea_repository_url = "https://git.sorells.me/whentimeslows/home-services.git"
 gitea_username     = "<gitea-user>"
 gitea_password     = "<gitea-token>"

terraform/variables.tf

@@ -15,16 +15,32 @@ variable "kubernetes_version" {
   description = "Kubernetes version supported by the selected Talos version."
 }
 
-variable "control_plane_ip" {
+variable "install_disk" {
   type        = string
-  description = "Control-plane node address."
+  description = "Disk on every VM where Talos should install itself."
-  default     = "10.0.0.99"
+  default     = "/dev/vda"
 }
 
-variable "worker_ip" {
+variable "nodes" {
-  type        = string
+  type = list(object({
-  description = "Worker node address."
+    name      = string
-  default     = "10.0.0.246"
+    ip        = string
+    role      = string
+    bootstrap = optional(bool, false)
+  }))
+  description = "Talos nodes to configure. Exactly one control-plane node must be marked bootstrap."
+
+  validation {
+    condition = (
+      length(var.nodes) > 0 &&
+      length(distinct([for node in var.nodes : node.name])) == length(var.nodes) &&
+      alltrue([for node in var.nodes : contains(["controlplane", "worker"], node.role)]) &&
+      length([for node in var.nodes : node if node.role == "controlplane"]) > 0 &&
+      length([for node in var.nodes : node if node.bootstrap]) == 1 &&
+      alltrue([for node in var.nodes : !node.bootstrap || node.role == "controlplane"])
+    )
+    error_message = "nodes must have unique names, use controlplane or worker roles, include a control-plane node, and mark exactly one control-plane node as bootstrap."
+  }
 }
 
 variable "cluster_endpoint" {