kubeadm: update references of "master" label/taint for 1.24

In 1.24 kubeadm will:
- stop using the "master" label on CP nodes.
- start tainting CP nodes with both "master" and "control-plane" taints.

In 1.25 the "master" taint will be removed.

Adjust references of the "master" label/taint to the above.
This commit is contained in:
Lubomir I. Ivanov
2022-01-18 18:08:20 +02:00
parent aef172862a
commit 806518df13
9 changed files with 49 additions and 29 deletions
@@ -173,8 +173,8 @@ kubectl get configmap myconfig \
-o jsonpath='{.data.ca\.crt}' -o jsonpath='{.data.ca\.crt}'
# Get all worker nodes (use a selector to exclude results that have a label # Get all worker nodes (use a selector to exclude results that have a label
# named 'node-role.kubernetes.io/master') # named 'node-role.kubernetes.io/control-plane')
kubectl get node --selector='!node-role.kubernetes.io/master' kubectl get node --selector='!node-role.kubernetes.io/control-plane'
# Get all running pods in the namespace # Get all running pods in the namespace
kubectl get pods --field-selector=status.phase=Running kubectl get pods --field-selector=status.phase=Running
@@ -318,11 +318,12 @@ Please note that:
As soon as the control plane is available, kubeadm executes following actions: As soon as the control plane is available, kubeadm executes following actions:
- Labels the node as control-plane with `node-role.kubernetes.io/master=""` - Labels the node as control-plane with `node-role.kubernetes.io/control-plane=""`
- Taints the node with `node-role.kubernetes.io/master:NoSchedule` - Taints the node with `node-role.kubernetes.io/master:NoSchedule` and `node-role.kubernetes.io/control-plane:NoSchedule`
Please note that: Please note that:
1. The `node-role.kubernetes.io/master` taint is deprecated and will be removed in kubeadm version 1.25
1. Mark control-plane phase phase can be invoked individually with the [`kubeadm init phase mark-control-plane`](/docs/reference/setup-tools/kubeadm/kubeadm-init-phase/#cmd-phase-mark-control-plane) command 1. Mark control-plane phase phase can be invoked individually with the [`kubeadm init phase mark-control-plane`](/docs/reference/setup-tools/kubeadm/kubeadm-init-phase/#cmd-phase-mark-control-plane) command
### Configure TLS-Bootstrapping for node joining ### Configure TLS-Bootstrapping for node joining
@@ -105,7 +105,7 @@ By default the certs and encryption key expire after two hours.
## kubeadm init phase mark-control-plane {#cmd-phase-mark-control-plane} ## kubeadm init phase mark-control-plane {#cmd-phase-mark-control-plane}
Use the following phase to label and taint the node with the `node-role.kubernetes.io/master=""` key-value pair. Use the following phase to label and taint the node as a control plane node.
{{< tabs name="tab-mark-control-plane" >}} {{< tabs name="tab-mark-control-plane" >}}
{{< tab name="mark-control-plane" include="generated/kubeadm_init_phase_mark-control-plane.md" />}} {{< tab name="mark-control-plane" include="generated/kubeadm_init_phase_mark-control-plane.md" />}}
@@ -285,26 +285,30 @@ for `kubeadm`.
### Control plane node isolation ### Control plane node isolation
By default, your cluster will not schedule Pods on the control-plane node for security By default, your cluster will not schedule Pods on the control plane nodes for security
reasons. If you want to be able to schedule Pods on the control-plane node, for example for a reasons. If you want to be able to schedule Pods on the control plane nodes,
single-machine Kubernetes cluster for development, run: for example for a single machine Kubernetes cluster, run:
```bash ```bash
kubectl taint nodes --all node-role.kubernetes.io/master- kubectl taint nodes --all node-role.kubernetes.io/control-plane- node-role.kubernetes.io/control-master-
``` ```
With output looking something like: The output will look something like:
``` ```
node "test-01" untainted node "test-01" untainted
taint "node-role.kubernetes.io/master:" not found ...
taint "node-role.kubernetes.io/master:" not found
``` ```
This will remove the `node-role.kubernetes.io/master` taint from any nodes that This will remove the `node-role.kubernetes.io/control-plane` and
have it, including the control-plane node, meaning that the scheduler will then be able `node-role.kubernetes.io/master` taints from any nodes that have them,
including the control plane nodes, meaning that the scheduler will then be able
to schedule Pods everywhere. to schedule Pods everywhere.
{{< note >}}
The `node-role.kubernetes.io/master` taint is deprecated and kubeadm will stop using it in version 1.25.
{{< /note >}}
### Joining your nodes {#join-nodes} ### Joining your nodes {#join-nodes}
The nodes are where your workloads (containers and Pods, etc) run. To add new nodes to your cluster do the following for each machine: The nodes are where your workloads (containers and Pods, etc) run. To add new nodes to your cluster do the following for each machine:
@@ -351,7 +351,7 @@ A known solution is to patch the kube-proxy DaemonSet to allow scheduling it on
nodes regardless of their conditions, keeping it off of other nodes until their initial guarding nodes regardless of their conditions, keeping it off of other nodes until their initial guarding
conditions abate: conditions abate:
``` ```
kubectl -n kube-system patch ds kube-proxy -p='{ "spec": { "template": { "spec": { "tolerations": [ { "key": "CriticalAddonsOnly", "operator": "Exists" }, { "effect": "NoSchedule", "key": "node-role.kubernetes.io/master" } ] } } } }' kubectl -n kube-system patch ds kube-proxy -p='{ "spec": { "template": { "spec": { "tolerations": [ { "key": "CriticalAddonsOnly", "operator": "Exists" }, { "effect": "NoSchedule", "key": "node-role.kubernetes.io/master" }, { "effect": "NoSchedule", "key": "node-role.kubernetes.io/control-plane" } ] } } } }'
``` ```
The tracking issue for this problem is [here](https://github.com/kubernetes/kubeadm/issues/1027). The tracking issue for this problem is [here](https://github.com/kubernetes/kubeadm/issues/1027).
@@ -59,9 +59,13 @@ spec:
- key: node.cloudprovider.kubernetes.io/uninitialized - key: node.cloudprovider.kubernetes.io/uninitialized
value: "true" value: "true"
effect: NoSchedule effect: NoSchedule
# this is to have the daemonset runnable on master nodes # these tolerations are to have the daemonset runnable on control plane nodes
# the taint may vary depending on your cluster setup # remove them if your control plane nodes should not run pods
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: NoSchedule
- key: node-role.kubernetes.io/master - key: node-role.kubernetes.io/master
operator: Exists
effect: NoSchedule effect: NoSchedule
# this is to restrict CCM to only run on master nodes # this is to restrict CCM to only run on master nodes
# the node selector may vary depending on your cluster setup # the node selector may vary depending on your cluster setup
@@ -15,8 +15,11 @@ spec:
name: fluentd-elasticsearch name: fluentd-elasticsearch
spec: spec:
tolerations: tolerations:
# this toleration is to have the daemonset runnable on master nodes # these tolerations are to have the daemonset runnable on control plane nodes
# remove it if your masters can't run pods # remove them if your control plane nodes should not run pods
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: NoSchedule
- key: node-role.kubernetes.io/master - key: node-role.kubernetes.io/master
operator: Exists operator: Exists
effect: NoSchedule effect: NoSchedule
@@ -19,9 +19,13 @@ spec:
name: fluentd-elasticsearch name: fluentd-elasticsearch
spec: spec:
tolerations: tolerations:
# this toleration is to have the daemonset runnable on master nodes # these tolerations are to have the daemonset runnable on control plane nodes
# remove it if your masters can't run pods # remove them if your control plane nodes should not run pods
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: NoSchedule
- key: node-role.kubernetes.io/master - key: node-role.kubernetes.io/master
operator: Exists
effect: NoSchedule effect: NoSchedule
containers: containers:
- name: fluentd-elasticsearch - name: fluentd-elasticsearch
@@ -19,9 +19,13 @@ spec:
name: fluentd-elasticsearch name: fluentd-elasticsearch
spec: spec:
tolerations: tolerations:
# this toleration is to have the daemonset runnable on master nodes # these tolerations are to have the daemonset runnable on control plane nodes
# remove it if your masters can't run pods # remove them if your control plane nodes should not run pods
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: NoSchedule
- key: node-role.kubernetes.io/master - key: node-role.kubernetes.io/master
operator: Exists
effect: NoSchedule effect: NoSchedule
containers: containers:
- name: fluentd-elasticsearch - name: fluentd-elasticsearch