kubeadm: update references of "master" label/taint for 1.24

In 1.24 kubeadm will:
- stop using the "master" label on CP nodes.
- start tainting CP nodes with both "master" and "control-plane" taints.

In 1.25 the "master" taint will be removed.

Adjust references of the "master" label/taint to the above.
This commit is contained in:
Lubomir I. Ivanov
2022-01-18 18:08:20 +02:00
parent aef172862a
commit 806518df13
9 changed files with 49 additions and 29 deletions
@@ -173,8 +173,8 @@ kubectl get configmap myconfig \
-o jsonpath='{.data.ca\.crt}'
# Get all worker nodes (use a selector to exclude results that have a label
# named 'node-role.kubernetes.io/master')
kubectl get node --selector='!node-role.kubernetes.io/master'
# named 'node-role.kubernetes.io/control-plane')
kubectl get node --selector='!node-role.kubernetes.io/control-plane'
# Get all running pods in the namespace
kubectl get pods --field-selector=status.phase=Running
@@ -318,11 +318,12 @@ Please note that:
As soon as the control plane is available, kubeadm executes following actions:
- Labels the node as control-plane with `node-role.kubernetes.io/master=""`
- Taints the node with `node-role.kubernetes.io/master:NoSchedule`
- Labels the node as control-plane with `node-role.kubernetes.io/control-plane=""`
- Taints the node with `node-role.kubernetes.io/master:NoSchedule` and `node-role.kubernetes.io/control-plane:NoSchedule`
Please note that:
1. The `node-role.kubernetes.io/master` taint is deprecated and will be removed in kubeadm version 1.25
1. Mark control-plane phase phase can be invoked individually with the [`kubeadm init phase mark-control-plane`](/docs/reference/setup-tools/kubeadm/kubeadm-init-phase/#cmd-phase-mark-control-plane) command
### Configure TLS-Bootstrapping for node joining
@@ -105,7 +105,7 @@ By default the certs and encryption key expire after two hours.
## kubeadm init phase mark-control-plane {#cmd-phase-mark-control-plane}
Use the following phase to label and taint the node with the `node-role.kubernetes.io/master=""` key-value pair.
Use the following phase to label and taint the node as a control plane node.
{{< tabs name="tab-mark-control-plane" >}}
{{< tab name="mark-control-plane" include="generated/kubeadm_init_phase_mark-control-plane.md" />}}
@@ -285,26 +285,30 @@ for `kubeadm`.
### Control plane node isolation
By default, your cluster will not schedule Pods on the control-plane node for security
reasons. If you want to be able to schedule Pods on the control-plane node, for example for a
single-machine Kubernetes cluster for development, run:
By default, your cluster will not schedule Pods on the control plane nodes for security
reasons. If you want to be able to schedule Pods on the control plane nodes,
for example for a single machine Kubernetes cluster, run:
```bash
kubectl taint nodes --all node-role.kubernetes.io/master-
kubectl taint nodes --all node-role.kubernetes.io/control-plane- node-role.kubernetes.io/control-master-
```
With output looking something like:
The output will look something like:
```
node "test-01" untainted
taint "node-role.kubernetes.io/master:" not found
taint "node-role.kubernetes.io/master:" not found
...
```
This will remove the `node-role.kubernetes.io/master` taint from any nodes that
have it, including the control-plane node, meaning that the scheduler will then be able
This will remove the `node-role.kubernetes.io/control-plane` and
`node-role.kubernetes.io/master` taints from any nodes that have them,
including the control plane nodes, meaning that the scheduler will then be able
to schedule Pods everywhere.
{{< note >}}
The `node-role.kubernetes.io/master` taint is deprecated and kubeadm will stop using it in version 1.25.
{{< /note >}}
### Joining your nodes {#join-nodes}
The nodes are where your workloads (containers and Pods, etc) run. To add new nodes to your cluster do the following for each machine:
@@ -351,7 +351,7 @@ A known solution is to patch the kube-proxy DaemonSet to allow scheduling it on
nodes regardless of their conditions, keeping it off of other nodes until their initial guarding
conditions abate:
```
kubectl -n kube-system patch ds kube-proxy -p='{ "spec": { "template": { "spec": { "tolerations": [ { "key": "CriticalAddonsOnly", "operator": "Exists" }, { "effect": "NoSchedule", "key": "node-role.kubernetes.io/master" } ] } } } }'
kubectl -n kube-system patch ds kube-proxy -p='{ "spec": { "template": { "spec": { "tolerations": [ { "key": "CriticalAddonsOnly", "operator": "Exists" }, { "effect": "NoSchedule", "key": "node-role.kubernetes.io/master" }, { "effect": "NoSchedule", "key": "node-role.kubernetes.io/control-plane" } ] } } } }'
```
The tracking issue for this problem is [here](https://github.com/kubernetes/kubeadm/issues/1027).
@@ -59,9 +59,13 @@ spec:
- key: node.cloudprovider.kubernetes.io/uninitialized
value: "true"
effect: NoSchedule
# this is to have the daemonset runnable on master nodes
# the taint may vary depending on your cluster setup
# these tolerations are to have the daemonset runnable on control plane nodes
# remove them if your control plane nodes should not run pods
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: NoSchedule
- key: node-role.kubernetes.io/master
operator: Exists
effect: NoSchedule
# this is to restrict CCM to only run on master nodes
# the node selector may vary depending on your cluster setup
@@ -15,8 +15,11 @@ spec:
name: fluentd-elasticsearch
spec:
tolerations:
# this toleration is to have the daemonset runnable on master nodes
# remove it if your masters can't run pods
# these tolerations are to have the daemonset runnable on control plane nodes
# remove them if your control plane nodes should not run pods
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: NoSchedule
- key: node-role.kubernetes.io/master
operator: Exists
effect: NoSchedule
@@ -19,9 +19,13 @@ spec:
name: fluentd-elasticsearch
spec:
tolerations:
# this toleration is to have the daemonset runnable on master nodes
# remove it if your masters can't run pods
# these tolerations are to have the daemonset runnable on control plane nodes
# remove them if your control plane nodes should not run pods
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: NoSchedule
- key: node-role.kubernetes.io/master
operator: Exists
effect: NoSchedule
containers:
- name: fluentd-elasticsearch
@@ -19,9 +19,13 @@ spec:
name: fluentd-elasticsearch
spec:
tolerations:
# this toleration is to have the daemonset runnable on master nodes
# remove it if your masters can't run pods
# these tolerations are to have the daemonset runnable on control plane nodes
# remove them if your control plane nodes should not run pods
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: NoSchedule
- key: node-role.kubernetes.io/master
operator: Exists
effect: NoSchedule
containers:
- name: fluentd-elasticsearch