whentimeslows/website
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #28360 from neolit123/1.22-add-v1beta3

Browse Source 
kubeadm: use the new v1beta3 instead of v1beta2
This commit is contained in:
Kubernetes Prow Robot
2021-07-08 09:42:54 -07:00
committed by GitHub
parent e10ad9024a 3437d0185c
commit 4c6a904242
10 changed files with 173 additions and 62 deletions
Download Patch File Download Diff File Expand all files Collapse all files
content/en/docs/setup/production-environment/tools/kubeadm/control-plane-flags.md
+140 -20
View File
@@ -1,79 +1,108 @@
---
reviewers:
- sig-cluster-lifecycle
title: Customizing control plane configuration with kubeadm
title: Customizing components with the kubeadm API
content_type: concept
weight: 40
---
<!-- overview -->
This page covers how to customize the components that kubeadm deploys. For control plane components
you can use flags in the `ClusteConfiguration` structure or patches per-node. For the kubelet
and kube-proxy you can use `KubeletConfiguration` and `KubeProxyConfiguration`, accordingly.
All of these options are possible via the kubeadm configuration API.
For more details on each field in the configuration you can navigate to our
[API reference pages](https://godoc.org/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta3).
{{< note >}}
Customizing the CoreDNS deployment of kubeadm is currently not supported. You must manually
patch the `kube-system/coredns` {{< glossary_tooltip text="ConfigMap" term_id="configmap" >}}
and recreate the CoreDNS {{< glossary_tooltip text="Pods" term_id="pod" >}} after that. Alternatively,
you can skip the default CoreDNS deployment and deploy your own variant.
For more details on that see [Using init phases with kubeadm](/docs/reference/setup-tools/kubeadm/kubeadm-init/#init-phases).
{{< /note >}}
<!-- body -->
{{< feature-state for_k8s_version="v1.12" state="stable" >}}
The kubeadm `ClusterConfiguration` object exposes the field `extraArgs` that can override the default flags passed to control plane
components such as the APIServer, ControllerManager and Scheduler. The components are defined using the following fields:
## Customizing the control plane with flags in `ClusterConfiguration`
The kubeadm `ClusterConfiguration` object exposes a way for users to override the default
flags passed to control plane components such as the APIServer, ControllerManager, Scheduler and Etcd.
The components are defined using the following structures:
- `apiServer`
- `controllerManager`
- `scheduler`
- `etcd`
The `extraArgs` field consist of `key: value` pairs. To override a flag for a control plane component:
These structures contain a common `extraArgs` field, that consists of `key: value` pairs.
To override a flag for a control plane component:
1. Add the appropriate fields to your configuration.
2. Add the flags to override to the field.
1. Add the appropriate `extraArgs` to your configuration.
2. Add flags to the `extraArgs` field.
3. Run `kubeadm init` with `--config <YOUR CONFIG YAML>`.
For more details on each field in the configuration you can navigate to our
[API reference pages](https://godoc.org/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta2#ClusterConfiguration).
{{< note >}}
You can generate a `ClusterConfiguration` object with default values by running `kubeadm config print init-defaults` and saving the output to a file of your choice.
You can generate a `ClusterConfiguration` object with default values by running `kubeadm config print init-defaults`
and saving the output to a file of your choice.
{{< /note >}}
{{< note >}}
The `ClusterConfiguration` object is currently global in kubeadm clusters. This means that any flags that you add,
will apply to all instances of the same component on different nodes. To apply individual configuration per component
on different nodes you can use [patches](#patches).
{{< /note >}}
{{< note >}}
Duplicate flags (keys), or passing the same flag `--foo` multiple times, is currently not supported.
To workaround that you must use [patches](#patches).
{{< /note >}}
<!-- body -->
## APIServer flags
### APIServer flags
For details, see the [reference documentation for kube-apiserver](/docs/reference/command-line-tools-reference/kube-apiserver/).
Example usage:
```yaml
apiVersion: kubeadm.k8s.io/v1beta2
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
kubernetesVersion: v1.16.0
apiServer:
extraArgs:
advertise-address: 192.168.0.103
anonymous-auth: "false"
enable-admission-plugins: AlwaysPullImages,DefaultStorageClass
audit-log-path: /home/johndoe/audit.log
```
## ControllerManager flags
### ControllerManager flags
For details, see the [reference documentation for kube-controller-manager](/docs/reference/command-line-tools-reference/kube-controller-manager/).
Example usage:
```yaml
apiVersion: kubeadm.k8s.io/v1beta2
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
kubernetesVersion: v1.16.0
controllerManager:
extraArgs:
cluster-signing-key-file: /home/johndoe/keys/ca.key
bind-address: 0.0.0.0
deployment-controller-sync-period: "50"
```
## Scheduler flags
### Scheduler flags
For details, see the [reference documentation for kube-scheduler](/docs/reference/command-line-tools-reference/kube-scheduler/).
Example usage:
```yaml
apiVersion: kubeadm.k8s.io/v1beta2
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
kubernetesVersion: v1.16.0
scheduler:
@@ -87,4 +116,95 @@ scheduler:
pathType: "File"
```
### Etcd flags
For details, see the [etcd server documentation](https://etcd.io/docs/).
Example usage:
```yaml
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
etcd:
local:
extraArgs:
election-timeout: 1000
```
## Customizing the control plane with patches {#patches}
{{< feature-state for_k8s_version="v1.22" state="beta" >}}
Kubeadm allows you to pass a directory with patch files to `InitConfiguration` and `JoinConfiguration`
on individual nodes. These patches can be used as the last customization step before the control
plane component manifests are written to disk.
You can pass this file to `kubeadm init` with `--config <YOUR CONFIG YAML>`:
```yaml
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
nodeRegistration:
patches:
directory: /home/user/somedir
```
{{< note >}}
For `kubeadm init` you can pass a file containing both a `ClusterConfiguration` and `InitConfiguration`
separated by `---`.
{{< /note >}}
You can pass this file to `kubeadm join` with `--config <YOUR CONFIG YAML>`:
```yaml
apiVersion: kubeadm.k8s.io/v1beta3
kind: JoinConfiguration
nodeRegistration:
patches:
directory: /home/user/somedir
```
The directory must contain files named `target[suffix][+patchtype].extension`.
For example, `kube-apiserver0+merge.yaml` or just `etcd.json`.
- `target` can be one of `kube-apiserver`, `kube-controller-manager`, `kube-scheduler` and `etcd`.
- `patchtype` can be one of `strategic`, `merge` or `json` and these must match the patching formats
[supported by kubectl](/docs/tasks/manage-kubernetes-objects/update-api-object-kubectl-patch).
The default `patchtype` is `strategic`.
- `extension` must be either `json` or `yaml`.
- `suffix` is an optional string that can be used to determine which patches are applied first
alpha-numerically.
{{< note >}}
If you are using `kubeadm upgrade` to upgrade your kubeadm nodes you must again provide the same
patches, so that the customization is preserved after upgrade. To do that you can use the `--patches`
flag, which must point to the same directory. `kubeadm upgrade` currently does not support a configuration
API structure that can be used for the same purpose.
{{< /note >}}
## Customizing the kubelet
To customize the kubelet you can add a `KubeletConfiguration` next to the `ClusterConfiguration` or
`InitConfiguration` separated by `---` within the same configuration file. This file can then be passed to `kubeadm init`.
{{< note >}}
kubeadm applies the same `KubeletConfiguration` to all nodes in the cluster. To apply node
specific settings you can use kubelet flags as overrides by passing them in the `nodeRegistration.kubeletExtraArgs`
field supported by both `InitConfiguration` and `JoinConfiguration`. Some kubelet flags are deprecated,
so check their status in the [kubelet reference documentation](/docs/reference/command-line-tools-reference/kubelet)
before using them.
{{< /note >}}
For more details see [Configuring each kubelet in your cluster using kubeadm](/docs/setup/production-environment/tools/kubeadm/kubelet-integration)
## Customizing kube-proxy
To customize kube-proxy you can pass a `KubeProxyConfiguration` next your `ClusterConfiguration` or
`InitConfiguration` to `kubeadm init` separated by `---`.
For more details you can navigate to our [API reference pages](https://godoc.org/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta3).
{{< note >}}
kubeadm deploys kube-proxy as a {{< glossary_tooltip text="DaemonSet" term_id="daemonset" >}}, which means
that the `KubeProxyConfiguration` would apply to all instances of kube-proxy in the cluster.
{{< /note >}}
content/en/docs/setup/production-environment/tools/kubeadm/dual-stack-support.md
+12 -12
View File
@@ -21,7 +21,7 @@ For each server that you want to use as a {{< glossary_tooltip text="node" term_
You need to have an IPv4 and and IPv6 address range to use. Cluster operators typically
use private address ranges for IPv4. For IPv6, a cluster operator typically chooses a global
unicast address block from within `2000::/3`, using a range that is assigned to the operator.
unicast address block from within `2000::/3`, using a range that is assigned to the operator.
You don't have to route the cluster's IP address ranges to the public internet.
The size of the IP address allocations should be suitable for the number of Pods and
@@ -30,7 +30,7 @@ Services that you are planning to run.
{{< note >}}
If you are upgrading an existing cluster then, by default, the `kubeadm upgrade` command
changes the [feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
`IPv6DualStack` to `true` if that is not already enabled.
`IPv6DualStack` to `true` if that is not already enabled.
However, `kubeadm` does not support making modifications to the pod IP address range
(“cluster CIDR”) nor to the cluster's Service address range (“Service CIDR”).
{{< /note >}}
@@ -45,11 +45,11 @@ similar to the following example:
kubeadm init --pod-network-cidr=10.244.0.0/16,2001:db8:42:0::/56 --service-cidr=10.96.0.0/16,2001:db8:42:1::/112
```
To make things clearer, here is an example kubeadm [configuration file](https://pkg.go.dev/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta2) `kubeadm-config.yaml` for the primary dual-stack control plane node.
To make things clearer, here is an example kubeadm [configuration file](https://pkg.go.dev/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta3) `kubeadm-config.yaml` for the primary dual-stack control plane node.
```yaml
---
apiVersion: kubeadm.k8s.io/v1beta2
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
featureGates:
IPv6DualStack: true
@@ -57,7 +57,7 @@ networking:
podSubnet: 10.244.0.0/16,2001:db8:42:0::/56
serviceSubnet: 10.96.0.0/16,2001:db8:42:1::/112
---
apiVersion: kubeadm.k8s.io/v1beta2
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: "10.100.0.1"
@@ -85,10 +85,10 @@ The `--apiserver-advertise-address` flag does not support dual-stack.
Before joining a node, make sure that the node has IPv6 routable network interface and allows IPv6 forwarding.
Here is an example kubeadm [configuration file](https://pkg.go.dev/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta2) `kubeadm-config.yaml` for joining a worker node to the cluster.
Here is an example kubeadm [configuration file](https://pkg.go.dev/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta3) `kubeadm-config.yaml` for joining a worker node to the cluster.
```yaml
apiVersion: kubeadm.k8s.io/v1beta2
apiVersion: kubeadm.k8s.io/v1beta3
kind: JoinConfiguration
discovery:
bootstrapToken:
@@ -98,9 +98,9 @@ nodeRegistration:
node-ip: 10.100.0.3,fd00:1:2:3::3
```
Also, here is an example kubeadm [configuration file](https://pkg.go.dev/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta2) `kubeadm-config.yaml` for joining another control plane node to the cluster.
Also, here is an example kubeadm [configuration file](https://pkg.go.dev/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta3) `kubeadm-config.yaml` for joining another control plane node to the cluster.
```yaml
apiVersion: kubeadm.k8s.io/v1beta2
apiVersion: kubeadm.k8s.io/v1beta3
kind: JoinConfiguration
controlPlane:
localAPIEndpoint:
@@ -124,7 +124,7 @@ kubeadm join --config=kubeadm-config.yaml ...
### Create a single-stack cluster
{{< note >}}
Enabling the dual-stack feature doesn't mean that you need to use dual-stack addressing.
Enabling the dual-stack feature doesn't mean that you need to use dual-stack addressing.
You can deploy a single-stack cluster that has the dual-stack networking feature enabled.
{{< /note >}}
@@ -134,10 +134,10 @@ In 1.21 the `IPv6DualStack` feature is Beta and the feature gate is defaulted to
kubeadm init --feature-gates IPv6DualStack=false
```
To make things more clear, here is an example kubeadm [configuration file](https://pkg.go.dev/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta2) `kubeadm-config.yaml` for the single-stack control plane node.
To make things more clear, here is an example kubeadm [configuration file](https://pkg.go.dev/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta3) `kubeadm-config.yaml` for the single-stack control plane node.
```yaml
apiVersion: kubeadm.k8s.io/v1beta2
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
featureGates:
IPv6DualStack: false
content/en/docs/setup/production-environment/tools/kubeadm/high-availability.md
+2 -2
View File
@@ -115,7 +115,7 @@ option. Your cluster requirements may need a different configuration.
{{< note >}}
The `kubeadm init` flags `--config` and `--certificate-key` cannot be mixed, therefore if you want
to use the [kubeadm configuration](https://godoc.org/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta2)
to use the [kubeadm configuration](https://godoc.org/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta3)
you must add the `certificateKey` field in the appropriate config locations
(under `InitConfiguration` and `JoinConfiguration: controlPlane`).
{{< /note >}}
@@ -230,7 +230,7 @@ in the kubeadm config file.
1. Create a file called `kubeadm-config.yaml` with the following contents:
apiVersion: kubeadm.k8s.io/v1beta2
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
kubernetesVersion: stable
controlPlaneEndpoint: "LOAD_BALANCER_DNS:LOAD_BALANCER_PORT"
content/en/docs/setup/production-environment/tools/kubeadm/setup-ha-etcd-with-kubeadm.md
+1 -1
View File
@@ -95,7 +95,7 @@ this example.
HOST=${ETCDHOSTS[$i]}
NAME=${NAMES[$i]}
cat << EOF > /tmp/${HOST}/kubeadmcfg.yaml
apiVersion: "kubeadm.k8s.io/v1beta2"
apiVersion: "kubeadm.k8s.io/v1beta3"
kind: ClusterConfiguration
etcd:
local:
content/en/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm.md
+4 -21
View File
@@ -367,23 +367,6 @@ kubectl -n kube-system patch ds kube-proxy -p='{ "spec": { "template": { "spec":
The tracking issue for this problem is [here](https://github.com/kubernetes/kubeadm/issues/1027).
## The NodeRegistration.Taints field is omitted when marshalling kubeadm configuration
*Note: This [issue](https://github.com/kubernetes/kubeadm/issues/1358) only applies to tools that marshal kubeadm types (e.g. to a YAML configuration file). It will be fixed in kubeadm API v1beta2.*
By default, kubeadm applies the `node-role.kubernetes.io/master:NoSchedule` taint to control-plane nodes.
If you prefer kubeadm to not taint the control-plane node, and set `InitConfiguration.NodeRegistration.Taints` to an empty slice,
the field will be omitted when marshalling. When the field is omitted, kubeadm applies the default taint.
There are at least two workarounds:
1. Use the `node-role.kubernetes.io/master:PreferNoSchedule` taint instead of an empty slice. [Pods will get scheduled on masters](/docs/concepts/scheduling-eviction/taint-and-toleration/), unless other nodes have capacity.
2. Remove the taint after kubeadm init exits:
```bash
kubectl taint nodes NODE_NAME node-role.kubernetes.io/master:NoSchedule-
```
## `/usr` is mounted read-only on nodes {#usr-mounted-read-only}
On Linux distributions such as Fedora CoreOS or Flatcar Container Linux, the directory `/usr` is mounted as a read-only filesystem.
@@ -393,19 +376,19 @@ Kubernetes components like the kubelet and kube-controller-manager use the defau
for the feature to work.
To workaround this issue you can configure the flex-volume directory using the kubeadm
[configuration file](https://godoc.org/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta2).
[configuration file](https://godoc.org/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta3).
On the primary control-plane Node (created using `kubeadm init`) pass the following
file using `--config`:
```yaml
apiVersion: kubeadm.k8s.io/v1beta2
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
nodeRegistration:
kubeletExtraArgs:
volume-plugin-dir: "/opt/libexec/kubernetes/kubelet-plugins/volume/exec/"
---
apiVersion: kubeadm.k8s.io/v1beta2
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
controllerManager:
extraArgs:
@@ -415,7 +398,7 @@ controllerManager:
On joining Nodes:
```yaml
apiVersion: kubeadm.k8s.io/v1beta2
apiVersion: kubeadm.k8s.io/v1beta3
kind: JoinConfiguration
nodeRegistration:
kubeletExtraArgs: