Add kubelet config reference
This is a reference for kubelet config generated from github.com/tengqm/genref tool using the following command: ./genref -include kubelet-config
This commit is contained in:
Qiming Teng
@@ -224,7 +224,7 @@ When a ConfigMap currently consumed in a volume is updated, projected keys are e
|
||||
The kubelet checks whether the mounted ConfigMap is fresh on every periodic sync.
|
||||
However, the kubelet uses its local cache for getting the current value of the ConfigMap.
|
||||
The type of the cache is configurable using the `ConfigMapAndSecretChangeDetectionStrategy` field in
|
||||
the [KubeletConfiguration struct](https://github.com/kubernetes/kubernetes/blob/{{< param "docsbranch" >}}/staging/src/k8s.io/kubelet/config/v1beta1/types.go).
|
||||
the [KubeletConfiguration struct](/docs/reference/config-api/kubelet-config.v1beta1/)).
|
||||
A ConfigMap can be either propagated by watch (default), ttl-based, or by redirecting
|
||||
all requests directly to the API server.
|
||||
As a result, the total delay from the moment when the ConfigMap is updated to the moment
|
||||
@@ -233,6 +233,7 @@ propagation delay, where the cache propagation delay depends on the chosen cache
|
||||
(it equals to watch propagation delay, ttl of cache, or zero correspondingly).
|
||||
|
||||
ConfigMaps consumed as environment variables are not updated automatically and require a pod restart.
|
||||
|
||||
## Immutable ConfigMaps {#configmap-immutable}
|
||||
|
||||
{{< feature-state for_k8s_version="v1.19" state="beta" >}}
|
||||
|
||||
@@ -21,9 +21,6 @@ allowed to use more of that resource than the limit you set. The kubelet also re
|
||||
at least the _request_ amount of that system resource specifically for that container
|
||||
to use.
|
||||
|
||||
|
||||
|
||||
|
||||
<!-- body -->
|
||||
|
||||
## Requests and limits
|
||||
@@ -442,12 +439,15 @@ If you want to use project quotas, you should:
|
||||
|
||||
* Enable the `LocalStorageCapacityIsolationFSQuotaMonitoring=true`
|
||||
[feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
|
||||
in the kubelet configuration.
|
||||
using the `featureGates` field in the
|
||||
[kubelet configuration](/docs/reference/config-api/kubelet-config.v1beta1/)
|
||||
or the `--feature-gates` command line flag.
|
||||
|
||||
* Ensure that the root filesystem (or optional runtime filesystem)
|
||||
has project quotas enabled. All XFS filesystems support project quotas.
|
||||
For ext4 filesystems, you need to enable the project quota tracking feature
|
||||
while the filesystem is not mounted.
|
||||
|
||||
```bash
|
||||
# For ext4, with /dev/block-device not mounted
|
||||
sudo tune2fs -O project -Q prjquota /dev/block-device
|
||||
|
||||
@@ -668,7 +668,7 @@ When a secret currently consumed in a volume is updated, projected keys are even
|
||||
The kubelet checks whether the mounted secret is fresh on every periodic sync.
|
||||
However, the kubelet uses its local cache for getting the current value of the Secret.
|
||||
The type of the cache is configurable using the `ConfigMapAndSecretChangeDetectionStrategy` field in
|
||||
the [KubeletConfiguration struct](https://github.com/kubernetes/kubernetes/blob/{{< param "docsbranch" >}}/staging/src/k8s.io/kubelet/config/v1beta1/types.go).
|
||||
the [KubeletConfiguration struct](/docs/reference/config-api/kubelet-config.v1beta1/).
|
||||
A Secret can be either propagated by watch (default), ttl-based, or by redirecting
|
||||
all requests directly to the API server.
|
||||
As a result, the total delay from the moment when the Secret is updated to the moment
|
||||
@@ -760,8 +760,8 @@ data has the following advantages:
|
||||
- improves performance of your cluster by significantly reducing load on kube-apiserver, by
|
||||
closing watches for secrets marked as immutable.
|
||||
|
||||
This feature is controlled by the `ImmutableEphemeralVolumes` [feature
|
||||
gate](/docs/reference/command-line-tools-reference/feature-gates/),
|
||||
This feature is controlled by the `ImmutableEphemeralVolumes`
|
||||
[feature gate](/docs/reference/command-line-tools-reference/feature-gates/),
|
||||
which is enabled by default since v1.19. You can create an immutable
|
||||
Secret by setting the `immutable` field to `true`. For example,
|
||||
```yaml
|
||||
@@ -865,6 +865,7 @@ start until all the Pod's volumes are mounted.
|
||||
### Use-Case: As container environment variables
|
||||
|
||||
Create a secret
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
@@ -877,6 +878,7 @@ data:
|
||||
```
|
||||
|
||||
Create the Secret:
|
||||
|
||||
```shell
|
||||
kubectl apply -f mysecret.yaml
|
||||
```
|
||||
@@ -992,7 +994,7 @@ For example, if your actual password is `S!B\*d$zDsb=`, you should execute the c
|
||||
kubectl create secret generic dev-db-secret --from-literal=username=devuser --from-literal=password='S!B\*d$zDsb='
|
||||
```
|
||||
|
||||
You do not need to escape special characters in passwords from files (`--from-file`).
|
||||
You do not need to escape special characters in passwords from files (`--from-file`).
|
||||
{{< /note >}}
|
||||
|
||||
Now make the Pods:
|
||||
@@ -1173,14 +1175,12 @@ privileged, system-level components.
|
||||
|
||||
Applications that need to access the Secret API should perform `get` requests on
|
||||
the secrets they need. This lets administrators restrict access to all secrets
|
||||
while [white-listing access to individual instances](
|
||||
/docs/reference/access-authn-authz/rbac/#referring-to-resources) that
|
||||
while [white-listing access to individual instances](/docs/reference/access-authn-authz/rbac/#referring-to-resources) that
|
||||
the app needs.
|
||||
|
||||
For improved performance over a looping `get`, clients can design resources that
|
||||
reference a secret then `watch` the resource, re-requesting the secret when the
|
||||
reference changes. Additionally, a ["bulk watch" API](
|
||||
https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/bulk_watch.md)
|
||||
reference changes. Additionally, a ["bulk watch" API](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/bulk_watch.md)
|
||||
to let clients `watch` individual resources has also been proposed, and will likely
|
||||
be available in future releases of Kubernetes.
|
||||
|
||||
|
||||
@@ -51,7 +51,9 @@ client libraries:
|
||||
|
||||
## Components
|
||||
|
||||
* [kubelet](/docs/reference/command-line-tools-reference/kubelet/) - The primary *node agent* that runs on each node. The kubelet takes a set of PodSpecs and ensures that the described containers are running and healthy.
|
||||
* [kubelet](/docs/reference/command-line-tools-reference/kubelet/) - The
|
||||
primary agent that runs on each node. The kubelet takes a set of PodSpecs
|
||||
and ensures that the described containers are running and healthy.
|
||||
* [kube-apiserver](/docs/reference/command-line-tools-reference/kube-apiserver/) - REST API that validates and configures data for API objects such as pods, services, replication controllers.
|
||||
* [kube-controller-manager](/docs/reference/command-line-tools-reference/kube-controller-manager/) - Daemon that embeds the core control loops shipped with Kubernetes.
|
||||
* [kube-proxy](/docs/reference/command-line-tools-reference/kube-proxy/) - Can
|
||||
@@ -66,6 +68,10 @@ client libraries:
|
||||
|
||||
* [kube-proxy configuration (v1alpha1)](/docs/reference/config-api/kube-proxy-config.v1alpha1/)
|
||||
|
||||
## Config APIs
|
||||
|
||||
* [kubelet config (v1beta1)](/docs/reference/config-api/kubelet-config.v1beta1/)
|
||||
|
||||
## Design Docs
|
||||
|
||||
An archive of the design docs for Kubernetes functionality. Good starting points are
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
+36
-18
@@ -24,11 +24,10 @@ found [here](https://github.com/kubernetes/kubernetes/pull/20439).
|
||||
This document describes the process of node initialization, how to set up TLS client certificate bootstrapping for
|
||||
kubelets, and how it works.
|
||||
|
||||
|
||||
|
||||
<!-- body -->
|
||||
|
||||
## Initialization Process
|
||||
|
||||
When a worker node starts up, the kubelet does the following:
|
||||
|
||||
1. Look for its `kubeconfig` file
|
||||
@@ -54,6 +53,7 @@ The TLS Bootstrapping described in this document is intended to simplify, and pa
|
||||
a cluster.
|
||||
|
||||
### Bootstrap Initialization
|
||||
|
||||
In the bootstrap initialization process, the following occurs:
|
||||
|
||||
1. kubelet begins
|
||||
@@ -77,6 +77,7 @@ In the bootstrap initialization process, the following occurs:
|
||||
The rest of this document describes the necessary steps to configure TLS Bootstrapping, and its limitations.
|
||||
|
||||
## Configuration
|
||||
|
||||
To configure for TLS bootstrapping and optional automatic approval, you must configure options on the following components:
|
||||
|
||||
* kube-apiserver
|
||||
@@ -87,6 +88,7 @@ To configure for TLS bootstrapping and optional automatic approval, you must con
|
||||
In addition, you need your Kubernetes Certificate Authority (CA).
|
||||
|
||||
## Certificate Authority
|
||||
|
||||
As without bootstrapping, you will need a Certificate Authority (CA) key and certificate. As without bootstrapping, these will be used
|
||||
to sign the kubelet certificate. As before, it is your responsibility to distribute them to master nodes.
|
||||
|
||||
@@ -96,6 +98,7 @@ We will refer to these as "Kubernetes CA certificate and key".
|
||||
All Kubernetes components that use these certificates - kubelet, kube-apiserver, kube-controller-manager - assume the key and certificate to be PEM-encoded.
|
||||
|
||||
## kube-apiserver configuration
|
||||
|
||||
The kube-apiserver has several requirements to enable TLS bootstrapping:
|
||||
|
||||
* Recognizing CA that signs the client certificate
|
||||
@@ -103,6 +106,7 @@ The kube-apiserver has several requirements to enable TLS bootstrapping:
|
||||
* Authorize the bootstrapping kubelet to create a certificate signing request (CSR)
|
||||
|
||||
### Recognizing client certificates
|
||||
|
||||
This is normal for all client certificate authentication.
|
||||
If not already set, add the `--client-ca-file=FILENAME` flag to the kube-apiserver command to enable
|
||||
client certificate authentication, referencing a certificate authority bundle
|
||||
@@ -110,6 +114,7 @@ containing the signing certificate, for example
|
||||
`--client-ca-file=/var/lib/kubernetes/ca.pem`.
|
||||
|
||||
### Initial bootstrap authentication
|
||||
|
||||
In order for the bootstrapping kubelet to connect to kube-apiserver and request a certificate, it must first authenticate to the server.
|
||||
You can use any [authenticator](/docs/reference/access-authn-authz/authentication/) that can authenticate the kubelet.
|
||||
|
||||
@@ -132,13 +137,13 @@ A kubelet authenticating using bootstrap tokens is authenticated as a user in th
|
||||
|
||||
As this feature matures, you
|
||||
should ensure tokens are bound to a Role Based Access Control (RBAC) policy
|
||||
which limits requests (using the [bootstrap
|
||||
token](/docs/reference/access-authn-authz/bootstrap-tokens/)) strictly to client
|
||||
which limits requests (using the [bootstrap token](/docs/reference/access-authn-authz/bootstrap-tokens/)) strictly to client
|
||||
requests related to certificate provisioning. With RBAC in place, scoping the
|
||||
tokens to a group allows for great flexibility. For example, you could disable a
|
||||
particular bootstrap group's access when you are done provisioning the nodes.
|
||||
|
||||
#### Bootstrap tokens
|
||||
|
||||
Bootstrap tokens are described in detail [here](/docs/reference/access-authn-authz/bootstrap-tokens/). These are tokens that are stored as secrets in the Kubernetes cluster,
|
||||
and then issued to the individual kubelet. You can use a single token for an entire cluster, or issue one per worker node.
|
||||
|
||||
@@ -148,7 +153,7 @@ The process is two-fold:
|
||||
2. Issue the token to the kubelet
|
||||
|
||||
From the kubelet's perspective, one token is like another and has no special meaning.
|
||||
From the kube-apiserver's perspective, however, the bootstrap token is special. Due to its `Type`, `namespace` and `name`, kube-apiserver recognizes it as a special token,
|
||||
From the kube-apiserver's perspective, however, the bootstrap token is special. Due to its `type`, `namespace` and `name`, kube-apiserver recognizes it as a special token,
|
||||
and grants anyone authenticating with that token special bootstrap rights, notably treating them as a member of the `system:bootstrappers` group. This fulfills a basic requirement
|
||||
for TLS bootstrapping.
|
||||
|
||||
@@ -156,17 +161,18 @@ The details for creating the secret are available [here](/docs/reference/access-
|
||||
|
||||
If you want to use bootstrap tokens, you must enable it on kube-apiserver with the flag:
|
||||
|
||||
```
|
||||
```console
|
||||
--enable-bootstrap-token-auth=true
|
||||
```
|
||||
|
||||
#### Token authentication file
|
||||
|
||||
kube-apiserver has an ability to accept tokens as authentication.
|
||||
These tokens are arbitrary but should represent at least 128 bits of entropy derived
|
||||
from a secure random number generator (such as `/dev/urandom` on most modern Linux
|
||||
systems). There are multiple ways you can generate a token. For example:
|
||||
|
||||
```
|
||||
```shell
|
||||
head -c 16 /dev/urandom | od -An -t x | tr -d ' '
|
||||
```
|
||||
|
||||
@@ -175,7 +181,7 @@ will generate tokens that look like `02b50b05283e98dd0fd71db496ef01e8`.
|
||||
The token file should look like the following example, where the first three
|
||||
values can be anything and the quoted group name should be as depicted:
|
||||
|
||||
```
|
||||
```console
|
||||
02b50b05283e98dd0fd71db496ef01e8,kubelet-bootstrap,10001,"system:bootstrappers"
|
||||
```
|
||||
|
||||
@@ -185,11 +191,16 @@ systemd unit file perhaps) to enable the token file. See docs
|
||||
further details.
|
||||
|
||||
### Authorize kubelet to create CSR
|
||||
Now that the bootstrapping node is _authenticated_ as part of the `system:bootstrappers` group, it needs to be _authorized_ to create a certificate signing request (CSR) as well as retrieve it when done. Fortunately, Kubernetes ships with a `ClusterRole` with precisely these (and only these) permissions, `system:node-bootstrapper`.
|
||||
|
||||
Now that the bootstrapping node is _authenticated_ as part of the
|
||||
`system:bootstrappers` group, it needs to be _authorized_ to create a
|
||||
certificate signing request (CSR) as well as retrieve it when done.
|
||||
Fortunately, Kubernetes ships with a `ClusterRole` with precisely these (and
|
||||
only these) permissions, `system:node-bootstrapper`.
|
||||
|
||||
To do this, you only need to create a `ClusterRoleBinding` that binds the `system:bootstrappers` group to the cluster role `system:node-bootstrapper`.
|
||||
|
||||
```
|
||||
```yaml
|
||||
# enable bootstrapping nodes to create CSR
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
@@ -206,6 +217,7 @@ roleRef:
|
||||
```
|
||||
|
||||
## kube-controller-manager configuration
|
||||
|
||||
While the apiserver receives the requests for certificates from the kubelet and authenticates those requests,
|
||||
the controller-manager is responsible for issuing actual signed certificates.
|
||||
|
||||
@@ -221,6 +233,7 @@ In order for the controller-manager to sign certificates, it needs the following
|
||||
* enabling CSR signing
|
||||
|
||||
### Access to key and certificate
|
||||
|
||||
As described earlier, you need to create a Kubernetes CA key and certificate, and distribute it to the master nodes.
|
||||
These will be used by the controller-manager to sign the kubelet certificates.
|
||||
|
||||
@@ -230,23 +243,24 @@ with the flag `--client-ca-file=FILENAME` (for example, `--client-ca-file=/var/l
|
||||
|
||||
To provide the Kubernetes CA key and certificate to kube-controller-manager, use the following flags:
|
||||
|
||||
```
|
||||
```shell
|
||||
--cluster-signing-cert-file="/etc/path/to/kubernetes/ca/ca.crt" --cluster-signing-key-file="/etc/path/to/kubernetes/ca/ca.key"
|
||||
```
|
||||
|
||||
for example:
|
||||
|
||||
```
|
||||
```shell
|
||||
--cluster-signing-cert-file="/var/lib/kubernetes/ca.pem" --cluster-signing-key-file="/var/lib/kubernetes/ca-key.pem"
|
||||
```
|
||||
|
||||
The validity duration of signed certificates can be configured with flag:
|
||||
|
||||
```
|
||||
```shell
|
||||
--cluster-signing-duration
|
||||
```
|
||||
|
||||
### Approval
|
||||
|
||||
In order to approve CSRs, you need to tell the controller-manager that it is acceptable to approve them. This is done by granting
|
||||
RBAC permissions to the correct group.
|
||||
|
||||
@@ -257,7 +271,7 @@ There are two distinct sets of permissions:
|
||||
|
||||
To enable the kubelet to request and receive a new certificate, create a `ClusterRoleBinding` that binds the group in which the bootstrapping node is a member `system:bootstrappers` to the `ClusterRole` that grants it permission, `system:certificates.k8s.io:certificatesigningrequests:nodeclient`:
|
||||
|
||||
```yml
|
||||
```yaml
|
||||
# Approve all CSRs for the group "system:bootstrappers"
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
@@ -276,7 +290,7 @@ roleRef:
|
||||
To enable the kubelet to renew its own client certificate, create a `ClusterRoleBinding` that binds the group in which the fully functioning node is a member `system:nodes` to the `ClusterRole` that
|
||||
grants it permission, `system:certificates.k8s.io:certificatesigningrequests:selfnodeclient`:
|
||||
|
||||
```yml
|
||||
```yaml
|
||||
# Approve renewal CSRs for the group "system:nodes"
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
@@ -294,8 +308,8 @@ roleRef:
|
||||
|
||||
The `csrapproving` controller that ships as part of
|
||||
[kube-controller-manager](/docs/admin/kube-controller-manager/) and is enabled
|
||||
by default. The controller uses the [`SubjectAccessReview`
|
||||
API](/docs/reference/access-authn-authz/authorization/#checking-api-access) to
|
||||
by default. The controller uses the
|
||||
[`SubjectAccessReview` API](/docs/reference/access-authn-authz/authorization/#checking-api-access) to
|
||||
determine if a given user is authorized to request a CSR, then approves based on
|
||||
the authorization outcome. To prevent conflicts with other approvers, the
|
||||
builtin approver doesn't explicitly deny CSRs. It only ignores unauthorized
|
||||
@@ -304,6 +318,7 @@ collection.
|
||||
|
||||
|
||||
## kubelet configuration
|
||||
|
||||
Finally, with the master nodes properly set up and all of the necessary authentication and authorization in place, we can configure the kubelet.
|
||||
|
||||
The kubelet requires the following configuration to bootstrap:
|
||||
@@ -317,7 +332,7 @@ The bootstrap `kubeconfig` should be in a path available to the kubelet, for exa
|
||||
|
||||
Its format is identical to a normal `kubeconfig` file. A sample file might look as follows:
|
||||
|
||||
```yml
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Config
|
||||
clusters:
|
||||
@@ -371,6 +386,7 @@ specified by `--kubeconfig`. The certificate and key file will be placed in the
|
||||
directory specified by `--cert-dir`.
|
||||
|
||||
### Client and Serving Certificates
|
||||
|
||||
All of the above relate to kubelet _client_ certificates, specifically, the certificates a kubelet
|
||||
uses to authenticate to kube-apiserver.
|
||||
|
||||
@@ -387,6 +403,7 @@ be used as serving certificates, or `server auth`.
|
||||
However, you _can_ enable its server certificate, at least partially, via certificate rotation.
|
||||
|
||||
### Certificate Rotation
|
||||
|
||||
Kubernetes v1.8 and higher kubelet implements __beta__ features for enabling
|
||||
rotation of its client and/or serving certificates. These can be enabled through
|
||||
the respective `RotateKubeletClientCertificate` and
|
||||
@@ -429,6 +446,7 @@ A deployment-specific approval process for kubelet serving certificates should t
|
||||
{{< /note >}}
|
||||
|
||||
## Other authenticating components
|
||||
|
||||
All of TLS bootstrapping described in this document relates to the kubelet. However,
|
||||
other components may need to communicate directly with kube-apiserver. Notable is kube-proxy, which
|
||||
is part of the Kubernetes control plane and runs on every node, but may also include other components such as monitoring or networking.
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -251,7 +251,7 @@ Other API server flags that are set unconditionally are:
|
||||
- `--requestheader-client-ca-file` to`front-proxy-ca.crt`
|
||||
- `--proxy-client-cert-file` to `front-proxy-client.crt`
|
||||
- `--proxy-client-key-file` to `front-proxy-client.key`
|
||||
- Other flags for securing the front proxy ([API Aggregation](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/aggregated-api-servers.md)) communications:
|
||||
- Other flags for securing the front proxy ([API Aggregation](/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation/)) communications:
|
||||
- `--requestheader-username-headers=X-Remote-User`
|
||||
- `--requestheader-group-headers=X-Remote-Group`
|
||||
- `--requestheader-extra-headers-prefix=X-Remote-Extra-`
|
||||
@@ -305,7 +305,7 @@ into `/var/lib/kubelet/config/init/kubelet` file.
|
||||
|
||||
The init configuration is used for starting the kubelet on this specific node, providing an alternative for the kubelet drop-in file;
|
||||
such configuration will be replaced by the kubelet base configuration as described in following steps.
|
||||
See [set Kubelet parameters via a config file](/docs/tasks/administer-cluster/kubelet-config-file) for additional info.
|
||||
See [set kubelet parameters via a config file](/docs/tasks/administer-cluster/kubelet-config-file) for additional information.
|
||||
|
||||
Please note that:
|
||||
|
||||
@@ -315,6 +315,9 @@ Please note that:
|
||||
a configuration file `--config some-file.yaml`. The `KubeletConfiguration` object can be separated from other objects such
|
||||
as `InitConfiguration` using the `---` separator. For more details have a look at the `kubeadm config print-default` command.
|
||||
|
||||
For more details about the `KubeletConfiguration` struct, take a look at the
|
||||
[`KubeletConfiguration` reference](/docs/reference/config-api/kubelet-config.v1beta1/).
|
||||
|
||||
### Wait for the control plane to come up
|
||||
|
||||
kubeadm waits (upto 4m0s) until `localhost:6443/healthz` (kube-apiserver liveness) returns `ok`. However in order to detect
|
||||
@@ -325,7 +328,8 @@ kubeadm relies on the kubelet to pull the control plane images and run them prop
|
||||
After the control plane is up, kubeadm completes the tasks described in following paragraphs.
|
||||
|
||||
### (optional) Write base kubelet configuration
|
||||
{{< feature-state for_k8s_version="v1.9" state="alpha" >}}
|
||||
|
||||
{{< feature-state for_k8s_version="v1.11" state="beta" >}}
|
||||
|
||||
If kubeadm is invoked with `--feature-gates=DynamicKubeletConfig`:
|
||||
|
||||
@@ -438,8 +442,8 @@ A ServiceAccount for `kube-proxy` is created in the `kube-system` namespace; the
|
||||
|
||||
- In Kubernetes version 1.18 kube-dns usage with kubeadm is deprecated and will be removed in a future release
|
||||
- The CoreDNS service is named `kube-dns`. This is done to prevent any interruption
|
||||
in service when the user is switching the cluster DNS from kube-dns to CoreDNS or vice-versa
|
||||
the `--config` method described [here](/docs/reference/setup-tools/kubeadm/kubeadm-init-phase/#cmd-phase-addon)
|
||||
in service when the user is switching the cluster DNS from kube-dns to CoreDNS or vice-versa
|
||||
the `--config` method described [here](/docs/reference/setup-tools/kubeadm/kubeadm-init-phase/#cmd-phase-addon)
|
||||
- A ServiceAccount for CoreDNS/kube-dns is created in the `kube-system` namespace.
|
||||
- The `kube-dns` ServiceAccount is bound to the privileges in the `system:kube-dns` ClusterRole
|
||||
|
||||
@@ -499,10 +503,9 @@ when the connection with the cluster is established, kubeadm try to access the `
|
||||
|
||||
## TLS Bootstrap
|
||||
|
||||
Once the cluster info are known, the file `bootstrap-kubelet.conf` is written, thus allowing kubelet to do TLS Bootstrapping
|
||||
(conversely until v.1.7 TLS bootstrapping were managed by kubeadm).
|
||||
Once the cluster info are known, the file `bootstrap-kubelet.conf` is written, thus allowing kubelet to do TLS Bootstrapping.
|
||||
|
||||
The TLS bootstrap mechanism uses the shared token to temporarily authenticate with the Kubernetes Master to submit a certificate
|
||||
The TLS bootstrap mechanism uses the shared token to temporarily authenticate with the Kubernetes API server to submit a certificate
|
||||
signing request (CSR) for a locally created key pair.
|
||||
|
||||
The request is then automatically approved and the operation completes saving `ca.crt` file and `kubelet.conf` file to be used
|
||||
@@ -512,17 +515,17 @@ Please note that:
|
||||
|
||||
- The temporary authentication is validated against the token saved during the `kubeadm init` process (or with additional tokens
|
||||
created with `kubeadm token`)
|
||||
- The temporary authentication resolve to a user member of `system:bootstrappers:kubeadm:default-node-token` group which was granted
|
||||
- The temporary authentication resolve to a user member of `system:bootstrappers:kubeadm:default-node-token` group which was granted
|
||||
access to CSR api during the `kubeadm init` process
|
||||
- The automatic CSR approval is managed by the csrapprover controller, according with configuration done the `kubeadm init` process
|
||||
|
||||
### (optional) Write init kubelet configuration
|
||||
|
||||
{{< feature-state for_k8s_version="v1.9" state="alpha" >}}
|
||||
{{< feature-state for_k8s_version="v1.11" state="beta" >}}
|
||||
|
||||
If kubeadm is invoked with `--feature-gates=DynamicKubeletConfig`:
|
||||
|
||||
1. Read the kubelet base configuration from the `kubelet-base-config-v1.9` ConfigMap in the `kube-system` namespace using the
|
||||
1. Read the kubelet base configuration from the `kubelet-base-config-v1.x` ConfigMap in the `kube-system` namespace using the
|
||||
Bootstrap Token credentials, and write it to disk as kubelet init configuration file `/var/lib/kubelet/config/init/kubelet`
|
||||
2. As soon as kubelet starts with the Node's own credential (`/etc/kubernetes/kubelet.conf`), update current node configuration
|
||||
specifying that the source for the node/kubelet configuration is the above ConfigMap.
|
||||
|
||||
@@ -71,7 +71,7 @@ For more details please see the [Network Plugin Requirements](/docs/concepts/ext
|
||||
|
||||
| Protocol | Direction | Port Range | Purpose | Used By |
|
||||
|----------|-----------|------------|-------------------------|---------------------------|
|
||||
| TCP | Inbound | 6443* | Kubernetes API server | All |
|
||||
| TCP | Inbound | 6443\* | Kubernetes API server | All |
|
||||
| TCP | Inbound | 2379-2380 | etcd server client API | kube-apiserver, etcd |
|
||||
| TCP | Inbound | 10250 | kubelet API | Self, Control plane |
|
||||
| TCP | Inbound | 10251 | kube-scheduler | Self |
|
||||
@@ -308,7 +308,8 @@ kind: KubeletConfiguration
|
||||
cgroupDriver: <value>
|
||||
```
|
||||
|
||||
For further details, please read [Using kubeadm init with a configuration file](/docs/reference/setup-tools/kubeadm/kubeadm-init/#config-file).
|
||||
For further details, please read [Using kubeadm init with a configuration file](/docs/reference/setup-tools/kubeadm/kubeadm-init/#config-file)
|
||||
and the [`KubeletConfiguration` reference](/docs/reference/config-api/kubelet-config.v1beta1/)
|
||||
|
||||
Please mind, that you **only** have to do that if the cgroup driver of your CRI
|
||||
is not `cgroupfs`, because that is the default value in the kubelet already.
|
||||
@@ -322,12 +323,11 @@ or `/etc/default/kubelet`(`/etc/sysconfig/kubelet` for RPMs), please remove it a
|
||||
The automatic detection of cgroup driver for other container runtimes
|
||||
like CRI-O and containerd is work in progress.
|
||||
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
If you are running into difficulties with kubeadm, please consult our [troubleshooting docs](/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm/).
|
||||
|
||||
## {{% heading "whatsnext" %}}
|
||||
|
||||
|
||||
* [Using kubeadm to Create a Cluster](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/)
|
||||
|
||||
|
||||
@@ -23,10 +23,8 @@ manager instead, but you need to configure it manually.
|
||||
Some kubelet configuration details need to be the same across all kubelets involved in the cluster, while
|
||||
other configuration aspects need to be set on a per-kubelet basis to accommodate the different
|
||||
characteristics of a given machine (such as OS, storage, and networking). You can manage the configuration
|
||||
of your kubelets manually, but kubeadm now provides a `KubeletConfiguration` API type for [managing your
|
||||
kubelet configurations centrally](#configure-kubelets-using-kubeadm).
|
||||
|
||||
|
||||
of your kubelets manually, but kubeadm now provides a `KubeletConfiguration` API type for
|
||||
[managing your kubelet configurations centrally](#configure-kubelets-using-kubeadm).
|
||||
|
||||
<!-- body -->
|
||||
|
||||
@@ -52,8 +50,9 @@ Virtual IPs for services are now allocated from this subnet. You also need to se
|
||||
by the kubelet, using the `--cluster-dns` flag. This setting needs to be the same for every kubelet
|
||||
on every manager and Node in the cluster. The kubelet provides a versioned, structured API object
|
||||
that can configure most parameters in the kubelet and push out this configuration to each running
|
||||
kubelet in the cluster. This object is called **the kubelet's ComponentConfig**.
|
||||
The ComponentConfig allows the user to specify flags such as the cluster DNS IP addresses expressed as
|
||||
kubelet in the cluster. This object is called
|
||||
[`KubeletConfiguration`](/docs/reference/config-api/kubelet-config.v1beta1/).
|
||||
The `KubeletConfiguration` allows the user to specify flags such as the cluster DNS IP addresses expressed as
|
||||
a list of values to a camelCased key, illustrated by the following example:
|
||||
|
||||
```yaml
|
||||
@@ -63,7 +62,7 @@ clusterDNS:
|
||||
- 10.96.0.10
|
||||
```
|
||||
|
||||
For more details on the ComponentConfig have a look at [this section](#configure-kubelets-using-kubeadm).
|
||||
For more details on the `KubeletConfiguration` have a look at [this section](#configure-kubelets-using-kubeadm).
|
||||
|
||||
### Providing instance-specific configuration details
|
||||
|
||||
@@ -99,8 +98,8 @@ API object is passed with a configuration file like so `kubeadm ... --config som
|
||||
By calling `kubeadm config print init-defaults --component-configs KubeletConfiguration` you can
|
||||
see all the default values for this structure.
|
||||
|
||||
Also have a look at the [API reference for the
|
||||
kubelet ComponentConfig](https://godoc.org/k8s.io/kubernetes/pkg/kubelet/apis/config#KubeletConfiguration)
|
||||
Also have a look at the
|
||||
[reference for the KubeletConfiguration](/docs/reference/config-api/kubelet-config.v1beta1/)
|
||||
for more information on the individual fields.
|
||||
|
||||
### Workflow when using `kubeadm init`
|
||||
@@ -160,9 +159,13 @@ has finished performing the TLS Bootstrap.
|
||||
`kubeadm` ships with configuration for how systemd should run the kubelet.
|
||||
Note that the kubeadm CLI command never touches this drop-in file.
|
||||
|
||||
This configuration file installed by the `kubeadm` [DEB](https://github.com/kubernetes/release/blob/master/cmd/kubepkg/templates/latest/deb/kubeadm/10-kubeadm.conf) or [RPM package](https://github.com/kubernetes/release/blob/master/cmd/kubepkg/templates/latest/rpm/kubeadm/10-kubeadm.conf) is written to
|
||||
This configuration file installed by the `kubeadm`
|
||||
[DEB](https://github.com/kubernetes/release/blob/master/cmd/kubepkg/templates/latest/deb/kubeadm/10-kubeadm.conf) or
|
||||
[RPM package](https://github.com/kubernetes/release/blob/master/cmd/kubepkg/templates/latest/rpm/kubeadm/10-kubeadm.conf) is written to
|
||||
`/etc/systemd/system/kubelet.service.d/10-kubeadm.conf` and is used by systemd.
|
||||
It augments the basic [`kubelet.service` for RPM](https://github.com/kubernetes/release/blob/master/cmd/kubepkg/templates/latest/rpm/kubelet/kubelet.service) or [`kubelet.service` for DEB](https://github.com/kubernetes/release/blob/master/cmd/kubepkg/templates/latest/deb/kubelet/lib/systemd/system/kubelet.service):
|
||||
It augments the basic
|
||||
[`kubelet.service` for RPM](https://github.com/kubernetes/release/blob/master/cmd/kubepkg/templates/latest/rpm/kubelet/kubelet.service) or
|
||||
[`kubelet.service` for DEB](https://github.com/kubernetes/release/blob/master/cmd/kubepkg/templates/latest/deb/kubelet/lib/systemd/system/kubelet.service):
|
||||
|
||||
```none
|
||||
[Service]
|
||||
|
||||
+11
-6
@@ -99,10 +99,11 @@ This may be caused by a number of problems. The most common are:
|
||||
|
||||
There are two common ways to fix the cgroup driver problem:
|
||||
|
||||
1. Install Docker again following instructions
|
||||
[here](/docs/setup/production-environment/container-runtimes/#docker).
|
||||
1. Install Docker again following instructions
|
||||
[here](/docs/setup/production-environment/container-runtimes/#docker).
|
||||
|
||||
1. Change the kubelet config to match the Docker cgroup driver manually, you can refer to [Configure cgroup driver used by kubelet on control-plane node](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#configure-cgroup-driver-used-by-kubelet-on-control-plane-node)
|
||||
1. Change the kubelet config to match the Docker cgroup driver manually, you can refer to
|
||||
[Configure cgroup driver used by kubelet on control-plane node](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#configure-cgroup-driver-used-by-kubelet-on-control-plane-node)
|
||||
|
||||
- control plane Docker containers are crashlooping or hanging. You can check this by running `docker ps` and investigating each container by running `docker logs`.
|
||||
|
||||
@@ -110,8 +111,11 @@ This may be caused by a number of problems. The most common are:
|
||||
|
||||
The following could happen if Docker halts and does not remove any Kubernetes-managed containers:
|
||||
|
||||
```bash
|
||||
```shell
|
||||
sudo kubeadm reset
|
||||
```
|
||||
|
||||
```console
|
||||
[preflight] Running pre-flight checks
|
||||
[reset] Stopping the kubelet service
|
||||
[reset] Unmounting mounted directories in "/var/lib/kubelet"
|
||||
@@ -121,14 +125,14 @@ sudo kubeadm reset
|
||||
|
||||
A possible solution is to restart the Docker service and then re-run `kubeadm reset`:
|
||||
|
||||
```bash
|
||||
```shell
|
||||
sudo systemctl restart docker.service
|
||||
sudo kubeadm reset
|
||||
```
|
||||
|
||||
Inspecting the logs for docker may also be useful:
|
||||
|
||||
```sh
|
||||
```shell
|
||||
journalctl -u docker
|
||||
```
|
||||
|
||||
@@ -415,3 +419,4 @@ If `/var/lib/kubelet` is being mounted, performing a `kubeadm reset` will effect
|
||||
To workaround the issue, re-mount the `/var/lib/kubelet` directory after performing the `kubeadm reset` operation.
|
||||
|
||||
This is a regression introduced in kubeadm 1.15. The issue is fixed in 1.20.
|
||||
|
||||
|
||||
@@ -7,31 +7,21 @@ content_type: task
|
||||
---
|
||||
|
||||
<!-- overview -->
|
||||
{{< feature-state for_k8s_version="v1.10" state="beta" >}}
|
||||
|
||||
A subset of the Kubelet's configuration parameters may be
|
||||
set via an on-disk config file, as a substitute for command-line flags.
|
||||
This functionality is considered beta in v1.10.
|
||||
|
||||
Providing parameters via a config file is the recommended approach because
|
||||
it simplifies node deployment and configuration management.
|
||||
|
||||
|
||||
|
||||
## {{% heading "prerequisites" %}}
|
||||
|
||||
|
||||
- A v1.10 or higher Kubelet binary must be installed for beta functionality.
|
||||
|
||||
|
||||
|
||||
<!-- steps -->
|
||||
|
||||
## Create the config file
|
||||
|
||||
The subset of the Kubelet's configuration that can be configured via a file
|
||||
is defined by the `KubeletConfiguration` struct
|
||||
[here (v1beta1)](https://github.com/kubernetes/kubernetes/blob/{{< param "docsbranch" >}}/staging/src/k8s.io/kubelet/config/v1beta1/types.go).
|
||||
is defined by the
|
||||
[`KubeletConfiguration`](/docs/reference/config-api/kubelet-config.v1beta1/)
|
||||
struct.
|
||||
|
||||
The configuration file must be a JSON or YAML representation of the parameters
|
||||
in this struct. Make sure the Kubelet has read permissions on the file.
|
||||
@@ -73,8 +63,6 @@ If `--config` is provided and the values are not specified via the command line,
|
||||
defaults for the `KubeletConfiguration` version apply.
|
||||
In the above example, this version is `kubelet.config.k8s.io/v1beta1`.
|
||||
|
||||
|
||||
|
||||
<!-- discussion -->
|
||||
|
||||
## Relationship to Dynamic Kubelet Config
|
||||
@@ -83,5 +71,9 @@ If you are using the [Dynamic Kubelet Configuration](/docs/tasks/administer-clus
|
||||
feature, the combination of configuration provided via `--config` and any flags which override these values
|
||||
is considered the default "last known good" configuration by the automatic rollback mechanism.
|
||||
|
||||
## {{% heading "whatsnext" %}}
|
||||
|
||||
- Learn more about kubelet configuration by checking the
|
||||
[`KubeletConfiguration`](/docs/reference/config-api/kubelet-config.v1beta1/)
|
||||
reference.
|
||||
|
||||
|
||||
@@ -22,8 +22,8 @@ but this is unsafe for some parameters. Before deciding to change a parameter
|
||||
dynamically, you need a strong understanding of how that change will affect your
|
||||
cluster's behavior. Always carefully test configuration changes on a small set
|
||||
of nodes before rolling them out cluster-wide. Advice on configuring specific
|
||||
fields is available in the inline `KubeletConfiguration`
|
||||
[type documentation (for v1.20)](https://github.com/kubernetes/kubernetes/blob/release-1.20/staging/src/k8s.io/kubelet/config/v1beta1/types.go).
|
||||
fields is available in the inline
|
||||
[`KubeletConfiguration`](/docs/reference/config-api/kubelet-config.v1beta1/).
|
||||
{{< /warning >}}
|
||||
|
||||
|
||||
@@ -55,7 +55,7 @@ For each node that you're reconfiguring, you must set the kubelet
|
||||
The basic workflow for configuring a kubelet in a live cluster is as follows:
|
||||
|
||||
1. Write a YAML or JSON configuration file containing the
|
||||
kubelet's configuration.
|
||||
kubelet's configuration.
|
||||
2. Wrap this file in a ConfigMap and save it to the Kubernetes control plane.
|
||||
3. Update the kubelet's corresponding Node object to use this ConfigMap.
|
||||
|
||||
@@ -135,24 +135,24 @@ To follow the tasks as written, you need to have `jq` installed. You can
|
||||
adapt the steps if you prefer to extract the `kubeletconfig` subobject manually.
|
||||
{{< /note >}}
|
||||
|
||||
1. Choose a Node to reconfigure. In this example, the name of this Node is
|
||||
referred to as `NODE_NAME`.
|
||||
2. Start the kubectl proxy in the background using the following command:
|
||||
1. Choose a Node to reconfigure. In this example, the name of this Node is
|
||||
referred to as `NODE_NAME`.
|
||||
2. Start the kubectl proxy in the background using the following command:
|
||||
|
||||
```bash
|
||||
kubectl proxy --port=8001 &
|
||||
```
|
||||
3. Run the following command to download and unpack the configuration from the
|
||||
`configz` endpoint. The command is long, so be careful when copying and
|
||||
pasting. **If you use zsh**, note that common zsh configurations add backslashes
|
||||
to escape the opening and closing curly braces around the variable name in the URL.
|
||||
For example: `${NODE_NAME}` will be rewritten as `$\{NODE_NAME\}` during the paste.
|
||||
You must remove the backslashes before running the command, or the command will fail.
|
||||
```shell
|
||||
kubectl proxy --port=8001 &
|
||||
```
|
||||
3. Run the following command to download and unpack the configuration from the
|
||||
`configz` endpoint. The command is long, so be careful when copying and
|
||||
pasting. **If you use zsh**, note that common zsh configurations add backslashes
|
||||
to escape the opening and closing curly braces around the variable name in the URL.
|
||||
For example: `${NODE_NAME}` will be rewritten as `$\{NODE_NAME\}` during the paste.
|
||||
You must remove the backslashes before running the command, or the command will fail.
|
||||
|
||||
|
||||
```bash
|
||||
NODE_NAME="the-name-of-the-node-you-are-reconfiguring"; curl -sSL "http://localhost:8001/api/v1/nodes/${NODE_NAME}/proxy/configz" | jq '.kubeletconfig|.kind="KubeletConfiguration"|.apiVersion="kubelet.config.k8s.io/v1beta1"' > kubelet_configz_${NODE_NAME}
|
||||
```
|
||||
```bash
|
||||
NODE_NAME="the-name-of-the-node-you-are-reconfiguring"; curl -sSL "http://localhost:8001/api/v1/nodes/${NODE_NAME}/proxy/configz" | jq '.kubeletconfig|.kind="KubeletConfiguration"|.apiVersion="kubelet.config.k8s.io/v1beta1"' > kubelet_configz_${NODE_NAME}
|
||||
```
|
||||
|
||||
{{< note >}}
|
||||
You need to manually add the `kind` and `apiVersion` to the downloaded
|
||||
@@ -312,8 +312,6 @@ empty, since all config sources have been reset to `nil`, which indicates that
|
||||
the local default config is `assigned`, `active`, and `lastKnownGood`, and no
|
||||
error is reported.
|
||||
|
||||
|
||||
|
||||
<!-- discussion -->
|
||||
## `kubectl patch` example
|
||||
|
||||
@@ -356,7 +354,7 @@ metadata and checkpoints. The structure of the kubelet's checkpointing directory
|
||||
| - ...
|
||||
```
|
||||
|
||||
## Understanding Node.Status.Config.Error messages {#understanding-node-config-status-errors}
|
||||
## Understanding `Node.Status.Config.Error` messages {#understanding-node-config-status-errors}
|
||||
|
||||
The following table describes error messages that can occur
|
||||
when using Dynamic Kubelet Config. You can search for the identical text
|
||||
@@ -378,6 +376,10 @@ internal failure, see Kubelet log for details | The kubelet encountered some int
|
||||
|
||||
## {{% heading "whatsnext" %}}
|
||||
|
||||
- For more information on configuring the kubelet via a configuration file, see
|
||||
- For more information on configuring the kubelet via a configuration file, see
|
||||
[Set kubelet parameters via a config file](/docs/tasks/administer-cluster/kubelet-config-file).
|
||||
- See the reference documentation for [`NodeConfigSource`](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#nodeconfigsource-v1-core)
|
||||
- Learn more about kubelet configuration by checking the
|
||||
[`KubeletConfiguration`](/docs/reference/config-api/kubelet-config.v1beta1/)
|
||||
reference.
|
||||
|
||||
|
||||
@@ -31,21 +31,14 @@ Pods to run a Pod on every node, you should probably be using a
|
||||
instead.
|
||||
{{< /note >}}
|
||||
|
||||
|
||||
|
||||
## {{% heading "prerequisites" %}}
|
||||
|
||||
|
||||
{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
|
||||
|
||||
This page assumes you're using {{< glossary_tooltip term_id="docker" >}} to run Pods,
|
||||
and that your nodes are running the Fedora operating system.
|
||||
Instructions for other distributions or Kubernetes installations may vary.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<!-- steps -->
|
||||
|
||||
## Create a static pod {#static-pod-creation}
|
||||
@@ -54,7 +47,9 @@ You can configure a static Pod with either a [file system hosted configuration f
|
||||
|
||||
### Filesystem-hosted static Pod manifest {#configuration-files}
|
||||
|
||||
Manifests are standard Pod definitions in JSON or YAML format in a specific directory. Use the `staticPodPath: <the directory>` field in the [kubelet configuration file](/docs/tasks/administer-cluster/kubelet-config-file), which periodically scans the directory and creates/deletes static Pods as YAML/JSON files appear/disappear there.
|
||||
Manifests are standard Pod definitions in JSON or YAML format in a specific directory. Use the `staticPodPath: <the directory>` field in the
|
||||
[kubelet configuration file](/docs/reference/config-api/kubelet-config.v1beta1/),
|
||||
which periodically scans the directory and creates/deletes static Pods as YAML/JSON files appear/disappear there.
|
||||
Note that the kubelet will ignore files starting with dots when scanning the specified directory.
|
||||
|
||||
For example, this is how to start a simple web server as a static Pod:
|
||||
@@ -90,17 +85,18 @@ For example, this is how to start a simple web server as a static Pod:
|
||||
|
||||
3. Configure your kubelet on the node to use this directory by running it with `--pod-manifest-path=/etc/kubelet.d/` argument. On Fedora edit `/etc/kubernetes/kubelet` to include this line:
|
||||
|
||||
```
|
||||
KUBELET_ARGS="--cluster-dns=10.254.0.10 --cluster-domain=kube.local --pod-manifest-path=/etc/kubelet.d/"
|
||||
```
|
||||
or add the `staticPodPath: <the directory>` field in the [kubelet configuration file](/docs/tasks/administer-cluster/kubelet-config-file).
|
||||
```
|
||||
KUBELET_ARGS="--cluster-dns=10.254.0.10 --cluster-domain=kube.local --pod-manifest-path=/etc/kubelet.d/"
|
||||
```
|
||||
or add the `staticPodPath: <the directory>` field in the
|
||||
[kubelet configuration file](/docs/reference/config-api/kubelet-config.v1beta1/).
|
||||
|
||||
4. Restart the kubelet. On Fedora, you would run:
|
||||
|
||||
```shell
|
||||
# Run this command on the node where the kubelet is running
|
||||
systemctl restart kubelet
|
||||
```
|
||||
```shell
|
||||
# Run this command on the node where the kubelet is running
|
||||
systemctl restart kubelet
|
||||
```
|
||||
|
||||
### Web-hosted static pod manifest {#pods-created-via-http}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user