Merge remote-tracking branch 'upstream/main' into dev-1.24
This commit is contained in:
@@ -1,7 +1,7 @@
|
||||
---
|
||||
title: Encrypting Secret Data at Rest
|
||||
reviewers:
|
||||
- smarterclayton
|
||||
title: Encrypting Secret Data at Rest
|
||||
content_type: task
|
||||
min-kubernetes-server-version: 1.13
|
||||
---
|
||||
@@ -9,27 +9,26 @@ min-kubernetes-server-version: 1.13
|
||||
<!-- overview -->
|
||||
This page shows how to enable and configure encryption of secret data at rest.
|
||||
|
||||
|
||||
## {{% heading "prerequisites" %}}
|
||||
|
||||
|
||||
* {{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
|
||||
|
||||
* etcd v3.0 or later is required
|
||||
|
||||
|
||||
|
||||
<!-- steps -->
|
||||
|
||||
## Configuration and determining whether encryption at rest is already enabled
|
||||
|
||||
The `kube-apiserver` process accepts an argument `--encryption-provider-config`
|
||||
that controls how API data is encrypted in etcd. An example configuration
|
||||
is provided below.
|
||||
that controls how API data is encrypted in etcd.
|
||||
The configuration is provided as an API named
|
||||
[`EncryptionConfiguration`](/docs/reference/config-api/apiserver-encryption.v1/).
|
||||
An example configuration is provided below.
|
||||
|
||||
{{< caution >}}
|
||||
**IMPORTANT:** For multi-master configurations (with two or more control plane nodes) the encryption configuration file must be the same!
|
||||
Otherwise, the kube-apiserver can't decrypt data stored inside the key-value store.
|
||||
**IMPORTANT:** For high-availability configurations (with two or more control plane nodes), the
|
||||
encryption configuration file must be the same! Otherwise, the `kube-apiserver` component cannot
|
||||
decrypt data stored in the etcd.
|
||||
{{< /caution >}}
|
||||
|
||||
## Understanding the encryption at rest configuration.
|
||||
@@ -39,39 +38,44 @@ apiVersion: apiserver.config.k8s.io/v1
|
||||
kind: EncryptionConfiguration
|
||||
resources:
|
||||
- resources:
|
||||
- secrets
|
||||
- secrets
|
||||
providers:
|
||||
- identity: {}
|
||||
- aesgcm:
|
||||
keys:
|
||||
- name: key1
|
||||
secret: c2VjcmV0IGlzIHNlY3VyZQ==
|
||||
- name: key2
|
||||
secret: dGhpcyBpcyBwYXNzd29yZA==
|
||||
- aescbc:
|
||||
keys:
|
||||
- name: key1
|
||||
secret: c2VjcmV0IGlzIHNlY3VyZQ==
|
||||
- name: key2
|
||||
secret: dGhpcyBpcyBwYXNzd29yZA==
|
||||
- secretbox:
|
||||
keys:
|
||||
- name: key1
|
||||
secret: YWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXoxMjM0NTY=
|
||||
- identity: {}
|
||||
- aesgcm:
|
||||
keys:
|
||||
- name: key1
|
||||
secret: c2VjcmV0IGlzIHNlY3VyZQ==
|
||||
- name: key2
|
||||
secret: dGhpcyBpcyBwYXNzd29yZA==
|
||||
- aescbc:
|
||||
keys:
|
||||
- name: key1
|
||||
secret: c2VjcmV0IGlzIHNlY3VyZQ==
|
||||
- name: key2
|
||||
secret: dGhpcyBpcyBwYXNzd29yZA==
|
||||
- secretbox:
|
||||
keys:
|
||||
- name: key1
|
||||
secret: YWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXoxMjM0NTY=
|
||||
```
|
||||
|
||||
Each `resources` array item is a separate config and contains a complete configuration. The
|
||||
`resources.resources` field is an array of Kubernetes resource names (`resource` or `resource.group`)
|
||||
that should be encrypted. The `providers` array is an ordered list of the possible encryption
|
||||
providers. Only one provider type may be specified per entry (`identity` or `aescbc` may be provided, but not both in the same item).
|
||||
providers.
|
||||
|
||||
The first provider in the list is used to encrypt resources going into storage. When reading
|
||||
resources from storage each provider that matches the stored data attempts to decrypt the data in
|
||||
order. If no provider can read the stored data due to a mismatch in format or secret key, an error
|
||||
Only one provider type may be specified per entry (`identity` or `aescbc` may be provided,
|
||||
but not both in the same item).
|
||||
The first provider in the list is used to encrypt resources written into the storage. When reading
|
||||
resources from storage, each provider that matches the stored data attempts in order to decrypt the
|
||||
data. If no provider can read the stored data due to a mismatch in format or secret key, an error
|
||||
is returned which prevents clients from accessing that resource.
|
||||
|
||||
For more detailed information about the `EncryptionConfiguration` struct, please refer to the
|
||||
[encryption configuration API](/docs/reference/config-api/apiserver-encryption.v1/).
|
||||
|
||||
{{< caution >}}
|
||||
**IMPORTANT:** If any resource is not readable via the encryption config (because keys were changed),
|
||||
If any resource is not readable via the encryption config (because keys were changed),
|
||||
the only recourse is to delete that key from the underlying etcd directly. Calls that attempt to
|
||||
read that resource will fail until it is deleted or a valid decryption key is provided.
|
||||
{{< /caution >}}
|
||||
@@ -90,15 +94,24 @@ Name | Encryption | Strength | Speed | Key Length | Other Considerations
|
||||
Each provider supports multiple keys - the keys are tried in order for decryption, and if the provider
|
||||
is the first provider, the first key is used for encryption.
|
||||
|
||||
__Storing the raw encryption key in the EncryptionConfig only moderately improves your security posture, compared to no encryption.
|
||||
Please use `kms` provider for additional security.__ By default, the `identity` provider is used to protect secrets in etcd, which
|
||||
provides no encryption. `EncryptionConfiguration` was introduced to encrypt secrets locally, with a locally managed key.
|
||||
|
||||
Encrypting secrets with a locally managed key protects against an etcd compromise, but it fails to protect against a host compromise.
|
||||
Since the encryption keys are stored on the host in the EncryptionConfig YAML file, a skilled attacker can access that file and
|
||||
extract the encryption keys.
|
||||
{{< caution >}}
|
||||
Storing the raw encryption key in the EncryptionConfig only moderately improves your security
|
||||
posture, compared to no encryption. Please use `kms` provider for additional security.
|
||||
{{< /caution >}}
|
||||
|
||||
Envelope encryption creates dependence on a separate key, not stored in Kubernetes. In this case, an attacker would need to compromise etcd, the kubeapi-server, and the third-party KMS provider to retrieve the plaintext values, providing a higher level of security than locally-stored encryption keys.
|
||||
By default, the `identity` provider is used to protect Secrets in etcd, which provides no
|
||||
encryption. `EncryptionConfiguration` was introduced to encrypt Secrets locally, with a locally
|
||||
managed key.
|
||||
|
||||
Encrypting Secrets with a locally managed key protects against an etcd compromise, but it fails to
|
||||
protect against a host compromise. Since the encryption keys are stored on the host in the
|
||||
EncryptionConfiguration YAML file, a skilled attacker can access that file and extract the encryption
|
||||
keys.
|
||||
|
||||
Envelope encryption creates dependence on a separate key, not stored in Kubernetes. In this case,
|
||||
an attacker would need to compromise etcd, the `kubeapi-server`, and the third-party KMS provider to
|
||||
retrieve the plaintext values, providing a higher level of security than locally stored encryption keys.
|
||||
|
||||
## Encrypting your data
|
||||
|
||||
@@ -109,113 +122,122 @@ apiVersion: apiserver.config.k8s.io/v1
|
||||
kind: EncryptionConfiguration
|
||||
resources:
|
||||
- resources:
|
||||
- secrets
|
||||
- secrets
|
||||
providers:
|
||||
- aescbc:
|
||||
keys:
|
||||
- name: key1
|
||||
secret: <BASE 64 ENCODED SECRET>
|
||||
- identity: {}
|
||||
- aescbc:
|
||||
keys:
|
||||
- name: key1
|
||||
secret: <BASE 64 ENCODED SECRET>
|
||||
- identity: {}
|
||||
```
|
||||
|
||||
To create a new secret perform the following steps:
|
||||
To create a new Secret, perform the following steps:
|
||||
|
||||
1. Generate a 32 byte random key and base64 encode it. If you're on Linux or macOS, run the following command:
|
||||
1. Generate a 32-byte random key and base64 encode it. If you're on Linux or macOS, run the following command:
|
||||
|
||||
```shell
|
||||
head -c 32 /dev/urandom | base64
|
||||
```
|
||||
|
||||
2. Place that value in the secret field.
|
||||
3. Set the `--encryption-provider-config` flag on the `kube-apiserver` to point to the location of the config file.
|
||||
4. Restart your API server.
|
||||
1. Place that value in the `secret` field of the `EncryptionConfiguration` struct.
|
||||
1. Set the `--encryption-provider-config` flag on the `kube-apiserver` to point to
|
||||
the location of the config file.
|
||||
1. Restart your API server.
|
||||
|
||||
{{< caution >}}
|
||||
Your config file contains keys that can decrypt content in etcd, so you must properly restrict permissions on your masters so only the user who runs the kube-apiserver can read it.
|
||||
Your config file contains keys that can decrypt the contents in etcd, so you must properly restrict
|
||||
permissions on your control-plane nodes so only the user who runs the `kube-apiserver` can read it.
|
||||
{{< /caution >}}
|
||||
|
||||
|
||||
## Verifying that data is encrypted
|
||||
|
||||
Data is encrypted when written to etcd. After restarting your `kube-apiserver`, any newly created or
|
||||
updated secret should be encrypted when stored. To check, you can use the `etcdctl` command line
|
||||
program to retrieve the contents of your secret.
|
||||
updated Secret should be encrypted when stored. To check this, you can use the `etcdctl` command line
|
||||
program to retrieve the contents of your Secret.
|
||||
|
||||
1. Create a new secret called `secret1` in the `default` namespace:
|
||||
1. Create a new Secret called `secret1` in the `default` namespace:
|
||||
|
||||
```shell
|
||||
kubectl create secret generic secret1 -n default --from-literal=mykey=mydata
|
||||
```
|
||||
|
||||
2. Using the etcdctl commandline, read that secret out of etcd:
|
||||
1. Using the `etcdctl` command line, read that Secret out of etcd:
|
||||
|
||||
`ETCDCTL_API=3 etcdctl get /registry/secrets/default/secret1 [...] | hexdump -C`
|
||||
|
||||
where `[...]` must be the additional arguments for connecting to the etcd server.
|
||||
|
||||
3. Verify the stored secret is prefixed with `k8s:enc:aescbc:v1:` which indicates the `aescbc` provider has encrypted the resulting data.
|
||||
1. Verify the stored Secret is prefixed with `k8s:enc:aescbc:v1:` which indicates
|
||||
the `aescbc` provider has encrypted the resulting data.
|
||||
|
||||
4. Verify the secret is correctly decrypted when retrieved via the API:
|
||||
1. Verify the Secret is correctly decrypted when retrieved via the API:
|
||||
|
||||
```shell
|
||||
kubectl describe secret secret1 -n default
|
||||
```
|
||||
|
||||
should match `mykey: bXlkYXRh`, mydata is encoded, check [decoding a secret](/docs/tasks/configmap-secret/managing-secret-using-kubectl/#decoding-secret) to
|
||||
completely decode the secret.
|
||||
The output should contain `mykey: bXlkYXRh`, with contents of `mydata` encoded, check
|
||||
[decoding a Secret](/docs/tasks/configmap-secret/managing-secret-using-kubectl/#decoding-secret)
|
||||
to completely decode the Secret.
|
||||
|
||||
## Ensure all Secrets are encrypted
|
||||
|
||||
## Ensure all secrets are encrypted
|
||||
|
||||
Since secrets are encrypted on write, performing an update on a secret will encrypt that content.
|
||||
Since Secrets are encrypted on write, performing an update on a Secret will encrypt that content.
|
||||
|
||||
```shell
|
||||
kubectl get secrets --all-namespaces -o json | kubectl replace -f -
|
||||
```
|
||||
|
||||
The command above reads all secrets and then updates them to apply server side encryption.
|
||||
The command above reads all Secrets and then updates them to apply server side encryption.
|
||||
|
||||
{{< note >}}
|
||||
If an error occurs due to a conflicting write, retry the command.
|
||||
For larger clusters, you may wish to subdivide the secrets by namespace or script an update.
|
||||
{{< /note >}}
|
||||
|
||||
|
||||
## Rotating a decryption key
|
||||
|
||||
Changing the secret without incurring downtime requires a multi step operation, especially in
|
||||
the presence of a highly available deployment where multiple `kube-apiserver` processes are running.
|
||||
Changing a Secret without incurring downtime requires a multi-step operation, especially in
|
||||
the presence of a highly-available deployment where multiple `kube-apiserver` processes are running.
|
||||
|
||||
1. Generate a new key and add it as the second key entry for the current provider on all servers
|
||||
2. Restart all `kube-apiserver` processes to ensure each server can decrypt using the new key
|
||||
3. Make the new key the first entry in the `keys` array so that it is used for encryption in the config
|
||||
4. Restart all `kube-apiserver` processes to ensure each server now encrypts using the new key
|
||||
5. Run `kubectl get secrets --all-namespaces -o json | kubectl replace -f -` to encrypt all existing secrets with the new key
|
||||
6. Remove the old decryption key from the config after you back up etcd with the new key in use and update all secrets
|
||||
|
||||
With a single `kube-apiserver`, step 2 may be skipped.
|
||||
1. Restart all `kube-apiserver` processes to ensure each server can decrypt using the new key
|
||||
1. Make the new key the first entry in the `keys` array so that it is used for encryption in the config
|
||||
1. Restart all `kube-apiserver` processes to ensure each server now encrypts using the new key
|
||||
1. Run `kubectl get secrets --all-namespaces -o json | kubectl replace -f -` to encrypt all
|
||||
existing Secrets with the new key
|
||||
1. Remove the old decryption key from the config after you have backed up etcd with the new key in use
|
||||
and updated all Secrets
|
||||
|
||||
When running a single `kube-apiserver` instance, step 2 may be skipped.
|
||||
|
||||
## Decrypting all data
|
||||
|
||||
To disable encryption at rest place the `identity` provider as the first entry in the config:
|
||||
To disable encryption at rest, place the `identity` provider as the first entry in the config
|
||||
and restart all `kube-apiserver` processes.
|
||||
|
||||
```yaml
|
||||
apiVersion: apiserver.config.k8s.io/v1
|
||||
kind: EncryptionConfiguration
|
||||
resources:
|
||||
- resources:
|
||||
- secrets
|
||||
- secrets
|
||||
providers:
|
||||
- identity: {}
|
||||
- aescbc:
|
||||
keys:
|
||||
- name: key1
|
||||
secret: <BASE 64 ENCODED SECRET>
|
||||
- identity: {}
|
||||
- aescbc:
|
||||
keys:
|
||||
- name: key1
|
||||
secret: <BASE 64 ENCODED SECRET>
|
||||
```
|
||||
|
||||
and restart all `kube-apiserver` processes. Then run:
|
||||
Then run the following command to force decrypt
|
||||
all Secrets:
|
||||
|
||||
```shell
|
||||
kubectl get secrets --all-namespaces -o json | kubectl replace -f -
|
||||
```
|
||||
to force all secrets to be decrypted.
|
||||
|
||||
## {{% heading "whatsnext" %}}
|
||||
|
||||
* Learn more about the [EncryptionConfiguration configuration API (v1)](/docs/reference/config-api/apiserver-encryption.v1/).
|
||||
|
||||
|
||||
@@ -10,7 +10,9 @@ weight: 10
|
||||
|
||||
{{< feature-state for_k8s_version="v1.15" state="stable" >}}
|
||||
|
||||
Client certificates generated by [kubeadm](/docs/reference/setup-tools/kubeadm/) expire after 1 year. This page explains how to manage certificate renewals with kubeadm.
|
||||
Client certificates generated by [kubeadm](/docs/reference/setup-tools/kubeadm/) expire after 1 year.
|
||||
This page explains how to manage certificate renewals with kubeadm. It also covers other tasks related
|
||||
to kubeadm certificate management.
|
||||
|
||||
## {{% heading "prerequisites" %}}
|
||||
|
||||
@@ -126,13 +128,13 @@ command. In that case, you should explicitly set `--certificate-renewal=true`.
|
||||
|
||||
You can renew your certificates manually at any time with the `kubeadm certs renew` command.
|
||||
|
||||
This command performs the renewal using CA (or front-proxy-CA) certificate and key stored in `/etc/kubernetes/pki`.
|
||||
This command performs the renewal using CA (or front-proxy-CA) certificate and key stored in `/etc/kubernetes/pki`.
|
||||
|
||||
After running the command you should restart the control plane Pods. This is required since
|
||||
dynamic certificate reload is currently not supported for all components and certificates.
|
||||
[Static Pods](/docs/tasks/configure-pod-container/static-pod/) are managed by the local kubelet
|
||||
and not by the API Server, thus kubectl cannot be used to delete and restart them.
|
||||
To restart a static Pod you can temporarily remove its manifest file from `/etc/kubernetes/manifests/`
|
||||
To restart a static Pod you can temporarily remove its manifest file from `/etc/kubernetes/manifests/`
|
||||
and wait for 20 seconds (see the `fileCheckFrequency` value in [KubeletConfiguration struct](/docs/reference/config-api/kubelet-config.v1beta1/).
|
||||
The kubelet will terminate the Pod if it's no longer in the manifest directory.
|
||||
You can then move the file back and after another `fileCheckFrequency` period, the kubelet will recreate
|
||||
@@ -289,3 +291,52 @@ Such a controller is not a secure mechanism unless it not only verifies the Comm
|
||||
in the CSR but also verifies the requested IPs and domain names. This would prevent
|
||||
a malicious actor that has access to a kubelet client certificate to create
|
||||
CSRs requesting serving certificates for any IP or domain name.
|
||||
|
||||
## Generating kubeconfig files for additional users {#kubeconfig-additional-users}
|
||||
|
||||
During cluster creation, kubeadm signs the certificate in the `admin.conf` to have
|
||||
`Subject: O = system:masters, CN = kubernetes-admin`.
|
||||
[`system:masters`](/docs/reference/access-authn-authz/rbac/#user-facing-roles)
|
||||
is a break-glass, super user group that bypasses the authorization layer (e.g. RBAC).
|
||||
Sharing the `admin.conf` with additional users is **not recommended**!
|
||||
|
||||
Instead, you can use the [`kubeadm kubeconfig user`](/docs/reference/setup-tools/kubeadm/kubeadm-kubeconfig)
|
||||
command to generate kubeconfig files for additional users.
|
||||
The command accepts a mixture of command line flags and
|
||||
[kubeadm configuration](/docs/reference/config-api/kubeadm-config.v1beta3/) options.
|
||||
The generated kubeconfig will be written to stdout and can be piped to a file
|
||||
using `kubeadm kubeconfig user ... > somefile.conf`.
|
||||
|
||||
Example configuration file that can be used with `--config`:
|
||||
|
||||
```yaml
|
||||
# example.yaml
|
||||
apiVersion: kubeadm.k8s.io/v1beta3
|
||||
kind: ClusterConfiguration
|
||||
# Will be used as the target "cluster" in the kubeconfig
|
||||
clusterName: "kubernetes"
|
||||
# Will be used as the "server" (IP or DNS name) of this cluster in the kubeconfig
|
||||
controlPlaneEndpoint: "some-dns-address:6443"
|
||||
# The cluster CA key and certificate will be loaded from this local directory
|
||||
certificatesDir: "/etc/kubernetes/pki"
|
||||
```
|
||||
|
||||
Make sure that these settings match the desired target cluster settings.
|
||||
To see the settings of an existing cluster use:
|
||||
|
||||
```shell
|
||||
kubectl get cm kubeadm-config -n kube-system -o=jsonpath="{.data.ClusterConfiguration}"
|
||||
```
|
||||
|
||||
The following example will generate a kubeconfig file with credentials valid for 24 hours
|
||||
for a new user `johndoe` that is part of the `appdevs` group:
|
||||
|
||||
```shell
|
||||
kubeadm kubeconfig user --config example.yaml --org appdevs --client-name johndoe --validity-period 24h
|
||||
```
|
||||
|
||||
The following example will generate a kubeconfig file with administrator credentials valid for 1 week:
|
||||
|
||||
```shell
|
||||
kubeadm kubeconfig user --config example.yaml --client-name admin --validity-period 168h
|
||||
```
|
||||
|
||||
+1
-1
@@ -13,7 +13,7 @@ This page shows how to configure default memory requests and limits for a
|
||||
{{< glossary_tooltip text="namespace" term_id="namespace" >}}.
|
||||
|
||||
A Kubernetes cluster can be divided into namespaces. Once you have a namespace that
|
||||
that has a default memory
|
||||
has a default memory
|
||||
[limit](/docs/concepts/configuration/manage-resources-containers/#requests-and-limits),
|
||||
and you then try to create a Pod with a container that does not specify its own memory
|
||||
limit its own memory limit, then the
|
||||
|
||||
@@ -138,7 +138,8 @@ The sum of their values will account for the total amount of reserved memory.
|
||||
A new `--reserved-memory` flag was added to Memory Manager to allow for this total reserved memory
|
||||
to be split (by a node administrator) and accordingly reserved across many NUMA nodes.
|
||||
|
||||
The flag specifies a comma-separated list of memory reservations per NUMA node.
|
||||
The flag specifies a comma-separated list of memory reservations of different memory types per NUMA node.
|
||||
Memory reservations across multiple NUMA nodes can be specified using semicolon as separator.
|
||||
This parameter is only useful in the context of the Memory Manager feature.
|
||||
The Memory Manager will not use this reserved memory for the allocation of container workloads.
|
||||
|
||||
@@ -180,6 +181,10 @@ or
|
||||
|
||||
`--reserved-memory 0:memory=1Gi --reserved-memory 1:memory=2Gi`
|
||||
|
||||
or
|
||||
|
||||
`--reserved-memory '0:memory=1Gi;1:memory=2Gi'`
|
||||
|
||||
When you specify values for `--reserved-memory` flag, you must comply with the setting that
|
||||
you prior provided via Node Allocatable Feature flags.
|
||||
That is, the following rule must be obeyed for each memory type:
|
||||
@@ -215,7 +220,7 @@ Here is an example of a correct configuration:
|
||||
--kube-reserved=cpu=4,memory=4Gi
|
||||
--system-reserved=cpu=1,memory=1Gi
|
||||
--memory-manager-policy=Static
|
||||
--reserved-memory 0:memory=3Gi --reserved-memory 1:memory=2148Mi
|
||||
--reserved-memory '0:memory=3Gi;1:memory=2148Mi'
|
||||
```
|
||||
|
||||
Let us validate the configuration above:
|
||||
|
||||
+154
@@ -0,0 +1,154 @@
|
||||
---
|
||||
title: "Changing the Container Runtime on a Node from Docker Engine to containerd"
|
||||
weight: 8
|
||||
content_type: task
|
||||
---
|
||||
|
||||
This task outlines the steps needed to update your container runtime to containerd from Docker. It is applicable for cluster operators running Kubernetes 1.23 or earlier. Also this covers an example scenario for migrating from dockershim to containerd and alternative container runtimes can be picked from this [page](https://kubernetes.io/docs/setup/production-environment/container-runtimes/).
|
||||
|
||||
## {{% heading "prerequisites" %}}
|
||||
|
||||
{{% thirdparty-content %}}
|
||||
|
||||
Install containerd. For more information see, [containerd's installation documentation](https://containerd.io/docs/getting-started/) and for specific prerequisite follow [this](/docs/setup/production-environment/container-runtimes/#containerd).
|
||||
|
||||
## Drain the node
|
||||
|
||||
```
|
||||
# replace <node-to-drain> with the name of your node you are draining
|
||||
kubectl drain <node-to-drain> --ignore-daemonsets
|
||||
```
|
||||
## Stop the Docker daemon
|
||||
|
||||
```shell
|
||||
systemctl stop kubelet
|
||||
systemctl disable docker.service --now
|
||||
```
|
||||
|
||||
## Install Containerd
|
||||
|
||||
This [page](/docs/setup/production-environment/container-runtimes/#containerd) contains detailed steps to install containerd.
|
||||
|
||||
{{< tabs name="tab-cri-containerd-installation" >}}
|
||||
{{% tab name="Linux" %}}
|
||||
|
||||
1. Install the `containerd.io` package from the official Docker repositories.
|
||||
Instructions for setting up the Docker repository for your respective Linux distribution and installing the `containerd.io` package can be found at
|
||||
[Install Docker Engine](https://docs.docker.com/engine/install/#server).
|
||||
|
||||
2. Configure containerd:
|
||||
|
||||
```shell
|
||||
sudo mkdir -p /etc/containerd
|
||||
containerd config default | sudo tee /etc/containerd/config.toml
|
||||
```
|
||||
|
||||
3. Restart containerd:
|
||||
|
||||
```shell
|
||||
sudo systemctl restart containerd
|
||||
```
|
||||
|
||||
{{% /tab %}}
|
||||
{{% tab name="Windows (PowerShell)" %}}
|
||||
|
||||
Start a Powershell session, set `$Version` to the desired version (ex: `$Version="1.4.3"`), and then run the following commands:
|
||||
|
||||
1. Download containerd:
|
||||
|
||||
```powershell
|
||||
curl.exe -L https://github.com/containerd/containerd/releases/download/v$Version/containerd-$Version-windows-amd64.tar.gz -o containerd-windows-amd64.tar.gz
|
||||
tar.exe xvf .\containerd-windows-amd64.tar.gz
|
||||
```
|
||||
|
||||
2. Extract and configure:
|
||||
|
||||
```powershell
|
||||
Copy-Item -Path ".\bin\" -Destination "$Env:ProgramFiles\containerd" -Recurse -Force
|
||||
cd $Env:ProgramFiles\containerd\
|
||||
.\containerd.exe config default | Out-File config.toml -Encoding ascii
|
||||
|
||||
# Review the configuration. Depending on setup you may want to adjust:
|
||||
# - the sandbox_image (Kubernetes pause image)
|
||||
# - cni bin_dir and conf_dir locations
|
||||
Get-Content config.toml
|
||||
|
||||
# (Optional - but highly recommended) Exclude containerd from Windows Defender Scans
|
||||
Add-MpPreference -ExclusionProcess "$Env:ProgramFiles\containerd\containerd.exe"
|
||||
```
|
||||
|
||||
3. Start containerd:
|
||||
|
||||
```powershell
|
||||
.\containerd.exe --register-service
|
||||
Start-Service containerd
|
||||
```
|
||||
|
||||
{{% /tab %}}
|
||||
{{< /tabs >}}
|
||||
|
||||
## Configure the kubelet to use containerd as its container runtime
|
||||
|
||||
Edit the file `/var/lib/kubelet/kubeadm-flags.env` and add the containerd runtime to the flags. `--container-runtime=remote` and `--container-runtime-endpoint=unix:///run/containerd/containerd.sock"`
|
||||
|
||||
For users using kubeadm should consider the following:
|
||||
|
||||
The `kubeadm` tool stores the CRI socket for each host as an annotation in the Node object for that host.
|
||||
|
||||
To change it you must do the following:
|
||||
|
||||
Execute `kubectl edit no <NODE-NAME>` on a machine that has the kubeadm `/etc/kubernetes/admin.conf` file.
|
||||
|
||||
This will start a text editor where you can edit the Node object.
|
||||
|
||||
To choose a text editor you can set the `KUBE_EDITOR` environment variable.
|
||||
|
||||
- Change the value of `kubeadm.alpha.kubernetes.io/cri-socket` from `/var/run/dockershim.sock`
|
||||
to the CRI socket path of your choice (for example `unix:///run/containerd/containerd.sock`).
|
||||
|
||||
Note that new CRI socket paths must be prefixed with `unix://` ideally.
|
||||
|
||||
- Save the changes in the text editor, which will update the Node object.
|
||||
|
||||
## Restart the kubelet
|
||||
|
||||
```shell
|
||||
systemctl start kubelet
|
||||
```
|
||||
|
||||
## Verify that the node is healthy
|
||||
|
||||
Run `kubectl get nodes -o wide` and containerd appears as the runtime for the node we just changed.
|
||||
|
||||
## Remove Docker Engine
|
||||
|
||||
{{% thirdparty-content %}}
|
||||
|
||||
Finally if everything goes well remove docker
|
||||
|
||||
{{< tabs name="tab-remove-docker-enigine" >}}
|
||||
{{% tab name="CentOS" %}}
|
||||
|
||||
```shell
|
||||
sudo yum remove docker-ce docker-ce-cli
|
||||
```
|
||||
{{% /tab %}}
|
||||
{{% tab name="Debian" %}}
|
||||
|
||||
```shell
|
||||
sudo apt-get purge docker-ce docker-ce-cli
|
||||
```
|
||||
{{% /tab %}}
|
||||
{{% tab name="Fedora" %}}
|
||||
|
||||
```shell
|
||||
sudo dnf remove docker-ce docker-ce-cli
|
||||
```
|
||||
{{% /tab %}}
|
||||
{{% tab name="Ubuntu" %}}
|
||||
|
||||
```shell
|
||||
sudo apt-get purge docker-ce docker-ce-cli
|
||||
```
|
||||
{{% /tab %}}
|
||||
{{< /tabs >}}
|
||||
@@ -5,7 +5,8 @@ content_type: task
|
||||
|
||||
<!--overview-->
|
||||
|
||||
This page shows you how to specify the type of [cascading deletion](/docs/concepts/workloads/controllers/garbage-collection/#cascading-deletion)
|
||||
This page shows you how to specify the type of
|
||||
[cascading deletion](/docs/concepts/architecture/garbage-collection/#cascading-deletion)
|
||||
to use in your cluster during {{<glossary_tooltip text="garbage collection" term_id="garbage-collection">}}.
|
||||
|
||||
## {{% heading "prerequisites" %}}
|
||||
@@ -26,7 +27,7 @@ kubectl get pods -l app=nginx --output=yaml
|
||||
|
||||
The output has an `ownerReferences` field similar to this:
|
||||
|
||||
```
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
...
|
||||
ownerReferences:
|
||||
@@ -41,7 +42,7 @@ apiVersion: v1
|
||||
|
||||
## Use foreground cascading deletion {#use-foreground-cascading-deletion}
|
||||
|
||||
By default, Kubernetes uses [background cascading deletion](/docs/concepts/workloads/controllers/garbage-collection/#background-deletion)
|
||||
By default, Kubernetes uses [background cascading deletion](/docs/concepts/architecture/garbage-collection/#background-deletion)
|
||||
to delete dependents of an object. You can switch to foreground cascading deletion
|
||||
using either `kubectl` or the Kubernetes API, depending on the Kubernetes
|
||||
version your cluster runs. {{<version-check>}}
|
||||
@@ -64,9 +65,9 @@ kubectl delete deployment nginx-deployment --cascade=foreground
|
||||
|
||||
1. Start a local proxy session:
|
||||
|
||||
```shell
|
||||
kubectl proxy --port=8080
|
||||
```
|
||||
```shell
|
||||
kubectl proxy --port=8080
|
||||
```
|
||||
|
||||
1. Use `curl` to trigger deletion:
|
||||
|
||||
@@ -80,19 +81,19 @@ kubectl delete deployment nginx-deployment --cascade=foreground
|
||||
like this:
|
||||
|
||||
```
|
||||
"kind": "Deployment",
|
||||
"apiVersion": "apps/v1",
|
||||
"metadata": {
|
||||
"name": "nginx-deployment",
|
||||
"namespace": "default",
|
||||
"uid": "d1ce1b02-cae8-4288-8a53-30e84d8fa505",
|
||||
"resourceVersion": "1363097",
|
||||
"creationTimestamp": "2021-07-08T20:24:37Z",
|
||||
"deletionTimestamp": "2021-07-08T20:27:39Z",
|
||||
"finalizers": [
|
||||
"foregroundDeletion"
|
||||
]
|
||||
...
|
||||
"kind": "Deployment",
|
||||
"apiVersion": "apps/v1",
|
||||
"metadata": {
|
||||
"name": "nginx-deployment",
|
||||
"namespace": "default",
|
||||
"uid": "d1ce1b02-cae8-4288-8a53-30e84d8fa505",
|
||||
"resourceVersion": "1363097",
|
||||
"creationTimestamp": "2021-07-08T20:24:37Z",
|
||||
"deletionTimestamp": "2021-07-08T20:27:39Z",
|
||||
"finalizers": [
|
||||
"foregroundDeletion"
|
||||
]
|
||||
...
|
||||
```
|
||||
|
||||
{{% /tab %}}
|
||||
@@ -104,9 +105,9 @@ For details, read the [documentation for your Kubernetes version](/docs/home/sup
|
||||
|
||||
1. Start a local proxy session:
|
||||
|
||||
```shell
|
||||
kubectl proxy --port=8080
|
||||
```
|
||||
```shell
|
||||
kubectl proxy --port=8080
|
||||
```
|
||||
|
||||
1. Use `curl` to trigger deletion:
|
||||
|
||||
@@ -120,19 +121,19 @@ For details, read the [documentation for your Kubernetes version](/docs/home/sup
|
||||
like this:
|
||||
|
||||
```
|
||||
"kind": "Deployment",
|
||||
"apiVersion": "apps/v1",
|
||||
"metadata": {
|
||||
"name": "nginx-deployment",
|
||||
"namespace": "default",
|
||||
"uid": "d1ce1b02-cae8-4288-8a53-30e84d8fa505",
|
||||
"resourceVersion": "1363097",
|
||||
"creationTimestamp": "2021-07-08T20:24:37Z",
|
||||
"deletionTimestamp": "2021-07-08T20:27:39Z",
|
||||
"finalizers": [
|
||||
"foregroundDeletion"
|
||||
]
|
||||
...
|
||||
"kind": "Deployment",
|
||||
"apiVersion": "apps/v1",
|
||||
"metadata": {
|
||||
"name": "nginx-deployment",
|
||||
"namespace": "default",
|
||||
"uid": "d1ce1b02-cae8-4288-8a53-30e84d8fa505",
|
||||
"resourceVersion": "1363097",
|
||||
"creationTimestamp": "2021-07-08T20:24:37Z",
|
||||
"deletionTimestamp": "2021-07-08T20:27:39Z",
|
||||
"finalizers": [
|
||||
"foregroundDeletion"
|
||||
]
|
||||
...
|
||||
```
|
||||
{{% /tab %}}
|
||||
{{</tabs>}}
|
||||
@@ -165,32 +166,32 @@ kubectl delete deployment nginx-deployment --cascade=background
|
||||
|
||||
1. Start a local proxy session:
|
||||
|
||||
```shell
|
||||
kubectl proxy --port=8080
|
||||
```
|
||||
```shell
|
||||
kubectl proxy --port=8080
|
||||
```
|
||||
|
||||
1. Use `curl` to trigger deletion:
|
||||
|
||||
```shell
|
||||
curl -X DELETE localhost:8080/apis/apps/v1/namespaces/default/deployments/nginx-deployment \
|
||||
-d '{"kind":"DeleteOptions","apiVersion":"v1","propagationPolicy":"Background"}' \
|
||||
-H "Content-Type: application/json"
|
||||
```
|
||||
```shell
|
||||
curl -X DELETE localhost:8080/apis/apps/v1/namespaces/default/deployments/nginx-deployment \
|
||||
-d '{"kind":"DeleteOptions","apiVersion":"v1","propagationPolicy":"Background"}' \
|
||||
-H "Content-Type: application/json"
|
||||
```
|
||||
|
||||
The output is similar to this:
|
||||
The output is similar to this:
|
||||
|
||||
```
|
||||
"kind": "Status",
|
||||
"apiVersion": "v1",
|
||||
...
|
||||
"status": "Success",
|
||||
"details": {
|
||||
"name": "nginx-deployment",
|
||||
"group": "apps",
|
||||
"kind": "deployments",
|
||||
"uid": "cc9eefb9-2d49-4445-b1c1-d261c9396456"
|
||||
}
|
||||
```
|
||||
```
|
||||
"kind": "Status",
|
||||
"apiVersion": "v1",
|
||||
...
|
||||
"status": "Success",
|
||||
"details": {
|
||||
"name": "nginx-deployment",
|
||||
"group": "apps",
|
||||
"kind": "deployments",
|
||||
"uid": "cc9eefb9-2d49-4445-b1c1-d261c9396456"
|
||||
}
|
||||
```
|
||||
{{% /tab %}}
|
||||
{{% tab name="Versions prior to Kubernetes 1.20.x" %}}
|
||||
Kubernetes uses background cascading deletion by default, and does so
|
||||
@@ -211,32 +212,32 @@ kubectl delete deployment nginx-deployment --cascade=true
|
||||
|
||||
1. Start a local proxy session:
|
||||
|
||||
```shell
|
||||
kubectl proxy --port=8080
|
||||
```
|
||||
```shell
|
||||
kubectl proxy --port=8080
|
||||
```
|
||||
|
||||
1. Use `curl` to trigger deletion:
|
||||
|
||||
```shell
|
||||
curl -X DELETE localhost:8080/apis/apps/v1/namespaces/default/deployments/nginx-deployment \
|
||||
-d '{"kind":"DeleteOptions","apiVersion":"v1","propagationPolicy":"Background"}' \
|
||||
-H "Content-Type: application/json"
|
||||
```
|
||||
```shell
|
||||
curl -X DELETE localhost:8080/apis/apps/v1/namespaces/default/deployments/nginx-deployment \
|
||||
-d '{"kind":"DeleteOptions","apiVersion":"v1","propagationPolicy":"Background"}' \
|
||||
-H "Content-Type: application/json"
|
||||
```
|
||||
|
||||
The output is similar to this:
|
||||
The output is similar to this:
|
||||
|
||||
```
|
||||
"kind": "Status",
|
||||
"apiVersion": "v1",
|
||||
...
|
||||
"status": "Success",
|
||||
"details": {
|
||||
"name": "nginx-deployment",
|
||||
"group": "apps",
|
||||
"kind": "deployments",
|
||||
"uid": "cc9eefb9-2d49-4445-b1c1-d261c9396456"
|
||||
}
|
||||
```
|
||||
```
|
||||
"kind": "Status",
|
||||
"apiVersion": "v1",
|
||||
...
|
||||
"status": "Success",
|
||||
"details": {
|
||||
"name": "nginx-deployment",
|
||||
"group": "apps",
|
||||
"kind": "deployments",
|
||||
"uid": "cc9eefb9-2d49-4445-b1c1-d261c9396456"
|
||||
}
|
||||
```
|
||||
{{% /tab %}}
|
||||
{{</tabs>}}
|
||||
|
||||
@@ -264,33 +265,33 @@ kubectl delete deployment nginx-deployment --cascade=orphan
|
||||
|
||||
1. Start a local proxy session:
|
||||
|
||||
```shell
|
||||
kubectl proxy --port=8080
|
||||
```
|
||||
```shell
|
||||
kubectl proxy --port=8080
|
||||
```
|
||||
|
||||
1. Use `curl` to trigger deletion:
|
||||
|
||||
```shell
|
||||
curl -X DELETE localhost:8080/apis/apps/v1/namespaces/default/deployments/nginx-deployment \
|
||||
-d '{"kind":"DeleteOptions","apiVersion":"v1","propagationPolicy":"Orphan"}' \
|
||||
-H "Content-Type: application/json"
|
||||
```
|
||||
```shell
|
||||
curl -X DELETE localhost:8080/apis/apps/v1/namespaces/default/deployments/nginx-deployment \
|
||||
-d '{"kind":"DeleteOptions","apiVersion":"v1","propagationPolicy":"Orphan"}' \
|
||||
-H "Content-Type: application/json"
|
||||
```
|
||||
|
||||
The output contains `orphan` in the `finalizers` field, similar to this:
|
||||
The output contains `orphan` in the `finalizers` field, similar to this:
|
||||
|
||||
```
|
||||
"kind": "Deployment",
|
||||
"apiVersion": "apps/v1",
|
||||
"namespace": "default",
|
||||
"uid": "6f577034-42a0-479d-be21-78018c466f1f",
|
||||
"creationTimestamp": "2021-07-09T16:46:37Z",
|
||||
"deletionTimestamp": "2021-07-09T16:47:08Z",
|
||||
"deletionGracePeriodSeconds": 0,
|
||||
"finalizers": [
|
||||
"orphan"
|
||||
],
|
||||
...
|
||||
```
|
||||
```
|
||||
"kind": "Deployment",
|
||||
"apiVersion": "apps/v1",
|
||||
"namespace": "default",
|
||||
"uid": "6f577034-42a0-479d-be21-78018c466f1f",
|
||||
"creationTimestamp": "2021-07-09T16:46:37Z",
|
||||
"deletionTimestamp": "2021-07-09T16:47:08Z",
|
||||
"deletionGracePeriodSeconds": 0,
|
||||
"finalizers": [
|
||||
"orphan"
|
||||
],
|
||||
...
|
||||
```
|
||||
|
||||
{{% /tab %}}
|
||||
{{% tab name="Versions prior to Kubernetes 1.20.x" %}}
|
||||
@@ -309,33 +310,33 @@ kubectl delete deployment nginx-deployment --cascade=orphan
|
||||
|
||||
1. Start a local proxy session:
|
||||
|
||||
```shell
|
||||
kubectl proxy --port=8080
|
||||
```
|
||||
```shell
|
||||
kubectl proxy --port=8080
|
||||
```
|
||||
|
||||
1. Use `curl` to trigger deletion:
|
||||
|
||||
```shell
|
||||
curl -X DELETE localhost:8080/apis/apps/v1/namespaces/default/deployments/nginx-deployment \
|
||||
-d '{"kind":"DeleteOptions","apiVersion":"v1","propagationPolicy":"Orphan"}' \
|
||||
-H "Content-Type: application/json"
|
||||
```
|
||||
```shell
|
||||
curl -X DELETE localhost:8080/apis/apps/v1/namespaces/default/deployments/nginx-deployment \
|
||||
-d '{"kind":"DeleteOptions","apiVersion":"v1","propagationPolicy":"Orphan"}' \
|
||||
-H "Content-Type: application/json"
|
||||
```
|
||||
|
||||
The output contains `orphan` in the `finalizers` field, similar to this:
|
||||
The output contains `orphan` in the `finalizers` field, similar to this:
|
||||
|
||||
```
|
||||
"kind": "Deployment",
|
||||
"apiVersion": "apps/v1",
|
||||
"namespace": "default",
|
||||
"uid": "6f577034-42a0-479d-be21-78018c466f1f",
|
||||
"creationTimestamp": "2021-07-09T16:46:37Z",
|
||||
"deletionTimestamp": "2021-07-09T16:47:08Z",
|
||||
"deletionGracePeriodSeconds": 0,
|
||||
"finalizers": [
|
||||
"orphan"
|
||||
],
|
||||
...
|
||||
```
|
||||
```
|
||||
"kind": "Deployment",
|
||||
"apiVersion": "apps/v1",
|
||||
"namespace": "default",
|
||||
"uid": "6f577034-42a0-479d-be21-78018c466f1f",
|
||||
"creationTimestamp": "2021-07-09T16:46:37Z",
|
||||
"deletionTimestamp": "2021-07-09T16:47:08Z",
|
||||
"deletionGracePeriodSeconds": 0,
|
||||
"finalizers": [
|
||||
"orphan"
|
||||
],
|
||||
...
|
||||
```
|
||||
{{% /tab %}}
|
||||
{{</tabs>}}
|
||||
|
||||
@@ -349,4 +350,4 @@ kubectl get pods -l app=nginx
|
||||
|
||||
* Learn about [owners and dependents](/docs/concepts/overview/working-with-objects/owners-dependents/) in Kubernetes.
|
||||
* Learn about Kubernetes [finalizers](/docs/concepts/overview/working-with-objects/finalizers/).
|
||||
* Learn about [garbage collection](/docs/concepts/workloads/controllers/garbage-collection/).
|
||||
* Learn about [garbage collection](/docs/concepts/architecture/garbage-collection/).
|
||||
|
||||
Reference in New Issue
Block a user