Merge 1.10 to master for release (#7861)
* 1.10 update (#7151) * Fix partition value expected behaviour explanation (#7123) Fixes issue #7057 * Correct "On-Premise" to "On-Premises" * Updates the Calico installation page (#7094) * All files for Haufe Groups case study (#7051) * Fix typo (#7127) * fix typo of device-plugins.md (#7106) * fix broken links (#7136) * Updated configure-service-account (#7147) Error from server resolved by escaping kubectl patch serviceaccount default -p '{"imagePullSecrets": [{"name": "myregistrykey"}]}' JSON string by '\' * Remove docs related to 'require-kubeconfig' (#7138) With kubernetes/kubernetes#58367 merged, v1.10 will not use the "require-kubeconfig" flag. The flag has become a no-op solely to ensure existing deployments won't break. * Added Verification Scenario for a Pod that Uses a PVC in Terminating State (#7164) The below PR: https://github.com/kubernetes/kubernetes/pull/55873 modified scheduler in such a way that scheduling of a pod that uses a PVC in Terminating state fails. That's why verification of such scenario was added to documentation. * fix LimitPodHardAntiAffinityTopology name (#7221) * Document the removal of the KubeletConfigFile feature gate (#7140) With kubernetes/kubernetes#58978 merged, the said feature gate is removed. This PR removes texts related to the gate and revises the Feature Gates reference to reflect this change. * deprecate three admission controller (#7363) * Document the removal of Accelerators feature gate (#7389) The `Accelerators` feature gate will be removed in 1.11. 1.10 will be its last mile. References: kubernetes/kubernetes#57384 * Update local storage docs for beta (#7473) * Document that HugePages feature gate is Beta (#7387) The `HugePages` feature gate has graduated to Beta in v1.10. This PR documents this fact. * Add HyperVContainer feature gates (#7502) * Remove the beta reference from Taints and Tolerations doc (#7493) * Kms provider doc (#7479) * Kms provider doc * issue# 7399, Create KMS-provider.md and update encrypt-data.md * address review comments * Document that Device Plugin feature is Beta (1.10) (#7512) * Add docs for CRD features for 1.10 (#7439) * Add docs for CRD features for 1.10 * Add CustomResourcesSubresources to list of feature gates * Add latest changes to custom resources doc * Add crds as abbreviated alias (#7437) * Bring PVC Protection Feature to Beta (#7165) * Bring PVC Protection Feature to Beta The PR: https://github.com/kubernetes/kubernetes/pull/59052 brought PVC Protection feature to beta. That's why the documentation is updated accordingly. * The PVC Protection feature was renamed to Storage Protection. That's why the documentation is updated. * promote PodNodeSelector to stable; document detailed behavior (#7134) * promote PodNodeSelector to stable; document detailed behavior * respond to feedback * Update CPU manager feature enabling (#7390) With `CPUManager` feature graduating to beta. No explicit enabling is required starting v1.10. References: kubernetes/kubernetes#55977 * Adding block volumeMode documentation for local volumes. (#7531) Code review comments. Changed property to field. Address tech review comment. * remove description kubectl --show-all (#7574) --show-all has been deprecated and set to true by default. https://github.com/kubernetes/kubernetes/pull/60210 * fix description about contribute style guide (#7592) * fix description about KUBECONFIG (#7589) s/envrionment/environment * fix description about cni (#7588) s/simultanously/simultaneously/ * fix description about MutatingAdmissionWebhook and ValidatingAdmissionWebhook (#7587) * fix description about persistent volume binding (#7590) s/slighty/slightly/ * Doc change for configurable pod resolv.conf Beta (#7611) * fix description about out of resource handling (#7597) s/threshhold/threshold * fix description about zookeeper (#7598) s/achive/achieve * fix description about kubeadm (#7594) s/compatability/compatibility/ * fix description about kubeadm (#7593) * fix description about kubeadm implementation details (#7595) * fix description about api concepts (#7596) * Storage Protection was renamed to Storage Object in Use Protection (#7576) * Storage Protection was renamed to Storage Object in Use Protection The K8s PR: https://github.com/kubernetes/kubernetes/pull/59901 renamed Storage Protection to Storage Object in Use Protection. That's why the same is also renamed in the documentation. * Moved Storage Object in Use Protection admission plugin description down according to alphabetic order. * Use PSP from policy API group. (#7562) * update kubeletconfig docs for v1.10, beta (#7561) * Update port-forwarding docs (#7575) * add pv protection description (#7620) * fix description about client library (#7634) * Add docs on configuring NodePort IP (#7631) * Document that LocalStorageCapacityIsolation is beta (#7635) A follow-up to the kubernetes/kubernetes#60159 change which has promoted the `LocalStorageCapacityIsolation` feature gate to Beta. * Update CoreDNS docs for beta (#7638) * Update CoreDNS docs for beta * Review comments * Fix typo (#7640) * Update feature gates move to beta (#7662) * Added the inability to use colon ':' character as environment variable names and described workaround (#7657) * merge master to 1.10, with fixes (#7682) * Flag names changed (s/admission-control/enable-admission-plugins); disable-admissions-plugin entry added; removed reference to admission controller/plugins requiring set order (for v1.10), redundant example enabling specific plugin, and redundant version-specific info (#7449) * Documentation for MountPropagation beta (#7655) * Remove job's scale-related operations (#7684) * authentication: document client-go exec plugins (#7648) * authentication: document client-go exec plugins * Update authentication.md * Update local ephemeral storage feature to beta (#7685) Update local ephemeral storage feature to beta * Update docs for windows container resources (#7653) * add server-side print docs (#7671) * Create a task describing Pod process namespace sharing (#7489) * Add external metrics to HPA docs (#7664) * Add external metrics to HPA docs * Update horizontal-pod-autoscale-walkthrough.md * Apply review comments to HPA walkthrough * remove description about "scale jobs" (#7712) * CSI Docs for K8s v1.10 (#7698) * Add a warning about increased memory consumption for audit logging feature. (#7725) Signed-off-by: Mik Vyatskov <vmik@google.com> * Update Audit Logging documentation for 1.10 (#7679) Signed-off-by: Mik Vyatskov <vmik@google.com> * Fix stage names in audit logging documentation (#7746) Signed-off-by: Mik Vyatskov <vmik@google.com> * Feature gate update for release 1.10 (#7742) * State in the docs that the value of default Node labels are not reliable. (#7794) * Kill the reference to --admission-control option (#7755) The `--admission-control` option has been replaced by two new options in v1.10. This PR kills the last appearance of the old option in the doc. * Pvcprotection toc (#7807) * Refreshing installation instructions (#7495) * Refreshing installation instructions Added conjure-up. Updated displays and juju versions to current versions. * Updated anchors * Fixed image value version typo (#7768) Was inconsistent with other values * Update flocker reference to the github repo (#7784) * Fix typo in federation document (#7779) * an user -> a user (#7778) * Events are namespaced (#7767) * fix 'monitoring' link lose efficacy problem' (#7764) * docs/concepts/policy/pod-security-policy.md: minor fix. (#7659) * Update downward-api-volume-expose-pod-information.md (#7771) * Update downward-api-volume-expose-pod-information.md The pod spec puts the downward api files into /etc/podinfo, not directly in /etc. Updated docs to reflect this fact. * Update downward-api-volume-expose-pod-information.md One more spot needed fixing. * Update downward-api-volume-expose-pod-information.md Yet another fix, in the container example. * Add Amadeus Case Study (#7783) * Add Amadeus Case Study * add Amadeus logo * Fixed Cyrillic с in 'kube-proxy-cm' (#7787) There was a typo (wrong character) in kube-proxy-cm.yaml - Cyrillic с (UTF-8 0x0441) was used instead of Latin c. * install-kubectl: choose one installation method (#7705) The previous text layout suggested that all installations had to be done, one after another. * Update install-kubeadm.md (#7781) Add note to kubeadm install instruction to help install in other arch i.e. aarch64, ppc64le etc. * repair failure link (#7788) * repair failure link * repair failure link * do change as required * Update k8s201.md (#7777) * Update k8s201.md Change instructions to download yams files directly from the website (as used in other pages.) Added instructions to delete labeled pod to avoid warnings in the subsequent deployment step. * Update k8s201.md Added example of using the exposed host from the a node running Kubernetes. (This works on AWS with Weave; not able to test it on other variations...) * Gramatical fix to kompose introduction (#7792) The original wording didn't through very well. As much of the original sentence has been preserved as possible, primarily to ensure the kompose web address is see both in text and as a href link. * update amadeus.html (#7800) * Fix a missing word in endpoint reconciler section (#7804) * add toc entry for pvcprotection downgrade issue doc * Pvcprotection toc (#7809) * Refreshing installation instructions (#7495) * Refreshing installation instructions Added conjure-up. Updated displays and juju versions to current versions. * Updated anchors * Fixed image value version typo (#7768) Was inconsistent with other values * Update flocker reference to the github repo (#7784) * Fix typo in federation document (#7779) * an user -> a user (#7778) * Events are namespaced (#7767) * fix 'monitoring' link lose efficacy problem' (#7764) * docs/concepts/policy/pod-security-policy.md: minor fix. (#7659) * Update downward-api-volume-expose-pod-information.md (#7771) * Update downward-api-volume-expose-pod-information.md The pod spec puts the downward api files into /etc/podinfo, not directly in /etc. Updated docs to reflect this fact. * Update downward-api-volume-expose-pod-information.md One more spot needed fixing. * Update downward-api-volume-expose-pod-information.md Yet another fix, in the container example. * Add Amadeus Case Study (#7783) * Add Amadeus Case Study * add Amadeus logo * Fixed Cyrillic с in 'kube-proxy-cm' (#7787) There was a typo (wrong character) in kube-proxy-cm.yaml - Cyrillic с (UTF-8 0x0441) was used instead of Latin c. * install-kubectl: choose one installation method (#7705) The previous text layout suggested that all installations had to be done, one after another. * Update install-kubeadm.md (#7781) Add note to kubeadm install instruction to help install in other arch i.e. aarch64, ppc64le etc. * repair failure link (#7788) * repair failure link * repair failure link * do change as required * Update k8s201.md (#7777) * Update k8s201.md Change instructions to download yams files directly from the website (as used in other pages.) Added instructions to delete labeled pod to avoid warnings in the subsequent deployment step. * Update k8s201.md Added example of using the exposed host from the a node running Kubernetes. (This works on AWS with Weave; not able to test it on other variations...) * Gramatical fix to kompose introduction (#7792) The original wording didn't through very well. As much of the original sentence has been preserved as possible, primarily to ensure the kompose web address is see both in text and as a href link. * update amadeus.html (#7800) * Fix a missing word in endpoint reconciler section (#7804) * add toc entry for pvcprotection downgrade issue doc * revert TOC change * Release 1.10 (#7818) * Refreshing installation instructions (#7495) * Refreshing installation instructions Added conjure-up. Updated displays and juju versions to current versions. * Updated anchors * Fixed image value version typo (#7768) Was inconsistent with other values * Update flocker reference to the github repo (#7784) * Fix typo in federation document (#7779) * an user -> a user (#7778) * Events are namespaced (#7767) * fix 'monitoring' link lose efficacy problem' (#7764) * docs/concepts/policy/pod-security-policy.md: minor fix. (#7659) * Update downward-api-volume-expose-pod-information.md (#7771) * Update downward-api-volume-expose-pod-information.md The pod spec puts the downward api files into /etc/podinfo, not directly in /etc. Updated docs to reflect this fact. * Update downward-api-volume-expose-pod-information.md One more spot needed fixing. * Update downward-api-volume-expose-pod-information.md Yet another fix, in the container example. * Add Amadeus Case Study (#7783) * Add Amadeus Case Study * add Amadeus logo * Fixed Cyrillic с in 'kube-proxy-cm' (#7787) There was a typo (wrong character) in kube-proxy-cm.yaml - Cyrillic с (UTF-8 0x0441) was used instead of Latin c. * install-kubectl: choose one installation method (#7705) The previous text layout suggested that all installations had to be done, one after another. * Update install-kubeadm.md (#7781) Add note to kubeadm install instruction to help install in other arch i.e. aarch64, ppc64le etc. * repair failure link (#7788) * repair failure link * repair failure link * do change as required * Update k8s201.md (#7777) * Update k8s201.md Change instructions to download yams files directly from the website (as used in other pages.) Added instructions to delete labeled pod to avoid warnings in the subsequent deployment step. * Update k8s201.md Added example of using the exposed host from the a node running Kubernetes. (This works on AWS with Weave; not able to test it on other variations...) * Gramatical fix to kompose introduction (#7792) The original wording didn't through very well. As much of the original sentence has been preserved as possible, primarily to ensure the kompose web address is see both in text and as a href link. * update amadeus.html (#7800) * Fix a missing word in endpoint reconciler section (#7804) * Partners page updates (#7802) * Partners page updates * Update to ZTE link * Make using sysctls a task instead of a concept (#6808) Closes: #4505 * add a note when mount a configmap to pod (#7745) * adjust a note format (#7812) * Update docker-cli-to-kubectl.md (#7748) * Update docker-cli-to-kubectl.md Edited the document for adherence to the style guide and word usage. * Update docker-cli-to-kubectl.md * Incorporated the changes suggested. * Mount propagation update to include docker config (#7854) * update overridden config for 1.10 (#7847) * update overridden config for 1.10 * fix config file per comments * Update Extended Resource doc wrt cluster-level resources (#7759)
This commit is contained in:
Jennifer Rondeau
committed by
Steve Perry
Steve Perry
parent
770440d20f
commit
409e77d3e5
@@ -5,7 +5,7 @@ title: Using CoreDNS for Service Discovery
|
||||
min-kubernetes-server-version: v1.9
|
||||
---
|
||||
|
||||
{% include feature-state-alpha.md %}
|
||||
{% include feature-state-beta.md %}
|
||||
|
||||
{% capture overview %}
|
||||
This page describes how to enable CoreDNS instead of kube-dns for service
|
||||
@@ -20,8 +20,9 @@ discovery.
|
||||
|
||||
## Installing CoreDNS with kubeadm
|
||||
|
||||
In Kubernetes 1.9, [CoreDNS](https://coredns.io) is available as an alpha feature and
|
||||
may be installed by setting the `CoreDNS` feature gate to `true` during `kubeadm init`:
|
||||
In Kubernetes 1.9, [CoreDNS](https://coredns.io) is available as an alpha feature, and
|
||||
in Kubernetes 1.10 it is available as a beta feature. In either case, you may install
|
||||
it during cluster creation by setting the `CoreDNS` feature gate to `true` during `kubeadm init`:
|
||||
|
||||
```
|
||||
kubeadm init --feature-gates=CoreDNS=true
|
||||
@@ -29,6 +30,21 @@ kubeadm init --feature-gates=CoreDNS=true
|
||||
|
||||
This installs CoreDNS instead of kube-dns.
|
||||
|
||||
## Upgrading an Existing Cluster with kubeadm
|
||||
|
||||
In Kubernetes 1.10, you can also move to CoreDNS when you use `kubeadm` to upgrade
|
||||
a cluster that is using `kube-dns`. In this case, `kubeadm` will generate the CoreDNS configuration
|
||||
("Corefile") based upon the `kube-dns` ConfigMap, preserving configurations for federation,
|
||||
stub domains, and upstream name server.
|
||||
|
||||
Note that if you are running CoreDNS in your cluster already, prior to upgrade, your existing Corefile will be
|
||||
**overwritten** by the one created during upgrade. **You should save your existing ConfigMap
|
||||
if you have customized it.** You may re-apply your customizations after the new ConfigMap is
|
||||
up and running.
|
||||
|
||||
This process will be modified for the GA release of this feature, such that an existing
|
||||
Corefile will not be overwritten.
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
{% capture whatsnext %}
|
||||
|
||||
@@ -33,9 +33,8 @@ management policies to determine some placement preferences on the node.
|
||||
|
||||
### Configuration
|
||||
|
||||
The CPU Manager is introduced as an alpha feature in Kubernetes v1.8. It
|
||||
must be explicitly enabled in the kubelet feature gates:
|
||||
`--feature-gates=CPUManager=true`.
|
||||
The CPU Manager is an alpha feature in Kubernetes v1.8. It was enabled by
|
||||
default as a beta feature since v1.10.
|
||||
|
||||
The CPU Manager policy is set with the `--cpu-manager-policy` kubelet
|
||||
option. There are two supported policies:
|
||||
|
||||
@@ -78,7 +78,8 @@ Name | Encryption | Strength | Speed | Key Length | Other Considerations
|
||||
`identity` | None | N/A | N/A | N/A | Resources written as-is without encryption. When set as the first provider, the resource will be decrypted as new values are written.
|
||||
`aescbc` | AES-CBC with PKCS#7 padding | Strongest | Fast | 32-byte | The recommended choice for encryption at rest but may be slightly slower than `secretbox`.
|
||||
`secretbox` | XSalsa20 and Poly1305 | Strong | Faster | 32-byte | A newer standard and may not be considered acceptable in environments that require high levels of review.
|
||||
`aesgcm` | AES-GCM with random nonce | Must be rotated every 200k writes | Fastest | 16, 24, or 32-byte | Is not recommended for use except when an automated key rotation scheme is implemented.
|
||||
`aesgcm` | AES-GCM with random nonce | Must be rotated every 200k writes | Fastest | 16, 24, or 32-byte | Is not recommended for use except when an automated key rotation scheme is implemented.
|
||||
`kms` | Uses envelope encryption scheme: Data is encrypted by data encryption keys (DEKs) using AES-CBC with PKCS#7 padding, DEKs are encrypted by key encryption keys (KEKs) according to configuration in Key Management Service (KMS) | Strongest | Fast | 32-bytes | The recommended choice for using a third party tool for key management. Simplifies key rotation, with a new DEK generated for each encryption, and KEK rotation controlled by the user. [Configure the KMS provider](/docs/tasks/administer-cluster/kms-provider/)
|
||||
|
||||
Each provider supports multiple keys - the keys are tried in order for decryption, and if the provider
|
||||
is the first provider, the first key is used for encryption.
|
||||
|
||||
@@ -0,0 +1,181 @@
|
||||
---
|
||||
approvers:
|
||||
- smarterclayton
|
||||
title: Using a KMS provider for data encryption
|
||||
---
|
||||
{% capture overview %}
|
||||
This page shows how to configure a Key Management Service (KMS) provider and plugin to enable secret data encryption.
|
||||
{% endcapture %}
|
||||
|
||||
{% capture prerequisites %}
|
||||
|
||||
* {% include task-tutorial-prereqs.md %}
|
||||
|
||||
* Kubernetes version 1.10.0 or later is required
|
||||
|
||||
* etcd v3 or later is required
|
||||
|
||||
{% assign for_k8s_version="v1.10" %}{% include feature-state-alpha.md %}
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
{% capture steps %}
|
||||
|
||||
The KMS encryption provider uses an envelope encryption scheme to encrypt data in etcd. The data is encrypted using a data encryption key (DEK); a new DEK is generated for each encryption. The DEKs are encrypted with a key encryption key (KEK) that is stored and managed in a remote KMS. The KMS provider uses gRPC to communicate with a specific KMS
|
||||
plugin. The KMS plugin, which is implemented as a gRPC server and deployed on the same host(s) as the Kubernetes master(s), is responsible for all communication with the remote KMS.
|
||||
|
||||
## Configuring the KMS provider
|
||||
|
||||
To configure a KMS provider on the API server, include a provider of type ```kms``` in the providers array in the encryption configuration file and set the following properties:
|
||||
|
||||
* `name`: Display name of the KMS plugin.
|
||||
* `endpoint`: Listen address of the gRPC server (KMS plugin). The endpoint is a UNIX domain socket.
|
||||
* `cachesize`: Number of data encryption keys (DEKs) to be cached in the clear. When cached, DEKs can be used without another call to the KMS; whereas DEKs that are not cached require a call to the KMS to unwrap..
|
||||
|
||||
See [Understanding the encryption at rest configuration.](/docs/tasks/administer-cluster/encrypt-data)
|
||||
|
||||
## Implementing a KMS plugin
|
||||
|
||||
To implement a KMS plugin, you can develop a new plugin gRPC server or enable a KMS plugin already provided by your cloud provider. You then integrate the plugin with the remote KMS and deploy it on the Kubernetes master.
|
||||
|
||||
### Enabling the KMS supported by your cloud provider
|
||||
Refer to your cloud provider for instructions on enabling the cloud provider-specific KMS plugin.
|
||||
|
||||
### Developing a KMS plugin gRPC server
|
||||
You can develop a KMS plugin gRPC server using a stub file available for Go. For other languages, you use a proto file to create a stub file that you can use to develop the gRPC server code.
|
||||
|
||||
* Using Go: Use the functions and data structures in the stub file: [service.pb.go](https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/apiserver/pkg/storage/value/encrypt/envelope/v1beta1/service.pb.go) to develop the gRPC server code
|
||||
|
||||
* Using languages other than Go: Use the protoc compiler with the proto file: [service.proto](https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/apiserver/pkg/storage/value/encrypt/envelope/v1beta1/service.proto) to generate a stub file for the specific language
|
||||
|
||||
Then use the functions and data structures in the stub file to develop the server code.
|
||||
|
||||
**Notes:**
|
||||
|
||||
* kms plugin version: `v1beta1`
|
||||
|
||||
In response to procedure call Version, a compatible KMS plugin should return v1beta1 as VersionResponse.version
|
||||
|
||||
* message version: `v1beta1`
|
||||
|
||||
All messages from KMS provider have the version field set to current version v1beta1
|
||||
|
||||
* protocol: UNIX domain socket (`unix`)
|
||||
|
||||
The gRPC server should listen at UNIX domain socket
|
||||
|
||||
### Integrating a KMS plugin with the remote KMS
|
||||
The KMS plugin can communicate with the remote KMS using any protocol supported by the KMS.
|
||||
All configuration data, including authentication credentials the KMS plugin uses to communicate with the remote KMS,
|
||||
are stored and managed by the KMS plugin independently. The KMS plugin can encode the ciphertext with additional metadata that may be required before sending it to the KMS for decryption.
|
||||
|
||||
### Deploying the KMS plugin
|
||||
Ensure that the KMS plugin runs on the same host(s) as the Kubernetes master(s).
|
||||
|
||||
## Encrypting your data with the KMS provider
|
||||
To encrypt the data:
|
||||
|
||||
1. Create a new encryption configuration file using the appropriate properties for the `kms` provider:
|
||||
|
||||
```yaml
|
||||
kind: EncryptionConfig
|
||||
apiVersion: v1
|
||||
resources:
|
||||
- resources:
|
||||
- secrets
|
||||
providers:
|
||||
- kms:
|
||||
name: myKmsPlugin
|
||||
endpoint: unix:///tmp/socketfile.sock
|
||||
cachesize: 100
|
||||
- identity: {}
|
||||
```
|
||||
|
||||
2. Set the `--experimental-encryption-provider-config` flag on the kube-apiserver to point to the location of the configuration file.
|
||||
3. Restart your API server.
|
||||
|
||||
## Verifying that the data is encrypted
|
||||
Data is encrypted when written to etcd. After restarting your kube-apiserver, any newly created or updated secret should be encrypted when stored. To verify, you can use the etcdctl command line program to retrieve the contents of your secret.
|
||||
|
||||
1. Create a new secret called secret1 in the default namespace:
|
||||
```
|
||||
kubectl create secret generic secret1 -n default --from-literal=mykey=mydata
|
||||
```
|
||||
2. Using the etcdctl command line, read that secret out of etcd:
|
||||
```
|
||||
ETCDCTL_API=3 etcdctl get /kubernetes.io/secrets/default/secret1 [...] | hexdump -C
|
||||
```
|
||||
where `[...]` must be the additional arguments for connecting to the etcd server.
|
||||
|
||||
3. Verify the stored secret is prefixed with `k8s:enc:kms:v1:`, which indicates that the `kms` provider has encrypted the resulting data.
|
||||
|
||||
4. Verify that the secret is correctly decrypted when retrieved via the API:
|
||||
```
|
||||
kubectl describe secret secret1 -n default
|
||||
```
|
||||
should match `mykey: mydata`
|
||||
|
||||
## Ensuring all secrets are encrypted
|
||||
Because secrets are encrypted on write, performing an update on a secret encrypts that content.
|
||||
|
||||
The following command reads all secrets and then updates them to apply server side encryption. If an error occurs due to a conflicting write, retry the command. For larger clusters, you may wish to subdivide the secrets by namespace or script an update.
|
||||
```
|
||||
kubectl get secrets --all-namespaces -o json | kubectl replace -f -
|
||||
```
|
||||
|
||||
## Switching from a local encryption provider to the KMS provider
|
||||
To switch from a local encryption provider to the `kms` provider and re-encrypt all of the secrets:
|
||||
|
||||
1. Add the `kms` provider as the first entry in the configuration file as shown in the following example.
|
||||
|
||||
```yaml
|
||||
kind: EncryptionConfig
|
||||
apiVersion: v1
|
||||
resources:
|
||||
- resources:
|
||||
- secrets
|
||||
providers:
|
||||
- kms:
|
||||
name : myKmsPlugin
|
||||
endpoint: unix:///tmp/socketfile.sock
|
||||
cachesize: 100
|
||||
- aescbc:
|
||||
keys:
|
||||
- name: key1
|
||||
secret: <BASE 64 ENCODED SECRET>
|
||||
```
|
||||
|
||||
2. Restart all kube-apiserver processes.
|
||||
|
||||
3. Run the following command to force all secrets to be re-encrypted using the `kms` provider.
|
||||
|
||||
```
|
||||
kubectl get secrets --all-namespaces -o json| kubectl replace -f -
|
||||
```
|
||||
|
||||
## Disabling encryption at rest
|
||||
To disable encryption at rest:
|
||||
|
||||
1. Place the `identity` provider as the first entry in the configuration file:
|
||||
|
||||
```yaml
|
||||
kind: EncryptionConfig
|
||||
apiVersion: v1
|
||||
resources:
|
||||
- resources:
|
||||
- secrets
|
||||
providers:
|
||||
- identity: {}
|
||||
- kms:
|
||||
name : myKmsPlugin
|
||||
endpoint: unix:///tmp/socketfile.sock
|
||||
cachesize: 100
|
||||
```
|
||||
2. Restart all kube-apiserver processes.
|
||||
3. Run the following command to force all secrets to be decrypted.
|
||||
```
|
||||
kubectl get secrets --all-namespaces -o json | kubectl replace -f -
|
||||
```
|
||||
{% endcapture %}
|
||||
|
||||
{% include templates/task.md %}
|
||||
@@ -6,18 +6,20 @@ title: Set Kubelet parameters via a config file
|
||||
---
|
||||
|
||||
{% capture overview %}
|
||||
{% include feature-state-alpha.md %}
|
||||
{% include feature-state-beta.md %}
|
||||
|
||||
As of Kubernetes 1.8, a subset of the Kubelet's configuration parameters may be
|
||||
set via an on-disk config file, as a substitute for command-line flags. In the
|
||||
future, most of the existing command-line flags will be deprecated in favor of
|
||||
providing parameters via a config file, which simplifies node deployment.
|
||||
A subset of the Kubelet's configuration parameters may be
|
||||
set via an on-disk config file, as a substitute for command-line flags.
|
||||
This functionality is considered beta in v1.10.
|
||||
|
||||
Providing parameters via a config file is the recommended approach because
|
||||
it simplifies node deployment and configuration management.
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
{% capture prerequisites %}
|
||||
|
||||
- A v1.8 or higher Kubelet binary must be installed.
|
||||
- A v1.10 or higher Kubelet binary must be installed for beta functionality.
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
@@ -27,25 +29,42 @@ providing parameters via a config file, which simplifies node deployment.
|
||||
|
||||
The subset of the Kubelet's configuration that can be configured via a file
|
||||
is defined by the `KubeletConfiguration` struct
|
||||
[here (v1alpha1)](https://github.com/kubernetes/kubernetes/blob/release-1.9/pkg/kubelet/apis/kubeletconfig/v1alpha1/types.go).
|
||||
The configuration file must be a JSON or YAML representation of the parameters
|
||||
in this struct. Note that this structure, and thus the config file API,
|
||||
is still considered alpha and is not subject to stability guarantees.
|
||||
[here (v1beta1)](https://github.com/kubernetes/kubernetes/blob/release-1.10/pkg/kubelet/apis/kubeletconfig/v1beta1/types.go).
|
||||
|
||||
Create a file named `kubelet` in its own directory and make sure the directory
|
||||
and file are both readable by the Kubelet. You should write your intended
|
||||
Kubelet configuration in this `kubelet` file.
|
||||
The configuration file must be a JSON or YAML representation of the parameters
|
||||
in this struct. Make sure the Kubelet has read permissions on the file.
|
||||
|
||||
Here is an example of what this file might look like:
|
||||
```
|
||||
kind: KubeletConfiguration
|
||||
apiVersion: kubelet.config.k8s.io/v1beta1
|
||||
evictionHard:
|
||||
memory.available: "200Mi"
|
||||
```
|
||||
|
||||
In the example, the Kubelet is configured to evict Pods when available memory drops below 200Mi.
|
||||
All other Kubelet configuration values are left at their built-in defaults, unless overridden
|
||||
by flags. Command line flags which target the same value as a config file will override that value.
|
||||
|
||||
For a trick to generate a configuration file from a live node, see
|
||||
[Reconfigure a Node's Kubelet in a Live Cluster](/docs/tasks/administer-cluster/reconfigure-kubelet).
|
||||
|
||||
## Start a Kubelet process configured via the config file
|
||||
|
||||
Start the Kubelet with the `KubeletConfigFile` feature gate enabled and the
|
||||
Kubelet's `--init-config-dir` flag set to the location of the directory
|
||||
containing the `kubelet` file. The Kubelet will then load the parameters defined
|
||||
by `KubeletConfiguration` from the `kubelet` file, rather than from their
|
||||
associated command-line flags.
|
||||
Start the Kubelet with the `--config` flag set to the path of the Kubelet's config file.
|
||||
The Kubelet will then load its config from this file.
|
||||
|
||||
Note that command line flags which target the same value as a config file will override that value.
|
||||
This helps ensure backwards compatibility with the command-line API.
|
||||
|
||||
Note that relative file paths in the Kubelet config file are resolved relative to the
|
||||
location of the Kubelet config file, whereas relative paths in command line flags are resolved
|
||||
relative to the Kubelet's current working directory.
|
||||
|
||||
Note that some default values differ between command-line flags and the Kubelet config file.
|
||||
If `--config` is provided and the values are not specified via the command line, the
|
||||
defaults for the `KubeletConfiguration` version apply.
|
||||
In the above example, this version is `kubelet.config.k8s.io/v1beta1`.
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
@@ -54,16 +73,8 @@ associated command-line flags.
|
||||
## Relationship to Dynamic Kubelet Config
|
||||
|
||||
If you are using the [Dynamic Kubelet Configuration](/docs/tasks/administer-cluster/reconfigure-kubelet)
|
||||
feature, the configuration provided via `--init-config-dir` will be considered
|
||||
the "last known good" configuration by the automatic rollback mechanism.
|
||||
|
||||
Note that the layout of the files in the `--init-config-dir` mirrors the layout
|
||||
of data in the ConfigMaps used for Dynamic Kubelet Config; the file names are
|
||||
the same as the keys of the ConfigMap, and the file contents are JSON or YAML
|
||||
representations of the same structures. Today, the only pair is
|
||||
`kubelet:KubeletConfiguration`, though more may emerge in the future.
|
||||
See [Reconfigure a Node's Kubelet in a Live Cluster](/docs/tasks/administer-cluster/reconfigure-kubelet)
|
||||
for more information.
|
||||
feature, the combination of configuration provided via `--config` and any flags which override these values
|
||||
is considered the default "last known good" configuration by the automatic rollback mechanism.
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
|
||||
@@ -42,21 +42,21 @@ The basic workflow for configuring a Kubelet in a live cluster is as follows:
|
||||
1. Write a YAML or JSON configuration file containing the
|
||||
Kubelet's configuration.
|
||||
2. Wrap this file in a ConfigMap and save it to the Kubernetes control plane.
|
||||
3. Update the Kubelet's correspoinding Node object to use this ConfigMap.
|
||||
3. Update the Kubelet's corresponding Node object to use this ConfigMap.
|
||||
|
||||
Each Kubelet watches a configuration reference on its respective Node object.
|
||||
When this reference changes, the Kubelet downloads the new configuration and
|
||||
exits. For the feature to work correctly, you must be running a process manager
|
||||
When this reference changes, the Kubelet downloads the new configuration,
|
||||
updates a local reference to refer to the file, and exits.
|
||||
For the feature to work correctly, you must be running a process manager
|
||||
(like systemd) which will restart the Kubelet when it exits. When the Kubelet is
|
||||
restarted, it will begin using the new configuration.
|
||||
|
||||
The new configuration completely overrides the old configuration; unspecified
|
||||
fields in the new configuration will receive their canonical default values.
|
||||
Some CLI flags do not have an associated configuration field, and will not be
|
||||
affected by the new configuration. These fields are defined by the KubeletFlags
|
||||
structure, [here](https://github.com/kubernetes/kubernetes/blob/master/cmd/kubelet/app/options/options.go).
|
||||
The new configuration completely overrides configuration provided by `--config`,
|
||||
and is overridden by command-line flags. Unspecified values in the new configuration
|
||||
will receive default values appropriate to the configuration version
|
||||
(e.g. `kubelet.config.k8s.io/v1beta1`), unless overridden by flags.
|
||||
|
||||
The status of the Node's Kubelet configuration is reported via the `ConfigOK`
|
||||
The status of the Node's Kubelet configuration is reported via the `KubeletConfigOK`
|
||||
condition in the Node status. Once you have updated a Node to use the new
|
||||
ConfigMap, you can observe this condition to confirm that the Node is using the
|
||||
intended configuration. A table describing the possible conditions can be found
|
||||
@@ -95,13 +95,13 @@ and you will simply edit a copy of this file (which, as a best practice, should
|
||||
live in version control) while creating the first Kubelet ConfigMap. Today,
|
||||
however, the Kubelet is still bootstrapped with command-line flags. Fortunately,
|
||||
there is a dirty trick you can use to generate a config file containing a Node's
|
||||
current configuration. The trick involves hitting the Kubelet server's `configz`
|
||||
current configuration. The trick involves accessing the Kubelet server's `configz`
|
||||
endpoint via the kubectl proxy. This endpoint, in its current implementation, is
|
||||
intended to be used only as a debugging aid, which is part of why this is a
|
||||
dirty trick. There is ongoing work to improve the endpoint, and in the future
|
||||
this will be a less "dirty" operation. This trick also requires the `jq` command
|
||||
to be installed on your machine, for unpacking and editing the JSON response
|
||||
from the endpoint.
|
||||
dirty trick. The endpoint may be improved in the future, but until then
|
||||
it should not be relied on for production scenarios.
|
||||
This trick also requires the `jq` command to be installed on your machine,
|
||||
for unpacking and editing the JSON response from the endpoint.
|
||||
|
||||
Do the following to generate the file:
|
||||
|
||||
@@ -112,12 +112,12 @@ configz endpoint:
|
||||
|
||||
```
|
||||
$ export NODE_NAME=the-name-of-the-node-you-are-reconfiguring
|
||||
$ curl -sSL http://localhost:8001/api/v1/proxy/nodes/${NODE_NAME}/configz | jq '.kubeletconfig|.kind="KubeletConfiguration"|.apiVersion="kubeletconfig/v1alpha1"' > kubelet_configz_${NODE_NAME}
|
||||
$ curl -sSL http://localhost:8001/api/v1/proxy/nodes/${NODE_NAME}/configz | jq '.kubeletconfig|.kind="KubeletConfiguration"|.apiVersion="kubelet.config.k8s.io/v1beta1"' > kubelet_configz_${NODE_NAME}
|
||||
```
|
||||
|
||||
Note that we have to manually add the `kind` and `apiVersion` to the downloaded
|
||||
object, as these are not reported by the configz endpoint. This is one of the
|
||||
limitations of the endpoint that is planned to be fixed in the future.
|
||||
limitations of the endpoint.
|
||||
|
||||
### Edit the configuration file
|
||||
|
||||
@@ -209,29 +209,29 @@ Be sure to specify all three of `name`, `namespace`, and `uid`.
|
||||
### Observe that the Node begins using the new configuration
|
||||
|
||||
Retrieve the Node with `kubectl get node ${NODE_NAME} -o yaml`, and look for the
|
||||
`ConfigOK` condition in `status.conditions`. You should see the message
|
||||
`KubeletConfigOK` condition in `status.conditions`. You should see the message
|
||||
`Using current (UID: CONFIG_MAP_UID)` when the Kubelet starts using the new
|
||||
configuration.
|
||||
|
||||
For convenience, you can use the following command (using `jq`) to filter down
|
||||
to the `ConfigOK` condition:
|
||||
to the `KubeletConfigOK` condition:
|
||||
|
||||
```
|
||||
$ kubectl get no ${NODE_NAME} -o json | jq '.status.conditions|map(select(.type=="ConfigOK"))'
|
||||
$ kubectl get no ${NODE_NAME} -o json | jq '.status.conditions|map(select(.type=="KubeletConfigOK"))'
|
||||
[
|
||||
{
|
||||
"lastHeartbeatTime": "2017-09-20T18:08:29Z",
|
||||
"lastTransitionTime": "2017-09-20T18:08:17Z",
|
||||
"message": "using current (UID: \"2ebc8d1a-9e2a-11e7-a8dd-42010a800006\")",
|
||||
"message": "using current: /api/v1/namespaces/kube-system/configmaps/my-node-config-gkt4c2m4b2",
|
||||
"reason": "passing all checks",
|
||||
"status": "True",
|
||||
"type": "ConfigOK"
|
||||
"type": "KubeletConfigOK"
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
If something goes wrong, you may see one of several different error conditions,
|
||||
detailed in the Table of ConfigOK Conditions, below. When this happens, you
|
||||
detailed in the table of KubeletConfigOK conditions, below. When this happens, you
|
||||
should check the Kubelet's log for more details.
|
||||
|
||||
### Edit the configuration file again
|
||||
@@ -282,16 +282,16 @@ the following, with `name` and `uid` substituted as necessary:
|
||||
```
|
||||
configSource:
|
||||
configMapRef:
|
||||
name: NEW_CONFIG_MAP_NAME
|
||||
name: ${NEW_CONFIG_MAP_NAME}
|
||||
namespace: kube-system
|
||||
uid: NEW_CONFIG_MAP_UID
|
||||
uid: ${NEW_CONFIG_MAP_UID}
|
||||
```
|
||||
|
||||
### Observe that the Kubelet is using the new configuration
|
||||
|
||||
Once more, retrieve the Node with `kubectl get node ${NODE_NAME} -o yaml`, and
|
||||
look for the `ConfigOK` condition in `status.conditions`. You should see the message
|
||||
`Using current (UID: NEW_CONFIG_MAP_UID)` when the Kubelet starts using the
|
||||
look for the `KubeletConfigOK` condition in `status.conditions`. You should see the message
|
||||
`using current: /api/v1/namespaces/kube-system/configmaps/${NEW_CONFIG_MAP_NAME}` when the Kubelet starts using the
|
||||
new configuration.
|
||||
|
||||
### Deauthorize your Node fom reading the old ConfigMap
|
||||
@@ -327,9 +327,8 @@ remove the `spec.configSource` subfield.
|
||||
|
||||
### Observe that the Node is using its local default configuration
|
||||
|
||||
After removing this subfield, you should eventually observe that the ConfigOK
|
||||
condition's message reverts to either `using current (default)` or
|
||||
`using current (init)`, depending on how the Node was provisioned.
|
||||
After removing this subfield, you should eventually observe that the KubeletConfigOK
|
||||
condition's message reverts to `using current: local`.
|
||||
|
||||
### Deauthorize your Node fom reading the old ConfigMap
|
||||
|
||||
@@ -366,9 +365,9 @@ Here is an example command that uses `kubectl patch`:
|
||||
kubectl patch node ${NODE_NAME} -p "{\"spec\":{\"configSource\":{\"configMapRef\":{\"name\":\"${CONFIG_MAP_NAME}\",\"namespace\":\"kube-system\",\"uid\":\"${CONFIG_MAP_UID}\"}}}}"
|
||||
```
|
||||
|
||||
## Understanding ConfigOK Conditions
|
||||
## Understanding KubeletConfigOK Conditions
|
||||
|
||||
The following table describes several of the `ConfigOK` Node conditions you
|
||||
The following table describes several of the `KubeletConfigOK` Node conditions you
|
||||
might encounter in a cluster that has Dynamic Kubelet Config enabled. If you
|
||||
observe a condition with `status=False`, you should check the Kubelet log for
|
||||
more error details by searching for the message or reason text.
|
||||
@@ -383,49 +382,33 @@ more error details by searching for the message or reason text.
|
||||
<th>Status</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>using current (default)</p></td>
|
||||
<td><p>current is set to the local default, and no init config was provided</p></td>
|
||||
<td><p>using current: local</p></td>
|
||||
<td><p>when the config source is nil, the Kubelet uses its local config</p></td>
|
||||
<td><p>True</p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>using current (init)</p></td>
|
||||
<td><p>current is set to the local default, and an init config was provided</p></td>
|
||||
<td><p>True</p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>using current (UID: CURRENT_CONFIG_MAP_UID)</p></td>
|
||||
<td><p>using current: /api/v1/namespaces/${CURRENT_CONFIG_MAP_NAMESPACE}/configmaps/${CURRENT_CONFIG_MAP_NAME}</p></td>
|
||||
<td><p>passing all checks</p></td>
|
||||
<td><p>True</p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>using last-known-good (default)</p></td>
|
||||
<td><p>using last-known-good: local</p></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li>failed to load current (UID: CURRENT_CONFIG_MAP_UID)</li>
|
||||
<li>failed to parse current (UID: CURRENT_CONFIG_MAP_UID)</li>
|
||||
<li>failed to validate current (UID: CURRENT_CONFIG_MAP_UID)</li>
|
||||
<li>failed to load current: /api/v1/namespaces/${CURRENT_CONFIG_MAP_NAMESPACE}/configmaps/${CURRENT_CONFIG_MAP_NAME}</li>
|
||||
<li>failed to parse current: /api/v1/namespaces/${CURRENT_CONFIG_MAP_NAMESPACE}/configmaps/${CURRENT_CONFIG_MAP_NAME}</li>
|
||||
<li>failed to validate current: /api/v1/namespaces/${CURRENT_CONFIG_MAP_NAMESPACE}/configmaps/${CURRENT_CONFIG_MAP_NAME}</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><p>False</p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>using last-known-good (init)</p></td>
|
||||
<td><p>using last-known-good: /api/v1/namespaces/${LAST_KNOWN_GOOD_CONFIG_MAP_NAMESPACE}/configmaps/${LAST_KNOWN_GOOD_CONFIG_MAP_NAME}</p></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li>failed to load current (UID: CURRENT_CONFIG_MAP_UID)</li>
|
||||
<li>failed to parse current (UID: CURRENT_CONFIG_MAP_UID)</li>
|
||||
<li>failed to validate current (UID: CURRENT_CONFIG_MAP_UID)</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><p>False</p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>using last-known-good (UID: LAST_KNOWN_GOOD_CONFIG_MAP_UID)</p></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li>failed to load current (UID: CURRENT_CONFIG_MAP_UID)</li>
|
||||
<li>failed to parse current (UID: CURRENT_CONFIG_MAP_UID)</li>
|
||||
<li>failed to validate current (UID: CURRENT_CONFIG_MAP_UID)</li>
|
||||
<li>failed to load current: /api/v1/namespaces/${CURRENT_CONFIG_MAP_NAMESPACE}/configmaps/${CURRENT_CONFIG_MAP_NAME}</li>
|
||||
<li>failed to parse current: /api/v1/namespaces/${CURRENT_CONFIG_MAP_NAMESPACE}/configmaps/${CURRENT_CONFIG_MAP_NAME}</li>
|
||||
<li>failed to validate current: /api/v1/namespaces/${CURRENT_CONFIG_MAP_NAMESPACE}/configmaps/${CURRENT_CONFIG_MAP_NAME}</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><p>False</p></td>
|
||||
@@ -451,15 +434,15 @@ more error details by searching for the message or reason text.
|
||||
<p>failed to sync, reason:</p>
|
||||
<ul>
|
||||
<li>failed to read Node from informer object cache</li>
|
||||
<li>failed to reset to local (default or init) config</li>
|
||||
<li>failed to reset to local config</li>
|
||||
<li>invalid NodeConfigSource, exactly one subfield must be non-nil, but all were nil</li>
|
||||
<li>invalid ObjectReference, all of UID, Name, and Namespace must be specified</li>
|
||||
<li>invalid ObjectReference, UID SOME_UID does not match UID of downloaded ConfigMap SOME_OTHER_UID</li>
|
||||
<li>failed to determine whether object with UID SOME_UID was already checkpointed</li>
|
||||
<li>failed to download ConfigMap with name SOME_NAME from namespace SOME_NAMESPACE</li>
|
||||
<li>failed to save config checkpoint for object with UID SOME_UID</li>
|
||||
<li>failed to set current config checkpoint to default</li>
|
||||
<li>failed to set current config checkpoint to object with UID SOME_UID</li>
|
||||
<li>invalid ConfigSource.ConfigMapRef.UID: ${UID} does not match ${API_PATH}.UID: ${UID_OF_CONFIG_MAP_AT_API_PATH}</li>
|
||||
<li>failed to determine whether object ${API_PATH} with UID ${UID} was already checkpointed</li>
|
||||
<li>failed to download ConfigMap with name ${NAME} from namespace ${NAMESPACE}</li>
|
||||
<li>failed to save config checkpoint for object ${API_PATH} with UID ${UID}</li>
|
||||
<li>failed to set current config checkpoint to local config</li>
|
||||
<li>failed to set current config checkpoint to object ${API_PATH} with UID ${UID}</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><p>False</p></td>
|
||||
|
||||
@@ -33,7 +33,7 @@ Successfully running cloud-controller-manager requires some changes to your clus
|
||||
|
||||
* `kube-apiserver` and `kube-controller-manager` MUST NOT specify the `--cloud-provider` flag. This ensures that it does not run any cloud specific loops that would be run by cloud controller manager. In the future, this flag will be deprecated and removed.
|
||||
* `kubelet` must run with `--cloud-provider=external`. This is to ensure that the kubelet is aware that it must be initialized by the cloud controller manager before it is scheduled any work.
|
||||
* `kube-apiserver` SHOULD NOT run the `PersistentVolumeLabel` admission controller since the cloud controller manager takes over labeling persistent volumes. To prevent the PersistentVolumeLabel admission plugin from running, make sure the `kube-apiserver` has a `--admission-control` flag with a value that does not include `PersistentVolumeLabel`.
|
||||
* `kube-apiserver` SHOULD NOT run the `PersistentVolumeLabel` admission controller since the cloud controller manager takes over labeling persistent volumes. To prevent the PersistentVolumeLabel admission plugin from running in `kube-apiserver`, include the `PersistentVolumeLabel` as a listed value in the `--disable-admission-plugins` flag.
|
||||
* For the `cloud-controller-manager` to label persistent volumes, initializers will need to be enabled and an InitializerConifguration needs to be added to the system. Follow [these instructions](/docs/admin/extensible-admission-controllers.md#enable-initializers-alpha-feature) to enable initializers. Use the following YAML to create the InitializerConfiguration:
|
||||
|
||||
{% include code.html language="yaml" file="persistent-volume-label-initializer-config.yaml" ghlink="/docs/tasks/administer-cluster/persistent-volume-label-initializer-config.yaml" %}
|
||||
|
||||
@@ -0,0 +1,315 @@
|
||||
---
|
||||
approvers:
|
||||
- msau42
|
||||
- jsafrane
|
||||
title: Storage Object in Use Protection
|
||||
---
|
||||
|
||||
{% capture overview %}
|
||||
{% assign for_k8s_version="v1.10" %}{% include feature-state-beta.md %}
|
||||
|
||||
Persistent volume claims (PVCs) that are in active use by a pod and persistent volumes (PVs) that are bound to PVCs can be protected from pre-mature removal.
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
{% capture prerequisites %}
|
||||
|
||||
- The Storage Object in Use Protection feature is enabled in a version of Kubernetes in which it is supported.
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
{% capture steps %}
|
||||
|
||||
## Storage Object in Use Protection feature used for PVC Protection
|
||||
|
||||
The example below uses a GCE PD `StorageClass`, however, similar steps can be performed for any volume type.
|
||||
|
||||
Create a `StorageClass` for convenient storage provisioning:
|
||||
```yaml
|
||||
apiVersion: storage.k8s.io/v1
|
||||
kind: StorageClass
|
||||
metadata:
|
||||
name: slow
|
||||
provisioner: kubernetes.io/gce-pd
|
||||
parameters:
|
||||
type: pd-standard
|
||||
```
|
||||
|
||||
Verification scenarios follow below.
|
||||
|
||||
### Scenario 1: The PVC is not in active use by a pod
|
||||
|
||||
- Create a PVC:
|
||||
|
||||
```yaml
|
||||
kind: PersistentVolumeClaim
|
||||
apiVersion: v1
|
||||
metadata:
|
||||
name: slzc
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
storageClassName: slow
|
||||
resources:
|
||||
requests:
|
||||
storage: 3.7Gi
|
||||
```
|
||||
|
||||
- Check that the PVC has the finalizer `kubernetes.io/pvc-protection` set:
|
||||
|
||||
```shell
|
||||
kubectl describe pvc slzc
|
||||
Name: slzc
|
||||
Namespace: default
|
||||
StorageClass: slow
|
||||
Status: Bound
|
||||
Volume: pvc-bee8c30a-d6a3-11e7-9af0-42010a800002
|
||||
Labels: <none>
|
||||
Annotations: pv.kubernetes.io/bind-completed=yes
|
||||
pv.kubernetes.io/bound-by-controller=yes
|
||||
volume.beta.kubernetes.io/storage-provisioner=kubernetes.io/gce-pd
|
||||
Finalizers: [kubernetes.io/pvc-protection]
|
||||
Capacity: 4Gi
|
||||
Access Modes: RWO
|
||||
Events:
|
||||
Type Reason Age From Message
|
||||
---- ------ ---- ---- -------
|
||||
Normal ProvisioningSucceeded 2m persistentvolume-controller Successfully provisioned volume pvc-bee8c30a-d6a3-11e7-9af0-42010a800002 using kubernetes.io/gce-pd
|
||||
```
|
||||
|
||||
- Delete the PVC and check that the PVC (not in active use by a pod) was removed successfully.
|
||||
|
||||
### Scenario 2: The PVC is in active use by a pod
|
||||
|
||||
- Again, create the same PVC.
|
||||
- Create a pod that uses the PVC:
|
||||
|
||||
```yaml
|
||||
kind: Pod
|
||||
apiVersion: v1
|
||||
metadata:
|
||||
name: app1
|
||||
spec:
|
||||
containers:
|
||||
- name: test-pod
|
||||
image: k8s.gcr.io/busybox:1.24
|
||||
command:
|
||||
- "/bin/sh"
|
||||
args:
|
||||
- "-c"
|
||||
- "date > /mnt/app1.txt; sleep 60 && exit 0 || exit 1"
|
||||
volumeMounts:
|
||||
- name: path-pvc
|
||||
mountPath: "/mnt"
|
||||
restartPolicy: "Never"
|
||||
volumes:
|
||||
- name: path-pvc
|
||||
persistentVolumeClaim:
|
||||
claimName: slzc
|
||||
```
|
||||
|
||||
- Wait until the pod status is `Running`, i.e. the PVC becomes in active use.
|
||||
- Delete the PVC that is now in active use by a pod and verify that the PVC is not removed but its status is `Terminating`:
|
||||
|
||||
```shell
|
||||
Name: slzc
|
||||
Namespace: default
|
||||
StorageClass: slow
|
||||
Status: Terminating (since Fri, 01 Dec 2017 14:47:55 +0000)
|
||||
Volume: pvc-803a1f4d-d6a6-11e7-9af0-42010a800002
|
||||
Labels: <none>
|
||||
Annotations: pv.kubernetes.io/bind-completed=yes
|
||||
pv.kubernetes.io/bound-by-controller=yes
|
||||
volume.beta.kubernetes.io/storage-provisioner=kubernetes.io/gce-pd
|
||||
Finalizers: [kubernetes.io/pvc-protection]
|
||||
Capacity: 4Gi
|
||||
Access Modes: RWO
|
||||
Events:
|
||||
Type Reason Age From Message
|
||||
---- ------ ---- ---- -------
|
||||
Normal ProvisioningSucceeded 52s persistentvolume-controller Successfully provisioned volume pvc-803a1f4d-d6a6-11e7-9af0-42010a800002 using kubernetes.io/gce-pd
|
||||
```
|
||||
- Wait until the pod status is `Terminated` (either delete the pod or wait until it finishes). Afterwards, check that the PVC is removed.
|
||||
|
||||
### Scenario 3: A pod starts using a PVC that is in Terminating state
|
||||
|
||||
- Again, create the same PVC.
|
||||
- Create a first pod that uses the PVC:
|
||||
|
||||
```yaml
|
||||
kind: Pod
|
||||
apiVersion: v1
|
||||
metadata:
|
||||
name: app1
|
||||
spec:
|
||||
containers:
|
||||
- name: test-pod
|
||||
image: k8s.gcr.io/busybox:1.24
|
||||
command:
|
||||
- "/bin/sh"
|
||||
args:
|
||||
- "-c"
|
||||
- "date > /mnt/app1.txt; sleep 600 && exit 0 || exit 1"
|
||||
volumeMounts:
|
||||
- name: path-pvc
|
||||
mountPath: "/mnt"
|
||||
restartPolicy: "Never"
|
||||
volumes:
|
||||
- name: path-pvc
|
||||
persistentVolumeClaim:
|
||||
claimName: slzc
|
||||
```
|
||||
|
||||
- Wait until the pod status is `Running`, i.e. the PVC becomes in active use.
|
||||
- Delete the PVC that is now in active use by a pod and verify that the PVC is not removed but its status is `Terminating`:
|
||||
|
||||
```shell
|
||||
Name: slzc
|
||||
Namespace: default
|
||||
StorageClass: slow
|
||||
Status: Terminating (since Fri, 01 Dec 2017 14:47:55 +0000)
|
||||
Volume: pvc-803a1f4d-d6a6-11e7-9af0-42010a800002
|
||||
Labels: <none>
|
||||
Annotations: pv.kubernetes.io/bind-completed=yes
|
||||
pv.kubernetes.io/bound-by-controller=yes
|
||||
volume.beta.kubernetes.io/storage-provisioner=kubernetes.io/gce-pd
|
||||
Finalizers: [kubernetes.io/pvc-protection]
|
||||
Capacity: 4Gi
|
||||
Access Modes: RWO
|
||||
Events:
|
||||
Type Reason Age From Message
|
||||
---- ------ ---- ---- -------
|
||||
Normal ProvisioningSucceeded 52s persistentvolume-controller Successfully provisioned volume pvc-803a1f4d-d6a6-11e7-9af0-42010a800002 using kubernetes.io/gce-pd
|
||||
```
|
||||
|
||||
- Create a second pod that uses the same PVC:
|
||||
|
||||
```
|
||||
kind: Pod
|
||||
apiVersion: v1
|
||||
metadata:
|
||||
name: app2
|
||||
spec:
|
||||
containers:
|
||||
- name: test-pod
|
||||
image: gcr.io/google_containers/busybox:1.24
|
||||
command:
|
||||
- "/bin/sh"
|
||||
args:
|
||||
- "-c"
|
||||
- "date > /mnt/app1.txt; sleep 600 && exit 0 || exit 1"
|
||||
volumeMounts:
|
||||
- name: path-pvc
|
||||
mountPath: "/mnt"
|
||||
restartPolicy: "Never"
|
||||
volumes:
|
||||
- name: path-pvc
|
||||
persistentVolumeClaim:
|
||||
claimName: slzc
|
||||
```
|
||||
|
||||
- Verify that the scheduling of the second pod fails with the below warning:
|
||||
|
||||
```
|
||||
Warning FailedScheduling 18s (x4 over 21s) default-scheduler persistentvolumeclaim "slzc" is being deleted
|
||||
```
|
||||
|
||||
- Wait until the pod status of both pods is `Terminated` or `Completed` (either delete the pods or wait until they finish). Afterwards, check that the PVC is removed.
|
||||
|
||||
## Storage Object in Use Protection feature used for PV Protection
|
||||
|
||||
The example below uses a `HostPath` PV.
|
||||
|
||||
Verification scenarios follow below.
|
||||
|
||||
### Scenario 1: The PV is not bound to a PVC
|
||||
|
||||
- Create a PV:
|
||||
|
||||
```yaml
|
||||
kind: PersistentVolume
|
||||
apiVersion: v1
|
||||
metadata:
|
||||
name: task-pv-volume
|
||||
labels:
|
||||
type: local
|
||||
spec:
|
||||
capacity:
|
||||
storage: 1Gi
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
persistentVolumeReclaimPolicy: Delete
|
||||
storageClassName: standard
|
||||
hostPath:
|
||||
path: "/tmp/data"
|
||||
```
|
||||
|
||||
- Check that the PV has the finalizer `kubernetes.io/pv-protection` set:
|
||||
|
||||
```shell
|
||||
Name: task-pv-volume
|
||||
Labels: type=local
|
||||
Annotations: pv.kubernetes.io/bound-by-controller=yes
|
||||
Finalizers: [kubernetes.io/pv-protection]
|
||||
StorageClass: standard
|
||||
Status: Terminating (lasts 1m)
|
||||
Claim: default/task-pv-claim
|
||||
Reclaim Policy: Delete
|
||||
Access Modes: RWO
|
||||
Capacity: 1Gi
|
||||
Message:
|
||||
Source:
|
||||
Type: HostPath (bare host directory volume)
|
||||
Path: /tmp/data
|
||||
HostPathType:
|
||||
Events: <none>
|
||||
```
|
||||
|
||||
- Delete the PV and check that the PV (not bound to a PVC) is removed successfully.
|
||||
|
||||
### Scenario 2: The PV is bound to a PVC
|
||||
|
||||
- Again, create the same PV.
|
||||
|
||||
- Create a PVC
|
||||
|
||||
```yaml
|
||||
kind: PersistentVolumeClaim
|
||||
apiVersion: v1
|
||||
metadata:
|
||||
name: task-pv-claim
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
resources:
|
||||
requests:
|
||||
storage: 1Gi
|
||||
```
|
||||
|
||||
- Wait until the PV and PVC are bound to each other.
|
||||
- Delete the PV and verify that the PV is not removed but its status is `Terminating`:
|
||||
|
||||
```shell
|
||||
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
|
||||
task-pv-volume 1Gi RWO Delete Terminating default/task-pv-claim standard 59s
|
||||
|
||||
```
|
||||
- Delete the PVC and verify that the PV is removed too.
|
||||
|
||||
```shell
|
||||
kubectl delete pvc task-pv-claim
|
||||
persistentvolumeclaim "task-pv-claim" deleted
|
||||
$ kubectl get pvc
|
||||
No resources found.
|
||||
$ kubectl get pv
|
||||
No resources found.
|
||||
```
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
{% capture discussion %}
|
||||
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
{% include templates/task.md %}
|
||||
Reference in New Issue
Block a user