Merge 1.10 to master for release (#7861)
* 1.10 update (#7151) * Fix partition value expected behaviour explanation (#7123) Fixes issue #7057 * Correct "On-Premise" to "On-Premises" * Updates the Calico installation page (#7094) * All files for Haufe Groups case study (#7051) * Fix typo (#7127) * fix typo of device-plugins.md (#7106) * fix broken links (#7136) * Updated configure-service-account (#7147) Error from server resolved by escaping kubectl patch serviceaccount default -p '{"imagePullSecrets": [{"name": "myregistrykey"}]}' JSON string by '\' * Remove docs related to 'require-kubeconfig' (#7138) With kubernetes/kubernetes#58367 merged, v1.10 will not use the "require-kubeconfig" flag. The flag has become a no-op solely to ensure existing deployments won't break. * Added Verification Scenario for a Pod that Uses a PVC in Terminating State (#7164) The below PR: https://github.com/kubernetes/kubernetes/pull/55873 modified scheduler in such a way that scheduling of a pod that uses a PVC in Terminating state fails. That's why verification of such scenario was added to documentation. * fix LimitPodHardAntiAffinityTopology name (#7221) * Document the removal of the KubeletConfigFile feature gate (#7140) With kubernetes/kubernetes#58978 merged, the said feature gate is removed. This PR removes texts related to the gate and revises the Feature Gates reference to reflect this change. * deprecate three admission controller (#7363) * Document the removal of Accelerators feature gate (#7389) The `Accelerators` feature gate will be removed in 1.11. 1.10 will be its last mile. References: kubernetes/kubernetes#57384 * Update local storage docs for beta (#7473) * Document that HugePages feature gate is Beta (#7387) The `HugePages` feature gate has graduated to Beta in v1.10. This PR documents this fact. * Add HyperVContainer feature gates (#7502) * Remove the beta reference from Taints and Tolerations doc (#7493) * Kms provider doc (#7479) * Kms provider doc * issue# 7399, Create KMS-provider.md and update encrypt-data.md * address review comments * Document that Device Plugin feature is Beta (1.10) (#7512) * Add docs for CRD features for 1.10 (#7439) * Add docs for CRD features for 1.10 * Add CustomResourcesSubresources to list of feature gates * Add latest changes to custom resources doc * Add crds as abbreviated alias (#7437) * Bring PVC Protection Feature to Beta (#7165) * Bring PVC Protection Feature to Beta The PR: https://github.com/kubernetes/kubernetes/pull/59052 brought PVC Protection feature to beta. That's why the documentation is updated accordingly. * The PVC Protection feature was renamed to Storage Protection. That's why the documentation is updated. * promote PodNodeSelector to stable; document detailed behavior (#7134) * promote PodNodeSelector to stable; document detailed behavior * respond to feedback * Update CPU manager feature enabling (#7390) With `CPUManager` feature graduating to beta. No explicit enabling is required starting v1.10. References: kubernetes/kubernetes#55977 * Adding block volumeMode documentation for local volumes. (#7531) Code review comments. Changed property to field. Address tech review comment. * remove description kubectl --show-all (#7574) --show-all has been deprecated and set to true by default. https://github.com/kubernetes/kubernetes/pull/60210 * fix description about contribute style guide (#7592) * fix description about KUBECONFIG (#7589) s/envrionment/environment * fix description about cni (#7588) s/simultanously/simultaneously/ * fix description about MutatingAdmissionWebhook and ValidatingAdmissionWebhook (#7587) * fix description about persistent volume binding (#7590) s/slighty/slightly/ * Doc change for configurable pod resolv.conf Beta (#7611) * fix description about out of resource handling (#7597) s/threshhold/threshold * fix description about zookeeper (#7598) s/achive/achieve * fix description about kubeadm (#7594) s/compatability/compatibility/ * fix description about kubeadm (#7593) * fix description about kubeadm implementation details (#7595) * fix description about api concepts (#7596) * Storage Protection was renamed to Storage Object in Use Protection (#7576) * Storage Protection was renamed to Storage Object in Use Protection The K8s PR: https://github.com/kubernetes/kubernetes/pull/59901 renamed Storage Protection to Storage Object in Use Protection. That's why the same is also renamed in the documentation. * Moved Storage Object in Use Protection admission plugin description down according to alphabetic order. * Use PSP from policy API group. (#7562) * update kubeletconfig docs for v1.10, beta (#7561) * Update port-forwarding docs (#7575) * add pv protection description (#7620) * fix description about client library (#7634) * Add docs on configuring NodePort IP (#7631) * Document that LocalStorageCapacityIsolation is beta (#7635) A follow-up to the kubernetes/kubernetes#60159 change which has promoted the `LocalStorageCapacityIsolation` feature gate to Beta. * Update CoreDNS docs for beta (#7638) * Update CoreDNS docs for beta * Review comments * Fix typo (#7640) * Update feature gates move to beta (#7662) * Added the inability to use colon ':' character as environment variable names and described workaround (#7657) * merge master to 1.10, with fixes (#7682) * Flag names changed (s/admission-control/enable-admission-plugins); disable-admissions-plugin entry added; removed reference to admission controller/plugins requiring set order (for v1.10), redundant example enabling specific plugin, and redundant version-specific info (#7449) * Documentation for MountPropagation beta (#7655) * Remove job's scale-related operations (#7684) * authentication: document client-go exec plugins (#7648) * authentication: document client-go exec plugins * Update authentication.md * Update local ephemeral storage feature to beta (#7685) Update local ephemeral storage feature to beta * Update docs for windows container resources (#7653) * add server-side print docs (#7671) * Create a task describing Pod process namespace sharing (#7489) * Add external metrics to HPA docs (#7664) * Add external metrics to HPA docs * Update horizontal-pod-autoscale-walkthrough.md * Apply review comments to HPA walkthrough * remove description about "scale jobs" (#7712) * CSI Docs for K8s v1.10 (#7698) * Add a warning about increased memory consumption for audit logging feature. (#7725) Signed-off-by: Mik Vyatskov <vmik@google.com> * Update Audit Logging documentation for 1.10 (#7679) Signed-off-by: Mik Vyatskov <vmik@google.com> * Fix stage names in audit logging documentation (#7746) Signed-off-by: Mik Vyatskov <vmik@google.com> * Feature gate update for release 1.10 (#7742) * State in the docs that the value of default Node labels are not reliable. (#7794) * Kill the reference to --admission-control option (#7755) The `--admission-control` option has been replaced by two new options in v1.10. This PR kills the last appearance of the old option in the doc. * Pvcprotection toc (#7807) * Refreshing installation instructions (#7495) * Refreshing installation instructions Added conjure-up. Updated displays and juju versions to current versions. * Updated anchors * Fixed image value version typo (#7768) Was inconsistent with other values * Update flocker reference to the github repo (#7784) * Fix typo in federation document (#7779) * an user -> a user (#7778) * Events are namespaced (#7767) * fix 'monitoring' link lose efficacy problem' (#7764) * docs/concepts/policy/pod-security-policy.md: minor fix. (#7659) * Update downward-api-volume-expose-pod-information.md (#7771) * Update downward-api-volume-expose-pod-information.md The pod spec puts the downward api files into /etc/podinfo, not directly in /etc. Updated docs to reflect this fact. * Update downward-api-volume-expose-pod-information.md One more spot needed fixing. * Update downward-api-volume-expose-pod-information.md Yet another fix, in the container example. * Add Amadeus Case Study (#7783) * Add Amadeus Case Study * add Amadeus logo * Fixed Cyrillic с in 'kube-proxy-cm' (#7787) There was a typo (wrong character) in kube-proxy-cm.yaml - Cyrillic с (UTF-8 0x0441) was used instead of Latin c. * install-kubectl: choose one installation method (#7705) The previous text layout suggested that all installations had to be done, one after another. * Update install-kubeadm.md (#7781) Add note to kubeadm install instruction to help install in other arch i.e. aarch64, ppc64le etc. * repair failure link (#7788) * repair failure link * repair failure link * do change as required * Update k8s201.md (#7777) * Update k8s201.md Change instructions to download yams files directly from the website (as used in other pages.) Added instructions to delete labeled pod to avoid warnings in the subsequent deployment step. * Update k8s201.md Added example of using the exposed host from the a node running Kubernetes. (This works on AWS with Weave; not able to test it on other variations...) * Gramatical fix to kompose introduction (#7792) The original wording didn't through very well. As much of the original sentence has been preserved as possible, primarily to ensure the kompose web address is see both in text and as a href link. * update amadeus.html (#7800) * Fix a missing word in endpoint reconciler section (#7804) * add toc entry for pvcprotection downgrade issue doc * Pvcprotection toc (#7809) * Refreshing installation instructions (#7495) * Refreshing installation instructions Added conjure-up. Updated displays and juju versions to current versions. * Updated anchors * Fixed image value version typo (#7768) Was inconsistent with other values * Update flocker reference to the github repo (#7784) * Fix typo in federation document (#7779) * an user -> a user (#7778) * Events are namespaced (#7767) * fix 'monitoring' link lose efficacy problem' (#7764) * docs/concepts/policy/pod-security-policy.md: minor fix. (#7659) * Update downward-api-volume-expose-pod-information.md (#7771) * Update downward-api-volume-expose-pod-information.md The pod spec puts the downward api files into /etc/podinfo, not directly in /etc. Updated docs to reflect this fact. * Update downward-api-volume-expose-pod-information.md One more spot needed fixing. * Update downward-api-volume-expose-pod-information.md Yet another fix, in the container example. * Add Amadeus Case Study (#7783) * Add Amadeus Case Study * add Amadeus logo * Fixed Cyrillic с in 'kube-proxy-cm' (#7787) There was a typo (wrong character) in kube-proxy-cm.yaml - Cyrillic с (UTF-8 0x0441) was used instead of Latin c. * install-kubectl: choose one installation method (#7705) The previous text layout suggested that all installations had to be done, one after another. * Update install-kubeadm.md (#7781) Add note to kubeadm install instruction to help install in other arch i.e. aarch64, ppc64le etc. * repair failure link (#7788) * repair failure link * repair failure link * do change as required * Update k8s201.md (#7777) * Update k8s201.md Change instructions to download yams files directly from the website (as used in other pages.) Added instructions to delete labeled pod to avoid warnings in the subsequent deployment step. * Update k8s201.md Added example of using the exposed host from the a node running Kubernetes. (This works on AWS with Weave; not able to test it on other variations...) * Gramatical fix to kompose introduction (#7792) The original wording didn't through very well. As much of the original sentence has been preserved as possible, primarily to ensure the kompose web address is see both in text and as a href link. * update amadeus.html (#7800) * Fix a missing word in endpoint reconciler section (#7804) * add toc entry for pvcprotection downgrade issue doc * revert TOC change * Release 1.10 (#7818) * Refreshing installation instructions (#7495) * Refreshing installation instructions Added conjure-up. Updated displays and juju versions to current versions. * Updated anchors * Fixed image value version typo (#7768) Was inconsistent with other values * Update flocker reference to the github repo (#7784) * Fix typo in federation document (#7779) * an user -> a user (#7778) * Events are namespaced (#7767) * fix 'monitoring' link lose efficacy problem' (#7764) * docs/concepts/policy/pod-security-policy.md: minor fix. (#7659) * Update downward-api-volume-expose-pod-information.md (#7771) * Update downward-api-volume-expose-pod-information.md The pod spec puts the downward api files into /etc/podinfo, not directly in /etc. Updated docs to reflect this fact. * Update downward-api-volume-expose-pod-information.md One more spot needed fixing. * Update downward-api-volume-expose-pod-information.md Yet another fix, in the container example. * Add Amadeus Case Study (#7783) * Add Amadeus Case Study * add Amadeus logo * Fixed Cyrillic с in 'kube-proxy-cm' (#7787) There was a typo (wrong character) in kube-proxy-cm.yaml - Cyrillic с (UTF-8 0x0441) was used instead of Latin c. * install-kubectl: choose one installation method (#7705) The previous text layout suggested that all installations had to be done, one after another. * Update install-kubeadm.md (#7781) Add note to kubeadm install instruction to help install in other arch i.e. aarch64, ppc64le etc. * repair failure link (#7788) * repair failure link * repair failure link * do change as required * Update k8s201.md (#7777) * Update k8s201.md Change instructions to download yams files directly from the website (as used in other pages.) Added instructions to delete labeled pod to avoid warnings in the subsequent deployment step. * Update k8s201.md Added example of using the exposed host from the a node running Kubernetes. (This works on AWS with Weave; not able to test it on other variations...) * Gramatical fix to kompose introduction (#7792) The original wording didn't through very well. As much of the original sentence has been preserved as possible, primarily to ensure the kompose web address is see both in text and as a href link. * update amadeus.html (#7800) * Fix a missing word in endpoint reconciler section (#7804) * Partners page updates (#7802) * Partners page updates * Update to ZTE link * Make using sysctls a task instead of a concept (#6808) Closes: #4505 * add a note when mount a configmap to pod (#7745) * adjust a note format (#7812) * Update docker-cli-to-kubectl.md (#7748) * Update docker-cli-to-kubectl.md Edited the document for adherence to the style guide and word usage. * Update docker-cli-to-kubectl.md * Incorporated the changes suggested. * Mount propagation update to include docker config (#7854) * update overridden config for 1.10 (#7847) * update overridden config for 1.10 * fix config file per comments * Update Extended Resource doc wrt cluster-level resources (#7759)
This commit is contained in:
committed by
Steve Perry
parent
770440d20f
commit
409e77d3e5
@@ -22,53 +22,93 @@ for database debugging.
|
||||
|
||||
{% capture steps %}
|
||||
|
||||
## Creating a pod to run a Redis server
|
||||
## Creating Redis deployment and service
|
||||
|
||||
1. Create a pod:
|
||||
1. Create a Redis deployment:
|
||||
|
||||
kubectl create -f https://k8s.io/docs/tasks/access-application-cluster/redis-master.yaml
|
||||
kubectl create -f https://k8s.io/docs/tutorials/stateless-application/guestbook/redis-master-deployment.yaml
|
||||
|
||||
The output of a successful command verifies that the pod was created:
|
||||
|
||||
pod "redis-master" created
|
||||
|
||||
1. Check to see whether the pod is running and ready:
|
||||
The output of a successful command verifies that the deployment was created:
|
||||
|
||||
deployment "redis-master" created
|
||||
|
||||
When the pod is ready, you can get:
|
||||
|
||||
kubectl get pods
|
||||
|
||||
When the pod is ready, the output displays a STATUS of Running:
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
redis-master-765d459796-258hz 1/1 Running 0 50s
|
||||
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
redis-master 2/2 Running 0 41s
|
||||
kubectl get deployment
|
||||
|
||||
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
|
||||
redis-master 1 1 1 1 55s
|
||||
|
||||
1. Verify that the Redis server is running in the pod and listening on port 6379:
|
||||
kubectl get rs
|
||||
NAME DESIRED CURRENT READY AGE
|
||||
redis-master-765d459796 1 1 1 1m
|
||||
|
||||
|
||||
2. Create a Redis service:
|
||||
|
||||
kubectl create -f https://k8s.io/docs/tutorials/stateless-application/guestbook/redis-master-service.yaml
|
||||
|
||||
The output of a successful command verifies that the service was created:
|
||||
|
||||
service "redis-master" created
|
||||
|
||||
Check the service created:
|
||||
|
||||
kubectl get svc | grep redis
|
||||
|
||||
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
|
||||
redis-master ClusterIP 10.0.0.213 <none> 6379/TCP 27s
|
||||
|
||||
3. Verify that the Redis server is running in the pod and listening on port 6379:
|
||||
|
||||
{% raw %}
|
||||
kubectl get pods redis-master --template='{{(index (index .spec.containers 0).ports 0).containerPort}}{{"\n"}}'
|
||||
kubectl get pods redis-master-765d459796-258hz --template='{{(index (index .spec.containers 0).ports 0).containerPort}}{{"\n"}}'
|
||||
{% endraw %}
|
||||
|
||||
The output displays the port:
|
||||
|
||||
6379
|
||||
|
||||
|
||||
## Forward a local port to a port on the pod
|
||||
|
||||
1. Forward port 6379 on the local workstation to port 6379 of redis-master pod:
|
||||
1. `kubectl port-forward` allows using resource name, such as a service name, to select a matching pod to port forward to since Kubernetes v1.10.
|
||||
|
||||
kubectl port-forward redis-master 6379:6379
|
||||
kubectl port-forward redis-master-765d459796-258hz 6379:6379
|
||||
|
||||
The output is similar to this:
|
||||
which is the same as
|
||||
|
||||
kubectl port-forward pods/redis-master-765d459796-258hz 6379:6379
|
||||
|
||||
or
|
||||
|
||||
kubectl port-forward deployment/redis-master 6379:6379
|
||||
|
||||
or
|
||||
|
||||
kubectl port-forward rs/redis-master 6379:6379
|
||||
|
||||
or
|
||||
|
||||
kubectl port-forward svc/redis-master 6379:6379
|
||||
|
||||
Any of the above commands works. The output is similar to this:
|
||||
|
||||
I0710 14:43:38.274550 3655 portforward.go:225] Forwarding from 127.0.0.1:6379 -> 6379
|
||||
I0710 14:43:38.274797 3655 portforward.go:225] Forwarding from [::1]:6379 -> 6379
|
||||
|
||||
1. Start the Redis command line interface:
|
||||
2. Start the Redis command line interface:
|
||||
|
||||
redis-cli
|
||||
redis-cli
|
||||
|
||||
1. At the Redis command line prompt, enter the `ping` command:
|
||||
3. At the Redis command line prompt, enter the `ping` command:
|
||||
|
||||
127.0.0.1:6379>ping
|
||||
127.0.0.1:6379>ping
|
||||
|
||||
A successful ping request returns PONG.
|
||||
|
||||
|
||||
@@ -216,8 +216,8 @@ Validation of custom objects is possible via
|
||||
[OpenAPI v3 schema](https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.0.md#schemaObject).
|
||||
Additionally, the following restrictions are applied to the schema:
|
||||
|
||||
- The fields `default`, `nullable`, `discriminator`, `readOnly`, `writeOnly`, `xml` and
|
||||
`deprecated` cannot be set.
|
||||
- The fields `default`, `nullable`, `discriminator`, `readOnly`, `writeOnly`, `xml`,
|
||||
`deprecated` and `$ref` cannot be set.
|
||||
- The field `uniqueItems` cannot be set to true.
|
||||
- The field `additionalProperties` cannot be set to false.
|
||||
|
||||
@@ -328,6 +328,218 @@ And create it:
|
||||
kubectl create -f my-crontab.yaml
|
||||
crontab "my-new-cron-object" created
|
||||
```
|
||||
|
||||
### Subresources
|
||||
|
||||
Custom resources support `/status` and `/scale` subresources.
|
||||
This feature is __alpha__ in v1.10 and may change in backward incompatible ways.
|
||||
|
||||
Enable this feature using the `CustomResourceSubresources` feature gate on
|
||||
the [kube-apiserver](/docs/admin/kube-apiserver):
|
||||
|
||||
```
|
||||
--feature-gates=CustomResourceSubresources=true
|
||||
```
|
||||
|
||||
When the `CustomResourceSubresources` feature gate is enabled, only the `properties` construct
|
||||
is allowed in the root schema for custom resource validation.
|
||||
|
||||
The status and scale subresources can be optionally enabled by
|
||||
defining them in the CustomResourceDefinition.
|
||||
|
||||
#### Status subresource
|
||||
|
||||
When the status subresource is enabled, the `/status` subresource for the custom resource is exposed.
|
||||
|
||||
- The status and the spec stanzas are represented by the `.status` and `.spec` JSONPaths respectively inside of a custom resource.
|
||||
- `PUT` requests to the `/status` subresource take a custom resource object and ignore changes to anything except the status stanza.
|
||||
- `PUT` requests to the `/status` subresource only validate the status stanza of the custom resource.
|
||||
- `PUT`/`POST`/`PATCH` requests to the custom resource ignore changes to the status stanza.
|
||||
- Any changes to the spec stanza increments the value at `.metadata.generation`.
|
||||
|
||||
#### Scale subresource
|
||||
|
||||
When the scale subresource is enabled, the `/scale` subresource for the custom resource is exposed.
|
||||
The `autoscaling/v1.Scale` object is sent as the payload for `/scale`.
|
||||
|
||||
To enable the scale subresource, the following values are defined in the CustomResourceDefinition.
|
||||
|
||||
- `SpecReplicasPath` defines the JSONPath inside of a custom resource that corresponds to `Scale.Spec.Replicas`.
|
||||
|
||||
- It is a required value.
|
||||
- Only JSONPaths under `.spec` and with the dot notation are allowed.
|
||||
- If there is no value under the `SpecReplicasPath` in the custom resource,
|
||||
the `/scale` subresource will return an error on GET.
|
||||
|
||||
- `StatusReplicasPath` defines the JSONPath inside of a custom resource that corresponds to `Scale.Status.Replicas`.
|
||||
|
||||
- It is a required value.
|
||||
- Only JSONPaths under `.status` and with the dotation are allowed.
|
||||
- If there is no value under the `StatusReplicasPath` in the custom resource,
|
||||
the status replica value in the `/scale` subresource will default to 0.
|
||||
|
||||
- `LabelSelectorPath` defines the JSONPath inside of a custom resource that corresponds to `Scale.Status.Selector`.
|
||||
|
||||
- It is an optional value.
|
||||
- It must be set to work with HPA.
|
||||
- Only JSONPaths under `.status` and with the dotation are allowed.
|
||||
- If there is no value under the `LabelSelectorPath` in the custom resource,
|
||||
the status selector value in the `/scale` subresource will default to the empty string.
|
||||
|
||||
In the following example, both status and scale subresources are enabled.
|
||||
|
||||
Save the CustomResourceDefinition to `resourcedefinition.yaml`:
|
||||
|
||||
```yaml
|
||||
apiVersion: apiextensions.k8s.io/v1beta1
|
||||
kind: CustomResourceDefinition
|
||||
metadata:
|
||||
name: crontabs.stable.example.com
|
||||
spec:
|
||||
group: stable.example.com
|
||||
version: v1
|
||||
scope: Namespaced
|
||||
names:
|
||||
plural: crontabs
|
||||
singular: crontab
|
||||
kind: CronTab
|
||||
shortNames:
|
||||
- ct
|
||||
# subresources describes the subresources for custom resources.
|
||||
subresources:
|
||||
# status enables the status subresource.
|
||||
status: {}
|
||||
# scale enables the scale subresource.
|
||||
scale:
|
||||
# specReplicasPath defines the JSONPath inside of a custom resource that corresponds to Scale.Spec.Replicas.
|
||||
specReplicasPath: .spec.replicas
|
||||
# statusReplicasPath defines the JSONPath inside of a custom resource that corresponds to Scale.Status.Replicas.
|
||||
statusReplicasPath: .status.replicas
|
||||
# labelSelectorPath defines the JSONPath inside of a custom resource that corresponds to Scale.Status.Selector.
|
||||
labelSelectorPath: .status.labelSelector
|
||||
```
|
||||
|
||||
And create it:
|
||||
|
||||
```shell
|
||||
kubectl create -f resourcedefinition.yaml
|
||||
```
|
||||
|
||||
After the CustomResourceDefinition object has been created, you can create custom objects.
|
||||
|
||||
If you save the following YAML to `my-crontab.yaml`:
|
||||
|
||||
```yaml
|
||||
apiVersion: "stable.example.com/v1"
|
||||
kind: CronTab
|
||||
metadata:
|
||||
name: my-new-cron-object
|
||||
spec:
|
||||
cronSpec: "* * * * */5"
|
||||
image: my-awesome-cron-image
|
||||
replicas: 3
|
||||
```
|
||||
|
||||
and create it:
|
||||
|
||||
```shell
|
||||
kubectl create -f my-crontab.yaml
|
||||
```
|
||||
|
||||
Then new namespaced RESTful API endpoints are created at:
|
||||
|
||||
```
|
||||
/apis/stable.example.com/v1/namespaces/*/crontabs/status
|
||||
```
|
||||
|
||||
and
|
||||
|
||||
```
|
||||
/apis/stable.example.com/v1/namespaces/*/crontabs/scale
|
||||
```
|
||||
|
||||
A custom resource can be scaled using the `kubectl scale` command.
|
||||
For example, the following command sets `.spec.replicas` of the
|
||||
custom resource created above to 5:
|
||||
|
||||
```shell
|
||||
kubectl scale --replicas=5 crontabs/my-new-cron-object
|
||||
crontabs "my-new-cron-object" scaled
|
||||
|
||||
kubectl get crontabs my-new-cron-object -o jsonpath='{.spec.replicas}'
|
||||
5
|
||||
```
|
||||
|
||||
### Categories
|
||||
|
||||
Categories is a list of grouped resources the custom resource belongs to (eg. `all`).
|
||||
You can use `kubectl get <category-name>` to list the resources belonging to the category.
|
||||
This feature is __beta__ and available for custom resources from v1.10.
|
||||
|
||||
The following example adds `all` in the list of categories in the CustomResourceDefinition
|
||||
and illustrates how to output the custom resource using `kubectl get all`.
|
||||
|
||||
Save the following CustomResourceDefinition to `resourcedefinition.yaml`:
|
||||
|
||||
```yaml
|
||||
apiVersion: apiextensions.k8s.io/v1beta1
|
||||
kind: CustomResourceDefinition
|
||||
metadata:
|
||||
name: crontabs.stable.example.com
|
||||
spec:
|
||||
group: stable.example.com
|
||||
version: v1
|
||||
scope: Namespaced
|
||||
names:
|
||||
plural: crontabs
|
||||
singular: crontab
|
||||
kind: CronTab
|
||||
shortNames:
|
||||
- ct
|
||||
# categories is a list of grouped resources the custom resource belongs to.
|
||||
categories:
|
||||
- all
|
||||
```
|
||||
|
||||
And create it:
|
||||
|
||||
```shell
|
||||
kubectl create -f resourcedefinition.yaml
|
||||
```
|
||||
|
||||
After the CustomResourceDefinition object has been created, you can create custom objects.
|
||||
|
||||
Save the following YAML to `my-crontab.yaml`:
|
||||
|
||||
```yaml
|
||||
apiVersion: "stable.example.com/v1"
|
||||
kind: CronTab
|
||||
metadata:
|
||||
name: my-new-cron-object
|
||||
spec:
|
||||
cronSpec: "* * * * */5"
|
||||
image: my-awesome-cron-image
|
||||
```
|
||||
|
||||
and create it:
|
||||
|
||||
```shell
|
||||
kubectl create -f my-crontab.yaml
|
||||
```
|
||||
|
||||
You can specify the category using `kubectl get`:
|
||||
|
||||
```
|
||||
kubectl get all
|
||||
```
|
||||
|
||||
and it will include the custom resources of kind `CronTab`:
|
||||
|
||||
```console
|
||||
NAME AGE
|
||||
crontabs/my-new-cron-object 3s
|
||||
```
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
{% capture whatsnext %}
|
||||
|
||||
@@ -5,7 +5,7 @@ title: Using CoreDNS for Service Discovery
|
||||
min-kubernetes-server-version: v1.9
|
||||
---
|
||||
|
||||
{% include feature-state-alpha.md %}
|
||||
{% include feature-state-beta.md %}
|
||||
|
||||
{% capture overview %}
|
||||
This page describes how to enable CoreDNS instead of kube-dns for service
|
||||
@@ -20,8 +20,9 @@ discovery.
|
||||
|
||||
## Installing CoreDNS with kubeadm
|
||||
|
||||
In Kubernetes 1.9, [CoreDNS](https://coredns.io) is available as an alpha feature and
|
||||
may be installed by setting the `CoreDNS` feature gate to `true` during `kubeadm init`:
|
||||
In Kubernetes 1.9, [CoreDNS](https://coredns.io) is available as an alpha feature, and
|
||||
in Kubernetes 1.10 it is available as a beta feature. In either case, you may install
|
||||
it during cluster creation by setting the `CoreDNS` feature gate to `true` during `kubeadm init`:
|
||||
|
||||
```
|
||||
kubeadm init --feature-gates=CoreDNS=true
|
||||
@@ -29,6 +30,21 @@ kubeadm init --feature-gates=CoreDNS=true
|
||||
|
||||
This installs CoreDNS instead of kube-dns.
|
||||
|
||||
## Upgrading an Existing Cluster with kubeadm
|
||||
|
||||
In Kubernetes 1.10, you can also move to CoreDNS when you use `kubeadm` to upgrade
|
||||
a cluster that is using `kube-dns`. In this case, `kubeadm` will generate the CoreDNS configuration
|
||||
("Corefile") based upon the `kube-dns` ConfigMap, preserving configurations for federation,
|
||||
stub domains, and upstream name server.
|
||||
|
||||
Note that if you are running CoreDNS in your cluster already, prior to upgrade, your existing Corefile will be
|
||||
**overwritten** by the one created during upgrade. **You should save your existing ConfigMap
|
||||
if you have customized it.** You may re-apply your customizations after the new ConfigMap is
|
||||
up and running.
|
||||
|
||||
This process will be modified for the GA release of this feature, such that an existing
|
||||
Corefile will not be overwritten.
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
{% capture whatsnext %}
|
||||
|
||||
@@ -33,9 +33,8 @@ management policies to determine some placement preferences on the node.
|
||||
|
||||
### Configuration
|
||||
|
||||
The CPU Manager is introduced as an alpha feature in Kubernetes v1.8. It
|
||||
must be explicitly enabled in the kubelet feature gates:
|
||||
`--feature-gates=CPUManager=true`.
|
||||
The CPU Manager is an alpha feature in Kubernetes v1.8. It was enabled by
|
||||
default as a beta feature since v1.10.
|
||||
|
||||
The CPU Manager policy is set with the `--cpu-manager-policy` kubelet
|
||||
option. There are two supported policies:
|
||||
|
||||
@@ -78,7 +78,8 @@ Name | Encryption | Strength | Speed | Key Length | Other Considerations
|
||||
`identity` | None | N/A | N/A | N/A | Resources written as-is without encryption. When set as the first provider, the resource will be decrypted as new values are written.
|
||||
`aescbc` | AES-CBC with PKCS#7 padding | Strongest | Fast | 32-byte | The recommended choice for encryption at rest but may be slightly slower than `secretbox`.
|
||||
`secretbox` | XSalsa20 and Poly1305 | Strong | Faster | 32-byte | A newer standard and may not be considered acceptable in environments that require high levels of review.
|
||||
`aesgcm` | AES-GCM with random nonce | Must be rotated every 200k writes | Fastest | 16, 24, or 32-byte | Is not recommended for use except when an automated key rotation scheme is implemented.
|
||||
`aesgcm` | AES-GCM with random nonce | Must be rotated every 200k writes | Fastest | 16, 24, or 32-byte | Is not recommended for use except when an automated key rotation scheme is implemented.
|
||||
`kms` | Uses envelope encryption scheme: Data is encrypted by data encryption keys (DEKs) using AES-CBC with PKCS#7 padding, DEKs are encrypted by key encryption keys (KEKs) according to configuration in Key Management Service (KMS) | Strongest | Fast | 32-bytes | The recommended choice for using a third party tool for key management. Simplifies key rotation, with a new DEK generated for each encryption, and KEK rotation controlled by the user. [Configure the KMS provider](/docs/tasks/administer-cluster/kms-provider/)
|
||||
|
||||
Each provider supports multiple keys - the keys are tried in order for decryption, and if the provider
|
||||
is the first provider, the first key is used for encryption.
|
||||
|
||||
@@ -0,0 +1,181 @@
|
||||
---
|
||||
approvers:
|
||||
- smarterclayton
|
||||
title: Using a KMS provider for data encryption
|
||||
---
|
||||
{% capture overview %}
|
||||
This page shows how to configure a Key Management Service (KMS) provider and plugin to enable secret data encryption.
|
||||
{% endcapture %}
|
||||
|
||||
{% capture prerequisites %}
|
||||
|
||||
* {% include task-tutorial-prereqs.md %}
|
||||
|
||||
* Kubernetes version 1.10.0 or later is required
|
||||
|
||||
* etcd v3 or later is required
|
||||
|
||||
{% assign for_k8s_version="v1.10" %}{% include feature-state-alpha.md %}
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
{% capture steps %}
|
||||
|
||||
The KMS encryption provider uses an envelope encryption scheme to encrypt data in etcd. The data is encrypted using a data encryption key (DEK); a new DEK is generated for each encryption. The DEKs are encrypted with a key encryption key (KEK) that is stored and managed in a remote KMS. The KMS provider uses gRPC to communicate with a specific KMS
|
||||
plugin. The KMS plugin, which is implemented as a gRPC server and deployed on the same host(s) as the Kubernetes master(s), is responsible for all communication with the remote KMS.
|
||||
|
||||
## Configuring the KMS provider
|
||||
|
||||
To configure a KMS provider on the API server, include a provider of type ```kms``` in the providers array in the encryption configuration file and set the following properties:
|
||||
|
||||
* `name`: Display name of the KMS plugin.
|
||||
* `endpoint`: Listen address of the gRPC server (KMS plugin). The endpoint is a UNIX domain socket.
|
||||
* `cachesize`: Number of data encryption keys (DEKs) to be cached in the clear. When cached, DEKs can be used without another call to the KMS; whereas DEKs that are not cached require a call to the KMS to unwrap..
|
||||
|
||||
See [Understanding the encryption at rest configuration.](/docs/tasks/administer-cluster/encrypt-data)
|
||||
|
||||
## Implementing a KMS plugin
|
||||
|
||||
To implement a KMS plugin, you can develop a new plugin gRPC server or enable a KMS plugin already provided by your cloud provider. You then integrate the plugin with the remote KMS and deploy it on the Kubernetes master.
|
||||
|
||||
### Enabling the KMS supported by your cloud provider
|
||||
Refer to your cloud provider for instructions on enabling the cloud provider-specific KMS plugin.
|
||||
|
||||
### Developing a KMS plugin gRPC server
|
||||
You can develop a KMS plugin gRPC server using a stub file available for Go. For other languages, you use a proto file to create a stub file that you can use to develop the gRPC server code.
|
||||
|
||||
* Using Go: Use the functions and data structures in the stub file: [service.pb.go](https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/apiserver/pkg/storage/value/encrypt/envelope/v1beta1/service.pb.go) to develop the gRPC server code
|
||||
|
||||
* Using languages other than Go: Use the protoc compiler with the proto file: [service.proto](https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/apiserver/pkg/storage/value/encrypt/envelope/v1beta1/service.proto) to generate a stub file for the specific language
|
||||
|
||||
Then use the functions and data structures in the stub file to develop the server code.
|
||||
|
||||
**Notes:**
|
||||
|
||||
* kms plugin version: `v1beta1`
|
||||
|
||||
In response to procedure call Version, a compatible KMS plugin should return v1beta1 as VersionResponse.version
|
||||
|
||||
* message version: `v1beta1`
|
||||
|
||||
All messages from KMS provider have the version field set to current version v1beta1
|
||||
|
||||
* protocol: UNIX domain socket (`unix`)
|
||||
|
||||
The gRPC server should listen at UNIX domain socket
|
||||
|
||||
### Integrating a KMS plugin with the remote KMS
|
||||
The KMS plugin can communicate with the remote KMS using any protocol supported by the KMS.
|
||||
All configuration data, including authentication credentials the KMS plugin uses to communicate with the remote KMS,
|
||||
are stored and managed by the KMS plugin independently. The KMS plugin can encode the ciphertext with additional metadata that may be required before sending it to the KMS for decryption.
|
||||
|
||||
### Deploying the KMS plugin
|
||||
Ensure that the KMS plugin runs on the same host(s) as the Kubernetes master(s).
|
||||
|
||||
## Encrypting your data with the KMS provider
|
||||
To encrypt the data:
|
||||
|
||||
1. Create a new encryption configuration file using the appropriate properties for the `kms` provider:
|
||||
|
||||
```yaml
|
||||
kind: EncryptionConfig
|
||||
apiVersion: v1
|
||||
resources:
|
||||
- resources:
|
||||
- secrets
|
||||
providers:
|
||||
- kms:
|
||||
name: myKmsPlugin
|
||||
endpoint: unix:///tmp/socketfile.sock
|
||||
cachesize: 100
|
||||
- identity: {}
|
||||
```
|
||||
|
||||
2. Set the `--experimental-encryption-provider-config` flag on the kube-apiserver to point to the location of the configuration file.
|
||||
3. Restart your API server.
|
||||
|
||||
## Verifying that the data is encrypted
|
||||
Data is encrypted when written to etcd. After restarting your kube-apiserver, any newly created or updated secret should be encrypted when stored. To verify, you can use the etcdctl command line program to retrieve the contents of your secret.
|
||||
|
||||
1. Create a new secret called secret1 in the default namespace:
|
||||
```
|
||||
kubectl create secret generic secret1 -n default --from-literal=mykey=mydata
|
||||
```
|
||||
2. Using the etcdctl command line, read that secret out of etcd:
|
||||
```
|
||||
ETCDCTL_API=3 etcdctl get /kubernetes.io/secrets/default/secret1 [...] | hexdump -C
|
||||
```
|
||||
where `[...]` must be the additional arguments for connecting to the etcd server.
|
||||
|
||||
3. Verify the stored secret is prefixed with `k8s:enc:kms:v1:`, which indicates that the `kms` provider has encrypted the resulting data.
|
||||
|
||||
4. Verify that the secret is correctly decrypted when retrieved via the API:
|
||||
```
|
||||
kubectl describe secret secret1 -n default
|
||||
```
|
||||
should match `mykey: mydata`
|
||||
|
||||
## Ensuring all secrets are encrypted
|
||||
Because secrets are encrypted on write, performing an update on a secret encrypts that content.
|
||||
|
||||
The following command reads all secrets and then updates them to apply server side encryption. If an error occurs due to a conflicting write, retry the command. For larger clusters, you may wish to subdivide the secrets by namespace or script an update.
|
||||
```
|
||||
kubectl get secrets --all-namespaces -o json | kubectl replace -f -
|
||||
```
|
||||
|
||||
## Switching from a local encryption provider to the KMS provider
|
||||
To switch from a local encryption provider to the `kms` provider and re-encrypt all of the secrets:
|
||||
|
||||
1. Add the `kms` provider as the first entry in the configuration file as shown in the following example.
|
||||
|
||||
```yaml
|
||||
kind: EncryptionConfig
|
||||
apiVersion: v1
|
||||
resources:
|
||||
- resources:
|
||||
- secrets
|
||||
providers:
|
||||
- kms:
|
||||
name : myKmsPlugin
|
||||
endpoint: unix:///tmp/socketfile.sock
|
||||
cachesize: 100
|
||||
- aescbc:
|
||||
keys:
|
||||
- name: key1
|
||||
secret: <BASE 64 ENCODED SECRET>
|
||||
```
|
||||
|
||||
2. Restart all kube-apiserver processes.
|
||||
|
||||
3. Run the following command to force all secrets to be re-encrypted using the `kms` provider.
|
||||
|
||||
```
|
||||
kubectl get secrets --all-namespaces -o json| kubectl replace -f -
|
||||
```
|
||||
|
||||
## Disabling encryption at rest
|
||||
To disable encryption at rest:
|
||||
|
||||
1. Place the `identity` provider as the first entry in the configuration file:
|
||||
|
||||
```yaml
|
||||
kind: EncryptionConfig
|
||||
apiVersion: v1
|
||||
resources:
|
||||
- resources:
|
||||
- secrets
|
||||
providers:
|
||||
- identity: {}
|
||||
- kms:
|
||||
name : myKmsPlugin
|
||||
endpoint: unix:///tmp/socketfile.sock
|
||||
cachesize: 100
|
||||
```
|
||||
2. Restart all kube-apiserver processes.
|
||||
3. Run the following command to force all secrets to be decrypted.
|
||||
```
|
||||
kubectl get secrets --all-namespaces -o json | kubectl replace -f -
|
||||
```
|
||||
{% endcapture %}
|
||||
|
||||
{% include templates/task.md %}
|
||||
@@ -6,18 +6,20 @@ title: Set Kubelet parameters via a config file
|
||||
---
|
||||
|
||||
{% capture overview %}
|
||||
{% include feature-state-alpha.md %}
|
||||
{% include feature-state-beta.md %}
|
||||
|
||||
As of Kubernetes 1.8, a subset of the Kubelet's configuration parameters may be
|
||||
set via an on-disk config file, as a substitute for command-line flags. In the
|
||||
future, most of the existing command-line flags will be deprecated in favor of
|
||||
providing parameters via a config file, which simplifies node deployment.
|
||||
A subset of the Kubelet's configuration parameters may be
|
||||
set via an on-disk config file, as a substitute for command-line flags.
|
||||
This functionality is considered beta in v1.10.
|
||||
|
||||
Providing parameters via a config file is the recommended approach because
|
||||
it simplifies node deployment and configuration management.
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
{% capture prerequisites %}
|
||||
|
||||
- A v1.8 or higher Kubelet binary must be installed.
|
||||
- A v1.10 or higher Kubelet binary must be installed for beta functionality.
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
@@ -27,25 +29,42 @@ providing parameters via a config file, which simplifies node deployment.
|
||||
|
||||
The subset of the Kubelet's configuration that can be configured via a file
|
||||
is defined by the `KubeletConfiguration` struct
|
||||
[here (v1alpha1)](https://github.com/kubernetes/kubernetes/blob/release-1.9/pkg/kubelet/apis/kubeletconfig/v1alpha1/types.go).
|
||||
The configuration file must be a JSON or YAML representation of the parameters
|
||||
in this struct. Note that this structure, and thus the config file API,
|
||||
is still considered alpha and is not subject to stability guarantees.
|
||||
[here (v1beta1)](https://github.com/kubernetes/kubernetes/blob/release-1.10/pkg/kubelet/apis/kubeletconfig/v1beta1/types.go).
|
||||
|
||||
Create a file named `kubelet` in its own directory and make sure the directory
|
||||
and file are both readable by the Kubelet. You should write your intended
|
||||
Kubelet configuration in this `kubelet` file.
|
||||
The configuration file must be a JSON or YAML representation of the parameters
|
||||
in this struct. Make sure the Kubelet has read permissions on the file.
|
||||
|
||||
Here is an example of what this file might look like:
|
||||
```
|
||||
kind: KubeletConfiguration
|
||||
apiVersion: kubelet.config.k8s.io/v1beta1
|
||||
evictionHard:
|
||||
memory.available: "200Mi"
|
||||
```
|
||||
|
||||
In the example, the Kubelet is configured to evict Pods when available memory drops below 200Mi.
|
||||
All other Kubelet configuration values are left at their built-in defaults, unless overridden
|
||||
by flags. Command line flags which target the same value as a config file will override that value.
|
||||
|
||||
For a trick to generate a configuration file from a live node, see
|
||||
[Reconfigure a Node's Kubelet in a Live Cluster](/docs/tasks/administer-cluster/reconfigure-kubelet).
|
||||
|
||||
## Start a Kubelet process configured via the config file
|
||||
|
||||
Start the Kubelet with the `KubeletConfigFile` feature gate enabled and the
|
||||
Kubelet's `--init-config-dir` flag set to the location of the directory
|
||||
containing the `kubelet` file. The Kubelet will then load the parameters defined
|
||||
by `KubeletConfiguration` from the `kubelet` file, rather than from their
|
||||
associated command-line flags.
|
||||
Start the Kubelet with the `--config` flag set to the path of the Kubelet's config file.
|
||||
The Kubelet will then load its config from this file.
|
||||
|
||||
Note that command line flags which target the same value as a config file will override that value.
|
||||
This helps ensure backwards compatibility with the command-line API.
|
||||
|
||||
Note that relative file paths in the Kubelet config file are resolved relative to the
|
||||
location of the Kubelet config file, whereas relative paths in command line flags are resolved
|
||||
relative to the Kubelet's current working directory.
|
||||
|
||||
Note that some default values differ between command-line flags and the Kubelet config file.
|
||||
If `--config` is provided and the values are not specified via the command line, the
|
||||
defaults for the `KubeletConfiguration` version apply.
|
||||
In the above example, this version is `kubelet.config.k8s.io/v1beta1`.
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
@@ -54,16 +73,8 @@ associated command-line flags.
|
||||
## Relationship to Dynamic Kubelet Config
|
||||
|
||||
If you are using the [Dynamic Kubelet Configuration](/docs/tasks/administer-cluster/reconfigure-kubelet)
|
||||
feature, the configuration provided via `--init-config-dir` will be considered
|
||||
the "last known good" configuration by the automatic rollback mechanism.
|
||||
|
||||
Note that the layout of the files in the `--init-config-dir` mirrors the layout
|
||||
of data in the ConfigMaps used for Dynamic Kubelet Config; the file names are
|
||||
the same as the keys of the ConfigMap, and the file contents are JSON or YAML
|
||||
representations of the same structures. Today, the only pair is
|
||||
`kubelet:KubeletConfiguration`, though more may emerge in the future.
|
||||
See [Reconfigure a Node's Kubelet in a Live Cluster](/docs/tasks/administer-cluster/reconfigure-kubelet)
|
||||
for more information.
|
||||
feature, the combination of configuration provided via `--config` and any flags which override these values
|
||||
is considered the default "last known good" configuration by the automatic rollback mechanism.
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
|
||||
@@ -42,21 +42,21 @@ The basic workflow for configuring a Kubelet in a live cluster is as follows:
|
||||
1. Write a YAML or JSON configuration file containing the
|
||||
Kubelet's configuration.
|
||||
2. Wrap this file in a ConfigMap and save it to the Kubernetes control plane.
|
||||
3. Update the Kubelet's correspoinding Node object to use this ConfigMap.
|
||||
3. Update the Kubelet's corresponding Node object to use this ConfigMap.
|
||||
|
||||
Each Kubelet watches a configuration reference on its respective Node object.
|
||||
When this reference changes, the Kubelet downloads the new configuration and
|
||||
exits. For the feature to work correctly, you must be running a process manager
|
||||
When this reference changes, the Kubelet downloads the new configuration,
|
||||
updates a local reference to refer to the file, and exits.
|
||||
For the feature to work correctly, you must be running a process manager
|
||||
(like systemd) which will restart the Kubelet when it exits. When the Kubelet is
|
||||
restarted, it will begin using the new configuration.
|
||||
|
||||
The new configuration completely overrides the old configuration; unspecified
|
||||
fields in the new configuration will receive their canonical default values.
|
||||
Some CLI flags do not have an associated configuration field, and will not be
|
||||
affected by the new configuration. These fields are defined by the KubeletFlags
|
||||
structure, [here](https://github.com/kubernetes/kubernetes/blob/master/cmd/kubelet/app/options/options.go).
|
||||
The new configuration completely overrides configuration provided by `--config`,
|
||||
and is overridden by command-line flags. Unspecified values in the new configuration
|
||||
will receive default values appropriate to the configuration version
|
||||
(e.g. `kubelet.config.k8s.io/v1beta1`), unless overridden by flags.
|
||||
|
||||
The status of the Node's Kubelet configuration is reported via the `ConfigOK`
|
||||
The status of the Node's Kubelet configuration is reported via the `KubeletConfigOK`
|
||||
condition in the Node status. Once you have updated a Node to use the new
|
||||
ConfigMap, you can observe this condition to confirm that the Node is using the
|
||||
intended configuration. A table describing the possible conditions can be found
|
||||
@@ -95,13 +95,13 @@ and you will simply edit a copy of this file (which, as a best practice, should
|
||||
live in version control) while creating the first Kubelet ConfigMap. Today,
|
||||
however, the Kubelet is still bootstrapped with command-line flags. Fortunately,
|
||||
there is a dirty trick you can use to generate a config file containing a Node's
|
||||
current configuration. The trick involves hitting the Kubelet server's `configz`
|
||||
current configuration. The trick involves accessing the Kubelet server's `configz`
|
||||
endpoint via the kubectl proxy. This endpoint, in its current implementation, is
|
||||
intended to be used only as a debugging aid, which is part of why this is a
|
||||
dirty trick. There is ongoing work to improve the endpoint, and in the future
|
||||
this will be a less "dirty" operation. This trick also requires the `jq` command
|
||||
to be installed on your machine, for unpacking and editing the JSON response
|
||||
from the endpoint.
|
||||
dirty trick. The endpoint may be improved in the future, but until then
|
||||
it should not be relied on for production scenarios.
|
||||
This trick also requires the `jq` command to be installed on your machine,
|
||||
for unpacking and editing the JSON response from the endpoint.
|
||||
|
||||
Do the following to generate the file:
|
||||
|
||||
@@ -112,12 +112,12 @@ configz endpoint:
|
||||
|
||||
```
|
||||
$ export NODE_NAME=the-name-of-the-node-you-are-reconfiguring
|
||||
$ curl -sSL http://localhost:8001/api/v1/proxy/nodes/${NODE_NAME}/configz | jq '.kubeletconfig|.kind="KubeletConfiguration"|.apiVersion="kubeletconfig/v1alpha1"' > kubelet_configz_${NODE_NAME}
|
||||
$ curl -sSL http://localhost:8001/api/v1/proxy/nodes/${NODE_NAME}/configz | jq '.kubeletconfig|.kind="KubeletConfiguration"|.apiVersion="kubelet.config.k8s.io/v1beta1"' > kubelet_configz_${NODE_NAME}
|
||||
```
|
||||
|
||||
Note that we have to manually add the `kind` and `apiVersion` to the downloaded
|
||||
object, as these are not reported by the configz endpoint. This is one of the
|
||||
limitations of the endpoint that is planned to be fixed in the future.
|
||||
limitations of the endpoint.
|
||||
|
||||
### Edit the configuration file
|
||||
|
||||
@@ -209,29 +209,29 @@ Be sure to specify all three of `name`, `namespace`, and `uid`.
|
||||
### Observe that the Node begins using the new configuration
|
||||
|
||||
Retrieve the Node with `kubectl get node ${NODE_NAME} -o yaml`, and look for the
|
||||
`ConfigOK` condition in `status.conditions`. You should see the message
|
||||
`KubeletConfigOK` condition in `status.conditions`. You should see the message
|
||||
`Using current (UID: CONFIG_MAP_UID)` when the Kubelet starts using the new
|
||||
configuration.
|
||||
|
||||
For convenience, you can use the following command (using `jq`) to filter down
|
||||
to the `ConfigOK` condition:
|
||||
to the `KubeletConfigOK` condition:
|
||||
|
||||
```
|
||||
$ kubectl get no ${NODE_NAME} -o json | jq '.status.conditions|map(select(.type=="ConfigOK"))'
|
||||
$ kubectl get no ${NODE_NAME} -o json | jq '.status.conditions|map(select(.type=="KubeletConfigOK"))'
|
||||
[
|
||||
{
|
||||
"lastHeartbeatTime": "2017-09-20T18:08:29Z",
|
||||
"lastTransitionTime": "2017-09-20T18:08:17Z",
|
||||
"message": "using current (UID: \"2ebc8d1a-9e2a-11e7-a8dd-42010a800006\")",
|
||||
"message": "using current: /api/v1/namespaces/kube-system/configmaps/my-node-config-gkt4c2m4b2",
|
||||
"reason": "passing all checks",
|
||||
"status": "True",
|
||||
"type": "ConfigOK"
|
||||
"type": "KubeletConfigOK"
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
If something goes wrong, you may see one of several different error conditions,
|
||||
detailed in the Table of ConfigOK Conditions, below. When this happens, you
|
||||
detailed in the table of KubeletConfigOK conditions, below. When this happens, you
|
||||
should check the Kubelet's log for more details.
|
||||
|
||||
### Edit the configuration file again
|
||||
@@ -282,16 +282,16 @@ the following, with `name` and `uid` substituted as necessary:
|
||||
```
|
||||
configSource:
|
||||
configMapRef:
|
||||
name: NEW_CONFIG_MAP_NAME
|
||||
name: ${NEW_CONFIG_MAP_NAME}
|
||||
namespace: kube-system
|
||||
uid: NEW_CONFIG_MAP_UID
|
||||
uid: ${NEW_CONFIG_MAP_UID}
|
||||
```
|
||||
|
||||
### Observe that the Kubelet is using the new configuration
|
||||
|
||||
Once more, retrieve the Node with `kubectl get node ${NODE_NAME} -o yaml`, and
|
||||
look for the `ConfigOK` condition in `status.conditions`. You should see the message
|
||||
`Using current (UID: NEW_CONFIG_MAP_UID)` when the Kubelet starts using the
|
||||
look for the `KubeletConfigOK` condition in `status.conditions`. You should see the message
|
||||
`using current: /api/v1/namespaces/kube-system/configmaps/${NEW_CONFIG_MAP_NAME}` when the Kubelet starts using the
|
||||
new configuration.
|
||||
|
||||
### Deauthorize your Node fom reading the old ConfigMap
|
||||
@@ -327,9 +327,8 @@ remove the `spec.configSource` subfield.
|
||||
|
||||
### Observe that the Node is using its local default configuration
|
||||
|
||||
After removing this subfield, you should eventually observe that the ConfigOK
|
||||
condition's message reverts to either `using current (default)` or
|
||||
`using current (init)`, depending on how the Node was provisioned.
|
||||
After removing this subfield, you should eventually observe that the KubeletConfigOK
|
||||
condition's message reverts to `using current: local`.
|
||||
|
||||
### Deauthorize your Node fom reading the old ConfigMap
|
||||
|
||||
@@ -366,9 +365,9 @@ Here is an example command that uses `kubectl patch`:
|
||||
kubectl patch node ${NODE_NAME} -p "{\"spec\":{\"configSource\":{\"configMapRef\":{\"name\":\"${CONFIG_MAP_NAME}\",\"namespace\":\"kube-system\",\"uid\":\"${CONFIG_MAP_UID}\"}}}}"
|
||||
```
|
||||
|
||||
## Understanding ConfigOK Conditions
|
||||
## Understanding KubeletConfigOK Conditions
|
||||
|
||||
The following table describes several of the `ConfigOK` Node conditions you
|
||||
The following table describes several of the `KubeletConfigOK` Node conditions you
|
||||
might encounter in a cluster that has Dynamic Kubelet Config enabled. If you
|
||||
observe a condition with `status=False`, you should check the Kubelet log for
|
||||
more error details by searching for the message or reason text.
|
||||
@@ -383,49 +382,33 @@ more error details by searching for the message or reason text.
|
||||
<th>Status</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>using current (default)</p></td>
|
||||
<td><p>current is set to the local default, and no init config was provided</p></td>
|
||||
<td><p>using current: local</p></td>
|
||||
<td><p>when the config source is nil, the Kubelet uses its local config</p></td>
|
||||
<td><p>True</p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>using current (init)</p></td>
|
||||
<td><p>current is set to the local default, and an init config was provided</p></td>
|
||||
<td><p>True</p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>using current (UID: CURRENT_CONFIG_MAP_UID)</p></td>
|
||||
<td><p>using current: /api/v1/namespaces/${CURRENT_CONFIG_MAP_NAMESPACE}/configmaps/${CURRENT_CONFIG_MAP_NAME}</p></td>
|
||||
<td><p>passing all checks</p></td>
|
||||
<td><p>True</p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>using last-known-good (default)</p></td>
|
||||
<td><p>using last-known-good: local</p></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li>failed to load current (UID: CURRENT_CONFIG_MAP_UID)</li>
|
||||
<li>failed to parse current (UID: CURRENT_CONFIG_MAP_UID)</li>
|
||||
<li>failed to validate current (UID: CURRENT_CONFIG_MAP_UID)</li>
|
||||
<li>failed to load current: /api/v1/namespaces/${CURRENT_CONFIG_MAP_NAMESPACE}/configmaps/${CURRENT_CONFIG_MAP_NAME}</li>
|
||||
<li>failed to parse current: /api/v1/namespaces/${CURRENT_CONFIG_MAP_NAMESPACE}/configmaps/${CURRENT_CONFIG_MAP_NAME}</li>
|
||||
<li>failed to validate current: /api/v1/namespaces/${CURRENT_CONFIG_MAP_NAMESPACE}/configmaps/${CURRENT_CONFIG_MAP_NAME}</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><p>False</p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>using last-known-good (init)</p></td>
|
||||
<td><p>using last-known-good: /api/v1/namespaces/${LAST_KNOWN_GOOD_CONFIG_MAP_NAMESPACE}/configmaps/${LAST_KNOWN_GOOD_CONFIG_MAP_NAME}</p></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li>failed to load current (UID: CURRENT_CONFIG_MAP_UID)</li>
|
||||
<li>failed to parse current (UID: CURRENT_CONFIG_MAP_UID)</li>
|
||||
<li>failed to validate current (UID: CURRENT_CONFIG_MAP_UID)</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><p>False</p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>using last-known-good (UID: LAST_KNOWN_GOOD_CONFIG_MAP_UID)</p></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li>failed to load current (UID: CURRENT_CONFIG_MAP_UID)</li>
|
||||
<li>failed to parse current (UID: CURRENT_CONFIG_MAP_UID)</li>
|
||||
<li>failed to validate current (UID: CURRENT_CONFIG_MAP_UID)</li>
|
||||
<li>failed to load current: /api/v1/namespaces/${CURRENT_CONFIG_MAP_NAMESPACE}/configmaps/${CURRENT_CONFIG_MAP_NAME}</li>
|
||||
<li>failed to parse current: /api/v1/namespaces/${CURRENT_CONFIG_MAP_NAMESPACE}/configmaps/${CURRENT_CONFIG_MAP_NAME}</li>
|
||||
<li>failed to validate current: /api/v1/namespaces/${CURRENT_CONFIG_MAP_NAMESPACE}/configmaps/${CURRENT_CONFIG_MAP_NAME}</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><p>False</p></td>
|
||||
@@ -451,15 +434,15 @@ more error details by searching for the message or reason text.
|
||||
<p>failed to sync, reason:</p>
|
||||
<ul>
|
||||
<li>failed to read Node from informer object cache</li>
|
||||
<li>failed to reset to local (default or init) config</li>
|
||||
<li>failed to reset to local config</li>
|
||||
<li>invalid NodeConfigSource, exactly one subfield must be non-nil, but all were nil</li>
|
||||
<li>invalid ObjectReference, all of UID, Name, and Namespace must be specified</li>
|
||||
<li>invalid ObjectReference, UID SOME_UID does not match UID of downloaded ConfigMap SOME_OTHER_UID</li>
|
||||
<li>failed to determine whether object with UID SOME_UID was already checkpointed</li>
|
||||
<li>failed to download ConfigMap with name SOME_NAME from namespace SOME_NAMESPACE</li>
|
||||
<li>failed to save config checkpoint for object with UID SOME_UID</li>
|
||||
<li>failed to set current config checkpoint to default</li>
|
||||
<li>failed to set current config checkpoint to object with UID SOME_UID</li>
|
||||
<li>invalid ConfigSource.ConfigMapRef.UID: ${UID} does not match ${API_PATH}.UID: ${UID_OF_CONFIG_MAP_AT_API_PATH}</li>
|
||||
<li>failed to determine whether object ${API_PATH} with UID ${UID} was already checkpointed</li>
|
||||
<li>failed to download ConfigMap with name ${NAME} from namespace ${NAMESPACE}</li>
|
||||
<li>failed to save config checkpoint for object ${API_PATH} with UID ${UID}</li>
|
||||
<li>failed to set current config checkpoint to local config</li>
|
||||
<li>failed to set current config checkpoint to object ${API_PATH} with UID ${UID}</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><p>False</p></td>
|
||||
|
||||
@@ -33,7 +33,7 @@ Successfully running cloud-controller-manager requires some changes to your clus
|
||||
|
||||
* `kube-apiserver` and `kube-controller-manager` MUST NOT specify the `--cloud-provider` flag. This ensures that it does not run any cloud specific loops that would be run by cloud controller manager. In the future, this flag will be deprecated and removed.
|
||||
* `kubelet` must run with `--cloud-provider=external`. This is to ensure that the kubelet is aware that it must be initialized by the cloud controller manager before it is scheduled any work.
|
||||
* `kube-apiserver` SHOULD NOT run the `PersistentVolumeLabel` admission controller since the cloud controller manager takes over labeling persistent volumes. To prevent the PersistentVolumeLabel admission plugin from running, make sure the `kube-apiserver` has a `--admission-control` flag with a value that does not include `PersistentVolumeLabel`.
|
||||
* `kube-apiserver` SHOULD NOT run the `PersistentVolumeLabel` admission controller since the cloud controller manager takes over labeling persistent volumes. To prevent the PersistentVolumeLabel admission plugin from running in `kube-apiserver`, include the `PersistentVolumeLabel` as a listed value in the `--disable-admission-plugins` flag.
|
||||
* For the `cloud-controller-manager` to label persistent volumes, initializers will need to be enabled and an InitializerConifguration needs to be added to the system. Follow [these instructions](/docs/admin/extensible-admission-controllers.md#enable-initializers-alpha-feature) to enable initializers. Use the following YAML to create the InitializerConfiguration:
|
||||
|
||||
{% include code.html language="yaml" file="persistent-volume-label-initializer-config.yaml" ghlink="/docs/tasks/administer-cluster/persistent-volume-label-initializer-config.yaml" %}
|
||||
|
||||
@@ -0,0 +1,315 @@
|
||||
---
|
||||
approvers:
|
||||
- msau42
|
||||
- jsafrane
|
||||
title: Storage Object in Use Protection
|
||||
---
|
||||
|
||||
{% capture overview %}
|
||||
{% assign for_k8s_version="v1.10" %}{% include feature-state-beta.md %}
|
||||
|
||||
Persistent volume claims (PVCs) that are in active use by a pod and persistent volumes (PVs) that are bound to PVCs can be protected from pre-mature removal.
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
{% capture prerequisites %}
|
||||
|
||||
- The Storage Object in Use Protection feature is enabled in a version of Kubernetes in which it is supported.
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
{% capture steps %}
|
||||
|
||||
## Storage Object in Use Protection feature used for PVC Protection
|
||||
|
||||
The example below uses a GCE PD `StorageClass`, however, similar steps can be performed for any volume type.
|
||||
|
||||
Create a `StorageClass` for convenient storage provisioning:
|
||||
```yaml
|
||||
apiVersion: storage.k8s.io/v1
|
||||
kind: StorageClass
|
||||
metadata:
|
||||
name: slow
|
||||
provisioner: kubernetes.io/gce-pd
|
||||
parameters:
|
||||
type: pd-standard
|
||||
```
|
||||
|
||||
Verification scenarios follow below.
|
||||
|
||||
### Scenario 1: The PVC is not in active use by a pod
|
||||
|
||||
- Create a PVC:
|
||||
|
||||
```yaml
|
||||
kind: PersistentVolumeClaim
|
||||
apiVersion: v1
|
||||
metadata:
|
||||
name: slzc
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
storageClassName: slow
|
||||
resources:
|
||||
requests:
|
||||
storage: 3.7Gi
|
||||
```
|
||||
|
||||
- Check that the PVC has the finalizer `kubernetes.io/pvc-protection` set:
|
||||
|
||||
```shell
|
||||
kubectl describe pvc slzc
|
||||
Name: slzc
|
||||
Namespace: default
|
||||
StorageClass: slow
|
||||
Status: Bound
|
||||
Volume: pvc-bee8c30a-d6a3-11e7-9af0-42010a800002
|
||||
Labels: <none>
|
||||
Annotations: pv.kubernetes.io/bind-completed=yes
|
||||
pv.kubernetes.io/bound-by-controller=yes
|
||||
volume.beta.kubernetes.io/storage-provisioner=kubernetes.io/gce-pd
|
||||
Finalizers: [kubernetes.io/pvc-protection]
|
||||
Capacity: 4Gi
|
||||
Access Modes: RWO
|
||||
Events:
|
||||
Type Reason Age From Message
|
||||
---- ------ ---- ---- -------
|
||||
Normal ProvisioningSucceeded 2m persistentvolume-controller Successfully provisioned volume pvc-bee8c30a-d6a3-11e7-9af0-42010a800002 using kubernetes.io/gce-pd
|
||||
```
|
||||
|
||||
- Delete the PVC and check that the PVC (not in active use by a pod) was removed successfully.
|
||||
|
||||
### Scenario 2: The PVC is in active use by a pod
|
||||
|
||||
- Again, create the same PVC.
|
||||
- Create a pod that uses the PVC:
|
||||
|
||||
```yaml
|
||||
kind: Pod
|
||||
apiVersion: v1
|
||||
metadata:
|
||||
name: app1
|
||||
spec:
|
||||
containers:
|
||||
- name: test-pod
|
||||
image: k8s.gcr.io/busybox:1.24
|
||||
command:
|
||||
- "/bin/sh"
|
||||
args:
|
||||
- "-c"
|
||||
- "date > /mnt/app1.txt; sleep 60 && exit 0 || exit 1"
|
||||
volumeMounts:
|
||||
- name: path-pvc
|
||||
mountPath: "/mnt"
|
||||
restartPolicy: "Never"
|
||||
volumes:
|
||||
- name: path-pvc
|
||||
persistentVolumeClaim:
|
||||
claimName: slzc
|
||||
```
|
||||
|
||||
- Wait until the pod status is `Running`, i.e. the PVC becomes in active use.
|
||||
- Delete the PVC that is now in active use by a pod and verify that the PVC is not removed but its status is `Terminating`:
|
||||
|
||||
```shell
|
||||
Name: slzc
|
||||
Namespace: default
|
||||
StorageClass: slow
|
||||
Status: Terminating (since Fri, 01 Dec 2017 14:47:55 +0000)
|
||||
Volume: pvc-803a1f4d-d6a6-11e7-9af0-42010a800002
|
||||
Labels: <none>
|
||||
Annotations: pv.kubernetes.io/bind-completed=yes
|
||||
pv.kubernetes.io/bound-by-controller=yes
|
||||
volume.beta.kubernetes.io/storage-provisioner=kubernetes.io/gce-pd
|
||||
Finalizers: [kubernetes.io/pvc-protection]
|
||||
Capacity: 4Gi
|
||||
Access Modes: RWO
|
||||
Events:
|
||||
Type Reason Age From Message
|
||||
---- ------ ---- ---- -------
|
||||
Normal ProvisioningSucceeded 52s persistentvolume-controller Successfully provisioned volume pvc-803a1f4d-d6a6-11e7-9af0-42010a800002 using kubernetes.io/gce-pd
|
||||
```
|
||||
- Wait until the pod status is `Terminated` (either delete the pod or wait until it finishes). Afterwards, check that the PVC is removed.
|
||||
|
||||
### Scenario 3: A pod starts using a PVC that is in Terminating state
|
||||
|
||||
- Again, create the same PVC.
|
||||
- Create a first pod that uses the PVC:
|
||||
|
||||
```yaml
|
||||
kind: Pod
|
||||
apiVersion: v1
|
||||
metadata:
|
||||
name: app1
|
||||
spec:
|
||||
containers:
|
||||
- name: test-pod
|
||||
image: k8s.gcr.io/busybox:1.24
|
||||
command:
|
||||
- "/bin/sh"
|
||||
args:
|
||||
- "-c"
|
||||
- "date > /mnt/app1.txt; sleep 600 && exit 0 || exit 1"
|
||||
volumeMounts:
|
||||
- name: path-pvc
|
||||
mountPath: "/mnt"
|
||||
restartPolicy: "Never"
|
||||
volumes:
|
||||
- name: path-pvc
|
||||
persistentVolumeClaim:
|
||||
claimName: slzc
|
||||
```
|
||||
|
||||
- Wait until the pod status is `Running`, i.e. the PVC becomes in active use.
|
||||
- Delete the PVC that is now in active use by a pod and verify that the PVC is not removed but its status is `Terminating`:
|
||||
|
||||
```shell
|
||||
Name: slzc
|
||||
Namespace: default
|
||||
StorageClass: slow
|
||||
Status: Terminating (since Fri, 01 Dec 2017 14:47:55 +0000)
|
||||
Volume: pvc-803a1f4d-d6a6-11e7-9af0-42010a800002
|
||||
Labels: <none>
|
||||
Annotations: pv.kubernetes.io/bind-completed=yes
|
||||
pv.kubernetes.io/bound-by-controller=yes
|
||||
volume.beta.kubernetes.io/storage-provisioner=kubernetes.io/gce-pd
|
||||
Finalizers: [kubernetes.io/pvc-protection]
|
||||
Capacity: 4Gi
|
||||
Access Modes: RWO
|
||||
Events:
|
||||
Type Reason Age From Message
|
||||
---- ------ ---- ---- -------
|
||||
Normal ProvisioningSucceeded 52s persistentvolume-controller Successfully provisioned volume pvc-803a1f4d-d6a6-11e7-9af0-42010a800002 using kubernetes.io/gce-pd
|
||||
```
|
||||
|
||||
- Create a second pod that uses the same PVC:
|
||||
|
||||
```
|
||||
kind: Pod
|
||||
apiVersion: v1
|
||||
metadata:
|
||||
name: app2
|
||||
spec:
|
||||
containers:
|
||||
- name: test-pod
|
||||
image: gcr.io/google_containers/busybox:1.24
|
||||
command:
|
||||
- "/bin/sh"
|
||||
args:
|
||||
- "-c"
|
||||
- "date > /mnt/app1.txt; sleep 600 && exit 0 || exit 1"
|
||||
volumeMounts:
|
||||
- name: path-pvc
|
||||
mountPath: "/mnt"
|
||||
restartPolicy: "Never"
|
||||
volumes:
|
||||
- name: path-pvc
|
||||
persistentVolumeClaim:
|
||||
claimName: slzc
|
||||
```
|
||||
|
||||
- Verify that the scheduling of the second pod fails with the below warning:
|
||||
|
||||
```
|
||||
Warning FailedScheduling 18s (x4 over 21s) default-scheduler persistentvolumeclaim "slzc" is being deleted
|
||||
```
|
||||
|
||||
- Wait until the pod status of both pods is `Terminated` or `Completed` (either delete the pods or wait until they finish). Afterwards, check that the PVC is removed.
|
||||
|
||||
## Storage Object in Use Protection feature used for PV Protection
|
||||
|
||||
The example below uses a `HostPath` PV.
|
||||
|
||||
Verification scenarios follow below.
|
||||
|
||||
### Scenario 1: The PV is not bound to a PVC
|
||||
|
||||
- Create a PV:
|
||||
|
||||
```yaml
|
||||
kind: PersistentVolume
|
||||
apiVersion: v1
|
||||
metadata:
|
||||
name: task-pv-volume
|
||||
labels:
|
||||
type: local
|
||||
spec:
|
||||
capacity:
|
||||
storage: 1Gi
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
persistentVolumeReclaimPolicy: Delete
|
||||
storageClassName: standard
|
||||
hostPath:
|
||||
path: "/tmp/data"
|
||||
```
|
||||
|
||||
- Check that the PV has the finalizer `kubernetes.io/pv-protection` set:
|
||||
|
||||
```shell
|
||||
Name: task-pv-volume
|
||||
Labels: type=local
|
||||
Annotations: pv.kubernetes.io/bound-by-controller=yes
|
||||
Finalizers: [kubernetes.io/pv-protection]
|
||||
StorageClass: standard
|
||||
Status: Terminating (lasts 1m)
|
||||
Claim: default/task-pv-claim
|
||||
Reclaim Policy: Delete
|
||||
Access Modes: RWO
|
||||
Capacity: 1Gi
|
||||
Message:
|
||||
Source:
|
||||
Type: HostPath (bare host directory volume)
|
||||
Path: /tmp/data
|
||||
HostPathType:
|
||||
Events: <none>
|
||||
```
|
||||
|
||||
- Delete the PV and check that the PV (not bound to a PVC) is removed successfully.
|
||||
|
||||
### Scenario 2: The PV is bound to a PVC
|
||||
|
||||
- Again, create the same PV.
|
||||
|
||||
- Create a PVC
|
||||
|
||||
```yaml
|
||||
kind: PersistentVolumeClaim
|
||||
apiVersion: v1
|
||||
metadata:
|
||||
name: task-pv-claim
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
resources:
|
||||
requests:
|
||||
storage: 1Gi
|
||||
```
|
||||
|
||||
- Wait until the PV and PVC are bound to each other.
|
||||
- Delete the PV and verify that the PV is not removed but its status is `Terminating`:
|
||||
|
||||
```shell
|
||||
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
|
||||
task-pv-volume 1Gi RWO Delete Terminating default/task-pv-claim standard 59s
|
||||
|
||||
```
|
||||
- Delete the PVC and verify that the PV is removed too.
|
||||
|
||||
```shell
|
||||
kubectl delete pvc task-pv-claim
|
||||
persistentvolumeclaim "task-pv-claim" deleted
|
||||
$ kubectl get pvc
|
||||
No resources found.
|
||||
$ kubectl get pv
|
||||
No resources found.
|
||||
```
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
{% capture discussion %}
|
||||
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
{% include templates/task.md %}
|
||||
@@ -178,7 +178,7 @@ myregistrykey kubernetes.io/.dockerconfigjson 1 1d
|
||||
Next, modify the default service account for the namespace to use this secret as an imagePullSecret.
|
||||
|
||||
```shell
|
||||
kubectl patch serviceaccount default -p '{"imagePullSecrets": [{"name": "myregistrykey"}]}'
|
||||
kubectl patch serviceaccount default -p '{\"imagePullSecrets\": [{\"name\": \"acrkey\"}]}'
|
||||
```
|
||||
|
||||
Interactive version requiring manual edit:
|
||||
|
||||
@@ -0,0 +1,111 @@
|
||||
---
|
||||
title: Share Process Namespace between Containers in a Pod
|
||||
min-kubernetes-server-version: v1.10
|
||||
approvers:
|
||||
- verb
|
||||
- yujuhong
|
||||
- dchen1107
|
||||
---
|
||||
|
||||
{% capture overview %}
|
||||
|
||||
{% include feature-state-alpha.md %}
|
||||
|
||||
This page shows how to configure process namespace sharing for a pod. When
|
||||
process namespace sharing is enabled, processes in a container are visible
|
||||
to all other containers in that pod.
|
||||
|
||||
You can use this feature to configure cooperating containers, such as a log
|
||||
handler sidecar container, or to troubleshoot container images that don't
|
||||
include debugging utilities like a shell.
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
{% capture prerequisites %}
|
||||
|
||||
{% include task-tutorial-prereqs.md %}
|
||||
|
||||
A special **alpha** feature gate `PodShareProcessNamespace` must be set to true
|
||||
across the system: `--feature-gates=PodShareProcessNamespace=true`.
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
{% capture steps %}
|
||||
|
||||
## Configure a Pod
|
||||
|
||||
Process Namespace Sharing is enabled using the `ShareProcessNamespace` field of
|
||||
`v1.PodSpec`. For example:
|
||||
|
||||
{% include code.html language="yaml" file="share-process-namespace.yaml" ghlink="/docs/tasks/configure-pod-container/share-process-namespace.yaml" %}
|
||||
|
||||
1. Create the pod `nginx` on your cluster:
|
||||
|
||||
$ kubectl create -f https://k8s.io/docs/tasks/configure-pod-container/share-process-namespace.yaml
|
||||
|
||||
1. Attach to the `shell` container and run `ps`:
|
||||
|
||||
$ kubectl attach -it nginx -c shell
|
||||
If you don't see a command prompt, try pressing enter.
|
||||
/ # ps ax
|
||||
PID USER TIME COMMAND
|
||||
1 root 0:00 /pause
|
||||
8 root 0:00 nginx: master process nginx -g daemon off;
|
||||
14 101 0:00 nginx: worker process
|
||||
15 root 0:00 sh
|
||||
21 root 0:00 ps ax
|
||||
|
||||
You can signal processes in other containers. For example, send `SIGHUP` to
|
||||
nginx to restart the worker process. This requires the `SYS_PTRACE` capability.
|
||||
|
||||
/ # kill -HUP 8
|
||||
/ # ps ax
|
||||
PID USER TIME COMMAND
|
||||
1 root 0:00 /pause
|
||||
8 root 0:00 nginx: master process nginx -g daemon off;
|
||||
15 root 0:00 sh
|
||||
22 101 0:00 nginx: worker process
|
||||
23 root 0:00 ps ax
|
||||
|
||||
It's even possible to access another container image using the
|
||||
`/proc/$pid/root` link.
|
||||
|
||||
/ # head /proc/8/root/etc/nginx/nginx.conf
|
||||
|
||||
user nginx;
|
||||
worker_processes 1;
|
||||
|
||||
error_log /var/log/nginx/error.log warn;
|
||||
pid /var/run/nginx.pid;
|
||||
|
||||
|
||||
events {
|
||||
worker_connections 1024;
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
{% capture discussion %}
|
||||
|
||||
## Understanding Process Namespace Sharing
|
||||
|
||||
Pods share many resources so it makes sense they would also share a process
|
||||
namespace. Some container images may expect to be isolated from other
|
||||
containers, though, so it's important to understand these differences:
|
||||
|
||||
1. **The container process no longer has PID 1.** Some container images refuse
|
||||
to start without PID 1 (for example, containers using `systemd`) or run
|
||||
commands like `kill -HUP 1` to signal the container process. In pods with a
|
||||
shared process namespace, `kill -HUP 1` will signal the pod sandbox.
|
||||
(`/pause` in the above example.)
|
||||
|
||||
1. **Processes are visible to other containers in the pod.** This includes all
|
||||
information visible in `/proc`, such as passwords that were passed as arguments
|
||||
or environment variables. These are protected only by regular Unix permissions.
|
||||
|
||||
1. **Container filesystems are visible to other containers in the pod through the
|
||||
`/proc/$pid/root` link.** This makes debugging easier, but it also means
|
||||
that filesystem secrets are protected only by filesystem permissions.
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
{% include templates/task.md %}
|
||||
@@ -0,0 +1,17 @@
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: nginx
|
||||
spec:
|
||||
shareProcessNamespace: true
|
||||
containers:
|
||||
- name: nginx
|
||||
image: nginx
|
||||
- name: shell
|
||||
image: busybox
|
||||
securityContext:
|
||||
capabilities:
|
||||
add:
|
||||
- SYS_PTRACE
|
||||
stdin: true
|
||||
tty: true
|
||||
@@ -29,6 +29,10 @@ of its execution generates an event, which is then pre-processed according to
|
||||
a certain policy and written to a backend. You can find more details about the
|
||||
pipeline in the [design proposal][auditing-proposal].
|
||||
|
||||
**Note,** that audit logging feature increases apiserver memory consumption, since some context
|
||||
required for auditing is stored for each request. Additionally, memory consumption depends on the
|
||||
audit logging configuration.
|
||||
|
||||
## Audit Policy
|
||||
|
||||
Audit policy defines rules about what events should be recorded and what data
|
||||
@@ -72,6 +76,24 @@ In both cases, audit events structure is defined by the API in the
|
||||
`audit.k8s.io` API group. The current version of the API is
|
||||
[`v1beta1`][auditing-api].
|
||||
|
||||
**Note:** In case of patches, request body is a JSON array with patch operations, not a JSON object
|
||||
with an appropriate Kubernetes API object. For example, the following request body is a valid patch
|
||||
request to `/apis/batch/v1/namespaces/some-namespace/jobs/some-job-name`.
|
||||
|
||||
```json
|
||||
[
|
||||
{
|
||||
"op": "replace",
|
||||
"path": "/spec/parallelism",
|
||||
"value": 0
|
||||
},
|
||||
{
|
||||
"op": "remove",
|
||||
"path": "/spec/template/spec/containers/0/terminationMessagePolicy"
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
### Log backend
|
||||
|
||||
Log backend writes audit events to a file in JSON format. You can configure
|
||||
@@ -91,14 +113,62 @@ audit backend using the following kube-apiserver flags:
|
||||
|
||||
- `--audit-webhook-config-file` specifies the path to a file with a webhook
|
||||
configuration. Webhook configuration is effectively a [kubeconfig][kubeconfig].
|
||||
- `--audit-webhook-mode` define the buffering strategy, one of the following:
|
||||
- `batch` - buffer events and asynchronously send the set of events to the external service
|
||||
This is the default
|
||||
- `blocking` - block API server responses on sending each event to the external service
|
||||
- `--audit-webhook-initial-backoff` specifies the amount of time to wait after the first failed
|
||||
request before retrying. Subsequent requests are retried with exponential backoff.
|
||||
|
||||
The webhook config file uses the kubeconfig format to specify the remote address of
|
||||
the service and credentials used to connect to it.
|
||||
|
||||
### Batching
|
||||
|
||||
Both log and webhook backends support batching. Using webhook as an example, here's the list of
|
||||
available flags. To get the same flag for log backend, replace `webhook` with `log` in the flag
|
||||
name. By default, batching is enabled in `webhook` and disabled in `log`. Similarly, by default
|
||||
throttling is enabled in `webhook` and disabled in `log`.
|
||||
|
||||
- `--audit-webhook-mode` defines the buffering strategy. One of the following:
|
||||
- `batch` - buffer events and asynchronously process them in batches. This is the default.
|
||||
- `blocking` - block API server responses on processing each individual event.
|
||||
|
||||
The following flags are used only in the `batch` mode.
|
||||
|
||||
- `--audit-webhook-batch-buffer-size` defines the number of events to buffer before batching.
|
||||
If the rate of incoming events overflows the buffer, events are dropped.
|
||||
- `--audit-webhook-batch-max-size` defines the maximum number of events in one batch.
|
||||
- `--audit-webhook-batch-max-wait` defines the maximum amount of time to wait before unconditionally
|
||||
batching events in the queue.
|
||||
- `--audit-webhook-batch-throttle-qps` defines the maximum average number of batches generated
|
||||
per second.
|
||||
- `--audit-webhook-batch-throttle-burst` defines the maximum number of batches generated at the same
|
||||
moment if the allowed QPS was underutilized previously.
|
||||
|
||||
#### Parameter tuning
|
||||
|
||||
Parameters should be set to accommodate the load on the apiserver.
|
||||
|
||||
For example, if kube-apiserver receives 100 requests each second, and each request is audited only
|
||||
on `ResponseStarted` and `ResponseComplete` stages, you should account for ~200 audit
|
||||
events being generated each second. Assuming that there are up to 100 events in a batch,
|
||||
you should set throttling level at at least 2 QPS. Assuming that the backend can take up to
|
||||
5 seconds to write events, you should set the buffer size to hold up to 5 seconds of events, i.e.
|
||||
10 batches, i.e. 1000 events.
|
||||
|
||||
In most cases however, the default parameters should be sufficient and you don't have to worry about
|
||||
setting them manually. You can look at the following Prometheus metrics exposed by kube-apiserver
|
||||
and in the logs to monitor the state of the auditing subsystem.
|
||||
|
||||
- `apiserver_audit_event_total` metric contains the total number of audit events exported.
|
||||
- `apiserver_audit_error_total` metric contains the total number of events dropped due to an error
|
||||
during exporting.
|
||||
|
||||
## Multi-cluster setup
|
||||
|
||||
If you're extending the Kubernetes API with the [aggregation layer][kube-aggregator], you can also
|
||||
set up audit logging for the aggregated apiserver. To do this, pass the configuration options in the
|
||||
same format as described above to the aggregated apiserver and set up the log ingesting pipeline
|
||||
to pick up audit logs. Different apiservers can have different audit configurations and different
|
||||
audit policies.
|
||||
|
||||
## Log Collector Examples
|
||||
|
||||
### Use fluentd to collect and distribute audit events from log file
|
||||
@@ -250,8 +320,8 @@ plugin which supports full-text search and analytics.
|
||||
|
||||
## Legacy Audit
|
||||
|
||||
__Note:__ Legacy Audit is deprecated and is disabled by default since Kubernetes 1.8.
|
||||
To fallback to this legacy audit, disable the advanced auditing feature
|
||||
__Note:__ Legacy Audit is deprecated and is disabled by default since Kubernetes 1.8. Legacy Audit
|
||||
will be removed in 1.12. To fallback to this legacy audit, disable the advanced auditing feature
|
||||
using the `AdvancedAuditing` feature gate in [kube-apiserver][kube-apiserver]:
|
||||
|
||||
```
|
||||
@@ -299,3 +369,4 @@ and `audit-log-maxage` options.
|
||||
[fluentd_install_doc]: http://docs.fluentd.org/v0.12/articles/quickstart#step1-installing-fluentd
|
||||
[logstash]: https://www.elastic.co/products/logstash
|
||||
[logstash_install_doc]: https://www.elastic.co/guide/en/logstash/current/installing-logstash.html
|
||||
[kube-aggregator]: /docs/concepts/api-extension/apiserver-aggregation
|
||||
|
||||
@@ -51,10 +51,10 @@ Admission Controller.
|
||||
|
||||
Update the Federation API server command line arguments to enable the Admission
|
||||
Controller and mount the ConfigMap into the container. If there's an existing
|
||||
`--admission-control` flag, append `,SchedulingPolicy` instead of adding
|
||||
`--enable-admission-plugins` flag, append `,SchedulingPolicy` instead of adding
|
||||
another line.
|
||||
|
||||
--admission-control=SchedulingPolicy
|
||||
--enable-admission-plugins=SchedulingPolicy
|
||||
--admission-control-config-file=/etc/kubernetes/admission/config.yml
|
||||
|
||||
Add the following volume to the Federation API server pod:
|
||||
|
||||
@@ -85,7 +85,7 @@ do not care to see.)
|
||||
We can check on the pods as well using the same label selector:
|
||||
|
||||
```shell
|
||||
$ kubectl get pods -l jobgroup=jobexample --show-all
|
||||
$ kubectl get pods -l jobgroup=jobexample
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
process-item-apple-kixwv 0/1 Completed 0 4m
|
||||
process-item-banana-wrsf7 0/1 Completed 0 4m
|
||||
@@ -96,7 +96,7 @@ There is not a single command to check on the output of all jobs at once,
|
||||
but looping over all the pods is pretty easy:
|
||||
|
||||
```shell
|
||||
$ for p in $(kubectl get pods -l jobgroup=jobexample --show-all -o name)
|
||||
$ for p in $(kubectl get pods -l jobgroup=jobexample -o name)
|
||||
do
|
||||
kubectl logs $p
|
||||
done
|
||||
@@ -184,11 +184,6 @@ If you have a large number of job objects, you may find that:
|
||||
- Even using labels, managing so many Job objects is cumbersome.
|
||||
- You exceed resource quota when creating all the Jobs at once,
|
||||
and do not want to wait to create them incrementally.
|
||||
- You need a way to easily scale the number of pods running
|
||||
concurrently. One reason would be to avoid using too many
|
||||
compute resources. Another would be to limit the number of
|
||||
concurrent requests to a shared resource, such as a database,
|
||||
used by all the pods in the job.
|
||||
- Very large numbers of jobs created at once overload the
|
||||
Kubernetes apiserver, controller, or scheduler.
|
||||
|
||||
|
||||
@@ -14,9 +14,10 @@ consume GPUs across different Kubernetes versions and the current limitations.
|
||||
**From 1.8 onwards, the recommended way to consume GPUs is to use [device
|
||||
plugins](/docs/concepts/cluster-administration/device-plugins).**
|
||||
|
||||
To enable GPU support through device plugins, a special **alpha** feature gate
|
||||
`DevicePlugins` has to be set to true across the system:
|
||||
`--feature-gates="DevicePlugins=true"`.
|
||||
To enable GPU support through device plugins before 1,10, the `DevicePlugins`
|
||||
feature gate has to be explicitly set to true across the system:
|
||||
`--feature-gates="DevicePlugins=true"`. This is no longer required starting
|
||||
from 1.10.
|
||||
|
||||
Then you have to install NVIDIA drivers on the nodes and run an NVIDIA GPU device
|
||||
plugin ([see below](#deploying-nvidia-gpu-device-plugin)).
|
||||
|
||||
@@ -5,10 +5,10 @@ title: Manage HugePages
|
||||
---
|
||||
|
||||
{% capture overview %}
|
||||
{% include feature-state-alpha.md %}
|
||||
{% include feature-state-beta.md %}
|
||||
|
||||
Kubernetes supports the allocation and consumption of pre-allocated huge pages
|
||||
by applications in a Pod as an **alpha** feature. This page describes how users
|
||||
by applications in a Pod as a **beta** feature. This page describes how users
|
||||
can consume huge pages and the current limitations.
|
||||
|
||||
{% endcapture %}
|
||||
@@ -18,8 +18,6 @@ can consume huge pages and the current limitations.
|
||||
1. Kubernetes nodes must pre-allocate huge pages in order for the node to report
|
||||
its huge page capacity. A node may only pre-allocate huge pages for a single
|
||||
size.
|
||||
1. A special **alpha** feature gate `HugePages` has to be set to true across the
|
||||
system: `--feature-gates=HugePages=true`.
|
||||
|
||||
The nodes will automatically discover and report all huge page resources as a
|
||||
schedulable resource.
|
||||
|
||||
@@ -26,7 +26,9 @@ heapster monitoring will be turned-on by default).
|
||||
|
||||
To specify multiple resource metrics for a Horizontal Pod Autoscaler, you must have a Kubernetes cluster
|
||||
and kubectl at version 1.6 or later. Furthermore, in order to make use of custom metrics, your cluster
|
||||
must be able to communicate with the API server providing the custom metrics API.
|
||||
must be able to communicate with the API server providing the custom metrics API. Finally, to use metrics
|
||||
not related to any Kubernetes object you must have a Kubernetes cluster at version 1.10 or later, and
|
||||
you must be able to communicate with the API server that provides the external metrics API.
|
||||
See the [Horizontal Pod Autoscaler user guide](/docs/tasks/run-application/horizontal-pod-autoscale/#support-for-custom-metrics) for more details.
|
||||
|
||||
## Step One: Run & expose php-apache server
|
||||
@@ -287,6 +289,37 @@ Then, your HorizontalPodAutoscaler would attempt to ensure that each pod was con
|
||||
50% of its requested CPU, serving 1000 packets per second, and that all pods behind the main-route
|
||||
Ingress were serving a total of 10000 requests per second.
|
||||
|
||||
### Autoscaling on metrics not related to Kubernetes objects
|
||||
|
||||
Applications running on Kubernetes may need to autoscale based on metrics that don't have an obvious
|
||||
relationship to any object in the Kubernetes cluster, such as metrics describing a hosted service with
|
||||
no direct correlation to Kubernetes namespaces. In Kubernetes 1.10 and later, you can address this use case
|
||||
with *external metrics*.
|
||||
|
||||
Using external metrics requires a certain level of knowledge of your monitoring system, and it requires a cluster
|
||||
monitoring setup similar to one required for using custom metrics. With external metrics, you can autoscale
|
||||
based on any metric available in your monitoring system by providing a `metricName` field in your
|
||||
HorizontalPodAutoscaler manifest. Additionally you can use a `metricSelector` field to limit which
|
||||
metrics' time series you want to use for autoscaling. If multiple time series are matched by `metricSelector`,
|
||||
the sum of their values is used by the HorizontalPodAutoscaler.
|
||||
|
||||
For example if your application processes tasks from a hosted queue service, you could add the following
|
||||
section to your HorizontalPodAutoscaler manifest to specify that you need one worker per 30 outstanding tasks.
|
||||
|
||||
```yaml
|
||||
- type: External
|
||||
external:
|
||||
metricName: queue_messages_ready
|
||||
metricSelector:
|
||||
matchLabels:
|
||||
queue: worker_tasks
|
||||
targetAverageValue: 30
|
||||
```
|
||||
|
||||
If your metric describes work or resources that can be divided between autoscaled pods the `targetAverageValue`
|
||||
field describes how much of that work each pod can handle. Instead of using the `targetAverageValue` field, you could use the
|
||||
`targetValue` to define a desired value of your external metric.
|
||||
|
||||
## Appendix: Horizontal Pod Autoscaler Status Conditions
|
||||
|
||||
When using the `autoscaling/v2beta1` form of the HorizontalPodAutoscaler, you will be able to see
|
||||
|
||||
@@ -160,13 +160,15 @@ To use custom metrics with your Horizontal Pod Autoscaler, you must set the nece
|
||||
|
||||
* [Enable the API aggregation layer](/docs/tasks/access-kubernetes-api/configure-aggregation-layer/) if you have not already done so.
|
||||
|
||||
* Register your resource metrics API and your
|
||||
custom metrics API with the API aggregation layer. Both of these API servers must be running *on* your cluster.
|
||||
* Register your resource metrics API, your
|
||||
custom metrics API and, optionally, external metrics API with the API aggregation layer. All of these API servers must be running *on* your cluster.
|
||||
|
||||
* *Resource Metrics API*: You can use Heapster's implementation of the resource metrics API, by running Heapster with its `--api-server` flag set to true.
|
||||
|
||||
* *Custom Metrics API*: This must be provided by a separate component. To get started with boilerplate code, see the [kubernetes-incubator/custom-metrics-apiserver](https://github.com/kubernetes-incubator/custom-metrics-apiserver) and the [k8s.io/metrics](https://github.com/kubernetes/metrics) repositories.
|
||||
|
||||
* *External Metrics API*: Starting from Kubernetes 1.10 you can use this API if you need to autoscale on metrics not related to any Kubernetes object. Similarly to *Custom Metrics API* this must be provided by a separate component.
|
||||
|
||||
* Set the appropriate flags for kube-controller-manager:
|
||||
|
||||
* `--horizontal-pod-autoscaler-use-rest-clients` should be true.
|
||||
|
||||
Reference in New Issue
Block a user