Official 1.14 Release Docs (#13174)

* Official documentation on Poseidon/Firmament, a new multi-scheduler support for K8S. (#11752)

* Added documentation about Poseidon-Firmament scheduler

* Fixed some style issues.

* Udpated the document as per the review comments.

* Fixed some typos and updated the document

* Updated the document as per the review comments.

* Document timeout attribute for kms-plugin. (#12158)

See 72540.

* Official documentation on Poseidon/Firmament, a new multi-scheduler  (#12343)

* Removed the old version of the Poseidon documentation. Incorrect location.

* Official documentation on Poseidon/Firmament, a new multi-scheduler support for K8S (#12069)

* Official documentation on Poseidon/Firmament, a new multi-scheduler support for K8S. (#11752)

* Added documentation about Poseidon-Firmament scheduler

* Fixed some style issues.

* Udpated the document as per the review comments.

* Fixed some typos and updated the document

* Updated the document as per the review comments.

* Updated the document as per review comments. Added config details.

* Updated the document as per the latest review comments. Fixed nits

* Made changes as per latest suggestions.

* Some more changes added.

* Updated as per suggestions.

* Changed the release process section.

* SIG Docs edits

Small edits to match style guidelines.

* add plus to feature state

* capitalization

* revert feature state shortcode

since this is a Kubernetes extension, not a direct feature, it shouldn't use the regular feature state tagging.

(cherry picked from commit 7730c1540b)

* Remove initializers from doc. It will be removed in 1.14 (#12331)

* kubeadm: Document CRI auto detection functionality (#12462)

Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>

* Minor doc change for GAing Pod DNS Config (#12514)

* Graduate ExpandInUsePersistentVolumes feature to beta (#10574)

* Rename 2018-11-07-grpc-load-balancing-with-linkerd.md.md file (#12594)

* Add dynamic percentage of node scoring to user docs (#12235)

* Add dynamic percentage of node scoring to user docs

* addressed review comments

* delete special symbol (#12445)

* Update documentation for VolumeSubpathEnvExpansion (#11843)

* Update documentation for VolumeSubpathEnvExpansion

* Address comments - improve descriptions

* Graduate Pod Priority and Preemption to GA (#12428)

* Added Instana links to the documentation (#12977)

* Added link to the Instana Kubernetes integration

* Added Instana link for services section

Added Instana and a link to the Kubernetes integration to the analytics services section and broadened the scope to APM, monitoring and analytics.

* Oxford comma /flex

* More Oxford commas, because they matter

* Update kubectl plugins to stable (#12847)

* documentation for CSI topology beta (#12889)

* Document changes to default RBAC discovery ClusterRole(Binding)s (#12888)

* Document changes to default RBAC discovery ClusterRole(Binding)s

Documentation for https://github.com/kubernetes/enhancements/issues/789 and https://github.com/kubernetes/kubernetes/pull/73807

* documentation review feedback

* CSI raw block to beta (#12931)

* Change incorrect string raw to block (#12926)

Fixes #12925

* Update documentation on node OS/arch labels (#12976)

These labels have been promoted to GA:
https://github.com/kubernetes/enhancements/issues/793

* local pv GA doc updates (#12915)

* Publish CRD OpenAPI Documentation (#12910)

* add documentation for CustomResourcePublishOpenAPI

* address comments

fix links, ordered lists, style and typo

* kubeadm: add document for upgrading from 1.13 to 1.14 (single CP and HA) (#13189)

* kubeadm: add document for upgrading from 1.13 to 1.14

- remove doc for upgrading 1.10 -> 1.11

* kubeadm: apply amends to upgrade-1.14 doc

* kubeadm: apply amends to upgrade-1.14 doc (part2)

* kubeadm: apply amends to upgrade-1.14 doc (part3)

* kubeadm: add note about "upgrade node experimental-control-plane"

+ add comment about `upgrade plan`

* kubeadm: add missing "You should see output similar to this"

* fix bullet indentation (#13214)

* mark PodReadinessGate GA (#12800)

* Update RuntimeClass documentation for beta (#13043)

* Update RuntimeClass documentation for beta

* Update feature gate & add upgrade section

* formatting fixes

* Highlight upgrade action required

* Address feedback

* CSI ephemeral volume alpha documentation (#10934)

* update kubectl documentation (#12867)

* update kubectl documentation

* add document for Secret/ConfigMap generators

* replace `kubectl create -f` by `kubectl apply -f`

* Add page for kustomization support in kubectl

* fix spelling errors and address comments

* Documentation for Windows GMSA feature (#12936)

* Documentation for Windows GMSA feature

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Enhancements to GMSA docs

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Fix links

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Fix GMSA link

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Add GMSA feature flag in feature flag list

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Relocate GMSA to container configuration

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Add example for container spec

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Remove changes in Windows index

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Update configure-gmsa.md

* Update configure-gmsa.md

* Update configure-gmsa.md

* Update configure-gmsa.md

* Rearrange the steps into two sections and other edits

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Fix links

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Add reference to script to generate GMSA YAMLs

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Some more clarifications for GMSA

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* HugePages graduated to GA (#13004)

* HugePages graduated to GA

* fixing nit for build

* Docs for node PID limiting (https://github.com/kubernetes/kubernetes/pull/73651) (#12932)

* kubeadm: update the reference documentation for 1.14 (#12911)

* kubeadm: update list of generated files for 1.14

NOTE: PLACEHOLDERS! these files are generated by SIG Docs each
release, but we need them to pass the k/website PR CI.

- add join_phase* (new sub phases of join)
- add init_phase_upload-certs.md (new upload certs phase for init)
- remove alpha-preflight (now both init and join have this)

* kubeadm: update reference docs includes for 1.14

- remove includes from alpha.md
- add upload-certs to init-phase.md
- add join-phase.md and it's phases

* kubeadm: update the editorial content of join and init

- cleanup master->control-plane node
- add some notes about phases and join
- remove table about pre-pulling images
- remove outdated info about self-hosting

* kubeadm: update target release for v1alpha3 removal

1.14 -> 1.15

* kubeadm: copy edits for 1.14 reference docs (part1)

* kubeadm: use "shell" for code blocks

* kubeadm: update the 1.14 HA guide (#13191)

* kubeadm: update the 1.14 HA guide

* kubeadm: try to fix note/caution indent in HA page

* kubeadm: fix missing sudo and minor amends in HA doc

* kubeadm: apply latest amends to the HA doc for 1.14

* fixed a few missed merge conflicts

* Admission Webhook new features doc (#12938)

- kubernetes/kubernetes#74998
- kubernetes/kubernetes#74477
- kubernetes/kubernetes#74562

* Clarifications and fixes in GMSA doc (#13226)

* Clarifications and fixes in GMSA doc

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Update configure-gmsa.md

* Reformat to align headings and pre-reqs better

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Reformat to align headings and pre-reqs better

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Reformat to fix bullets

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Reword application of sample gmsa

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Update configure-gmsa.md

* Address feedback to use active voice

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Address feedback to use active voice

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* RunAsGroup documentation for Progressing this to Beta (#12297)

* start serverside-apply documentation (#13077)

* start serverside-apply documentation

* add more concept info on server side apply

* Update api concepts

* Update api-concepts.md

* fix style issues

* Document CSI update (#12928)

* Document CSI update

* Finish CSI documentation

Also fix mistake with  ExpandInUsePersistentVolumes documented as beta

* Overall docs for CSI Migration feature (#12935)

* Placeholder docs for CSI Migration feature

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Address CR comments and update feature gates

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Add mappings for CSI plugins

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Add sections for AWS and GCE PD migration

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Add docs for Cinder and CSI Migration info

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Clarify scope to volumes with file system

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Change the format of EBS and Cinder CSI Migration sections to follow the GCE template

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Windows documentation updates for 1.14 (#12929)

* Updated the note to indicate doc work for 1.14

* first attempt at md export from gdoc

* simplifyig

* big attempt

* moving DRAFT windows content to PR for review

* moving content to PR in markdown for review

* updated note tags

* Delete windows-contributing.md

deleting this file as it is already ported to the github contributor guide

* fixed formatting in intro and cluster setup guide

* updating formatting for running containers guide

* rejiggered end of troubleshooting

* fixed minor typos

* Clarified the windows binary download step

* Update _index.md

making updates based on feedback

* Update _index.md

updating ovn-kubernetes docs

* Update _index.md

* Update _index.md

* updating relative docs links

updating all the links to be relative links to /docs

* Update _index.md

* Update _index.md

updates for windows services and ovn-kubernetes

* formatted for correct step numbering

* fix typos

* Update _index.md

updates for flannel PR in troubleshooting

* Update _index.md

* Update _index.md

updating a few sections like roadmap, services, troubleshooting/filing tickets

* Update _index.md

* Update _index.md

* Update _index.md

* Fixed a few whitespace issues

* Update _index.md

* Update _index.md

* Update _index.md

* add section on upgrading CoreDNS (#12909)

* documentation for kubelet resource metrics endpoint (#12934)

* windows docs updates for 1.14 (#13279)

* Delete sample-l2bridge-wincni-config.json

this file is not used anywhere

* Update _index.md

* Update _index.md

* Update _index.md

* Update _index.md

* Update _index.md

* Rename content/en/docs/getting-started-guides/windows/_index.md to content/en/docs/setup/windows/_index.md

moving to new location

* Delete flannel-master-kubectl-get-ds.png

* Delete flannel-master-kubeclt-get-pods.png

* Delete windows-docker-error.png

* Add files via upload

* Rename _index.md to add-windows-nodes.md

* Create _index.md

* Update _index.md

* Update add-windows-nodes.md

* Update add-windows-nodes.md

* Create user-guide-windows-nodes.md

* Create user-guide-windows-containers.md

* Update and rename add-windows-nodes.md to intro-windows-nodes.md

* Update user-guide-windows-containers.md

* Rename intro-windows-nodes.md to intro-windows-in-kubernetes.md

* Update user-guide-windows-nodes.md

* Update user-guide-windows-containers.md

* Update user-guide-windows-containers.md

* Update user-guide-windows-nodes.md

* Update user-guide-windows-containers.md

* Update _index.md

* Update intro-windows-in-kubernetes.md

* Update intro-windows-in-kubernetes.md

fixing the pause image

* Update intro-windows-in-kubernetes.md

changing tables from html to MD

* Update user-guide-windows-nodes.md

converting tables from HTML to MD

* Update intro-windows-in-kubernetes.md

* Update user-guide-windows-nodes.md

* Update user-guide-windows-nodes.md

* Update user-guide-windows-nodes.md

updating the numbering , even though it messes up the notes a little bit. Jim will file a ticket to follow up

* Update user-guide-windows-nodes.md

* update to windows docs for 1.14 (#13322)

* Update intro-windows-in-kubernetes.md

* Update intro-windows-in-kubernetes.md

* Update intro-windows-in-kubernetes.md

* Update intro-windows-in-kubernetes.md

* Update intro-windows-in-kubernetes.md

* Update user-guide-windows-containers.md

* Update user-guide-windows-nodes.md

* Update intro-windows-in-kubernetes.md (#13344)

* server side apply followup (#13321)

* change some parts of serverside apply docs in response to comments

* fix typos and wording

* Update config.toml (#13365)
This commit is contained in:
Jim Angel
2019-03-25 17:06:16 -05:00
committed by Kubernetes Prow Robot
parent 28dd4d5154
commit 851ef58fa8
160 changed files with 4319 additions and 1812 deletions
@@ -35,7 +35,7 @@ a container that writes some text to standard output once per second.
To run this pod, use the following command:
```shell
kubectl create -f https://k8s.io/examples/debug/counter-pod.yaml
kubectl apply -f https://k8s.io/examples/debug/counter-pod.yaml
pod/counter created
```
@@ -26,7 +26,7 @@ Many applications require multiple resources to be created, such as a Deployment
Multiple resources can be created the same way as a single resource:
```shell
kubectl create -f https://k8s.io/examples/application/nginx-app.yaml
kubectl apply -f https://k8s.io/examples/application/nginx-app.yaml
```
```shell
@@ -36,16 +36,16 @@ deployment.apps/my-nginx created
The resources will be created in the order they appear in the file. Therefore, it's best to specify the service first, since that will ensure the scheduler can spread the pods associated with the service as they are created by the controller(s), such as Deployment.
`kubectl create` also accepts multiple `-f` arguments:
`kubectl apply` also accepts multiple `-f` arguments:
```shell
kubectl create -f https://k8s.io/examples/application/nginx/nginx-svc.yaml -f https://k8s.io/examples/application/nginx/nginx-deployment.yaml
kubectl apply -f https://k8s.io/examples/application/nginx/nginx-svc.yaml -f https://k8s.io/examples/application/nginx/nginx-deployment.yaml
```
And a directory can be specified rather than or in addition to individual files:
```shell
kubectl create -f https://k8s.io/examples/application/nginx/
kubectl apply -f https://k8s.io/examples/application/nginx/
```
`kubectl` will read any files with suffixes `.yaml`, `.yml`, or `.json`.
@@ -55,7 +55,7 @@ It is a recommended practice to put resources related to the same microservice o
A URL can also be specified as a configuration source, which is handy for deploying directly from configuration files checked into github:
```shell
kubectl create -f https://raw.githubusercontent.com/kubernetes/website/master/content/en/examples/application/nginx/nginx-deployment.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes/website/master/content/en/examples/application/nginx/nginx-deployment.yaml
```
```shell
@@ -123,7 +123,7 @@ project/k8s/development
By default, performing a bulk operation on `project/k8s/development` will stop at the first level of the directory, not processing any subdirectories. If we had tried to create the resources in this directory using the following command, we would have encountered an error:
```shell
kubectl create -f project/k8s/development
kubectl apply -f project/k8s/development
```
```shell
@@ -133,7 +133,7 @@ error: you must provide one or more resources by argument or filename (.json|.ya
Instead, specify the `--recursive` or `-R` flag with the `--filename,-f` flag as such:
```shell
kubectl create -f project/k8s/development --recursive
kubectl apply -f project/k8s/development --recursive
```
```shell
@@ -147,7 +147,7 @@ The `--recursive` flag works with any operation that accepts the `--filename,-f`
The `--recursive` flag also works when multiple `-f` arguments are provided:
```shell
kubectl create -f project/k8s/namespaces -f project/k8s/development --recursive
kubectl apply -f project/k8s/namespaces -f project/k8s/development --recursive
```
```shell
@@ -193,7 +193,7 @@ and
The labels allow us to slice and dice our resources along any dimension specified by a label:
```shell
kubectl create -f examples/guestbook/all-in-one/guestbook-all-in-one.yaml
kubectl apply -f examples/guestbook/all-in-one/guestbook-all-in-one.yaml
kubectl get pods -Lapp -Ltier -Lrole
```
@@ -366,7 +366,6 @@ This command will compare the version of the configuration that you're pushing w
```shell
kubectl apply -f https://k8s.io/examples/application/nginx/nginx-deployment.yaml
```
```shell
deployment.apps/my-nginx configured
```
@@ -377,10 +376,6 @@ Currently, resources are created without this annotation, so the first invocatio
All subsequent calls to `kubectl apply`, and other commands that modify the configuration, such as `kubectl replace` and `kubectl edit`, will update the annotation, allowing subsequent calls to `kubectl apply` to detect and perform deletions using a three-way diff.
{{< note >}}
To use apply, always create resource initially with either `kubectl apply` or `kubectl create --save-config`.
{{< /note >}}
### kubectl edit
Alternatively, you may also update resources with `kubectl edit`:
@@ -430,8 +425,7 @@ deployment.apps/my-nginx replaced
At some point, you'll eventually need to update your deployed application, typically by specifying a new image or image tag, as in the canary deployment scenario above. `kubectl` supports several update operations, each of which is applicable to different scenarios.
We'll guide you through how to create and update applications with Deployments. If your deployed application is managed by Replication Controllers,
you should read [how to use `kubectl rolling-update`](/docs/tasks/run-application/rolling-update-replication-controller/) instead.
We'll guide you through how to create and update applications with Deployments.
Let's say you were running version 1.7.9 of nginx:
@@ -69,7 +69,7 @@ Then add a nodeSelector like so:
{{< codenew file="pods/pod-nginx.yaml" >}}
When you then run `kubectl create -f https://k8s.io/examples/pods/pod-nginx.yaml`,
When you then run `kubectl apply -f https://k8s.io/examples/pods/pod-nginx.yaml`,
the Pod will get scheduled on the node that you attached the label to. You can
verify that it worked by running `kubectl get pods -o wide` and looking at the
"NODE" that the Pod was assigned to.
@@ -83,8 +83,8 @@ with a standard set of labels. As of Kubernetes v1.4 these labels are
* `failure-domain.beta.kubernetes.io/zone`
* `failure-domain.beta.kubernetes.io/region`
* `beta.kubernetes.io/instance-type`
* `beta.kubernetes.io/os`
* `beta.kubernetes.io/arch`
* `kubernetes.io/os`
* `kubernetes.io/arch`
{{< note >}}
The value of these labels is cloud provider specific and is not guaranteed to be reliable.
@@ -23,7 +23,7 @@ This is a living document. If you think of something that is not on this list bu
- Group related objects into a single file whenever it makes sense. One file is often easier to manage than several. See the [guestbook-all-in-one.yaml](https://github.com/kubernetes/examples/tree/{{< param "githubbranch" >}}/guestbook/all-in-one/guestbook-all-in-one.yaml) file as an example of this syntax.
- Note also that many `kubectl` commands can be called on a directory. For example, you can call `kubectl create` on a directory of config files.
- Note also that many `kubectl` commands can be called on a directory. For example, you can call `kubectl apply` on a directory of config files.
- Don't specify default values unnecessarily: simple, minimal configuration will make errors less likely.
@@ -100,7 +100,7 @@ The caching semantics of the underlying image provider make even `imagePullPolic
## Using kubectl
- Use `kubectl apply -f <directory>` or `kubectl create -f <directory>`. This looks for Kubernetes configuration in all `.yaml`, `.yml`, and `.json` files in `<directory>` and passes it to `apply` or `create`.
- Use `kubectl apply -f <directory>`. This looks for Kubernetes configuration in all `.yaml`, `.yml`, and `.json` files in `<directory>` and passes it to `apply`.
- Use label selectors for `get` and `delete` operations instead of specific object names. See the sections on [label selectors](/docs/concepts/overview/working-with-objects/labels/#label-selectors) and [using labels effectively](/docs/concepts/cluster-administration/manage-deployment/#using-labels-effectively).
@@ -9,7 +9,7 @@ weight: 70
{{% capture overview %}}
{{< feature-state for_k8s_version="1.11" state="beta" >}}
{{< feature-state for_k8s_version="1.14" state="stable" >}}
[Pods](/docs/user-guide/pods) can have _priority_. Priority indicates the
importance of a Pod relative to other Pods. If a Pod cannot be scheduled, the
@@ -19,8 +19,8 @@ pending Pod possible.
In Kubernetes 1.9 and later, Priority also affects scheduling order of Pods and
out-of-resource eviction ordering on the Node.
Pod priority and preemption are moved to beta since Kubernetes 1.11 and are
enabled by default in this release and later.
Pod priority and preemption graduated to beta in Kubernetes 1.11 and to GA in
Kubernetes 1.14. They have been enabled by default since 1.11.
In Kubernetes versions where Pod priority and preemption is still an alpha-level
feature, you need to explicitly enable it. To use these features in the older
@@ -34,6 +34,7 @@ Kubernetes Version | Priority and Preemption State | Enabled by default
1.9 | alpha | no
1.10 | alpha | no
1.11 | beta | yes
1.14 | GA | yes
{{< warning >}}In a cluster where not all users are trusted, a
malicious user could create pods at the highest possible priorities, causing
@@ -71,15 +72,15 @@ Pods.
## How to disable preemption
{{< note >}}
In Kubernetes 1.11, critical pods (except DaemonSet pods, which are
still scheduled by the DaemonSet controller) rely on scheduler preemption to be
scheduled when a cluster is under resource pressure. For this reason, you will
need to run an older version of Rescheduler if you decide to disable preemption.
More on this is provided below.
In Kubernetes 1.12+, critical pods rely on scheduler preemption to be scheduled
when a cluster is under resource pressure. For this reason, it is not
recommended to disable preemption.
{{< /note >}}
In Kubernetes 1.11 and later, preemption is controlled by a kube-scheduler flag
`disablePreemption`, which is set to `false` by default.
If you want to disable preemption despite the above note, you can set
`disablePreemption` to `true`.
This option is available in component configs only and is not available in
old-style command line options. Below is a sample component config to disable
@@ -96,20 +97,6 @@ algorithmSource:
disablePreemption: true
```
### Start an older version of Rescheduler in the cluster
When priority or preemption is disabled, we must run Rescheduler v0.3.1 (instead
of v0.4.0) to ensure that critical Pods are scheduled when nodes or cluster are
under resource pressure. Since critical Pod annotation is still supported in
this release, running Rescheduler should be enough and no other changes to the
configuration of Pods should be needed.
Rescheduler images can be found at:
[gcr.io/k8s-image-staging/rescheduler](http://gcr.io/k8s-image-staging/rescheduler).
In the code, changing the Rescheduler version back to v.0.3.1 is the reverse of
[this PR](https://github.com/kubernetes/kubernetes/pull/65454).
## PriorityClass
A PriorityClass is a non-namespaced object that defines a mapping from a
@@ -8,15 +8,15 @@ weight: 70
{{% capture overview %}}
{{< feature-state for_k8s_version="1.12" >}}
{{< feature-state for_k8s_version="1.14" state="beta" >}}
Kube-scheduler is the Kubernetes default scheduler. It is responsible for
placement of Pods on Nodes in a cluster. Nodes in a cluster that meet the
scheduling requirements of a Pod are called "feasible" Nodes for the Pod. The
scheduler finds feasible Nodes for a Pod and then runs a set of functions to
score the feasible Nodes and picks a Node with the highest score among the
feasible ones to run the Pod. The scheduler then notifies the API server about this
decision in a process called "Binding".
feasible ones to run the Pod. The scheduler then notifies the API server about
this decision in a process called "Binding".
{{% /capture %}}
@@ -24,15 +24,23 @@ decision in a process called "Binding".
## Percentage of Nodes to Score
Before Kubernetes 1.12, Kube-scheduler used to check the feasibility of all the
nodes in a cluster and then scored the feasible ones. Kubernetes 1.12 has a new
feature that allows the scheduler to stop looking for more feasible nodes once
it finds a certain number of them. This improves the scheduler's performance in
large clusters. The number is specified as a percentage of the cluster size and
is controlled by a configuration option called `percentageOfNodesToScore`. The
range should be between 1 and 100. Other values are considered as 100%. The
default value of this option is 50%. A cluster administrator can change this value by providing a
different value in the scheduler configuration. However, it may not be necessary to change this value.
Before Kubernetes 1.12, Kube-scheduler used to check the feasibility of all
nodes in a cluster and then scored the feasible ones. Kubernetes 1.12 added a
new feature that allows the scheduler to stop looking for more feasible nodes
once it finds a certain number of them. This improves the scheduler's
performance in large clusters. The number is specified as a percentage of the
cluster size. The percentage can be controlled by a configuration option called
`percentageOfNodesToScore`. The range should be between 1 and 100. Larger values
are considered as 100%. Zero is equivalent to not providing the config option.
Kubernetes 1.14 has logic to find the percentage of nodes to score based on the
size of the cluster if it is not specified in the configuration. It uses a
linear formula which yields 50% for a 100-node cluster. The formula yields 10%
for a 5000-node cluster. The lower bound for the automatic value is 5%. In other
words, the scheduler always scores at least 5% of the cluster no matter how
large the cluster is, unless the user provides the config option with a value
smaller than 5.
Below is an example configuration that sets `percentageOfNodesToScore` to 50%.
```yaml
apiVersion: componentconfig/v1alpha1
@@ -45,26 +53,22 @@ algorithmSource:
percentageOfNodesToScore: 50
```
{{< note >}}
In clusters with zero or less than 50 feasible nodes, the
scheduler still checks all the nodes, simply because there are not enough
feasible nodes to stop the scheduler's search early.
{{< /note >}}
{{< note >}} In clusters with less than 50 feasible nodes, the scheduler still
checks all the nodes, simply because there are not enough feasible nodes to stop
the scheduler's search early. {{< /note >}}
**To disable this feature**, you can set `percentageOfNodesToScore` to 100.
### Tuning percentageOfNodesToScore
`percentageOfNodesToScore` must be a value between 1 and 100
with the default value of 50. There is also a hardcoded minimum value of 50
nodes which is applied internally. The scheduler tries to find at
least 50 nodes regardless of the value of `percentageOfNodesToScore`. This means
that changing this option to lower values in clusters with several hundred nodes
will not have much impact on the number of feasible nodes that the scheduler
tries to find. This is intentional as this option is unlikely to improve
performance noticeably in smaller clusters. In large clusters with over a 1000
nodes setting this value to lower numbers may show a noticeable performance
improvement.
`percentageOfNodesToScore` must be a value between 1 and 100 with the default
value being calculated based on the cluster size. There is also a hardcoded
minimum value of 50 nodes. This means that changing
this option to lower values in clusters with several hundred nodes will not have
much impact on the number of feasible nodes that the scheduler tries to find.
This is intentional as this option is unlikely to improve performance noticeably
in smaller clusters. In large clusters with over a 1000 nodes setting this value
to lower numbers may show a noticeable performance improvement.
An important note to consider when setting this value is that when a smaller
number of nodes in a cluster are checked for feasibility, some nodes are not
@@ -72,14 +76,14 @@ sent to be scored for a given Pod. As a result, a Node which could possibly
score a higher value for running the given Pod might not even be passed to the
scoring phase. This would result in a less than ideal placement of the Pod. For
this reason, the value should not be set to very low percentages. A general rule
of thumb is to never set the value to anything lower than 30. Lower values
of thumb is to never set the value to anything lower than 10. Lower values
should be used only when the scheduler's throughput is critical for your
application and the score of nodes is not important. In other words, you prefer
to run the Pod on any Node as long as it is feasible.
It is not recommended to lower this value from its default if your cluster has
only several hundred Nodes. It is unlikely to improve the scheduler's
performance significantly.
If your cluster has several hundred Nodes or fewer, we do not recommend lowering
the default value of this configuration option. It is unlikely to improve the
scheduler's performance significantly.
### How the scheduler iterates over Nodes
@@ -91,8 +95,8 @@ for running Pods, the scheduler iterates over the nodes in a round robin
fashion. You can imagine that Nodes are in an array. The scheduler starts from
the start of the array and checks feasibility of the nodes until it finds enough
Nodes as specified by `percentageOfNodesToScore`. For the next Pod, the
scheduler continues from the point in the Node array that it stopped at when checking
feasibility of Nodes for the previous Pod.
scheduler continues from the point in the Node array that it stopped at when
checking feasibility of Nodes for the previous Pod.
If Nodes are in multiple zones, the scheduler iterates over Nodes in various
zones to ensure that Nodes from different zones are considered in the
@@ -75,6 +75,12 @@ kubectl create secret generic db-user-pass --from-file=./username.txt --from-fil
```
secret "db-user-pass" created
```
{{< note >}}
Special characters such as `$`, `\*`, and `!` require escaping.
If the password you are using has special characters, you need to escape them using the `\\` character. For example, if your actual password is `S!B\*d$zDsb`, you should execute the command this way:
kubectl create secret generic dev-db-secret --from-literal=username=devuser --from-literal=password=S\\!B\\\*d\\$zDsb
You do not need to escape special characters in passwords from files (`--from-file`).
{{< /note >}}
You can check that the secret was created like this:
@@ -143,10 +149,10 @@ data:
password: MWYyZDFlMmU2N2Rm
```
Now create the Secret using [`kubectl create`](/docs/reference/generated/kubectl/kubectl-commands#create):
Now create the Secret using [`kubectl apply`](/docs/reference/generated/kubectl/kubectl-commands#apply):
```shell
kubectl create -f ./secret.yaml
kubectl apply -f ./secret.yaml
```
```
secret "mysecret" created
@@ -184,7 +190,7 @@ stringData:
```
Your deployment tool could then replace the `{{username}}` and `{{password}}`
template variables before running `kubectl create`.
template variables before running `kubectl apply`.
stringData is a write-only convenience field. It is never output when
retrieving Secrets. For example, if you run the following command:
@@ -254,6 +260,73 @@ using the `-b` option to split long lines. Conversely Linux users *should* add
the option `-w 0` to `base64` commands or the pipeline `base64 | tr -d '\n'` if
`-w` option is not available.
#### Creating a Secret from Generator
Kubectl supports [managing objects using Kustomize](/docs/concepts/overview/object-management-kubectl/kustomization/)
since 1.14. With this new feature,
you can also create a Secret from generators and then apply it to create the object on
the Apiserver. The generators
should be specified in a `kustomization.yaml` inside a directory.
For example, to generate a Secret from files `./username.txt` and `./password.txt`
```shell
# Create a kustomization.yaml file with SecretGenerator
cat <<EOF >./kustomization.yaml
secretGenerator:
- name: db-user-pass
files:
- username.txt
- password.txt
EOF
```
Apply the kustomization directory to create the Secret object.
```shell
$ kubectl apply -k .
secret/db-user-pass-96mffmfh4k created
```
You can check that the secret was created like this:
```shell
$ kubectl get secrets
NAME TYPE DATA AGE
db-user-pass-96mffmfh4k Opaque 2 51s
$ kubectl describe secrets/db-user-pass-96mffmfh4k
Name: db-user-pass
Namespace: default
Labels: <none>
Annotations: <none>
Type: Opaque
Data
====
password.txt: 12 bytes
username.txt: 5 bytes
```
For example, to generate a Secret from literals `username=admin` and `password=secret`,
you can specify the secret generator in `kusotmization.yaml` as
```shell
# Create a kustomization.yaml file with SecretGenerator
$ cat <<EOF >./kustomization.yaml
secretGenerator:
- name: db-user-pass
literals:
- username=admin
- password=secret
EOF
```
Apply the kustomization directory to create the Secret object.
```shell
$ kubectl apply -k .
secret/db-user-pass-dddghtt9b5 created
```
{{< note >}}
The generated Secrets name has a suffix appended by hashing the contents. This ensures that a new
Secret is generated each time the contents is modified.
{{< /note >}}
#### Decoding a Secret
Secrets can be retrieved via the `kubectl get secret` command. For example, to retrieve the secret created in the previous section:
@@ -620,8 +693,7 @@ start until all the pod's volumes are mounted.
### Use-Case: Pod with ssh keys
Create a secret containing some ssh keys:
Create a kustomization.yaml with SecretGenerator containing some ssh keys:
```shell
kubectl create secret generic ssh-key-secret --from-file=ssh-privatekey=/path/to/.ssh/id_rsa --from-file=ssh-publickey=/path/to/.ssh/id_rsa.pub
```
@@ -673,8 +745,7 @@ This example illustrates a pod which consumes a secret containing prod
credentials and another pod which consumes a secret with test environment
credentials.
Make the secrets:
Make the kustomization.yaml with SecretGenerator
```shell
kubectl create secret generic prod-db-secret --from-literal=username=produser --from-literal=password=Y4nys7f11
```
@@ -682,24 +753,10 @@ kubectl create secret generic prod-db-secret --from-literal=username=produser --
secret "prod-db-secret" created
```
```shell
kubectl create secret generic test-db-secret --from-literal=username=testuser --from-literal=password=iluvtests
```
```
secret "test-db-secret" created
```
{{< note >}}
Special characters such as `$`, `\*`, and `!` require escaping.
If the password you are using has special characters, you need to escape them using the `\\` character. For example, if your actual password is `S!B\*d$zDsb`, you should execute the command this way:
kubectl create secret generic dev-db-secret --from-literal=username=devuser --from-literal=password=S\\!B\\\*d\\$zDsb
You do not need to escape special characters in passwords from files (`--from-file`).
{{< /note >}}
Now make the pods:
```yaml
```shell
$ cat <<EOF > pod.yaml
apiVersion: v1
kind: List
items:
@@ -739,6 +796,21 @@ items:
- name: secret-volume
readOnly: true
mountPath: "/etc/secret-volume"
EOF
```
Add the pods to the same kustomization.yaml
```shell
$ cat <<EOF >> kustomization.yaml
resources:
- pod.yaml
EOF
```
Apply all those objects on the Apiserver by
```shell
kubectl apply --k .
```
Both containers will have the following files present on their filesystems with the values for each container's environment:
+22 -4
View File
@@ -205,7 +205,7 @@ example, run these on your desktop/laptop:
Verify by creating a pod that uses a private image, e.g.:
```yaml
kubectl create -f - <<EOF
kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
@@ -279,8 +279,19 @@ Kubernetes supports specifying registry keys on a pod.
Run the following command, substituting the appropriate uppercase values:
```shell
kubectl create secret docker-registry myregistrykey --docker-server=DOCKER_REGISTRY_SERVER --docker-username=DOCKER_USER --docker-password=DOCKER_PASSWORD --docker-email=DOCKER_EMAIL
secret/myregistrykey created.
cat <<EOF > ./kustomization.yaml
secretGenerator:
- name: myregistrykey
type: docker-registry
literals:
- docker-server=DOCKER_REGISTRY_SERVER
- docker-username=DOCKER_USER
- docker-password=DOCKER_PASSWORD
- docker-email=DOCKER_EMAIL
EOF
kubectl apply -k .
secret/myregistrykey-66h7d4d986 created
```
If you already have a Docker credentials file then, rather than using the above
@@ -300,7 +311,8 @@ so this process needs to be done one time per namespace.
Now, you can create pods which reference that secret by adding an `imagePullSecrets`
section to a pod definition.
```yaml
```shell
cat <<EOF > pod.yaml
apiVersion: v1
kind: Pod
metadata:
@@ -312,6 +324,12 @@ spec:
image: janedoe/awesomeapp:v1
imagePullSecrets:
- name: myregistrykey
EOF
cat <<EOF >> ./kustomization.yaml
resources:
- pod.yaml
EOF
```
This needs to be done for each pod that is using a private registry.
@@ -9,10 +9,16 @@ weight: 20
{{% capture overview %}}
{{< feature-state for_k8s_version="v1.12" state="alpha" >}}
{{< feature-state for_k8s_version="v1.14" state="beta" >}}
This page describes the RuntimeClass resource and runtime selection mechanism.
{{< warning >}}
RuntimeClass includes *breaking* changes in the beta upgrade in v1.14. If you were using
RuntimeClass prior to v1.14, see [Upgrading RuntimeClass from Alpha to
Beta](#upgrading-runtimeclass-from-alpha-to-beta).
{{< /warning >}}
{{% /capture %}}
@@ -20,72 +26,51 @@ This page describes the RuntimeClass resource and runtime selection mechanism.
## Runtime Class
RuntimeClass is an alpha feature for selecting the container runtime configuration to use to run a
pod's containers.
RuntimeClass is a feature for selecting the container runtime configuration. The container runtime
configuration is used to run a Pod's containers.
### Set Up
As an early alpha feature, there are some additional setup steps that must be taken in order to use
the RuntimeClass feature:
Ensure the RuntimeClass feature gate is enabled (it is by default). See [Feature
Gates](/docs/reference/command-line-tools-reference/feature-gates/) for an explanation of enabling
feature gates. The `RuntimeClass` feature gate must be enabled on apiservers _and_ kubelets.
1. Enable the RuntimeClass feature gate (on apiservers & kubelets, requires version 1.12+)
2. Install the RuntimeClass CRD
3. Configure the CRI implementation on nodes (runtime dependent)
4. Create the corresponding RuntimeClass resources
1. Configure the CRI implementation on nodes (runtime dependent)
2. Create the corresponding RuntimeClass resources
#### 1. Enable the RuntimeClass feature gate
#### 1. Configure the CRI implementation on nodes
See [Feature Gates](/docs/reference/command-line-tools-reference/feature-gates/) for an explanation
of enabling feature gates. The `RuntimeClass` feature gate must be enabled on apiservers _and_
kubelets.
#### 2. Install the RuntimeClass CRD
The RuntimeClass [CustomResourceDefinition][] (CRD) can be found in the addons directory of the
Kubernetes git repo: [kubernetes/cluster/addons/runtimeclass/runtimeclass_crd.yaml][runtimeclass_crd]
Install the CRD with `kubectl apply -f runtimeclass_crd.yaml`.
[CustomResourceDefinition]: /docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/
[runtimeclass_crd]: https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/runtimeclass/runtimeclass_crd.yaml
#### 3. Configure the CRI implementation on nodes
The configurations to select between with RuntimeClass are CRI implementation dependent. See the
corresponding documentation for your CRI implementation for how to configure. As this is an alpha
feature, not all CRIs support multiple RuntimeClasses yet.
The configurations available through RuntimeClass are Container Runtime Interface (CRI)
implementation dependent. See the corresponding documentation ([below](#cri-documentation)) for your
CRI implementation for how to configure.
{{< note >}}
RuntimeClass currently assumes a homogeneous node configuration across the cluster
(which means that all nodes are configured the same way with respect to container runtimes). Any heterogeneity (varying configurations) must be
managed independently of RuntimeClass through scheduling features (see [Assigning Pods to
Nodes](/docs/concepts/configuration/assign-pod-node/)).
RuntimeClass currently assumes a homogeneous node configuration across the cluster (which means that
all nodes are configured the same way with respect to container runtimes). Any heterogeneity
(varying configurations) must be managed independently of RuntimeClass through scheduling features
(see [Assigning Pods to Nodes](/docs/concepts/configuration/assign-pod-node/)).
{{< /note >}}
The configurations have a corresponding `RuntimeHandler` name, referenced by the RuntimeClass. The
RuntimeHandler must be a valid DNS 1123 subdomain (alpha-numeric + `-` and `.` characters).
The configurations have a corresponding `handler` name, referenced by the RuntimeClass. The
handler must be a valid DNS 1123 label (alpha-numeric + `-` characters).
#### 4. Create the corresponding RuntimeClass resources
#### 2. Create the corresponding RuntimeClass resources
The configurations setup in step 3 should each have an associated `RuntimeHandler` name, which
identifies the configuration. For each RuntimeHandler (and optionally the empty `""` handler),
create a corresponding RuntimeClass object.
The configurations setup in step 1 should each have an associated `handler` name, which identifies
the configuration. For each handler, create a corresponding RuntimeClass object.
The RuntimeClass resource currently only has 2 significant fields: the RuntimeClass name
(`metadata.name`) and the RuntimeHandler (`spec.runtimeHandler`). The object definition looks like this:
(`metadata.name`) and the handler (`handler`). The object definition looks like this:
```yaml
apiVersion: node.k8s.io/v1alpha1 # RuntimeClass is defined in the node.k8s.io API group
apiVersion: node.k8s.io/v1beta1 # RuntimeClass is defined in the node.k8s.io API group
kind: RuntimeClass
metadata:
name: myclass # The name the RuntimeClass will be referenced by
# RuntimeClass is a non-namespaced resource
spec:
runtimeHandler: myconfiguration # The name of the corresponding CRI configuration
handler: myconfiguration # The name of the corresponding CRI configuration
```
{{< note >}}
It is recommended that RuntimeClass write operations (create/update/patch/delete) be
restricted to the cluster administrator. This is typically the default. See [Authorization
@@ -116,4 +101,66 @@ error message.
If no `runtimeClassName` is specified, the default RuntimeHandler will be used, which is equivalent
to the behavior when the RuntimeClass feature is disabled.
### CRI Configuration
For more details on setting up CRI runtimes, see [CRI installation](/docs/setup/cri/).
#### dockershim
Kubernetes built-in dockershim CRI does not support runtime handlers.
#### [containerd](https://containerd.io/)
Runtime handlers are configured through containerd's configuration at
`/etc/containerd/config.toml`. Valid handlers are configured under the runtimes section:
```
[plugins.cri.containerd.runtimes.${HANDLER_NAME}]
```
See containerd's config documentation for more details:
https://github.com/containerd/cri/blob/master/docs/config.md
#### [cri-o](https://cri-o.io/)
Runtime handlers are configured through cri-o's configuration at `/etc/crio/crio.conf`. Valid
handlers are configured under the [crio.runtime
table](https://github.com/kubernetes-sigs/cri-o/blob/master/docs/crio.conf.5.md#crioruntime-table):
```
[crio.runtime.runtimes.${HANDLER_NAME}]
runtime_path = "${PATH_TO_BINARY}"
```
See cri-o's config documentation for more details:
https://github.com/kubernetes-sigs/cri-o/blob/master/cmd/crio/config.go
### Upgrading RuntimeClass from Alpha to Beta
The RuntimeClass Beta feature includes the following changes:
- The `node.k8s.io` API group and `runtimeclasses.node.k8s.io` resource have been migrated to a
built-in API from a CustomResourceDefinition.
- The `spec` has been inlined in the RuntimeClass definition (i.e. there is no more
RuntimeClassSpec).
- The `runtimeHandler` field has been renamed `handler`.
- The `handler` field is now required in all API versions. This means the `runtimeHandler` field in
the Alpha API is also required.
- The `handler` field must be a valid DNS label ([RFC 1123](https://tools.ietf.org/html/rfc1123)),
meaning it can no longer contain `.` characters (in all versions). Valid handlers match the
following regular expression: `^[a-z0-9]([-a-z0-9]*[a-z0-9])?$`.
**Action Required:** The following actions are required to upgrade from the alpha version of the
RuntimeClass feature to the beta version:
- RuntimeClass resources must be recreated *after* upgrading to v1.14, and the
`runtimeclasses.node.k8s.io` CRD should be manually deleted:
```
kubectl delete customresourcedefinitions.apiextensions.k8s.io runtimeclasses.node.k8s.io
```
- Alpha RuntimeClasses with an unspecified or empty `runtimeHandler` or those using a `.` character
in the handler are no longer valid, and must be migrated to a valid handler configuration (see
above).
{{% /capture %}}
@@ -0,0 +1,758 @@
---
title: Declarative Management of Kubernetes Objects Using Kustomize
content_template: templates/concept
weight: 40
---
{{% capture overview %}}
[Kustomize](https://github.com/kubernetes-sigs/kustomize) is a standalone tool
to customize Kubernetes objects
through a [kustomization file](https://github.com/kubernetes-sigs/kustomize/blob/master/docs/kustomization.yaml).
Since 1.14, Kubectl also
supports the management of Kubernetes objects using a kustomization file.
To view Resources found in a directory containing a kustomization file, run the following command:
```shell
kubectl kustomize <kustomization_directory>
```
To apply those Resources, run `kubectl apply` with `--kustomize` or `-k` flag:
```shell
kubectl apply -k <kustomization_directory>
```
{{% /capture %}}
{{% capture body %}}
## Overview of Kustomize
Kustomize is a tool for customizing Kubernetes configurations. It has the following features to manage application configuration files:
* generating resources from other sources
* setting cross-cutting fields for resources
* composing and customizing collections of resources
### Generating Resources
ConfigMap and Secret hold config or sensitive data that are used by other Kubernetes objects, such as Pods. The source
of truth of ConfigMap or Secret are usually from somewhere else, such as a `.properties` file or a ssh key file.
Kustomize has `secretGenerator` and `configMapGenerator`, which generate Secret and ConfigMap from files or literals.
#### configMapGenerator
To generate a ConfigMap from a file, add an entry to `files` list in `configMapGenerator`. Here is an example of generating a ConfigMap with a data item from a file content.
```shell
# Create a application.properties file
cat <<EOF >application.properties
FOO=Bar
EOF
cat <<EOF >./kustomization.yaml
configMapGenerator:
- name: example-configmap-1
files:
- application.properties
EOF
```
The generated ConfigMap can be checked by the following command:
```shell
kubectl kustomize ./
```
The generated ConfigMap is
```yaml
apiVersion: v1
data:
application.properties: |
FOO=Bar
kind: ConfigMap
metadata:
name: example-configmap-1-8mbdf7882g
```
ConfigMap can also be generated from literal key-value pairs. To generate a ConfigMap from a literal key-value pair, add an entry to `literals` list in configMapGenerator. Here is an example of generating a ConfigMap with a data item from a key-value pair.
```shell
cat <<EOF >./kustomization.yaml
configMapGenerator:
- name: example-configmap-2
literals:
- FOO=Bar
EOF
```
The generated ConfigMap can be checked by the following command:
```shell
kubectl kustomize ./
```
The generated ConfigMap is
```yaml
apiVersion: v1
data:
FOO: Bar
kind: ConfigMap
metadata:
name: example-configmap-2-g2hdhfc6tk
```
#### secretGenerator
Secret can also be generated from files or literal key-value pairs. To generate a Secret from a file, add an entry to `files` list in `secretGenerator`. Here is an example of generating a Secret with a data item from a file.
```shell
# Create a password.txt file
cat <<EOF >./password.txt
username=admin
password=secret
EOF
cat <<EOF >./kustomization.yaml
secretGenerator:
- name: example-secret-1
files:
- password.txt
EOF
```
The generated Secret is as follows:
```yaml
apiVersion: v1
data:
password.txt: dXNlcm5hbWU9YWRtaW4KcGFzc3dvcmQ9c2VjcmV0Cg==
kind: Secret
metadata:
name: example-secret-1-t2kt65hgtb
type: Opaque
```
To generate a Secret from a literal key-value pair, add an entry to `literals` list in `secretGenerator`. Here is an example of generating a Secret with a data item from a key-value pair.
```shell
cat <<EOF >./kustomization.yaml
secretGenerator:
- name: example-secret-2
literals:
- username=admin
- password=secert
EOF
```
The generated Secret is as follows:
```yaml
apiVersion: v1
data:
password: c2VjZXJ0
username: YWRtaW4=
kind: Secret
metadata:
name: example-secret-2-t52t6g96d8
type: Opaque
```
#### generatorOptions
The generated ConfigMaps and Secrets have a suffix appended by hashing the contents. This ensures that a new ConfigMap or Secret is generated when the content is changed. To disable the behavior of appending a suffix, one can use `generatorOptions`. Besides that, it is also possible to specify cross-cutting options for generated ConfigMaps and Secrets.
```shell
cat <<EOF >./kustomization.yaml
configMapGenerator:
- name: example-configmap-3
literals:
- FOO=Bar
generatorOptions:
disableNameSuffixHash: true
labels:
type: generated
annotations:
note: generated
EOF
```
Run`kubectl kustomize ./` to view the generated ConfigMap:
```yaml
apiVersion: v1
data:
FOO: Bar
kind: ConfigMap
metadata:
annotations:
note: generated
labels:
type: generated
name: example-configmap-3
```
### Setting cross-cutting fields
It is quite common to set cross-cutting fields for all Kubernetes resources in a project.
Some use cases for setting cross-cutting fields:
* setting the same namespace for all Resource
* adding the same name prefix or suffix
* adding the same set of labels
* adding the same set of annotations
Here is an example:
```shell
# Create a deployment.yaml
cat <<EOF >./deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
EOF
cat <<EOF >./kustomization.yaml
namespace: my-namespace
namePrefix: dev-
nameSuffix: "-001"
commonLabels:
app: bingo
commonAnnotations:
oncallPager: 800-555-1212
resources:
- deployment.yaml
EOF
```
Run `kubectl kustomize ./` to view those fields are all set in the Deployment Resource:
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
oncallPager: 800-555-1212
labels:
app: bingo
name: dev-nginx-deployment-001
namespace: my-namespace
spec:
selector:
matchLabels:
app: bingo
template:
metadata:
annotations:
oncallPager: 800-555-1212
labels:
app: bingo
spec:
containers:
- image: nginx
name: nginx
```
### Composing and Customizing Resources
It is common to compose a set of Resources in a project and manage them inside
the same file or directory.
Kustomize offers composing Resources from different files and applying patches or other customization to them.
#### Composing
Kustomize supports composition of different resources. The `resources` field, in the `kustomization.yaml` file, defines the list of resources to include in a configuration. Set the path to a resource's configuration file in the `resources` list.
Here is an example for an nginx application with a Deployment and a Service.
```shell
# Create a deployment.yaml file
cat <<EOF > deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-nginx
spec:
selector:
matchLabels:
run: my-nginx
replicas: 2
template:
metadata:
labels:
run: my-nginx
spec:
containers:
- name: my-nginx
image: nginx
ports:
- containerPort: 80
EOF
# Create a service.yaml file
cat <<EOF > service.yaml
apiVersion: v1
kind: Service
metadata:
name: my-nginx
labels:
run: my-nginx
spec:
ports:
- port: 80
protocol: TCP
selector:
run: my-nginx
EOF
# Create a kustomization.yaml composing them
cat <<EOF >./kustomization.yaml
resources:
- deployment.yaml
- service.yaml
EOF
```
The Resources from `kubectl kustomize ./` contains both the Deployment and the Service objects.
#### Customizing
On top of Resources, one can apply different customizations by applying patches. Kustomize supports different patching
mechanisms through `patchesStrategicMerge` and `patchesJson6902`. `patchesStrategicMerge` is a list of file paths. Each file should be resolved to a [strategic merge patch](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-api-machinery/strategic-merge-patch.md). The names inside the patches must match Resource names that are already loaded. Small patches that do one thing are recommended. For example, create one patch for increasing the deployment replica number and another patch for setting the memory limit.
```shell
# Create a deployment.yaml file
cat <<EOF > deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-nginx
spec:
selector:
matchLabels:
run: my-nginx
replicas: 2
template:
metadata:
labels:
run: my-nginx
spec:
containers:
- name: my-nginx
image: nginx
ports:
- containerPort: 80
EOF
# Create a patch increase_replicas.yaml
cat <<EOF > increase_replicas.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-nginx
spec:
replicas: 3
EOF
# Create another patch set_memory.yaml
cat <<EOF > set_memory.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-nginx
spec:
template:
spec:
containers:
- name: my-nginx
resources:
limits:
memory: 512Mi
EOF
cat <<EOF >./kustomization.yaml
resources:
- deployment.yaml
patchesStrategicMerge:
- increase_replicas.yaml
- set_memory.yaml
EOF
```
Run `kubectl kustomize ./` to view the Deployment:
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-nginx
spec:
replicas: 3
selector:
matchLabels:
run: my-nginx
template:
metadata:
labels:
run: my-nginx
spec:
containers:
- image: nginx
limits:
memory: 512Mi
name: my-nginx
ports:
- containerPort: 80
```
Not all Resources or fields support strategic merge patches. To support modifying arbitrary fields in arbitrary Resources,
Kustomize offers applying [JSON patch](https://tools.ietf.org/html/rfc6902) through `patchesJson6902`.
To find the correct Resource for a Json patch, the group, version, kind and name of that Resource need to be
specified in `kustomization.yaml`. For example, increasing the replica number of a Deployment object can also be done
through `patchesJson6902`.
```shell
# Create a deployment.yaml file
cat <<EOF > deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-nginx
spec:
selector:
matchLabels:
run: my-nginx
replicas: 2
template:
metadata:
labels:
run: my-nginx
spec:
containers:
- name: my-nginx
image: nginx
ports:
- containerPort: 80
EOF
# Create a json patch
cat <<EOF > patch.yaml
- op: replace
path: /spec/replicas
value: 3
EOF
# Create a kustomization.yaml
cat <<EOF >./kustomization.yaml
resources:
- deployment.yaml
patchesJson6902:
- target:
group: apps
version: v1
kind: Deployment
name: my-nginx
path: patch.yaml
EOF
```
Run `kubectl kustomize ./` to see the `replicas` field is updated:
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-nginx
spec:
replicas: 3
selector:
matchLabels:
run: my-nginx
template:
metadata:
labels:
run: my-nginx
spec:
containers:
- image: nginx
name: my-nginx
ports:
- containerPort: 80
```
In addition to patches, Kustomize also offers customizing container images or injecting field values from other objects into containers
without creating patches. For example, you can change the image used inside containers by specifying the new image in `images` field in `kustomization.yaml`.
```shell
cat <<EOF > deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-nginx
spec:
selector:
matchLabels:
run: my-nginx
replicas: 2
template:
metadata:
labels:
run: my-nginx
spec:
containers:
- name: my-nginx
image: nginx
ports:
- containerPort: 80
EOF
cat <<EOF >./kustomization.yaml
resources:
- deployment.yaml
images:
- name: nginx
newName: my.image.registry/nginx
newTag: 1.4.0
EOF
```
Run `kubectl kustomize ./` to see that the image being used is updated:
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-nginx
spec:
replicas: 2
selector:
matchLabels:
run: my-nginx
template:
metadata:
labels:
run: my-nginx
spec:
containers:
- image: my.image.registry/nginx:1.4.0
name: my-nginx
ports:
- containerPort: 80
```
Sometimes, the application running in a Pod may need to use configuration values from other objects. For example,
a Pod from a Deployment object need to read the corresponding Service name from Env or as a command argument.
Since the Service name may change as `namePrefix` or `nameSuffix` is added in the `kustomization.yaml` file. It is
not recommended to hard code the Service name in the command argument. For this usage, Kustomize can inject the Service name into containers through `vars`.
```shell
# Create a deployment.yaml file
cat <<EOF > deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-nginx
spec:
selector:
matchLabels:
run: my-nginx
replicas: 2
template:
metadata:
labels:
run: my-nginx
spec:
containers:
- name: my-nginx
image: nginx
command: ["start", "--host", "\$(MY_SERVICE_NAME)"]
EOF
# Create a service.yaml file
cat <<EOF > service.yaml
apiVersion: v1
kind: Service
metadata:
name: my-nginx
labels:
run: my-nginx
spec:
ports:
- port: 80
protocol: TCP
selector:
run: my-nginx
EOF
cat <<EOF >./kustomization.yaml
namePrefix: dev-
nameSuffix: "-001"
resources:
- deployment.yaml
- service.yaml
vars:
- name: MY_SERVICE_NAME
objref:
kind: Service
name: my-nginx
apiVersion: v1
EOF
```
Run `kubectl kustomize ./` to see that the Service name injected into containers is `dev-my-nginx-001`:
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: dev-my-nginx-001
spec:
replicas: 2
selector:
matchLabels:
run: my-nginx
template:
metadata:
labels:
run: my-nginx
spec:
containers:
- command:
- start
- --host
- dev-my-nginx-001
image: nginx
name: my-nginx
```
## Bases and Overlays
Kustomize has the concepts of **bases** and **overlays**. A **base** is a directory with a `kustomization.yaml`, which contains a
set of resources and associated customization. A base could be either a local directory or a directory from a remote repo,
as long as a `kustomization.yaml` is present inside. An **overlay** is a directory with a `kustomization.yaml` that refers to other
kustomization directories as its `bases`. A **base** has no knowledge of an overlay and can be used in multiple overlays.
An overlay may have multiple bases and it composes all resources
from bases and may also have customization on top of them.
Here is an example of a base.
```shell
# Create a directory to hold the base
mkdir base
# Create a base/deployment.yaml
cat <<EOF > base/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-nginx
spec:
selector:
matchLabels:
run: my-nginx
replicas: 2
template:
metadata:
labels:
run: my-nginx
spec:
containers:
- name: my-nginx
image: nginx
EOF
# Create a base/service.yaml file
cat <<EOF > base/service.yaml
apiVersion: v1
kind: Service
metadata:
name: my-nginx
labels:
run: my-nginx
spec:
ports:
- port: 80
protocol: TCP
selector:
run: my-nginx
EOF
# Create a base/kustomization.yaml
cat <<EOF > base/kustomization.yaml
resources:
- deployment.yaml
- service.yaml
```
This base can be used in multiple overlays. You can add different `namePrefix` or other cross-cutting fields
in different overlays. Here are two overlays using the same base.
```shell
mkdir dev
cat <<EOF > dev/kustomization.yaml
bases:
- ../base
namePrefix: dev-
EOF
mkdir prod
cat <<EOF > prod/kustomization.yaml
bases:
- ../base
namePrefix: prod-
EOF
```
## How to apply/view/delete objects using Kustomize
Use `--kustomize` or `-k` in `kubectl` commands to recognize Resources managed by `kustomization.yaml`.
Note that `-k` should point to a kustomization directory, such as
```shell
kubectl apply -k <kustomization directory>/
```
Given the following `kustomization.yaml`,
```shell
# Create a deployment.yaml file
cat <<EOF > deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-nginx
spec:
selector:
matchLabels:
run: my-nginx
replicas: 2
template:
metadata:
labels:
run: my-nginx
spec:
containers:
- name: my-nginx
image: nginx
ports:
- containerPort: 80
EOF
# Create a kustomization.yaml
cat <<EOF >./kustomization.yaml
namePrefix: dev-
commonLabels:
app: my-nginx
resources:
- deployment.yaml
EOF
```
Running the following command will apply the Deployment object `dev-my-nginx`:
```shell
> kubectl apply -k ./
deployment.apps/dev-my-nginx created
```
Running the following command will get he Deployment object `dev-my-nginx`:
```shell
kubectl get -k ./
```
or
```shell
kubectl describe -k ./
```
Running the following command will delete the Deployment object `dev-my-nginx`:
```shell
> kubectl delete -k ./
deployment.apps "dev-my-nginx" deleted
```
## Kustomize Feature List
Here is a list of all the features in Kustomize.
| Field | Type | Explanation |
|-----------------------|--------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------|
| namespace | string | add namespace to all resources |
| namePrefix | string | value of this field is prepended to the names of all resources |
| nameSuffix | string | value of this field is appended to the names of all resources |
| commonlabels | map[string]string | labels to add to all resources and selectors |
| commonAnnotations | map[string]string | annotations to add to all resources |
| resources | []string | each entry in this list must resolve to an existing resource configuration file |
| configmapGenerator | [][ConfigMapArgs](https://github.com/kubernetes-sigs/kustomize/blob/master/pkg/types/kustomization.go#L195) | Each entry in this list generates a ConfigMap |
| secretGenerator | [][SecretArgs](https://github.com/kubernetes-sigs/kustomize/blob/master/pkg/types/kustomization.go#L201) | Each entry in this list generates a Secret |
| generatorOptions | [GeneratorOptions](https://github.com/kubernetes-sigs/kustomize/blob/master/pkg/types/kustomization.go#L239) | Modify behaviors of all ConfigMap and Secret generatos |
| bases | []string | Each entry in this list should resolve to a directory containing a kustomization.yaml file |
| patchesStrategicMerge | []string | Each entry in this list should resolve a strategic merge patch of a Kubernetes object |
| patchesJson6902 | [][Json6902](https://github.com/kubernetes-sigs/kustomize/blob/master/pkg/patch/json6902.go#L23) | Each entry in this list should resolve to a Kubernetes object and a Json Patch |
| vars | [][Var](https://github.com/kubernetes-sigs/kustomize/blob/master/pkg/types/var.go#L31) | Each entry is to capture text from one resource's field |
| images | [][Image](https://github.com/kubernetes-sigs/kustomize/blob/master/pkg/image/image.go#L23) | Each entry is to modify the name, tags and/or digest for one image without creating patches |
| configurations | []string | Each entry in this list should resolve to a file containing [Kustomize transformer configurations](https://github.com/kubernetes-sigs/kustomize/tree/master/examples/transformerconfigs) |
| crds | []string | Each entry in this list should resolve to an OpenAPI definition file for Kubernetes types |
{{% capture whatsnext %}}
- [Kustomize](https://github.com/kubernetes-sigs/kustomize)
- [Kubectl Book](https://kubectl.docs.kubernetes.io)
- [Kubectl Command Reference](/docs/reference/generated/kubectl/kubectl/)
- [Kubernetes API Reference](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/)
{{% /capture %}}
@@ -7,7 +7,8 @@ weight: 10
{{% capture overview %}}
The `kubectl` command-line tool supports several different ways to create and manage
Kubernetes objects. This document provides an overview of the different
approaches.
approaches. Read the [Kubectl book](https://kubectl.docs.kubernetes.io) for
details of managing objects by Kubectl.
{{% /capture %}}
{{% capture body %}}
@@ -179,6 +180,7 @@ Disadvantages compared to imperative object configuration:
- [Managing Kubernetes Objects Using Object Configuration (Imperative)](/docs/concepts/overview/object-management-kubectl/imperative-config/)
- [Managing Kubernetes Objects Using Object Configuration (Declarative)](/docs/concepts/overview/object-management-kubectl/declarative-config/)
- [Kubectl Command Reference](/docs/reference/generated/kubectl/kubectl-commands/)
- [Kubectl Book](https://kubectl.docs.kubernetes.io)
- [Kubernetes API Reference](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/)
{{< comment >}}
@@ -42,11 +42,11 @@ Here's an example `.yaml` file that shows the required fields and object spec fo
{{< codenew file="application/deployment.yaml" >}}
One way to create a Deployment using a `.yaml` file like the one above is to use the
[`kubectl create`](/docs/reference/generated/kubectl/kubectl-commands#create) command
[`kubectl apply`](/docs/reference/generated/kubectl/kubectl-commands#apply) command
in the `kubectl` command-line interface, passing the `.yaml` file as an argument. Here's an example:
```shell
kubectl create -f https://k8s.io/examples/application/deployment.yaml --record
kubectl apply -f https://k8s.io/examples/application/deployment.yaml --record
```
The output is similar to this:
@@ -35,7 +35,7 @@ Create an nginx Pod, and note that it has a container port specification:
This makes it accessible from any node in your cluster. Check the nodes the Pod is running on:
```shell
kubectl create -f ./run-my-nginx.yaml
kubectl apply -f ./run-my-nginx.yaml
kubectl get pods -l run=my-nginx -o wide
```
```
@@ -71,7 +71,7 @@ kubectl expose deployment/my-nginx
service/my-nginx exposed
```
This is equivalent to `kubectl create -f` the following yaml:
This is equivalent to `kubectl apply -f` the following yaml:
{{< codenew file="service/networking/nginx-svc.yaml" >}}
@@ -232,7 +232,7 @@ You can acquire all these from the [nginx https example](https://github.com/kube
```shell
make keys secret KEY=/tmp/nginx.key CERT=/tmp/nginx.crt SECRET=/tmp/secret.json
kubectl create -f /tmp/secret.json
kubectl apply -f /tmp/secret.json
```
```
secret/nginxsecret created
@@ -269,7 +269,7 @@ data:
Now create the secrets using the file:
```shell
kubectl create -f nginxsecrets.yaml
kubectl apply -f nginxsecrets.yaml
kubectl get secrets
```
```
@@ -312,7 +312,7 @@ Let's test this from a pod (the same secret is being reused for simplicity, the
{{< codenew file="service/networking/curlpod.yaml" >}}
```shell
kubectl create -f ./curlpod.yaml
kubectl apply -f ./curlpod.yaml
kubectl get pods -l app=curlpod
```
```
@@ -170,10 +170,10 @@ following pod-specific DNS policies. These policies are specified in the
for details on how DNS queries are handled in those cases.
- "`ClusterFirstWithHostNet`": For Pods running with hostNetwork, you should
explicitly set its DNS policy "`ClusterFirstWithHostNet`".
- "`None`": A new option value introduced in Kubernetes v1.9 (Beta in v1.10). It
allows a Pod to ignore DNS settings from the Kubernetes environment. All DNS
settings are supposed to be provided using the `dnsConfig` field in the Pod Spec.
See [DNS config](#dns-config) subsection below.
- "`None`": It allows a Pod to ignore DNS settings from the Kubernetes
environment. All DNS settings are supposed to be provided using the
`dnsConfig` field in the Pod Spec.
See [Pod's DNS config](#pod-s-dns-config) subsection below.
{{< note >}}
"Default" is not the default DNS policy. If `dnsPolicy` is not
@@ -205,13 +205,7 @@ spec:
### Pod's DNS Config
Kubernetes v1.9 introduces an Alpha feature (Beta in v1.10) that allows users more
control on the DNS settings for a Pod. This feature is enabled by default in v1.10.
To enable this feature in v1.9, the cluster administrator
needs to enable the `CustomPodDNS` feature gate on the apiserver and the kubelet,
for example, "`--feature-gates=CustomPodDNS=true,...`".
When the feature gate is enabled, users can set the `dnsPolicy` field of a Pod
to "`None`" and they can add a new field `dnsConfig` to a Pod Spec.
Pod's DNS Config allows users more control on the DNS settings for a Pod.
The `dnsConfig` field is optional and it can work with any `dnsPolicy` settings.
However, when a Pod's `dnsPolicy` is set to "`None`", the `dnsConfig` field has
@@ -257,6 +251,16 @@ search default.svc.cluster.local svc.cluster.local cluster.local
options ndots:5
```
### Feature availability
The availability of Pod DNS Config and DNS Policy "`None`"" is shown as below.
| k8s version | Feature support |
| :---------: |:-----------:|
| 1.14 | Stable |
| 1.10 | Beta (on by default)|
| 1.9 | Alpha |
{{% /capture %}}
{{% capture whatsnext %}}
@@ -136,7 +136,7 @@ There are existing Kubernetes concepts that allow you to expose a single Service
{{< codenew file="service/networking/ingress.yaml" >}}
If you create it using `kubectl create -f` you should see:
If you create it using `kubectl apply -f` you should see:
```shell
kubectl get ingress test-ingress
@@ -190,7 +190,7 @@ spec:
servicePort: 8080
```
When you create the Ingress with `kubectl create -f`:
When you create the ingress with `kubectl apply -f`:
```shell
kubectl describe ingress simple-fanout-example
@@ -179,8 +179,8 @@ However, the particular path specified in the custom recycler pod template in th
### Expanding Persistent Volumes Claims
{{< feature-state for_k8s_version="v1.8" state="alpha" >}}
{{< feature-state for_k8s_version="v1.11" state="beta" >}}
Support for expanding PersistentVolumeClaims (PVCs) is now enabled by default. You can expand
the following types of volumes:
@@ -193,6 +193,7 @@ the following types of volumes:
* Azure Disk
* Portworx
* FlexVolumes
* CSI
You can only expand a PVC if its storage class's `allowVolumeExpansion` field is set to true.
@@ -214,6 +215,13 @@ To request a larger volume for a PVC, edit the PVC object and specify a larger
size. This triggers expansion of the volume that backs the underlying `PersistentVolume`. A
new `PersistentVolume` is never created to satisfy the claim. Instead, an existing volume is resized.
#### CSI Volume expansion
{{< feature-state for_k8s_version="v1.14" state="alpha" >}}
CSI volume expansion requires enabling `ExpandCSIVolumes` feature gate and also requires specific CSI driver to support volume expansion. Please refer to documentation of specific CSI driver for more information.
#### Resizing a volume containing a file system
You can only resize volumes containing a file system if the file system is XFS, Ext3, or Ext4.
@@ -312,7 +320,7 @@ Currently, storage size is the only resource that can be set or requested. Futu
{{< feature-state for_k8s_version="v1.13" state="beta" >}}
Prior to Kubernetes 1.9, all volume plugins created a filesystem on the persistent volume.
Now, you can set the value of `volumeMode` to `raw` to use a raw block device, or `filesystem`
Now, you can set the value of `volumeMode` to `block` to use a raw block device, or `filesystem`
to use a filesystem. `filesystem` is the default if the value is omitted. This is an optional API
parameter.
@@ -151,6 +151,11 @@ The following plugins support `WaitForFirstConsumer` with pre-created Persistent
* All of the above
* [Local](#local)
{{< feature-state state="beta" for_k8s_version="1.14" >}}
[CSI volumes](/docs/concepts/storage/volumes/#csi) are also supported with dynamic provisioning
and pre-created PVs, but you'll need to look at the documentation for a specific CSI driver
to see its supported topology keys and examples. The `CSINodeInfo` feature gate must be enabled.
### Allowed Topologies
When a cluster operator specifies the `WaitForFirstConsumer` volume binding mode, it is no longer necessary
@@ -739,7 +744,7 @@ references it.
### Local
{{< feature-state for_k8s_version="v1.10" state="beta" >}}
{{< feature-state for_k8s_version="v1.14" state="stable" >}}
```yaml
kind: StorageClass
@@ -750,7 +755,7 @@ provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
```
Local volumes do not support dynamic provisioning yet, however a StorageClass
Local volumes do not currently support dynamic provisioning, however a StorageClass
should still be created to delay volume binding until pod scheduling. This is
specified by the `WaitForFirstConsumer` volume binding mode.
+132 -22
View File
@@ -70,6 +70,7 @@ Kubernetes supports several types of Volumes:
* [azureDisk](#azuredisk)
* [azureFile](#azurefile)
* [cephfs](#cephfs)
* [cinder](#cinder)
* [configMap](#configmap)
* [csi](#csi)
* [downwardAPI](#downwardapi)
@@ -148,6 +149,17 @@ spec:
fsType: ext4
```
#### CSI Migration
{{< feature-state for_k8s_version="v1.14" state="alpha" >}}
The CSI Migration feature for awsElasticBlockStore, when enabled, shims all plugin operations
from the existing in-tree plugin to the `ebs.csi.aws.com` Container
Storage Interface (CSI) Driver. In order to use this feature, the [AWS EBS CSI
Driver](https://github.com/kubernetes-sigs/aws-ebs-csi-driver)
must be installed on the cluster and the `CSIMigration` and `CSIMigrationAWS`
Alpha features must be enabled.
### azureDisk {#azuredisk}
A `azureDisk` is used to mount a Microsoft Azure [Data Disk](https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-linux-about-disks-vhds/) into a Pod.
@@ -176,6 +188,48 @@ You must have your own Ceph server running with the share exported before you ca
See the [CephFS example](https://github.com/kubernetes/examples/tree/{{< param "githubbranch" >}}/staging/volumes/cephfs/) for more details.
### cinder {#cinder}
{{< note >}}
Prerequisite: Kubernetes with OpenStack Cloud Provider configured. For cloudprovider
configuration please refer [cloud provider openstack](https://kubernetes.io/docs/concepts/cluster-administration/cloud-providers/#openstack).
{{< /note >}}
`cinder` is used to mount OpenStack Cinder Volume into your Pod.
#### Cinder Volume Example configuration
```yaml
apiVersion: v1
kind: Pod
metadata:
name: test-cinder
spec:
containers:
- image: k8s.gcr.io/test-webserver
name: test-cinder-container
volumeMounts:
- mountPath: /test-cinder
name: test-volume
volumes:
- name: test-volume
# This OpenStack volume must already exist.
cinder:
volumeID: <volume-id>
fsType: ext4
```
#### CSI Migration
{{< feature-state for_k8s_version="v1.14" state="alpha" >}}
The CSI Migration feature for Cinder, when enabled, shims all plugin operations
from the existing in-tree plugin to the `cinder.csi.openstack.org` Container
Storage Interface (CSI) Driver. In order to use this feature, the [Openstack Cinder CSI
Driver](https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/using-cinder-csi-plugin.md)
must be installed on the cluster and the `CSIMigration` and `CSIMigrationOpenStack`
Alpha features must be enabled.
### configMap {#configmap}
The [`configMap`](/docs/tasks/configure-pod-container/configure-pod-configmap/) resource
@@ -401,6 +455,17 @@ spec:
fsType: ext4
```
#### CSI Migration
{{< feature-state for_k8s_version="v1.14" state="alpha" >}}
The CSI Migration feature for GCE PD, when enabled, shims all plugin operations
from the existing in-tree plugin to the `pd.csi.storage.gke.io` Container
Storage Interface (CSI) Driver. In order to use this feature, the [GCE PD CSI
Driver](https://github.com/kubernetes-sigs/gcp-compute-persistent-disk-csi-driver)
must be installed on the cluster and the `CSIMigration` and `CSIMigrationGCE`
Alpha features must be enabled.
### gitRepo (deprecated) {#gitrepo}
{{< warning >}}
@@ -535,14 +600,7 @@ See the [iSCSI example](https://github.com/kubernetes/examples/tree/{{< param "g
### local {#local}
{{< feature-state for_k8s_version="v1.10" state="beta" >}}
{{< note >}}
The alpha PersistentVolume NodeAffinity annotation has been deprecated
and will be removed in a future release. Existing PersistentVolumes using this
annotation must be updated by the user to use the new PersistentVolume
`NodeAffinity` field.
{{< /note >}}
{{< feature-state for_k8s_version="v1.14" state="stable" >}}
A `local` volume represents a mounted local storage device such as a disk,
partition or directory.
@@ -608,7 +666,8 @@ selectors, Pod affinity, and Pod anti-affinity.
An external static provisioner can be run separately for improved management of
the local volume lifecycle. Note that this provisioner does not support dynamic
provisioning yet. For an example on how to run an external local provisioner,
see the [local volume provisioner user guide](https://github.com/kubernetes-incubator/external-storage/tree/master/local-volume).
see the [local volume provisioner user
guide](https://github.com/kubernetes-sigs/sig-storage-local-static-provisioner).
{{< note >}}
The local PersistentVolume requires manual cleanup and deletion by the
@@ -1073,13 +1132,14 @@ spec:
### Using subPath with expanded environment variables
{{< feature-state for_k8s_version="v1.11" state="alpha" >}}
{{< feature-state for_k8s_version="v1.14" state="alpha" >}}
`subPath` directory names can also be constructed from Downward API environment variables.
Use the `subPathExpr` field to construct `subPath` directory names from Downward API environment variables.
Before you use this feature, you must enable the `VolumeSubpathEnvExpansion` feature gate.
The `subPath` and `subPathExpr` properties are mutually exclusive.
In this example, a Pod uses `subPath` to create a directory `pod1` within the hostPath volume `/var/log/pods`, using the pod name from the Downward API. The host directory `/var/log/pods/pod1` is mounted at `/logs` in the container.
In this example, a Pod uses `subPathExpr` to create a directory `pod1` within the hostPath volume `/var/log/pods`, using the pod name from the Downward API. The host directory `/var/log/pods/pod1` is mounted at `/logs` in the container.
```yaml
apiVersion: v1
@@ -1100,7 +1160,7 @@ spec:
volumeMounts:
- name: workdir1
mountPath: /logs
subPath: $(POD_NAME)
subPathExpr: $(POD_NAME)
restartPolicy: Never
volumes:
- name: workdir1
@@ -1217,28 +1277,78 @@ persistent volume:
#### CSI raw block volume support
{{< feature-state for_k8s_version="v1.11" state="alpha" >}}
{{< feature-state for_k8s_version="v1.14" state="beta" >}}
Starting with version 1.11, CSI introduced support for raw block volumes, which
relies on the raw block volume feature that was introduced in a previous version of
Kubernetes. This feature will make it possible for vendors with external CSI drivers to
implement raw block volumes support in Kubernetes workloads.
CSI block volume support is feature-gated and turned off by default. To run CSI with
block volume support enabled, a cluster administrator must enable the feature for each
Kubernetes component using the following feature gate flags:
```
--feature-gates=BlockVolume=true,CSIBlockVolume=true
```
CSI block volume support is feature-gated, but enabled by default. The two
feature gates which must be enabled for this feature are `BlockVolume` and
`CSIBlockVolume`.
Learn how to
[setup your PV/PVC with raw block volume support](/docs/concepts/storage/persistent-volumes/#raw-block-volume-support).
#### Developer resources
#### CSI ephemeral volumes
{{< feature-state for_k8s_version="v1.14" state="alpha" >}}
This feature allows CSI volumes to be directly embedded in the Pod specification instead of a PersistentVolume. Volumes specified in this way are ephemeral and do not persist across Pod restarts.
Example:
```yaml
kind: Pod
apiVersion: v1
metadata:
name: my-csi-app
spec:
containers:
- name: my-frontend
image: busybox
volumeMounts:
- mountPath: "/data"
name: my-csi-inline-vol
command: [ "sleep", "1000000" ]
volumes:
- name: my-csi-inline-vol
csi:
driver: inline.storage.kubernetes.io
volumeAttributes:
foo: bar
```
This feature requires CSIInlineVolume feature gate to be enabled:
```
--feature-gates=CSIInlineVolume=true
```
CSI ephemeral volumes are only supported by a subset of CSI drivers. Please see the list of CSI drivers [here](https://kubernetes-csi.github.io/docs/drivers.html).
# Developer resources
For more information on how to develop a CSI driver, refer to the [kubernetes-csi
documentation](https://kubernetes-csi.github.io/docs/)
#### Migrating to CSI drivers from in-tree plugins
{{< feature-state for_k8s_version="v1.14" state="alpha" >}}
The CSI Migration feature, when enabled, directs operations against existing in-tree
plugins to corresponding CSI plugins (which are expected to be installed and configured).
The feature implements the necessary translation logic and shims to re-route the
operations in a seamless fashion. As a result, operators do not have to make any
configuration changes to existing Storage Classes, PVs or PVCs (referring to
in-tree plugins) when transitioning to a CSI driver that supersedes an in-tree plugin.
In the alpha state, the operations and features that are supported include
provisioning/delete, attach/detach and mount/unmount of volumes with `volumeMode` set to `filesystem`
In-tree plugins that support CSI Migration and have a corresponding CSI driver implemented
are listed in the "Types of Volumes" section above.
### Flexvolume {#flexVolume}
Flexvolume is an out-of-tree plugin interface that has existed in Kubernetes
@@ -42,7 +42,7 @@ You can describe a DaemonSet in a YAML file. For example, the `daemonset.yaml` f
* Create a DaemonSet based on the YAML file:
```
kubectl create -f https://k8s.io/examples/controllers/daemonset.yaml
kubectl apply -f https://k8s.io/examples/controllers/daemonset.yaml
```
### Required Fields
@@ -73,7 +73,7 @@ In this example:
To create this Deployment, run the following command:
```shell
kubectl create -f https://k8s.io/examples/controllers/nginx-deployment.yaml
kubectl apply -f https://k8s.io/examples/controllers/nginx-deployment.yaml
```
{{< note >}}
@@ -455,7 +455,7 @@ kubectl rollout history deployment.v1.apps/nginx-deployment
```
deployments "nginx-deployment"
REVISION CHANGE-CAUSE
1 kubectl create --filename=https://k8s.io/examples/controllers/nginx-deployment.yaml --record=true
1 kubectl apply --filename=https://k8s.io/examples/controllers/nginx-deployment.yaml --record=true
2 kubectl set image deployment.v1.apps/nginx-deployment nginx=nginx:1.9.1 --record=true
3 kubectl set image deployment.v1.apps/nginx-deployment nginx=nginx:1.91 --record=true
```
@@ -39,7 +39,7 @@ If you create the ReplicaSet and then view the Pod metadata, you can see
OwnerReferences field:
```shell
kubectl create -f https://k8s.io/examples/controllers/replicaset.yaml
kubectl apply -f https://k8s.io/examples/controllers/replicaset.yaml
kubectl get pods --output=yaml
```
@@ -39,7 +39,7 @@ It takes around 10s to complete.
You can run the example with this command:
```shell
kubectl create -f https://k8s.io/examples/controllers/job.yaml
kubectl apply -f https://k8s.io/examples/controllers/job.yaml
```
```
job "pi" created
@@ -54,7 +54,7 @@ Saving this manifest into `frontend.yaml` and submitting it to a Kubernetes clus
create the defined ReplicaSet and the Pods that it manages.
```shell
kubectl create -f http://k8s.io/examples/controllers/frontend.yaml
kubectl apply -f http://k8s.io/examples/controllers/frontend.yaml
```
You can then get the current ReplicaSets deployed:
@@ -162,7 +162,7 @@ Suppose you create the Pods after the frontend ReplicaSet has been deployed and
fulfill its replica count requirement:
```shell
kubectl create -f http://k8s.io/examples/pods/pod-rs.yaml
kubectl apply -f http://k8s.io/examples/pods/pod-rs.yaml
```
The new Pods will be acquired by the ReplicaSet, and then immediately terminated as the ReplicaSet would be over
@@ -184,12 +184,12 @@ pod2 0/1 Terminating 0 4s
If you create the Pods first:
```shell
kubectl create -f http://k8s.io/examples/pods/pod-rs.yaml
kubectl apply -f http://k8s.io/examples/pods/pod-rs.yaml
```
And then create the ReplicaSet however:
```shell
kubectl create -f http://k8s.io/examples/controllers/frontend.yaml
kubectl apply -f http://k8s.io/examples/controllers/frontend.yaml
```
You shall see that the ReplicaSet has acquired the Pods and has only created new ones according to its spec until the
@@ -308,7 +308,7 @@ create the defined HPA that autoscales the target ReplicaSet depending on the CP
of the replicated Pods.
```shell
kubectl create -f https://k8s.io/examples/controllers/hpa-rs.yaml
kubectl apply -f https://k8s.io/examples/controllers/hpa-rs.yaml
```
Alternatively, you can use the `kubectl autoscale` command to accomplish the same
@@ -55,7 +55,7 @@ This example ReplicationController config runs three copies of the nginx web ser
Run the example job by downloading the example file and then running this command:
```shell
kubectl create -f https://k8s.io/examples/controllers/replication.yaml
kubectl apply -f https://k8s.io/examples/controllers/replication.yaml
```
```
replicationcontroller/nginx created
@@ -180,7 +180,7 @@ spec:
This Pod can be started and debugged with the following commands:
```shell
kubectl create -f myapp.yaml
kubectl apply -f myapp.yaml
```
```
pod/myapp-pod created
@@ -240,7 +240,7 @@ Once we start the `mydb` and `myservice` services, we can see the Init Container
complete and the `myapp-pod` is created:
```shell
kubectl create -f services.yaml
kubectl apply -f services.yaml
```
```
service/myservice created
@@ -193,7 +193,7 @@ Once Pod is assigned to a node by scheduler, kubelet starts creating containers
## Pod readiness gate
{{< feature-state for_k8s_version="v1.12" state="beta" >}}
{{< feature-state for_k8s_version="v1.14" state="stable" >}}
In order to add extensibility to Pod readiness by enabling the injection of
extra feedbacks or signals into `PodStatus`, Kubernetes 1.11 introduced a