Merge branch 'master' of https://github.com/kubernetes/kubernetes.github.io into release-1.8
* 'master' of https://github.com/kubernetes/kubernetes.github.io: (33 commits) Added a prerequisite to using CoreDNS provider in federation (#5159) Edits "Creating a Deployment" section (#5195) Updating vSphere Cloud Provider Documentation (#5241) Update disruptions.md update OWNERS file Cilium network policy, update link: configure-pod-container -> administer-cluster Update volumes.md (#5214) Update images.md (#5212) fix resource quota redirect Fix link to Pod overview from concepts index fix the command output fix the command output Improve taint and toleration documentation Update deployment.md Update deployment.md Fix invalid internal links in federation doc Update images.md (#5034) Update docker-cli-to-kubectl.md (#5040) Update parallel-processing-expansion.md (#5060) Update cluster-management.md (#5130) ...
This commit is contained in:
@@ -171,7 +171,7 @@ Starting in Kubernetes 1.6, the NodeController is also responsible for evicting
|
||||
pods that are running on nodes with `NoExecute` taints, when the pods do not tolerate
|
||||
the taints. Additionally, as an alpha feature that is disabled by default, the
|
||||
NodeController is responsible for adding taints corresponding to node problems like
|
||||
node unreachable or not ready. See [this documentation](/docs/concepts/configuration/assign-pod-node/#taints-and-tolerations-beta-feature)
|
||||
node unreachable or not ready. See [this documentation](/docs/concepts/configuration/taint-and-toleration)
|
||||
for details about `NoExecute` taints and the alpha feature.
|
||||
|
||||
### Self-Registration of Nodes
|
||||
|
||||
@@ -38,7 +38,7 @@ why you might want multiple clusters are:
|
||||
* Scalability: There are scalability limits to a single kubernetes cluster (this
|
||||
should not be the case for most users. For more details:
|
||||
[Kubernetes Scaling and Performance Goals](https://git.k8s.io/community/sig-scalability/goals.md)).
|
||||
* [Hybrid cloud](###hybrid-cloud-capabilities): You can have multiple clusters on different cloud providers or
|
||||
* [Hybrid cloud](#hybrid-cloud-capabilities): You can have multiple clusters on different cloud providers or
|
||||
on-premises data centers.
|
||||
|
||||
### Caveats
|
||||
@@ -70,7 +70,7 @@ register each cluster's API endpoint and credentials with your
|
||||
Federation API Server (See the
|
||||
[federation admin guide](/docs/admin/federation/) for details).
|
||||
|
||||
Thereafter, your [API resources](##api-resources) can span different clusters
|
||||
Thereafter, your [API resources](#api-resources) can span different clusters
|
||||
and cloud providers.
|
||||
|
||||
## Setting up federation
|
||||
|
||||
@@ -118,4 +118,5 @@ spec:
|
||||
**Note**: a pod with the _unsafe_ sysctls specified above will fail to launch on
|
||||
any node which has not enabled those two _unsafe_ sysctls explicitly. As with
|
||||
_node-level_ sysctls it is recommended to use [_taints and toleration_
|
||||
feature](/docs/user-guide/kubectl/v1.6/#taint) or [labels on nodes](/docs/concepts/configuration/assign-pod-node/) to schedule those pods onto the right nodes.
|
||||
feature](/docs/user-guide/kubectl/v1.6/#taint) or [taints on nodes](/docs/concepts/configuration/taint-and-toleration/)
|
||||
to schedule those pods onto the right nodes.
|
||||
|
||||
@@ -298,231 +298,5 @@ Highly Available database statefulset has one master and three replicas, one may
|
||||
For more information on inter-pod affinity/anti-affinity, see the design doc
|
||||
[here](https://git.k8s.io/community/contributors/design-proposals/podaffinity.md).
|
||||
|
||||
## Taints and tolerations (beta feature)
|
||||
|
||||
Node affinity, described earlier, is a property of *pods* that *attracts* them to a set
|
||||
of nodes (either as a preference or a hard requirement). Taints are the opposite --
|
||||
they allow a *node* to *repel* a set of pods.
|
||||
|
||||
Taints and tolerations work together to ensure that pods are not scheduled
|
||||
onto inappropriate nodes. One or more taints are applied to a node; this
|
||||
marks that the node should not accept any pods that do not tolerate the taints.
|
||||
Tolerations are applied to pods, and allow (but do not require) the pods to schedule
|
||||
onto nodes with matching taints.
|
||||
|
||||
You add a taint to a node using [kubectl taint](/docs/user-guide/kubectl/v1.7/#taint).
|
||||
For example,
|
||||
|
||||
```shell
|
||||
kubectl taint nodes node1 key=value:NoSchedule
|
||||
```
|
||||
|
||||
places a taint on node `node1`. The taint has key `key`, value `value`, and taint effect `NoSchedule`.
|
||||
This means that no pod will be able to schedule onto `node1` unless it has a matching toleration.
|
||||
You specify a toleration for a pod in the PodSpec. Both of the following tolerations "match" the
|
||||
taint created by the `kubectl taint` line above, and thus a pod with either toleration would be able
|
||||
to schedule onto `node1`:
|
||||
|
||||
```yaml
|
||||
tolerations:
|
||||
- key: "key"
|
||||
operator: "Equal"
|
||||
value: "value"
|
||||
effect: "NoSchedule"
|
||||
```
|
||||
|
||||
```yaml
|
||||
tolerations:
|
||||
- key: "key"
|
||||
operator: "Exists"
|
||||
effect: "NoSchedule"
|
||||
```
|
||||
|
||||
A toleration "matches" a taint if the keys are the same and the effects are the same, and:
|
||||
|
||||
* the `operator` is `Exists` (in which case no `value` should be specified), or
|
||||
* the `operator` is `Equal` and the `value`s are equal
|
||||
|
||||
`Operator` defaults to `Equal` if not specified.
|
||||
|
||||
**NOTE:** There are two special cases:
|
||||
|
||||
* An empty `key` with operator `Exists` matches all keys, values and effects which means this
|
||||
will tolerate everything.
|
||||
|
||||
```yaml
|
||||
tolerations:
|
||||
- operator: "Exists"
|
||||
```
|
||||
|
||||
* An empty `effect` matches all effects with key `key`.
|
||||
|
||||
```yaml
|
||||
tolerations:
|
||||
- key: "key"
|
||||
operator: "Exists"
|
||||
```
|
||||
|
||||
The above example used `effect` of `NoSchedule`. Alternatively, you can use `effect` of `PreferNoSchedule`.
|
||||
This is a "preference" or "soft" version of `NoSchedule` -- the system will *try* to avoid placing a
|
||||
pod that does not tolerate the taint on the node, but it is not required. The third kind of `effect` is
|
||||
`NoExecute`, described later.
|
||||
|
||||
You can put multiple taints on the same node and multiple tolerations on the same pod.
|
||||
The way Kubernetes processes multiple taints and tolerations is like a filter: start
|
||||
with all of a node's taints, then ignore the ones for which the pod has a matching toleration; the
|
||||
remaining un-ignored taints have the indicated effects on the pod. In particular,
|
||||
|
||||
* if there is at least one un-ignored taint with effect `NoSchedule` then Kubernetes will not schedule
|
||||
the pod onto that node
|
||||
* if there is no un-ignored taint with effect `NoSchedule` but there is at least one un-ignored taint with
|
||||
effect `PreferNoSchedule` then Kubernetes will *try* to not schedule the pod onto the node
|
||||
* if there is at least one un-ignored taint with effect `NoExecute` then the pod will be evicted from
|
||||
the node (if it is already running on the node), and will not be
|
||||
scheduled onto the node (if it is not yet running on the node).
|
||||
|
||||
For example, imagine you taint a node like this
|
||||
|
||||
```shell
|
||||
kubectl taint nodes node1 key1=value1:NoSchedule
|
||||
kubectl taint nodes node1 key1=value1:NoExecute
|
||||
kubectl taint nodes node1 key2=value2:NoSchedule
|
||||
```
|
||||
|
||||
And a pod has two tolerations:
|
||||
|
||||
```yaml
|
||||
tolerations:
|
||||
- key: "key1"
|
||||
operator: "Equal"
|
||||
value: "value1"
|
||||
effect: "NoSchedule"
|
||||
- key: "key1"
|
||||
operator: "Equal"
|
||||
value: "value1"
|
||||
effect: "NoExecute"
|
||||
```
|
||||
|
||||
In this case, the pod will not be able to schedule onto the node, because there is no
|
||||
toleration matching the third taint. But it will be able to continue running if it is
|
||||
already running on the node when the taint is added, because the third taint is the only
|
||||
one of the three that is not tolerated by the pod.
|
||||
|
||||
Normally, if a taint with effect `NoExecute` is added to a node, then any pods that do
|
||||
not tolerate the taint will be evicted immediately, and any pods that do tolerate the
|
||||
taint will never be evicted. However, a toleration with `NoExecute` effect can specify
|
||||
an optional `tolerationSeconds` field that dictates how long the pod will stay bound
|
||||
to the node after the taint is added. For example,
|
||||
|
||||
```yaml
|
||||
tolerations:
|
||||
- key: "key1"
|
||||
operator: "Equal"
|
||||
value: "value1"
|
||||
effect: "NoExecute"
|
||||
tolerationSeconds: 3600
|
||||
```
|
||||
|
||||
means that if this pod is running and a matching taint is added to the node, then
|
||||
the pod will stay bound to the node for 3600 seconds, and then be evicted. If the
|
||||
taint is removed before that time, the pod will not be evicted.
|
||||
|
||||
### Example use cases
|
||||
|
||||
Taints and tolerations are a flexible way to steer pods away from nodes or evict
|
||||
pods that shouldn't be running. A few of the use cases are
|
||||
|
||||
* **dedicated nodes**: If you want to dedicate a set of nodes for exclusive use by
|
||||
a particular set of users, you can add a taint to those nodes (say,
|
||||
`kubectl taint nodes nodename dedicated=groupName:NoSchedule`) and then add a corresponding
|
||||
toleration to their pods (this would be done most easily by writing a custom
|
||||
[admission controller](/docs/admin/admission-controllers/)).
|
||||
The pods with the tolerations will then be allowed to use the tainted (dedicated) nodes as
|
||||
well as any other nodes in the cluster. If you want to dedicate the nodes to them *and*
|
||||
ensure they *only* use the dedicated nodes, then you should additionally add a label similar
|
||||
to the taint to the same set of nodes (e.g. `dedicated=groupName`), and the admission
|
||||
controller should additionally add a node affinity to require that the pods can only schedule
|
||||
onto nodes labeled with `dedicated=groupName`.
|
||||
|
||||
* **nodes with special hardware**: In a cluster where a small subset of nodes have specialized
|
||||
hardware (for example GPUs), it is desirable to keep pods that don't need the specialized
|
||||
hardware off of those nodes, thus leaving room for later-arriving pods that do need the
|
||||
specialized hardware. This can be done by tainting the nodes that have the specialized
|
||||
hardware (e.g. `kubectl taint nodes nodename special=true:NoSchedule` or
|
||||
`kubectl taint nodes nodename special=true:PreferNoSchedule`) and adding a corresponding
|
||||
toleration to pods that use the special hardware. As in the dedicated nodes use case,
|
||||
it is probably easiest to apply the tolerations using a custom
|
||||
[admission controller](/docs/admin/admission-controllers/)).
|
||||
For example, the admission controller could use
|
||||
some characteristic(s) of the pod to determine that the pod should be allowed to use
|
||||
the special nodes and hence the admission controller should add the toleration.
|
||||
To ensure that the pods that need
|
||||
the special hardware *only* schedule onto the nodes that have the special hardware, you will need some
|
||||
additional mechanism, e.g. you could represent the special resource using
|
||||
[opaque integer resources](/docs/concepts/configuration/manage-compute-resources-container/#opaque-integer-resources-alpha-feature)
|
||||
and request it as a resource in the PodSpec, or you could label the nodes that have
|
||||
the special hardware and use node affinity on the pods that need the hardware.
|
||||
|
||||
* **per-pod-configurable eviction behavior when there are node problems (alpha feature)**,
|
||||
which is described in the next section.
|
||||
|
||||
### Per-pod-configurable eviction behavior when there are node problems (alpha feature)
|
||||
|
||||
Earlier we mentioned the `NoExecute` taint effect, which affects pods that are already
|
||||
running on the node as follows
|
||||
|
||||
* pods that do not tolerate the taint are evicted immediately
|
||||
* pods that tolerate the taint without specifying `tolerationSeconds` in
|
||||
their toleration specification remain bound forever
|
||||
* pods that tolerate the taint with a specified `tolerationSeconds` remain
|
||||
bound for the specified amount of time
|
||||
|
||||
The above behavior is a beta feature. In addition, Kubernetes 1.6 has alpha
|
||||
support for representing node problems (currently only "node unreachable" and
|
||||
"node not ready", corresponding to the NodeCondition "Ready" being "Unknown" or
|
||||
"False" respectively) as taints. When the `TaintBasedEvictions` alpha feature
|
||||
is enabled (you can do this by including `TaintBasedEvictions=true` in `--feature-gates`, such as
|
||||
`--feature-gates=FooBar=true,TaintBasedEvictions=true`), the taints are automatically
|
||||
added by the NodeController and the normal logic for evicting pods from nodes
|
||||
based on the Ready NodeCondition is disabled.
|
||||
(Note: To maintain the existing [rate limiting](/docs/concepts/architecture/nodes/)
|
||||
behavior of pod evictions due to node problems, the system actually adds the taints
|
||||
in a rate-limited way. This prevents massive pod evictions in scenarios such
|
||||
as the master becoming partitioned from the nodes.)
|
||||
This alpha feature, in combination with `tolerationSeconds`, allows a pod
|
||||
to specify how long it should stay bound to a node that has one or both of these problems.
|
||||
|
||||
For example, an application with a lot of local state might want to stay
|
||||
bound to node for a long time in the event of network partition, in the hope
|
||||
that the partition will recover and thus the pod eviction can be avoided.
|
||||
The toleration the pod would use in that case would look like
|
||||
|
||||
```yaml
|
||||
tolerations:
|
||||
- key: "node.alpha.kubernetes.io/unreachable"
|
||||
operator: "Exists"
|
||||
effect: "NoExecute"
|
||||
tolerationSeconds: 6000
|
||||
```
|
||||
|
||||
(For the node not ready case, change the key to `node.alpha.kubernetes.io/notReady`.)
|
||||
|
||||
Note that Kubernetes automatically adds a toleration for
|
||||
`node.alpha.kubernetes.io/notReady` with `tolerationSeconds=300`
|
||||
unless the pod configuration provided
|
||||
by the user already has a toleration for `node.alpha.kubernetes.io/notReady`.
|
||||
Likewise it adds a toleration for
|
||||
`node.alpha.kubernetes.io/unreachable` with `tolerationSeconds=300`
|
||||
unless the pod configuration provided
|
||||
by the user already has a toleration for `node.alpha.kubernetes.io/unreachable`.
|
||||
|
||||
These automatically-added tolerations ensure that
|
||||
the default pod behavior of remaining bound for 5 minutes after one of these
|
||||
problems is detected is maintained.
|
||||
The two default tolerations are added by the [DefaultTolerationSeconds
|
||||
admission controller](https://git.k8s.io/kubernetes/plugin/pkg/admission/defaulttolerationseconds).
|
||||
|
||||
[DaemonSet](/docs/concepts/workloads/controllers/daemonset/) pods are created with
|
||||
`NoExecute` tolerations for `node.alpha.kubernetes.io/unreachable` and `node.alpha.kubernetes.io/notReady`
|
||||
with no `tolerationSeconds`. This ensures that DaemonSet pods are never evicted due
|
||||
to these problems, which matches the behavior when this feature is disabled.
|
||||
You may want to check [Taints](/docs/concepts/configuration/taint-and-toleration/)
|
||||
as well, which allow a *node* to *repel* a set of pods.
|
||||
|
||||
@@ -137,7 +137,7 @@ the option `-w 0` to `base64` commands or the pipeline `base64 | tr -d '\n'` if
|
||||
|
||||
#### Decoding a Secret
|
||||
|
||||
Get back the secret created in the previous section:
|
||||
Secrets can be retrieved via the `kubectl get secret` command. For example, to retrieve the secret created in the previous section:
|
||||
|
||||
```shell
|
||||
$ kubectl get secret mysecret -o yaml
|
||||
|
||||
@@ -0,0 +1,257 @@
|
||||
---
|
||||
approvers:
|
||||
- davidopp
|
||||
- kevin-wangzefeng
|
||||
- bsalamat
|
||||
title: Taints and Tolerations
|
||||
---
|
||||
|
||||
Node affinity, described [here](/docs/concepts/configuration/assign-pod-node/#node-affinity-beta-feature),
|
||||
is a property of *pods* that *attracts* them to a set of nodes (either as a
|
||||
preference or a hard requirement). Taints are the opposite -- they allow a
|
||||
*node* to *repel* a set of pods.
|
||||
|
||||
Taints and tolerations work together to ensure that pods are not scheduled
|
||||
onto inappropriate nodes. One or more taints are applied to a node; this
|
||||
marks that the node should not accept any pods that do not tolerate the taints.
|
||||
Tolerations are applied to pods, and allow (but do not require) the pods to schedule
|
||||
onto nodes with matching taints.
|
||||
|
||||
## Concepts
|
||||
|
||||
You add a taint to a node using [kubectl taint](/docs/user-guide/kubectl/v1.7/#taint).
|
||||
For example,
|
||||
|
||||
```shell
|
||||
kubectl taint nodes node1 key=value:NoSchedule
|
||||
```
|
||||
|
||||
places a taint on node `node1`. The taint has key `key`, value `value`, and taint effect `NoSchedule`.
|
||||
This means that no pod will be able to schedule onto `node1` unless it has a matching toleration.
|
||||
You specify a toleration for a pod in the PodSpec. Both of the following tolerations "match" the
|
||||
taint created by the `kubectl taint` line above, and thus a pod with either toleration would be able
|
||||
to schedule onto `node1`:
|
||||
|
||||
```yaml
|
||||
tolerations:
|
||||
- key: "key"
|
||||
operator: "Equal"
|
||||
value: "value"
|
||||
effect: "NoSchedule"
|
||||
```
|
||||
|
||||
```yaml
|
||||
tolerations:
|
||||
- key: "key"
|
||||
operator: "Exists"
|
||||
effect: "NoSchedule"
|
||||
```
|
||||
|
||||
A toleration "matches" a taint if the keys are the same and the effects are the same, and:
|
||||
|
||||
* the `operator` is `Exists` (in which case no `value` should be specified), or
|
||||
* the `operator` is `Equal` and the `value`s are equal
|
||||
|
||||
`Operator` defaults to `Equal` if not specified.
|
||||
|
||||
**NOTE:** There are two special cases:
|
||||
|
||||
* An empty `key` with operator `Exists` matches all keys, values and effects which means this
|
||||
will tolerate everything.
|
||||
|
||||
```yaml
|
||||
tolerations:
|
||||
- operator: "Exists"
|
||||
```
|
||||
|
||||
* An empty `effect` matches all effects with key `key`.
|
||||
|
||||
```yaml
|
||||
tolerations:
|
||||
- key: "key"
|
||||
operator: "Exists"
|
||||
```
|
||||
|
||||
The above example used `effect` of `NoSchedule`. Alternatively, you can use `effect` of `PreferNoSchedule`.
|
||||
This is a "preference" or "soft" version of `NoSchedule` -- the system will *try* to avoid placing a
|
||||
pod that does not tolerate the taint on the node, but it is not required. The third kind of `effect` is
|
||||
`NoExecute`, described later.
|
||||
|
||||
You can put multiple taints on the same node and multiple tolerations on the same pod.
|
||||
The way Kubernetes processes multiple taints and tolerations is like a filter: start
|
||||
with all of a node's taints, then ignore the ones for which the pod has a matching toleration; the
|
||||
remaining un-ignored taints have the indicated effects on the pod. In particular,
|
||||
|
||||
* if there is at least one un-ignored taint with effect `NoSchedule` then Kubernetes will not schedule
|
||||
the pod onto that node
|
||||
* if there is no un-ignored taint with effect `NoSchedule` but there is at least one un-ignored taint with
|
||||
effect `PreferNoSchedule` then Kubernetes will *try* to not schedule the pod onto the node
|
||||
* if there is at least one un-ignored taint with effect `NoExecute` then the pod will be evicted from
|
||||
the node (if it is already running on the node), and will not be
|
||||
scheduled onto the node (if it is not yet running on the node).
|
||||
|
||||
For example, imagine you taint a node like this
|
||||
|
||||
```shell
|
||||
kubectl taint nodes node1 key1=value1:NoSchedule
|
||||
kubectl taint nodes node1 key1=value1:NoExecute
|
||||
kubectl taint nodes node1 key2=value2:NoSchedule
|
||||
```
|
||||
|
||||
And a pod has two tolerations:
|
||||
|
||||
```yaml
|
||||
tolerations:
|
||||
- key: "key1"
|
||||
operator: "Equal"
|
||||
value: "value1"
|
||||
effect: "NoSchedule"
|
||||
- key: "key1"
|
||||
operator: "Equal"
|
||||
value: "value1"
|
||||
effect: "NoExecute"
|
||||
```
|
||||
|
||||
In this case, the pod will not be able to schedule onto the node, because there is no
|
||||
toleration matching the third taint. But it will be able to continue running if it is
|
||||
already running on the node when the taint is added, because the third taint is the only
|
||||
one of the three that is not tolerated by the pod.
|
||||
|
||||
Normally, if a taint with effect `NoExecute` is added to a node, then any pods that do
|
||||
not tolerate the taint will be evicted immediately, and any pods that do tolerate the
|
||||
taint will never be evicted. However, a toleration with `NoExecute` effect can specify
|
||||
an optional `tolerationSeconds` field that dictates how long the pod will stay bound
|
||||
to the node after the taint is added. For example,
|
||||
|
||||
```yaml
|
||||
tolerations:
|
||||
- key: "key1"
|
||||
operator: "Equal"
|
||||
value: "value1"
|
||||
effect: "NoExecute"
|
||||
tolerationSeconds: 3600
|
||||
```
|
||||
|
||||
means that if this pod is running and a matching taint is added to the node, then
|
||||
the pod will stay bound to the node for 3600 seconds, and then be evicted. If the
|
||||
taint is removed before that time, the pod will not be evicted.
|
||||
|
||||
## Example Use Cases
|
||||
|
||||
Taints and tolerations are a flexible way to steer pods *away* from nodes or evict
|
||||
pods that shouldn't be running. A few of the use cases are
|
||||
|
||||
* **Dedicated Nodes**: If you want to dedicate a set of nodes for exclusive use by
|
||||
a particular set of users, you can add a taint to those nodes (say,
|
||||
`kubectl taint nodes nodename dedicated=groupName:NoSchedule`) and then add a corresponding
|
||||
toleration to their pods (this would be done most easily by writing a custom
|
||||
[admission controller](/docs/admin/admission-controllers/)).
|
||||
The pods with the tolerations will then be allowed to use the tainted (dedicated) nodes as
|
||||
well as any other nodes in the cluster. If you want to dedicate the nodes to them *and*
|
||||
ensure they *only* use the dedicated nodes, then you should additionally add a label similar
|
||||
to the taint to the same set of nodes (e.g. `dedicated=groupName`), and the admission
|
||||
controller should additionally add a node affinity to require that the pods can only schedule
|
||||
onto nodes labeled with `dedicated=groupName`.
|
||||
|
||||
* **Nodes with Special Hardware**: In a cluster where a small subset of nodes have specialized
|
||||
hardware (for example GPUs), it is desirable to keep pods that don't need the specialized
|
||||
hardware off of those nodes, thus leaving room for later-arriving pods that do need the
|
||||
specialized hardware. This can be done by tainting the nodes that have the specialized
|
||||
hardware (e.g. `kubectl taint nodes nodename special=true:NoSchedule` or
|
||||
`kubectl taint nodes nodename special=true:PreferNoSchedule`) and adding a corresponding
|
||||
toleration to pods that use the special hardware. As in the dedicated nodes use case,
|
||||
it is probably easiest to apply the tolerations using a custom
|
||||
[admission controller](/docs/admin/admission-controllers/)).
|
||||
For example, the admission controller could use
|
||||
some characteristic(s) of the pod to determine that the pod should be allowed to use
|
||||
the special nodes and hence the admission controller should add the toleration.
|
||||
To ensure that the pods that need
|
||||
the special hardware *only* schedule onto the nodes that have the special hardware, you will need some
|
||||
additional mechanism, e.g. you could represent the special resource using
|
||||
[opaque integer resources](/docs/concepts/configuration/manage-compute-resources-container/#opaque-integer-resources-alpha-feature)
|
||||
and request it as a resource in the PodSpec, or you could label the nodes that have
|
||||
the special hardware and use node affinity on the pods that need the hardware.
|
||||
|
||||
* **Taint based Evictions (alpha feature)**: A per-pod-configurable eviction behavior
|
||||
when there are node problems, which is described in the next section.
|
||||
|
||||
## Taint based Evictions
|
||||
|
||||
Earlier we mentioned the `NoExecute` taint effect, which affects pods that are already
|
||||
running on the node as follows
|
||||
|
||||
* pods that do not tolerate the taint are evicted immediately
|
||||
* pods that tolerate the taint without specifying `tolerationSeconds` in
|
||||
their toleration specification remain bound forever
|
||||
* pods that tolerate the taint with a specified `tolerationSeconds` remain
|
||||
bound for the specified amount of time
|
||||
|
||||
The above behavior is a beta feature. In addition, Kubernetes 1.6 has alpha
|
||||
support for representing node problems. In other words, the node controller
|
||||
automatically taints a node when certain condition is true. The builtin taints
|
||||
currently include:
|
||||
|
||||
* `node.alpha.kubernetes.io/notReady`: Node is not ready. This corresponds to
|
||||
the NodeCondition `Ready` being "`False`".
|
||||
* `node.alpha.kubernetes.io/unreachable`: Node is unreachable from the node
|
||||
controller. This corresponds to the NodeCondition `Ready` being "`Unknown`".
|
||||
* `node.kubernetes.io/outOfDisk`: Node becomes out of disk.
|
||||
* `node.kubernetes.io/memoryPressure`: Node has memory pressure.
|
||||
* `node.kubernetes.io/diskPressure`: Node has disk pressure.
|
||||
* `node.kubernetes.io/networkUnavailable`: Node's network is unavailable.
|
||||
* `node.cloudprovider.kubernetes.io/uninitialized`: When kubelet is started
|
||||
with "external" cloud provider, it sets this taint on a node to mark it
|
||||
as unusable. When a controller from the cloud-controller-manager initializes
|
||||
this node, kubelet removes this taint.
|
||||
|
||||
When the `TaintBasedEvictions` alpha feature is enabled (you can do this by
|
||||
including `TaintBasedEvictions=true` in `--feature-gates`, such as
|
||||
`--feature-gates=FooBar=true,TaintBasedEvictions=true`), the taints are automatically
|
||||
added by the NodeController (or kubelet) and the normal logic for evicting pods from nodes
|
||||
based on the Ready NodeCondition is disabled.
|
||||
(Note: To maintain the existing [rate limiting](/docs/concepts/architecture/nodes/)
|
||||
behavior of pod evictions due to node problems, the system actually adds the taints
|
||||
in a rate-limited way. This prevents massive pod evictions in scenarios such
|
||||
as the master becoming partitioned from the nodes.)
|
||||
This alpha feature, in combination with `tolerationSeconds`, allows a pod
|
||||
to specify how long it should stay bound to a node that has one or both of these problems.
|
||||
|
||||
For example, an application with a lot of local state might want to stay
|
||||
bound to node for a long time in the event of network partition, in the hope
|
||||
that the partition will recover and thus the pod eviction can be avoided.
|
||||
The toleration the pod would use in that case would look like
|
||||
|
||||
```yaml
|
||||
tolerations:
|
||||
- key: "node.alpha.kubernetes.io/unreachable"
|
||||
operator: "Exists"
|
||||
effect: "NoExecute"
|
||||
tolerationSeconds: 6000
|
||||
```
|
||||
|
||||
Note that Kubernetes automatically adds a toleration for
|
||||
`node.alpha.kubernetes.io/notReady` with `tolerationSeconds=300`
|
||||
unless the pod configuration provided
|
||||
by the user already has a toleration for `node.alpha.kubernetes.io/notReady`.
|
||||
Likewise it adds a toleration for
|
||||
`node.alpha.kubernetes.io/unreachable` with `tolerationSeconds=300`
|
||||
unless the pod configuration provided
|
||||
by the user already has a toleration for `node.alpha.kubernetes.io/unreachable`.
|
||||
|
||||
These automatically-added tolerations ensure that
|
||||
the default pod behavior of remaining bound for 5 minutes after one of these
|
||||
problems is detected is maintained.
|
||||
The two default tolerations are added by the [DefaultTolerationSeconds
|
||||
admission controller](https://git.k8s.io/kubernetes/plugin/pkg/admission/defaulttolerationseconds).
|
||||
|
||||
[DaemonSet](/docs/concepts/workloads/controllers/daemonset/) pods are created with
|
||||
`NoExecute` tolerations for the following taints with no `tolerationSeconds`:
|
||||
|
||||
* `node.alpha.kubernetes.io/unreachable`
|
||||
* `node.alpha.kubernetes.io/notReady`
|
||||
* `node.kubernetes.io/memoryPressure`
|
||||
* `node.kubernetes.io/diskPressure`
|
||||
* `node.kubernetes.io/outOfDisk` (*only for critical pods*)
|
||||
|
||||
This ensures that DaemonSet pods are never evicted due to these problems,
|
||||
which matches the behavior when this feature is disabled.
|
||||
@@ -113,6 +113,7 @@ You first need to create a registry and generate credentials, complete documenta
|
||||
the [Azure container registry documentation](https://docs.microsoft.com/en-us/azure/container-registry/container-registry-get-started-azure-cli).
|
||||
|
||||
Once you have created your container registry, you will use the following credentials to login:
|
||||
|
||||
* `DOCKER_USER` : service principal, or admin username
|
||||
* `DOCKER_PASSWORD`: service principal password, or admin user password
|
||||
* `DOCKER_REGISTRY_SERVER`: `${some-registry-name}.azurecr.io`
|
||||
@@ -263,7 +264,7 @@ type: kubernetes.io/dockerconfigjson
|
||||
```
|
||||
|
||||
If you get the error message `error: no objects passed to create`, it may mean the base64 encoded string is invalid.
|
||||
If you get an error message like `Secret "myregistrykey" is invalid: data[.dockerconfigjson]: invalid value ...` it means
|
||||
If you get an error message like `Secret "myregistrykey" is invalid: data[.dockerconfigjson]: invalid value ...`, it means
|
||||
the data was successfully un-base64 encoded, but could not be parsed as a `.docker/config.json` file.
|
||||
|
||||
#### Referring to an imagePullSecrets on a Pod
|
||||
@@ -300,26 +301,26 @@ common use cases and suggested solutions.
|
||||
|
||||
1. Cluster running only non-proprietary (e.g. open-source) images. No need to hide images.
|
||||
- Use public images on the Docker hub.
|
||||
- no configuration required
|
||||
- on GCE/GKE, a local mirror is automatically used for improved speed and availability
|
||||
- No configuration required.
|
||||
- On GCE/GKE, a local mirror is automatically used for improved speed and availability.
|
||||
1. Cluster running some proprietary images which should be hidden to those outside the company, but
|
||||
visible to all cluster users.
|
||||
- Use a hosted private [Docker registry](https://docs.docker.com/registry/)
|
||||
- may be hosted on the [Docker Hub](https://hub.docker.com/account/signup/), or elsewhere.
|
||||
- manually configure .docker/config.json on each node as described above
|
||||
- Use a hosted private [Docker registry](https://docs.docker.com/registry/).
|
||||
- It may be hosted on the [Docker Hub](https://hub.docker.com/account/signup/), or elsewhere.
|
||||
- Manually configure .docker/config.json on each node as described above.
|
||||
- Or, run an internal private registry behind your firewall with open read access.
|
||||
- no Kubernetes configuration required
|
||||
- No Kubernetes configuration is required.
|
||||
- Or, when on GCE/GKE, use the project's Google Container Registry.
|
||||
- will work better with cluster autoscaling than manual node configuration
|
||||
- It will work better with cluster autoscaling than manual node configuration.
|
||||
- Or, on a cluster where changing the node configuration is inconvenient, use `imagePullSecrets`.
|
||||
1. Cluster with a proprietary images, a few of which require stricter access control
|
||||
- ensure [AlwaysPullImages admission controller](/docs/admin/admission-controllers/#alwayspullimages) is active, otherwise, all Pods potentially have access to all images
|
||||
1. Cluster with a proprietary images, a few of which require stricter access control.
|
||||
- Ensure [AlwaysPullImages admission controller](/docs/admin/admission-controllers/#alwayspullimages) is active. Otherwise, all Pods potentially have access to all images.
|
||||
- Move sensitive data into a "Secret" resource, instead of packaging it in an image.
|
||||
1. A multi-tenant cluster where each tenant needs own private registry
|
||||
- ensure [AlwaysPullImages admission controller](/docs/admin/admission-controllers/#alwayspullimages) is active, otherwise, all Pods of all tenants potentially have access to all images
|
||||
- run a private registry with authorization required.
|
||||
- generate registry credential for each tenant, put into secret, and populate secret to each tenant namespace.
|
||||
- tenant adds that secret to imagePullSecrets of each namespace.
|
||||
1. A multi-tenant cluster where each tenant needs own private registry.
|
||||
- Ensure [AlwaysPullImages admission controller](/docs/admin/admission-controllers/#alwayspullimages) is active. Otherwise, all Pods of all tenants potentially have access to all images.
|
||||
- Run a private registry with authorization required.
|
||||
- Generate registry credential for each tenant, put into secret, and populate secret to each tenant namespace.
|
||||
- The tenant adds that secret to imagePullSecrets of each namespace.
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
|
||||
@@ -21,18 +21,18 @@ Kubernetes contains a number of abstractions that represent the state of your sy
|
||||
|
||||
The basic Kubernetes objects include:
|
||||
|
||||
* [Pod](/docs/concepts/abstractions/pod/)
|
||||
* Service
|
||||
* Volume
|
||||
* Namespace
|
||||
* [Pod](/docs/concepts/workloads/pods/pod-overview/)
|
||||
* [Service](/docs/concepts/services-networking/service/)
|
||||
* [Volume](/docs/concepts/storage/volumes/)
|
||||
* [Namespace](/docs/concepts/overview/working-with-objects/namespaces/)
|
||||
|
||||
In addition, Kubernetes contains a number of higher-level abstractions called Controllers. Controllers build upon the basic objects, and provide additional functionality and convenience features. They include:
|
||||
|
||||
* ReplicaSet
|
||||
* Deployment
|
||||
* [StatefulSet](/docs/concepts/abstractions/controllers/statefulsets/)
|
||||
* DaemonSet
|
||||
* Job
|
||||
* [ReplicaSet](/docs/concepts/workloads/controllers/replicaset/)
|
||||
* [Deployment](/docs/concepts/workloads/controllers/deployment/)
|
||||
* [StatefulSet](/docs/concepts/workloads/controllers/statefulsets/)
|
||||
* [DaemonSet](/docs/concepts/workloads/controllers/daemonset/)
|
||||
* [Job](/docs/concepts/workloads/controllers/jobs-run-to-completion/)
|
||||
|
||||
## Kubernetes Control Plane
|
||||
|
||||
|
||||
@@ -99,9 +99,10 @@ exists as long as that Pod is running on that node. As the name says, it is
|
||||
initially empty. Containers in the pod can all read and write the same
|
||||
files in the `emptyDir` volume, though that volume can be mounted at the same
|
||||
or different paths in each container. When a Pod is removed from a node for
|
||||
any reason, the data in the `emptyDir` is deleted forever. NOTE: a container
|
||||
crashing does *NOT* remove a pod from a node, so the data in an `emptyDir`
|
||||
volume is safe across container crashes.
|
||||
any reason, the data in the `emptyDir` is deleted forever.
|
||||
|
||||
**Note:** a container crashing does *NOT* remove a pod from a node, so the data in an `emptyDir` volume is safe across container crashes.
|
||||
{: .note}
|
||||
|
||||
Some uses for an `emptyDir` are:
|
||||
|
||||
@@ -189,8 +190,8 @@ Disk](http://cloud.google.com/compute/docs/disks) into your pod. Unlike
|
||||
preserved and the volume is merely unmounted. This means that a PD can be
|
||||
pre-populated with data, and that data can be "handed off" between pods.
|
||||
|
||||
__Important: You must create a PD using `gcloud` or the GCE API or UI
|
||||
before you can use it__
|
||||
**Important:** You must create a PD using `gcloud` or the GCE API or UI before you can use it.
|
||||
{: .caution}
|
||||
|
||||
There are some restrictions when using a `gcePersistentDisk`:
|
||||
|
||||
@@ -245,8 +246,8 @@ volume are preserved and the volume is merely unmounted. This means that an
|
||||
EBS volume can be pre-populated with data, and that data can be "handed off"
|
||||
between pods.
|
||||
|
||||
__Important: You must create an EBS volume using `aws ec2 create-volume` or
|
||||
the AWS API before you can use it__
|
||||
**Important:** You must create an EBS volume using `aws ec2 create-volume` or the AWS API before you can use it.
|
||||
{: .caution}
|
||||
|
||||
There are some restrictions when using an awsElasticBlockStore volume:
|
||||
|
||||
@@ -259,7 +260,7 @@ There are some restrictions when using an awsElasticBlockStore volume:
|
||||
Before you can use an EBS volume with a pod, you need to create it.
|
||||
|
||||
```shell
|
||||
aws ec2 create-volume --availability-zone eu-west-1a --size 10 --volume-type gp2
|
||||
aws ec2 create-volume --availability-zone=eu-west-1a --size=10 --volume-type=gp2
|
||||
```
|
||||
|
||||
Make sure the zone matches the zone you brought up your cluster in. (And also check that the size and EBS volume
|
||||
@@ -296,8 +297,8 @@ unmounted. This means that an NFS volume can be pre-populated with data, and
|
||||
that data can be "handed off" between pods. NFS can be mounted by multiple
|
||||
writers simultaneously.
|
||||
|
||||
__Important: You must have your own NFS server running with the share exported
|
||||
before you can use it__
|
||||
**Important:** You must have your own NFS server running with the share exported before you can use it.
|
||||
{: .caution}
|
||||
|
||||
See the [NFS example](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/volumes/nfs) for more details.
|
||||
|
||||
@@ -309,8 +310,8 @@ contents of an `iscsi` volume are preserved and the volume is merely
|
||||
unmounted. This means that an iscsi volume can be pre-populated with data, and
|
||||
that data can be "handed off" between pods.
|
||||
|
||||
__Important: You must have your own iSCSI server running with the volume
|
||||
created before you can use it__
|
||||
**Important:** You must have your own iSCSI server running with the volume created before you can use it.
|
||||
{: .caution}
|
||||
|
||||
A feature of iSCSI is that it can be mounted as read-only by multiple consumers
|
||||
simultaneously. This means that you can pre-populate a volume with your dataset
|
||||
@@ -327,9 +328,8 @@ You can specify single or multiple target World Wide Names using the parameter
|
||||
`targetWWNs` in your volume configuration. If multiple WWNs are specified,
|
||||
targetWWNs expect that those WWNs are from multi-path connections.
|
||||
|
||||
__Important: You must configure FC SAN Zoning to allocate and mask those
|
||||
LUNs (volumes) to the target WWNs beforehand so that Kubernetes hosts
|
||||
can access them__
|
||||
**Important:** You must configure FC SAN Zoning to allocate and mask those LUNs (volumes) to the target WWNs beforehand so that Kubernetes hosts can access them.
|
||||
{: .caution}
|
||||
|
||||
See the [FC example](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/volumes/fibre_channel) for more details.
|
||||
|
||||
@@ -344,7 +344,8 @@ CLI or by using the Flocker API. If the dataset already exists it will be
|
||||
reattached by Flocker to the node that the pod is scheduled. This means data
|
||||
can be "handed off" between pods as required.
|
||||
|
||||
__Important: You must have your own Flocker installation running before you can use it__
|
||||
**Important:** You must have your own Flocker installation running before you can use it.
|
||||
{: .caution}
|
||||
|
||||
See the [Flocker example](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/volumes/flocker) for more details.
|
||||
|
||||
@@ -358,8 +359,8 @@ means that a glusterfs volume can be pre-populated with data, and that data can
|
||||
be "handed off" between pods. GlusterFS can be mounted by multiple writers
|
||||
simultaneously.
|
||||
|
||||
__Important: You must have your own GlusterFS installation running before you
|
||||
can use it__
|
||||
**Important:** You must have your own GlusterFS installation running before you can use it.
|
||||
{: .caution}
|
||||
|
||||
See the [GlusterFS example](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/volumes/glusterfs) for more details.
|
||||
|
||||
@@ -372,8 +373,8 @@ a `rbd` volume are preserved and the volume is merely unmounted. This
|
||||
means that a RBD volume can be pre-populated with data, and that data can
|
||||
be "handed off" between pods.
|
||||
|
||||
__Important: You must have your own Ceph installation running before you
|
||||
can use RBD__
|
||||
**Important:** You must have your own Ceph installation running before you can use RBD.
|
||||
{: .caution}
|
||||
|
||||
A feature of RBD is that it can be mounted as read-only by multiple consumers
|
||||
simultaneously. This means that you can pre-populate a volume with your dataset
|
||||
@@ -392,8 +393,8 @@ unmounted. This means that a CephFS volume can be pre-populated with data, and
|
||||
that data can be "handed off" between pods. CephFS can be mounted by multiple
|
||||
writers simultaneously.
|
||||
|
||||
__Important: You must have your own Ceph server running with the share exported
|
||||
before you can use it__
|
||||
**Important:** You must have your own Ceph server running with the share exported before you can use it.
|
||||
{: .caution}
|
||||
|
||||
See the [CephFS example](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/volumes/cephfs/) for more details.
|
||||
|
||||
@@ -433,8 +434,8 @@ use by pods without coupling to Kubernetes directly. `secret` volumes are
|
||||
backed by tmpfs (a RAM-backed filesystem) so they are never written to
|
||||
non-volatile storage.
|
||||
|
||||
__Important: You must create a secret in the Kubernetes API before you can use
|
||||
it__
|
||||
**Important:** You must create a secret in the Kubernetes API before you can use it.
|
||||
{: .caution}
|
||||
|
||||
Secrets are described in more detail [here](/docs/user-guide/secrets).
|
||||
|
||||
@@ -467,7 +468,7 @@ Currently, the following types of volume sources can be projected:
|
||||
|
||||
All sources are required to be in the same namespace as the pod. For more details, see the [all-in-one volume design document](https://github.com/kubernetes/community/blob/{{page.githubbranch}}/contributors/design-proposals/all-in-one-volume.md).
|
||||
|
||||
#### Example pod with a secret, a downward API, and a configmap
|
||||
#### Example pod with a secret, a downward API, and a configmap.
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
@@ -507,7 +508,7 @@ spec:
|
||||
path: my-group/my-config
|
||||
```
|
||||
|
||||
#### Example pod with multiple secrets with a non-default permission mode set
|
||||
#### Example pod with multiple secrets with a non-default permission mode set.
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
@@ -554,30 +555,31 @@ A `FlexVolume` enables users to mount vendor volumes into a pod. It expects vend
|
||||
drivers are installed in the volume plugin path on each kubelet node. This is
|
||||
an alpha feature and may change in future.
|
||||
|
||||
More details are in [here](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/volumes/flexvolume/README.md)
|
||||
More details are in [here](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/volumes/flexvolume/README.md).
|
||||
|
||||
### AzureFileVolume
|
||||
|
||||
A `AzureFileVolume` is used to mount a Microsoft Azure File Volume (SMB 2.1 and 3.0)
|
||||
into a Pod.
|
||||
|
||||
More details can be found [here](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/volumes/azure_file/README.md)
|
||||
More details can be found [here](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/volumes/azure_file/README.md).
|
||||
|
||||
### AzureDiskVolume
|
||||
|
||||
A `AzureDiskVolume` is used to mount a Microsoft Azure [Data Disk](https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-linux-about-disks-vhds/) into a Pod.
|
||||
|
||||
More details can be found [here](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/volumes/azure_disk/README.md)
|
||||
More details can be found [here](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/volumes/azure_disk/README.md).
|
||||
|
||||
### vsphereVolume
|
||||
|
||||
__Prerequisite: Kubernetes with vSphere Cloud Provider configured.
|
||||
For cloudprovider configuration please refer [vSphere getting started guide](/docs/getting-started-guides/vsphere/).__
|
||||
**Prerequisite:** Kubernetes with vSphere Cloud Provider configured. For cloudprovider configuration please refer [vSphere getting started guide](/docs/getting-started-guides/vsphere/).
|
||||
{: .note}
|
||||
|
||||
A `vsphereVolume` is used to mount a vSphere VMDK Volume into your Pod. The contents
|
||||
of a volume are preserved when it is unmounted. It supports both VMFS and VSAN datastore.
|
||||
|
||||
__Important: You must create VMDK using one of the following method before using with POD.__
|
||||
**Important:** You must create VMDK using one of the following method before using with POD.
|
||||
{: .caution}
|
||||
|
||||
#### Creating a VMDK volume
|
||||
|
||||
@@ -631,8 +633,8 @@ More examples can be found [here](https://git.k8s.io/kubernetes/examples/volumes
|
||||
|
||||
A `Quobyte` volume allows an existing [Quobyte](http://www.quobyte.com) volume to be mounted into your pod.
|
||||
|
||||
__Important: You must have your own Quobyte setup running with the volumes created
|
||||
before you can use it__
|
||||
**Important:** You must have your own Quobyte setup running with the volumes created before you can use it.
|
||||
{: .caution}
|
||||
|
||||
See the [Quobyte example](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/volumes/quobyte) for more details.
|
||||
|
||||
@@ -664,9 +666,10 @@ spec:
|
||||
fsType: "<fs-type>"
|
||||
```
|
||||
|
||||
__Important: Make sure you have an existing PortworxVolume with name `pxvol` before using it in the pod__
|
||||
**Important:** Make sure you have an existing PortworxVolume with name `pxvol` before using it in the pod.
|
||||
{: .caution}
|
||||
|
||||
More details and examples can be found [here](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/volumes/portworx/README.md)
|
||||
More details and examples can be found [here](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/volumes/portworx/README.md).
|
||||
|
||||
### ScaleIO
|
||||
ScaleIO is a software-based storage platform that can use existing hardware to create clusters of scalable
|
||||
@@ -674,8 +677,8 @@ shared block networked storage. The ScaleIO volume plugin allows deployed pods
|
||||
volumes (or it can dynamically provision new volumes for persistent volume claims, see
|
||||
[ScaleIO Persistent Volumes](/docs/user-guide/persistent-volumes/#scaleio)).
|
||||
|
||||
__Important: You must have an existing ScaleIO cluster already setup and running with the volumes created
|
||||
before you can use them__
|
||||
**Important:** You must have an existing ScaleIO cluster already setup and running with the volumes created before you can use them.
|
||||
{: .caution}
|
||||
|
||||
The following is an example pod configuration with ScaleIO:
|
||||
|
||||
@@ -713,7 +716,8 @@ At its core, StorageOS provides block storage to containers, accessible via a fi
|
||||
|
||||
The StorageOS container requires 64-bit Linux and has no additional dependencies. A free developer licence is available.
|
||||
|
||||
__Important: You must run the StorageOS container on each node that wants to access StorageOS volumes or that will contribute storage capacity to the pool. For installation instructions, consult the [StorageOS documentation](https://docs.storageos.com)__
|
||||
**Important:** You must run the StorageOS container on each node that wants to access StorageOS volumes or that will contribute storage capacity to the pool. For installation instructions, consult the [StorageOS documentation](https://docs.storageos.com).
|
||||
{: .caution}
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
@@ -791,11 +795,11 @@ spec:
|
||||
path: /mnt/disks/ssd1
|
||||
```
|
||||
|
||||
Note that local PersistentVolume cleanup and deletion requires manual
|
||||
intervention without the external provisioner.
|
||||
**Note:** The local PersistentVolume cleanup and deletion requires manual intervention without the external provisioner.
|
||||
{: .note}
|
||||
|
||||
For details on the `local` volume type, see the [Local Persistent Storage
|
||||
user guide](https://github.com/kubernetes-incubator/external-storage/tree/master/local-volume)
|
||||
user guide](https://github.com/kubernetes-incubator/external-storage/tree/master/local-volume).
|
||||
|
||||
## Using subPath
|
||||
|
||||
|
||||
@@ -27,84 +27,106 @@ The following are typical use cases for Deployments:
|
||||
* [Create a Deployment to rollout a ReplicaSet](#creating-a-deployment). The ReplicaSet creates Pods in the background. Check the status of the rollout to see if it succeeds or not.
|
||||
* [Declare the new state of the Pods](#updating-a-deployment) by updating the PodTemplateSpec of the Deployment. A new ReplicaSet is created and the Deployment manages moving the Pods from the old ReplicaSet to the new one at a controlled rate. Each new ReplicaSet updates the revision of the Deployment.
|
||||
* [Rollback to an earlier Deployment revision](#rolling-back-a-deployment) if the current state of the Deployment is not stable. Each rollback updates the revision of the Deployment.
|
||||
* [Scale up the Deployment to facilitate more load.](#scaling-a-deployment)
|
||||
* [Scale up the Deployment to facilitate more load](#scaling-a-deployment).
|
||||
* [Pause the Deployment](#pausing-and-resuming-a-deployment) to apply multiple fixes to its PodTemplateSpec and then resume it to start a new rollout.
|
||||
* [Use the status of the Deployment](#deployment-status) as an indicator that a rollout has stuck
|
||||
* [Clean up older ReplicaSets](#clean-up-policy) that you don't need anymore
|
||||
* [Use the status of the Deployment](#deployment-status) as an indicator that a rollout has stuck.
|
||||
* [Clean up older ReplicaSets](#clean-up-policy) that you don't need anymore.
|
||||
|
||||
|
||||
## Creating a Deployment
|
||||
|
||||
Here is an example Deployment. It creates a ReplicaSet to bring up three nginx Pods.
|
||||
The following is an example of a Deployment. It creates a ReplicaSet to bring up three `nginx` Pods:
|
||||
|
||||
{% include code.html language="yaml" file="nginx-deployment.yaml" ghlink="/docs/concepts/workloads/controllers/nginx-deployment.yaml" %}
|
||||
|
||||
Run the example by downloading the example file and then running this command:
|
||||
In this example:
|
||||
|
||||
* A Deployment named `nginx` is created.
|
||||
* The `nginx` Deployment creates three replicated Pods.
|
||||
* The Pods are created from the `template` field.
|
||||
|
||||
The `template` field contains the following instructions:
|
||||
|
||||
* Create one container in each Pod.
|
||||
* Label the container `app: nginx`.
|
||||
* Run the [Docker Hub](https://hub.docker.com) image `nginx` at version `1.7.9`.
|
||||
* Open port `80` so that the container can send and accept traffic.
|
||||
|
||||
To create this Deployment, run the following command:
|
||||
|
||||
```shell
|
||||
$ kubectl create -f docs/user-guide/nginx-deployment.yaml --record
|
||||
deployment "nginx-deployment" created
|
||||
kubectl create -f https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/master/docs/concepts/workloads/controllers/nginx-deployment.yaml
|
||||
```
|
||||
|
||||
Setting the kubectl flag `--record` to `true` allows you to record current command in the annotations of
|
||||
the resources being created or updated. It is useful for future introspection: for example, to see the
|
||||
commands executed in each Deployment revision.
|
||||
Note: You can append `--record` to this command to record the current command in the annotations of
|
||||
the created or updated resource. This is useful for future review, such as investigating which
|
||||
commands were executed in each Deployment revision.
|
||||
|
||||
Then running `get` immediately will give:
|
||||
Next, run `kubectl get deployments`. The output is similar to the following:
|
||||
|
||||
```shell
|
||||
$ kubectl get deployments
|
||||
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
|
||||
nginx-deployment 3 0 0 0 1s
|
||||
```
|
||||
|
||||
This indicates that the Deployment's number of desired replicas is 3 (according to deployment's `.spec.replicas`),
|
||||
the number of current replicas (`.status.replicas`) is 0, the number of up-to-date replicas (`.status.updatedReplicas`)
|
||||
is 0, and the number of available replicas (`.status.availableReplicas`) is also 0.
|
||||
When you inspect the Deployments in your cluster, the following fields are displayed:
|
||||
|
||||
To see the Deployment rollout status, run:
|
||||
* `NAME` lists the names of the Deployments in the cluster.
|
||||
* `DESIRED` displays the desired number of _replicas_ of the application, which
|
||||
you define when you create the Deployment. This is the _desired state_.
|
||||
* `CURRENT` displays how many replicas are currently running.
|
||||
* `UP-TO-DATE` displays the number of replicas that have been updated to achieve
|
||||
the desired state.
|
||||
* `AVAILABLE` displays how many replicas of the application are available to
|
||||
your users.
|
||||
* `AGE` displays the amount of time that the application has been running.
|
||||
|
||||
Notice how the values in each field correspond to the values in the Deployment specification:
|
||||
|
||||
* The number of desired replicas is 3 according to `spec: replicas` field.
|
||||
* The number of current replicas is 0 according to the `.status.replicas` field.
|
||||
* The number of up-to-date replicas is 0 accoridng to the `.status.updatedReplicas` field.
|
||||
* The number of available replicas is 0 according to the `.status.availableReplicas` field.
|
||||
|
||||
To see the Deployment rollout status, run `kubectl rollout status deployment/nginx-deployment`. This command returns the following output:
|
||||
|
||||
```shell
|
||||
$ kubectl rollout status deployment/nginx-deployment
|
||||
Waiting for rollout to finish: 2 out of 3 new replicas have been updated...
|
||||
deployment "nginx-deployment" successfully rolled out
|
||||
```
|
||||
|
||||
Running the `get` again a few seconds later should give:
|
||||
Run the `kubectl get deployments` again a few seconds later:
|
||||
|
||||
```shell
|
||||
$ kubectl get deployments
|
||||
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
|
||||
nginx-deployment 3 3 3 3 18s
|
||||
```
|
||||
|
||||
This indicates that the Deployment has created all three replicas, and all replicas are up-to-date (contains the
|
||||
latest pod template) and available (pod status is ready for at least Deployment's `.spec.minReadySeconds`). Running
|
||||
`kubectl get rs` and `kubectl get pods` will show the ReplicaSet (RS) and Pods created.
|
||||
Notice that the Deployment has created all three replicas, and all replicas are up-to-date (they contain the
|
||||
latest Pod template) and available (the Pod status is Ready for at least the value of the Deployment's `.spec.minReadySeconds` field).
|
||||
|
||||
To see the ReplicaSet (`rs`) created by the deployment, run `kubectl get rs`:
|
||||
|
||||
```shell
|
||||
$ kubectl get rs
|
||||
NAME DESIRED CURRENT READY AGE
|
||||
nginx-deployment-2035384211 3 3 3 18s
|
||||
```
|
||||
|
||||
You may notice that the name of the ReplicaSet is always `<the name of the Deployment>-<hash value of the pod template>`.
|
||||
Notice that the name of the ReplicaSet is always formatted as `[DEPLOYMENT-NAME]-[POD-TEMPALTE-HASH-VALUE]`. The hash value is automatically generated when the Deployemnt is created.
|
||||
|
||||
To see the labels automatically generated for each pod, run `kubectl get pods --show-labels`. The following output is returned:
|
||||
|
||||
```shell
|
||||
$ kubectl get pods --show-labels
|
||||
NAME READY STATUS RESTARTS AGE LABELS
|
||||
nginx-deployment-2035384211-7ci7o 1/1 Running 0 18s app=nginx,pod-template-hash=2035384211
|
||||
nginx-deployment-2035384211-kzszj 1/1 Running 0 18s app=nginx,pod-template-hash=2035384211
|
||||
nginx-deployment-2035384211-qqcnn 1/1 Running 0 18s app=nginx,pod-template-hash=2035384211
|
||||
```
|
||||
|
||||
The created ReplicaSet ensures that there are three nginx Pods at all times.
|
||||
The created ReplicaSet ensures that there are three `nginx` Pods running at all times.
|
||||
|
||||
**Note:** You must specify an appropriate selector and pod template labels in a Deployment (in this case,
|
||||
`app = nginx`). That is, don't overlap with other controllers (including other Deployments, ReplicaSets,
|
||||
StatefulSets, etc.). Kubernetes doesn't stop you from overlapping, and if multiple
|
||||
controllers have overlapping selectors, those controllers may fight with each other and won't behave
|
||||
correctly.
|
||||
**Note:** You must specify an appropriate selector and Pod template labels in a Deployment (in this case,
|
||||
`app: nginx`). Do not overlap labels or selectors with other controllers (including other Deployments and StatefulSets). Kubernetes doesn't stop you from overlapping, and if multiple controllers have overlapping selectors those controllers might conflict and behave unexpectedly.
|
||||
{: .note}
|
||||
|
||||
### Pod-template-hash label
|
||||
@@ -112,11 +134,10 @@ correctly.
|
||||
**Note:** Do not change this label.
|
||||
{: .note}
|
||||
|
||||
Note the pod-template-hash label in the example output in the pod labels above. This label is added by the
|
||||
Deployment controller to every ReplicaSet that a Deployment creates or adopts. Its purpose is to make sure that child
|
||||
ReplicaSets of a Deployment do not overlap. It is computed by hashing the PodTemplate of the ReplicaSet
|
||||
and using the resulting hash as the label value that will be added in the ReplicaSet selector, pod template labels,
|
||||
and in any existing Pods that the ReplicaSet may have.
|
||||
The `pod-template-hash label` is added by the Deployment controller to every ReplicaSet that a Deployment creates or adopts.
|
||||
|
||||
This label ensures that child ReplicaSets of a Deployment do not overlap. It is generated by hashing the `PodTemplate` of the ReplicaSet and using the resulting hash as the label value that is added to the ReplicaSet selector, Pod template labels,
|
||||
and in any existing Pods that the ReplicaSet might have.
|
||||
|
||||
## Updating a Deployment
|
||||
|
||||
@@ -262,9 +283,9 @@ removed label still exists in any existing Pods and ReplicaSets.
|
||||
|
||||
Sometimes you may want to rollback a Deployment; for example, when the Deployment is not stable, such as crash looping.
|
||||
By default, all of the Deployment's rollout history is kept in the system so that you can rollback anytime you want
|
||||
(you can change that by modifying revision history limit]).
|
||||
(you can change that by modifying revision history limit).
|
||||
|
||||
**Note:** a Deployment's revision is created when a Deployment's rollout is triggered. This means that the
|
||||
**Note:** A Deployment's revision is created when a Deployment's rollout is triggered. This means that the
|
||||
new revision is created if and only if the Deployment's pod template (`.spec.template`) is changed,
|
||||
for example if you update the labels or container images of the template. Other updates, such as scaling the Deployment,
|
||||
do not create a Deployment revision, so that we can facilitate simultaneous manual- or auto-scaling.
|
||||
|
||||
@@ -141,7 +141,7 @@ will have to manage the deletion yourself (see [below](#working-with-replication
|
||||
|
||||
You can specify how many pods should run concurrently by setting `.spec.replicas` to the number
|
||||
of pods you would like to have running concurrently. The number running at any time may be higher
|
||||
or lower, such as if the replicas was just increased or decreased, or if a pod is gracefully
|
||||
or lower, such as if the replicas were just increased or decreased, or if a pod is gracefully
|
||||
shutdown, and a replacement starts early.
|
||||
|
||||
If you do not specify `.spec.replicas`, then it defaults to 1.
|
||||
|
||||
@@ -32,7 +32,7 @@ an application. Examples are:
|
||||
- cluster administrator deletes VM (instance) by mistake
|
||||
- cloud provider or hypervisor failure makes VM disappear
|
||||
- a kernel panic
|
||||
- if the node to disappears from the cluster due to cluster network partition
|
||||
- the node disappears from the cluster due to cluster network partition
|
||||
- eviction of a pod due to the node being [out-of-resources](/docs/tasks/administer-cluster/out-of-resource/).
|
||||
|
||||
Except for the out-of-resources condition, all these conditions
|
||||
@@ -145,7 +145,7 @@ Initially, the pods are laid out as follows:
|
||||
| pod-a *available* | pod-b *available* | pod-c *available* |
|
||||
| pod-x *available* | | |
|
||||
|
||||
All 3 pods are part of an deployment, and they collectively have a PDB which requires
|
||||
All 3 pods are part of a deployment, and they collectively have a PDB which requires
|
||||
there be at least 2 of the 3 pods to be available at all times.
|
||||
|
||||
For example, assume the cluster administrator wants to reboot into a new kernel version to fix a bug in the kernel.
|
||||
@@ -174,7 +174,7 @@ Now the cluster is in this state:
|
||||
| pod-a *terminating* | pod-b *available* | pod-c *available* |
|
||||
| pod-x *terminating* | pod-d *starting* | pod-y |
|
||||
|
||||
At some point, the pods terminate, and the cluster look like this:
|
||||
At some point, the pods terminate, and the cluster looks like this:
|
||||
|
||||
| node-1 *drained* | node-2 | node-3 |
|
||||
|:--------------------:|:-------------------:|:------------------:|
|
||||
|
||||
Reference in New Issue
Block a user