Merge branch 'dev-1.16' into merged-master-dev-1.16
This commit is contained in:
simplytunde
@@ -290,6 +290,13 @@ includes all containers started by the kubelet, but not containers started direc
|
||||
If you want to explicitly reserve resources for non-Pod processes, follow this tutorial to
|
||||
[reserve resources for system daemons](/docs/tasks/administer-cluster/reserve-compute-resources/#system-reserved).
|
||||
|
||||
## Node topology
|
||||
|
||||
{{< feature-state state="alpha" >}}
|
||||
|
||||
If you have enabled the `TopologyManager`
|
||||
[feature gate](/docs/reference/command-line-tools-reference/feature-gates/), then
|
||||
the kubelet can use topology hints when making resource assignment decisions.
|
||||
|
||||
## API Object
|
||||
|
||||
@@ -298,3 +305,7 @@ API object can be found at:
|
||||
[Node API object](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#node-v1-core).
|
||||
|
||||
{{% /capture %}}
|
||||
{{% capture whatsnext %}}
|
||||
* Read about [node components](https://kubernetes.io/docs/concepts/overview/components/#node-components)
|
||||
* Read about node-level topology: [Control Topology Management Policies on a node](/docs/tasks/administer-cluster/topology-manager/)
|
||||
{{% /capture %}}
|
||||
|
||||
@@ -394,4 +394,8 @@ The design documents for
|
||||
[node affinity](https://git.k8s.io/community/contributors/design-proposals/scheduling/nodeaffinity.md)
|
||||
and for [inter-pod affinity/anti-affinity](https://git.k8s.io/community/contributors/design-proposals/scheduling/podaffinity.md) contain extra background information about these features.
|
||||
|
||||
Once a Pod is assigned to a Node, the kubelet runs the Pod and allocates node-local resources.
|
||||
The [topology manager](/docs/tasks/administer-cluster/topology-manager/) can take part in node-level
|
||||
resource allocation decisions.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
@@ -0,0 +1,62 @@
|
||||
---
|
||||
reviewers:
|
||||
- dchen1107
|
||||
- egernst
|
||||
- tallclair
|
||||
title: Pod Overhead
|
||||
content_template: templates/concept
|
||||
weight: 20
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
{{< feature-state for_k8s_version="v1.16" state="alpha" >}}
|
||||
|
||||
|
||||
When you run a Pod on a Node, the Pod itself takes an amount of system resources. These
|
||||
resources are additional to the resources needed to run the container(s) inside the Pod.
|
||||
_Pod Overhead_ is a feature for accounting for the resources consumed by the pod infrastructure
|
||||
on top of the container requests & limits.
|
||||
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
{{% capture body %}}
|
||||
|
||||
## Pod Overhead
|
||||
|
||||
In Kubernetes, the pod's overhead is set at
|
||||
[admission](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#what-are-admission-webhooks)
|
||||
time according to the overhead associated with the pod's
|
||||
[RuntimeClass](https://kubernetes.io/docs/concepts/containers/runtime-class/).
|
||||
|
||||
When Pod Overhead is enabled, the overhead is considered in addition to the sum of container
|
||||
resource requests when scheduling a pod. Similarly, Kubelet will include the pod overhead when sizing
|
||||
the pod cgroup, and when carrying out pod eviction ranking.
|
||||
|
||||
### Set Up
|
||||
|
||||
You need to make sure that the `PodOverhead`
|
||||
[feature gate](/docs/reference/command-line-tools-reference/feature-gates/) is enabled (it is off by default)
|
||||
across your cluster. This means:
|
||||
|
||||
- in {{< glossary_tooltip text="kube-scheduler" term_id="kube-scheduler" >}}
|
||||
- in {{< glossary_tooltip text="kube-apiserver" term_id="kube-apiserver" >}}
|
||||
- in the {{< glossary_tooltip text="kubelet" term_id="kubelet" >}} on each Node
|
||||
- in any custom API servers that use feature gates
|
||||
|
||||
{{< note >}}
|
||||
Users who can write to RuntimeClass resources are able to have cluster-wide impact on
|
||||
workload performance. You can limit access to this ability using Kubernetes access controls.
|
||||
See [Authorization Overview](/docs/reference/access-authn-authz/authorization/) for more details.
|
||||
{{< /note >}}
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture whatsnext %}}
|
||||
|
||||
* [RuntimeClass](/docs/concepts/containers/runtime-class/)
|
||||
* [PodOverhead Design](https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/20190226-pod-overhead.md)
|
||||
|
||||
{{% /capture %}}
|
||||
@@ -0,0 +1,197 @@
|
||||
---
|
||||
reviewers:
|
||||
- bsalamat
|
||||
- k82cn
|
||||
- ahg-g
|
||||
title: Resource Bin Packing for Extended Resources
|
||||
content_template: templates/concept
|
||||
weight: 10
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
{{< feature-state for_k8s_version="1.16" state="alpha" >}}
|
||||
|
||||
The kube-scheduler can be configured to enable bin packing of resources along with extended resources using `RequestedToCapacityRatioResourceAllocation` priority function. Priority functions can be used to fine-tune the kube-scheduler as per custom needs.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture body %}}
|
||||
|
||||
## Enabling Bin Packing using RequestedToCapacityRatioResourceAllocation
|
||||
|
||||
Before Kubernetes 1.15, Kube-scheduler used to allow scoring nodes based on the request to capacity ratio of primary resources like CPU and Memory. Kubernetes 1.16 added a new parameter to the priority function that allows the users to specify the resources along with weights for each resource to score nodes based on the request to capacity ratio. This allows users to bin pack extended resources by using appropriate parameters improves the utilization of scarce resources in large clusters. The behavior of the `RequestedToCapacityRatioResourceAllocation` priority function can be controlled by a configuration option called `requestedToCapacityRatioArguments`. This argument consists of two parameters `shape` and `resources`. Shape allows the user to tune the function as least requested or most requested based on `utilization` and `score` values. Resources
|
||||
consists of `name` which specifies the resource to be considered during scoring and `weight` specify the weight of each resource.
|
||||
|
||||
Below is an example configuration that sets `requestedToCapacityRatioArguments` to bin packing behavior for extended resources `intel.com/foo` and `intel.com/bar`
|
||||
|
||||
```json
|
||||
{
|
||||
"kind" : "Policy",
|
||||
"apiVersion" : "v1",
|
||||
|
||||
...
|
||||
|
||||
"priorities" : [
|
||||
|
||||
...
|
||||
|
||||
{
|
||||
"name": "RequestedToCapacityRatioPriority",
|
||||
"weight": 2,
|
||||
"argument": {
|
||||
"requestedToCapacityRatioArguments": {
|
||||
"shape": [
|
||||
{"utilization": 0, "score": 0},
|
||||
{"utilization": 100, "score": 10}
|
||||
],
|
||||
"resources": [
|
||||
{"name": "intel.com/foo", "weight": 3},
|
||||
{"name": "intel.com/bar", "weight": 5}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
}
|
||||
```
|
||||
|
||||
**This feature is disabled by default**
|
||||
|
||||
### Tuning RequestedToCapacityRatioResourceAllocation Priority Function
|
||||
|
||||
`shape` is used to specify the behavior of the `RequestedToCapacityRatioPriority` function.
|
||||
|
||||
```yaml
|
||||
{"utilization": 0, "score": 0},
|
||||
{"utilization": 100, "score": 10}
|
||||
```
|
||||
|
||||
The above arguments give the node a score of 0 if utilization is 0% and 10 for utilization 100%, thus enabling bin packing behavior. To enable least requested the score value must be reversed as follows.
|
||||
|
||||
```yaml
|
||||
{"utilization": 0, "score": 100},
|
||||
{"utilization": 100, "score": 0}
|
||||
```
|
||||
|
||||
`resources` is an optional parameter which by defaults is set to:
|
||||
|
||||
``` yaml
|
||||
"resources": [
|
||||
{"name": "CPU", "weight": 1},
|
||||
{"name": "Memory", "weight": 1}
|
||||
]
|
||||
```
|
||||
|
||||
It can be used to add extended resources as follows:
|
||||
|
||||
```yaml
|
||||
"resources": [
|
||||
{"name": "intel.com/foo", "weight": 5},
|
||||
{"name": "CPU", "weight": 3},
|
||||
{"name": "Memory", "weight": 1}
|
||||
]
|
||||
```
|
||||
|
||||
The weight parameter is optional and is set to 1 if not specified. Also, the weight cannot be set to a negative value.
|
||||
|
||||
### How the RequestedToCapacityRatioResourceAllocation Priority Function Scores Nodes
|
||||
|
||||
This section is intended for those who want to understand the internal details
|
||||
of this feature.
|
||||
Below is an example of how the node score is calculated for a given set of values.
|
||||
|
||||
```
|
||||
Requested Resources
|
||||
|
||||
intel.com/foo : 2
|
||||
Memory: 256MB
|
||||
CPU: 2
|
||||
|
||||
Resource Weights
|
||||
|
||||
intel.com/foo : 5
|
||||
Memory: 1
|
||||
CPU: 3
|
||||
|
||||
FunctionShapePoint {{0, 0}, {100, 10}}
|
||||
|
||||
Node 1 Spec
|
||||
|
||||
Available:
|
||||
intel.com/foo : 4
|
||||
Memory : 1 GB
|
||||
CPU: 8
|
||||
|
||||
Used:
|
||||
intel.com/foo: 1
|
||||
Memory: 256MB
|
||||
CPU: 1
|
||||
|
||||
|
||||
Node Score:
|
||||
|
||||
intel.com/foo = resourceScoringFunction((2+1),4)
|
||||
= (100 - ((4-3)*100/4)
|
||||
= (100 - 25)
|
||||
= 75
|
||||
= rawScoringFunction(75)
|
||||
= 7
|
||||
|
||||
Memory = resourceScoringFunction((256+256),1024)
|
||||
= (100 -((1024-512)*100/1024))
|
||||
= 50
|
||||
= rawScoringFunction(50)
|
||||
= 5
|
||||
|
||||
CPU = resourceScoringFunction((2+1),8)
|
||||
= (100 -((8-3)*100/8))
|
||||
= 37.5
|
||||
= rawScoringFunction(37.5)
|
||||
= 3
|
||||
|
||||
NodeScore = (7 * 5) + (5 * 1) + (3 * 3) / (5 + 1 + 3)
|
||||
= 5
|
||||
|
||||
|
||||
Node 2 Spec
|
||||
|
||||
Available:
|
||||
intel.com/foo: 8
|
||||
Memory: 1GB
|
||||
CPU: 8
|
||||
|
||||
Used:
|
||||
|
||||
intel.com/foo: 2
|
||||
Memory: 512MB
|
||||
CPU: 6
|
||||
|
||||
|
||||
Node Score:
|
||||
|
||||
intel.com/foo = resourceScoringFunction((2+2),8)
|
||||
= (100 - ((8-4)*100/8)
|
||||
= (100 - 25)
|
||||
= 50
|
||||
= rawScoringFunction(50)
|
||||
= 5
|
||||
|
||||
Memory = resourceScoringFunction((256+512),1024)
|
||||
= (100 -((1024-768)*100/1024))
|
||||
= 75
|
||||
= rawScoringFunction(75)
|
||||
= 7
|
||||
|
||||
CPU = resourceScoringFunction((2+6),8)
|
||||
= (100 -((8-8)*100/8))
|
||||
= 100
|
||||
= rawScoringFunction(100)
|
||||
= 10
|
||||
|
||||
NodeScore = (5 * 5) + (7 * 1) + (10 * 3) / (5 + 1 + 3)
|
||||
= 7
|
||||
|
||||
```
|
||||
|
||||
{{% /capture %}}
|
||||
@@ -57,10 +57,9 @@ implementation dependent. See the corresponding documentation ([below](#cri-conf
|
||||
CRI implementation for how to configure.
|
||||
|
||||
{{< note >}}
|
||||
RuntimeClass currently assumes a homogeneous node configuration across the cluster (which means that
|
||||
all nodes are configured the same way with respect to container runtimes). Any heterogeneity
|
||||
(varying configurations) must be managed independently of RuntimeClass through scheduling features
|
||||
(see [Assigning Pods to Nodes](/docs/concepts/configuration/assign-pod-node/)).
|
||||
RuntimeClass assumes a homogeneous node configuration across the cluster by default (which means
|
||||
that all nodes are configured the same way with respect to container runtimes). To support
|
||||
heterogenous node configurations, see [Scheduling](#scheduling) below.
|
||||
{{< /note >}}
|
||||
|
||||
The configurations have a corresponding `handler` name, referenced by the RuntimeClass. The
|
||||
@@ -147,6 +146,44 @@ table](https://github.com/kubernetes-sigs/cri-o/blob/master/docs/crio.conf.5.md#
|
||||
See cri-o's config documentation for more details:
|
||||
https://github.com/kubernetes-sigs/cri-o/blob/master/cmd/crio/config.go
|
||||
|
||||
### Scheduling
|
||||
|
||||
{{< feature-state for_k8s_version="v1.16" state="beta" >}}
|
||||
|
||||
As of Kubernetes v1.16, RuntimeClass includes support for heterogenous clusters through its
|
||||
`scheduling` fields. Through the use of these fields, you can ensure that pods running with this
|
||||
RuntimeClass are scheduled to nodes that support it. To use the scheduling support, you must have
|
||||
the RuntimeClass [admission controller][] enabled (the default, as of 1.16).
|
||||
|
||||
To ensure pods land on nodes supporting a specific RuntimeClass, that set of nodes should have a
|
||||
common label which is then selected by the `runtimeclass.scheduling.nodeSelector` field. The
|
||||
RuntimeClass's nodeSelector is merged with the pod's nodeSelector in admission, effectively taking
|
||||
the intersection of the set of nodes selected by each. If there is a conflict, the pod will be
|
||||
rejected.
|
||||
|
||||
If the supported nodes are tainted to prevent other RuntimeClass pods from running on the node, you
|
||||
can add `tolerations` to the RuntimeClass. As with the `nodeSelector`, the tolerations are merged
|
||||
with the pod's tolerations in admission, effectively taking the union of the set of nodes tolerated
|
||||
by each.
|
||||
|
||||
To learn more about configuring the node selector and tolerations, see [Assigning Pods to
|
||||
Nodes](/docs/concepts/configuration/assign-pod-node/).
|
||||
|
||||
[admission controller]: /docs/reference/access-authn-authz/admission-controllers/
|
||||
|
||||
### Pod Overhead
|
||||
|
||||
{{< feature-state for_k8s_version="v1.16" state="alpha" >}}
|
||||
|
||||
As of Kubernetes v1.16, RuntimeClass includes support for specifying overhead associated with
|
||||
running a pod, as part of the [`PodOverhead`](/docs/concepts/configuration/pod-overhead.md) feature.
|
||||
To use `PodOverhead`, you must have the PodOverhead [feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
|
||||
enabled (it is off by default).
|
||||
|
||||
|
||||
Pod overhead is defined in RuntimeClass through the `Overhead` fields. Through the use of these fields,
|
||||
you can specify the overhead of running pods utilizing this RuntimeClass and ensure these overheads
|
||||
are accounted for in Kubernetes.
|
||||
|
||||
### Upgrading RuntimeClass from Alpha to Beta
|
||||
|
||||
@@ -175,4 +212,11 @@ RuntimeClass feature to the beta version:
|
||||
in the handler are no longer valid, and must be migrated to a valid handler configuration (see
|
||||
above).
|
||||
|
||||
### Further Reading
|
||||
|
||||
- [RuntimeClass Design](https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/runtime-class.md)
|
||||
- [RuntimeClass Scheduling Design](https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/runtime-class-scheduling.md)
|
||||
- Read about the [Pod Overhead](/docs/concepts/configuration/pod-overhead/) concept
|
||||
- [PodOverhead Feature Design](https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/20190226-pod-overhead.md)
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
@@ -174,7 +174,7 @@ Aggregated APIs offer more advanced API features and customization of other feat
|
||||
| Feature | Description | CRDs | Aggregated API |
|
||||
| ------- | ----------- | ---- | -------------- |
|
||||
| Validation | Help users prevent errors and allow you to evolve your API independently of your clients. These features are most useful when there are many clients who can't all update at the same time. | Yes. Most validation can be specified in the CRD using [OpenAPI v3.0 validation](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/#validation). Any other validations supported by addition of a [Validating Webhook](/docs/reference/access-authn-authz/admission-controllers/#validatingadmissionwebhook-alpha-in-1-8-beta-in-1-9). | Yes, arbitrary validation checks |
|
||||
| Defaulting | See above | Yes, either via [OpenAPI v3.0 validation](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/#defaulting) `default` keyword (alpha in 1.15), or via a [Mutating Webhook](/docs/reference/access-authn-authz/admission-controllers/#mutatingadmissionwebhook-beta-in-1-9) | Yes |
|
||||
| Defaulting | See above | Yes, either via [OpenAPI v3.0 validation](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/#defaulting) `default` keyword (beta in 1.16), or via a [Mutating Webhook](/docs/reference/access-authn-authz/admission-controllers/#mutatingadmissionwebhook-beta-in-1-9) | Yes |
|
||||
| Multi-versioning | Allows serving the same object through two API versions. Can help ease API changes like renaming fields. Less important if you control your client versions. | [Yes](/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definition-versioning) | Yes |
|
||||
| Custom Storage | If you need storage with a different performance mode (for example, time-series database instead of key-value store) or isolation for security (for example, encryption secrets or different | No | Yes |
|
||||
| Custom Business Logic | Perform arbitrary checks or actions when creating, reading, updating or deleting an object | Yes, using [Webhooks](/docs/reference/access-authn-authz/extensible-admission-controllers/#admission-webhooks). | Yes |
|
||||
@@ -183,7 +183,7 @@ Aggregated APIs offer more advanced API features and customization of other feat
|
||||
| Other Subresources | Add operations other than CRUD, such as "logs" or "exec". | No | Yes |
|
||||
| strategic-merge-patch | The new endpoints support PATCH with `Content-Type: application/strategic-merge-patch+json`. Useful for updating objects that may be modified both locally, and by the server. For more information, see ["Update API Objects in Place Using kubectl patch"](/docs/tasks/run-application/update-api-object-kubectl-patch/) | No | Yes |
|
||||
| Protocol Buffers | The new resource supports clients that want to use Protocol Buffers | No | Yes |
|
||||
| OpenAPI Schema | Is there an OpenAPI (swagger) schema for the types that can be dynamically fetched from the server? Is the user protected from misspelling field names by ensuring only allowed fields are set? Are types enforced (in other words, don't put an `int` in a `string` field?) | Yes, based on the [OpenAPI v3.0 validation](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/#validation) schema (beta in 1.15) | Yes |
|
||||
| OpenAPI Schema | Is there an OpenAPI (swagger) schema for the types that can be dynamically fetched from the server? Is the user protected from misspelling field names by ensuring only allowed fields are set? Are types enforced (in other words, don't put an `int` in a `string` field?) | Yes, based on the [OpenAPI v3.0 validation](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/#validation) schema (GA in 1.16) | Yes |
|
||||
|
||||
### Common Features
|
||||
|
||||
|
||||
@@ -181,6 +181,9 @@ kube-scheduler has a default set of scheduling policies.
|
||||
{{% /capture %}}
|
||||
{{% capture whatsnext %}}
|
||||
* Read about [scheduler performance tuning](/docs/concepts/scheduling/scheduler-perf-tuning/)
|
||||
* Read about [Pod topology spread constraints](/docs/concepts/workloads/pods/pod-topology-spread-constraints/)
|
||||
* Read the [reference documentation](/docs/reference/command-line-tools-reference/kube-scheduler/) for kube-scheduler
|
||||
* Learn about [configuring multiple schedulers](https://kubernetes.io/docs/tasks/administer-cluster/configure-multiple-schedulers/)
|
||||
* Learn about [configuring multiple schedulers](/docs/tasks/administer-cluster/configure-multiple-schedulers/)
|
||||
* Learn about [topology management policies](/docs/tasks/administer-cluster/topology-manager/)
|
||||
* Learn about [Pod Overhead](/docs/concepts/configuration/pod-overhead/)
|
||||
{{% /capture %}}
|
||||
|
||||
@@ -98,7 +98,7 @@ Name: hostpath
|
||||
Namespace: default
|
||||
StorageClass: example-hostpath
|
||||
Status: Terminating
|
||||
Volume:
|
||||
Volume:
|
||||
Labels: <none>
|
||||
Annotations: volume.beta.kubernetes.io/storage-class=example-hostpath
|
||||
volume.beta.kubernetes.io/storage-provisioner=example.com/hostpath
|
||||
@@ -116,15 +116,15 @@ Annotations: <none>
|
||||
Finalizers: [kubernetes.io/pv-protection]
|
||||
StorageClass: standard
|
||||
Status: Available
|
||||
Claim:
|
||||
Claim:
|
||||
Reclaim Policy: Delete
|
||||
Access Modes: RWO
|
||||
Capacity: 1Gi
|
||||
Message:
|
||||
Message:
|
||||
Source:
|
||||
Type: HostPath (bare host directory volume)
|
||||
Path: /tmp/data
|
||||
HostPathType:
|
||||
HostPathType:
|
||||
Events: <none>
|
||||
```
|
||||
|
||||
@@ -217,9 +217,9 @@ new `PersistentVolume` is never created to satisfy the claim. Instead, an existi
|
||||
|
||||
#### CSI Volume expansion
|
||||
|
||||
{{< feature-state for_k8s_version="v1.14" state="alpha" >}}
|
||||
{{< feature-state for_k8s_version="v1.16" state="beta" >}}
|
||||
|
||||
CSI volume expansion requires enabling `ExpandCSIVolumes` feature gate and also requires specific CSI driver to support volume expansion. Please refer to documentation of specific CSI driver for more information.
|
||||
Support for expanding CSI volumes is enabled by default but it also requires specific CSI driver to support volume expansion. Please refer to documentation of specific CSI driver for more information.
|
||||
|
||||
|
||||
#### Resizing a volume containing a file system
|
||||
@@ -228,10 +228,10 @@ You can only resize volumes containing a file system if the file system is XFS,
|
||||
|
||||
When a volume contains a file system, the file system is only resized when a new Pod is using
|
||||
the `PersistentVolumeClaim` in ReadWrite mode. File system expansion is either done when Pod is starting up
|
||||
or is done when Pod is running and underlying file system supports online expansion.
|
||||
or is done when Pod is running and underlying file system supports online expansion.
|
||||
|
||||
FlexVolumes allow resize if the driver is set with the `RequiresFSResize` capability to true.
|
||||
The FlexVolume can be resized on pod restart.
|
||||
FlexVolumes allow resize if the driver is set with the `RequiresFSResize` capability to true.
|
||||
The FlexVolume can be resized on pod restart.
|
||||
|
||||
#### Resizing an in-use PersistentVolumeClaim
|
||||
|
||||
@@ -682,7 +682,7 @@ spec:
|
||||
|
||||
## Volume Cloning
|
||||
|
||||
{{< feature-state for_k8s_version="v1.15" state="alpha" >}}
|
||||
{{< feature-state for_k8s_version="v1.16" state="beta" >}}
|
||||
|
||||
Volume clone feature was added to support CSI Volume Plugins only. For details, see [volume cloning](/docs/concepts/storage/volume-pvc-datasource/).
|
||||
|
||||
|
||||
@@ -11,7 +11,7 @@ weight: 30
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
{{< feature-state for_k8s_version="v1.15" state="alpha" >}}
|
||||
{{< feature-state for_k8s_version="v1.16" state="beta" >}}
|
||||
This document describes the concept of cloning existing CSI Volumes in Kubernetes. Familiarity with [Volumes](/docs/concepts/storage/volumes) is suggested.
|
||||
|
||||
This feature requires VolumePVCDataSource feature gate to be enabled:
|
||||
@@ -32,7 +32,7 @@ The {{< glossary_tooltip text="CSI" term_id="csi" >}} Volume Cloning feature add
|
||||
|
||||
A Clone is defined as a duplicate of an existing Kubernetes Volume that can be consumed as any standard Volume would be. The only difference is that upon provisioning, rather than creating a "new" empty Volume, the back end device creates an exact duplicate of the specified Volume.
|
||||
|
||||
The implementation of cloning, from the perspective of the Kubernetes API simply adds the ability to specify an existing unbound PVC as a dataSource during new pvc creation.
|
||||
The implementation of cloning, from the perspective of the Kubernetes API simply adds the ability to specify an existing unbound PVC as a dataSource during new pvc creation.
|
||||
|
||||
Users need to be aware of the following when using this feature:
|
||||
|
||||
@@ -40,6 +40,10 @@ Users need to be aware of the following when using this feature:
|
||||
* Cloning support is only available for dynamic provisioners.
|
||||
* CSI drivers may or may not have implemented the volume cloning functionality.
|
||||
* You can only clone a PVC when it exists in the same namespace as the destination PVC (source and destination must be in the same namespace).
|
||||
* Cloning is only supported within the same Storage Class.
|
||||
- Destination volume must be the same storage class as the source
|
||||
- Default storage class can be used and storageClassName omitted in the spec
|
||||
|
||||
|
||||
## Provisioning
|
||||
|
||||
@@ -54,6 +58,7 @@ metadata:
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
storageClassName: cloning
|
||||
resources:
|
||||
requests:
|
||||
storage: 5Gi
|
||||
|
||||
@@ -0,0 +1,209 @@
|
||||
---
|
||||
reviewers:
|
||||
- verb
|
||||
- yujuhong
|
||||
title: Ephemeral Containers
|
||||
content_template: templates/concept
|
||||
weight: 80
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
{{< feature-state state="alpha" >}}
|
||||
|
||||
This page provides an overview of ephemeral containers: a special type of container
|
||||
that runs temporarily in an existing {{< glossary_tooltip term_id="pod" >}} to accomplish user-initiated actions such
|
||||
as troubleshooting. You use ephemeral containers to inspect services rather than
|
||||
to build applications.
|
||||
|
||||
{{< warning >}}
|
||||
Ephemeral containers are in early alpha state and are not suitable for production
|
||||
clusters. You should expect the feature not to work in some situations, such as
|
||||
when targeting the namespaces of a container. In accordance with the [Kubernetes
|
||||
Deprecation Policy](/docs/reference/using-api/deprecation-policy/), this alpha
|
||||
feature could change significantly in the future or be removed entirely.
|
||||
{{< /warning >}}
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture body %}}
|
||||
|
||||
## Understanding ephemeral containers
|
||||
|
||||
{{< glossary_tooltip text="Pods" term_id="pod" >}} are the fundamental building
|
||||
block of Kubernetes applications. Since Pods are intended to be disposable and
|
||||
replaceable, you cannot add a container to a Pod once it has been created.
|
||||
Instead, you usually delete and replace Pods in a controlled fashion using
|
||||
{{< glossary_tooltip text="deployments" term_id="deployment" >}}.
|
||||
|
||||
Sometimes it's necessary to inspect the state of an existing Pod, however, for
|
||||
example to troubleshoot a hard-to-reproduce bug. In these cases you can run
|
||||
an ephemeral container in an existing Pod to inspect its state and run
|
||||
arbitrary commands.
|
||||
|
||||
### What is an ephemeral container?
|
||||
|
||||
Ephemeral containers differ from other containers in that they lack guarantees
|
||||
for resources or execution, and they will never be automatically restarted, so
|
||||
they are not appropriate for building applications. Ephemeral containers are
|
||||
described using the same `ContainerSpec` as regular containers, but many fields
|
||||
are incompatible and disallowed for ephemeral containers.
|
||||
|
||||
- Ephemeral containers may not have ports, so fields such as `ports`,
|
||||
`livenessProbe`, `readinessProbe` are disallowed.
|
||||
- Pod resource allocations are immutable, so setting `resources` is disallowed.
|
||||
- For a complete list of allowed fields, see the [EphemeralContainer reference
|
||||
documentation](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#ephemeralcontainer-v1-core).
|
||||
|
||||
Ephemeral containers are created using a special `ephemeralcontainers` handler
|
||||
in the API rather than by adding them directly to `pod.spec`, so it's not
|
||||
possible to add an ephemeral container using `kubectl edit`.
|
||||
|
||||
Like regular containers, you may not change or remove an ephemeral container
|
||||
after you have added it to a Pod.
|
||||
|
||||
## Uses for ephemeral containers
|
||||
|
||||
Ephemeral containers are useful for interactive troubleshooting when `kubectl
|
||||
exec` is insufficient because a container has crashed or a container image
|
||||
doesn't include debugging utilities.
|
||||
|
||||
In particular, [distroless images](https://github.com/GoogleContainerTools/distroless)
|
||||
enable you to deploy minimal container images that reduce attack surface
|
||||
and exposure to bugs and vulnerabilities. Since distroless images do not include a
|
||||
shell or any debugging utilities, it's difficult to troubleshoot distroless
|
||||
images using `kubectl exec` alone.
|
||||
|
||||
When using ephemeral containers, it's helpful to enable [process namespace
|
||||
sharing](/docs/tasks/configure-pod-container/share-process-namespace/) so
|
||||
you can view processes in other containers.
|
||||
|
||||
### Examples
|
||||
|
||||
{{< note >}}
|
||||
The examples in this section require the `EphemeralContainers` [feature
|
||||
gate](/docs/reference/command-line-tools-reference/feature-gates/) to be
|
||||
enabled and kubernetes client and server version v1.16 or later.
|
||||
{{< /note >}}
|
||||
|
||||
The examples in this section demonstrate how ephemeral containers appear in
|
||||
the API. Users would normally use a `kubectl` plugin for troubleshooting that
|
||||
would automate these steps.
|
||||
|
||||
Ephemeral containers are created using the `ephemeralcontainers` subresource
|
||||
of Pod, which can be demonstrated using `kubectl --raw`. First describe
|
||||
the ephemeral container to add as an `EphemeralContainers` list:
|
||||
|
||||
```json
|
||||
{
|
||||
"apiVersion": "v1",
|
||||
"kind": "EphemeralContainers",
|
||||
"metadata": {
|
||||
"name": "example-pod"
|
||||
},
|
||||
"ephemeralContainers": [{
|
||||
"command": [
|
||||
"sh"
|
||||
],
|
||||
"image": "busybox",
|
||||
"imagePullPolicy": "IfNotPresent",
|
||||
"name": "debugger",
|
||||
"stdin": true,
|
||||
"tty": true,
|
||||
"terminationMessagePolicy": "File"
|
||||
}]
|
||||
}
|
||||
```
|
||||
|
||||
To update the ephemeral containers of the already running `example-pod`:
|
||||
|
||||
```shell
|
||||
kubectl replace --raw /api/v1/namespaces/default/pods/example-pod/ephemeralcontainers -f ec.json
|
||||
```
|
||||
|
||||
This will return the new list of ephemeral containers:
|
||||
|
||||
```json
|
||||
{
|
||||
"kind":"EphemeralContainers",
|
||||
"apiVersion":"v1",
|
||||
"metadata":{
|
||||
"name":"example-pod",
|
||||
"namespace":"default",
|
||||
"selfLink":"/api/v1/namespaces/default/pods/example-pod/ephemeralcontainers",
|
||||
"uid":"a14a6d9b-62f2-4119-9d8e-e2ed6bc3a47c",
|
||||
"resourceVersion":"15886",
|
||||
"creationTimestamp":"2019-08-29T06:41:42Z"
|
||||
},
|
||||
"ephemeralContainers":[
|
||||
{
|
||||
"name":"debugger",
|
||||
"image":"busybox",
|
||||
"command":[
|
||||
"sh"
|
||||
],
|
||||
"resources":{
|
||||
|
||||
},
|
||||
"terminationMessagePolicy":"File",
|
||||
"imagePullPolicy":"IfNotPresent",
|
||||
"stdin":true,
|
||||
"tty":true
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
You can view the state of the newly created ephemeral container using `kubectl describe`:
|
||||
|
||||
```shell
|
||||
kubectl describe pod example-pod
|
||||
```
|
||||
|
||||
```
|
||||
...
|
||||
Ephemeral Containers:
|
||||
debugger:
|
||||
Container ID: docker://cf81908f149e7e9213d3c3644eda55c72efaff67652a2685c1146f0ce151e80f
|
||||
Image: busybox
|
||||
Image ID: docker-pullable://busybox@sha256:9f1003c480699be56815db0f8146ad2e22efea85129b5b5983d0e0fb52d9ab70
|
||||
Port: <none>
|
||||
Host Port: <none>
|
||||
Command:
|
||||
sh
|
||||
State: Running
|
||||
Started: Thu, 29 Aug 2019 06:42:21 +0000
|
||||
Ready: False
|
||||
Restart Count: 0
|
||||
Environment: <none>
|
||||
Mounts: <none>
|
||||
...
|
||||
```
|
||||
|
||||
You can attach to the new ephemeral container using `kubectl attach`:
|
||||
|
||||
```shell
|
||||
kubectl attach -it example-pod -c debugger
|
||||
```
|
||||
|
||||
If process namespace sharing is enabled, you can see processes from all the containers in that Pod. For example:
|
||||
|
||||
```shell
|
||||
/ # ps auxww
|
||||
PID USER TIME COMMAND
|
||||
1 root 0:00 /pause
|
||||
6 root 0:00 nginx: master process nginx -g daemon off;
|
||||
11 101 0:00 nginx: worker process
|
||||
12 101 0:00 nginx: worker process
|
||||
13 101 0:00 nginx: worker process
|
||||
14 101 0:00 nginx: worker process
|
||||
15 101 0:00 nginx: worker process
|
||||
16 101 0:00 nginx: worker process
|
||||
17 101 0:00 nginx: worker process
|
||||
18 101 0:00 nginx: worker process
|
||||
19 root 0:00 /pause
|
||||
24 root 0:00 sh
|
||||
29 root 0:00 ps auxww
|
||||
```
|
||||
|
||||
{{% /capture %}}
|
||||
@@ -101,7 +101,7 @@ Each probe has one of three results:
|
||||
* Failure: The Container failed the diagnostic.
|
||||
* Unknown: The diagnostic failed, so no action should be taken.
|
||||
|
||||
The kubelet can optionally perform and react to two kinds of probes on running
|
||||
The kubelet can optionally perform and react to three kinds of probes on running
|
||||
Containers:
|
||||
|
||||
* `livenessProbe`: Indicates whether the Container is running. If
|
||||
@@ -115,7 +115,15 @@ Containers:
|
||||
state of readiness before the initial delay is `Failure`. If a Container does
|
||||
not provide a readiness probe, the default state is `Success`.
|
||||
|
||||
### When should you use liveness or readiness probes?
|
||||
* `startupProbe`: Indicates whether the application within the Container is started.
|
||||
All other probes are disabled if a startup probe is provided, until it succeeds.
|
||||
If the startup probe fails, the kubelet kills the Container, and the Container
|
||||
is subjected to its [restart policy](#restart-policy). If a Container does not
|
||||
provide a startup probe, the default state is `Success`.
|
||||
|
||||
### When should you use a liveness probe?
|
||||
|
||||
{{< feature-state for_k8s_version="v1.0" state="stable" >}}
|
||||
|
||||
If the process in your Container is able to crash on its own whenever it
|
||||
encounters an issue or becomes unhealthy, you do not necessarily need a liveness
|
||||
@@ -125,6 +133,10 @@ with the Pod's `restartPolicy`.
|
||||
If you'd like your Container to be killed and restarted if a probe fails, then
|
||||
specify a liveness probe, and specify a `restartPolicy` of Always or OnFailure.
|
||||
|
||||
### When should you use a readiness probe?
|
||||
|
||||
{{< feature-state for_k8s_version="v1.0" state="stable" >}}
|
||||
|
||||
If you'd like to start sending traffic to a Pod only when a probe succeeds,
|
||||
specify a readiness probe. In this case, the readiness probe might be the same
|
||||
as the liveness probe, but the existence of the readiness probe in the spec means
|
||||
@@ -142,6 +154,13 @@ puts itself into an unready state regardless of whether the readiness probe exis
|
||||
The Pod remains in the unready state while it waits for the Containers in the Pod
|
||||
to stop.
|
||||
|
||||
### When should you use a startup probe?
|
||||
|
||||
{{< feature-state for_k8s_version="v1.16" state="alpha" >}}
|
||||
|
||||
If your Container usually starts in more than `initialDelaySeconds + failureThreshold × periodSeconds`, you should specify a startup probe that checks the same endpoint as the liveness probe. The default for `periodSeconds` is 30s.
|
||||
You should then set its `failureThreshold` high enough to allow the Container to start, without changing the default values of the liveness probe. This helps to protect against deadlocks.
|
||||
|
||||
For more information about how to set up a liveness or readiness probe, see
|
||||
[Configure Liveness and Readiness Probes](/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/).
|
||||
|
||||
|
||||
@@ -0,0 +1,209 @@
|
||||
---
|
||||
title: Pod Topology Spread Constraints
|
||||
content_template: templates/concept
|
||||
weight: 50
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
{{< feature-state for_k8s_version="v1.16" state="alpha" >}}
|
||||
|
||||
You can use _topology spread constraints_ to control how {{< glossary_tooltip text="Pods" term_id="Pod" >}} are spread across your cluster among failure-domains such as regions, zones, nodes, and other user-defined topology domains. This can help to achieve high availability as well as efficient resource utilization.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture body %}}
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### Enable Feature Gate
|
||||
|
||||
Ensure the `EvenPodsSpread` feature gate is enabled (it is disabled by default
|
||||
in 1.16). See [Feature Gates](/docs/reference/command-line-tools-reference/feature-gates/)
|
||||
for an explanation of enabling feature gates. The `EvenPodsSpread` feature gate must be enabled for the
|
||||
{{< glossary_tooltip text="API Server" term_id="kube-apiserver" >}} **and**
|
||||
{{< glossary_tooltip text="scheduler" term_id="kube-scheduler" >}}.
|
||||
|
||||
### Node Labels
|
||||
|
||||
Topology spread constraints rely on node labels to identify the topology domain(s) that each Node is in. For example, a Node might have labels: `node=node1,zone=us-east-1a,region=us-east-1`
|
||||
|
||||
Suppose you have a 4-node cluster with the following labels:
|
||||
|
||||
```
|
||||
NAME STATUS ROLES AGE VERSION LABELS
|
||||
node1 Ready <none> 4m26s v1.16.0 node=node1,zone=zoneA
|
||||
node2 Ready <none> 3m58s v1.16.0 node=node2,zone=zoneA
|
||||
node3 Ready <none> 3m17s v1.16.0 node=node3,zone=zoneB
|
||||
node4 Ready <none> 2m43s v1.16.0 node=node4,zone=zoneB
|
||||
```
|
||||
|
||||
Then the cluster is logically viewed as below:
|
||||
|
||||
```
|
||||
+---------------+---------------+
|
||||
| zoneA | zoneB |
|
||||
+-------+-------+-------+-------+
|
||||
| node1 | node2 | node3 | node4 |
|
||||
+-------+-------+-------+-------+
|
||||
```
|
||||
|
||||
Instead of manually applying labels, you can also reuse the [well-known labels](/docs/reference/kubernetes-api/labels-annotations-taints/) that are created and populated automatically on most clusters.
|
||||
|
||||
## Spread Constraints for Pods
|
||||
|
||||
### API
|
||||
|
||||
The field `pod.spec.topologySpreadConstraints` is introduced in 1.16 as below:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: mypod
|
||||
spec:
|
||||
topologySpreadConstraints:
|
||||
- maxSkew: <integer>
|
||||
topologyKey: <string>
|
||||
whenUnsatisfiable: <string>
|
||||
labelSelector: <object>
|
||||
```
|
||||
|
||||
You can define one or multiple `topologySpreadConstraint` to instruct the kube-scheduler how to place each incoming Pod in relation to the existing Pods across your cluster. The fields are:
|
||||
|
||||
- **maxSkew** describes the degree to which Pods may be unevenly distributed. It's the maximum permitted difference between the number of matching Pods in any two topology domains of a given topology type. It must be greater than zero.
|
||||
- **topologyKey** is the key of node labels. If two Nodes are labelled with this key and have identical values for that label, the scheduler treats both Nodes as being in the same topology. The scheduler tries to place a balanced number of Pods into each topology domain.
|
||||
- **whenUnsatisfiable** indicates how to deal with a Pod if it doesn't satisfy the spread constraint:
|
||||
- `DoNotSchedule` (default) tells the scheduler not to schedule it.
|
||||
- `ScheduleAnyway` tells the scheduler to still schedule it while prioritizing nodes that minimize the skew.
|
||||
- **labelSelector** is used to find matching Pods. Pods that match this label selector are counted to determine the number of Pods in their corresponding topology domain. See [Label Selectors](/docs/concepts/overview/working-with-objects/labels/#label-selectors) for more details.
|
||||
|
||||
You can read more about this field by running `kubectl explain Pod.spec.topologySpreadConstraints`.
|
||||
|
||||
### Example: One TopologySpreadConstraint
|
||||
|
||||
Suppose you have a 4-node cluster where 3 Pods labeled `foo:bar` are located in node1, node2 and node3 respectively (`P` represents Pod):
|
||||
|
||||
```
|
||||
+---------------+---------------+
|
||||
| zoneA | zoneB |
|
||||
+-------+-------+-------+-------+
|
||||
| node1 | node2 | node3 | node4 |
|
||||
+-------+-------+-------+-------+
|
||||
| P | P | P | |
|
||||
+-------+-------+-------+-------+
|
||||
```
|
||||
|
||||
If we want an incoming Pod to be evenly spread with existing Pods across zones, the spec can be given as:
|
||||
|
||||
{{< codenew file="pods/topology-spread-constraints/one-constraint.yaml" >}}
|
||||
|
||||
`topologyKey: zone` implies the even distribution will only be applied to the nodes which have label pair "zone:<any value>" present. `whenUnsatisfiable: DoNotSchedule` tells the scheduler to let it stay pending if the incoming Pod can’t satisfy the constraint.
|
||||
|
||||
If the scheduler placed this incoming Pod into "zoneA", the Pods distribution would become [3, 1], hence the actual skew is 2 (3 - 1) - which violates `maxSkew: 1`. In this example, the incoming Pod can only be placed onto "zoneB":
|
||||
|
||||
```
|
||||
+---------------+---------------+ +---------------+---------------+
|
||||
| zoneA | zoneB | | zoneA | zoneB |
|
||||
+-------+-------+-------+-------+ +-------+-------+-------+-------+
|
||||
| node1 | node2 | node3 | node4 | OR | node1 | node2 | node3 | node4 |
|
||||
+-------+-------+-------+-------+ +-------+-------+-------+-------+
|
||||
| P | P | P | P | | P | P | P P | |
|
||||
+-------+-------+-------+-------+ +-------+-------+-------+-------+
|
||||
```
|
||||
|
||||
You can tweak the Pod spec to meet various kinds of requirements:
|
||||
|
||||
- Change `maxSkew` to a bigger value like "2" so that the incoming Pod can be placed onto "zoneA" as well.
|
||||
- Change `topologyKey` to "node" so as to distribute the Pods evenly across nodes instead of zones. In the above example, if `maxSkew` remains "1", the incoming Pod can only be placed onto "node4".
|
||||
- Change `whenUnsatisfiable: DoNotSchedule` to `whenUnsatisfiable: ScheduleAnyway` to ensure the incoming Pod to be always schedulable (suppose other scheduling APIs are satisfied). However, it’s preferred to be placed onto the topology domain which has fewer matching Pods. (Be aware that this preferability is jointly normalized with other internal scheduling priorities like resource usage ratio, etc.)
|
||||
|
||||
### Example: Multiple TopologySpreadConstraints
|
||||
|
||||
This builds upon the previous example. Suppose you have a 4-node cluster where 3 Pods labeled `foo:bar` are located in node1, node2 and node3 respectively (`P` represents Pod):
|
||||
|
||||
```
|
||||
+---------------+---------------+
|
||||
| zoneA | zoneB |
|
||||
+-------+-------+-------+-------+
|
||||
| node1 | node2 | node3 | node4 |
|
||||
+-------+-------+-------+-------+
|
||||
| P | P | P | |
|
||||
+-------+-------+-------+-------+
|
||||
```
|
||||
|
||||
You can use 2 TopologySpreadConstraints to control the Pods spreading on both zone and node:
|
||||
|
||||
{{< codenew file="pods/topology-spread-constraints/two-constraints.yaml" >}}
|
||||
|
||||
In this case, to match the first constraint, the incoming Pod can only be placed onto "zoneB"; while in terms of the second constraint, the incoming Pod can only be placed onto "node4". Then the results of 2 constraints are ANDed, so the only viable option is to place on "node4".
|
||||
|
||||
Multiple constraints can lead to conflicts. Suppose you have a 3-node cluster across 2 zones:
|
||||
|
||||
```
|
||||
+---------------+-------+
|
||||
| zoneA | zoneB |
|
||||
+-------+-------+-------+
|
||||
| node1 | node2 | nod3 |
|
||||
+-------+-------+-------+
|
||||
| P P | P | P P |
|
||||
+-------+-------+-------+
|
||||
```
|
||||
|
||||
If you apply "two-constraints.yaml" to this cluster, you will notice "mypod" stays in `Pending` state. This is because: to satisfy the first constraint, "mypod" can only be put to "zoneB"; while in terms of the second constraint, "mypod" can only put to "node2". Then a joint result of "zoneB" and "node2" returns nothing.
|
||||
|
||||
To overcome this situation, you can either increase the `maxSkew` or modify one of the constraints to use `whenUnsatisfiable: ScheduleAnyway`.
|
||||
|
||||
### Conventions
|
||||
|
||||
There are some implicit conventions worth noting here:
|
||||
|
||||
- Only the Pods holding the same namespace as the incoming Pod can be matching candidates.
|
||||
|
||||
- Nodes without `topologySpreadConstraints[*].topologyKey` present will be bypassed. It implies that:
|
||||
1. the Pods located on those nodes do not impact `maxSkew` calculation - in the above example, suppose "node1" does not have label "zone", then the 2 Pods will be disregarded, hence the incomingPod will be scheduled into "zoneA".
|
||||
2. the incoming Pod has no chances to be scheduled onto this kind of nodes - in the above example, suppose a "node5" carrying label `{zone-typo: zoneC}` joins the cluster, it will be bypassed due to the absence of label key "zone".
|
||||
|
||||
- Be aware of what will happen if the incomingPod’s `topologySpreadConstraints[*].labelSelector` doesn’t match its own labels. In the above example, if we remove the incoming Pod’s labels, it can still be placed onto "zoneB" since the constraints are still satisfied. However, after the placement, the degree of imbalance of the cluster remains unchanged - it’s still zoneA having 2 Pods which hold label {foo:bar}, and zoneB having 1 Pod which holds label {foo:bar}. So if this is not what you expect, we recommend the workload’s `topologySpreadConstraints[*].labelSelector` to match its own labels.
|
||||
|
||||
- If the incoming Pod has `spec.nodeSelector` or `spec.affinity.nodeAffinity` defined, nodes not matching them will be bypassed.
|
||||
|
||||
Suppose you have a 5-node cluster ranging from zoneA to zoneC:
|
||||
|
||||
```
|
||||
+---------------+---------------+-------+
|
||||
| zoneA | zoneB | zoneC |
|
||||
+-------+-------+-------+-------+-------+
|
||||
| node1 | node2 | node3 | node4 | node5 |
|
||||
+-------+-------+-------+-------+-------+
|
||||
| P | P | P | | |
|
||||
+-------+-------+-------+-------+-------+
|
||||
```
|
||||
|
||||
and you know that "zoneC" must be excluded. In this case, you can compose the yaml as below, so that "mypod" will be placed onto "zoneB" instead of "zoneC". Similarly `spec.nodeSelector` is also respected.
|
||||
|
||||
{{< codenew file="pods/topology-spread-constraints/one-constraint-with-nodeaffinity.yaml" >}}
|
||||
|
||||
## Comparison with PodAffinity/PodAntiAffinity
|
||||
|
||||
In Kubernetes, directives related to "Affinity" control how Pods are
|
||||
scheduled - more packed or more scattered.
|
||||
|
||||
- For `PodAffinity`, you can try to pack any number of Pods into qualifying
|
||||
topology domain(s)
|
||||
- For `PodAntiAffinity`, only one Pod can be scheduled into a
|
||||
single topology domain.
|
||||
|
||||
The "EvenPodsSpread" feature provides flexible options to distribute Pods evenly across different
|
||||
topology domains - to achieve high availability or cost-saving. This can also help on rolling update
|
||||
workloads and scaling out replicas smoothly.
|
||||
See [Motivation](https://github.com/kubernetes/enhancements/blob/master/keps/sig-scheduling/20190221-even-pods-spreading.md#motivation) for more details.
|
||||
|
||||
## Known Limitations
|
||||
|
||||
As of 1.16, at which this feature is Alpha, there are some known limitations:
|
||||
|
||||
- Scaling down a `Deployment` may result in imbalanced Pods distribution.
|
||||
- Pods matched on tainted nodes are respected. See [Issue 80921](https://github.com/kubernetes/kubernetes/issues/80921)
|
||||
|
||||
{{% /capture %}}
|
||||
Reference in New Issue
Block a user