Switch language name 'zh' to 'zh-cn'

This is the first step to rename 'zh' to 'zh-cn'. There are several reasons why we rename the language name.

- The upstream docsy theme changed the language name, leading to many warnings during site build;
  The side-effect is that the i18n strings are no longer working.
- We believe renaming the language is the right thing to do, because this move can make room for other variants of Chinese language, such as 'zh-tw', 'zh-sg' etc.

There would be several follow-ups to this PR, such as fixing the intra-site links, adding redirects etc.

We will lock up changes to zh/zh-cn pages for the moment, until this one gets in.

This PR is based on commit cdad0a7342.
This commit is contained in:
Qiming Teng
2022-06-10 20:25:27 +08:00
parent 3d345b1816
commit c52818c03d
1347 changed files with 8 additions and 6 deletions
@@ -0,0 +1,52 @@
---
title: 调度,抢占和驱逐
weight: 90
content_type: concept
description: >
在Kubernetes中,调度 (scheduling) 指的是确保 Pods 匹配到合适的节点,
以便 kubelet 能够运行它们。抢占 (Preemption) 指的是终止低优先级的 Pods 以便高优先级的 Pods 可以
调度运行的过程。驱逐 (Eviction) 是在资源匮乏的节点上,主动让一个或多个 Pods 失效的过程。
no_list: true
---
<!--
title: "Scheduling, Preemption and Eviction"
weight: 90
content_type: concept
description: >
In Kubernetes, scheduling refers to making sure that Pods are matched to Nodes
so that the kubelet can run them. Preemption is the process of terminating
Pods with lower Priority so that Pods with higher Priority can schedule on
Nodes. Eviction is the process of proactively terminating one or more Pods on
resource-starved Nodes.
no_list: true
-->
<!--
In Kubernetes, scheduling refers to making sure that {{<glossary_tooltip text="Pods" term_id="pod">}}
are matched to {{<glossary_tooltip text="Nodes" term_id="node">}} so that the
{{<glossary_tooltip text="kubelet" term_id="kubelet">}} can run them. Preemption
is the process of terminating Pods with lower {{<glossary_tooltip text="Priority" term_id="pod-priority">}}
so that Pods with higher Priority can schedule on Nodes. Eviction is the process
of terminating one or more Pods on Nodes.
-->
<!-- ## Scheduling -->
## 调度
* [Kubernetes 调度器](/zh/docs/concepts/scheduling-eviction/kube-scheduler/)
* [将 Pods 指派到节点](/zh/docs/concepts/scheduling-eviction/assign-pod-node/)
* [Pod 开销](/zh/docs/concepts/scheduling-eviction/pod-overhead/)
* [污点和容忍](/zh/docs/concepts/scheduling-eviction/taint-and-toleration/)
* [调度框架](/zh/docs/concepts/scheduling-eviction/scheduling-framework)
* [调度器的性能调试](/zh/docs/concepts/scheduling-eviction/scheduler-perf-tuning/)
* [扩展资源的资源装箱](/zh/docs/concepts/scheduling-eviction/resource-bin-packing/)
<!-- ## Pod Disruption -->
## Pod 干扰
* [Pod 优先级和抢占](/zh/docs/concepts/scheduling-eviction/pod-priority-preemption/)
* [节点压力驱逐](/zh/docs/concepts/scheduling-eviction/pod-priority-preemption/)
* [API发起的驱逐](/zh/docs/concepts/scheduling-eviction/api-eviction/)
@@ -0,0 +1,206 @@
---
title: API 发起的驱逐
content_type: concept
weight: 70
---
<!--
---
title: API-initiated Eviction
content_type: concept
weight: 70
---
-->
{{< glossary_definition term_id="api-eviction" length="short" >}} </br>
<!--
You can request eviction by calling the Eviction API directly, or programmatically
using a client of the {{<glossary_tooltip term_id="kube-apiserver" text="API server">}}, like the `kubectl drain` command. This
creates an `Eviction` object, which causes the API server to terminate the Pod.
API-initiated evictions respect your configured [`PodDisruptionBudgets`](/docs/tasks/run-application/configure-pdb/)
and [`terminationGracePeriodSeconds`](/docs/concepts/workloads/pods/pod-lifecycle#pod-termination).
Using the API to create an Eviction object for a Pod is like performing a
policy-controlled [`DELETE` operation](/docs/reference/kubernetes-api/workload-resources/pod-v1/#delete-delete-a-pod)
on the Pod.
-->
你可以通过直接调用 Eviction API 发起驱逐,也可以通过编程的方式使用
{{<glossary_tooltip term_id="kube-apiserver" text="API 服务器">}}的客户端来发起驱逐,
比如 `kubectl drain` 命令。
此操作创建一个 `Eviction` 对象,该对象再驱动 API 服务器终止选定的 Pod。
API 发起的驱逐将遵从你的
[`PodDisruptionBudgets`](/zh/docs/tasks/run-application/configure-pdb/)
和 [`terminationGracePeriodSeconds`](/zh/docs/concepts/workloads/pods/pod-lifecycle#pod-termination)
配置。
使用 API 创建 Eviction 对象,就像对 Pod 执行策略控制的
[`DELETE` 操作](/zh/docs/reference/kubernetes-api/workload-resources/pod-v1/#delete-delete-a-pod)
<!--
## Calling the Eviction API
You can use a [Kubernetes language client](/docs/tasks/administer-cluster/access-cluster-api/#programmatic-access-to-the-api)
to access the Kubernetes API and create an `Eviction` object. To do this, you
POST the attempted operation, similar to the following example:
-->
## 调用 Eviction API
你可以使用 [Kubernetes 语言客户端](/zh/docs/tasks/administer-cluster/access-cluster-api/#programmatic-access-to-the-api)
来访问 Kubernetes API 并创建 `Eviction` 对象。
要执行此操作,你应该用 POST 发出要尝试的请求,类似于下面的示例:
{{< tabs name="Eviction_example" >}}
{{% tab name="policy/v1" %}}
{{< note >}}
<!-- `policy/v1` Eviction is available in v1.22+. Use `policy/v1beta1` with prior releases. -->
`policy/v1` 版本的 Eviction 在 v1.22 以及更高的版本中可用,之前的发行版本使用 `policy/v1beta1` 版本。
{{< /note >}}
```json
{
"apiVersion": "policy/v1",
"kind": "Eviction",
"metadata": {
"name": "quux",
"namespace": "default"
}
}
```
{{% /tab %}}
{{% tab name="policy/v1beta1" %}}
{{< note >}}
<!-- Deprecated in v1.22 in favor of `policy/v1` -->
在 v1.22 版本废弃以支持 `policy/v1`
{{< /note >}}
```json
{
"apiVersion": "policy/v1beta1",
"kind": "Eviction",
"metadata": {
"name": "quux",
"namespace": "default"
}
}
```
{{% /tab %}}
{{< /tabs >}}
<!--
Alternatively, you can attempt an eviction operation by accessing the API using
`curl` or `wget`, similar to the following example:
-->
或者,你可以通过使用 `curl` 或者 `wget` 来访问 API 以尝试驱逐操作,类似于以下示例:
```bash
curl -v -H 'Content-type: application/json' https://your-cluster-api-endpoint.example/api/v1/namespaces/default/pods/quux/eviction -d @eviction.json
```
<!--
## How API-initiated eviction works
When you request an eviction using the API, the API server performs admission
checks and responds in one of the following ways:
-->
## API 发起驱逐的工作原理
当你使用 API 来请求驱逐时,API 服务器将执行准入检查,并通过以下方式之一做出响应:
<!--
* `200 OK`: the eviction is allowed, the `Eviction` subresource is created, and
the Pod is deleted, similar to sending a `DELETE` request to the Pod URL.
* `429 Too Many Requests`: the eviction is not currently allowed because of the
configured {{<glossary_tooltip term_id="pod-disruption-budget" text="PodDisruptionBudget">}}.
You may be able to attempt the eviction again later. You might also see this
response because of API rate limiting.
* `500 Internal Server Error`: the eviction is not allowed because there is a
misconfiguration, like if multiple PodDisruptionBudgets reference the same Pod.
-->
* `200 OK`:允许驱逐,子资源 `Eviction` 被创建,并且 Pod 被删除,
类似于发送一个 `DELETE` 请求到 Pod 地址。
* `429 Too Many Requests`:当前不允许驱逐,因为配置了 {{<glossary_tooltip term_id="pod-disruption-budget" text="PodDisruptionBudget">}}。
你可以稍后再尝试驱逐。你也可能因为 API 速率限制而看到这种响应。
* `500 Internal Server Error`:不允许驱逐,因为存在配置错误,
例如存在多个 PodDisruptionBudgets 引用同一个 Pod。
<!--
If the Pod you want to evict isn't part of a workload that has a
PodDisruptionBudget, the API server always returns `200 OK` and allows the
eviction.
If the API server allows the eviction, the Pod is deleted as follows:
-->
如果你想驱逐的 Pod 不属于有 PodDisruptionBudget 的工作负载,
API 服务器总是返回 `200 OK` 并且允许驱逐。
如果 API 服务器允许驱逐,Pod 按照如下方式删除:
<!--
1. The `Pod` resource in the API server is updated with a deletion timestamp,
after which the API server considers the `Pod` resource to be terminated. The
`Pod` resource is also marked with the configured grace period.
1. The {{<glossary_tooltip term_id="kubelet" text="kubelet">}} on the node where the local Pod is running notices that the `Pod`
resource is marked for termination and starts to gracefully shut down the
local Pod.
1. While the kubelet is shutting the Pod down, the control plane removes the Pod
from {{<glossary_tooltip term_id="endpoint" text="Endpoint">}} and
{{<glossary_tooltip term_id="endpoint-slice" text="EndpointSlice">}}
objects. As a result, controllers no longer consider the Pod as a valid object.
1. After the grace period for the Pod expires, the kubelet forcefully terminates
the local Pod.
1. The kubelet tells the API server to remove the `Pod` resource.
1. The API server deletes the `Pod` resource.
-->
1. API 服务器中的 `Pod` 资源会更新上删除时间戳,之后 API 服务器会认为此 `Pod` 资源将被终止。
`Pod` 资源还会标记上配置的宽限期。
1. 本地运行状态的 Pod 所处的节点上的 {{<glossary_tooltip term_id="kubelet" text="kubelet">}}
注意到 `Pod` 资源被标记为终止,并开始优雅停止本地 Pod。
1. 当 kubelet 停止 Pod 时,控制面从 {{<glossary_tooltip term_id="endpoint" text="Endpoint">}}
和 {{<glossary_tooltip term_id="endpoint-slice" text="EndpointSlice">}}
对象中移除该 Pod。因此,控制器不再将此 Pod 视为有用对象。
1. Pod 的宽限期到期后,kubelet 强制终止本地 Pod。
1. kubelet 告诉 API 服务器删除 `Pod` 资源。
1. API 服务器删除 `Pod` 资源。
<!--
## Troubleshooting stuck evictions
In some cases, your applications may enter a broken state, where the Eviction
API will only return `429` or `500` responses until you intervene. This can
happen if, for example, a ReplicaSet creates pods for your application but new
pods do not enter a `Ready` state. You may also notice this behavior in cases
where the last evicted Pod had a long termination grace period.
-->
## 解决驱逐被卡住的问题
在某些情况下,你的应用可能进入中断状态,
在你干预之前,驱逐 API 总是返回 `429``500`
例如,如果 ReplicaSet 为你的应用程序创建了 Pod,
但新的 Pod 没有进入 `Ready` 状态,就会发生这种情况。
在最后一个被驱逐的 Pod 有很长的终止宽限期的情况下,你可能也会注意到这种行为。
<!--
If you notice stuck evictions, try one of the following solutions:
* Abort or pause the automated operation causing the issue. Investigate the stuck
application before you restart the operation.
* Wait a while, then directly delete the Pod from your cluster control plane
instead of using the Eviction API.
-->
如果你注意到驱逐被卡住,请尝试以下解决方案之一:
* 终止或暂停导致问题的自动化操作,重新启动操作之前,请检查被卡住的应用程序。
* 等待一段时间后,直接从集群控制平面删除 Pod,而不是使用 Eviction API。
## {{% heading "whatsnext" %}}
<!--
* Learn how to protect your applications with a [Pod Disruption Budget](/docs/tasks/run-application/configure-pdb/).
* Learn about [Node-pressure Eviction](/docs/concepts/scheduling-eviction/node-pressure-eviction/).
* Learn about [Pod Priority and Preemption](/docs/concepts/scheduling-eviction/pod-priority-preemption/).
-->
* 了解如何使用 [Pod 干扰预算](/zh/docs/tasks/run-application/configure-pdb/) 保护你的应用。
* 了解[节点压力引发的驱逐](/zh/docs/concepts/scheduling-eviction/node-pressure-eviction/)。
* 了解 [Pod 优先级和抢占](/zh/docs/concepts/scheduling-eviction/pod-priority-preemption/)。
@@ -0,0 +1,821 @@
---
title: 将 Pod 指派给节点
content_type: concept
weight: 20
---
<!--
reviewers:
- davidopp
- kevin-wangzefeng
- bsalamat
title: Assigning Pods to Nodes
content_type: concept
weight: 20
-->
<!-- overview -->
<!--
You can constrain a {{< glossary_tooltip text="Pod" term_id="pod" >}} so that it can only run on particular set of
{{< glossary_tooltip text="Node(s)" term_id="node" >}}.
There are several ways to do this, and the recommended approaches all use
[label selectors](/docs/concepts/overview/working-with-objects/labels/) to facilitate the selection.
Generally such constraints are unnecessary, as the scheduler will automatically do a reasonable placement
(e.g. spread your pods across nodes so as not place the pod on a node with insufficient free resources, etc.)
However, there are some circumstances where you may want to control which node
the Pod deploys to, for example, to ensure that a Pod ends up on a node with an SSD attached to it, or to co-locate pods from two different
services that communicate a lot into the same availability zone.
-->
你可以约束一个 {{< glossary_tooltip text="Pod" term_id="pod" >}}
只能在特定的{{< glossary_tooltip text="节点" term_id="node" >}}上运行。
有几种方法可以实现这点,推荐的方法都是用
[标签选择算符](/zh/docs/concepts/overview/working-with-objects/labels/)来进行选择。
通常这样的约束不是必须的,因为调度器将自动进行合理的放置(比如,将 Pod 分散到节点上,
而不是将 Pod 放置在可用资源不足的节点上等等)。但在某些情况下,你可能需要进一步控制
Pod 被部署到的节点。例如,确保 Pod 最终落在连接了 SSD 的机器上,
或者将来自两个不同的服务且有大量通信的 Pods 被放置在同一个可用区。
<!-- body -->
<!--
You can use any of the following methods to choose where Kubernetes schedules
specific Pods:
* [nodeSelector](#nodeselector) field matching against [node labels](#built-in-node-labels)
* [Affinity and anti-affinity](#affinity-and-anti-affinity)
* [nodeName](#nodename) field
-->
你可以使用下列方法中的任何一种来选择 Kubernetes 对特定 Pod 的调度:
* 与[节点标签](#built-in-node-labels)匹配的 [nodeSelector](#nodeSelector)
* [亲和性与反亲和性](#affinity-and-anti-affinity)
* [nodeName](#nodename) 字段
<!--
## Node labels {#built-in-node-labels}
Like many other Kubernetes objects, nodes have
[labels](/docs/concepts/overview/working-with-objects/labels/). You can [attach labels manually](/docs/tasks/configure-pod-container/assign-pods-nodes/#add-a-label-to-a-node).
Kubernetes also populates a standard set of labels on all nodes in a cluster. See [Well-Known Labels, Annotations and Taints](/docs/reference/labels-annotations-taints/)
for a list of common node labels.
-->
## 节点标签 {#built-in-node-labels}
与很多其他 Kubernetes 对象类似,节点也有[标签](/zh/docs/concepts/overview/working-with-objects/labels/)。
你可以[手动地添加标签](/zh/docs/tasks/configure-pod-container/assign-pods-nodes/#add-a-label-to-a-node)。
Kubernetes 也会为集群中所有节点添加一些标准的标签。
参见[常用的标签、注解和污点](/zh/docs/reference/labels-annotations-taints/)以了解常见的节点标签。
{{< note >}}
<!--
The value of these labels is cloud provider specific and is not guaranteed to be reliable.
For example, the value of `kubernetes.io/hostname` may be the same as the node name in some environments
and a different value in other environments.
-->
这些标签的取值是取决于云提供商的,并且是无法在可靠性上给出承诺的。
例如,`kubernetes.io/hostname` 的取值在某些环境中可能与节点名称相同,
而在其他环境中会取不同的值。
{{< /note >}}
<!--
### Node isolation/restriction
Adding labels to nodes allows you to target Pods for scheduling on specific
nodes or groups of nodes. You can use this functionality to ensure that specific
Pods only run on nodes with certain isolation, security, or regulatory
properties.
-->
## 节点隔离/限制 {#node-isolation-restriction}
通过为节点添加标签,你可以准备让 Pod 调度到特定节点或节点组上。
你可以使用这个功能来确保特定的 Pod 只能运行在具有一定隔离性,安全性或监管属性的节点上。
<!--
If you use labels for node isolation, choose label keys that the {{<glossary_tooltip text="kubelet" term_id="kubelet">}}
cannot modify. This prevents a compromised node from setting those labels on
itself so that the scheduler schedules workloads onto the compromised node.
-->
如果使用标签来实现节点隔离,建议选择节点上的
{{<glossary_tooltip text="kubelet" term_id="kubelet">}}
无法修改的标签键。
这可以防止受感染的节点在自身上设置这些标签,进而影响调度器将工作负载调度到受感染的节点。
<!--
The [`NodeRestriction` admission plugin](/docs/reference/access-authn-authz/admission-controllers/#noderestriction)
prevents the kubelet from setting or modifying labels with a
`node-restriction.kubernetes.io/` prefix.
To make use of that label prefix for node isolation:
-->
[`NodeRestriction` 准入插件](/zh/docs/reference/access-authn-authz/admission-controllers/#noderestriction)防止
kubelet 使用 `node-restriction.kubernetes.io/` 前缀设置或修改标签。
要使用该标签前缀进行节点隔离:
<!--
1. Ensure you are using the [Node authorizer](/docs/reference/access-authn-authz/node/) and have _enabled_ the `NodeRestriction` admission plugin.
2. Add labels with the `node-restriction.kubernetes.io/` prefix to your nodes, and use those labels in your [node selectors](#nodeselector).
For example, `example.com.node-restriction.kubernetes.io/fips=true` or `example.com.node-restriction.kubernetes.io/pci-dss=true`.
-->
1. 确保你在使用[节点鉴权](/zh/docs/reference/access-authn-authz/node/)机制并且已经启用了
[NodeRestriction 准入插件](/zh/docs/reference/access-authn-authz/admission-controllers/#noderestriction)。
2. 将带有 `node-restriction.kubernetes.io/` 前缀的标签添加到 Node 对象,
然后在[节点选择器](#nodeSelector)中使用这些标签。
例如,`example.com.node-restriction.kubernetes.io/fips=true`
`example.com.node-restriction.kubernetes.io/pci-dss=true`
## nodeSelector
<!--
`nodeSelector` is the simplest recommended form of node selection constraint.
You can add the `nodeSelector` field to your Pod specification and specify the
[node labels](#built-in-node-labels) you want the target node to have.
Kubernetes only schedules the Pod onto nodes that have each of the labels you
specify.
-->
`nodeSelector` 是节点选择约束的最简单推荐形式。你可以将 `nodeSelector` 字段添加到
Pod 的规约中设置你希望目标节点所具有的[节点标签](#built-in-node-labels)。
Kubernetes 只会将 Pod 调度到拥有你所指定的每个标签的节点上。
<!--
See [Assign Pods to Nodes](/docs/tasks/configure-pod-container/assign-pods-nodes) for more
information.
-->
进一步的信息可参见[将 Pod 指派给节点](/zh/docs/tasks/configure-pod-container/assign-pods-nodes)。
<!--
## Affinity and anti-affinity
`nodeSelector` is the simplest way to constrain Pods to nodes with specific
labels. Affinity and anti-affinity expands the types of constraints you can
define. Some of the benefits of affinity and anti-affinity include:
-->
## 亲和性与反亲和性 {#affinity-and-anti-affinity}
`nodeSelector` 提供了一种最简单的方法来将 Pod 约束到具有特定标签的节点上。
亲和性和反亲和性扩展了你可以定义的约束类型。使用亲和性与反亲和性的一些好处有:
<!--
* The affinity/anti-affinity language is more expressive. `nodeSelector` only
selects nodes with all the specified labels. Affinity/anti-affinity gives you
more control over the selection logic.
* You can indicate that a rule is *soft* or *preferred*, so that the scheduler
still schedules the Pod even if it can't find a matching node.
* You can constrain a Pod using labels on other Pods running on the node (or other topological domain),
instead of just node labels, which allows you to define rules for which Pods
can be co-located on a node.
-->
* 亲和性、反亲和性语言的表达能力更强。`nodeSelector` 只能选择拥有所有指定标签的节点。
亲和性、反亲和性为你提供对选择逻辑的更强控制能力。
* 你可以标明某规则是“软需求”或者“偏好”,这样调度器在无法找到匹配节点时仍然调度该 Pod。
* 你可以使用节点上(或其他拓扑域中)运行的其他 Pod 的标签来实施调度约束,
而不是只能使用节点本身的标签。这个能力让你能够定义规则允许哪些 Pod 可以被放置在一起。
<!--
### Node affinity
Node affinity is conceptually similar to `nodeSelector`, allowing you to constrain which nodes your
Pod can be scheduled on based on node labels. There are two types of node
affinity:
-->
### 节点亲和性 {#node-affinity}
节点亲和性概念上类似于 `nodeSelector`
它使你可以根据节点上的标签来约束 Pod 可以调度到哪些节点上。
节点亲和性有两种:
<!--
* `requiredDuringSchedulingIgnoredDuringExecution`: The scheduler can't
schedule the Pod unless the rule is met. This functions like `nodeSelector`,
but with a more expressive syntax.
* `preferredDuringSchedulingIgnoredDuringExecution`: The scheduler tries to
find a node that meets the rule. If a matching node is not available, the
scheduler still schedules the Pod.
-->
* `requiredDuringSchedulingIgnoredDuringExecution`
调度器只有在规则被满足的时候才能执行调度。此功能类似于 `nodeSelector`
但其语法表达能力更强。
* `preferredDuringSchedulingIgnoredDuringExecution`
调度器会尝试寻找满足对应规则的节点。如果找不到匹配的节点,调度器仍然会调度该 Pod。
{{<note>}}
<!--
In the preceding types, `IgnoredDuringExecution` means that if the node labels
change after Kubernetes schedules the Pod, the Pod continues to run.
-->
在上述类型中,`IgnoredDuringExecution` 意味着如果节点标签在 Kubernetes
调度 Pod 时发生了变更,Pod 仍将继续运行。
{{</note>}}
<!--
You can specify node affinities using the `.spec.affinity.nodeAffinity` field in
your Pod spec.
For example, consider the following Pod spec:
-->
你可以使用 Pod 规约中的 `.spec.affinity.nodeAffinity` 字段来设置节点亲和性。
例如,考虑下面的 Pod 规约:
{{< codenew file="pods/pod-with-node-affinity.yaml" >}}
<!--
In this example, the following rules apply:
* The node *must* have a label with the key `kubernetes.io/os` and
the value `linux`.
* The node *preferably* has a label with the key `another-node-label-key` and
the value `another-node-label-value`.
-->
在这一示例中,所应用的规则如下:
* 节点必须包含键名为 `kubernetes.io/os` 的标签,并且其取值为 `linux`
* 节点 **最好** 具有键名为 `another-node-label-key` 且取值为
`another-node-label-value` 的标签。
<!--
You can use the `operator` field to specify a logical operator for Kubernetes to use when
interpreting the rules. You can use `In`, `NotIn`, `Exists`, `DoesNotExist`,
`Gt` and `Lt`.
-->
你可以使用 `operator` 字段来为 Kubernetes 设置在解释规则时要使用的逻辑操作符。
你可以使用 `In``NotIn``Exists``DoesNotExist``Gt``Lt` 之一作为操作符。
<!--
`NotIn` and `DoesNotExist` allow you to define node anti-affinity behavior.
Alternatively, you can use [node taints](/docs/concepts/scheduling-eviction/taint-and-toleration/)
to repel Pods from specific nodes.
-->
`NotIn``DoesNotExist` 可用来实现节点反亲和性行为。
你也可以使用[节点污点](/zh/docs/concepts/scheduling-eviction/taint-and-toleration/)
将 Pod 从特定节点上驱逐。
{{< note >}}
<!--
If you specify both `nodeSelector` and `nodeAffinity`, *both* must be satisfied
for the Pod to be scheduled onto a node.
-->
如果你同时指定了 `nodeSelector``nodeAffinity`**两者** 必须都要满足,
才能将 Pod 调度到候选节点上。
<!--
If you specify multiple `nodeSelectorTerms` associated with `nodeAffinity`
types, then the Pod can be scheduled onto a node if one of the specified `nodeSelectorTerms` can be
satisfied.
-->
如果你指定了多个与 `nodeAffinity` 类型关联的 `nodeSelectorTerms`
只要其中一个 `nodeSelectorTerms` 满足的话,Pod 就可以被调度到节点上。
<!--
If you specify multiple `matchExpressions` associated with a single `nodeSelectorTerms`,
then the Pod can be scheduled onto a node only if all the `matchExpressions` are
satisfied.
-->
如果你指定了多个与同一 `nodeSelectorTerms` 关联的 `matchExpressions`
则只有当所有 `matchExpressions` 都满足时 Pod 才可以被调度到节点上。
{{< /note >}}
<!--
See [Assign Pods to Nodes using Node Affinity](/docs/tasks/configure-pod-container/assign-pods-nodes-using-node-affinity/)
for more information.
-->
参阅[使用节点亲和性来为 Pod 指派节点](/zh/docs/tasks/configure-pod-container/assign-pods-nodes-using-node-affinity/)
以了解进一步的信息。
<!--
#### Node affinity weight
You can specify a `weight` between 1 and 100 for each instance of the
`preferredDuringSchedulingIgnoredDuringExecution` affinity type. When the
scheduler finds nodes that meet all the other scheduling requirements of the Pod, the
scheduler iterates through every preferred rule that the node satisfies and adds the
value of the `weight` for that expression to a sum.
-->
#### 节点亲和性权重 {#node-affinity-weight}
你可以为 `preferredDuringSchedulingIgnoredDuringExecution` 亲和性类型的每个实例设置
`weight` 字段,其取值范围是 1 到 100。
当调度器找到能够满足 Pod 的其他调度请求的节点时,调度器会遍历节点满足的所有的偏好性规则,
并将对应表达式的 `weight` 值加和。
<!--
The final sum is added to the score of other priority functions for the node.
Nodes with the highest total score are prioritized when the scheduler makes a
scheduling decision for the Pod.
For example, consider the following Pod spec:
-->
最终的加和值会添加到该节点的其他优先级函数的评分之上。
在调度器为 Pod 作出调度决定时,总分最高的节点的优先级也最高。
例如,考虑下面的 Pod 规约:
{{< codenew file="pods/pod-with-affinity-anti-affinity.yaml" >}}
<!--
If there are two possible nodes that match the
`requiredDuringSchedulingIgnoredDuringExecution` rule, one with the
`label-1:key-1` label and another with the `label-2:key-2` label, the scheduler
considers the `weight` of each node and adds the weight to the other scores for
that node, and schedules the Pod onto the node with the highest final score.
-->
如果存在两个候选节点,都满足 `requiredDuringSchedulingIgnoredDuringExecution` 规则,
其中一个节点具有标签 `label-1:key-1`,另一个节点具有标签 `label-2:key-2`
调度器会考察各个节点的 `weight` 取值,并将该权重值添加到节点的其他得分值之上,
{{< note >}}
<!--
If you want Kubernetes to successfully schedule the Pods in this example, you
must have existing nodes with the `kubernetes.io/os=linux` label.
-->
如果你希望 Kubernetes 能够成功地调度此例中的 Pod,你必须拥有打了
`kubernetes.io/os=linux` 标签的节点。
{{< /note >}}
<!--
#### Node affinity per scheduling profile
-->
#### 逐个调度方案中设置节点亲和性 {#node-affinity-per-scheduling-profile}
{{< feature-state for_k8s_version="v1.20" state="beta" >}}
<!--
When configuring multiple [scheduling profiles](/docs/reference/scheduling/config/#multiple-profiles), you can associate
a profile with a Node affinity, which is useful if a profile only applies to a specific set of nodes.
To do so, add an `addedAffinity` to the `args` field of the [`NodeAffinity` plugin](/docs/reference/scheduling/config/#scheduling-plugins)
in the [scheduler configuration](/docs/reference/scheduling/config/). For example:
-->
在配置多个[调度方案](/zh/docs/reference/scheduling/config/#multiple-profiles)时,
你可以将某个方案与节点亲和性关联起来,如果某个调度方案仅适用于某组特殊的节点时,
这样做是很有用的。
要实现这点,可以在[调度器配置](/zh/docs/reference/scheduling/config/)中为
[`NodeAffinity` 插件](/zh/docs/reference/scheduling/config/#scheduling-plugins)的
`args` 字段添加 `addedAffinity`。例如:
```yaml
apiVersion: kubescheduler.config.k8s.io/v1beta3
kind: KubeSchedulerConfiguration
profiles:
- schedulerName: default-scheduler
- schedulerName: foo-scheduler
pluginConfig:
- name: NodeAffinity
args:
addedAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: scheduler-profile
operator: In
values:
- foo
```
<!--
The `addedAffinity` is applied to all Pods that set `.spec.schedulerName` to `foo-scheduler`, in addition to the
NodeAffinity specified in the PodSpec.
That is, in order to match the Pod, nodes need to satisfy `addedAffinity` and
the Pod's `.spec.NodeAffinity`.
-->
这里的 `addedAffinity` 除遵从 Pod 规约中设置的节点亲和性之外,还
适用于将 `.spec.schedulerName` 设置为 `foo-scheduler`
换言之,为了匹配 Pod,节点需要满足 `addedAffinity` 和 Pod 的 `.spec.NodeAffinity`
<!--
Since the `addedAffinity` is not visible to end users, its behavior might be
unexpected to them. Use node labels that have a clear correlation to the
scheduler profile name.
-->
由于 `addedAffinity` 对最终用户不可见,其行为可能对用户而言是出乎意料的。
应该使用与调度方案名称有明确关联的节点标签。
{{< note >}}
<!--
The DaemonSet controller, which [creates Pods for DaemonSets](/docs/concepts/workloads/controllers/daemonset/#scheduled-by-default-scheduler),
does not support scheduling profiles. When the DaemonSet controller creates
Pods, the default Kubernetes scheduler places those Pods and honors any
`nodeAffinity` rules in the DaemonSet controller.
-->
DaemonSet 控制器[为 DaemonSet 创建 Pods](/zh/docs/concepts/workloads/controllers/daemonset/#scheduled-by-default-scheduler)
但该控制器不理会调度方案。
DaemonSet 控制器创建 Pod 时,默认的 Kubernetes 调度器负责放置 Pod
并遵从 DaemonSet 控制器中奢侈的 `nodeAffinity` 规则。
{{< /note >}}
<!--
### Inter-pod affinity and anti-affinity
Inter-pod affinity and anti-affinity allow you to constrain which nodes your
Pods can be scheduled on based on the labels of **Pods** already running on that
node, instead of the node labels.
-->
### pod 间亲和性与反亲和性 {#inter-pod-affinity-and-anti-affinity}
Pod 间亲和性与反亲和性使你可以基于已经在节点上运行的 **Pod** 的标签来约束
Pod 可以调度到的节点,而不是基于节点上的标签。
<!--
Inter-pod affinity and anti-affinity rules take the form "this
Pod should (or, in the case of anti-affinity, should not) run in an X if that X
is already running one or more Pods that meet rule Y", where X is a topology
domain like node, rack, cloud provider zone or region, or similar and Y is the
rule Kubernetes tries to satisfy.
-->
Pod 间亲和性与反亲和性的规则格式为“如果 X 上已经运行了一个或多个满足规则 Y 的 Pod,
则这个 Pod 应该(或者在反亲和性的情况下不应该)运行在 X 上”。
这里的 X 可以是节点、机架、云提供商可用区或地理区域或类似的拓扑域,
Y 则是 Kubernetes 尝试满足的规则。
<!--
You express these rules (Y) as [label selectors](/docs/concepts/overview/working-with-objects/labels/#label-selectors)
with an optional associated list of namespaces. Pods are namespaced objects in
Kubernetes, so Pod labels also implicitly have namespaces. Any label selectors
for Pod labels should specify the namespaces in which Kubernetes should look for those
labels.
-->
你通过[标签选择算符](/zh/docs/concepts/overview/working-with-objects/labels/#label-selectors)
的形式来表达规则(Y),并可根据需要指定选关联的名字空间列表。
Pod 在 Kubernetes 中是名字空间作用域的对象,因此 Pod 的标签也隐式地具有名字空间属性。
针对 Pod 标签的所有标签选择算符都要指定名字空间,Kubernetes
会在指定的名字空间内寻找标签。
<!--
You express the topology domain (X) using a `topologyKey`, which is the key for
the node label that the system uses to denote the domain. For examples, see
[Well-Known Labels, Annotations and Taints](/docs/reference/labels-annotations-taints/).
-->
你会通过 `topologyKey` 来表达拓扑域(X)的概念,其取值是系统用来标示域的节点标签键。
相关示例可参见[常用标签、注解和污点](/zh/docs/reference/labels-annotations-taints/)。
{{< note >}}
<!--
Inter-pod affinity and anti-affinity require substantial amount of
processing which can slow down scheduling in large clusters significantly. We do
not recommend using them in clusters larger than several hundred nodes.
-->
Pod 间亲和性和反亲和性都需要相当的计算量,因此会在大规模集群中显著降低调度速度。
我们不建议在包含数百个节点的集群中使用这类设置。
{{< /note >}}
{{< note >}}
<!--
Pod anti-affinity requires nodes to be consistently labelled, in other words,
every node in the cluster must have an appropriate label matching `topologyKey`.
If some or all nodes are missing the specified `topologyKey` label, it can lead
to unintended behavior.
-->
Pod 反亲和性需要节点上存在一致性的标签。换言之,
集群中每个节点都必须拥有与 `topologyKey` 匹配的标签。
如果某些或者所有节点上不存在所指定的 `topologyKey` 标签,调度行为可能与预期的不同。
{{< /note >}}
<!--
#### Types of inter-pod affinity and anti-affinity
Similar to [node affinity](#node-affinity) are two types of Pod affinity and
anti-affinity as follows:
-->
#### Pod 间亲和性与反亲和性的类型
与[节点亲和性](#node-affinity)类似,Pod 的亲和性与反亲和性也有两种类型:
* `requiredDuringSchedulingIgnoredDuringExecution`
* `preferredDuringSchedulingIgnoredDuringExecution`
<!--
For example, you could use
`requiredDuringSchedulingIgnoredDuringExecution` affinity to tell the scheduler to
co-locate Pods of two services in the same cloud provider zone because they
communicate with each other a lot. Similarly, you could use
`preferredDuringSchedulingIgnoredDuringExecution` anti-affinity to spread Pods
from a service across multiple cloud provider zones.
-->
例如,你可以使用 `requiredDuringSchedulingIgnoredDuringExecution` 亲和性来告诉调度器,
将两个服务的 Pod 放到同一个云提供商可用区内,因为它们彼此之间通信非常频繁。
类似地,你可以使用 `preferredDuringSchedulingIgnoredDuringExecution`
反亲和性来将同一服务的多个 Pod 分布到多个云提供商可用区中。
<!--
To use inter-pod affinity, use the `affinity.podAffinity` field in the Pod spec.
For inter-pod anti-affinity, use the `affinity.podAntiAffinity` field in the Pod
spec.
-->
要使用 Pod 间亲和性,可以使用 Pod 规约中的 `.affinity.podAffinity` 字段。
对于 Pod 间反亲和性,可以使用 Pod 规约中的 `.affinity.podAntiAffinity` 字段。
<!--
#### Pod affinity example {#an-example-of-a-pod-that-uses-pod-affinity}
Consider the following Pod spec:
-->
#### Pod 亲和性示例 {#an-example-of-a-pod-that-uses-pod-affinity}
考虑下面的 Pod 规约:
{{< codenew file="pods/pod-with-pod-affinity.yaml" >}}
<!--
This example defines one Pod affinity rule and one Pod anti-affinity rule. The
Pod affinity rule uses the "hard"
`requiredDuringSchedulingIgnoredDuringExecution`, while the anti-affinity rule
uses the "soft" `preferredDuringSchedulingIgnoredDuringExecution`.
-->
本示例定义了一条 Pod 亲和性规则和一条 Pod 反亲和性规则。Pod 亲和性规则配置为
`requiredDuringSchedulingIgnoredDuringExecution`,而 Pod 反亲和性配置为
`preferredDuringSchedulingIgnoredDuringExecution`
<!--
The affinity rule says that the scheduler can only schedule a Pod onto a node if
the node is in the same zone as one or more existing Pods with the label
`security=S1`. More precisely, the scheduler must place the Pod on a node that has the
`topology.kubernetes.io/zone=V` label, as long as there is at least one node in
that zone that currently has one or more Pods with the Pod label `security=S1`.
-->
亲和性规则表示,仅当节点和至少一个已运行且有 `security=S1` 的标签的
Pod 处于同一区域时,才可以将该 Pod 调度到节点上。
更确切的说,调度器必须将 Pod 调度到具有 `topology.kubernetes.io/zone=V`
标签的节点上,并且集群中至少有一个位于该可用区的节点上运行着带有
`security=S1` 标签的 Pod。
<!--
The anti-affinity rule says that the scheduler should try to avoid scheduling
the Pod onto a node that is in the same zone as one or more Pods with the label
`security=S2`. More precisely, the scheduler should try to avoid placing the Pod on a node that has the
`topology.kubernetes.io/zone=R` label if there are other nodes in the
same zone currently running Pods with the `Security=S2` Pod label.
-->
反亲和性规则表示,如果节点处于 Pod 所在的同一可用区且至少一个 Pod 具有
`security=S2` 标签,则该 Pod 不应被调度到该节点上。
更确切地说, 如果同一可用区中存在其他运行着带有 `security=S2` 标签的 Pod 节点,
并且节点具有标签 `topology.kubernetes.io/zone=R`Pod 不能被调度到该节点上。
<!--
See the
[design doc](https://git.k8s.io/community/contributors/design-proposals/scheduling/podaffinity.md)
for many more examples of Pod affinity and anti-affinity.
-->
查阅[设计文档](https://git.k8s.io/community/contributors/design-proposals/scheduling/podaffinity.md)
以了解 Pod 亲和性与反亲和性的更多示例。
<!--
You can use the `In`, `NotIn`, `Exists` and `DoesNotExist` values in the
`operator` field for Pod affinity and anti-affinity.
In principle, the `topologyKey` can be any allowed label key with the following
exceptions for performance and security reasons:
-->
你可以针对 Pod 间亲和性与反亲和性为其 `operator` 字段使用 `In``NotIn``Exists`
`DoesNotExist` 等值。
原则上,`topologyKey` 可以是任何合法的标签键。出于性能和安全原因,`topologyKey`
有一些限制:
<!--
* For Pod affinity and anti-affinity, an empty `topologyKey` field is not allowed in both
`requiredDuringSchedulingIgnoredDuringExecution`
and `preferredDuringSchedulingIgnoredDuringExecution`.
* For `requiredDuringSchedulingIgnoredDuringExecution` Pod anti-affinity rules,
the admission controller `LimitPodHardAntiAffinityTopology` limits
`topologyKey` to `kubernetes.io/hostname`. You can modify or disable the
admission controller if you want to allow custom topologies.
-->
* 对于 Pod 亲和性而言,在 `requiredDuringSchedulingIgnoredDuringExecution`
`preferredDuringSchedulingIgnoredDuringExecution` 中,`topologyKey`
不允许为空。
* 对于 `requiredDuringSchedulingIgnoredDuringExecution` 要求的 Pod 反亲和性,
准入控制器 `LimitPodHardAntiAffinityTopology` 要求 `topologyKey` 只能是
`kubernetes.io/hostname`。如果你希望使用其他定制拓扑逻辑,
你可以更改准入控制器或者禁用之。
<!--
In addition to `labelSelector` and `topologyKey`, you can optionally specify a list
of namespaces which the `labelSelector` should match against using the
`namespaces` field at the same level as `labelSelector` and `topologyKey`.
If omitted or empty, `namespaces` defaults to the namespace of the Pod where the
affinity/anti-affinity definition appears.
-->
除了 `labelSelector``topologyKey`,你也可以指定 `labelSelector`
要匹配的命名空间列表,方法是在 `labelSelector``topologyKey`
所在层同一层次上设置 `namespaces`
如果 `namespaces` 被忽略或者为空,则默认为 Pod 亲和性/反亲和性的定义所在的命名空间。
<!--
#### Namespace selector
-->
#### 名字空间选择算符 {#namespace-selector}
{{< feature-state for_k8s_version="v1.24" state="stable" >}}
<!--
You can also select matching namespaces using `namespaceSelector`, which is a label query over the set of namespaces.
The affinity term is applied to namespaces selected by both `namespaceSelector` and the `namespaces` field.
Note that an empty `namespaceSelector` ({}) matches all namespaces, while a null or empty `namespaces` list and
null `namespaceSelector` matches the namespace of the Pod where the rule is defined.
-->
用户也可以使用 `namespaceSelector` 选择匹配的名字空间,`namespaceSelector`
是对名字空间集合进行标签查询的机制。
亲和性条件会应用到 `namespaceSelector` 所选择的名字空间和 `namespaces` 字段中
所列举的名字空间之上。
注意,空的 `namespaceSelector``{}`)会匹配所有名字空间,而 null 或者空的
`namespaces` 列表以及 null 值 `namespaceSelector` 意味着“当前 Pod 的名字空间”。
<!--
#### More practical use-cases
Inter-pod affinity and anti-affinity can be even more useful when they are used with higher
level collections such as ReplicaSets, StatefulSets, Deployments, etc. These
rules allow you to configure that a set of workloads should
be co-located in the same defined topology, eg., the same node.
-->
#### 更实际的用例
Pod 间亲和性与反亲和性在与更高级别的集合(例如 ReplicaSet、StatefulSet、
Deployment 等)一起使用时,它们可能更加有用。
这些规则使得你可以配置一组工作负载,使其位于相同定义拓扑(例如,节点)中。
<!--
In the following example Deployment for the redis cache, the replicas get the label `app=store`. The
`podAntiAffinity` rule tells the scheduler to avoid placing multiple replicas
with the `app=store` label on a single node. This creates each cache in a
separate node.
-->
在下面的 Redis 缓存 Deployment 示例中,副本上设置了标签 `app=store`
`podAntiAffinity` 规则告诉调度器避免将多个带有 `app=store` 标签的副本部署到同一节点上。
因此,每个独立节点上会创建一个缓存实例。
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: redis-cache
spec:
selector:
matchLabels:
app: store
replicas: 3
template:
metadata:
labels:
app: store
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- store
topologyKey: "kubernetes.io/hostname"
containers:
- name: redis-server
image: redis:3.2-alpine
```
<!--
The following Deployment for the web servers creates replicas with the label `app=web-store`. The
Pod affinity rule tells the scheduler to place each replica on a node that has a
Pod with the label `app=store`. The Pod anti-affinity rule tells the scheduler
to avoid placing multiple `app=web-store` servers on a single node.
-->
下面的 Deployment 用来提供 Web 服务器服务,会创建带有标签 `app=web-store` 的副本。
Pod 亲和性规则告诉调度器将副本放到运行有标签包含 `app=store` Pod 的节点上。
Pod 反亲和性规则告诉调度器不要在同一节点上放置多个 `app=web-store` 的服务器。
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: web-server
spec:
selector:
matchLabels:
app: web-store
replicas: 3
template:
metadata:
labels:
app: web-store
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- web-store
topologyKey: "kubernetes.io/hostname"
podAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- store
topologyKey: "kubernetes.io/hostname"
containers:
- name: web-app
image: nginx:1.16-alpine
```
<!--
Creating the two preceding Deployments results in the following cluster layout,
where each web server is co-located with a cache, on three separate nodes.
-->
创建前面两个 Deployment 会产生如下的集群布局,每个 Web 服务器与一个缓存实例并置,
并分别运行在三个独立的节点上。
| node-1 | node-2 | node-3 |
|:--------------------:|:-------------------:|:------------------:|
| *webserver-1* | *webserver-2* | *webserver-3* |
| *cache-1* | *cache-2* | *cache-3* |
<!--
See the [ZooKeeper tutorial](/docs/tutorials/stateful-application/zookeeper/#tolerating-node-failure)
for an example of a StatefulSet configured with anti-affinity for high
availability, using the same technique as this example.
-->
参阅 [ZooKeeper 教程](/zh/docs/tutorials/stateful-application/zookeeper/#tolerating-node-failure)
了解一个 StatefulSet 的示例,该 StatefulSet 配置了反亲和性以实现高可用,
所使用的是与此例相同的技术。
<!--
## nodeName
`nodeName` is a more direct form of node selection than affinity or
`nodeSelector`. `nodeName` is a field in the Pod spec. If the `nodeName` field
is not empty, the scheduler ignores the Pod and the kubelet on the named node
tries to place the Pod on that node. Using `nodeName` overrules using
`nodeSelector` or affinity and anti-affinity rules.
-->
## nodeName
`nodeName` 是比亲和性或者 `nodeSelector` 更为直接的形式。`nodeName` 是 Pod
规约中的一个字段。如果 `nodeName` 字段不为空,调度器会忽略该 Pod,
而指定节点上的 kubelet 会尝试将 Pod 放到该节点上。
使用 `nodeName` 规则的优先级会高于使用 `nodeSelector` 或亲和性与非亲和性的规则。
<!--
Some of the limitations of using `nodeName` to select nodes are:
- If the named node does not exist, the Pod will not run, and in
some cases may be automatically deleted.
- If the named node does not have the resources to accommodate the
Pod, the Pod will fail and its reason will indicate why,
for example OutOfmemory or OutOfcpu.
- Node names in cloud environments are not always predictable or stable.
-->
使用 `nodeName` 来选择节点的方式有一些局限性:
- 如果所指代的节点不存在,则 Pod 无法运行,而且在某些情况下可能会被自动删除。
- 如果所指代的节点无法提供用来运行 Pod 所需的资源,Pod 会失败,
而其失败原因中会给出是否因为内存或 CPU 不足而造成无法运行。
- 在云环境中的节点名称并不总是可预测的,也不总是稳定的。
<!--
Here is an example of a Pod spec using the `nodeName` field:
-->
下面是一个使用 `nodeName` 字段的 Pod 规约示例:
```yaml
apiVersion: v1
kind: Pod
metadata:
name: nginx
spec:
containers:
- name: nginx
image: nginx
nodeName: kube-01
```
<!--
The above Pod will only run on the node `kube-01`.
-->
上面的 Pod 只能运行在节点 `kube-01` 之上。
## {{% heading "whatsnext" %}}
<!--
* Read more about [taints and tolerations](/docs/concepts/scheduling-eviction/taint-and-toleration/) .
* Read the design docs for [node affinity](https://git.k8s.io/community/contributors/design-proposals/scheduling/nodeaffinity.md)
and for [inter-pod affinity/anti-affinity](https://git.k8s.io/community/contributors/design-proposals/scheduling/podaffinity.md).
* Learn about how the [topology manager](/docs/tasks/administer-cluster/topology-manager/) takes part in node-level
resource allocation decisions.
* Learn how to use [nodeSelector](/docs/tasks/configure-pod-container/assign-pods-nodes/).
* Learn how to use [affinity and anti-affinity](/docs/tasks/configure-pod-container/assign-pods-nodes-using-node-affinity/).
-->
* 进一步阅读[污点与容忍度](/zh/docs/concepts/scheduling-eviction/taint-and-toleration/)文档。
* 阅读[节点亲和性](https://git.k8s.io/community/contributors/design-proposals/scheduling/nodeaffinity.md)
和[Pod 间亲和性与反亲和性](https://git.k8s.io/community/contributors/design-proposals/scheduling/podaffinity.md)
的设计文档。
* 了解[拓扑管理器](/zh/docs/tasks/administer-cluster/topology-manager/)如何参与节点层面资源分配决定。
* 了解如何使用 [nodeSelector](/zh/docs/tasks/configure-pod-container/assign-pods-nodes/)。
* 了解如何使用[亲和性和反亲和性](/zh/docs/tasks/configure-pod-container/assign-pods-nodes-using-node-affinity/)。
@@ -0,0 +1,187 @@
---
title: Kubernetes 调度器
content_type: concept
weight: 10
---
<!--
title: Kubernetes Scheduler
content_type: concept
weight: 10
-->
<!-- overview -->
<!--
In Kubernetes, _scheduling_ refers to making sure that {{< glossary_tooltip text="Pods" term_id="pod" >}}
are matched to {{< glossary_tooltip text="Nodes" term_id="node" >}} so that
{{< glossary_tooltip term_id="kubelet" >}} can run them.
-->
在 Kubernetes 中,_调度_ 是指将 {{< glossary_tooltip text="Pod" term_id="pod" >}} 放置到合适的
{{< glossary_tooltip text="Node" term_id="node" >}} 上,然后对应 Node 上的
{{< glossary_tooltip term_id="kubelet" >}} 才能够运行这些 pod。
<!-- body -->
<!--
## Scheduling overview {#scheduling}
-->
## 调度概览 {#scheduling}
<!--
A scheduler watches for newly created Pods that have no Node assigned. For
every Pod that the scheduler discovers, the scheduler becomes responsible
for finding the best Node for that Pod to run on. The scheduler reaches
this placement decision taking into account the scheduling principles
described below.
-->
调度器通过 kubernetes 的监测(Watch)机制来发现集群中新创建且尚未被调度到 Node 上的 Pod。
调度器会将发现的每一个未调度的 Pod 调度到一个合适的 Node 上来运行。
调度器会依据下文的调度原则来做出调度选择。
<!--
If you want to understand why Pods are placed onto a particular Node,
or if you're planning to implement a custom scheduler yourself, this
page will help you learn about scheduling.
-->
如果你想要理解 Pod 为什么会被调度到特定的 Node 上,或者你想要尝试实现
一个自定义的调度器,这篇文章将帮助你了解调度。
<!--
## kube-scheduler
-->
## kube-scheduler
<!--
[kube-scheduler](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-scheduler/)
is the default scheduler for Kubernetes and runs as part of the
{{< glossary_tooltip text="control plane" term_id="control-plane" >}}.
kube-scheduler is designed so that, if you want and need to, you can
write your own scheduling component and use that instead.
-->
[kube-scheduler](/zh/docs/reference/command-line-tools-reference/kube-scheduler/)
是 Kubernetes 集群的默认调度器,并且是集群
{{< glossary_tooltip text="控制面" term_id="control-plane" >}} 的一部分。
如果你真的希望或者有这方面的需求,kube-scheduler 在设计上是允许
你自己写一个调度组件并替换原有的 kube-scheduler。
<!--
For every newly created pods or other unscheduled pods, kube-scheduler
selects a optimal node for them to run on. However, every container in
pods has different requirements for resources and every pod also has
different requirements. Therefore, existing nodes need to be filtered
according to the specific scheduling requirements.
-->
对每一个新创建的 Pod 或者是未被调度的 Podkube-scheduler 会选择一个最优的
Node 去运行这个 Pod。然而,Pod 内的每一个容器对资源都有不同的需求,而且
Pod 本身也有不同的资源需求。因此,Pod 在被调度到 Node 上之前,
根据这些特定的资源调度需求,需要对集群中的 Node 进行一次过滤。
<!--
In a cluster, Nodes that meet the scheduling requirements for a Pod
are called _feasible_ nodes. If none of the nodes are suitable, the pod
remains unscheduled until the scheduler is able to place it.
-->
在一个集群中,满足一个 Pod 调度请求的所有 Node 称之为 _可调度节点_
如果没有任何一个 Node 能满足 Pod 的资源请求,那么这个 Pod 将一直停留在
未调度状态直到调度器能够找到合适的 Node。
<!--
The scheduler finds feasible Nodes for a Pod and then runs a set of
functions to score the feasible Nodes and picks a Node with the highest
score among the feasible ones to run the Pod. The scheduler then notifies
the API server about this decision in a process called _binding_.
-->
调度器先在集群中找到一个 Pod 的所有可调度节点,然后根据一系列函数对这些可调度节点打分,
选出其中得分最高的 Node 来运行 Pod。之后,调度器将这个调度决定通知给
kube-apiserver,这个过程叫做 _绑定_
<!--
Factors that need to be taken into account for scheduling decisions include
individual and collective resource requirements, hardware / software /
policy constraints, affinity and anti-affinity specifications, data
locality, inter-workload interference, and so on.
-->
在做调度决定时需要考虑的因素包括:单独和整体的资源请求、硬件/软件/策略限制、
亲和以及反亲和要求、数据局域性、负载间的干扰等等。
<!--
## Scheduling with kube-scheduler {#kube-scheduler-implementation}
-->
## kube-scheduler 调度流程 {#kube-scheduler-implementation}
<!--
kube-scheduler selects a node for the pod in a 2-step operation:
1. Filtering
2. Scoring
-->
kube-scheduler 给一个 pod 做调度选择包含两个步骤:
1. 过滤
2. 打分
<!--
The _filtering_ step finds the set of Nodes where it's feasible to
schedule the Pod. For example, the PodFitsResources filter checks whether a
candidate Node has enough available resource to meet a Pod's specific
resource requests. After this step, the node list contains any suitable
Nodes; often, there will be more than one. If the list is empty, that
Pod isn't (yet) schedulable.
-->
过滤阶段会将所有满足 Pod 调度需求的 Node 选出来。
例如,PodFitsResources 过滤函数会检查候选 Node 的可用资源能否满足 Pod 的资源请求。
在过滤之后,得出一个 Node 列表,里面包含了所有可调度节点;通常情况下,
这个 Node 列表包含不止一个 Node。如果这个列表是空的,代表这个 Pod 不可调度。
<!--
In the _scoring_ step, the scheduler ranks the remaining nodes to choose
the most suitable Pod placement. The scheduler assigns a score to each Node
that survived filtering, basing this score on the active scoring rules.
-->
在打分阶段,调度器会为 Pod 从所有可调度节点中选取一个最合适的 Node。
根据当前启用的打分规则,调度器会给每一个可调度节点进行打分。
<!--
Finally, kube-scheduler assigns the Pod to the Node with the highest ranking.
If there is more than one node with equal scores, kube-scheduler selects
one of these at random.
-->
最后,kube-scheduler 会将 Pod 调度到得分最高的 Node 上。
如果存在多个得分最高的 Nodekube-scheduler 会从中随机选取一个。
<!--
There are two supported ways to configure the filtering and scoring behavior
of the scheduler:
-->
支持以下两种方式配置调度器的过滤和打分行为:
<!--
1. [Scheduling Policies](/docs/reference/scheduling/policies) allow you to
configure _Predicates_ for filtering and _Priorities_ for scoring.
1. [Scheduling Profiles](/docs/reference/scheduling/config/#profiles) allow you to
configure Plugins that implement different scheduling stages, including:
`QueueSort`, `Filter`, `Score`, `Bind`, `Reserve`, `Permit`, and others. You
can also configure the kube-scheduler to run different profiles.
-->
1. [调度策略](/zh/docs/reference/scheduling/policies) 允许你配置过滤的 _断言(Predicates)_
和打分的 _优先级(Priorities)_
2. [调度配置](/zh/docs/reference/scheduling/config/#profiles) 允许你配置实现不同调度阶段的插件,
包括:`QueueSort`, `Filter`, `Score`, `Bind`, `Reserve`, `Permit` 等等。
你也可以配置 kube-scheduler 运行不同的配置文件。
## {{% heading "whatsnext" %}}
<!--
* Read about [scheduler performance tuning](/docs/concepts/scheduling-eviction/scheduler-perf-tuning/)
* Read about [Pod topology spread constraints](/docs/concepts/workloads/pods/pod-topology-spread-constraints/)
* Read the [reference documentation](/docs/reference/command-line-tools-reference/kube-scheduler/) for kube-scheduler
* Read the [kube-scheduler config (v1beta3)](/docs/reference/config-api/kube-scheduler-config.v1beta3/) reference
* Learn about [configuring multiple schedulers](/docs/tasks/extend-kubernetes/configure-multiple-schedulers/)
* Learn about [topology management policies](/docs/tasks/administer-cluster/topology-manager/)
* Learn about [Pod Overhead](/docs/concepts/scheduling-eviction/pod-overhead/)
-->
* 阅读关于 [调度器性能调优](/zh/docs/concepts/scheduling-eviction/scheduler-perf-tuning/)
* 阅读关于 [Pod 拓扑分布约束](/zh/docs/concepts/workloads/pods/pod-topology-spread-constraints/)
* 阅读关于 kube-scheduler 的 [参考文档](/zh/docs/reference/command-line-tools-reference/kube-scheduler/)
* 阅读 [kube-scheduler 配置参考 (v1beta3)](/zh/docs/reference/config-api/kube-scheduler-config.v1beta3/)
* 了解关于 [配置多个调度器](/zh/docs/tasks/extend-kubernetes/configure-multiple-schedulers/) 的方式
* 了解关于 [拓扑结构管理策略](/zh/docs/tasks/administer-cluster/topology-manager/)
* 了解关于 [Pod 额外开销](/zh/docs/concepts/scheduling-eviction/pod-overhead/)
@@ -0,0 +1,778 @@
---
title: 节点压力驱逐
content_type: concept
weight: 60
---
<!--
title: Node-pressure Eviction
content_type: concept
weight: 60
-->
{{<glossary_definition term_id="node-pressure-eviction" length="short">}}</br>
<!--
The {{<glossary_tooltip term_id="kubelet" text="kubelet">}} monitors resources
like CPU, memory, disk space, and filesystem inodes on your cluster's nodes.
When one or more of these resources reach specific consumption levels, the
kubelet can proactively fail one or more pods on the node to reclaim resources
and prevent starvation.
During a node-pressure eviction, the kubelet sets the `PodPhase` for the
selected pods to `Failed`. This terminates the pods.
Node-pressure eviction is not the same as
[API-initiated eviction](/docs/reference/generated/kubernetes-api/v1.23/).
-->
{{<glossary_tooltip term_id="kubelet" text="kubelet">}}
监控集群节点的 CPU、内存、磁盘空间和文件系统的 inode 等资源。
当这些资源中的一个或者多个达到特定的消耗水平,
kubelet 可以主动地使节点上一个或者多个 Pod 失效,以回收资源防止饥饿。
在节点压力驱逐期间,kubelet 将所选 Pod 的 `PodPhase` 设置为 `Failed`。这将终止 Pod。
节点压力驱逐不同于 [API 发起的驱逐](/docs/reference/generated/kubernetes-api/v1.23/)。
<!--
The kubelet does not respect your configured `PodDisruptionBudget` or the pod's
`terminationGracePeriodSeconds`. If you use [soft eviction thresholds](#soft-eviction-thresholds),
the kubelet respects your configured `eviction-max-pod-grace-period`. If you use
[hard eviction thresholds](#hard-eviction-thresholds), it uses a `0s` grace period for termination.
If the pods are managed by a {{< glossary_tooltip text="workload" term_id="workload" >}}
resource (such as {{< glossary_tooltip text="StatefulSet" term_id="statefulset" >}}
or {{< glossary_tooltip text="Deployment" term_id="deployment" >}}) that
replaces failed pods, the control plane or `kube-controller-manager` creates new
pods in place of the evicted pods.
-->
kubelet 并不理会你配置的 `PodDisruptionBudget` 或者是 Pod 的 `terminationGracePeriodSeconds`
如果你使用了[软驱逐条件](#soft-eviction-thresholds)kubelet 会考虑你所配置的
`eviction-max-pod-grace-period`
如果你使用了[硬驱逐条件](#hard-eviction-thresholds),它使用 `0s` 宽限期来终止 Pod。
如果 Pod 是由替换失败 Pod 的{{< glossary_tooltip text="工作负载" term_id="workload" >}}资源
(例如 {{< glossary_tooltip text="StatefulSet" term_id="statefulset" >}}
或者 {{< glossary_tooltip text="Deployment" term_id="deployment" >}})管理,
则控制平面或 `kube-controller-manager` 会创建新的 Pod 来代替被驱逐的 Pod。
{{<note>}}
<!--
The kubelet attempts to [reclaim node-level resources](#reclaim-node-resources)
before it terminates end-user pods. For example, it removes unused container
images when disk resources are starved.
-->
kubelet 在终止最终用户 Pod 之前会尝试[回收节点级资源](#reclaim-node-resources)。
例如,它会在磁盘资源不足时删除未使用的容器镜像。
{{</note>}}
<!--
The kubelet uses various parameters to make eviction decisions, like the following:
* Eviction signals
* Eviction thresholds
* Monitoring intervals
-->
kubelet 使用各种参数来做出驱逐决定,如下所示:
* 驱逐信号
* 驱逐条件
* 监控间隔
<!--
### Eviction signals {#eviction-signals}
Eviction signals are the current state of a particular resource at a specific
point in time. Kubelet uses eviction signals to make eviction decisions by
comparing the signals to eviction thresholds, which are the minimum amount of
the resource that should be available on the node.
Kubelet uses the following eviction signals:
-->
### 驱逐信号 {#eviction-signals}
驱逐信号是特定资源在特定时间点的当前状态。
kubelet 使用驱逐信号,通过将信号与驱逐条件进行比较来做出驱逐决定,
驱逐条件是节点上应该可用资源的最小量。
kubelet 使用以下驱逐信号:
| 驱逐信号 | 描述 |
|----------------------|---------------------------------------------------------------------------------------|
| `memory.available` | `memory.available` := `node.status.capacity[memory]` - `node.stats.memory.workingSet` |
| `nodefs.available` | `nodefs.available` := `node.stats.fs.available` |
| `nodefs.inodesFree` | `nodefs.inodesFree` := `node.stats.fs.inodesFree` |
| `imagefs.available` | `imagefs.available` := `node.stats.runtime.imagefs.available` |
| `imagefs.inodesFree` | `imagefs.inodesFree` := `node.stats.runtime.imagefs.inodesFree` |
| `pid.available` | `pid.available` := `node.stats.rlimit.maxpid` - `node.stats.rlimit.curproc` |
<!--
In this table, the `Description` column shows how kubelet gets the value of the
signal. Each signal supports either a percentage or a literal value. Kubelet
calculates the percentage value relative to the total capacity associated with
the signal.
-->
在上表中,`描述`列显示了 kubelet 如何获取信号的值。每个信号支持百分比值或者是字面值。
kubelet 计算相对于与信号有关的总量的百分比值。
<!--
The value for `memory.available` is derived from the cgroupfs instead of tools
like `free -m`. This is important because `free -m` does not work in a
container, and if users use the [node
allocatable](/docs/tasks/administer-cluster/reserve-compute-resources/#node-allocatable) feature, out of resource decisions
are made local to the end user Pod part of the cgroup hierarchy as well as the
root node. This [script](/examples/admin/resource/memory-available.sh)
reproduces the same set of steps that the kubelet performs to calculate
`memory.available`. The kubelet excludes inactive_file (i.e. # of bytes of
file-backed memory on inactive LRU list) from its calculation as it assumes that
memory is reclaimable under pressure.
-->
`memory.available` 的值来自 cgroupfs,而不是像 `free -m` 这样的工具。
这很重要,因为 `free -m` 在容器中不起作用,如果用户使用
[节点可分配资源](/zh/docs/tasks/administer-cluster/reserve-compute-resources/#node-allocatable)
这一功能特性,资源不足的判定是基于 CGroup 层次结构中的用户 Pod 所处的局部及 CGroup 根节点作出的。
这个[脚本](/zh/examples/admin/resource/memory-available.sh)
重现了 kubelet 为计算 `memory.available` 而执行的相同步骤。
kubelet 在其计算中排除了 inactive_file(即非活动 LRU 列表上基于文件来虚拟的内存的字节数),
因为它假定在压力下内存是可回收的。
<!--
The kubelet supports the following filesystem partitions:
1. `nodefs`: The node's main filesystem, used for local disk volumes, emptyDir,
log storage, and more. For example, `nodefs` contains `/var/lib/kubelet/`.
1. `imagefs`: An optional filesystem that container runtimes use to store container
images and container writable layers.
Kubelet auto-discovers these filesystems and ignores other filesystems. Kubelet
does not support other configurations.
-->
kubelet 支持以下文件系统分区:
1. `nodefs`:节点的主要文件系统,用于本地磁盘卷、emptyDir、日志存储等。
例如,`nodefs` 包含 `/var/lib/kubelet/`
1. `imagefs`:可选文件系统,供容器运行时存储容器镜像和容器可写层。
kubelet 会自动发现这些文件系统并忽略其他文件系统。kubelet 不支持其他配置。
{{<note>}}
<!--
Some kubelet garbage collection features are deprecated in favor of eviction.
For a list of the deprecated features, see [kubelet garbage collection deprecation](/docs/concepts/cluster-administration/kubelet-garbage-collection/#deprecation).
-->
一些 kubelet 垃圾收集功能已被弃用,以支持驱逐。
有关已弃用功能的列表,请参阅
[kubelet 垃圾收集弃用](/zh/docs/concepts/cluster-administration/kubelet-garbage-collection/#deprecation)。
{{</note>}}
<!--
### Eviction thresholds
You can specify custom eviction thresholds for the kubelet to use when it makes
eviction decisions.
Eviction thresholds have the form `[eviction-signal][operator][quantity]`, where:
* `eviction-signal` is the [eviction signal](#eviction-signals) to use.
* `operator` is the [relational operator](https://en.wikipedia.org/wiki/Relational_operator#Standard_relational_operators)
you want, such as `<` (less than).
* `quantity` is the eviction threshold amount, such as `1Gi`. The value of `quantity`
must match the quantity representation used by Kubernetes. You can use either
literal values or percentages (`%`).
-->
### 驱逐条件 {#eviction-thresholds}
你可以为 kubelet 指定自定义驱逐条件,以便在作出驱逐决定时使用。
驱逐条件的形式为 `[eviction-signal][operator][quantity]`,其中:
* `eviction-signal` 是要使用的[驱逐信号](#eviction-signals)。
* `operator` 是你想要的[关系运算符](https://en.wikipedia.org/wiki/Relational_operator#Standard_relational_operators)
比如 `<`(小于)。
* `quantity` 是驱逐条件数量,例如 `1Gi`
`quantity` 的值必须与 Kubernetes 使用的数量表示相匹配。
你可以使用文字值或百分比(`%`)。
<!--
For example, if a node has `10Gi` of total memory and you want trigger eviction if
the available memory falls below `1Gi`, you can define the eviction threshold as
either `memory.available<10%` or `memory.available<1Gi`. You cannot use both.
You can configure soft and hard eviction thresholds.
-->
例如,如果一个节点的总内存为 10Gi 并且你希望在可用内存低于 1Gi 时触发驱逐,
则可以将驱逐条件定义为 `memory.available<10%``memory.available< 1G`
你不能同时使用二者。
你可以配置软和硬驱逐条件。
<!--
#### Soft eviction thresholds {#soft-eviction-thresholds}
A soft eviction threshold pairs an eviction threshold with a required
administrator-specified grace period. The kubelet does not evict pods until the
grace period is exceeded. The kubelet returns an error on startup if there is no
specified grace period.
-->
#### 软驱逐条件 {#soft-eviction-thresholds}
软驱逐条件将驱逐条件与管理员所必须指定的宽限期配对。
在超过宽限期之前,kubelet 不会驱逐 Pod。
如果没有指定的宽限期,kubelet 会在启动时返回错误。
<!--
You can specify both a soft eviction threshold grace period and a maximum
allowed pod termination grace period for kubelet to use during evictions. If you
specify a maximum allowed grace period and the soft eviction threshold is met,
the kubelet uses the lesser of the two grace periods. If you do not specify a
maximum allowed grace period, the kubelet kills evicted pods immediately without
graceful termination.
-->
你可以既指定软驱逐条件宽限期,又指定 Pod 终止宽限期的上限,,给 kubelet 在驱逐期间使用。
如果你指定了宽限期的上限并且 Pod 满足软驱逐阈条件,则 kubelet 将使用两个宽限期中的较小者。
如果你没有指定宽限期上限,kubelet 会立即杀死被驱逐的 Pod,不允许其体面终止。
<!--
You can use the following flags to configure soft eviction thresholds:
* `eviction-soft`: A set of eviction thresholds like `memory.available<1.5Gi`
that can trigger pod eviction if held over the specified grace period.
* `eviction-soft-grace-period`: A set of eviction grace periods like `memory.available=1m30s`
that define how long a soft eviction threshold must hold before triggering a Pod eviction.
* `eviction-max-pod-grace-period`: The maximum allowed grace period (in seconds)
to use when terminating pods in response to a soft eviction threshold being met.
-->
你可以使用以下标志来配置软驱逐条件:
* `eviction-soft`:一组驱逐条件,如 `memory.available<1.5Gi`
如果驱逐条件持续时长超过指定的宽限期,可以触发 Pod 驱逐。
* `eviction-soft-grace-period`:一组驱逐宽限期,
`memory.available=1m30s`,定义软驱逐条件在触发 Pod 驱逐之前必须保持多长时间。
* `eviction-max-pod-grace-period`:在满足软驱逐条件而终止 Pod 时使用的最大允许宽限期(以秒为单位)。
<!--
#### Hard eviction thresholds {#hard-eviction-thresholds}
A hard eviction threshold has no grace period. When a hard eviction threshold is
met, the kubelet kills pods immediately without graceful termination to reclaim
the starved resource.
You can use the `eviction-hard` flag to configure a set of hard eviction
thresholds like `memory.available<1Gi`.
-->
#### 硬驱逐条件 {#hard-eviction-thresholds}
硬驱逐条件没有宽限期。当达到硬驱逐条件时,
kubelet 会立即杀死 pod,而不会正常终止以回收紧缺的资源。
你可以使用 `eviction-hard` 标志来配置一组硬驱逐条件,
例如 `memory.available<1Gi`
<!--
The kubelet has the following default hard eviction thresholds:
* `memory.available<100Mi`
* `nodefs.available<10%`
* `imagefs.available<15%`
* `nodefs.inodesFree<5%` (Linux nodes)
-->
kubelet 具有以下默认硬驱逐条件:
* `memory.available<100Mi`
* `nodefs.available<10%`
* `imagefs.available<15%`
* `nodefs.inodesFree<5%`Linux 节点)
<!--
### Eviction monitoring interval
The kubelet evaluates eviction thresholds based on its configured `housekeeping-interval`
which defaults to `10s`.
-->
### 驱逐监测间隔
kubelet 根据其配置的 `housekeeping-interval`(默认为 `10s`)评估驱逐条件。
<!--
### Node conditions {#node-conditions}
The kubelet reports node conditions to reflect that the node is under pressure
because hard or soft eviction threshold is met, independent of configured grace
periods.
-->
### 节点条件 {#node-conditions}
kubelet 报告节点状况以反映节点处于压力之下,因为满足硬或软驱逐条件,与配置的宽限期无关。
<!--
The kubelet maps eviction signals to node conditions as follows:
| Node Condition | Eviction Signal | Description |
|-------------------|---------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------|
| `MemoryPressure` | `memory.available` | Available memory on the node has satisfied an eviction threshold |
| `DiskPressure` | `nodefs.available`, `nodefs.inodesFree`, `imagefs.available`, or `imagefs.inodesFree` | Available disk space and inodes on either the node's root filesystem or image filesystem has satisfied an eviction threshold |
| `PIDPressure` | `pid.available` | Available processes identifiers on the (Linux) node has fallen below an eviction threshold |
The kubelet updates the node conditions based on the configured
`--node-status-update-frequency`, which defaults to `10s`.
-->
kubelet 根据下表将驱逐信号映射为节点状况:
| 节点条件 | 驱逐信号 | 描述 |
|---------|--------|------|
| `MemoryPressure` | `memory.available` | 节点上的可用内存已满足驱逐条件 |
| `DiskPressure` | `nodefs.available``nodefs.inodesFree``imagefs.available``imagefs.inodesFree` | 节点的根文件系统或映像文件系统上的可用磁盘空间和 inode 已满足驱逐条件 |
| `PIDPressure` | `pid.available` | (Linux) 节点上的可用进程标识符已低于驱逐条件 |
kubelet 根据配置的 `--node-status-update-frequency` 更新节点条件,默认为 `10s`
<!--
#### Node condition oscillation
In some cases, nodes oscillate above and below soft eviction thresholds without
holding for the defined grace periods. This causes the reported node condition
to constantly switch between `true` and `false`, leading to bad eviction decisions.
To protect against oscillation, you can use the `eviction-pressure-transition-period`
flag, which controls how long the kubelet must wait before transitioning a node
condition to a different state. The transition period has a default value of `5m`.
-->
#### 节点条件振荡
在某些情况下,节点在软驱逐条件上下振荡,而没有保持定义的宽限期。
这会导致报告的节点条件在 `true``false` 之间不断切换,从而导致错误的驱逐决策。
为了防止振荡,你可以使用 `eviction-pressure-transition-period` 标志,
该标志控制 kubelet 在将节点条件转换为不同状态之前必须等待的时间。
过渡期的默认值为 `5m`
<!--
### Reclaiming node level resources {#reclaim-node-resources}
The kubelet tries to reclaim node-level resources before it evicts end-user pods.
When a `DiskPressure` node condition is reported, the kubelet reclaims node-level
resources based on the filesystems on the node.
-->
### 回收节点级资源 {#reclaim-node-resources}
kubelet 在驱逐最终用户 Pod 之前会先尝试回收节点级资源。
当报告 `DiskPressure` 节点状况时,kubelet 会根据节点上的文件系统回收节点级资源。
<!--
#### With `imagefs`
If the node has a dedicated `imagefs` filesystem for container runtimes to use,
the kubelet does the following:
* If the `nodefs` filesystem meets the eviction thresholds, the kubelet garbage collects
dead pods and containers.
* If the `imagefs` filesystem meets the eviction thresholds, the kubelet
deletes all unused images.
-->
#### 有 `imagefs`
如果节点有一个专用的 `imagefs` 文件系统供容器运行时使用,kubelet 会执行以下操作:
* 如果 `nodefs` 文件系统满足驱逐条件,kubelet 垃圾收集死亡 Pod 和容器。
* 如果 `imagefs` 文件系统满足驱逐条件,kubelet 将删除所有未使用的镜像。
<!--
#### Without `imagefs`
If the node only has a `nodefs` filesystem that meets eviction thresholds,
the kubelet frees up disk space in the following order:
1. Garbage collect dead pods and containers
1. Delete unused images
-->
#### 没有 `imagefs`
如果节点只有一个满足驱逐条件的 `nodefs` 文件系统,
kubelet 按以下顺序释放磁盘空间:
1. 对死亡的 Pod 和容器进行垃圾收集
1. 删除未使用的镜像
<!--
### Pod selection for kubelet eviction
If the kubelet's attempts to reclaim node-level resources don't bring the eviction
signal below the threshold, the kubelet begins to evict end-user pods.
The kubelet uses the following parameters to determine pod eviction order:
1. Whether the pod's resource usage exceeds requests
1. [Pod Priority](/docs/concepts/scheduling-eviction/pod-priority-preemption/)
1. The pod's resource usage relative to requests
-->
### kubelet 驱逐时 Pod 的选择
如果 kubelet 回收节点级资源的尝试没有使驱逐信号低于条件,
则 kubelet 开始驱逐最终用户 Pod。
kubelet 使用以下参数来确定 Pod 驱逐顺序:
1. Pod 的资源使用是否超过其请求
1. [Pod 优先级](/zh/docs/concepts/scheduling-eviction/pod-priority-preemption/)
1. Pod 相对于请求的资源使用情况
<!--
As a result, kubelet ranks and evicts pods in the following order:
1. `BestEffort` or `Burstable` pods where the usage exceeds requests. These pods
are evicted based on their Priority and then by how much their usage level
exceeds the request.
1. `Guaranteed` pods and `Burstable` pods where the usage is less than requests
are evicted last, based on their Priority.
-->
因此,kubelet 按以下顺序排列和驱逐 Pod:
1. 首先考虑资源使用量超过其请求的 `BestEffort``Burstable` Pod。
这些 Pod 会根据它们的优先级以及它们的资源使用级别超过其请求的程度被逐出。
1. 资源使用量少于请求量的 `Guaranteed` Pod 和 `Burstable` Pod 根据其优先级被最后驱逐。
{{<note>}}
<!--
The kubelet does not use the pod's QoS class to determine the eviction order.
You can use the QoS class to estimate the most likely pod eviction order when
reclaiming resources like memory. QoS does not apply to EphemeralStorage requests,
so the above scenario will not apply if the node is, for example, under `DiskPressure`.
-->
kubelet 不使用 Pod 的 QoS 类来确定驱逐顺序。
在回收内存等资源时,你可以使用 QoS 类来估计最可能的 Pod 驱逐顺序。
QoS 不适用于临时存储(EphemeralStorage)请求,
因此如果节点在 `DiskPressure` 下,则上述场景将不适用。
{{</note>}}
<!--
`Guaranteed` pods are guaranteed only when requests and limits are specified for
all the containers and they are equal. These pods will never be evicted because
of another pod's resource consumption. If a system daemon (such as `kubelet`,
and `journald`) is consuming more resources than were reserved via
`system-reserved` or `kube-reserved` allocations, and the node only has
`Guaranteed` or `Burstable` pods using less resources than requests left on it,
then the kubelet must choose to evict one of these pods to preserve node stability
and to limit the impact of resource starvation on other pods. In this case, it
will choose to evict pods of lowest Priority first.
-->
仅当 `Guaranteed` Pod 中所有容器都被指定了请求和限制并且二者相等时,才保证 Pod 不被驱逐。
这些 Pod 永远不会因为另一个 Pod 的资源消耗而被驱逐。
如果系统守护进程(例如 `kubelet``journald`
消耗的资源比通过 `system-reserved``kube-reserved` 分配保留的资源多,
并且该节点只有 `Guaranteed``Burstable` Pod 使用的资源少于其上剩余的请求,
那么 kubelet 必须选择驱逐这些 Pod 中的一个以保持节点稳定性并减少资源匮乏对其他 Pod 的影响。
在这种情况下,它会选择首先驱逐最低优先级的 Pod。
<!--
When the kubelet evicts pods in response to `inode` or `PID` starvation, it uses
the Priority to determine the eviction order, because `inodes` and `PIDs` have no
requests.
The kubelet sorts pods differently based on whether the node has a dedicated
`imagefs` filesystem:
-->
当 kubelet 因 inode 或 PID 不足而驱逐 pod 时,
它使用优先级来确定驱逐顺序,因为 inode 和 PID 没有请求。
kubelet 根据节点是否具有专用的 `imagefs` 文件系统对 Pod 进行不同的排序:
<!--
#### With `imagefs`
If `nodefs` is triggering evictions, the kubelet sorts pods based on `nodefs`
usage (`local volumes + logs of all containers`).
If `imagefs` is triggering evictions, the kubelet sorts pods based on the
writable layer usage of all containers.
#### Without `imagefs`
If `nodefs` is triggering evictions, the kubelet sorts pods based on their total
disk usage (`local volumes + logs & writable layer of all containers`)
-->
#### 有 `imagefs`
如果 `nodefs` 触发驱逐,
kubelet 会根据 `nodefs` 使用情况(`本地卷 + 所有容器的日志`)对 Pod 进行排序。
如果 `imagefs` 触发驱逐,kubelet 会根据所有容器的可写层使用情况对 Pod 进行排序。
#### 没有 `imagefs`
如果 `nodefs` 触发驱逐,
kubelet 会根据磁盘总用量(`本地卷 + 日志和所有容器的可写层`)对 Pod 进行排序。
<!--
### Minimum eviction reclaim
In some cases, pod eviction only reclaims a small amount of the starved resource.
This can lead to the kubelet repeatedly hitting the configured eviction thresholds
and triggering multiple evictions.
-->
### 最小驱逐回收 {#minimum-eviction-reclaim}
在某些情况下,驱逐 Pod 只会回收少量的紧俏资源。
这可能导致 kubelet 反复达到配置的驱逐条件并触发多次驱逐。
<!--
You can use the `--eviction-minimum-reclaim` flag or a [kubelet config file](/docs/tasks/administer-cluster/kubelet-config-file/)
to configure a minimum reclaim amount for each resource. When the kubelet notices
that a resource is starved, it continues to reclaim that resource until it
reclaims the quantity you specify.
For example, the following configuration sets minimum reclaim amounts:
-->
你可以使用 `--eviction-minimum-reclaim` 标志或
[kubelet 配置文件](/zh/docs/tasks/administer-cluster/kubelet-config-file/)
为每个资源配置最小回收量。
当 kubelet 注意到某个资源耗尽时,它会继续回收该资源,直到回收到你所指定的数量为止。
例如,以下配置设置最小回收量:
```yaml
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
evictionHard:
memory.available: "500Mi"
nodefs.available: "1Gi"
imagefs.available: "100Gi"
evictionMinimumReclaim:
memory.available: "0Mi"
nodefs.available: "500Mi"
imagefs.available: "2Gi"
```
<!--
In this example, if the `nodefs.available` signal meets the eviction threshold,
the kubelet reclaims the resource until the signal reaches the threshold of `1Gi`,
and then continues to reclaim the minimum amount of `500Mi` it until the signal
reaches `1.5Gi`.
Similarly, the kubelet reclaims the `imagefs` resource until the `imagefs.available`
signal reaches `102Gi`.
The default `eviction-minimum-reclaim` is `0` for all resources.
-->
在这个例子中,如果 `nodefs.available` 信号满足驱逐条件,
kubelet 会回收资源,直到信号达到 `1Gi` 的条件,
然后继续回收至少 `500Mi` 直到信号达到 `1.5Gi`
类似地,kubelet 会回收 `imagefs` 资源,直到 `imagefs.available` 信号达到 `102Gi`
对于所有资源,默认的 `eviction-minimum-reclaim``0`
<!--
### Node out of memory behavior
If the node experiences an out of memory (OOM) event prior to the kubelet
being able to reclaim memory, the node depends on the [oom_killer](https://lwn.net/Articles/391222/)
to respond.
The kubelet sets an `oom_score_adj` value for each container based on the QoS for the pod.
-->
### 节点内存不足行为
如果节点在 kubelet 能够回收内存之前遇到内存不足(OOM)事件,
则节点依赖 [oom_killer](https://lwn.net/Articles/391222/) 来响应。
kubelet 根据 Pod 的服务质量(QoS)为每个容器设置一个 `oom_score_adj` 值。
| 服务质量 | oom_score_adj |
|--------------------|-----------------------------------------------------------------------------------|
| `Guaranteed` | -997 |
| `BestEffort` | 1000 |
| `Burstable` | min(max(2, 1000 - (1000 * memoryRequestBytes) / machineMemoryCapacityBytes), 999) |
{{<note>}}
<!--
The kubelet also sets an `oom_score_adj` value of `-997` for containers in Pods that have
`system-node-critical` {{<glossary_tooltip text="Priority" term_id="pod-priority">}}
-->
kubelet 还将具有 `system-node-critical`
{{<glossary_tooltip text="优先级" term_id="pod-priority">}}
的 Pod 中的容器 `oom_score_adj` 值设为 `-997`
{{</note>}}
<!--
If the kubelet can't reclaim memory before a node experiences OOM, the
`oom_killer` calculates an `oom_score` based on the percentage of memory it's
using on the node, and then adds the `oom_score_adj` to get an effective `oom_score`
for each container. It then kills the container with the highest score.
This means that containers in low QoS pods that consume a large amount of memory
relative to their scheduling requests are killed first.
Unlike pod eviction, if a container is OOM killed, the `kubelet` can restart it
based on its `RestartPolicy`.
-->
如果 kubelet 在节点遇到 OOM 之前无法回收内存,
`oom_killer` 根据它在节点上使用的内存百分比计算 `oom_score`
然后加上 `oom_score_adj` 得到每个容器有效的 `oom_score`
然后它会杀死得分最高的容器。
这意味着低 QoS Pod 中相对于其调度请求消耗内存较多的容器,将首先被杀死。
与 Pod 驱逐不同,如果容器被 OOM 杀死,
`kubelet` 可以根据其 `RestartPolicy` 重新启动它。
<!--
### Best practices {#node-pressure-eviction-good-practices}
The following sections describe best practices for eviction configuration.
-->
### 最佳实践 {#node-pressure-eviction-good-practices}
以下部分描述了驱逐配置的最佳实践。
<!--
#### Schedulable resources and eviction policies
When you configure the kubelet with an eviction policy, you should make sure that
the scheduler will not schedule pods if they will trigger eviction because they
immediately induce memory pressure.
-->
#### 可调度的资源和驱逐策略
当你为 kubelet 配置驱逐策略时,
你应该确保调度程序不会在 Pod 触发驱逐时对其进行调度,因为这类 Pod 会立即引起内存压力。
<!--
Consider the following scenario:
* Node memory capacity: `10Gi`
* Operator wants to reserve 10% of memory capacity for system daemons (kernel, `kubelet`, etc.)
* Operator wants to evict Pods at 95% memory utilization to reduce incidence of system OOM.
-->
考虑以下场景:
* 节点内存容量:`10Gi`
* 操作员希望为系统守护进程(内核、`kubelet` 等)保留 10% 的内存容量
* 操作员希望驱逐内存利用率为 95% 的Pod,以减少系统 OOM 的概率。
<!--
For this to work, the kubelet is launched as follows:
-->
为此,kubelet 启动设置如下:
```
--eviction-hard=memory.available<500Mi
--system-reserved=memory=1.5Gi
```
<!--
In this configuration, the `--system-reserved` flag reserves `1.5Gi` of memory
for the system, which is `10% of the total memory + the eviction threshold amount`.
The node can reach the eviction threshold if a pod is using more than its request,
or if the system is using more than `1Gi` of memory, which makes the `memory.available`
signal fall below `500Mi` and triggers the threshold.
-->
在此配置中,`--system-reserved` 标志为系统预留了 `1.5Gi` 的内存,
`总内存的 10% + 驱逐条件量`
如果 Pod 使用的内存超过其请求值或者系统使用的内存超过 `1Gi`
则节点可以达到驱逐条件,这使得 `memory.available` 信号低于 `500Mi` 并触发条件。
<!--
#### DaemonSet
Pod Priority is a major factor in making eviction decisions. If you do not want
the kubelet to evict pods that belong to a `DaemonSet`, give those pods a high
enough `priorityClass` in the pod spec. You can also use a lower `priorityClass`
or the default to only allow `DaemonSet` pods to run when there are enough
resources.
-->
### DaemonSet
Pod 优先级是做出驱逐决定的主要因素。
如果你不希望 kubelet 驱逐属于 `DaemonSet` 的 Pod
请在 Pod 规约中为这些 Pod 提供足够高的 `priorityClass`
你还可以使用优先级较低的 `priorityClass` 或默认配置,
仅在有足够资源时才运行 `DaemonSet` Pod。
<!--
### Known issues
The following sections describe known issues related to out of resource handling.
-->
### 已知问题
以下部分描述了与资源不足处理相关的已知问题。
<!--
#### kubelet may not observe memory pressure right away
By default, the kubelet polls `cAdvisor` to collect memory usage stats at a
regular interval. If memory usage increases within that window rapidly, the
kubelet may not observe `MemoryPressure` fast enough, and the `OOMKiller`
will still be invoked.
-->
#### kubelet 可能不会立即观察到内存压力
默认情况下,kubelet 轮询 `cAdvisor` 以定期收集内存使用情况统计信息。
如果该轮询时间窗口内内存使用量迅速增加,kubelet 可能无法足够快地观察到 `MemoryPressure`
但是 `OOMKiller` 仍将被调用。
<!--
You can use the `--kernel-memcg-notification` flag to enable the `memcg`
notification API on the kubelet to get notified immediately when a threshold
is crossed.
If you are not trying to achieve extreme utilization, but a sensible measure of
overcommit, a viable workaround for this issue is to use the `--kube-reserved`
and `--system-reserved` flags to allocate memory for the system.
-->
你可以使用 `--kernel-memcg-notification`
标志在 kubelet 上启用 `memcg` 通知 API,以便在超过条件时立即收到通知。
如果你不是追求极端利用率,而是要采取合理的过量使用措施,
则解决此问题的可行方法是使用 `--kube-reserved``--system-reserved` 标志为系统分配内存。
<!--
#### active_file memory is not considered as available memory
On Linux, the kernel tracks the number of bytes of file-backed memory on active
LRU list as the `active_file` statistic. The kubelet treats `active_file` memory
areas as not reclaimable. For workloads that make intensive use of block-backed
local storage, including ephemeral local storage, kernel-level caches of file
and block data means that many recently accessed cache pages are likely to be
counted as `active_file`. If enough of these kernel block buffers are on the
active LRU list, the kubelet is liable to observe this as high resource use and
taint the node as experiencing memory pressure - triggering pod eviction.
-->
#### active_file 内存未被视为可用内存
在 Linux 上,内核跟踪活动 LRU 列表上的基于文件所虚拟的内存字节数作为 `active_file` 统计信息。
kubelet 将 `active_file` 内存区域视为不可回收。
对于大量使用块设备形式的本地存储(包括临时本地存储)的工作负载,
文件和块数据的内核级缓存意味着许多最近访问的缓存页面可能被计为 `active_file`
如果这些内核块缓冲区中在活动 LRU 列表上有足够多,
kubelet 很容易将其视为资源用量过量并为节点设置内存压力污点,从而触发 Pod 驱逐。
<!--
For more more details, see [https://github.com/kubernetes/kubernetes/issues/43916](https://github.com/kubernetes/kubernetes/issues/43916)
You can work around that behavior by setting the memory limit and memory request
the same for containers likely to perform intensive I/O activity. You will need
to estimate or measure an optimal memory limit value for that container.
-->
更多细节请参见 [https://github.com/kubernetes/kubernetes/issues/43916](https://github.com/kubernetes/kubernetes/issues/43916)
你可以通过为可能执行 I/O 密集型活动的容器设置相同的内存限制和内存请求来应对该行为。
你将需要估计或测量该容器的最佳内存限制值。
## {{% heading "whatsnext" %}}
<!--
* Learn about [API-initiated Eviction](/docs/reference/generated/kubernetes-api/v1.23/)
* Learn about [Pod Priority and Preemption](/docs/concepts/scheduling-eviction/pod-priority-preemption/)
* Learn about [PodDisruptionBudgets](/docs/tasks/run-application/configure-pdb/)
* Learn about [Quality of Service](/docs/tasks/configure-pod-container/quality-service-pod/) (QoS)
* Check out the [Eviction API](/docs/reference/generated/kubernetes-api/{{<param "version">}}/#create-eviction-pod-v1-core)
-->
* 了解 [API 发起的驱逐](/docs/reference/generated/kubernetes-api/v1.23/)
* 了解 [Pod 优先级和驱逐](/zh/docs/concepts/scheduling-eviction/pod-priority-preemption/)
* 了解 [PodDisruptionBudgets](/docs/tasks/run-application/configure-pdb/)
* 了解[服务质量](/zh/docs/tasks/configure-pod-container/quality-service-pod/)QoS
* 查看[驱逐 API](/docs/reference/generated/kubernetes-api/{{<param "version">}}/#create-eviction-pod-v1-core)
@@ -0,0 +1,335 @@
---
title: Pod 开销
content_type: concept
weight: 30
---
<!--
---
reviewers:
- dchen1107
- egernst
- tallclair
title: Pod Overhead
content_type: concept
weight: 30
---
-->
<!-- overview -->
{{< feature-state for_k8s_version="v1.24" state="stable" >}}
<!--
When you run a Pod on a Node, the Pod itself takes an amount of system resources. These
resources are additional to the resources needed to run the container(s) inside the Pod.
In Kubernetes, _Pod Overhead_ is a way to account for the resources consumed by the Pod
infrastructure on top of the container requests & limits.
-->
在节点上运行 Pod 时,Pod 本身占用大量系统资源。这些是运行 Pod 内容器所需资源之外的资源。
在 Kubernetes 中,_POD 开销_ 是一种方法,用于计算 Pod 基础设施在容器请求和限制之上消耗的资源。
<!-- body -->
<!--
In Kubernetes, the Pod's overhead is set at
[admission](/docs/reference/access-authn-authz/extensible-admission-controllers/#what-are-admission-webhooks)
time according to the overhead associated with the Pod's
[RuntimeClass](/docs/concepts/containers/runtime-class/).
-->
在 Kubernetes 中,Pod 的开销是根据与 Pod 的 [RuntimeClass](/zh/docs/concepts/containers/runtime-class/)
相关联的开销在[准入](/zh/docs/reference/access-authn-authz/extensible-admission-controllers/#what-are-admission-webhooks)时设置的。
<!--
When Pod Overhead is enabled, the overhead is considered in addition to the sum of container
resource requests when scheduling a Pod. Similarly,the kubelet will include the Pod overhead when sizing
the Pod cgroup, and when carrying out Pod eviction ranking.
-->
如果启用了 Pod Overhead,在调度 Pod 时,除了考虑容器资源请求的总和外,还要考虑 Pod 开销。
类似地,kubelet 将在确定 Pod cgroups 的大小和执行 Pod 驱逐排序时也会考虑 Pod 开销。
<!--
## Configuring Pod overhead {#set-up}
-->
## 配置 Pod 开销 {#set-up}
<!--
You need to make sure a `RuntimeClass` is utilized which defines the `overhead` field.
-->
你需要确保使用一个定义了 `overhead` 字段的 `RuntimeClass`
<!--
## Usage example
-->
## 使用示例
<!--
To work with Pod overhead, you need a RuntimeClass that defines the `overhead` field. As
an example, you could use the following RuntimeClass definition with a virtualization container
runtime that uses around 120MiB per Pod for the virtual machine and the guest OS:
-->
要使用 Pod 开销,你需要一个定义了 `overhead` 字段的 RuntimeClass。
作为例子,下面的 RuntimeClass 定义中包含一个虚拟化所用的容器运行时,
RuntimeClass 如下,其中每个 Pod 大约使用 120MiB 用来运行虚拟机和寄宿操作系统:
```yaml
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
name: kata-fc
handler: kata-fc
overhead:
podFixed:
memory: "120Mi"
cpu: "250m"
```
<!--
Workloads which are created which specify the `kata-fc` RuntimeClass handler will take the memory and
cpu overheads into account for resource quota calculations, node scheduling, as well as Pod cgroup sizing.
Consider running the given example workload, test-pod:
-->
通过指定 `kata-fc` RuntimeClass 处理程序创建的工作负载会将内存和 CPU
开销计入资源配额计算、节点调度以及 Pod cgroup 尺寸确定。
假设我们运行下面给出的工作负载示例 test-pod:
```yaml
apiVersion: v1
kind: Pod
metadata:
name: test-pod
spec:
runtimeClassName: kata-fc
containers:
- name: busybox-ctr
image: busybox:1.28
stdin: true
tty: true
resources:
limits:
cpu: 500m
memory: 100Mi
- name: nginx-ctr
image: nginx
resources:
limits:
cpu: 1500m
memory: 100Mi
```
<!--
At admission time the RuntimeClass [admission controller](/docs/reference/access-authn-authz/admission-controllers/)
updates the workload's PodSpec to include the `overhead` as described in the RuntimeClass. If the PodSpec already has this field defined,
the Pod will be rejected. In the given example, since only the RuntimeClass name is specified, the admission controller mutates the Pod
to include an `overhead`.
-->
在准入阶段 RuntimeClass [准入控制器](/zh/docs/reference/access-authn-authz/admission-controllers/)
更新工作负载的 PodSpec 以包含
RuntimeClass 中定义的 `overhead`。如果 PodSpec 中已定义该字段,该 Pod 将会被拒绝。
在这个例子中,由于只指定了 RuntimeClass 名称,所以准入控制器更新了 Pod,使之包含 `overhead`
<!--
After the RuntimeClass admission controller, you can check the updated PodSpec:
-->
在 RuntimeClass 准入控制器进行修改后,你可以查看更新后的 PodSpec:
```bash
kubectl get pod test-pod -o jsonpath='{.spec.overhead}'
```
<!--
The output is:
-->
输出:
```
map[cpu:250m memory:120Mi]
```
<!--
If a ResourceQuota is defined, the sum of container requests as well as the
`overhead` field are counted.
-->
如果定义了 ResourceQuata, 则容器请求的总量以及 `overhead` 字段都将计算在内。
<!--
When the kube-scheduler is deciding which node should run a new Pod, the scheduler considers that Pod's
`overhead` as well as the sum of container requests for that Pod. For this example, the scheduler adds the
requests and the overhead, then looks for a node that has 2.25 CPU and 320 MiB of memory available.
-->
当 kube-scheduler 决定在哪一个节点调度运行新的 Pod 时,调度器会兼顾该 Pod 的
`overhead` 以及该 Pod 的容器请求总量。在这个示例中,调度器将资源请求和开销相加,
然后寻找具备 2.25 CPU 和 320 MiB 内存可用的节点。
<!--
Once a Pod is scheduled to a node, the kubelet on that node creates a new {{< glossary_tooltip
text="cgroup" term_id="cgroup" >}} for the Pod. It is within this pod that the underlying
container runtime will create containers.
-->
一旦 Pod 被调度到了某个节点, 该节点上的 kubelet 将为该 Pod 新建一个
{{< glossary_tooltip text="cgroup" term_id="cgroup" >}}。 底层容器运行时将在这个
Pod 中创建容器。
<!--
If the resource has a limit defined for each container (Guaranteed QoS or Burstable QoS with limits defined),
the kubelet will set an upper limit for the pod cgroup associated with that resource (cpu.cfs_quota_us for CPU
and memory.limit_in_bytes memory). This upper limit is based on the sum of the container limits plus the `overhead`
defined in the PodSpec.
-->
如果该资源对每一个容器都定义了一个限制(定义了限制值的 Guaranteed QoS 或者
Burstable QoS),kubelet 会为与该资源(CPU 的 `cpu.cfs_quota_us` 以及内存的
`memory.limit_in_bytes`
相关的 Pod cgroup 设定一个上限。该上限基于 PodSpec 中定义的容器限制总量与 `overhead` 之和。
<!--
For CPU, if the Pod is Guaranteed or Burstable QoS, the kubelet will set `cpu.shares` based on the
sum of container requests plus the `overhead` defined in the PodSpec.
-->
对于 CPU,如果 Pod 的 QoS 是 Guaranteed 或者 Burstablekubelet 会基于容器请求总量与
PodSpec 中定义的 `overhead` 之和设置 `cpu.shares`
<!--
Looking at our example, verify the container requests for the workload:
-->
请看这个例子,验证工作负载的容器请求:
```bash
kubectl get pod test-pod -o jsonpath='{.spec.containers[*].resources.limits}'
```
<!--
The total container requests are 2000m CPU and 200MiB of memory:
-->
容器请求总计 2000m CPU 和 200MiB 内存:
```
map[cpu: 500m memory:100Mi] map[cpu:1500m memory:100Mi]
```
<!--
Check this against what is observed by the node:
-->
对照从节点观察到的情况来检查一下:
```bash
kubectl describe node | grep test-pod -B2
```
<!--
The output shows requests for 2250m CPU, and for 320MiB of memory. The requests include Pod overhead:
-->
该输出显示请求了 2250m CPU 以及 320MiB 内存。请求包含了 Pod 开销在内:
```
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits AGE
--------- ---- ------------ ---------- --------------- ------------- ---
default test-pod 2250m (56%) 2250m (56%) 320Mi (1%) 320Mi (1%) 36m
```
<!--
## Verify Pod cgroup limits
-->
## 验证 Pod cgroup 限制
<!--
Check the Pod's memory cgroups on the node where the workload is running. In the following example,
[`crictl`](https://github.com/kubernetes-sigs/cri-tools/blob/master/docs/crictl.md)
is used on the node, which provides a CLI for CRI-compatible container runtimes. This is an
advanced example to show Pod overhead behavior, and it is not expected that users should need to check
cgroups directly on the node.
First, on the particular node, determine the Pod identifier:
-->
在工作负载所运行的节点上检查 Pod 的内存 cgroups。在接下来的例子中,
将在该节点上使用具备 CRI 兼容的容器运行时命令行工具
[`crictl`](https://github.com/kubernetes-sigs/cri-tools/blob/master/docs/crictl.md)。
这是一个显示 Pod 开销行为的高级示例, 预计用户不需要直接在节点上检查 cgroups。
首先在特定的节点上确定该 Pod 的标识符:
<!--
```bash
# Run this on the node where the Pod is scheduled
-->
```bash
# 在该 Pod 被调度到的节点上执行如下命令:
POD_ID="$(sudo crictl pods --name test-pod -q)"
```
<!--
From this, you can determine the cgroup path for the Pod:
-->
可以依此判断该 Pod 的 cgroup 路径:
<!--
```bash
# Run this on the node where the Pod is scheduled
-->
```bash
# 在该 Pod 被调度到的节点上执行如下命令:
sudo crictl inspectp -o=json $POD_ID | grep cgroupsPath
```
<!--
The resulting cgroup path includes the Pod's `pause` container. The Pod level cgroup is one directory above.
-->
执行结果的 cgroup 路径中包含了该 Pod 的 `pause` 容器。Pod 级别的 cgroup 在即上一层目录。
```
"cgroupsPath": "/kubepods/podd7f4b509-cf94-4951-9417-d1087c92a5b2/7ccf55aee35dd16aca4189c952d83487297f3cd760f1bbf09620e206e7d0c27a"
```
<!--
In this specific case, the pod cgroup path is `kubepods/podd7f4b509-cf94-4951-9417-d1087c92a5b2`.
Verify the Pod level cgroup setting for memory:
-->
在这个例子中,该 Pod 的 cgroup 路径是 `kubepods/podd7f4b509-cf94-4951-9417-d1087c92a5b2`
验证内存的 Pod 级别 cgroup 设置:
<!--
```bash
# Run this on the node where the Pod is scheduled.
# Also, change the name of the cgroup to match the cgroup allocated for your pod.
-->
```bash
# 在该 Pod 被调度到的节点上执行这个命令。
# 另外,修改 cgroup 的名称以匹配为该 Pod 分配的 cgroup。
cat /sys/fs/cgroup/memory/kubepods/podd7f4b509-cf94-4951-9417-d1087c92a5b2/memory.limit_in_bytes
```
<!--
This is 320 MiB, as expected:
-->
和预期的一样,这一数值为 320 MiB。
```
335544320
```
<!--
### Observability
-->
### 可观察性
<!--
Some `kube_pod_overhead_*` metrics are available in [kube-state-metrics](https://github.com/kubernetes/kube-state-metrics)
to help identify when Pod overhead is being utilized and to help observe stability of workloads
running with a defined overhead.
-->
在 [kube-state-metrics](https://github.com/kubernetes/kube-state-metrics) 中可以通过
`kube_pod_overhead_*` 指标来协助确定何时使用 Pod 开销,
以及协助观察以一个既定开销运行的工作负载的稳定性。
该特性在 kube-state-metrics 的 1.9 发行版本中不可用,不过预计将在后续版本中发布。
在此之前,用户需要从源代码构建 kube-state-metrics。
## {{% heading "whatsnext" %}}
<!--
* Learn more about [RuntimeClass](/docs/concepts/containers/runtime-class/)
* Read the [PodOverhead Design](https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/688-pod-overhead)
enhancement proposal for extra context
-->
* 学习更多关于 [RuntimeClass](/zh/docs/concepts/containers/runtime-class/) 的信息
* 阅读 [PodOverhead 设计](https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/688-pod-overhead)增强建议以获取更多上下文
@@ -0,0 +1,669 @@
---
title: Pod 优先级和抢占
content_type: concept
weight: 50
---
<!--
reviewers:
- davidopp
- wojtek-t
title: Pod Priority and Preemption
content_type: concept
weight: 50
-->
<!-- overview -->
{{< feature-state for_k8s_version="v1.14" state="stable" >}}
<!--
[Pods](/docs/concepts/workloads/pods/) can have _priority_. Priority indicates the
importance of a Pod relative to other Pods. If a Pod cannot be scheduled, the
scheduler tries to preempt (evict) lower priority Pods to make scheduling of the
pending Pod possible.
-->
[Pod](/zh/docs/concepts/workloads/pods/) 可以有 _优先级_
优先级表示一个 Pod 相对于其他 Pod 的重要性。
如果一个 Pod 无法被调度,调度程序会尝试抢占(驱逐)较低优先级的 Pod,
以使悬决 Pod 可以被调度。
<!-- body -->
{{< warning >}}
<!--
In a cluster where not all users are trusted, a malicious user could create Pods
at the highest possible priorities, causing other Pods to be evicted/not get
scheduled.
An administrator can use ResourceQuota to prevent users from creating pods at
high priorities.
See [limit Priority Class consumption by default](/docs/concepts/policy/resource-quotas/#limit-priority-class-consumption-by-default)
for details.
-->
在一个并非所有用户都是可信的集群中,恶意用户可能以最高优先级创建 Pod,
导致其他 Pod 被驱逐或者无法被调度。
管理员可以使用 ResourceQuota 来阻止用户创建高优先级的 Pod。
参见[默认限制优先级消费](/zh/docs/concepts/policy/resource-quotas/#limit-priority-class-consumption-by-default)。
{{< /warning >}}
<!--
## How to use priority and preemption
To use priority and preemption:
1. Add one or more [PriorityClasses](#priorityclass).
1. Create Pods with[`priorityClassName`](#pod-priority) set to one of the added
PriorityClasses. Of course you do not need to create the Pods directly;
normally you would add `priorityClassName` to the Pod template of a
collection object like a Deployment.
Keep reading for more information about these steps.
-->
## 如何使用优先级和抢占
要使用优先级和抢占:
1. 新增一个或多个 [PriorityClass](#priorityclass)。
1. 创建 Pod,并将其 [`priorityClassName`](#pod-priority) 设置为新增的 PriorityClass。
当然你不需要直接创建 Pod;通常,你将会添加 `priorityClassName` 到集合对象(如 Deployment
的 Pod 模板中。
继续阅读以获取有关这些步骤的更多信息。
{{< note >}}
<!--
Kubernetes already ships with two PriorityClasses:
`system-cluster-critical` and `system-node-critical`.
These are common classes and are used to [ensure that critical components are always scheduled first](/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods/).
-->
Kubernetes 已经提供了 2 个 PriorityClass
`system-cluster-critical``system-node-critical`
这些是常见的类,用于[确保始终优先调度关键组件](/zh/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods/)。
{{< /note >}}
<!--
## PriorityClass
A PriorityClass is a non-namespaced object that defines a mapping from a
priority class name to the integer value of the priority. The name is specified
in the `name` field of the PriorityClass object's metadata. The value is
specified in the required `value` field. The higher the value, the higher the
priority.
The name of a PriorityClass object must be a valid
[DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names),
and it cannot be prefixed with `system-`.
-->
## PriorityClass {#priorityclass}
PriorityClass 是一个无名称空间对象,它定义了从优先级类名称到优先级整数值的映射。
名称在 PriorityClass 对象元数据的 `name` 字段中指定。
值在必填的 `value` 字段中指定。值越大,优先级越高。
PriorityClass 对象的名称必须是有效的
[DNS 子域名](/zh/docs/concepts/overview/working-with-objects/names#dns-subdomain-names)
并且它不能以 `system-` 为前缀。
<!--
A PriorityClass object can have any 32-bit integer value smaller than or equal
to 1 billion. Larger numbers are reserved for critical system Pods that should
not normally be preempted or evicted. A cluster admin should create one
PriorityClass object for each such mapping that they want.
PriorityClass also has two optional fields: `globalDefault` and `description`.
The `globalDefault` field indicates that the value of this PriorityClass should
be used for Pods without a `priorityClassName`. Only one PriorityClass with
`globalDefault` set to true can exist in the system. If there is no
PriorityClass with `globalDefault` set, the priority of Pods with no
`priorityClassName` is zero.
The `description` field is an arbitrary string. It is meant to tell users of the
cluster when they should use this PriorityClass.
-->
PriorityClass 对象可以设置任何小于或等于 10 亿的 32 位整数值。
较大的数字是为通常不应被抢占或驱逐的关键的系统 Pod 所保留的。
集群管理员应该为这类映射分别创建独立的 PriorityClass 对象。
PriorityClass 还有两个可选字段:`globalDefault``description`
`globalDefault` 字段表示这个 PriorityClass 的值应该用于没有 `priorityClassName` 的 Pod。
系统中只能存在一个 `globalDefault` 设置为 true 的 PriorityClass。
如果不存在设置了 `globalDefault` 的 PriorityClass
则没有 `priorityClassName` 的 Pod 的优先级为零。
`description` 字段是一个任意字符串。
它用来告诉集群用户何时应该使用此 PriorityClass。
<!--
### Notes about PodPriority and existing clusters
- If you upgrade an existing cluster without this feature, the priority
of your existing Pods is effectively zero.
- Addition of a PriorityClass with `globalDefault` set to `true` does not
change the priorities of existing Pods. The value of such a PriorityClass is
used only for Pods created after the PriorityClass is added.
- If you delete a PriorityClass, existing Pods that use the name of the
deleted PriorityClass remain unchanged, but you cannot create more Pods that
use the name of the deleted PriorityClass.
-->
### 关于 PodPriority 和现有集群的注意事项
- 如果你升级一个已经存在的但尚未使用此特性的集群,该集群中已经存在的 Pod 的优先级等效于零。
- 添加一个将 `globalDefault` 设置为 `true` 的 PriorityClass 不会改变现有 Pod 的优先级。
此类 PriorityClass 的值仅用于添加 PriorityClass 后创建的 Pod。
- 如果你删除了某个 PriorityClass 对象,则使用被删除的 PriorityClass 名称的现有 Pod 保持不变,
但是你不能再创建使用已删除的 PriorityClass 名称的 Pod。
<!-- ### Example PriorityClass -->
### PriorityClass 示例
```yaml
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
name: high-priority
value: 1000000
globalDefault: false
description: "此优先级类应仅用于 XYZ 服务 Pod。"
```
<!--
## Non-preempting PriorityClass {#non-preempting-priority-class}
{{< feature-state for_k8s_version="v1.24" state="stable" >}}
Pods with `preemptionPolicy: Never` will be placed in the scheduling queue
ahead of lower-priority pods,
but they cannot preempt other pods.
A non-preempting pod waiting to be scheduled will stay in the scheduling queue,
until sufficient resources are free,
and it can be scheduled.
Non-preempting pods,
like other pods,
are subject to scheduler back-off.
This means that if the scheduler tries these pods and they cannot be scheduled,
they will be retried with lower frequency,
allowing other pods with lower priority to be scheduled before them.
Non-preempting pods may still be preempted by other,
high-priority pods.
-->
## 非抢占式 PriorityClass {#non-preempting-priority-class}
{{< feature-state for_k8s_version="v1.24" state="stable" >}}
配置了 `preemptionPolicy: Never` 的 Pod 将被放置在调度队列中较低优先级 Pod 之前,
但它们不能抢占其他 Pod。等待调度的非抢占式 Pod 将留在调度队列中,直到有足够的可用资源,
它才可以被调度。非抢占式 Pod,像其他 Pod 一样,受调度程序回退的影响。
这意味着如果调度程序尝试这些 Pod 并且无法调度它们,它们将以更低的频率被重试,
从而允许其他优先级较低的 Pod 排在它们之前。
非抢占式 Pod 仍可能被其他高优先级 Pod 抢占。
<!--
`preemptionPolicy` defaults to `PreemptLowerPriority`,
which will allow pods of that PriorityClass to preempt lower-priority pods
(as is existing default behavior).
If `preemptionPolicy` is set to `Never`,
pods in that PriorityClass will be non-preempting.
An example use case is for data science workloads.
A user may submit a job that they want to be prioritized above other workloads,
but do not wish to discard existing work by preempting running pods.
The high priority job with `preemptionPolicy: Never` will be scheduled
ahead of other queued pods,
as soon as sufficient cluster resources "naturally" become free.
-->
`preemptionPolicy` 默认为 `PreemptLowerPriority`
这将允许该 PriorityClass 的 Pod 抢占较低优先级的 Pod(现有默认行为也是如此)。
如果 `preemptionPolicy` 设置为 `Never`,则该 PriorityClass 中的 Pod 将是非抢占式的。
数据科学工作负载是一个示例用例。用户可以提交他们希望优先于其他工作负载的作业,
但不希望因为抢占运行中的 Pod 而导致现有工作被丢弃。
设置为 `preemptionPolicy: Never` 的高优先级作业将在其他排队的 Pod 之前被调度,
只要足够的集群资源“自然地”变得可用。
<!-- ### Example Non-preempting PriorityClass -->
### 非抢占式 PriorityClass 示例
```yaml
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
name: high-priority-nonpreempting
value: 1000000
preemptionPolicy: Never
globalDefault: false
description: "This priority class will not cause other pods to be preempted."
```
<!--
## Pod priority
After you have one or more PriorityClasses, you can create Pods that specify one
of those PriorityClass names in their specifications. The priority admission
controller uses the `priorityClassName` field and populates the integer value of
the priority. If the priority class is not found, the Pod is rejected.
The following YAML is an example of a Pod configuration that uses the
PriorityClass created in the preceding example. The priority admission
controller checks the specification and resolves the priority of the Pod to
1000000.
-->
## Pod 优先级 {#pod-priority}
在你拥有一个或多个 PriorityClass 对象之后,
你可以创建在其规约中指定这些 PriorityClass 名称之一的 Pod。
优先级准入控制器使用 `priorityClassName` 字段并填充优先级的整数值。
如果未找到所指定的优先级类,则拒绝 Pod。
以下 YAML 是 Pod 配置的示例,它使用在前面的示例中创建的 PriorityClass。
优先级准入控制器检查 Pod 规约并将其优先级解析为 1000000。
```yaml
apiVersion: v1
kind: Pod
metadata:
name: nginx
labels:
env: test
spec:
containers:
- name: nginx
image: nginx
imagePullPolicy: IfNotPresent
priorityClassName: high-priority
```
<!--
### Effect of Pod priority on scheduling order
When Pod priority is enabled, the scheduler orders pending Pods by
their priority and a pending Pod is placed ahead of other pending Pods
with lower priority in the scheduling queue. As a result, the higher
priority Pod may be scheduled sooner than Pods with lower priority if
its scheduling requirements are met. If such Pod cannot be scheduled,
scheduler will continue and tries to schedule other lower priority Pods.
-->
### Pod 优先级对调度顺序的影响
当启用 Pod 优先级时,调度程序会按优先级对悬决 Pod 进行排序,
并且每个悬决的 Pod 会被放置在调度队列中其他优先级较低的悬决 Pod 之前。
因此,如果满足调度要求,较高优先级的 Pod 可能会比具有较低优先级的 Pod 更早调度。
如果无法调度此类 Pod,调度程序将继续并尝试调度其他较低优先级的 Pod。
<!--
## Preemption
When Pods are created, they go to a queue and wait to be scheduled. The
scheduler picks a Pod from the queue and tries to schedule it on a Node. If no
Node is found that satisfies all the specified requirements of the Pod,
preemption logic is triggered for the pending Pod. Let's call the pending Pod P.
Preemption logic tries to find a Node where removal of one or more Pods with
lower priority than P would enable P to be scheduled on that Node. If such a
Node is found, one or more lower priority Pods get evicted from the Node. After
the Pods are gone, P can be scheduled on the Node.
-->
## 抢占 {#preemption}
Pod 被创建后会进入队列等待调度。
调度器从队列中挑选一个 Pod 并尝试将它调度到某个节点上。
如果没有找到满足 Pod 的所指定的所有要求的节点,则触发对悬决 Pod 的抢占逻辑。
让我们将悬决 Pod 称为 P。抢占逻辑试图找到一个节点,
在该节点中删除一个或多个优先级低于 P 的 Pod,则可以将 P 调度到该节点上。
如果找到这样的节点,一个或多个优先级较低的 Pod 会被从节点中驱逐。
被驱逐的 Pod 消失后,P 可以被调度到该节点上。
<!--
### User exposed information
When Pod P preempts one or more Pods on Node N, `nominatedNodeName` field of Pod
P's status is set to the name of Node N. This field helps scheduler track
resources reserved for Pod P and also gives users information about preemptions
in their clusters.
Please note that Pod P is not necessarily scheduled to the "nominated Node".
The scheduler always tries the "nominated Node" before iterating over any other nodes.
After victim Pods are preempted, they get their graceful termination period. If
another node becomes available while scheduler is waiting for the victim Pods to
terminate, scheduler may use the other node to schedule Pod P. As a result
`nominatedNodeName` and `nodeName` of Pod spec are not always the same. Also, if
scheduler preempts Pods on Node N, but then a higher priority Pod than Pod P
arrives, scheduler may give Node N to the new higher priority Pod. In such a
case, scheduler clears `nominatedNodeName` of Pod P. By doing this, scheduler
makes Pod P eligible to preempt Pods on another Node.
-->
### 用户暴露的信息
当 Pod P 抢占节点 N 上的一个或多个 Pod 时,
Pod P 状态的 `nominatedNodeName` 字段被设置为节点 N 的名称。
该字段帮助调度程序跟踪为 Pod P 保留的资源,并为用户提供有关其集群中抢占的信息。
请注意,Pod P 不一定会调度到“被提名的节点(Nominated Node)”。
调度程序总是在迭代任何其他节点之前尝试“指定节点”。
在 Pod 因抢占而牺牲时,它们将获得体面终止期。
如果调度程序正在等待牺牲者 Pod 终止时另一个节点变得可用,
则调度程序可以使用另一个节点来调度 Pod P。
因此,Pod 规约中的 `nominatedNodeName``nodeName` 并不总是相同。
此外,如果调度程序抢占节点 N 上的 Pod,但随后比 Pod P 更高优先级的 Pod 到达,
则调度程序可能会将节点 N 分配给新的更高优先级的 Pod。
在这种情况下,调度程序会清除 Pod P 的 `nominatedNodeName`
通过这样做,调度程序使 Pod P 有资格抢占另一个节点上的 Pod。
<!--
### Limitations of preemption
#### Graceful termination of preemption victims
When Pods are preempted, the victims get their
[graceful termination period](/docs/concepts/workloads/pods/pod-lifecycle/#pod-termination).
They have that much time to finish their work and exit. If they don't, they are
killed. This graceful termination period creates a time gap between the point
that the scheduler preempts Pods and the time when the pending Pod (P) can be
scheduled on the Node (N). In the meantime, the scheduler keeps scheduling other
pending Pods. As victims exit or get terminated, the scheduler tries to schedule
Pods in the pending queue. Therefore, there is usually a time gap between the
point that scheduler preempts victims and the time that Pod P is scheduled. In
order to minimize this gap, one can set graceful termination period of lower
priority Pods to zero or a small number.
-->
### 抢占的限制
#### 被抢占牺牲者的体面终止
当 Pod 被抢占时,牺牲者会得到他们的
[体面终止期](/zh/docs/concepts/workloads/pods/pod-lifecycle/#pod-termination)。
它们可以在体面终止期内完成工作并退出。如果它们不这样做就会被杀死。
这个体面终止期在调度程序抢占 Pod 的时间点和待处理的 Pod (P)
可以在节点 (N) 上调度的时间点之间划分出了一个时间跨度。
同时,调度器会继续调度其他待处理的 Pod。当牺牲者退出或被终止时,
调度程序会尝试在待处理队列中调度 Pod。
因此,调度器抢占牺牲者的时间点与 Pod P 被调度的时间点之间通常存在时间间隔。
为了最小化这个差距,可以将低优先级 Pod 的体面终止时间设置为零或一个小数字。
<!--
#### PodDisruptionBudget is supported, but not guaranteed
A [PodDisruptionBudget](/docs/concepts/workloads/pods/disruptions/) (PDB)
allows application owners to limit the number of Pods of a replicated application
that are down simultaneously from voluntary disruptions. Kubernetes supports
PDB when preempting Pods, but respecting PDB is best effort. The scheduler tries
to find victims whose PDB are not violated by preemption, but if no such victims
are found, preemption will still happen, and lower priority Pods will be removed
despite their PDBs being violated.
-->
#### 支持 PodDisruptionBudget,但不保证
[PodDisruptionBudget](/zh/docs/concepts/workloads/pods/disruptions/)
(PDB) 允许多副本应用程序的所有者限制因自愿性质的干扰而同时终止的 Pod 数量。
Kubernetes 在抢占 Pod 时支持 PDB,但对 PDB 的支持是基于尽力而为原则的。
调度器会尝试寻找不会因被抢占而违反 PDB 的牺牲者,但如果没有找到这样的牺牲者,
抢占仍然会发生,并且即使违反了 PDB 约束也会删除优先级较低的 Pod。
<!--
#### Inter-Pod affinity on lower-priority Pods
A Node is considered for preemption only when the answer to this question is
yes: "If all the Pods with lower priority than the pending Pod are removed from
the Node, can the pending Pod be scheduled on the Node?"
{{< note >}}
Preemption does not necessarily remove all lower-priority
Pods. If the pending Pod can be scheduled by removing fewer than all
lower-priority Pods, then only a portion of the lower-priority Pods are removed.
Even so, the answer to the preceding question must be yes. If the answer is no,
the Node is not considered for preemption.
{{< /note >}}
-->
#### 与低优先级 Pod 之间的 Pod 间亲和性
只有当这个问题的答案是肯定的时,才考虑在一个节点上执行抢占操作:
“如果从此节点上删除优先级低于悬决 Pod 的所有 Pod,悬决 Pod 是否可以在该节点上调度?”
{{< note >}}
抢占并不一定会删除所有较低优先级的 Pod。
如果悬决 Pod 可以通过删除少于所有较低优先级的 Pod 来调度,
那么只有一部分较低优先级的 Pod 会被删除。
即便如此,上述问题的答案必须是肯定的。
如果答案是否定的,则不考虑在该节点上执行抢占。
{{< /note >}}
<!--
If a pending Pod has inter-pod {{< glossary_tooltip text="affinity" term_id="affinity" >}}
to one or more of the lower-priority Pods on the Node, the inter-Pod affinity
rule cannot be satisfied in the absence of those lower-priority Pods. In this case,
the scheduler does not preempt any Pods on the Node. Instead, it looks for another
Node. The scheduler might find a suitable Node or it might not. There is no
guarantee that the pending Pod can be scheduled.
Our recommended solution for this problem is to create inter-Pod affinity only
towards equal or higher priority Pods.
-->
如果悬决 Pod 与节点上的一个或多个较低优先级 Pod 具有 Pod 间{{< glossary_tooltip text="亲和性" term_id="affinity" >}}
则在没有这些较低优先级 Pod 的情况下,无法满足 Pod 间亲和性规则。
在这种情况下,调度程序不会抢占节点上的任何 Pod。
相反,它寻找另一个节点。调度程序可能会找到合适的节点,
也可能不会。无法保证悬决 Pod 可以被调度。
我们针对此问题推荐的解决方案是仅针对同等或更高优先级的 Pod 设置 Pod 间亲和性。
<!--
#### Cross node preemption
Suppose a Node N is being considered for preemption so that a pending Pod P can
be scheduled on N. P might become feasible on N only if a Pod on another Node is
preempted. Here's an example:
* Pod P is being considered for Node N.
* Pod Q is running on another Node in the same Zone as Node N.
* Pod P has Zone-wide anti-affinity with Pod Q (`topologyKey:
topology.kubernetes.io/zone`).
* There are no other cases of anti-affinity between Pod P and other Pods in
the Zone.
* In order to schedule Pod P on Node N, Pod Q can be preempted, but scheduler
does not perform cross-node preemption. So, Pod P will be deemed
unschedulable on Node N.
If Pod Q were removed from its Node, the Pod anti-affinity violation would be
gone, and Pod P could possibly be scheduled on Node N.
We may consider adding cross Node preemption in future versions if there is
enough demand and if we find an algorithm with reasonable performance.
-->
#### 跨节点抢占
假设正在考虑在一个节点 N 上执行抢占,以便可以在 N 上调度待处理的 Pod P。
只有当另一个节点上的 Pod 被抢占时,P 才可能在 N 上变得可行。
下面是一个例子:
* 正在考虑将 Pod P 调度到节点 N 上。
* Pod Q 正在与节点 N 位于同一区域的另一个节点上运行。
* Pod P 与 Pod Q 具有 Zone 维度的反亲和(`topologyKey:topology.kubernetes.io/zone`)。
* Pod P 与 Zone 中的其他 Pod 之间没有其他反亲和性设置。
* 为了在节点 N 上调度 Pod P,可以抢占 Pod Q,但调度器不会进行跨节点抢占。
因此,Pod P 将被视为在节点 N 上不可调度。
如果将 Pod Q 从所在节点中移除,则不会违反 Pod 间反亲和性约束,
并且 Pod P 可能会被调度到节点 N 上。
如果有足够的需求,并且如果我们找到性能合理的算法,
我们可能会考虑在未来版本中添加跨节点抢占。
<!--
## Troubleshooting
Pod priority and pre-emption can have unwanted side effects. Here are some
examples of potential problems and ways to deal with them.
-->
## 故障排除
Pod 优先级和抢占可能会产生不必要的副作用。以下是一些潜在问题的示例以及处理这些问题的方法。
<!--
### Pods are preempted unnecessarily
Preemption removes existing Pods from a cluster under resource pressure to make
room for higher priority pending Pods. If you give high priorities to
certain Pods by mistake, these unintentionally high priority Pods may cause
preemption in your cluster. Pod priority is specified by setting the
`priorityClassName` field in the Pod's specification. The integer value for
priority is then resolved and populated to the `priority` field of `podSpec`.
To address the problem, you can change the `priorityClassName` for those Pods
to use lower priority classes, or leave that field empty. An empty
`priorityClassName` is resolved to zero by default.
When a Pod is preempted, there will be events recorded for the preempted Pod.
Preemption should happen only when a cluster does not have enough resources for
a Pod. In such cases, preemption happens only when the priority of the pending
Pod (preemptor) is higher than the victim Pods. Preemption must not happen when
there is no pending Pod, or when the pending Pods have equal or lower priority
than the victims. If preemption happens in such scenarios, please file an issue.
-->
### Pod 被不必要地抢占
抢占在资源压​​力较大时从集群中删除现有 Pod,为更高优先级的悬决 Pod 腾出空间。
如果你错误地为某些 Pod 设置了高优先级,这些无意的高优先级 Pod 可能会导致集群中出现抢占行为。
Pod 优先级是通过设置 Pod 规约中的 `priorityClassName` 字段来指定的。
优先级的整数值然后被解析并填充到 `podSpec` 的 `priority` 字段。
为了解决这个问题,你可以将这些 Pod 的 `priorityClassName` 更改为使用较低优先级的类,
或者将该字段留空。默认情况下,空的 `priorityClassName` 解析为零。
当 Pod 被抢占时,集群会为被抢占的 Pod 记录事件。只有当集群没有足够的资源用于 Pod 时,
才会发生抢占。在这种情况下,只有当悬决 Pod(抢占者)的优先级高于受害 Pod 时才会发生抢占。
当没有悬决 Pod,或者悬决 Pod 的优先级等于或低于牺牲者时,不得发生抢占。
如果在这种情况下发生抢占,请提出问题。
<!--
### Pods are preempted, but the preemptor is not scheduled
When pods are preempted, they receive their requested graceful termination
period, which is by default 30 seconds. If the victim Pods do not terminate within
this period, they are forcibly terminated. Once all the victims go away, the
preemptor Pod can be scheduled.
While the preemptor Pod is waiting for the victims to go away, a higher priority
Pod may be created that fits on the same Node. In this case, the scheduler will
schedule the higher priority Pod instead of the preemptor.
This is expected behavior: the Pod with the higher priority should take the place
of a Pod with a lower priority.
-->
### 有 Pod 被抢占,但抢占者并没有被调度
当 Pod 被抢占时,它们会收到请求的体面终止期,默认为 30 秒。
如果受害 Pod 在此期限内没有终止,它们将被强制终止。
一旦所有牺牲者都离开,就可以调度抢占者 Pod。
在抢占者 Pod 等待牺牲者离开的同时,可能某个适合同一个节点的更高优先级的 Pod 被创建。
在这种情况下,调度器将调度优先级更高的 Pod 而不是抢占者。
这是预期的行为:具有较高优先级的 Pod 应该取代具有较低优先级的 Pod。
<!--
### Higher priority Pods are preempted before lower priority pods
The scheduler tries to find nodes that can run a pending Pod. If no node is
found, the scheduler tries to remove Pods with lower priority from an arbitrary
node in order to make room for the pending pod.
If a node with low priority Pods is not feasible to run the pending Pod, the scheduler
may choose another node with higher priority Pods (compared to the Pods on the
other node) for preemption. The victims must still have lower priority than the
preemptor Pod.
When there are multiple nodes available for preemption, the scheduler tries to
choose the node with a set of Pods with lowest priority. However, if such Pods
have PodDisruptionBudget that would be violated if they are preempted then the
scheduler may choose another node with higher priority Pods.
When multiple nodes exist for preemption and none of the above scenarios apply,
the scheduler chooses a node with the lowest priority.
-->
### 优先级较高的 Pod 在优先级较低的 Pod 之前被抢占
调度程序尝试查找可以运行悬决 Pod 的节点。如果没有找到这样的节点,
调度程序会尝试从任意节点中删除优先级较低的 Pod,以便为悬决 Pod 腾出空间。
如果具有低优先级 Pod 的节点无法运行悬决 Pod,
调度器可能会选择另一个具有更高优先级 Pod 的节点(与其他节点上的 Pod 相比)进行抢占。
牺牲者的优先级必须仍然低于抢占者 Pod。
当有多个节点可供执行抢占操作时,调度器会尝试选择具有一组优先级最低的 Pod 的节点。
但是,如果此类 Pod 具有 PodDisruptionBudget,当它们被抢占时,
则会违反 PodDisruptionBudget,那么调度程序可能会选择另一个具有更高优先级 Pod 的节点。
当存在多个节点抢占且上述场景均不适用时,调度器会选择优先级最低的节点。
<!--
## Interactions between Pod priority and quality of service {#interactions-of-pod-priority-and-qos}
Pod priority and {{< glossary_tooltip text="QoS class" term_id="qos-class" >}}
are two orthogonal features with few interactions and no default restrictions on
setting the priority of a Pod based on its QoS classes. The scheduler's
preemption logic does not consider QoS when choosing preemption targets.
Preemption considers Pod priority and attempts to choose a set of targets with
the lowest priority. Higher-priority Pods are considered for preemption only if
the removal of the lowest priority Pods is not sufficient to allow the scheduler
to schedule the preemptor Pod, or if the lowest priority Pods are protected by
`PodDisruptionBudget`.
-->
## Pod 优先级和服务质量之间的相互作用 {#interactions-of-pod-priority-and-qos}
Pod 优先级和 {{<glossary_tooltip text="QoS 类" term_id="qos-class" >}}
是两个正交特征,交互很少,并且对基于 QoS 类设置 Pod 的优先级没有默认限制。
调度器的抢占逻辑在选择抢占目标时不考虑 QoS。
抢占会考虑 Pod 优先级并尝试选择一组优先级最低的目标。
仅当移除优先级最低的 Pod 不足以让调度程序调度抢占式 Pod,
或者最低优先级的 Pod 受 PodDisruptionBudget 保护时,才会考虑优先级较高的 Pod。
<!--
The kubelet uses Priority to determine pod order for [node-pressure eviction](/docs/concepts/scheduling-eviction/node-pressure-eviction/).
You can use the QoS class to estimate the order in which pods are most likely
to get evicted. The kubelet ranks pods for eviction based on the following factors:
1. Whether the starved resource usage exceeds requests
1. Pod Priority
1. Amount of resource usage relative to requests
See [evicting end-user pods](/docs/concepts/scheduling-eviction/node-pressure-eviction/#pod-selection-for-kubelet-eviction)
for more details.
kubelet node-pressure eviction does not evict Pods when their
usage does not exceed their requests. If a Pod with lower priority is not
exceeding its requests, it won't be evicted. Another Pod with higher priority
that exceeds its requests may be evicted.
-->
kubelet 使用优先级来确定
[节点压力驱逐](/zh/docs/concepts/scheduling-eviction/pod-priority-preemption/) Pod 的顺序。
你可以使用 QoS 类来估计 Pod 最有可能被驱逐的顺序。kubelet 根据以下因素对 Pod 进行驱逐排名:
1. 对紧俏资源的使用是否超过请求值
1. Pod 优先级
1. 相对于请求的资源使用量
有关更多详细信息,请参阅
[kubelet 驱逐时 Pod 的选择](/zh/docs/concepts/scheduling-eviction/node-pressure-eviction/#pod-selection-for-kubelet-eviction)。
当某 Pod 的资源用量未超过其请求时,kubelet 节点压力驱逐不会驱逐该 Pod。
如果优先级较低的 Pod 没有超过其请求,则不会被驱逐。
另一个优先级高于其请求的 Pod 可能会被驱逐。
## {{% heading "whatsnext" %}}
<!--
* Read about using ResourceQuotas in connection with PriorityClasses:
[limit Priority Class consumption by default](/docs/concepts/policy/resource-quotas/#limit-priority-class-consumption-by-default)
* Learn about [Pod Disruption](/docs/concepts/workloads/pods/disruptions/)
* Learn about [API-initiated Eviction](/docs/reference/generated/kubernetes-api/v1.23/)
* Learn about [Node-pressure Eviction](/docs/concepts/scheduling-eviction/node-pressure-eviction/)
-->
* 阅读有关将 ResourceQuota 与 PriorityClass 结合使用的信息:
[默认限制优先级消费](/zh/docs/concepts/policy/resource-quotas/#limit-priority-class-consumption-by-default)
* 了解 [Pod 干扰](/zh/docs/concepts/workloads/pods/disruptions/)
* 了解 [API 发起的驱逐](/docs/reference/generated/kubernetes-api/v1.23/)
* 了解[节点压力驱逐](/zh/docs/concepts/scheduling-eviction/pod-priority-preemption/)
@@ -0,0 +1,272 @@
---
title: 扩展资源的资源装箱
content_type: concept
weight: 80
---
<!--
---
reviewers:
- bsalamat
- k82cn
- ahg-g
title: Resource Bin Packing for Extended Resources
content_type: concept
weight: 80
---
-->
<!-- overview -->
{{< feature-state for_k8s_version="1.16" state="alpha" >}}
<!--
The kube-scheduler can be configured to enable bin packing of resources along with extended resources using `RequestedToCapacityRatioResourceAllocation` priority function. Priority functions can be used to fine-tune the kube-scheduler as per custom needs.
-->
使用 `RequestedToCapacityRatioResourceAllocation` 优先级函数,可以将 kube-scheduler
配置为支持包含扩展资源在内的资源装箱操作。
优先级函数可用于根据自定义需求微调 kube-scheduler 。
<!-- body -->
<!--
## Enabling Bin Packing using RequestedToCapacityRatioResourceAllocation
Kubernetes allows the users to specify the resources along with weights for
each resource to score nodes based on the request to capacity ratio. This
allows users to bin pack extended resources by using appropriate parameters
and improves the utilization of scarce resources in large clusters. The
behavior of the `RequestedToCapacityRatioResourceAllocation` priority function
can be controlled by a configuration option called `RequestedToCapacityRatioArgs`.
This argument consists of two parameters `shape` and `resources`. The `shape`
parameter allows the user to tune the function as least requested or most
requested based on `utilization` and `score` values. The `resources` parameter
consists of `name` of the resource to be considered during scoring and `weight`
specify the weight of each resource.
-->
## 使用 RequestedToCapacityRatioResourceAllocation 启用装箱
Kubernetes 允许用户指定资源以及每类资源的权重,
以便根据请求数量与可用容量之比率为节点评分。
这就使得用户可以通过使用适当的参数来对扩展资源执行装箱操作,从而提高了大型集群中稀缺资源的利用率。
`RequestedToCapacityRatioResourceAllocation` 优先级函数的行为可以通过名为
`RequestedToCapacityRatioArgs` 的配置选项进行控制。
该标志由两个参数 `shape``resources` 组成。
`shape` 允许用户根据 `utilization``score` 值将函数调整为
最少请求(least requested)或最多请求(most requested)计算。
`resources` 包含由 `name``weight` 组成,`name` 指定评分时要考虑的资源,
`weight` 指定每种资源的权重。
<!--
Below is an example configuration that sets
`requestedToCapacityRatioArguments` to bin packing behavior for extended
resources `intel.com/foo` and `intel.com/bar`.
-->
以下是一个配置示例,该配置将 `requestedToCapacityRatioArguments` 设置为对扩展资源
`intel.com/foo``intel.com/bar` 的装箱行为
```yaml
apiVersion: kubescheduler.config.k8s.io/v1beta3
kind: KubeSchedulerConfiguration
profiles:
# ...
pluginConfig:
- name: RequestedToCapacityRatio
args:
shape:
- utilization: 0
score: 10
- utilization: 100
score: 0
resources:
- name: intel.com/foo
weight: 3
- name: intel.com/bar
weight: 5
```
<!--
Referencing the `KubeSchedulerConfiguration` file with the kube-scheduler
flag `--config=/path/to/config/file` will pass the configuration to the
scheduler.
-->
使用 kube-scheduler 标志 `--config=/path/to/config/file`
引用 `KubeSchedulerConfiguration` 文件将配置传递给调度器。
<!--
**This feature is disabled by default**
-->
**默认情况下此功能处于被禁用状态**
<!--
### Tuning RequestedToCapacityRatioResourceAllocation Priority Function
`shape` is used to specify the behavior of the `RequestedToCapacityRatioPriority` function.
-->
### 调整 RequestedToCapacityRatioResourceAllocation 优先级函数
`shape` 用于指定 `RequestedToCapacityRatioPriority` 函数的行为。
```yaml
shape:
- utilization: 0
score: 0
- utilization: 100
score: 10
```
<!--
The above arguments give the node a score of 0 if utilization is 0% and 10 for utilization 100%, thus enabling bin packing behavior. To enable least requested the score value must be reversed as follows.
-->
上面的参数在 `utilization` 为 0% 时给节点评分为 0,在 `utilization`
100% 时给节点评分为 10,因此启用了装箱行为。
要启用最少请求(least requested)模式,必须按如下方式反转得分值。
```yaml
shape:
- utilization: 0
score: 10
- utilization: 100
score: 0
```
<!--
`resources` is an optional parameter which by defaults is set to:
-->
`resources` 是一个可选参数,默认情况下设置为:
``` yaml
resources:
- name: cpu
weight: 1
- name: memory
weight: 1
```
<!--
It can be used to add extended resources as follows:
-->
它可以用来添加扩展资源,如下所示:
```yaml
resources:
- name: intel.com/foo
weight: 5
- name: cpu
weight: 3
- name: memory
weight: 1
```
<!--
The weight parameter is optional and is set to 1 if not specified. Also, the weight cannot be set to a negative value.
-->
weight 参数是可选的,如果未指定,则设置为 1。
同时,weight 不能设置为负值。
<!--
### Node scoring for capacity allocation
This section is intended for those who want to understand the internal details
of this feature.
Below is an example of how the node score is calculated for a given set of values.
-->
### 节点容量分配的评分
本节适用于希望了解此功能的内部细节的人员。
以下是如何针对给定的一组值来计算节点得分的示例。
```
请求的资源
intel.com/foo : 2
memory: 256MB
cpu: 2
资源权重
intel.com/foo : 5
memory: 1
cpu: 3
FunctionShapePoint {{0, 0}, {100, 10}}
节点 Node 1 配置
可用:
intel.com/foo : 4
memory : 1 GB
cpu: 8
已用:
intel.com/foo: 1
memory: 256MB
cpu: 1
节点得分:
intel.com/foo = resourceScoringFunction((2+1),4)
= (100 - ((4-3)*100/4)
= (100 - 25)
= 75
= rawScoringFunction(75)
= 7
memory = resourceScoringFunction((256+256),1024)
= (100 -((1024-512)*100/1024))
= 50
= rawScoringFunction(50)
= 5
cpu = resourceScoringFunction((2+1),8)
= (100 -((8-3)*100/8))
= 37.5
= rawScoringFunction(37.5)
= 3
NodeScore = (7 * 5) + (5 * 1) + (3 * 3) / (5 + 1 + 3)
= 5
节点 Node 2 配置
可用:
intel.com/foo: 8
memory: 1GB
cpu: 8
已用:
intel.com/foo: 2
memory: 512MB
cpu: 6
节点得分:
intel.com/foo = resourceScoringFunction((2+2),8)
= (100 - ((8-4)*100/8)
= (100 - 50)
= 50
= rawScoringFunction(50)
= 5
memory = resourceScoringFunction((256+512),1024)
= (100 -((1024-768)*100/1024))
= 75
= rawScoringFunction(75)
= 7
cpu = resourceScoringFunction((2+6),8)
= (100 -((8-8)*100/8))
= 100
= rawScoringFunction(100)
= 10
NodeScore = (5 * 5) + (7 * 1) + (10 * 3) / (5 + 1 + 3)
= 7
```
@@ -0,0 +1,303 @@
---
title: 调度器性能调优
content_type: concept
weight: 100
---
<!--
---
reviewers:
- bsalamat
title: Scheduler Performance Tuning
content_type: concept
weight: 100
---
-->
<!-- overview -->
{{< feature-state for_k8s_version="1.14" state="beta" >}}
<!--
[kube-scheduler](/docs/concepts/scheduling-eviction/kube-scheduler/#kube-scheduler)
is the Kubernetes default scheduler. It is responsible for placement of Pods
on Nodes in a cluster.
-->
作为 kubernetes 集群的默认调度器,
[kube-scheduler](/zh/docs/concepts/scheduling-eviction/kube-scheduler/#kube-scheduler)
主要负责将 Pod 调度到集群的 Node 上。
<!--
Nodes in a cluster that meet the scheduling requirements of a Pod are
called _feasible_ Nodes for the Pod. The scheduler finds feasible Nodes
for a Pod and then runs a set of functions to score the feasible Nodes,
picking a Node with the highest score among the feasible ones to run
the Pod. The scheduler then notifies the API server about this decision
in a process called _Binding_.
-->
在一个集群中,满足一个 Pod 调度请求的所有 Node 称之为 _可调度_ Node。
调度器先在集群中找到一个 Pod 的可调度 Node,然后根据一系列函数对这些可调度 Node 打分,
之后选出其中得分最高的 Node 来运行 Pod。
最后,调度器将这个调度决定告知 kube-apiserver,这个过程叫做 _绑定(Binding_
<!--
This page explains performance tuning optimizations that are relevant for
large Kubernetes clusters.
-->
这篇文章将会介绍一些在大规模 Kubernetes 集群下调度器性能优化的方式。
<!-- body -->
<!--
In large clusters, you can tune the scheduler's behaviour balancing
scheduling outcomes between latency (new Pods are placed quickly) and
accuracy (the scheduler rarely makes poor placement decisions).
You configure this tuning setting via kube-scheduler setting
`percentageOfNodesToScore`. This KubeSchedulerConfiguration setting determines
a threshold for scheduling nodes in your cluster.
-->
在大规模集群中,你可以调节调度器的表现来平衡调度的延迟(新 Pod 快速就位)
和精度(调度器很少做出糟糕的放置决策)。
你可以通过设置 kube-scheduler 的 `percentageOfNodesToScore` 来配置这个调优设置。
这个 KubeSchedulerConfiguration 设置决定了调度集群中节点的阈值。
<!--
### Setting the threshold
-->
### 设置阈值
<!--
The `percentageOfNodesToScore` option accepts whole numeric values between 0
and 100. The value 0 is a special number which indicates that the kube-scheduler
should use its compiled-in default.
If you set `percentageOfNodesToScore` above 100, kube-scheduler acts as if you
had set a value of 100.
-->
`percentageOfNodesToScore` 选项接受从 0 到 100 之间的整数值。
0 值比较特殊,表示 kube-scheduler 应该使用其编译后的默认值。
如果你设置 `percentageOfNodesToScore` 的值超过了 100
kube-scheduler 的表现等价于设置值为 100。
<!--
To change the value, edit the
[kube-scheduler configuration file](/docs/reference/config-api/kube-scheduler-config.v1beta2/)
and then restart the scheduler.
In many cases, the configuration file can be found at `/etc/kubernetes/config/kube-scheduler.yaml`
-->
要修改这个值,先编辑 [kube-scheduler 的配置文件](/zh/docs/reference/config-api/kube-scheduler-config.v1beta3/)
然后重启调度器。
大多数情况下,这个配置文件是 `/etc/kubernetes/config/kube-scheduler.yaml`
<!--
After you have made this change, you can run
-->
修改完成后,你可以执行
```bash
kubectl get pods -n kube-system | grep kube-scheduler
```
<!--
to verify that the kube-scheduler component is healthy.
-->
来检查该 kube-scheduler 组件是否健康。
<!--
## Node scoring threshold {#percentage-of-nodes-to-score}
-->
## 节点打分阈值 {#percentage-of-nodes-to-score}
<!--
To improve scheduling performance, the kube-scheduler can stop looking for
feasible nodes once it has found enough of them. In large clusters, this saves
time compared to a naive approach that would consider every node.
-->
要提升调度性能,kube-scheduler 可以在找到足够的可调度节点之后停止查找。
在大规模集群中,比起考虑每个节点的简单方法相比可以节省时间。
<!--
You specify a threshold for how many nodes are enough, as a whole number percentage
of all the nodes in your cluster. The kube-scheduler converts this into an
integer number of nodes. During scheduling, if the kube-scheduler has identified
enough feasible nodes to exceed the configured percentage, the kube-scheduler
stops searching for more feasible nodes and moves on to the
[scoring phase](/docs/concepts/scheduling-eviction/kube-scheduler/#kube-scheduler-implementation).
-->
你可以使用整个集群节点总数的百分比作为阈值来指定需要多少节点就足够。
kube-scheduler 会将它转换为节点数的整数值。在调度期间,如果
kube-scheduler 已确认的可调度节点数足以超过了配置的百分比数量,
kube-scheduler 将停止继续查找可调度节点并继续进行
[打分阶段](/zh/docs/concepts/scheduling-eviction/kube-scheduler/#kube-scheduler-implementation)。
<!--
[How the scheduler iterates over Nodes](#how-the-scheduler-iterates-over-nodes)
describes the process in detail.
-->
[调度器如何遍历节点](#how-the-scheduler-iterates-over-nodes) 详细介绍了这个过程。
<!--
### Default threshold
-->
### 默认阈值
<!--
If you don't specify a threshold, Kubernetes calculates a figure using a
linear formula that yields 50% for a 100-node cluster and yields 10%
for a 5000-node cluster. The lower bound for the automatic value is 5%.
-->
如果你不指定阈值,Kubernetes 使用线性公式计算出一个比例,在 100-节点集群
下取 50%,在 5000-节点的集群下取 10%。这个自动设置的参数的最低值是 5%。
<!--
This means that, the kube-scheduler always scores at least 5% of your cluster no
matter how large the cluster is, unless you have explicitly set
`percentageOfNodesToScore` to be smaller than 5.
-->
这意味着,调度器至少会对集群中 5% 的节点进行打分,除非用户将该参数设置的低于 5。
<!--
If you want the scheduler to score all nodes in your cluster, set
`percentageOfNodesToScore` to 100.
-->
如果你想让调度器对集群内所有节点进行打分,则将 `percentageOfNodesToScore` 设置为 100。
<!--
## Example
-->
## 示例
<!--
Below is an example configuration that sets `percentageOfNodesToScore` to 50%.
-->
下面就是一个将 `percentageOfNodesToScore` 参数设置为 50% 的例子。
```yaml
apiVersion: kubescheduler.config.k8s.io/v1alpha1
kind: KubeSchedulerConfiguration
algorithmSource:
provider: DefaultProvider
...
percentageOfNodesToScore: 50
```
<!--
### Tuning percentageOfNodesToScore
-->
### 调节 percentageOfNodesToScore 参数
<!--
`percentageOfNodesToScore` must be a value between 1 and 100 with the default
value being calculated based on the cluster size. There is also a hardcoded
minimum value of 50 nodes.
-->
`percentageOfNodesToScore` 的值必须在 1 到 100 之间,而且其默认值是通过集群的规模计算得来的。
另外,还有一个 50 个 Node 的最小值是硬编码在程序中。
<!--
{{< note >}} In clusters with less than 50 feasible nodes, the scheduler still
checks all the nodes because there are not enough feasible nodes to stop
the scheduler's search early.
In a small cluster, if you set a low value for `percentageOfNodesToScore`, your
change will have no or little effect, for a similar reason.
If your cluster has several hundred Nodes or fewer, leave this configuration option
at its default value. Making changes is unlikely to improve the
scheduler's performance significantly.
-->
{{< note >}}
当集群中的可调度节点少于 50 个时,调度器仍然会去检查所有的 Node,
因为可调度节点太少,不足以停止调度器最初的过滤选择。
同理,在小规模集群中,如果你将 `percentageOfNodesToScore` 设置为
一个较低的值,则没有或者只有很小的效果。
如果集群只有几百个节点或者更少,请保持这个配置的默认值。
改变基本不会对调度器的性能有明显的提升。
{{< /note >}}
<!--
An important detail to consider when setting this value is that when a smaller
number of nodes in a cluster are checked for feasibility, some nodes are not
sent to be scored for a given Pod. As a result, a Node which could possibly
score a higher value for running the given Pod might not even be passed to the
scoring phase. This would result in a less than ideal placement of the Pod.
You should avoid setting `percentageOfNodesToScore` very low so that kube-scheduler
does not make frequent, poor Pod placement decisions. Avoid setting the
percentage to anything below 10%, unless the scheduler's throughput is critical
for your application and the score of nodes is not important. In other words, you
prefer to run the Pod on any Node as long as it is feasible.
-->
值得注意的是,该参数设置后可能会导致只有集群中少数节点被选为可调度节点,
很多节点都没有进入到打分阶段。这样就会造成一种后果,
一个本来可以在打分阶段得分很高的节点甚至都不能进入打分阶段。
由于这个原因,这个参数不应该被设置成一个很低的值。
通常的做法是不会将这个参数的值设置的低于 10。
很低的参数值一般在调度器的吞吐量很高且对节点的打分不重要的情况下才使用。
换句话说,只有当你更倾向于在可调度节点中任意选择一个节点来运行这个 Pod 时,
才使用很低的参数设置。
<!--
### How the scheduler iterates over Nodes
-->
### 调度器做调度选择的时候如何覆盖所有的 Node {#how-the-scheduler-iterates-over-nodes}
<!--
This section is intended for those who want to understand the internal details
of this feature.
-->
如果你想要理解这一个特性的内部细节,那么请仔细阅读这一章节。
<!--
In order to give all the Nodes in a cluster a fair chance of being considered
for running Pods, the scheduler iterates over the nodes in a round robin
fashion. You can imagine that Nodes are in an array. The scheduler starts from
the start of the array and checks feasibility of the nodes until it finds enough
Nodes as specified by `percentageOfNodesToScore`. For the next Pod, the
scheduler continues from the point in the Node array that it stopped at when
checking feasibility of Nodes for the previous Pod.
-->
在将 Pod 调度到节点上时,为了让集群中所有节点都有公平的机会去运行这些 Pod,
调度器将会以轮询的方式覆盖全部的 Node。
你可以将 Node 列表想象成一个数组。调度器从数组的头部开始筛选可调度节点,
依次向后直到可调度节点的数量达到 `percentageOfNodesToScore` 参数的要求。
在对下一个 Pod 进行调度的时候,前一个 Pod 调度筛选停止的 Node 列表的位置,
将会来作为这次调度筛选 Node 开始的位置。
<!--
If Nodes are in multiple zones, the scheduler iterates over Nodes in various
zones to ensure that Nodes from different zones are considered in the
feasibility checks. As an example, consider six nodes in two zones:
-->
如果集群中的 Node 在多个区域,那么调度器将从不同的区域中轮询 Node,
来确保不同区域的 Node 接受可调度性检查。如下例,考虑两个区域中的六个节点:
```
Zone 1: Node 1, Node 2, Node 3, Node 4
Zone 2: Node 5, Node 6
```
<!--
The Scheduler evaluates feasibility of the nodes in this order:
-->
调度器将会按照如下的顺序去评估 Node 的可调度性:
```
Node 1, Node 5, Node 2, Node 6, Node 3, Node 4
```
<!--
After going over all the Nodes, it goes back to Node 1.
-->
在评估完所有 Node 后,将会返回到 Node 1,从头开始。
## {{% heading "whatsnext" %}}
<!-- * Check the [kube-scheduler configuration reference (v1beta3)](/docs/reference/config-api/kube-scheduler-config.v1beta3/) -->
* 参见 [kube-scheduler 配置参考 (v1beta3)](/zh/docs/reference/config-api/kube-scheduler-config.v1beta3/)
@@ -0,0 +1,475 @@
---
title: 调度框架
content_type: concept
weight: 90
---
<!--
---
reviewers:
- ahg-g
title: Scheduling Framework
content_type: concept
weight: 90
---
-->
<!-- overview -->
{{< feature-state for_k8s_version="1.19" state="stable" >}}
<!--
The scheduling framework is a pluggable architecture for the Kubernetes scheduler.
It adds a new set of "plugin" APIs to the existing scheduler. Plugins are compiled into the scheduler. The APIs allow most scheduling features to be implemented as plugins, while keeping the
scheduling "core" lightweight and maintainable. Refer to the [design proposal of the
scheduling framework][kep] for more technical information on the design of the
framework.
-->
调度框架是面向 Kubernetes 调度器的一种插件架构,
它为现有的调度器添加了一组新的“插件” API。插件会被编译到调度器之中。
这些 API 允许大多数调度功能以插件的形式实现,同时使调度“核心”保持简单且可维护。
请参考[调度框架的设计提案](https://github.com/kubernetes/enhancements/blob/master/keps/sig-scheduling/624-scheduling-framework/README.md)
获取框架设计的更多技术信息。
<!-- body -->
<!--
# Framework workflow
-->
# 框架工作流程
<!--
The Scheduling Framework defines a few extension points. Scheduler plugins
register to be invoked at one or more extension points. Some of these plugins
can change the scheduling decisions and some are informational only.
-->
调度框架定义了一些扩展点。调度器插件注册后在一个或多个扩展点处被调用。
这些插件中的一些可以改变调度决策,而另一些仅用于提供信息。
<!--
Each attempt to schedule one Pod is split into two phases, the **scheduling
cycle** and the **binding cycle**.
-->
每次调度一个 Pod 的尝试都分为两个阶段,即 **调度周期****绑定周期**
<!--
## Scheduling Cycle & Binding Cycle
-->
## 调度周期和绑定周期
<!--
The scheduling cycle selects a node for the Pod, and the binding cycle applies
that decision to the cluster. Together, a scheduling cycle and binding cycle are
referred to as a "scheduling context".
-->
调度周期为 Pod 选择一个节点,绑定周期将该决策应用于集群。
调度周期和绑定周期一起被称为“调度上下文”。
<!--
Scheduling cycles are run serially, while binding cycles may run concurrently.
-->
调度周期是串行运行的,而绑定周期可能是同时运行的。
<!--
A scheduling or binding cycle can be aborted if the Pod is determined to
be unschedulable or if there is an internal error. The Pod will be returned to
the queue and retried.
-->
如果确定 Pod 不可调度或者存在内部错误,则可以终止调度周期或绑定周期。
Pod 将返回队列并重试。
<!--
## Extension points
-->
## 扩展点
<!--
The following picture shows the scheduling context of a Pod and the extension
points that the scheduling framework exposes. In this picture "Filter" is
equivalent to "Predicate" and "Scoring" is equivalent to "Priority function".
-->
下图显示了一个 Pod 的调度上下文以及调度框架公开的扩展点。
在此图片中,“过滤器”等同于“断言”,“评分”相当于“优先级函数”。
<!--
One plugin may register at multiple extension points to perform more complex or
stateful tasks.
-->
一个插件可以在多个扩展点处注册,以执行更复杂或有状态的任务。
<!--
{{< figure src="/images/docs/scheduling-framework-extensions.png" title="scheduling framework extension points" class="diagram-large">}}
-->
{{< figure src="/images/docs/scheduling-framework-extensions.png" title="调度框架扩展点" class="diagram-large">}}
<!--
### QueueSort {#queue-sort}
-->
### 队列排序 {#queue-sort}
<!--
These plugins are used to sort Pods in the scheduling queue. A queue sort plugin
essentially provides a `less(Pod1, Pod2)` function. Only one queue sort
plugin may be enabled at a time.
-->
这些插件用于对调度队列中的 Pod 进行排序。
队列排序插件本质上提供 `less(Pod1, Pod2)` 函数。
一次只能启动一个队列插件。
<!--
### PreFilter {#pre-filter}
-->
### PreFilter {#pre-filter}
<!--
These plugins are used to pre-process info about the Pod, or to check certain
conditions that the cluster or the Pod must meet. If a PreFilter plugin returns
an error, the scheduling cycle is aborted.
-->
这些插件用于预处理 Pod 的相关信息,或者检查集群或 Pod 必须满足的某些条件。
如果 PreFilter 插件返回错误,则调度周期将终止。
<!--
### Filter
-->
### Filter
<!--
These plugins are used to filter out nodes that cannot run the Pod. For each
node, the scheduler will call filter plugins in their configured order. If any
filter plugin marks the node as infeasible, the remaining plugins will not be
called for that node. Nodes may be evaluated concurrently.
-->
这些插件用于过滤出不能运行该 Pod 的节点。对于每个节点,
调度器将按照其配置顺序调用这些过滤插件。如果任何过滤插件将节点标记为不可行,
则不会为该节点调用剩下的过滤插件。节点可以被同时进行评估。
<!--
### PostFilter {#post-filter}
-->
### PostFilter {#post-filter}
<!--
These plugins are called after Filter phase, but only when no feasible nodes
were found for the pod. Plugins are called in their configured order. If
any postFilter plugin marks the node as `Schedulable`, the remaining plugins
will not be called. A typical PostFilter implementation is preemption, which
tries to make the pod schedulable by preempting other Pods.
-->
这些插件在 Filter 阶段后调用,但仅在该 Pod 没有可行的节点时调用。
插件按其配置的顺序调用。如果任何 PostFilter 插件标记节点为“Schedulable”,
则其余的插件不会调用。典型的 PostFilter 实现是抢占,试图通过抢占其他 Pod
的资源使该 Pod 可以调度。
<!--
### PreScore {#pre-score}
-->
### PreScore {#pre-score}
<!--
These plugins are used to perform "pre-scoring" work, which generates a sharable
state for Score plugins to use. If a PreScore plugin returns an error, the
scheduling cycle is aborted.
-->
这些插件用于执行 “前置评分(pre-scoring)” 工作,即生成一个可共享状态供 Score 插件使用。
如果 PreScore 插件返回错误,则调度周期将终止。
<!--
### Score {#scoring}
-->
### Score {#scoring}
<!--
These plugins are used to rank nodes that have passed the filtering phase. The
scheduler will call each scoring plugin for each node. There will be a well
defined range of integers representing the minimum and maximum scores. After the
[NormalizeScore](#normalize-scoring) phase, the scheduler will combine node
scores from all plugins according to the configured plugin weights.
-->
这些插件用于对通过过滤阶段的节点进行排序。调度器将为每个节点调用每个评分插件。
将有一个定义明确的整数范围,代表最小和最大分数。
在[标准化评分](#normalize-scoring)阶段之后,调度器将根据配置的插件权重
合并所有插件的节点分数。
<!--
### NormalizeScore {#normalize-scoring}
-->
### NormalizeScore {#normalize-scoring}
<!--
These plugins are used to modify scores before the scheduler computes a final
ranking of Nodes. A plugin that registers for this extension point will be
called with the [Score](#scoring) results from the same plugin. This is called
once per plugin per scheduling cycle.
-->
这些插件用于在调度器计算 Node 排名之前修改分数。
在此扩展点注册的插件被调用时会使用同一插件的 [Score](#scoring) 结果。
每个插件在每个调度周期调用一次。
<!--
For example, suppose a plugin `BlinkingLightScorer` ranks Nodes based on how
many blinking lights they have.
-->
例如,假设一个 `BlinkingLightScorer` 插件基于具有的闪烁指示灯数量来对节点进行排名。
```go
func ScoreNode(_ *v1.pod, n *v1.Node) (int, error) {
return getBlinkingLightCount(n)
}
```
<!--
However, the maximum count of blinking lights may be small compared to
`NodeScoreMax`. To fix this, `BlinkingLightScorer` should also register for this
extension point.
-->
然而,最大的闪烁灯个数值可能比 `NodeScoreMax` 小。要解决这个问题,
`BlinkingLightScorer` 插件还应该注册该扩展点。
```go
func NormalizeScores(scores map[string]int) {
highest := 0
for _, score := range scores {
highest = max(highest, score)
}
for node, score := range scores {
scores[node] = score*NodeScoreMax/highest
}
}
```
<!--
If any NormalizeScore plugin returns an error, the scheduling cycle is
aborted.
-->
如果任何 NormalizeScore 插件返回错误,则调度阶段将终止。
<!--
Plugins wishing to perform "pre-reserve" work should use the
NormalizeScore extension point.
-->
{{< note >}}
希望执行“预保留”工作的插件应该使用 NormalizeScore 扩展点。
{{< /note >}}
<!--
### Reserve
-->
### Reserve
<!--
This is an informational extension point. Plugins which maintain runtime state
(aka "stateful plugins") should use this extension point to be notified by the
scheduler when resources on a node are being reserved for a given Pod. This
happens before the scheduler actually binds the Pod to the Node, and it exists
to prevent race conditions while the scheduler waits for the bind to succeed.
-->
Reserve 是一个信息性的扩展点。
管理运行时状态的插件(也成为“有状态插件”)应该使用此扩展点,以便
调度器在节点给指定 Pod 预留了资源时能够通知该插件。
这是在调度器真正将 Pod 绑定到节点之前发生的,并且它存在是为了防止
在调度器等待绑定成功时发生竞争情况。
<!--
This is the last step in a scheduling cycle. Once a Pod is in the reserved
state, it will either trigger [Unreserve](#unreserve) plugins (on failure) or
[PostBind](#post-bind) plugins (on success) at the end of the binding cycle.
-->
这个是调度周期的最后一步。
一旦 Pod 处于保留状态,它将在绑定周期结束时触发 [Unreserve](#unreserve) 插件
(失败时)或 [PostBind](#post-bind) 插件(成功时)。
<!--
### Permit
-->
### Permit
<!--
_Permit_ plugins are invoked at the end of the scheduling cycle for each Pod, to
prevent or delay the binding to the candidate node. A permit plugin can do one of
the three things:
-->
_Permit_ 插件在每个 Pod 调度周期的最后调用,用于防止或延迟 Pod 的绑定。
一个允许插件可以做以下三件事之一:
<!--
1. **approve** \
Once all Permit plugins approve a Pod, it is sent for binding.
-->
1. **批准** \
一旦所有 Permit 插件批准 Pod 后,该 Pod 将被发送以进行绑定。
<!--
1. **deny** \
If any Permit plugin denies a Pod, it is returned to the scheduling queue.
This will trigger [Unreserve](#unreserve) plugins.
-->
1. **拒绝** \
如果任何 Permit 插件拒绝 Pod,则该 Pod 将被返回到调度队列。
这将触发[Unreserve](#unreserve) 插件。
<!--
1. **wait** (with a timeout) \
If a Permit plugin returns "wait", then the Pod is kept in an internal "waiting"
Pods list, and the binding cycle of this Pod starts but directly blocks until it
gets [approved](#frameworkhandle). If a timeout occurs, **wait** becomes **deny**
and the Pod is returned to the scheduling queue, triggering [Unreserve](#unreserve)
plugins.
-->
1. **等待**(带有超时) \
如果一个 Permit 插件返回 “等待” 结果,则 Pod 将保持在一个内部的 “等待中”
的 Pod 列表,同时该 Pod 的绑定周期启动时即直接阻塞直到得到
[批准](#frameworkhandle)。如果超时发生,**等待** 变成 **拒绝**,并且 Pod
将返回调度队列,从而触发 [Unreserve](#unreserve) 插件。
<!--
While any plugin can access the list of "waiting" Pods and approve them
(see [`FrameworkHandle`](https://git.k8s.io/enhancements/keps/sig-scheduling/624-scheduling-framework#frameworkhandle)), we expect only the permit
plugins to approve binding of reserved Pods that are in "waiting" state. Once a Pod
is approved, it is sent to the [PreBind](#pre-bind) phase.
-->
{{< note >}}
尽管任何插件可以访问 “等待中” 状态的 Pod 列表并批准它们
(查看 [`FrameworkHandle`](https://git.k8s.io/enhancements/keps/sig-scheduling/624-scheduling-framework#frameworkhandle))。
我们期望只有允许插件可以批准处于 “等待中” 状态的预留 Pod 的绑定。
一旦 Pod 被批准了,它将发送到 [PreBind](#pre-bind) 阶段。
{{< /note >}}
<!--
### Pre-bind {#pre-bind}
-->
### PreBind {#pre-bind}
<!--
These plugins are used to perform any work required before a Pod is bound. For
example, a pre-bind plugin may provision a network volume and mount it on the
target node before allowing the Pod to run there.
-->
这些插件用于执行 Pod 绑定前所需的所有工作。
例如,一个 PreBind 插件可能需要制备网络卷并且在允许 Pod 运行在该节点之前
将其挂载到目标节点上。
<!--
If any PreBind plugin returns an error, the Pod is [rejected](#unreserve) and
returned to the scheduling queue.
-->
如果任何 PreBind 插件返回错误,则 Pod 将被 [拒绝](#unreserve) 并且
退回到调度队列中。
<!--
### Bind
-->
### Bind
<!--
These plugins are used to bind a Pod to a Node. Bind plugins will not be called
until all PreBind plugins have completed. Each bind plugin is called in the
configured order. A bind plugin may choose whether or not to handle the given
Pod. If a bind plugin chooses to handle a Pod, **the remaining bind plugins are
skipped**.
-->
Bind 插件用于将 Pod 绑定到节点上。直到所有的 PreBind 插件都完成,Bind 插件才会被调用。
各 Bind 插件按照配置顺序被调用。Bind 插件可以选择是否处理指定的 Pod。
如果某 Bind 插件选择处理某 Pod,**剩余的 Bind 插件将被跳过**。
<!--
### PostBind {#post-bind}
-->
### PostBind {#post-bind}
<!--
This is an informational extension point. Post-bind plugins are called after a
Pod is successfully bound. This is the end of a binding cycle, and can be used
to clean up associated resources.
-->
这是个信息性的扩展点。
PostBind 插件在 Pod 成功绑定后被调用。这是绑定周期的结尾,可用于清理相关的资源。
<!--
### Unreserve
-->
### Unreserve
<!--
This is an informational extension point. If a Pod was reserved and then
rejected in a later phase, then unreserve plugins will be notified. Unreserve
plugins should clean up state associated with the reserved Pod.
-->
这是个信息性的扩展点。
如果 Pod 被保留,然后在后面的阶段中被拒绝,则 Unreserve 插件将被通知。
Unreserve 插件应该清楚保留 Pod 的相关状态。
<!--
Plugins that use this extension point usually should also use
[Reserve](#reserve).
-->
使用此扩展点的插件通常也使用 [Reserve](#reserve)。
<!--
## Plugin API
-->
## 插件 API
<!--
There are two steps to the plugin API. First, plugins must register and get
configured, then they use the extension point interfaces. Extension point
interfaces have the following form.
-->
插件 API 分为两个步骤。首先,插件必须完成注册并配置,然后才能使用扩展点接口。
扩展点接口具有以下形式。
```go
type Plugin interface {
Name() string
}
type QueueSortPlugin interface {
Plugin
Less(*v1.pod, *v1.pod) bool
}
type PreFilterPlugin interface {
Plugin
PreFilter(context.Context, *framework.CycleState, *v1.pod) error
}
// ...
```
<!--
# Plugin Configuration
-->
# 插件配置
<!--
You can enable or disable plugins in the scheduler configuration. If you are using
Kubernetes v1.18 or later, most scheduling
[plugins](/docs/reference/scheduling/config/#scheduling-plugins) are in use and
enabled by default.
-->
你可以在调度器配置中启用或禁用插件。
如果你在使用 Kubernetes v1.18 或更高版本,大部分调度
[插件](/zh/docs/reference/scheduling/config/#scheduling-plugins)
都在使用中且默认启用。
<!--
In addition to default plugins, you can also implement your own scheduling
plugins and get them configured along with default plugins. You can visit
[scheduler-plugins](https://github.com/kubernetes-sigs/scheduler-plugins) for more details.
-->
除了默认的插件,你还可以实现自己的调度插件并且将它们与默认插件一起配置。
你可以访问 [scheduler-plugins](https://github.com/kubernetes-sigs/scheduler-plugins)
了解更多信息。
<!--
If you are using Kubernetes v1.18 or later, you can configure a set of plugins as
a scheduler profile and then define multiple profiles to fit various kinds of workload.
Learn more at [multiple profiles](/docs/reference/scheduling/config/#multiple-profiles).
-->
如果你正在使用 Kubernetes v1.18 或更高版本,你可以将一组插件设置为
一个调度器配置文件,然后定义不同的配置文件来满足各类工作负载。
了解更多关于[多配置文件](/zh/docs/reference/scheduling/config/#multiple-profiles)。
@@ -0,0 +1,548 @@
---
title: 污点和容忍度
content_type: concept
weight: 40
---
<!-- overview -->
<!--
[_Node affinity_](/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity)
is a property of {{< glossary_tooltip text="Pods" term_id="pod" >}} that *attracts* them to
a set of {{< glossary_tooltip text="nodes" term_id="node" >}} (either as a preference or a
hard requirement). _Taints_ are the opposite -- they allow a node to repel a set of pods.
-->
[_节点亲和性_](/zh/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity)
是 {{< glossary_tooltip text="Pod" term_id="pod" >}} 的一种属性,它使 Pod
被吸引到一类特定的{{< glossary_tooltip text="节点" term_id="node" >}}
(这可能出于一种偏好,也可能是硬性要求)。
_污点_(Taint)则相反——它使节点能够排斥一类特定的 Pod。
<!--
_Tolerations_ are applied to pods, and allow (but do not require) the pods to schedule
onto nodes with matching taints.
Taints and tolerations work together to ensure that pods are not scheduled
onto inappropriate nodes. One or more taints are applied to a node; this
marks that the node should not accept any pods that do not tolerate the taints.
-->
容忍度(Toleration)是应用于 Pod 上的,允许(但并不要求)Pod
调度到带有与之匹配的污点的节点上。
污点和容忍度(Toleration)相互配合,可以用来避免 Pod 被分配到不合适的节点上。
每个节点上都可以应用一个或多个污点,这表示对于那些不能容忍这些污点的 Pod,是不会被该节点接受的。
<!-- body -->
<!--
## Concepts
-->
## 概念
<!--
You add a taint to a node using [kubectl taint](/docs/reference/generated/kubectl/kubectl-commands#taint).
For example,
-->
你可以使用命令 [kubectl taint](/docs/reference/generated/kubectl/kubectl-commands#taint) 给节点增加一个污点。比如,
```shell
kubectl taint nodes node1 key1=value1:NoSchedule
```
<!--
places a taint on node `node1`. The taint has key `key1`, value `value1`, and taint effect `NoSchedule`.
This means that no pod will be able to schedule onto `node1` unless it has a matching toleration.
```shell
kubectl taint nodes node1 key:NoSchedule
```
To remove the taint added by the command above, you can run:
```shell
kubectl taint nodes node1 key1=value1:NoSchedule-
```
-->
给节点 `node1` 增加一个污点,它的键名是 `key1`,键值是 `value1`,效果是 `NoSchedule`
这表示只有拥有和这个污点相匹配的容忍度的 Pod 才能够被分配到 `node1` 这个节点。
若要移除上述命令所添加的污点,你可以执行:
```shell
kubectl taint nodes node1 key1=value1:NoSchedule-
```
<!--
You specify a toleration for a pod in the PodSpec. Both of the following tolerations "match" the
taint created by the `kubectl taint` line above, and thus a pod with either toleration would be able
to schedule onto `node1`:
-->
你可以在 PodSpec 中定义 Pod 的容忍度。
下面两个容忍度均与上面例子中使用 `kubectl taint` 命令创建的污点相匹配,
因此如果一个 Pod 拥有其中的任何一个容忍度都能够被分配到 `node1`
```yaml
tolerations:
- key: "key1"
operator: "Equal"
value: "value1"
effect: "NoSchedule"
```
```yaml
tolerations:
- key: "key1"
operator: "Exists"
effect: "NoSchedule"
```
<!--
Heres an example of a pod that uses tolerations:
-->
这里是一个使用了容忍度的 Pod
{{< codenew file="pods/pod-with-toleration.yaml" >}}
<!--
The default value for `operator` is `Equal`.
-->
`operator` 的默认值是 `Equal`
<!--
A toleration "matches" a taint if the keys are the same and the effects are the same, and:
* the `operator` is `Exists` (in which case no `value` should be specified), or
* the `operator` is `Equal` and the `value`s are equal
-->
一个容忍度和一个污点相“匹配”是指它们有一样的键名和效果,并且:
* 如果 `operator``Exists` (此时容忍度不能指定 `value`),或者
* 如果 `operator``Equal` ,则它们的 `value` 应该相等
<!--
There are two special cases:
An empty `key` with operator `Exists` matches all keys, values and effects which means this
will tolerate everything.
An empty `effect` matches all effects with key `key1`.
-->
{{< note >}}
存在两种特殊情况:
如果一个容忍度的 `key` 为空且 operator 为 `Exists`
表示这个容忍度与任意的 key 、value 和 effect 都匹配,即这个容忍度能容忍任意 taint。
如果 `effect` 为空,则可以与所有键名 `key1` 的效果相匹配。
{{< /note >}}
<!--
The above example used `effect` of `NoSchedule`. Alternatively, you can use `effect` of `PreferNoSchedule`.
This is a "preference" or "soft" version of `NoSchedule` - the system will *try* to avoid placing a
pod that does not tolerate the taint on the node, but it is not required. The third kind of `effect` is
`NoExecute`, described later.
-->
上述例子中 `effect` 使用的值为 `NoSchedule`,你也可以使用另外一个值 `PreferNoSchedule`
这是“优化”或“软”版本的 `NoSchedule` —— 系统会 *尽量* 避免将 Pod 调度到存在其不能容忍污点的节点上,
但这不是强制的。`effect` 的值还可以设置为 `NoExecute`,下文会详细描述这个值。
<!--
You can put multiple taints on the same node and multiple tolerations on the same pod.
The way Kubernetes processes multiple taints and tolerations is like a filter: start
with all of a node's taints, then ignore the ones for which the pod has a matching toleration; the
remaining un-ignored taints have the indicated effects on the pod. In particular,
-->
你可以给一个节点添加多个污点,也可以给一个 Pod 添加多个容忍度设置。
Kubernetes 处理多个污点和容忍度的过程就像一个过滤器:从一个节点的所有污点开始遍历,
过滤掉那些 Pod 中存在与之相匹配的容忍度的污点。余下未被过滤的污点的 effect 值决定了
Pod 是否会被分配到该节点,特别是以下情况:
<!--
* if there is at least one un-ignored taint with effect `NoSchedule` then Kubernetes will not schedule
the pod onto that node
* if there is no un-ignored taint with effect `NoSchedule` but there is at least one un-ignored taint with
effect `PreferNoSchedule` then Kubernetes will *try* to not schedule the pod onto the node
* if there is at least one un-ignored taint with effect `NoExecute` then the pod will be evicted from
the node (if it is already running on the node), and will not be
scheduled onto the node (if it is not yet running on the node).
-->
* 如果未被过滤的污点中存在至少一个 effect 值为 `NoSchedule` 的污点,
则 Kubernetes 不会将 Pod 分配到该节点。
* 如果未被过滤的污点中不存在 effect 值为 `NoSchedule` 的污点,
但是存在 effect 值为 `PreferNoSchedule` 的污点,
则 Kubernetes 会 *尝试* 不将 Pod 分配到该节点。
* 如果未被过滤的污点中存在至少一个 effect 值为 `NoExecute` 的污点,
则 Kubernetes 不会将 Pod 分配到该节点(如果 Pod 还未在节点上运行),
或者将 Pod 从该节点驱逐(如果 Pod 已经在节点上运行)。
<!--
For example, imagine you taint a node like this
-->
例如,假设你给一个节点添加了如下污点
```shell
kubectl taint nodes node1 key1=value1:NoSchedule
kubectl taint nodes node1 key1=value1:NoExecute
kubectl taint nodes node1 key2=value2:NoSchedule
```
<!--
And a pod has two tolerations:
-->
假定有一个 Pod,它有两个容忍度:
```yaml
tolerations:
- key: "key1"
operator: "Equal"
value: "value1"
effect: "NoSchedule"
- key: "key1"
operator: "Equal"
value: "value1"
effect: "NoExecute"
```
<!--
In this case, the pod will not be able to schedule onto the node, because there is no
toleration matching the third taint. But it will be able to continue running if it is
already running on the node when the taint is added, because the third taint is the only
one of the three that is not tolerated by the pod.
-->
在这种情况下,上述 Pod 不会被分配到上述节点,因为其没有容忍度和第三个污点相匹配。
但是如果在给节点添加上述污点之前,该 Pod 已经在上述节点运行,
那么它还可以继续运行在该节点上,因为第三个污点是三个污点中唯一不能被这个 Pod 容忍的。
<!--
Normally, if a taint with effect `NoExecute` is added to a node, then any pods that do
not tolerate the taint will be evicted immediately, and any pods that do tolerate the
taint will never be evicted. However, a toleration with `NoExecute` effect can specify
an optional `tolerationSeconds` field that dictates how long the pod will stay bound
to the node after the taint is added. For example,
-->
通常情况下,如果给一个节点添加了一个 effect 值为 `NoExecute` 的污点,
则任何不能忍受这个污点的 Pod 都会马上被驱逐,
任何可以忍受这个污点的 Pod 都不会被驱逐。
但是,如果 Pod 存在一个 effect 值为 `NoExecute` 的容忍度指定了可选属性
`tolerationSeconds` 的值,则表示在给节点添加了上述污点之后,
Pod 还能继续在节点上运行的时间。例如,
```yaml
tolerations:
- key: "key1"
operator: "Equal"
value: "value1"
effect: "NoExecute"
tolerationSeconds: 3600
```
<!--
means that if this pod is running and a matching taint is added to the node, then
the pod will stay bound to the node for 3600 seconds, and then be evicted. If the
taint is removed before that time, the pod will not be evicted.
-->
这表示如果这个 Pod 正在运行,同时一个匹配的污点被添加到其所在的节点,
那么 Pod 还将继续在节点上运行 3600 秒,然后被驱逐。
如果在此之前上述污点被删除了,则 Pod 不会被驱逐。
<!--
## Example Use Cases
-->
## 使用例子
<!--
Taints and tolerations are a flexible way to steer pods *away* from nodes or evict
pods that shouldn't be running. A few of the use cases are
-->
通过污点和容忍度,可以灵活地让 Pod **避开** 某些节点或者将 Pod 从某些节点驱逐。下面是几个使用例子:
<!--
* **Dedicated Nodes**: If you want to dedicate a set of nodes for exclusive use by
a particular set of users, you can add a taint to those nodes (say,
`kubectl taint nodes nodename dedicated=groupName:NoSchedule`) and then add a corresponding
toleration to their pods (this would be done most easily by writing a custom
[admission controller](/docs/reference/access-authn-authz/admission-controllers/)).
The pods with the tolerations will then be allowed to use the tainted (dedicated) nodes as
well as any other nodes in the cluster. If you want to dedicate the nodes to them *and*
ensure they *only* use the dedicated nodes, then you should additionally add a label similar
to the taint to the same set of nodes (e.g. `dedicated=groupName`), and the admission
controller should additionally add a node affinity to require that the pods can only schedule
onto nodes labeled with `dedicated=groupName`.
-->
* **专用节点**:如果你想将某些节点专门分配给特定的一组用户使用,你可以给这些节点添加一个污点(即,
`kubectl taint nodes nodename dedicated=groupName:NoSchedule`),
然后给这组用户的 Pod 添加一个相对应的 toleration(通过编写一个自定义的
[准入控制器](/zh/docs/reference/access-authn-authz/admission-controllers/),很容易就能做到)。
拥有上述容忍度的 Pod 就能够被分配到上述专用节点,同时也能够被分配到集群中的其它节点。
如果你希望这些 Pod 只能被分配到上述专用节点,那么你还需要给这些专用节点另外添加一个和上述
污点类似的 label (例如:`dedicated=groupName`),同时 还要在上述准入控制器中给 Pod
增加节点亲和性要求上述 Pod 只能被分配到添加了 `dedicated=groupName` 标签的节点上。
<!--
* **Nodes with Special Hardware**: In a cluster where a small subset of nodes have specialized
hardware (for example GPUs), it is desirable to keep pods that don't need the specialized
hardware off of those nodes, thus leaving room for later-arriving pods that do need the
specialized hardware. This can be done by tainting the nodes that have the specialized
hardware (e.g. `kubectl taint nodes nodename special=true:NoSchedule` or
`kubectl taint nodes nodename special=true:PreferNoSchedule`) and adding a corresponding
toleration to pods that use the special hardware. As in the dedicated nodes use case,
it is probably easiest to apply the tolerations using a custom
[admission controller](/docs/reference/access-authn-authz/admission-controllers/).
For example, it is recommended to use [Extended
Resources](/docs/concepts/configuration/manage-compute-resources-container/#extended-resources)
to represent the special hardware, taint your special hardware nodes with the
extended resource name and run the
[ExtendedResourceToleration](/docs/reference/access-authn-authz/admission-controllers/#extendedresourcetoleration)
admission controller. Now, because the nodes are tainted, no pods without the
toleration will schedule on them. But when you submit a pod that requests the
extended resource, the `ExtendedResourceToleration` admission controller will
automatically add the correct toleration to the pod and that pod will schedule
on the special hardware nodes. This will make sure that these special hardware
nodes are dedicated for pods requesting such hardware and you don't have to
manually add tolerations to your pods.
-->
* **配备了特殊硬件的节点**:在部分节点配备了特殊硬件(比如 GPU)的集群中,
我们希望不需要这类硬件的 Pod 不要被分配到这些特殊节点,以便为后继需要这类硬件的 Pod 保留资源。
要达到这个目的,可以先给配备了特殊硬件的节点添加 taint
(例如 `kubectl taint nodes nodename special=true:NoSchedule`
`kubectl taint nodes nodename special=true:PreferNoSchedule`)
然后给使用了这类特殊硬件的 Pod 添加一个相匹配的 toleration。
和专用节点的例子类似,添加这个容忍度的最简单的方法是使用自定义
[准入控制器](/zh/docs/reference/access-authn-authz/admission-controllers/)。
比如,我们推荐使用[扩展资源](/zh/docs/concepts/configuration/manage-resources-containers/#extended-resources)
来表示特殊硬件,给配置了特殊硬件的节点添加污点时包含扩展资源名称,
然后运行一个 [ExtendedResourceToleration](/zh/docs/reference/access-authn-authz/admission-controllers/#extendedresourcetoleration)
准入控制器。此时,因为节点已经被设置污点了,没有对应容忍度的 Pod
不会被调度到这些节点。但当你创建一个使用了扩展资源的 Pod 时,
`ExtendedResourceToleration` 准入控制器会自动给 Pod 加上正确的容忍度,
这样 Pod 就会被自动调度到这些配置了特殊硬件件的节点上。
这样就能够确保这些配置了特殊硬件的节点专门用于运行需要使用这些硬件的 Pod,
并且你无需手动给这些 Pod 添加容忍度。
<!--
* **Taint based Evictions**: A per-pod-configurable eviction behavior
when there are node problems, which is described in the next section.
-->
* **基于污点的驱逐**: 这是在每个 Pod 中配置的在节点出现问题时的驱逐行为,接下来的章节会描述这个特性。
<!--
## Taint based Evictions
-->
## 基于污点的驱逐 {#taint-based-evictions}
{{< feature-state for_k8s_version="v1.18" state="stable" >}}
<!--
The `NoExecute` taint effect, which affects pods that are already
running on the node as follows
* pods that do not tolerate the taint are evicted immediately
* pods that tolerate the taint without specifying `tolerationSeconds` in
their toleration specification remain bound forever
* pods that tolerate the taint with a specified `tolerationSeconds` remain
bound for the specified amount of time
-->
前文提到过污点的 effect 值 `NoExecute` 会影响已经在节点上运行的 Pod
* 如果 Pod 不能忍受 effect 值为 `NoExecute` 的污点,那么 Pod 将马上被驱逐
* 如果 Pod 能够忍受 effect 值为 `NoExecute` 的污点,但是在容忍度定义中没有指定
`tolerationSeconds`,则 Pod 还会一直在这个节点上运行。
* 如果 Pod 能够忍受 effect 值为 `NoExecute` 的污点,而且指定了 `tolerationSeconds`
则 Pod 还能在这个节点上继续运行这个指定的时间长度。
<!--
The node controller automatically taints a node when certain conditions are
true. The following taints are built in:
* `node.kubernetes.io/not-ready`: Node is not ready. This corresponds to
the NodeCondition `Ready` being "`False`".
* `node.kubernetes.io/unreachable`: Node is unreachable from the node
controller. This corresponds to the NodeCondition `Ready` being "`Unknown`".
* `node.kubernetes.io/memory-pressure`: Node has memory pressure.
* `node.kubernetes.io/disk-pressure`: Node has disk pressure.
* `node.kubernetes.io/pid-pressure`: Node has PID pressure.
* `node.kubernetes.io/network-unavailable`: Node's network is unavailable.
* `node.kubernetes.io/unschedulable`: Node is unschedulable.
* `node.cloudprovider.kubernetes.io/uninitialized`: When the kubelet is started
with "external" cloud provider, this taint is set on a node to mark it
as unusable. After a controller from the cloud-controller-manager initializes
this node, the kubelet removes this taint.
-->
当某种条件为真时,节点控制器会自动给节点添加一个污点。当前内置的污点包括:
* `node.kubernetes.io/not-ready`:节点未准备好。这相当于节点状态 `Ready` 的值为 "`False`"。
* `node.kubernetes.io/unreachable`:节点控制器访问不到节点. 这相当于节点状态 `Ready` 的值为 "`Unknown`"。
* `node.kubernetes.io/memory-pressure`:节点存在内存压力。
* `node.kubernetes.io/disk-pressure`:节点存在磁盘压力。
* `node.kubernetes.io/pid-pressure`: 节点的 PID 压力。
* `node.kubernetes.io/network-unavailable`:节点网络不可用。
* `node.kubernetes.io/unschedulable`: 节点不可调度。
* `node.cloudprovider.kubernetes.io/uninitialized`:如果 kubelet 启动时指定了一个 "外部" 云平台驱动,
它将给当前节点添加一个污点将其标志为不可用。在 cloud-controller-manager
的一个控制器初始化这个节点后,kubelet 将删除这个污点。
<!--
In case a node is to be evicted, the node controller or the kubelet adds relevant taints
with `NoExecute` effect. If the fault condition returns to normal the kubelet or node
controller can remove the relevant taint(s).
-->
在节点被驱逐时,节点控制器或者 kubelet 会添加带有 `NoExecute` 效应的相关污点。
如果异常状态恢复正常,kubelet 或节点控制器能够移除相关的污点。
<!--
To maintain the existing [rate limiting](/docs/concepts/architecture/nodes/)
behavior of pod evictions due to node problems, the system actually adds the taints
in a rate-limited way. This prevents massive pod evictions in scenarios such
as the master becoming partitioned from the nodes.
-->
{{< note >}}
为了保证由于节点问题引起的 Pod 驱逐
[速率限制](/zh/docs/concepts/architecture/nodes/)行为正常,
系统实际上会以限定速率的方式添加污点。在像主控节点与工作节点间通信中断等场景下,
这样做可以避免 Pod 被大量驱逐。
{{< /note >}}
<!--
This feature, in combination with `tolerationSeconds`, allows a pod
to specify how long it should stay bound to a node that has one or both of these problems.
-->
使用这个功能特性,结合 `tolerationSeconds`Pod 就可以指定当节点出现一个
或全部上述问题时还将在这个节点上运行多长的时间。
<!--
For example, an application with a lot of local state might want to stay
bound to node for a long time in the event of network partition, in the hope
that the partition will recover and thus the pod eviction can be avoided.
The toleration the pod would use in that case would look like
-->
比如,一个使用了很多本地状态的应用程序在网络断开时,仍然希望停留在当前节点上运行一段较长的时间,
愿意等待网络恢复以避免被驱逐。在这种情况下,Pod 的容忍度可能是下面这样的:
```yaml
tolerations:
- key: "node.kubernetes.io/unreachable"
operator: "Exists"
effect: "NoExecute"
tolerationSeconds: 6000
```
<!--
Note that Kubernetes automatically adds a toleration for
`node.kubernetes.io/not-ready` with `tolerationSeconds=300`
unless the pod configuration provided
by the user already has a toleration for `node.kubernetes.io/not-ready`.
Likewise it adds a toleration for
`node.kubernetes.io/unreachable` with `tolerationSeconds=300`
unless the pod configuration provided
by the user already has a toleration for `node.kubernetes.io/unreachable`.
-->
{{< note >}}
Kubernetes 会自动给 Pod 添加一个 key 为 `node.kubernetes.io/not-ready` 的容忍度
并配置 `tolerationSeconds=300`,除非用户提供的 Pod 配置中已经已存在了 key 为
`node.kubernetes.io/not-ready` 的容忍度。
同样,Kubernetes 会给 Pod 添加一个 key 为 `node.kubernetes.io/unreachable` 的容忍度
并配置 `tolerationSeconds=300`,除非用户提供的 Pod 配置中已经已存在了 key 为
`node.kubernetes.io/unreachable` 的容忍度。
{{< /note >}}
<!--
These automatically-added tolerations mean that Pods remain bound to
Nodes for 5 minutes after one of these problems is detected.
-->
这种自动添加的容忍度意味着在其中一种问题被检测到时 Pod
默认能够继续停留在当前节点运行 5 分钟。
<!--
[DaemonSet](/docs/concepts/workloads/controllers/daemonset/) pods are created with
`NoExecute` tolerations for the following taints with no `tolerationSeconds`:
* `node.kubernetes.io/unreachable`
* `node.kubernetes.io/not-ready`
This ensures that DaemonSet pods are never evicted due to these problems.
-->
[DaemonSet](/zh/docs/concepts/workloads/controllers/daemonset/) 中的 Pod 被创建时,
针对以下污点自动添加的 `NoExecute` 的容忍度将不会指定 `tolerationSeconds`
* `node.kubernetes.io/unreachable`
* `node.kubernetes.io/not-ready`
这保证了出现上述问题时 DaemonSet 中的 Pod 永远不会被驱逐。
<!--
## Taint Nodes by Condition
-->
## 基于节点状态添加污点
<!--
The control plane, using the node {{<glossary_tooltip text="controller" term_id="controller">}},
automatically creates taints with a `NoSchedule` effect for [node conditions](/docs/concepts/scheduling-eviction/node-pressure-eviction/#node-conditions).
The scheduler checks taints, not node conditions, when it makes scheduling
decisions. This ensures that node conditions don't directly affect scheduling.
For example, if the `DiskPressure` node condition is active, the control plane
adds the `node.kubernetes.io/disk-pressure` taint and does not schedule new pods
onto the affected node. If the `MemoryPressure` node condition is active, the
control plane adds the `node.kubernetes.io/memory-pressure` taint.
-->
控制平面使用节点{{<glossary_tooltip text="控制器" term_id="controller">}}自动创建
与[节点状况](/zh/docs/concepts/scheduling-eviction/node-pressure-eviction/#node-conditions)对应的带有 `NoSchedule` 效应的污点。
调度器在进行调度时检查污点,而不是检查节点状况。这确保节点状况不会直接影响调度。
例如,如果 `DiskPressure` 节点状况处于活跃状态,则控制平面
添加 `node.kubernetes.io/disk-pressure` 污点并且不会调度新的 pod
到受影响的节点。如果 `MemoryPressure` 节点状况处于活跃状态,则
控制平面添加 `node.kubernetes.io/memory-pressure` 污点。
<!--
You can ignore node conditions for newly created pods by adding the corresponding
Pod tolerations. The control plane also adds the `node.kubernetes.io/memory-pressure`
toleration on pods that have a {{< glossary_tooltip text="QoS class" term_id="qos-class" >}}
other than `BestEffort`. This is because Kubernetes treats pods in the `Guaranteed`
or `Burstable` QoS classes (even pods with no memory request set) as if they are
able to cope with memory pressure, while new `BestEffort` pods are not scheduled
onto the affected node.
-->
对于新创建的 Pod,可以通过添加相应的 Pod 容忍度来忽略节点状况。
控制平面还在具有除 `BestEffort` 之外的 {{<glossary_tooltip text="QoS 类" term_id="qos-class" >}}的 Pod 上
添加 `node.kubernetes.io/memory-pressure` 容忍度。
这是因为 Kubernetes 将 `Guaranteed``Burstable` QoS 类中的 Pod(甚至没有设置内存请求的 Pod)
视为能够应对内存压力,而新创建的 `BestEffort` Pod 不会被调度到受影响的节点上。
<!--
The DaemonSet controller automatically adds the
following `NoSchedule` tolerations to all daemons, to prevent DaemonSets from
breaking.
* `node.kubernetes.io/memory-pressure`
* `node.kubernetes.io/disk-pressure`
* `node.kubernetes.io/pid-pressure` (1.14 or later)
* `node.kubernetes.io/unschedulable` (1.10 or later)
* `node.kubernetes.io/network-unavailable` (*host network only*)
-->
DaemonSet 控制器自动为所有守护进程添加如下 `NoSchedule` 容忍度以防 DaemonSet 崩溃:
* `node.kubernetes.io/memory-pressure`
* `node.kubernetes.io/disk-pressure`
* `node.kubernetes.io/pid-pressure` (1.14 或更高版本)
* `node.kubernetes.io/unschedulable` (1.10 或更高版本)
* `node.kubernetes.io/network-unavailable` (**只适合主机网络配置**)
<!--
Adding these tolerations ensures backward compatibility. You can also add
arbitrary tolerations to DaemonSets.
-->
添加上述容忍度确保了向后兼容,你也可以选择自由向 DaemonSet 添加容忍度。
## {{% heading "whatsnext" %}}
<!--
* Read about [Node-pressure Eviction](/docs/concepts/scheduling-eviction/node-pressure-eviction/) and how you can configure it
* Read about [Pod Priority](/docs/concepts/scheduling-eviction/pod-priority-preemption/)
-->
* 阅读[节点压力驱逐](/zh/docs/concepts/scheduling-eviction/pod-priority-preemption/),以及如何配置其行为
* 阅读 [Pod 优先级](/zh/docs/concepts/scheduling-eviction/pod-priority-preemption/)