|
|
|
@@ -1,30 +1,41 @@
|
|
|
|
|
---
|
|
|
|
|
title: Scheduling Profiles
|
|
|
|
|
title: Scheduler Configuration
|
|
|
|
|
content_type: concept
|
|
|
|
|
weight: 20
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
{{< feature-state for_k8s_version="v1.19" state="beta" >}}
|
|
|
|
|
|
|
|
|
|
You can customize the behavior of the `kube-scheduler` by writing a configuration
|
|
|
|
|
file and passing its path as a command line argument.
|
|
|
|
|
|
|
|
|
|
<!-- overview -->
|
|
|
|
|
|
|
|
|
|
{{< feature-state for_k8s_version="v1.18" state="alpha" >}}
|
|
|
|
|
|
|
|
|
|
A scheduling Profile allows you to configure the different stages of scheduling
|
|
|
|
|
in the {{< glossary_tooltip text="kube-scheduler" term_id="kube-scheduler" >}}.
|
|
|
|
|
Each stage is exposed in a extension point. Plugins provide scheduling behaviors
|
|
|
|
|
by implementing one or more of these extension points.
|
|
|
|
|
|
|
|
|
|
You can specify scheduling profiles by running `kube-scheduler --config <filename>`,
|
|
|
|
|
using the component config APIs
|
|
|
|
|
([`v1alpha1`](https://pkg.go.dev/k8s.io/kube-scheduler@{{< param "fullversion" >}}/config/v1alpha1?tab=doc#KubeSchedulerConfiguration)
|
|
|
|
|
or [`v1alpha2`](https://pkg.go.dev/k8s.io/kube-scheduler@{{< param "fullversion" >}}/config/v1alpha2?tab=doc#KubeSchedulerConfiguration)).
|
|
|
|
|
The `v1alpha2` API allows you to configure kube-scheduler to run
|
|
|
|
|
[multiple profiles](#multiple-profiles).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<!-- body -->
|
|
|
|
|
|
|
|
|
|
## Extension points
|
|
|
|
|
## Minimal Configuration
|
|
|
|
|
|
|
|
|
|
A minimal configuration looks as follows:
|
|
|
|
|
|
|
|
|
|
```yaml
|
|
|
|
|
apiVersion: kubescheduler.config.k8s.io/v1beta1
|
|
|
|
|
kind: KubeSchedulerConfiguration
|
|
|
|
|
clientConnection:
|
|
|
|
|
kubeconfig: /etc/srv/kubernetes/kube-scheduler/kubeconfig
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
## Profiles
|
|
|
|
|
|
|
|
|
|
A scheduling Profile allows you to configure the different stages of scheduling
|
|
|
|
|
in the {{< glossary_tooltip text="kube-scheduler" term_id="kube-scheduler" >}}.
|
|
|
|
|
Each stage is exposed in an [extension point](#extension-points).
|
|
|
|
|
[Plugins](#scheduling-plugins) provide scheduling behaviors by implementing one
|
|
|
|
|
or more of these extension points.
|
|
|
|
|
|
|
|
|
|
You can configure a single instance of `kube-scheduler` to run
|
|
|
|
|
[multiple profiles](#multiple-profiles).
|
|
|
|
|
|
|
|
|
|
### Extension points
|
|
|
|
|
|
|
|
|
|
Scheduling happens in a series of stages that are exposed through the following
|
|
|
|
|
extension points:
|
|
|
|
@@ -33,17 +44,21 @@ extension points:
|
|
|
|
|
sort pending Pods in the scheduling queue. Exactly one queue sort plugin
|
|
|
|
|
may be enabled at a time.
|
|
|
|
|
1. `PreFilter`: These plugins are used to pre-process or check information
|
|
|
|
|
about a Pod or the cluster before filtering.
|
|
|
|
|
about a Pod or the cluster before filtering. They can mark a pod as
|
|
|
|
|
unschedulable.
|
|
|
|
|
1. `Filter`: These plugins are the equivalent of Predicates in a scheduling
|
|
|
|
|
Policy and are used to filter out nodes that can not run the Pod. Filters
|
|
|
|
|
are called in the configured order.
|
|
|
|
|
are called in the configured order. A pod is marked as unschedulable if no
|
|
|
|
|
nodes pass all the filters.
|
|
|
|
|
1. `PreScore`: This is an informational extension point that can be used
|
|
|
|
|
for doing pre-scoring work.
|
|
|
|
|
1. `Score`: These plugins provide a score to each node that has passed the
|
|
|
|
|
filtering phase. The scheduler will then select the node with the highest
|
|
|
|
|
weighted scores sum.
|
|
|
|
|
1. `Reserve`: This is an informational extension point that notifies plugins
|
|
|
|
|
when resources have being reserved for a given Pod.
|
|
|
|
|
when resources have been reserved for a given Pod. Plugins also implement an
|
|
|
|
|
`Unreserve` call that gets called in the case of failure during or after
|
|
|
|
|
`Reserve`.
|
|
|
|
|
1. `Permit`: These plugins can prevent or delay the binding of a Pod.
|
|
|
|
|
1. `PreBind`: These plugins perform any work required before a Pod is bound.
|
|
|
|
|
1. `Bind`: The plugins bind a Pod to a Node. Bind plugins are called in order
|
|
|
|
@@ -51,15 +66,35 @@ extension points:
|
|
|
|
|
least one bind plugin is required.
|
|
|
|
|
1. `PostBind`: This is an informational extension point that is called after
|
|
|
|
|
a Pod has been bound.
|
|
|
|
|
1. `UnReserve`: This is an informational extension point that is called if
|
|
|
|
|
a Pod is rejected after being reserved and put on hold by a `Permit` plugin.
|
|
|
|
|
|
|
|
|
|
For each extension point, you could disable specific [default plugins](#scheduling-plugins)
|
|
|
|
|
or enable your own. For example:
|
|
|
|
|
|
|
|
|
|
```yaml
|
|
|
|
|
apiVersion: kubescheduler.config.k8s.io/v1beta1
|
|
|
|
|
kind: KubeSchedulerConfiguration
|
|
|
|
|
profiles:
|
|
|
|
|
- plugins:
|
|
|
|
|
score:
|
|
|
|
|
disabled:
|
|
|
|
|
- name: NodeResourcesLeastAllocated
|
|
|
|
|
enabled:
|
|
|
|
|
- name: MyCustomPluginA
|
|
|
|
|
weight: 2
|
|
|
|
|
- name: MyCustomPluginB
|
|
|
|
|
weight: 1
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
You can use `*` as name in the disabled array to disable all default plugins
|
|
|
|
|
for that extension point. This can also be used to rearrange plugins order, if
|
|
|
|
|
desired.
|
|
|
|
|
|
|
|
|
|
## Scheduling plugins
|
|
|
|
|
### Scheduling plugins
|
|
|
|
|
|
|
|
|
|
The following plugins, enabled by default, implement one or more of these
|
|
|
|
|
extension points:
|
|
|
|
|
|
|
|
|
|
- `DefaultTopologySpread`: Favors spreading across nodes for Pods that belong to
|
|
|
|
|
- `SelectorSpread`: Favors spreading across nodes for Pods that belong to
|
|
|
|
|
{{< glossary_tooltip text="Services" term_id="service" >}},
|
|
|
|
|
{{< glossary_tooltip text="ReplicaSets" term_id="replica-set" >}} and
|
|
|
|
|
{{< glossary_tooltip text="StatefulSets" term_id="statefulset" >}}
|
|
|
|
@@ -99,7 +134,7 @@ extension points:
|
|
|
|
|
Extension points: `Score`.
|
|
|
|
|
- `VolumeBinding`: Checks if the node has or if it can bind the requested
|
|
|
|
|
{{< glossary_tooltip text="volumes" term_id="volume" >}}.
|
|
|
|
|
Extension points: `Filter`.
|
|
|
|
|
Extension points: `PreFilter`, `Filter`, `Reserve`, `PreBind`.
|
|
|
|
|
- `VolumeRestrictions`: Checks that volumes mounted in the node satisfy
|
|
|
|
|
restrictions that are specific to the volume provider.
|
|
|
|
|
Extension points: `Filter`.
|
|
|
|
@@ -147,10 +182,31 @@ that are not enabled by default:
|
|
|
|
|
Service across nodes.
|
|
|
|
|
Extension points: `PreFilter`, `Filter`, `Score`.
|
|
|
|
|
|
|
|
|
|
## Multiple profiles
|
|
|
|
|
### Multiple profiles
|
|
|
|
|
|
|
|
|
|
You can configure `kube-scheduler` to run more than one profile.
|
|
|
|
|
Each profile has an associated scheduler name and can have a different set of
|
|
|
|
|
plugins configured in its [extension points](#extension-points).
|
|
|
|
|
|
|
|
|
|
With the following sample configuration, the scheduler will run with two
|
|
|
|
|
profiles: one with the default plugins and one with all scoring plugins
|
|
|
|
|
disabled.
|
|
|
|
|
|
|
|
|
|
```yaml
|
|
|
|
|
apiVersion: kubescheduler.config.k8s.io/v1beta1
|
|
|
|
|
kind: KubeSchedulerConfiguration
|
|
|
|
|
profiles:
|
|
|
|
|
- schedulerName: default-scheduler
|
|
|
|
|
- schedulerName: no-scoring-scheduler
|
|
|
|
|
plugins:
|
|
|
|
|
prescore:
|
|
|
|
|
disabled:
|
|
|
|
|
- name: '*'
|
|
|
|
|
score:
|
|
|
|
|
disabled:
|
|
|
|
|
- name: '*'
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
When using the component config API v1alpha2, a scheduler can be configured to
|
|
|
|
|
run more than one profile. Each profile has an associated scheduler name.
|
|
|
|
|
Pods that want to be scheduled according to a specific profile can include
|
|
|
|
|
the corresponding scheduler name in its `.spec.schedulerName`.
|
|
|
|
|
|
|
|
|
@@ -174,9 +230,24 @@ the same configuration parameters (if applicable). This is because the scheduler
|
|
|
|
|
only has one pending pods queue.
|
|
|
|
|
{{< /note >}}
|
|
|
|
|
|
|
|
|
|
## Upgrading from `v1alpha2` to `v1beta1` {#beta-changes}
|
|
|
|
|
|
|
|
|
|
When migrating from `kubescheduler.config.k8s.io/v1alpha2` to `kubescheduler.config.k8s.io/v1beta1`,
|
|
|
|
|
beware of the following changes, if applicable:
|
|
|
|
|
|
|
|
|
|
- `.bindTimeoutSeconds` was moved as part of plugin args for `VolumeBinding`,
|
|
|
|
|
which can be configured separately per [profile](#profiles).
|
|
|
|
|
- `.extenders` are updated to satisfy API standards. In particular:
|
|
|
|
|
- `.extenders` decoding is case sensitive. All fields are affected.
|
|
|
|
|
- `.extenders[*].httpTimeout` is of type `metav1.Duration`.
|
|
|
|
|
- `.extenders[*].enableHttps` is renamed to `.extenders[*].enableHTTPS`.
|
|
|
|
|
- `RequestedToCapacityRatio` args decoding is case sensitive. All fields are affected.
|
|
|
|
|
- `DefaultPodTopologySpread` [plugin](#scheduling-plugins) is renamed to `SelectorSpread`.
|
|
|
|
|
- `Unreserve` extension point is removed from Profile definition. All `Reserve`
|
|
|
|
|
plugins implement an `Unreserve` call.
|
|
|
|
|
|
|
|
|
|
## {{% heading "whatsnext" %}}
|
|
|
|
|
|
|
|
|
|
* Read the [kube-scheduler reference](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-scheduler/)
|
|
|
|
|
* Learn about [scheduling](/docs/concepts/scheduling-eviction/kube-scheduler/)
|
|
|
|
|
|
Kubernetes Prow Robot