Add docs on scheduler MultiPoint config
This commit is contained in:
@@ -78,6 +78,8 @@ extension points:
|
|||||||
least one bind plugin is required.
|
least one bind plugin is required.
|
||||||
1. `postBind`: This is an informational extension point that is called after
|
1. `postBind`: This is an informational extension point that is called after
|
||||||
a Pod has been bound.
|
a Pod has been bound.
|
||||||
|
1. `multiPoint`: This is a config-only field that allows plugins to be enabled
|
||||||
|
or disabled for all of their applicable extension points simultaneously.
|
||||||
|
|
||||||
For each extension point, you could disable specific [default plugins](#scheduling-plugins)
|
For each extension point, you could disable specific [default plugins](#scheduling-plugins)
|
||||||
or enable your own. For example:
|
or enable your own. For example:
|
||||||
@@ -251,6 +253,186 @@ the same configuration parameters (if applicable). This is because the scheduler
|
|||||||
only has one pending pods queue.
|
only has one pending pods queue.
|
||||||
{{< /note >}}
|
{{< /note >}}
|
||||||
|
|
||||||
|
### Plugins that apply to multiple extension points {#multipoint}
|
||||||
|
|
||||||
|
Starting from `kubescheduler.config.k8s.io/v1beta3`, there is an additional field in the
|
||||||
|
profile config, `multiPoint`, which allows for easily enabling or disabling a plugin
|
||||||
|
across several extension points. The intent of `multiPoint` config is to simplify the
|
||||||
|
configuration needed for users and administrators when using custom profiles.
|
||||||
|
|
||||||
|
Consider a plugin, `MyPlugin`, which implements the `preScore`, `score`, `preFilter`,
|
||||||
|
and `filter` extension points. To enable `MyPlugin` for all its available extension
|
||||||
|
points, the profile config looks like:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
apiVersion: kubescheduler.config.k8s.io/v1beta3
|
||||||
|
kind: KubeSchedulerConfiguration
|
||||||
|
profiles:
|
||||||
|
- schedulerName: multipoint-scheduler
|
||||||
|
plugins:
|
||||||
|
multiPoint:
|
||||||
|
enabled:
|
||||||
|
- name: MyPlugin
|
||||||
|
```
|
||||||
|
|
||||||
|
This would equate to manually enabling `MyPlugin` for all of its extension
|
||||||
|
points, like so:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
apiVersion: kubescheduler.config.k8s.io/v1beta3
|
||||||
|
kind: KubeSchedulerConfiguration
|
||||||
|
profiles:
|
||||||
|
- schedulerName: non-multipoint-scheduler
|
||||||
|
plugins:
|
||||||
|
preScore:
|
||||||
|
enabled:
|
||||||
|
- name: MyPlugin
|
||||||
|
score:
|
||||||
|
enabled:
|
||||||
|
- name: MyPlugin
|
||||||
|
preFilter:
|
||||||
|
enabled:
|
||||||
|
- name: MyPlugin
|
||||||
|
filter:
|
||||||
|
enabled:
|
||||||
|
- name: MyPlugin
|
||||||
|
```
|
||||||
|
|
||||||
|
One benefit of using `multiPoint` here is that if `MyPlugin` implements another
|
||||||
|
extension point in the future, the `multiPoint` config will automatically enable it
|
||||||
|
for the new extension.
|
||||||
|
|
||||||
|
Specific extension points can be excluded from `MultiPoint` expansion using
|
||||||
|
the `disabled` field for that extension point. This works with disabling default
|
||||||
|
plugins, non-default plugins, or with the wildcard (`'*'`) to disable all plugins.
|
||||||
|
An example of this, disabling `Score` and `PreScore`, would be:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
apiVersion: kubescheduler.config.k8s.io/v1beta3
|
||||||
|
kind: KubeSchedulerConfiguration
|
||||||
|
profiles:
|
||||||
|
- schedulerName: non-multipoint-scheduler
|
||||||
|
plugins:
|
||||||
|
multiPoint:
|
||||||
|
enabled:
|
||||||
|
- name: 'MyPlugin'
|
||||||
|
preScore:
|
||||||
|
disabled:
|
||||||
|
- name: '*'
|
||||||
|
score:
|
||||||
|
disabled:
|
||||||
|
- name: '*'
|
||||||
|
```
|
||||||
|
|
||||||
|
In `v1beta3`, all [default plugins](#scheduling-plugins) are enabled internally through `MultiPoint`.
|
||||||
|
However, individual extension points are still available to allow flexible
|
||||||
|
reconfiguration of the default values (such as ordering and Score weights). For
|
||||||
|
example, consider two Score plugins `DefaultScore1` and `DefaultScore2`, each with
|
||||||
|
a weight of `1`. They can be reordered with different weights like so:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
apiVersion: kubescheduler.config.k8s.io/v1beta3
|
||||||
|
kind: KubeSchedulerConfiguration
|
||||||
|
profiles:
|
||||||
|
- schedulerName: multipoint-scheduler
|
||||||
|
plugins:
|
||||||
|
score:
|
||||||
|
enabled:
|
||||||
|
- name: 'DefaultScore2'
|
||||||
|
weight: 5
|
||||||
|
```
|
||||||
|
|
||||||
|
In this example, it's unnecessary to specify the plugins in `MultiPoint` explicitly
|
||||||
|
because they are default plugins. And the only plugin specified in `Score` is `DefaultScore2`.
|
||||||
|
This is because plugins set through specific extension points will always take precedence
|
||||||
|
over `MultiPoint` plugins. So, this snippet essentially re-orders the two plugins
|
||||||
|
without needing to specify both of them.
|
||||||
|
|
||||||
|
The general hierarchy for precedence when configuring `MultiPoint` plugins is as follows:
|
||||||
|
1. Specific extension points run first, and their settings override whatever is set elsewhere
|
||||||
|
2. Plugins manually configured through `MultiPoint` and their settings
|
||||||
|
3. Default plugins and their default settings
|
||||||
|
|
||||||
|
To demonstrate the above hierarchy, the following example is based on these plugins:
|
||||||
|
|Plugin|Extension Points|
|
||||||
|
|---|---|
|
||||||
|
|`DefaultQueueSort`|`QueueSort`|
|
||||||
|
|`CustomQueueSort`|`QueueSort`|
|
||||||
|
|`DefaultPlugin1`|`Score`, `Filter`|
|
||||||
|
|`DefaultPlugin2`|`Score`|
|
||||||
|
|`CustomPlugin1`|`Score`, `Filter`|
|
||||||
|
|`CustomPlugin2`|`Score`, `Filter`|
|
||||||
|
|
||||||
|
A valid sample configuration for these plugins would be:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
apiVersion: kubescheduler.config.k8s.io/v1beta3
|
||||||
|
kind: KubeSchedulerConfiguration
|
||||||
|
profiles:
|
||||||
|
- schedulerName: multipoint-scheduler
|
||||||
|
plugins:
|
||||||
|
multiPoint:
|
||||||
|
enabled:
|
||||||
|
- name: 'CustomQueueSort'
|
||||||
|
- name: 'CustomPlugin1'
|
||||||
|
weight: 3
|
||||||
|
- name: 'CustomPlugin2'
|
||||||
|
disabled:
|
||||||
|
- name: 'DefaultQueueSort'
|
||||||
|
filter:
|
||||||
|
disabled:
|
||||||
|
- name: 'DefaultPlugin1'
|
||||||
|
score:
|
||||||
|
enabled:
|
||||||
|
- name: 'DefaultPlugin2'
|
||||||
|
```
|
||||||
|
|
||||||
|
Note that there is no error for re-declaring a `MultiPoint` plugin in a specific
|
||||||
|
extension point. The re-declaration is ignored (and logged), as specific extension points
|
||||||
|
take precedence.
|
||||||
|
|
||||||
|
Besides keeping most of the config in one spot, this sample does a few things:
|
||||||
|
* Enables the custom `queueSort` plugin and disables the default one
|
||||||
|
* Enables `CustomPlugin1` and `CustomPlugin2`, which will run first for all of their extension points
|
||||||
|
* Disables `DefaultPlugin1`, but only for `filter`
|
||||||
|
* Reorders `DefaultPlugin2` to run first in `score` (even before the custom plugins)
|
||||||
|
|
||||||
|
In versions of the config before `v1beta3`, without `multiPoint`, the above snippet would equate to this:
|
||||||
|
```yaml
|
||||||
|
apiVersion: kubescheduler.config.k8s.io/v1beta2
|
||||||
|
kind: KubeSchedulerConfiguration
|
||||||
|
profiles:
|
||||||
|
- schedulerName: multipoint-scheduler
|
||||||
|
plugins:
|
||||||
|
|
||||||
|
# Disable the default QueueSort plugin
|
||||||
|
queueSort:
|
||||||
|
enabled:
|
||||||
|
- name: 'CustomQueueSort'
|
||||||
|
disabled:
|
||||||
|
- name: 'DefaultQueueSort'
|
||||||
|
|
||||||
|
# Enable custom Filter plugins
|
||||||
|
filter:
|
||||||
|
enabled:
|
||||||
|
- name: 'CustomPlugin1'
|
||||||
|
- name: 'CustomPlugin2'
|
||||||
|
- name: 'DefaultPlugin2'
|
||||||
|
disabled:
|
||||||
|
- name: 'DefaultPlugin1'
|
||||||
|
|
||||||
|
# Enable and reorder custom score plugins
|
||||||
|
score:
|
||||||
|
enabled:
|
||||||
|
- name: 'DefaultPlugin2'
|
||||||
|
weight: 1
|
||||||
|
- name: 'DefaultPlugin1'
|
||||||
|
weight: 3
|
||||||
|
```
|
||||||
|
|
||||||
|
While this is a complicated example, it demonstrates the flexibility of `MultiPoint` config
|
||||||
|
as well as its seamless integration with the existing methods for configuring extension points.
|
||||||
|
|
||||||
## Scheduler configuration migrations
|
## Scheduler configuration migrations
|
||||||
|
|
||||||
{{< tabs name="tab_with_md" >}}
|
{{< tabs name="tab_with_md" >}}
|
||||||
|
|||||||
Reference in New Issue
Block a user