b76366329d
This PR updates the scheduler configuration YAML snippets to use the 'v1beta3' version API. 'v1beta1' is gone and not recommended. We don't have config API reference for 'v1beta1' now.
225 lines
5.4 KiB
Markdown
225 lines
5.4 KiB
Markdown
---
|
|
reviewers:
|
|
- bsalamat
|
|
- k82cn
|
|
- ahg-g
|
|
title: Resource Bin Packing for Extended Resources
|
|
content_type: concept
|
|
weight: 80
|
|
---
|
|
|
|
<!-- overview -->
|
|
|
|
{{< feature-state for_k8s_version="v1.16" state="alpha" >}}
|
|
|
|
The kube-scheduler can be configured to enable bin packing of resources along
|
|
with extended resources using `RequestedToCapacityRatioResourceAllocation`
|
|
priority function. Priority functions can be used to fine-tune the
|
|
kube-scheduler as per custom needs.
|
|
|
|
<!-- body -->
|
|
|
|
## Enabling Bin Packing using RequestedToCapacityRatioResourceAllocation
|
|
|
|
Kubernetes allows the users to specify the resources along with weights for
|
|
each resource to score nodes based on the request to capacity ratio. This
|
|
allows users to bin pack extended resources by using appropriate parameters
|
|
and improves the utilization of scarce resources in large clusters. The
|
|
behavior of the `RequestedToCapacityRatioResourceAllocation` priority function
|
|
can be controlled by a configuration option called `RequestedToCapacityRatioArgs`.
|
|
This argument consists of two parameters `shape` and `resources`. The `shape`
|
|
parameter allows the user to tune the function as least requested or most
|
|
requested based on `utilization` and `score` values. The `resources` parameter
|
|
consists of `name` of the resource to be considered during scoring and `weight`
|
|
specify the weight of each resource.
|
|
|
|
Below is an example configuration that sets
|
|
`requestedToCapacityRatioArguments` to bin packing behavior for extended
|
|
resources `intel.com/foo` and `intel.com/bar`.
|
|
|
|
```yaml
|
|
apiVersion: kubescheduler.config.k8s.io/v1beta3
|
|
kind: KubeSchedulerConfiguration
|
|
profiles:
|
|
# ...
|
|
pluginConfig:
|
|
- name: RequestedToCapacityRatio
|
|
args:
|
|
shape:
|
|
- utilization: 0
|
|
score: 10
|
|
- utilization: 100
|
|
score: 0
|
|
resources:
|
|
- name: intel.com/foo
|
|
weight: 3
|
|
- name: intel.com/bar
|
|
weight: 5
|
|
```
|
|
|
|
Referencing the `KubeSchedulerConfiguration` file with the kube-scheduler
|
|
flag `--config=/path/to/config/file` will pass the configuration to the
|
|
scheduler.
|
|
|
|
**This feature is disabled by default**
|
|
|
|
### Tuning the Priority Function
|
|
|
|
`shape` is used to specify the behavior of the
|
|
`RequestedToCapacityRatioPriority` function.
|
|
|
|
```yaml
|
|
shape:
|
|
- utilization: 0
|
|
score: 0
|
|
- utilization: 100
|
|
score: 10
|
|
```
|
|
|
|
The above arguments give the node a `score` of 0 if `utilization` is 0% and 10 for
|
|
`utilization` 100%, thus enabling bin packing behavior. To enable least
|
|
requested the score value must be reversed as follows.
|
|
|
|
```yaml
|
|
shape:
|
|
- utilization: 0
|
|
score: 10
|
|
- utilization: 100
|
|
score: 0
|
|
```
|
|
|
|
`resources` is an optional parameter which defaults to:
|
|
|
|
``` yaml
|
|
resources:
|
|
- name: cpu
|
|
weight: 1
|
|
- name: memory
|
|
weight: 1
|
|
```
|
|
|
|
It can be used to add extended resources as follows:
|
|
|
|
```yaml
|
|
resources:
|
|
- name: intel.com/foo
|
|
weight: 5
|
|
- name: cpu
|
|
weight: 3
|
|
- name: memory
|
|
weight: 1
|
|
```
|
|
|
|
The `weight` parameter is optional and is set to 1 if not specified. Also, the
|
|
`weight` cannot be set to a negative value.
|
|
|
|
### Node scoring for capacity allocation
|
|
|
|
This section is intended for those who want to understand the internal details
|
|
of this feature.
|
|
Below is an example of how the node score is calculated for a given set of values.
|
|
|
|
Requested resources:
|
|
|
|
```
|
|
intel.com/foo : 2
|
|
memory: 256MB
|
|
cpu: 2
|
|
```
|
|
|
|
Resource weights:
|
|
|
|
```
|
|
intel.com/foo : 5
|
|
memory: 1
|
|
cpu: 3
|
|
```
|
|
|
|
FunctionShapePoint {{0, 0}, {100, 10}}
|
|
|
|
Node 1 spec:
|
|
|
|
```
|
|
Available:
|
|
intel.com/foo: 4
|
|
memory: 1 GB
|
|
cpu: 8
|
|
|
|
Used:
|
|
intel.com/foo: 1
|
|
memory: 256MB
|
|
cpu: 1
|
|
```
|
|
|
|
Node score:
|
|
|
|
```
|
|
intel.com/foo = resourceScoringFunction((2+1),4)
|
|
= (100 - ((4-3)*100/4)
|
|
= (100 - 25)
|
|
= 75 # requested + used = 75% * available
|
|
= rawScoringFunction(75)
|
|
= 7 # floor(75/10)
|
|
|
|
memory = resourceScoringFunction((256+256),1024)
|
|
= (100 -((1024-512)*100/1024))
|
|
= 50 # requested + used = 50% * available
|
|
= rawScoringFunction(50)
|
|
= 5 # floor(50/10)
|
|
|
|
cpu = resourceScoringFunction((2+1),8)
|
|
= (100 -((8-3)*100/8))
|
|
= 37.5 # requested + used = 37.5% * available
|
|
= rawScoringFunction(37.5)
|
|
= 3 # floor(37.5/10)
|
|
|
|
NodeScore = (7 * 5) + (5 * 1) + (3 * 3) / (5 + 1 + 3)
|
|
= 5
|
|
```
|
|
|
|
Node 2 spec:
|
|
|
|
```
|
|
Available:
|
|
intel.com/foo: 8
|
|
memory: 1GB
|
|
cpu: 8
|
|
Used:
|
|
intel.com/foo: 2
|
|
memory: 512MB
|
|
cpu: 6
|
|
```
|
|
|
|
Node score:
|
|
|
|
```
|
|
intel.com/foo = resourceScoringFunction((2+2),8)
|
|
= (100 - ((8-4)*100/8)
|
|
= (100 - 50)
|
|
= 50
|
|
= rawScoringFunction(50)
|
|
= 5
|
|
|
|
memory = resourceScoringFunction((256+512),1024)
|
|
= (100 -((1024-768)*100/1024))
|
|
= 75
|
|
= rawScoringFunction(75)
|
|
= 7
|
|
|
|
cpu = resourceScoringFunction((2+6),8)
|
|
= (100 -((8-8)*100/8))
|
|
= 100
|
|
= rawScoringFunction(100)
|
|
= 10
|
|
|
|
NodeScore = (5 * 5) + (7 * 1) + (10 * 3) / (5 + 1 + 3)
|
|
= 7
|
|
|
|
```
|
|
|
|
## {{% heading "whatsnext" %}}
|
|
|
|
- Read more about the [scheduling framework](/docs/concepts/scheduling-eviction/scheduling-framework/)
|
|
- Read more about [scheduler configuration](/docs/reference/scheduling/config/)
|
|
|