From 95a6e055d822c5269986e1923ec78d6f538ca004 Mon Sep 17 00:00:00 2001 From: Kubernetes Prow Robot Date: Thu, 5 Mar 2020 09:44:38 -0800 Subject: [PATCH] Update hugepages documentation (#19008) * Update hugepages documentation - described support for multiple huge page sizes - described container isolation of the huge pages * Add HugePageStorageMediumSize description * update description for container isolation of hugepages Signed-off-by: Byonggon Chun Co-authored-by: Byonggon Chun --- .../feature-gates.md | 2 + .../manage-hugepages/scheduling-hugepages.md | 69 +++++++++++++++---- 2 files changed, 58 insertions(+), 13 deletions(-) diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates.md b/content/en/docs/reference/command-line-tools-reference/feature-gates.md index b02b3e48bd..2dc2a84b57 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates.md @@ -103,6 +103,7 @@ different Kubernetes components. | `ExperimentalHostUserNamespaceDefaulting` | `false` | Beta | 1.5 | | | `EvenPodsSpread` | `false` | Alpha | 1.16 | | | `HPAScaleToZero` | `false` | Alpha | 1.16 | | +| `HugePageStorageMediumSize` | `false` | Alpha | 1.18 | | | `HyperVContainer` | `false` | Alpha | 1.10 | | | `KubeletPodResources` | `false` | Alpha | 1.13 | 1.14 | | `KubeletPodResources` | `true` | Beta | 1.15 | | @@ -395,6 +396,7 @@ Each feature gate is designed for enabling/disabling a specific feature: See [Enabling Endpoint Slices](/docs/tasks/administer-cluster/enabling-endpointslices/). - `GCERegionalPersistentDisk`: Enable the regional PD feature on GCE. - `HugePages`: Enable the allocation and consumption of pre-allocated [huge pages](/docs/tasks/manage-hugepages/scheduling-hugepages/). +- `HugePageStorageMediumSize`: Enable support for multiple sizes pre-allocated [huge pages](/docs/tasks/manage-hugepages/scheduling-hugepages/). - `HyperVContainer`: Enable [Hyper-V isolation](https://docs.microsoft.com/en-us/virtualization/windowscontainers/manage-containers/hyperv-container) for Windows containers. - `HPAScaleToZero`: Enables setting `minReplicas` to 0 for `HorizontalPodAutoscaler` resources when using custom or external metrics. - `KubeletConfigFile`: Enable loading kubelet configuration from a file specified using a config file. diff --git a/content/en/docs/tasks/manage-hugepages/scheduling-hugepages.md b/content/en/docs/tasks/manage-hugepages/scheduling-hugepages.md index 890170a988..ad6b969c87 100644 --- a/content/en/docs/tasks/manage-hugepages/scheduling-hugepages.md +++ b/content/en/docs/tasks/manage-hugepages/scheduling-hugepages.md @@ -17,11 +17,11 @@ can consume huge pages and the current limitations. {{% capture prerequisites %}} 1. Kubernetes nodes must pre-allocate huge pages in order for the node to report - its huge page capacity. A node may only pre-allocate huge pages for a single - size. + its huge page capacity. A node can pre-allocate huge pages for multiple + sizes. -The nodes will automatically discover and report all huge page resources as a -schedulable resource. +The nodes will automatically discover and report all huge page resources as +schedulable resources. {{% /capture %}} @@ -30,12 +30,51 @@ schedulable resource. ## API Huge pages can be consumed via container level resource requirements using the -resource name `hugepages-`, where size is the most compact binary notation -using integer values supported on a particular node. For example, if a node -supports 2048KiB page sizes, it will expose a schedulable resource -`hugepages-2Mi`. Unlike CPU or memory, huge pages do not support overcommit. Note -that when requesting hugepage resources, either memory or CPU resources must -be requested as well. +resource name `hugepages-`, where `` is the most compact binary +notation using integer values supported on a particular node. For example, if a +node supports 2048KiB and 1048576KiB page sizes, it will expose a schedulable +resources `hugepages-2Mi` and `hugepages-1Gi`. Unlike CPU or memory, huge pages +do not support overcommit. Note that when requesting hugepage resources, either +memory or CPU resources must be requested as well. + +A pod may consume multiple huge page sizes in a single pod spec. In this case it +must use `medium: HugePages-` notation for all volume mounts. + + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: huge-pages-example +spec: + containers: + - name: example + image: fedora:latest + command: + - sleep + - inf + volumeMounts: + - mountPath: /hugepages-2Mi + name: hugepage-2mi + - mountPath: /hugepages-1Gi + name: hugepage-1gi + resources: + limits: + hugepages-2Mi: 100Mi + hugepages-1Gi: 2Gi + memory: 100Mi + requests: + memory: 100Mi + volumes: + - name: hugepage-2mi + emptyDir: + medium: HugePages-2Mi + - name: hugepage-1gi + emptyDir: + medium: HugePages-1Gi +``` + +A pod may use `medium: HugePages` only if it requests huge pages of one size. ```yaml apiVersion: v1 @@ -66,8 +105,7 @@ spec: - Huge page requests must equal the limits. This is the default if limits are specified, but requests are not. -- Huge pages are isolated at a pod scope, container isolation is planned in a - future iteration. +- Huge pages are isolated at a container scope, so each container has own limit on their cgroup sandbox as requested in a container spec. - EmptyDir volumes backed by huge pages may not consume more huge page memory than the pod request. - Applications that consume huge pages via `shmget()` with `SHM_HUGETLB` must @@ -75,10 +113,15 @@ spec: - Huge page usage in a namespace is controllable via ResourceQuota similar to other compute resources like `cpu` or `memory` using the `hugepages-` token. +- Support of multiple sizes huge pages is feature gated. It can be + enabled with the `HugePageStorageMediumSize` [feature +gate](/docs/reference/command-line-tools-reference/feature-gates/) on the {{< +glossary_tooltip text="kubelet" term_id="kubelet" >}} and {{< +glossary_tooltip text="kube-apiserver" +term_id="kube-apiserver" >}} (`--feature-gates=HugePageStorageMediumSize=true`). ## Future -- Support container isolation of huge pages in addition to pod isolation. - NUMA locality guarantees as a feature of quality of service. - LimitRange support.