From 8ba585e8f7a3e990cfab510c502da8ec0ea1efb1 Mon Sep 17 00:00:00 2001 From: Hemant Kumar Date: Thu, 13 Sep 2018 00:54:41 -0400 Subject: [PATCH] Dynamic volume limit updates for 1.12 (#10211) * add a placeholder commit * Update docs for csi volume limits * Update storage-limits.md --- .../docs/concepts/storage/storage-limits.md | 37 ++++++++++--------- 1 file changed, 20 insertions(+), 17 deletions(-) diff --git a/content/en/docs/concepts/storage/storage-limits.md b/content/en/docs/concepts/storage/storage-limits.md index d291c6eca2..b01d54e89d 100644 --- a/content/en/docs/concepts/storage/storage-limits.md +++ b/content/en/docs/concepts/storage/storage-limits.md @@ -11,11 +11,11 @@ content_template: templates/concept {{% capture overview %}} This page describes the maximum number of volumes that can be attached -to a node for various cloud providers. +to a Node for various cloud providers. Cloud providers like Google, Amazon, and Microsoft typically have a limit on -how many volumes can be attached to a node. It is important for Kubernetes to -respect those limits. Otherwise, Pods scheduled on a node could get stuck +how many volumes can be attached to a Node. It is important for Kubernetes to +respect those limits. Otherwise, Pods scheduled on a Node could get stuck waiting for volumes to attach. {{% /capture %}} @@ -25,10 +25,10 @@ waiting for volumes to attach. ## Kubernetes default limits The Kubernetes scheduler has default limits on the number of volumes -that can be attached to a node: +that can be attached to a Node: - + @@ -40,28 +40,28 @@ You can change these limits by setting the value of the `KUBE_MAX_PD_VOLS` environment variable, and then starting the scheduler. Use caution if you set a limit that is higher than the default limit. Consult -the cloud provider's documentation to make sure that nodes can actually support +the cloud provider's documentation to make sure that Nodes can actually support the limit you set. -The limit applies to the entire cluster, so it affects all nodes. +The limit applies to the entire cluster, so it affects all Nodes. ## Dynamic volume limits -{{< feature-state state="alpha" for_k8s_version="v1.11" >}} +{{< feature-state state="beta" for_k8s_version="v1.12" >}} -Kubernetes 1.11 introduces dynamic volume limits based on node type. This -is an alpha feature that supports these services: +Kubernetes 1.11 introduced support for dynamic volume limits based on Node type as an Alpha feature. +In Kubernetes 1.12 this feature is graduating to Beta and will be enabled by default. + +Dynamic volume limits is supported for following volume types. - Amazon EBS - Google Persistent Disk - Azure Disk +- CSI -To enable dynamic volume limits, set the `AttachVolumeLimit` -[feature gate](/docs/reference/command-line-tools-reference/feature-gates/) -to True. When the dynamic volume limits feature is enabled, Kubernetes automatically -determines the node type and supports the appropriate number of attachable +determines the Node type and enforces the appropriate number of attachable volumes for the node. For example: * On @@ -69,11 +69,14 @@ volumes for the node. For example: up to 128 volumes can be attached to a node, [depending on the node type](https://cloud.google.com/compute/docs/disks/#pdnumberlimits). -* For Amazon EBS disks on M5/C5 instance types, Kubernetes allows only 25 -volumes to be attached to a node. For other instance types on +* For Amazon EBS disks on M5,C5,R5,T3 and Z1D instance types, Kubernetes allows only 25 +volumes to be attached to a Node. For other instance types on Amazon Elastic Compute Cloud (EC2), -Kubernetes allows 39 volumes to be attached to a node. +Kubernetes allows 39 volumes to be attached to a Node. * On Azure, up to 64 disks can be attached to a node, depending on the node type. For more details, refer to [Sizes for virtual machines in Azure](https://docs.microsoft.com/en-us/azure/virtual-machines/windows/sizes). +* For CSI, any driver that advertises volume attach limits via CSI specs will have those limits available as the Node's allocatable property + and the Scheduler will not schedule Pods with volumes on any Node that is already at its capacity. Refer to the [CSI specs](https://github.com/container-storage-interface/spec/blob/master/spec.md#nodegetinfo) for more details. + {{% /capture %}}
Cloud serviceMaximum volumes per node
Cloud serviceMaximum volumes per Node
Amazon Elastic Block Store (EBS)39
Google Persistent Disk16
Microsoft Azure Disk Storage16