From d7de46dec8a36fed373e8e20a1a38ccbb4698720 Mon Sep 17 00:00:00 2001 From: simplytunde Date: Mon, 23 Sep 2019 02:31:23 -0500 Subject: [PATCH] Tune feature-gates table (#16389) (#16410) This PR separates the graduated/deprecated feature gates into a separate table. When updating the table contents by browsing the source code, we also found some missing items which are added in this PR as well. (cherry picked from commit faae88f4784f2e48dda23ee3e7237066000a998d) --- .../feature-gates.md | 157 +++++++++++------- 1 file changed, 97 insertions(+), 60 deletions(-) diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates.md b/content/en/docs/reference/command-line-tools-reference/feature-gates.md index 6610fba3c9..c2c1f966e8 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates.md @@ -24,7 +24,7 @@ To set feature gates for a component, such as kubelet, use the `--feature-gates` --feature-gates="...,DynamicKubeletConfig=true" ``` -The following table is a summary of the feature gates that you can set on +The following tables are a summary of the feature gates that you can set on different Kubernetes components. - The "Since" column contains the Kubernetes release when a feature is introduced @@ -32,22 +32,18 @@ different Kubernetes components. - The "Until" column, if not empty, contains the last Kubernetes release in which you can still use a feature gate. +{{< table caption="Feature gates for features in Alpha or Beta states" >}} + | Feature | Default | Stage | Since | Until | |---------|---------|-------|-------|-------| -| `Accelerators` | `false` | Alpha | 1.6 | 1.10 | -| `AdvancedAuditing` | `false` | Alpha | 1.7 | 1.7 | -| `AdvancedAuditing` | `true` | Beta | 1.8 | 1.11 | -| `AdvancedAuditing` | `true` | GA | 1.12 | - | -| `AffinityInAnnotations` | `false` | Alpha | 1.6 | 1.7 | -| `AllowExtTrafficLocalEndpoints` | `false` | Beta | 1.4 | 1.6 | -| `AllowExtTrafficLocalEndpoints` | `true` | GA | 1.7 | - | | `APIListChunking` | `false` | Alpha | 1.8 | 1.8 | | `APIListChunking` | `true` | Beta | 1.9 | | | `APIResponseCompression` | `false` | Alpha | 1.7 | | | `AppArmor` | `true` | Beta | 1.4 | | | `AttachVolumeLimit` | `true` | Alpha | 1.11 | 1.11 | | `AttachVolumeLimit` | `true` | Beta | 1.12 | | -| `BlockVolume` | `false` | Alpha | 1.9 | | +| `BalanceAttachedNodeVolumes` | `false` | Alpha | 1.11 | | +| `BlockVolume` | `false` | Alpha | 1.9 | 1.12 | | `BlockVolume` | `true` | Beta | 1.13 | - | | `BoundServiceAccountTokenVolume` | `false` | Alpha | 1.13 | | | `CPUManager` | `false` | Alpha | 1.8 | 1.9 | @@ -67,17 +63,11 @@ different Kubernetes components. | `CSIMigrationOpenStack` | `false` | Alpha | 1.14 | | | `CSINodeInfo` | `false` | Alpha | 1.12 | 1.13 | | `CSINodeInfo` | `true` | Beta | 1.14 | | -| `CSIPersistentVolume` | `false` | Alpha | 1.9 | 1.9 | -| `CSIPersistentVolume` | `true` | Beta | 1.10 | 1.12 | -| `CSIPersistentVolume` | `true` | GA | 1.13 | - | | `CustomCPUCFSQuotaPeriod` | `false` | Alpha | 1.12 | | -| `CustomPodDNS` | `false` | Alpha | 1.9 | 1.9 | -| `CustomPodDNS` | `true` | Beta| 1.10 | 1.13 | -| `CustomPodDNS` | `true` | GA | 1.14 | - | | `CustomResourceDefaulting` | `false` | Alpha| 1.15 | | | `CustomResourcePublishOpenAPI` | `false` | Alpha| 1.14 | 1.14 | | `CustomResourcePublishOpenAPI` | `true` | Beta| 1.15 | | -| `CustomResourceSubresources` | `false` | Alpha | 1.10 | 1.11 | +| `CustomResourceSubresources` | `false` | Alpha | 1.10 | 1.10 | | `CustomResourceSubresources` | `true` | Beta | 1.11 | - | | `CustomResourceValidation` | `false` | Alpha | 1.8 | 1.8 | | `CustomResourceValidation` | `true` | Beta | 1.9 | | @@ -86,83 +76,54 @@ different Kubernetes components. | `DebugContainers` | `false` | Alpha | 1.10 | | | `DevicePlugins` | `false` | Alpha | 1.8 | 1.9 | | `DevicePlugins` | `true` | Beta | 1.10 | | +| `DryRun` | `false` | Alpha | 1.12 | 1.12 | | `DryRun` | `true` | Beta | 1.13 | | | `DynamicAuditing` | `false` | Alpha | 1.13 | | | `DynamicKubeletConfig` | `false` | Alpha | 1.4 | 1.10 | | `DynamicKubeletConfig` | `true` | Beta | 1.11 | | | `DynamicProvisioningScheduling` | `false` | Alpha | 1.11 | 1.11 | -| `DynamicVolumeProvisioning` | `true` | Alpha | 1.3 | 1.7 | -| `DynamicVolumeProvisioning` | `true` | GA | 1.8 | | | `EnableEquivalenceClassCache` | `false` | Alpha | 1.8 | | | `ExpandCSIVolumes` | `false` | Alpha | 1.14 | | | `ExpandInUsePersistentVolumes` | `false` | Alpha | 1.11 | 1.14 | | `ExpandInUsePersistentVolumes` | `true` | Beta | 1.15 | | | `ExpandPersistentVolumes` | `false` | Alpha | 1.8 | 1.10 | | `ExpandPersistentVolumes` | `true` | Beta | 1.11 | | -| `ExperimentalCriticalPodAnnotation` | `false` | Alpha | 1.5 | | | `ExperimentalHostUserNamespaceDefaulting` | `false` | Beta | 1.5 | | -| `GCERegionalPersistentDisk` | `true` | Beta | 1.10 | 1.12 | -| `GCERegionalPersistentDisk` | `true` | GA | 1.13 | - | -| `HugePages` | `false` | Alpha | 1.8 | 1.9 | -| `HugePages` | `true` | Beta| 1.10 | 1.13 | -| `HugePages` | `true` | GA | 1.14 | | | `HyperVContainer` | `false` | Alpha | 1.10 | | -| `Initializers` | `false` | Alpha | 1.7 | 1.13 | -| `Initializers` | - | Deprecated | 1.14 | | -| `KubeletConfigFile` | `false` | Alpha | 1.8 | 1.9 | -| `KubeletPluginsWatcher` | `false` | Alpha | 1.11 | 1.11 | -| `KubeletPluginsWatcher` | `true` | Beta | 1.12 | 1.12 | -| `KubeletPluginsWatcher` | `true` | GA | 1.13 | - | | `KubeletPodResources` | `false` | Alpha | 1.13 | 1.14 | | `KubeletPodResources` | `true` | Beta | 1.15 | | -| `LegacyNodeRoleBehavior` | `true` | Alpha | 1.16 | | | `LocalStorageCapacityIsolation` | `false` | Alpha | 1.7 | 1.9 | -| `LocalStorageCapacityIsolation` | `true` | Beta| 1.10 | | -| `LocalStorageCapacityIsolationFSQuotaMonitoring` | `false` | Alpha| 1.15 | | +| `LocalStorageCapacityIsolation` | `true` | Beta | 1.10 | | +| `LocalStorageCapacityIsolationFSQuotaMonitoring` | `false` | Alpha | 1.15 | | | `MountContainers` | `false` | Alpha | 1.9 | | -| `MountPropagation` | `false` | Alpha | 1.8 | 1.9 | -| `MountPropagation` | `true` | Beta | 1.10 | 1.11 | -| `MountPropagation` | `true` | GA | 1.12 | | -| `NodeDisruptionExclusion` | `false` | Alpha | 1.16 | | | `NodeLease` | `false` | Alpha | 1.12 | 1.13 | | `NodeLease` | `true` | Beta | 1.14 | | | `NonPreemptingPriority` | `false` | Alpha | 1.15 | | -| `PersistentLocalVolumes` | `false` | Alpha | 1.7 | 1.9 | -| `PersistentLocalVolumes` | `true` | Beta | 1.10 | 1.13 | -| `PersistentLocalVolumes` | `true` | GA | 1.14 | | -| `PodPriority` | `false` | Alpha | 1.8 | 1.10 | -| `PodPriority` | `true` | Beta | 1.11 | 1.13 | -| `PodPriority` | `true` | GA | 1.14 | | -| `PodReadinessGates` | `false` | Alpha | 1.11 | 1.11 | -| `PodReadinessGates` | `true` | Beta | 1.12 | 1.13 | -| `PodReadinessGates` | `true` | GA | 1.14 | - | -| `PodShareProcessNamespace` | `false` | Alpha | 1.10 | | +| `PodShareProcessNamespace` | `false` | Alpha | 1.10 | 1.11 | | `PodShareProcessNamespace` | `true` | Beta | 1.12 | | | `ProcMountType` | `false` | Alpha | 1.12 | | -| `PVCProtection` | `false` | Alpha | 1.9 | 1.9 | +| `QOSReserved` | `false` | Alpha | 1.11 | | | `RemainingItemCount` | `false` | Alpha | 1.15 | | -| `ResourceLimitsPriorityFunction` | `false` | Alpha | 1.9 | | | `RequestManagement` | `false` | Alpha | 1.15 | | +| `ResourceLimitsPriorityFunction` | `false` | Alpha | 1.9 | | | `ResourceQuotaScopeSelectors` | `false` | Alpha | 1.11 | 1.11 | | `ResourceQuotaScopeSelectors` | `true` | Beta | 1.12 | | | `RotateKubeletClientCertificate` | `true` | Beta | 1.8 | | | `RotateKubeletServerCertificate` | `false` | Alpha | 1.7 | 1.11 | | `RotateKubeletServerCertificate` | `true` | Beta | 1.12 | | | `RunAsGroup` | `true` | Beta | 1.14 | | +| `RuntimeClass` | `false` | Alpha | 1.12 | 1.13 | | `RuntimeClass` | `true` | Beta | 1.14 | | +| `ScheduleDaemonSetPods` | `false` | Alpha | 1.11 | 1.11 | +| `ScheduleDaemonSetPods` | `true` | Beta | 1.12 | | | `SCTPSupport` | `false` | Alpha | 1.12 | | | `ServerSideApply` | `false` | Alpha | 1.14 | | | `ServiceLoadBalancerFinalizer` | `false` | Alpha | 1.15 | | | `ServiceNodeExclusion` | `false` | Alpha | 1.8 | | -| `StorageObjectInUseProtection` | `true` | Beta | 1.10 | 1.10 | -| `StorageObjectInUseProtection` | `true` | GA | 1.11 | | | `StorageVersionHash` | `false` | Alpha | 1.14 | 1.14 | | `StorageVersionHash` | `true` | Beta | 1.15 | | -| `StreamingProxyRedirects` | `true` | Beta | 1.5 | | -| `SupportIPVSProxyMode` | `false` | Alpha | 1.8 | 1.8 | -| `SupportIPVSProxyMode` | `false` | Beta | 1.9 | 1.9 | -| `SupportIPVSProxyMode` | `true` | Beta | 1.10 | 1.10 | -| `SupportIPVSProxyMode` | `true` | GA | 1.11 | | +| `StreamingProxyRedirects` | `false` | Beta | 1.5 | 1.5 | +| `StreamingProxyRedirects` | `true` | Beta | 1.6 | | | `SupportNodePidsLimit` | `false` | Alpha | 1.14 | 1.14 | | `SupportNodePidsLimit` | `true` | Beta | 1.15 | | | `SupportPodPidsLimit` | `false` | Alpha | 1.10 | 1.13 | @@ -178,9 +139,6 @@ different Kubernetes components. | `TokenRequestProjection` | `true` | Beta | 1.12 | | | `TTLAfterFinished` | `false` | Alpha | 1.12 | | | `VolumePVCDataSource` | `false` | Alpha | 1.15 | | -| `VolumeScheduling` | `false` | Alpha | 1.9 | 1.9 | -| `VolumeScheduling` | `true` | Beta | 1.10 | 1.12 | -| `VolumeScheduling` | `true` | GA | 1.13 | | | `VolumeSubpathEnvExpansion` | `false` | Alpha | 1.14 | 1.14 | | `VolumeSubpathEnvExpansion` | `true` | Beta | 1.15 | | | `VolumeSnapshotDataSource` | `false` | Alpha | 1.12 | - | @@ -188,6 +146,76 @@ different Kubernetes components. | `ScheduleDaemonSetPods` | `true` | Beta | 1.12 | | | `WatchBookmark` | `false` | Alpha | 1.15 | | | `WindowsGMSA` | `false` | Alpha | 1.14 | | +| `WinDSR` | `false` | Alpha | 1.14 | | +| `WinOverlay` | `false` | Alpha | 1.14 | | +{{< /table >}} + +The following table contains feature gates for graduated or deprecated features. + +{{< table caption="Feature Gates for Graduated or Deprecated Features" >}} + +| Feature | Default | Stage | Since | Until | +|---------|---------|-------|-------|-------| +| `Accelerators` | `false` | Alpha | 1.6 | 1.10 | +| `Accelerators` | - | Deprecated | 1.11 | - | +| `AdvancedAuditing` | `false` | Alpha | 1.7 | 1.7 | +| `AdvancedAuditing` | `true` | Beta | 1.8 | 1.11 | +| `AdvancedAuditing` | `true` | GA | 1.12 | - | +| `AffinityInAnnotations` | `false` | Alpha | 1.6 | 1.7 | +| `AffinityInAnnotations` | - | Deprecated | 1.8 | - | +| `AllowExtTrafficLocalEndpoints` | `false` | Beta | 1.4 | 1.6 | +| `AllowExtTrafficLocalEndpoints` | `true` | GA | 1.7 | - | +| `CSIPersistentVolume` | `false` | Alpha | 1.9 | 1.9 | +| `CSIPersistentVolume` | `true` | Beta | 1.10 | 1.12 | +| `CSIPersistentVolume` | `true` | GA | 1.13 | - | +| `CustomPodDNS` | `false` | Alpha | 1.9 | 1.9 | +| `CustomPodDNS` | `true` | Beta| 1.10 | 1.13 | +| `CustomPodDNS` | `true` | GA | 1.14 | - | +| `DynamicProvisioningScheduling` | `false` | Alpha | 1.11 | 1.11 | +| `DynamicProvisioningScheduling` | - | Deprecated| 1.12 | - | +| `DynamicVolumeProvisioning` | `true` | Alpha | 1.3 | 1.7 | +| `DynamicVolumeProvisioning` | `true` | GA | 1.8 | - | +| `EnableEquivalenceClassCache` | `false` | Alpha | 1.8 | 1.14 | +| `EnableEquivalenceClassCache` | - | Deprecated | 1.15 | - | +| `ExperimentalCriticalPodAnnotation` | `false` | Alpha | 1.5 | 1.12 | +| `ExperimentalCriticalPodAnnotation` | `false` | Deprecated | 1.13 | - | +| `GCERegionalPersistentDisk` | `true` | Beta | 1.10 | 1.12 | +| `GCERegionalPersistentDisk` | `true` | GA | 1.13 | - | +| `HugePages` | `false` | Alpha | 1.8 | 1.9 | +| `HugePages` | `true` | Beta| 1.10 | 1.13 | +| `HugePages` | `true` | GA | 1.14 | - | +| `Initializers` | `false` | Alpha | 1.7 | 1.13 | +| `Initializers` | - | Deprecated | 1.14 | - | +| `KubeletConfigFile` | `false` | Alpha | 1.8 | 1.9 | +| `KubeletConfigFile` | - | Deprecated | 1.10 | - | +| `KubeletPluginsWatcher` | `false` | Alpha | 1.11 | 1.11 | +| `KubeletPluginsWatcher` | `true` | Beta | 1.12 | 1.12 | +| `KubeletPluginsWatcher` | `true` | GA | 1.13 | - | +| `MountPropagation` | `false` | Alpha | 1.8 | 1.9 | +| `MountPropagation` | `true` | Beta | 1.10 | 1.11 | +| `MountPropagation` | `true` | GA | 1.12 | - | +| `PersistentLocalVolumes` | `false` | Alpha | 1.7 | 1.9 | +| `PersistentLocalVolumes` | `true` | Beta | 1.10 | 1.13 | +| `PersistentLocalVolumes` | `true` | GA | 1.14 | - | +| `PodPriority` | `false` | Alpha | 1.8 | 1.10 | +| `PodPriority` | `true` | Beta | 1.11 | 1.13 | +| `PodPriority` | `true` | GA | 1.14 | - | +| `PodReadinessGates` | `false` | Alpha | 1.11 | 1.11 | +| `PodReadinessGates` | `true` | Beta | 1.12 | 1.13 | +| `PodReadinessGates` | `true` | GA | 1.14 | - | +| `PVCProtection` | `false` | Alpha | 1.9 | 1.9 | +| `PVCProtection` | - | Deprecated | 1.10 | - | +| `StorageObjectInUseProtection` | `true` | Beta | 1.10 | 1.10 | +| `StorageObjectInUseProtection` | `true` | GA | 1.11 | | +| `SupportIPVSProxyMode` | `false` | Alpha | 1.8 | 1.8 | +| `SupportIPVSProxyMode` | `false` | Beta | 1.9 | 1.9 | +| `SupportIPVSProxyMode` | `true` | Beta | 1.10 | 1.10 | +| `SupportIPVSProxyMode` | `true` | GA | 1.11 | - | +| `VolumeScheduling` | `false` | Alpha | 1.9 | 1.9 | +| `VolumeScheduling` | `true` | Beta | 1.10 | 1.12 | +| `VolumeScheduling` | `true` | GA | 1.13 | - | +| `VolumeSubpath` | `true` | GA | 1.13 | - | +{{< /table >}} ## Using a Feature @@ -242,6 +270,9 @@ Each feature gate is designed for enabling/disabling a specific feature: - `AttachVolumeLimit`: Enable volume plugins to report limits on number of volumes that can be attached to a node. See [dynamic volume limits](/docs/concepts/storage/storage-limits/#dynamic-volume-limits) for more details. +- `BalanceAttachedNodeVolumes`: Include volume count on node to be considered for balanced resource allocation + while scheduling. A node which has closer CPU, memory utilization, and volume count is favored by the scheduler + while making decisions. - `BlockVolume`: Enable the definition and consumption of raw block devices in Pods. See [Raw Block Volume Support](/docs/concepts/storage/persistent-volumes/#raw-block-volume-support) for more details. @@ -281,7 +312,8 @@ Each feature gate is designed for enabling/disabling a specific feature: troubleshoot a running Pod. - `DevicePlugins`: Enable the [device-plugins](/docs/concepts/cluster-administration/device-plugins/) based resource provisioning on nodes. -- `DryRun`: Enable server-side [dry run](/docs/reference/using-api/api-concepts/#dry-run) requests. +- `DryRun`: Enable server-side [dry run](/docs/reference/using-api/api-concepts/#dry-run) requests + so that validation, merging, and mutation can be tested without committing. - `DynamicAuditing`: Enable [dynamic auditing](/docs/tasks/debug-application-cluster/audit/#dynamic-backend) - `DynamicKubeletConfig`: Enable the dynamic configuration of kubelet. See [Reconfigure kubelet](/docs/tasks/administer-cluster/reconfigure-kubelet/). - `DynamicProvisioningScheduling`: Extend the default scheduler to be aware of volume topology and handle PV provisioning. @@ -291,6 +323,7 @@ Each feature gate is designed for enabling/disabling a specific feature: - `ExpandInUsePersistentVolumes`: Enable expanding in-use PVCs. See [Resizing an in-use PersistentVolumeClaim](/docs/concepts/storage/persistent-volumes/#resizing-an-in-use-persistentvolumeclaim). - `ExpandPersistentVolumes`: Enable the expanding of persistent volumes. See [Expanding Persistent Volumes Claims](/docs/concepts/storage/persistent-volumes/#expanding-persistent-volumes-claims). - `ExperimentalCriticalPodAnnotation`: Enable annotating specific pods as *critical* so that their [scheduling is guaranteed](/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods/). + This feature is deprecated by Pod Priority and Preemption as of v1.13. - `ExperimentalHostUserNamespaceDefaultingGate`: Enabling the defaulting user namespace to host. This is for containers that are using other host namespaces, host mounts, or containers that are privileged or using specific non-namespaced @@ -324,6 +357,8 @@ Each feature gate is designed for enabling/disabling a specific feature: - `PVCProtection`: Enable the prevention of a PersistentVolumeClaim (PVC) from being deleted when it is still used by any Pod. More details can be found [here](/docs/tasks/administer-cluster/storage-object-in-use-protection/). +- `QOSReserved`: Allows resource reservations at the QoS level preventing pods at lower QoS levels from + bursting into resources requested at higher QoS levels (memory only for now). - `ResourceLimitsPriorityFunction`: Enable a scheduler priority function that assigns a lowest possible score of 1 to a node that satisfies at least one of the input Pod's cpu and memory limits. The intent is to break ties between @@ -369,5 +404,7 @@ Each feature gate is designed for enabling/disabling a specific feature: - `VolumeSubpathEnvExpansion`: Enable `subPathExpr` field for expanding environment variables into a `subPath`. - `WatchBookmark`: Enable support for watch bookmark events. - `WindowsGMSA`: Enables passing of GMSA credential specs from pods to container runtimes. +- `WinDSR`: Allows kube-proxy to create DSR loadbalancers for Windows. +- `WinOverlay`: Allows kube-proxy to run in overlay mode for Windows. {{% /capture %}}