From 3502b36f86fafbf298ef298df1d29c75853cf021 Mon Sep 17 00:00:00 2001 From: Qiming Teng Date: Tue, 19 Jan 2021 15:24:44 +0800 Subject: [PATCH] Fix outdated content in feature gates Also applied some line wrapping to unreasonable long lines. --- .../feature-gates.md | 488 ++++++++++++------ 1 file changed, 337 insertions(+), 151 deletions(-) diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates.md b/content/en/docs/reference/command-line-tools-reference/feature-gates.md index ed18f36c77..d9754afb56 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates.md @@ -1,6 +1,6 @@ --- -weight: 10 title: Feature Gates +weight: 10 content_type: concept --- @@ -48,13 +48,15 @@ different Kubernetes components. | Feature | Default | Stage | Since | Until | |---------|---------|-------|-------|-------| -| `AnyVolumeDataSource` | `false` | Alpha | 1.18 | | | `APIListChunking` | `false` | Alpha | 1.8 | 1.8 | | `APIListChunking` | `true` | Beta | 1.9 | | | `APIPriorityAndFairness` | `false` | Alpha | 1.17 | 1.19 | | `APIPriorityAndFairness` | `true` | Beta | 1.20 | | -| `APIResponseCompression` | `false` | Alpha | 1.7 | | +| `APIResponseCompression` | `false` | Alpha | 1.7 | 1.15 | +| `APIResponseCompression` | `false` | Beta | 1.16 | | | `APIServerIdentity` | `false` | Alpha | 1.20 | | +| `AllowInsecureBackendProxy` | `true` | Beta | 1.17 | | +| `AnyVolumeDataSource` | `false` | Alpha | 1.18 | | | `AppArmor` | `true` | Beta | 1.4 | | | `BalanceAttachedNodeVolumes` | `false` | Alpha | 1.11 | | | `BoundServiceAccountTokenVolume` | `false` | Alpha | 1.13 | | @@ -77,7 +79,8 @@ different Kubernetes components. | `CSIMigrationGCE` | `false` | Alpha | 1.14 | 1.16 | | `CSIMigrationGCE` | `false` | Beta | 1.17 | | | `CSIMigrationGCEComplete` | `false` | Alpha | 1.17 | | -| `CSIMigrationOpenStack` | `false` | Alpha | 1.14 | | +| `CSIMigrationOpenStack` | `false` | Alpha | 1.14 | 1.17 | +| `CSIMigrationOpenStack` | `true` | Beta | 1.18 | | | `CSIMigrationOpenStackComplete` | `false` | Alpha | 1.17 | | | `CSIMigrationvSphere` | `false` | Beta | 1.19 | | | `CSIMigrationvSphereComplete` | `false` | Beta | 1.19 | | @@ -89,26 +92,23 @@ different Kubernetes components. | `ConfigurableFSGroupPolicy` | `true` | Beta | 1.20 | | | `CronJobControllerV2` | `false` | Alpha | 1.20 | | | `CustomCPUCFSQuotaPeriod` | `false` | Alpha | 1.12 | | -| `CustomResourceDefaulting` | `false` | Alpha| 1.15 | 1.15 | -| `CustomResourceDefaulting` | `true` | Beta | 1.16 | | | `DefaultPodTopologySpread` | `false` | Alpha | 1.19 | 1.19 | | `DefaultPodTopologySpread` | `true` | Beta | 1.20 | | | `DevicePlugins` | `false` | Alpha | 1.8 | 1.9 | | `DevicePlugins` | `true` | Beta | 1.10 | | | `DisableAcceleratorUsageMetrics` | `false` | Alpha | 1.19 | 1.19 | -| `DisableAcceleratorUsageMetrics` | `true` | Beta | 1.20 | 1.22 | +| `DisableAcceleratorUsageMetrics` | `true` | Beta | 1.20 | | | `DownwardAPIHugePages` | `false` | Alpha | 1.20 | | -| `DryRun` | `false` | Alpha | 1.12 | 1.12 | -| `DryRun` | `true` | Beta | 1.13 | | | `DynamicKubeletConfig` | `false` | Alpha | 1.4 | 1.10 | | `DynamicKubeletConfig` | `true` | Beta | 1.11 | | +| `EfficientWatchResumption` | `false` | Alpha | 1.20 | | | `EndpointSlice` | `false` | Alpha | 1.16 | 1.16 | | `EndpointSlice` | `false` | Beta | 1.17 | | | `EndpointSlice` | `true` | Beta | 1.18 | | | `EndpointSliceNodeName` | `false` | Alpha | 1.20 | | | `EndpointSliceProxying` | `false` | Alpha | 1.18 | 1.18 | | `EndpointSliceProxying` | `true` | Beta | 1.19 | | -| `EndpointSliceTerminating` | `false` | Alpha | 1.20 | | +| `EndpointSliceTerminatingCondition` | `false` | Alpha | 1.20 | | | `EphemeralContainers` | `false` | Alpha | 1.16 | | | `ExpandCSIVolumes` | `false` | Alpha | 1.14 | 1.15 | | `ExpandCSIVolumes` | `true` | Beta | 1.16 | | @@ -119,19 +119,22 @@ different Kubernetes components. | `ExperimentalHostUserNamespaceDefaulting` | `false` | Beta | 1.5 | | | `GenericEphemeralVolume` | `false` | Alpha | 1.19 | | | `GracefulNodeShutdown` | `false` | Alpha | 1.20 | | +| `HPAContainerMetrics` | `false` | Alpha | 1.20 | | | `HPAScaleToZero` | `false` | Alpha | 1.16 | | | `HugePageStorageMediumSize` | `false` | Alpha | 1.18 | 1.18 | | `HugePageStorageMediumSize` | `true` | Beta | 1.19 | | -| `HyperVContainer` | `false` | Alpha | 1.10 | | +| `IPv6DualStack` | `false` | Alpha | 1.15 | | | `ImmutableEphemeralVolumes` | `false` | Alpha | 1.18 | 1.18 | | `ImmutableEphemeralVolumes` | `true` | Beta | 1.19 | | -| `IPv6DualStack` | `false` | Alpha | 1.16 | | -| `LegacyNodeRoleBehavior` | `true` | Alpha | 1.16 | | +| `KubeletCredentialProviders` | `false` | Alpha | 1.20 | | +| `KubeletPodResources` | `true` | Alpha | 1.13 | 1.14 | +| `KubeletPodResources` | `true` | Beta | 1.15 | | +| `LegacyNodeRoleBehavior` | `false` | Alpha | 1.16 | 1.18 | +| `LegacyNodeRoleBehavior` | `true` | True | 1.19 | | | `LocalStorageCapacityIsolation` | `false` | Alpha | 1.7 | 1.9 | | `LocalStorageCapacityIsolation` | `true` | Beta | 1.10 | | | `LocalStorageCapacityIsolationFSQuotaMonitoring` | `false` | Alpha | 1.15 | | | `MixedProtocolLBService` | `false` | Alpha | 1.20 | | -| `MountContainers` | `false` | Alpha | 1.9 | | | `NodeDisruptionExclusion` | `false` | Alpha | 1.16 | 1.18 | | `NodeDisruptionExclusion` | `true` | Beta | 1.19 | | | `NonPreemptingPriority` | `false` | Alpha | 1.15 | 1.18 | @@ -143,25 +146,27 @@ different Kubernetes components. | `ProcMountType` | `false` | Alpha | 1.12 | | | `QOSReserved` | `false` | Alpha | 1.11 | | | `RemainingItemCount` | `false` | Alpha | 1.15 | | +| `RemoveSelfLink` | `false` | Alpha | 1.16 | 1.19 | +| `RemoveSelfLink` | `true` | Beta | 1.20 | | | `RootCAConfigMap` | `false` | Alpha | 1.13 | 1.19 | | `RootCAConfigMap` | `true` | Beta | 1.20 | | | `RotateKubeletServerCertificate` | `false` | Alpha | 1.7 | 1.11 | | `RotateKubeletServerCertificate` | `true` | Beta | 1.12 | | | `RunAsGroup` | `true` | Beta | 1.14 | | -| `RuntimeClass` | `false` | Alpha | 1.12 | 1.13 | -| `RuntimeClass` | `true` | Beta | 1.14 | | | `SCTPSupport` | `false` | Alpha | 1.12 | 1.18 | | `SCTPSupport` | `true` | Beta | 1.19 | | | `ServerSideApply` | `false` | Alpha | 1.14 | 1.15 | | `ServerSideApply` | `true` | Beta | 1.16 | | -| `ServiceAccountIssuerDiscovery` | `false` | Alpha | 1.18 | | -| `ServiceLBNodePortControl` | `false` | Alpha | 1.20 | 1.20 | +| `ServiceAccountIssuerDiscovery` | `false` | Alpha | 1.18 | 1.19 | +| `ServiceAccountIssuerDiscovery` | `true` | Beta | 1.20 | | +| `ServiceLBNodePortControl` | `false` | Alpha | 1.20 | | | `ServiceNodeExclusion` | `false` | Alpha | 1.8 | 1.18 | | `ServiceNodeExclusion` | `true` | Beta | 1.19 | | | `ServiceTopology` | `false` | Alpha | 1.17 | | -| `SizeMemoryBackedVolumes` | `false` | Alpha | 1.20 | | | `SetHostnameAsFQDN` | `false` | Alpha | 1.19 | 1.19 | | `SetHostnameAsFQDN` | `true` | Beta | 1.20 | | +| `SizeMemoryBackedVolumes` | `false` | Alpha | 1.20 | | +| `StorageVersionAPI` | `false` | Alpha | 1.20 | | | `StorageVersionHash` | `false` | Alpha | 1.14 | 1.14 | | `StorageVersionHash` | `true` | Beta | 1.15 | | | `Sysctls` | `true` | Beta | 1.11 | | @@ -170,11 +175,11 @@ different Kubernetes components. | `TopologyManager` | `true` | Beta | 1.18 | | | `ValidateProxyRedirects` | `false` | Alpha | 1.12 | 1.13 | | `ValidateProxyRedirects` | `true` | Beta | 1.14 | | -| `WindowsEndpointSliceProxying` | `false` | Alpha | 1.19 | | -| `WindowsGMSA` | `false` | Alpha | 1.14 | | -| `WindowsGMSA` | `true` | Beta | 1.16 | | +| `WarningHeaders` | `true` | Beta | 1.19 | | | `WinDSR` | `false` | Alpha | 1.14 | | -| `WinOverlay` | `false` | Alpha | 1.14 | | +| `WinOverlay` | `false` | Alpha | 1.14 | 1.19 | +| `WinOverlay` | `true` | Beta | 1.20 | | +| `WindowsEndpointSliceProxying` | `false` | Alpha | 1.19 | | {{< /table >}} ### Feature gates for graduated or deprecated features @@ -228,6 +233,9 @@ different Kubernetes components. | `CustomResourceWebhookConversion` | `false` | Alpha | 1.13 | 1.14 | | `CustomResourceWebhookConversion` | `true` | Beta | 1.15 | 1.15 | | `CustomResourceWebhookConversion` | `true` | GA | 1.16 | - | +| `DryRun` | `false` | Alpha | 1.12 | 1.12 | +| `DryRun` | `true` | Beta | 1.13 | 1.18 | +| `DryRun` | `true` | GA | 1.19 | - | | `DynamicAuditing` | `false` | Alpha | 1.13 | 1.18 | | `DynamicAuditing` | - | Deprecated | 1.19 | - | | `DynamicProvisioningScheduling` | `false` | Alpha | 1.11 | 1.11 | @@ -247,23 +255,28 @@ different Kubernetes components. | `HugePages` | `false` | Alpha | 1.8 | 1.9 | | `HugePages` | `true` | Beta| 1.10 | 1.13 | | `HugePages` | `true` | GA | 1.14 | - | +| `HyperVContainer` | `false` | Alpha | 1.10 | 1.19 | +| `HyperVContainer` | `false` | Deprecated | 1.20 | - | | `Initializers` | `false` | Alpha | 1.7 | 1.13 | | `Initializers` | - | Deprecated | 1.14 | - | | `KubeletConfigFile` | `false` | Alpha | 1.8 | 1.9 | | `KubeletConfigFile` | - | Deprecated | 1.10 | - | -| `KubeletCredentialProviders` | `false` | Alpha | 1.20 | 1.20 | | `KubeletPluginsWatcher` | `false` | Alpha | 1.11 | 1.11 | | `KubeletPluginsWatcher` | `true` | Beta | 1.12 | 1.12 | | `KubeletPluginsWatcher` | `true` | GA | 1.13 | - | | `KubeletPodResources` | `false` | Alpha | 1.13 | 1.14 | | `KubeletPodResources` | `true` | Beta | 1.15 | | | `KubeletPodResources` | `true` | GA | 1.20 | | +| `MountContainers` | `false` | Alpha | 1.9 | 1.16 | +| `MountContainers` | `false` | Deprecated | 1.17 | - | | `MountPropagation` | `false` | Alpha | 1.8 | 1.9 | | `MountPropagation` | `true` | Beta | 1.10 | 1.11 | | `MountPropagation` | `true` | GA | 1.12 | - | | `NodeLease` | `false` | Alpha | 1.12 | 1.13 | | `NodeLease` | `true` | Beta | 1.14 | 1.16 | | `NodeLease` | `true` | GA | 1.17 | - | +| `PVCProtection` | `false` | Alpha | 1.9 | 1.9 | +| `PVCProtection` | - | Deprecated | 1.10 | - | | `PersistentLocalVolumes` | `false` | Alpha | 1.7 | 1.9 | | `PersistentLocalVolumes` | `true` | Beta | 1.10 | 1.13 | | `PersistentLocalVolumes` | `true` | GA | 1.14 | - | @@ -276,8 +289,6 @@ different Kubernetes components. | `PodShareProcessNamespace` | `false` | Alpha | 1.10 | 1.11 | | `PodShareProcessNamespace` | `true` | Beta | 1.12 | 1.16 | | `PodShareProcessNamespace` | `true` | GA | 1.17 | - | -| `PVCProtection` | `false` | Alpha | 1.9 | 1.9 | -| `PVCProtection` | - | Deprecated | 1.10 | - | | `RequestManagement` | `false` | Alpha | 1.15 | 1.16 | | `ResourceLimitsPriorityFunction` | `false` | Alpha | 1.9 | 1.18 | | `ResourceLimitsPriorityFunction` | - | Deprecated | 1.19 | - | @@ -398,65 +409,134 @@ A *General Availability* (GA) feature is also referred to as a *stable* feature. Each feature gate is designed for enabling/disabling a specific feature: +- `APIListChunking`: Enable the API clients to retrieve (`LIST` or `GET`) + resources from API server in chunks. +- `APIPriorityAndFairness`: Enable managing request concurrency with + prioritization and fairness at each server. (Renamed from `RequestManagement`) +- `APIResponseCompression`: Compress the API responses for `LIST` or `GET` requests. +- `APIServerIdentity`: Assign each API server an ID in a cluster. - `Accelerators`: Enable Nvidia GPU support when using Docker - `AdvancedAuditing`: Enable [advanced auditing](/docs/tasks/debug-application-cluster/audit/#advanced-audit) -- `AffinityInAnnotations`(*deprecated*): Enable setting [Pod affinity or anti-affinity](/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity). +- `AffinityInAnnotations`(*deprecated*): Enable setting + [Pod affinity or anti-affinity](/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity). - `AllowExtTrafficLocalEndpoints`: Enable a service to route external requests to node local endpoints. +- `AllowInsecureBackendProxy`: Enable the users to skip TLS verification of + kubelets on Pod log requests. - `AnyVolumeDataSource`: Enable use of any custom resource as the `DataSource` of a {{< glossary_tooltip text="PVC" term_id="persistent-volume-claim" >}}. -- `APIListChunking`: Enable the API clients to retrieve (`LIST` or `GET`) resources from API server in chunks. -- `APIPriorityAndFairness`: Enable managing request concurrency with prioritization and fairness at each server. (Renamed from `RequestManagement`) -- `APIResponseCompression`: Compress the API responses for `LIST` or `GET` requests. -- `APIServerIdentity`: Assign each kube-apiserver an ID in a cluster. - `AppArmor`: Enable AppArmor based mandatory access control on Linux nodes when using Docker. - See [AppArmor Tutorial](/docs/tutorials/clusters/apparmor/) for more details. + See [AppArmor Tutorial](/docs/tutorials/clusters/apparmor/) for more details. - `AttachVolumeLimit`: Enable volume plugins to report limits on number of volumes that can be attached to a node. - See [dynamic volume limits](/docs/concepts/storage/storage-limits/#dynamic-volume-limits) for more details. + See [dynamic volume limits](/docs/concepts/storage/storage-limits/#dynamic-volume-limits) for more details. - `BalanceAttachedNodeVolumes`: Include volume count on node to be considered for balanced resource allocation while scheduling. A node which has closer CPU, memory utilization, and volume count is favored by the scheduler while making decisions. - `BlockVolume`: Enable the definition and consumption of raw block devices in Pods. - See [Raw Block Volume Support](/docs/concepts/storage/persistent-volumes/#raw-block-volume-support) - for more details. + See [Raw Block Volume Support](/docs/concepts/storage/persistent-volumes/#raw-block-volume-support) + for more details. - `BoundServiceAccountTokenVolume`: Migrate ServiceAccount volumes to use a projected volume consisting of a - ServiceAccountTokenVolumeProjection. Cluster admins can use metric `serviceaccount_stale_tokens_total` to - monitor workloads that are depending on the extended tokens. If there are no such workloads, turn off - extended tokens by starting `kube-apiserver` with flag `--service-account-extend-token-expiration=false`. - Check [Bound Service Account Tokens](https://github.com/kubernetes/enhancements/blob/master/keps/sig-auth/1205-bound-service-account-tokens/README.md) + ServiceAccountTokenVolumeProjection. Cluster admins can use metric `serviceaccount_stale_tokens_total` to + monitor workloads that are depending on the extended tokens. If there are no such workloads, turn off + extended tokens by starting `kube-apiserver` with flag `--service-account-extend-token-expiration=false`. + Check [Bound Service Account Tokens](https://github.com/kubernetes/enhancements/blob/master/keps/sig-auth/1205-bound-service-account-tokens/README.md) for more details. -- `ConfigurableFSGroupPolicy`: Allows user to configure volume permission change policy for fsGroups when mounting a volume in a Pod. See [Configure volume permission and ownership change policy for Pods](/docs/tasks/configure-pod-container/security-context/#configure-volume-permission-and-ownership-change-policy-for-pods) for more details. -- `CronJobControllerV2`: Use an alternative implementation of the {{< glossary_tooltip text="CronJob" term_id="cronjob" >}} controller. Otherwise, version 1 of the same controller is selected. The version 2 controller provides experimental performance improvements. -- `CPUManager`: Enable container level CPU affinity support, see [CPU Management Policies](/docs/tasks/administer-cluster/cpu-management-policies/). +- `CPUManager`: Enable container level CPU affinity support, see + [CPU Management Policies](/docs/tasks/administer-cluster/cpu-management-policies/). - `CRIContainerLogRotation`: Enable container log rotation for cri container runtime. -- `CSIBlockVolume`: Enable external CSI volume drivers to support block storage. See the [`csi` raw block volume support](/docs/concepts/storage/volumes/#csi-raw-block-volume-support) documentation for more details. -- `CSIDriverRegistry`: Enable all logic related to the CSIDriver API object in csi.storage.k8s.io. +- `CSIBlockVolume`: Enable external CSI volume drivers to support block storage. + See the [`csi` raw block volume support](/docs/concepts/storage/volumes/#csi-raw-block-volume-support) + documentation for more details. +- `CSIDriverRegistry`: Enable all logic related to the CSIDriver API object in + csi.storage.k8s.io. - `CSIInlineVolume`: Enable CSI Inline volumes support for pods. -- `CSIMigration`: Enables shims and translation logic to route volume operations from in-tree plugins to corresponding pre-installed CSI plugins -- `CSIMigrationAWS`: Enables shims and translation logic to route volume operations from the AWS-EBS in-tree plugin to EBS CSI plugin. Supports falling back to in-tree EBS plugin if a node does not have EBS CSI plugin installed and configured. Requires CSIMigration feature flag enabled. -- `CSIMigrationAWSComplete`: Stops registering the EBS in-tree plugin in kubelet and volume controllers and enables shims and translation logic to route volume operations from the AWS-EBS in-tree plugin to EBS CSI plugin. Requires CSIMigration and CSIMigrationAWS feature flags enabled and EBS CSI plugin installed and configured on all nodes in the cluster. -- `CSIMigrationAzureDisk`: Enables shims and translation logic to route volume operations from the Azure-Disk in-tree plugin to AzureDisk CSI plugin. Supports falling back to in-tree AzureDisk plugin if a node does not have AzureDisk CSI plugin installed and configured. Requires CSIMigration feature flag enabled. -- `CSIMigrationAzureDiskComplete`: Stops registering the Azure-Disk in-tree plugin in kubelet and volume controllers and enables shims and translation logic to route volume operations from the Azure-Disk in-tree plugin to AzureDisk CSI plugin. Requires CSIMigration and CSIMigrationAzureDisk feature flags enabled and AzureDisk CSI plugin installed and configured on all nodes in the cluster. -- `CSIMigrationAzureFile`: Enables shims and translation logic to route volume operations from the Azure-File in-tree plugin to AzureFile CSI plugin. Supports falling back to in-tree AzureFile plugin if a node does not have AzureFile CSI plugin installed and configured. Requires CSIMigration feature flag enabled. -- `CSIMigrationAzureFileComplete`: Stops registering the Azure-File in-tree plugin in kubelet and volume controllers and enables shims and translation logic to route volume operations from the Azure-File in-tree plugin to AzureFile CSI plugin. Requires CSIMigration and CSIMigrationAzureFile feature flags enabled and AzureFile CSI plugin installed and configured on all nodes in the cluster. -- `CSIMigrationGCE`: Enables shims and translation logic to route volume operations from the GCE-PD in-tree plugin to PD CSI plugin. Supports falling back to in-tree GCE plugin if a node does not have PD CSI plugin installed and configured. Requires CSIMigration feature flag enabled. -- `CSIMigrationGCEComplete`: Stops registering the GCE-PD in-tree plugin in kubelet and volume controllers and enables shims and translation logic to route volume operations from the GCE-PD in-tree plugin to PD CSI plugin. Requires CSIMigration and CSIMigrationGCE feature flags enabled and PD CSI plugin installed and configured on all nodes in the cluster. -- `CSIMigrationOpenStack`: Enables shims and translation logic to route volume operations from the Cinder in-tree plugin to Cinder CSI plugin. Supports falling back to in-tree Cinder plugin if a node does not have Cinder CSI plugin installed and configured. Requires CSIMigration feature flag enabled. -- `CSIMigrationOpenStackComplete`: Stops registering the Cinder in-tree plugin in kubelet and volume controllers and enables shims and translation logic to route volume operations from the Cinder in-tree plugin to Cinder CSI plugin. Requires CSIMigration and CSIMigrationOpenStack feature flags enabled and Cinder CSI plugin installed and configured on all nodes in the cluster. -- `CSIMigrationvSphere`: Enables shims and translation logic to route volume operations from the vSphere in-tree plugin to vSphere CSI plugin. Supports falling back to in-tree vSphere plugin if a node does not have vSphere CSI plugin installed and configured. Requires CSIMigration feature flag enabled. -- `CSIMigrationvSphereComplete`: Stops registering the vSphere in-tree plugin in kubelet and volume controllers and enables shims and translation logic to route volume operations from the vSphere in-tree plugin to vSphere CSI plugin. Requires CSIMigration and CSIMigrationvSphere feature flags enabled and vSphere CSI plugin installed and configured on all nodes in the cluster. +- `CSIMigration`: Enables shims and translation logic to route volume + operations from in-tree plugins to corresponding pre-installed CSI plugins +- `CSIMigrationAWS`: Enables shims and translation logic to route volume + operations from the AWS-EBS in-tree plugin to EBS CSI plugin. Supports + falling back to in-tree EBS plugin if a node does not have EBS CSI plugin + installed and configured. Requires CSIMigration feature flag enabled. +- `CSIMigrationAWSComplete`: Stops registering the EBS in-tree plugin in + kubelet and volume controllers and enables shims and translation logic to + route volume operations from the AWS-EBS in-tree plugin to EBS CSI plugin. + Requires CSIMigration and CSIMigrationAWS feature flags enabled and EBS CSI + plugin installed and configured on all nodes in the cluster. +- `CSIMigrationAzureDisk`: Enables shims and translation logic to route volume + operations from the Azure-Disk in-tree plugin to AzureDisk CSI plugin. + Supports falling back to in-tree AzureDisk plugin if a node does not have + AzureDisk CSI plugin installed and configured. Requires CSIMigration feature + flag enabled. +- `CSIMigrationAzureDiskComplete`: Stops registering the Azure-Disk in-tree + plugin in kubelet and volume controllers and enables shims and translation + logic to route volume operations from the Azure-Disk in-tree plugin to + AzureDisk CSI plugin. Requires CSIMigration and CSIMigrationAzureDisk feature + flags enabled and AzureDisk CSI plugin installed and configured on all nodes + in the cluster. +- `CSIMigrationAzureFile`: Enables shims and translation logic to route volume + operations from the Azure-File in-tree plugin to AzureFile CSI plugin. + Supports falling back to in-tree AzureFile plugin if a node does not have + AzureFile CSI plugin installed and configured. Requires CSIMigration feature + flag enabled. +- `CSIMigrationAzureFileComplete`: Stops registering the Azure-File in-tree + plugin in kubelet and volume controllers and enables shims and translation + logic to route volume operations from the Azure-File in-tree plugin to + AzureFile CSI plugin. Requires CSIMigration and CSIMigrationAzureFile feature + flags enabled and AzureFile CSI plugin installed and configured on all nodes + in the cluster. +- `CSIMigrationGCE`: Enables shims and translation logic to route volume + operations from the GCE-PD in-tree plugin to PD CSI plugin. Supports falling + back to in-tree GCE plugin if a node does not have PD CSI plugin installed and + configured. Requires CSIMigration feature flag enabled. +- `CSIMigrationGCEComplete`: Stops registering the GCE-PD in-tree plugin in + kubelet and volume controllers and enables shims and translation logic to + route volume operations from the GCE-PD in-tree plugin to PD CSI plugin. + Requires CSIMigration and CSIMigrationGCE feature flags enabled and PD CSI + plugin installed and configured on all nodes in the cluster. +- `CSIMigrationOpenStack`: Enables shims and translation logic to route volume + operations from the Cinder in-tree plugin to Cinder CSI plugin. Supports + falling back to in-tree Cinder plugin if a node does not have Cinder CSI + plugin installed and configured. Requires CSIMigration feature flag enabled. +- `CSIMigrationOpenStackComplete`: Stops registering the Cinder in-tree plugin in + kubelet and volume controllers and enables shims and translation logic to route + volume operations from the Cinder in-tree plugin to Cinder CSI plugin. + Requires CSIMigration and CSIMigrationOpenStack feature flags enabled and Cinder + CSI plugin installed and configured on all nodes in the cluster. +- `CSIMigrationvSphere`: Enables shims and translation logic to route volume operations + from the vSphere in-tree plugin to vSphere CSI plugin. + Supports falling back to in-tree vSphere plugin if a node does not have vSphere + CSI plugin installed and configured. Requires CSIMigration feature flag enabled. +- `CSIMigrationvSphereComplete`: Stops registering the vSphere in-tree plugin in kubelet + and volume controllers and enables shims and translation logic to route volume operations + from the vSphere in-tree plugin to vSphere CSI plugin. Requires CSIMigration and + CSIMigrationvSphere feature flags enabled and vSphere CSI plugin installed and + configured on all nodes in the cluster. - `CSINodeInfo`: Enable all logic related to the CSINodeInfo API object in csi.storage.k8s.io. - `CSIPersistentVolume`: Enable discovering and mounting volumes provisioned through a [CSI (Container Storage Interface)](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/container-storage-interface.md) compatible volume plugin. -- `CSIServiceAccountToken`: Enable CSI drivers to receive the pods' service account token that they mount volumes for. See [Token Requests](https://kubernetes-csi.github.io/docs/token-requests.html). -- `CSIStorageCapacity`: Enables CSI drivers to publish storage capacity information and the Kubernetes scheduler to use that information when scheduling pods. See [Storage Capacity](/docs/concepts/storage/storage-capacity/). +- `CSIServiceAccountToken`: Enable CSI drivers to receive the pods' service account token + that they mount volumes for. See + [Token Requests](https://kubernetes-csi.github.io/docs/token-requests.html). +- `CSIStorageCapacity`: Enables CSI drivers to publish storage capacity information + and the Kubernetes scheduler to use that information when scheduling pods. See + [Storage Capacity](/docs/concepts/storage/storage-capacity/). Check the [`csi` volume type](/docs/concepts/storage/volumes/#csi) documentation for more details. -- `CSIVolumeFSGroupPolicy`: Allows CSIDrivers to use the `fsGroupPolicy` field. This field controls whether volumes created by a CSIDriver support volume ownership and permission modifications when these volumes are mounted. -- `CustomCPUCFSQuotaPeriod`: Enable nodes to change CPUCFSQuotaPeriod. +- `CSIVolumeFSGroupPolicy`: Allows CSIDrivers to use the `fsGroupPolicy` field. + This field controls whether volumes created by a CSIDriver support volume ownership + and permission modifications when these volumes are mounted. +- `ConfigurableFSGroupPolicy`: Allows user to configure volume permission change policy + for fsGroups when mounting a volume in a Pod. See + [Configure volume permission and ownership change policy for Pods](/docs/tasks/configure-pod-container/security-context/#configure-volume-permission-and-ownership-change-policy-for-pods) + for more details. +- `CronJobControllerV2`: Use an alternative implementation of the + {{< glossary_tooltip text="CronJob" term_id="cronjob" >}} controller. Otherwise, + version 1 of the same controller is selected. + The version 2 controller provides experimental performance improvements. +- `CustomCPUCFSQuotaPeriod`: Enable nodes to change `cpuCFSQuotaPeriod` in + [kubelet config](/docs/tasks/administer-cluster/kubelet-config-file/). - `CustomPodDNS`: Enable customizing the DNS settings for a Pod using its `dnsConfig` property. - Check [Pod's DNS Config](/docs/concepts/services-networking/dns-pod-service/#pods-dns-config) - for more details. + Check [Pod's DNS Config](/docs/concepts/services-networking/dns-pod-service/#pods-dns-config) + for more details. - `CustomResourceDefaulting`: Enable CRD support for default values in OpenAPI v3 validation schemas. - `CustomResourcePublishOpenAPI`: Enables publishing of CRD OpenAPI specs. - `CustomResourceSubresources`: Enable `/status` and `/scale` subresources @@ -466,147 +546,253 @@ Each feature gate is designed for enabling/disabling a specific feature: - `CustomResourceWebhookConversion`: Enable webhook-based conversion on resources created from [CustomResourceDefinition](/docs/concepts/extend-kubernetes/api-extension/custom-resources/). troubleshoot a running Pod. -- `DisableAcceleratorUsageMetrics`: [Disable accelerator metrics collected by the kubelet](/docs/concepts/cluster-administration/system-metrics/#disable-accelerator-metrics). -- `DevicePlugins`: Enable the [device-plugins](/docs/concepts/cluster-administration/device-plugins/) - based resource provisioning on nodes. - `DefaultPodTopologySpread`: Enables the use of `PodTopologySpread` scheduling plugin to do [default spreading](/docs/concepts/workloads/pods/pod-topology-spread-constraints/#internal-default-constraints). -- `DownwardAPIHugePages`: Enables usage of hugepages in downward API. +- `DevicePlugins`: Enable the [device-plugins](/docs/concepts/cluster-administration/device-plugins/) + based resource provisioning on nodes. +- `DisableAcceleratorUsageMetrics`: + [Disable accelerator metrics collected by the kubelet](/docs/concepts/cluster-administration/system-metrics/#disable-accelerator-metrics). +- `DownwardAPIHugePages`: Enables usage of hugepages in + [downward API](/docs/tasks/inject-data-application/downward-api-volume-expose-pod-information). - `DryRun`: Enable server-side [dry run](/docs/reference/using-api/api-concepts/#dry-run) requests so that validation, merging, and mutation can be tested without committing. - `DynamicAuditing`(*deprecated*): Used to enable dynamic auditing before v1.19. -- `DynamicKubeletConfig`: Enable the dynamic configuration of kubelet. See [Reconfigure kubelet](/docs/tasks/administer-cluster/reconfigure-kubelet/). -- `DynamicProvisioningScheduling`: Extend the default scheduler to be aware of volume topology and handle PV provisioning. +- `DynamicKubeletConfig`: Enable the dynamic configuration of kubelet. See + [Reconfigure kubelet](/docs/tasks/administer-cluster/reconfigure-kubelet/). +- `DynamicProvisioningScheduling`: Extend the default scheduler to be aware of + volume topology and handle PV provisioning. This feature is superseded by the `VolumeScheduling` feature completely in v1.12. -- `DynamicVolumeProvisioning`(*deprecated*): Enable the [dynamic provisioning](/docs/concepts/storage/dynamic-provisioning/) of persistent volumes to Pods. -- `EnableAggregatedDiscoveryTimeout` (*deprecated*): Enable the five second timeout on aggregated discovery calls. -- `EnableEquivalenceClassCache`: Enable the scheduler to cache equivalence of nodes when scheduling Pods. -- `EphemeralContainers`: Enable the ability to add {{< glossary_tooltip text="ephemeral containers" - term_id="ephemeral-container" >}} to running pods. -- `EvenPodsSpread`: Enable pods to be scheduled evenly across topology domains. See [Pod Topology Spread Constraints](/docs/concepts/workloads/pods/pod-topology-spread-constraints/). -- `ExecProbeTimeout`: Ensure kubelet respects exec probe timeouts. This feature gate exists in case any of your existing workloads depend on a now-corrected fault where Kubernetes ignored exec probe timeouts. See [readiness probes](/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#configure-probes). -- `ExpandInUsePersistentVolumes`: Enable expanding in-use PVCs. See [Resizing an in-use PersistentVolumeClaim](/docs/concepts/storage/persistent-volumes/#resizing-an-in-use-persistentvolumeclaim). -- `ExpandPersistentVolumes`: Enable the expanding of persistent volumes. See [Expanding Persistent Volumes Claims](/docs/concepts/storage/persistent-volumes/#expanding-persistent-volumes-claims). -- `ExperimentalCriticalPodAnnotation`: Enable annotating specific pods as *critical* so that their [scheduling is guaranteed](/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods/). - This feature is deprecated by Pod Priority and Preemption as of v1.13. -- `ExperimentalHostUserNamespaceDefaultingGate`: Enabling the defaulting user - namespace to host. This is for containers that are using other host namespaces, - host mounts, or containers that are privileged or using specific non-namespaced - capabilities (e.g. `MKNODE`, `SYS_MODULE` etc.). This should only be enabled - if user namespace remapping is enabled in the Docker daemon. -- `EndpointSlice`: Enables Endpoint Slices for more scalable and extensible - network endpoints. See [Enabling Endpoint Slices](/docs/tasks/administer-cluster/enabling-endpointslices/). +- `DynamicVolumeProvisioning`(*deprecated*): Enable the + [dynamic provisioning](/docs/concepts/storage/dynamic-provisioning/) of persistent volumes to Pods. +- `EfficientWatchResumption`: Allows for storage-originated bookmark (progress + notify) events to be delivered to the users. This is only applied to watch + operations. +- `EnableAggregatedDiscoveryTimeout` (*deprecated*): Enable the five second + timeout on aggregated discovery calls. +- `EnableEquivalenceClassCache`: Enable the scheduler to cache equivalence of + nodes when scheduling Pods. +- `EndpointSlice`: Enables EndpointSlices for more scalable and extensible + network endpoints. See [Enabling EndpointSlices](/docs/tasks/administer-cluster/enabling-endpointslices/). - `EndpointSliceNodeName`: Enables EndpointSlice `nodeName` field. -- `EndpointSliceTerminating`: Enables EndpointSlice `terminating` and `serving` - condition fields. -- `EndpointSliceProxying`: When this feature gate is enabled, kube-proxy running +- `EndpointSliceProxying`: When enabled, kube-proxy running on Linux will use EndpointSlices as the primary data source instead of Endpoints, enabling scalability and performance improvements. See [Enabling Endpoint Slices](/docs/tasks/administer-cluster/enabling-endpointslices/). -- `WindowsEndpointSliceProxying`: When this feature gate is enabled, kube-proxy - running on Windows will use EndpointSlices as the primary data source instead - of Endpoints, enabling scalability and performance improvements. See - [Enabling Endpoint Slices](/docs/tasks/administer-cluster/enabling-endpointslices/). +- `EndpointSliceTerminatingCondition`: Enables EndpointSlice `terminating` and `serving` + condition fields. +- `EphemeralContainers`: Enable the ability to add + {{< glossary_tooltip text="ephemeral containers" term_id="ephemeral-container" >}} + to running pods. +- `EvenPodsSpread`: Enable pods to be scheduled evenly across topology domains. See + [Pod Topology Spread Constraints](/docs/concepts/workloads/pods/pod-topology-spread-constraints/). +- `ExecProbeTimeout`: Ensure kubelet respects exec probe timeouts. + This feature gate exists in case any of your existing workloads depend on a + now-corrected fault where Kubernetes ignored exec probe timeouts. See + [readiness probes](/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#configure-probes). +- `ExpandCSIVolumes`: Enable the expanding of CSI volumes. +- `ExpandInUsePersistentVolumes`: Enable expanding in-use PVCs. See + [Resizing an in-use PersistentVolumeClaim](/docs/concepts/storage/persistent-volumes/#resizing-an-in-use-persistentvolumeclaim). +- `ExpandPersistentVolumes`: Enable the expanding of persistent volumes. See + [Expanding Persistent Volumes Claims](/docs/concepts/storage/persistent-volumes/#expanding-persistent-volumes-claims). +- `ExperimentalCriticalPodAnnotation`: Enable annotating specific pods as *critical* + so that their [scheduling is guaranteed](/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods/). + This feature is deprecated by Pod Priority and Preemption as of v1.13. +- `ExperimentalHostUserNamespaceDefaulting`: Enabling the defaulting user + namespace to host. This is for containers that are using other host namespaces, + host mounts, or containers that are privileged or using specific non-namespaced + capabilities (e.g. `MKNODE`, `SYS_MODULE` etc.). This should only be enabled + if user namespace remapping is enabled in the Docker daemon. - `GCERegionalPersistentDisk`: Enable the regional PD feature on GCE. -- `GenericEphemeralVolume`: Enables ephemeral, inline volumes that support all features of normal volumes (can be provided by third-party storage vendors, storage capacity tracking, restore from snapshot, etc.). See [Ephemeral Volumes](/docs/concepts/storage/ephemeral-volumes/). -- `GracefulNodeShutdown`: Enables support for graceful shutdown in kubelet. During a system shutdown, kubelet will attempt to detect the shutdown event and gracefully terminate pods running on the node. See [Graceful Node Shutdown](/docs/concepts/architecture/nodes/#graceful-node-shutdown) for more details. -- `HugePages`: Enable the allocation and consumption of pre-allocated [huge pages](/docs/tasks/manage-hugepages/scheduling-hugepages/). -- `HugePageStorageMediumSize`: Enable support for multiple sizes pre-allocated [huge pages](/docs/tasks/manage-hugepages/scheduling-hugepages/). -- `HyperVContainer`: Enable [Hyper-V isolation](https://docs.microsoft.com/en-us/virtualization/windowscontainers/manage-containers/hyperv-container) for Windows containers. -- `HPAScaleToZero`: Enables setting `minReplicas` to 0 for `HorizontalPodAutoscaler` resources when using custom or external metrics. -- `ImmutableEphemeralVolumes`: Allows for marking individual Secrets and ConfigMaps as immutable for better safety and performance. -- `KubeletConfigFile`: Enable loading kubelet configuration from a file specified using a config file. - See [setting kubelet parameters via a config file](/docs/tasks/administer-cluster/kubelet-config-file/) for more details. +- `GenericEphemeralVolume`: Enables ephemeral, inline volumes that support all features + of normal volumes (can be provided by third-party storage vendors, storage capacity tracking, + restore from snapshot, etc.). + See [Ephemeral Volumes](/docs/concepts/storage/ephemeral-volumes/). +- `GracefulNodeShutdown`: Enables support for graceful shutdown in kubelet. + During a system shutdown, kubelet will attempt to detect the shutdown event + and gracefully terminate pods running on the node. See + [Graceful Node Shutdown](/docs/concepts/architecture/nodes/#graceful-node-shutdown) + for more details. +- `HPAContainerMetrics`: Enable the `HorizontalPodAutoscaler` to scale based on + metrics from individual containers in target pods. +- `HPAScaleToZero`: Enables setting `minReplicas` to 0 for `HorizontalPodAutoscaler` + resources when using custom or external metrics. +- `HugePages`: Enable the allocation and consumption of pre-allocated + [huge pages](/docs/tasks/manage-hugepages/scheduling-hugepages/). +- `HugePageStorageMediumSize`: Enable support for multiple sizes pre-allocated + [huge pages](/docs/tasks/manage-hugepages/scheduling-hugepages/). +- `HyperVContainer`: Enable + [Hyper-V isolation](https://docs.microsoft.com/en-us/virtualization/windowscontainers/manage-containers/hyperv-container) + for Windows containers. +- `IPv6DualStack`: Enable [dual stack](/docs/concepts/services-networking/dual-stack/) + support for IPv6. +- `ImmutableEphemeralVolumes`: Allows for marking individual Secrets and ConfigMaps as + immutable for better safety and performance. +- `KubeletConfigFile` (*deprecated*): Enable loading kubelet configuration from + a file specified using a config file. + See [setting kubelet parameters via a config file](/docs/tasks/administer-cluster/kubelet-config-file/) + for more details. - `KubeletCredentialProviders`: Enable kubelet exec credential providers for image pull credentials. - `KubeletPluginsWatcher`: Enable probe-based plugin watcher utility to enable kubelet to discover plugins such as [CSI volume drivers](/docs/concepts/storage/volumes/#csi). -- `KubeletPodResources`: Enable the kubelet's pod resources grpc endpoint. - See [Support Device Monitoring](https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/compute-device-assignment.md) for more details. -- `LegacyNodeRoleBehavior`: When disabled, legacy behavior in service load balancers and node disruption will ignore the `node-role.kubernetes.io/master` label in favor of the feature-specific labels provided by `NodeDisruptionExclusion` and `ServiceNodeExclusion`. -- `LocalStorageCapacityIsolation`: Enable the consumption of [local ephemeral storage](/docs/concepts/configuration/manage-resources-containers/) and also the `sizeLimit` property of an [emptyDir volume](/docs/concepts/storage/volumes/#emptydir). -- `LocalStorageCapacityIsolationFSQuotaMonitoring`: When `LocalStorageCapacityIsolation` is enabled for [local ephemeral storage](/docs/concepts/configuration/manage-resources-containers/) and the backing filesystem for [emptyDir volumes](/docs/concepts/storage/volumes/#emptydir) supports project quotas and they are enabled, use project quotas to monitor [emptyDir volume](/docs/concepts/storage/volumes/#emptydir) storage consumption rather than filesystem walk for better performance and accuracy. -- `MixedProtocolLBService`: Enable using different protocols in the same LoadBalancer type Service instance. -- `MountContainers`: Enable using utility containers on host as the volume mounter. +- `KubeletPodResources`: Enable the kubelet's pod resources GRPC endpoint. See + [Support Device Monitoring](https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/compute-device-assignment.md) + for more details. +- `LegacyNodeRoleBehavior`: When disabled, legacy behavior in service load balancers and + node disruption will ignore the `node-role.kubernetes.io/master` label in favor of the + feature-specific labels provided by `NodeDisruptionExclusion` and `ServiceNodeExclusion`. +- `LocalStorageCapacityIsolation`: Enable the consumption of + [local ephemeral storage](/docs/concepts/configuration/manage-resources-containers/) + and also the `sizeLimit` property of an + [emptyDir volume](/docs/concepts/storage/volumes/#emptydir). +- `LocalStorageCapacityIsolationFSQuotaMonitoring`: When `LocalStorageCapacityIsolation` + is enabled for + [local ephemeral storage](/docs/concepts/configuration/manage-resources-containers/) + and the backing filesystem for [emptyDir volumes](/docs/concepts/storage/volumes/#emptydir) + supports project quotas and they are enabled, use project quotas to monitor + [emptyDir volume](/docs/concepts/storage/volumes/#emptydir) storage consumption rather than + filesystem walk for better performance and accuracy. +- `MixedProtocolLBService`: Enable using different protocols in the same `LoadBalancer` type + Service instance. +- `MountContainers` (*deprecated*): Enable using utility containers on host as + the volume mounter. - `MountPropagation`: Enable sharing volume mounted by one container to other containers or pods. For more details, please see [mount propagation](/docs/concepts/storage/volumes/#mount-propagation). -- `NodeDisruptionExclusion`: Enable use of the node label `node.kubernetes.io/exclude-disruption` which prevents nodes from being evacuated during zone failures. +- `NodeDisruptionExclusion`: Enable use of the Node label `node.kubernetes.io/exclude-disruption` + which prevents nodes from being evacuated during zone failures. - `NodeLease`: Enable the new Lease API to report node heartbeats, which could be used as a node health signal. -- `NonPreemptingPriority`: Enable NonPreempting option for PriorityClass and Pod. +- `NonPreemptingPriority`: Enable `preemptionPolicy` field for PriorityClass and Pod. +- `PVCProtection`: Enable the prevention of a PersistentVolumeClaim (PVC) from + being deleted when it is still used by any Pod. - `PersistentLocalVolumes`: Enable the usage of `local` volume type in Pods. Pod affinity has to be specified if requesting a `local` volume. - `PodDisruptionBudget`: Enable the [PodDisruptionBudget](/docs/tasks/run-application/configure-pdb/) feature. -- `PodOverhead`: Enable the [PodOverhead](/docs/concepts/scheduling-eviction/pod-overhead/) feature to account for pod overheads. -- `PodPriority`: Enable the descheduling and preemption of Pods based on their [priorities](/docs/concepts/configuration/pod-priority-preemption/). +- `PodOverhead`: Enable the [PodOverhead](/docs/concepts/scheduling-eviction/pod-overhead/) + feature to account for pod overheads. +- `PodPriority`: Enable the descheduling and preemption of Pods based on their + [priorities](/docs/concepts/configuration/pod-priority-preemption/). - `PodReadinessGates`: Enable the setting of `PodReadinessGate` field for extending Pod readiness evaluation. See [Pod readiness gate](/docs/concepts/workloads/pods/pod-lifecycle/#pod-readiness-gate) for more details. - `PodShareProcessNamespace`: Enable the setting of `shareProcessNamespace` in a Pod for sharing a single process namespace between containers running in a pod. More details can be found in [Share Process Namespace between Containers in a Pod](/docs/tasks/configure-pod-container/share-process-namespace/). -- `ProcMountType`: Enables control over ProcMountType for containers. -- `PVCProtection`: Enable the prevention of a PersistentVolumeClaim (PVC) from - being deleted when it is still used by any Pod. -- `QOSReserved`: Allows resource reservations at the QoS level preventing pods at lower QoS levels from - bursting into resources requested at higher QoS levels (memory only for now). +- `ProcMountType`: Enables control over the type proc mounts for containers + by setting the `procMount` field of a SecurityContext. +- `QOSReserved`: Allows resource reservations at the QoS level preventing pods + at lower QoS levels from bursting into resources requested at higher QoS levels + (memory only for now). +- `RemainingItemCount`: Allow the API servers to show a count of remaining + items in the response to a + [chunking list request](/docs/reference/using-api/api-concepts/#retrieving-large-results-sets-in-chunks). +- `RemoveSelfLink`: Deprecates and removes `selfLink` from ObjectMeta and + ListMeta. - `ResourceLimitsPriorityFunction` (*deprecated*): Enable a scheduler priority function that assigns a lowest possible score of 1 to a node that satisfies at least one of the input Pod's cpu and memory limits. The intent is to break ties between nodes with same scores. - `ResourceQuotaScopeSelectors`: Enable resource quota scope selectors. -- `RootCAConfigMap`: Configure the kube-controller-manager to publish a {{< glossary_tooltip text="ConfigMap" term_id="configmap" >}} named `kube-root-ca.crt` to every namespace. This ConfigMap contains a CA bundle used for verifying connections to the kube-apiserver. - See [Bound Service Account Tokens](https://github.com/kubernetes/enhancements/blob/master/keps/sig-auth/1205-bound-service-account-tokens/README.md) for more details. +- `RootCAConfigMap`: Configure the `kube-controller-manager` to publish a + {{< glossary_tooltip text="ConfigMap" term_id="configmap" >}} named `kube-root-ca.crt` + to every namespace. This ConfigMap contains a CA bundle used for verifying connections + to the kube-apiserver. See + [Bound Service Account Tokens](https://github.com/kubernetes/enhancements/blob/master/keps/sig-auth/1205-bound-service-account-tokens/README.md) + for more details. - `RotateKubeletClientCertificate`: Enable the rotation of the client TLS certificate on the kubelet. See [kubelet configuration](/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping/#kubelet-configuration) for more details. - `RotateKubeletServerCertificate`: Enable the rotation of the server TLS certificate on the kubelet. - See [kubelet configuration](/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping/#kubelet-configuration) for more details. -- `RunAsGroup`: Enable control over the primary group ID set on the init processes of containers. -- `RuntimeClass`: Enable the [RuntimeClass](/docs/concepts/containers/runtime-class/) feature for selecting container runtime configurations. -- `ScheduleDaemonSetPods`: Enable DaemonSet Pods to be scheduled by the default scheduler instead of the DaemonSet controller. -- `SCTPSupport`: Enables the _SCTP_ `protocol` value in Pod, Service, Endpoints, EndpointSlice, and NetworkPolicy definitions. -- `ServerSideApply`: Enables the [Sever Side Apply (SSA)](/docs/reference/using-api/server-side-apply/) path at the API Server. -- `ServiceAccountIssuerDiscovery`: Enable OIDC discovery endpoints (issuer and JWKS URLs) for the service account issuer in the API server. See [Configure Service Accounts for Pods](/docs/tasks/configure-pod-container/configure-service-account/#service-account-issuer-discovery) for more details. + See [kubelet configuration](/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping/#kubelet-configuration) + for more details. +- `RunAsGroup`: Enable control over the primary group ID set on the init + processes of containers. +- `RuntimeClass`: Enable the [RuntimeClass](/docs/concepts/containers/runtime-class/) feature + for selecting container runtime configurations. +- `ScheduleDaemonSetPods`: Enable DaemonSet Pods to be scheduled by the default scheduler + instead of the DaemonSet controller. +- `SCTPSupport`: Enables the _SCTP_ `protocol` value in Pod, Service, + Endpoints, EndpointSlice, and NetworkPolicy definitions. +- `ServerSideApply`: Enables the [Sever Side Apply (SSA)](/docs/reference/using-api/server-side-apply/) + feature on the API Server. +- `ServiceAccountIssuerDiscovery`: Enable OIDC discovery endpoints (issuer and + JWKS URLs) for the service account issuer in the API server. See + [Configure Service Accounts for Pods](/docs/tasks/configure-pod-container/configure-service-account/#service-account-issuer-discovery) + for more details. - `ServiceAppProtocol`: Enables the `AppProtocol` field on Services and Endpoints. -- `ServiceLBNodePortControl`: Enables the `spec.allocateLoadBalancerNodePorts` field on Services. +- `ServiceLBNodePortControl`: Enables the `spec.allocateLoadBalancerNodePorts` + field on Services. - `ServiceLoadBalancerFinalizer`: Enable finalizer protection for Service load balancers. -- `ServiceNodeExclusion`: Enable the exclusion of nodes from load balancers created by a cloud provider. - A node is eligible for exclusion if labelled with "`alpha.service-controller.kubernetes.io/exclude-balancer`" key or `node.kubernetes.io/exclude-from-external-load-balancers`. -- `ServiceTopology`: Enable service to route traffic based upon the Node topology of the cluster. See [ServiceTopology](/docs/concepts/services-networking/service-topology/) for more details. -- `SizeMemoryBackedVolumes`: Enables kubelet support to size memory backed volumes. See [volumes](docs/concepts/storage/volumes) for more details. -- `SetHostnameAsFQDN`: Enable the ability of setting Fully Qualified Domain Name(FQDN) as hostname of pod. See [Pod's `setHostnameAsFQDN` field](/docs/concepts/services-networking/dns-pod-service/#pod-sethostnameasfqdn-field). -- `StartupProbe`: Enable the [startup](/docs/concepts/workloads/pods/pod-lifecycle/#when-should-you-use-a-startup-probe) probe in the kubelet. +- `ServiceNodeExclusion`: Enable the exclusion of nodes from load balancers + created by a cloud provider. A node is eligible for exclusion if labelled with + "`node.kubernetes.io/exclude-from-external-load-balancers`". +- `ServiceTopology`: Enable service to route traffic based upon the Node + topology of the cluster. See + [ServiceTopology](/docs/concepts/services-networking/service-topology/) + for more details. +- `SizeMemoryBackedVolumes`: Enables kubelet support to size memory backed volumes. + See [volumes](docs/concepts/storage/volumes) for more details. +- `SetHostnameAsFQDN`: Enable the ability of setting Fully Qualified Domain + Name(FQDN) as the hostname of a pod. See + [Pod's `setHostnameAsFQDN` field](/docs/concepts/services-networking/dns-pod-service/#pod-sethostnameasfqdn-field). +- `SizeMemoryBackedVolumes`: Enable kubelets to determine the size limit for + memory-backed volumes (mainly `emptyDir` volumes). +- `StartupProbe`: Enable the + [startup](/docs/concepts/workloads/pods/pod-lifecycle/#when-should-you-use-a-startup-probe) + probe in the kubelet. - `StorageObjectInUseProtection`: Postpone the deletion of PersistentVolume or PersistentVolumeClaim objects if they are still being used. -- `StorageVersionHash`: Allow apiservers to expose the storage version hash in the discovery. +- `StorageVersionAPI`: Enable the + [storage version API](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#storageversion-v1alpha1-internal-apiserver-k8s-io). +- `StorageVersionHash`: Allow API servers to expose the storage version hash in the + discovery. - `StreamingProxyRedirects`: Instructs the API server to intercept (and follow) - redirects from the backend (kubelet) for streaming requests. + redirects from the backend (kubelet) for streaming requests. Examples of streaming requests include the `exec`, `attach` and `port-forward` requests. - `SupportIPVSProxyMode`: Enable providing in-cluster service load balancing using IPVS. See [service proxies](/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies) for more details. - `SupportPodPidsLimit`: Enable the support to limiting PIDs in Pods. -- `SupportNodePidsLimit`: Enable the support to limiting PIDs on the Node. The parameter `pid=` in the `--system-reserved` and `--kube-reserved` options can be specified to ensure that the specified number of process IDs will be reserved for the system as a whole and for Kubernetes system daemons respectively. -- `Sysctls`: Enable support for namespaced kernel parameters (sysctls) that can be set for each pod. - See [sysctls](/docs/tasks/administer-cluster/sysctl-cluster/) for more details. -- `TaintBasedEvictions`: Enable evicting pods from nodes based on taints on nodes and tolerations on Pods. - See [taints and tolerations](/docs/concepts/scheduling-eviction/taint-and-toleration/) for more details. -- `TaintNodesByCondition`: Enable automatic tainting nodes based on [node conditions](/docs/concepts/architecture/nodes/#condition). +- `SupportNodePidsLimit`: Enable the support to limiting PIDs on the Node. + The parameter `pid=` in the `--system-reserved` and `--kube-reserved` + options can be specified to ensure that the specified number of process IDs + will be reserved for the system as a whole and for Kubernetes system daemons + respectively. +- `Sysctls`: Enable support for namespaced kernel parameters (sysctls) that can be + set for each pod. See + [sysctls](/docs/tasks/administer-cluster/sysctl-cluster/) for more details. +- `TTLAfterFinished`: Allow a + [TTL controller](/docs/concepts/workloads/controllers/ttlafterfinished/) + to clean up resources after they finish execution. +- `TaintBasedEvictions`: Enable evicting pods from nodes based on taints on Nodes + and tolerations on Pods. + See [taints and tolerations](/docs/concepts/scheduling-eviction/taint-and-toleration/) + for more details. +- `TaintNodesByCondition`: Enable automatic tainting nodes based on + [node conditions](/docs/concepts/architecture/nodes/#condition). - `TokenRequest`: Enable the `TokenRequest` endpoint on service account resources. -- `TokenRequestProjection`: Enable the injection of service account tokens into - a Pod through the [`projected` volume](/docs/concepts/storage/volumes/#projected). -- `TopologyManager`: Enable a mechanism to coordinate fine-grained hardware resource assignments for different components in Kubernetes. See [Control Topology Management Policies on a node](/docs/tasks/administer-cluster/topology-manager/). -- `TTLAfterFinished`: Allow a [TTL controller](/docs/concepts/workloads/controllers/ttlafterfinished/) to clean up resources after they finish execution. +- `TokenRequestProjection`: Enable the injection of service account tokens into a + Pod through a [`projected` volume](/docs/concepts/storage/volumes/#projected). +- `TopologyManager`: Enable a mechanism to coordinate fine-grained hardware resource + assignments for different components in Kubernetes. See + [Control Topology Management Policies on a node](/docs/tasks/administer-cluster/topology-manager/). - `VolumePVCDataSource`: Enable support for specifying an existing PVC as a DataSource. - `VolumeScheduling`: Enable volume topology aware scheduling and make the PersistentVolumeClaim (PVC) binding aware of scheduling decisions. It also enables the usage of [`local`](/docs/concepts/storage/volumes/#local) volume type when used together with the `PersistentLocalVolumes` feature gate. - `VolumeSnapshotDataSource`: Enable volume snapshot data source support. -- `VolumeSubpathEnvExpansion`: Enable `subPathExpr` field for expanding environment variables into a `subPath`. +- `VolumeSubpathEnvExpansion`: Enable `subPathExpr` field for expanding environment + variables into a `subPath`. +- `WarningHeaders`: Allow sending warning headers in API responses. - `WatchBookmark`: Enable support for watch bookmark events. -- `WindowsGMSA`: Enables passing of GMSA credential specs from pods to container runtimes. -- `WindowsRunAsUserName` : Enable support for running applications in Windows containers with as a non-default user. - See [Configuring RunAsUserName](/docs/tasks/configure-pod-container/configure-runasusername) for more details. - `WinDSR`: Allows kube-proxy to create DSR loadbalancers for Windows. - `WinOverlay`: Allows kube-proxy to run in overlay mode for Windows. +- `WindowsGMSA`: Enables passing of GMSA credential specs from pods to container runtimes. +- `WindowsRunAsUserName` : Enable support for running applications in Windows containers + with as a non-default user. See + [Configuring RunAsUserName](/docs/tasks/configure-pod-container/configure-runasusername) + for more details. +- `WindowsEndpointSliceProxying`: When enabled, kube-proxy running on Windows + will use EndpointSlices as the primary data source instead of Endpoints, + enabling scalability and performance improvements. See + [Enabling Endpoint Slices](/docs/tasks/administer-cluster/enabling-endpointslices/). ## {{% heading "whatsnext" %}}