From 220a7b201b42c919824c679cbf08ef7997ce78cd Mon Sep 17 00:00:00 2001 From: Sergey Kanzhelev Date: Tue, 10 Nov 2020 00:10:11 +0000 Subject: [PATCH] ExecProbeTimeout feature gate introduction --- .../feature-gates.md | 2 ++ ...igure-liveness-readiness-startup-probes.md | 23 +++++++++++++++---- 2 files changed, 21 insertions(+), 4 deletions(-) diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates.md b/content/en/docs/reference/command-line-tools-reference/feature-gates.md index 303c90aae5..b16b4d722b 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates.md @@ -227,6 +227,7 @@ different Kubernetes components. | `EvenPodsSpread` | `false` | Alpha | 1.16 | 1.17 | | `EvenPodsSpread` | `true` | Beta | 1.18 | 1.18 | | `EvenPodsSpread` | `true` | GA | 1.19 | - | +| `ExecProbeTimeout` | `true` | GA | 1.20 | - | | `GCERegionalPersistentDisk` | `true` | Beta | 1.10 | 1.12 | | `GCERegionalPersistentDisk` | `true` | GA | 1.13 | - | | `HugePages` | `false` | Alpha | 1.8 | 1.9 | @@ -450,6 +451,7 @@ Each feature gate is designed for enabling/disabling a specific feature: - `EphemeralContainers`: Enable the ability to add {{< glossary_tooltip text="ephemeral containers" term_id="ephemeral-container" >}} to running pods. - `EvenPodsSpread`: Enable pods to be scheduled evenly across topology domains. See [Pod Topology Spread Constraints](/docs/concepts/workloads/pods/pod-topology-spread-constraints/). +- `ExecProbeTimeout`: Ensure kubelet respects exec probe timeouts. This feature gate exists in case any of your existing workloads depend on a now-corrected fault where Kubernetes ignored exec probe timeouts. See [readiness probes](/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#configure-probes). - `ExpandInUsePersistentVolumes`: Enable expanding in-use PVCs. See [Resizing an in-use PersistentVolumeClaim](/docs/concepts/storage/persistent-volumes/#resizing-an-in-use-persistentvolumeclaim). - `ExpandPersistentVolumes`: Enable the expanding of persistent volumes. See [Expanding Persistent Volumes Claims](/docs/concepts/storage/persistent-volumes/#expanding-persistent-volumes-claims). - `ExperimentalCriticalPodAnnotation`: Enable annotating specific pods as *critical* so that their [scheduling is guaranteed](/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods/). diff --git a/content/en/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes.md b/content/en/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes.md index 150fe720e1..e55f8cb789 100644 --- a/content/en/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes.md +++ b/content/en/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes.md @@ -336,6 +336,25 @@ liveness. Minimum value is 1. try `failureThreshold` times before giving up. Giving up in case of liveness probe means restarting the container. In case of readiness probe the Pod will be marked Unready. Defaults to 3. Minimum value is 1. +{{< note >}} +Before Kubernetes 1.20, the field `timeoutSeconds` was not respected for exec probes: +probes continued running indefinitely, even past their configured deadline, +until a result was returned. + +This defect was corrected in Kubernetes v1.20. You may have been relying on the previous behavior, +even without realizing it, as the default timeout is 1 second. +As a cluster administrator, you can disable the feature gate `ExecProbeTimeout` (set it to `false`) +on kubelet to restore the behavior from older versions, then remove that override +once all the exec probes in the cluster have a `timeoutSeconds` value set. + +With the fix of the defect, for exec probes, on Kubernetes `1.20+` with the `dockershim` container runtime, +the process inside the container may keep running even after probe returned failure because of the timeout. +{{< /note >}} +{{< caution >}} +Incorrect implementation of readiness probes may result in an ever growing number +of processes in the container, and resource starvation if this is left unchecked. +{{< /caution >}} + ### HTTP probes [HTTP probes](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#httpgetaction-v1-core) @@ -406,7 +425,3 @@ You can also read the API references for: * [Pod](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#pod-v1-core) * [Container](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#container-v1-core) * [Probe](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#probe-v1-core) - - - -