From 9bcc29be911c03044f4683b396d6493339a9c307 Mon Sep 17 00:00:00 2001 From: RaviSantosh Gudimetla Date: Mon, 29 Jan 2018 09:30:49 -0500 Subject: [PATCH 1/6] Update documentation to include priorities. Updated documentation to include deprecation for rescheduler and include priorities and preemption. --- ...aranteed-scheduling-critical-addon-pods.md | 29 ++++++++++++++++--- 1 file changed, 25 insertions(+), 4 deletions(-) diff --git a/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods.md b/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods.md index 85c387f0ab..f7fd8b0d6c 100644 --- a/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods.md +++ b/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods.md @@ -19,6 +19,12 @@ and becomes pending (for example when the cluster is highly utilized and either vacated by the evicted critical add-on pod or the amount of resources available on the node changed for some other reason). ## Rescheduler: guaranteed scheduling of critical add-ons +**Rescheduler is deprecated as of Kubernetes 1.10 and will be removed in version 1.11 in +accordance with the [deprecation policy](/docs/reference/deprecation-policy) for beta features.** + +**To avoid eviction of critical pods, you must +[enable priorities in scheduler](docs/concepts/configuration/pod-priority-preemption/) +before upgrading to Kubernetes 1.8 or higher.** Rescheduler ensures that critical add-ons are always scheduled (assuming the cluster has enough resources to run the critical add-on pods in the absence of regular pods). @@ -35,18 +41,33 @@ while the other pods shouldn't tolerate the taint. The taint is removed once the *Warning:* currently there is no guarantee which node is chosen and which pods are being killed in order to schedule critical pods, so if rescheduler is enabled your pods might be occasionally -killed for this purpose. +killed for this purpose. Please ensure that rescheduler should not enabled along with priorities & preemptions in default scheduler as it may cause unwanted side-effects. ## Config -Rescheduler should be enabled by default. It doesn't have any user facing configuration (component config) or API. +Rescheduler should be enabled by default(unless you enabled priorities & preemption). It doesn't have any user facing configuration (component config) or API. Rescheduler may also be disabled. -### Marking add-on as critical +### Marking pod as critical when using Rescheduler(This will be deprecated). To be critical an add-on has to run in `kube-system` namespace (configurable via flag) and * have the `scheduler.alpha.kubernetes.io/critical-pod` annotation set to empty string, and -* have the PodSpec's `tolerations` field set to `[{"key":"CriticalAddonsOnly", "operator":"Exists"}]` +* have the PodSpec's `tolerations` field set to `[{"key":"CriticalAddonsOnly", "operator":"Exists"}]`. The first one marks a pod a critical. The second one is required by Rescheduler algorithm. + +### Marking pod as critical when priorites are enabled. + +To be critical an add-on has to run in `kube-system` namespace (configurable via flag) and + +* have the priorityClass set as "system-cluster-critical" or "system-node-critical", the latter being the highest for entire cluster. + +or + +* have the `scheduler.alpha.kubernetes.io/critical-pod` annotation set to empty string(This will be deprecated too). + + + + + From c707b24b1f9a3ea11a9f0e0dcf3a045097377220 Mon Sep 17 00:00:00 2001 From: RaviSantosh Gudimetla Date: Mon, 29 Jan 2018 10:43:15 -0500 Subject: [PATCH 2/6] Updated as per reviewers comments --- .../guaranteed-scheduling-critical-addon-pods.md | 14 ++++---------- 1 file changed, 4 insertions(+), 10 deletions(-) diff --git a/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods.md b/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods.md index f7fd8b0d6c..e8daab72b8 100644 --- a/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods.md +++ b/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods.md @@ -41,16 +41,15 @@ while the other pods shouldn't tolerate the taint. The taint is removed once the *Warning:* currently there is no guarantee which node is chosen and which pods are being killed in order to schedule critical pods, so if rescheduler is enabled your pods might be occasionally -killed for this purpose. Please ensure that rescheduler should not enabled along with priorities & preemptions in default scheduler as it may cause unwanted side-effects. +killed for this purpose. Please ensure that rescheduler is enabled along with priorities & preemptions in default-scheduler as it may have unwanted side-effects. ## Config -Rescheduler should be enabled by default(unless you enabled priorities & preemption). It doesn't have any user facing configuration (component config) or API. -Rescheduler may also be disabled. +Rescheduler doesn't have any user facing configuration (component config) or API. ### Marking pod as critical when using Rescheduler(This will be deprecated). -To be critical an add-on has to run in `kube-system` namespace (configurable via flag) and +To be considered critical, the pod has to run in the `kube-system` namespace (configurable via flag) and * have the `scheduler.alpha.kubernetes.io/critical-pod` annotation set to empty string, and * have the PodSpec's `tolerations` field set to `[{"key":"CriticalAddonsOnly", "operator":"Exists"}]`. @@ -59,15 +58,10 @@ The first one marks a pod a critical. The second one is required by Rescheduler ### Marking pod as critical when priorites are enabled. -To be critical an add-on has to run in `kube-system` namespace (configurable via flag) and +To be considered critical, the pod has to run in the `kube-system` namespace (configurable via flag) and * have the priorityClass set as "system-cluster-critical" or "system-node-critical", the latter being the highest for entire cluster. or * have the `scheduler.alpha.kubernetes.io/critical-pod` annotation set to empty string(This will be deprecated too). - - - - - From 57c9a5bba745a1ebe67f76db63171d24bf32482f Mon Sep 17 00:00:00 2001 From: RaviSantosh Gudimetla Date: Mon, 29 Jan 2018 11:32:26 -0500 Subject: [PATCH 3/6] "Updated the priorityClass changes" --- .../guaranteed-scheduling-critical-addon-pods.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods.md b/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods.md index e8daab72b8..c992e26935 100644 --- a/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods.md +++ b/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods.md @@ -64,4 +64,4 @@ To be considered critical, the pod has to run in the `kube-system` namespace (co or -* have the `scheduler.alpha.kubernetes.io/critical-pod` annotation set to empty string(This will be deprecated too). +* have the `scheduler.alpha.kubernetes.io/critical-pod` annotation set to empty string(This will be deprecated too). This ensures that the pod gets "system-cluster-critical" priority. From e242b92de409a305b67a94f847a7def5b25aeb8f Mon Sep 17 00:00:00 2001 From: RaviSantosh Gudimetla Date: Wed, 31 Jan 2018 12:27:47 -0500 Subject: [PATCH 4/6] Update guaranteed-scheduling-critical-addon-pods.md --- .../guaranteed-scheduling-critical-addon-pods.md | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods.md b/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods.md index c992e26935..85db60f231 100644 --- a/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods.md +++ b/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods.md @@ -47,7 +47,8 @@ killed for this purpose. Please ensure that rescheduler is enabled along with pr Rescheduler doesn't have any user facing configuration (component config) or API. -### Marking pod as critical when using Rescheduler(This will be deprecated). +### Marking pod as critical when using Rescheduler. +** Marking pod as critical when using Rescheduler. To be considered critical, the pod has to run in the `kube-system` namespace (configurable via flag) and @@ -60,8 +61,4 @@ The first one marks a pod a critical. The second one is required by Rescheduler To be considered critical, the pod has to run in the `kube-system` namespace (configurable via flag) and -* have the priorityClass set as "system-cluster-critical" or "system-node-critical", the latter being the highest for entire cluster. - -or - -* have the `scheduler.alpha.kubernetes.io/critical-pod` annotation set to empty string(This will be deprecated too). This ensures that the pod gets "system-cluster-critical" priority. +* Have the priorityClass set as "system-cluster-critical" or "system-node-critical", the latter being the highest for entire cluster and `scheduler.alpha.kubernetes.io/critical-pod` annotation set to empty string(This will be deprecated too). From effcc3db9448f3d58d27558d45578b3f567162ae Mon Sep 17 00:00:00 2001 From: RaviSantosh Gudimetla Date: Thu, 1 Feb 2018 10:57:05 -0500 Subject: [PATCH 5/6] Update guaranteed-scheduling-critical-addon-pods.md --- .../guaranteed-scheduling-critical-addon-pods.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods.md b/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods.md index 85db60f231..e1652edafc 100644 --- a/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods.md +++ b/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods.md @@ -41,7 +41,7 @@ while the other pods shouldn't tolerate the taint. The taint is removed once the *Warning:* currently there is no guarantee which node is chosen and which pods are being killed in order to schedule critical pods, so if rescheduler is enabled your pods might be occasionally -killed for this purpose. Please ensure that rescheduler is enabled along with priorities & preemptions in default-scheduler as it may have unwanted side-effects. +killed for this purpose. Please ensure that rescheduler is not enabled along with priorities & preemptions in default-scheduler as rescheduler is oblivious to priorities and it may evict high priority pods, instead of low priority ones. ## Config From 237bf27cc3b16a7b77d5615e155f7e2f76af9036 Mon Sep 17 00:00:00 2001 From: RaviSantosh Gudimetla Date: Fri, 2 Feb 2018 09:48:09 -0500 Subject: [PATCH 6/6] Update guaranteed-scheduling-critical-addon-pods.md --- .../guaranteed-scheduling-critical-addon-pods.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods.md b/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods.md index e1652edafc..bf54698b95 100644 --- a/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods.md +++ b/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods.md @@ -24,7 +24,7 @@ accordance with the [deprecation policy](/docs/reference/deprecation-policy) for **To avoid eviction of critical pods, you must [enable priorities in scheduler](docs/concepts/configuration/pod-priority-preemption/) -before upgrading to Kubernetes 1.8 or higher.** +before upgrading to Kubernetes 1.10 or higher.** Rescheduler ensures that critical add-ons are always scheduled (assuming the cluster has enough resources to run the critical add-on pods in the absence of regular pods).