whentimeslows/website
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

PodDisruptionBudget documentation Improvements (#4140)

Browse Source 
* Changes from #3885

Title: Update PDB documentation to explain new field
Author: foxish

* Added Placeholder Disruptions Concept Guide

New file: docs/concepts/workloads/pods/disruptions.md
Intented contents: concept for Pod Disruption Budget,
 cross reference to Eviction and Preemption docs.
Linked from: concepts > workloads > pods

* Added placeholder Configuring PDB Task

New file: docs/tasks/run-application/configure-pdb.md
Intented contents: task for writing a Pod Disruption Budget.
Linked from: tasks > configuring-applications > configure pdb.

* Add refs to the "drain a node" task.

* Refactor PDB docs.

Move the "Requesting an eviction" section from:
docs/tasks/administer-cluster/configure-pod-disruption-budget.md
-- which is going away -- to:
docs/tasks/administer-cluster/safely-drain-node.md

The move is verbatim, except for an introductory sentence.

Also added assignees.

* Refactor of PDB docs

Moved the section:
Specifying a PodDisruptionBudget
from:
docs/tasks/administer-cluster/configure-pod-disruption-budget.md
to:
docs/tasks/run-application/configure-pdb.md
because that former file is going away.
Move is verbatim.

* Explain how Eviction tools should handle failures

* Refactor PDB docs

Move text from:
docs/tasks/administer-cluster/configure-pod-disruption-budget.md
to:
docs/concepts/workloads/pods/disruptions.md

Delete the now empty:
docs/tasks/administer-cluster/configure-pod-disruption-budget.md

Added a redirects_from section to the new doc, containing the path
of the now-deleted doc, plus all the redirects from the deleted
doc.

* Expand PDB Concept guide

Building on a little content from the old task,
greatly expanded the Disruptions concept
guide, including an abstract example.

* Update creating a pdb Task.

* Address review comments.

* Fixed for all cody-clark's review comments

* Address review comments from mml

* Address review comments from maisem

* Fix missing backtick
This commit is contained in:
Eric Tune
2017-06-26 13:54:25 -07:00
committed by Andrew Chen
parent 098b4a84ef
commit 4ff5587583
6 changed files with 578 additions and 172 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/tasks/administer-cluster/configure-pod-disruption-budget.md
-169
View File
@@ -1,169 +0,0 @@
---
assignees:
- davidopp
- erictune
- foxish
- kow3ns
title: Configure a Pod Disruption Budget
redirect_from:
- "/docs/admin/disruptions/"
- "/docs/admin/disruptions.html"
- "/docs/tasks/configure-pod-container/configure-pod-disruption-budget/"
- "/docs/tasks/configure-pod-container/configure-pod-disruption-budget/"
---
This guide is for anyone wishing to specify safety constraints on pods or anyone
wishing to write software (typically automation software) that respects those
constraints.
* TOC
{:toc}
## Rationale
Various cluster management operations may voluntarily evict pods. "Voluntary"
means an eviction can be safely delayed for a reasonable period of time. The
principal examples today are draining a node for maintenance or upgrade
(`kubectl drain`), and cluster autoscaling down. In the future the
[rescheduler](https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/rescheduling.md)
may also perform voluntary evictions. By contrast, something like evicting pods
because a node has become unreachable or reports `NotReady`, is not "voluntary."
For voluntary evictions, it can be useful for applications to be able to limit
the number of pods that are down simultaneously. For example, a quorum-based application would
like to ensure that the number of replicas running is never brought below the
number needed for a quorum, even temporarily. Or a web front end might want to
ensure that the number of replicas serving load never falls below a certain
percentage of the total, even briefly. `PodDisruptionBudget` is an API object
that specifies the minimum number or percentage of replicas of a collection that
must be up at a time. Components that wish to evict a pod subject to disruption
budget use the `/eviction` subresource; unlike a regular pod deletion, this
operation may be rejected by the API server if the eviction would cause a
disruption budget to be violated.
## Specifying a PodDisruptionBudget
A `PodDisruptionBudget` has three fields:
* A label selector `selector` to specify the set of
pods to which it applies. This is a required field.
* `minAvailable` which is a description of the number of pods from that
set that must still be available after the eviction, i.e. even in the absence
of the evicted pod. `minAvailable` can be either an absolute number or a percentage.
* `maxUnavailable` (available in Kubernetes 1.7 and higher) which is a description
of the number of pods from that set that can be unavailable after the eviction.
It can also be either an absolute number or a percentage.
You can specify only one of `maxUnavailable` and `minAvailable` in a single `PodDisruptionBudget`.
`maxUnavailable` can only be used to control the eviction of pods
that have an associated controller managing them. In the examples below, "desired replicas"
is the `scale` of the controller managing the pods being selected by the
`PodDisruptionBudget`.
Example 1: With a `minAvailable` of 5, evictions will be allowed as long as they leave behind
5 or more healthy pods among those selected by the PodDisruptionBudget's `selector`.
Example 2: With a `minAvailable` of 30%, evictions will be allowed as long as at least 30%
of the number of desired replicas are healthy.
Example 3: With a `maxUnavailable` of 5, evictions will be allowed as long as there are at most 5
unhealthy replicas among the total number of desired replicas.
Example 4: With a `maxUnavailable` of 30%, evictions will be allowed as long as no more than 30%
of the desired replicas are unhealthy.
In typical usage, a single budget would be used for a collection of pods managed by
a controller—for example, the pods in a single ReplicaSet or StatefulSet.
Note that a disruption budget does not truly guarantee that the specified
number/percentage of pods will always be up. For example, a node that hosts a
pod from the collection may fail when the collection is at the minimum size
specified in the budget, thus bringing the number of available pods from the
collection below the specified size. The budget can only protect against
voluntary evictions, not all causes of unavailability.
A `maxUnavailable` of 0% (or 0) or a `minAvailable` of 100% (or equal to the
number of replicas) may block node drains entirely. This is permitted as per the
semantics of `PodDisruptionBudget`.
You can find examples of pod disruption budgets defined below. They match pods with the label
`app: zookeeper`.
Example PDB Using maxUnavailable:
```yaml
apiVersion: policy/v1beta1
kind: PodDisruptionBudget
metadata:
name: zk-pdb
spec:
minAvailable: 2
selector:
matchLabels:
app: zookeeper
```
Example PDB Using maxUnavailable (Kubernetes 1.7 or higher):
```yaml
apiVersion: policy/v1beta1
kind: PodDisruptionBudget
metadata:
name: zk-pdb
spec:
maxUnavailable: 1
selector:
matchLabels:
app: zookeeper
```
For example, if the above `zk-pdb` object selects the pods of a StatefulSet of size 3, both
specifications have the exact same meaning. The use of `maxUnavailable` is recommended as it
automatically responds to changes in the number of replicas of the corresponding controller.
## Requesting an eviction
See the task explaining [draining nodes](/docs/tasks/administer-cluster/safely-drain-node/) for a higher level construct used to trigger evictions of pods on chosen nodes.
If you are writing infrastructure software that wants to produce these voluntary
evictions, you will need to use the eviction API. The eviction subresource of a
pod can be thought of as a kind of policy-controlled DELETE operation on the pod
itself. To attempt an eviction (perhaps more REST-precisely, to attempt to
*create* an eviction), you POST an attempted operation. Here's an example:
```json
{
"apiVersion": "policy/v1beta1",
"kind": "Eviction",
"metadata": {
"name": "quux",
"namespace": "default"
}
}
```
You can attempt an eviction using `curl`:
```bash
$ curl -v -H 'Content-type: application/json' http://127.0.0.1:8080/api/v1/namespaces/default/pods/quux/eviction -d @eviction.json
```
The API can respond in one of three ways.
1. If the eviction is granted, then the pod is deleted just as if you had sent
a `DELETE` request to the pod's URL and you get back `200 OK`.
2. If the current state of affairs wouldn't allow an eviction by the rules set
forth in the budget, you get back `429 Too Many Requests`. This is
typically used for generic rate limiting of *any* requests, but here we mean
that this request isn't allowed *right now* but it may be allowed later.
Currently, callers do not get any `Retry-After` advice, but they may in
future versions.
3. If there is some kind of misconfiguration, like multiple budgets pointing at
the same pod, you will get `500 Internal Server Error`.
For a given eviction request, there are two cases.
1. There is no budget that matches this pod. In this case, the server always
returns `200 OK`.
2. There is at least one budget. In this case, any of the three above responses may
apply.
docs/tasks/administer-cluster/safely-drain-node.md
+74 -3
View File
@@ -1,6 +1,9 @@
---
assignees:
- davidopp
- mml
- foxish
- kow3ns
title: Safely Drain a Node while Respecting Application SLOs
---
@@ -14,8 +17,12 @@ disruption SLOs you have specified using PodDisruptionBudget.
This task assumes that you have met the following prerequisites:
* You are using Kubernetes release >= 1.5.
* You have created [PodDisruptionBudget(s)](/docs/tasks/configure-pod-container/configure-pod-disruption-budget/) to express the
application-level disruption SLOs you want the system to enforce.
* Either:
1. You do not require your applications to be highly available during the
node drain, or
1. You have read about the [PodDisruptionBudget concept](/docs/concepts/workloads/pods/disruptions.md)
and [Configured PodDisruptionBudgets](/docs/tasks/run-application/configure-pdb.md) for
applications that need them.
{% endcapture %}
@@ -81,11 +88,75 @@ that only one pod is unavailable at any given time. Any drains that
would cause the number of ready replicas to fall below the specified
budget are blocked.
## The Eviction API
If you prefer not to use [kubectl drain](/docs/user-guide/kubectl/v1.6/#drain) (such as
to avoid calling to an external command, or to get finer control over over the pod
eviction process), you can also programmatically cause evictions using the eviction API.
You should first be familiar with using [Kubernetes language clients](/docs/tasks/administer-cluster/access-cluster-api.md#programmatic-access-to-the-api).
The eviction subresource of a
pod can be thought of as a kind of policy-controlled DELETE operation on the pod
itself. To attempt an eviction (perhaps more REST-precisely, to attempt to
*create* an eviction), you POST an attempted operation. Here's an example:
```json
{
"apiVersion": "policy/v1beta1",
"kind": "Eviction",
"metadata": {
"name": "quux",
"namespace": "default"
}
}
```
You can attempt an eviction using `curl`:
```bash
$ curl -v -H 'Content-type: application/json' http://127.0.0.1:8080/api/v1/namespaces/default/pods/quux/eviction -d @eviction.json
```
The API can respond in one of three ways.
- If the eviction is granted, then the pod is deleted just as if you had sent
a `DELETE` request to the pod's URL and you get back `200 OK`.
- If the current state of affairs wouldn't allow an eviction by the rules set
forth in the budget, you get back `429 Too Many Requests`. This is
typically used for generic rate limiting of *any* requests, but here we mean
that this request isn't allowed *right now* but it may be allowed later.
Currently, callers do not get any `Retry-After` advice, but they may in
future versions.
- If there is some kind of misconfiguration, like multiple budgets pointing at
the same pod, you will get `500 Internal Server Error`.
For a given eviction request, there are two cases.
- There is no budget that matches this pod. In this case, the server always
returns `200 OK`.
- There is at least one budget. In this case, any of the three above responses may
apply.
In some cases, an application may reach a broken state where it will never return anything
other than 429 or 500. This can happen, for example, if the replacement pod created by the
application's controller does not become ready, or if the last pod evicted has a very long
termination grace period.
In this case, there are two potential solutions:
- Abort or pause the automated operation. Investigate the reason for the stuck application, and restart the automation.
- After a suitably long wait, `DELETE` the pod instead of using the eviction API.
Kubernetes does not specify what the behavior should be in this case; it is up to the
application owners and cluster owners to establish an agreement on behavior in these cases.
{% endcapture %}
{% capture whatsnext %}
*TODO: link to other docs about Stateful Set?*
* Follow steps to protect your application by [configuring a Pod Disruption Budget](/docs/tasks/run-application//configure-pdb.md).
{% endcapture %}
{% include templates/task.md %}