Merge branch 'master' into release-1.12

This commit is contained in:
Tim Fogarty
2018-11-01 14:10:31 -07:00
146 changed files with 4631 additions and 773 deletions
@@ -32,7 +32,9 @@ the corresponding `PersistentVolume` is not be deleted. Instead, it is moved to
1. List the PersistentVolumes in your cluster:
kubectl get pv
```shell
kubectl get pv
```
The output is similar to this:
@@ -46,13 +48,17 @@ the corresponding `PersistentVolume` is not be deleted. Instead, it is moved to
1. Choose one of your PersistentVolumes and change its reclaim policy:
kubectl patch pv <your-pv-name> -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}'
```shell
kubectl patch pv <your-pv-name> -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}'
```
where `<your-pv-name>` is the name of your chosen PersistentVolume.
1. Verify that your chosen PersistentVolume has the right policy:
kubectl get pv
```shell
kubectl get pv
```
The output is similar to this:
@@ -15,7 +15,6 @@ running cluster.
{{% /capture %}}
{{< toc >}}
{{% capture body %}}
@@ -21,7 +21,6 @@ in the Kubernetes source directory for a canonical example.
{{% /capture %}}
{{< toc >}}
{{% capture prerequisites %}}
@@ -12,7 +12,6 @@ content_template: templates/task
{{% /capture %}}
{{< toc >}}
{{% capture prerequisites %}}
@@ -20,7 +20,6 @@ directives.
{{% /capture %}}
{{< toc >}}
{{% capture prerequisites %}}
@@ -178,7 +177,7 @@ spec:
```
This pod runs in the `Guaranteed` QoS class because `requests` are equal to `limits`.
And the container's resource limit for the CPU resource is an integer greater than
And the container's resource limit for the CPU resource is an integer greater than
or equal to one. The `nginx` container is granted 2 exclusive CPUs.
@@ -213,8 +212,8 @@ spec:
```
This pod runs in the `Guaranteed` QoS class because only `limits` are specified
and `requests` are set equal to `limits` when not explicitly specified. And the
container's resource limit for the CPU resource is an integer greater than or
and `requests` are set equal to `limits` when not explicitly specified. And the
container's resource limit for the CPU resource is an integer greater than or
equal to one. The `nginx` container is granted 2 exclusive CPUs.
{{% /capture %}}
@@ -22,7 +22,6 @@ To dive a little deeper into implementation details, all cloud controller manage
{{% /capture %}}
{{< toc >}}
{{% capture body %}}
@@ -18,54 +18,15 @@ vacated by the evicted critical add-on pod or the amount of resources available
{{% /capture %}}
{{< toc >}}
{{% capture body %}}
## Rescheduler: guaranteed scheduling of critical add-ons
**Rescheduler is deprecated as of Kubernetes 1.10 and will be removed in version 1.12 in
accordance with the [deprecation policy](/docs/reference/deprecation-policy) for beta features.**
**To avoid eviction of critical pods, you must
[enable priorities in scheduler](/docs/concepts/configuration/pod-priority-preemption/)
before upgrading to Kubernetes 1.10 or higher.**
Rescheduler ensures that critical pods created by DaemonSet controller are always scheduled
(assuming the cluster has enough resources to run the critical add-on pods in the absence of regular pods).
If the scheduler determines that no node has enough free resources to run the critical add-on pod
given the pods that are already running in the cluster
(indicated by critical add-on pod's pod condition PodScheduled set to false, the reason set to Unschedulable)
the rescheduler tries to free up space for the DaemonSet critical pod by evicting some pods; then the scheduler will schedule the add-on pod.
To avoid situation when another pod is scheduled into the space prepared for the critical add-on,
the chosen node gets a temporary taint "CriticalAddonsOnly" before the eviction(s)
(see [more details](https://git.k8s.io/community/contributors/design-proposals/scheduling/taint-toleration-dedicated.md)).
Each critical add-on has to tolerate it,
while the other pods shouldn't tolerate the taint. The taint is removed once the add-on is successfully scheduled.
*Warning:* currently there is no guarantee which node is chosen and which pods are being killed
in order to schedule critical pods, so if rescheduler is enabled your pods might be occasionally
killed for this purpose. Please ensure that rescheduler is not enabled along with priorities & preemptions in default-scheduler as rescheduler is oblivious to priorities and it may evict high priority pods, instead of low priority ones.
## Config
Rescheduler doesn't have any user facing configuration (component config) or API.
### Marking pod as critical when using Rescheduler.
### Marking pod as critical
To be considered critical, the pod has to run in the `kube-system` namespace (configurable via flag) and
* have the `scheduler.alpha.kubernetes.io/critical-pod` annotation set to empty string, and
* have the PodSpec's `tolerations` field set to `[{"key":"CriticalAddonsOnly", "operator":"Exists"}]`.
* Have the priorityClassName set as "system-cluster-critical" or "system-node-critical", the latter being the highest for entire cluster. Alternatively, you could add an annotation `scheduler.alpha.kubernetes.io/critical-pod` as key and empty string as value to your pod, but this annotation is deprecated as of version 1.13 and will be removed in 1.14.
The first one marks a pod a critical. The second one is required by Rescheduler algorithm.
A pod could also be considered critical, if its priority is greater than or equal to system-critical-priority.
### Marking pod as critical when priorites are enabled.
To be considered critical, the pod has to run in the `kube-system` namespace (configurable via flag) and
* Have the priorityClass set as "system-cluster-critical" or "system-node-critical", the latter being the highest for entire cluster and `scheduler.alpha.kubernetes.io/critical-pod` annotation set to empty string(This will be deprecated too).
{{% /capture %}}
@@ -14,7 +14,6 @@ This document describes how to use kube-up/down scripts to manage highly availab
{{% /capture %}}
{{< toc >}}
{{% capture prerequisites %}}
@@ -21,7 +21,7 @@ Before proceeding:
- You need to have a `kubeadm` HA cluster running version 1.11 or higher.
- Make sure you read the [release notes](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.12.md) carefully.
- Make sure to back up any important components, such as app-level state stored in a database. `kubeadm upgrade` does not touch your workloads, only components internal to Kubernetes, but backups are always a best practice.
- Check the prerequisites for [Upgrading/downgrading kubeadm clusters between v1.11 to v1.12](/docs/tasks/administer-cluster/kubeadm-upgrade-1-12/).
- Check the prerequisites for [Upgrading/downgrading kubeadm clusters between v1.11 to v1.12](/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade-1-12/).
{{< note >}}
**Note**: All commands on any control plane or etcd node should be
@@ -68,7 +68,7 @@ kubectl get pod default-cpu-demo --output=yaml --namespace=default-cpu-example
The output shows that the Pod's Container has a CPU request of 500 millicpus and
a CPU limit of 1 cpu. These are the default values specified by the LimitRange.
```shel
```shell
containers:
- image: nginx
imagePullPolicy: Always
@@ -70,7 +70,7 @@ kubectl get pod default-mem-demo --output=yaml --namespace=default-mem-example
The output shows that the Pod's Container has a memory request of 256 MiB and
a memory limit of 512 MiB. These are the default values specified by the LimitRange.
```shel
```shell
containers:
- image: nginx
imagePullPolicy: Always
@@ -20,7 +20,6 @@ This example demonstrates how to use Kubernetes namespaces to subdivide your clu
{{% /capture %}}
{{< toc >}}
{{% capture prerequisites %}}
@@ -18,7 +18,6 @@ nodes become unstable.
{{% /capture %}}
{{< toc >}}
{{% capture body %}}
@@ -205,7 +204,7 @@ If `nodefs` filesystem has met eviction thresholds, `kubelet` frees up disk spac
If the `kubelet` is unable to reclaim sufficient resource on the node, `kubelet` begins evicting Pods.
The `kubelet` ranks Pods for eviction first by whether or not their usage of the starved resource exceeds requests,
The `kubelet` ranks Pods for eviction first by whether or not their usage of the starved resource exceeds requests,
then by [Priority](https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/), and then by the consumption of the starved compute resource relative to the Pods' scheduling requests.
As a result, `kubelet` ranks and evicts Pods in the following order:
@@ -213,15 +212,15 @@ As a result, `kubelet` ranks and evicts Pods in the following order:
* `BestEffort` or `Burstable` Pods whose usage of a starved resource exceeds its request.
Such pods are ranked by Priority, and then usage above request.
* `Guaranteed` pods and `Burstable` pods whose usage is beneath requests are evicted last.
`Guaranteed` Pods are guaranteed only when requests and limits are specified for all
the containers and they are equal. Such pods are guaranteed to never be evicted because
`Guaranteed` Pods are guaranteed only when requests and limits are specified for all
the containers and they are equal. Such pods are guaranteed to never be evicted because
of another Pod's resource consumption. If a system daemon (such as `kubelet`, `docker`,
and `journald`) is consuming more resources than were reserved via `system-reserved` or
`kube-reserved` allocations, and the node only has `Guaranteed` or `Burstable` Pods using
less than requests remaining, then the node must choose to evict such a Pod in order to
`kube-reserved` allocations, and the node only has `Guaranteed` or `Burstable` Pods using
less than requests remaining, then the node must choose to evict such a Pod in order to
preserve node stability and to limit the impact of the unexpected consumption to other Pods.
In this case, it will choose to evict pods of Lowest Priority first.
If necessary, `kubelet` evicts Pods one at a time to reclaim disk when `DiskPressure`
is encountered. If the `kubelet` is responding to `inode` starvation, it reclaims
`inodes` by evicting Pods with the lowest quality of service first. If the `kubelet`
@@ -23,7 +23,6 @@ on each node.
{{% /capture %}}
{{< toc >}}
{{% capture prerequisites %}}
@@ -17,7 +17,6 @@ The `cloud-controller-manager` can be linked to any cloud provider that satisfie
{{% /capture %}}
{{< toc >}}
{{% capture body %}}
@@ -16,7 +16,6 @@ This means that the pods are visible on the API server but cannot be controlled
{{% /capture %}}
{{< toc >}}
{{% capture body %}}
@@ -103,7 +102,7 @@ Notice we cannot delete the pod with the API server (e.g. via [`kubectl`](/docs/
{{<note>}}
**Note**: Make sure the kubelet has permission to create the mirror pod in the API server.
If not, the creation request is rejected by the API server. See
If not, the creation request is rejected by the API server. See
[PodSecurityPolicy](/docs/concepts/policy/pod-security-policy/).
{{</note>}}
@@ -216,7 +216,7 @@ spec:
- Verify that the scheduling of the second pod fails with the below warning:
```shell
Warning FailedScheduling 18s (x4 over 21s) default-scheduler persistentvolumeclaim "slzc" is being deleted
Warning FailedScheduling 18s (x4 over 21s) default-scheduler persistentvolumeclaim "slzc" is being deleted
```
- Wait until the pod status of both pods is `Terminated` or `Completed` (either delete the pods or wait until they finish). Afterwards, check that the PVC is removed.