Release 1.7 (#4094)
* Minor fixes in the Deployment doc Signed-off-by: Michail Kargakis <mkargaki@redhat.com> * add NodeRestriction to admission-controllers (#3842) * Admins Can Configure Zones in Storage Class The PR #38505 (https://github.com/kubernetes/kubernetes/pull/38505) added zones optional parameter to Storage Class for AWS and GCE provisioners. That's why documentation needs to be updated accordingly. * document custom resource definitions * add host paths to psp (#3971) * add host paths to psp * add italics * Update ConfigMap doc to explain TTL-based cache updates (#3989) * Update ConfigMap doc to explain TTL-based cache updates * swap word order Change "When a ConfigMap being already consumed..." to "When a ConfigMap already being consumed..." * Update NetworkPolicy docs for v1 * StorageOS Volume plugin * Update GPU docs * docs: HPA autoscaling/v2alpha1 status conditions This commit documents the new status conditions feature for HPA autoscaling/v2alpha1. It demonstrates how to get the status conditions using `kubectl describe`, and how to interpret them. * Update description about NodeRestriction kubelet node can alse create mirror pods for their own static pods. * adding storage as a supported resource to node allocatable Signed-off-by: Vishnu kannan <vishnuk@google.com> * Add documentation for podpreset opt-out annotation This adds the annotation for having the podpreset admission controller to skip (opt-out) manipulating the pod spec. Also, the annotation format for what presets have acted on a pod has been modified to add a prefix of "podpreset-". The new naming makes it such that there is no chance of collision with the newly introduced opt-out annotation (or future ones yet to be added). Opt-out annotation PR: kubernetes/kubernetes#44965 * Update PDB documentation to explain new field (#3885) * update-docs-pdb * Addressed erictune@'s comments * Fix title and add a TOC to the logging concept page * Patch #4118 for typos * Describe setting coredns server in nameserver resolv chain * Address comments in PR #3997. Comment is in https://github.com/kubernetes/kubernetes.github.io/pull/3997/files/f6eb59c67e28efc298c87b1ef49a96bc6adacd1e#diff-7a14981f3dd8eb203f897ce6c11d9828 * Update task for DaemonSet history and rollback (#4098) * Update task for DaemonSet history and rollback Also remove mentions of templateGeneration field because it's deprecated * Address comments * removed lt and gt as operators (#4152) * removed lt and gt as operators * replace lt and gt for node-affinfity * updated based on bsalamat review * Initial draft of upgrade guide for kubeadm clusters. In-place upgrades are supported between 1.6 and 1.7 releases. Rollback instructions to come in a separate commit. Fixes https://github.com/kubernetes/kubeadm/issues/278 * Add local volume documentation (#4050) * Add local volume documentation * Add PV local volume example * Patch PR #3999 * Add documentation for Stackdriver event exporter * Add documentation about controller metrics * Federation: Add task for setting up placement policies (#4075) * Add task for setting up placement policies * Update version of management sidecar in policy engine deployment * Address @nikhiljindal's comments - Lower case filenames - Comments in policy - Typo fixes - Removed type LoadBalancer from OPA Service * Add example that sets cluster selector Per-@nikhiljindal's suggestion * Fix wording and templating per @chenopis * PodDisruptionBudget documentation Improvements (#4140) * Changes from #3885 Title: Update PDB documentation to explain new field Author: foxish * Added Placeholder Disruptions Concept Guide New file: docs/concepts/workloads/pods/disruptions.md Intented contents: concept for Pod Disruption Budget, cross reference to Eviction and Preemption docs. Linked from: concepts > workloads > pods * Added placeholder Configuring PDB Task New file: docs/tasks/run-application/configure-pdb.md Intented contents: task for writing a Pod Disruption Budget. Linked from: tasks > configuring-applications > configure pdb. * Add refs to the "drain a node" task. * Refactor PDB docs. Move the "Requesting an eviction" section from: docs/tasks/administer-cluster/configure-pod-disruption-budget.md -- which is going away -- to: docs/tasks/administer-cluster/safely-drain-node.md The move is verbatim, except for an introductory sentence. Also added assignees. * Refactor of PDB docs Moved the section: Specifying a PodDisruptionBudget from: docs/tasks/administer-cluster/configure-pod-disruption-budget.md to: docs/tasks/run-application/configure-pdb.md because that former file is going away. Move is verbatim. * Explain how Eviction tools should handle failures * Refactor PDB docs Move text from: docs/tasks/administer-cluster/configure-pod-disruption-budget.md to: docs/concepts/workloads/pods/disruptions.md Delete the now empty: docs/tasks/administer-cluster/configure-pod-disruption-budget.md Added a redirects_from section to the new doc, containing the path of the now-deleted doc, plus all the redirects from the deleted doc. * Expand PDB Concept guide Building on a little content from the old task, greatly expanded the Disruptions concept guide, including an abstract example. * Update creating a pdb Task. * Address review comments. * Fixed for all cody-clark's review comments * Address review comments from mml * Address review comments from maisem * Fix missing backtick * Api and Kubectl reference docs updates for 1.7 (#4193) * Fix includes groups * Generated kubectl docs for 1.7 * Generated references docs for 1.7 api * Document node authorization mode * API Aggregator (#4173) * API Aggregator * Additional bullet points * incorporated feedback for apiserver-aggregation.md * split setup-api-aggregator.md into two docs and address feedback * fix link * addressed docs feedback * incorporate feedback * integrate feedback * Add documentation for DNS stub domains (#4063) * Add documentation for DNS stub domains * add additional prereq * fix image path * review feedback * minor grammar and style nits * documentation for using hostAliases to manage hosts file (#4080) * documentation for using hostAliases to manage hosts file * add to table of contents * review comments * update the right command to see hosts file * reformat doc based on suggestion and change some wording * Fix typo for #4080 * Patch PR #4063 * Fix wording in placement policy task introduction * Add update to statefulset concepts and basic tutorial (#4174) * Add update to statefulset concpets and basic tutorial * Address tech comments. * Update ESIPP docs for new added API fields * Custom resource docs * update audit document with advanced audit features added in 1.7 * kubeadm v1.7 documentation updates (#4018) * v1.7 updates for kubeadm * Address review comments * Address Luke's comments * Encrypting secrets at rest and cluster security guide * Edits for Custom DNS Documentation (#4207) * reorganize custom dns doc * format fixes * Update version numbers to 1.7 * Patch PR #4140 (#4215) * Patch PR #4140 * fix link and typos * Update PR template * Update TLS bootstrapping with 1.7 features This includes documenting the new CSR approver built into the controller manager and the kubelet alpha features for certificate rotation. Since the CSR approver changed over the 1.7 release cycle we need to call out the migration steps for those using the alpha feature. This document as a whole could probably use some updates, but the main focus of this PR is just to get these features minimally documented before the release. * Federated ClusterSelector formatting updates from review * complete PR #4181 (#4223) * complete PR #4181 * fix security link * Extensible admission controller (#4092) * extensible-admission-controllers * Update extensible-admission-controllers.md * more on initializers * fixes * Expand external admission webhooks documentation * wrap at 80 chars * more * add reference * Use correct apigroup for network policy * Docs changes to PR #4092 (#4224) * Docs changes to PR #4092 * address feedback * add doc for --as-group in cli Add doc for this pr: https://github.com/kubernetes/kubernetes/pull/43696
This commit is contained in:
zhangchaolei
@@ -0,0 +1,35 @@
|
||||
---
|
||||
title: Extending the Kubernetes API with the aggregation layer
|
||||
assignees:
|
||||
- lavalamp
|
||||
- cheftako
|
||||
- chenopis
|
||||
---
|
||||
|
||||
{% capture overview %}
|
||||
|
||||
The aggregation layer allows Kubernetes to be extended with additional APIs, beyond what is offered by the core Kubernetes APIs.
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
{% capture body %}
|
||||
|
||||
## Overview
|
||||
|
||||
The aggregation layer enables installing additional Kubernetes-style APIs in your cluster. These can either be pre-built, existing 3rd party solutions, such as [service-catalog](https://github.com/kubernetes-incubator/service-catalog/blob/master/README.md), or user-created APIs like [apiserver-builder](https://github.com/kubernetes-incubator/apiserver-builder/blob/master/README.md), which can get you started.
|
||||
|
||||
In 1.7 the aggregation layer runs in-process with the kube-apiserver. Until an extension resource is registered, the aggregation layer will do nothing. To register their API, users must add an APIService object, which "claims" the URL path in the Kubernetes API. At that point, the aggregation layer will proxy anything sent to that API path (e.g. /apis/myextension.mycompany.io/v1/…) to the registered APIService.
|
||||
|
||||
Ordinarily, the APIService will be implemented by an *extension-apiserver* in a pod running in the cluster. This extension-apiserver will normally need to be paired with one or more controllers if active management of the added resources is needed. As a result, the apiserver-builder will actually provide a skeleton for both. As another example, when the service-catalog is installed, it provides both the extension-apiserver and controller for the services it provides.
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
{% capture whatsnext %}
|
||||
|
||||
* To get the aggregator working in your environment, [configure the aggregation layer](/docs/tasks/access-kubernetes-api/configure-aggregation-layer/).
|
||||
* Then, [setup an extension api-server](/docs/tasks/access-kubernetes-api/setup-extension-api-server/) to work with the aggregation layer.
|
||||
* Also, learn how to [extend the Kubernetes API using Custom Resource Definitions](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/).
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
{% include templates/concept.md %}
|
||||
@@ -0,0 +1,79 @@
|
||||
---
|
||||
title: Custom Resources
|
||||
assignees:
|
||||
- enisoc
|
||||
- deads2k
|
||||
---
|
||||
|
||||
{% capture overview %}
|
||||
This page explains the concept of *custom resources*, which are extensions of the Kubernetes API.
|
||||
{% endcapture %}
|
||||
|
||||
{% capture body %}
|
||||
## Custom resources
|
||||
|
||||
A *resource* is an endpoint in the [Kubernetes API](/docs/reference/api-overview/) that stores a
|
||||
collection of [API objects](/docs/concepts/overview/working-with-objects/kubernetes-objects/) of a
|
||||
certain kind.
|
||||
For example, the built-in *pods* resource contains a collection of Pod objects.
|
||||
|
||||
A *custom resource* is an extension of the Kubernetes API that is not necessarily available on every
|
||||
Kubernetes cluster.
|
||||
In other words, it represents a customization of a particular Kubernetes installation.
|
||||
|
||||
Custom resources can appear and disappear in a running cluster through dynamic registration,
|
||||
and cluster admins can update custom resources independently of the cluster itself.
|
||||
Once a custom resource is installed, users can create and access its objects with
|
||||
[kubectl](/docs/user-guide/kubectl-overview/), just as they do for built-in resources like *pods*.
|
||||
|
||||
## Custom controllers
|
||||
|
||||
On their own, custom resources simply let you store and retrieve structured data.
|
||||
It is only when combined with a *controller* that they become a true
|
||||
[declarative API](/docs/concepts/overview/working-with-objects/kubernetes-objects/#understanding-kubernetes-objects).
|
||||
The controller interprets the structured data as a record of the user's desired state,
|
||||
and continually takes action to achieve and maintain that state.
|
||||
|
||||
A *custom controller* is a controller that users can deploy and update on a running cluster,
|
||||
independently of the cluster's own lifecycle.
|
||||
Custom controllers can work with any kind of resource, but they are especially effective when
|
||||
combined with custom resources.
|
||||
The [Operator](https://coreos.com/blog/introducing-operators.html) pattern is one example of such a
|
||||
combination. It allows developers to encode domain knowledge for specific applications into an
|
||||
extension of the Kubernetes API.
|
||||
|
||||
## CustomResourceDefinitions
|
||||
|
||||
[CustomResourceDefinition](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/)
|
||||
(CRD) is a built-in API that offers a simple way to create custom resources.
|
||||
Deploying a CRD into the cluster causes the Kubernetes API server to begin serving the specified
|
||||
custom resource on your behalf.
|
||||
|
||||
This frees you from writing your own API server to handle the custom resource,
|
||||
but the generic nature of the implementation means you have less flexibility than with
|
||||
[API server aggregation](#api-server-aggregation).
|
||||
|
||||
CRD is the successor to the deprecated *ThirdPartyResource* (TPR) API, and is available as of
|
||||
Kubernetes 1.7.
|
||||
|
||||
## API server aggregation
|
||||
|
||||
Usually, each resource in the Kubernetes API requires code that handles REST requests and manages
|
||||
persistent storage of objects.
|
||||
The main Kubernetes API server handles built-in resources like *pods* and *services*,
|
||||
and can also handle custom resources in a generic way through [CustomResourceDefinitions](#customresourcedefinitions).
|
||||
|
||||
The [aggregation layer](/docs/concepts/api-extension/) allows you to provide specialized
|
||||
implementations for your custom resources by writing and deploying your own standalone API server.
|
||||
The main API server delegates requests to you for the custom resources that you handle,
|
||||
making them available to all of its clients.
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
{% capture whatsnext %}
|
||||
* Learn how to [Extend the Kubernetes API with the aggregation layer](/docs/concepts/api-extension/apiserver-aggregation/).
|
||||
* Learn how to [Extend the Kubernetes API with CustomResourceDefinition](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/).
|
||||
* Learn how to [Migrate a ThirdPartyResource to CustomResourceDefinition](/docs/tasks/access-kubernetes-api/migrate-third-party-resource/).
|
||||
{% endcapture %}
|
||||
|
||||
{% include templates/concept.md %}
|
||||
@@ -0,0 +1,48 @@
|
||||
---
|
||||
title: Controller manager metrics
|
||||
---
|
||||
|
||||
{% capture overview %}
|
||||
Controller manager metrics provide important insight into the performance and health of
|
||||
the controller manager.
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
{% capture body %}
|
||||
## What are controller manager metrics
|
||||
|
||||
Controller manager metrics provide important insight into the performance and health of the controller manager.
|
||||
These metrics include common Go language runtime metrics such as go_routine count and controller specific metrics such as
|
||||
etcd request latencies or Cloudprovider (AWS, GCE, Openstack) API latencies that can be used
|
||||
to gauge the health of a cluster.
|
||||
|
||||
Starting from Kubernetes 1.7, detailed Cloudprovider metrics are available for storage operations for GCE, AWS, Vsphere and Openstack.
|
||||
These metrics can be used to monitor health of persistent volume operations.
|
||||
|
||||
For example, for GCE these metrics are called:
|
||||
|
||||
```
|
||||
cloudprovider_gce_api_request_duration_seconds { request = "instance_list"}
|
||||
cloudprovider_gce_api_request_duration_seconds { request = "disk_insert"}
|
||||
cloudprovider_gce_api_request_duration_seconds { request = "disk_delete"}
|
||||
cloudprovider_gce_api_request_duration_seconds { request = "attach_disk"}
|
||||
cloudprovider_gce_api_request_duration_seconds { request = "detach_disk"}
|
||||
cloudprovider_gce_api_request_duration_seconds { request = "list_disk"}
|
||||
```
|
||||
|
||||
|
||||
|
||||
## Configuration
|
||||
|
||||
|
||||
In a cluster, controller-manager metrics are available from `http://localhost:10252/metrics`
|
||||
from the host where the controller-manager is running.
|
||||
|
||||
The metrics are emitted in [prometheus format](https://prometheus.io/docs/instrumenting/exposition_formats/) and are human readable.
|
||||
|
||||
In a production environment you may want to configure prometheus or some other metrics scraper
|
||||
to periodically gather these metrics and make them available in some kind of time series database.
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
{% include templates/concept.md %}
|
||||
@@ -2,7 +2,7 @@
|
||||
assignees:
|
||||
- crassirostris
|
||||
- piosz
|
||||
title: Logging and Monitoring Cluster Activity
|
||||
title: Logging Architecture
|
||||
redirect_from:
|
||||
- "/docs/concepts/clusters/logging/"
|
||||
- "/docs/concepts/clusters/logging.html"
|
||||
@@ -15,13 +15,13 @@ Application and systems logs can help you understand what is happening inside yo
|
||||
|
||||
However, the native functionality provided by a container engine or runtime is usually not enough for a complete logging solution. For example, if a container crashes, a pod is evicted, or a node dies, you'll usually still want to access your application's logs. As such, logs should have a separate storage and lifecycle independent of nodes, pods, or containers. This concept is called _cluster-level-logging_. Cluster-level logging requires a separate backend to store, analyze, and query logs. Kubernetes provides no native storage solution for log data, but you can integrate many existing logging solutions into your Kubernetes cluster.
|
||||
|
||||
This document includes:
|
||||
* TOC
|
||||
{:toc}
|
||||
|
||||
* A basic demonstration of logging in Kubernetes using the standard output stream
|
||||
* A detailed description of the node logging architecture in Kubernetes
|
||||
* Guidance for implementing cluster-level logging in Kubernetes
|
||||
|
||||
The guidance for cluster-level logging assumes that a logging backend is present inside or outside of your cluster. If you're not interested in having cluster-level logging, you might still find the description of how logs are stored and handled on the node to be useful.
|
||||
Cluster-level logging architectures are described in assumption that
|
||||
a logging backend is present inside or outside of your cluster. If you're
|
||||
not interested in having cluster-level logging, you might still find
|
||||
the description of how logs are stored and handled on the node to be useful.
|
||||
|
||||
## Basic logging in Kubernetes
|
||||
|
||||
|
||||
@@ -188,7 +188,7 @@ label having key "security" and value "S2".) See the [design doc](https://git.k8
|
||||
for many more examples of pod affinity and anti-affinity, both the `requiredDuringSchedulingIgnoredDuringExecution`
|
||||
flavor and the `preferredDuringSchedulingIgnoredDuringExecution` flavor.
|
||||
|
||||
As with node affinity, the legal operators for pod affinity and anti-affinity are `In`, `NotIn`, `Exists`, `DoesNotExist`, `Gt`, `Lt`.
|
||||
The legal operators for pod affinity and anti-affinity are `In`, `NotIn`, `Exists`, `DoesNotExist`.
|
||||
|
||||
In principle, the `topologyKey` can be any legal label-key. However,
|
||||
for performance and security reasons, there are some constraints on topologyKey:
|
||||
|
||||
@@ -82,9 +82,10 @@ Currently there are several API groups in use:
|
||||
(e.g. `apiVersion: batch/v1`). Full list of supported API groups can be seen in [Kubernetes API reference](/docs/reference/).
|
||||
|
||||
|
||||
There are two supported paths to extending the API.
|
||||
1. [Third Party Resources](https://git.k8s.io/community/contributors/design-proposals/extending-api.md)
|
||||
are for users with very basic CRUD needs.
|
||||
There are two supported paths to extending the API with [custom resources](/docs/concepts/api-extension/custom-resources/):
|
||||
|
||||
1. [CustomResourceDefinition](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/)
|
||||
is for users with very basic CRUD needs.
|
||||
1. Coming soon: users needing the full set of Kubernetes API semantics can implement their own apiserver
|
||||
and use the [aggregator](https://git.k8s.io/community/contributors/design-proposals/aggregated-api-servers.md)
|
||||
to make it seamless for clients.
|
||||
|
||||
@@ -35,6 +35,7 @@ administrator to control the following:
|
||||
| The use of host ports | `hostPorts` |
|
||||
| The use of host's PID namespace | `hostPID` |
|
||||
| The use of host's IPC namespace | `hostIPC` |
|
||||
| The use of host paths | [`allowedHostPaths`](#allowed-host-paths) |
|
||||
| The SELinux context of the container | [`seLinux`](#selinux) |
|
||||
| The user ID | [`runAsUser`](#runasuser) |
|
||||
| Configuring allowable supplemental groups | [`supplementalGroups`](#supplementalgroups) |
|
||||
@@ -119,6 +120,7 @@ to the volume sources that are defined when creating a volume:
|
||||
1. projected
|
||||
1. portworxVolume
|
||||
1. scaleIO
|
||||
1. storageos
|
||||
1. \* (allow all volumes)
|
||||
|
||||
The recommended minimum set of allowed volumes for new PSPs are
|
||||
@@ -126,7 +128,10 @@ configMap, downwardAPI, emptyDir, persistentVolumeClaim, secret, and projected.
|
||||
|
||||
### Host Network
|
||||
- *HostPorts*, default `empty`. List of `HostPortRange`, defined by `min`(inclusive) and `max`(inclusive), which define the allowed host ports.
|
||||
|
||||
|
||||
### Allowed Host Paths
|
||||
- *AllowedHostPaths* is a white list of allowed host path prefixes. Empty indicates that all host paths may be used.
|
||||
|
||||
## Admission
|
||||
|
||||
_Admission control_ with `PodSecurityPolicy` allows for control over the
|
||||
|
||||
@@ -0,0 +1,75 @@
|
||||
---
|
||||
assignees:
|
||||
- rickypai
|
||||
- thockin
|
||||
title: Adding entries to Pod /etc/hosts with HostAliases
|
||||
redirect_from:
|
||||
- "/docs/user-guide/add-entries-to-pod-etc-hosts-with-host-aliases/"
|
||||
- "/docs/user-guide/add-entries-to-pod-etc-hosts-with-host-aliases.md"
|
||||
---
|
||||
|
||||
* TOC
|
||||
{:toc}
|
||||
|
||||
Adding entries to a Pod's /etc/hosts file provides Pod-level override of hostname resolution when DNS and other options are not applicable. In 1.7, users can add these custom entries with the HostAliases field in PodSpec.
|
||||
|
||||
Modification not using HostAliases is not suggested because the file is managed by Kubelet and can be overwritten on during Pod creation/restart.
|
||||
|
||||
## Default Hosts File Content
|
||||
|
||||
Lets start an Nginx Pod which is assigned an Pod IP:
|
||||
```
|
||||
$ kubectl get pods --output=wide
|
||||
NAME READY STATUS RESTARTS AGE IP NODE
|
||||
nginx 1/1 Running 0 13s 10.200.0.4 worker0
|
||||
```
|
||||
|
||||
The hosts file content would look like this:
|
||||
```
|
||||
$ kubectl exec nginx -- cat /etc/hosts
|
||||
# Kubernetes-managed hosts file.
|
||||
127.0.0.1 localhost
|
||||
::1 localhost ip6-localhost ip6-loopback
|
||||
fe00::0 ip6-localnet
|
||||
fe00::0 ip6-mcastprefix
|
||||
fe00::1 ip6-allnodes
|
||||
fe00::2 ip6-allrouters
|
||||
10.200.0.4 nginx
|
||||
```
|
||||
|
||||
by default, the hosts file only includes ipv4 and ipv6 boilerplates like `localhost` and its own hostname.
|
||||
|
||||
## Adding Additional Entries with HostAliases
|
||||
|
||||
In addition the the default boilerplate, we can add additional entries to the hosts file to resolve `foo.local`, `bar.local` to `127.0.0.1` and `foo.remote`, `bar.remote` to `10.1.2.3`, we can by adding HostAliases to the Pod under `.spec.hostAliases`:
|
||||
|
||||
{% include code.html language="yaml" file="hostaliases-pod.yaml" ghlink="/docs/concepts/services-networking/hostaliases-pod.yaml" %}
|
||||
|
||||
The hosts file content would look like this:
|
||||
```
|
||||
$ kubectl logs hostaliases-pod
|
||||
# Kubernetes-managed hosts file.
|
||||
127.0.0.1 localhost
|
||||
::1 localhost ip6-localhost ip6-loopback
|
||||
fe00::0 ip6-localnet
|
||||
fe00::0 ip6-mcastprefix
|
||||
fe00::1 ip6-allnodes
|
||||
fe00::2 ip6-allrouters
|
||||
10.200.0.4 hostaliases-pod
|
||||
127.0.0.1 foo.local
|
||||
127.0.0.1 bar.local
|
||||
10.1.2.3 foo.remote
|
||||
10.1.2.3 bar.remote
|
||||
```
|
||||
|
||||
With the additional entries specified at the bottom.
|
||||
|
||||
## Limitations
|
||||
|
||||
As of 1.7, Pods with hostNetwork enabled will not be able to use this feature. This is because kubelet only manages the hosts file for non-hostNetwork Pods. There are ongoing discussions to change this.
|
||||
|
||||
## Why Does Kubelet Manage the Hosts File?
|
||||
|
||||
kubelet [manages](https://github.com/kubernetes/kubernetes/issues/14633) the hosts file for each container of the Pod to prevent Docker from [modifying](https://github.com/moby/moby/issues/17190) the file after the containers have already been started.
|
||||
|
||||
Because of the managed-nature of the file, any user-written content will be overwritten whenever the hosts file is remounted by Kubelet in the event of a container restart or a Pod reschedule. Thus, it is not suggested to modify the contents of the file.
|
||||
@@ -0,0 +1,21 @@
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: hostaliases-pod
|
||||
spec:
|
||||
hostAliases:
|
||||
- ip: "127.0.0.1"
|
||||
hostnames:
|
||||
- "foo.local"
|
||||
- "bar.local"
|
||||
- ip: "10.1.2.3"
|
||||
hostnames:
|
||||
- "foo.remote"
|
||||
- "bar.remote"
|
||||
containers:
|
||||
- name: cat-hosts
|
||||
image: busybox
|
||||
command:
|
||||
- cat
|
||||
args:
|
||||
- "/etc/hosts"
|
||||
@@ -2,6 +2,7 @@
|
||||
assignees:
|
||||
- thockin
|
||||
- caseydavenport
|
||||
- danwinship
|
||||
title: Network Policies
|
||||
redirect_from:
|
||||
- "/docs/user-guide/networkpolicies/"
|
||||
@@ -13,53 +14,26 @@ redirect_from:
|
||||
|
||||
A network policy is a specification of how groups of pods are allowed to communicate with each other and other network endpoints.
|
||||
|
||||
`NetworkPolicy` resources use labels to select pods and define whitelist rules which allow traffic to the selected pods in addition to what is allowed by the isolation policy for a given namespace.
|
||||
`NetworkPolicy` resources use labels to select pods and define rules which specify what traffic is allowed to the selected pods.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
You must enable the `extensions/v1beta1/networkpolicies` runtime config in your apiserver to enable this resource.
|
||||
Network policies are implemented by the network plugin, so you must be using a networking solution which supports `NetworkPolicy` - simply creating the resource without a controller to implement it will have no effect.
|
||||
|
||||
You must also be using a networking solution which supports `NetworkPolicy` - simply creating the
|
||||
resource without a controller to implement it will have no effect.
|
||||
## Isolated and Non-isolated Pods
|
||||
|
||||
By default, pods are non-isolated; they accept traffic from any source.
|
||||
|
||||
## Configuring Namespace Isolation
|
||||
|
||||
By default, all traffic is allowed between all pods (and `NetworkPolicy` resources have no effect).
|
||||
|
||||
Isolation can be configured on a per-namespace basis. Currently, only isolation on inbound traffic (ingress) can be defined. When a namespace has been configured to isolate inbound traffic, all traffic to pods in that namespace (even from other pods in the same namespace) will be blocked. `NetworkPolicy` objects can then be added to the isolated namespace to specify what traffic should be allowed.
|
||||
|
||||
Ingress isolation can be enabled using an annotation on the Namespace.
|
||||
|
||||
```yaml
|
||||
kind: Namespace
|
||||
apiVersion: v1
|
||||
metadata:
|
||||
annotations:
|
||||
net.beta.kubernetes.io/network-policy: |
|
||||
{
|
||||
"ingress": {
|
||||
"isolation": "DefaultDeny"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
To configure the annotation via `kubectl`:
|
||||
|
||||
```shell
|
||||
{% raw %}
|
||||
kubectl annotate ns <namespace> "net.beta.kubernetes.io/network-policy={\"ingress\": {\"isolation\": \"DefaultDeny\"}}"
|
||||
{% endraw %}
|
||||
```
|
||||
Pods become isolated by having a NetworkPolicy that selects them. Once there is any NetworkPolicy in a Namespace selecting a particular pod, that pod will reject any connections that are not allowed by any NetworkPolicy. (Other pods in the Namespace that are not selected by any NetworkPolicy will continue to accept all traffic.)
|
||||
|
||||
## The `NetworkPolicy` Resource
|
||||
|
||||
See the [api-reference](/docs/api-reference/extensions/v1beta1/definitions/#_v1beta1_networkpolicy) for a full definition of the resource.
|
||||
See the [api-reference](/docs/api-reference/networking/v1/definitions/#_v1_networkpolicy) for a full definition of the resource.
|
||||
|
||||
An example `NetworkPolicy` might look like this:
|
||||
|
||||
```yaml
|
||||
apiVersion: extensions/v1beta1
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: test-network-policy
|
||||
@@ -93,8 +67,36 @@ __ingress__: Each `NetworkPolicy` includes a list of whitelist `ingress` rules.
|
||||
|
||||
So, the example NetworkPolicy:
|
||||
|
||||
1. allows connections to tcp port 6379 of "role=db" pods in the "default" namespace from any pod in the "default" namespace with the label "role=frontend"
|
||||
2. allows connections to tcp port 6379 of "role=db" pods in the "default" namespace from any pod in a namespace with the label "project=myproject"
|
||||
1. isolates "role=db" pods in the "default" namespace (if they weren't already isolated)
|
||||
2. allows connections to tcp port 6379 of "role=db" pods in the "default" namespace from any pod in the "default" namespace with the label "role=frontend"
|
||||
3. allows connections to tcp port 6379 of "role=db" pods in the "default" namespace from any pod in a namespace with the label "project=myproject"
|
||||
|
||||
See the [NetworkPolicy getting started guide](/docs/getting-started-guides/network-policy/walkthrough) for further examples.
|
||||
|
||||
## Default policies
|
||||
|
||||
You can create a "default" isolation policy for a Namespace by creating a NetworkPolicy that selects all pods but does not allow any traffic:
|
||||
|
||||
```yaml
|
||||
apiVersion: networking/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: default-deny
|
||||
spec:
|
||||
podSelector:
|
||||
```
|
||||
|
||||
This ensures that even pods that aren't selected by any other NetworkPolicy will still be isolated.
|
||||
|
||||
Alternatively, if you want to allow all traffic for all pods in a Namespace (even if policies are added that cause some pods to be treated as "isolated"), you can create a policy that explicitly allows all traffic:
|
||||
|
||||
```yaml
|
||||
apiVersion: networking/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: allow-all
|
||||
spec:
|
||||
podSelector:
|
||||
ingress:
|
||||
- {}
|
||||
```
|
||||
|
||||
@@ -127,6 +127,7 @@ However, the particular path specified in the custom recycler pod template in th
|
||||
* VMware Photon
|
||||
* Portworx Volumes
|
||||
* ScaleIO Volumes
|
||||
* StorageOS
|
||||
|
||||
## Persistent Volumes
|
||||
|
||||
@@ -195,6 +196,7 @@ In the CLI, the access modes are abbreviated to:
|
||||
| VsphereVolume | ✓ | - | - |
|
||||
| PortworxVolume | ✓ | - | ✓ |
|
||||
| ScaleIO | ✓ | ✓ | - |
|
||||
| StorageOS | ✓ | - | - |
|
||||
|
||||
### Class
|
||||
|
||||
@@ -472,12 +474,13 @@ metadata:
|
||||
provisioner: kubernetes.io/aws-ebs
|
||||
parameters:
|
||||
type: io1
|
||||
zone: us-east-1d
|
||||
zones: us-east-1d, us-east-1c
|
||||
iopsPerGB: "10"
|
||||
```
|
||||
|
||||
* `type`: `io1`, `gp2`, `sc1`, `st1`. See [AWS docs](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSVolumeTypes.html) for details. Default: `gp2`.
|
||||
* `zone`: AWS zone. If not specified, volumes are generally round-robin-ed across all active zones where Kubernetes cluster has a node.
|
||||
* `zone`: AWS zone. If neither `zone` nor `zones` is specified, volumes are generally round-robin-ed across all active zones where Kubernetes cluster has a node. `zone` and `zones` parameters must not be used at the same time.
|
||||
* `zones`: A comma separated list of AWS zone(s). If neither `zone` nor `zones` is specified, volumes are generally round-robin-ed across all active zones where Kubernetes cluster has a node. `zone` and `zones` parameters must not be used at the same time.
|
||||
* `iopsPerGB`: only for `io1` volumes. I/O operations per second per GiB. AWS volume plugin multiplies this with size of requested volume to compute IOPS of the volume and caps it at 20 000 IOPS (maximum supported by AWS, see [AWS docs](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSVolumeTypes.html). A string is expected here, i.e. `"10"`, not `10`.
|
||||
* `encrypted`: denotes whether the EBS volume should be encrypted or not. Valid values are `"true"` or `"false"`. A string is expected here, i.e. `"true"`, not `true`.
|
||||
* `kmsKeyId`: optional. The full Amazon Resource Name of the key to use when encrypting the volume. If none is supplied but `encrypted` is true, a key is generated by AWS. See AWS docs for valid ARN value.
|
||||
@@ -492,11 +495,12 @@ metadata:
|
||||
provisioner: kubernetes.io/gce-pd
|
||||
parameters:
|
||||
type: pd-standard
|
||||
zone: us-central1-a
|
||||
zones: us-central1-a, us-central1-b
|
||||
```
|
||||
|
||||
* `type`: `pd-standard` or `pd-ssd`. Default: `pd-standard`
|
||||
* `zone`: GCE zone. If not specified, volumes are generally round-robin-ed across all active zones where Kubernetes cluster has a node.
|
||||
* `zone`: GCE zone. If neither `zone` nor `zones` is specified, volumes are generally round-robin-ed across all active zones where Kubernetes cluster has a node. `zone` and `zones` parameters must not be used at the same time.
|
||||
* `zones`: A comma separated list of GCE zone(s). If neither `zone` nor `zones` is specified, volumes are generally round-robin-ed across all active zones where Kubernetes cluster has a node. `zone` and `zones` parameters must not be used at the same time.
|
||||
|
||||
#### Glusterfs
|
||||
|
||||
@@ -761,6 +765,36 @@ as shown in the following command:
|
||||
$> kubectl create secret generic sio-secret --type="kubernetes.io/scaleio" --from-literal=username=sioadmin --from-literal=password=d2NABDNjMA== --namespace=default
|
||||
```
|
||||
|
||||
#### StorageOS
|
||||
```yaml
|
||||
kind: StorageClass
|
||||
apiVersion: storage.k8s.io/v1
|
||||
metadata:
|
||||
name: fast
|
||||
provisioner: kubernetes.io/storageos
|
||||
parameters:
|
||||
pool: default
|
||||
description: Kubernetes volume
|
||||
fsType: ext4
|
||||
adminSecretNamespace: default
|
||||
adminSecretName: storageos-secret
|
||||
```
|
||||
|
||||
* `pool`: The name of the StorageOS distributed capacity pool to provision the volume from. Uses the `default` pool which is normally present if not specified.
|
||||
* `description`: The description to assign to volumes that were created dynamically. All volume descriptions will be the same for the storage class, but different storage classes can be used to allow descriptions for different use cases. Defaults to `Kubernetes volume`.
|
||||
* `fsType`: The default filesystem type to request. Note that user-defined rules within StorageOS may override this value. Defaults to `ext4`.
|
||||
* `adminSecretNamespace`: The namespace where the API configuration secret is located. Required if adminSecretName set.
|
||||
* `adminSecretName`: The name of the secret to use for obtaining the StorageOS API credentials. If not specified, default values will be attempted.
|
||||
|
||||
The StorageOS Kubernetes volume plugin can use a Secret object to specify an endpoint and credentials to access the StorageOS API. This is only required when the defaults have been changed.
|
||||
The secret must be created with type `kubernetes.io/storageos` as shown in the following command:
|
||||
|
||||
```
|
||||
$ kubectl create secret generic storageos-secret --type="kubernetes.io/storageos" --from-literal=apiAddress=tcp://localhost:5705 --from-literal=apiUsername=storageos --from-literal=apiPassword=storageos --namespace=default
|
||||
```
|
||||
|
||||
Secrets used for dynamically provisioned volumes may be created in any namespace and referenced with the `adminSecretNamespace` parameter. Secrets used by pre-provisioned volumes must be created in the same namespace as the PVC that references it.
|
||||
|
||||
## Writing Portable Configuration
|
||||
|
||||
If you're writing configuration templates or examples that run on a wide range of clusters
|
||||
|
||||
@@ -90,6 +90,8 @@ Kubernetes supports several types of Volumes:
|
||||
* `Quobyte`
|
||||
* `PortworxVolume`
|
||||
* `ScaleIO`
|
||||
* `StorageOS`
|
||||
* `local`
|
||||
|
||||
We welcome additional contributions.
|
||||
|
||||
@@ -696,6 +698,96 @@ spec:
|
||||
|
||||
For further detail, plese the see the [ScaleIO examples](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/volumes/scaleio).
|
||||
|
||||
### StorageOS
|
||||
A `storageos` volume allows an existing [StorageOS](https://www.storageos.com) volume to be mounted into your pod.
|
||||
|
||||
StorageOS runs as a container within your Kubernetes environment, making local or attached storage accessible from any node within the Kubernetes cluster. Data can be replicated to protect against node failure. Thin provisioning and compression can improve utilization and reduce cost.
|
||||
|
||||
At its core, StorageOS provides block storage to containers, accessible via a file system.
|
||||
|
||||
The StorageOS container requires 64-bit Linux and has no additional dependencies. A free developer licence is available.
|
||||
|
||||
__Important: You must run the StorageOS container on each node that wants to access StorageOS volumes or that will contribute storage capacity to the pool. For installation instructions, consult the [StorageOS documentation](https://docs.storageos.com)__
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
labels:
|
||||
name: redis
|
||||
role: master
|
||||
name: test-storageos-redis
|
||||
spec:
|
||||
containers:
|
||||
- name: master
|
||||
image: kubernetes/redis:v1
|
||||
env:
|
||||
- name: MASTER
|
||||
value: "true"
|
||||
ports:
|
||||
- containerPort: 6379
|
||||
volumeMounts:
|
||||
- mountPath: /redis-master-data
|
||||
name: redis-data
|
||||
volumes:
|
||||
- name: redis-data
|
||||
storageos:
|
||||
# The `redis-vol01` volume must already exist within StorageOS in the `default` namespace.
|
||||
volumeName: redis-vol01
|
||||
fsType: ext4
|
||||
```
|
||||
|
||||
For more information including Dynamic Provisioning and Persistent Volume Claims, please see the [StorageOS examples](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/volumes/storageos).
|
||||
|
||||
### local
|
||||
|
||||
This volume type is alpha in 1.7.
|
||||
|
||||
A `local` volume represents a mounted local storage device such as a disk,
|
||||
partition or directory.
|
||||
|
||||
Local volumes can only be used as a statically created PersistentVolume.
|
||||
|
||||
Compared to HostPath volumes, local volumes can be used in a durable manner
|
||||
without manually scheduling pods to nodes, as the system is aware of the volume's
|
||||
node constraints.
|
||||
|
||||
However, local volumes are still subject to the availability of the underlying
|
||||
node and are not suitable for all applications.
|
||||
|
||||
The following is an example PersistentVolume spec using a `local` volume:
|
||||
|
||||
``` yaml
|
||||
apiVersion: v1
|
||||
kind: PersistentVolume
|
||||
metadata:
|
||||
name: example-pv
|
||||
annotations:
|
||||
"volume.alpha.kubernetes.io/node-affinity": '{
|
||||
"requiredDuringSchedulingIgnoredDuringExecution": {
|
||||
"nodeSelectorTerms": [
|
||||
{ "matchExpressions": [
|
||||
{ "key": "kubernetes.io/hostname",
|
||||
"operator": "In",
|
||||
"values": ["example-node"]
|
||||
}
|
||||
]}
|
||||
]}
|
||||
}'
|
||||
spec:
|
||||
capacity:
|
||||
storage: 100Gi
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
persistentVolumeReclaimPolicy: Delete
|
||||
storageClassName: local-storage
|
||||
local:
|
||||
path: /mnt/disks/ssd1
|
||||
```
|
||||
|
||||
For details on the `local` volume type, see the [Local Persistent Storage
|
||||
user guide](https://github.com/kubernetes-incubator/external-storage/tree/master/local-volume)
|
||||
|
||||
## Using subPath
|
||||
|
||||
Sometimes, it is useful to share one volume for multiple uses in a single pod. The `volumeMounts.subPath`
|
||||
|
||||
@@ -589,10 +589,10 @@ rolling out a new ReplicaSet, it can be [complete](#complete-deployment), or it
|
||||
|
||||
Kubernetes marks a Deployment as _progressing_ when one of the following tasks is performed:
|
||||
|
||||
* The Deployment is in the process of creating a new ReplicaSet.
|
||||
* The Deployment is scaling up an existing ReplicaSet.
|
||||
* The Deployment is scaling down an existing ReplicaSet.
|
||||
* New pods become available.
|
||||
* The Deployment creates a new ReplicaSet.
|
||||
* The Deployment is scaling up its newest ReplicaSet.
|
||||
* The Deployment is scaling down its older ReplicaSet(s).
|
||||
* New Pods become ready or available (ready for at least [MinReadySeconds](#min-ready-seconds)).
|
||||
|
||||
You can monitor the progress for a Deployment by using `kubectl rollout status`.
|
||||
|
||||
@@ -600,11 +600,10 @@ You can monitor the progress for a Deployment by using `kubectl rollout status`.
|
||||
|
||||
Kubernetes marks a Deployment as _complete_ when it has the following characteristics:
|
||||
|
||||
* The Deployment has minimum availability. Minimum availability means that the Deployment's number of available replicas
|
||||
equals or exceeds the number required by the Deployment strategy.
|
||||
* All of the replicas associated with the Deployment have been updated to the latest version you've specified, meaning any
|
||||
updates you've requested have been completed.
|
||||
* No old pods for the Deployment are running.
|
||||
* All of the replicas associated with the Deployment are available.
|
||||
* No old replicas for the Deployment are running.
|
||||
|
||||
You can check if a Deployment has completed by using `kubectl rollout status`. If the rollout completed
|
||||
successfully, `kubectl rollout status` returns a zero exit code.
|
||||
|
||||
@@ -154,11 +154,9 @@ kubectl delete replicaset my-repset --cascade=false
|
||||
```
|
||||
|
||||
## Known issues
|
||||
* In 1.6, garbage collection does not support non-core resources, e.g.,
|
||||
resources added via ThirdPartyResource or via aggregated API servers. It will
|
||||
support non-core resources in the future. When it does, garbage collector will
|
||||
delete objects with ownerRefereneces referring to non-existent object of a
|
||||
valid non-core resource.
|
||||
* As of 1.7, garbage collection does not yet support
|
||||
[custom resources](/docs/concepts/api-extension/custom-resources/),
|
||||
such as those added through CustomResourceDefinition or aggregated API servers.
|
||||
|
||||
[Other known issues](https://github.com/kubernetes/kubernetes/issues/26120)
|
||||
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
---
|
||||
assignees:
|
||||
- bprashanth
|
||||
- enisoc
|
||||
- erictune
|
||||
- foxish
|
||||
@@ -14,7 +13,7 @@ redirect_from:
|
||||
---
|
||||
|
||||
{% capture overview %}
|
||||
**StatefulSets are a beta feature in 1.5. This feature replaces the
|
||||
**StatefulSets are a beta feature in 1.7. This feature replaces the
|
||||
PetSets feature from 1.4. Users of PetSets are referred to the 1.5
|
||||
[Upgrade Guide](/docs/tasks/manage-stateful-set/upgrade-pet-set-to-stateful-set/)
|
||||
for further information on how to upgrade existing PetSets to StatefulSets.**
|
||||
@@ -34,8 +33,9 @@ following.
|
||||
* Stable, persistent storage.
|
||||
* Ordered, graceful deployment and scaling.
|
||||
* Ordered, graceful deletion and termination.
|
||||
* Ordered, automated rolling updates.
|
||||
|
||||
In the above, stable is synonymous with persistence across Pod (re)schedulings.
|
||||
In the above, stable is synonymous with persistence across Pod (re)scheduling.
|
||||
If an application doesn't require any stable identifiers or ordered deployment,
|
||||
deletion, or scaling, you should deploy your application with a controller that
|
||||
provides a set of stateless replicas. Controllers such as
|
||||
@@ -48,7 +48,6 @@ provides a set of stateless replicas. Controllers such as
|
||||
* The storage for a given Pod must either be provisioned by a [PersistentVolume Provisioner](http://releases.k8s.io/{{page.githubbranch}}/examples/persistent-volume-provisioning/README.md) based on the requested `storage class`, or pre-provisioned by an admin.
|
||||
* Deleting and/or scaling a StatefulSet down will *not* delete the volumes associated with the StatefulSet. This is done to ensure data safety, which is generally more valuable than an automatic purge of all related StatefulSet resources.
|
||||
* StatefulSets currently require a [Headless Service](/docs/concepts/services-networking/service/#headless-services) to be responsible for the network identity of the Pods. You are responsible for creating this Service.
|
||||
* Updating an existing StatefulSet is currently a [manual process](/docs/tutorials/stateful-application/basic-stateful-set/#updating-containers).
|
||||
|
||||
## Components
|
||||
The example below demonstrates the components of a StatefulSet.
|
||||
@@ -59,7 +58,6 @@ The example below demonstrates the components of a StatefulSet.
|
||||
PersistentVolume Provisioner.
|
||||
|
||||
```yaml
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
@@ -154,7 +152,7 @@ PersistentVolume Claims. Note that, the PersistentVolumes associated with the
|
||||
Pods' PersistentVolume Claims are not deleted when the Pods, or StatefulSet are deleted.
|
||||
This must be done manually.
|
||||
|
||||
## Deployment and Scaling Guarantee
|
||||
## Deployment and Scaling Guarantees
|
||||
|
||||
* For a StatefulSet with N replicas, when Pods are being deployed, they are created sequentially, in order from {0..N-1}.
|
||||
* When Pods are being deleted, they are terminated in reverse order, from {N-1..0}.
|
||||
@@ -175,5 +173,62 @@ If a user were to scale the deployed example by patching the StatefulSet such th
|
||||
is fully shutdown and deleted. If web-0 were to fail after web-2 has been terminated and
|
||||
is completely shutdown, but prior to web-1's termination, web-1 would not be terminated
|
||||
until web-0 is Running and Ready.
|
||||
|
||||
### Pod Management Policies
|
||||
In Kubernetes 1.7 and later, StatefulSet allows you to relax its ordering guarantees while
|
||||
preserving its uniqueness and identity guarantees via its `.spec.podManagementPolicy` field.
|
||||
|
||||
#### OrderedReady Pod Management
|
||||
|
||||
`OrderedReady` pod management is the default for StatefulSets. It implements the behavior
|
||||
described [above](#deployment-and-scaling-guarantees).
|
||||
|
||||
#### Parallel Pod Management
|
||||
|
||||
`Parallel` pod management tells the StatefulSet controller to launch or
|
||||
terminate all Pods in parallel, and to not wait for Pods to become Running
|
||||
and Ready or completely terminated prior to launching or terminating another
|
||||
Pod.
|
||||
|
||||
## Update Strategies
|
||||
|
||||
In Kuberentes 1.7 and later, StatefulSet's `.spec.updateStrategy` field allows you to configure
|
||||
and disable automated rolling updates for containers, labels, resource request/limits, and
|
||||
annotations for the Pods in a StatefulSet.
|
||||
|
||||
### On Delete
|
||||
|
||||
The `OnDelete` update strategy implements the legacy (1.6 and prior) behavior. It is the default
|
||||
strategy when `spec.updateStrategy` is left unspecified. When a StatefulSet's
|
||||
`.spec.updateStrategy.type` is set to `OnDelete`, the StatefulSet controller will not automatically
|
||||
update the Pods in a StatefulSet. Users must manually delete Pods to cause the controller to
|
||||
create new Pods that reflect modifications made to a StatefulSet's `.spec.template`.
|
||||
|
||||
### Rolling Updates
|
||||
|
||||
The `RollingUpdate` update strategy implements automated, rolling update for the Pods in a
|
||||
StatefulSet. When a StatefulSet's `.spec.updateStrategy.type` is set to `RollingUpdate`, the
|
||||
StatefulSet controller will delete and recreate each Pod in the StatefulSet. It will proceed
|
||||
in the same order as Pod termination (from the largest ordinal to the smallest), updating
|
||||
each Pod one at a time. It will wait until an updated Pod is Running and Ready prior to
|
||||
updating its predecessor.
|
||||
|
||||
#### Partitions
|
||||
|
||||
The `RollingUpdate` update strategy can be partitioned, by specifying a
|
||||
`.spec.updateStrategy.rollingUpdate.partition`. If a partition is specified, all Pods with an
|
||||
ordinal that is greater than or equal to the partition will be updated when the StatefulSet's
|
||||
`.spec.template` is updated. All Pods with an ordinal that is less than the partition will not
|
||||
be updated, and, even if they are deleted, they will be recreated at the previous version. If a
|
||||
StatefulSet's `.spec.updateStrategy.rollingUpdate.partition` is greater than its `.spec.replicas`,
|
||||
updates to its `.spec.template` will not be propagated to its Pods.
|
||||
In most cases you will not need to use a partition, but they are useful if you want to stage an
|
||||
update, roll out a canary, or perform a phased roll out.
|
||||
|
||||
{% endcapture %}
|
||||
{% capture whatsnext %}
|
||||
|
||||
* Follow an example of [deploying a stateful application](/docs/tutorials/stateful-application/basic-stateful-set).
|
||||
|
||||
{% endcapture %}
|
||||
{% include templates/concept.md %}
|
||||
|
||||
@@ -0,0 +1,276 @@
|
||||
---
|
||||
assignees:
|
||||
- erictune
|
||||
- foxish
|
||||
- davidopp
|
||||
title: Disruptions
|
||||
redirect_from:
|
||||
- "/docs/admin/disruptions/"
|
||||
- "/docs/admin/disruptions.html"
|
||||
- "/docs/tasks/configure-pod-container/configure-pod-disruption-budget/"
|
||||
- "/docs/tasks/administer-cluster/configure-pod-disruption-budget/"
|
||||
---
|
||||
|
||||
{% capture overview %}
|
||||
This guide is for application owners who want to build
|
||||
highly available applications, and thus need to understand
|
||||
what types of Disruptions can happen to Pods.
|
||||
|
||||
It is also for Cluster Administrators who want to perform automated
|
||||
cluster actions, like upgrading and autoscaling clusters.
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
{:toc}
|
||||
|
||||
{% capture body %}
|
||||
|
||||
## Voluntary and Involuntary Disruptions
|
||||
|
||||
Pods do not disappear until someone (a person or a controller) destroys them, or
|
||||
there is an unavoidable hardware or system software error.
|
||||
|
||||
We call these unavoidable cases *involuntary disruptions* to
|
||||
an application. Examples are:
|
||||
|
||||
- a hardware failure of the physical machine backing the node
|
||||
- cluster administrator deletes VM (instance) by mistake
|
||||
- cloud provider or hypervisor failure makes VM disappear
|
||||
- a kernel panic
|
||||
- if the node to disappears from the cluster due to cluster network partition
|
||||
- eviction of a pod due to the node being [out-of-resources](/docs/tasks/administer-cluster/out-of-resource.md).
|
||||
|
||||
Except for the out-of-resources condition, all these conditions
|
||||
should be familiar to most users; they are are not specific
|
||||
to Kubernetes.
|
||||
|
||||
We call other cases *voluntary disruptions*. These include both
|
||||
actions initiated by the application owner and those initiated by a Cluster
|
||||
Administrator. Typical application owner actions include:
|
||||
|
||||
- deleting the deployment or other controller that manages the pod
|
||||
- updating a deployment's pod template causing a restart
|
||||
- directly deleting a pod (e.g. by accident)
|
||||
|
||||
Cluster Administrator actions include:
|
||||
|
||||
- [Draining a node](/docs//tasks/administer-cluster/safely-drain-node.md) for repair or upgrade.
|
||||
- Draining a node from a cluster to scale the cluster down (learn about
|
||||
[Cluster Autoscaling](/docs/tasks/administer-cluster/cluster-management/#cluster-autoscaler)
|
||||
).
|
||||
- Removing a pod from a node to permit something else to fit on that node.
|
||||
|
||||
These actions might be taken directly by the cluster administrator, or by automation
|
||||
run by the cluster administrator, or by your cluster hosting provider.
|
||||
|
||||
Ask your cluster administrator or consult your cloud provider or distribution documentation
|
||||
to determine if any sources of voluntary disruptions are enabled for your cluster.
|
||||
If none are enabled, you can skip creating Pod Disruption Budgets.
|
||||
|
||||
## Dealing with Disruptions
|
||||
|
||||
Here are some ways to mitigate involuntary disruptions:
|
||||
|
||||
- Ensure your pod [requests the resources](/docs/tasks/configure-pod-container/assign-cpu-ram-container) it needs.
|
||||
- Replicate your application if you need higher availability. (Learn about running replicated
|
||||
[stateless](/docs/tasks/run-application/run-stateless-application-deployment.md)
|
||||
and [stateful](/docs/tasks/run-application/run-replicated-stateful-application.md) applications.)
|
||||
- For even higher availability when running replicated applications,
|
||||
spread applications across racks (using
|
||||
[anti-affinity](/docs/user-guide/node-selection/#inter-pod-affinity-and-anti-affinity-beta-feature))
|
||||
or across zones (if using a
|
||||
[multi-zone cluster](/docs/admin/multiple-zones).)
|
||||
|
||||
The frequency of voluntary disruptions varies. On a basic Kubernetes cluster, there are
|
||||
no voluntary disruptions at all. However, your cluster administrator or hosting provider
|
||||
may run some additional services which cause voluntary disruptions. For example,
|
||||
rolling out node software updates can cause voluntary updates. Also, some implementations
|
||||
of cluster (node) autoscaling may cause voluntary disruptions to defragment and compact nodes.
|
||||
You cluster administrator or hosting provider should have documented what level of voluntary
|
||||
disruptions, if any, to expect.
|
||||
|
||||
Kubernetes offers features to help run highly available applications at the same
|
||||
time as frequent voluntary disruptions. We call this set of features
|
||||
*Disruption Budgets*.
|
||||
|
||||
|
||||
## How Disruption Budgets Work
|
||||
|
||||
An Application Owner can create a `PodDisruptionBudget` object (PDB) for each application.
|
||||
A PDB limits the number pods of a replicated application that are down simultaneously from
|
||||
voluntary disruptions. For example, a quorum-based application would
|
||||
like to ensure that the number of replicas running is never brought below the
|
||||
number needed for a quorum. A web front end might want to
|
||||
ensure that the number of replicas serving load never falls below a certain
|
||||
percentage of the total.
|
||||
|
||||
Cluster managers and hosting providers should use tools which
|
||||
respect Pod Disruption Budgets by calling the [Eviction API](/docs/tasks/administer-cluster/safely-drain-node/#the-eviction-api)
|
||||
instead of directly deleting pods. Examples are the `kubectl drain` command
|
||||
and the Kubernetes-on-GCE cluster upgrade script (`cluster/gce/upgrade.sh`).
|
||||
|
||||
When a cluster administrator wants to drain a node
|
||||
they use the `kubectl drain` command. That tool tries to evict all
|
||||
the pods on the machine. The eviction request may be temporarily rejected,
|
||||
and the tool periodically retries all failed requests until all pods
|
||||
are terminated, or until a configurable timeout is reached.
|
||||
|
||||
A PDB specifies the number of replicas that an application can tolerate having, relative to how
|
||||
many it is intended to have. For example, a Deployment which has a `spec.replicas: 5` is
|
||||
supposed to have 5 pods at any given time. If its PDB allows for there to be 4 at a time,
|
||||
then the Eviction API will allow voluntary disruption of one, but not two pods, at a time.
|
||||
|
||||
The group of pods that comprise the application is specified using a label selector, the same
|
||||
as the one used by the application's controller (deployment, stateful-set, etc).
|
||||
|
||||
The "intended" number of pods is computed from the `.spec.replicas` of the pods controller.
|
||||
The controller is discovered from the pods using the `.metadata.ownerReferences` of the object.
|
||||
|
||||
PDBs cannot prevent [involuntary disruptions](#voluntary-and-involuntary-disruptions) from
|
||||
occuring, but they do count against the budget.
|
||||
|
||||
Pods which are deleted or unavailable due to a rolling upgrade to an application do count
|
||||
against the disruption budget, but controllers (like deployment and stateful-set)
|
||||
are not limited by PDBs when doing rolling upgrades -- the handling of failures
|
||||
during application updates is configured in the controller spec.
|
||||
(Learn about [updating a deployment](/docs/concepts/cluster-administration/manage-deployment/#updating-your-application-without-a-service-outage).)
|
||||
|
||||
When a pod is evicted using the eviction API, it is gracefully terminated (see
|
||||
`terminationGracePeriodSeconds` in [PodSpec](/docs/resources-reference/v1.6/#podspec-v1-core).)
|
||||
|
||||
## PDB Example
|
||||
|
||||
Consider a cluster with 3 nodes, `node-1` through `node-3`.
|
||||
The cluster is running several applications. One of them has 3 replicas initially called
|
||||
`pod-a`, `pod-b`, and `pod-c`. Another, unrelated pod without a PDB, called `pod-x`, is also shown.
|
||||
Initially, the pods are laid out as follows:
|
||||
|
||||
| node-1 | node-2 | node-3 |
|
||||
|:--------------------:|:-------------------:|:------------------:|
|
||||
| pod-a *available* | pod-b *available* | pod-c *available* |
|
||||
| pod-x *available* | | |
|
||||
|
||||
All 3 pods are part of an deployment, and they collectively have a PDB which requires
|
||||
there be at least 2 of the 3 pods to be available at all times.
|
||||
|
||||
For example, assume the cluster administrator wants to reboot into a new kernel version to fix a bug in the kernel.
|
||||
The cluster administrator first tries to drain `node-1` using the `kubectl drain` command.
|
||||
That tool tries to evict `pod-a` and `pod-x`. This succeeds immediately.
|
||||
Both pods go into the `terminating` state at the same time.
|
||||
This puts the cluster in this state:
|
||||
|
||||
| node-1 *draining* | node-2 | node-3 |
|
||||
|:--------------------:|:-------------------:|:------------------:|
|
||||
| pod-a *terminating* | pod-b *available* | pod-c *available* |
|
||||
| pod-x *terminating* | | |
|
||||
|
||||
The deployment notices that one of the pods is terminating, so it creates a replacement
|
||||
called `pod-d`. Since `node-1` is cordoned, it lands on another node. Something has
|
||||
also created `pod-y` as a replacement for `pod-x`.
|
||||
|
||||
(Note: for a StatefulSet, `pod-a`, which would be called something like `pod-1`, would need
|
||||
to terminate completely before its replacement, which is also called `pod-1` but has a
|
||||
different UID, could be created. Otherwise, the example applies to a StatefulSet as well.)
|
||||
|
||||
Now the cluster is in this state:
|
||||
|
||||
| node-1 *draining* | node-2 | node-3 |
|
||||
|:--------------------:|:-------------------:|:------------------:|
|
||||
| pod-a *terminating* | pod-b *available* | pod-c *available* |
|
||||
| pod-x *terminating* | pod-d *starting* | pod-y |
|
||||
|
||||
At some point, the pods terminate, and the cluster look like this:
|
||||
|
||||
| node-1 *drained* | node-2 | node-3 |
|
||||
|:--------------------:|:-------------------:|:------------------:|
|
||||
| | pod-b *available* | pod-c *available* |
|
||||
| | pod-d *starting* | pod-y |
|
||||
|
||||
At this point, if an impatient cluster administrator tries to drain `node-2` or
|
||||
`node-3`, the drain command will block, because there are only 2 available
|
||||
pods for the deployment, and its PDB requires at least 2. After some time
|
||||
|
||||
asses, `pod-d` becomes available.
|
||||
|
||||
The cluster state now looks like this:
|
||||
|
||||
| node-1 *drained* | node-2 | node-3 |
|
||||
|:--------------------:|:-------------------:|:------------------:|
|
||||
| | pod-b *available* | pod-c *available* |
|
||||
| | pod-d *available* | pod-y |
|
||||
|
||||
Now, the cluster admin tries to drain `node-2`.
|
||||
The drain command will try to evict the two pods in some order, say
|
||||
`pod-b` first and then `pod-d`. It will succeed at evicting `pod-b`.
|
||||
But, when it tries to evict `pod-d`, it will be refused because that would leave only
|
||||
one pod available for the deployment.
|
||||
|
||||
The deployment creates a replacement for `pod-b` called `pod-e`.
|
||||
However, not there are not enough resources in the cluster to schedule
|
||||
`pod-e`. So, the drain then the drain will block. The cluster may end up in this
|
||||
state:
|
||||
|
||||
| node-1 *drained* | node-2 | node-3 | *no node* |
|
||||
|:--------------------:|:-------------------:|:------------------:|:------------------:|
|
||||
| | pod-b *available* | pod-c *available* | pod-e *pending* |
|
||||
| | pod-d *available* | pod-y | |
|
||||
|
||||
At this point, the cluster administrator needs to
|
||||
add a node back to the cluster to proceed with the upgrade.
|
||||
|
||||
You can see how Kubernetes varies the rate at which disruptions
|
||||
can happen, according to:
|
||||
|
||||
- how many replicas an application needs
|
||||
- how long it takes to gracefully shutdown an instance
|
||||
- how long it takes a new instance to start up
|
||||
- the type of controller
|
||||
- the cluster's resource capacity
|
||||
|
||||
## Separating Cluster Owner and Application Owner Roles
|
||||
|
||||
Often, it is useful to think of the Cluster Manager
|
||||
and Application Owner as separate roles with limited knowledge
|
||||
of each other. This separation of responsibilities
|
||||
may make sense in these scenarios:
|
||||
|
||||
- when there are many application teams sharing a Kubernetes cluster, and
|
||||
there is natural specialization of roles
|
||||
- when third-party tools or services are used to automate cluster management
|
||||
|
||||
Pod Disruption Budgets support this separation of roles by providing an
|
||||
interface between the roles.
|
||||
|
||||
If you do not have such a separation of responsibilities in your organization,
|
||||
you may not need to use Pod Disruption Budgets.
|
||||
|
||||
## How to perform Distruptive Actions your Cluster
|
||||
|
||||
If you are a Cluster Administrator, and you need to perform a disruptive action on all
|
||||
the nodes in your cluster, such as a node or system software upgrade, here are some options:
|
||||
|
||||
- Accept downtime during the upgrade.
|
||||
- Fail over to another complete replica cluster.
|
||||
- No downtime, but may be costly both for the duplicated nodes,
|
||||
and for human effort to orchestrate the switchover.
|
||||
- Write disruption tolerant applications and use PDBs.
|
||||
- No downtime.
|
||||
- Minimal resource duplication.
|
||||
- Allows more automation of cluster administration.
|
||||
- Writing disruption-tolerant applications is tricky, but the work to tolerate voluntary
|
||||
disruptions largely overlaps with work to support autoscaling and tolerating
|
||||
involuntary disruptions.
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
|
||||
{% capture whatsnext %}
|
||||
|
||||
* Follow steps to protect your application by [configuring a Pod Disruption Budget](/docs/tasks/run-application//configure-pdb.md).
|
||||
|
||||
* Learn more about [draining nodes](/docs/tasks/administer-cluster//safely-drain-node.md)
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
|
||||
{% include templates/concept.md %}
|
||||
Reference in New Issue
Block a user