delete un-translated docs
This commit is contained in:
@@ -1,93 +0,0 @@
|
||||
---
|
||||
approvers:
|
||||
- mikedanese
|
||||
title: Configuration Best Practices
|
||||
---
|
||||
|
||||
{% capture overview %}
|
||||
This document highlights and consolidates configuration best practices that are introduced throughout the user-guide, getting-started documentation, and examples.
|
||||
|
||||
This is a living document. If you think of something that is not on this list but might be useful to others, please don't hesitate to file an issue or submit a PR.
|
||||
{% endcapture %}
|
||||
|
||||
{% capture body %}
|
||||
## General Config Tips
|
||||
|
||||
- When defining configurations, specify the latest stable API version (currently v1).
|
||||
|
||||
- Configuration files should be stored in version control before being pushed to the cluster. This allows quick roll-back of a configuration if needed. It also aids with cluster re-creation and restoration if necessary.
|
||||
|
||||
- Write your configuration files using YAML rather than JSON. Though these formats can be used interchangeably in almost all scenarios, YAML tends to be more user-friendly.
|
||||
|
||||
- Group related objects into a single file whenever it makes sense. One file is often easier to manage than several. See the [guestbook-all-in-one.yaml](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/guestbook/all-in-one/guestbook-all-in-one.yaml) file as an example of this syntax.
|
||||
|
||||
Note also that many `kubectl` commands can be called on a directory, so you can also call `kubectl create` on a directory of config files. See below for more details.
|
||||
|
||||
- Don't specify default values unnecessarily, in order to simplify and minimize configs, and to reduce error. For example, omit the selector and labels in a `ReplicationController` if you want them to be the same as the labels in its `podTemplate`, since those fields are populated from the `podTemplate` labels by default. See the [guestbook app's](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/guestbook/) .yaml files for some [examples](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/guestbook/frontend-deployment.yaml) of this.
|
||||
|
||||
- Put an object description in an annotation to allow better introspection.
|
||||
|
||||
|
||||
## "Naked" Pods vs Replication Controllers and Jobs
|
||||
|
||||
- If there is a viable alternative to naked pods (in other words: pods not bound to a [replication controller](/docs/user-guide/replication-controller)), go with the alternative. Naked pods will not be rescheduled in the event of node failure.
|
||||
|
||||
Replication controllers are almost always preferable to creating pods, except for some explicit [`restartPolicy: Never`](/docs/concepts/workloads/pods/pod-lifecycle/#restart-policy) scenarios. A [Job](/docs/concepts/jobs/run-to-completion-finite-workloads/) object (currently in Beta) may also be appropriate.
|
||||
|
||||
|
||||
## Services
|
||||
|
||||
- It's typically best to create a [service](/docs/concepts/services-networking/service/) before corresponding [replication controllers](/docs/concepts/workloads/controllers/replicationcontroller/). This lets the scheduler spread the pods that comprise the service.
|
||||
|
||||
You can also use this process to ensure that at least one replica works before creating lots of them:
|
||||
|
||||
1. Create a replication controller without specifying replicas (this will set replicas=1);
|
||||
2. Create a service;
|
||||
3. Then scale up the replication controller.
|
||||
|
||||
- Don't use `hostPort` unless it is absolutely necessary (for example: for a node daemon). It specifies the port number to expose on the host. When you bind a Pod to a `hostPort`, there are a limited number of places to schedule a pod due to port conflicts— you can only schedule as many such Pods as there are nodes in your Kubernetes cluster.
|
||||
|
||||
If you only need access to the port for debugging purposes, you can use the [kubectl proxy and apiserver proxy](/docs/tasks/access-kubernetes-api/http-proxy-access-api/) or [kubectl port-forward](/docs/tasks/access-application-cluster/port-forward-access-application-cluster/).
|
||||
You can use a [Service](/docs/concepts/services-networking/service/) object for external service access.
|
||||
|
||||
If you explicitly need to expose a pod's port on the host machine, consider using a [NodePort](/docs/user-guide/services/#type-nodeport) service before resorting to `hostPort`.
|
||||
|
||||
- Avoid using `hostNetwork`, for the same reasons as `hostPort`.
|
||||
|
||||
- Use _headless services_ for easy service discovery when you don't need kube-proxy load balancing. See [headless services](/docs/user-guide/services/#headless-services).
|
||||
|
||||
## Using Labels
|
||||
|
||||
- Define and use [labels](/docs/user-guide/labels/) that identify __semantic attributes__ of your application or deployment. For example, instead of attaching a label to a set of pods to explicitly represent some service (For example, `service: myservice`), or explicitly representing the replication controller managing the pods (for example, `controller: mycontroller`), attach labels that identify semantic attributes, such as `{ app: myapp, tier: frontend, phase: test, deployment: v3 }`. This will let you select the object groups appropriate to the context— for example, a service for all "tier: frontend" pods, or all "test" phase components of app "myapp". See the [guestbook](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/guestbook/) app for an example of this approach.
|
||||
|
||||
A service can be made to span multiple deployments, such as is done across [rolling updates](/docs/tasks/run-application/rolling-update-replication-controller/), by simply omitting release-specific labels from its selector, rather than updating a service's selector to match the replication controller's selector fully.
|
||||
|
||||
- To facilitate rolling updates, include version info in replication controller names, for example as a suffix to the name. It is useful to set a 'version' label as well. The rolling update creates a new controller as opposed to modifying the existing controller. So, there will be issues with version-agnostic controller names. See the [documentation](/docs/tasks/run-application/rolling-update-replication-controller/) on the rolling-update command for more detail.
|
||||
|
||||
Note that the [Deployment](/docs/concepts/workloads/controllers/deployment/) object obviates the need to manage replication controller 'version names'. A desired state of an object is described by a Deployment, and if changes to that spec are _applied_, the deployment controller changes the actual state to the desired state at a controlled rate. (Deployment objects are currently part of the [`extensions` API Group](/docs/concepts/overview/kubernetes-api/#api-groups).)
|
||||
|
||||
- You can manipulate labels for debugging. Because Kubernetes replication controllers and services match to pods using labels, this allows you to remove a pod from being considered by a controller, or served traffic by a service, by removing the relevant selector labels. If you remove the labels of an existing pod, its controller will create a new pod to take its place. This is a useful way to debug a previously "live" pod in a quarantine environment. See the [`kubectl label`](/docs/concepts/overview/working-with-objects/labels/) command.
|
||||
|
||||
## Container Images
|
||||
|
||||
- The [default container image pull policy](/docs/concepts/containers/images/) is `IfNotPresent`, which causes the [Kubelet](/docs/admin/kubelet/) to not pull an image if it already exists. If you would like to always force a pull, you must specify a pull image policy of `Always` in your .yaml file (`imagePullPolicy: Always`) or specify a `:latest` tag on your image.
|
||||
|
||||
That is, if you're specifying an image with other than the `:latest` tag, for example `myimage:v1`, and there is an image update to that same tag, the Kubelet won't pull the updated image. You can address this by ensuring that any updates to an image bump the image tag as well (for example, `myimage:v2`), and ensuring that your configs point to the correct version.
|
||||
|
||||
**Note:** You should avoid using `:latest` tag when deploying containers in production, because this makes it hard to track which version of the image is running and hard to roll back.
|
||||
|
||||
- To work only with a specific version of an image, you can specify an image with its digest (SHA256). This approach guarantees that the image will never update. For detailed information about working with image digests, see [the Docker documentation](https://docs.docker.com/engine/reference/commandline/pull/#pull-an-image-by-digest-immutable-identifier).
|
||||
|
||||
## Using kubectl
|
||||
|
||||
- Use `kubectl create -f <directory>` where possible. This looks for config objects in all `.yaml`, `.yml`, and `.json` files in `<directory>` and passes them to `create`.
|
||||
|
||||
- Use `kubectl delete` rather than `stop`. `Delete` has a superset of the functionality of `stop`, and `stop` is deprecated.
|
||||
|
||||
- Use kubectl bulk operations (via files and/or labels) for get and delete. See [label selectors](/docs/user-guide/labels/#label-selectors) and [using labels effectively](/docs/concepts/cluster-administration/manage-deployment/#using-labels-effectively).
|
||||
|
||||
- Use `kubectl run` and `expose` to quickly create and expose single container Deployments. See the [quick start guide](/docs/user-guide/quick-start/) for an example.
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
{% include templates/concept.md %}
|
||||
@@ -1,298 +0,0 @@
|
||||
---
|
||||
approvers:
|
||||
- bprashanth
|
||||
title: Ingress Resources
|
||||
---
|
||||
|
||||
* TOC
|
||||
{:toc}
|
||||
|
||||
__Terminology__
|
||||
|
||||
Throughout this doc you will see a few terms that are sometimes used interchangeably elsewhere, that might cause confusion. This section attempts to clarify them.
|
||||
|
||||
* Node: A single virtual or physical machine in a Kubernetes cluster.
|
||||
* Cluster: A group of nodes firewalled from the internet, that are the primary compute resources managed by Kubernetes.
|
||||
* Edge router: A router that enforces the firewall policy for your cluster. This could be a gateway managed by a cloud provider or a physical piece of hardware.
|
||||
* Cluster network: A set of links, logical or physical, that facilitate communication within a cluster according to the [Kubernetes networking model](/docs/concepts/cluster-administration/networking/). Examples of a Cluster network include Overlays such as [flannel](https://github.com/coreos/flannel#flannel) or SDNs such as [OVS](/docs/admin/ovs-networking/).
|
||||
* Service: A Kubernetes [Service](/docs/concepts/services-networking/service/) that identifies a set of pods using label selectors. Unless mentioned otherwise, Services are assumed to have virtual IPs only routable within the cluster network.
|
||||
|
||||
## What is Ingress?
|
||||
|
||||
Typically, services and pods have IPs only routable by the cluster network. All traffic that ends up at an edge router is either dropped or forwarded elsewhere. Conceptually, this might look like:
|
||||
|
||||
```
|
||||
internet
|
||||
|
|
||||
------------
|
||||
[ Services ]
|
||||
```
|
||||
|
||||
An Ingress is a collection of rules that allow inbound connections to reach the cluster services.
|
||||
|
||||
```
|
||||
internet
|
||||
|
|
||||
[ Ingress ]
|
||||
--|-----|--
|
||||
[ Services ]
|
||||
```
|
||||
|
||||
It can be configured to give services externally-reachable URLs, load balance traffic, terminate SSL, offer name based virtual hosting etc. Users request ingress by POSTing the Ingress resource to the API server. An [Ingress controller](#ingress-controllers) is responsible for fulfilling the Ingress, usually with a loadbalancer, though it may also configure your edge router or additional frontends to help handle the traffic in an HA manner.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Before you start using the Ingress resource, there are a few things you should understand. The Ingress is a beta resource, not available in any Kubernetes release prior to 1.1. You need an Ingress controller to satisfy an Ingress, simply creating the resource will have no effect.
|
||||
|
||||
GCE/GKE deploys an ingress controller on the master. You can deploy any number of custom ingress controllers in a pod. You must annotate each ingress with the appropriate class, as indicated [here](https://git.k8s.io/ingress/controllers/nginx#running-multiple-ingress-controllers) and [here](https://git.k8s.io/ingress/controllers/gce/BETA_LIMITATIONS.md#disabling-glbc).
|
||||
|
||||
Make sure you review the [beta limitations](https://git.k8s.io/ingress/controllers/gce/BETA_LIMITATIONS.md) of this controller. In environments other than GCE/GKE, you need to [deploy a controller](https://git.k8s.io/ingress/controllers) as a pod.
|
||||
|
||||
## The Ingress Resource
|
||||
|
||||
A minimal Ingress might look like:
|
||||
|
||||
```yaml
|
||||
apiVersion: extensions/v1beta1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: test-ingress
|
||||
annotations:
|
||||
ingress.kubernetes.io/rewrite-target: /
|
||||
spec:
|
||||
rules:
|
||||
- http:
|
||||
paths:
|
||||
- path: /testpath
|
||||
backend:
|
||||
serviceName: test
|
||||
servicePort: 80
|
||||
```
|
||||
|
||||
*POSTing this to the API server will have no effect if you have not configured an [Ingress controller](#ingress-controllers).*
|
||||
|
||||
__Lines 1-6__: As with all other Kubernetes config, an Ingress needs `apiVersion`, `kind`, and `metadata` fields. For general information about working with config files, see [deploying applications](/docs/tasks/run-application/run-stateless-application-deployment/), [configuring containers](/docs/tasks/configure-pod-container/configmap/), [managing resources](/docs/concepts/cluster-administration/manage-deployment/) and [ingress configuration rewrite](https://github.com/kubernetes/ingress/blob/master/controllers/nginx/configuration.md#rewrite).
|
||||
|
||||
__Lines 7-9__: Ingress [spec](https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status) has all the information needed to configure a loadbalancer or proxy server. Most importantly, it contains a list of rules matched against all incoming requests. Currently the Ingress resource only supports http rules.
|
||||
|
||||
__Lines 10-11__: Each http rule contains the following information: A host (e.g.: foo.bar.com, defaults to * in this example), a list of paths (e.g.: /testpath) each of which has an associated backend (test:80). Both the host and path must match the content of an incoming request before the loadbalancer directs traffic to the backend.
|
||||
|
||||
__Lines 12-14__: A backend is a service:port combination as described in the [services doc](/docs/concepts/services-networking/service/). Ingress traffic is typically sent directly to the endpoints matching a backend.
|
||||
|
||||
__Global Parameters__: For the sake of simplicity the example Ingress has no global parameters, see the [API reference](https://releases.k8s.io/{{page.githubbranch}}/staging/src/k8s.io/api/extensions/v1beta1/types.go) for a full definition of the resource. One can specify a global default backend in the absence of which requests that don't match a path in the spec are sent to the default backend of the Ingress controller.
|
||||
|
||||
## Ingress controllers
|
||||
|
||||
In order for the Ingress resource to work, the cluster must have an Ingress controller running. This is unlike other types of controllers, which typically run as part of the `kube-controller-manager` binary, and which are typically started automatically as part of cluster creation. You need to choose the ingress controller implementation that is the best fit for your cluster, or implement one. Examples and instructions can be found [here](https://git.k8s.io/ingress/controllers).
|
||||
|
||||
## Before you begin
|
||||
|
||||
The following document describes a set of cross platform features exposed through the Ingress resource. Ideally, all Ingress controllers should fulfill this specification, but we're not there yet. The docs for the GCE and nginx controllers are [here](https://git.k8s.io/ingress/controllers/gce/README.md) and [here](https://git.k8s.io/ingress/controllers/nginx/README.md) respectively. **Make sure you review controller specific docs so you understand the caveats of each one**.
|
||||
|
||||
## Types of Ingress
|
||||
|
||||
### Single Service Ingress
|
||||
|
||||
There are existing Kubernetes concepts that allow you to expose a single service (see [alternatives](#alternatives)), however you can do so through an Ingress as well, by specifying a *default backend* with no rules.
|
||||
|
||||
{% include code.html language="yaml" file="ingress.yaml" ghlink="/docs/concepts/services-networking/ingress.yaml" %}
|
||||
|
||||
If you create it using `kubectl create -f` you should see:
|
||||
|
||||
```shell
|
||||
$ kubectl get ing
|
||||
NAME RULE BACKEND ADDRESS
|
||||
test-ingress - testsvc:80 107.178.254.228
|
||||
```
|
||||
|
||||
Where `107.178.254.228` is the IP allocated by the Ingress controller to satisfy this Ingress. The `RULE` column shows that all traffic send to the IP is directed to the Kubernetes Service listed under `BACKEND`.
|
||||
|
||||
### Simple fanout
|
||||
|
||||
As described previously, pods within kubernetes have IPs only visible on the cluster network, so we need something at the edge accepting ingress traffic and proxying it to the right endpoints. This component is usually a highly available loadbalancer. An Ingress allows you to keep the number of loadbalancers down to a minimum, for example, a setup like:
|
||||
|
||||
```shell
|
||||
foo.bar.com -> 178.91.123.132 -> / foo s1:80
|
||||
/ bar s2:80
|
||||
```
|
||||
|
||||
would require an Ingress such as:
|
||||
|
||||
```yaml
|
||||
apiVersion: extensions/v1beta1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: test
|
||||
annotations:
|
||||
ingress.kubernetes.io/rewrite-target: /
|
||||
spec:
|
||||
rules:
|
||||
- host: foo.bar.com
|
||||
http:
|
||||
paths:
|
||||
- path: /foo
|
||||
backend:
|
||||
serviceName: s1
|
||||
servicePort: 80
|
||||
- path: /bar
|
||||
backend:
|
||||
serviceName: s2
|
||||
servicePort: 80
|
||||
```
|
||||
|
||||
When you create the Ingress with `kubectl create -f`:
|
||||
|
||||
```shell
|
||||
$ kubectl get ing
|
||||
NAME RULE BACKEND ADDRESS
|
||||
test -
|
||||
foo.bar.com
|
||||
/foo s1:80
|
||||
/bar s2:80
|
||||
```
|
||||
The Ingress controller will provision an implementation specific loadbalancer that satisfies the Ingress, as long as the services (s1, s2) exist. When it has done so, you will see the address of the loadbalancer under the last column of the Ingress.
|
||||
|
||||
### Name based virtual hosting
|
||||
|
||||
Name-based virtual hosts use multiple host names for the same IP address.
|
||||
|
||||
```
|
||||
foo.bar.com --| |-> foo.bar.com s1:80
|
||||
| 178.91.123.132 |
|
||||
bar.foo.com --| |-> bar.foo.com s2:80
|
||||
```
|
||||
|
||||
The following Ingress tells the backing loadbalancer to route requests based on the [Host header](https://tools.ietf.org/html/rfc7230#section-5.4).
|
||||
|
||||
```yaml
|
||||
apiVersion: extensions/v1beta1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: test
|
||||
spec:
|
||||
rules:
|
||||
- host: foo.bar.com
|
||||
http:
|
||||
paths:
|
||||
- backend:
|
||||
serviceName: s1
|
||||
servicePort: 80
|
||||
- host: bar.foo.com
|
||||
http:
|
||||
paths:
|
||||
- backend:
|
||||
serviceName: s2
|
||||
servicePort: 80
|
||||
```
|
||||
|
||||
__Default Backends__: An Ingress with no rules, like the one shown in the previous section, sends all traffic to a single default backend. You can use the same technique to tell a loadbalancer where to find your website's 404 page, by specifying a set of rules *and* a default backend. Traffic is routed to your default backend if none of the Hosts in your Ingress match the Host in the request header, and/or none of the paths match the URL of the request.
|
||||
|
||||
### TLS
|
||||
|
||||
You can secure an Ingress by specifying a [secret](/docs/user-guide/secrets) that contains a TLS private key and certificate. Currently the Ingress only supports a single TLS port, 443, and assumes TLS termination. If the TLS configuration section in an Ingress specifies different hosts, they will be multiplexed on the same port according to the hostname specified through the SNI TLS extension (provided the Ingress controller supports SNI). The TLS secret must contain keys named `tls.crt` and `tls.key` that contain the certificate and private key to use for TLS, e.g.:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
data:
|
||||
tls.crt: base64 encoded cert
|
||||
tls.key: base64 encoded key
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: testsecret
|
||||
namespace: default
|
||||
type: Opaque
|
||||
```
|
||||
|
||||
Referencing this secret in an Ingress will tell the Ingress controller to secure the channel from the client to the loadbalancer using TLS:
|
||||
|
||||
```yaml
|
||||
apiVersion: extensions/v1beta1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: no-rules-map
|
||||
spec:
|
||||
tls:
|
||||
- secretName: testsecret
|
||||
backend:
|
||||
serviceName: s1
|
||||
servicePort: 80
|
||||
```
|
||||
|
||||
Note that there is a gap between TLS features supported by various Ingress controllers. Please refer to documentation on [nginx](https://git.k8s.io/ingress/controllers/nginx/README.md#https), [GCE](https://git.k8s.io/ingress/controllers/gce/README.md#tls), or any other platform specific Ingress controller to understand how TLS works in your environment.
|
||||
|
||||
### Loadbalancing
|
||||
|
||||
An Ingress controller is bootstrapped with some loadbalancing policy settings that it applies to all Ingress, such as the loadbalancing algorithm, backend weight scheme etc. More advanced loadbalancing concepts (e.g.: persistent sessions, dynamic weights) are not yet exposed through the Ingress. You can still get these features through the [service loadbalancer](https://git.k8s.io/contrib/service-loadbalancer). With time, we plan to distill loadbalancing patterns that are applicable cross platform into the Ingress resource.
|
||||
|
||||
It's also worth noting that even though health checks are not exposed directly through the Ingress, there exist parallel concepts in Kubernetes such as [readiness probes](/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/) which allow you to achieve the same end result. Please review the controller specific docs to see how they handle health checks ([nginx](https://git.k8s.io/ingress/controllers/nginx/README.md), [GCE](https://git.k8s.io/ingress/controllers/gce/README.md#health-checks)).
|
||||
|
||||
## Updating an Ingress
|
||||
|
||||
Say you'd like to add a new Host to an existing Ingress, you can update it by editing the resource:
|
||||
|
||||
```shell
|
||||
$ kubectl get ing
|
||||
NAME RULE BACKEND ADDRESS
|
||||
test - 178.91.123.132
|
||||
foo.bar.com
|
||||
/foo s1:80
|
||||
$ kubectl edit ing test
|
||||
```
|
||||
|
||||
This should pop up an editor with the existing yaml, modify it to include the new Host.
|
||||
|
||||
```yaml
|
||||
spec:
|
||||
rules:
|
||||
- host: foo.bar.com
|
||||
http:
|
||||
paths:
|
||||
- backend:
|
||||
serviceName: s1
|
||||
servicePort: 80
|
||||
path: /foo
|
||||
- host: bar.baz.com
|
||||
http:
|
||||
paths:
|
||||
- backend:
|
||||
serviceName: s2
|
||||
servicePort: 80
|
||||
path: /foo
|
||||
..
|
||||
```
|
||||
|
||||
saving it will update the resource in the API server, which should tell the Ingress controller to reconfigure the loadbalancer.
|
||||
|
||||
```shell
|
||||
$ kubectl get ing
|
||||
NAME RULE BACKEND ADDRESS
|
||||
test - 178.91.123.132
|
||||
foo.bar.com
|
||||
/foo s1:80
|
||||
bar.baz.com
|
||||
/foo s2:80
|
||||
```
|
||||
|
||||
You can achieve the same by invoking `kubectl replace -f` on a modified Ingress yaml file.
|
||||
|
||||
## Failing across availability zones
|
||||
|
||||
Techniques for spreading traffic across failure domains differs between cloud providers. Please check the documentation of the relevant Ingress controller for details. Please refer to the federation [doc](/docs/concepts/cluster-administration/federation/) for details on deploying Ingress in a federated cluster.
|
||||
|
||||
## Future Work
|
||||
|
||||
* Various modes of HTTPS/TLS support (e.g.: SNI, re-encryption)
|
||||
* Requesting an IP or Hostname via claims
|
||||
* Combining L4 and L7 Ingress
|
||||
* More Ingress controllers
|
||||
|
||||
Please track the [L7 and Ingress proposal](https://github.com/kubernetes/kubernetes/pull/12827) for more details on the evolution of the resource, and the [Ingress repository](https://github.com/kubernetes/ingress/tree/master) for more details on the evolution of various Ingress controllers.
|
||||
|
||||
## Alternatives
|
||||
|
||||
You can expose a Service in multiple ways that don't directly involve the Ingress resource:
|
||||
|
||||
* Use [Service.Type=LoadBalancer](/docs/user-guide/services/#type-loadbalancer)
|
||||
* Use [Service.Type=NodePort](/docs/user-guide/services/#type-nodeport)
|
||||
* Use a [Port Proxy](https://git.k8s.io/contrib/for-demos/proxy-to-service)
|
||||
* Deploy the [Service loadbalancer](https://git.k8s.io/contrib/service-loadbalancer). This allows you to share a single IP among multiple Services and achieve more advanced loadbalancing through Service Annotations.
|
||||
@@ -1,234 +0,0 @@
|
||||
---
|
||||
approvers:
|
||||
- enisoc
|
||||
- erictune
|
||||
- foxish
|
||||
- janetkuo
|
||||
- kow3ns
|
||||
- smarterclayton
|
||||
title: StatefulSets
|
||||
---
|
||||
|
||||
{% capture overview %}
|
||||
**StatefulSet is the workload API object used to manage stateful applications.
|
||||
StatefulSets are beta in 1.8.**
|
||||
|
||||
{% include templates/glossary/snippet.md term="statefulset" length="long" %}
|
||||
{% endcapture %}
|
||||
|
||||
{% capture body %}
|
||||
|
||||
## Using StatefulSets
|
||||
|
||||
StatefulSets are valuable for applications that require one or more of the
|
||||
following.
|
||||
|
||||
* Stable, unique network identifiers.
|
||||
* Stable, persistent storage.
|
||||
* Ordered, graceful deployment and scaling.
|
||||
* Ordered, graceful deletion and termination.
|
||||
* Ordered, automated rolling updates.
|
||||
|
||||
In the above, stable is synonymous with persistence across Pod (re)scheduling.
|
||||
If an application doesn't require any stable identifiers or ordered deployment,
|
||||
deletion, or scaling, you should deploy your application with a controller that
|
||||
provides a set of stateless replicas. Controllers such as
|
||||
[Deployment](/docs/concepts/workloads/controllers/deployment/) or
|
||||
[ReplicaSet](/docs/concepts/workloads/controllers/replicaset/) may be better suited to your stateless needs.
|
||||
|
||||
## Limitations
|
||||
|
||||
* StatefulSet is a beta resource, not available in any Kubernetes release prior to 1.5.
|
||||
* As with all alpha/beta resources, you can disable StatefulSet through the `--runtime-config` option passed to the apiserver.
|
||||
* The storage for a given Pod must either be provisioned by a [PersistentVolume Provisioner](https://github.com/kubernetes/examples/tree/{{page.githubbranch}}/staging/persistent-volume-provisioning/README.md) based on the requested `storage class`, or pre-provisioned by an admin.
|
||||
* Deleting and/or scaling a StatefulSet down will *not* delete the volumes associated with the StatefulSet. This is done to ensure data safety, which is generally more valuable than an automatic purge of all related StatefulSet resources.
|
||||
* StatefulSets currently require a [Headless Service](/docs/concepts/services-networking/service/#headless-services) to be responsible for the network identity of the Pods. You are responsible for creating this Service.
|
||||
|
||||
## Components
|
||||
The example below demonstrates the components of a StatefulSet.
|
||||
|
||||
* A Headless Service, named nginx, is used to control the network domain.
|
||||
* The StatefulSet, named web, has a Spec that indicates that 3 replicas of the nginx container will be launched in unique Pods.
|
||||
* The volumeClaimTemplates will provide stable storage using [PersistentVolumes](/docs/concepts/storage/volumes/) provisioned by a PersistentVolume Provisioner.
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: nginx
|
||||
labels:
|
||||
app: nginx
|
||||
spec:
|
||||
ports:
|
||||
- port: 80
|
||||
name: web
|
||||
clusterIP: None
|
||||
selector:
|
||||
app: nginx
|
||||
---
|
||||
apiVersion: apps/v1beta2
|
||||
kind: StatefulSet
|
||||
metadata:
|
||||
name: web
|
||||
spec:
|
||||
selector:
|
||||
matchLabels:
|
||||
app: nginx # has to match .spec.template.metadata.labels
|
||||
serviceName: "nginx"
|
||||
replicas: 3 # by default is 1
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: nginx # has to match .spec.selector.matchLabels
|
||||
spec:
|
||||
terminationGracePeriodSeconds: 10
|
||||
containers:
|
||||
- name: nginx
|
||||
image: gcr.io/google_containers/nginx-slim:0.8
|
||||
ports:
|
||||
- containerPort: 80
|
||||
name: web
|
||||
volumeMounts:
|
||||
- name: www
|
||||
mountPath: /usr/share/nginx/html
|
||||
volumeClaimTemplates:
|
||||
- metadata:
|
||||
name: www
|
||||
spec:
|
||||
accessModes: [ "ReadWriteOnce" ]
|
||||
storageClassName: my-storage-class
|
||||
resources:
|
||||
requests:
|
||||
storage: 1Gi
|
||||
```
|
||||
|
||||
## Pod Selector
|
||||
You must set the `spec.selector` field of a StatefulSet to match the labels of its `.spec.template.metadata.labels`. Prior to Kubernetes 1.8, the `spec.selector` field was defaulted when omitted. In 1.8 and later versions, failing to specify a matching Pod Selector will result in a validation error during StatefulSet creation.
|
||||
|
||||
## Pod Identity
|
||||
StatefulSet Pods have a unique identity that is comprised of an ordinal, a
|
||||
stable network identity, and stable storage. The identity sticks to the Pod,
|
||||
regardless of which node it's (re)scheduled on.
|
||||
|
||||
### Ordinal Index
|
||||
|
||||
For a StatefulSet with N replicas, each Pod in the StatefulSet will be
|
||||
assigned an integer ordinal, in the range [0,N), that is unique over the Set.
|
||||
|
||||
### Stable Network ID
|
||||
|
||||
Each Pod in a StatefulSet derives its hostname from the name of the StatefulSet
|
||||
and the ordinal of the Pod. The pattern for the constructed hostname
|
||||
is `$(statefulset name)-$(ordinal)`. The example above will create three Pods
|
||||
named `web-0,web-1,web-2`.
|
||||
A StatefulSet can use a [Headless Service](/docs/concepts/services-networking/service/#headless-services)
|
||||
to control the domain of its Pods. The domain managed by this Service takes the form:
|
||||
`$(service name).$(namespace).svc.cluster.local`, where "cluster.local"
|
||||
is the [cluster domain](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/dns/README.md).
|
||||
As each Pod is created, it gets a matching DNS subdomain, taking the form:
|
||||
`$(podname).$(governing service domain)`, where the governing service is defined
|
||||
by the `serviceName` field on the StatefulSet.
|
||||
|
||||
Here are some examples of choices for Cluster Domain, Service name,
|
||||
StatefulSet name, and how that affects the DNS names for the StatefulSet's Pods.
|
||||
|
||||
Cluster Domain | Service (ns/name) | StatefulSet (ns/name) | StatefulSet Domain | Pod DNS | Pod Hostname |
|
||||
-------------- | ----------------- | ----------------- | -------------- | ------- | ------------ |
|
||||
cluster.local | default/nginx | default/web | nginx.default.svc.cluster.local | web-{0..N-1}.nginx.default.svc.cluster.local | web-{0..N-1} |
|
||||
cluster.local | foo/nginx | foo/web | nginx.foo.svc.cluster.local | web-{0..N-1}.nginx.foo.svc.cluster.local | web-{0..N-1} |
|
||||
kube.local | foo/nginx | foo/web | nginx.foo.svc.kube.local | web-{0..N-1}.nginx.foo.svc.kube.local | web-{0..N-1} |
|
||||
|
||||
Note that Cluster Domain will be set to `cluster.local` unless
|
||||
[otherwise configured](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/dns/README.md).
|
||||
|
||||
### Stable Storage
|
||||
|
||||
Kubernetes creates one [PersistentVolume](/docs/concepts/storage/volumes/) for each
|
||||
VolumeClaimTemplate. In the nginx example above, each Pod will receive a single PersistentVolume
|
||||
with a StorageClass of `my-storage-class` and 1 Gib of provisioned storage. If no StorageClass
|
||||
is specified, then the default StorageClass will be used. When a Pod is (re)scheduled
|
||||
onto a node, its `volumeMounts` mount the PersistentVolumes associated with its
|
||||
PersistentVolume Claims. Note that, the PersistentVolumes associated with the
|
||||
Pods' PersistentVolume Claims are not deleted when the Pods, or StatefulSet are deleted.
|
||||
This must be done manually.
|
||||
|
||||
## Deployment and Scaling Guarantees
|
||||
|
||||
* For a StatefulSet with N replicas, when Pods are being deployed, they are created sequentially, in order from {0..N-1}.
|
||||
* When Pods are being deleted, they are terminated in reverse order, from {N-1..0}.
|
||||
* Before a scaling operation is applied to a Pod, all of its predecessors must be Running and Ready.
|
||||
* Before a Pod is terminated, all of its successors must be completely shutdown.
|
||||
|
||||
The StatefulSet should not specify a `pod.Spec.TerminationGracePeriodSeconds` of 0. This practice is unsafe and strongly discouraged. For further explanation, please refer to [force deleting StatefulSet Pods](/docs/tasks/run-application/force-delete-stateful-set-pod/).
|
||||
|
||||
When the nginx example above is created, three Pods will be deployed in the order
|
||||
web-0, web-1, web-2. web-1 will not be deployed before web-0 is
|
||||
[Running and Ready](/docs/user-guide/pod-states), and web-2 will not be deployed until
|
||||
web-1 is Running and Ready. If web-0 should fail, after web-1 is Running and Ready, but before
|
||||
web-2 is launched, web-2 will not be launched until web-0 is successfully relaunched and
|
||||
becomes Running and Ready.
|
||||
|
||||
If a user were to scale the deployed example by patching the StatefulSet such that
|
||||
`replicas=1`, web-2 would be terminated first. web-1 would not be terminated until web-2
|
||||
is fully shutdown and deleted. If web-0 were to fail after web-2 has been terminated and
|
||||
is completely shutdown, but prior to web-1's termination, web-1 would not be terminated
|
||||
until web-0 is Running and Ready.
|
||||
|
||||
### Pod Management Policies
|
||||
In Kubernetes 1.7 and later, StatefulSet allows you to relax its ordering guarantees while
|
||||
preserving its uniqueness and identity guarantees via its `.spec.podManagementPolicy` field.
|
||||
|
||||
#### OrderedReady Pod Management
|
||||
|
||||
`OrderedReady` pod management is the default for StatefulSets. It implements the behavior
|
||||
described [above](#deployment-and-scaling-guarantees).
|
||||
|
||||
#### Parallel Pod Management
|
||||
|
||||
`Parallel` pod management tells the StatefulSet controller to launch or
|
||||
terminate all Pods in parallel, and to not wait for Pods to become Running
|
||||
and Ready or completely terminated prior to launching or terminating another
|
||||
Pod.
|
||||
|
||||
## Update Strategies
|
||||
|
||||
In Kubernetes 1.7 and later, StatefulSet's `.spec.updateStrategy` field allows you to configure
|
||||
and disable automated rolling updates for containers, labels, resource request/limits, and
|
||||
annotations for the Pods in a StatefulSet.
|
||||
|
||||
### On Delete
|
||||
|
||||
The `OnDelete` update strategy implements the legacy (1.6 and prior) behavior. It is the default
|
||||
strategy when `spec.updateStrategy` is left unspecified. When a StatefulSet's
|
||||
`.spec.updateStrategy.type` is set to `OnDelete`, the StatefulSet controller will not automatically
|
||||
update the Pods in a StatefulSet. Users must manually delete Pods to cause the controller to
|
||||
create new Pods that reflect modifications made to a StatefulSet's `.spec.template`.
|
||||
|
||||
### Rolling Updates
|
||||
|
||||
The `RollingUpdate` update strategy implements automated, rolling update for the Pods in a
|
||||
StatefulSet. When a StatefulSet's `.spec.updateStrategy.type` is set to `RollingUpdate`, the
|
||||
StatefulSet controller will delete and recreate each Pod in the StatefulSet. It will proceed
|
||||
in the same order as Pod termination (from the largest ordinal to the smallest), updating
|
||||
each Pod one at a time. It will wait until an updated Pod is Running and Ready prior to
|
||||
updating its predecessor.
|
||||
|
||||
#### Partitions
|
||||
|
||||
The `RollingUpdate` update strategy can be partitioned, by specifying a
|
||||
`.spec.updateStrategy.rollingUpdate.partition`. If a partition is specified, all Pods with an
|
||||
ordinal that is greater than or equal to the partition will be updated when the StatefulSet's
|
||||
`.spec.template` is updated. All Pods with an ordinal that is less than the partition will not
|
||||
be updated, and, even if they are deleted, they will be recreated at the previous version. If a
|
||||
StatefulSet's `.spec.updateStrategy.rollingUpdate.partition` is greater than its `.spec.replicas`,
|
||||
updates to its `.spec.template` will not be propagated to its Pods.
|
||||
In most cases you will not need to use a partition, but they are useful if you want to stage an
|
||||
update, roll out a canary, or perform a phased roll out.
|
||||
|
||||
{% endcapture %}
|
||||
{% capture whatsnext %}
|
||||
|
||||
* Follow an example of [deploying a stateful application](/docs/tutorials/stateful-application/basic-stateful-set).
|
||||
|
||||
{% endcapture %}
|
||||
{% include templates/concept.md %}
|
||||
Reference in New Issue
Block a user