Convert site to Hugo (#8316)
This commit converts content and layout to use Hugo.
This commit is contained in:
Bjørn Erik Pedersen
committed by
k8s-ci-robot
k8s-ci-robot
parent
7745f0e0c5
commit
7f3b633aa0
+5
@@ -0,0 +1,5 @@
|
||||
---
|
||||
title: "Run Applications"
|
||||
weight: 50
|
||||
---
|
||||
|
||||
@@ -0,0 +1,233 @@
|
||||
---
|
||||
title: Specifying a Disruption Budget for your Application
|
||||
content_template: templates/task
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
This page shows how to limit the number of concurrent disruptions
|
||||
that your application experiences, allowing for higher availability
|
||||
while permitting the cluster administrator to manage the clusters
|
||||
nodes.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture prerequisites %}}
|
||||
* You are the owner of an application running on a Kubernetes cluster that requires
|
||||
high availability.
|
||||
* You should know how to deploy [Replicated Stateless Applications](/docs/tasks/run-application/run-stateless-application-deployment/)
|
||||
and/or [Replicated Stateful Applications](/docs/tasks/run-application/run-replicated-stateful-application/).
|
||||
* You should have read about [Pod Disruptions](/docs/concepts/workloads/pods/disruptions/).
|
||||
* You should confirm with your cluster owner or service provider that they respect
|
||||
Pod Disruption Budgets.
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture steps %}}
|
||||
|
||||
## Protecting an Application with a PodDisruptionBudget
|
||||
|
||||
1. Identify what application you want to protect with a PodDisruptionBudget (PDB).
|
||||
1. Think about how your application reacts to disruptions.
|
||||
1. Create a PDB definition as a YAML file.
|
||||
1. Create the PDB object from the YAML file.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture discussion %}}
|
||||
|
||||
## Identify an Application to Protect
|
||||
|
||||
The most common use case when you want to protect an application
|
||||
specified by one of the built-in Kubernetes controllers:
|
||||
|
||||
- Deployment
|
||||
- ReplicationController
|
||||
- ReplicaSet
|
||||
- StatefulSet
|
||||
|
||||
In this case, make a note of the controller's `.spec.selector`; the same
|
||||
selector goes into the PDBs `.spec.selector`.
|
||||
|
||||
You can also use PDBs with pods which are not controlled by one of the above
|
||||
controllers, or arbitrary groups of pods, but there are some restrictions,
|
||||
described in [Arbitrary Controllers and Selectors](#arbitrary-controllers-and-selectors).
|
||||
|
||||
|
||||
## Think about how your application reacts to disruptions
|
||||
|
||||
Decide how many instances can be down at the same time for a short period
|
||||
due to a voluntary disruption.
|
||||
|
||||
- Stateless frontends:
|
||||
- Concern: don't reduce serving capacity by more than 10%.
|
||||
- Solution: use PDB with minAvailable 90% for example.
|
||||
- Single-instance Stateful Application:
|
||||
- Concern: do not terminate this application without talking to me.
|
||||
- Possible Solution 1: Do not use a PDB and tolerate occasional downtime.
|
||||
- Possible Solution 2: Set PDB with maxUnavailable=0. Have an understanding
|
||||
(outside of Kubernetes) that the cluster operator needs to consult you before
|
||||
termination. When the cluster operator contacts you, prepare for downtime,
|
||||
and then delete the PDB to indicate readiness for disruption. Recreate afterwards.
|
||||
- Multiple-instance Stateful application such as Consul, ZooKeeper, or etcd:
|
||||
- Concern: Do not reduce number of instances below quorum, otherwise writes fail.
|
||||
- Possible Solution 1: set maxUnavailable to 1 (works with varying scale of application).
|
||||
- Possible Solution 2: set minAvailable to quorum-size (e.g. 3 when scale is 5). (Allows more disruptions at once).
|
||||
- Restartable Batch Job:
|
||||
- Concern: Job needs to complete in case of voluntary disruption.
|
||||
- Possible solution: Do not create a PDB. The Job controller will create a replacement pod.
|
||||
|
||||
## Specifying a PodDisruptionBudget
|
||||
|
||||
A `PodDisruptionBudget` has three fields:
|
||||
|
||||
* A label selector `.spec.selector` to specify the set of
|
||||
pods to which it applies. This field is required.
|
||||
* `.spec.minAvailable` which is a description of the number of pods from that
|
||||
set that must still be available after the eviction, even in the absence
|
||||
of the evicted pod. `minAvailable` can be either an absolute number or a percentage.
|
||||
* `.spec.maxUnavailable` (available in Kubernetes 1.7 and higher) which is a description
|
||||
of the number of pods from that set that can be unavailable after the eviction.
|
||||
It can be either an absolute number or a percentage.
|
||||
|
||||
{{< note >}}
|
||||
**Note:** For versions 1.8 and earlier: When creating a `PodDisruptionBudget`
|
||||
object using the `kubectl` command line tool, the `minAvailable` field has a
|
||||
default value of 1 if neither `minAvailable` nor `maxUnavailable` is specified.
|
||||
{{< /note >}}
|
||||
|
||||
You can specify only one of `maxUnavailable` and `minAvailable` in a single `PodDisruptionBudget`.
|
||||
`maxUnavailable` can only be used to control the eviction of pods
|
||||
that have an associated controller managing them. In the examples below, "desired replicas"
|
||||
is the `scale` of the controller managing the pods being selected by the
|
||||
`PodDisruptionBudget`.
|
||||
|
||||
Example 1: With a `minAvailable` of 5, evictions are be allowed as long as they leave behind
|
||||
5 or more healthy pods among those selected by the PodDisruptionBudget's `selector`.
|
||||
|
||||
Example 2: With a `minAvailable` of 30%, evictions are allowed as long as at least 30%
|
||||
of the number of desired replicas are healthy.
|
||||
|
||||
Example 3: With a `maxUnavailable` of 5, evictions are allowed as long as there are at most 5
|
||||
unhealthy replicas among the total number of desired replicas.
|
||||
|
||||
Example 4: With a `maxUnavailable` of 30%, evictions are allowed as long as no more than 30%
|
||||
of the desired replicas are unhealthy.
|
||||
|
||||
In typical usage, a single budget would be used for a collection of pods managed by
|
||||
a controller—for example, the pods in a single ReplicaSet or StatefulSet.
|
||||
|
||||
**Note:** A disruption budget does not truly guarantee that the specified
|
||||
number/percentage of pods will always be up. For example, a node that hosts a
|
||||
pod from the collection may fail when the collection is at the minimum size
|
||||
specified in the budget, thus bringing the number of available pods from the
|
||||
collection below the specified size. The budget can only protect against
|
||||
voluntary evictions, not all causes of unavailability.
|
||||
|
||||
A `maxUnavailable` of 0% (or 0) or a `minAvailable` of 100% (or equal to the
|
||||
number of replicas) may block node drains entirely. This is permitted as per the
|
||||
semantics of `PodDisruptionBudget`.
|
||||
|
||||
You can find examples of pod disruption budgets defined below. They match pods with the label
|
||||
`app: zookeeper`.
|
||||
|
||||
Example PDB Using minAvailable:
|
||||
|
||||
```yaml
|
||||
apiVersion: policy/v1beta1
|
||||
kind: PodDisruptionBudget
|
||||
metadata:
|
||||
name: zk-pdb
|
||||
spec:
|
||||
minAvailable: 2
|
||||
selector:
|
||||
matchLabels:
|
||||
app: zookeeper
|
||||
```
|
||||
|
||||
Example PDB Using maxUnavailable (Kubernetes 1.7 or higher):
|
||||
|
||||
```yaml
|
||||
apiVersion: policy/v1beta1
|
||||
kind: PodDisruptionBudget
|
||||
metadata:
|
||||
name: zk-pdb
|
||||
spec:
|
||||
maxUnavailable: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: zookeeper
|
||||
```
|
||||
|
||||
For example, if the above `zk-pdb` object selects the pods of a StatefulSet of size 3, both
|
||||
specifications have the exact same meaning. The use of `maxUnavailable` is recommended as it
|
||||
automatically responds to changes in the number of replicas of the corresponding controller.
|
||||
|
||||
# Create the PDB object
|
||||
|
||||
You can create the PDB object with a command like `kubectl create -f mypdb.yaml`.
|
||||
|
||||
You cannot update PDB objects. They must be deleted and re-created.
|
||||
|
||||
# Check the status of the PDB
|
||||
|
||||
Use kubectl to check that your PDB is created.
|
||||
|
||||
Assuming you don't actually have pods matching `app: zookeeper` in your namespace,
|
||||
then you'll see something like this:
|
||||
|
||||
```shell
|
||||
$ kubectl get poddisruptionbudgets
|
||||
NAME MIN-AVAILABLE ALLOWED-DISRUPTIONS AGE
|
||||
zk-pdb 2 0 7s
|
||||
```
|
||||
|
||||
If there are matching pods (say, 3), then you would see something like this:
|
||||
|
||||
```shell
|
||||
$ kubectl get poddisruptionbudgets
|
||||
NAME MIN-AVAILABLE ALLOWED-DISRUPTIONS AGE
|
||||
zk-pdb 2 1 7s
|
||||
```
|
||||
|
||||
The non-zero value for `ALLOWED-DISRUPTIONS` means that the disruption controller has seen the pods,
|
||||
counted the matching pods, and update the status of the PDB.
|
||||
|
||||
You can get more information about the status of a PDB with this command:
|
||||
|
||||
```shell
|
||||
$ kubectl get poddisruptionbudgets zk-pdb -o yaml
|
||||
apiVersion: policy/v1beta1
|
||||
kind: PodDisruptionBudget
|
||||
metadata:
|
||||
creationTimestamp: 2017-08-28T02:38:26Z
|
||||
generation: 1
|
||||
name: zk-pdb
|
||||
...
|
||||
status:
|
||||
currentHealthy: 3
|
||||
desiredHealthy: 3
|
||||
disruptedPods: null
|
||||
disruptionsAllowed: 1
|
||||
expectedPods: 3
|
||||
observedGeneration: 1
|
||||
```
|
||||
|
||||
# Arbitrary Controllers and Selectors
|
||||
|
||||
You can skip this section if you only use PDBs with the built-in
|
||||
application controllers (Deployment, ReplicationController, ReplicaSet, and StatefulSet),
|
||||
with the PDB selector matching the controller's selector.
|
||||
|
||||
You can use a PDB with pods controlled by another type of controller, by an
|
||||
"operator", or bare pods, but with these restrictions:
|
||||
|
||||
- only `.spec.minAvailable` can be used, not `.spec.maxUnavailable`.
|
||||
- only an integer value can be used with `.spec.minAvailable`, not a percentage.
|
||||
|
||||
You can use a selector which selects a subset or superset of the pods belonging to a built-in
|
||||
controller. However, when there are multiple PDBs in a namespace, you must be careful not
|
||||
to create PDBs whose selectors overlap.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
@@ -0,0 +1,89 @@
|
||||
---
|
||||
reviewers:
|
||||
- bprashanth
|
||||
- erictune
|
||||
- foxish
|
||||
- janetkuo
|
||||
- smarterclayton
|
||||
title: Delete a StatefulSet
|
||||
content_template: templates/task
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
This task shows you how to delete a StatefulSet.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture prerequisites %}}
|
||||
|
||||
* This task assumes you have an application running on your cluster represented by a StatefulSet.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture steps %}}
|
||||
|
||||
## Deleting a StatefulSet
|
||||
|
||||
You can delete a StatefulSet in the same way you delete other resources in Kubernetes: use the `kubectl delete` command, and specify the StatefulSet either by file or by name.
|
||||
|
||||
```shell
|
||||
kubectl delete -f <file.yaml>
|
||||
```
|
||||
|
||||
```shell
|
||||
kubectl delete statefulsets <statefulset-name>
|
||||
```
|
||||
|
||||
You may need to delete the associated headless service separately after the StatefulSet itself is deleted.
|
||||
|
||||
```shell
|
||||
kubectl delete service <service-name>
|
||||
```
|
||||
|
||||
Deleting a StatefulSet through kubectl will scale it down to 0, thereby deleting all pods that are a part of it.
|
||||
If you want to delete just the StatefulSet and not the pods, use `--cascade=false`.
|
||||
|
||||
```shell
|
||||
kubectl delete -f <file.yaml> --cascade=false
|
||||
```
|
||||
|
||||
By passing `--cascade=false` to `kubectl delete`, the Pods managed by the StatefulSet are left behind even after the StatefulSet object itself is deleted. If the pods have a label `app=myapp`, you can then delete them as follows:
|
||||
|
||||
```shell
|
||||
kubectl delete pods -l app=myapp
|
||||
```
|
||||
|
||||
### Persistent Volumes
|
||||
|
||||
Deleting the Pods in a StatefulSet will not delete the associated volumes. This is to ensure that you have the chance to copy data off the volume before deleting it. Deleting the PVC after the pods have left the [terminating state](/docs/concepts/workloads/pods/pod/#termination-of-pods) might trigger deletion of the backing Persistent Volumes depending on the storage class and reclaim policy. You should never assume ability to access a volume after claim deletion.
|
||||
|
||||
**Note: Use caution when deleting a PVC, as it may lead to data loss.**
|
||||
|
||||
### Complete deletion of a StatefulSet
|
||||
|
||||
To simply delete everything in a StatefulSet, including the associated pods, you can run a series of commands similar to the following:
|
||||
|
||||
```shell
|
||||
grace=$(kubectl get pods <stateful-set-pod> --template '{{.spec.terminationGracePeriodSeconds}}')
|
||||
kubectl delete statefulset -l app=myapp
|
||||
sleep $grace
|
||||
kubectl delete pvc -l app=myapp
|
||||
|
||||
```
|
||||
|
||||
In the example above, the Pods have the label `app=myapp`; substitute your own label as appropriate.
|
||||
|
||||
### Force deletion of StatefulSet pods
|
||||
|
||||
If you find that some pods in your StatefulSet are stuck in the 'Terminating' or 'Unknown' states for an extended period of time, you may need to manually intervene to forcefully delete the pods from the apiserver. This is a potentially dangerous task. Refer to [Deleting StatefulSet Pods](/docs/tasks/manage-stateful-set/delete-pods/) for details.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture whatsnext %}}
|
||||
|
||||
Learn more about [force deleting StatefulSet Pods](/docs/tasks/run-application/force-delete-stateful-set-pod/).
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: patch-demo
|
||||
spec:
|
||||
replicas: 2
|
||||
selector:
|
||||
matchLabels:
|
||||
app: nginx
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: nginx
|
||||
spec:
|
||||
containers:
|
||||
- name: patch-demo-ctr
|
||||
image: nginx
|
||||
tolerations:
|
||||
- effect: NoSchedule
|
||||
key: dedicated
|
||||
value: test-team
|
||||
@@ -0,0 +1,19 @@
|
||||
apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: nginx-deployment
|
||||
spec:
|
||||
selector:
|
||||
matchLabels:
|
||||
app: nginx
|
||||
replicas: 4 # Update the replicas from 2 to 4
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: nginx
|
||||
spec:
|
||||
containers:
|
||||
- name: nginx
|
||||
image: nginx:1.8
|
||||
ports:
|
||||
- containerPort: 80
|
||||
@@ -0,0 +1,19 @@
|
||||
apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: nginx-deployment
|
||||
spec:
|
||||
selector:
|
||||
matchLabels:
|
||||
app: nginx
|
||||
replicas: 2
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: nginx
|
||||
spec:
|
||||
containers:
|
||||
- name: nginx
|
||||
image: nginx:1.8 # Update the version of nginx from 1.7.9 to 1.8
|
||||
ports:
|
||||
- containerPort: 80
|
||||
@@ -0,0 +1,21 @@
|
||||
apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: nginx-deployment
|
||||
spec:
|
||||
selector:
|
||||
matchLabels:
|
||||
app: nginx
|
||||
replicas: 2 # tells deployment to run 2 pods matching the template
|
||||
template: # create pods using pod definition in this template
|
||||
metadata:
|
||||
# unlike pod-nginx.yaml, the name is not included in the meta data as a unique name is
|
||||
# generated from the deployment name
|
||||
labels:
|
||||
app: nginx
|
||||
spec:
|
||||
containers:
|
||||
- name: nginx
|
||||
image: nginx:1.7.9
|
||||
ports:
|
||||
- containerPort: 80
|
||||
@@ -0,0 +1,79 @@
|
||||
---
|
||||
reviewers:
|
||||
- bprashanth
|
||||
- erictune
|
||||
- foxish
|
||||
- smarterclayton
|
||||
title: Force Delete StatefulSet Pods
|
||||
content_template: templates/task
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
This page shows how to delete Pods which are part of a stateful set, and explains the considerations to keep in mind when doing so.
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture prerequisites %}}
|
||||
|
||||
* This is a fairly advanced task and has the potential to violate some of the properties inherent to StatefulSet.
|
||||
* Before proceeding, make yourself familiar with the considerations enumerated below.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture steps %}}
|
||||
|
||||
|
||||
## StatefulSet considerations
|
||||
|
||||
In normal operation of a StatefulSet, there is **never** a need to force delete a StatefulSet Pod. The StatefulSet controller is responsible for creating, scaling and deleting members of the StatefulSet. It tries to ensure that the specified number of Pods from ordinal 0 through N-1 are alive and ready. StatefulSet ensures that, at any time, there is at most one Pod with a given identity running in a cluster. This is referred to as *at most one* semantics provided by a StatefulSet.
|
||||
|
||||
Manual force deletion should be undertaken with caution, as it has the potential to violate the at most one semantics inherent to StatefulSet. StatefulSets may be used to run distributed and clustered applications which have a need for a stable network identity and stable storage. These applications often have configuration which relies on an ensemble of a fixed number of members with fixed identities. Having multiple members with the same identity can be disastrous and may lead to data loss (e.g. split brain scenario in quorum-based systems).
|
||||
|
||||
## Delete Pods
|
||||
|
||||
You can perform a graceful pod deletion with the following command:
|
||||
|
||||
```shell
|
||||
kubectl delete pods <pod>
|
||||
```
|
||||
|
||||
For the above to lead to graceful termination, the Pod **must not** specify a `pod.Spec.TerminationGracePeriodSeconds` of 0. The practice of setting a `pod.Spec.TerminationGracePeriodSeconds` of 0 seconds is unsafe and strongly discouraged for StatefulSet Pods. Graceful deletion is safe and will ensure that the [Pod shuts down gracefully](/docs/user-guide/pods/#termination-of-pods) before the kubelet deletes the name from the apiserver.
|
||||
|
||||
Kubernetes (versions 1.5 or newer) will not delete Pods just because a Node is unreachable. The Pods running on an unreachable Node enter the 'Terminating' or 'Unknown' state after a [timeout](/docs/admin/node/#node-condition). Pods may also enter these states when the user attempts graceful deletion of a Pod on an unreachable Node. The only ways in which a Pod in such a state can be removed from the apiserver are as follows:
|
||||
|
||||
* The Node object is deleted (either by you, or by the [Node Controller](/docs/admin/node)).<br/>
|
||||
* The kubelet on the unresponsive Node starts responding, kills the Pod and removes the entry from the apiserver.<br/>
|
||||
* Force deletion of the Pod by the user.
|
||||
|
||||
The recommended best practice is to use the first or second approach. If a Node is confirmed to be dead (e.g. permanently disconnected from the network, powered down, etc), then delete the Node object. If the Node is suffering from a network partition, then try to resolve this or wait for it to resolve. When the partition heals, the kubelet will complete the deletion of the Pod and free up its name in the apiserver.
|
||||
|
||||
Normally, the system completes the deletion once the Pod is no longer running on a Node, or the Node is deleted by an administrator. You may override this by force deleting the Pod.
|
||||
|
||||
### Force Deletion
|
||||
|
||||
Force deletions **do not** wait for confirmation from the kubelet that the Pod has been terminated. Irrespective of whether a force deletion is successful in killing a Pod, it will immediately free up the name from the apiserver. This would let the StatefulSet controller create a replacement Pod with that same identity; this can lead to the duplication of a still-running Pod, and if said Pod can still communicate with the other members of the StatefulSet, will violate the at most one semantics that StatefulSet is designed to guarantee.
|
||||
|
||||
When you force delete a StatefulSet pod, you are asserting that the Pod in question will never again make contact with other Pods in the StatefulSet and its name can be safely freed up for a replacement to be created.
|
||||
|
||||
If you want to delete a Pod forcibly using kubectl version >= 1.5, do the following:
|
||||
|
||||
```shell
|
||||
kubectl delete pods <pod> --grace-period=0 --force
|
||||
```
|
||||
|
||||
If you're using any version of kubectl <= 1.4, you should omit the `--force` option and use:
|
||||
|
||||
```shell
|
||||
kubectl delete pods <pod> --grace-period=0
|
||||
```
|
||||
|
||||
Always perform force deletion of StatefulSet Pods carefully and with complete knowledge of the risks involved.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture whatsnext %}}
|
||||
|
||||
Learn more about [debugging a StatefulSet](/docs/tasks/manage-stateful-set/debugging-a-statefulset/).
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
@@ -0,0 +1,378 @@
|
||||
---
|
||||
reviewers:
|
||||
- fgrzadkowski
|
||||
- jszczepkowski
|
||||
- justinsb
|
||||
- directxman12
|
||||
title: Horizontal Pod Autoscaler Walkthrough
|
||||
---
|
||||
|
||||
{{< toc >}}
|
||||
|
||||
Horizontal Pod Autoscaler automatically scales the number of pods
|
||||
in a replication controller, deployment or replica set based on observed CPU utilization
|
||||
(or, with beta support, on some other, application-provided metrics).
|
||||
|
||||
This document walks you through an example of enabling Horizontal Pod Autoscaler for the php-apache server. For more information on how Horizontal Pod Autoscaler behaves, see the [Horizontal Pod Autoscaler user guide](/docs/tasks/run-application/horizontal-pod-autoscale/).
|
||||
|
||||
## Prerequisites
|
||||
|
||||
This example requires a running Kubernetes cluster and kubectl, version 1.2 or later.
|
||||
[Heapster](https://github.com/kubernetes/heapster) monitoring needs to be deployed in the cluster
|
||||
as Horizontal Pod Autoscaler uses it to collect metrics
|
||||
(if you followed [getting started on GCE guide](/docs/getting-started-guides/gce.md),
|
||||
heapster monitoring will be turned-on by default).
|
||||
|
||||
To specify multiple resource metrics for a Horizontal Pod Autoscaler, you must have a Kubernetes cluster
|
||||
and kubectl at version 1.6 or later. Furthermore, in order to make use of custom metrics, your cluster
|
||||
must be able to communicate with the API server providing the custom metrics API. Finally, to use metrics
|
||||
not related to any Kubernetes object you must have a Kubernetes cluster at version 1.10 or later, and
|
||||
you must be able to communicate with the API server that provides the external metrics API.
|
||||
See the [Horizontal Pod Autoscaler user guide](/docs/tasks/run-application/horizontal-pod-autoscale/#support-for-custom-metrics) for more details.
|
||||
|
||||
## Step One: Run & expose php-apache server
|
||||
|
||||
To demonstrate Horizontal Pod Autoscaler we will use a custom docker image based on the php-apache image.
|
||||
The Dockerfile has the following content:
|
||||
|
||||
```
|
||||
FROM php:5-apache
|
||||
ADD index.php /var/www/html/index.php
|
||||
RUN chmod a+rx index.php
|
||||
```
|
||||
|
||||
It defines an index.php page which performs some CPU intensive computations:
|
||||
|
||||
```
|
||||
<?php
|
||||
$x = 0.0001;
|
||||
for ($i = 0; $i <= 1000000; $i++) {
|
||||
$x += sqrt($x);
|
||||
}
|
||||
echo "OK!";
|
||||
?>
|
||||
```
|
||||
|
||||
First, we will start a deployment running the image and expose it as a service:
|
||||
|
||||
```shell
|
||||
$ kubectl run php-apache --image=k8s.gcr.io/hpa-example --requests=cpu=200m --expose --port=80
|
||||
service "php-apache" created
|
||||
deployment "php-apache" created
|
||||
```
|
||||
|
||||
## Step Two: Create Horizontal Pod Autoscaler
|
||||
|
||||
Now that the server is running, we will create the autoscaler using
|
||||
[kubectl autoscale](https://github.com/kubernetes/kubernetes/blob/{{< param "githubbranch" >}}/docs/user-guide/kubectl/kubectl_autoscale.md).
|
||||
The following command will create a Horizontal Pod Autoscaler that maintains between 1 and 10 replicas of the Pods
|
||||
controlled by the php-apache deployment we created in the first step of these instructions.
|
||||
Roughly speaking, HPA will increase and decrease the number of replicas
|
||||
(via the deployment) to maintain an average CPU utilization across all Pods of 50%
|
||||
(since each pod requests 200 milli-cores by [kubectl run](https://github.com/kubernetes/kubernetes/blob/{{< param "githubbranch" >}}/docs/user-guide/kubectl/kubectl_run.md), this means average CPU usage of 100 milli-cores).
|
||||
See [here](https://git.k8s.io/community/contributors/design-proposals/autoscaling/horizontal-pod-autoscaler.md#autoscaling-algorithm) for more details on the algorithm.
|
||||
|
||||
```shell
|
||||
$ kubectl autoscale deployment php-apache --cpu-percent=50 --min=1 --max=10
|
||||
deployment "php-apache" autoscaled
|
||||
```
|
||||
|
||||
We may check the current status of autoscaler by running:
|
||||
|
||||
```shell
|
||||
$ kubectl get hpa
|
||||
NAME REFERENCE TARGET MINPODS MAXPODS REPLICAS AGE
|
||||
php-apache Deployment/php-apache/scale 0% / 50% 1 10 1 18s
|
||||
|
||||
```
|
||||
|
||||
Please note that the current CPU consumption is 0% as we are not sending any requests to the server
|
||||
(the ``CURRENT`` column shows the average across all the pods controlled by the corresponding deployment).
|
||||
|
||||
## Step Three: Increase load
|
||||
|
||||
Now, we will see how the autoscaler reacts to increased load.
|
||||
We will start a container, and send an infinite loop of queries to the php-apache service (please run it in a different terminal):
|
||||
|
||||
```shell
|
||||
$ kubectl run -i --tty load-generator --image=busybox /bin/sh
|
||||
|
||||
Hit enter for command prompt
|
||||
|
||||
$ while true; do wget -q -O- http://php-apache.default.svc.cluster.local; done
|
||||
```
|
||||
|
||||
Within a minute or so, we should see the higher CPU load by executing:
|
||||
|
||||
```shell
|
||||
$ kubectl get hpa
|
||||
NAME REFERENCE TARGET CURRENT MINPODS MAXPODS REPLICAS AGE
|
||||
php-apache Deployment/php-apache/scale 305% / 50% 305% 1 10 1 3m
|
||||
|
||||
```
|
||||
|
||||
Here, CPU consumption has increased to 305% of the request.
|
||||
As a result, the deployment was resized to 7 replicas:
|
||||
|
||||
```shell
|
||||
$ kubectl get deployment php-apache
|
||||
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
|
||||
php-apache 7 7 7 7 19m
|
||||
```
|
||||
|
||||
**Note** Sometimes it may take a few minutes to stabilize the number of replicas.
|
||||
Since the amount of load is not controlled in any way it may happen that the final number of replicas will
|
||||
differ from this example.
|
||||
|
||||
## Step Four: Stop load
|
||||
|
||||
We will finish our example by stopping the user load.
|
||||
|
||||
In the terminal where we created the container with `busybox` image, terminate
|
||||
the load generation by typing `<Ctrl> + C`.
|
||||
|
||||
Then we will verify the result state (after a minute or so):
|
||||
|
||||
```shell
|
||||
$ kubectl get hpa
|
||||
NAME REFERENCE TARGET MINPODS MAXPODS REPLICAS AGE
|
||||
php-apache Deployment/php-apache/scale 0% / 50% 1 10 1 11m
|
||||
|
||||
$ kubectl get deployment php-apache
|
||||
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
|
||||
php-apache 1 1 1 1 27m
|
||||
```
|
||||
|
||||
Here CPU utilization dropped to 0, and so HPA autoscaled the number of replicas back down to 1.
|
||||
|
||||
**Note** autoscaling the replicas may take a few minutes.
|
||||
|
||||
## Autoscaling on multiple metrics and custom metrics
|
||||
|
||||
You can introduce additional metrics to use when autoscaling the `php-apache` Deployment
|
||||
by making use of the `autoscaling/v2beta1` API version.
|
||||
|
||||
First, get the YAML of your HorizontalPodAutoscaler in the `autoscaling/v2beta1` form:
|
||||
|
||||
```shell
|
||||
$ kubectl get hpa.v2beta1.autoscaling -o yaml > /tmp/hpa-v2.yaml
|
||||
```
|
||||
|
||||
Open the `/tmp/hpa-v2.yaml` file in an editor, and you should see YAML which looks like this:
|
||||
|
||||
```yaml
|
||||
apiVersion: autoscaling/v2beta1
|
||||
kind: HorizontalPodAutoscaler
|
||||
metadata:
|
||||
name: php-apache
|
||||
namespace: default
|
||||
spec:
|
||||
scaleTargetRef:
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
name: php-apache
|
||||
minReplicas: 1
|
||||
maxReplicas: 10
|
||||
metrics:
|
||||
- type: Resource
|
||||
resource:
|
||||
name: cpu
|
||||
targetAverageUtilization: 50
|
||||
status:
|
||||
observedGeneration: 1
|
||||
lastScaleTime: <some-time>
|
||||
currentReplicas: 1
|
||||
desiredReplicas: 1
|
||||
currentMetrics:
|
||||
- type: Resource
|
||||
resource:
|
||||
name: cpu
|
||||
currentAverageUtilization: 0
|
||||
currentAverageValue: 0
|
||||
```
|
||||
|
||||
Notice that the `targetCPUUtilizationPercentage` field has been replaced with an array called `metrics`.
|
||||
The CPU utilization metric is a *resource metric*, since it is represented as a percentage of a resource
|
||||
specified on pod containers. Notice that you can specify other resource metrics besides CPU. By default,
|
||||
the only other supported resource metric is memory. These resources do not change names from cluster
|
||||
to cluster, and should always be available, as long as Heapster is deployed.
|
||||
|
||||
You can also specify resource metrics in terms of direct values, instead of as percentages of the
|
||||
requested value. To do so, use the `targetAverageValue` field instead of the `targetAverageUtilization`
|
||||
field.
|
||||
|
||||
There are two other types of metrics, both of which are considered *custom metrics*: pod metrics and
|
||||
object metrics. These metrics may have names which are cluster specific, and require a more
|
||||
advanced cluster monitoring setup.
|
||||
|
||||
The first of these alternative metric types is *pod metrics*. These metrics describe pods, and
|
||||
are averaged together across pods and compared with a target value to determine the replica count.
|
||||
They work much like resource metrics, except that they *only* have the `targetAverageValue` field.
|
||||
|
||||
Pod metrics are specified using a metric block like this:
|
||||
|
||||
```yaml
|
||||
type: Pods
|
||||
pods:
|
||||
metricName: packets-per-second
|
||||
targetAverageValue: 1k
|
||||
```
|
||||
|
||||
The second alternative metric type is *object metrics*. These metrics describe a different
|
||||
object in the same namespace, instead of describing pods. Note that the metrics are not
|
||||
fetched from the object -- they simply describe it. Object metrics do not involve averaging,
|
||||
and look like this:
|
||||
|
||||
```yaml
|
||||
type: Object
|
||||
object:
|
||||
metricName: requests-per-second
|
||||
target:
|
||||
apiVersion: extensions/v1beta1
|
||||
kind: Ingress
|
||||
name: main-route
|
||||
targetValue: 2k
|
||||
```
|
||||
|
||||
If you provide multiple such metric blocks, the HorizontalPodAutoscaler will consider each metric in turn.
|
||||
The HorizontalPodAutoscaler will calculate proposed replica counts for each metric, and then choose the
|
||||
one with the highest replica count.
|
||||
|
||||
For example, if you had your monitoring system collecting metrics about network traffic,
|
||||
you could update the definition above using `kubectl edit` to look like this:
|
||||
|
||||
```yaml
|
||||
apiVersion: autoscaling/v2beta1
|
||||
kind: HorizontalPodAutoscaler
|
||||
metadata:
|
||||
name: php-apache
|
||||
namespace: default
|
||||
spec:
|
||||
scaleTargetRef:
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
name: php-apache
|
||||
minReplicas: 1
|
||||
maxReplicas: 10
|
||||
metrics:
|
||||
- type: Resource
|
||||
resource:
|
||||
name: cpu
|
||||
targetAverageUtilization: 50
|
||||
- type: Pods
|
||||
pods:
|
||||
metricName: packets-per-second
|
||||
targetAverageValue: 1k
|
||||
- type: Object
|
||||
object:
|
||||
metricName: requests-per-second
|
||||
target:
|
||||
apiVersion: extensions/v1beta1
|
||||
kind: Ingress
|
||||
name: main-route
|
||||
targetValue: 10k
|
||||
status:
|
||||
observedGeneration: 1
|
||||
lastScaleTime: <some-time>
|
||||
currentReplicas: 1
|
||||
desiredReplicas: 1
|
||||
currentMetrics:
|
||||
- type: Resource
|
||||
resource:
|
||||
name: cpu
|
||||
currentAverageUtilization: 0
|
||||
currentAverageValue: 0
|
||||
```
|
||||
|
||||
Then, your HorizontalPodAutoscaler would attempt to ensure that each pod was consuming roughly
|
||||
50% of its requested CPU, serving 1000 packets per second, and that all pods behind the main-route
|
||||
Ingress were serving a total of 10000 requests per second.
|
||||
|
||||
### Autoscaling on metrics not related to Kubernetes objects
|
||||
|
||||
Applications running on Kubernetes may need to autoscale based on metrics that don't have an obvious
|
||||
relationship to any object in the Kubernetes cluster, such as metrics describing a hosted service with
|
||||
no direct correlation to Kubernetes namespaces. In Kubernetes 1.10 and later, you can address this use case
|
||||
with *external metrics*.
|
||||
|
||||
Using external metrics requires a certain level of knowledge of your monitoring system, and it requires a cluster
|
||||
monitoring setup similar to one required for using custom metrics. With external metrics, you can autoscale
|
||||
based on any metric available in your monitoring system by providing a `metricName` field in your
|
||||
HorizontalPodAutoscaler manifest. Additionally you can use a `metricSelector` field to limit which
|
||||
metrics' time series you want to use for autoscaling. If multiple time series are matched by `metricSelector`,
|
||||
the sum of their values is used by the HorizontalPodAutoscaler.
|
||||
|
||||
For example if your application processes tasks from a hosted queue service, you could add the following
|
||||
section to your HorizontalPodAutoscaler manifest to specify that you need one worker per 30 outstanding tasks.
|
||||
|
||||
```yaml
|
||||
- type: External
|
||||
external:
|
||||
metricName: queue_messages_ready
|
||||
metricSelector:
|
||||
matchLabels:
|
||||
queue: worker_tasks
|
||||
targetAverageValue: 30
|
||||
```
|
||||
|
||||
If your metric describes work or resources that can be divided between autoscaled pods the `targetAverageValue`
|
||||
field describes how much of that work each pod can handle. Instead of using the `targetAverageValue` field, you could use the
|
||||
`targetValue` to define a desired value of your external metric.
|
||||
|
||||
## Appendix: Horizontal Pod Autoscaler Status Conditions
|
||||
|
||||
When using the `autoscaling/v2beta1` form of the HorizontalPodAutoscaler, you will be able to see
|
||||
*status conditions* set by Kubernetes on the HorizontalPodAutoscaler. These status conditions indicate
|
||||
whether or not the HorizontalPodAutoscaler is able to scale, and whether or not it is currently restricted
|
||||
in any way.
|
||||
|
||||
The conditions appear in the `status.conditions` field. To see the conditions affecting a HorizontalPodAutoscaler,
|
||||
we can use `kubectl describe hpa`:
|
||||
|
||||
```shell
|
||||
$ kubectl describe hpa cm-test
|
||||
Name: cm-test
|
||||
Namespace: prom
|
||||
Labels: <none>
|
||||
Annotations: <none>
|
||||
CreationTimestamp: Fri, 16 Jun 2017 18:09:22 +0000
|
||||
Reference: ReplicationController/cm-test
|
||||
Metrics: ( current / target )
|
||||
"http_requests" on pods: 66m / 500m
|
||||
Min replicas: 1
|
||||
Max replicas: 4
|
||||
ReplicationController pods: 1 current / 1 desired
|
||||
Conditions:
|
||||
Type Status Reason Message
|
||||
---- ------ ------ -------
|
||||
AbleToScale True ReadyForNewScale the last scale time was sufficiently old as to warrant a new scale
|
||||
ScalingActive True ValidMetricFound the HPA was able to successfully calculate a replica count from pods metric http_requests
|
||||
ScalingLimited False DesiredWithinRange the desired replica count is within the acceptable range
|
||||
Events:
|
||||
```
|
||||
|
||||
For this HorizontalPodAutoscaler, we can see several conditions in a healthy state. The first,
|
||||
`AbleToScale`, indicates whether or not the HPA is able to fetch and update scales, as well as
|
||||
whether or not any backoff-related conditions would prevent scaling. The second, `ScalingActive`,
|
||||
indicates whether or not the HPA is enabled (i.e. the replica count of the target is not zero) and
|
||||
is able to calculate desired scales. When it is `False`, it generally indicates problems with
|
||||
fetching metrics. Finally, the last condition, `ScalingLimited`, indicates that the desired scale
|
||||
was capped by the maximum or minimum of the HorizontalPodAutoscaler. This is an indication that
|
||||
you may wish to raise or lower the minimum or maximum replica count constraints on your
|
||||
HorizontalPodAutoscaler.
|
||||
|
||||
## Appendix: Other possible scenarios
|
||||
|
||||
### Creating the autoscaler declaratively
|
||||
|
||||
Instead of using `kubectl autoscale` command to create a HorizontalPodAutoscaler imperatively we
|
||||
can use the following file to create it declaratively:
|
||||
|
||||
{{< code file="hpa-php-apache.yaml" >}}
|
||||
|
||||
We will create the autoscaler by executing the following command:
|
||||
|
||||
```shell
|
||||
$ kubectl create -f https://k8s.io/docs/tasks/run-application/hpa-php-apache.yaml
|
||||
horizontalpodautoscaler "php-apache" created
|
||||
```
|
||||
@@ -0,0 +1,186 @@
|
||||
---
|
||||
reviewers:
|
||||
- fgrzadkowski
|
||||
- jszczepkowski
|
||||
- directxman12
|
||||
title: Horizontal Pod Autoscaler
|
||||
---
|
||||
|
||||
{{< toc >}}
|
||||
|
||||
This document describes the current state of the Horizontal Pod Autoscaler in Kubernetes.
|
||||
|
||||
## What is the Horizontal Pod Autoscaler?
|
||||
|
||||
The Horizontal Pod Autoscaler automatically scales the number of pods
|
||||
in a replication controller, deployment or replica set based on observed CPU utilization (or, with
|
||||
[custom metrics](https://git.k8s.io/community/contributors/design-proposals/instrumentation/custom-metrics-api.md)
|
||||
support, on some other application-provided metrics). Note that Horizontal
|
||||
Pod Autoscaling does not apply to objects that can't be scaled, for example, DaemonSets.
|
||||
|
||||
The Horizontal Pod Autoscaler is implemented as a Kubernetes API resource and a controller.
|
||||
The resource determines the behavior of the controller.
|
||||
The controller periodically adjusts the number of replicas in a replication controller or deployment
|
||||
to match the observed average CPU utilization to the target specified by user.
|
||||
|
||||
## How does the Horizontal Pod Autoscaler work?
|
||||
|
||||
|
||||
|
||||
The Horizontal Pod Autoscaler is implemented as a control loop, with a period controlled
|
||||
by the controller manager's `--horizontal-pod-autoscaler-sync-period` flag (with a default
|
||||
value of 30 seconds).
|
||||
|
||||
During each period, the controller manager queries the resource utilization against the
|
||||
metrics specified in each HorizontalPodAutoscaler definition. The controller manager
|
||||
obtains the metrics from either the resource metrics API (for per-pod resource metrics),
|
||||
or the custom metrics API (for all other metrics).
|
||||
|
||||
* For per-pod resource metrics (like CPU), the controller fetches the metrics
|
||||
from the resource metrics API for each pod targeted by the HorizontalPodAutoscaler.
|
||||
Then, if a target utilization value is set, the controller calculates the utilization
|
||||
value as a percentage of the equivalent resource request on the containers in
|
||||
each pod. If a target raw value is set, the raw metric values are used directly.
|
||||
The controller then takes the mean of the utilization or the raw value (depending on the type
|
||||
of target specified) across all targeted pods, and produces a ratio used to scale
|
||||
the number of desired replicas.
|
||||
|
||||
Please note that if some of the pod's containers do not have the relevant resource request set,
|
||||
CPU utilization for the pod will not be defined and the autoscaler will not take any action
|
||||
for that metric. See the [autoscaling algorithm design document](https://git.k8s.io/community/contributors/design-proposals/autoscaling/horizontal-pod-autoscaler.md#autoscaling-algorithm) for further
|
||||
details about how the autoscaling algorithm works.
|
||||
|
||||
* For per-pod custom metrics, the controller functions similarly to per-pod resource metrics,
|
||||
except that it works with raw values, not utilization values.
|
||||
|
||||
* For object metrics, a single metric is fetched (which describes the object
|
||||
in question), and compared to the target value, to produce a ratio as above.
|
||||
|
||||
The HorizontalPodAutoscaler controller can fetch metrics in two different ways: direct Heapster
|
||||
access, and REST client access.
|
||||
|
||||
When using direct Heapster access, the HorizontalPodAutoscaler queries Heapster directly
|
||||
through the API server's service proxy subresource. Heapster needs to be deployed on the
|
||||
cluster and running in the kube-system namespace.
|
||||
|
||||
See [Support for custom metrics](#support-for-custom-metrics) for more details on REST client access.
|
||||
|
||||
The autoscaler accesses corresponding replication controller, deployment or replica set by scale sub-resource.
|
||||
Scale is an interface that allows you to dynamically set the number of replicas and examine each of their current states.
|
||||
More details on scale sub-resource can be found [here](https://git.k8s.io/community/contributors/design-proposals/autoscaling/horizontal-pod-autoscaler.md#scale-subresource).
|
||||
|
||||
|
||||
## API Object
|
||||
|
||||
The Horizontal Pod Autoscaler is an API resource in the Kubernetes `autoscaling` API group.
|
||||
The current stable version, which only includes support for CPU autoscaling,
|
||||
can be found in the `autoscaling/v1` API version.
|
||||
|
||||
The beta version, which includes support for scaling on memory and custom metrics,
|
||||
can be found in `autoscaling/v2beta1`. The new fields introduced in `autoscaling/v2beta1`
|
||||
are preserved as annotations when working with `autoscaling/v1`.
|
||||
|
||||
More details about the API object can be found at
|
||||
[HorizontalPodAutoscaler Object](https://git.k8s.io/community/contributors/design-proposals/autoscaling/horizontal-pod-autoscaler.md#horizontalpodautoscaler-object).
|
||||
|
||||
## Support for Horizontal Pod Autoscaler in kubectl
|
||||
|
||||
Horizontal Pod Autoscaler, like every API resource, is supported in a standard way by `kubectl`.
|
||||
We can create a new autoscaler using `kubectl create` command.
|
||||
We can list autoscalers by `kubectl get hpa` and get detailed description by `kubectl describe hpa`.
|
||||
Finally, we can delete an autoscaler using `kubectl delete hpa`.
|
||||
|
||||
In addition, there is a special `kubectl autoscale` command for easy creation of a Horizontal Pod Autoscaler.
|
||||
For instance, executing `kubectl autoscale rc foo --min=2 --max=5 --cpu-percent=80`
|
||||
will create an autoscaler for replication controller *foo*, with target CPU utilization set to `80%`
|
||||
and the number of replicas between 2 and 5.
|
||||
The detailed documentation of `kubectl autoscale` can be found [here](/docs/reference/generated/kubectl/kubectl-commands/#autoscale).
|
||||
|
||||
|
||||
## Autoscaling during rolling update
|
||||
|
||||
Currently in Kubernetes, it is possible to perform a [rolling update](/docs/tasks/run-application/rolling-update-replication-controller/) by managing replication controllers directly,
|
||||
or by using the deployment object, which manages the underlying replica sets for you.
|
||||
Horizontal Pod Autoscaler only supports the latter approach: the Horizontal Pod Autoscaler is bound to the deployment object,
|
||||
it sets the size for the deployment object, and the deployment is responsible for setting sizes of underlying replica sets.
|
||||
|
||||
Horizontal Pod Autoscaler does not work with rolling update using direct manipulation of replication controllers,
|
||||
i.e. you cannot bind a Horizontal Pod Autoscaler to a replication controller and do rolling update (e.g. using `kubectl rolling-update`).
|
||||
The reason this doesn't work is that when rolling update creates a new replication controller,
|
||||
the Horizontal Pod Autoscaler will not be bound to the new replication controller.
|
||||
|
||||
## Support for cooldown/delay
|
||||
|
||||
When managing the scale of a group of replicas using the Horizontal Pod Autoscaler,
|
||||
it is possible that the number of replicas keeps fluctuating frequently due to the
|
||||
dynamic nature of the metrics evaluated. This is sometimes referred to as *thrashing*.
|
||||
|
||||
Starting from v1.6, a cluster operator can mitigate this problem by tuning
|
||||
the global HPA settings exposed as flags for the `kube-controller-manager` component:
|
||||
|
||||
- `--horizontal-pod-autoscaler-downscale-delay`: The value for this option is a
|
||||
duration that specifies how long the autoscaler has to wait before another
|
||||
downscale operation can be performed after the current one has completed.
|
||||
The default value is 5 minutes (`5m0s`).
|
||||
|
||||
- `--horizontal-pod-autoscaler-upscale-delay`: The value for this option is a
|
||||
duration that specifies how long the autoscaler has to wait before another
|
||||
upscale operation can be performed after the current one has completed.
|
||||
The default value is 3 minutes (`3m0s`).
|
||||
|
||||
{{< note >}}
|
||||
**Note**: When tuning these parameter values, a cluster operator should be aware of
|
||||
the possible consequences. If the delay (cooldown) value is set too long, there
|
||||
could be complaints that the Horizontal Pod Autoscaler is not responsive to workload
|
||||
changes. However, if the delay value is set too short, the scale of the replicas set
|
||||
may keep thrashing as usual.
|
||||
{{< /note >}}
|
||||
|
||||
## Support for multiple metrics
|
||||
|
||||
Kubernetes 1.6 adds support for scaling based on multiple metrics. You can use the `autoscaling/v2beta1` API
|
||||
version to specify multiple metrics for the Horizontal Pod Autoscaler to scale on. Then, the Horizontal Pod
|
||||
Autoscaler controller will evaluate each metric, and propose a new scale based on that metric. The largest of the
|
||||
proposed scales will be used as the new scale.
|
||||
|
||||
## Support for custom metrics
|
||||
|
||||
**Note**: Kubernetes 1.2 added alpha support for scaling based on application-specific metrics using special annotations.
|
||||
Support for these annotations was removed in Kubernetes 1.6 in favor of the new autoscaling API. While the old method for collecting
|
||||
custom metrics is still available, these metrics will not be available for use by the Horizontal Pod Autoscaler, and the former
|
||||
annotations for specifying which custom metrics to scale on are no longer honored by the Horizontal Pod Autoscaler controller.
|
||||
|
||||
Kubernetes 1.6 adds support for making use of custom metrics in the Horizontal Pod Autoscaler.
|
||||
You can add custom metrics for the Horizontal Pod Autoscaler to use in the `autoscaling/v2beta1` API.
|
||||
Kubernetes then queries the new custom metrics API to fetch the values of the appropriate custom metrics.
|
||||
|
||||
### Requirements
|
||||
|
||||
To use custom metrics with your Horizontal Pod Autoscaler, you must set the necessary configurations when deploying your cluster:
|
||||
|
||||
* [Enable the API aggregation layer](/docs/tasks/access-kubernetes-api/configure-aggregation-layer/) if you have not already done so.
|
||||
|
||||
* Register your resource metrics API, your
|
||||
custom metrics API and, optionally, external metrics API with the API aggregation layer. All of these API servers must be running *on* your cluster.
|
||||
|
||||
* *Resource Metrics API*: You can use Heapster's implementation of the resource metrics API, by running Heapster with its `--api-server` flag set to true.
|
||||
|
||||
* *Custom Metrics API*: This must be provided by a separate component. To get started with boilerplate code, see the [kubernetes-incubator/custom-metrics-apiserver](https://github.com/kubernetes-incubator/custom-metrics-apiserver) and the [k8s.io/metrics](https://github.com/kubernetes/metrics) repositories.
|
||||
|
||||
* *External Metrics API*: Starting from Kubernetes 1.10 you can use this API if you need to autoscale on metrics not related to any Kubernetes object. Similarly to *Custom Metrics API* this must be provided by a separate component.
|
||||
|
||||
* Set the appropriate flags for kube-controller-manager:
|
||||
|
||||
* `--horizontal-pod-autoscaler-use-rest-clients` should be true.
|
||||
|
||||
* `--kubeconfig <path-to-kubeconfig>` OR `--master <ip-address-of-apiserver>`
|
||||
|
||||
Note that either the `--master` or `--kubeconfig` flag can be used; `--master` will override `--kubeconfig` if both are specified. These flags specify the location of the API aggregation layer, allowing the controller manager to communicate to the API server.
|
||||
|
||||
In Kubernetes 1.7, the standard aggregation layer that Kubernetes provides runs in-process with the kube-apiserver, so the target IP address can be found with `kubectl get pods --selector k8s-app=kube-apiserver --namespace kube-system -o jsonpath='{.items[0].status.podIP}'`.
|
||||
|
||||
## Further reading
|
||||
|
||||
* Design documentation: [Horizontal Pod Autoscaling](https://git.k8s.io/community/contributors/design-proposals/autoscaling/horizontal-pod-autoscaler.md).
|
||||
* kubectl autoscale command: [kubectl autoscale](/docs/reference/generated/kubectl/kubectl-commands/#autoscale).
|
||||
* Usage example of [Horizontal Pod Autoscaler](/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/).
|
||||
@@ -0,0 +1,13 @@
|
||||
apiVersion: autoscaling/v1
|
||||
kind: HorizontalPodAutoscaler
|
||||
metadata:
|
||||
name: php-apache
|
||||
namespace: default
|
||||
spec:
|
||||
scaleTargetRef:
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
name: php-apache
|
||||
minReplicas: 1
|
||||
maxReplicas: 10
|
||||
targetCPUUtilizationPercentage: 50
|
||||
@@ -0,0 +1,16 @@
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: mysql
|
||||
labels:
|
||||
app: mysql
|
||||
data:
|
||||
master.cnf: |
|
||||
# Apply this config only on the master.
|
||||
[mysqld]
|
||||
log-bin
|
||||
slave.cnf: |
|
||||
# Apply this config only on slaves.
|
||||
[mysqld]
|
||||
super-read-only
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: mysql
|
||||
spec:
|
||||
ports:
|
||||
- port: 3306
|
||||
selector:
|
||||
app: mysql
|
||||
clusterIP: None
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: mysql-pv-claim
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
resources:
|
||||
requests:
|
||||
storage: 20Gi
|
||||
---
|
||||
apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: mysql
|
||||
spec:
|
||||
selector:
|
||||
matchLabels:
|
||||
app: mysql
|
||||
strategy:
|
||||
type: Recreate
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: mysql
|
||||
spec:
|
||||
containers:
|
||||
- image: mysql:5.6
|
||||
name: mysql
|
||||
env:
|
||||
# Use secret in real usage
|
||||
- name: MYSQL_ROOT_PASSWORD
|
||||
value: password
|
||||
ports:
|
||||
- containerPort: 3306
|
||||
name: mysql
|
||||
volumeMounts:
|
||||
- name: mysql-persistent-storage
|
||||
mountPath: /var/lib/mysql
|
||||
volumes:
|
||||
- name: mysql-persistent-storage
|
||||
persistentVolumeClaim:
|
||||
claimName: mysql-pv-claim
|
||||
@@ -0,0 +1,30 @@
|
||||
# Headless service for stable DNS entries of StatefulSet members.
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: mysql
|
||||
labels:
|
||||
app: mysql
|
||||
spec:
|
||||
ports:
|
||||
- name: mysql
|
||||
port: 3306
|
||||
clusterIP: None
|
||||
selector:
|
||||
app: mysql
|
||||
---
|
||||
# Client service for connecting to any MySQL instance for reads.
|
||||
# For writes, you must instead connect to the master: mysql-0.mysql.
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: mysql-read
|
||||
labels:
|
||||
app: mysql
|
||||
spec:
|
||||
ports:
|
||||
- name: mysql
|
||||
port: 3306
|
||||
selector:
|
||||
app: mysql
|
||||
|
||||
@@ -0,0 +1,167 @@
|
||||
apiVersion: apps/v1
|
||||
kind: StatefulSet
|
||||
metadata:
|
||||
name: mysql
|
||||
spec:
|
||||
selector:
|
||||
matchLabels:
|
||||
app: mysql
|
||||
serviceName: mysql
|
||||
replicas: 3
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: mysql
|
||||
spec:
|
||||
initContainers:
|
||||
- name: init-mysql
|
||||
image: mysql:5.7
|
||||
command:
|
||||
- bash
|
||||
- "-c"
|
||||
- |
|
||||
set -ex
|
||||
# Generate mysql server-id from pod ordinal index.
|
||||
[[ `hostname` =~ -([0-9]+)$ ]] || exit 1
|
||||
ordinal=${BASH_REMATCH[1]}
|
||||
echo [mysqld] > /mnt/conf.d/server-id.cnf
|
||||
# Add an offset to avoid reserved server-id=0 value.
|
||||
echo server-id=$((100 + $ordinal)) >> /mnt/conf.d/server-id.cnf
|
||||
# Copy appropriate conf.d files from config-map to emptyDir.
|
||||
if [[ $ordinal -eq 0 ]]; then
|
||||
cp /mnt/config-map/master.cnf /mnt/conf.d/
|
||||
else
|
||||
cp /mnt/config-map/slave.cnf /mnt/conf.d/
|
||||
fi
|
||||
volumeMounts:
|
||||
- name: conf
|
||||
mountPath: /mnt/conf.d
|
||||
- name: config-map
|
||||
mountPath: /mnt/config-map
|
||||
- name: clone-mysql
|
||||
image: gcr.io/google-samples/xtrabackup:1.0
|
||||
command:
|
||||
- bash
|
||||
- "-c"
|
||||
- |
|
||||
set -ex
|
||||
# Skip the clone if data already exists.
|
||||
[[ -d /var/lib/mysql/mysql ]] && exit 0
|
||||
# Skip the clone on master (ordinal index 0).
|
||||
[[ `hostname` =~ -([0-9]+)$ ]] || exit 1
|
||||
ordinal=${BASH_REMATCH[1]}
|
||||
[[ $ordinal -eq 0 ]] && exit 0
|
||||
# Clone data from previous peer.
|
||||
ncat --recv-only mysql-$(($ordinal-1)).mysql 3307 | xbstream -x -C /var/lib/mysql
|
||||
# Prepare the backup.
|
||||
xtrabackup --prepare --target-dir=/var/lib/mysql
|
||||
volumeMounts:
|
||||
- name: data
|
||||
mountPath: /var/lib/mysql
|
||||
subPath: mysql
|
||||
- name: conf
|
||||
mountPath: /etc/mysql/conf.d
|
||||
containers:
|
||||
- name: mysql
|
||||
image: mysql:5.7
|
||||
env:
|
||||
- name: MYSQL_ALLOW_EMPTY_PASSWORD
|
||||
value: "1"
|
||||
ports:
|
||||
- name: mysql
|
||||
containerPort: 3306
|
||||
volumeMounts:
|
||||
- name: data
|
||||
mountPath: /var/lib/mysql
|
||||
subPath: mysql
|
||||
- name: conf
|
||||
mountPath: /etc/mysql/conf.d
|
||||
resources:
|
||||
requests:
|
||||
cpu: 500m
|
||||
memory: 1Gi
|
||||
livenessProbe:
|
||||
exec:
|
||||
command: ["mysqladmin", "ping"]
|
||||
initialDelaySeconds: 30
|
||||
periodSeconds: 10
|
||||
timeoutSeconds: 5
|
||||
readinessProbe:
|
||||
exec:
|
||||
# Check we can execute queries over TCP (skip-networking is off).
|
||||
command: ["mysql", "-h", "127.0.0.1", "-e", "SELECT 1"]
|
||||
initialDelaySeconds: 5
|
||||
periodSeconds: 2
|
||||
timeoutSeconds: 1
|
||||
- name: xtrabackup
|
||||
image: gcr.io/google-samples/xtrabackup:1.0
|
||||
ports:
|
||||
- name: xtrabackup
|
||||
containerPort: 3307
|
||||
command:
|
||||
- bash
|
||||
- "-c"
|
||||
- |
|
||||
set -ex
|
||||
cd /var/lib/mysql
|
||||
|
||||
# Determine binlog position of cloned data, if any.
|
||||
if [[ -f xtrabackup_slave_info ]]; then
|
||||
# XtraBackup already generated a partial "CHANGE MASTER TO" query
|
||||
# because we're cloning from an existing slave.
|
||||
mv xtrabackup_slave_info change_master_to.sql.in
|
||||
# Ignore xtrabackup_binlog_info in this case (it's useless).
|
||||
rm -f xtrabackup_binlog_info
|
||||
elif [[ -f xtrabackup_binlog_info ]]; then
|
||||
# We're cloning directly from master. Parse binlog position.
|
||||
[[ `cat xtrabackup_binlog_info` =~ ^(.*?)[[:space:]]+(.*?)$ ]] || exit 1
|
||||
rm xtrabackup_binlog_info
|
||||
echo "CHANGE MASTER TO MASTER_LOG_FILE='${BASH_REMATCH[1]}',\
|
||||
MASTER_LOG_POS=${BASH_REMATCH[2]}" > change_master_to.sql.in
|
||||
fi
|
||||
|
||||
# Check if we need to complete a clone by starting replication.
|
||||
if [[ -f change_master_to.sql.in ]]; then
|
||||
echo "Waiting for mysqld to be ready (accepting connections)"
|
||||
until mysql -h 127.0.0.1 -e "SELECT 1"; do sleep 1; done
|
||||
|
||||
echo "Initializing replication from clone position"
|
||||
# In case of container restart, attempt this at-most-once.
|
||||
mv change_master_to.sql.in change_master_to.sql.orig
|
||||
mysql -h 127.0.0.1 <<EOF
|
||||
$(<change_master_to.sql.orig),
|
||||
MASTER_HOST='mysql-0.mysql',
|
||||
MASTER_USER='root',
|
||||
MASTER_PASSWORD='',
|
||||
MASTER_CONNECT_RETRY=10;
|
||||
START SLAVE;
|
||||
EOF
|
||||
fi
|
||||
|
||||
# Start a server to send backups when requested by peers.
|
||||
exec ncat --listen --keep-open --send-only --max-conns=1 3307 -c \
|
||||
"xtrabackup --backup --slave-info --stream=xbstream --host=127.0.0.1 --user=root"
|
||||
volumeMounts:
|
||||
- name: data
|
||||
mountPath: /var/lib/mysql
|
||||
subPath: mysql
|
||||
- name: conf
|
||||
mountPath: /etc/mysql/conf.d
|
||||
resources:
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 100Mi
|
||||
volumes:
|
||||
- name: conf
|
||||
emptyDir: {}
|
||||
- name: config-map
|
||||
configMap:
|
||||
name: mysql
|
||||
volumeClaimTemplates:
|
||||
- metadata:
|
||||
name: data
|
||||
spec:
|
||||
accessModes: ["ReadWriteOnce"]
|
||||
resources:
|
||||
requests:
|
||||
storage: 10Gi
|
||||
@@ -0,0 +1,262 @@
|
||||
---
|
||||
reviewers:
|
||||
- janetkuo
|
||||
title: Perform Rolling Update Using a Replication Controller
|
||||
---
|
||||
|
||||
{{< toc >}}
|
||||
|
||||
## Overview
|
||||
|
||||
{{< note >}}
|
||||
**Note**: The preferred way to create a replicated application is to use a
|
||||
[Deployment](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#deployment-v1-apps),
|
||||
which in turn uses a
|
||||
[ReplicaSet](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#replicaset-v1-apps).
|
||||
For more information, see
|
||||
[Running a Stateless Application Using a Deployment](/docs/tasks/run-application/run-stateless-application-deployment/).
|
||||
{{< /note >}}
|
||||
|
||||
To update a service without an outage, `kubectl` supports what is called [rolling update](/docs/reference/generated/kubectl/kubectl-commands/#rolling-update), which updates one pod at a time, rather than taking down the entire service at the same time. See the [rolling update design document](https://git.k8s.io/community/contributors/design-proposals/cli/simple-rolling-update.md) for more information.
|
||||
|
||||
Note that `kubectl rolling-update` only supports Replication Controllers. However, if you deploy applications with Replication Controllers,
|
||||
consider switching them to [Deployments](/docs/concepts/workloads/controllers/deployment/). A Deployment is a higher-level controller that automates rolling updates
|
||||
of applications declaratively, and therefore is recommended. If you still want to keep your Replication Controllers and use `kubectl rolling-update`, keep reading:
|
||||
|
||||
A rolling update applies changes to the configuration of pods being managed by
|
||||
a replication controller. The changes can be passed as a new replication
|
||||
controller configuration file; or, if only updating the image, a new container
|
||||
image can be specified directly.
|
||||
|
||||
A rolling update works by:
|
||||
|
||||
1. Creating a new replication controller with the updated configuration.
|
||||
2. Increasing/decreasing the replica count on the new and old controllers until
|
||||
the correct number of replicas is reached.
|
||||
3. Deleting the original replication controller.
|
||||
|
||||
Rolling updates are initiated with the `kubectl rolling-update` command:
|
||||
|
||||
$ kubectl rolling-update NAME \
|
||||
([NEW_NAME] --image=IMAGE | -f FILE)
|
||||
|
||||
## Passing a configuration file
|
||||
|
||||
To initiate a rolling update using a configuration file, pass the new file to
|
||||
`kubectl rolling-update`:
|
||||
|
||||
$ kubectl rolling-update NAME -f FILE
|
||||
|
||||
The configuration file must:
|
||||
|
||||
* Specify a different `metadata.name` value.
|
||||
|
||||
* Overwrite at least one common label in its `spec.selector` field.
|
||||
|
||||
* Use the same `metadata.namespace`.
|
||||
|
||||
Replication controller configuration files are described in
|
||||
[Creating Replication Controllers](/docs/tutorials/stateless-application/run-stateless-ap-replication-controller/).
|
||||
|
||||
### Examples
|
||||
|
||||
// Update pods of frontend-v1 using new replication controller data in frontend-v2.json.
|
||||
$ kubectl rolling-update frontend-v1 -f frontend-v2.json
|
||||
|
||||
// Update pods of frontend-v1 using JSON data passed into stdin.
|
||||
$ cat frontend-v2.json | kubectl rolling-update frontend-v1 -f -
|
||||
|
||||
## Updating the container image
|
||||
|
||||
To update only the container image, pass a new image name and tag with the
|
||||
`--image` flag and (optionally) a new controller name:
|
||||
|
||||
$ kubectl rolling-update NAME [NEW_NAME] --image=IMAGE:TAG
|
||||
|
||||
The `--image` flag is only supported for single-container pods. Specifying
|
||||
`--image` with multi-container pods returns an error.
|
||||
|
||||
If no `NEW_NAME` is specified, a new replication controller is created with
|
||||
a temporary name. Once the rollout is complete, the old controller is deleted,
|
||||
and the new controller is updated to use the original name.
|
||||
|
||||
The update will fail if `IMAGE:TAG` is identical to the
|
||||
current value. For this reason, we recommend the use of versioned tags as
|
||||
opposed to values such as `:latest`. Doing a rolling update from `image:latest`
|
||||
to a new `image:latest` will fail, even if the image at that tag has changed.
|
||||
Moreover, the use of `:latest` is not recommended, see
|
||||
[Best Practices for Configuration](/docs/concepts/configuration/overview/#container-images) for more information.
|
||||
|
||||
### Examples
|
||||
|
||||
// Update the pods of frontend-v1 to frontend-v2
|
||||
$ kubectl rolling-update frontend-v1 frontend-v2 --image=image:v2
|
||||
|
||||
// Update the pods of frontend, keeping the replication controller name
|
||||
$ kubectl rolling-update frontend --image=image:v2
|
||||
|
||||
## Required and optional fields
|
||||
|
||||
Required fields are:
|
||||
|
||||
* `NAME`: The name of the replication controller to update.
|
||||
|
||||
as well as either:
|
||||
|
||||
* `-f FILE`: A replication controller configuration file, in either JSON or
|
||||
YAML format. The configuration file must specify a new top-level `id` value
|
||||
and include at least one of the existing `spec.selector` key:value pairs.
|
||||
See the
|
||||
[Run Stateless AP Replication Controller](/docs/tutorials/stateless-application/run-stateless-ap-replication-controller/#replication-controller-configuration-file)
|
||||
page for details.
|
||||
<br>
|
||||
<br>
|
||||
or:
|
||||
<br>
|
||||
<br>
|
||||
* `--image IMAGE:TAG`: The name and tag of the image to update to. Must be
|
||||
different than the current image:tag currently specified.
|
||||
|
||||
Optional fields are:
|
||||
|
||||
* `NEW_NAME`: Only used in conjunction with `--image` (not with `-f FILE`). The
|
||||
name to assign to the new replication controller.
|
||||
* `--poll-interval DURATION`: The time between polling the controller status
|
||||
after update. Valid units are `ns` (nanoseconds), `us` or `µs` (microseconds),
|
||||
`ms` (milliseconds), `s` (seconds), `m` (minutes), or `h` (hours). Units can
|
||||
be combined (e.g. `1m30s`). The default is `3s`.
|
||||
* `--timeout DURATION`: The maximum time to wait for the controller to update a
|
||||
pod before exiting. Default is `5m0s`. Valid units are as described for
|
||||
`--poll-interval` above.
|
||||
* `--update-period DURATION`: The time to wait between updating pods. Default
|
||||
is `1m0s`. Valid units are as described for `--poll-interval` above.
|
||||
|
||||
Additional information about the `kubectl rolling-update` command is available
|
||||
from the [`kubectl` reference](/docs/reference/generated/kubectl/kubectl-commands/#rolling-update).
|
||||
|
||||
## Walkthrough
|
||||
|
||||
Let's say you were running version 1.7.9 of nginx:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ReplicationController
|
||||
metadata:
|
||||
name: my-nginx
|
||||
spec:
|
||||
replicas: 5
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: nginx
|
||||
spec:
|
||||
containers:
|
||||
- name: nginx
|
||||
image: nginx:1.7.9
|
||||
ports:
|
||||
- containerPort: 80
|
||||
```
|
||||
|
||||
To update to version 1.9.1, you can use [`kubectl rolling-update --image`](https://git.k8s.io/community/contributors/design-proposals/cli/simple-rolling-update.md) to specify the new image:
|
||||
|
||||
```shell
|
||||
$ kubectl rolling-update my-nginx --image=nginx:1.9.1
|
||||
Created my-nginx-ccba8fbd8cc8160970f63f9a2696fc46
|
||||
```
|
||||
|
||||
In another window, you can see that `kubectl` added a `deployment` label to the pods, whose value is a hash of the configuration, to distinguish the new pods from the old:
|
||||
|
||||
```shell
|
||||
$ kubectl get pods -l app=nginx -L deployment
|
||||
NAME READY STATUS RESTARTS AGE DEPLOYMENT
|
||||
my-nginx-ccba8fbd8cc8160970f63f9a2696fc46-k156z 1/1 Running 0 1m ccba8fbd8cc8160970f63f9a2696fc46
|
||||
my-nginx-ccba8fbd8cc8160970f63f9a2696fc46-v95yh 1/1 Running 0 35s ccba8fbd8cc8160970f63f9a2696fc46
|
||||
my-nginx-divi2 1/1 Running 0 2h 2d1d7a8f682934a254002b56404b813e
|
||||
my-nginx-o0ef1 1/1 Running 0 2h 2d1d7a8f682934a254002b56404b813e
|
||||
my-nginx-q6all 1/1 Running 0 8m 2d1d7a8f682934a254002b56404b813e
|
||||
```
|
||||
|
||||
`kubectl rolling-update` reports progress as it progresses:
|
||||
|
||||
```
|
||||
Scaling up my-nginx-ccba8fbd8cc8160970f63f9a2696fc46 from 0 to 3, scaling down my-nginx from 3 to 0 (keep 3 pods available, don't exceed 4 pods)
|
||||
Scaling my-nginx-ccba8fbd8cc8160970f63f9a2696fc46 up to 1
|
||||
Scaling my-nginx down to 2
|
||||
Scaling my-nginx-ccba8fbd8cc8160970f63f9a2696fc46 up to 2
|
||||
Scaling my-nginx down to 1
|
||||
Scaling my-nginx-ccba8fbd8cc8160970f63f9a2696fc46 up to 3
|
||||
Scaling my-nginx down to 0
|
||||
Update succeeded. Deleting old controller: my-nginx
|
||||
Renaming my-nginx-ccba8fbd8cc8160970f63f9a2696fc46 to my-nginx
|
||||
replicationcontroller "my-nginx" rolling updated
|
||||
```
|
||||
|
||||
If you encounter a problem, you can stop the rolling update midway and revert to the previous version using `--rollback`:
|
||||
|
||||
```shell
|
||||
$ kubectl rolling-update my-nginx --rollback
|
||||
Setting "my-nginx" replicas to 1
|
||||
Continuing update with existing controller my-nginx.
|
||||
Scaling up nginx from 1 to 1, scaling down my-nginx-ccba8fbd8cc8160970f63f9a2696fc46 from 1 to 0 (keep 1 pods available, don't exceed 2 pods)
|
||||
Scaling my-nginx-ccba8fbd8cc8160970f63f9a2696fc46 down to 0
|
||||
Update succeeded. Deleting my-nginx-ccba8fbd8cc8160970f63f9a2696fc46
|
||||
replicationcontroller "my-nginx" rolling updated
|
||||
```
|
||||
|
||||
This is one example where the immutability of containers is a huge asset.
|
||||
|
||||
If you need to update more than just the image (e.g., command arguments, environment variables), you can create a new replication controller, with a new name and distinguishing label value, such as:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ReplicationController
|
||||
metadata:
|
||||
name: my-nginx-v4
|
||||
spec:
|
||||
replicas: 5
|
||||
selector:
|
||||
app: nginx
|
||||
deployment: v4
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: nginx
|
||||
deployment: v4
|
||||
spec:
|
||||
containers:
|
||||
- name: nginx
|
||||
image: nginx:1.9.2
|
||||
args: ["nginx", "-T"]
|
||||
ports:
|
||||
- containerPort: 80
|
||||
```
|
||||
|
||||
and roll it out:
|
||||
|
||||
```shell
|
||||
$ kubectl rolling-update my-nginx -f ./nginx-rc.yaml
|
||||
Created my-nginx-v4
|
||||
Scaling up my-nginx-v4 from 0 to 5, scaling down my-nginx from 4 to 0 (keep 4 pods available, don't exceed 5 pods)
|
||||
Scaling my-nginx-v4 up to 1
|
||||
Scaling my-nginx down to 3
|
||||
Scaling my-nginx-v4 up to 2
|
||||
Scaling my-nginx down to 2
|
||||
Scaling my-nginx-v4 up to 3
|
||||
Scaling my-nginx down to 1
|
||||
Scaling my-nginx-v4 up to 4
|
||||
Scaling my-nginx down to 0
|
||||
Scaling my-nginx-v4 up to 5
|
||||
Update succeeded. Deleting old controller: my-nginx
|
||||
replicationcontroller "my-nginx-v4" rolling updated
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
If the `timeout` duration is reached during a rolling update, the operation will
|
||||
fail with some pods belonging to the new replication controller, and some to the
|
||||
original controller.
|
||||
|
||||
To continue the update from where it failed, retry using the same command.
|
||||
|
||||
To roll back to the original state before the attempted update, append the
|
||||
`--rollback=true` flag to the original command. This will revert all changes.
|
||||
@@ -0,0 +1,533 @@
|
||||
---
|
||||
reviewers:
|
||||
- enisoc
|
||||
- erictune
|
||||
- foxish
|
||||
- janetkuo
|
||||
- kow3ns
|
||||
- smarterclayton
|
||||
title: Run a Replicated Stateful Application
|
||||
content_template: templates/tutorial
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
This page shows how to run a replicated stateful application using a
|
||||
[StatefulSet](/docs/concepts/workloads/controllers/statefulset/) controller.
|
||||
The example is a MySQL single-master topology with multiple slaves running
|
||||
asynchronous replication.
|
||||
|
||||
Note that **this is not a production configuration**.
|
||||
In particular, MySQL settings remain on insecure defaults to keep the focus
|
||||
on general patterns for running stateful applications in Kubernetes.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture prerequisites %}}
|
||||
|
||||
* {{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
|
||||
* {{< include "default-storage-class-prereqs.md" >}}
|
||||
* This tutorial assumes you are familiar with
|
||||
[PersistentVolumes](/docs/concepts/storage/persistent-volumes/)
|
||||
and [StatefulSets](/docs/concepts/workloads/controllers/statefulset/),
|
||||
as well as other core concepts like [Pods](/docs/concepts/workloads/pods/pod/),
|
||||
[Services](/docs/concepts/services-networking/service/), and
|
||||
[ConfigMaps](/docs/tasks/configure-pod-container/configure-pod-configmap/).
|
||||
* Some familiarity with MySQL helps, but this tutorial aims to present
|
||||
general patterns that should be useful for other systems.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture objectives %}}
|
||||
|
||||
* Deploy a replicated MySQL topology with a StatefulSet controller.
|
||||
* Send MySQL client traffic.
|
||||
* Observe resistance to downtime.
|
||||
* Scale the StatefulSet up and down.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture lessoncontent %}}
|
||||
|
||||
## Deploy MySQL
|
||||
|
||||
The example MySQL deployment consists of a ConfigMap, two Services,
|
||||
and a StatefulSet.
|
||||
|
||||
### ConfigMap
|
||||
|
||||
Create the ConfigMap from the following YAML configuration file:
|
||||
|
||||
```shell
|
||||
kubectl create -f https://k8s.io/docs/tasks/run-application/mysql-configmap.yaml
|
||||
```
|
||||
|
||||
{{< code file="mysql-configmap.yaml" >}}
|
||||
|
||||
This ConfigMap provides `my.cnf` overrides that let you independently control
|
||||
configuration on the MySQL master and slaves.
|
||||
In this case, you want the master to be able to serve replication logs to slaves
|
||||
and you want slaves to reject any writes that don't come via replication.
|
||||
|
||||
There's nothing special about the ConfigMap itself that causes different
|
||||
portions to apply to different Pods.
|
||||
Each Pod decides which portion to look at as it's initializing,
|
||||
based on information provided by the StatefulSet controller.
|
||||
|
||||
### Services
|
||||
|
||||
Create the Services from the following YAML configuration file:
|
||||
|
||||
```shell
|
||||
kubectl create -f https://k8s.io/docs/tasks/run-application/mysql-services.yaml
|
||||
```
|
||||
|
||||
{{< code file="mysql-services.yaml" >}}
|
||||
|
||||
The Headless Service provides a home for the DNS entries that the StatefulSet
|
||||
controller creates for each Pod that's part of the set.
|
||||
Because the Headless Service is named `mysql`, the Pods are accessible by
|
||||
resolving `<pod-name>.mysql` from within any other Pod in the same Kubernetes
|
||||
cluster and namespace.
|
||||
|
||||
The Client Service, called `mysql-read`, is a normal Service with its own
|
||||
cluster IP that distributes connections across all MySQL Pods that report
|
||||
being Ready. The set of potential endpoints includes the MySQL master and all
|
||||
slaves.
|
||||
|
||||
Note that only read queries can use the load-balanced Client Service.
|
||||
Because there is only one MySQL master, clients should connect directly to the
|
||||
MySQL master Pod (through its DNS entry within the Headless Service) to execute
|
||||
writes.
|
||||
|
||||
### StatefulSet
|
||||
|
||||
Finally, create the StatefulSet from the following YAML configuration file:
|
||||
|
||||
```shell
|
||||
kubectl create -f https://k8s.io/docs/tasks/run-application/mysql-statefulset.yaml
|
||||
```
|
||||
|
||||
{{< code file="mysql-statefulset.yaml" >}}
|
||||
|
||||
You can watch the startup progress by running:
|
||||
|
||||
```shell
|
||||
kubectl get pods -l app=mysql --watch
|
||||
```
|
||||
|
||||
After a while, you should see all 3 Pods become Running:
|
||||
|
||||
```
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
mysql-0 2/2 Running 0 2m
|
||||
mysql-1 2/2 Running 0 1m
|
||||
mysql-2 2/2 Running 0 1m
|
||||
```
|
||||
|
||||
Press **Ctrl+C** to cancel the watch.
|
||||
If you don't see any progress, make sure you have a dynamic PersistentVolume
|
||||
provisioner enabled as mentioned in the [prerequisites](#before-you-begin).
|
||||
|
||||
This manifest uses a variety of techniques for managing stateful Pods as part of
|
||||
a StatefulSet. The next section highlights some of these techniques to explain
|
||||
what happens as the StatefulSet creates Pods.
|
||||
|
||||
## Understanding stateful Pod initialization
|
||||
|
||||
The StatefulSet controller starts Pods one at a time, in order by their
|
||||
ordinal index.
|
||||
It waits until each Pod reports being Ready before starting the next one.
|
||||
|
||||
In addition, the controller assigns each Pod a unique, stable name of the form
|
||||
`<statefulset-name>-<ordinal-index>`.
|
||||
In this case, that results in Pods named `mysql-0`, `mysql-1`, and `mysql-2`.
|
||||
|
||||
The Pod template in the above StatefulSet manifest takes advantage of these
|
||||
properties to perform orderly startup of MySQL replication.
|
||||
|
||||
### Generating configuration
|
||||
|
||||
Before starting any of the containers in the Pod spec, the Pod first runs any
|
||||
[Init Containers](/docs/concepts/workloads/pods/init-containers/)
|
||||
in the order defined.
|
||||
|
||||
The first Init Container, named `init-mysql`, generates special MySQL config
|
||||
files based on the ordinal index.
|
||||
|
||||
The script determines its own ordinal index by extracting it from the end of
|
||||
the Pod name, which is returned by the `hostname` command.
|
||||
Then it saves the ordinal (with a numeric offset to avoid reserved values)
|
||||
into a file called `server-id.cnf` in the MySQL `conf.d` directory.
|
||||
This translates the unique, stable identity provided by the StatefulSet
|
||||
controller into the domain of MySQL server IDs, which require the same
|
||||
properties.
|
||||
|
||||
The script in the `init-mysql` container also applies either `master.cnf` or
|
||||
`slave.cnf` from the ConfigMap by copying the contents into `conf.d`.
|
||||
Because the example topology consists of a single MySQL master and any number of
|
||||
slaves, the script simply assigns ordinal `0` to be the master, and everyone
|
||||
else to be slaves.
|
||||
Combined with the StatefulSet controller's
|
||||
[deployment order guarantee](/docs/concepts/workloads/controllers/statefulset/#deployment-and-scaling-guarantees/),
|
||||
this ensures the MySQL master is Ready before creating slaves, so they can begin
|
||||
replicating.
|
||||
|
||||
### Cloning existing data
|
||||
|
||||
In general, when a new Pod joins the set as a slave, it must assume the MySQL
|
||||
master might already have data on it. It also must assume that the replication
|
||||
logs might not go all the way back to the beginning of time.
|
||||
These conservative assumptions are the key to allow a running StatefulSet
|
||||
to scale up and down over time, rather than being fixed at its initial size.
|
||||
|
||||
The second Init Container, named `clone-mysql`, performs a clone operation on
|
||||
a slave Pod the first time it starts up on an empty PersistentVolume.
|
||||
That means it copies all existing data from another running Pod,
|
||||
so its local state is consistent enough to begin replicating from the master.
|
||||
|
||||
MySQL itself does not provide a mechanism to do this, so the example uses a
|
||||
popular open-source tool called Percona XtraBackup.
|
||||
During the clone, the source MySQL server might suffer reduced performance.
|
||||
To minimize impact on the MySQL master, the script instructs each Pod to clone
|
||||
from the Pod whose ordinal index is one lower.
|
||||
This works because the StatefulSet controller always ensures Pod `N` is
|
||||
Ready before starting Pod `N+1`.
|
||||
|
||||
### Starting replication
|
||||
|
||||
After the Init Containers complete successfully, the regular containers run.
|
||||
The MySQL Pods consist of a `mysql` container that runs the actual `mysqld`
|
||||
server, and an `xtrabackup` container that acts as a
|
||||
[sidecar](http://blog.kubernetes.io/2015/06/the-distributed-system-toolkit-patterns.html).
|
||||
|
||||
The `xtrabackup` sidecar looks at the cloned data files and determines if
|
||||
it's necessary to initialize MySQL replication on the slave.
|
||||
If so, it waits for `mysqld` to be ready and then executes the
|
||||
`CHANGE MASTER TO` and `START SLAVE` commands with replication parameters
|
||||
extracted from the XtraBackup clone files.
|
||||
|
||||
Once a slave begins replication, it remembers its MySQL master and
|
||||
reconnects automatically if the server restarts or the connection dies.
|
||||
Also, because slaves look for the master at its stable DNS name
|
||||
(`mysql-0.mysql`), they automatically find the master even if it gets a new
|
||||
Pod IP due to being rescheduled.
|
||||
|
||||
Lastly, after starting replication, the `xtrabackup` container listens for
|
||||
connections from other Pods requesting a data clone.
|
||||
This server remains up indefinitely in case the StatefulSet scales up, or in
|
||||
case the next Pod loses its PersistentVolumeClaim and needs to redo the clone.
|
||||
|
||||
## Sending client traffic
|
||||
|
||||
You can send test queries to the MySQL master (hostname `mysql-0.mysql`)
|
||||
by running a temporary container with the `mysql:5.7` image and running the
|
||||
`mysql` client binary.
|
||||
|
||||
```shell
|
||||
kubectl run mysql-client --image=mysql:5.7 -i --rm --restart=Never --\
|
||||
mysql -h mysql-0.mysql <<EOF
|
||||
CREATE DATABASE test;
|
||||
CREATE TABLE test.messages (message VARCHAR(250));
|
||||
INSERT INTO test.messages VALUES ('hello');
|
||||
EOF
|
||||
```
|
||||
|
||||
Use the hostname `mysql-read` to send test queries to any server that reports
|
||||
being Ready:
|
||||
|
||||
```shell
|
||||
kubectl run mysql-client --image=mysql:5.7 -i -t --rm --restart=Never --\
|
||||
mysql -h mysql-read -e "SELECT * FROM test.messages"
|
||||
```
|
||||
|
||||
You should get output like this:
|
||||
|
||||
```
|
||||
Waiting for pod default/mysql-client to be running, status is Pending, pod ready: false
|
||||
+---------+
|
||||
| message |
|
||||
+---------+
|
||||
| hello |
|
||||
+---------+
|
||||
pod "mysql-client" deleted
|
||||
```
|
||||
|
||||
To demonstrate that the `mysql-read` Service distributes connections across
|
||||
servers, you can run `SELECT @@server_id` in a loop:
|
||||
|
||||
```shell
|
||||
kubectl run mysql-client-loop --image=mysql:5.7 -i -t --rm --restart=Never --\
|
||||
bash -ic "while sleep 1; do mysql -h mysql-read -e 'SELECT @@server_id,NOW()'; done"
|
||||
```
|
||||
|
||||
You should see the reported `@@server_id` change randomly, because a different
|
||||
endpoint might be selected upon each connection attempt:
|
||||
|
||||
```
|
||||
+-------------+---------------------+
|
||||
| @@server_id | NOW() |
|
||||
+-------------+---------------------+
|
||||
| 100 | 2006-01-02 15:04:05 |
|
||||
+-------------+---------------------+
|
||||
+-------------+---------------------+
|
||||
| @@server_id | NOW() |
|
||||
+-------------+---------------------+
|
||||
| 102 | 2006-01-02 15:04:06 |
|
||||
+-------------+---------------------+
|
||||
+-------------+---------------------+
|
||||
| @@server_id | NOW() |
|
||||
+-------------+---------------------+
|
||||
| 101 | 2006-01-02 15:04:07 |
|
||||
+-------------+---------------------+
|
||||
```
|
||||
|
||||
You can press **Ctrl+C** when you want to stop the loop, but it's useful to keep
|
||||
it running in another window so you can see the effects of the following steps.
|
||||
|
||||
## Simulating Pod and Node downtime
|
||||
|
||||
To demonstrate the increased availability of reading from the pool of slaves
|
||||
instead of a single server, keep the `SELECT @@server_id` loop from above
|
||||
running while you force a Pod out of the Ready state.
|
||||
|
||||
### Break the Readiness Probe
|
||||
|
||||
The [readiness probe](/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/#define-readiness-probes)
|
||||
for the `mysql` container runs the command `mysql -h 127.0.0.1 -e 'SELECT 1'`
|
||||
to make sure the server is up and able to execute queries.
|
||||
|
||||
One way to force this readiness probe to fail is to break that command:
|
||||
|
||||
```shell
|
||||
kubectl exec mysql-2 -c mysql -- mv /usr/bin/mysql /usr/bin/mysql.off
|
||||
```
|
||||
|
||||
This reaches into the actual container's filesystem for Pod `mysql-2` and
|
||||
renames the `mysql` command so the readiness probe can't find it.
|
||||
After a few seconds, the Pod should report one of its containers as not Ready,
|
||||
which you can check by running:
|
||||
|
||||
```shell
|
||||
kubectl get pod mysql-2
|
||||
```
|
||||
|
||||
Look for `1/2` in the `READY` column:
|
||||
|
||||
```
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
mysql-2 1/2 Running 0 3m
|
||||
```
|
||||
|
||||
At this point, you should see your `SELECT @@server_id` loop continue to run,
|
||||
although it never reports `102` anymore.
|
||||
Recall that the `init-mysql` script defined `server-id` as `100 + $ordinal`,
|
||||
so server ID `102` corresponds to Pod `mysql-2`.
|
||||
|
||||
Now repair the Pod and it should reappear in the loop output
|
||||
after a few seconds:
|
||||
|
||||
```shell
|
||||
kubectl exec mysql-2 -c mysql -- mv /usr/bin/mysql.off /usr/bin/mysql
|
||||
```
|
||||
|
||||
### Delete Pods
|
||||
|
||||
The StatefulSet also recreates Pods if they're deleted, similar to what a
|
||||
ReplicaSet does for stateless Pods.
|
||||
|
||||
```shell
|
||||
kubectl delete pod mysql-2
|
||||
```
|
||||
|
||||
The StatefulSet controller notices that no `mysql-2` Pod exists anymore,
|
||||
and creates a new one with the same name and linked to the same
|
||||
PersistentVolumeClaim.
|
||||
You should see server ID `102` disappear from the loop output for a while
|
||||
and then return on its own.
|
||||
|
||||
### Drain a Node
|
||||
|
||||
If your Kubernetes cluster has multiple Nodes, you can simulate Node downtime
|
||||
(such as when Nodes are upgraded) by issuing a
|
||||
[drain](/docs/reference/generated/kubectl/kubectl-commands/#drain).
|
||||
|
||||
First determine which Node one of the MySQL Pods is on:
|
||||
|
||||
```shell
|
||||
kubectl get pod mysql-2 -o wide
|
||||
```
|
||||
|
||||
The Node name should show up in the last column:
|
||||
|
||||
```
|
||||
NAME READY STATUS RESTARTS AGE IP NODE
|
||||
mysql-2 2/2 Running 0 15m 10.244.5.27 kubernetes-minion-group-9l2t
|
||||
```
|
||||
|
||||
Then drain the Node by running the following command, which cordons it so
|
||||
no new Pods may schedule there, and then evicts any existing Pods.
|
||||
Replace `<node-name>` with the name of the Node you found in the last step.
|
||||
|
||||
This might impact other applications on the Node, so it's best to
|
||||
**only do this in a test cluster**.
|
||||
|
||||
```shell
|
||||
kubectl drain <node-name> --force --delete-local-data --ignore-daemonsets
|
||||
```
|
||||
|
||||
Now you can watch as the Pod reschedules on a different Node:
|
||||
|
||||
```shell
|
||||
kubectl get pod mysql-2 -o wide --watch
|
||||
```
|
||||
|
||||
It should look something like this:
|
||||
|
||||
```
|
||||
NAME READY STATUS RESTARTS AGE IP NODE
|
||||
mysql-2 2/2 Terminating 0 15m 10.244.1.56 kubernetes-minion-group-9l2t
|
||||
[...]
|
||||
mysql-2 0/2 Pending 0 0s <none> kubernetes-minion-group-fjlm
|
||||
mysql-2 0/2 Init:0/2 0 0s <none> kubernetes-minion-group-fjlm
|
||||
mysql-2 0/2 Init:1/2 0 20s 10.244.5.32 kubernetes-minion-group-fjlm
|
||||
mysql-2 0/2 PodInitializing 0 21s 10.244.5.32 kubernetes-minion-group-fjlm
|
||||
mysql-2 1/2 Running 0 22s 10.244.5.32 kubernetes-minion-group-fjlm
|
||||
mysql-2 2/2 Running 0 30s 10.244.5.32 kubernetes-minion-group-fjlm
|
||||
```
|
||||
|
||||
And again, you should see server ID `102` disappear from the
|
||||
`SELECT @@server_id` loop output for a while and then return.
|
||||
|
||||
Now uncordon the Node to return it to a normal state:
|
||||
|
||||
```shell
|
||||
kubectl uncordon <node-name>
|
||||
```
|
||||
|
||||
## Scaling the number of slaves
|
||||
|
||||
With MySQL replication, you can scale your read query capacity by adding slaves.
|
||||
With StatefulSet, you can do this with a single command:
|
||||
|
||||
```shell
|
||||
kubectl scale statefulset mysql --replicas=5
|
||||
```
|
||||
|
||||
Watch the new Pods come up by running:
|
||||
|
||||
```shell
|
||||
kubectl get pods -l app=mysql --watch
|
||||
```
|
||||
|
||||
Once they're up, you should see server IDs `103` and `104` start appearing in
|
||||
the `SELECT @@server_id` loop output.
|
||||
|
||||
You can also verify that these new servers have the data you added before they
|
||||
existed:
|
||||
|
||||
```shell
|
||||
kubectl run mysql-client --image=mysql:5.7 -i -t --rm --restart=Never --\
|
||||
mysql -h mysql-3.mysql -e "SELECT * FROM test.messages"
|
||||
```
|
||||
|
||||
```
|
||||
Waiting for pod default/mysql-client to be running, status is Pending, pod ready: false
|
||||
+---------+
|
||||
| message |
|
||||
+---------+
|
||||
| hello |
|
||||
+---------+
|
||||
pod "mysql-client" deleted
|
||||
```
|
||||
|
||||
Scaling back down is also seamless:
|
||||
|
||||
```shell
|
||||
kubectl scale statefulset mysql --replicas=3
|
||||
```
|
||||
|
||||
Note, however, that while scaling up creates new PersistentVolumeClaims
|
||||
automatically, scaling down does not automatically delete these PVCs.
|
||||
This gives you the choice to keep those initialized PVCs around to make
|
||||
scaling back up quicker, or to extract data before deleting them.
|
||||
|
||||
You can see this by running:
|
||||
|
||||
```shell
|
||||
kubectl get pvc -l app=mysql
|
||||
```
|
||||
|
||||
Which shows that all 5 PVCs still exist, despite having scaled the
|
||||
StatefulSet down to 3:
|
||||
|
||||
```
|
||||
NAME STATUS VOLUME CAPACITY ACCESSMODES AGE
|
||||
data-mysql-0 Bound pvc-8acbf5dc-b103-11e6-93fa-42010a800002 10Gi RWO 20m
|
||||
data-mysql-1 Bound pvc-8ad39820-b103-11e6-93fa-42010a800002 10Gi RWO 20m
|
||||
data-mysql-2 Bound pvc-8ad69a6d-b103-11e6-93fa-42010a800002 10Gi RWO 20m
|
||||
data-mysql-3 Bound pvc-50043c45-b1c5-11e6-93fa-42010a800002 10Gi RWO 2m
|
||||
data-mysql-4 Bound pvc-500a9957-b1c5-11e6-93fa-42010a800002 10Gi RWO 2m
|
||||
```
|
||||
|
||||
If you don't intend to reuse the extra PVCs, you can delete them:
|
||||
|
||||
```shell
|
||||
kubectl delete pvc data-mysql-3
|
||||
kubectl delete pvc data-mysql-4
|
||||
```
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture cleanup %}}
|
||||
|
||||
1. Cancel the `SELECT @@server_id` loop by pressing **Ctrl+C** in its terminal,
|
||||
or running the following from another terminal:
|
||||
|
||||
```shell
|
||||
kubectl delete pod mysql-client-loop --now
|
||||
```
|
||||
|
||||
1. Delete the StatefulSet. This also begins terminating the Pods.
|
||||
|
||||
```shell
|
||||
kubectl delete statefulset mysql
|
||||
```
|
||||
|
||||
1. Verify that the Pods disappear.
|
||||
They might take some time to finish terminating.
|
||||
|
||||
```shell
|
||||
kubectl get pods -l app=mysql
|
||||
```
|
||||
|
||||
You'll know the Pods have terminated when the above returns:
|
||||
|
||||
```
|
||||
No resources found.
|
||||
```
|
||||
|
||||
1. Delete the ConfigMap, Services, and PersistentVolumeClaims.
|
||||
|
||||
```shell
|
||||
kubectl delete configmap,service,pvc -l app=mysql
|
||||
```
|
||||
|
||||
1. If you manually provisioned PersistentVolumes, you also need to manually
|
||||
delete them, as well as release the underlying resources.
|
||||
If you used a dynamic provisioner, it automatically deletes the
|
||||
PersistentVolumes when it sees that you deleted the PersistentVolumeClaims.
|
||||
Some dynamic provisioners (such as those for EBS and PD) also release the
|
||||
underlying resources upon deleting the PersistentVolumes.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture whatsnext %}}
|
||||
|
||||
* Look in the [Helm Charts repository](https://github.com/kubernetes/charts)
|
||||
for other stateful application examples.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,191 @@
|
||||
---
|
||||
title: Run a Single-Instance Stateful Application
|
||||
content_template: templates/tutorial
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
This page shows you how to run a single-instance stateful application
|
||||
in Kubernetes using a PersistentVolume and a Deployment. The
|
||||
application is MySQL.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
{{% capture objectives %}}
|
||||
|
||||
* Create a PersistentVolume referencing a disk in your environment.
|
||||
* Create a MySQL Deployment.
|
||||
* Expose MySQL to other pods in the cluster at a known DNS name.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
{{% capture prerequisites %}}
|
||||
|
||||
* {{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
|
||||
|
||||
* {{< include "default-storage-class-prereqs.md" >}}
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
{{% capture lessoncontent %}}
|
||||
|
||||
## Deploy MySQL
|
||||
|
||||
You can run a stateful application by creating a Kubernetes Deployment
|
||||
and connecting it to an existing PersistentVolume using a
|
||||
PersistentVolumeClaim. For example, this YAML file describes a
|
||||
Deployment that runs MySQL and references the PersistentVolumeClaim. The file
|
||||
defines a volume mount for /var/lib/mysql, and then creates a
|
||||
PersistentVolumeClaim that looks for a 20G volume. This claim is
|
||||
satisfied by any existing volume that meets the requirements,
|
||||
or by a dynamic provisioner.
|
||||
|
||||
Note: The password is defined in the config yaml, and this is insecure. See
|
||||
[Kubernetes Secrets](/docs/concepts/configuration/secret/)
|
||||
for a secure solution.
|
||||
|
||||
{{< code file="mysql-deployment.yaml" >}}
|
||||
|
||||
1. Deploy the contents of the YAML file:
|
||||
|
||||
kubectl create -f https://k8s.io/docs/tasks/run-application/mysql-deployment.yaml
|
||||
|
||||
1. Display information about the Deployment:
|
||||
|
||||
kubectl describe deployment mysql
|
||||
|
||||
Name: mysql
|
||||
Namespace: default
|
||||
CreationTimestamp: Tue, 01 Nov 2016 11:18:45 -0700
|
||||
Labels: app=mysql
|
||||
Annotations: deployment.kubernetes.io/revision=1
|
||||
Selector: app=mysql
|
||||
Replicas: 1 desired | 1 updated | 1 total | 0 available | 1 unavailable
|
||||
StrategyType: Recreate
|
||||
MinReadySeconds: 0
|
||||
Pod Template:
|
||||
Labels: app=mysql
|
||||
Containers:
|
||||
mysql:
|
||||
Image: mysql:5.6
|
||||
Port: 3306/TCP
|
||||
Environment:
|
||||
MYSQL_ROOT_PASSWORD: password
|
||||
Mounts:
|
||||
/var/lib/mysql from mysql-persistent-storage (rw)
|
||||
Volumes:
|
||||
mysql-persistent-storage:
|
||||
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
|
||||
ClaimName: mysql-pv-claim
|
||||
ReadOnly: false
|
||||
Conditions:
|
||||
Type Status Reason
|
||||
---- ------ ------
|
||||
Available False MinimumReplicasUnavailable
|
||||
Progressing True ReplicaSetUpdated
|
||||
OldReplicaSets: <none>
|
||||
NewReplicaSet: mysql-63082529 (1/1 replicas created)
|
||||
Events:
|
||||
FirstSeen LastSeen Count From SubobjectPath Type Reason Message
|
||||
--------- -------- ----- ---- ------------- -------- ------ -------
|
||||
33s 33s 1 {deployment-controller } Normal ScalingReplicaSet Scaled up replica set mysql-63082529 to 1
|
||||
|
||||
1. List the pods created by the Deployment:
|
||||
|
||||
kubectl get pods -l app=mysql
|
||||
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
mysql-63082529-2z3ki 1/1 Running 0 3m
|
||||
|
||||
1. Inspect the PersistentVolumeClaim:
|
||||
|
||||
kubectl describe pvc mysql-pv-claim
|
||||
|
||||
Name: mysql-pv-claim
|
||||
Namespace: default
|
||||
StorageClass:
|
||||
Status: Bound
|
||||
Volume: mysql-pv
|
||||
Labels: <none>
|
||||
Annotations: pv.kubernetes.io/bind-completed=yes
|
||||
pv.kubernetes.io/bound-by-controller=yes
|
||||
Capacity: 20Gi
|
||||
Access Modes: RWO
|
||||
Events: <none>
|
||||
|
||||
## Accessing the MySQL instance
|
||||
|
||||
The preceding YAML file creates a service that
|
||||
allows other Pods in the cluster to access the database. The Service option
|
||||
`clusterIP: None` lets the Service DNS name resolve directly to the
|
||||
Pod's IP address. This is optimal when you have only one Pod
|
||||
behind a Service and you don't intend to increase the number of Pods.
|
||||
|
||||
Run a MySQL client to connect to the server:
|
||||
|
||||
```
|
||||
kubectl run -it --rm --image=mysql:5.6 --restart=Never mysql-client -- mysql -h mysql -ppassword
|
||||
```
|
||||
|
||||
This command creates a new Pod in the cluster running a MySQL client
|
||||
and connects it to the server through the Service. If it connects, you
|
||||
know your stateful MySQL database is up and running.
|
||||
|
||||
```
|
||||
Waiting for pod default/mysql-client-274442439-zyp6i to be running, status is Pending, pod ready: false
|
||||
If you don't see a command prompt, try pressing enter.
|
||||
|
||||
mysql>
|
||||
```
|
||||
|
||||
## Updating
|
||||
|
||||
The image or any other part of the Deployment can be updated as usual
|
||||
with the `kubectl apply` command. Here are some precautions that are
|
||||
specific to stateful apps:
|
||||
|
||||
* Don't scale the app. This setup is for single-instance apps
|
||||
only. The underlying PersistentVolume can only be mounted to one
|
||||
Pod. For clustered stateful apps, see the
|
||||
[StatefulSet documentation](/docs/concepts/workloads/controllers/statefulset/).
|
||||
* Use `strategy:` `type: Recreate` in the Deployment configuration
|
||||
YAML file. This instructs Kubernetes to _not_ use rolling
|
||||
updates. Rolling updates will not work, as you cannot have more than
|
||||
one Pod running at a time. The `Recreate` strategy will stop the
|
||||
first pod before creating a new one with the updated configuration.
|
||||
|
||||
## Deleting a deployment
|
||||
|
||||
Delete the deployed objects by name:
|
||||
|
||||
```
|
||||
kubectl delete deployment,svc mysql
|
||||
kubectl delete pvc mysql-pv-claim
|
||||
```
|
||||
|
||||
If you manually provisioned a PersistentVolume, you also need to manually
|
||||
delete it, as well as release the underlying resource.
|
||||
If you used a dynamic provisioner, it automatically deletes the
|
||||
PersistentVolume when it sees that you deleted the PersistentVolumeClaim.
|
||||
Some dynamic provisioners (such as those for EBS and PD) also release the
|
||||
underlying resource upon deleting the PersistentVolume.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
{{% capture whatsnext %}}
|
||||
|
||||
* Learn more about [Deployment objects](/docs/concepts/workloads/controllers/deployment/).
|
||||
|
||||
* Learn more about [Deploying applications](/docs/user-guide/deploying-applications/)
|
||||
|
||||
* [kubectl run documentation](/docs/reference/generated/kubectl/kubectl-commands/#run)
|
||||
|
||||
* [Volumes](/docs/concepts/storage/volumes/) and [Persistent Volumes](/docs/concepts/storage/persistent-volumes/)
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
@@ -0,0 +1,157 @@
|
||||
---
|
||||
title: Run a Stateless Application Using a Deployment
|
||||
min-kubernetes-server-version: v1.9
|
||||
content_template: templates/tutorial
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
This page shows how to run an application using a Kubernetes Deployment object.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
{{% capture objectives %}}
|
||||
|
||||
* Create an nginx deployment.
|
||||
* Use kubectl to list information about the deployment.
|
||||
* Update the deployment.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
{{% capture prerequisites %}}
|
||||
|
||||
{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
{{% capture lessoncontent %}}
|
||||
|
||||
## Creating and exploring an nginx deployment
|
||||
|
||||
You can run an application by creating a Kubernetes Deployment object, and you
|
||||
can describe a Deployment in a YAML file. For example, this YAML file describes
|
||||
a Deployment that runs the nginx:1.7.9 Docker image:
|
||||
|
||||
{{< code file="deployment.yaml" >}}
|
||||
|
||||
|
||||
1. Create a Deployment based on the YAML file:
|
||||
|
||||
kubectl apply -f https://k8s.io/docs/tasks/run-application/deployment.yaml
|
||||
|
||||
1. Display information about the Deployment:
|
||||
|
||||
kubectl describe deployment nginx-deployment
|
||||
|
||||
The output is similar to this:
|
||||
|
||||
user@computer:~/website$ kubectl describe deployment nginx-deployment
|
||||
Name: nginx-deployment
|
||||
Namespace: default
|
||||
CreationTimestamp: Tue, 30 Aug 2016 18:11:37 -0700
|
||||
Labels: app=nginx
|
||||
Annotations: deployment.kubernetes.io/revision=1
|
||||
Selector: app=nginx
|
||||
Replicas: 2 desired | 2 updated | 2 total | 2 available | 0 unavailable
|
||||
StrategyType: RollingUpdate
|
||||
MinReadySeconds: 0
|
||||
RollingUpdateStrategy: 1 max unavailable, 1 max surge
|
||||
Pod Template:
|
||||
Labels: app=nginx
|
||||
Containers:
|
||||
nginx:
|
||||
Image: nginx:1.7.9
|
||||
Port: 80/TCP
|
||||
Environment: <none>
|
||||
Mounts: <none>
|
||||
Volumes: <none>
|
||||
Conditions:
|
||||
Type Status Reason
|
||||
---- ------ ------
|
||||
Available True MinimumReplicasAvailable
|
||||
Progressing True NewReplicaSetAvailable
|
||||
OldReplicaSets: <none>
|
||||
NewReplicaSet: nginx-deployment-1771418926 (2/2 replicas created)
|
||||
No events.
|
||||
|
||||
1. List the pods created by the deployment:
|
||||
|
||||
kubectl get pods -l app=nginx
|
||||
|
||||
The output is similar to this:
|
||||
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
nginx-deployment-1771418926-7o5ns 1/1 Running 0 16h
|
||||
nginx-deployment-1771418926-r18az 1/1 Running 0 16h
|
||||
|
||||
1. Display information about a pod:
|
||||
|
||||
kubectl describe pod <pod-name>
|
||||
|
||||
where `<pod-name>` is the name of one of your pods.
|
||||
|
||||
## Updating the deployment
|
||||
|
||||
You can update the deployment by applying a new YAML file. This YAML file
|
||||
specifies that the deployment should be updated to use nginx 1.8.
|
||||
|
||||
{{< code file="deployment-update.yaml" >}}
|
||||
|
||||
1. Apply the new YAML file:
|
||||
|
||||
kubectl apply -f https://k8s.io/docs/tasks/run-application/deployment-update.yaml
|
||||
|
||||
1. Watch the deployment create pods with new names and delete the old pods:
|
||||
|
||||
kubectl get pods -l app=nginx
|
||||
|
||||
## Scaling the application by increasing the replica count
|
||||
|
||||
You can increase the number of pods in your Deployment by applying a new YAML
|
||||
file. This YAML file sets `replicas` to 4, which specifies that the Deployment
|
||||
should have four pods:
|
||||
|
||||
{{< code file="deployment-scale.yaml" >}}
|
||||
|
||||
1. Apply the new YAML file:
|
||||
|
||||
kubectl apply -f https://k8s.io/docs/tasks/run-application/deployment-scale.yaml
|
||||
|
||||
1. Verify that the Deployment has four pods:
|
||||
|
||||
kubectl get pods -l app=nginx
|
||||
|
||||
The output is similar to this:
|
||||
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
nginx-deployment-148880595-4zdqq 1/1 Running 0 25s
|
||||
nginx-deployment-148880595-6zgi1 1/1 Running 0 25s
|
||||
nginx-deployment-148880595-fxcez 1/1 Running 0 2m
|
||||
nginx-deployment-148880595-rwovn 1/1 Running 0 2m
|
||||
|
||||
## Deleting a deployment
|
||||
|
||||
Delete the deployment by name:
|
||||
|
||||
kubectl delete deployment nginx-deployment
|
||||
|
||||
## ReplicationControllers -- the Old Way
|
||||
|
||||
The preferred way to create a replicated application is to use a Deployment,
|
||||
which in turn uses a ReplicaSet. Before the Deployment and ReplicaSet were
|
||||
added to Kubernetes, replicated applications were configured by using a
|
||||
[ReplicationController](/docs/concepts/workloads/controllers/replicationcontroller/).
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
{{% capture whatsnext %}}
|
||||
|
||||
* Learn more about [Deployment objects](/docs/concepts/workloads/controllers/deployment/).
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
@@ -0,0 +1,102 @@
|
||||
---
|
||||
reviewers:
|
||||
- bprashanth
|
||||
- enisoc
|
||||
- erictune
|
||||
- foxish
|
||||
- janetkuo
|
||||
- kow3ns
|
||||
- smarterclayton
|
||||
title: Scale a StatefulSet
|
||||
content_template: templates/task
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
This page shows how to scale a StatefulSet.
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture prerequisites %}}
|
||||
|
||||
* StatefulSets are only available in Kubernetes version 1.5 or later.
|
||||
* **Not all stateful applications scale nicely.** You need to understand your StatefulSets well before continuing. If you're unsure, remember that it might not be safe to scale your StatefulSets.
|
||||
* You should perform scaling only when you're sure that your stateful application
|
||||
cluster is completely healthy.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture steps %}}
|
||||
|
||||
## Use `kubectl` to scale StatefulSets
|
||||
|
||||
Make sure you have `kubectl` upgraded to Kubernetes version 1.5 or later before
|
||||
continuing. If you're unsure, run `kubectl version` and check `Client Version`
|
||||
for which kubectl you're using.
|
||||
|
||||
### `kubectl scale`
|
||||
|
||||
First, find the StatefulSet you want to scale. Remember, you need to first understand if you can scale it or not.
|
||||
|
||||
```shell
|
||||
kubectl get statefulsets <stateful-set-name>
|
||||
```
|
||||
|
||||
Change the number of replicas of your StatefulSet:
|
||||
|
||||
```shell
|
||||
kubectl scale statefulsets <stateful-set-name> --replicas=<new-replicas>
|
||||
```
|
||||
|
||||
### Alternative: `kubectl apply` / `kubectl edit` / `kubectl patch`
|
||||
|
||||
Alternatively, you can do [in-place updates](/docs/concepts/cluster-administration/manage-deployment/#in-place-updates-of-resources) on your StatefulSets.
|
||||
|
||||
If your StatefulSet was initially created with `kubectl apply` or `kubectl create --save-config`,
|
||||
update `.spec.replicas` of the StatefulSet manifests, and then do a `kubectl apply`:
|
||||
|
||||
```shell
|
||||
kubectl apply -f <stateful-set-file-updated>
|
||||
```
|
||||
|
||||
Otherwise, edit that field with `kubectl edit`:
|
||||
|
||||
```shell
|
||||
kubectl edit statefulsets <stateful-set-name>
|
||||
```
|
||||
|
||||
Or use `kubectl patch`:
|
||||
|
||||
```shell
|
||||
kubectl patch statefulsets <stateful-set-name> -p '{"spec":{"replicas":<new-replicas>}}'
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Scaling down doesn't work right
|
||||
|
||||
You cannot scale down a StatefulSet when any of the stateful Pods it manages is unhealthy. Scaling down only takes place
|
||||
after those stateful Pods become running and ready.
|
||||
|
||||
With a StatefulSet of size > 1, if there is an unhealthy Pod, there is no way
|
||||
for Kubernetes to know (yet) if it is due to a permanent fault or a transient
|
||||
one (upgrade/maintenance/node reboot). If the Pod is unhealthy due to a permanent fault, scaling
|
||||
without correcting the fault may lead to a state where the StatefulSet membership
|
||||
drops below a certain minimum number of "replicas" that are needed to function
|
||||
correctly. This may cause your StatefulSet to become unavailable.
|
||||
|
||||
If the Pod is unhealthy due to a transient fault and the Pod might become available again,
|
||||
the transient error may interfere with your scale-up/scale-down operation. Some distributed
|
||||
databases have issues when nodes join and leave at the same time. It is better
|
||||
to reason about scaling operations at the application level in these cases, and
|
||||
perform scaling only when you're sure that your stateful application cluster is
|
||||
completely healthy.
|
||||
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture whatsnext %}}
|
||||
|
||||
Learn more about [deleting a StatefulSet](/docs/tasks/manage-stateful-set/deleting-a-statefulset/).
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
@@ -0,0 +1,339 @@
|
||||
---
|
||||
title: Update API Objects in Place Using kubectl patch
|
||||
description: Use kubectl patch to update Kubernetes API objects in place. Do a strategic merge patch or a JSON merge patch.
|
||||
content_template: templates/task
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
This task shows how to use `kubectl patch` to update an API object in place. The exercises
|
||||
in this task demonstrate a strategic merge patch and a JSON merge patch.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture prerequisites %}}
|
||||
|
||||
{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
{{% capture steps %}}
|
||||
|
||||
## Use a strategic merge patch to update a Deployment
|
||||
|
||||
Here's the configuration file for a Deployment that has two replicas. Each replica
|
||||
is a Pod that has one container:
|
||||
|
||||
{{< code file="deployment-patch-demo.yaml" >}}
|
||||
|
||||
Create the Deployment:
|
||||
|
||||
```shell
|
||||
kubectl create -f https://k8s.io/docs/tasks/run-application/deployment-patch-demo.yaml
|
||||
```
|
||||
|
||||
View the Pods associated with your Deployment:
|
||||
|
||||
```shell
|
||||
kubectl get pods
|
||||
```
|
||||
|
||||
The output shows that the Deployment has two Pods. The `1/1` indicates that
|
||||
each Pod has one container:
|
||||
|
||||
|
||||
```
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
patch-demo-28633765-670qr 1/1 Running 0 23s
|
||||
patch-demo-28633765-j5qs3 1/1 Running 0 23s
|
||||
```
|
||||
|
||||
Make a note of the names of the running Pods. Later, you will see that these Pods
|
||||
get terminated and replaced by new ones.
|
||||
|
||||
At this point, each Pod has one Container that runs the nginx image. Now suppose
|
||||
you want each Pod to have two containers: one that runs nginx and one that runs redis.
|
||||
|
||||
Create a file named `patch-file-containers.yaml` that has this content:
|
||||
|
||||
```yaml
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
containers:
|
||||
- name: patch-demo-ctr-2
|
||||
image: redis
|
||||
```
|
||||
|
||||
Patch your Deployment:
|
||||
|
||||
```shell
|
||||
kubectl patch deployment patch-demo --patch "$(cat patch-file-containers.yaml)"
|
||||
```
|
||||
|
||||
View the patched Deployment:
|
||||
|
||||
```shell
|
||||
kubectl get deployment patch-demo --output yaml
|
||||
```
|
||||
|
||||
The output shows that the PodSpec in the Deployment has two Containers:
|
||||
|
||||
```shell
|
||||
containers:
|
||||
- image: redis
|
||||
imagePullPolicy: Always
|
||||
name: patch-demo-ctr-2
|
||||
...
|
||||
- image: nginx
|
||||
imagePullPolicy: Always
|
||||
name: patch-demo-ctr
|
||||
...
|
||||
```
|
||||
|
||||
View the Pods associated with your patched Deployment:
|
||||
|
||||
```shell
|
||||
kubectl get pods
|
||||
```
|
||||
|
||||
The output shows that the running Pods have different names from the Pods that
|
||||
were running previously. The Deployment terminated the old Pods and created two
|
||||
new Pods that comply with the updated Deployment spec. The `2/2` indicates that
|
||||
each Pod has two Containers:
|
||||
|
||||
```
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
patch-demo-1081991389-2wrn5 2/2 Running 0 1m
|
||||
patch-demo-1081991389-jmg7b 2/2 Running 0 1m
|
||||
```
|
||||
|
||||
Take a closer look at one of the patch-demo Pods:
|
||||
|
||||
```shell
|
||||
kubectl get pod <your-pod-name> --output yaml
|
||||
```
|
||||
|
||||
The output shows that the Pod has two Containers: one running nginx and one running redis:
|
||||
|
||||
```
|
||||
containers:
|
||||
- image: redis
|
||||
...
|
||||
- image: nginx
|
||||
...
|
||||
```
|
||||
|
||||
### Notes on the strategic merge patch
|
||||
|
||||
The patch you did in the preceding exercise is called a *strategic merge patch*.
|
||||
Notice that the patch did not replace the `containers` list. Instead it added a new
|
||||
Container to the list. In other words, the list in the patch was merged with the
|
||||
existing list. This is not always what happens when you use a strategic merge patch on a list.
|
||||
In some cases, the list is replaced, not merged.
|
||||
|
||||
With a strategic merge patch, a list is either replaced or merged depending on its
|
||||
patch strategy. The patch strategy is specified by the value of the `patchStrategy` key
|
||||
in a field tag in the Kubernetes source code. For example, the `Containers` field of `PodSpec`
|
||||
struct has a `patchStrategy` of `merge`:
|
||||
|
||||
```go
|
||||
type PodSpec struct {
|
||||
...
|
||||
Containers []Container `json:"containers" patchStrategy:"merge" patchMergeKey:"name" ...`
|
||||
```
|
||||
|
||||
You can also see the patch strategy in the
|
||||
[OpenApi spec](https://raw.githubusercontent.com/kubernetes/kubernetes/master/api/openapi-spec/swagger.json):
|
||||
|
||||
```json
|
||||
"io.k8s.api.core.v1.PodSpec": {
|
||||
...
|
||||
"containers": {
|
||||
"description": "List of containers belonging to the pod. ...
|
||||
},
|
||||
"x-kubernetes-patch-merge-key": "name",
|
||||
"x-kubernetes-patch-strategy": "merge"
|
||||
},
|
||||
```
|
||||
|
||||
And you can see the patch strategy in the
|
||||
[Kubernetes API documentation](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podspec-v1-core).
|
||||
|
||||
Create a file named `patch-file-tolerations.yaml` that has this content:
|
||||
|
||||
```yaml
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
tolerations:
|
||||
- effect: NoSchedule
|
||||
key: disktype
|
||||
value: ssd
|
||||
```
|
||||
|
||||
Patch your Deployment:
|
||||
|
||||
```shell
|
||||
kubectl patch deployment patch-demo --patch "$(cat patch-file-tolerations.yaml)"
|
||||
```
|
||||
|
||||
View the patched Deployment:
|
||||
|
||||
```shell
|
||||
kubectl get deployment patch-demo --output yaml
|
||||
```
|
||||
|
||||
The output shows that the PodSpec in the Deployment has only one Toleration:
|
||||
|
||||
```shell
|
||||
tolerations:
|
||||
- effect: NoSchedule
|
||||
key: disktype
|
||||
value: ssd
|
||||
```
|
||||
|
||||
Notice that the `tolerations` list in the PodSpec was replaced, not merged. This is because
|
||||
the Tolerations field of PodSpec does not have a `patchStrategy` key in its field tag. So the
|
||||
strategic merge patch uses the default patch strategy, which is `replace`.
|
||||
|
||||
```go
|
||||
type PodSpec struct {
|
||||
...
|
||||
Tolerations []Toleration `json:"tolerations,omitempty" protobuf:"bytes,22,opt,name=tolerations"`
|
||||
```
|
||||
|
||||
## Use a JSON merge patch to update a Deployment
|
||||
|
||||
A strategic merge patch is different from a
|
||||
[JSON merge patch](https://tools.ietf.org/html/rfc7386).
|
||||
With a JSON merge patch, if you
|
||||
want to update a list, you have to specify the entire new list. And the new list completely
|
||||
replaces the existing list.
|
||||
|
||||
The `kubectl patch` command has a `type` parameter that you can set to one of these values:
|
||||
|
||||
<table>
|
||||
<tr><th>Parameter value</th><th>Merge type</th></tr>
|
||||
<tr><td>json</td><td><a href="https://tools.ietf.org/html/rfc6902">JSON Patch, RFC 6902</a></td></tr>
|
||||
<tr><td>merge</td><td><a href="https://tools.ietf.org/html/rfc7386">JSON Merge Patch, RFC 7386</a></td></tr>
|
||||
<tr><td>strategic</td><td>Strategic merge patch</td></tr>
|
||||
</table>
|
||||
|
||||
For a comparison of JSON patch and JSON merge patch, see
|
||||
[JSON Patch and JSON Merge Patch](http://erosb.github.io/post/json-patch-vs-merge-patch/).
|
||||
|
||||
The default value for the `type` parameter is `strategic`. So in the preceding exercise, you
|
||||
did a strategic merge patch.
|
||||
|
||||
Next, do a JSON merge patch on your same Deployment. Create a file named `patch-file-2.yaml`
|
||||
that has this content:
|
||||
|
||||
```yaml
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
containers:
|
||||
- name: patch-demo-ctr-3
|
||||
image: gcr.io/google-samples/node-hello:1.0
|
||||
```
|
||||
|
||||
In your patch command, set `type` to `merge`:
|
||||
|
||||
```shell
|
||||
kubectl patch deployment patch-demo --type merge --patch "$(cat patch-file-2.yaml)"
|
||||
```
|
||||
|
||||
View the patched Deployment:
|
||||
|
||||
```shell
|
||||
kubectl get deployment patch-demo --output yaml
|
||||
```
|
||||
|
||||
The `containers` list that you specified in the patch has only one Container.
|
||||
The output shows that your list of one Container replaced the existing `containers` list.
|
||||
|
||||
```shell
|
||||
spec:
|
||||
containers:
|
||||
- image: gcr.io/google-samples/node-hello:1.0
|
||||
...
|
||||
name: patch-demo-ctr-3
|
||||
```
|
||||
|
||||
List the running Pods:
|
||||
|
||||
```shell
|
||||
kubectl get pods
|
||||
```
|
||||
|
||||
In the output, you can see that the existing Pods were terminated, and new Pods
|
||||
were created. The `1/1` indicates that each new Pod is running only one Container.
|
||||
|
||||
```shell
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
patch-demo-1307768864-69308 1/1 Running 0 1m
|
||||
patch-demo-1307768864-c86dc 1/1 Running 0 1m
|
||||
```
|
||||
|
||||
## Alternate forms of the kubectl patch command
|
||||
|
||||
The `kubectl patch` command takes YAML or JSON. It can take the patch as a file or
|
||||
directly on the command line.
|
||||
|
||||
Create a file named `patch-file.json` that has this content:
|
||||
|
||||
```json
|
||||
{
|
||||
"spec": {
|
||||
"template": {
|
||||
"spec": {
|
||||
"containers": [
|
||||
{
|
||||
"name": "patch-demo-ctr-2",
|
||||
"image": "redis"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The following commands are equivalent:
|
||||
|
||||
|
||||
```shell
|
||||
kubectl patch deployment patch-demo --patch "$(cat patch-file.yaml)"
|
||||
kubectl patch deployment patch-demo --patch $'spec:\n template:\n spec:\n containers:\n - name: patch-demo-ctr-2\n image: redis'
|
||||
|
||||
kubectl patch deployment patch-demo --patch "$(cat patch-file.json)"
|
||||
kubectl patch deployment patch-demo --patch '{"spec": {"template": {"spec": {"containers": [{"name": "patch-demo-ctr-2","image": "redis"}]}}}}'
|
||||
```
|
||||
|
||||
## Summary
|
||||
|
||||
In this exercise, you used `kubectl patch` to change the live configuration
|
||||
of a Deployment object. You did not change the configuration file that you originally used to
|
||||
create the Deployment object. Other commands for updating API objects include
|
||||
[kubectl annotate](/docs/reference/generated/kubectl/kubectl-commands/#annotate),
|
||||
[kubectl edit](/docs/reference/generated/kubectl/kubectl-commands/#edit),
|
||||
[kubectl replace](/docs/reference/generated/kubectl/kubectl-commands/#replace),
|
||||
[kubectl scale](/docs/reference/generated/kubectl/kubectl-commands/#scale),
|
||||
and
|
||||
[kubectl apply](/docs/reference/generated/kubectl/kubectl-commands/#apply).
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
{{% capture whatsnext %}}
|
||||
|
||||
* [Kubernetes Object Management](/docs/concepts/overview/object-management-kubectl/overview/)
|
||||
* [Managing Kubernetes Objects Using Imperative Commands](/docs/concepts/overview/object-management-kubectl/imperative-command/)
|
||||
* [Imperative Management of Kubernetes Objects Using Configuration Files](/docs/concepts/overview/object-management-kubectl/imperative-config/)
|
||||
* [Declarative Management of Kubernetes Objects Using Configuration Files](/docs/concepts/overview/object-management-kubectl/declarative-config/)
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user