Move StatefulSet topics. (#3659)

* Move StatefulSet topics.

* Fix toc paths.
This commit is contained in:
Steve Perry
2017-05-05 13:30:15 -07:00
committed by GitHub
parent cbe9501834
commit daecd7213d
5 changed files with 16 additions and 7 deletions
@@ -0,0 +1,90 @@
---
assignees:
- bprashanth
- erictune
- foxish
- janetkuo
- smarterclayton
title: Deleting a Stateful Set
redirect_from:
- "/docs/tasks/manage-stateful-set/deleting-a-statefulset/"
- "/docs/tasks/manage-stateful-set/deleting-a-statefulset.html"
---
{% capture overview %}
This task shows you how to delete a StatefulSet.
{% endcapture %}
{% capture prerequisites %}
* This task assumes you have an application running on your cluster represented by a StatefulSet.
{% endcapture %}
{% capture steps %}
## Deleting a StatefulSet
You can delete a StatefulSet in the same way you delete other resources in Kubernetes: use the `kubectl delete` command, and specify the StatefulSet either by file or by name.
```shell
kubectl delete -f <file.yaml>
```
```shell
kubectl delete statefulsets <statefulset-name>
```
You may need to delete the associated headless service separately after the StatefulSet itself is deleted.
```shell
kubectl delete service <service-name>
```
Deleting a StatefulSet through kubectl will scale it down to 0, thereby deleting all pods that are a part of it. If you want to delete just the StatefulSet and not the pods, use `--cascade=false`.
```shell
kubectl delete -f <file.yaml> --cascade=false
```
By passing `--cascade=false` to `kubectl delete`, the Pods managed by the StatefulSet are left behind even after the StatefulSet object itself is deleted. If the pods have a label `app=myapp`, you can then delete them as follows:
```shell
kubectl delete pods -l app=myapp
```
### Persistent Volumes
Deleting the Pods in a StatefulSet will not delete the associated volumes. This is to ensure that you have the chance to copy data off the volume before deleting it. Deleting the PVC after the pods have left the [terminating state](/docs/user-guide/pods/index#termination-of-pods) might trigger deletion of the backing Persistent Volumes depending on the storage class and reclaim policy. You should never assume ability to access a volume after claim deletion.
**Note: Use caution when deleting a PVC, as it may lead to data loss.**
### Complete deletion of a StatefulSet
To simply delete everything in a StatefulSet, including the associated pods, you can run a series of commands similar to the following:
```shell{% raw %}
grace=$(kubectl get pods <stateful-set-pod> --template '{{.spec.terminationGracePeriodSeconds}}')
kubectl delete statefulset -l app=myapp
sleep $grace
kubectl delete pvc -l app=myapp
{% endraw %}
```
In the example above, the Pods have the label `app=myapp`; substitute your own label as appropriate.
### Force deletion of StatefulSet pods
If you find that some pods in your StatefulSet are stuck in the 'Terminating' or 'Unknown' states for an extended period of time, you may need to manually intervene to forcefully delete the pods from the apiserver. This is a potentially dangerous task. Refer to [Deleting StatefulSet Pods](/docs/tasks/manage-stateful-set/delete-pods/) for details.
{% endcapture %}
{% capture whatsnext %}
Learn more about [force deleting StatefulSet Pods](/docs/tasks/manage-stateful-set/delete-pods/).
{% endcapture %}
{% include templates/task.md %}
@@ -0,0 +1,80 @@
---
assignees:
- bprashanth
- erictune
- foxish
- smarterclayton
title: Force Deleting StatefulSet Pods
redirect:from:
- "/docs/tasks/manage-stateful-set/delete-pods/"
- "/docs/tasks/manage-stateful-set/delete-pods.html"
---
{% capture overview %}
This page shows how to delete Pods which are part of a stateful set, and explains the considerations to keep in mind when doing so.
{% endcapture %}
{% capture prerequisites %}
* This is a fairly advanced task and has the potential to violate some of the properties inherent to StatefulSet.
* Before proceeding, make yourself familiar with the considerations enumerated below.
{% endcapture %}
{% capture steps %}
## StatefulSet considerations
In normal operation of a StatefulSet, there is **never** a need to force delete a StatefulSet Pod. The StatefulSet controller is responsible for creating, scaling and deleting members of the StatefulSet. It tries to ensure that the specified number of Pods from ordinal 0 through N-1 are alive and ready. StatefulSet ensures that, at any time, there is at most one Pod with a given identity running in a cluster. This is referred to as *at most one* semantics provided by a StatefulSet.
Manual force deletion should be undertaken with caution, as it has the potential to violate the at most one semantics inherent to StatefulSet. StatefulSets may be used to run distributed and clustered applications which have a need for a stable network identity and stable storage. These applications often have configuration which relies on an ensemble of a fixed number of members with fixed identities. Having multiple members with the same identity can be disastrous and may lead to data loss (e.g. split brain scenario in quorum-based systems).
## Deleting Pods
You can perform a graceful pod deletion with the following command:
```shell
kubectl delete pods <pod>
```
For the above to lead to graceful termination, the Pod **must not** specify a `pod.Spec.TerminationGracePeriodSeconds` of 0. The practice of setting a `pod.Spec.TerminationGracePeriodSeconds` of 0 seconds is unsafe and strongly discouraged for StatefulSet Pods. Graceful deletion is safe and will ensure that the [Pod shuts down gracefully](/docs/user-guide/pods/#termination-of-pods) before the kubelet deletes the name from the apiserver.
Kubernetes (versions 1.5 or newer) will not delete Pods just because a Node is unreachable. The Pods running on an unreachable Node enter the 'Terminating' or 'Unknown' state after a [timeout](/docs/admin/node/#node-condition). Pods may also enter these states when the user attempts graceful deletion of a Pod on an unreachable Node. The only ways in which a Pod in such a state can be removed from the apiserver are as follows:
* The Node object is deleted (either by you, or by the [Node Controller](/docs/admin/node)).
* The kubelet on the unresponsive Node starts responding, kills the Pod and removes the entry from the apiserver.
* Force deletion of the Pod by the user.
The recommended best practice is to use the first or second approach. If a Node is confirmed to be dead (e.g. permanently disconnected from the network, powered down, etc), then delete the node object. If the node is suffering from a network partition, then try to resolve this or wait for it to resolve. When the partition heals, the kubelet will complete the deletion of the Pod and free up its name in the apiserver.
Normally, the system completes the deletion once the Pod is no longer running on a Node, or the Node is deleted by an administrator. You may override this by force deleting the Pod.
### Force Deletion
Force deletions **do not** wait for confirmation from the kubelet that the Pod has been terminated. Irrespective of whether a force deletion is successful in killing a Pod, it will immediately free up the name from the apiserver. This would let the StatefulSet controller create a replacement Pod with that same identity; this can lead to the duplication of a still-running Pod, and if said Pod can still communicate with the other members of the StatefulSet, will violate the at most one semantics that StatefulSet is designed to guarantee.
When you force delete a StatefulSet pod, you are asserting that the Pod in question will never again make contact with other Pods in the StatefulSet and its name can be safely freed up for a replacement to be created.
If you want to delete a Pod forcibly using kubectl version >= 1.5, do the following:
```shell
kubectl delete pods <pod> --grace-period=0 --force
```
If you're using any version of kubectl <= 1.4, you should omit the `--force` option and use:
```shell
kubectl delete pods <pod> --grace-period=0
```
Always perform force deletion of StatefulSet Pods carefully and with complete knowledge of the risks involved.
{% endcapture %}
{% capture whatsnext %}
Learn more about [debugging a StatefulSet](/docs/tasks/manage-stateful-set/debugging-a-statefulset/).
{% endcapture %}
{% include templates/task.md %}
@@ -0,0 +1,104 @@
---
assignees:
- bprashanth
- enisoc
- erictune
- foxish
- janetkuo
- kow3ns
- smarterclayton
title: Scaling a StatefulSet
redirect_from:
- "/docs/tasks/manage-stateful-set/scale-stateful-set/"
- "/docs/tasks/manage-stateful-set/scale-stateful-set.html"
---
{% capture overview %}
This page shows how to scale a StatefulSet.
{% endcapture %}
{% capture prerequisites %}
* StatefulSets are only available in Kubernetes version 1.5 or later.
* **Not all stateful applications scale nicely.** You need to understand your StatefulSets well before continuing. If you're unsure, remember that it might not be safe to scale your StatefulSets.
* You should perform scaling only when you're sure that your stateful application
cluster is completely healthy.
{% endcapture %}
{% capture steps %}
## Use `kubectl` to scale StatefulSets
Make sure you have `kubectl` upgraded to Kubernetes version 1.5 or later before
continuing. If you're unsure, run `kubectl version` and check `Client Version`
for which kubectl you're using.
### `kubectl scale`
First, find the StatefulSet you want to scale. Remember, you need to first understand if you can scale it or not.
```shell
kubectl get statefulsets <stateful-set-name>
```
Change the number of replicas of your StatefulSet:
```shell
kubectl scale statefulsets <stateful-set-name> --replicas=<new-replicas>
```
### Alternative: `kubectl apply` / `kubectl edit` / `kubectl patch`
Alternatively, you can do [in-place updates](/docs/concepts/cluster-administration/manage-deployment/#in-place-updates-of-resources) on your StatefulSets.
If your StatefulSet was initially created with `kubectl apply` or `kubectl create --save-config`,
update `.spec.replicas` of the StatefulSet manifests, and then do a `kubectl apply`:
```shell
kubectl apply -f <stateful-set-file-updated>
```
Otherwise, edit that field with `kubectl edit`:
```shell
kubectl edit statefulsets <stateful-set-name>
```
Or use `kubectl patch`:
```shell
kubectl patch statefulsets <stateful-set-name> -p '{"spec":{"replicas":<new-replicas>}}'
```
## Troubleshooting
### Scaling down doesn't not work right
You cannot scale down a StatefulSet when any of the stateful Pods it manages is unhealthy. Scaling down only takes place
after those stateful Pods become running and ready.
With a StatefulSet of size > 1, if there is an unhealthy Pod, there is no way
for Kubernetes to know (yet) if it is due to a permanent fault or a transient
one (upgrade/maintenance/node reboot). If the Pod is unhealthy due to a permanent fault, scaling
without correcting the fault may lead to a state where the StatefulSet membership
drops below a certain minimum number of "replicas" that are needed to function
correctly. This may cause your StatefulSet to become unavailable.
If the Pod is unhealthy due to a transient fault and the Pod might become available again,
the transient error may interfere with your scale-up/scale-down operation. Some distributed
databases have issues when nodes join and leave at the same time. It is better
to reason about scaling operations at the application level in these cases, and
perform scaling only when you're sure that your stateful application cluster is
completely healthy.
{% endcapture %}
{% capture whatsnext %}
Learn more about [deleting a StatefulSet](/docs/tasks/manage-stateful-set/deleting-a-statefulset/).
{% endcapture %}
{% include templates/task.md %}