From fe10be967770b794c651ef143a4a86ab4858639b Mon Sep 17 00:00:00 2001 From: Steve Perry Date: Mon, 10 Apr 2017 14:24:07 -0700 Subject: [PATCH] Move Guide topic: kubelet Garbage Collection. (#3301) --- _data/concepts.yml | 1 + docs/admin/garbage-collection.md | 69 +---------------- .../kubelet-garbage-collection.md | 75 +++++++++++++++++++ 3 files changed, 78 insertions(+), 67 deletions(-) create mode 100644 docs/concepts/cluster-administration/kubelet-garbage-collection.md diff --git a/_data/concepts.yml b/_data/concepts.yml index 73a1ea192c..1031271611 100644 --- a/_data/concepts.yml +++ b/_data/concepts.yml @@ -55,6 +55,7 @@ toc: - docs/concepts/cluster-administration/audit.md - docs/concepts/cluster-administration/resource-usage-monitoring.md - docs/concepts/cluster-administration/out-of-resource.md + - docs/concepts/cluster-administration/kubelet-garbage-collection.md - docs/concepts/cluster-administration/cluster-management.md - docs/concepts/cluster-administration/multiple-clusters.md - docs/concepts/cluster-administration/federation.md diff --git a/docs/admin/garbage-collection.md b/docs/admin/garbage-collection.md index c6cd861d58..829d9cef6f 100644 --- a/docs/admin/garbage-collection.md +++ b/docs/admin/garbage-collection.md @@ -4,72 +4,7 @@ assignees: title: Configuring kubelet Garbage Collection --- -* TOC -{:toc} -Garbage collection is a helpful function of kubelet that will clean up unused images and unused containers. Kubelet will perform garbage collection for containers every minute and garbage collection for images every five minutes. +{% include user-guide-content-moved.md %} -External garbage collection tools are not recommended as these tools can potentially break the behavior of kubelet by removing containers expected to exist. - -### Image Collection - -Kubernetes manages lifecycle of all images through imageManager, with the cooperation -of cadvisor. - -The policy for garbage collecting images takes two factors into consideration: -`HighThresholdPercent` and `LowThresholdPercent`. Disk usage above the high threshold -will trigger garbage collection. The garbage collection will delete least recently used images until the low -threshold has been met. - -### Container Collection - -The policy for garbage collecting containers considers three user-defined variables. `MinAge` is the minimum age at which a container can be garbage collected. `MaxPerPodContainer` is the maximum number of dead containers every single -pod (UID, container name) pair is allowed to have. `MaxContainers` is the maximum number of total dead containers. These variables can be individually disabled by setting `MinAge` to zero and setting `MaxPerPodContainer` and `MaxContainers` respectively to less than zero. - -Kubelet will act on containers that are unidentified, deleted, or outside of the boundaries set by the previously mentioned flags. The oldest containers will generally be removed first. `MaxPerPodContainer` and `MaxContainer` may potentially conflict with each other in situations where retaining the maximum number of containers per pod (`MaxPerPodContainer`) would go outside the allowable range of global dead containers (`MaxContainers`). `MaxPerPodContainer` would be adjusted in this situation: A worst case scenario would be to downgrade `MaxPerPodContainer` to 1 and evict the oldest containers. Additionally, containers owned by pods that have been deleted are removed once they are older than `MinAge`. - -Containers that are not managed by kubelet are not subject to container garbage collection. - -### User Configuration - -Users can adjust the following thresholds to tune image garbage collection with the following kubelet flags : - -1. `image-gc-high-threshold`, the percent of disk usage which triggers image garbage collection. -Default is 90%. -2. `image-gc-low-threshold`, the percent of disk usage to which image garbage collection attempts -to free. Default is 80%. - -We also allow users to customize garbage collection policy through the following kubelet flags: - -1. `minimum-container-ttl-duration`, minimum age for a finished container before it is -garbage collected. Default is 0 minute, which means every finished container will be garbaged collected. -2. `maximum-dead-containers-per-container`, maximum number of old instances to be retained -per container. Default is 1. -3. `maximum-dead-containers`, maximum number of old instances of containers to retain globally. -Default is -1, which means there is no global limit. - -Containers can potentially be garbage collected before their usefulness has expired. These containers -can contain logs and other data that can be useful for troubleshooting. A sufficiently large value for -`maximum-dead-containers-per-container` is highly recommended to allow at least 1 dead container to be -retained per expected container. A larger value for `maximum-dead-containers` is also recommended for a -similar reason. -See [this issue](https://github.com/kubernetes/kubernetes/issues/13287) for more details. - - -### Deprecation - -Some kubelet Garbage Collection features in this doc will be replaced by kubelet eviction in the future. - -Including: - -| Existing Flag | New Flag | Rationale | -| ------------- | -------- | --------- | -| `--image-gc-high-threshold` | `--eviction-hard` or `--eviction-soft` | existing eviction signals can trigger image garbage collection | -| `--image-gc-low-threshold` | `--eviction-minimum-reclaim` | eviction reclaims achieve the same behavior | -| `--maximum-dead-containers` | | deprecated once old logs are stored outside of container's context | -| `--maximum-dead-containers-per-container` | | deprecated once old logs are stored outside of container's context | -| `--minimum-container-ttl-duration` | | deprecated once old logs are stored outside of container's context | -| `--low-diskspace-threshold-mb` | `--eviction-hard` or `eviction-soft` | eviction generalizes disk thresholds to other resources | -| `--outofdisk-transition-frequency` | `--eviction-pressure-transition-period` | eviction generalizes disk pressure transition to other resources | - -See [Configuring Out Of Resource Handling](/docs/admin/out-of-resource/) for more details. +[kubelet Garbage Collection](/docs/concepts/cluster-administration/kubelet-garbage-collection/) diff --git a/docs/concepts/cluster-administration/kubelet-garbage-collection.md b/docs/concepts/cluster-administration/kubelet-garbage-collection.md new file mode 100644 index 0000000000..c6cd861d58 --- /dev/null +++ b/docs/concepts/cluster-administration/kubelet-garbage-collection.md @@ -0,0 +1,75 @@ +--- +assignees: +- mikedanese +title: Configuring kubelet Garbage Collection +--- + +* TOC +{:toc} + +Garbage collection is a helpful function of kubelet that will clean up unused images and unused containers. Kubelet will perform garbage collection for containers every minute and garbage collection for images every five minutes. + +External garbage collection tools are not recommended as these tools can potentially break the behavior of kubelet by removing containers expected to exist. + +### Image Collection + +Kubernetes manages lifecycle of all images through imageManager, with the cooperation +of cadvisor. + +The policy for garbage collecting images takes two factors into consideration: +`HighThresholdPercent` and `LowThresholdPercent`. Disk usage above the high threshold +will trigger garbage collection. The garbage collection will delete least recently used images until the low +threshold has been met. + +### Container Collection + +The policy for garbage collecting containers considers three user-defined variables. `MinAge` is the minimum age at which a container can be garbage collected. `MaxPerPodContainer` is the maximum number of dead containers every single +pod (UID, container name) pair is allowed to have. `MaxContainers` is the maximum number of total dead containers. These variables can be individually disabled by setting `MinAge` to zero and setting `MaxPerPodContainer` and `MaxContainers` respectively to less than zero. + +Kubelet will act on containers that are unidentified, deleted, or outside of the boundaries set by the previously mentioned flags. The oldest containers will generally be removed first. `MaxPerPodContainer` and `MaxContainer` may potentially conflict with each other in situations where retaining the maximum number of containers per pod (`MaxPerPodContainer`) would go outside the allowable range of global dead containers (`MaxContainers`). `MaxPerPodContainer` would be adjusted in this situation: A worst case scenario would be to downgrade `MaxPerPodContainer` to 1 and evict the oldest containers. Additionally, containers owned by pods that have been deleted are removed once they are older than `MinAge`. + +Containers that are not managed by kubelet are not subject to container garbage collection. + +### User Configuration + +Users can adjust the following thresholds to tune image garbage collection with the following kubelet flags : + +1. `image-gc-high-threshold`, the percent of disk usage which triggers image garbage collection. +Default is 90%. +2. `image-gc-low-threshold`, the percent of disk usage to which image garbage collection attempts +to free. Default is 80%. + +We also allow users to customize garbage collection policy through the following kubelet flags: + +1. `minimum-container-ttl-duration`, minimum age for a finished container before it is +garbage collected. Default is 0 minute, which means every finished container will be garbaged collected. +2. `maximum-dead-containers-per-container`, maximum number of old instances to be retained +per container. Default is 1. +3. `maximum-dead-containers`, maximum number of old instances of containers to retain globally. +Default is -1, which means there is no global limit. + +Containers can potentially be garbage collected before their usefulness has expired. These containers +can contain logs and other data that can be useful for troubleshooting. A sufficiently large value for +`maximum-dead-containers-per-container` is highly recommended to allow at least 1 dead container to be +retained per expected container. A larger value for `maximum-dead-containers` is also recommended for a +similar reason. +See [this issue](https://github.com/kubernetes/kubernetes/issues/13287) for more details. + + +### Deprecation + +Some kubelet Garbage Collection features in this doc will be replaced by kubelet eviction in the future. + +Including: + +| Existing Flag | New Flag | Rationale | +| ------------- | -------- | --------- | +| `--image-gc-high-threshold` | `--eviction-hard` or `--eviction-soft` | existing eviction signals can trigger image garbage collection | +| `--image-gc-low-threshold` | `--eviction-minimum-reclaim` | eviction reclaims achieve the same behavior | +| `--maximum-dead-containers` | | deprecated once old logs are stored outside of container's context | +| `--maximum-dead-containers-per-container` | | deprecated once old logs are stored outside of container's context | +| `--minimum-container-ttl-duration` | | deprecated once old logs are stored outside of container's context | +| `--low-diskspace-threshold-mb` | `--eviction-hard` or `eviction-soft` | eviction generalizes disk thresholds to other resources | +| `--outofdisk-transition-frequency` | `--eviction-pressure-transition-period` | eviction generalizes disk pressure transition to other resources | + +See [Configuring Out Of Resource Handling](/docs/admin/out-of-resource/) for more details.