Merge pull request #6807 from tengqm/fix-2896
Fix out-of-resource future work
This commit is contained in:
@@ -0,0 +1,31 @@
|
||||
#!/bin/bash
|
||||
#!/usr/bin/env bash
|
||||
|
||||
# This script reproduces what the kubelet does
|
||||
# to calculate memory.available relative to root cgroup.
|
||||
|
||||
# current memory usage
|
||||
memory_capacity_in_kb=$(cat /proc/meminfo | grep MemTotal | awk '{print $2}')
|
||||
memory_capacity_in_bytes=$((memory_capacity_in_kb * 1024))
|
||||
memory_usage_in_bytes=$(cat /sys/fs/cgroup/memory/memory.usage_in_bytes)
|
||||
memory_total_inactive_file=$(cat /sys/fs/cgroup/memory/memory.stat | grep total_inactive_file | awk '{print $2}')
|
||||
|
||||
memory_working_set=$memory_usage_in_bytes
|
||||
if [ "$memory_working_set" -lt "$memory_total_inactive_file" ];
|
||||
then
|
||||
memory_working_set=0
|
||||
else
|
||||
memory_working_set=$((memory_usage_in_bytes - memory_total_inactive_file))
|
||||
fi
|
||||
|
||||
memory_available_in_bytes=$((memory_capacity_in_bytes - memory_working_set))
|
||||
memory_available_in_kb=$((memory_available_in_bytes / 1024))
|
||||
memory_available_in_mb=$((memory_available_in_kb / 1024))
|
||||
|
||||
echo "memory.capacity_in_bytes $memory_capacity_in_bytes"
|
||||
echo "memory.usage_in_bytes $memory_usage_in_bytes"
|
||||
echo "memory.total_inactive_file $memory_total_inactive_file"
|
||||
echo "memory.working_set $memory_working_set"
|
||||
echo "memory.available_in_bytes $memory_available_in_bytes"
|
||||
echo "memory.available_in_kb $memory_available_in_kb"
|
||||
echo "memory.available_in_mb $memory_available_in_mb"
|
||||
@@ -370,10 +370,3 @@ to prevent system OOMs, and promote eviction of workloads so cluster state can r
|
||||
|
||||
The Pod eviction may evict more Pods than needed due to stats collection timing gap. This can be mitigated by adding
|
||||
the ability to get root container stats on an on-demand basis [(https://github.com/google/cadvisor/issues/1247)](https://github.com/google/cadvisor/issues/1247) in the future.
|
||||
|
||||
### How kubelet ranks Pods for eviction in response to inode exhaustion
|
||||
|
||||
At this time, it is not possible to know how many inodes were consumed by a particular container. If the `kubelet` observes
|
||||
inode exhaustion, it evicts Pods by ranking them by quality of service. The following issue has been opened in cadvisor
|
||||
to track per container inode consumption [(https://github.com/google/cadvisor/issues/1422)](https://github.com/google/cadvisor/issues/1422) which would allow us to rank Pods
|
||||
by inode consumption. For example, this would let us identify a container that created large numbers of 0 byte files, and evict that Pod over others.
|
||||
|
||||
Reference in New Issue
Block a user