Revert v1.17 release changes on v1.16 branch (#18123)
This reverts commits:1131f8603e.cf5b8b200c.75de70a431.dff2f7b960.cf925bb688.ffd1363c0e.87a2cafb20.ffe259d6f2.
This commit is contained in:
Tim Bannister
committed by
Kubernetes Prow Robot
Kubernetes Prow Robot
parent
5733771d62
commit
065ccd94be
@@ -154,8 +154,6 @@ spec:
|
||||
|
||||
## Garbage Collecting Load Balancers
|
||||
|
||||
{{< feature-state for_k8s_version="v1.17" state="stable" >}}
|
||||
|
||||
In usual case, the correlating load balancer resources in cloud provider should
|
||||
be cleaned up soon after a LoadBalancer type Service is deleted. But it is known
|
||||
that there are various corner cases where cloud resources are orphaned after the
|
||||
@@ -169,6 +167,10 @@ The finalizer will only be removed after the load balancer resource is cleaned u
|
||||
This prevents dangling load balancer resources even in corner cases such as the
|
||||
service controller crashing.
|
||||
|
||||
This feature is beta and enabled by default since Kubernetes v1.16. You can also
|
||||
enable it in v1.15 (alpha) via the [feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
|
||||
`ServiceLoadBalancerFinalizer`.
|
||||
|
||||
## External Load Balancer Providers
|
||||
|
||||
It is important to note that the datapath for this functionality is provided by a load balancer external to the Kubernetes cluster.
|
||||
|
||||
+3
-3
@@ -628,7 +628,7 @@ Additionally, the following restrictions are applied to the schema:
|
||||
|
||||
These fields can only be set with specific features enabled:
|
||||
|
||||
- `default`: can be set for `apiextensions.k8s.io/v1` CustomResourceDefinitions. Defaulting is in GA since 1.17 (beta since 1.16 with the `CustomResourceDefaulting` feature gate to be enabled, which is the case automatically for many clusters for beta features). Compare [Validation Schema Defaulting](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/#defaulting).
|
||||
- `default`: can be set for `apiextensions.k8s.io/v1` CustomResourceDefinitions. Defaulting is in beta since 1.16 and requires the `CustomResourceDefaulting` feature gate to be enabled (which is the case automatically for many clusters for beta features). Compare [Validation Schema Defaulting](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/#defaulting).
|
||||
|
||||
Note: compare with [structural schemas](#specifying-a-structural-schema) for further restriction required for certain CustomResourceDefinition features.
|
||||
|
||||
@@ -781,10 +781,10 @@ crontab "my-new-cron-object" created
|
||||
|
||||
### Defaulting
|
||||
|
||||
{{< feature-state state="stable" for_kubernetes_version="1.17" >}}
|
||||
{{< feature-state state="beta" for_kubernetes_version="1.16" >}}
|
||||
|
||||
{{< note >}}
|
||||
To use defaulting, your CustomResourceDefinition must use API version `apiextensions.k8s.io/v1`.
|
||||
Defaulting is available as beta since 1.16 in `apiextensions.k8s.io/v1` CustomResourceDefinitions, and hence enabled by default for most clusters (feature gate `CustomResourceDefaulting`, refer to the [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) documentation).
|
||||
{{< /note >}}
|
||||
|
||||
Defaulting allows to specify default values in the [OpenAPI v3 validation schema](#validation):
|
||||
|
||||
@@ -85,10 +85,7 @@ state file `cpu_manager_state` in the kubelet root directory.
|
||||
This policy manages a shared pool of CPUs that initially contains all CPUs in the
|
||||
node. The amount of exclusively allocatable CPUs is equal to the total
|
||||
number of CPUs in the node minus any CPU reservations by the kubelet `--kube-reserved` or
|
||||
`--system-reserved` options. From 1.17, the CPU reservation list can be specified
|
||||
explictly by kubelet `--reserved-cpus` option. The explicit CPU list specified by
|
||||
`--reserved-cpus` takes precedence over the CPU reservation specified by
|
||||
`--kube-reserved` and `--system-reserved`. CPUs reserved by these options are taken, in
|
||||
`--system-reserved` options. CPUs reserved by these options are taken, in
|
||||
integer quantity, from the initial shared pool in ascending order by physical
|
||||
core ID. This shared pool is the set of CPUs on which any containers in
|
||||
`BestEffort` and `Burstable` pods run. Containers in `Guaranteed` pods with fractional
|
||||
@@ -98,8 +95,8 @@ exclusive CPUs.
|
||||
|
||||
{{< note >}}
|
||||
The kubelet requires a CPU reservation greater than zero be made
|
||||
using either `--kube-reserved` and/or `--system-reserved` or `--reserved-cpus` when
|
||||
the static policy is enabled. This is because zero CPU reservation would allow the shared
|
||||
using either `--kube-reserved` and/or `--system-reserved` when the static
|
||||
policy is enabled. This is because zero CPU reservation would allow the shared
|
||||
pool to become empty.
|
||||
{{< /note >}}
|
||||
|
||||
|
||||
+15
-3
@@ -27,7 +27,7 @@ resources instead of a single large Endpoints resource.
|
||||
|
||||
## Enabling Endpoint Slices
|
||||
|
||||
{{< feature-state for_k8s_version="v1.17" state="beta" >}}
|
||||
{{< feature-state for_k8s_version="v1.16" state="alpha" >}}
|
||||
|
||||
{{< note >}}
|
||||
Although Endpoint Slices may eventually replace Endpoints, many Kubernetes
|
||||
@@ -36,7 +36,19 @@ seen as an addition to Endpoints in a cluster, not a replacement for them.
|
||||
{{< /note >}}
|
||||
|
||||
As an alpha feature, Endpoint Slices are not enabled by default in Kubernetes.
|
||||
To enable them, the EndpointSlice feature gate will need to be enabled
|
||||
Enabling Endpoint Slices requires as many as 3 changes to Kubernetes cluster
|
||||
configuration.
|
||||
|
||||
To enable the Discovery API group that includes Endpoint Slices, use the runtime
|
||||
config flag (`--runtime-config=discovery.k8s.io/v1alpha1=true`).
|
||||
|
||||
The logic responsible for watching services, pods, and nodes and creating or
|
||||
updating associated Endpoint Slices lives within the EndpointSlice controller.
|
||||
This is disabled by default but can be enabled with the controllers flag on
|
||||
kube-controller-manager (`--controllers=endpointslice`).
|
||||
|
||||
For Kubernetes components like kube-proxy to actually start using Endpoint
|
||||
Slices, the EndpointSlice feature gate will need to be enabled
|
||||
(`--feature-gates=EndpointSlice=true`).
|
||||
|
||||
## Using Endpoint Slices
|
||||
@@ -45,4 +57,4 @@ With Endpoint Slices fully enabled in your cluster, you should see corresponding
|
||||
EndpointSlice resources for each Endpoints resource. In addition to supporting
|
||||
existing Endpoints functionality, Endpoint Slices should include new bits of
|
||||
information such as topology. They will allow for greater scalability and
|
||||
extensibility of network endpoints in your cluster.
|
||||
extensibility of network endpoints in your cluster.
|
||||
@@ -8,14 +8,14 @@ content_template: templates/task
|
||||
{{% capture overview %}}
|
||||
|
||||
This page explains how to upgrade a Kubernetes cluster created with kubeadm from version
|
||||
1.16.x to version 1.17.x, and from version 1.17.x to 1.17.y (where `y > x`).
|
||||
1.15.x to version 1.16.x, and from version 1.16.x to 1.16.y (where `y > x`).
|
||||
|
||||
To see information about upgrading clusters created using older versions of kubeadm,
|
||||
please refer to following pages instead:
|
||||
|
||||
- [Upgrading kubeadm cluster from 1.15 to 1.16](https://v1-16.docs.kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/)
|
||||
- [Upgrading kubeadm cluster from 1.14 to 1.15](https://v1-15.docs.kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade-1-15/)
|
||||
- [Upgrading kubeadm cluster from 1.13 to 1.14](https://v1-15.docs.kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade-1-14/)
|
||||
- [Upgrading kubeadm cluster from 1.12 to 1.13](https://v1-15.docs.kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade-1-13/) or [Upgrading kubeadm HA clusters from v1.12 to v1.13](https://v1-15.docs.kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade-ha-1-13/)
|
||||
|
||||
The upgrade workflow at high level is the following:
|
||||
|
||||
@@ -27,7 +27,7 @@ The upgrade workflow at high level is the following:
|
||||
|
||||
{{% capture prerequisites %}}
|
||||
|
||||
- You need to have a kubeadm Kubernetes cluster running version 1.16.0 or later.
|
||||
- You need to have a kubeadm Kubernetes cluster running version 1.15.0 or later.
|
||||
- [Swap must be disabled](https://serverfault.com/questions/684771/best-way-to-disable-swap-in-linux).
|
||||
- The cluster should use a static control plane and etcd pods or external etcd.
|
||||
- Make sure you read the [release notes](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.16.md) carefully.
|
||||
@@ -47,19 +47,19 @@ The upgrade workflow at high level is the following:
|
||||
|
||||
## Determine which version to upgrade to
|
||||
|
||||
1. Find the latest stable 1.17 version:
|
||||
1. Find the latest stable 1.16 version:
|
||||
|
||||
{{< tabs name="k8s_install_versions" >}}
|
||||
{{% tab name="Ubuntu, Debian or HypriotOS" %}}
|
||||
apt update
|
||||
apt-cache policy kubeadm
|
||||
# find the latest 1.17 version in the list
|
||||
# it should look like 1.17.x-00, where x is the latest patch
|
||||
# find the latest 1.16 version in the list
|
||||
# it should look like 1.16.x-00, where x is the latest patch
|
||||
{{% /tab %}}
|
||||
{{% tab name="CentOS, RHEL or Fedora" %}}
|
||||
yum list --showduplicates kubeadm --disableexcludes=kubernetes
|
||||
# find the latest 1.17 version in the list
|
||||
# it should look like 1.17.x-0, where x is the latest patch
|
||||
# find the latest 1.16 version in the list
|
||||
# it should look like 1.16.x-0, where x is the latest patch
|
||||
{{% /tab %}}
|
||||
{{< /tabs >}}
|
||||
|
||||
@@ -71,14 +71,14 @@ The upgrade workflow at high level is the following:
|
||||
|
||||
{{< tabs name="k8s_install_kubeadm_first_cp" >}}
|
||||
{{% tab name="Ubuntu, Debian or HypriotOS" %}}
|
||||
# replace x in 1.17.x-00 with the latest patch version
|
||||
# replace x in 1.16.x-00 with the latest patch version
|
||||
apt-mark unhold kubeadm && \
|
||||
apt-get update && apt-get install -y kubeadm=1.17.x-00 && \
|
||||
apt-get update && apt-get install -y kubeadm=1.16.x-00 && \
|
||||
apt-mark hold kubeadm
|
||||
{{% /tab %}}
|
||||
{{% tab name="CentOS, RHEL or Fedora" %}}
|
||||
# replace x in 1.17.x-0 with the latest patch version
|
||||
yum install -y kubeadm-1.17.x-0 --disableexcludes=kubernetes
|
||||
# replace x in 1.16.x-0 with the latest patch version
|
||||
yum install -y kubeadm-1.16.x-0 --disableexcludes=kubernetes
|
||||
{{% /tab %}}
|
||||
{{< /tabs >}}
|
||||
|
||||
@@ -91,7 +91,7 @@ The upgrade workflow at high level is the following:
|
||||
1. Drain the control plane node:
|
||||
|
||||
```shell
|
||||
kubectl drain $CP_NODE --ignore-daemonsets
|
||||
kubectl drain $MASTER --ignore-daemonsets
|
||||
```
|
||||
|
||||
1. On the control plane node, run:
|
||||
@@ -109,26 +109,26 @@ The upgrade workflow at high level is the following:
|
||||
[preflight] Running pre-flight checks.
|
||||
[upgrade] Making sure the cluster is healthy:
|
||||
[upgrade] Fetching available versions to upgrade to
|
||||
[upgrade/versions] Cluster version: v1.16.0
|
||||
[upgrade/versions] kubeadm version: v1.17.0
|
||||
[upgrade/versions] Cluster version: v1.15.2
|
||||
[upgrade/versions] kubeadm version: v1.16.0
|
||||
|
||||
Components that must be upgraded manually after you have upgraded the control plane with 'kubeadm upgrade apply':
|
||||
COMPONENT CURRENT AVAILABLE
|
||||
Kubelet 1 x v1.16.0 v1.17.0
|
||||
Kubelet 1 x v1.15.2 v1.16.0
|
||||
|
||||
Upgrade to the latest version in the v1.16 series:
|
||||
|
||||
COMPONENT CURRENT AVAILABLE
|
||||
API Server v1.16.0 v1.17.0
|
||||
Controller Manager v1.16.0 v1.17.0
|
||||
Scheduler v1.16.0 v1.17.0
|
||||
Kube Proxy v1.16.0 v1.17.0
|
||||
CoreDNS 1.6.2 1.6.5
|
||||
Etcd 3.3.15 3.4.3-0
|
||||
API Server v1.15.2 v1.16.0
|
||||
Controller Manager v1.15.2 v1.16.0
|
||||
Scheduler v1.15.2 v1.16.0
|
||||
Kube Proxy v1.15.2 v1.16.0
|
||||
CoreDNS 1.3.1 1.6.2
|
||||
Etcd 3.3.10 3.3.15
|
||||
|
||||
You can now apply the upgrade by executing the following command:
|
||||
|
||||
kubeadm upgrade apply v1.17.0
|
||||
kubeadm upgrade apply v1.16.0
|
||||
|
||||
_____________________________________________________________________
|
||||
```
|
||||
@@ -144,7 +144,7 @@ The upgrade workflow at high level is the following:
|
||||
1. Choose a version to upgrade to, and run the appropriate command. For example:
|
||||
|
||||
```shell
|
||||
sudo kubeadm upgrade apply v1.17.x
|
||||
sudo kubeadm upgrade apply v1.16.x
|
||||
```
|
||||
|
||||
- Replace `x` with the patch version you picked for this upgrade.
|
||||
@@ -157,9 +157,9 @@ The upgrade workflow at high level is the following:
|
||||
[upgrade/config] Making sure the configuration is correct:
|
||||
[upgrade/config] Reading configuration from the cluster...
|
||||
[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
|
||||
[upgrade/version] You have chosen to change the cluster version to "v1.17.0"
|
||||
[upgrade/versions] Cluster version: v1.16.0
|
||||
[upgrade/versions] kubeadm version: v1.17.0
|
||||
[upgrade/version] You have chosen to change the cluster version to "v1.16.0"
|
||||
[upgrade/versions] Cluster version: v1.15.2
|
||||
[upgrade/versions] kubeadm version: v1.16.0
|
||||
[upgrade/confirm] Are you sure you want to proceed with the upgrade? [y/N]: y
|
||||
[upgrade/prepull] Will prepull images for components [kube-apiserver kube-controller-manager kube-scheduler etcd]
|
||||
[upgrade/prepull] Prepulling image for component etcd.
|
||||
@@ -177,7 +177,7 @@ The upgrade workflow at high level is the following:
|
||||
[upgrade/prepull] Prepulled image for component kube-apiserver.
|
||||
[upgrade/prepull] Prepulled image for component kube-scheduler.
|
||||
[upgrade/prepull] Successfully prepulled the images for all the control plane components
|
||||
[upgrade/apply] Upgrading your Static Pod-hosted control plane to version "v1.17.0"...
|
||||
[upgrade/apply] Upgrading your Static Pod-hosted control plane to version "v1.16.0"...
|
||||
Static pod: kube-apiserver-luboitvbox hash: 8d931c2296a38951e95684cbcbe3b923
|
||||
Static pod: kube-controller-manager-luboitvbox hash: 2480bf6982ad2103c05f6764e20f2787
|
||||
Static pod: kube-scheduler-luboitvbox hash: 9b290132363a92652555896288ca3f88
|
||||
@@ -215,8 +215,8 @@ The upgrade workflow at high level is the following:
|
||||
[upgrade/staticpods] Component "kube-scheduler" upgraded successfully!
|
||||
[upgrade/staticpods] Renewing certificate embedded in "admin.conf"
|
||||
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
|
||||
[kubelet] Creating a ConfigMap "kubelet-config-1.17" in namespace kube-system with the configuration for the kubelets in the cluster
|
||||
[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.17" ConfigMap in the kube-system namespace
|
||||
[kubelet] Creating a ConfigMap "kubelet-config-1.16" in namespace kube-system with the configuration for the kubelets in the cluster
|
||||
[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.16" ConfigMap in the kube-system namespace
|
||||
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
|
||||
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
|
||||
[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
|
||||
@@ -224,7 +224,7 @@ The upgrade workflow at high level is the following:
|
||||
[addons] Applied essential addon: CoreDNS
|
||||
[addons] Applied essential addon: kube-proxy
|
||||
|
||||
[upgrade/successful] SUCCESS! Your cluster was upgraded to "v1.17.0". Enjoy!
|
||||
[upgrade/successful] SUCCESS! Your cluster was upgraded to "v1.16.0". Enjoy!
|
||||
|
||||
[upgrade/kubelet] Now that your control plane is upgraded, please proceed with upgrading your kubelets if you haven't already done so.
|
||||
```
|
||||
@@ -240,7 +240,7 @@ The upgrade workflow at high level is the following:
|
||||
1. Uncordon the control plane node
|
||||
|
||||
```shell
|
||||
kubectl uncordon $CP_NODE
|
||||
kubectl uncordon $MASTER
|
||||
```
|
||||
|
||||
### Upgrade additional control plane nodes
|
||||
@@ -265,14 +265,14 @@ Also `sudo kubeadm upgrade plan` is not needed.
|
||||
|
||||
{{< tabs name="k8s_install_kubelet" >}}
|
||||
{{% tab name="Ubuntu, Debian or HypriotOS" %}}
|
||||
# replace x in 1.17.x-00 with the latest patch version
|
||||
# replace x in 1.16.x-00 with the latest patch version
|
||||
apt-mark unhold kubelet kubectl && \
|
||||
apt-get update && apt-get install -y kubelet=1.17.x-00 kubectl=1.17.x-00 && \
|
||||
apt-get update && apt-get install -y kubelet=1.16.x-00 kubectl=1.16.x-00 && \
|
||||
apt-mark hold kubelet kubectl
|
||||
{{% /tab %}}
|
||||
{{% tab name="CentOS, RHEL or Fedora" %}}
|
||||
# replace x in 1.17.x-0 with the latest patch version
|
||||
yum install -y kubelet-1.17.x-0 kubectl-1.17.x-0 --disableexcludes=kubernetes
|
||||
# replace x in 1.16.x-0 with the latest patch version
|
||||
yum install -y kubelet-1.16.x-0 kubectl-1.16.x-0 --disableexcludes=kubernetes
|
||||
{{% /tab %}}
|
||||
{{< /tabs >}}
|
||||
|
||||
@@ -293,14 +293,14 @@ without compromising the minimum required capacity for running your workloads.
|
||||
|
||||
{{< tabs name="k8s_install_kubeadm_worker_nodes" >}}
|
||||
{{% tab name="Ubuntu, Debian or HypriotOS" %}}
|
||||
# replace x in 1.17.x-00 with the latest patch version
|
||||
# replace x in 1.16.x-00 with the latest patch version
|
||||
apt-mark unhold kubeadm && \
|
||||
apt-get update && apt-get install -y kubeadm=1.17.x-00 && \
|
||||
apt-get update && apt-get install -y kubeadm=1.16.x-00 && \
|
||||
apt-mark hold kubeadm
|
||||
{{% /tab %}}
|
||||
{{% tab name="CentOS, RHEL or Fedora" %}}
|
||||
# replace x in 1.17.x-0 with the latest patch version
|
||||
yum install -y kubeadm-1.17.x-0 --disableexcludes=kubernetes
|
||||
# replace x in 1.16.x-0 with the latest patch version
|
||||
yum install -y kubeadm-1.16.x-0 --disableexcludes=kubernetes
|
||||
{{% /tab %}}
|
||||
{{< /tabs >}}
|
||||
|
||||
@@ -334,14 +334,14 @@ without compromising the minimum required capacity for running your workloads.
|
||||
|
||||
{{< tabs name="k8s_kubelet_and_kubectl" >}}
|
||||
{{% tab name="Ubuntu, Debian or HypriotOS" %}}
|
||||
# replace x in 1.17.x-00 with the latest patch version
|
||||
# replace x in 1.16.x-00 with the latest patch version
|
||||
apt-mark unhold kubelet kubectl && \
|
||||
apt-get update && apt-get install -y kubelet=1.17.x-00 kubectl=1.17.x-00 && \
|
||||
apt-get update && apt-get install -y kubelet=1.16.x-00 kubectl=1.16.x-00 && \
|
||||
apt-mark hold kubelet kubectl
|
||||
{{% /tab %}}
|
||||
{{% tab name="CentOS, RHEL or Fedora" %}}
|
||||
# replace x in 1.17.x-0 with the latest patch version
|
||||
yum install -y kubelet-1.17.x-0 kubectl-1.17.x-0 --disableexcludes=kubernetes
|
||||
# replace x in 1.16.x-0 with the latest patch version
|
||||
yum install -y kubelet-1.16.x-0 kubectl-1.16.x-0 --disableexcludes=kubernetes
|
||||
{{% /tab %}}
|
||||
{{< /tabs >}}
|
||||
|
||||
|
||||
@@ -146,28 +146,6 @@ control group (`system.slice` on systemd machines for example).
|
||||
Note that Kubelet **does not** create `--system-reserved-cgroup` if it doesn't
|
||||
exist. Kubelet will fail if an invalid cgroup is specified.
|
||||
|
||||
### Explictly Reserved CPU List
|
||||
{{< feature-state for_k8s_version="v1.17" state="stable" >}}
|
||||
|
||||
- **Kubelet Flag**: `--reserved-cpus=0-3`
|
||||
|
||||
`reserved-cpus` is meant to define an explict CPU set for OS system daemons and
|
||||
kubernetes system daemons. This option is added in 1.17 release. `reserved-cpus`
|
||||
is for systems that do not intent to define seperate top level cgroups for
|
||||
OS system daemons and kubernetes system daemons with regard to cpuset resource.
|
||||
If the Kubelet **does not** have `--system-reserved-cgroup` and `--kube-reserved-cgroup`,
|
||||
the explicit cpuset provided by `reserved-cpus` will take precedence over the CPUs
|
||||
defined by `--kube-reserved` and `--system-reserved` options.
|
||||
|
||||
This option is specifically designed for Telco/NFV use cases where uncontrolled
|
||||
interrupts/timers may impact the workload performance. you can use this option
|
||||
to define the explict cpuset for the system/kubernetes daemons as well as the
|
||||
interrupts/timers, so the rest CPUs on the system can be used exclusively for
|
||||
workloads, with less impact from uncontrolled interrupts/timers. To move the
|
||||
system daemon, kubernetes daemons and interrupts/timers to the explict cpuset
|
||||
defined by this option, other mechanism outside Kubernetes should be used.
|
||||
For example: in Centos, you can do this using the tuned toolset.
|
||||
|
||||
### Eviction Thresholds
|
||||
|
||||
- **Kubelet Flag**: `--eviction-hard=[memory.available<500Mi]`
|
||||
@@ -281,8 +259,4 @@ for `kube-reserved` and `system-reserved`.
|
||||
As of Kubernetes version 1.8, the `storage` key name was changed to `ephemeral-storage`
|
||||
for the alpha release.
|
||||
|
||||
As of Kubernetes version 1.17, you can optionally specify
|
||||
explicit cpuset by `reserved-cpus` as CPUs reserved for OS system
|
||||
daemons/interrupts/timers and Kubernetes daemons.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
---
|
||||
title: Control Topology Management Policies on a node
|
||||
|
||||
reviewers:
|
||||
- ConnorDoyle
|
||||
- klueska
|
||||
@@ -49,9 +48,9 @@ The hint is then stored in the Topology Manager for use by the *Hint Providers*
|
||||
The Topology Manager currently:
|
||||
|
||||
- Works on Nodes with the `static` CPU Manager Policy enabled. See [control CPU Management Policies](/docs/tasks/administer-cluster/cpu-management-policies/)
|
||||
- Works on Pods making CPU requests or Device requests via extended resources
|
||||
- Works on Pods in the `Guaranteed` {{< glossary_tooltip text="QoS class" term_id="qos-class" >}}
|
||||
|
||||
If these conditions are met, Topology Manager will align the requested resources.
|
||||
If these conditions are met, Topology Manager will align CPU and device requests.
|
||||
|
||||
Topology Manager supports four allocation policies. You can set a policy via a Kubelet flag, `--topology-manager-policy`.
|
||||
There are four supported policies:
|
||||
@@ -84,9 +83,6 @@ Using this information, the Topology Manager stores the
|
||||
preferred NUMA Node affinity for that container. If the affinity is not preferred,
|
||||
Topology Manager will reject this pod from the node. This will result in a pod in a `Terminated` state with a pod admission failure.
|
||||
|
||||
Once the pod is in a `Terminated` state, the Kubernetes scheduler will **not** attempt to reschedule the pod. It is recommended to use a ReplicaSet or Deployment to trigger a redeploy of the pod.
|
||||
An external control loop could be also implemented to trigger a redeployment of pods that have the `Topology Affinity` error.
|
||||
|
||||
If the pod is admitted, the *Hint Providers* can then use this information when making the
|
||||
resource allocation decision.
|
||||
|
||||
@@ -99,8 +95,6 @@ If it is, Topology Manager will store this and the *Hint Providers* can then use
|
||||
resource allocation decision.
|
||||
If, however, this is not possible then the Topology Manager will reject the pod from the node. This will result in a pod in a `Terminated` state with a pod admission failure.
|
||||
|
||||
Once the pod is in a `Terminated` state, the Kubernetes scheduler will **not** attempt to reschedule the pod. It is recommended a Deployment with Replicas to trigger a redeploy of the pod.
|
||||
An external control loop could be also implemented to trigger a redeployment of pods that have the `Topology Affinity` error.
|
||||
|
||||
### Pod Interactions with Topology Manager Policies
|
||||
|
||||
@@ -152,36 +146,9 @@ spec:
|
||||
|
||||
This pod runs in the `Guaranteed` QoS class because `requests` are equal to `limits`.
|
||||
|
||||
Topology Manager would consider this Pod. The Topology Manager consults the CPU Manager `static` policy, which returns the topology of available CPUs.
|
||||
Topology Manager also consults Device Manager to discover the topology of available devices for example.com/device.
|
||||
|
||||
```yaml
|
||||
spec:
|
||||
containers:
|
||||
- name: nginx
|
||||
image: nginx
|
||||
resources:
|
||||
limits:
|
||||
example.com/deviceA: "1"
|
||||
example.com/deviceB: "1"
|
||||
requests:
|
||||
example.com/deviceA: "1"
|
||||
example.com/deviceB: "1"
|
||||
```
|
||||
This pod runs in the `BestEffort` QoS class because there are no CPU and memory requests.
|
||||
|
||||
The Topology Manager would consider both of the above pods. The Topology Manager would consult the Hint Providers, which are CPU and Device Manager to get topology hints for the pods.
|
||||
In the case of the `Guaranteed` pod the `static` CPU Manager policy would return hints relating to the CPU request and the Device Manager would send back hints for the requested device.
|
||||
|
||||
In the case of the `BestEffort` pod the CPU Manager would send back the default hint as there is no CPU request and the Device Manager would send back the hints for each of the requested devices.
|
||||
|
||||
Using this information the Topology Manager calculates the optimal hint for the pod and stores this information, which will be used by the Hint Providers when they are making their resource assignments.
|
||||
|
||||
### Known Limitations
|
||||
1. As of K8s 1.16 the Topology Manager is currently only guaranteed to work if a *single* container in the pod spec requires aligned resources. This is due to the hint generation being based on current resource allocations, and all containers in a pod generate hints before any resource allocation has been made. This results in unreliable hints for all but the first container in a pod.
|
||||
*Due to this limitation if multiple pods/containers are considered by Kubelet in quick succession they may not respect the Topology Manager policy.
|
||||
|
||||
2. The maximum number of NUMA nodes that Topology Manager will allow is 8, past this there will be a state explosion when trying to enumerate the possible NUMA affinities and generating their hints.
|
||||
|
||||
3. The scheduler is not topology-aware, so it is possible to be scheduled on a node and then fail on the node due to the Topology Manager.
|
||||
|
||||
Topology Manager will use this information to store the best Topology for this container. In the case of this Pod, CPU and Device Manager will use this stored information at the resource allocation stage.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
@@ -6,12 +6,12 @@ weight: 20
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
{{< feature-state for_k8s_version="v1.17" state="beta" >}}
|
||||
{{< feature-state for_k8s_version="v1.16" state="alpha" >}}
|
||||
|
||||
This page shows how to enable and use the `RunAsUserName` feature for pods and containers that will run on Windows nodes. This feature is meant to be the Windows equivalent of the Linux-specific `runAsUser` feature, allowing users to run the container entrypoints with a different username that their default ones.
|
||||
|
||||
{{< note >}}
|
||||
This feature is in beta. The overall functionality for `RunAsUserName` will not change, but there may be some changes regarding the username validation.
|
||||
Currently this feature is in alpha state. The overall functionality of the feature will not change, but there may be some changes regarding the username validation. Please take this into consideration when testing or adopting this feature.
|
||||
{{< /note >}}
|
||||
|
||||
{{% /capture %}}
|
||||
@@ -20,6 +20,16 @@ This feature is in beta. The overall functionality for `RunAsUserName` will not
|
||||
|
||||
You need to have a Kubernetes cluster and the kubectl command-line tool must be configured to communicate with your cluster. The cluster is expected to have Windows worker nodes where pods with containers running Windows workloads will get scheduled.
|
||||
|
||||
|
||||
### Enable the WindowsRunAsUserName feature gate
|
||||
|
||||
In the alpha state, the `WindowsRunAsUserName` feature gate needs to be enabled on the `kube-apiserver` service. Without it, the `runAsUserName` field will be dropped from the pod's, container's, and init container's SecurityContexts. See [Feature Gates](/docs/reference/command-line-tools-reference/feature-gates/) for an explanation of enabling feature gates. Please make sure `feature-gates=WindowsRunAsUserName=true` parameter exists in the `kube-apiserver` command line.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture steps %}}
|
||||
|
||||
|
||||
## Set the Username for a Pod
|
||||
|
||||
To specify the username with which to execute the Pod's container processes, include the `securityContext` field ([PodSecurityContext](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podsecuritycontext-v1-core) in the Pod specification, and within it, the `windowsOptions` ([WindowsSecurityContextOptions](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#windowssecuritycontextoptions-v1-core) field containing the `runAsUserName` field.
|
||||
|
||||
@@ -11,7 +11,7 @@ weight: 160
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
{{< feature-state state="stable" for_k8s_version="v1.17" >}}
|
||||
{{< feature-state state="beta" >}}
|
||||
|
||||
This page shows how to configure process namespace sharing for a pod. When
|
||||
process namespace sharing is enabled, processes in a container are visible
|
||||
@@ -27,6 +27,9 @@ include debugging utilities like a shell.
|
||||
|
||||
{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
|
||||
|
||||
Process Namespace Sharing is a **beta** feature that is enabled by default. It
|
||||
may be disabled by setting `--feature-gates=PodShareProcessNamespace=false`.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture steps %}}
|
||||
|
||||
@@ -58,41 +58,6 @@ kubectl get pods pod01 -o go-template --template='{{range .status.podIPs}}{{prin
|
||||
a00:100::4
|
||||
```
|
||||
|
||||
You can also validate Pod IPs using the Downward API via the `status.podIPs` fieldPath. The following snippet demonstrates how you can expose the Pod IPs via an environment variable called `MY_POD_IPS` within a container.
|
||||
|
||||
```
|
||||
env:
|
||||
- name: MY_POD_IPS
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: status.podIPs
|
||||
```
|
||||
|
||||
The following command prints the value of the `MY_POD_IPS` environment variable from within a container. The value is a comma separated list that corresponds to the Pod's IPv4 and IPv6 addresses.
|
||||
```shell
|
||||
kubectl exec -it pod01 -- set | grep MY_POD_IPS
|
||||
```
|
||||
```
|
||||
MY_POD_IPS=10.244.1.4,a00:100::4
|
||||
```
|
||||
|
||||
The Pod's IP addresses will also be written to `/etc/hosts` within a container. The following command executes a cat on `/etc/hosts` on a dual stack Pod. From the output you can verify both the IPv4 and IPv6 IP address for the Pod.
|
||||
|
||||
```shell
|
||||
kubectl exec -it pod01 -- cat /etc/hosts
|
||||
```
|
||||
```
|
||||
# Kubernetes-managed hosts file.
|
||||
127.0.0.1 localhost
|
||||
::1 localhost ip6-localhost ip6-loopback
|
||||
fe00::0 ip6-localnet
|
||||
fe00::0 ip6-mcastprefix
|
||||
fe00::1 ip6-allnodes
|
||||
fe00::2 ip6-allrouters
|
||||
10.244.1.4 pod01
|
||||
a00:100::4 pod01
|
||||
```
|
||||
|
||||
## Validate Services
|
||||
|
||||
Create the following Service without the `ipFamily` field set. When this field is not set, the Service gets an IP from the first configured range via `--service-cluster-ip-range` flag on the kube-controller-manager.
|
||||
@@ -158,3 +123,4 @@ my-service ClusterIP fe80:20d::d06b 2001:db8:f100:4002::9d37:c0d7 80:318
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user