whentimeslows/website
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'upstream/main' into dev-1.24

Browse Source
This commit is contained in:
Nate W
2022-02-21 10:36:03 -08:00
parent ad150d9c0b 0c8eab971c
commit e8b19637df
48 changed files with 2470 additions and 387 deletions
Download Patch File Download Diff File Expand all files Collapse all files
content/en/docs/concepts/cluster-administration/addons.md
-1
View File
@@ -21,7 +21,6 @@ This page lists some of the available add-ons and links to their respective inst
* [Canal](https://github.com/tigera/canal/tree/master/k8s-install) unites Flannel and Calico, providing networking and network policy.
* [Cilium](https://github.com/cilium/cilium) is a L3 network and network policy plugin that can enforce HTTP/API/L7 policies transparently. Both routing and overlay/encapsulation mode are supported, and it can work on top of other CNI plugins.
* [CNI-Genie](https://github.com/Huawei-PaaS/CNI-Genie) enables Kubernetes to seamlessly connect to a choice of CNI plugins, such as Calico, Canal, Flannel, Romana, or Weave.
* [Contiv](https://contiv.github.io) provides configurable networking (native L3 using BGP, overlay using vxlan, classic L2, and Cisco-SDN/ACI) for various use cases and a rich policy framework. Contiv project is fully [open sourced](https://github.com/contiv). The [installer](https://github.com/contiv/install) provides both kubeadm and non-kubeadm based installation options.
* [Contrail](https://www.juniper.net/us/en/products-services/sdn/contrail/contrail-networking/), based on [Tungsten Fabric](https://tungsten.io), is an open source, multi-cloud network virtualization and policy management platform. Contrail and Tungsten Fabric are integrated with orchestration systems such as Kubernetes, OpenShift, OpenStack and Mesos, and provide isolation modes for virtual machines, containers/pods and bare metal workloads.
* [Flannel](https://github.com/flannel-io/flannel#deploying-flannel-manually) is an overlay network provider that can be used with Kubernetes.
* [Knitter](https://github.com/ZTE/Knitter/) is a plugin to support multiple network interfaces in a Kubernetes pod.
content/en/docs/concepts/cluster-administration/system-traces.md
+2 -2
View File
@@ -66,7 +66,7 @@ with `--tracing-config-file=<path-to-config>`. This is an example config that re
spans for 1 in 10000 requests, and uses the default OpenTelemetry endpoint:
```yaml
apiVersion: apiserver.config.k8s.io/v1beta1
apiVersion: apiserver.config.k8s.io/v1alpha1
kind: TracingConfiguration
# default value
#endpoint: localhost:4317
@@ -74,7 +74,7 @@ samplingRatePerMillion: 100
```
For more information about the `TracingConfiguration` struct, see
[API server config API (v1beta1)](/docs/reference/config-api/apiserver-config.v1beta1/#apiserver-k8s-io-v1beta1-TracingConfiguration).
[API server config API (v1alpha1)](/docs/reference/config-api/apiserver-config.v1alpha1/#apiserver-k8s-io-v1alpha1-TracingConfiguration).
## Stability
content/en/docs/concepts/configuration/manage-resources-containers.md
+1 -1
View File
@@ -231,7 +231,7 @@ The kubelet reports the resource usage of a Pod as part of the Pod
If optional [tools for monitoring](/docs/tasks/debug-application-cluster/resource-usage-monitoring/)
are available in your cluster, then Pod resource usage can be retrieved either
from the [Metrics API](/docs/tasks/debug-application-cluster/resource-metrics-pipeline/#the-metrics-api)
from the [Metrics API](/docs/tasks/debug-application-cluster/resource-metrics-pipeline/#metrics-api)
directly or from your monitoring tools.
## Local ephemeral storage
content/en/docs/concepts/extend-kubernetes/operator.md
+1
View File
@@ -111,6 +111,7 @@ Operator.
{{% thirdparty-content %}}
* [Charmed Operator Framework](https://juju.is/)
* [Kopf](https://github.com/nolar/kopf) (Kubernetes Operator Pythonic Framework)
* [kubebuilder](https://book.kubebuilder.io/)
* [KubeOps](https://buehler.github.io/dotnet-operator-sdk/) (.NET operator SDK)
* [KUDO](https://kudo.dev/) (Kubernetes Universal Declarative Operator)
content/en/docs/concepts/overview/working-with-objects/kubernetes-objects.md
+1 -1
View File
@@ -63,7 +63,7 @@ One way to create a Deployment using a `.yaml` file like the one above is to use
in the `kubectl` command-line interface, passing the `.yaml` file as an argument. Here's an example:
```shell
kubectl apply -f https://k8s.io/examples/application/deployment.yaml --record
kubectl apply -f https://k8s.io/examples/application/deployment.yaml
```
The output is similar to this:
content/en/docs/concepts/storage/persistent-volumes.md
+1 -1
View File
@@ -339,7 +339,7 @@ If that has happened, or you suspect that it might have, you can retry expansion
size that is within the capacity limits of underlying storage provider. You can monitor status of resize operation by watching `.status.resizeStatus` and events on the PVC.
Note that,
although you can a specify a lower amount of storage than what was requested previously,
although you can specify a lower amount of storage than what was requested previously,
the new value must still be higher than `.status.capacity`.
Kubernetes does not support shrinking a PVC to less than its current size.
{{% /tab %}}
content/en/docs/concepts/storage/storage-classes.md
+3 -3
View File
@@ -49,7 +49,7 @@ metadata:
name: standard
provisioner: kubernetes.io/aws-ebs
parameters:
type: gp2
type: gp3
reclaimPolicy: Retain
allowVolumeExpansion: true
mountOptions:
@@ -271,9 +271,9 @@ parameters:
fsType: ext4
```
* `type`: `io1`, `gp2`, `sc1`, `st1`. See
* `type`: `io1`, `gp2`, `gp3`, `sc1`, `st1`. See
[AWS docs](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSVolumeTypes.html)
for details. Default: `gp2`.
for details. Default: `gp3`.
* `zone` (Deprecated): AWS zone. If neither `zone` nor `zones` is specified, volumes are
generally round-robin-ed across all active zones where Kubernetes cluster
has a node. `zone` and `zones` parameters must not be used at the same time.
content/en/docs/contribute/advanced.md
+1
View File
@@ -86,6 +86,7 @@ Responsibilities for New Contributor Ambassadors include:
- Mentoring new contributors through their first few PRs to the docs repo.
- Helping new contributors create the more complex PRs they need to become Kubernetes members.
- [Sponsoring contributors](/docs/contribute/advanced/#sponsor-a-new-contributor) on their path to becoming Kubernetes members.
- Hosting a monthly meeting to help and mentor new contributors.
Current New Contributor Ambassadors are announced at each SIG-Docs meeting and in the [Kubernetes #sig-docs channel](https://kubernetes.slack.com).
content/en/docs/home/_index.md
+1 -1
View File
@@ -60,7 +60,7 @@ cards:
title: K8s Release Notes
description: If you are installing Kubernetes or upgrading to the newest version, refer to the current release notes.
button: "Download Kubernetes"
button_path: "/docs/setup/release/notes"
button_path: "/releases/download"
- name: about
title: About the documentation
description: This website contains documentation for the current and previous 4 versions of Kubernetes.
content/en/docs/reference/_index.md
+1
View File
@@ -74,6 +74,7 @@ by the API server in a RESTful way though they are essential for a user or an
operator to use or manage a cluster.
* [kube-apiserver configuration (v1alpha1)](/docs/reference/config-api/apiserver-config.v1alpha1/)
* [kube-apiserver configuration (v1)](/docs/reference/config-api/apiserver-config.v1/)
* [kube-apiserver encryption (v1)](/docs/reference/config-api/apiserver-encryption.v1/)
* [kubelet configuration (v1alpha1)](/docs/reference/config-api/kubelet-config.v1alpha1/) and
content/en/docs/reference/command-line-tools-reference/feature-gates.md
+2 -2
View File
@@ -401,7 +401,7 @@ different Kubernetes components.
| `PodShareProcessNamespace` | `true` | Beta | 1.12 | 1.16 |
| `PodShareProcessNamespace` | `true` | GA | 1.17 | - |
| `RequestManagement` | `false` | Alpha | 1.15 | 1.16 |
| `RequestManagement` | - | Derecated | 1.17 | - |
| `RequestManagement` | - | Deprecated | 1.17 | - |
| `ResourceLimitsPriorityFunction` | `false` | Alpha | 1.9 | 1.18 |
| `ResourceLimitsPriorityFunction` | - | Deprecated | 1.19 | - |
| `ResourceQuotaScopeSelectors` | `false` | Alpha | 1.11 | 1.11 |
@@ -816,7 +816,7 @@ Each feature gate is designed for enabling/disabling a specific feature:
and gracefully terminate pods running on the node. See
[Graceful Node Shutdown](/docs/concepts/architecture/nodes/#graceful-node-shutdown)
for more details.
= `GracefulNodeShutdownBasedOnPodPriority`: Enables the kubelet to check Pod priorities
- `GracefulNodeShutdownBasedOnPodPriority`: Enables the kubelet to check Pod priorities
when shutting down a node gracefully.
- `GRPCContainerProbe`: Enables the gRPC probe method for {Liveness,Readiness,Startup}Probe. See [Configure Liveness, Readiness and Startup Probes](/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#define-a-grpc-liveness-probe).
- `HonorPVReclaimPolicy`: Honor persistent volume reclaim policy when it is `Delete` irrespective of PV-PVC deletion ordering.
content/en/docs/reference/command-line-tools-reference/kube-apiserver.md
+1 -1
View File
@@ -946,7 +946,7 @@ kube-apiserver [flags]
<td colspan="2">--service-account-key-file strings</td>
</tr>
<tr>
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>File containing PEM-encoded x509 RSA or ECDSA private or public keys, used to verify ServiceAccount tokens. The specified file can contain multiple keys, and the flag can be specified multiple times with different files. If unspecified, --tls-private-key-file is used. Must be specified when --service-account-signing-key is provided</p></td>
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>File containing PEM-encoded x509 RSA or ECDSA private or public keys, used to verify ServiceAccount tokens. The specified file can contain multiple keys, and the flag can be specified multiple times with different files. If unspecified, --tls-private-key-file is used. Must be specified when --service-account-signing-key-file is provided</p></td>
</tr>
<tr>
content/en/docs/reference/config-api/apiserver-config.v1alpha1.md
+375
View File
@@ -0,0 +1,375 @@
---
title: kube-apiserver Configuration (v1alpha1)
content_type: tool-reference
package: apiserver.k8s.io/v1alpha1
auto_generated: true
---
<p>Package v1alpha1 is the v1alpha1 version of the API.</p>
## Resource Types
- [AdmissionConfiguration](#apiserver-k8s-io-v1alpha1-AdmissionConfiguration)
- [EgressSelectorConfiguration](#apiserver-k8s-io-v1alpha1-EgressSelectorConfiguration)
- [TracingConfiguration](#apiserver-k8s-io-v1alpha1-TracingConfiguration)
## `AdmissionConfiguration` {#apiserver-k8s-io-v1alpha1-AdmissionConfiguration}
<p>AdmissionConfiguration provides versioned configuration for admission controllers.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>apiVersion</code><br/>string</td><td><code>apiserver.k8s.io/v1alpha1</code></td></tr>
<tr><td><code>kind</code><br/>string</td><td><code>AdmissionConfiguration</code></td></tr>
<tr><td><code>plugins</code><br/>
<a href="#apiserver-k8s-io-v1alpha1-AdmissionPluginConfiguration"><code>[]AdmissionPluginConfiguration</code></a>
</td>
<td>
<p>Plugins allows specifying a configuration per admission control plugin.</p>
</td>
</tr>
</tbody>
</table>
## `EgressSelectorConfiguration` {#apiserver-k8s-io-v1alpha1-EgressSelectorConfiguration}
<p>EgressSelectorConfiguration provides versioned configuration for egress selector clients.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>apiVersion</code><br/>string</td><td><code>apiserver.k8s.io/v1alpha1</code></td></tr>
<tr><td><code>kind</code><br/>string</td><td><code>EgressSelectorConfiguration</code></td></tr>
<tr><td><code>egressSelections</code> <B>[Required]</B><br/>
<a href="#apiserver-k8s-io-v1alpha1-EgressSelection"><code>[]EgressSelection</code></a>
</td>
<td>
<p>connectionServices contains a list of egress selection client configurations</p>
</td>
</tr>
</tbody>
</table>
## `TracingConfiguration` {#apiserver-k8s-io-v1alpha1-TracingConfiguration}
<p>TracingConfiguration provides versioned configuration for tracing clients.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>apiVersion</code><br/>string</td><td><code>apiserver.k8s.io/v1alpha1</code></td></tr>
<tr><td><code>kind</code><br/>string</td><td><code>TracingConfiguration</code></td></tr>
<tr><td><code>endpoint</code><br/>
<code>string</code>
</td>
<td>
<p>Endpoint of the collector that's running on the control-plane node.
The APIServer uses the egressType ControlPlane when sending data to the collector.
The syntax is defined in https://github.com/grpc/grpc/blob/master/doc/naming.md.
Defaults to the otlpgrpc default, localhost:4317
The connection is insecure, and does not support TLS.</p>
</td>
</tr>
<tr><td><code>samplingRatePerMillion</code><br/>
<code>int32</code>
</td>
<td>
<p>SamplingRatePerMillion is the number of samples to collect per million spans.
Defaults to 0.</p>
</td>
</tr>
</tbody>
</table>
## `AdmissionPluginConfiguration` {#apiserver-k8s-io-v1alpha1-AdmissionPluginConfiguration}
**Appears in:**
- [AdmissionConfiguration](#apiserver-k8s-io-v1alpha1-AdmissionConfiguration)
<p>AdmissionPluginConfiguration provides the configuration for a single plug-in.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>name</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>Name is the name of the admission controller.
It must match the registered admission plugin name.</p>
</td>
</tr>
<tr><td><code>path</code><br/>
<code>string</code>
</td>
<td>
<p>Path is the path to a configuration file that contains the plugin's
configuration</p>
</td>
</tr>
<tr><td><code>configuration</code><br/>
<a href="https://pkg.go.dev/k8s.io/apimachinery/pkg/runtime#Unknown"><code>k8s.io/apimachinery/pkg/runtime.Unknown</code></a>
</td>
<td>
<p>Configuration is an embedded configuration object to be used as the plugin's
configuration. If present, it will be used instead of the path to the configuration file.</p>
</td>
</tr>
</tbody>
</table>
## `Connection` {#apiserver-k8s-io-v1alpha1-Connection}
**Appears in:**
- [EgressSelection](#apiserver-k8s-io-v1alpha1-EgressSelection)
<p>Connection provides the configuration for a single egress selection client.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>proxyProtocol</code> <B>[Required]</B><br/>
<a href="#apiserver-k8s-io-v1alpha1-ProtocolType"><code>ProtocolType</code></a>
</td>
<td>
<p>Protocol is the protocol used to connect from client to the konnectivity server.</p>
</td>
</tr>
<tr><td><code>transport</code><br/>
<a href="#apiserver-k8s-io-v1alpha1-Transport"><code>Transport</code></a>
</td>
<td>
<p>Transport defines the transport configurations we use to dial to the konnectivity server.
This is required if ProxyProtocol is HTTPConnect or GRPC.</p>
</td>
</tr>
</tbody>
</table>
## `EgressSelection` {#apiserver-k8s-io-v1alpha1-EgressSelection}
**Appears in:**
- [EgressSelectorConfiguration](#apiserver-k8s-io-v1alpha1-EgressSelectorConfiguration)
<p>EgressSelection provides the configuration for a single egress selection client.</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>name</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>name is the name of the egress selection.
Currently supported values are &quot;controlplane&quot;, &quot;master&quot;, &quot;etcd&quot; and &quot;cluster&quot;
The &quot;master&quot; egress selector is deprecated in favor of &quot;controlplane&quot;</p>
</td>
</tr>
<tr><td><code>connection</code> <B>[Required]</B><br/>
<a href="#apiserver-k8s-io-v1alpha1-Connection"><code>Connection</code></a>
</td>
<td>
<p>connection is the exact information used to configure the egress selection</p>
</td>
</tr>
</tbody>
</table>
## `ProtocolType` {#apiserver-k8s-io-v1alpha1-ProtocolType}
(Alias of `string`)
**Appears in:**
- [Connection](#apiserver-k8s-io-v1alpha1-Connection)
<p>ProtocolType is a set of valid values for Connection.ProtocolType</p>
## `TCPTransport` {#apiserver-k8s-io-v1alpha1-TCPTransport}
**Appears in:**
- [Transport](#apiserver-k8s-io-v1alpha1-Transport)
<p>TCPTransport provides the information to connect to konnectivity server via TCP</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>url</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>URL is the location of the konnectivity server to connect to.
As an example it might be &quot;https://127.0.0.1:8131&quot;</p>
</td>
</tr>
<tr><td><code>tlsConfig</code><br/>
<a href="#apiserver-k8s-io-v1alpha1-TLSConfig"><code>TLSConfig</code></a>
</td>
<td>
<p>TLSConfig is the config needed to use TLS when connecting to konnectivity server</p>
</td>
</tr>
</tbody>
</table>
## `TLSConfig` {#apiserver-k8s-io-v1alpha1-TLSConfig}
**Appears in:**
- [TCPTransport](#apiserver-k8s-io-v1alpha1-TCPTransport)
<p>TLSConfig provides the authentication information to connect to konnectivity server
Only used with TCPTransport</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>caBundle</code><br/>
<code>string</code>
</td>
<td>
<p>caBundle is the file location of the CA to be used to determine trust with the konnectivity server.
Must be absent/empty if TCPTransport.URL is prefixed with http://
If absent while TCPTransport.URL is prefixed with https://, default to system trust roots.</p>
</td>
</tr>
<tr><td><code>clientKey</code><br/>
<code>string</code>
</td>
<td>
<p>clientKey is the file location of the client key to be used in mtls handshakes with the konnectivity server.
Must be absent/empty if TCPTransport.URL is prefixed with http://
Must be configured if TCPTransport.URL is prefixed with https://</p>
</td>
</tr>
<tr><td><code>clientCert</code><br/>
<code>string</code>
</td>
<td>
<p>clientCert is the file location of the client certificate to be used in mtls handshakes with the konnectivity server.
Must be absent/empty if TCPTransport.URL is prefixed with http://
Must be configured if TCPTransport.URL is prefixed with https://</p>
</td>
</tr>
</tbody>
</table>
## `Transport` {#apiserver-k8s-io-v1alpha1-Transport}
**Appears in:**
- [Connection](#apiserver-k8s-io-v1alpha1-Connection)
<p>Transport defines the transport configurations we use to dial to the konnectivity server</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>tcp</code><br/>
<a href="#apiserver-k8s-io-v1alpha1-TCPTransport"><code>TCPTransport</code></a>
</td>
<td>
<p>TCP is the TCP configuration for communicating with the konnectivity server via TCP
ProxyProtocol of GRPC is not supported with TCP transport at the moment
Requires at least one of TCP or UDS to be set</p>
</td>
</tr>
<tr><td><code>uds</code><br/>
<a href="#apiserver-k8s-io-v1alpha1-UDSTransport"><code>UDSTransport</code></a>
</td>
<td>
<p>UDS is the UDS configuration for communicating with the konnectivity server via UDS
Requires at least one of TCP or UDS to be set</p>
</td>
</tr>
</tbody>
</table>
## `UDSTransport` {#apiserver-k8s-io-v1alpha1-UDSTransport}
**Appears in:**
- [Transport](#apiserver-k8s-io-v1alpha1-Transport)
<p>UDSTransport provides the information to connect to konnectivity server via UDS</p>
<table class="table">
<thead><tr><th width="30%">Field</th><th>Description</th></tr></thead>
<tbody>
<tr><td><code>udsName</code> <B>[Required]</B><br/>
<code>string</code>
</td>
<td>
<p>UDSName is the name of the unix domain socket to connect to konnectivity server
This does not use a unix:// prefix. (Eg: /etc/srv/kubernetes/konnectivity-server/konnectivity-server.socket)</p>
</td>
</tr>
</tbody>
</table>
content/en/docs/reference/labels-annotations-taints.md
+9 -1
View File
@@ -72,6 +72,14 @@ This annotation is a best guess at why something was changed.
It is populated when adding `--record` to a `kubectl` command that may change an object.
## kubernetes.io/description {#description}
Example: `kubernetes.io/description: "Description of K8s object."`
Used on: All Objects
This annotation is used for describing specific behaviour of given object.
## controller.kubernetes.io/pod-deletion-cost {#pod-deletion-cost}
Example: `controller.kubernetes.io/pod-deletion-cost=10`
@@ -464,4 +472,4 @@ This annotation has been deprecated since Kubernetes v1.19 and will become non-f
The tutorial [Restrict a Container's Syscalls with seccomp](/docs/tutorials/clusters/seccomp/) takes
you through the steps you follow to apply a seccomp profile to a Pod or to one of
its containers. That tutorial covers the supported mechanism for configuring seccomp in Kubernetes,
based on setting `securityContext` within the Pod's `.spec`.
based on setting `securityContext` within the Pod's `.spec`.
content/en/docs/reference/node/topics-on-dockershim-and-cri-compatible-runtimes.md
+1 -1
View File
@@ -12,7 +12,7 @@ with that removal.
## Kubernetes project
* Kubernetes blog: [Dockershim Deprecation FAQ](/blog/2020/12/02/dockershim-faq/) (originally published 2020/12/02)
* Kubernetes blog: [Dockershim Removal FAQ](/blog/2022/02/17/dockershim-faq/) (originally published 2022/02/17)
* Kubernetes blog: [Kubernetes is Moving on From Dockershim: Commitments and Next Steps](/blog/2022/01/07/kubernetes-is-moving-on-from-dockershim/) (published 2022/01/07)
content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md
+11
View File
@@ -327,6 +327,17 @@ In a cluster that includes Windows nodes, you can use the following types of Ser
* `LoadBalancer`
* `ExternalName`
{{< warning >}}
There are known issue with NodePort services on overlay networking, if the target destination node is running Windows Server 2022.
To avoid the issue entirely, you can configure the service with `externalTrafficPolicy: Local`.
There are known issues with pod to pod connectivity on l2bridge network on Windows Server 2022 with KB5005619 or higher installed.
To workaround the issue and restore pod-pod connectivity, you can disable the WinDSR feature in kube-proxy.
These issues require OS fixes.
Please follow https://github.com/microsoft/Windows-Containers/issues/204 for updates.
{{< /warning >}}
Windows container networking differs in some important ways from Linux networking.
The [Microsoft documentation for Windows Container Networking](https://docs.microsoft.com/en-us/virtualization/windowscontainers/container-networking/architecture) provides
additional details and background.
content/en/docs/tasks/administer-cluster/manage-resources/memory-default-namespace.md
+1 -1
View File
@@ -16,7 +16,7 @@ A Kubernetes cluster can be divided into namespaces. Once you have a namespace t
has a default memory
[limit](/docs/concepts/configuration/manage-resources-containers/#requests-and-limits),
and you then try to create a Pod with a container that does not specify its own memory
limit its own memory limit, then the
limit, then the
{{< glossary_tooltip text="control plane" term_id="control-plane" >}} assigns the default
memory limit to that container.
content/en/docs/tasks/administer-cluster/migrating-from-dockershim/_index.md
+2 -1
View File
@@ -11,7 +11,8 @@ dockershim to other container runtimes.
Since the announcement of [dockershim deprecation](/blog/2020/12/08/kubernetes-1-20-release-announcement/#dockershim-deprecation)
in Kubernetes 1.20, there were questions on how this will affect various workloads and Kubernetes
installations. You can find this blog post useful to understand the problem better: [Dockershim Deprecation FAQ](/blog/2020/12/02/dockershim-faq/)
installations. Our [Dockershim Removal FAQ](/blog/2022/02/17/dockershim-faq/) is there to help you
to understand the problem better.
It is recommended to migrate from dockershim to alternative container runtimes.
Check out [container runtimes](/docs/setup/production-environment/container-runtimes/)
content/en/docs/tasks/debug-application-cluster/resource-metrics-pipeline.md
+179 -46
View File
@@ -2,75 +2,208 @@
reviewers:
- fgrzadkowski
- piosz
title: Resource metrics pipeline
title: Resource metrics pipeline
content_type: concept
---
<!-- overview -->
Resource usage metrics, such as container CPU and memory usage,
are available in Kubernetes through the Metrics API. These metrics can be accessed either directly
by the user with the `kubectl top` command, or by a controller in the cluster, for example
Horizontal Pod Autoscaler, to make decisions.
For Kubernetes, the _Metrics API_ offers a basic set of metrics to support automatic scaling and similar use cases.
This API makes information available about resource usage for node and pod, including metrics for CPU and memory.
If you deploy the Metrics API into your cluster, clients of the Kubernetes API can then query for this information, and
you can use Kubernetes' access control mechanisms to manage permissions to do so.
<!-- body -->
The [HorizontalPodAutoscaler](/docs/tasks/run-application/horizontal-pod-autoscale/) (HPA) and [VerticalPodAutoscaler](https://github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler#readme) (VPA) use data from the metrics API to adjust workload replicas and resources to meet customer demand.
## The Metrics API
Through the Metrics API, you can get the amount of resource currently used
by a given node or a given pod. This API doesn't store the metric values,
so it's not possible, for example, to get the amount of resources used by a
given node 10 minutes ago.
The API is no different from any other API:
- it is discoverable through the same endpoint as the other Kubernetes APIs under the path: `/apis/metrics.k8s.io/`
- it offers the same security, scalability, and reliability guarantees
The API is defined in [k8s.io/metrics](https://github.com/kubernetes/metrics/blob/master/pkg/apis/metrics/v1beta1/types.go)
repository. You can find more information about the API there.
You can also view the resource metrics using the [`kubectl top`](https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#top) command.
{{< note >}}
The API requires the metrics server to be deployed in the cluster. Otherwise it will be not available.
The Metrics API, and the metrics pipeline that it enables, only offers the minimum
CPU and memory metrics to enable automatic scaling using HPA and / or VPA.
If you would like to provide a more complete set of metrics, you can complement
the simpler Metrics API by deploying a second
[metrics pipeline](/docs/tasks/debug-application-cluster/resource-usage-monitoring/#full-metrics-pipeline)
that uses the _Custom Metrics API_.
{{< /note >}}
## Measuring Resource Usage
Figure 1 illustrates the architecture of the resource metrics pipeline.
{{< mermaid >}}
flowchart RL
subgraph cluster[Cluster]
direction RL
S[ <br><br> ]
A[Metrics-<br>Server]
subgraph B[Nodes]
direction TB
D[cAdvisor] --> C[kubelet]
E[Container<br>runtime] --> D
E1[Container<br>runtime] --> D
P[pod data] -.- C
end
L[API<br>server]
W[HPA]
C ---->|Summary<br>API| A -->|metrics<br>API| L --> W
end
L ---> K[kubectl<br>top]
classDef box fill:#fff,stroke:#000,stroke-width:1px,color:#000;
class W,B,P,K,cluster,D,E,E1 box
classDef spacewhite fill:#ffffff,stroke:#fff,stroke-width:0px,color:#000
class S spacewhite
classDef k8s fill:#326ce5,stroke:#fff,stroke-width:1px,color:#fff;
class A,L,C k8s
{{< /mermaid >}}
Figure 1. Resource Metrics Pipeline
The architecture components, from right to left in the figure, consist of the following:
* [cAdvisor](https://github.com/google/cadvisor): Daemon for collecting, aggregating and exposing container metrics included in Kubelet.
* [kubelet](/docs/concepts/overview/components/#kubelet): Node agent for managing container resources. Resource metrics are accessible using the `/metrics/resource` and `/stats` kubelet API endpoints.
* [Summary API](#summary-api-source): API provided by the kubelet for discovering and retrieving per-node summarized stats available through the `/stats` endpoint.
* [metrics-server](#metrics-server): Cluster addon component that collects and aggregates resource metrics pulled from each kubelet. The API server serves Metrics API for use by HPA, VPA, and by the `kubectl top` command. Metrics Server is a reference implementation of the Metrics API.
* [Metrics API](#metrics-api): Kubernetes API supporting access to CPU and memory used for workload autoscaling. To make this work in your cluster, you need an API extension server that provides the Metrics API.
{{< note >}}
cAdvisor supports reading metrics from cgroups, which works with typical container runtimes on Linux.
If you use a container runtime that uses another resource isolation mechanism, for example virtualization, then that container runtime must support [CRI Container Metrics](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-node/cri-container-stats.md) in order for metrics to be available to the kubelet.
{{< /note >}}
<!-- body -->
## Metrics API
The metrics-server implements the Metrics API. This API allows you to access CPU and memory usage for the nodes and pods in your cluster. Its primary role is to feed resource usage metrics to K8s autoscaler components.
Here is an example of the Metrics API request for a `minikube` node piped through `jq` for easier reading:
```shell
kubectl get --raw "/apis/metrics.k8s.io/v1beta1/nodes/minikube" | jq '.'
```
Here is the same API call using `curl`:
```shell
curl http://localhost:8080/apis/metrics.k8s.io/v1beta1/nodes/minikube
```
Sample reply:
```json
{
"kind": "NodeMetrics",
"apiVersion": "metrics.k8s.io/v1beta1",
"metadata": {
"name": "minikube",
"selfLink": "/apis/metrics.k8s.io/v1beta1/nodes/minikube",
"creationTimestamp": "2022-01-27T18:48:43Z"
},
"timestamp": "2022-01-27T18:48:33Z",
"window": "30s",
"usage": {
"cpu": "487558164n",
"memory": "732212Ki"
}
}
```
Here is an example of the Metrics API request for a `kube-scheduler-minikube` pod contained in the `kube-system` namespace and piped through `jq` for easier reading:
```shell
kubectl get --raw "/apis/metrics.k8s.io/v1beta1/namespaces/kube-system/pods/kube-scheduler-minikube" | jq '.'
```
Here is the same API call using `curl`:
```shell
curl http://localhost:8080/apis/metrics.k8s.io/v1beta1/namespaces/kube-system/pods/kube-scheduler-minikube
```
Sample reply:
```json
{
"kind": "PodMetrics",
"apiVersion": "metrics.k8s.io/v1beta1",
"metadata": {
"name": "kube-scheduler-minikube",
"namespace": "kube-system",
"selfLink": "/apis/metrics.k8s.io/v1beta1/namespaces/kube-system/pods/kube-scheduler-minikube",
"creationTimestamp": "2022-01-27T19:25:00Z"
},
"timestamp": "2022-01-27T19:24:31Z",
"window": "30s",
"containers": [
{
"name": "kube-scheduler",
"usage": {
"cpu": "9559630n",
"memory": "22244Ki"
}
}
]
}
```
The Metrics API is defined in the [k8s.io/metrics](https://github.com/kubernetes/metrics) repository. You must enable the [API aggregation layer](/docs/tasks/extend-kubernetes/configure-aggregation-layer/) and register an [APIService](/docs/reference/kubernetes-api/cluster-resources/api-service-v1/) for the `metrics.k8s.io` API.
To learn more about the Metrics API, see [resource metrics API design](https://github.com/kubernetes/design-proposals-archive/blob/main/instrumentation/resource-metrics-api.md), the [metrics-server repository](https://github.com/kubernetes-sigs/metrics-server) and the [resource metrics API](https://github.com/kubernetes/metrics#resource-metrics-api).
{{< note >}} You must deploy the metrics-server or alternative adapter that serves the Metrics API to be able to access it. {{< /note >}}
## Measuring resource usage
### CPU
CPU is reported as the average usage, in
[CPU cores](/docs/concepts/configuration/manage-resources-containers/#meaning-of-cpu),
over a period of time. This value is derived by taking a rate over a cumulative CPU counter
provided by the kernel (in both Linux and Windows kernels).
The kubelet chooses the window for the rate calculation.
CPU is reported as the average core usage measured in cpu units. One cpu, in Kubernetes, is equivalent to 1 vCPU/Core for cloud providers, and 1 hyper-thread on bare-metal Intel processors.
This value is derived by taking a rate over a cumulative CPU counter provided by the kernel (in both Linux and Windows kernels). The time window used to calculate CPU is shown under window field in Metrics API.
To learn more about how Kubernetes allocates and measures CPU resources, see [meaning of CPU](/docs/concepts/configuration/manage-compute-resources-container/#meaning-of-cpu).
### Memory
Memory is reported as the working set, in bytes, at the instant the metric was collected.
In an ideal world, the "working set" is the amount of memory in-use that cannot be freed under memory pressure.
However, calculation of the working set varies by host OS, and generally makes heavy use of heuristics to produce an estimate.
It includes all anonymous (non-file-backed) memory since Kubernetes does not support swap.
The metric typically also includes some cached (file-backed) memory, because the host OS cannot always reclaim such pages.
Memory is reported as the working set, measured in bytes, at the instant the metric was collected.
In an ideal world, the "working set" is the amount of memory in-use that cannot be freed under memory pressure. However, calculation of the working set varies by host OS, and generally makes heavy use of heuristics to produce an estimate.
The Kubernetes model for a container's working set expects that the container runtime counts anonymous memory associated with the container in question. The working set metric typically also includes some cached (file-backed) memory, because the host OS cannot always reclaim pages.
To learn more about how Kubernetes allocates and measures memory resources, see [meaning of memory](/docs/concepts/configuration/manage-compute-resources-container/#meaning-of-memory).
## Metrics Server
[Metrics Server](https://github.com/kubernetes-sigs/metrics-server) is a cluster-wide aggregator of resource usage data.
By default, it is deployed in clusters created by `kube-up.sh` script
as a Deployment object. If you use a different Kubernetes setup mechanism, you can deploy it using the provided
[deployment components.yaml](https://github.com/kubernetes-sigs/metrics-server/releases) file.
The metrics-server fetches resource metrics from the kubelets and exposes them in the Kubernetes API server through the Metrics API for use by the HPA and VPA. You can also view these metrics using the `kubectl top` command.
Metrics Server collects metrics from the Summary API, exposed by
[Kubelet](/docs/reference/command-line-tools-reference/kubelet/) on each node, and is registered with the main API server via
[Kubernetes aggregator](/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation/).
The metrics-server uses the Kubernetes API to track nodes and pods in your cluster. The metrics-server queries each node over HTTP to fetch metrics. The metrics-server also builds an internal view of pod metadata, and keeps a cache of pod health. That cached pod health information is available via the extension API that the metrics-server makes available.
Learn more about the metrics server in
[the design doc](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/instrumentation/metrics-server.md).
For example with an HPA query, the metrics-server needs to identify which pods fulfill the label selectors in the deployment.
### Summary API Source
The [Kubelet](/docs/reference/command-line-tools-reference/kubelet/) gathers stats at node, volume, pod and container level, and emits their statistics in
The metrics-server calls the [kubelet](/docs/reference/command-line-tools-reference/kubelet/) API to collect metrics from each node. Depending on the metrics-server version it uses:
* Metrics resource endpoint `/metrics/resource` in version v0.6.0+ or
* Summary API endpoint `/stats/summary` in older versions
To learn more about the metrics-server, see the [metrics-server repository](https://github.com/kubernetes-sigs/metrics-server).
You can also check out the following:
* [metrics-server design](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/instrumentation/metrics-server.md)
* [metrics-server FAQ](https://github.com/kubernetes-sigs/metrics-server/blob/master/FAQ.md)
* [metrics-server known issues](https://github.com/kubernetes-sigs/metrics-server/blob/master/KNOWN_ISSUES.md)
* [metrics-server releases](https://github.com/kubernetes-sigs/metrics-server/releases)
* [Horizontal Pod Autoscaling](/docs/tasks/run-application/horizontal-pod-autoscale/)
### Summary API source
The [kubelet](/docs/reference/command-line-tools-reference/kubelet/) gathers stats at the node, volume, pod and container level, and emits this information in
the [Summary API](https://github.com/kubernetes/kubernetes/blob/7d309e0104fedb57280b261e5677d919cb2a0e2d/staging/src/k8s.io/kubelet/pkg/apis/stats/v1alpha1/types.go)
for consumers to read.
Pre-1.23, these resources have been primarily gathered from [cAdvisor](https://github.com/google/cadvisor). However, in 1.23 with the
introduction of the `PodAndContainerStatsFromCRI` FeatureGate, container and pod level stats can be gathered by the CRI implementation.
Note: this also requires support from the CRI implementations (containerd >= 1.6.0, CRI-O >= 1.23.0).
Here is an example of a Summary API request for a `minikube` node:
```shell
kubectl get --raw "/api/v1/nodes/minikube/proxy/stats/summary"
```
Here is the same API call using `curl`:
```shell
curl http://localhost:8080/api/v1/nodes/minikube/proxy/stats/summary
```
{{< note >}}
The summary API `/stats/summary` endpoint will be replaced by the `/metrics/resource` endpoint beginning with metrics-server 0.6.x.
{{< /note >}}