Merge remote-tracking branch 'upstream/main' into dev-1.24

This commit is contained in:
Nate W
2022-03-28 08:48:23 -07:00
189 changed files with 7876 additions and 5343 deletions
@@ -205,35 +205,10 @@ See documentation for other libraries for how they authenticate.
## Accessing the API from a Pod
When accessing the API from a pod, locating and authenticating
to the apiserver are somewhat different.
to the API server are somewhat different.
The recommended way to locate the apiserver within the pod is with
the `kubernetes.default.svc` DNS name, which resolves to a Service IP which in turn
will be routed to an apiserver.
The recommended way to authenticate to the apiserver is with a
[service account](/docs/tasks/configure-pod-container/configure-service-account/) credential. By kube-system, a pod
is associated with a service account, and a credential (token) for that
service account is placed into the filesystem tree of each container in that pod,
at `/var/run/secrets/kubernetes.io/serviceaccount/token`.
If available, a certificate bundle is placed into the filesystem tree of each
container at `/var/run/secrets/kubernetes.io/serviceaccount/ca.crt`, and should be
used to verify the serving certificate of the apiserver.
Finally, the default namespace to be used for namespaced API operations is placed in a file
at `/var/run/secrets/kubernetes.io/serviceaccount/namespace` in each container.
From within a pod the recommended ways to connect to API are:
- Run `kubectl proxy` in a sidecar container in the pod, or as a background
process within the container. This proxies the
Kubernetes API to the localhost interface of the pod, so that other processes
in any container of the pod can access it.
- Use the Go client library, and create a client using the `rest.InClusterConfig()` and `kubernetes.NewForConfig()` functions.
They handle locating and authenticating to the apiserver. [example](https://git.k8s.io/client-go/examples/in-cluster-client-configuration/main.go)
In each case, the credentials of the pod are used to communicate securely with the apiserver.
Please check [Accessing the API from within a Pod](/docs/tasks/run-application/access-api-from-pod/)
for more details.
## Accessing services running on the cluster
@@ -240,13 +240,13 @@ The following manifest defines an Ingress that sends traffic to your Service via
following lines at the end:
```yaml
- path: /v2
pathType: Prefix
backend:
service:
name: web2
port:
number: 8080
- path: /v2
pathType: Prefix
backend:
service:
name: web2
port:
number: 8080
```
1. Apply the changes:
@@ -7,14 +7,10 @@ content_type: task
This page shows how to change the reclaim policy of a Kubernetes
PersistentVolume.
## {{% heading "prerequisites" %}}
{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
<!-- steps -->
## Why change reclaim policy of a PersistentVolume
@@ -33,55 +29,58 @@ Released phase, where all of its data can be manually recovered.
1. List the PersistentVolumes in your cluster:
```shell
kubectl get pv
```
```shell
kubectl get pv
```
The output is similar to this:
The output is similar to this:
NAME CAPACITY ACCESSMODES RECLAIMPOLICY STATUS CLAIM STORAGECLASS REASON AGE
pvc-b6efd8da-b7b5-11e6-9d58-0ed433a7dd94 4Gi RWO Delete Bound default/claim1 manual 10s
pvc-b95650f8-b7b5-11e6-9d58-0ed433a7dd94 4Gi RWO Delete Bound default/claim2 manual 6s
pvc-bb3ca71d-b7b5-11e6-9d58-0ed433a7dd94 4Gi RWO Delete Bound default/claim3 manual 3s
```none
NAME CAPACITY ACCESSMODES RECLAIMPOLICY STATUS CLAIM STORAGECLASS REASON AGE
pvc-b6efd8da-b7b5-11e6-9d58-0ed433a7dd94 4Gi RWO Delete Bound default/claim1 manual 10s
pvc-b95650f8-b7b5-11e6-9d58-0ed433a7dd94 4Gi RWO Delete Bound default/claim2 manual 6s
pvc-bb3ca71d-b7b5-11e6-9d58-0ed433a7dd94 4Gi RWO Delete Bound default/claim3 manual 3s
```
This list also includes the name of the claims that are bound to each volume
This list also includes the name of the claims that are bound to each volume
for easier identification of dynamically provisioned volumes.
1. Choose one of your PersistentVolumes and change its reclaim policy:
```shell
kubectl patch pv <your-pv-name> -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}'
```
```shell
kubectl patch pv <your-pv-name> -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}'
```
where `<your-pv-name>` is the name of your chosen PersistentVolume.
where `<your-pv-name>` is the name of your chosen PersistentVolume.
{{< note >}}
On Windows, you must _double_ quote any JSONPath template that contains spaces (not single quote as shown above for bash). This in turn means that you must use a single quote or escaped double quote around any literals in the template. For example:
{{< note >}}
On Windows, you must _double_ quote any JSONPath template that contains spaces (not single
quote as shown above for bash). This in turn means that you must use a single quote or escaped
double quote around any literals in the template. For example:
```cmd
kubectl patch pv <your-pv-name> -p "{\"spec\":{\"persistentVolumeReclaimPolicy\":\"Retain\"}}"
```
{{< /note >}}
```cmd
kubectl patch pv <your-pv-name> -p "{\"spec\":{\"persistentVolumeReclaimPolicy\":\"Retain\"}}"
```
{{< /note >}}
1. Verify that your chosen PersistentVolume has the right policy:
```shell
kubectl get pv
```
```shell
kubectl get pv
```
The output is similar to this:
NAME CAPACITY ACCESSMODES RECLAIMPOLICY STATUS CLAIM STORAGECLASS REASON AGE
pvc-b6efd8da-b7b5-11e6-9d58-0ed433a7dd94 4Gi RWO Delete Bound default/claim1 manual 40s
pvc-b95650f8-b7b5-11e6-9d58-0ed433a7dd94 4Gi RWO Delete Bound default/claim2 manual 36s
pvc-bb3ca71d-b7b5-11e6-9d58-0ed433a7dd94 4Gi RWO Retain Bound default/claim3 manual 33s
In the preceding output, you can see that the volume bound to claim
`default/claim3` has reclaim policy `Retain`. It will not be automatically
deleted when a user deletes claim `default/claim3`.
The output is similar to this:
```none
NAME CAPACITY ACCESSMODES RECLAIMPOLICY STATUS CLAIM STORAGECLASS REASON AGE
pvc-b6efd8da-b7b5-11e6-9d58-0ed433a7dd94 4Gi RWO Delete Bound default/claim1 manual 40s
pvc-b95650f8-b7b5-11e6-9d58-0ed433a7dd94 4Gi RWO Delete Bound default/claim2 manual 36s
pvc-bb3ca71d-b7b5-11e6-9d58-0ed433a7dd94 4Gi RWO Retain Bound default/claim3 manual 33s
```
In the preceding output, you can see that the volume bound to claim
`default/claim3` has reclaim policy `Retain`. It will not be automatically
deleted when a user deletes claim `default/claim3`.
## {{% heading "whatsnext" %}}
@@ -91,8 +90,8 @@ kubectl patch pv <your-pv-name> -p "{\"spec\":{\"persistentVolumeReclaimPolicy\"
### References {#reference}
* {{< api-reference page="config-and-storage-resources/persistent-volume-v1" >}}
* Pay attention to the `.spec.persistentVolumeReclaimPolicy` [field](https://kubernetes.io/docs/reference/kubernetes-api/config-and-storage-resources/persistent-volume-v1/#PersistentVolumeSpec) of PersistentVolume.
* Pay attention to the `.spec.persistentVolumeReclaimPolicy`
[field](/docs/reference/kubernetes-api/config-and-storage-resources/persistent-volume-v1/#PersistentVolumeSpec)
of PersistentVolume.
* {{< api-reference page="config-and-storage-resources/persistent-volume-claim-v1" >}}
@@ -68,7 +68,7 @@ pod/nginx-701339712-e0qfq 1/1 Running 0 35s
You should be able to access the new `nginx` service from other Pods. To access the `nginx` Service from another Pod in the `default` namespace, start a busybox container:
```console
kubectl run busybox --rm -ti --image=busybox -- /bin/sh
kubectl run busybox --rm -ti --image=busybox:1.28 -- /bin/sh
```
In your shell, run the following command:
@@ -111,7 +111,7 @@ networkpolicy.networking.k8s.io/access-nginx created
When you attempt to access the `nginx` Service from a Pod without the correct labels, the request times out:
```console
kubectl run busybox --rm -ti --image=busybox -- /bin/sh
kubectl run busybox --rm -ti --image=busybox:1.28 -- /bin/sh
```
In your shell, run the command:
@@ -130,7 +130,7 @@ wget: download timed out
You can create a Pod with the correct labels to see that the request is allowed:
```console
kubectl run busybox --rm -ti --labels="access=true" --image=busybox -- /bin/sh
kubectl run busybox --rm -ti --labels="access=true" --image=busybox:1.28 -- /bin/sh
```
In your shell, run the command:
@@ -4,31 +4,27 @@ content_type: task
---
<!-- overview -->
This page shows how to configure and enable the ip-masq-agent.
This page shows how to configure and enable the `ip-masq-agent`.
## {{% heading "prerequisites" %}}
{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
<!-- discussion -->
## IP Masquerade Agent User Guide
The ip-masq-agent configures iptables rules to hide a pod's IP address behind the cluster node's IP address. This is typically done when sending traffic to destinations outside the cluster's pod [CIDR](https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing) range.
The `ip-masq-agent` configures iptables rules to hide a pod's IP address behind the cluster node's IP address. This is typically done when sending traffic to destinations outside the cluster's pod [CIDR](https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing) range.
### **Key Terms**
* **NAT (Network Address Translation)**
Is a method of remapping one IP address to another by modifying either the source and/or destination address information in the IP header. Typically performed by a device doing IP routing.
* **Masquerading**
A form of NAT that is typically used to perform a many to one address translation, where multiple source IP addresses are masked behind a single address, which is typically the device doing the IP routing. In Kubernetes this is the Node's IP address.
* **CIDR (Classless Inter-Domain Routing)**
Based on the variable-length subnet masking, allows specifying arbitrary-length prefixes. CIDR introduced a new method of representation for IP addresses, now commonly known as **CIDR notation**, in which an address or routing prefix is written with a suffix indicating the number of bits of the prefix, such as 192.168.2.0/24.
* **Link Local**
A link-local address is a network address that is valid only for communications within the network segment or the broadcast domain that the host is connected to. Link-local addresses for IPv4 are defined in the address block 169.254.0.0/16 in CIDR notation.
* **NAT (Network Address Translation)**
Is a method of remapping one IP address to another by modifying either the source and/or destination address information in the IP header. Typically performed by a device doing IP routing.
* **Masquerading**
A form of NAT that is typically used to perform a many to one address translation, where multiple source IP addresses are masked behind a single address, which is typically the device doing the IP routing. In Kubernetes this is the Node's IP address.
* **CIDR (Classless Inter-Domain Routing)**
Based on the variable-length subnet masking, allows specifying arbitrary-length prefixes. CIDR introduced a new method of representation for IP addresses, now commonly known as **CIDR notation**, in which an address or routing prefix is written with a suffix indicating the number of bits of the prefix, such as 192.168.2.0/24.
* **Link Local**
A link-local address is a network address that is valid only for communications within the network segment or the broadcast domain that the host is connected to. Link-local addresses for IPv4 are defined in the address block 169.254.0.0/16 in CIDR notation.
The ip-masq-agent configures iptables rules to handle masquerading node/pod IP addresses when sending traffic to destinations outside the cluster node's IP and the Cluster IP range. This essentially hides pod IP addresses behind the cluster node's IP address. In some environments, traffic to "external" addresses must come from a known machine address. For example, in Google Cloud, any traffic to the internet must come from a VM's IP. When containers are used, as in Google Kubernetes Engine, the Pod IP will be rejected for egress. To avoid this, we must hide the Pod IP behind the VM's own IP address - generally known as "masquerade". By default, the agent is configured to treat the three private IP ranges specified by [RFC 1918](https://tools.ietf.org/html/rfc1918) as non-masquerade [CIDR](https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing). These ranges are 10.0.0.0/8, 172.16.0.0/12, and 192.168.0.0/16. The agent will also treat link-local (169.254.0.0/16) as a non-masquerade CIDR by default. The agent is configured to reload its configuration from the location */etc/config/ip-masq-agent* every 60 seconds, which is also configurable.
@@ -36,14 +32,20 @@ The ip-masq-agent configures iptables rules to handle masquerading node/pod IP a
The agent configuration file must be written in YAML or JSON syntax, and may contain three optional keys:
* **nonMasqueradeCIDRs:** A list of strings in [CIDR](https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing) notation that specify the non-masquerade ranges.
* **masqLinkLocal:** A Boolean (true / false) which indicates whether to masquerade traffic to the link local prefix 169.254.0.0/16. False by default.
* **resyncInterval:** A time interval at which the agent attempts to reload config from disk. For example: '30s', where 's' means seconds, 'ms' means milliseconds, etc...
* `nonMasqueradeCIDRs`: A list of strings in
[CIDR](https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing) notation that specify the non-masquerade ranges.
* `masqLinkLocal`: A Boolean (true/false) which indicates whether to masquerade traffic to the
link local prefix `169.254.0.0/16`. False by default.
* `resyncInterval`: A time interval at which the agent attempts to reload config from disk.
For example: '30s', where 's' means seconds, 'ms' means milliseconds.
Traffic to 10.0.0.0/8, 172.16.0.0/12 and 192.168.0.0/16) ranges will NOT be masqueraded. Any other traffic (assumed to be internet) will be masqueraded. An example of a local destination from a pod could be its Node's IP address as well as another node's address or one of the IP addresses in Cluster's IP range. Any other traffic will be masqueraded by default. The below entries show the default set of rules that are applied by the ip-masq-agent:
```
```shell
iptables -t nat -L IP-MASQ-AGENT
```
```none
RETURN all -- anywhere 169.254.0.0/16 /* ip-masq-agent: cluster-local traffic should not be subject to MASQUERADE */ ADDRTYPE match dst-type !LOCAL
RETURN all -- anywhere 10.0.0.0/8 /* ip-masq-agent: cluster-local traffic should not be subject to MASQUERADE */ ADDRTYPE match dst-type !LOCAL
RETURN all -- anywhere 172.16.0.0/12 /* ip-masq-agent: cluster-local traffic should not be subject to MASQUERADE */ ADDRTYPE match dst-type !LOCAL
@@ -52,33 +54,35 @@ MASQUERADE all -- anywhere anywhere /* ip-masq-agent:
```
By default, in GCE/Google Kubernetes Engine starting with Kubernetes version 1.7.0, if network policy is enabled or you are using a cluster CIDR not in the 10.0.0.0/8 range, the ip-masq-agent will run in your cluster. If you are running in another environment, you can add the ip-masq-agent [DaemonSet](/docs/concepts/workloads/controllers/daemonset/) to your cluster:
By default, in GCE/Google Kubernetes Engine, if network policy is enabled or
you are using a cluster CIDR not in the 10.0.0.0/8 range, the `ip-masq-agent`
will run in your cluster. If you are running in another environment,
you can add the `ip-masq-agent` [DaemonSet](/docs/concepts/workloads/controllers/daemonset/)
to your cluster.
<!-- steps -->
## Create an ip-masq-agent
To create an ip-masq-agent, run the following kubectl command:
`
```shell
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/ip-masq-agent/master/ip-masq-agent.yaml
`
```
You must also apply the appropriate node label to any nodes in your cluster that you want the agent to run on.
`
kubectl label nodes my-node beta.kubernetes.io/masq-agent-ds-ready=true
`
```shell
kubectl label nodes my-node node.kubernetes.io/masq-agent-ds-ready=true
```
More information can be found in the ip-masq-agent documentation [here](https://github.com/kubernetes-sigs/ip-masq-agent)
In most cases, the default set of rules should be sufficient; however, if this is not the case for your cluster, you can create and apply a [ConfigMap](/docs/tasks/configure-pod-container/configure-pod-configmap/) to customize the IP ranges that are affected. For example, to allow only 10.0.0.0/8 to be considered by the ip-masq-agent, you can create the following [ConfigMap](/docs/tasks/configure-pod-container/configure-pod-configmap/) in a file called "config".
{{< note >}}
It is important that the file is called config since, by default, that will be used as the key for lookup by the ip-masq-agent:
It is important that the file is called config since, by default, that will be used as the key for lookup by the `ip-masq-agent`:
```
```yaml
nonMasqueradeCIDRs:
- 10.0.0.0/8
resyncInterval: 60s
@@ -87,15 +91,18 @@ resyncInterval: 60s
Run the following command to add the config map to your cluster:
```
```shell
kubectl create configmap ip-masq-agent --from-file=config --namespace=kube-system
```
This will update a file located at */etc/config/ip-masq-agent* which is periodically checked every *resyncInterval* and applied to the cluster node.
This will update a file located at `/etc/config/ip-masq-agent` which is periodically checked every `resyncInterval` and applied to the cluster node.
After the resync interval has expired, you should see the iptables rules reflect your changes:
```
```shell
iptables -t nat -L IP-MASQ-AGENT
```
```none
Chain IP-MASQ-AGENT (1 references)
target prot opt source destination
RETURN all -- anywhere 169.254.0.0/16 /* ip-masq-agent: cluster-local traffic should not be subject to MASQUERADE */ ADDRTYPE match dst-type !LOCAL
@@ -103,9 +110,9 @@ RETURN all -- anywhere 10.0.0.0/8 /* ip-masq-agent:
MASQUERADE all -- anywhere anywhere /* ip-masq-agent: outbound traffic should be subject to MASQUERADE (this match must come after cluster-local CIDR matches) */ ADDRTYPE match dst-type !LOCAL
```
By default, the link local range (169.254.0.0/16) is also handled by the ip-masq agent, which sets up the appropriate iptables rules. To have the ip-masq-agent ignore link local, you can set *masqLinkLocal* to true in the config map.
By default, the link local range (169.254.0.0/16) is also handled by the ip-masq agent, which sets up the appropriate iptables rules. To have the ip-masq-agent ignore link local, you can set `masqLinkLocal` to true in the ConfigMap.
```
```yaml
nonMasqueradeCIDRs:
- 10.0.0.0/8
resyncInterval: 60s
@@ -81,13 +81,13 @@ kubectl describe pod liveness-exec
The output indicates that no liveness probes have failed yet:
```
FirstSeen LastSeen Count From SubobjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
24s 24s 1 {default-scheduler } Normal Scheduled Successfully assigned liveness-exec to worker0
23s 23s 1 {kubelet worker0} spec.containers{liveness} Normal Pulling pulling image "k8s.gcr.io/busybox"
23s 23s 1 {kubelet worker0} spec.containers{liveness} Normal Pulled Successfully pulled image "k8s.gcr.io/busybox"
23s 23s 1 {kubelet worker0} spec.containers{liveness} Normal Created Created container with docker id 86849c15382e; Security:[seccomp=unconfined]
23s 23s 1 {kubelet worker0} spec.containers{liveness} Normal Started Started container with docker id 86849c15382e
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 11s default-scheduler Successfully assigned default/liveness-exec to node01
Normal Pulling 9s kubelet, node01 Pulling image "k8s.gcr.io/busybox"
Normal Pulled 7s kubelet, node01 Successfully pulled image "k8s.gcr.io/busybox"
Normal Created 7s kubelet, node01 Created container liveness
Normal Started 7s kubelet, node01 Started container liveness
```
After 35 seconds, view the Pod events again:
@@ -100,14 +100,15 @@ At the bottom of the output, there are messages indicating that the liveness
probes have failed, and the containers have been killed and recreated.
```
FirstSeen LastSeen Count From SubobjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
37s 37s 1 {default-scheduler } Normal Scheduled Successfully assigned liveness-exec to worker0
36s 36s 1 {kubelet worker0} spec.containers{liveness} Normal Pulling pulling image "k8s.gcr.io/busybox"
36s 36s 1 {kubelet worker0} spec.containers{liveness} Normal Pulled Successfully pulled image "k8s.gcr.io/busybox"
36s 36s 1 {kubelet worker0} spec.containers{liveness} Normal Created Created container with docker id 86849c15382e; Security:[seccomp=unconfined]
36s 36s 1 {kubelet worker0} spec.containers{liveness} Normal Started Started container with docker id 86849c15382e
2s 2s 1 {kubelet worker0} spec.containers{liveness} Warning Unhealthy Liveness probe failed: cat: can't open '/tmp/healthy': No such file or directory
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 57s default-scheduler Successfully assigned default/liveness-exec to node01
Normal Pulling 55s kubelet, node01 Pulling image "k8s.gcr.io/busybox"
Normal Pulled 53s kubelet, node01 Successfully pulled image "k8s.gcr.io/busybox"
Normal Created 53s kubelet, node01 Created container liveness
Normal Started 53s kubelet, node01 Started container liveness
Warning Unhealthy 10s (x3 over 20s) kubelet, node01 Liveness probe failed: cat: can't open '/tmp/healthy': No such file or directory
Normal Killing 10s kubelet, node01 Container liveness failed liveness probe, will be restarted
```
Wait another 30 seconds, and verify that the container has been restarted:
@@ -208,7 +208,6 @@ you need is an existing `docker-compose.yml` file.
- CLI
- [`kompose convert`](#kompose-convert)
- Documentation
- [Build and Push Docker Images](#build-and-push-docker-images)
- [Alternative Conversions](#alternative-conversions)
- [Labels](#labels)
- [Restart](#restart)
@@ -326,55 +325,6 @@ INFO OpenShift file "foo-buildconfig.yaml" created
If you are manually pushing the OpenShift artifacts using ``oc create -f``, you need to ensure that you push the imagestream artifact before the buildconfig artifact, to workaround this OpenShift issue: https://github.com/openshift/origin/issues/4518 .
{{< /note >}}
## Build and Push Docker Images
Kompose supports both building and pushing Docker images. When using the `build` key within your Docker Compose file, your image will:
- Automatically be built with Docker using the `image` key specified within your file
- Be pushed to the correct Docker repository using local credentials (located at `.docker/config`)
Using an [example Docker Compose file](https://raw.githubusercontent.com/kubernetes/kompose/master/examples/buildconfig/docker-compose.yml):
```yaml
version: "2"
services:
foo:
build: "./build"
image: docker.io/foo/bar
```
Using `kompose up` with a `build` key:
```none
kompose up
INFO Build key detected. Attempting to build and push image 'docker.io/foo/bar'
INFO Building image 'docker.io/foo/bar' from directory 'build'
INFO Image 'docker.io/foo/bar' from directory 'build' built successfully
INFO Pushing image 'foo/bar:latest' to registry 'docker.io'
INFO Attempting authentication credentials 'https://index.docker.io/v1/
INFO Successfully pushed image 'foo/bar:latest' to registry 'docker.io'
INFO We are going to create Kubernetes Deployments, Services and PersistentVolumeClaims for your Dockerized application. If you need different kind of resources, use the 'kompose convert' and 'kubectl apply -f' commands instead.
INFO Deploying application in "default" namespace
INFO Successfully created Service: foo
INFO Successfully created Deployment: foo
Your application has been deployed to Kubernetes. You can run 'kubectl get deployment,svc,pods,pvc' for details.
```
In order to disable the functionality, or choose to use BuildConfig generation (with OpenShift) `--build (local|build-config|none)` can be passed.
```sh
# Disable building/pushing Docker images
kompose up --build none
# Generate Build Config artifacts for OpenShift
kompose up --provider openshift --build build-config
```
## Alternative Conversions
The default `kompose` transformation will generate Kubernetes [Deployments](/docs/concepts/workloads/controllers/deployment/) and [Services](/docs/concepts/services-networking/service/), in yaml format. You have alternative option to generate json with `-j`. Also, you can alternatively generate [Replication Controllers](/docs/concepts/workloads/controllers/replicationcontroller/) objects, [Daemon Sets](/docs/concepts/workloads/controllers/daemonset/), or [Helm](https://github.com/helm/helm) charts.
@@ -110,7 +110,7 @@ specify the `-i`/`--interactive` argument, `kubectl` will automatically attach
to the console of the Ephemeral Container.
```shell
kubectl debug -it ephemeral-demo --image=busybox --target=ephemeral-demo
kubectl debug -it ephemeral-demo --image=busybox:1.28 --target=ephemeral-demo
```
```
@@ -182,7 +182,7 @@ but you need debugging utilities not included in `busybox`. You can simulate
this scenario using `kubectl run`:
```shell
kubectl run myapp --image=busybox --restart=Never -- sleep 1d
kubectl run myapp --image=busybox:1.28 --restart=Never -- sleep 1d
```
Run this command to create a copy of `myapp` named `myapp-debug` that adds a
@@ -225,7 +225,7 @@ To simulate a crashing application, use `kubectl run` to create a container
that immediately exits:
```
kubectl run --image=busybox myapp -- false
kubectl run --image=busybox:1.28 myapp -- false
```
You can see using `kubectl describe pod myapp` that this container is crashing:
@@ -283,7 +283,7 @@ additional utilities.
As an example, create a Pod using `kubectl run`:
```
kubectl run myapp --image=busybox --restart=Never -- sleep 1d
kubectl run myapp --image=busybox:1.28 --restart=Never -- sleep 1d
```
Now use `kubectl debug` to make a copy and change its container image
@@ -8,14 +8,19 @@ content_type: concept
<!-- overview -->
For Kubernetes, the _Metrics API_ offers a basic set of metrics to support automatic scaling and similar use cases.
This API makes information available about resource usage for node and pod, including metrics for CPU and memory.
If you deploy the Metrics API into your cluster, clients of the Kubernetes API can then query for this information, and
you can use Kubernetes' access control mechanisms to manage permissions to do so.
For Kubernetes, the _Metrics API_ offers a basic set of metrics to support automatic scaling and
similar use cases. This API makes information available about resource usage for node and pod,
including metrics for CPU and memory. If you deploy the Metrics API into your cluster, clients of
the Kubernetes API can then query for this information, and you can use Kubernetes' access control
mechanisms to manage permissions to do so.
The [HorizontalPodAutoscaler](/docs/tasks/run-application/horizontal-pod-autoscale/) (HPA) and [VerticalPodAutoscaler](https://github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler#readme) (VPA) use data from the metrics API to adjust workload replicas and resources to meet customer demand.
The [HorizontalPodAutoscaler](/docs/tasks/run-application/horizontal-pod-autoscale/) (HPA) and
[VerticalPodAutoscaler](https://github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler#readme) (VPA)
use data from the metrics API to adjust workload replicas and resources to meet customer demand.
You can also view the resource metrics using the [`kubectl top`](https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#top) command.
You can also view the resource metrics using the
[`kubectl top`](/docs/reference/generated/kubectl/kubectl-commands#top)
command.
{{< note >}}
The Metrics API, and the metrics pipeline that it enables, only offers the minimum
@@ -59,34 +64,51 @@ Figure 1. Resource Metrics Pipeline
The architecture components, from right to left in the figure, consist of the following:
* [cAdvisor](https://github.com/google/cadvisor): Daemon for collecting, aggregating and exposing container metrics included in Kubelet.
* [kubelet](/docs/concepts/overview/components/#kubelet): Node agent for managing container resources. Resource metrics are accessible using the `/metrics/resource` and `/stats` kubelet API endpoints.
* [Summary API](#summary-api-source): API provided by the kubelet for discovering and retrieving per-node summarized stats available through the `/stats` endpoint.
* [metrics-server](#metrics-server): Cluster addon component that collects and aggregates resource metrics pulled from each kubelet. The API server serves Metrics API for use by HPA, VPA, and by the `kubectl top` command. Metrics Server is a reference implementation of the Metrics API.
* [Metrics API](#metrics-api): Kubernetes API supporting access to CPU and memory used for workload autoscaling. To make this work in your cluster, you need an API extension server that provides the Metrics API.
* [cAdvisor](https://github.com/google/cadvisor): Daemon for collecting, aggregating and exposing
container metrics included in Kubelet.
* [kubelet](/docs/concepts/overview/components/#kubelet): Node agent for managing container
resources. Resource metrics are accessible using the `/metrics/resource` and `/stats` kubelet
API endpoints.
* [Summary API](#summary-api-source): API provided by the kubelet for discovering and retrieving
per-node summarized stats available through the `/stats` endpoint.
* [metrics-server](#metrics-server): Cluster addon component that collects and aggregates resource
metrics pulled from each kubelet. The API server serves Metrics API for use by HPA, VPA, and by
the `kubectl top` command. Metrics Server is a reference implementation of the Metrics API.
* [Metrics API](#metrics-api): Kubernetes API supporting access to CPU and memory used for
workload autoscaling. To make this work in your cluster, you need an API extension server that
provides the Metrics API.
{{< note >}}
cAdvisor supports reading metrics from cgroups, which works with typical container runtimes on Linux.
If you use a container runtime that uses another resource isolation mechanism, for example virtualization, then that container runtime must support [CRI Container Metrics](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-node/cri-container-stats.md) in order for metrics to be available to the kubelet.
If you use a container runtime that uses another resource isolation mechanism, for example
virtualization, then that container runtime must support
[CRI Container Metrics](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-node/cri-container-stats.md)
in order for metrics to be available to the kubelet.
{{< /note >}}
<!-- body -->
## Metrics API
The metrics-server implements the Metrics API. This API allows you to access CPU and memory usage for the nodes and pods in your cluster. Its primary role is to feed resource usage metrics to K8s autoscaler components.
The metrics-server implements the Metrics API. This API allows you to access CPU and memory usage
for the nodes and pods in your cluster. Its primary role is to feed resource usage metrics to K8s
autoscaler components.
Here is an example of the Metrics API request for a `minikube` node piped through `jq` for easier
reading:
Here is an example of the Metrics API request for a `minikube` node piped through `jq` for easier reading:
```shell
kubectl get --raw "/apis/metrics.k8s.io/v1beta1/nodes/minikube" | jq '.'
```
Here is the same API call using `curl`:
```shell
curl http://localhost:8080/apis/metrics.k8s.io/v1beta1/nodes/minikube
```
Sample reply:
Sample response:
```json
{
"kind": "NodeMetrics",
@@ -104,16 +126,22 @@ Sample reply:
}
}
```
Here is an example of the Metrics API request for a `kube-scheduler-minikube` pod contained in the `kube-system` namespace and piped through `jq` for easier reading:
Here is an example of the Metrics API request for a `kube-scheduler-minikube` pod contained in the
`kube-system` namespace and piped through `jq` for easier reading:
```shell
kubectl get --raw "/apis/metrics.k8s.io/v1beta1/namespaces/kube-system/pods/kube-scheduler-minikube" | jq '.'
```
Here is the same API call using `curl`:
```shell
curl http://localhost:8080/apis/metrics.k8s.io/v1beta1/namespaces/kube-system/pods/kube-scheduler-minikube
```
Sample reply:
Sample response:
```json
{
"kind": "PodMetrics",
@@ -138,47 +166,72 @@ Sample reply:
}
```
The Metrics API is defined in the [k8s.io/metrics](https://github.com/kubernetes/metrics) repository. You must enable the [API aggregation layer](/docs/tasks/extend-kubernetes/configure-aggregation-layer/) and register an [APIService](/docs/reference/kubernetes-api/cluster-resources/api-service-v1/) for the `metrics.k8s.io` API.
The Metrics API is defined in the [k8s.io/metrics](https://github.com/kubernetes/metrics)
repository. You must enable the [API aggregation layer](/docs/tasks/extend-kubernetes/configure-aggregation-layer/)
and register an [APIService](/docs/reference/kubernetes-api/cluster-resources/api-service-v1/)
for the `metrics.k8s.io` API.
To learn more about the Metrics API, see [resource metrics API design](https://github.com/kubernetes/design-proposals-archive/blob/main/instrumentation/resource-metrics-api.md), the [metrics-server repository](https://github.com/kubernetes-sigs/metrics-server) and the [resource metrics API](https://github.com/kubernetes/metrics#resource-metrics-api).
To learn more about the Metrics API, see [resource metrics API design](https://github.com/kubernetes/design-proposals-archive/blob/main/instrumentation/resource-metrics-api.md),
the [metrics-server repository](https://github.com/kubernetes-sigs/metrics-server) and the
[resource metrics API](https://github.com/kubernetes/metrics#resource-metrics-api).
{{< note >}} You must deploy the metrics-server or alternative adapter that serves the Metrics API to be able to access it. {{< /note >}}
{{< note >}}
You must deploy the metrics-server or alternative adapter that serves the Metrics API to be able
to access it.
{{< /note >}}
## Measuring resource usage
### CPU
CPU is reported as the average core usage measured in cpu units. One cpu, in Kubernetes, is equivalent to 1 vCPU/Core for cloud providers, and 1 hyper-thread on bare-metal Intel processors.
CPU is reported as the average core usage measured in cpu units. One cpu, in Kubernetes, is
equivalent to 1 vCPU/Core for cloud providers, and 1 hyper-thread on bare-metal Intel processors.
This value is derived by taking a rate over a cumulative CPU counter provided by the kernel (in both Linux and Windows kernels). The time window used to calculate CPU is shown under window field in Metrics API.
This value is derived by taking a rate over a cumulative CPU counter provided by the kernel (in
both Linux and Windows kernels). The time window used to calculate CPU is shown under window field
in Metrics API.
To learn more about how Kubernetes allocates and measures CPU resources, see [meaning of CPU](/docs/concepts/configuration/manage-compute-resources-container/#meaning-of-cpu).
To learn more about how Kubernetes allocates and measures CPU resources, see
[meaning of CPU](/docs/concepts/configuration/manage-resources-container/#meaning-of-cpu).
### Memory
Memory is reported as the working set, measured in bytes, at the instant the metric was collected.
In an ideal world, the "working set" is the amount of memory in-use that cannot be freed under memory pressure. However, calculation of the working set varies by host OS, and generally makes heavy use of heuristics to produce an estimate.
In an ideal world, the "working set" is the amount of memory in-use that cannot be freed under
memory pressure. However, calculation of the working set varies by host OS, and generally makes
heavy use of heuristics to produce an estimate.
The Kubernetes model for a container's working set expects that the container runtime counts anonymous memory associated with the container in question. The working set metric typically also includes some cached (file-backed) memory, because the host OS cannot always reclaim pages.
The Kubernetes model for a container's working set expects that the container runtime counts
anonymous memory associated with the container in question. The working set metric typically also
includes some cached (file-backed) memory, because the host OS cannot always reclaim pages.
To learn more about how Kubernetes allocates and measures memory resources, see [meaning of memory](/docs/concepts/configuration/manage-compute-resources-container/#meaning-of-memory).
To learn more about how Kubernetes allocates and measures memory resources, see
[meaning of memory](/docs/concepts/configuration/manage-resources-container/#meaning-of-memory).
## Metrics Server
The metrics-server fetches resource metrics from the kubelets and exposes them in the Kubernetes API server through the Metrics API for use by the HPA and VPA. You can also view these metrics using the `kubectl top` command.
The metrics-server fetches resource metrics from the kubelets and exposes them in the Kubernetes
API server through the Metrics API for use by the HPA and VPA. You can also view these metrics
using the `kubectl top` command.
The metrics-server uses the Kubernetes API to track nodes and pods in your cluster. The metrics-server queries each node over HTTP to fetch metrics. The metrics-server also builds an internal view of pod metadata, and keeps a cache of pod health. That cached pod health information is available via the extension API that the metrics-server makes available.
The metrics-server uses the Kubernetes API to track nodes and pods in your cluster. The
metrics-server queries each node over HTTP to fetch metrics. The metrics-server also builds an
internal view of pod metadata, and keeps a cache of pod health. That cached pod health information
is available via the extension API that the metrics-server makes available.
For example with an HPA query, the metrics-server needs to identify which pods fulfill the label selectors in the deployment.
For example with an HPA query, the metrics-server needs to identify which pods fulfill the label
selectors in the deployment.
The metrics-server calls the [kubelet](/docs/reference/command-line-tools-reference/kubelet/) API
to collect metrics from each node. Depending on the metrics-server version it uses:
The metrics-server calls the [kubelet](/docs/reference/command-line-tools-reference/kubelet/) API to collect metrics from each node. Depending on the metrics-server version it uses:
* Metrics resource endpoint `/metrics/resource` in version v0.6.0+ or
* Summary API endpoint `/stats/summary` in older versions
To learn more about the metrics-server, see the [metrics-server repository](https://github.com/kubernetes-sigs/metrics-server).
To learn more about the metrics-server, see the
[metrics-server repository](https://github.com/kubernetes-sigs/metrics-server).
You can also check out the following:
@@ -190,20 +243,25 @@ You can also check out the following:
### Summary API source
The [kubelet](/docs/reference/command-line-tools-reference/kubelet/) gathers stats at the node, volume, pod and container level, and emits this information in
The [kubelet](/docs/reference/command-line-tools-reference/kubelet/) gathers stats at the node,
volume, pod and container level, and emits this information in
the [Summary API](https://github.com/kubernetes/kubernetes/blob/7d309e0104fedb57280b261e5677d919cb2a0e2d/staging/src/k8s.io/kubelet/pkg/apis/stats/v1alpha1/types.go)
for consumers to read.
Here is an example of a Summary API request for a `minikube` node:
```shell
kubectl get --raw "/api/v1/nodes/minikube/proxy/stats/summary"
```
Here is the same API call using `curl`:
```shell
curl http://localhost:8080/api/v1/nodes/minikube/proxy/stats/summary
```
{{< note >}}
The summary API `/stats/summary` endpoint will be replaced by the `/metrics/resource` endpoint beginning with metrics-server 0.6.x.
{{< /note >}}
The summary API `/stats/summary` endpoint will be replaced by the `/metrics/resource` endpoint
beginning with metrics-server 0.6.x.
{{< /note >}}
@@ -201,7 +201,7 @@ spec:
spec:
containers:
- name: c
image: busybox
image: busybox:1.28
command: ["sh", "-c", "echo Processing URL {{ url }} && sleep 5"]
restartPolicy: Never
{% endfor %}
@@ -34,11 +34,11 @@ DaemonSet has two update strategy types:
To enable the rolling update feature of a DaemonSet, you must set its
`.spec.updateStrategy.type` to `RollingUpdate`.
You may want to set
[`.spec.updateStrategy.rollingUpdate.maxUnavailable`](/docs/concepts/workloads/controllers/deployment/#max-unavailable)
You may want to set
[`.spec.updateStrategy.rollingUpdate.maxUnavailable`](/docs/reference/kubernetes-api/workload-resources/daemon-set-v1/#DaemonSetSpec)
(default to 1),
[`.spec.minReadySeconds`](/docs/concepts/workloads/controllers/deployment/#min-ready-seconds)
(default to 0) and
[`.spec.minReadySeconds`](/docs/reference/kubernetes-api/workload-resources/daemon-set-v1/#DaemonSetSpec)
(default to 0) and
[`.spec.updateStrategy.rollingUpdate.maxSurge`](/docs/reference/kubernetes-api/workload-resources/daemon-set-v1/#DaemonSetSpec)
(a beta feature and defaults to 0) as well.
@@ -189,11 +189,11 @@ kubectl get deployment patch-demo --output yaml
The output shows that the PodSpec in the Deployment has only one Toleration:
```shell
```yaml
tolerations:
- effect: NoSchedule
key: disktype
value: ssd
- effect: NoSchedule
key: disktype
value: ssd
```
Notice that the `tolerations` list in the PodSpec was replaced, not merged. This is because
@@ -48,7 +48,8 @@ While running in a Pod, the Kubernetes apiserver is accessible via a Service nam
do this automatically.
The recommended way to authenticate to the API server is with a
[service account](/docs/tasks/configure-pod-container/configure-service-account/) credential. By default, a Pod
[service account](/docs/tasks/configure-pod-container/configure-service-account/)
credential. By default, a Pod
is associated with a service account, and a credential (token) for that
service account is placed into the filesystem tree of each container in that Pod,
at `/var/run/secrets/kubernetes.io/serviceaccount/token`.
@@ -152,7 +152,7 @@ runs in an infinite loop, sending queries to the php-apache service.
```shell
# Run this in a separate terminal
# so that the load generation continues and you can carry on with the rest of the steps
kubectl run -i --tty load-generator --rm --image=busybox --restart=Never -- /bin/sh -c "while sleep 0.01; do wget -q -O- http://php-apache; done"
kubectl run -i --tty load-generator --rm --image=busybox:1.28 --restart=Never -- /bin/sh -c "while sleep 0.01; do wget -q -O- http://php-apache; done"
```
Now run:
@@ -52,7 +52,7 @@ For example, to download version {{< param "fullversion" >}} on Linux, type:
Validate the kubectl binary against the checksum file:
```bash
echo "$(<kubectl.sha256) kubectl" | sha256sum --check
echo "$(cat kubectl.sha256) kubectl" | sha256sum --check
```
If valid, the output is:
@@ -212,7 +212,7 @@ Below are the procedures to set up autocompletion for Bash, Fish, and Zsh.
Validate the kubectl-convert binary against the checksum file:
```bash
echo "$(<kubectl-convert.sha256) kubectl-convert" | sha256sum --check
echo "$(cat kubectl-convert.sha256) kubectl-convert" | sha256sum --check
```
If valid, the output is:
@@ -69,7 +69,7 @@ The following methods exist for installing kubectl on macOS:
Validate the kubectl binary against the checksum file:
```bash
echo "$(<kubectl.sha256) kubectl" | shasum -a 256 --check
echo "$(cat kubectl.sha256) kubectl" | shasum -a 256 --check
```
If valid, the output is:
@@ -114,7 +114,7 @@ The following methods exist for installing kubectl on macOS:
Or use this for detailed view of version:
```cmd
kubectl version --client --output=yaml
kubectl version --client --output=yaml
```
### Install with Homebrew on macOS
@@ -124,7 +124,7 @@ If you are on macOS and using [Homebrew](https://brew.sh/) package manager, you
1. Run the installation command:
```bash
brew install kubectl
brew install kubectl
```
or
@@ -205,7 +205,7 @@ Below are the procedures to set up autocompletion for Bash, Fish, and Zsh.
Validate the kubectl-convert binary against the checksum file:
```bash
echo "$(<kubectl-convert.sha256) kubectl-convert" | shasum -a 256 --check
echo "$(cat kubectl-convert.sha256) kubectl-convert" | shasum -a 256 --check
```
If valid, the output is:
@@ -38,13 +38,13 @@ The following methods exist for installing kubectl on Windows:
1. Validate the binary (optional)
Download the kubectl checksum file:
Download the `kubectl` checksum file:
```powershell
curl -LO "https://dl.k8s.io/{{< param "fullversion" >}}/bin/windows/amd64/kubectl.exe.sha256"
```
Validate the kubectl binary against the checksum file:
Validate the `kubectl` binary against the checksum file:
- Using Command Prompt to manually compare `CertUtil`'s output to the checksum file downloaded:
@@ -59,7 +59,7 @@ The following methods exist for installing kubectl on Windows:
$($(CertUtil -hashfile .\kubectl.exe SHA256)[1] -replace " ", "") -eq $(type .\kubectl.exe.sha256)
```
1. Append or prepend the kubectl binary folder to your `PATH` environment variable.
1. Append or prepend the `kubectl` binary folder to your `PATH` environment variable.
1. Test to ensure the version of `kubectl` is the same as downloaded:
@@ -156,13 +156,13 @@ Below are the procedures to set up autocompletion for PowerShell.
1. Validate the binary (optional)
Download the kubectl-convert checksum file:
Download the `kubectl-convert` checksum file:
```powershell
curl -LO "https://dl.k8s.io/{{< param "fullversion" >}}/bin/windows/amd64/kubectl-convert.exe.sha256"
```
Validate the kubectl-convert binary against the checksum file:
Validate the `kubectl-convert` binary against the checksum file:
- Using Command Prompt to manually compare `CertUtil`'s output to the checksum file downloaded:
@@ -177,7 +177,7 @@ Below are the procedures to set up autocompletion for PowerShell.
$($(CertUtil -hashfile .\kubectl-convert.exe SHA256)[1] -replace " ", "") -eq $(type .\kubectl-convert.exe.sha256)
```
1. Append or prepend the kubectl binary folder to your `PATH` environment variable.
1. Append or prepend the `kubectl-convert` binary folder to your `PATH` environment variable.
1. Verify plugin is successfully installed