Die Gewissheit, dass Kubernetes überall und für alle gut funktioniert.
-
Verbinden Sie sich mit der Kubernetes-Community in unserem Slack Kanal, Diskussionsforum, oder beteiligen Sie sich an der Kubernetes-dev-Google-Gruppe. Eine wöchentliches Community-Meeting findet per Videokonferenz statt, um den Stand der Dinge zu diskutieren, folgen Sie
+
Verbinden Sie sich mit der Kubernetes-Community in unserem Slack Kanal, Diskussionsforum, oder beteiligen Sie sich an der Kubernetes-dev-Google-Gruppe. Eine wöchentliches Community-Meeting findet per Videokonferenz statt, um den Stand der Dinge zu diskutieren, folgen Sie
diesen Anweisungen für Informationen wie Sie teilnehmen können.
Sie können Kubernetes auch auf der ganzen Welt über unsere
Kubernetes Meetup Community und der
diff --git a/content/de/community/static/cncf-code-of-conduct.md b/content/de/community/static/cncf-code-of-conduct.md
index e94bc7b7fa..7a7bee34e7 100644
--- a/content/de/community/static/cncf-code-of-conduct.md
+++ b/content/de/community/static/cncf-code-of-conduct.md
@@ -23,7 +23,7 @@ Dieser Verhaltenskodex gilt sowohl innerhalb von Projekträumen als auch in öff
Fälle von missbräuchlichem, belästigendem oder anderweitig unzumutbarem Verhalten in Kubernetes können gemeldet werden, indem Sie sich an das [Kubernetes Komitee für Verhaltenskodex](https://git.k8s.io/community/committee-code-of-conduct) wenden unter . Für andere Projekte wenden Sie sich bitte an einen CNCF-Projektbetreuer oder an unseren Mediator, Mishi Choudhary .
-Dieser Verhaltenskodex wurde aus dem Contributor Covenant übernommen (http://contributor-covenant.org), Version 1.2.0, verfügbar unter http://contributor-covenant.org/version/1/2/0/
+Dieser Verhaltenskodex wurde aus dem Contributor Covenant übernommen (https://contributor-covenant.org), Version 1.2.0, verfügbar unter https://contributor-covenant.org/version/1/2/0/
### CNCF Verhaltenskodex für Veranstaltungen
diff --git a/content/en/blog/_posts/2016-01-00-Why-Kubernetes-Doesnt-Use-Libnetwork.md b/content/en/blog/_posts/2016-01-00-Why-Kubernetes-Doesnt-Use-Libnetwork.md
index 590548aea2..553bd6ad96 100644
--- a/content/en/blog/_posts/2016-01-00-Why-Kubernetes-Doesnt-Use-Libnetwork.md
+++ b/content/en/blog/_posts/2016-01-00-Why-Kubernetes-Doesnt-Use-Libnetwork.md
@@ -26,7 +26,7 @@ On the other hand, CNI is more philosophically aligned with Kubernetes. It's far
Additionally, it's trivial to wrap a CNI plugin and produce a more customized CNI plugin — it can be done with a simple shell script. CNM is much more complex in this regard. This makes CNI an attractive option for rapid development and iteration. Early prototypes have proven that it's possible to eject almost 100% of the currently hard-coded network logic in kubelet into a plugin.
-We investigated [writing a "bridge" CNM driver](https://groups.google.com/forum/#!topic/kubernetes-sig-network/5MWRPxsURUw) for Docker that ran CNI drivers. This turned out to be very complicated. First, the CNM and CNI models are very different, so none of the "methods" lined up. We still have the global vs. local and key-value issues discussed above. Assuming this driver would declare itself local, we have to get info about logical networks from Kubernetes.
+We investigated [writing a "bridge" CNM driver](https://groups.google.com/g/kubernetes-sig-network/c/5MWRPxsURUw) for Docker that ran CNI drivers. This turned out to be very complicated. First, the CNM and CNI models are very different, so none of the "methods" lined up. We still have the global vs. local and key-value issues discussed above. Assuming this driver would declare itself local, we have to get info about logical networks from Kubernetes.
Unfortunately, Docker drivers are hard to map to other control planes like Kubernetes. Specifically, drivers are not told the name of the network to which a container is being attached — just an ID that Docker allocates internally. This makes it hard for a driver to map back to any concept of network that exists in another system.
@@ -34,6 +34,6 @@ This and other issues have been brought up to Docker developers by network vendo
For all of these reasons we have chosen to invest in CNI as the Kubernetes plugin model. There will be some unfortunate side-effects of this. Most of them are relatively minor (for example, `docker inspect` will not show an IP address), but some are significant. In particular, containers started by `docker run` might not be able to communicate with containers started by Kubernetes, and network integrators will have to provide CNI drivers if they want to fully integrate with Kubernetes. On the other hand, Kubernetes will get simpler and more flexible, and a lot of the ugliness of early bootstrapping (such as configuring Docker to use our bridge) will go away.
-As we proceed down this path, we’ll certainly keep our eyes and ears open for better ways to integrate and simplify. If you have thoughts on how we can do that, we really would like to hear them — find us on [slack](http://slack.k8s.io/) or on our [network SIG mailing-list](https://groups.google.com/forum/#!forum/kubernetes-sig-network).
+As we proceed down this path, we’ll certainly keep our eyes and ears open for better ways to integrate and simplify. If you have thoughts on how we can do that, we really would like to hear them — find us on [slack](http://slack.k8s.io/) or on our [network SIG mailing-list](https://groups.google.com/g/kubernetes-sig-network).
Tim Hockin, Software Engineer, Google
diff --git a/content/en/blog/_posts/2017-11-00-Containerd-Container-Runtime-Options-Kubernetes.md b/content/en/blog/_posts/2017-11-00-Containerd-Container-Runtime-Options-Kubernetes.md
index 3edf5625c2..2a105c6396 100644
--- a/content/en/blog/_posts/2017-11-00-Containerd-Container-Runtime-Options-Kubernetes.md
+++ b/content/en/blog/_posts/2017-11-00-Containerd-Container-Runtime-Options-Kubernetes.md
@@ -56,13 +56,13 @@ Cri-containerd uses containerd to manage the full container lifecycle and all co
Let’s use an example to demonstrate how cri-containerd works for the case when Kubelet creates a single-container pod:
-1. 1.Kubelet calls cri-containerd, via the CRI runtime service API, to create a pod;
-2. 2.cri-containerd uses containerd to create and start a special [pause container](https://www.ianlewis.org/en/almighty-pause-container) (the _sandbox container_) and put that container inside the pod’s cgroups and namespace (steps omitted for brevity);
-3. 3.cri-containerd configures the pod’s network namespace using CNI;
-4. 4.Kubelet subsequently calls cri-containerd, via the CRI image service API, to pull the application container image;
-5. 5.cri-containerd further uses containerd to pull the image if the image is not present on the node;
-6. 6.Kubelet then calls cri-containerd, via the CRI runtime service API, to create and start the application container inside the pod using the pulled container image;
-7. 7.cri-containerd finally calls containerd to create the application container, put it inside the pod’s cgroups and namespace, then to start the pod’s new application container.
+1. Kubelet calls cri-containerd, via the CRI runtime service API, to create a pod;
+2. cri-containerd uses containerd to create and start a special [pause container](https://www.ianlewis.org/en/almighty-pause-container) (the _sandbox container_) and put that container inside the pod’s cgroups and namespace (steps omitted for brevity);
+3. cri-containerd configures the pod’s network namespace using CNI;
+4. Kubelet subsequently calls cri-containerd, via the CRI image service API, to pull the application container image;
+5. cri-containerd further uses containerd to pull the image if the image is not present on the node;
+6. Kubelet then calls cri-containerd, via the CRI runtime service API, to create and start the application container inside the pod using the pulled container image;
+7. cri-containerd finally calls containerd to create the application container, put it inside the pod’s cgroups and namespace, then to start the pod’s new application container.
After these steps, a pod and its corresponding application container is created and running.
diff --git a/content/en/blog/_posts/2019-12-09-volume-snapshot-beta.md b/content/en/blog/_posts/2019-12-09-volume-snapshot-beta.md
index e6d8d0cc2b..b18b54aeb4 100644
--- a/content/en/blog/_posts/2019-12-09-volume-snapshot-beta.md
+++ b/content/en/blog/_posts/2019-12-09-volume-snapshot-beta.md
@@ -44,7 +44,7 @@ As mentioned above, with the promotion of Volume Snapshot to beta, the feature i
In order to use the Kubernetes Volume Snapshot feature, you must ensure the following components have been deployed on your Kubernetes cluster:
-- [Kubernetes Volume Snapshot CRDs](https://github.com/kubernetes-csi/external-snapshotter/tree/master/config/crd)
+- [Kubernetes Volume Snapshot CRDs](https://github.com/kubernetes-csi/external-snapshotter/tree/53469c21962339229dd150cbba50c34359acec73/config/crd)
- [Volume snapshot controller](https://github.com/kubernetes-csi/external-snapshotter/tree/master/pkg/common-controller)
- CSI Driver supporting Kubernetes volume snapshot beta
@@ -180,7 +180,7 @@ If your cluster does not come pre-installed with the correct components, you may
#### Install Snapshot Beta CRDs
- `kubectl create -f config/crd`
-- [https://github.com/kubernetes-csi/external-snapshotter/tree/master/config/crd](https://github.com/kubernetes-csi/external-snapshotter/tree/master/config/crd)
+- [https://github.com/kubernetes-csi/external-snapshotter/tree/53469c21962339229dd150cbba50c34359acec73/config/crd](https://github.com/kubernetes-csi/external-snapshotter/tree/53469c21962339229dd150cbba50c34359acec73/config/crd)
- Do this once per cluster
diff --git a/content/en/blog/_posts/2020-09-30-writing-crl-scheduler/image01.png b/content/en/blog/_posts/2020-09-30-writing-crl-scheduler/image01.png
new file mode 100644
index 0000000000..91e8856139
Binary files /dev/null and b/content/en/blog/_posts/2020-09-30-writing-crl-scheduler/image01.png differ
diff --git a/content/en/blog/_posts/2020-09-30-writing-crl-scheduler/image02.png b/content/en/blog/_posts/2020-09-30-writing-crl-scheduler/image02.png
new file mode 100644
index 0000000000..dfd14d7cdc
Binary files /dev/null and b/content/en/blog/_posts/2020-09-30-writing-crl-scheduler/image02.png differ
diff --git a/content/en/blog/_posts/2020-09-30-writing-crl-scheduler/image03.png b/content/en/blog/_posts/2020-09-30-writing-crl-scheduler/image03.png
new file mode 100644
index 0000000000..443a6f2d67
Binary files /dev/null and b/content/en/blog/_posts/2020-09-30-writing-crl-scheduler/image03.png differ
diff --git a/content/en/blog/_posts/2020-09-30-writing-crl-scheduler/image04.png b/content/en/blog/_posts/2020-09-30-writing-crl-scheduler/image04.png
new file mode 100644
index 0000000000..e107adc88b
Binary files /dev/null and b/content/en/blog/_posts/2020-09-30-writing-crl-scheduler/image04.png differ
diff --git a/content/en/blog/_posts/2020-09-30-writing-crl-scheduler/image05.png b/content/en/blog/_posts/2020-09-30-writing-crl-scheduler/image05.png
new file mode 100644
index 0000000000..6d80447d09
Binary files /dev/null and b/content/en/blog/_posts/2020-09-30-writing-crl-scheduler/image05.png differ
diff --git a/content/en/blog/_posts/2020-09-30-writing-crl-scheduler/image06.png b/content/en/blog/_posts/2020-09-30-writing-crl-scheduler/image06.png
new file mode 100644
index 0000000000..d40b2eb0b6
Binary files /dev/null and b/content/en/blog/_posts/2020-09-30-writing-crl-scheduler/image06.png differ
diff --git a/content/en/blog/_posts/2020-09-30-writing-crl-scheduler/image07.png b/content/en/blog/_posts/2020-09-30-writing-crl-scheduler/image07.png
new file mode 100644
index 0000000000..fc3976040f
Binary files /dev/null and b/content/en/blog/_posts/2020-09-30-writing-crl-scheduler/image07.png differ
diff --git a/content/en/blog/_posts/2020-09-30-writing-crl-scheduler/index.md b/content/en/blog/_posts/2020-09-30-writing-crl-scheduler/index.md
new file mode 100644
index 0000000000..0fab185f98
--- /dev/null
+++ b/content/en/blog/_posts/2020-09-30-writing-crl-scheduler/index.md
@@ -0,0 +1,96 @@
+---
+layout: blog
+title: "A Custom Kubernetes Scheduler to Orchestrate Highly Available Applications"
+date: 2020-12-21
+slug: writing-crl-scheduler
+---
+
+**Author**: Chris Seto (Cockroach Labs)
+
+As long as you're willing to follow the rules, deploying on Kubernetes and air travel can be quite pleasant. More often than not, things will "just work". However, if one is interested in travelling with an alligator that must remain alive or scaling a database that must remain available, the situation is likely to become a bit more complicated. It may even be easier to build one's own plane or database for that matter. Travelling with reptiles aside, scaling a highly available stateful system is no trivial task.
+
+Scaling any system has two main components:
+1. Adding or removing infrastructure that the system will run on, and
+2. Ensuring that the system knows how to handle additional instances of itself being added and removed.
+
+Most stateless systems, web servers for example, are created without the need to be aware of peers. Stateful systems, which includes databases like CockroachDB, have to coordinate with their peer instances and shuffle around data. As luck would have it, CockroachDB handles data redistribution and replication. The tricky part is being able to tolerate failures during these operations by ensuring that data and instances are distributed across many failure domains (availability zones).
+
+One of Kubernetes' responsibilities is to place "resources" (e.g, a disk or container) into the cluster and satisfy the constraints they request. For example: "I must be in availability zone _A_" (see [Running in multiple zones](/docs/setup/best-practices/multiple-zones/#nodes-are-labeled)), or "I can't be placed onto the same node as this other Pod" (see [Affinity and anti-affinity](/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity)).
+
+As an addition to those constraints, Kubernetes offers [Statefulsets](/docs/concepts/workloads/controllers/statefulset/) that provide identity to Pods as well as persistent storage that "follows" these identified pods. Identity in a StatefulSet is handled by an increasing integer at the end of a pod's name. It's important to note that this integer must always be contiguous: in a StatefulSet, if pods 1 and 3 exist then pod 2 must also exist.
+
+Under the hood, CockroachCloud deploys each region of CockroachDB as a StatefulSet in its own Kubernetes cluster - see [Orchestrate CockroachDB in a Single Kubernetes Cluster](https://www.cockroachlabs.com/docs/stable/orchestrate-cockroachdb-with-kubernetes.html).
+In this article, I'll be looking at an individual region, one StatefulSet and one Kubernetes cluster which is distributed across at least three availability zones.
+
+A three-node CockroachCloud cluster would look something like this:
+
+
+
+When adding additional resources to the cluster we also distribute them across zones. For the speediest user experience, we add all Kubernetes nodes at the same time and then scale up the StatefulSet.
+
+
+
+Note that anti-affinities are satisfied no matter the order in which pods are assigned to Kubernetes nodes. In the example, pods 0, 1 and 2 were assigned to zones A, B, and C respectively, but pods 3 and 4 were assigned in a different order, to zones B and A respectively. The anti-affinity is still satisfied because the pods are still placed in different zones.
+
+To remove resources from a cluster, we perform these operations in reverse order.
+
+We first scale down the StatefulSet and then remove from the cluster any nodes lacking a CockroachDB pod.
+
+
+
+Now, remember that pods in a StatefulSet of size _n_ must have ids in the range `[0,n)`. When scaling down a StatefulSet by _m_, Kubernetes removes _m_ pods, starting from the highest ordinals and moving towards the lowest, [the reverse in which they were added](/docs/concepts/workloads/controllers/statefulset/#deployment-and-scaling-guarantees).
+Consider the cluster topology below:
+
+
+
+As ordinals 5 through 3 are removed from this cluster, the statefulset continues to have a presence across all 3 availability zones.
+
+
+
+However, Kubernetes' scheduler doesn't _guarantee_ the placement above as we expected at first.
+
+Our combined knowledge of the following is what lead to this misconception.
+* Kubernetes' ability to [automatically spread Pods across zone](/docs/setup/best-practices/multiple-zones/#pods-are-spread-across-zones)
+* The behavior that a StatefulSet with _n_ replicas, when Pods are being deployed, they are created sequentially, in order from `{0..n-1}`. See [StatefulSet](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#deployment-and-scaling-guarantees) for more details.
+
+Consider the following topology:
+
+
+
+These pods were created in order and they are spread across all availability zones in the cluster. When ordinals 5 through 3 are terminated, this cluster will lose its presence in zone C!
+
+
+
+Worse yet, our automation, at the time, would remove Nodes A-2, B-2, and C-2. Leaving CRDB-1 in an unscheduled state as persistent volumes are only available in the zone they are initially created in.
+
+To correct the latter issue, we now employ a "hunt and peck" approach to removing machines from a cluster. Rather than blindly removing Kubernetes nodes from the cluster, only nodes without a CockroachDB pod would be removed. The much more daunting task was to wrangle the Kubernetes scheduler.
+
+## A session of brainstorming left us with 3 options:
+
+### 1. Upgrade to kubernetes 1.18 and make use of Pod Topology Spread Constraints
+
+While this seems like it could have been the perfect solution, at the time of writing Kubernetes 1.18 was unavailable on the two most common managed Kubernetes services in public cloud, EKS and GKE.
+Furthermore, [pod topology spread constraints](/docs/concepts/workloads/pods/pod-topology-spread-constraints/) were still a [beta feature in 1.18](https://v1-18.docs.kubernetes.io/docs/concepts/workloads/pods/pod-topology-spread-constraints/) which meant that it [wasn't guaranteed to be available in managed clusters](https://cloud.google.com/kubernetes-engine/docs/concepts/types-of-clusters#kubernetes_feature_choices) even when v1.18 became available.
+The entire endeavour was concerningly reminiscent of checking [caniuse.com](https://caniuse.com/) when Internet Explorer 8 was still around.
+
+### 2. Deploy a statefulset _per zone_.
+
+Rather than having one StatefulSet distributed across all availability zones, a single StatefulSet with node affinities per zone would allow manual control over our zonal topology.
+Our team had considered this as an option in the past which made it particularly appealing.
+Ultimately, we decided to forego this option as it would have required a massive overhaul to our codebase and performing the migration on existing customer clusters would have been an equally large undertaking.
+
+### 3. Write a custom Kubernetes scheduler.
+
+Thanks to an example from [Kelsey Hightower](https://github.com/kelseyhightower/scheduler) and a blog post from [Banzai Cloud](https://banzaicloud.com/blog/k8s-custom-scheduler/), we decided to dive in head first and write our own [custom Kubernetes scheduler](/docs/tasks/extend-kubernetes/configure-multiple-schedulers/).
+Once our proof-of-concept was deployed and running, we quickly discovered that the Kubernetes' scheduler is also responsible for mapping persistent volumes to the Pods that it schedules.
+The output of [`kubectl get events`](/docs/tasks/extend-kubernetes/configure-multiple-schedulers/#verifying-that-the-pods-were-scheduled-using-the-desired-schedulers) had led us to believe there was another system at play.
+In our journey to find the component responsible for storage claim mapping, we discovered the [kube-scheduler plugin system](/docs/concepts/scheduling-eviction/scheduling-framework/). Our next POC was a `Filter` plugin that determined the appropriate availability zone by pod ordinal, and it worked flawlessly!
+
+Our [custom scheduler plugin](https://github.com/cockroachlabs/crl-scheduler) is open source and runs in all of our CockroachCloud clusters.
+Having control over how our StatefulSet pods are being scheduled has let us scale out with confidence.
+We may look into retiring our plugin once pod topology spread constraints are available in GKE and EKS, but the maintenance overhead has been surprisingly low.
+Better still: the plugin's implementation is orthogonal to our business logic. Deploying it, or retiring it for that matter, is as simple as changing the `schedulerName` field in our StatefulSet definitions.
+
+---
+
+_[Chris Seto](https://twitter.com/_ostriches) is a software engineer at Cockroach Labs and works on their Kubernetes automation for [CockroachCloud](https://cockroachlabs.cloud), CockroachDB._
diff --git a/content/en/blog/_posts/2020-12-02-dockershim-faq.md b/content/en/blog/_posts/2020-12-02-dockershim-faq.md
index 687921a7ce..b9676fc742 100644
--- a/content/en/blog/_posts/2020-12-02-dockershim-faq.md
+++ b/content/en/blog/_posts/2020-12-02-dockershim-faq.md
@@ -141,13 +141,14 @@ runtime where possible.
Another thing to look out for is anything expecting to run for system maintenance
or nested inside a container when building images will no longer work. For the
former, you can use the [`crictl`][cr] tool as a drop-in replacement (see [mapping from docker cli to crictl](https://kubernetes.io/docs/tasks/debug-application-cluster/crictl/#mapping-from-docker-cli-to-crictl)) and for the
-latter you can use newer container build options like [img], [buildah], or
-[kaniko] that don’t require Docker.
+latter you can use newer container build options like [img], [buildah],
+[kaniko], or [buildkit-cli-for-kubectl] that don’t require Docker.
[cr]: https://github.com/kubernetes-sigs/cri-tools
[img]: https://github.com/genuinetools/img
[buildah]: https://github.com/containers/buildah
[kaniko]: https://github.com/GoogleContainerTools/kaniko
+[buildkit-cli-for-kubectl]: https://github.com/vmware-tanzu/buildkit-cli-for-kubectl
For containerd, you can start with their [documentation] to see what configuration
options are available as you migrate things over.
diff --git a/content/en/blog/_posts/2020-12-10-Kubernetes-Volume-Snapshot-Moves-to-GA.md b/content/en/blog/_posts/2020-12-10-Kubernetes-Volume-Snapshot-Moves-to-GA.md
new file mode 100644
index 0000000000..0924cbcc91
--- /dev/null
+++ b/content/en/blog/_posts/2020-12-10-Kubernetes-Volume-Snapshot-Moves-to-GA.md
@@ -0,0 +1,224 @@
+---
+layout: blog
+title: 'Kubernetes 1.20: Kubernetes Volume Snapshot Moves to GA'
+date: 2020-12-10
+slug: kubernetes-1.20-volume-snapshot-moves-to-ga
+---
+
+**Authors**: Xing Yang, VMware & Xiangqian Yu, Google
+
+The Kubernetes Volume Snapshot feature is now GA in Kubernetes v1.20. It was introduced as [alpha](https://kubernetes.io/blog/2018/10/09/introducing-volume-snapshot-alpha-for-kubernetes/) in Kubernetes v1.12, followed by a [second alpha](https://kubernetes.io/blog/2019/01/17/update-on-volume-snapshot-alpha-for-kubernetes/) with breaking changes in Kubernetes v1.13, and promotion to [beta](https://kubernetes.io/blog/2019/12/09/kubernetes-1-17-feature-cis-volume-snapshot-beta/) in Kubernetes 1.17. This blog post summarizes the changes releasing the feature from beta to GA.
+
+## What is a volume snapshot?
+
+Many storage systems (like Google Cloud Persistent Disks, Amazon Elastic Block Storage, and many on-premise storage systems) provide the ability to create a “snapshot” of a persistent volume. A snapshot represents a point-in-time copy of a volume. A snapshot can be used either to rehydrate a new volume (pre-populated with the snapshot data) or to restore an existing volume to a previous state (represented by the snapshot).
+
+## Why add volume snapshots to Kubernetes?
+
+Kubernetes aims to create an abstraction layer between distributed applications and underlying clusters so that applications can be agnostic to the specifics of the cluster they run on and application deployment requires no “cluster-specific” knowledge.
+
+The Kubernetes Storage SIG identified snapshot operations as critical functionality for many stateful workloads. For example, a database administrator may want to snapshot a database’s volumes before starting a database operation.
+
+By providing a standard way to trigger volume snapshot operations in Kubernetes, this feature allows Kubernetes users to incorporate snapshot operations in a portable manner on any Kubernetes environment regardless of the underlying storage.
+
+Additionally, these Kubernetes snapshot primitives act as basic building blocks that unlock the ability to develop advanced enterprise-grade storage administration features for Kubernetes, including application or cluster level backup solutions.
+
+## What’s new since beta?
+
+With the promotion of Volume Snapshot to GA, the feature is enabled by default on standard Kubernetes deployments and cannot be turned off.
+
+Many enhancements have been made to improve the quality of this feature and to make it production-grade.
+
+- The Volume Snapshot APIs and client library were moved to a separate Go module.
+
+- A snapshot validation webhook has been added to perform necessary validation on volume snapshot objects. More details can be found in the [Volume Snapshot Validation Webhook Kubernetes Enhancement Proposal](https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/1900-volume-snapshot-validation-webhook).
+
+- Along with the validation webhook, the volume snapshot controller will start labeling invalid snapshot objects that already existed. This allows users to identify, remove any invalid objects, and correct their workflows. Once the API is switched to the v1 type, those invalid objects will not be deletable from the system.
+
+- To provide better insights into how the snapshot feature is performing, an initial set of operation metrics has been added to the volume snapshot controller.
+
+- There are more end-to-end tests, running on GCP, that validate the feature in a real Kubernetes cluster. Stress tests (based on Google Persistent Disk and `hostPath` CSI Drivers) have been introduced to test the robustness of the system.
+
+Other than introducing tightening validation, there is no difference between the v1beta1 and v1 Kubernetes volume snapshot API. In this release (with Kubernetes 1.20), both v1 and v1beta1 are served while the stored API version is still v1beta1. Future releases will switch the stored version to v1 and gradually remove v1beta1 support.
+
+## Which CSI drivers support volume snapshots?
+
+Snapshots are only supported for CSI drivers, not for in-tree or FlexVolume drivers. Ensure the deployed CSI driver on your cluster has implemented the snapshot interfaces. For more information, see [Container Storage Interface (CSI) for Kubernetes GA](https://kubernetes.io/blog/2019/01/15/container-storage-interface-ga/).
+
+Currently more than [50 CSI drivers](https://kubernetes-csi.github.io/docs/drivers.html) support the Volume Snapshot feature. The [GCE Persistent Disk CSI Driver](https://github.com/kubernetes-sigs/gcp-compute-persistent-disk-csi-driver) has gone through the tests for upgrading from volume snapshots beta to GA. GA level support for other CSI drivers should be available soon.
+
+## Who builds products using volume snapshots?
+
+As of the publishing of this blog, the following participants from the [Kubernetes Data Protection Working Group](https://github.com/kubernetes/community/tree/master/wg-data-protection) are building products or have already built products using Kubernetes volume snapshots.
+
+- [Dell-EMC: PowerProtect](https://www.delltechnologies.com/en-us/data-protection/powerprotect-data-manager.htm)
+- [Druva](https://www.druva.com/)
+- [Kasten K10](https://www.kasten.io/)
+- [NetApp: Project Astra](https://cloud.netapp.com/project-astra)
+- [Portworx (PX-Backup)](https://portworx.com/products/px-backup/)
+- [Pure Storage (Pure Service Orchestrator)](https://github.com/purestorage/pso-csi)
+- [Red Hat OpenShift Container Storage](https://www.redhat.com/en/technologies/cloud-computing/openshift-container-storage)
+- [Robin Cloud Native Storage](https://robin.io/storage/)
+- [TrilioVault for Kubernetes](https://docs.trilio.io/kubernetes/)
+- [Velero plugin for CSI](https://github.com/vmware-tanzu/velero-plugin-for-csi)
+
+## How to deploy volume snapshots?
+
+Volume Snapshot feature contains the following components:
+
+- [Kubernetes Volume Snapshot CRDs](https://github.com/kubernetes-csi/external-snapshotter/tree/master/client/config/crd)
+- [Volume snapshot controller](https://github.com/kubernetes-csi/external-snapshotter/tree/master/pkg/common-controller)
+- [Snapshot validation webhook](https://github.com/kubernetes-csi/external-snapshotter/tree/master/pkg/validation-webhook)
+- CSI Driver along with [CSI Snapshotter sidecar](https://github.com/kubernetes-csi/external-snapshotter/tree/master/pkg/sidecar-controller)
+
+It is strongly recommended that Kubernetes distributors bundle and deploy the volume snapshot controller, CRDs, and validation webhook as part of their Kubernetes cluster management process (independent of any CSI Driver).
+
+{{< warning >}}
+
+The snapshot validation webhook serves as a critical component to transition smoothly from using v1beta1 to v1 API. Not installing the snapshot validation webhook makes prevention of invalid volume snapshot objects from creation/updating impossible, which in turn will block deletion of invalid volume snapshot objects in coming upgrades.
+
+{{< /warning >}}
+
+If your cluster does not come pre-installed with the correct components, you may manually install them. See the [CSI Snapshotter](https://github.com/kubernetes-csi/external-snapshotter#readme) README for details.
+
+## How to use volume snapshots?
+
+Assuming all the required components (including CSI driver) have been already deployed and running on your cluster, you can create volume snapshots using the `VolumeSnapshot` API object, or use an existing `VolumeSnapshot` to restore a PVC by specifying the VolumeSnapshot data source on it. For more details, see the [volume snapshot documentation](/docs/concepts/storage/volume-snapshots/).
+
+{{< note >}} The Kubernetes Snapshot API does not provide any application consistency guarantees. You have to prepare your application (pause application, freeze filesystem etc.) before taking the snapshot for data consistency either manually or using higher level APIs/controllers. {{< /note >}}
+
+### Dynamically provision a volume snapshot
+
+To dynamically provision a volume snapshot, create a `VolumeSnapshotClass` API object first.
+
+```yaml
+apiVersion: snapshot.storage.k8s.io/v1
+kind: VolumeSnapshotClass
+metadata:
+ name: test-snapclass
+driver: testdriver.csi.k8s.io
+deletionPolicy: Delete
+parameters:
+ csi.storage.k8s.io/snapshotter-secret-name: mysecret
+ csi.storage.k8s.io/snapshotter-secret-namespace: mysecretnamespace
+```
+
+Then create a `VolumeSnapshot` API object from a PVC by specifying the volume snapshot class.
+
+```yaml
+apiVersion: snapshot.storage.k8s.io/v1
+kind: VolumeSnapshot
+metadata:
+ name: test-snapshot
+ namespace: ns1
+spec:
+ volumeSnapshotClassName: test-snapclass
+ source:
+ persistentVolumeClaimName: test-pvc
+```
+
+### Importing an existing volume snapshot with Kubernetes
+
+To import a pre-existing volume snapshot into Kubernetes, manually create a `VolumeSnapshotContent` object first.
+
+```yaml
+apiVersion: snapshot.storage.k8s.io/v1
+kind: VolumeSnapshotContent
+metadata:
+ name: test-content
+spec:
+ deletionPolicy: Delete
+ driver: testdriver.csi.k8s.io
+ source:
+ snapshotHandle: 7bdd0de3-xxx
+ volumeSnapshotRef:
+ name: test-snapshot
+ namespace: default
+```
+
+Then create a `VolumeSnapshot` object pointing to the `VolumeSnapshotContent` object.
+
+```yaml
+apiVersion: snapshot.storage.k8s.io/v1
+kind: VolumeSnapshot
+metadata:
+ name: test-snapshot
+spec:
+ source:
+ volumeSnapshotContentName: test-content
+```
+
+### Rehydrate volume from snapshot
+
+A bound and ready `VolumeSnapshot` object can be used to rehydrate a new volume with data pre-populated from snapshotted data as shown here:
+
+```yaml
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+ name: pvc-restore
+ namespace: demo-namespace
+spec:
+ storageClassName: test-storageclass
+ dataSource:
+ name: test-snapshot
+ kind: VolumeSnapshot
+ apiGroup: snapshot.storage.k8s.io
+ accessModes:
+ - ReadWriteOnce
+ resources:
+ requests:
+ storage: 1Gi
+```
+
+## How to add support for snapshots in a CSI driver?
+
+See the [CSI spec](https://github.com/container-storage-interface/spec/blob/master/spec.md) and the [Kubernetes-CSI Driver Developer Guide](https://kubernetes-csi.github.io/docs/snapshot-restore-feature.html) for more details on how to implement the snapshot feature in a CSI driver.
+
+## What are the limitations?
+
+The GA implementation of volume snapshots for Kubernetes has the following limitations:
+
+- Does not support reverting an existing PVC to an earlier state represented by a snapshot (only supports provisioning a new volume from a snapshot).
+
+### How to learn more?
+
+The code repository for snapshot APIs and controller is here: https://github.com/kubernetes-csi/external-snapshotter
+
+Check out additional documentation on the snapshot feature here: http://k8s.io/docs/concepts/storage/volume-snapshots and https://kubernetes-csi.github.io/docs/
+
+## How to get involved?
+
+This project, like all of Kubernetes, is the result of hard work by many contributors from diverse backgrounds working together.
+
+We offer a huge thank you to the contributors who stepped up these last few quarters to help the project reach GA. We want to thank Saad Ali, Michelle Au, Tim Hockin, and Jordan Liggitt for their insightful reviews and thorough consideration with the design, thank Andi Li for his work on adding the support of the snapshot validation webhook, thank Grant Griffiths on implementing metrics support in the snapshot controller and handling password rotation in the validation webhook, thank Chris Henzie, Raunak Shah, and Manohar Reddy for writing critical e2e tests to meet the scalability and stability requirements for graduation, thank Kartik Sharma for moving snapshot APIs and client lib to a separate go module, and thank Raunak Shah and Prafull Ladha for their help with upgrade testing from beta to GA.
+
+There are many more people who have helped to move the snapshot feature from beta to GA. We want to thank everyone who has contributed to this effort:
+- [Andi Li](https://github.com/AndiLi99)
+- [Ben Swartzlander](https://github.com/bswartz)
+- [Chris Henzie](https://github.com/chrishenzie)
+- [Christian Huffman](https://github.com/huffmanca)
+- [Grant Griffiths](https://github.com/ggriffiths)
+- [Humble Devassy Chirammal](https://github.com/humblec)
+- [Jan Šafránek](https://github.com/jsafrane)
+- [Jiawei Wang](https://github.com/Jiawei0227)
+- [Jing Xu](https://github.com/jingxu97)
+- [Jordan Liggitt](https://github.com/liggitt)
+- [Kartik Sharma](https://github.com/Kartik494)
+- [Madhu Rajanna](https://github.com/Madhu-1)
+- [Manohar Reddy](https://github.com/boddumanohar)
+- [Michelle Au](https://github.com/msau42)
+- [Patrick Ohly](https://github.com/pohly)
+- [Prafull Ladha](https://github.com/prafull01)
+- [Prateek Pandey](https://github.com/prateekpandey14)
+- [Raunak Shah](https://github.com/RaunakShah)
+- [Saad Ali](https://github.com/saad-ali)
+- [Saikat Roychowdhury](https://github.com/saikat-royc)
+- [Tim Hockin](https://github.com/thockin)
+- [Xiangqian Yu](https://github.com/yuxiangqian)
+- [Xing Yang](https://github.com/xing-yang)
+- [Zhu Can](https://github.com/zhucan)
+
+For those interested in getting involved with the design and development of CSI or any part of the Kubernetes Storage system, join the [Kubernetes Storage Special Interest Group](https://github.com/kubernetes/community/tree/master/sig-storage) (SIG). We’re rapidly growing and always welcome new contributors.
+
+We also hold regular [Data Protection Working Group meetings](https://docs.google.com/document/d/15tLCV3csvjHbKb16DVk-mfUmFry_Rlwo-2uG6KNGsfw/edit#). New attendees are welcome to join in discussions.
diff --git a/content/en/blog/_posts/2020-12-10-Pod-Impersonation-and-Short-lived-Volumes-in-CSI-Drivers.md b/content/en/blog/_posts/2020-12-10-Pod-Impersonation-and-Short-lived-Volumes-in-CSI-Drivers.md
new file mode 100644
index 0000000000..ef20ad5210
--- /dev/null
+++ b/content/en/blog/_posts/2020-12-10-Pod-Impersonation-and-Short-lived-Volumes-in-CSI-Drivers.md
@@ -0,0 +1,53 @@
+---
+layout: blog
+title: 'Kubernetes 1.20: Pod Impersonation and Short-lived Volumes in CSI Drivers'
+date: 2020-12-18
+slug: kubernetes-1.20-pod-impersonation-short-lived-volumes-in-csi
+---
+
+**Author**: Shihang Zhang (Google)
+
+Typically when a [CSI](https://github.com/container-storage-interface/spec/blob/baa71a34651e5ee6cb983b39c03097d7aa384278/spec.md) driver mounts credentials such as secrets and certificates, it has to authenticate against storage providers to access the credentials. However, the access to those credentials are controlled on the basis of the pods' identities rather than the CSI driver's identity. CSI drivers, therefore, need some way to retrieve pod's service account token.
+
+Currently there are two suboptimal approaches to achieve this, either by granting CSI drivers the permission to use TokenRequest API or by reading tokens directly from the host filesystem.
+
+Both of them exhibit the following drawbacks:
+
+- Violating the principle of least privilege
+- Every CSI driver needs to re-implement the logic of getting the pod’s service account token
+
+The second approach is more problematic due to:
+
+- The audience of the token defaults to the kube-apiserver
+- The token is not guaranteed to be available (e.g. `AutomountServiceAccountToken=false`)
+- The approach does not work for CSI drivers that run as a different (non-root) user from the pods. See [file permission section for service account token](https://github.com/kubernetes/enhancements/blob/f40c24a5da09390bd521be535b38a4dbab09380c/keps/sig-storage/20180515-svcacct-token-volumes.md#file-permission)
+- The token might be legacy Kubernetes service account token which doesn’t expire if `BoundServiceAccountTokenVolume=false`
+
+Kubernetes 1.20 introduces an alpha feature, `CSIServiceAccountToken`, to improve the security posture. The new feature allows CSI drivers to receive pods' [bound service account tokens](https://github.com/kubernetes/enhancements/blob/master/keps/sig-auth/1205-bound-service-account-tokens/README.md).
+
+This feature also provides a knob to re-publish volumes so that short-lived volumes can be refreshed.
+
+## Pod Impersonation
+
+### Using GCP APIs
+
+Using [Workload Identity](https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity), a Kubernetes service account can authenticate as a Google service account when accessing Google Cloud APIs. If a CSI driver needs to access GCP APIs on behalf of the pods that it is mounting volumes for, it can use the pod's service account token to [exchange for GCP tokens](https://cloud.google.com/iam/docs/reference/sts/rest). The pod's service account token is plumbed through the volume context in `NodePublishVolume` RPC calls when the feature `CSIServiceAccountToken` is enabled. For example: accessing [Google Secret Manager](https://cloud.google.com/secret-manager/) via a [secret store CSI driver](https://github.com/GoogleCloudPlatform/secrets-store-csi-driver-provider-gcp).
+
+### Using Vault
+
+If users configure [Kubernetes as an auth method](https://www.vaultproject.io/docs/auth/kubernetes), Vault uses the `TokenReview` API to validate the Kubernetes service account token. For CSI drivers using Vault as resources provider, they need to present the pod's service account to Vault. For example, [secrets store CSI driver](https://github.com/hashicorp/secrets-store-csi-driver-provider-vault) and [cert manager CSI driver](https://github.com/jetstack/cert-manager-csi).
+
+## Short-lived Volumes
+
+To keep short-lived volumes such as certificates effective, CSI drivers can specify `RequiresRepublish=true` in their`CSIDriver` object to have the kubelet periodically call `NodePublishVolume` on mounted volumes. These republishes allow CSI drivers to ensure that the volume content is up-to-date.
+
+## Next steps
+
+This feature is alpha and projected to move to beta in 1.21. See more in the following KEP and CSI documentation:
+
+- [KEP-1855: Service Account Token for CSI Driver](https://github.com/kubernetes/enhancements/blob/master/keps/sig-storage/1855-csi-driver-service-account-token/README.md)
+- [Token Requests](https://kubernetes-csi.github.io/docs/token-requests.html)
+
+Your feedback is always welcome!
+- SIG-Auth [meets regularly](https://github.com/kubernetes/community/tree/master/sig-auth#meetings) and can be reached via [Slack and the mailing list](https://github.com/kubernetes/community/tree/master/sig-auth#contact)
+- SIG-Storage [meets regularly](https://github.com/kubernetes/community/tree/master/sig-storage#meetings) and can be reached via [Slack and the mailing list](https://github.com/kubernetes/community/tree/master/sig-storage#contact).
diff --git a/content/en/blog/_posts/2020-12-14-Granular-Control-of-Volume-Permission-Changes.md b/content/en/blog/_posts/2020-12-14-Granular-Control-of-Volume-Permission-Changes.md
new file mode 100644
index 0000000000..0a0ad0879c
--- /dev/null
+++ b/content/en/blog/_posts/2020-12-14-Granular-Control-of-Volume-Permission-Changes.md
@@ -0,0 +1,59 @@
+---
+layout: blog
+title: 'Kubernetes 1.20: Granular Control of Volume Permission Changes'
+date: 2020-12-14
+slug: kubernetes-release-1.20-fsGroupChangePolicy-fsGroupPolicy
+---
+
+**Authors**: Hemant Kumar, Red Hat & Christian Huffman, Red Hat
+
+Kubernetes 1.20 brings two important beta features, allowing Kubernetes admins and users alike to have more adequate control over how volume permissions are applied when a volume is mounted inside a Pod.
+
+### Allow users to skip recursive permission changes on mount
+Traditionally if your pod is running as a non-root user ([which you should](https://twitter.com/thockin/status/1333892204490735617)), you must specify a `fsGroup` inside the pod’s security context so that the volume can be readable and writable by the Pod. This requirement is covered in more detail in [here](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/).
+
+But one side-effect of setting `fsGroup` is that, each time a volume is mounted, Kubernetes must recursively `chown()` and `chmod()` all the files and directories inside the volume - with a few exceptions noted below. This happens even if group ownership of the volume already matches the requested `fsGroup`, and can be pretty expensive for larger volumes with lots of small files, which causes pod startup to take a long time. This scenario has been a [known problem](https://github.com/kubernetes/kubernetes/issues/69699) for a while, and in Kubernetes 1.20 we are providing knobs to opt-out of recursive permission changes if the volume already has the correct permissions.
+
+When configuring a pod’s security context, set `fsGroupChangePolicy` to "OnRootMismatch" so if the root of the volume already has the correct permissions, the recursive permission change can be skipped. Kubernetes ensures that permissions of the top-level directory are changed last the first time it applies permissions.
+
+```yaml
+securityContext:
+ runAsUser: 1000
+ runAsGroup: 3000
+ fsGroup: 2000
+ fsGroupChangePolicy: "OnRootMismatch"
+```
+You can learn more about this in [Configure volume permission and ownership change policy for Pods](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#configure-volume-permission-and-ownership-change-policy-for-pods).
+
+### Allow CSI Drivers to declare support for fsGroup based permissions
+
+Although the previous section implied that Kubernetes _always_ recursively changes permissions of a volume if a Pod has a `fsGroup`, this is not strictly true. For certain multi-writer volume types, such as NFS or Gluster, the cluster doesn’t perform recursive permission changes even if the pod has a `fsGroup`. Other volume types may not even support `chown()`/`chmod()`, which rely on Unix-style permission control primitives.
+
+So how do we know when to apply recursive permission changes and when we shouldn't? For in-tree storage drivers, this was relatively simple. For [CSI](https://kubernetes-csi.github.io/docs/introduction.html#introduction) drivers that could span a multitude of platforms and storage types, this problem can be a bigger challenge.
+
+Previously, whenever a CSI volume was mounted to a Pod, Kubernetes would attempt to automatically determine if the permissions and ownership should be modified. These methods were imprecise and could cause issues as we already mentioned, depending on the storage type.
+
+The CSIDriver custom resource now has a `.spec.fsGroupPolicy` field, allowing storage drivers to explicitly opt in or out of these recursive modifications. By having the CSI driver specify a policy for the backing volumes, Kubernetes can avoid needless modification attempts. This optimization helps to reduce volume mount time and also cuts own reporting errors about modifications that would never succeed.
+
+#### CSIDriver FSGroupPolicy API
+
+Three FSGroupPolicy values are available as of Kubernetes 1.20, with more planned for future releases.
+
+- **ReadWriteOnceWithFSType** - This is the default policy, applied if no `fsGroupPolicy` is defined; this preserves the behavior from previous Kubernetes releases. Each volume is examined at mount time to determine if permissions should be recursively applied.
+- **File** - Always attempt to apply permission modifications, regardless of the filesystem type or PersistentVolumeClaim’s access mode.
+- **None** - Never apply permission modifications.
+
+#### How do I use it?
+The only configuration needed is defining `fsGroupPolicy` inside of the `.spec` for a CSIDriver. Once that element is defined, any subsequently mounted volumes will automatically use the defined policy. There’s no additional deployment required!
+
+#### What’s next?
+
+Depending on feedback and adoption, the Kubernetes team plans to push these implementations to GA in either 1.21 or 1.22.
+
+### How can I learn more?
+This feature is explained in more detail in Kubernetes project documentation: [CSI Driver fsGroup Support](https://kubernetes-csi.github.io/docs/support-fsgroup.html) and [Configure volume permission and ownership change policy for Pods ](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#configure-volume-permission-and-ownership-change-policy-for-pods).
+
+### How do I get involved?
+The [Kubernetes Slack channel #csi](https://kubernetes.slack.com/messages/csi) and any of the [standard SIG Storage communication channels](https://github.com/kubernetes/community/blob/master/sig-storage/README.md#contact) are great mediums to reach out to the SIG Storage and the CSI team.
+
+Those interested in getting involved with the design and development of CSI or any part of the Kubernetes Storage system, join the [Kubernetes Storage Special Interest Group (SIG)](https://github.com/kubernetes/community/tree/master/sig-storage). We’re rapidly growing and always welcome new contributors.
diff --git a/content/en/blog/_posts/2020-12-16-third-party-device-metrics-hits-ga.md b/content/en/blog/_posts/2020-12-16-third-party-device-metrics-hits-ga.md
new file mode 100644
index 0000000000..a4fa02e4fc
--- /dev/null
+++ b/content/en/blog/_posts/2020-12-16-third-party-device-metrics-hits-ga.md
@@ -0,0 +1,134 @@
+---
+layout: blog
+title: 'Third Party Device Metrics Reaches GA'
+date: 2020-12-16
+slug: third-party-device-metrics-reaches-ga
+---
+
+ **Authors:** Renaud Gaubert (NVIDIA), David Ashpole (Google), and Pramod Ramarao (NVIDIA)
+
+ With Kubernetes 1.20, infrastructure teams who manage large scale Kubernetes clusters, are seeing the graduation of two exciting and long awaited features:
+ * The Pod Resources API (introduced in 1.13) is finally graduating to GA. This allows Kubernetes plugins to obtain information about the node’s resource usage and assignment; for example: which pod/container consumes which device.
+ * The `DisableAcceleratorMetrics` feature (introduced in 1.19) is graduating to beta and will be enabled by default. This removes device metrics reported by the kubelet in favor of the new plugin architecture.
+
+Many of the features related to fundamental device support (device discovery, plugin, and monitoring) are reaching a strong level of stability.
+Kubernetes users should see these features as stepping stones to enable more complex use cases (networking, scheduling, storage, etc.)!
+
+One such example is Non Uniform Memory Access (NUMA) placement where, when selecting a device, an application typically wants to ensure that data transfer between CPU Memory and Device Memory is as fast as possible. In some cases, incorrect NUMA placement can nullify the benefit of offloading compute to an external device.
+
+If these are topics of interest to you, consider joining the [Kubernetes Node Special Insterest Group](https://github.com/kubernetes/community/tree/master/sig-node) (SIG) for all topics related to the Kubernetes node, the COD (container orchestrated device) workgroup for topics related to runtimes, or the resource management forum for topics related to resource management!
+
+## The Pod Resources API - Why does it need to exist?
+
+Kubernetes is a vendor neutral platform. If we want it to support device monitoring, adding vendor-specific code in the Kubernetes code base is not an ideal solution. Ultimately, devices are a domain where deep expertise is needed and the best people to add and maintain code in that area are the device vendors themselves.
+
+The Pod Resources API was built as a solution to this issue. Each vendor can build and maintain their own out-of-tree monitoring plugin. This monitoring plugin, often deployed as a separate pod within a cluster, can then associate the metrics a device emits with the associated pod that's using it.
+
+For example, use the NVIDIA GPU dcgm-exporter to scrape metrics in Prometheus format:
+
+```
+$ curl -sL http://127.0.01:8080/metrics
+
+
+# HELP DCGM_FI_DEV_SM_CLOCK SM clock frequency (in MHz).
+# TYPE DCGM_FI_DEV_SM_CLOCK gauge
+# HELP DCGM_FI_DEV_MEM_CLOCK Memory clock frequency (in MHz).
+# TYPE DCGM_FI_DEV_MEM_CLOCK gauge
+# HELP DCGM_FI_DEV_MEMORY_TEMP Memory temperature (in C).
+# TYPE DCGM_FI_DEV_MEMORY_TEMP gauge
+...
+DCGM_FI_DEV_SM_CLOCK{gpu="0", UUID="GPU-604ac76c-d9cf-fef3-62e9-d92044ab6e52",container="foo",namespace="bar",pod="baz"} 139
+DCGM_FI_DEV_MEM_CLOCK{gpu="0", UUID="GPU-604ac76c-d9cf-fef3-62e9-d92044ab6e52",container="foo",namespace="bar",pod="baz"} 405
+DCGM_FI_DEV_MEMORY_TEMP{gpu="0", UUID="GPU-604ac76c-d9cf-fef3-62e9-d92044ab6e52",container="foo",namespace="bar",pod="baz"} 9223372036854775794
+```
+
+Each agent is expected to adhere to the node monitoring guidelines. In other words, plugins are expected to generate metrics in Prometheus format, and new metrics should not have any dependency on the Kubernetes base directly.
+
+This allows consumers of the metrics to use a compatible monitoring pipeline to collect and analyze metrics from a variety of agents, even if they are maintained by different vendors.
+
+
+
+## Disabling the NVIDIA GPU metrics - Warning {#nvidia-gpu-metrics-deprecated}
+
+With the graduation of the plugin monitoring system, Kubernetes is deprecating the NVIDIA GPU metrics that are being reported by the kubelet.
+
+With the [DisableAcceleratorMetrics](/docs/concepts/cluster-administration/system-metrics/#disable-accelerator-metrics) feature being enabled by default in Kubernetes 1.20, NVIDIA GPUs are no longer special citizens in Kubernetes. This is a good thing in the spirit of being vendor-neutral, and enables the most suited people to maintain their plugin on their own release schedule!
+
+Users will now need to either install the [NVIDIA GDGM exporter](https://github.com/NVIDIA/gpu-monitoring-tools) or use [bindings](https://github.com/nvidia/go-nvml) to gather more accurate and complete metrics about NVIDIA GPUs. This deprecation means that you can no longer rely on metrics that were reported by kubelet, such as `container_accelerator_duty_cycle` or `container_accelerator_memory_used_bytes` which were used to gather NVIDIA GPU memory utilization.
+
+This means that users who used to rely on the NVIDIA GPU metrics reported by the kubelet, will need to update their reference and deploy the NVIDIA plugin. Namely the different metrics reported by Kubernetes map to the following metrics:
+
+| Kubernetes Metrics | NVIDIA dcgm-exporter metric |
+| ------------------------------------------ | ------------------------------------------- |
+| `container_accelerator_duty_cycle` | `DCGM_FI_DEV_GPU_UTIL` |
+| `container_accelerator_memory_used_bytes` | `DCGM_FI_DEV_FB_USED` |
+| `container_accelerator_memory_total_bytes` | `DCGM_FI_DEV_FB_FREE + DCGM_FI_DEV_FB_USED` |
+
+You might also be interested in other metrics such as `DCGM_FI_DEV_GPU_TEMP` (the GPU temperature) or DCGM_FI_DEV_POWER_USAGE (the power usage). The [default set](https://github.com/NVIDIA/gpu-monitoring-tools/blob/d5c9bb55b4d1529ca07068b7f81e690921ce2b59/etc/dcgm-exporter/default-counters.csv) is available in Nvidia's [Data Center GPU Manager documentation](https://docs.nvidia.com/datacenter/dcgm/latest/dcgm-api/group__dcgmFieldIdentifiers.html).
+
+Note that for this release you can still set the `DisableAcceleratorMetrics` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) to _false_, effectively re-enabling the ability for the kubelet to report NVIDIA GPU metrics.
+
+Paired with the graduation of the Pod Resources API, these tools can be used to generate GPU telemetry [that can be used in visualization dashboards](https://grafana.com/grafana/dashboards/12239), below is an example:
+
+
+
+## The Pod Resources API - What can I go on to do with this?
+
+As soon as this interface was introduced, many vendors started using it for widely different use cases! To list a few examples:
+
+The [kuryr-kubernetes](https://github.com/openstack/kuryr-kubernetes) CNI plugin in tandem with [intel-sriov-device-plugin](https://github.com/intel/sriov-network-device-plugin). This allowed the CNI plugin to know which allocation of SR-IOV Virtual Functions (VFs) the kubelet made and use that information to correctly setup the container network namespace and use a device with the appropriate NUMA node. We also expect this interface to be used to track the allocated and available resources with information about the NUMA topology of the worker node.
+
+Another use-case is GPU telemetry, where GPU metrics can be associated with the containers and pods that the GPU is assigned to. One such example is the NVIDIA `dcgm-exporter`, but others can be easily built in the same paradigm.
+
+The Pod Resources API is a simple gRPC service which informs clients of the pods the kubelet knows. The information concerns the devices assignment the kubelet made and the assignment of CPUs. This information is obtained from the internal state of the kubelet's Device Manager and CPU Manager respectively.
+
+You can see below a sample example of the API and how a go client could use that information in a few lines:
+
+```
+service PodResourcesLister {
+ rpc List(ListPodResourcesRequest) returns (ListPodResourcesResponse) {}
+ rpc GetAllocatableResources(AllocatableResourcesRequest) returns (AllocatableResourcesResponse) {}
+
+ // Kubernetes 1.21
+ rpc Watch(WatchPodResourcesRequest) returns (stream WatchPodResourcesResponse) {}
+}
+```
+
+```go
+func main() {
+ ctx, cancel := context.WithTimeout(context.Background(), connectionTimeout)
+ defer cancel()
+
+ socket := "/var/lib/kubelet/pod-resources/kubelet.sock"
+ conn, err := grpc.DialContext(ctx, socket, grpc.WithInsecure(), grpc.WithBlock(),
+ grpc.WithDialer(func(addr string, timeout time.Duration) (net.Conn, error) {
+ return net.DialTimeout("unix", addr, timeout)
+ }),
+ )
+
+ if err != nil {
+ panic(err)
+ }
+
+ client := podresourcesapi.NewPodResourcesListerClient(conn)
+ resp, err := client.List(ctx, &podresourcesapi.ListPodResourcesRequest{})
+ if err != nil {
+ panic(err)
+ }
+ net.Printf("%+v\n", resp)
+}
+```
+
+Finally, note that you can watch the number of requests made to the Pod Resources endpoint by watching the new kubelet metric called `pod_resources_endpoint_requests_total` on the kubelet's `/metrics` endpoint.
+
+## Is device monitoring suitable for production? Can I extend it? Can I contribute?
+
+Yes! This feature released in 1.13, almost 2 years ago, has seen broad adoption, is already used by different cloud managed services, and with its graduation to G.A in Kubernetes 1.20 is production ready!
+
+If you are a device vendor, you can start using it today! If you just want to monitor the devices in your cluster, go get the latest version of your monitoring plugin!
+
+If you feel passionate about that area, join the kubernetes community, help improve the API or contribute the device monitoring plugins!
+
+## Acknowledgements
+
+We thank the members of the community who have contributed to this feature or given feedback including members of WG-Resource-Management, SIG-Node and the Resource management forum!
\ No newline at end of file
diff --git a/content/en/blog/_posts/image01.png b/content/en/blog/_posts/image01.png
new file mode 100644
index 0000000000..91e8856139
Binary files /dev/null and b/content/en/blog/_posts/image01.png differ
diff --git a/content/en/blog/_posts/image02.png b/content/en/blog/_posts/image02.png
new file mode 100644
index 0000000000..dfd14d7cdc
Binary files /dev/null and b/content/en/blog/_posts/image02.png differ
diff --git a/content/en/blog/_posts/image03.png b/content/en/blog/_posts/image03.png
new file mode 100644
index 0000000000..443a6f2d67
Binary files /dev/null and b/content/en/blog/_posts/image03.png differ
diff --git a/content/en/blog/_posts/image04.png b/content/en/blog/_posts/image04.png
new file mode 100644
index 0000000000..e107adc88b
Binary files /dev/null and b/content/en/blog/_posts/image04.png differ
diff --git a/content/en/blog/_posts/image05.png b/content/en/blog/_posts/image05.png
new file mode 100644
index 0000000000..6d80447d09
Binary files /dev/null and b/content/en/blog/_posts/image05.png differ
diff --git a/content/en/blog/_posts/image06.png b/content/en/blog/_posts/image06.png
new file mode 100644
index 0000000000..d40b2eb0b6
Binary files /dev/null and b/content/en/blog/_posts/image06.png differ
diff --git a/content/en/blog/_posts/image07.png b/content/en/blog/_posts/image07.png
new file mode 100644
index 0000000000..fc3976040f
Binary files /dev/null and b/content/en/blog/_posts/image07.png differ
diff --git a/content/en/case-studies/appdirect/index.html b/content/en/case-studies/appdirect/index.html
index ffb06c67b9..dbb902d06b 100644
--- a/content/en/case-studies/appdirect/index.html
+++ b/content/en/case-studies/appdirect/index.html
@@ -13,7 +13,7 @@ new_case_study_styles: true
heading_background: /images/case-studies/appdirect/banner1.jpg
heading_title_logo: /images/appdirect_logo.png
subheading: >
- AppDirect: How AppDirect Supported the 10x Growth of Its Engineering Staff with Kubernetess
+ AppDirect: How AppDirect Supported the 10x Growth of Its Engineering Staff with Kubernetes
case_study_details:
- Company: AppDirect
- Location: San Francisco, California
diff --git a/content/en/community/static/cncf-code-of-conduct.md b/content/en/community/static/cncf-code-of-conduct.md
index 05aefcc9fe..d07444c418 100644
--- a/content/en/community/static/cncf-code-of-conduct.md
+++ b/content/en/community/static/cncf-code-of-conduct.md
@@ -37,8 +37,8 @@ when an individual is representing the project or its community.
Instances of abusive, harassing, or otherwise unacceptable behavior in Kubernetes may be reported by contacting the [Kubernetes Code of Conduct Committee](https://git.k8s.io/community/committee-code-of-conduct) via . For other projects, please contact a CNCF project maintainer or our mediator, Mishi Choudhary .
This Code of Conduct is adapted from the Contributor Covenant
-(http://contributor-covenant.org), version 1.2.0, available at
-http://contributor-covenant.org/version/1/2/0/
+(https://contributor-covenant.org), version 1.2.0, available at
+https://contributor-covenant.org/version/1/2/0/
### CNCF Events Code of Conduct
diff --git a/content/en/docs/concepts/cluster-administration/kubelet-garbage-collection.md b/content/en/docs/concepts/cluster-administration/kubelet-garbage-collection.md
index 9f6dc7599d..ea51a566ac 100644
--- a/content/en/docs/concepts/cluster-administration/kubelet-garbage-collection.md
+++ b/content/en/docs/concepts/cluster-administration/kubelet-garbage-collection.md
@@ -1,13 +1,13 @@
---
reviewers:
-title: Configuring kubelet Garbage Collection
+title: Garbage collection for container images
content_type: concept
weight: 70
---
-Garbage collection is a helpful function of kubelet that will clean up unused images and unused containers. Kubelet will perform garbage collection for containers every minute and garbage collection for images every five minutes.
+Garbage collection is a helpful function of kubelet that will clean up unused [images](/docs/concepts/containers/#container-images) and unused [containers](/docs/concepts/containers/). Kubelet will perform garbage collection for containers every minute and garbage collection for images every five minutes.
External garbage collection tools are not recommended as these tools can potentially break the behavior of kubelet by removing containers expected to exist.
diff --git a/content/en/docs/concepts/cluster-administration/networking.md b/content/en/docs/concepts/cluster-administration/networking.md
index 15f4222224..c517b13175 100644
--- a/content/en/docs/concepts/cluster-administration/networking.md
+++ b/content/en/docs/concepts/cluster-administration/networking.md
@@ -114,7 +114,7 @@ Additionally, the CNI can be run alongside [Calico for network policy enforcemen
### Azure CNI for Kubernetes
[Azure CNI](https://docs.microsoft.com/en-us/azure/virtual-network/container-networking-overview) is an [open source](https://github.com/Azure/azure-container-networking/blob/master/docs/cni.md) plugin that integrates Kubernetes Pods with an Azure Virtual Network (also known as VNet) providing network performance at par with VMs. Pods can connect to peered VNet and to on-premises over Express Route or site-to-site VPN and are also directly reachable from these networks. Pods can access Azure services, such as storage and SQL, that are protected by Service Endpoints or Private Link. You can use VNet security policies and routing to filter Pod traffic. The plugin assigns VNet IPs to Pods by utilizing a pool of secondary IPs pre-configured on the Network Interface of a Kubernetes node.
-Azure CNI is available natively in the [Azure Kubernetes Service (AKS)] (https://docs.microsoft.com/en-us/azure/aks/configure-azure-cni).
+Azure CNI is available natively in the [Azure Kubernetes Service (AKS)](https://docs.microsoft.com/en-us/azure/aks/configure-azure-cni).
### Big Cloud Fabric from Big Switch Networks
diff --git a/content/en/docs/concepts/cluster-administration/system-metrics.md b/content/en/docs/concepts/cluster-administration/system-metrics.md
index e2e037ff88..bbfdf0cc0c 100644
--- a/content/en/docs/concepts/cluster-administration/system-metrics.md
+++ b/content/en/docs/concepts/cluster-administration/system-metrics.md
@@ -50,39 +50,41 @@ rules:
## Metric lifecycle
-Alpha metric → Stable metric → Deprecated metric → Hidden metric → Deletion
+Alpha metric → Stable metric → Deprecated metric → Hidden metric → Deleted metric
-Alpha metrics have no stability guarantees; as such they can be modified or deleted at any time.
+Alpha metrics have no stability guarantees. These metrics can be modified or deleted at any time.
-Stable metrics can be guaranteed to not change; Specifically, stability means:
+Stable metrics are guaranteed to not change. This means:
+* A stable metric without a deprecated signature will not be deleted or renamed
+* A stable metric's type will not be modified
-* the metric itself will not be deleted (or renamed)
-* the type of metric will not be modified
+Deprecated metrics are slated for deletion, but are still available for use.
+These metrics include an annotation about the version in which they became deprecated.
-Deprecated metric signal that the metric will eventually be deleted; to find which version, you need to check annotation, which includes from which kubernetes version that metric will be considered deprecated.
+For example:
-Before deprecation:
+* Before deprecation
-```
-# HELP some_counter this counts things
-# TYPE some_counter counter
-some_counter 0
-```
+ ```
+ # HELP some_counter this counts things
+ # TYPE some_counter counter
+ some_counter 0
+ ```
-After deprecation:
+* After deprecation
-```
-# HELP some_counter (Deprecated since 1.15.0) this counts things
-# TYPE some_counter counter
-some_counter 0
-```
+ ```
+ # HELP some_counter (Deprecated since 1.15.0) this counts things
+ # TYPE some_counter counter
+ some_counter 0
+ ```
-Once a metric is hidden then by default the metrics is not published for scraping. To use a hidden metric, you need to override the configuration for the relevant cluster component.
+Hidden metrics are no longer published for scraping, but are still available for use. To use a hidden metric, please refer to the [Show hidden metrics](#show-hidden-metrics) section.
-Once a metric is deleted, the metric is not published. You cannot change this using an override.
+Deleted metrics are no longer published and cannot be used.
-## Show Hidden Metrics
+## Show hidden metrics
As described above, admins can enable hidden metrics through a command-line flag on a specific binary. This intends to be used as an escape hatch for admins if they missed the migration of the metrics deprecated in the last release.
diff --git a/content/en/docs/concepts/configuration/configmap.md b/content/en/docs/concepts/configuration/configmap.md
index d9889604ce..9a134dfc99 100644
--- a/content/en/docs/concepts/configuration/configmap.md
+++ b/content/en/docs/concepts/configuration/configmap.md
@@ -40,7 +40,7 @@ separate database or file service.
A ConfigMap is an API [object](/docs/concepts/overview/working-with-objects/kubernetes-objects/)
that lets you store configuration for other objects to use. Unlike most
Kubernetes objects that have a `spec`, a ConfigMap has `data` and `binaryData`
-fields. These fields accepts key-value pairs as their values. Both the `data`
+fields. These fields accept key-value pairs as their values. Both the `data`
field and the `binaryData` are optional. The `data` field is designed to
contain UTF-8 byte sequences while the `binaryData` field is designed to
contain binary data.
diff --git a/content/en/docs/concepts/configuration/manage-resources-containers.md b/content/en/docs/concepts/configuration/manage-resources-containers.md
index d267b83dd2..2668050d26 100644
--- a/content/en/docs/concepts/configuration/manage-resources-containers.md
+++ b/content/en/docs/concepts/configuration/manage-resources-containers.md
@@ -396,7 +396,7 @@ The kubelet supports different ways to measure Pod storage use:
{{< tabs name="resource-emphemeralstorage-measurement" >}}
{{% tab name="Periodic scanning" %}}
-The kubelet performs regular, schedules checks that scan each
+The kubelet performs regular, scheduled checks that scan each
`emptyDir` volume, container log directory, and writeable container layer.
The scan measures how much space is used.
diff --git a/content/en/docs/concepts/configuration/secret.md b/content/en/docs/concepts/configuration/secret.md
index d2bfe274c2..3674c3ce4c 100644
--- a/content/en/docs/concepts/configuration/secret.md
+++ b/content/en/docs/concepts/configuration/secret.md
@@ -271,6 +271,13 @@ However, using the builtin Secret type helps unify the formats of your credentia
and the API server does verify if the required keys are provided in a Secret
configuration.
+{{< caution >}}
+SSH private keys do not establish trusted communication between an SSH client and
+host server on their own. A secondary means of establishing trust is needed to
+mitigate "man in the middle" attacks, such as a `known_hosts` file added to a
+ConfigMap.
+{{< /caution >}}
+
### TLS secrets
Kubernetes provides a builtin Secret type `kubernetes.io/tls` for to storing
@@ -351,7 +358,7 @@ data:
A bootstrap type Secret has the following keys specified under `data`:
-- `token_id`: A random 6 character string as the token identifier. Required.
+- `token-id`: A random 6 character string as the token identifier. Required.
- `token-secret`: A random 16 character string as the actual token secret. Required.
- `description`: A human-readable string that describes what the token is
used for. Optional.
diff --git a/content/en/docs/concepts/extend-kubernetes/operator.md b/content/en/docs/concepts/extend-kubernetes/operator.md
index 140cf2557a..0e9d227a53 100644
--- a/content/en/docs/concepts/extend-kubernetes/operator.md
+++ b/content/en/docs/concepts/extend-kubernetes/operator.md
@@ -103,7 +103,7 @@ as well as keeping the existing service in good shape.
## Writing your own Operator {#writing-operator}
If there isn't an Operator in the ecosystem that implements the behavior you
-want, you can code your own. In [What's next](#whats-next) you'll find a few
+want, you can code your own. In [What's next](#what-s-next) you'll find a few
links to libraries and tools you can use to write your own cloud native
Operator.
diff --git a/content/en/docs/concepts/overview/kubernetes-api.md b/content/en/docs/concepts/overview/kubernetes-api.md
index 506c76cde7..07b5d559d7 100644
--- a/content/en/docs/concepts/overview/kubernetes-api.md
+++ b/content/en/docs/concepts/overview/kubernetes-api.md
@@ -19,7 +19,7 @@ is the {{< glossary_tooltip text="API server" term_id="kube-apiserver" >}}. The
exposes an HTTP API that lets end users, different parts of your cluster, and
external components communicate with one another.
-The Kubernetes API lets you query and manipulate the state of objects in the Kubernetes API
+The Kubernetes API lets you query and manipulate the state of API objects in Kubernetes
(for example: Pods, Namespaces, ConfigMaps, and Events).
Most operations can be performed through the
diff --git a/content/en/docs/concepts/overview/working-with-objects/common-labels.md b/content/en/docs/concepts/overview/working-with-objects/common-labels.md
index a0a68c6dff..29af899b4e 100644
--- a/content/en/docs/concepts/overview/working-with-objects/common-labels.md
+++ b/content/en/docs/concepts/overview/working-with-objects/common-labels.md
@@ -59,8 +59,8 @@ metadata:
## Applications And Instances Of Applications
An application can be installed one or more times into a Kubernetes cluster and,
-in some cases, the same namespace. For example, wordpress can be installed more
-than once where different websites are different installations of wordpress.
+in some cases, the same namespace. For example, WordPress can be installed more
+than once where different websites are different installations of WordPress.
The name of an application and the instance name are recorded separately. For
example, WordPress has a `app.kubernetes.io/name` of `wordpress` while it has
@@ -168,6 +168,6 @@ metadata:
...
```
-With the MySQL `StatefulSet` and `Service` you'll notice information about both MySQL and Wordpress, the broader application, are included.
+With the MySQL `StatefulSet` and `Service` you'll notice information about both MySQL and WordPress, the broader application, are included.
diff --git a/content/en/docs/concepts/policy/pod-security-policy.md b/content/en/docs/concepts/policy/pod-security-policy.md
index 063f1234fe..17f30906bf 100644
--- a/content/en/docs/concepts/policy/pod-security-policy.md
+++ b/content/en/docs/concepts/policy/pod-security-policy.md
@@ -216,12 +216,17 @@ kubectl-user create -f- <}}
Volume snapshots only support the out-of-tree CSI volume plugins. For details, see [Volume Snapshots](/docs/concepts/storage/volume-snapshots/).
-In-tree volume plugins are deprecated. You can read about the deprecated volume plugins in the [Volume Plugin FAQ] (https://github.com/kubernetes/community/blob/master/sig-storage/volume-plugin-faq.md).
+In-tree volume plugins are deprecated. You can read about the deprecated volume plugins in the [Volume Plugin FAQ](https://github.com/kubernetes/community/blob/master/sig-storage/volume-plugin-faq.md).
### Create a PersistentVolumeClaim from a Volume Snapshot {#create-persistent-volume-claim-from-volume-snapshot}
diff --git a/content/en/docs/concepts/storage/storage-capacity.md b/content/en/docs/concepts/storage/storage-capacity.md
index 836d5d2c36..d5993d4f59 100644
--- a/content/en/docs/concepts/storage/storage-capacity.md
+++ b/content/en/docs/concepts/storage/storage-capacity.md
@@ -34,7 +34,7 @@ text="Container Storage Interface" term_id="csi" >}} (CSI) drivers and
## API
There are two API extensions for this feature:
-- [CSIStorageCapacity](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#csistoragecapacity-v1alpha1-storage-k8s-io) objects:
+- CSIStorageCapacity objects:
these get produced by a CSI driver in the namespace
where the driver is installed. Each object contains capacity
information for one storage class and defines which nodes have
diff --git a/content/en/docs/concepts/workloads/_index.md b/content/en/docs/concepts/workloads/_index.md
index 21e57aed03..2c9dd8aa8e 100644
--- a/content/en/docs/concepts/workloads/_index.md
+++ b/content/en/docs/concepts/workloads/_index.md
@@ -8,50 +8,74 @@ no_list: true
{{< glossary_definition term_id="workload" length="short" >}}
Whether your workload is a single component or several that work together, on Kubernetes you run
-it inside a set of [Pods](/docs/concepts/workloads/pods).
-In Kubernetes, a Pod represents a set of running {{< glossary_tooltip text="containers" term_id="container" >}}
-on your cluster.
+it inside a set of [_pods_](/docs/concepts/workloads/pods).
+In Kubernetes, a `Pod` represents a set of running
+{{< glossary_tooltip text="containers" term_id="container" >}} on your cluster.
-A Pod has a defined lifecycle. For example, once a Pod is running in your cluster then
-a critical failure on the {{< glossary_tooltip text="node" term_id="node" >}} where that
-Pod is running means that all the Pods on that node fail. Kubernetes treats that level
-of failure as final: you would need to create a new Pod even if the node later recovers.
+Kubernetes pods have a [defined lifecycle](/docs/concepts/workloads/pods/pod-lifecycle/).
+For example, once a pod is running in your cluster then a critical fault on the
+{{< glossary_tooltip text="node" term_id="node" >}} where that pod is running means that
+all the pods on that node fail. Kubernetes treats that level of failure as final: you
+would need to create a new `Pod` to recover, even if the node later becomes healthy.
-However, to make life considerably easier, you don't need to manage each Pod directly.
-Instead, you can use _workload resources_ that manage a set of Pods on your behalf.
+However, to make life considerably easier, you don't need to manage each `Pod` directly.
+Instead, you can use _workload resources_ that manage a set of pods on your behalf.
These resources configure {{< glossary_tooltip term_id="controller" text="controllers" >}}
-that make sure the right number of the right kind of Pod are running, to match the state
+that make sure the right number of the right kind of pod are running, to match the state
you specified.
-Those workload resources include:
+Kubernetes provides several built-in workload resources:
-* [Deployment](/docs/concepts/workloads/controllers/deployment/) and [ReplicaSet](/docs/concepts/workloads/controllers/replicaset/)
- (replacing the legacy resource {{< glossary_tooltip text="ReplicationController" term_id="replication-controller" >}});
-* [StatefulSet](/docs/concepts/workloads/controllers/statefulset/);
-* [DaemonSet](/docs/concepts/workloads/controllers/daemonset/) for running Pods that provide
- node-local facilities, such as a storage driver or network plugin;
-* [Job](/docs/concepts/workloads/controllers/job/) and
- [CronJob](/docs/concepts/workloads/controllers/cron-jobs/)
- for tasks that run to completion.
+* [`Deployment`](/docs/concepts/workloads/controllers/deployment/) and [`ReplicaSet`](/docs/concepts/workloads/controllers/replicaset/)
+ (replacing the legacy resource
+ {{< glossary_tooltip text="ReplicationController" term_id="replication-controller" >}}).
+ `Deployment` is a good fit for managing a stateless application workload on your cluster,
+ where any `Pod` in the `Deployment` is interchangeable and can be replaced if needed.
+* [`StatefulSet`](/docs/concepts/workloads/controllers/statefulset/) lets you
+ run one or more related Pods that do track state somehow. For example, if your workload
+ records data persistently, you can run a `StatefulSet` that matches each `Pod` with a
+ [`PersistentVolume`](/docs/concepts/storage/persistent-volumes/). Your code, running in the
+ `Pods` for that `StatefulSet`, can replicate data to other `Pods` in the same `StatefulSet`
+ to improve overall resilience.
+* [`DaemonSet`](/docs/concepts/workloads/controllers/daemonset/) defines `Pods` that provide
+ node-local facilities. These might be fundamental to the operation of your cluster, such
+ as a networking helper tool, or be part of an
+ {{< glossary_tooltip text="add-on" term_id="addons" >}}.
+ Every time you add a node to your cluster that matches the specification in a `DaemonSet`,
+ the control plane schedules a `Pod` for that `DaemonSet` onto the new node.
+* [`Job`](/docs/concepts/workloads/controllers/job/) and
+ [`CronJob`](/docs/concepts/workloads/controllers/cron-jobs/)
+ define tasks that run to completion and then stop. Jobs represent one-off tasks, whereas
+ `CronJobs` recur according to a schedule.
-There are also two supporting concepts that you might find relevant:
-* [Garbage collection](/docs/concepts/workloads/controllers/garbage-collection/) tidies up objects
- from your cluster after their _owning resource_ has been removed.
-* The [_time-to-live after finished_ controller](/docs/concepts/workloads/controllers/ttlafterfinished/)
- removes Jobs once a defined time has passed since they completed.
+In the wider Kubernetes ecosystem, you can find third-party workload resources that provide
+additional behaviors. Using a
+[custom resource definition](/docs/concepts/extend-kubernetes/api-extension/custom-resources/),
+you can add in a third-party workload resource if you want a specific behavior that's not part
+of Kubernetes' core. For example, if you wanted to run a group of `Pods` for your application but
+stop work unless _all_ the Pods are available (perhaps for some high-throughput distributed task),
+then you can implement or install an extension that does provide that feature.
## {{% heading "whatsnext" %}}
As well as reading about each resource, you can learn about specific tasks that relate to them:
-* [Run a stateless application using a Deployment](/docs/tasks/run-application/run-stateless-application-deployment/)
+* [Run a stateless application using a `Deployment`](/docs/tasks/run-application/run-stateless-application-deployment/)
* Run a stateful application either as a [single instance](/docs/tasks/run-application/run-single-instance-stateful-application/)
or as a [replicated set](/docs/tasks/run-application/run-replicated-stateful-application/)
-* [Run Automated Tasks with a CronJob](/docs/tasks/job/automated-tasks-with-cron-jobs/)
+* [Run automated tasks with a `CronJob`](/docs/tasks/job/automated-tasks-with-cron-jobs/)
+
+To learn about Kubernetes' mechanisms for separating code from configuration,
+visit [Configuration](/docs/concepts/configuration/).
+
+There are two supporting concepts that provide backgrounds about how Kubernetes manages pods
+for applications:
+* [Garbage collection](/docs/concepts/workloads/controllers/garbage-collection/) tidies up objects
+ from your cluster after their _owning resource_ has been removed.
+* The [_time-to-live after finished_ controller](/docs/concepts/workloads/controllers/ttlafterfinished/)
+ removes Jobs once a defined time has passed since they completed.
Once your application is running, you might want to make it available on the internet as
-a [Service](/docs/concepts/services-networking/service/) or, for web application only,
-using an [Ingress](/docs/concepts/services-networking/ingress).
+a [`Service`](/docs/concepts/services-networking/service/) or, for web application only,
+using an [`Ingress`](/docs/concepts/services-networking/ingress).
-You can also visit [Configuration](/docs/concepts/configuration/) to learn about Kubernetes'
-mechanisms for separating code from configuration.
diff --git a/content/en/docs/concepts/workloads/controllers/cron-jobs.md b/content/en/docs/concepts/workloads/controllers/cron-jobs.md
index c4206facb5..481c6f5017 100644
--- a/content/en/docs/concepts/workloads/controllers/cron-jobs.md
+++ b/content/en/docs/concepts/workloads/controllers/cron-jobs.md
@@ -49,6 +49,37 @@ This example CronJob manifest prints the current time and a hello message every
([Running Automated Tasks with a CronJob](/docs/tasks/job/automated-tasks-with-cron-jobs/)
takes you through this example in more detail).
+### Cron schedule syntax
+
+```
+# ┌───────────── minute (0 - 59)
+# │ ┌───────────── hour (0 - 23)
+# │ │ ┌───────────── day of the month (1 - 31)
+# │ │ │ ┌───────────── month (1 - 12)
+# │ │ │ │ ┌───────────── day of the week (0 - 6) (Sunday to Saturday;
+# │ │ │ │ │ 7 is also Sunday on some systems)
+# │ │ │ │ │
+# │ │ │ │ │
+# * * * * *
+```
+
+
+| Entry | Description | Equivalent to |
+| ------------- | ------------- |------------- |
+| @yearly (or @annually) | Run once a year at midnight of 1 January | 0 0 1 1 * |
+| @monthly | Run once a month at midnight of the first day of the month | 0 0 1 * * |
+| @weekly | Run once a week at midnight on Sunday morning | 0 0 * * 0 |
+| @daily (or @midnight) | Run once a day at midnight | 0 0 * * * |
+| @hourly | Run once an hour at the beginning of the hour | 0 * * * * |
+
+
+
+For example, the line below states that the task must be started every Friday at midnight, as well as on the 13th of each month at midnight:
+
+`0 0 13 * 5`
+
+To generate CronJob schedule expressions, you can also use web tools like [crontab.guru](https://crontab.guru/).
+
## CronJob limitations {#cron-job-limitations}
A cron job creates a job object _about_ once per execution time of its schedule. We say "about" because there
diff --git a/content/en/docs/concepts/workloads/controllers/job.md b/content/en/docs/concepts/workloads/controllers/job.md
index b65ed27ae2..14f218ddc8 100644
--- a/content/en/docs/concepts/workloads/controllers/job.md
+++ b/content/en/docs/concepts/workloads/controllers/job.md
@@ -38,6 +38,7 @@ You can run the example with this command:
```shell
kubectl apply -f https://kubernetes.io/examples/controllers/job.yaml
```
+The output is similar to this:
```
job.batch/pi created
```
@@ -47,6 +48,7 @@ Check on the status of the Job with `kubectl`:
```shell
kubectl describe jobs/pi
```
+The output is similar to this:
```
Name: pi
Namespace: default
@@ -91,6 +93,7 @@ To list all the Pods that belong to a Job in a machine readable form, you can us
pods=$(kubectl get pods --selector=job-name=pi --output=jsonpath='{.items[*].metadata.name}')
echo $pods
```
+The output is similar to this:
```
pi-5rwd7
```
@@ -398,10 +401,11 @@ Therefore, you delete Job `old` but _leave its pods
running_, using `kubectl delete jobs/old --cascade=false`.
Before deleting it, you make a note of what selector it uses:
-```
+```shell
kubectl get job old -o yaml
```
-```
+The output is similar to this:
+```yaml
kind: Job
metadata:
name: old
@@ -420,7 +424,7 @@ they are controlled by Job `new` as well.
You need to specify `manualSelector: true` in the new Job since you are not using
the selector that the system normally generates for you automatically.
-```
+```yaml
kind: Job
metadata:
name: new
diff --git a/content/en/docs/concepts/workloads/controllers/replicationcontroller.md b/content/en/docs/concepts/workloads/controllers/replicationcontroller.md
index fa481fb3f8..36ae4a880a 100644
--- a/content/en/docs/concepts/workloads/controllers/replicationcontroller.md
+++ b/content/en/docs/concepts/workloads/controllers/replicationcontroller.md
@@ -54,6 +54,7 @@ Run the example job by downloading the example file and then running this comman
```shell
kubectl apply -f https://k8s.io/examples/controllers/replication.yaml
```
+The output is similar to this:
```
replicationcontroller/nginx created
```
@@ -63,6 +64,7 @@ Check on the status of the ReplicationController using this command:
```shell
kubectl describe replicationcontrollers/nginx
```
+The output is similar to this:
```
Name: nginx
Namespace: default
@@ -101,6 +103,7 @@ To list all the pods that belong to the ReplicationController in a machine reada
pods=$(kubectl get pods --selector=app=nginx --output=jsonpath={.items..metadata.name})
echo $pods
```
+The output is similar to this:
```
nginx-3ntk0 nginx-4ok8v nginx-qrm3m
```
diff --git a/content/en/docs/concepts/workloads/pods/_index.md b/content/en/docs/concepts/workloads/pods/_index.md
index 5bd954dcef..af548a79cb 100644
--- a/content/en/docs/concepts/workloads/pods/_index.md
+++ b/content/en/docs/concepts/workloads/pods/_index.md
@@ -191,6 +191,35 @@ details are abstracted away. That abstraction and separation of concerns simplif
system semantics, and makes it feasible to extend the cluster's behavior without
changing existing code.
+## Pod update and replacement
+
+As mentioned in the previous section, when the Pod template for a workload
+resource is changed, the controller creates new Pods based on the updated
+template instead of updating or patching the existing Pods.
+
+Kubernetes doesn't prevent you from managing Pods directly. It is possible to
+update some fields of a running Pod, in place. However, Pod update operations
+like
+[`patch`](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#patch-pod-v1-core), and
+[`replace`](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#replace-pod-v1-core)
+have some limitations:
+
+- Most of the metadata about a Pod is immutable. For example, you cannot
+ change the `namespace`, `name`, `uid`, or `creationTimestamp` fields;
+ the `generation` field is unique. It only accepts updates that increment the
+ field's current value.
+- If the `metadata.deletionTimestamp` is set, no new entry can be added to the
+ `metadata.finalizers` list.
+- Pod updates may not change fields other than `spec.containers[*].image`,
+ `spec.initContainers[*].image`, `spec.activeDeadlineSeconds` or
+ `spec.tolerations`. For `spec.tolerations`, you can only add new entries.
+- When updating the `spec.activeDeadlineSeconds` field, two types of updates
+ are allowed:
+
+ 1. setting the unassigned field to a positive number;
+ 1. updating the field from a positive number to a smaller, non-negative
+ number.
+
## Resource sharing and communication
Pods enable data sharing and communication among their constituent
diff --git a/content/en/docs/concepts/workloads/pods/init-containers.md b/content/en/docs/concepts/workloads/pods/init-containers.md
index 5c92f07423..869ba66f66 100644
--- a/content/en/docs/concepts/workloads/pods/init-containers.md
+++ b/content/en/docs/concepts/workloads/pods/init-containers.md
@@ -49,9 +49,9 @@ as documented in [Resources](#resources).
Also, init containers do not support `lifecycle`, `livenessProbe`, `readinessProbe`, or
`startupProbe` because they must run to completion before the Pod can be ready.
-If you specify multiple init containers for a Pod, Kubelet runs each init
+If you specify multiple init containers for a Pod, kubelet runs each init
container sequentially. Each init container must succeed before the next can run.
-When all of the init containers have run to completion, Kubelet initializes
+When all of the init containers have run to completion, kubelet initializes
the application containers for the Pod and runs them as usual.
## Using init containers
@@ -257,7 +257,7 @@ if the Pod `restartPolicy` is set to Always, the init containers use
A Pod cannot be `Ready` until all init containers have succeeded. The ports on an
init container are not aggregated under a Service. A Pod that is initializing
-is in the `Pending` state but should have a condition `Initialized` set to true.
+is in the `Pending` state but should have a condition `Initialized` set to false.
If the Pod [restarts](#pod-restart-reasons), or is restarted, all init containers
must execute again.
diff --git a/content/en/docs/concepts/workloads/pods/pod-topology-spread-constraints.md b/content/en/docs/concepts/workloads/pods/pod-topology-spread-constraints.md
index 4b33db8703..2e8a915c62 100644
--- a/content/en/docs/concepts/workloads/pods/pod-topology-spread-constraints.md
+++ b/content/en/docs/concepts/workloads/pods/pod-topology-spread-constraints.md
@@ -66,7 +66,7 @@ Instead of manually applying labels, you can also reuse the [well-known labels](
The API field `pod.spec.topologySpreadConstraints` is defined as below:
-```
+```yaml
apiVersion: v1
kind: Pod
metadata:
diff --git a/content/en/docs/contribute/new-content/new-features.md b/content/en/docs/contribute/new-content/new-features.md
index 4e70f6c9ec..a0e3600562 100644
--- a/content/en/docs/contribute/new-content/new-features.md
+++ b/content/en/docs/contribute/new-content/new-features.md
@@ -95,14 +95,16 @@ deadlines.
### Open a placeholder PR
-1. Open a pull request against the
+1. Open a **draft** pull request against the
`dev-{{< skew nextMinorVersion >}}` branch in the `kubernetes/website` repository, with a small
-commit that you will amend later.
+commit that you will amend later. To create a draft pull request, use the
+Create Pull Request drop-down and select **Create Draft Pull Request**,
+then click **Draft Pull Request**.
2. Edit the pull request description to include links to [kubernetes/kubernetes](https://github.com/kubernetes/kubernetes)
PR(s) and [kubernetes/enhancements](https://github.com/kubernetes/enhancements) issue(s).
-3. Use the Prow command `/milestone {{< skew nextMinorVersion >}}` to
-assign the PR to the relevant milestone. This alerts the docs person managing
-this release that the feature docs are coming.
+3. Leave a comment on the related [kubernetes/enhancements](https://github.com/kubernetes/enhancements)
+issue with a link to the PR to notify the docs person managing this release that
+the feature docs are coming and should be tracked for the release.
If your feature does not need
any documentation changes, make sure the sig-release team knows this, by
@@ -112,7 +114,9 @@ milestone.
### PR ready for review
-When ready, populate your placeholder PR with feature documentation.
+When ready, populate your placeholder PR with feature documentation and change
+the state of the PR from draft to **ready for review**. To mark a pull request
+as ready for review, navigate to the merge box and click **Ready for review**.
Do your best to describe your feature and how to use it. If you need help structuring your documentation, ask in the `#sig-docs` slack channel.
@@ -120,6 +124,13 @@ When you complete your content, the documentation person assigned to your featur
To ensure technical accuracy, the content may also require a technical review from corresponding SIG(s).
Use their suggestions to get the content to a release ready state.
+If your feature is an Alpha or Beta feature and is behind a feature gate,
+make sure you add it to [Alpha/Beta Feature gates](/docs/reference/command-line-tools-reference/feature-gates/#feature-gates-for-alpha-or-beta-features)
+table as part of your pull request. With new feature gates, a description of
+the feature gate is also required. If your feature is GA'ed or deprecated,
+make sure to move it from that table to [Feature gates for graduated or deprecated features](/docs/reference/command-line-tools-reference/feature-gates/#feature-gates-for-graduated-or-deprecated-features)
+table with Alpha and Beta history intact.
+
If your feature needs documentation and the first draft
content is not received, the feature may be removed from the milestone.
@@ -128,10 +139,4 @@ content is not received, the feature may be removed from the milestone.
If your PR has not yet been merged into the `dev-{{< skew nextMinorVersion >}}` branch by the release deadline, work with the
docs person managing the release to get it in by the deadline. If your feature needs
documentation and the docs are not ready, the feature may be removed from the
-milestone.
-
-If your feature is an Alpha feature and is behind a feature gate, make sure you
-add it to [Alpha/Beta Feature gates](/docs/reference/command-line-tools-reference/feature-gates/#feature-gates-for-alpha-or-beta-features) table
-as part of your pull request. If your feature is moving out of Alpha, make sure to
-remove it from that table.
-
+milestone.
\ No newline at end of file
diff --git a/content/en/docs/contribute/style/style-guide.md b/content/en/docs/contribute/style/style-guide.md
index c0efac7959..b4864dbabf 100644
--- a/content/en/docs/contribute/style/style-guide.md
+++ b/content/en/docs/contribute/style/style-guide.md
@@ -143,7 +143,7 @@ Do | Don't
:--| :-----
Set the value of the `replicas` field in the configuration file. | Set the value of the "replicas" field in the configuration file.
The value of the `exec` field is an ExecAction object. | The value of the "exec" field is an ExecAction object.
-Run the process as a Daemonset in the `kube-system` namespace. | Run the process as a Daemonset in the kube-system namespace.
+Run the process as a DaemonSet in the `kube-system` namespace. | Run the process as a DaemonSet in the kube-system namespace.
{{< /table >}}
### Use code style for Kubernetes command tool and component names
diff --git a/content/en/docs/reference/_index.md b/content/en/docs/reference/_index.md
index bc95e7efeb..5b22271c86 100644
--- a/content/en/docs/reference/_index.md
+++ b/content/en/docs/reference/_index.md
@@ -18,7 +18,7 @@ This section of the Kubernetes documentation contains references.
## API Reference
-* [Kubernetes API Reference {{< latest-version >}}](/docs/reference/generated/kubernetes-api/{{< latest-version >}}/)
+* [API Reference for Kubernetes {{< param "version" >}}](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/)
* [Using The Kubernetes API](/docs/reference/using-api/) - overview of the API for Kubernetes.
## API Client Libraries
@@ -54,4 +54,3 @@ An archive of the design docs for Kubernetes functionality. Good starting points
[Kubernetes Architecture](https://git.k8s.io/community/contributors/design-proposals/architecture/architecture.md) and
[Kubernetes Design Overview](https://git.k8s.io/community/contributors/design-proposals).
-
diff --git a/content/en/docs/reference/access-authn-authz/rbac.md b/content/en/docs/reference/access-authn-authz/rbac.md
index 8ff03a171c..d3e6dc118e 100644
--- a/content/en/docs/reference/access-authn-authz/rbac.md
+++ b/content/en/docs/reference/access-authn-authz/rbac.md
@@ -514,7 +514,7 @@ subjects:
namespace: kube-system
```
-For all service accounts in the "qa" namespace:
+For all service accounts in the "qa" group in any namespace:
```yaml
subjects:
@@ -522,6 +522,15 @@ subjects:
name: system:serviceaccounts:qa
apiGroup: rbac.authorization.k8s.io
```
+For all service accounts in the "dev" group in the "development" namespace:
+
+```yaml
+subjects:
+- kind: Group
+ name: system:serviceaccounts:dev
+ apiGroup: rbac.authorization.k8s.io
+ namespace: development
+```
For all service accounts in any namespace:
diff --git a/content/en/docs/reference/command-line-tools-reference/kubelet.md b/content/en/docs/reference/command-line-tools-reference/kubelet.md
index 501a62fb9c..e56e84bc57 100644
--- a/content/en/docs/reference/command-line-tools-reference/kubelet.md
+++ b/content/en/docs/reference/command-line-tools-reference/kubelet.md
@@ -73,13 +73,6 @@ kubelet [flags]
| Enables anonymous requests to the Kubelet server. Requests that are not rejected by another authentication method are treated as anonymous requests. Anonymous requests have a username of `system:anonymous`, and a group name of `system:unauthenticated`. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's `--config` flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) |
-
-| --application-metrics-count-limit int Default: 100 |
-
-
- | Max number of application metrics to store (per container) (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns,it will follow the standard CLI deprecation timeline before being removed.) |
-
-
| --authentication-token-webhook |
@@ -122,13 +115,6 @@ kubelet [flags]
| Path to the file containing Azure container registry configuration information. |
-
-| --boot-id-file string Default: `/proc/sys/kernel/random/boot_id` |
-
-
- | Comma-separated list of files to check for `boot-id`. Use the first one that exists. (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) |
-
-
| --bootstrap-kubeconfig string |
@@ -234,13 +220,6 @@ kubelet [flags]
| The Kubelet will load its initial configuration from this file. The path may be absolute or relative; relative paths start at the Kubelet's current working directory. Omit this flag to use the built-in default configuration values. Command-line flags override configuration from this file. |
-
-| --container-hints string Default: `/etc/cadvisor/container_hints.json` |
-
-
- | location of the container hints file. (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) |
-
-
| --container-log-max-files int32 Default: 5 |
@@ -269,13 +248,7 @@ kubelet [flags]
| [Experimental] The endpoint of remote runtime service. Currently unix socket endpoint is supported on Linux, while npipe and tcp endpoints are supported on windows. Examples: `unix:///var/run/dockershim.sock`, `npipe:////./pipe/dockershim`. |
-
-| --containerd string Default: `/run/containerd/containerd.sock` |
-
-
- | The `containerd` endpoint. (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) |
-
-
+
| --contention-profiling |
@@ -311,13 +284,6 @@ kubelet [flags]
| <Warning: Alpha feature> CPU Manager reconciliation period. Examples: `10s`, or `1m`. If not supplied, defaults to node status update frequency. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's `--config` flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) |
-
-| --docker string Default: `unix:///var/run/docker.sock` |
-
-
- | The `docker` endpoint. (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) |
-
-
| --docker-endpoint string Default: `unix:///var/run/docker.sock` |
@@ -325,55 +291,6 @@ kubelet [flags]
| Use this for the `docker` endpoint to communicate with. This docker-specific flag only works when container-runtime is set to `docker`. |
-
-| --docker-env-metadata-whitelist string |
-
-
- | a comma-separated list of environment variable keys that needs to be collected for docker containers (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) |
-
-
-
-| --docker-only |
-
-
- | Only report docker containers in addition to root stats (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) |
-
-
-
-| --docker-root string Default: `/var/lib/docker` |
-
-
- | DEPRECATED: docker root is read from docker info (this is a fallback). |
-
-
-
-| --docker-tls |
-
-
- | use TLS to connect to docker (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) |
-
-
-
-| --docker-tls-ca string Default: `ca.pem` |
-
-
- | path to trusted CA. (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) |
-
-
-
-| --docker-tls-cert string Default: `cert.pem` |
-
-
- | path to client certificate. (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) |
-
-
-
-| --docker-tls-key string Default: `key.pem` |
-
-
- | Path to private key. (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) |
-
-
| --dynamic-config-dir string |
@@ -402,13 +319,6 @@ kubelet [flags]
| Enables server endpoints for log collection and local running of containers and commands. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's `--config` flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) |
-
-| --enable-load-reader |
-
-
- | Whether to enable CPU load reader (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) |
-
-
| --enable-server Default: `true` |
@@ -438,21 +348,7 @@ kubelet [flags]
-| --event-storage-age-limit string Default: `default=0` |
-
-
- | Max length of time for which to store events (per type). Value is a comma separated list of key values, where the keys are event types (e.g.: `creation`, `oom`) or `default` and the value is a duration. Default is applied to all non-specified event types. (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) |
-
-
-
-| --event-storage-event-limit string Default: `default=0` |
-
-
- | Max number of events to store (per type). Value is a comma separated list of key values, where the keys are event types (e.g.: `creation`, `oom`) or `default` and the value is an integer. Default is applied to all non-specified event types. (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) |
-
-
-
-| --eviction-hard mapStringString Default: `imagefs.available<15%,memory.available<100Mi,nodefs.available<10%,nodefs.inodesFree<5%` |
+--eviction-hard mapStringString Default: `imagefs.available<15%,memory.available<100Mi,nodefs.available<10%` |
| A set of eviction thresholds (e.g. `memory.available<1Gi`) that if met would trigger a pod eviction. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's `--config` flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) |
@@ -528,6 +424,13 @@ kubelet [flags]
| If enabled, the kubelet will integrate with the kernel memcg notification to determine if memory eviction thresholds are crossed rather than polling. This flag will be removed in 1.23. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's `--config` flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) |
+
+| --experimental-log-sanitization bool |
+
+
+ | [Experimental] When enabled prevents logging of fields tagged as sensitive (passwords, keys, tokens). Runtime log sanitization may introduce significant computation overhead and therefore should not be enabled in production. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
+ |
+
| --experimental-mounter-path string Default: `mount` |
@@ -548,8 +451,9 @@ kubelet [flags]
| A set of `key=value` pairs that describe feature gates for alpha/experimental features. Options are:
APIListChunking=true|false (BETA - default=true)
-APIPriorityAndFairness=true|false (ALPHA - default=false)
+APIPriorityAndFairness=true|false (BETA - default=true)
APIResponseCompression=true|false (BETA - default=true)
+APIServerIdentity=true|false (ALPHA - default=false)
AllAlpha=true|false (ALPHA - default=false)
AllBeta=true|false (BETA - default=false)
AllowInsecureBackendProxy=true|false (BETA - default=true)
@@ -573,31 +477,40 @@ CSIMigrationOpenStack=true|false (BETA - default=false)
CSIMigrationOpenStackComplete=true|false (ALPHA - default=false)
CSIMigrationvSphere=true|false (BETA - default=false)
CSIMigrationvSphereComplete=true|false (BETA - default=false)
+CSIServiceAccountToken=true|false (ALPHA - default=false)
CSIStorageCapacity=true|false (ALPHA - default=false)
-CSIVolumeFSGroupPolicy=true|false (ALPHA - default=false)
-ConfigurableFSGroupPolicy=true|false (ALPHA - default=false)
+CSIVolumeFSGroupPolicy=true|false (BETA - default=true)
+ConfigurableFSGroupPolicy=true|false (BETA - default=true)
+CronJobControllerV2=true|false (ALPHA - default=false)
CustomCPUCFSQuotaPeriod=true|false (ALPHA - default=false)
-DefaultPodTopologySpread=true|false (ALPHA - default=false)
+DefaultPodTopologySpread=true|false (BETA - default=true)
DevicePlugins=true|false (BETA - default=true)
DisableAcceleratorUsageMetrics=true|false (BETA - default=true)
+DownwardAPIHugePages=true|false (ALPHA - default=false)
DynamicKubeletConfig=true|false (BETA - default=true)
+EfficientWatchResumption=true|false (ALPHA - default=false)
EndpointSlice=true|false (BETA - default=true)
+EndpointSliceNodeName=true|false (ALPHA - default=false)
EndpointSliceProxying=true|false (BETA - default=true)
+EndpointSliceTerminatingCondition=true|false (ALPHA - default=false)
EphemeralContainers=true|false (ALPHA - default=false)
ExpandCSIVolumes=true|false (BETA - default=true)
ExpandInUsePersistentVolumes=true|false (BETA - default=true)
ExpandPersistentVolumes=true|false (BETA - default=true)
ExperimentalHostUserNamespaceDefaulting=true|false (BETA - default=false)
GenericEphemeralVolume=true|false (ALPHA - default=false)
+GracefulNodeShutdown=true|false (ALPHA - default=false)
+HPAContainerMetrics=true|false (ALPHA - default=false)
HPAScaleToZero=true|false (ALPHA - default=false)
HugePageStorageMediumSize=true|false (BETA - default=true)
-HyperVContainer=true|false (ALPHA - default=false)
IPv6DualStack=true|false (ALPHA - default=false)
ImmutableEphemeralVolumes=true|false (BETA - default=true)
+KubeletCredentialProviders=true|false (ALPHA - default=false)
KubeletPodResources=true|false (BETA - default=true)
LegacyNodeRoleBehavior=true|false (BETA - default=true)
LocalStorageCapacityIsolation=true|false (BETA - default=true)
LocalStorageCapacityIsolationFSQuotaMonitoring=true|false (ALPHA - default=false)
+MixedProtocolLBService=true|false (ALPHA - default=false)
NodeDisruptionExclusion=true|false (BETA - default=true)
NonPreemptingPriority=true|false (BETA - default=true)
PodDisruptionBudget=true|false (BETA - default=true)
@@ -605,31 +518,26 @@ PodOverhead=true|false (BETA - default=true)
ProcMountType=true|false (ALPHA - default=false)
QOSReserved=true|false (ALPHA - default=false)
RemainingItemCount=true|false (BETA - default=true)
-RemoveSelfLink=true|false (ALPHA - default=false)
+RemoveSelfLink=true|false (BETA - default=true)
+RootCAConfigMap=true|false (BETA - default=true)
RotateKubeletServerCertificate=true|false (BETA - default=true)
RunAsGroup=true|false (BETA - default=true)
-RuntimeClass=true|false (BETA - default=true)
-SCTPSupport=true|false (BETA - default=true)
-SelectorIndex=true|false (BETA - default=true)
ServerSideApply=true|false (BETA - default=true)
-ServiceAccountIssuerDiscovery=true|false (ALPHA - default=false)
-ServiceAppProtocol=true|false (BETA - default=true)
+ServiceAccountIssuerDiscovery=true|false (BETA - default=true)
+ServiceLBNodePortControl=true|false (ALPHA - default=false)
ServiceNodeExclusion=true|false (BETA - default=true)
ServiceTopology=true|false (ALPHA - default=false)
-SetHostnameAsFQDN=true|false (ALPHA - default=false)
+SetHostnameAsFQDN=true|false (BETA - default=true)
+SizeMemoryBackedVolumes=true|false (ALPHA - default=false)
+StorageVersionAPI=true|false (ALPHA - default=false)
StorageVersionHash=true|false (BETA - default=true)
-SupportNodePidsLimit=true|false (BETA - default=true)
-SupportPodPidsLimit=true|false (BETA - default=true)
Sysctls=true|false (BETA - default=true)
TTLAfterFinished=true|false (ALPHA - default=false)
-TokenRequest=true|false (BETA - default=true)
-TokenRequestProjection=true|false (BETA - default=true)
TopologyManager=true|false (BETA - default=true)
ValidateProxyRedirects=true|false (BETA - default=true)
-VolumeSnapshotDataSource=true|false (BETA - default=true)
WarningHeaders=true|false (BETA - default=true)
WinDSR=true|false (ALPHA - default=false)
-WinOverlay=true|false (ALPHA - default=false)
+WinOverlay=true|false (BETA - default=true)
WindowsEndpointSliceProxying=true|false (ALPHA - default=false)
(DEPRECATED: This parameter should be set via the config file specified by the Kubelet's `--config` flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) |
@@ -641,13 +549,6 @@ WindowsEndpointSliceProxying=true|false (ALPHA - default=false)
| Duration between checking config files for new data. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's `--config` flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) |
-
-| --global-housekeeping-interval duration Default: `1m0s` |
-
-
- | Interval between global housekeepings. (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) |
-
-
| --hairpin-mode string Default: `promiscuous-bridge` |
@@ -697,6 +598,20 @@ WindowsEndpointSliceProxying=true|false (ALPHA - default=false)
| Duration between checking HTTP for new data. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's `--config` flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) |
+
+| --image-credential-provider-bin-dir string |
+
+
+ | The path to the directory where credential provider plugin binaries are located. |
+
+
+
+| --image-credential-provider-config string |
+
+
+ | The path to the credential provider plugin config file. |
+
+
| --image-gc-high-threshold int32 Default: 85 |
@@ -757,7 +672,7 @@ WindowsEndpointSliceProxying=true|false (ALPHA - default=false)
--kube-api-burst int32 Default: 10 |
- | Burst to use while talking with kubernetes API server. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's `--config` flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) |
+ | Burst to use while talking with kubernetes API server. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's `--config` flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) |
@@ -778,7 +693,7 @@ WindowsEndpointSliceProxying=true|false (ALPHA - default=false)
--kube-reserved mapStringString Default: <None> |
- | A set of `=` (e.g. `cpu=200m,memory=500Mi,ephemeral-storage=1Gi`) pairs that describe resources reserved for kubernetes system components. Currently `cpu`, `memory` and local `ephemeral-storage` for root file system are supported. See http://kubernetes.io/docs/user-guide/compute-resources for more detail. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's `--config` flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) |
+ | A set of `=` (e.g. `cpu=200m,memory=500Mi,ephemeral-storage=1Gi,pid='100'`) pairs that describe resources reserved for kubernetes system components. Currently `cpu`, `memory` and local `ephemeral-storage` for root file system are supported. See http://kubernetes.io/docs/user-guide/compute-resources for more detail. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's `--config` flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) |
@@ -816,13 +731,6 @@ WindowsEndpointSliceProxying=true|false (ALPHA - default=false)
| When logging hits line `:`, emit a stack trace. |
-
-| --log-cadvisor-usage |
-
-
- | Whether to log the usage of the cAdvisor container (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) |
-
-
| --log-dir string |
@@ -855,7 +763,7 @@ WindowsEndpointSliceProxying=true|false (ALPHA - default=false)
--logging-format string Default: `text` |
- | Sets the log format. Permitted formats: `text`, `json`.\nNon-default formats don't honor these flags: `--add-dir-header`, `--alsologtostderr`, `--log-backtrace-at`, `--log_dir`, `--log-file`, `--log-file-max-size`, `--logtostderr`, `--skip_headers`, `--skip_log_headers`, `--stderrthreshold`, `--log-flush-frequency`.\nNon-default choices are currently alpha and subject to change without warning. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's `--config` flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) |
+ | Sets the log format. Permitted formats: `text`, `json`.\nNon-default formats don't honor these flags: `--add-dir-header`, `--alsologtostderr`, `--log-backtrace-at`, `--log-dir`, `--log-file`, `--log-file-max-size`, `--logtostderr`, `--skip_headers`, `--skip_log_headers`, `--stderrthreshold`, `--log-flush-frequency`.\nNon-default choices are currently alpha and subject to change without warning. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's `--config` flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) |
@@ -865,13 +773,6 @@ WindowsEndpointSliceProxying=true|false (ALPHA - default=false)
| log to standard error instead of files. |
-
-| --machine-id-file string Default: `/etc/machine-id,/var/lib/dbus/machine-id` |
-
-
- | Comma-separated list of files to check for `machine-id`. Use the first one that exists. (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) |
-
-
| --make-iptables-util-chains Default: `true` |
@@ -990,6 +891,14 @@ WindowsEndpointSliceProxying=true|false (ALPHA - default=false)
| Traffic to IPs outside this range will use IP masquerade. Set to `0.0.0.0/0` to never masquerade. (DEPRECATED: will be removed in a future version) |
+
+| --one-output |
+
+
+ | If true, only write logs to their native severity level (vs also writing to each lower severity level.
+ |
+
+
| --oom-score-adj int32 Default: -999 |
@@ -1082,7 +991,7 @@ WindowsEndpointSliceProxying=true|false (ALPHA - default=false)
-| --register-node |
+--register-node Default: `true` |
| Register the node with the API server. If `--kubeconfig` is not provided, this flag is irrelevant, as the Kubelet won't have an API server to register with. Default to `true`. |
@@ -1096,7 +1005,7 @@ WindowsEndpointSliceProxying=true|false (ALPHA - default=false)
-| --register-with-taints []api.Taint |
+--register-with-taints mapStringString |
| Register the node with the given list of taints (comma separated `=:`). No-op if `--register-node` is `false`. |
@@ -1202,61 +1111,12 @@ WindowsEndpointSliceProxying=true|false (ALPHA - default=false)
-| --stderrthreshold severity Default: 2 |
+--stderrthreshold int Default: 2 |
| logs at or above this threshold go to stderr. |
-
-| --storage-driver-buffer-duration duration Default: `1m0s` |
-
-
- | Writes in the storage driver will be buffered for this duration, and committed to the non memory backends as a single transaction. (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) |
-
-
-
-| --storage-driver-db string Default: `cadvisor` |
-
-
- | Database name. (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) |
-
-
-
-| --storage-driver-host string Default: `localhost:8086` |
-
-
- | Database `host:port`. (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) |
-
-
-
-| --storage-driver-password string Default: `root` |
-
-
- | Database password. (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) |
-
-
-
-| --storage-driver-secure |
-
-
- | Use secure connection with database (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) |
-
-
-
-| --storage-driver-table string Default: `stats` |
-
-
- | Table name. (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) |
-
-
-
-| --storage-driver-user string Default: `root` |
-
-
- | Database username. (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.) |
-
-
| --streaming-connection-idle-timeout duration Default: `4h0m0s` |
@@ -1282,7 +1142,7 @@ WindowsEndpointSliceProxying=true|false (ALPHA - default=false)
--system-reserved mapStringString Default: \ |
- | A set of `=` (e.g. `cpu=200m,memory=500Mi,ephemeral-storage=1Gi`) pairs that describe resources reserved for non-kubernetes components. Currently only `cpu` and `memory` are supported. See http://kubernetes.io/docs/user-guide/compute-resources for more detail. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's `--config` flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) |
+ | A set of `=` (e.g. `cpu=200m,memory=500Mi,ephemeral-storage=1Gi,pid='100'`) pairs that describe resources reserved for non-kubernetes components. Currently only `cpu` and `memory` are supported. See http://kubernetes.io/docs/user-guide/compute-resources for more detail. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's `--config` flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) |
@@ -1331,6 +1191,13 @@ Insecure values: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_R
| Topology Manager policy to use. Possible values: `none`, `best-effort`, `restricted`, `single-numa-node`. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's `--config` flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) |
+
+| --topology-manager-scope string Default: `container` |
+
+
+ | Scope to which topology hints applied. Topology Manager collects hints from Hint Providers and applies them to defined scope to ensure the pod admission. Possible values: 'container' (default), 'pod'. (default "container") (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) |
+
+
| -v, --v Level |
diff --git a/content/en/docs/reference/kubectl/cheatsheet.md b/content/en/docs/reference/kubectl/cheatsheet.md
index ab4fde5469..8a5d15b0b2 100644
--- a/content/en/docs/reference/kubectl/cheatsheet.md
+++ b/content/en/docs/reference/kubectl/cheatsheet.md
@@ -391,6 +391,7 @@ Verbosity | Description
`--v=2` | Useful steady state information about the service and important log messages that may correlate to significant changes in the system. This is the recommended default log level for most systems.
`--v=3` | Extended information about changes.
`--v=4` | Debug level verbosity.
+`--v=5` | Trace level verbosity.
`--v=6` | Display requested resources.
`--v=7` | Display HTTP request headers.
`--v=8` | Display HTTP request contents.
diff --git a/content/en/docs/reference/using-api/client-libraries.md b/content/en/docs/reference/using-api/client-libraries.md
index c14f0fa9c7..96589c6a55 100644
--- a/content/en/docs/reference/using-api/client-libraries.md
+++ b/content/en/docs/reference/using-api/client-libraries.md
@@ -74,6 +74,7 @@ their authors, not the Kubernetes team.
| Rust | [github.com/ynqa/kubernetes-rust](https://github.com/ynqa/kubernetes-rust) |
| Scala | [github.com/doriordan/skuber](https://github.com/doriordan/skuber) |
| Scala | [github.com/joan38/kubernetes-client](https://github.com/joan38/kubernetes-client) |
+| Swift | [github.com/swiftkube/client](https://github.com/swiftkube/client) |
| DotNet | [github.com/tonnyeremin/kubernetes_gen](https://github.com/tonnyeremin/kubernetes_gen) |
| DotNet (RestSharp) | [github.com/masroorhasan/Kubernetes.DotNet](https://github.com/masroorhasan/Kubernetes.DotNet) |
| Elixir | [github.com/obmarg/kazan](https://github.com/obmarg/kazan/) |
diff --git a/content/en/docs/setup/production-environment/container-runtimes.md b/content/en/docs/setup/production-environment/container-runtimes.md
index 8ba5300375..fdc7214d00 100644
--- a/content/en/docs/setup/production-environment/container-runtimes.md
+++ b/content/en/docs/setup/production-environment/container-runtimes.md
@@ -125,6 +125,24 @@ sudo mkdir -p /etc/containerd
sudo containerd config default | sudo tee /etc/containerd/config.toml
```
+```shell
+# Restart containerd
+sudo systemctl restart containerd
+```
+{{% /tab %}}
+{{% tab name="Ubuntu 18.04/20.04" %}}
+
+```shell
+# (Install containerd)
+sudo apt-get update && sudo apt-get install -y containerd
+```
+
+```shell
+# Configure containerd
+sudo mkdir -p /etc/containerd
+sudo containerd config default | sudo tee /etc/containerd/config.toml
+```
+
```shell
# Restart containerd
sudo systemctl restart containerd
@@ -218,6 +236,13 @@ For more information, see the [CRI-O compatibility matrix](https://github.com/cr
Install and configure prerequisites:
```shell
+
+# Create the .conf file to load the modules at bootup
+cat <}}
-Since `--cgroup-driver` flag has been deprecated by kubelet, if you have that in `/var/lib/kubelet/kubeadm-flags.env`
+Since `--cgroup-driver` flag has been deprecated by the kubelet, if you have that in `/var/lib/kubelet/kubeadm-flags.env`
or `/etc/default/kubelet`(`/etc/sysconfig/kubelet` for RPMs), please remove it and use the KubeletConfiguration instead
(stored in `/var/lib/kubelet/config.yaml` by default).
{{< /note >}}
diff --git a/content/en/docs/setup/release/notes.md b/content/en/docs/setup/release/notes.md
index 85399c55b9..34d669e746 100644
--- a/content/en/docs/setup/release/notes.md
+++ b/content/en/docs/setup/release/notes.md
@@ -1168,23 +1168,7 @@ filename | sha512 hash
### Other (Cleanup or Flake)
-- **Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.**:
-
- ([#96443](https://github.com/kubernetes/kubernetes/pull/96443), [@alaypatel07](https://github.com/alaypatel07)) [SIG Apps]
+- Handle slow cronjob lister in cronjob controller v2 and improve memory footprint. ([#96443](https://github.com/kubernetes/kubernetes/pull/96443), [@alaypatel07](https://github.com/alaypatel07)) [SIG Apps]
- --redirect-container-streaming is no longer functional. The flag will be removed in v1.22 ([#95935](https://github.com/kubernetes/kubernetes/pull/95935), [@tallclair](https://github.com/tallclair)) [SIG Node]
- A new metric `requestAbortsTotal` has been introduced that counts aborted requests for each `group`, `version`, `verb`, `resource`, `subresource` and `scope`. ([#95002](https://github.com/kubernetes/kubernetes/pull/95002), [@p0lyn0mial](https://github.com/p0lyn0mial)) [SIG API Machinery, Cloud Provider, Instrumentation and Scheduling]
- API priority and fairness metrics use snake_case in label names ([#96236](https://github.com/kubernetes/kubernetes/pull/96236), [@adtac](https://github.com/adtac)) [SIG API Machinery, Cluster Lifecycle, Instrumentation and Testing]
@@ -1350,23 +1334,7 @@ filename | sha512 hash
### Deprecation
-- **Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.**:
-
- ([#95856](https://github.com/kubernetes/kubernetes/pull/95856), [@knight42](https://github.com/knight42)) [SIG API Machinery, Node and Testing]
+- ACTION REQUIRED: The kube-apiserver ability to serve on an insecure port, deprecated since v1.10, has been removed. The insecure address flags `--address` and `--insecure-bind-address` have no effect in kube-apiserver and will be removed in v1.24. The insecure port flags `--port` and `--insecure-port` may only be set to 0 and will be removed in v1.24. ([#95856](https://github.com/kubernetes/kubernetes/pull/95856), [@knight42](https://github.com/knight42)) [SIG API Machinery, Node and Testing]
### API Change
@@ -1510,23 +1478,7 @@ filename | sha512 hash
### Bug or Regression
-- **Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.**:
-
- ([#95725](https://github.com/kubernetes/kubernetes/pull/95725), [@p0lyn0mial](https://github.com/p0lyn0mial)) [SIG API Machinery and Cloud Provider]
+- Exposes and sets a default timeout for the SubjectAccessReview client for DelegatingAuthorizationOptions. ([#95725](https://github.com/kubernetes/kubernetes/pull/95725), [@p0lyn0mial](https://github.com/p0lyn0mial)) [SIG API Machinery and Cloud Provider]
- Alter wording to describe pods using a pvc ([#95635](https://github.com/kubernetes/kubernetes/pull/95635), [@RaunakShah](https://github.com/RaunakShah)) [SIG CLI]
- If we set SelectPolicy MinPolicySelect on scaleUp behavior or scaleDown behavior,Horizontal Pod Autoscaler doesn`t automatically scale the number of pods correctly ([#95647](https://github.com/kubernetes/kubernetes/pull/95647), [@JoshuaAndrew](https://github.com/JoshuaAndrew)) [SIG Apps and Autoscaling]
- Ignore apparmor for non-linux operating systems ([#93220](https://github.com/kubernetes/kubernetes/pull/93220), [@wawa0210](https://github.com/wawa0210)) [SIG Node and Windows]
diff --git a/content/en/docs/tasks/access-application-cluster/port-forward-access-application-cluster.md b/content/en/docs/tasks/access-application-cluster/port-forward-access-application-cluster.md
index 3eceb4f6d2..62ddbdcbbc 100644
--- a/content/en/docs/tasks/access-application-cluster/port-forward-access-application-cluster.md
+++ b/content/en/docs/tasks/access-application-cluster/port-forward-access-application-cluster.md
@@ -158,10 +158,16 @@ for database debugging.
Any of the above commands works. The output is similar to this:
```
- I0710 14:43:38.274550 3655 portforward.go:225] Forwarding from 127.0.0.1:7000 -> 6379
- I0710 14:43:38.274797 3655 portforward.go:225] Forwarding from [::1]:7000 -> 6379
+ Forwarding from 127.0.0.1:7000 -> 6379
+ Forwarding from [::1]:7000 -> 6379
```
+{{< note >}}
+
+`kubectl port-forward` does not return. To continue with the exercises, you will need to open another terminal.
+
+{{< /note >}}
+
2. Start the Redis command line interface:
```shell
@@ -180,7 +186,23 @@ for database debugging.
PONG
```
+### Optionally let _kubectl_ choose the local port {#let-kubectl-choose-local-port}
+If you don't need a specific local port, you can let `kubectl` choose and allocate
+the local port and thus relieve you from having to manage local port conflicts, with
+the slightly simpler syntax:
+
+```shell
+kubectl port-forward deployment/redis-master :6379
+```
+
+The `kubectl` tool finds a local port number that is not in use (avoiding low ports numbers,
+because these might be used by other applications). The output is similar to:
+
+```
+Forwarding from 127.0.0.1:62162 -> 6379
+Forwarding from [::1]:62162 -> 6379
+```
@@ -203,4 +225,3 @@ The support for UDP protocol is tracked in
## {{% heading "whatsnext" %}}
Learn more about [kubectl port-forward](/docs/reference/generated/kubectl/kubectl-commands/#port-forward).
-
diff --git a/content/en/docs/tasks/administer-cluster/developing-cloud-controller-manager.md b/content/en/docs/tasks/administer-cluster/developing-cloud-controller-manager.md
index 0f6579d915..a3732c68de 100644
--- a/content/en/docs/tasks/administer-cluster/developing-cloud-controller-manager.md
+++ b/content/en/docs/tasks/administer-cluster/developing-cloud-controller-manager.md
@@ -30,7 +30,7 @@ The Kubernetes project provides skeleton cloud-controller-manager code with Go i
To build an out-of-tree cloud-controller-manager for your cloud:
1. Create a go package with an implementation that satisfies [cloudprovider.Interface](https://github.com/kubernetes/cloud-provider/blob/master/cloud.go).
-2. Use [`main.go` in cloud-controller-manager](https://github.com/kubernetes/kubernetes/blob/master/cmd/cloud-controller-manager/controller-manager.go) from Kubernetes core as a template for your `main.go`. As mentioned above, the only difference should be the cloud package that will be imported.
+2. Use [`main.go` in cloud-controller-manager](https://github.com/kubernetes/kubernetes/blob/master/cmd/cloud-controller-manager/main.go) from Kubernetes core as a template for your `main.go`. As mentioned above, the only difference should be the cloud package that will be imported.
3. Import your cloud package in `main.go`, ensure your package has an `init` block to run [`cloudprovider.RegisterCloudProvider`](https://github.com/kubernetes/cloud-provider/blob/master/plugins.go).
Many cloud providers publish their controller manager code as open source. If you are creating
diff --git a/content/en/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade.md b/content/en/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade.md
index 0164d5aea1..5af9d27b82 100644
--- a/content/en/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade.md
+++ b/content/en/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade.md
@@ -231,6 +231,14 @@ without compromising the minimum required capacity for running your workloads.
{{% /tab %}}
{{< /tabs >}}
+### Call "kubeadm upgrade"
+
+- For worker nodes this upgrades the local kubelet configuration:
+
+ ```shell
+ sudo kubeadm upgrade node
+ ```
+
### Drain the node
- Prepare the node for maintenance by marking it unschedulable and evicting the workloads:
@@ -240,14 +248,6 @@ without compromising the minimum required capacity for running your workloads.
kubectl drain --ignore-daemonsets
```
-### Call "kubeadm upgrade"
-
-- For worker nodes this upgrades the local kubelet configuration:
-
- ```shell
- sudo kubeadm upgrade node
- ```
-
### Upgrade kubelet and kubectl
- Upgrade the kubelet and kubectl:
diff --git a/content/en/docs/tasks/administer-cluster/limit-storage-consumption.md b/content/en/docs/tasks/administer-cluster/limit-storage-consumption.md
index 1347dc85a7..89f130a010 100644
--- a/content/en/docs/tasks/administer-cluster/limit-storage-consumption.md
+++ b/content/en/docs/tasks/administer-cluster/limit-storage-consumption.md
@@ -12,7 +12,6 @@ The following resources are used in the demonstration: [ResourceQuota](/docs/con
and [PersistentVolumeClaim](/docs/concepts/storage/persistent-volumes/).
-
## {{% heading "prerequisites" %}}
@@ -41,7 +40,7 @@ the values set by the admin.
In this example, a PVC requesting 10Gi of storage would be rejected because it exceeds the 2Gi max.
-```
+```yaml
apiVersion: v1
kind: LimitRange
metadata:
@@ -67,7 +66,7 @@ In this example, a 6th PVC in the namespace would be rejected because it exceeds
a 5Gi maximum quota when combined with the 2Gi max limit above, cannot have 3 PVCs where each has 2Gi. That would be 6Gi requested
for a namespace capped at 5Gi.
-```
+```yaml
apiVersion: v1
kind: ResourceQuota
metadata:
@@ -78,8 +77,6 @@ spec:
requests.storage: "5Gi"
```
-
-
## Summary
@@ -87,7 +84,3 @@ spec:
A limit range can put a ceiling on how much storage is requested while a resource quota can effectively cap the storage
consumed by a namespace through claim counts and cumulative storage capacity. The allows a cluster-admin to plan their
cluster's storage budget without risk of any one project going over their allotment.
-
-
-
-
diff --git a/content/en/docs/tasks/administer-cluster/namespaces.md b/content/en/docs/tasks/administer-cluster/namespaces.md
index 00dec41774..08b2868806 100644
--- a/content/en/docs/tasks/administer-cluster/namespaces.md
+++ b/content/en/docs/tasks/administer-cluster/namespaces.md
@@ -13,7 +13,7 @@ This page shows how to view, work in, and delete {{< glossary_tooltip text="name
## {{% heading "prerequisites" %}}
* Have an [existing Kubernetes cluster](/docs/setup/).
-2. You have a basic understanding of Kubernetes {{< glossary_tooltip text="Pods" term_id="pod" >}}, {{< glossary_tooltip term_id="service" text="Services" >}}, and {{< glossary_tooltip text="Deployments" term_id="deployment" >}}.
+* You have a basic understanding of Kubernetes {{< glossary_tooltip text="Pods" term_id="pod" >}}, {{< glossary_tooltip term_id="service" text="Services" >}}, and {{< glossary_tooltip text="Deployments" term_id="deployment" >}}.
diff --git a/content/en/docs/tasks/configure-pod-container/attach-handler-lifecycle-event.md b/content/en/docs/tasks/configure-pod-container/attach-handler-lifecycle-event.md
index 00b9251be8..4576b0f02b 100644
--- a/content/en/docs/tasks/configure-pod-container/attach-handler-lifecycle-event.md
+++ b/content/en/docs/tasks/configure-pod-container/attach-handler-lifecycle-event.md
@@ -9,7 +9,7 @@ weight: 140
This page shows how to attach handlers to Container lifecycle events. Kubernetes supports
the postStart and preStop events. Kubernetes sends the postStart event immediately
after a Container is started, and it sends the preStop event immediately before the
-Container is terminated.
+Container is terminated. A Container may specify one handler per event.
diff --git a/content/en/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes.md b/content/en/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes.md
index 5896ab8357..45d56531f2 100644
--- a/content/en/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes.md
+++ b/content/en/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes.md
@@ -366,7 +366,7 @@ have additional fields that can be set on `httpGet`:
* `host`: Host name to connect to, defaults to the pod IP. You probably want to
set "Host" in httpHeaders instead.
* `scheme`: Scheme to use for connecting to the host (HTTP or HTTPS). Defaults to HTTP.
-* `path`: Path to access on the HTTP server.
+* `path`: Path to access on the HTTP server. Defaults to /.
* `httpHeaders`: Custom headers to set in the request. HTTP allows repeated headers.
* `port`: Name or number of the port to access on the container. Number must be
in the range 1 to 65535.
@@ -389,24 +389,32 @@ You can override the default headers by defining `.httpHeaders` for the probe; f
```yaml
livenessProbe:
- httpHeaders:
- Accept: application/json
+ httpGet:
+ httpHeaders:
+ - name: Accept
+ value: application/json
startupProbe:
- httpHeaders:
- User-Agent: MyUserAgent
+ httpGet:
+ httpHeaders:
+ - name: User-Agent
+ value: MyUserAgent
```
You can also remove these two headers by defining them with an empty value.
```yaml
livenessProbe:
- httpHeaders:
- Accept: ""
+ httpGet:
+ httpHeaders:
+ - name: Accept
+ value: ""
startupProbe:
- httpHeaders:
- User-Agent: ""
+ httpGet:
+ httpHeaders:
+ - name: User-Agent
+ value: ""
```
### TCP probes
diff --git a/content/en/docs/tasks/debug-application-cluster/crictl.md b/content/en/docs/tasks/debug-application-cluster/crictl.md
index 5f8214b610..343e3b1cb0 100644
--- a/content/en/docs/tasks/debug-application-cluster/crictl.md
+++ b/content/en/docs/tasks/debug-application-cluster/crictl.md
@@ -360,7 +360,9 @@ for more information.
The exact versions for below mapping table are for docker cli v1.40 and crictl v1.19.0. Please note that the list is not exhaustive. For example, it doesn't include experimental commands of docker cli.
-Warn: the output format of CRICTL is similar to Docker CLI, despite some missing columns for some CLI. Make sure to check output for the specific command if your script output parsing.
+{{< note >}}
+The output format of CRICTL is similar to Docker CLI, despite some missing columns for some CLI. Make sure to check output for the specific command if your script output parsing.
+{{< /note >}}
### Retrieve Debugging Information
diff --git a/content/en/docs/tasks/debug-application-cluster/debug-pod-replication-controller.md b/content/en/docs/tasks/debug-application-cluster/debug-pod-replication-controller.md
index 8f7c97c4e8..8a972e1365 100644
--- a/content/en/docs/tasks/debug-application-cluster/debug-pod-replication-controller.md
+++ b/content/en/docs/tasks/debug-application-cluster/debug-pod-replication-controller.md
@@ -49,7 +49,7 @@ case you can try several things:
* Add more nodes to the cluster.
-* [Terminate unneeded pods](/docs/concepts/workloads/pods/#pod-termination)
+* [Terminate unneeded pods](/docs/concepts/workloads/pods/pod-lifecycle/#pod-termination)
to make room for pending pods.
* Check that the pod is not larger than your nodes. For example, if all
diff --git a/content/en/docs/tasks/extend-kubernetes/setup-konnectivity.md b/content/en/docs/tasks/extend-kubernetes/setup-konnectivity.md
index 82eecc9c38..34936014f6 100644
--- a/content/en/docs/tasks/extend-kubernetes/setup-konnectivity.md
+++ b/content/en/docs/tasks/extend-kubernetes/setup-konnectivity.md
@@ -37,8 +37,22 @@ by providing the following flags to the kube-apiserver:
1. Create an egress configuration file such as `admin/konnectivity/egress-selector-configuration.yaml`.
1. Set the `--egress-selector-config-file` flag of the API Server to the path of
your API Server egress configuration file.
+1. If you use UDS connection, add volumes config to the kube-apiserver:
+ ```yaml
+ spec:
+ containers:
+ volumeMounts:
+ - name: konnectivity-uds
+ mountPath: /etc/kubernetes/konnectivity-server
+ readOnly: false
+ volumes:
+ - name: konnectivity-uds
+ hostPath:
+ path: /etc/kubernetes/konnectivity-server
+ type: DirectoryOrCreate
+ ```
-Generate or obtain a certificate and kubeconfig for konnectivity-server.
+Generate or obtain a certificate and kubeconfig for konnectivity-server.
For example, you can use the OpenSSL command line tool to issue a X.509 certificate,
using the cluster CA certificate `/etc/kubernetes/pki/ca.crt` from a control-plane host.
diff --git a/content/en/docs/tasks/manage-hugepages/scheduling-hugepages.md b/content/en/docs/tasks/manage-hugepages/scheduling-hugepages.md
index 36b92ac18a..8d4cd4afe6 100644
--- a/content/en/docs/tasks/manage-hugepages/scheduling-hugepages.md
+++ b/content/en/docs/tasks/manage-hugepages/scheduling-hugepages.md
@@ -104,17 +104,19 @@ spec:
- Huge page requests must equal the limits. This is the default if limits are
specified, but requests are not.
-- Huge pages are isolated at a container scope, so each container has own limit on their cgroup sandbox as requested in a container spec.
+- Huge pages are isolated at a container scope, so each container has own
+ limit on their cgroup sandbox as requested in a container spec.
- EmptyDir volumes backed by huge pages may not consume more huge page memory
than the pod request.
- Applications that consume huge pages via `shmget()` with `SHM_HUGETLB` must
run with a supplemental group that matches `proc/sys/vm/hugetlb_shm_group`.
- Huge page usage in a namespace is controllable via ResourceQuota similar
-to other compute resources like `cpu` or `memory` using the `hugepages-`
-token.
+ to other compute resources like `cpu` or `memory` using the `hugepages-`
+ token.
- Support of multiple sizes huge pages is feature gated. It can be
- disabled with the `HugePageStorageMediumSize` [feature
-gate](/docs/reference/command-line-tools-reference/feature-gates/) on the {{<
-glossary_tooltip text="kubelet" term_id="kubelet" >}} and {{<
-glossary_tooltip text="kube-apiserver"
-term_id="kube-apiserver" >}} (`--feature-gates=HugePageStorageMediumSize=true`).
+ disabled with the `HugePageStorageMediumSize`
+ [feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
+ on the {{< glossary_tooltip text="kubelet" term_id="kubelet" >}} and
+ {{< glossary_tooltip text="kube-apiserver" term_id="kube-apiserver" >}}
+ (`--feature-gates=HugePageStorageMediumSize=false`).
+
diff --git a/content/en/docs/tasks/manage-kubernetes-objects/kustomization.md b/content/en/docs/tasks/manage-kubernetes-objects/kustomization.md
index f5e20a94c0..d655d39be9 100644
--- a/content/en/docs/tasks/manage-kubernetes-objects/kustomization.md
+++ b/content/en/docs/tasks/manage-kubernetes-objects/kustomization.md
@@ -590,7 +590,7 @@ spec:
containers:
- name: my-nginx
image: nginx
- command: ["start", "--host", "\$(MY_SERVICE_NAME)"]
+ command: ["start", "--host", "$(MY_SERVICE_NAME)"]
EOF
# Create a service.yaml file
diff --git a/content/en/docs/tasks/run-application/configure-pdb.md b/content/en/docs/tasks/run-application/configure-pdb.md
index 8113e07128..3823cac4ee 100644
--- a/content/en/docs/tasks/run-application/configure-pdb.md
+++ b/content/en/docs/tasks/run-application/configure-pdb.md
@@ -236,9 +236,6 @@ You can use a PDB with pods controlled by another type of controller, by an
- only an integer value can be used with `.spec.minAvailable`, not a percentage.
You can use a selector which selects a subset or superset of the pods belonging to a built-in
-controller. However, when there are multiple PDBs in a namespace, you must be careful not
-to create PDBs whose selectors overlap.
-
-
-
-
+controller. The eviction API will disallow eviction of any pod covered by multiple PDBs,
+so most users will want to avoid overlapping selectors. One reasonable use of overlapping
+PDBs is when pods are being transitioned from one PDB to another.
diff --git a/content/en/docs/tutorials/stateless-application/expose-external-ip-address.md b/content/en/docs/tutorials/stateless-application/expose-external-ip-address.md
index 5babc2c0b0..8368d24132 100644
--- a/content/en/docs/tutorials/stateless-application/expose-external-ip-address.md
+++ b/content/en/docs/tutorials/stateless-application/expose-external-ip-address.md
@@ -9,78 +9,73 @@ weight: 10
This page shows how to create a Kubernetes Service object that exposes an
external IP address.
-
-
-
## {{% heading "prerequisites" %}}
-
- * Install [kubectl](/docs/tasks/tools/install-kubectl/).
-
- * Use a cloud provider like Google Kubernetes Engine or Amazon Web Services to
- create a Kubernetes cluster. This tutorial creates an
- [external load balancer](/docs/tasks/access-application-cluster/create-external-load-balancer/),
- which requires a cloud provider.
-
- * Configure `kubectl` to communicate with your Kubernetes API server. For
- instructions, see the documentation for your cloud provider.
-
-
-
+* Install [kubectl](/docs/tasks/tools/install-kubectl/).
+* Use a cloud provider like Google Kubernetes Engine or Amazon Web Services to
+ create a Kubernetes cluster. This tutorial creates an
+ [external load balancer](/docs/tasks/access-application-cluster/create-external-load-balancer/),
+ which requires a cloud provider.
+* Configure `kubectl` to communicate with your Kubernetes API server. For instructions, see the
+ documentation for your cloud provider.
## {{% heading "objectives" %}}
-
* Run five instances of a Hello World application.
* Create a Service object that exposes an external IP address.
* Use the Service object to access the running application.
-
-
-
## Creating a service for an application running in five pods
1. Run a Hello World application in your cluster:
-{{< codenew file="service/load-balancer-example.yaml" >}}
+ {{< codenew file="service/load-balancer-example.yaml" >}}
-```shell
-kubectl apply -f https://k8s.io/examples/service/load-balancer-example.yaml
-```
-
-
-The preceding command creates a
- {{< glossary_tooltip text="Deployment" term_id="deployment" >}}
- and an associated
- {{< glossary_tooltip term_id="replica-set" text="ReplicaSet" >}}.
- The ReplicaSet has five
- {{< glossary_tooltip text="Pods" term_id="pod" >}}
- each of which runs the Hello World application.
+ ```shell
+ kubectl apply -f https://k8s.io/examples/service/load-balancer-example.yaml
+ ```
+ The preceding command creates a
+ {{< glossary_tooltip text="Deployment" term_id="deployment" >}}
+ and an associated
+ {{< glossary_tooltip term_id="replica-set" text="ReplicaSet" >}}.
+ The ReplicaSet has five
+ {{< glossary_tooltip text="Pods" term_id="pod" >}}
+ each of which runs the Hello World application.
1. Display information about the Deployment:
- kubectl get deployments hello-world
- kubectl describe deployments hello-world
+ ```shell
+ kubectl get deployments hello-world
+ kubectl describe deployments hello-world
+ ```
1. Display information about your ReplicaSet objects:
- kubectl get replicasets
- kubectl describe replicasets
+ ```shell
+ kubectl get replicasets
+ kubectl describe replicasets
+ ```
1. Create a Service object that exposes the deployment:
- kubectl expose deployment hello-world --type=LoadBalancer --name=my-service
+ ```shell
+ kubectl expose deployment hello-world --type=LoadBalancer --name=my-service
+ ```
1. Display information about the Service:
- kubectl get services my-service
+ ```shell
+ kubectl get services my-service
+ ```
- The output is similar to this:
+ The output is similar to:
- NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
- my-service LoadBalancer 10.3.245.137 104.198.205.71 8080/TCP 54s
+ ```console
+ NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
+ my-service LoadBalancer 10.3.245.137 104.198.205.71 8080/TCP 54s
+ ```
{{< note >}}
@@ -96,23 +91,27 @@ The preceding command creates a
1. Display detailed information about the Service:
- kubectl describe services my-service
+ ```shell
+ kubectl describe services my-service
+ ```
- The output is similar to this:
+ The output is similar to:
- Name: my-service
- Namespace: default
- Labels: app.kubernetes.io/name=load-balancer-example
- Annotations:
- Selector: app.kubernetes.io/name=load-balancer-example
- Type: LoadBalancer
- IP: 10.3.245.137
- LoadBalancer Ingress: 104.198.205.71
- Port: 8080/TCP
- NodePort: 32377/TCP
- Endpoints: 10.0.0.6:8080,10.0.1.6:8080,10.0.1.7:8080 + 2 more...
- Session Affinity: None
- Events:
+ ```console
+ Name: my-service
+ Namespace: default
+ Labels: app.kubernetes.io/name=load-balancer-example
+ Annotations:
+ Selector: app.kubernetes.io/name=load-balancer-example
+ Type: LoadBalancer
+ IP: 10.3.245.137
+ LoadBalancer Ingress: 104.198.205.71
+ Port: 8080/TCP
+ NodePort: 32377/TCP
+ Endpoints: 10.0.0.6:8080,10.0.1.6:8080,10.0.1.7:8080 + 2 more...
+ Session Affinity: None
+ Events:
+ ```
Make a note of the external IP address (`LoadBalancer Ingress`) exposed by
your service. In this example, the external IP address is 104.198.205.71.
@@ -124,21 +123,27 @@ The preceding command creates a
addresses of the pods that are running the Hello World application. To
verify these are pod addresses, enter this command:
- kubectl get pods --output=wide
+ ```shell
+ kubectl get pods --output=wide
+ ```
- The output is similar to this:
+ The output is similar to:
- NAME ... IP NODE
- hello-world-2895499144-1jaz9 ... 10.0.1.6 gke-cluster-1-default-pool-e0b8d269-1afc
- hello-world-2895499144-2e5uh ... 10.0.1.8 gke-cluster-1-default-pool-e0b8d269-1afc
- hello-world-2895499144-9m4h1 ... 10.0.0.6 gke-cluster-1-default-pool-e0b8d269-5v7a
- hello-world-2895499144-o4z13 ... 10.0.1.7 gke-cluster-1-default-pool-e0b8d269-1afc
- hello-world-2895499144-segjf ... 10.0.2.5 gke-cluster-1-default-pool-e0b8d269-cpuc
+ ```console
+ NAME ... IP NODE
+ hello-world-2895499144-1jaz9 ... 10.0.1.6 gke-cluster-1-default-pool-e0b8d269-1afc
+ hello-world-2895499144-2e5uh ... 10.0.1.8 gke-cluster-1-default-pool-e0b8d269-1afc
+ hello-world-2895499144-9m4h1 ... 10.0.0.6 gke-cluster-1-default-pool-e0b8d269-5v7a
+ hello-world-2895499144-o4z13 ... 10.0.1.7 gke-cluster-1-default-pool-e0b8d269-1afc
+ hello-world-2895499144-segjf ... 10.0.2.5 gke-cluster-1-default-pool-e0b8d269-cpuc
+ ```
1. Use the external IP address (`LoadBalancer Ingress`) to access the Hello
World application:
- curl http://:
+ ```shell
+ curl http://:
+ ```
where `` is the external IP address (`LoadBalancer Ingress`)
of your Service, and `` is the value of `Port` in your Service
@@ -148,29 +153,26 @@ The preceding command creates a
The response to a successful request is a hello message:
- Hello Kubernetes!
-
-
-
+ ```shell
+ Hello Kubernetes!
+ ```
## {{% heading "cleanup" %}}
-
To delete the Service, enter this command:
- kubectl delete services my-service
+```shell
+kubectl delete services my-service
+```
To delete the Deployment, the ReplicaSet, and the Pods that are running
the Hello World application, enter this command:
- kubectl delete deployment hello-world
-
-
-
+```shell
+kubectl delete deployment hello-world
+```
## {{% heading "whatsnext" %}}
-
Learn more about
[connecting applications with services](/docs/concepts/services-networking/connect-applications-service/).
-
diff --git a/content/es/community/_index.html b/content/es/community/_index.html
index 8f557f3526..56e42c4899 100644
--- a/content/es/community/_index.html
+++ b/content/es/community/_index.html
@@ -9,7 +9,7 @@ cid: community
Garantizando el funcionamiento de Kubernetes para todo el mundo y en cualquier lugar.
Conecte con la comunidad Kubernetes en nuestro canal de Slack, foro, o únete al
- Kubernetes-dev Google group.
+ Kubernetes-dev Google group.
Cada semana se lleva a cabo una reunión de la comunidad por videoconferencia para discutir el estado de cosas, revise el documento
Community Meeting para obtener información sobre cómo participar.
También puede formar parte de la comunidad en cualquier parte del mundo a través de la
@@ -59,4 +59,4 @@ cid: community
Cette page vous
-apprend comment installer la boîte à outils `kubeadm`.
-Pour plus d'informations sur la création d'un cluster avec kubeadm, une fois que vous avez
-effectué ce processus d'installation, voir la page: [Utiliser kubeadm pour créer un cluster](/fr/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/).
+
Cette page vous apprend comment installer la boîte à outils `kubeadm`.
+Pour plus d'informations sur la création d'un cluster avec kubeadm, une fois que vous avez effectué ce processus d'installation, voir la page: [Utiliser kubeadm pour créer un cluster](/fr/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/).
@@ -19,39 +20,53 @@ effectué ce processus d'installation, voir la page: [Utiliser kubeadm pour cré
* Une ou plusieurs machines exécutant:
- Ubuntu 16.04+
- - Debian 9
+ - Debian 9+
- CentOS 7
- - RHEL 7
- - Fedora 25/26 (best-effort)
+ - Red Hat Enterprise Linux (RHEL) 7
+ - Fedora 25+
- HypriotOS v1.0.1+
- - Container Linux (testé avec 1800.6.0)
+ - Flatcar Container Linux (testé avec 2512.3.0)
* 2 Go ou plus de RAM par machine (toute quantité inférieure laissera peu de place à vos applications)
* 2 processeurs ou plus
* Connectivité réseau complète entre toutes les machines du cluster (réseau public ou privé)
* Nom d'hôte, adresse MAC et product_uuid uniques pour chaque nœud. Voir [ici](#verify-the-mac-address-and-product-uuid-are-unique-for-every-node) pour plus de détails.
* Certains ports doivent êtres ouverts sur vos machines. Voir [ici](#check-required-ports) pour plus de détails.
-* Swap désactivé. Vous devez impérativement désactiver le swap pour que la kubelet fonctionne correctement.
+* Swap désactivé. Vous **devez** impérativement désactiver le swap pour que la kubelet fonctionne correctement.
-## Vérifiez que les adresses MAC et product_uuid sont uniques pour chaque nœud {#verify-the-mac-address-and-product-uuid-are-unique-for-every-node}
+## Vérifiez que les adresses MAC et product_uuid sont uniques pour chaque nœud {#verify-mac-address}
* Vous pouvez obtenir l'adresse MAC des interfaces réseau en utilisant la commande `ip link` ou` ifconfig -a`
* Le product_uuid peut être vérifié en utilisant la commande `sudo cat/sys/class/dmi/id/product_uuid`
Il est très probable que les périphériques matériels aient des adresses uniques, bien que
-certaines machines virtuelles puissent avoir des valeurs identiques. Kubernetes utilise
-ces valeurs pour identifier de manière unique les nœuds du cluster.
+certaines machines virtuelles puissent avoir des valeurs identiques. Kubernetes utilise ces valeurs pour identifier de manière unique les nœuds du cluster.
Si ces valeurs ne sont pas uniques à chaque nœud, le processus d'installation
peut [échouer](https://github.com/kubernetes/kubeadm/issues/31).
## Vérifiez les cartes réseaux
-Si vous avez plusieurs cartes réseaux et que vos composants Kubernetes ne sont pas accessibles par la
-route par défaut, nous vous recommandons d’ajouter une ou plusieurs routes IP afin que les adresses
-de cluster Kubernetes soient acheminées via la carte approprié.
+Si vous avez plusieurs cartes réseaux et que vos composants Kubernetes ne sont pas accessibles par la route par défaut,
+nous vous recommandons d’ajouter une ou plusieurs routes IP afin que les adresses de cluster Kubernetes soient acheminées via la carte approprié.
+
+## Permettre à iptables de voir le trafic ponté
+
+Assurez-vous que le module `br_netfilter` est chargé. Cela peut être fait en exécutant `lsmod | grep br_netfilter`. Pour le charger explicitement, appelez `sudo modprobe br_netfilter`.
+
+Pour que les iptables de votre nœud Linux voient correctement le trafic ponté, vous devez vous assurer que `net.bridge.bridge-nf-call-iptables` est défini sur 1 dans votre configuration` sysctl`, par ex.
+
+```bash
+cat <
+ 
+ 
+