Merge remote-tracking branch 'upstream/main' into dev-1.24

2022-01-31 11:57:45 -08:00
parent 5e35828808 b2063eedeb
commit 2fa88c51b6
73 changed files with 2005 additions and 430 deletions
@@ -190,7 +190,8 @@ kubectl get configmap
 No resources found in default namespace.
 ```

-To sum things up, when there's an override owner reference from a child to a parent, deleting the parent deletes the children automatically. This is called `cascade`. The default for cascade is `true`, however, you can use the --cascade=orphan option for `kubectl delete` to delete an object and orphan its children. 
+To sum things up, when there's an override owner reference from a child to a parent, deleting the parent deletes the children automatically. This is called `cascade`. The default for cascade is `true`, however, you can use the --cascade=orphan option for `kubectl delete` to delete an object and orphan its children.  *Update: starting with kubectl v1.20, the default for cascade is `background`.*
+

 In the following example, there is a parent and a child. Notice the owner references are still included. If I delete the parent using --cascade=orphan, the parent is deleted but the child still exists:

@@ -138,4 +138,5 @@ Stay tuned for what comes next, and if you have any questions, comments or sugge
 * Chat with us on the Kubernetes [Slack](http://slack.k8s.io/):[#cluster-api](https://kubernetes.slack.com/archives/C8TSNPY4T)
 * Join the SIG Cluster Lifecycle [Google Group](https://groups.google.com/g/kubernetes-sig-cluster-lifecycle) to receive calendar invites and gain access to documents
 * Join our [Zoom meeting](https://zoom.us/j/861487554), every Wednesday at 10:00 Pacific Time
-* Check out the [ClusterClass tutorial](https://cluster-api.sigs.k8s.io/tasks/experimental-features/cluster-classes.html) in the Cluster API book.
+* Check out the [ClusterClass quick-start](https://cluster-api.sigs.k8s.io/user/quick-start.html) for the Docker provider (CAPD) in the Cluster API book.
+* _UPDATE_: Check out the [ClusterClass experimental feature](https://cluster-api.sigs.k8s.io/tasks/experimental-features/cluster-class/index.html) documentation in the Cluster API book.
@@ -20,7 +20,7 @@ case_study_details:

 <h2>Solution</h2>

-<p>In 2016, the company began moving their code from Heroku to <a href="https://www.docker.com/">Docker</a> containers running on <a href="https://cloud.google.com/kubernetes-engine/">Google Kubernetes Engine</a>, orchestrated by <a href="http://kubernetes.io/">Kubernetes</a> and monitored with <a href="https://prometheus.io/">Prometheus</a>.</p>
+<p>In 2016, the company began moving their code from Heroku to containers running on <a href="https://cloud.google.com/kubernetes-engine/">Google Kubernetes Engine</a>, orchestrated by <a href="http://kubernetes.io/">Kubernetes</a> and monitored with <a href="https://prometheus.io/">Prometheus</a>.</p>

 <h2>Impact</h2>

@@ -42,7 +42,7 @@ With the speed befitting a startup, Pear Deck delivered its first prototype to c

 <p>On top of that, many of Pear Deck's customers are behind government firewalls and connect through Firebase, not Pear Deck's servers, making troubleshooting even more difficult.</p>

-<p>The team began looking around for another solution, and finally decided in early 2016 to start moving the app from Heroku to <a href="https://www.docker.com/">Docker</a> containers running on <a href="https://cloud.google.com/kubernetes-engine/">Google Kubernetes Engine</a>, orchestrated by <a href="http://kubernetes.io/">Kubernetes</a> and monitored with <a href="https://prometheus.io/">Prometheus</a>.</p>
+<p>The team began looking around for another solution, and finally decided in early 2016 to start moving the app from Heroku to containers running on <a href="https://cloud.google.com/kubernetes-engine/">Google Kubernetes Engine</a>, orchestrated by <a href="http://kubernetes.io/">Kubernetes</a> and monitored with <a href="https://prometheus.io/">Prometheus</a>.</p>

 {{< case-studies/quote image="/images/case-studies/peardeck/banner1.jpg" >}}
 "When it became clear that Google Kubernetes Engine was going to have a lot of support from Google and be a fully-managed Kubernetes platform, it seemed very obvious to us that was the way to go," says Eynon-Lynch.
@@ -70,7 +70,7 @@ If you haven't set foot in a school in awhile, you might be surprised by what yo

 <p>Recently, the team launched a new single sign-on solution for use in an internal application. "Due to the resource based architecture of the Kubernetes platform, we were able to bring that application into an entirely new production environment in less than a day, most of that time used for testing after applying the already well-known resource definitions from staging to the new environment," says van den Bosch. "On a traditional VM this would have likely cost a day or two, and then probably a few weeks to iron out the kinks in our provisioning scripts as we apply updates."</p>

-<p>Legacy applications are also being moved to Kubernetes. Not long ago, the team needed to set up a Java-based application for compiling and running a frontend. "On a traditional VM, it would have taken quite a bit of time to set it up and keep it up to date, not to mention maintenance for that setup down the line," says van den Bosch. Instead, it took less than half a day to Dockerize it and get it running on Kubernetes. "It was much easier, and we were able to save costs too because we didn't have to spin up new VMs specially for it."</p>
+<p>Legacy applications are also being moved to Kubernetes. Not long ago, the team needed to set up a Java-based application for compiling and running a frontend. "On a traditional VM, it would have taken quite a bit of time to set it up and keep it up to date, not to mention maintenance for that setup down the line," says van den Bosch. Instead, it took less than half a day to containerize it and get it running on Kubernetes. "It was much easier, and we were able to save costs too because we didn't have to spin up new VMs specially for it."</p>

 {{< case-studies/quote author="VICTOR VAN DEN BOSCH, SENIOR DEVOPS ENGINEER, PROWISE" >}}
 "We're really trying to deliver integrated solutions with our hardware and software and making it as easy as possible for users to use and collaborate from different places," says van den Bosch. And, says Haalstra, "We cannot do it without Kubernetes."
@@ -46,7 +46,7 @@ Since it was started in a dorm room in 2003, Squarespace has made it simple for
 After experimenting with another container orchestration platform and "breaking it in very painful ways," Lynch says, the team began experimenting with Kubernetes in mid-2016 and found that it "answered all the questions that we had."
 {{< /case-studies/quote >}}

-<p>Within a couple months, they had a stable cluster for their internal use, and began rolling out Kubernetes for production. They also added Zipkin and CNCF projects <a href="https://prometheus.io/">Prometheus</a> and <a href="https://www.fluentd.org/">fluentd</a> to their cloud native stack. "We switched to Kubernetes, a new world, and we revamped all our other tooling as well," says Lynch. "It allowed us to streamline our process, so we can now easily create an entire microservice project from templates, generate the code and deployment pipeline for that, generate the Docker file, and then immediately just ship a workable, deployable project to Kubernetes." Deployments across Dev/QA/Stage/Prod were also "simplified drastically," Lynch adds. "Now there is little configuration variation."</p>
+<p>Within a couple months, they had a stable cluster for their internal use, and began rolling out Kubernetes for production. They also added Zipkin and CNCF projects <a href="https://prometheus.io/">Prometheus</a> and <a href="https://www.fluentd.org/">fluentd</a> to their cloud native stack. "We switched to Kubernetes, a new world, and we revamped all our other tooling as well," says Lynch. "It allowed us to streamline our process, so we can now easily create an entire microservice project from templates, generate the code and deployment pipeline for that, generate the Dockerfile, and then immediately just ship a workable, deployable project to Kubernetes." Deployments across Dev/QA/Stage/Prod were also "simplified drastically," Lynch adds. "Now there is little configuration variation."</p>

 <p>And the whole process takes only five minutes, an almost 85% reduction in time compared to their VM deployment. "From end to end that probably took half an hour, and that's not accounting for the fact that an infrastructure engineer would be responsible for doing that, so there's some business delay in there as well."</p>

@@ -58,9 +58,9 @@ How many people does it take to turn on a light bulb?

 <p>In addition, Wink had other requirements: horizontal scalability, the ability to encrypt everything quickly, connections that could be easily brought back up if something went wrong. "Looking at this whole structure we started, we decided to make a secure socket-based service," says Klein. "We've always used, I would say, some sort of clustering technology to deploy our services and so the decision we came to was, this thing is going to be containerized, running on Docker."</p>

-<p>At the time – just over two years ago – Docker wasn't yet widely used, but as Klein points out, "it was certainly understood by the people who were on the frontier of technology. We started looking at potential technologies that existed. One of the limiting factors was that we needed to deploy multi-port non-http/https services. It wasn't really appropriate for some of the early cluster technology. We liked the project a lot and we ended up using it on other stuff for a while, but initially it was too targeted toward http workloads."</p>
+<p>In 2015, Docker wasn't yet widely used, but as Klein points out, "it was certainly understood by the people who were on the frontier of technology. We started looking at potential technologies that existed. One of the limiting factors was that we needed to deploy multi-port non-http/https services. It wasn't really appropriate for some of the early cluster technology. We liked the project a lot and we ended up using it on other stuff for a while, but initially it was too targeted toward http workloads."</p>

-<p>Once Wink's backend engineering team decided on a Dockerized workload, they had to make decisions about the OS and the container orchestration platform. "Obviously you can't just start the containers and hope everything goes well," Klein says with a laugh. "You need to have a system that is helpful [in order] to manage where the workloads are being distributed out to. And when the container inevitably dies or something like that, to restart it, you have a load balancer. All sorts of housekeeping work is needed to have a robust infrastructure."</p>
+<p>Once Wink's backend engineering team decided on a containerized workload, they had to make decisions about the OS and the container orchestration platform. "Obviously you can't just start the containers and hope everything goes well," Klein says with a laugh. "You need to have a system that is helpful [in order] to manage where the workloads are being distributed out to. And when the container inevitably dies or something like that, to restart it, you have a load balancer. All sorts of housekeeping work is needed to have a robust infrastructure."</p>

 {{< case-studies/quote image="/images/case-studies/wink/banner4.jpg" >}}
 "Obviously you can't just start the containers and hope everything goes well," Klein says with a laugh. "You need to have a system that is helpful [in order] to manage where the workloads are being distributed out to. And when the container inevitably dies or something like that, to restart it, you have a load balancer. All sorts of housekeeping work is needed to have a robust infrastructure."
@@ -68,7 +68,7 @@ How many people does it take to turn on a light bulb?

 <p>Wink considered building directly on a general purpose Linux distro like Ubuntu (which would have required installing tools to run a containerized workload) and cluster management systems like Mesos (which was targeted toward enterprises with larger teams/workloads), but ultimately set their sights on CoreOS Container Linux. "A container-optimized Linux distribution system was exactly what we needed," he says. "We didn't have to futz around with trying to take something like a Linux distro and install everything. It's got a built-in container orchestration system, which is Fleet, and an easy-to-use API. It's not as feature-rich as some of the heavier solutions, but we realized that, at that moment, it was exactly what we needed."</p>

-<p>Wink's hub (along with a revamped app) was introduced in July 2014 with a short-term deployment, and within the first month, they had moved the service to the Dockerized CoreOS deployment. Since then, they've moved almost every other piece of their infrastructure – from third-party cloud-to-cloud integrations to their customer service and payment portals – onto CoreOS Container Linux clusters.</p>
+<p>Wink's hub (along with a revamped app) was introduced in July 2014 with a short-term deployment, and within the first month, they had moved the service to the containerized CoreOS deployment. Since then, they've moved almost every other piece of their infrastructure – from third-party cloud-to-cloud integrations to their customer service and payment portals – onto CoreOS Container Linux clusters.</p>

 <p>Using this setup did require some customization. "Fleet is really nice as a basic container orchestration system, but it doesn't take care of routing, sharing configurations, secrets, et cetera, among instances of a service," Klein says. "All of those layers of functionality can be implemented, of course, but if you don't want to spend a lot of time writing unit files manually – which of course nobody does – you need to create a tool to automate some of that, which we did."</p>

@@ -12,7 +12,7 @@ weight: 60
 Application logs can help you understand what is happening inside your application. The logs are particularly useful for debugging problems and monitoring cluster activity. Most modern applications have some kind of logging mechanism. Likewise, container engines are designed to support logging. The easiest and most adopted logging method for containerized applications is writing to standard output and standard error streams.

 However, the native functionality provided by a container engine or runtime is usually not enough for a complete logging solution.
-For example, you may want access your application's logs if a container crashes; a pod gets evicted; or a node dies.
+For example, you may want to access your application's logs if a container crashes, a pod gets evicted, or a node dies.
 In a cluster, logs should have a separate storage and lifecycle independent of nodes, pods, or containers. This concept is called _cluster-level logging_.

 <!-- body -->
@@ -98,6 +98,24 @@ is local to a namespace.  This is useful for using the same configuration across
 multiple namespaces such as Development, Staging and Production.  If you want to reach
 across namespaces, you need to use the fully qualified domain name (FQDN).

+As a result, all namespace names must be valid
+[RFC 1123 DNS labels](/docs/concepts/overview/working-with-objects/names/#dns-label-names).
+
+{{< warning >}}
+By creating namespaces with the same name as [public top-level
+domains](https://data.iana.org/TLD/tlds-alpha-by-domain.txt), Services in these
+namespaces can have short DNS names that overlap with public DNS records.
+Workloads from any namespace performing a DNS lookup without a [trailing dot](https://datatracker.ietf.org/doc/html/rfc1034#page-8) will
+be redirected to those services, taking precedence over public DNS. 
+
+To mitigate this, limit privileges for creating namespaces to trusted users. If
+required, you could additionally configure third-party security controls, such
+as [admission
+webhooks](/docs/reference/access-authn-authz/extensible-admission-controllers/),
+to block creating any namespace with the name of [public
+TLDs](https://data.iana.org/TLD/tlds-alpha-by-domain.txt).
+{{< /warning >}}
+
 ## Not All Objects are in a Namespace

 Most Kubernetes resources (e.g. pods, services, replication controllers, and others) are
@@ -11,7 +11,8 @@ weight: 30

 {{< feature-state for_k8s_version="v1.21" state="deprecated" >}}

-PodSecurityPolicy is deprecated as of Kubernetes v1.21, and will be removed in v1.25. For more information on the deprecation,
+PodSecurityPolicy is deprecated as of Kubernetes v1.21, and will be removed in v1.25. It has been replaced by
+[Pod Security Admission](/docs/concepts/security/pod-security-admission/). For more information on the deprecation,
 see [PodSecurityPolicy Deprecation: Past, Present, and Future](/blog/2021/04/06/podsecuritypolicy-deprecation-past-present-and-future/).

 Pod Security Policies enable fine-grained authorization of pod creation and
@@ -305,34 +305,22 @@ fail validation.
 		<tr>
 			<td style="white-space: nowrap">Volume Types</td>
 			<td>
-				<p>In addition to restricting HostPath volumes, the restricted policy limits usage of non-core volume types to those defined through PersistentVolumes.</p>
+				<p>The restricted policy only permits the following volume types.</p>
 				<p><strong>Restricted Fields</strong></p>
 				<ul>
-					<li><code>spec.volumes[*].hostPath</code></li>
-					<li><code>spec.volumes[*].gcePersistentDisk</code></li>
-					<li><code>spec.volumes[*].awsElasticBlockStore</code></li>
-					<li><code>spec.volumes[*].gitRepo</code></li>
-					<li><code>spec.volumes[*].nfs</code></li>
-					<li><code>spec.volumes[*].iscsi</code></li>
-					<li><code>spec.volumes[*].glusterfs</code></li>
-					<li><code>spec.volumes[*].rbd</code></li>
-					<li><code>spec.volumes[*].flexVolume</code></li>
-					<li><code>spec.volumes[*].cinder</code></li>
-					<li><code>spec.volumes[*].cephfs</code></li>
-					<li><code>spec.volumes[*].flocker</code></li>
-					<li><code>spec.volumes[*].fc</code></li>
-					<li><code>spec.volumes[*].azureFile</code></li>
-					<li><code>spec.volumes[*].vsphereVolume</code></li>
-					<li><code>spec.volumes[*].quobyte</code></li>
-					<li><code>spec.volumes[*].azureDisk</code></li>
-					<li><code>spec.volumes[*].portworxVolume</code></li>
-					<li><code>spec.volumes[*].scaleIO</code></li>
-					<li><code>spec.volumes[*].storageos</code></li>
-					<li><code>spec.volumes[*].photonPersistentDisk</code></li>
+					<li><code>spec.volumes[*]</code></li>
 				</ul>
 				<p><strong>Allowed Values</strong></p>
+				Every item in the <code>spec.volumes[*]</code> list must set one of the following fields to a non-null value:
 				<ul>
-					<li>Undefined/nil</li>
+					<li><code>spec.volumes[*].configMap</code></li>
+					<li><code>spec.volumes[*].csi</code></li>
+					<li><code>spec.volumes[*].downwardAPI</code></li>
+					<li><code>spec.volumes[*].emptyDir</code></li>
+					<li><code>spec.volumes[*].ephemeral</code></li>
+					<li><code>spec.volumes[*].persistentVolumeClaim</code></li>
+					<li><code>spec.volumes[*].projected</code></li>
+					<li><code>spec.volumes[*].secret</code></li>
 				</ul>
 			</td>
 		</tr>
@@ -391,26 +379,6 @@ fail validation.
 				</ul>
 			</td>
 		</tr>
-		<tr>
-			<td style="white-space: nowrap">Non-root groups <em>(optional)</em></td>
-			<td>
-				<p>Containers should be forbidden from running with a root primary or supplementary GID.</p>
-				<p><strong>Restricted Fields</strong></p>
-				<ul>
-					<li><code>spec.securityContext.runAsGroup</code></li>
-					<li><code>spec.securityContext.supplementalGroups[*]</code></li>
-					<li><code>spec.securityContext.fsGroup</code></li>
-					<li><code>spec.containers[*].securityContext.runAsGroup</code></li>
-					<li><code>spec.initContainers[*].securityContext.runAsGroup</code></li>
-					<li><code>spec.ephemeralContainers[*].securityContext.runAsGroup</code></li>
-				</ul>
-				<p><strong>Allowed Values</strong></p>
-				<ul>
-					<li>Undefined/nil (except for <code>*.runAsGroup</code>)</li>
-					<li>Non-zero</li>
-				</ul>
-			</td>
-		</tr>
 		<tr>
  			<td style="white-space: nowrap">Seccomp (v1.19+)</td>
  			<td>
@@ -106,10 +106,9 @@ and the domain name for your cluster is `cluster.local`, then the Pod has a DNS

 `172-17-0-3.default.pod.cluster.local`.

-Any pods created by a Deployment or DaemonSet exposed by a Service have the
-following DNS resolution available:
+Any pods exposed by a Service have the following DNS resolution available:

-`pod-ip-address.deployment-name.my-namespace.svc.cluster-domain.example`.
+`pod-ip-address.service-name.my-namespace.svc.cluster-domain.example`.

 ### Pod's hostname and subdomain fields

@@ -88,6 +88,15 @@ has all the information needed to configure a load balancer or proxy server. Mos
 contains a list of rules matched against all incoming requests. Ingress resource only supports rules
 for directing HTTP(S) traffic.

+If the `ingressClassName` is omitted, a [default Ingress class](#default-ingress-class) 
+should be defined.
+
+There are some ingress controllers, that work without the definition of a 
+default `IngressClass`. For example, the Ingress-NGINX controller can be 
+configured with a [flag](https://kubernetes.github.io/ingress-nginx/#what-is-the-flag-watch-ingress-without-class) 
+`--watch-ingress-without-class`. It is [recommended](https://kubernetes.github.io/ingress-nginx/#i-have-only-one-instance-of-the-ingresss-nginx-controller-in-my-cluster-what-should-i-do)  though, to specify the 
+default `IngressClass` as shown [below](#default-ingress-class).
+
 ### Ingress rules

 Each HTTP rule contains the following information:
@@ -339,6 +348,14 @@ an `ingressClassName` specified. You can resolve this by ensuring that at most 1
 IngressClass is marked as default in your cluster.
 {{< /caution >}}

+There are some ingress controllers, that work without the definition of a
+default `IngressClass`. For example, the Ingress-NGINX controller can be
+configured with a [flag](https://kubernetes.github.io/ingress-nginx/#what-is-the-flag-watch-ingress-without-class)
+`--watch-ingress-without-class`. It is [recommended](https://kubernetes.github.io/ingress-nginx/#i-have-only-one-instance-of-the-ingresss-nginx-controller-in-my-cluster-what-should-i-do)  though, to specify the
+default `IngressClass`:
+
+{{< codenew file="service/networking/default-ingressclass.yaml" >}}
+
 ## Types of Ingress

 ### Ingress backed by a single Service {#single-service-ingress}
@@ -468,9 +485,7 @@ web traffic to the IP address of your Ingress controller can be matched without
 virtual host being required.

 For example, the following Ingress routes traffic
-requested for `first.bar.com` to `service1`, `second.bar.com` to `service2`, and any traffic
-to the IP address without a hostname defined in request (that is, without a request header being
-presented) to `service3`.
+requested for `first.bar.com` to `service1`, `second.bar.com` to `service2`, and any traffic whose request host header doesn't match `first.bar.com` and `second.bar.com` to `service3`.

 {{< codenew file="service/networking/name-virtual-host-ingress-no-third-host.yaml" >}}

@@ -422,7 +422,7 @@ Helper programs relating to the volume type may be required for consumption of a

 ### Capacity

-Generally, a PV will have a specific storage capacity.  This is set using the PV's `capacity` attribute.  See the Kubernetes [Resource Model](https://git.k8s.io/community/contributors/design-proposals/scheduling/resources.md) to understand the units expected by `capacity`.
+Generally, a PV will have a specific storage capacity.  This is set using the PV's `capacity` attribute.  Read the glossary term [Quantity](/docs/reference/glossary/?all=true#term-quantity) to understand the units expected by `capacity`.

 Currently, storage size is the only resource that can be set or requested.  Future attributes may include IOPS, throughput, etc.

@@ -535,19 +535,19 @@ Not all Persistent Volume types support mount options.

 The following volume types support mount options:

-* AWSElasticBlockStore
-* AzureDisk
-* AzureFile
-* CephFS
-* Cinder (OpenStack block storage)
-* GCEPersistentDisk
-* Glusterfs
-* NFS
-* Quobyte Volumes
-* RBD (Ceph Block Device)
-* StorageOS
-* VsphereVolume
-* iSCSI
+* `awsElasticBlockStore`
+* `azureDisk`
+* `azureFile`
+* `cephfs`
+* `cinder` (**deprecated** in v1.18)
+* `gcePersistentDisk`
+* `glusterfs`
+* `iscsi`
+* `nfs`
+* `quobyte` (**deprecated** in v1.22)
+* `rbd`
+* `storageos` (**deprecated** in v1.22)
+* `vsphereVolume`

 Mount options are not validated. If a mount option is invalid, the mount fails.

@@ -26,7 +26,7 @@ Currently, the following types of volume sources can be projected:
 * `serviceAccountToken`

 All sources are required to be in the same namespace as the Pod. For more details,
-see the [all-in-one volume design document](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/node/all-in-one-volume.md).
+see the [all-in-one volume](https://github.com/kubernetes/design-proposals-archive/blob/main/node/all-in-one-volume.md) design document.

 ### Example configuration with a secret, a downwardAPI, and a configMap {#example-configuration-secret-downwardapi-configmap}

@@ -71,9 +71,7 @@ volume mount will not receive updates for those volume sources.

 ## SecurityContext interactions

-The [proposal for file permission handling in projected service account volume](https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/2451-service-account-token-volumes#token-volume-projection)
-enhancement introduced the projected files having the the correct owner
-permissions set.
+The [proposal](https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/2451-service-account-token-volumes#proposal) for file permission handling in projected service account volume enhancement introduced the projected files having the the correct owner permissions set.

 ### Linux

@@ -842,6 +842,13 @@ Kubernetes marks a Deployment as _progressing_ when one of the following tasks i
 * The Deployment is scaling down its older ReplicaSet(s).
 * New Pods become ready or available (ready for at least [MinReadySeconds](#min-ready-seconds)).

+When the rollout becomes “progressing”, the Deployment controller adds a condition with the following
+attributes to the Deployment's `.status.conditions`:
+
+* `type: Progressing`
+* `status: "True"`
+* `reason: NewReplicaSetCreated` | `reason: FoundNewReplicaSet` | `reason: ReplicaSetUpdated`
+
 You can monitor the progress for a Deployment by using `kubectl rollout status`.

 ### Complete Deployment
@@ -853,6 +860,17 @@ updates you've requested have been completed.
 * All of the replicas associated with the Deployment are available.
 * No old replicas for the Deployment are running.

+When the rollout becomes “complete”, the Deployment controller sets a condition with the following
+attributes to the Deployment's `.status.conditions`:
+
+* `type: Progressing`
+* `status: "True"`
+* `reason: NewReplicaSetAvailable`
+
+This `Progressing` condition will retain a status value of `"True"` until a new rollout
+is initiated. The condition holds even when availability of replicas changes (which
+does instead affect the `Available` condition).
+
 You can check if a Deployment has completed by using `kubectl rollout status`. If the rollout completed
 successfully, `kubectl rollout status` returns a zero exit code.

@@ -890,7 +908,7 @@ number of seconds the Deployment controller waits before indicating (in the Depl
 Deployment progress has stalled.

 The following `kubectl` command sets the spec with `progressDeadlineSeconds` to make the controller report
-lack of progress for a Deployment after 10 minutes:
+lack of progress of a rollout for a Deployment after 10 minutes:

 ```shell
 kubectl patch deployment/nginx-deployment -p '{"spec":{"progressDeadlineSeconds":600}}'
@@ -902,15 +920,18 @@ deployment.apps/nginx-deployment patched
 Once the deadline has been exceeded, the Deployment controller adds a DeploymentCondition with the following
 attributes to the Deployment's `.status.conditions`:

-* Type=Progressing
-* Status=False
-* Reason=ProgressDeadlineExceeded
+* `type: Progressing`
+* `status: "False"`
+* `reason: ProgressDeadlineExceeded`
+
+This condition can also fail early and is then set to status value of `"False"` due to reasons as `ReplicaSetCreateError`.
+Also, the deadline is not taken into account anymore once the Deployment rollout completes.

 See the [Kubernetes API conventions](https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#typical-status-properties) for more information on status conditions.

 {{< note >}}
 Kubernetes takes no action on a stalled Deployment other than to report a status condition with
-`Reason=ProgressDeadlineExceeded`. Higher level orchestrators can take advantage of it and act accordingly, for
+`reason: ProgressDeadlineExceeded`. Higher level orchestrators can take advantage of it and act accordingly, for
 example, rollback the Deployment to its previous version.
 {{< /note >}}

@@ -984,7 +1005,7 @@ Conditions:
 You can address an issue of insufficient quota by scaling down your Deployment, by scaling down other
 controllers you may be running, or by increasing quota in your namespace. If you satisfy the quota
 conditions and the Deployment controller then completes the Deployment rollout, you'll see the
-Deployment's status update with a successful condition (`Status=True` and `Reason=NewReplicaSetAvailable`).
+Deployment's status update with a successful condition (`status: "True"` and `reason: NewReplicaSetAvailable`).

 ```
 Conditions:
@@ -994,11 +1015,11 @@ Conditions:
  Progressing   True    NewReplicaSetAvailable
 ```

-`Type=Available` with `Status=True` means that your Deployment has minimum availability. Minimum availability is dictated
-by the parameters specified in the deployment strategy. `Type=Progressing` with `Status=True` means that your Deployment
+`type: Available` with `status: "True"` means that your Deployment has minimum availability. Minimum availability is dictated
+by the parameters specified in the deployment strategy. `type: Progressing` with `status: "True"` means that your Deployment
 is either in the middle of a rollout and it is progressing or that it has successfully completed its progress and the minimum
 required new replicas are available (see the Reason of the condition for the particulars - in our case
-`Reason=NewReplicaSetAvailable` means that the Deployment is complete).
+`reason: NewReplicaSetAvailable` means that the Deployment is complete).

 You can check if a Deployment has failed to progress by using `kubectl rollout status`. `kubectl rollout status`
 returns a non-zero exit code if the Deployment has exceeded the progression deadline.
@@ -1155,8 +1176,8 @@ total number of Pods running at any time during the update is at most 130% of de

 `.spec.progressDeadlineSeconds` is an optional field that specifies the number of seconds you want
 to wait for your Deployment to progress before the system reports back that the Deployment has
-[failed progressing](#failed-deployment) - surfaced as a condition with `Type=Progressing`, `Status=False`.
-and `Reason=ProgressDeadlineExceeded` in the status of the resource. The Deployment controller will keep
+[failed progressing](#failed-deployment) - surfaced as a condition with `type: Progressing`, `status: "False"`.
+and `reason: ProgressDeadlineExceeded` in the status of the resource. The Deployment controller will keep
 retrying the Deployment. This defaults to 600. In the future, once automatic rollback will be implemented, the Deployment
 controller will roll back a Deployment as soon as it observes such a condition.

@@ -313,7 +313,7 @@ ensures that a desired number of Pods with a matching label selector are availab
 When scaling down, the ReplicaSet controller chooses which pods to delete by sorting the available pods to
 prioritize scaling down pods based on the following general algorithm:
 1. Pending (and unschedulable) pods are scaled down first
- 2. If controller.kubernetes.io/pod-deletion-cost annotation is set, then
+ 2. If `controller.kubernetes.io/pod-deletion-cost` annotation is set, then
    the pod with the lower value will come first.
 3. Pods on nodes with more replicas come before pods on nodes with fewer replicas.
 4. If the pods' creation times differ, the pod that was created more recently
@@ -562,7 +562,10 @@ Each feature gate is designed for enabling/disabling a specific feature:
 - `APIResponseCompression`: Compress the API responses for `LIST` or `GET` requests.
 - `APIServerIdentity`: Assign each API server an ID in a cluster.
 - `APIServerTracing`: Add support for distributed tracing in the API server.
- `Accelerators`: Enable Nvidia GPU support when using Docker
+- `Accelerators`: Provided an early form of plugin to enable Nvidia GPU support when using
+  Docker Engine; no longer available. See
+  [Device Plugins](/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/) for
+  an alternative.
 - `AdvancedAuditing`: Enable [advanced auditing](/docs/tasks/debug-application-cluster/audit/#advanced-audit)
 - `AffinityInAnnotations`: Enable setting
  [Pod affinity or anti-affinity](/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity).
@@ -571,7 +574,7 @@ Each feature gate is designed for enabling/disabling a specific feature:
  kubelets on Pod log requests.
 - `AnyVolumeDataSource`: Enable use of any custom resource as the `DataSource` of a
  {{< glossary_tooltip text="PVC" term_id="persistent-volume-claim" >}}.
- `AppArmor`: Enable AppArmor based mandatory access control on Linux nodes when using Docker.
+- `AppArmor`: Enable use of AppArmor mandatory access control for Pods running on Linux nodes.
  See [AppArmor Tutorial](/docs/tutorials/clusters/apparmor/) for more details.
 - `AttachVolumeLimit`: Enable volume plugins to report limits on number of volumes
  that can be attached to a node.
@@ -0,0 +1,25 @@
+---
+title: Event
+id: event
+date: 2022-01-16
+full_link: /docs/reference/kubernetes-api/cluster-resources/event-v1/
+short_description: >
+   A report of an event somewhere in the cluster. It generally denotes some state change in the system.
+aka: 
+tags:
+- core-object
+- fundamental
+---
+Each Event is a report of an event somewhere in the {{< glossary_tooltip text="cluster" term_id="cluster" >}}. 
+It generally denotes some state change in the system.
+
+<!--more-->
+Events have a limited retention time and triggers and messages may evolve with time. 
+Event consumers should not rely on the timing of an event with a given reason reflecting a consistent underlying trigger, 
+or the continued existence of events with that reason. 
+
+
+Events should be treated as informative, best-effort, supplemental data.
+
+In Kubernetes, [auditing](/docs/tasks/debug-application-cluster/audit/) generates a different kind of
+Event record (API group `audit.k8s.io`).
@@ -159,6 +159,20 @@ The scheduler (through the _VolumeZonePredicate_ predicate) also will ensure tha
 If `PersistentVolumeLabel` does not support automatic labeling of your PersistentVolumes, you should consider
 adding the labels manually (or adding support for `PersistentVolumeLabel`). With `PersistentVolumeLabel`, the scheduler prevents Pods from mounting volumes in a different zone. If your infrastructure doesn't have this constraint, you don't need to add the zone labels to the volumes at all.

+## volume.beta.kubernetes.io/storage-provisioner (deprecated)
+
+Example: `volume.beta.kubernetes.io/storage-provisioner: k8s.io/minikube-hostpath`
+
+Used on: PersistentVolumeClaim
+
+This annotation has been deprecated.
+
+## volume.kubernetes.io/storage-provisioner
+
+Used on: PersistentVolumeClaim
+
+This annotation will be added to dynamic provisioning required PVC.
+
 ## node.kubernetes.io/windows-build {#nodekubernetesiowindows-build}

 Example: `node.kubernetes.io/windows-build=10.0.17763`
@@ -73,21 +73,23 @@ knows how to convert between them in both directions.  Additionally, any new
 field added in v2 must be able to round-trip to v1 and back, which means v1
 might have to add an equivalent field or represent it as an annotation.

-**Rule #3: An API version in a given track may not be deprecated until a new
-API version at least as stable is released.**
+**Rule #3: An API version in a given track may not be deprecated in favor of a less stable API version.**

-GA API versions can replace GA API versions as well as beta and alpha API
-versions.  Beta API versions *may not* replace GA API versions.
+  * GA API versions can replace beta and alpha API versions.
+  * Beta API versions can replace earlier beta and alpha API versions, but *may not* replace GA API versions.
+  * Alpha API versions can replace earlier alpha API versions, but *may not* replace GA or beta API versions.

-**Rule #4a: Other than the most recent API versions in each track, older API
-versions must be supported after their announced deprecation for a duration of
-no less than:**
+**Rule #4a: minimum API lifetime is determined by the API stability level**

-   * **GA: 12 months or 3 releases (whichever is longer)**
-   * **Beta: 9 months or 3 releases (whichever is longer)**
-   * **Alpha: 0 releases**
+   * **GA API versions may be marked as deprecated, but must not be removed within a major version of Kubernetes**
+   * **Beta API versions must be supported for 9 months or 3 releases (whichever is longer) after deprecation**
+   * **Alpha API versions may be removed in any release without prior deprecation notice**

-This covers the [maximum supported version skew of 2 releases](/docs/setup/release/version-skew-policy/).
+This ensures beta API support covers the [maximum supported version skew of 2 releases](/docs/setup/release/version-skew-policy/).
+
+{{< note >}}
+There are no current plans for a major version revision of Kubernetes that removes GA APIs.
+{{< /note >}}

 {{< note >}}
 Until [#52185](https://github.com/kubernetes/kubernetes/issues/52185) is
@@ -237,7 +239,7 @@ API versions are supported in a series of subsequent releases.
      <td>
        <ul>
          <li>v2beta2 is deprecated, "action required" relnote</li>
-          <li>v1 is deprecated, "action required" relnote</li>
+          <li>v1 is deprecated in favor of v2, but will not be removed</li>
        </ul>
      </td>
    </tr>
@@ -267,22 +269,6 @@ API versions are supported in a series of subsequent releases.
        </ul>
      </td>
    </tr>
-    <tr>
-      <td>X+16</td>
-      <td>v2, v1 (deprecated)</td>
-      <td>v2</td>
-      <td></td>
-    </tr>
-    <tr>
-      <td>X+17</td>
-      <td>v2</td>
-      <td>v2</td>
-      <td>
-        <ul>
-          <li>v1 is removed, "action required" relnote</li>
-        </ul>
-      </td>
-    </tr>
  </tbody>
 </table>

@@ -2,14 +2,18 @@
 title: Configure Minimum and Maximum CPU Constraints for a Namespace
 content_type: task
 weight: 40
+description: >-
+  Define a range of valid CPU resource limits for a namespace, so that every new Pod
+  in that namespace falls within the range you configure.
 ---


 <!-- overview -->

-This page shows how to set minimum and maximum values for the CPU resources used by Containers
-and Pods in a namespace. You specify minimum and maximum CPU values in a
-[LimitRange](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#limitrange-v1-core)
+This page shows how to set minimum and maximum values for the CPU resources used by containers
+and Pods in a {{< glossary_tooltip text="namespace" term_id="namespace" >}}. You specify minimum
+and maximum CPU values in a
+[LimitRange](/docs/reference/kubernetes-api/policy-resources/limit-range-v1/)
 object. If a Pod does not meet the constraints imposed by the LimitRange, it cannot be created
 in the namespace.

@@ -19,11 +23,13 @@ in the namespace.
 ## {{% heading "prerequisites" %}}


-{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
-
-Your cluster must have at least 1 CPU available for use to run the task examples.
+{{< include "task-tutorial-prereqs.md" >}}

+You must have access to create namespaces in your cluster.

+Your cluster must have at least 1.0 CPU available for use to run the task examples.
+See [meaning of CPU](/docs/concepts/configuration/manage-resources-containers/#meaning-of-cpu)
+to learn what Kubernetes means by “1 CPU”.


 <!-- steps -->
@@ -39,7 +45,7 @@ kubectl create namespace constraints-cpu-example

 ## Create a LimitRange and a Pod

-Here's the configuration file for a LimitRange:
+Here's an example manifest for a LimitRange:

 {{< codenew file="admin/resource/cpu-constraints.yaml" >}}

@@ -72,15 +78,15 @@ limits:
  type: Container
 ```

-Now whenever a Container is created in the constraints-cpu-example namespace, Kubernetes
-performs these steps:
+Now whenever you create a Pod in the constraints-cpu-example namespace (or some other client
+of the Kubernetes API creates an equivalent Pod), Kubernetes performs these steps:

-* If the Container does not specify its own CPU request and limit, assign the default
-CPU request and limit to the Container.
+* If any container in that Pod does not specify its own CPU request and limit, the control plane
+  assigns the default CPU request and limit to that container.

-* Verify that the Container specifies a CPU request that is greater than or equal to 200 millicpu.
+* Verify that every container in that Pod specifies a CPU request that is greater than or equal to 200 millicpu.

-* Verify that the Container specifies a CPU limit that is less than or equal to 800 millicpu.
+* Verify that every container in that Pod specifies a CPU limit that is less than or equal to 800 millicpu.

 {{< note >}}
 When creating a `LimitRange` object, you can specify limits on huge-pages
@@ -88,7 +94,7 @@ or GPUs as well. However, when both `default` and `defaultRequest` are specified
 on these resources, the two values must be the same.
 {{< /note >}}

-Here's the configuration file for a Pod that has one Container. The Container manifest
+Here's a manifest for a Pod that has one container. The container manifest
 specifies a CPU request of 500 millicpu and a CPU limit of 800 millicpu. These satisfy the
 minimum and maximum CPU constraints imposed by the LimitRange.

@@ -100,7 +106,7 @@ Create the Pod:
 kubectl apply -f https://k8s.io/examples/admin/resource/cpu-constraints-pod.yaml --namespace=constraints-cpu-example
 ```

-Verify that the Pod's Container is running:
+Verify that the Pod is running and that its container is healthy:

 ```shell
 kubectl get pod constraints-cpu-demo --namespace=constraints-cpu-example
@@ -112,7 +118,7 @@ View detailed information about the Pod:
 kubectl get pod constraints-cpu-demo --output=yaml --namespace=constraints-cpu-example
 ```

-The output shows that the Container has a CPU request of 500 millicpu and CPU limit
+The output shows that the Pod's only container has a CPU request of 500 millicpu and CPU limit
 of 800 millicpu. These satisfy the constraints imposed by the LimitRange.

 ```yaml
@@ -131,7 +137,7 @@ kubectl delete pod constraints-cpu-demo --namespace=constraints-cpu-example

 ## Attempt to create a Pod that exceeds the maximum CPU constraint

-Here's the configuration file for a Pod that has one Container. The Container specifies a
+Here's a manifest for a Pod that has one container. The container specifies a
 CPU request of 500 millicpu and a cpu limit of 1.5 cpu.

 {{< codenew file="admin/resource/cpu-constraints-pod-2.yaml" >}}
@@ -142,8 +148,8 @@ Attempt to create the Pod:
 kubectl apply -f https://k8s.io/examples/admin/resource/cpu-constraints-pod-2.yaml --namespace=constraints-cpu-example
 ```

-The output shows that the Pod does not get created, because the Container specifies a CPU limit that is
-too large:
+The output shows that the Pod does not get created, because it defines an unacceptable container.
+That container is not acceptable because it specifies a CPU limit that is too large:

 ```
 Error from server (Forbidden): error when creating "examples/admin/resource/cpu-constraints-pod-2.yaml":
@@ -152,7 +158,7 @@ pods "constraints-cpu-demo-2" is forbidden: maximum cpu usage per Container is 8

 ## Attempt to create a Pod that does not meet the minimum CPU request

-Here's the configuration file for a Pod that has one Container. The Container specifies a
+Here's a manifest for a Pod that has one container. The container specifies a
 CPU request of 100 millicpu and a CPU limit of 800 millicpu.

 {{< codenew file="admin/resource/cpu-constraints-pod-3.yaml" >}}
@@ -163,8 +169,9 @@ Attempt to create the Pod:
 kubectl apply -f https://k8s.io/examples/admin/resource/cpu-constraints-pod-3.yaml --namespace=constraints-cpu-example
 ```

-The output shows that the Pod does not get created, because the Container specifies a CPU
-request that is too small:
+The output shows that the Pod does not get created, because it defines an unacceptable container.
+That container is not acceptable because it specifies a CPU limit that is lower than the
+enforced minimum:

 ```
 Error from server (Forbidden): error when creating "examples/admin/resource/cpu-constraints-pod-3.yaml":
@@ -173,8 +180,8 @@ pods "constraints-cpu-demo-3" is forbidden: minimum cpu usage per Container is 2

 ## Create a Pod that does not specify any CPU request or limit

-Here's the configuration file for a Pod that has one Container. The Container does not
-specify a CPU request, and it does not specify a CPU limit.
+Here's a manifest for a Pod that has one container. The container does not
+specify a CPU request, nor does it specify a CPU limit.

 {{< codenew file="admin/resource/cpu-constraints-pod-4.yaml" >}}

@@ -190,8 +197,9 @@ View detailed information about the Pod:
 kubectl get pod constraints-cpu-demo-4 --namespace=constraints-cpu-example --output=yaml
 ```

-The output shows that the Pod's Container has a CPU request of 800 millicpu and a CPU limit of 800 millicpu.
-How did the Container get those values?
+The output shows that the Pod's single container has a CPU request of 800 millicpu and a
+CPU limit of 800 millicpu.
+How did that container get those values?

 ```yaml
 resources:
@@ -201,11 +209,12 @@ resources:
    cpu: 800m
 ```

-Because your Container did not specify its own CPU request and limit, it was given the
+Because that container did not specify its own CPU request and limit, the control plane
+applied the
 [default CPU request and limit](/docs/tasks/administer-cluster/manage-resources/cpu-default-namespace/)
-from the LimitRange.
+from the LimitRange for this namespace.

-At this point, your Container might be running or it might not be running. Recall that a prerequisite for this task is that your cluster must have at least 1 CPU available for use. If each of your Nodes has only 1 CPU, then there might not be enough allocatable CPU on any Node to accommodate a request of 800 millicpu. If you happen to be using Nodes with 2 CPU, then you probably have enough CPU to accommodate the 800 millicpu request.
+At this point, your Pod might be running or it might not be running. Recall that a prerequisite for this task is that your cluster must have at least 1 CPU available for use. If each of your Nodes has only 1 CPU, then there might not be enough allocatable CPU on any Node to accommodate a request of 800 millicpu. If you happen to be using Nodes with 2 CPU, then you probably have enough CPU to accommodate the 800 millicpu request.

 Delete your Pod:

@@ -2,23 +2,36 @@
 title: Configure Default CPU Requests and Limits for a Namespace
 content_type: task
 weight: 20
+description: >-
+  Define a default CPU resource limits for a namespace, so that every new Pod
+  in that namespace has a CPU resource limit configured.
 ---

 <!-- overview -->

-This page shows how to configure default CPU requests and limits for a namespace.
-A Kubernetes cluster can be divided into namespaces. If a Container is created in a namespace
-that has a default CPU limit, and the Container does not specify its own CPU limit, then
-the Container is assigned the default CPU limit. Kubernetes assigns a default CPU request
-under certain conditions that are explained later in this topic.
+This page shows how to configure default CPU requests and limits for a
+{{< glossary_tooltip text="namespace" term_id="namespace" >}}.

+A Kubernetes cluster can be divided into namespaces. If you create a Pod within a
+namespace that has a default CPU
+[limit](/docs/concepts/configuration/manage-resources-containers/#requests-and-limits), and any container in that Pod does not specify
+its own CPU limit, then the
+{{< glossary_tooltip text="control plane" term_id="control-plane" >}} assigns the default
+CPU limit to that container.

+Kubernetes assigns a default CPU
+[request](/docs/concepts/configuration/manage-resources-containers/#requests-and-limits),
+but only under certain conditions that are explained later in this page.

 ## {{% heading "prerequisites" %}}


-{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
+{{< include "task-tutorial-prereqs.md" >}}

+You must have access to create namespaces in your cluster.
+
+If you're not already familiar with what Kubernetes means by 1.0 CPU,
+read [meaning of CPU](/docs/concepts/configuration/manage-resources-containers/#meaning-of-cpu).

 <!-- steps -->

@@ -33,8 +46,8 @@ kubectl create namespace default-cpu-example

 ## Create a LimitRange and a Pod

-Here's the configuration file for a LimitRange object. The configuration specifies
-a default CPU request and a default CPU limit.
+Here's a manifest for an example {{< glossary_tooltip text="LimitRange" term_id="limitrange" >}}.
+The manifest specifies a default CPU request and a default CPU limit.

 {{< codenew file="admin/resource/cpu-defaults.yaml" >}}

@@ -44,12 +57,12 @@ Create the LimitRange in the default-cpu-example namespace:
 kubectl apply -f https://k8s.io/examples/admin/resource/cpu-defaults.yaml --namespace=default-cpu-example
 ```

-Now if a Container is created in the default-cpu-example namespace, and the
-Container does not specify its own values for CPU request and CPU limit,
-the Container is given a default CPU request of 0.5 and a default
+Now if you create a Pod in the default-cpu-example namespace, and any container
+in that Pod does not specify its own values for CPU request and CPU limit,
+then the control plane applies default values: a CPU request of 0.5 and a default
 CPU limit of 1.

-Here's the configuration file for a Pod that has one Container. The Container
+Here's a manifest for a Pod that has one container. The container
 does not specify a CPU request and limit.

 {{< codenew file="admin/resource/cpu-defaults-pod.yaml" >}}
@@ -66,8 +79,9 @@ View the Pod's specification:
 kubectl get pod default-cpu-demo --output=yaml --namespace=default-cpu-example
 ```

-The output shows that the Pod's Container has a CPU request of 500 millicpus and
-a CPU limit of 1 cpu. These are the default values specified by the LimitRange.
+The output shows that the Pod's only container has a CPU request of 500m `cpu`
+(which you can read as “500 millicpu”), and a CPU limit of 1 `cpu`.
+These are the default values specified by the LimitRange.

 ```shell
 containers:
@@ -81,9 +95,9 @@ containers:
      cpu: 500m
 ```

-## What if you specify a Container's limit, but not its request?
+## What if you specify a container's limit, but not its request?

-Here's the configuration file for a Pod that has one Container. The Container
+Here's a manifest for a Pod that has one container. The container
 specifies a CPU limit, but not a request:

 {{< codenew file="admin/resource/cpu-defaults-pod-2.yaml" >}}
@@ -95,14 +109,15 @@ Create the Pod:
 kubectl apply -f https://k8s.io/examples/admin/resource/cpu-defaults-pod-2.yaml --namespace=default-cpu-example
 ```

-View the Pod specification:
+View the [specification](/docs/concepts/overview/working-with-objects/kubernetes-objects/#object-spec-and-status)
+of the Pod that you created:

 ```
 kubectl get pod default-cpu-demo-2 --output=yaml --namespace=default-cpu-example
 ```

-The output shows that the Container's CPU request is set to match its CPU limit.
-Notice that the Container was not assigned the default CPU request value of 0.5 cpu.
+The output shows that the container's CPU request is set to match its CPU limit.
+Notice that the container was not assigned the default CPU request value of 0.5 `cpu`:

 ```
 resources:
@@ -112,9 +127,9 @@ resources:
    cpu: "1"
 ```

-## What if you specify a Container's request, but not its limit?
+## What if you specify a container's request, but not its limit?

-Here's the configuration file for a Pod that has one Container. The Container
+Here's an example manifest for a Pod that has one container. The container
 specifies a CPU request, but not a limit:

 {{< codenew file="admin/resource/cpu-defaults-pod-3.yaml" >}}
@@ -125,15 +140,16 @@ Create the Pod:
 kubectl apply -f https://k8s.io/examples/admin/resource/cpu-defaults-pod-3.yaml --namespace=default-cpu-example
 ```

-View the Pod specification:
+View the specification of the Pod that you created:

 ```
 kubectl get pod default-cpu-demo-3 --output=yaml --namespace=default-cpu-example
 ```

-The output shows that the Container's CPU request is set to the value specified in the
-Container's configuration file. The Container's CPU limit is set to 1 cpu, which is the
-default CPU limit for the namespace.
+The output shows that the container's CPU request is set to the value you specified at
+the time you created the Pod (in other words: it matches the manifest).
+However, the same container's CPU limit is set to 1 `cpu`, which is the default CPU limit
+for that namespace.

 ```
 resources:
@@ -145,16 +161,22 @@ resources:

 ## Motivation for default CPU limits and requests

-If your namespace has a
-[resource quota](/docs/tasks/administer-cluster/manage-resources/quota-memory-cpu-namespace/),
+If your namespace has a CPU {{< glossary_tooltip text="resource quota" term_id="resource-quota" >}}
+configured,
 it is helpful to have a default value in place for CPU limit.
-Here are two of the restrictions that a resource quota imposes on a namespace:
+Here are two of the restrictions that a CPU resource quota imposes on a namespace:

-* Every Container that runs in the namespace must have its own CPU limit.
-* The total amount of CPU used by all Containers in the namespace must not exceed a specified limit.
+* For every Pod that runs in the namespace, each of its containers must have a CPU limit.
+* CPU limits apply a resource reservation on the node where the Pod in question is scheduled.
+  The total amount of CPU that is reserved for use by all Pods in the namespace must not
+  exceed a specified limit.
+
+When you add a LimitRange:
+
+If any Pod in that namespace that includes a container does not specify its own CPU limit,
+the control plane applies the default CPU limit to that container, and the Pod can be
+allowed to run in a namespace that is restricted by a CPU ResourceQuota.

-If a Container does not specify its own CPU limit, it is given the default limit, and then
-it can be allowed to run in a namespace that is restricted by a quota.

 ## Clean up

@@ -2,12 +2,15 @@
 title: Configure Minimum and Maximum Memory Constraints for a Namespace
 content_type: task
 weight: 30
+description: >-
+  Define a range of valid memory resource limits for a namespace, so that every new Pod
+  in that namespace falls within the range you configure.
 ---


 <!-- overview -->

-This page shows how to set minimum and maximum values for memory used by Containers
+This page shows how to set minimum and maximum values for memory used by containers
 running in a namespace. You specify minimum and maximum memory values in a
 [LimitRange](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#limitrange-v1-core)
 object. If a Pod does not meet the constraints imposed by the LimitRange,
@@ -15,16 +18,14 @@ it cannot be created in the namespace.



-
 ## {{% heading "prerequisites" %}}


-{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
-
-Each node in your cluster must have at least 1 GiB of memory.
-
+{{< include "task-tutorial-prereqs.md" >}}

+You must have access to create namespaces in your cluster.

+Each node in your cluster must have at least 1 GiB of memory available for Pods.

 <!-- steps -->

@@ -39,7 +40,7 @@ kubectl create namespace constraints-mem-example

 ## Create a LimitRange and a Pod

-Here's the configuration file for a LimitRange:
+Here's an example manifest for a LimitRange:

 {{< codenew file="admin/resource/memory-constraints.yaml" >}}

@@ -72,18 +73,19 @@ file for the LimitRange, they were created automatically.
    type: Container
 ```

-Now whenever a Container is created in the constraints-mem-example namespace, Kubernetes
+Now whenever you define a Pod within the constraints-mem-example namespace, Kubernetes
 performs these steps:

-* If the Container does not specify its own memory request and limit, assign the default
-memory request and limit to the Container.
+* If any container in that Pod does not specify its own memory request and limit, assign
+the default memory request and limit to that container.

-* Verify that the Container has a memory request that is greater than or equal to 500 MiB.
+* Verify that every container in that Pod requests at least 500 MiB of memory.

-* Verify that the Container has a memory limit that is less than or equal to 1 GiB.
+* Verify that every container in that Pod requests no more than 1024 MiB (1 GiB)
+  of memory.

-Here's the configuration file for a Pod that has one Container. The Container manifest
-specifies a memory request of 600 MiB and a memory limit of 800 MiB. These satisfy the
+Here's a manifest for a Pod that has one container. Within the Pod spec, the sole
+container specifies a memory request of 600 MiB and a memory limit of 800 MiB. These satisfy the
 minimum and maximum memory constraints imposed by the LimitRange.

 {{< codenew file="admin/resource/memory-constraints-pod.yaml" >}}
@@ -94,7 +96,7 @@ Create the Pod:
 kubectl apply -f https://k8s.io/examples/admin/resource/memory-constraints-pod.yaml --namespace=constraints-mem-example
 ```

-Verify that the Pod's Container is running:
+Verify that the Pod is running and that its container is healthy:

 ```shell
 kubectl get pod constraints-mem-demo --namespace=constraints-mem-example
@@ -106,8 +108,9 @@ View detailed information about the Pod:
 kubectl get pod constraints-mem-demo --output=yaml --namespace=constraints-mem-example
 ```

-The output shows that the Container has a memory request of 600 MiB and a memory limit
-of 800 MiB. These satisfy the constraints imposed by the LimitRange.
+The output shows that the container within that Pod has a memory request of 600 MiB and
+a memory limit of 800 MiB. These satisfy the constraints imposed by the LimitRange for
+this namespace:

 ```yaml
 resources:
@@ -125,7 +128,7 @@ kubectl delete pod constraints-mem-demo --namespace=constraints-mem-example

 ## Attempt to create a Pod that exceeds the maximum memory constraint

-Here's the configuration file for a Pod that has one Container. The Container specifies a
+Here's a manifest for a Pod that has one container. The container specifies a
 memory request of 800 MiB and a memory limit of 1.5 GiB.

 {{< codenew file="admin/resource/memory-constraints-pod-2.yaml" >}}
@@ -136,8 +139,8 @@ Attempt to create the Pod:
 kubectl apply -f https://k8s.io/examples/admin/resource/memory-constraints-pod-2.yaml --namespace=constraints-mem-example
 ```

-The output shows that the Pod does not get created, because the Container specifies a memory limit that is
-too large:
+The output shows that the Pod does not get created, because it defines a container that
+requests more memory than is allowed:

 ```
 Error from server (Forbidden): error when creating "examples/admin/resource/memory-constraints-pod-2.yaml":
@@ -146,7 +149,7 @@ pods "constraints-mem-demo-2" is forbidden: maximum memory usage per Container i

 ## Attempt to create a Pod that does not meet the minimum memory request

-Here's the configuration file for a Pod that has one Container. The Container specifies a
+Here's a manifest for a Pod that has one container. That container specifies a
 memory request of 100 MiB and a memory limit of 800 MiB.

 {{< codenew file="admin/resource/memory-constraints-pod-3.yaml" >}}
@@ -157,8 +160,8 @@ Attempt to create the Pod:
 kubectl apply -f https://k8s.io/examples/admin/resource/memory-constraints-pod-3.yaml --namespace=constraints-mem-example
 ```

-The output shows that the Pod does not get created, because the Container specifies a memory
-request that is too small:
+The output shows that the Pod does not get created, because it defines a container
+that requests less memory than the enforced minimum:

 ```
 Error from server (Forbidden): error when creating "examples/admin/resource/memory-constraints-pod-3.yaml":
@@ -167,9 +170,7 @@ pods "constraints-mem-demo-3" is forbidden: minimum memory usage per Container i

 ## Create a Pod that does not specify any memory request or limit

-
-
-Here's the configuration file for a Pod that has one Container. The Container does not
+Here's a manifest for a Pod that has one container. The container does not
 specify a memory request, and it does not specify a memory limit.

 {{< codenew file="admin/resource/memory-constraints-pod-4.yaml" >}}
@@ -182,12 +183,12 @@ kubectl apply -f https://k8s.io/examples/admin/resource/memory-constraints-pod-4

 View detailed information about the Pod:

-```
+```shell
 kubectl get pod constraints-mem-demo-4 --namespace=constraints-mem-example --output=yaml
 ```

-The output shows that the Pod's Container has a memory request of 1 GiB and a memory limit of 1 GiB.
-How did the Container get those values?
+The output shows that the Pod's only container has a memory request of 1 GiB and a memory limit of 1 GiB.
+How did that container get those values?

 ```
 resources:
@@ -197,11 +198,20 @@ resources:
    memory: 1Gi
 ```

-Because your Container did not specify its own memory request and limit, it was given the
+Because your Pod did not define any memory request and limit for that container, the cluster
+applied a
 [default memory request and limit](/docs/tasks/administer-cluster/manage-resources/memory-default-namespace/)
 from the LimitRange.

-At this point, your Container might be running or it might not be running. Recall that a prerequisite
+This means that the definition of that Pod shows those values. You can check it using
+`kubectl describe`:
+
+```shell
+# Look for the "Requests:" section of the output
+kubectl describe pod constraints-mem-demo-4 --namespace=constraints-mem-example
+```
+
+At this point, your Pod might be running or it might not be running. Recall that a prerequisite
 for this task is that your Nodes have at least 1 GiB of memory. If each of your Nodes has only
 1 GiB of memory, then there is not enough allocatable memory on any Node to accommodate a memory
 request of 1 GiB. If you happen to be using Nodes with 2 GiB of memory, then you probably have
@@ -209,7 +219,7 @@ enough space to accommodate the 1 GiB request.

 Delete your Pod:

-```
+```shell
 kubectl delete pod constraints-mem-demo-4 --namespace=constraints-mem-example
 ```

@@ -224,12 +234,12 @@ Pods that were created previously.
 As a cluster administrator, you might want to impose restrictions on the amount of memory that Pods can use.
 For example:

-* Each Node in a cluster has 2 GB of memory. You do not want to accept any Pod that requests
-more than 2 GB of memory, because no Node in the cluster can support the request.
+* Each Node in a cluster has 2 GiB of memory. You do not want to accept any Pod that requests
+more than 2 GiB of memory, because no Node in the cluster can support the request.

 * A cluster is shared by your production and development departments.
-You want to allow production workloads to consume up to 8 GB of memory, but
-you want development workloads to be limited to 512 MB. You create separate namespaces
+You want to allow production workloads to consume up to 8 GiB of memory, but
+you want development workloads to be limited to 512 MiB. You create separate namespaces
 for production and development, and you apply memory constraints to each namespace.

 ## Clean up
@@ -241,7 +251,6 @@ kubectl delete namespace constraints-mem-example
 ```


-
 ## {{% heading "whatsnext" %}}


@@ -2,21 +2,35 @@
 title: Configure Default Memory Requests and Limits for a Namespace
 content_type: task
 weight: 10
+description: >-
+  Define a default memory resource limit for a namespace, so that every new Pod
+  in that namespace has a memory resource limit configured.
 ---

 <!-- overview -->

-This page shows how to configure default memory requests and limits for a namespace.
-If a Container is created in a namespace that has a default memory limit, and the Container
-does not specify its own memory limit, then the Container is assigned the default memory limit.
+This page shows how to configure default memory requests and limits for a
+{{< glossary_tooltip text="namespace" term_id="namespace" >}}.
+
+A Kubernetes cluster can be divided into namespaces. Once you have a namespace that
+that has a default memory
+[limit](/docs/concepts/configuration/manage-resources-containers/#requests-and-limits),
+and you then try to create a Pod with a container that does not specify its own memory
+limit its own memory limit, then the
+{{< glossary_tooltip text="control plane" term_id="control-plane" >}} assigns the default
+memory limit to that container.
+
 Kubernetes assigns a default memory request under certain conditions that are explained later in this topic.



+
 ## {{% heading "prerequisites" %}}


-{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
+{{< include "task-tutorial-prereqs.md" >}}
+
+You must have access to create namespaces in your cluster.

 Each node in your cluster must have at least 2 GiB of memory.

@@ -35,8 +49,9 @@ kubectl create namespace default-mem-example

 ## Create a LimitRange and a Pod

-Here's the configuration file for a LimitRange object. The configuration specifies
-a default memory request and a default memory limit.
+Here's a manifest for an example {{< glossary_tooltip text="LimitRange" term_id="limitrange" >}}.
+The manifest specifies a default memory
+request and a default memory limit.

 {{< codenew file="admin/resource/memory-defaults.yaml" >}}

@@ -46,12 +61,13 @@ Create the LimitRange in the default-mem-example namespace:
 kubectl apply -f https://k8s.io/examples/admin/resource/memory-defaults.yaml --namespace=default-mem-example
 ```

-Now if a Container is created in the default-mem-example namespace, and the
-Container does not specify its own values for memory request and memory limit,
-the Container is given a default memory request of 256 MiB and a default
-memory limit of 512 MiB.
+Now if you create a Pod in the default-mem-example namespace, and any container
+within that Pod does not specify its own values for memory request and memory limit,
+then the {{< glossary_tooltip text="control plane" term_id="control-plane" >}}
+applies default values: a memory request of 256MiB and a memory limit of 512MiB.

-Here's the configuration file for a Pod that has one Container. The Container
+
+Here's an example manifest for a Pod that has one container. The container
 does not specify a memory request and limit.

 {{< codenew file="admin/resource/memory-defaults-pod.yaml" >}}
@@ -68,7 +84,7 @@ View detailed information about the Pod:
 kubectl get pod default-mem-demo --output=yaml --namespace=default-mem-example
 ```

-The output shows that the Pod's Container has a memory request of 256 MiB and
+The output shows that the Pod's container has a memory request of 256 MiB and
 a memory limit of 512 MiB. These are the default values specified by the LimitRange.

 ```shell
@@ -89,9 +105,9 @@ Delete your Pod:
 kubectl delete pod default-mem-demo --namespace=default-mem-example
 ```

-## What if you specify a Container's limit, but not its request?
+## What if you specify a container's limit, but not its request?

-Here's the configuration file for a Pod that has one Container. The Container
+Here's a manifest for a Pod that has one container. The container
 specifies a memory limit, but not a request:

 {{< codenew file="admin/resource/memory-defaults-pod-2.yaml" >}}
@@ -109,8 +125,8 @@ View detailed information about the Pod:
 kubectl get pod default-mem-demo-2 --output=yaml --namespace=default-mem-example
 ```

-The output shows that the Container's memory request is set to match its memory limit.
-Notice that the Container was not assigned the default memory request value of 256Mi.
+The output shows that the container's memory request is set to match its memory limit.
+Notice that the container was not assigned the default memory request value of 256Mi.

 ```
 resources:
@@ -120,9 +136,9 @@ resources:
    memory: 1Gi
 ```

-## What if you specify a Container's request, but not its limit?
+## What if you specify a container's request, but not its limit?

-Here's the configuration file for a Pod that has one Container. The Container
+Here's a manifest for a Pod that has one container. The container
 specifies a memory request, but not a limit:

 {{< codenew file="admin/resource/memory-defaults-pod-3.yaml" >}}
@@ -139,9 +155,9 @@ View the Pod's specification:
 kubectl get pod default-mem-demo-3 --output=yaml --namespace=default-mem-example
 ```

-The output shows that the Container's memory request is set to the value specified in the
-Container's configuration file. The Container's memory limit is set to 512Mi, which is the
-default memory limit for the namespace.
+The output shows that the container's memory request is set to the value specified in the
+container's manifest. The container is limited to use no more than 512MiB of
+memory, which matches the default memory limit for the namespace.

 ```
 resources:
@@ -153,15 +169,23 @@ resources:

 ## Motivation for default memory limits and requests

-If your namespace has a resource quota,
+If your namespace has a memory {{< glossary_tooltip text="resource quota" term_id="resource-quota" >}}
+configured,
 it is helpful to have a default value in place for memory limit.
 Here are two of the restrictions that a resource quota imposes on a namespace:

-* Every Container that runs in the namespace must have its own memory limit.
-* The total amount of memory used by all Containers in the namespace must not exceed a specified limit.
+* For every Pod that runs in the namespace, the Pod and each of its containers must have a memory limit.
+  (If you specify a memory limit for every container in a Pod, Kubernetes can infer the Pod-level memory
+  limit by adding up the limits for its containers).
+* CPU limits apply a resource reservation on the node where the Pod in question is scheduled.
+  The total amount of memory reserved for all Pods in the namespace must not exceed a specified limit.
+* The total amount of memory actually used by all Pods in the namespace must also not exceed a specified limit.

-If a Container does not specify its own memory limit, it is given the default limit, and then
-it can be allowed to run in a namespace that is restricted by a quota.
+When you add a LimitRange:
+
+If any Pod in that namespace that includes a container does not specify its own memory limit,
+the control plane applies the default memory limit to that container, and the Pod can be
+allowed to run in a namespace that is restricted by a memory ResourceQuota.

 ## Clean up

@@ -2,14 +2,17 @@
 title: Configure Memory and CPU Quotas for a Namespace
 content_type: task
 weight: 50
+description: >-
+  Define overall memory and CPU resource limits for a namespace.
 ---


 <!-- overview -->

 This page shows how to set quotas for the total amount memory and CPU that
-can be used by all Containers running in a namespace. You specify quotas in a
-[ResourceQuota](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#resourcequota-v1-core)
+can be used by all Pods running in a {{< glossary_tooltip text="namespace" term_id="namespace" >}}.
+You specify quotas in a
+[ResourceQuota](/docs/reference/kubernetes-api/policy-resources/resource-quota-v1/)
 object.


@@ -17,14 +20,13 @@ object.

 ## {{% heading "prerequisites" %}}

+{{< include "task-tutorial-prereqs.md" >}}

-{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
+You must have access to create namespaces in your cluster.

 Each node in your cluster must have at least 1 GiB of memory.


-
-
 <!-- steps -->

 ## Create a namespace
@@ -38,7 +40,7 @@ kubectl create namespace quota-mem-cpu-example

 ## Create a ResourceQuota

-Here is the configuration file for a ResourceQuota object:
+Here is a manifest for an example ResourceQuota:

 {{< codenew file="admin/resource/quota-mem-cpu.yaml" >}}

@@ -56,15 +58,18 @@ kubectl get resourcequota mem-cpu-demo --namespace=quota-mem-cpu-example --outpu

 The ResourceQuota places these requirements on the quota-mem-cpu-example namespace:

-* Every Container must have a memory request, memory limit, cpu request, and cpu limit.
-* The memory request total for all Containers must not exceed 1 GiB.
-* The memory limit total for all Containers must not exceed 2 GiB.
-* The CPU request total for all Containers must not exceed 1 cpu.
-* The CPU limit total for all Containers must not exceed 2 cpu.
+* For every Pod in the namespace, each container must have a memory request, memory limit, cpu request, and cpu limit.
+* The memory request total for all Pods in that namespace must not exceed 1 GiB.
+* The memory limit total for all Pods in that namespace must not exceed 2 GiB.
+* The CPU request total for all Pods in that namespace must not exceed 1 cpu.
+* The CPU limit total for all Pods in that namespace must not exceed 2 cpu.
+
+See [meaning of CPU](/docs/concepts/configuration/manage-resources-containers/#meaning-of-cpu)
+to learn what Kubernetes means by “1 CPU”.

 ## Create a Pod

-Here is the configuration file for a Pod:
+Here is a manifest for an example Pod:

 {{< codenew file="admin/resource/quota-mem-cpu-pod.yaml" >}}

@@ -75,15 +80,15 @@ Create the Pod:
 kubectl apply -f https://k8s.io/examples/admin/resource/quota-mem-cpu-pod.yaml --namespace=quota-mem-cpu-example
 ```

-Verify that the Pod's Container is running:
+Verify that the Pod is running and that its (only) container is healthy:

-```
+```shell
 kubectl get pod quota-mem-cpu-demo --namespace=quota-mem-cpu-example
 ```

 Once again, view detailed information about the ResourceQuota:

-```
+```shell
 kubectl get resourcequota mem-cpu-demo --namespace=quota-mem-cpu-example --output=yaml
 ```

@@ -105,15 +110,22 @@ status:
    requests.memory: 600Mi
 ```

+If you have the `jq` tool, you can also query (using [JSONPath](/docs/reference/kubectl/jsonpath/))
+for just the `used` values, **and** pretty-print that that of the output. For example:
+
+```shell
+kubectl get resourcequota mem-cpu-demo --namespace=quota-mem-cpu-example -o jsonpath='{ .status.used }' | jq .
+```
+
 ## Attempt to create a second Pod

-Here is the configuration file for a second Pod:
+Here is a manifest for a second Pod:

 {{< codenew file="admin/resource/quota-mem-cpu-pod-2.yaml" >}}

-In the configuration file, you can see that the Pod has a memory request of 700 MiB.
+In the manifest, you can see that the Pod has a memory request of 700 MiB.
 Notice that the sum of the used memory request and this new memory
-request exceeds the memory request quota. 600 MiB + 700 MiB > 1 GiB.
+request exceeds the memory request quota: 600 MiB + 700 MiB > 1 GiB.

 Attempt to create the Pod:

@@ -133,11 +145,12 @@ requested: requests.memory=700Mi,used: requests.memory=600Mi, limited: requests.
 ## Discussion

 As you have seen in this exercise, you can use a ResourceQuota to restrict
-the memory request total for all Containers running in a namespace.
+the memory request total for all Pods running in a namespace.
 You can also restrict the totals for memory limit, cpu request, and cpu limit.

-If you want to restrict individual Containers, instead of totals for all Containers, use a
-[LimitRange](/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace/).
+Instead of managing total resource use within a namespace, you might want to restrict
+individual Pods, or the containers in those Pods. To achieve that kind of limiting, use a
+[LimitRange](/docs/concepts/policy/limit-range/).

 ## Clean up

@@ -2,14 +2,16 @@
 title: Configure a Pod Quota for a Namespace
 content_type: task
 weight: 60
+description: >-
+  Restrict how many Pods you can create within a namespace.
 ---


 <!-- overview -->

 This page shows how to set a quota for the total number of Pods that can run
-in a namespace. You specify quotas in a
-[ResourceQuota](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#resourcequota-v1-core)
+in a {{< glossary_tooltip text="Namespace" term_id="namespace" >}}. You specify quotas in a
+[ResourceQuota](/docs/reference/kubernetes-api/policy-resources/resource-quota-v1/)
 object.


@@ -18,10 +20,9 @@ object.
 ## {{% heading "prerequisites" %}}


-{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
-
-
+{{< include "task-tutorial-prereqs.md" >}}

+You must have access to create namespaces in your cluster.

 <!-- steps -->

@@ -36,7 +37,7 @@ kubectl create namespace quota-pod-example

 ## Create a ResourceQuota

-Here is the configuration file for a ResourceQuota object:
+Here is an example manifest for a ResourceQuota:

 {{< codenew file="admin/resource/quota-pod.yaml" >}}

@@ -66,11 +67,12 @@ status:
    pods: "0"
 ```

-Here is the configuration file for a Deployment:
+Here is an example manifest for a {{< glossary_tooltip term_id="deployment" >}}:

 {{< codenew file="admin/resource/quota-pod-deployment.yaml" >}}

-In the configuration file, `replicas: 3` tells Kubernetes to attempt to create three Pods, all running the same application.
+In that manifest, `replicas: 3` tells Kubernetes to attempt to create three new Pods, all
+running the same application.

 Create the Deployment:

@@ -85,7 +87,7 @@ kubectl get deployment pod-quota-demo --namespace=quota-pod-example --output=yam
 ```

 The output shows that even though the Deployment specifies three replicas, only two
-Pods were created because of the quota.
+Pods were created because of the quota you defined earlier:

 ```yaml
 spec:
@@ -95,11 +97,18 @@ spec:
 status:
  availableReplicas: 2
 ...
-lastUpdateTime: 2017-07-07T20:57:05Z
+lastUpdateTime: 2021-04-02T20:57:05Z
    message: 'unable to create pods: pods "pod-quota-demo-1650323038-" is forbidden:
      exceeded quota: pod-demo, requested: pods=1, used: pods=2, limited: pods=2'
 ```

+### Choice of resource
+
+In this task you have defined a ResourceQuota that limited the total number of Pods, but
+you could also limit the total number of other kinds of object. For example, you
+might decide to limit how many {{< glossary_tooltip text="CronJobs" term_id="cronjob" >}}
+that can live in a single namespace.
+
 ## Clean up

 Delete your namespace:
@@ -8,37 +8,35 @@ weight: 70

 <!-- overview -->

-With Kubernetes 1.20 dockershim was deprecated. From the
-[Dockershim Deprecation FAQ](/blog/2020/12/02/dockershim-faq/)
-you might already know that most apps do not have a direct dependency on runtime hosting
-containers. However, there are still a lot of telemetry and security agents
-that has a dependency on docker to collect containers metadata, logs and
-metrics. This document aggregates information on how to detect these
-dependencies and links on how to migrate these agents to use generic tools or
-alternative runtimes.
+Kubernetes' support for direct integration with Docker Engine is deprecated, and will be removed. Most apps do not have a direct dependency on runtime hosting containers. However, there are still a lot of telemetry and monitoring agents that has a dependency on docker to collect containers metadata, logs and metrics. This document aggregates information on how to detect these dependencies and links on how to migrate these agents to use generic tools or alternative runtimes.

 ## Telemetry and security agents

-There are a few ways agents may run on Kubernetes cluster. Agents may run on
-nodes directly or as DaemonSets.
+Within a Kubernetes cluster there are a few different ways to run telemetry or security agents.
+Some agents have a direct  dependency on Docker Engine when they as DaemonSets or
+directly on nodes.

-### Why do telemetry agents rely on Docker?
+### Why do some telemetry agents communicate with Docker Engine?

-Historically, Kubernetes was built on top of Docker. Kubernetes is managing
-networking and scheduling, Docker was placing and operating containers on a
-node. So you can get scheduling-related metadata like a pod name from Kubernetes
-and containers state information from Docker. Over time more runtimes were
-created to manage containers. Also there are projects and Kubernetes features
-that generalize container status information extraction across many runtimes.
+Historically, Kubernetes was written to work specifically with Docker Engine.
+Kubernetes took care of networking and scheduling, relying on Docker Engine for launching
+and running containers (within Pods) on a node. Some information that is relevant to telemetry,
+such as a pod name, is only available from Kubernetes components. Other data, such as container
+metrics, is not the responsibility of the container runtime. Early yelemetry agents needed to query the
+container runtime **and** Kubernetes to report an accurate picture. Over time, Kubernetes gained
+the ability to support multiple runtimes, and now supports any runtime that is compatible with
+the container runtime interface.

-Some agents are tied specifically to the Docker tool. The agents may run
-commands like [`docker ps`](https://docs.docker.com/engine/reference/commandline/ps/)
+Some telemetry agents rely specifically on Docker Engine tooling. For example, an agent
+might run a command such as
+[`docker ps`](https://docs.docker.com/engine/reference/commandline/ps/)
 or [`docker top`](https://docs.docker.com/engine/reference/commandline/top/) to list
-containers and processes or [docker logs](https://docs.docker.com/engine/reference/commandline/logs/)
-to subscribe on docker logs. With the deprecating of Docker as a container runtime,
+containers and processes or [`docker logs`](https://docs.docker.com/engine/reference/commandline/logs/)
+to receive streamed logs. If nodes in your existing cluster use
+Docker Engine, and you switch to a different container runtime,
 these commands will not work any longer.

-### Identify DaemonSets that depend on Docker {#identify-docker-dependency}
+### Identify DaemonSets that depend on Docker Engine {#identify-docker-dependency}

 If a pod wants to make calls to the `dockerd` running on the node, the pod must either:

@@ -67,7 +67,7 @@ transient slices for resources that are supported by that init system.
 Depending on the configuration of the associated container runtime,
 operators may have to choose a particular cgroup driver to ensure
 proper system behavior. For example, if operators use the `systemd`
-cgroup driver provided by the `docker` runtime, the `kubelet` must
+cgroup driver provided by the `containerd` runtime, the `kubelet` must
 be configured to use the `systemd` cgroup driver.

 ### Kube Reserved
@@ -174,7 +174,7 @@ The output shows that the Container was killed because it is out of memory (OOM)
 ```shell
 lastState:
   terminated:
-     containerID: docker://65183c1877aaec2e8427bc95609cc52677a454b56fcb24340dbd22917c23b10f
+     containerID: 65183c1877aaec2e8427bc95609cc52677a454b56fcb24340dbd22917c23b10f
     exitCode: 137
     finishedAt: 2017-06-20T20:52:19Z
     reason: OOMKilled
@@ -7,8 +7,10 @@ weight: 100
 <!-- overview -->

 This page shows how to create a Pod that uses a
-{{< glossary_tooltip text="Secret" term_id="secret" >}} to pull an image from a
-private container image registry or repository.
+{{< glossary_tooltip text="Secret" term_id="secret" >}} to pull an image 
+from a private container image registry or repository. There are many private 
+registries in use. This task uses [Docker Hub](https://www.docker.com/products/docker-hub)
+as an example registry.

 {{% thirdparty-content single="true" %}}

@@ -18,6 +20,8 @@ private container image registry or repository.

 * To do this exercise, you need the `docker` command line tool, and a
  [Docker ID](https://docs.docker.com/docker-id/) for which you know the password.
+* If you are using a different private container registry, you need the command 
+  line tool for that registry and any login information for the registry. 

 <!-- steps -->

@@ -59,12 +63,13 @@ The output contains a section similar to this:
 If you use a Docker credentials store, you won't see that `auth` entry but a `credsStore` entry with the name of the store as value.
 {{< /note >}}

-## Create a Secret based on existing Docker credentials {#registry-secret-existing-credentials}
+## Create a Secret based on existing credentials {#registry-secret-existing-credentials}

 A Kubernetes cluster uses the Secret of `kubernetes.io/dockerconfigjson` type to authenticate with
 a container registry to pull a private image.

-If you already ran `docker login`, you can copy that credential into Kubernetes:
+If you already ran `docker login`, you can copy
+that credential into Kubernetes:

 ```shell
 kubectl create secret generic regcred \
@@ -77,7 +82,7 @@ secret) then you can customise the Secret before storing it.
 Be sure to:

 - set the name of the data item to `.dockerconfigjson`
- base64 encode the docker file and paste that string, unbroken
+- base64 encode the Docker configuration file and then paste that string, unbroken
  as the value for field `data[".dockerconfigjson"]`
 - set `type` to `kubernetes.io/dockerconfigjson`

@@ -213,4 +218,3 @@ kubectl get pod private-reg
 * Learn more about [adding image pull secrets to a service account](/docs/tasks/configure-pod-container/configure-service-account/#add-imagepullsecrets-to-a-service-account).
 * See [kubectl create secret docker-registry](/docs/reference/generated/kubectl/kubectl-commands/#-em-secret-docker-registry-em-).
 * See the `imagePullSecrets` field within the [container definitions](/docs/reference/kubernetes-api/workload-resources/pod-v1/#containers) of a Pod
-
@@ -42,7 +42,7 @@ The `spec` of a static Pod cannot refer to other API objects

 {{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}

-This page assumes you're using {{< glossary_tooltip term_id="docker" >}} to run Pods,
+This page assumes you're using {{< glossary_tooltip term_id="cri-o" >}} to run Pods,
 and that your nodes are running the Fedora operating system.
 Instructions for other distributions or Kubernetes installations may vary.

@@ -156,15 +156,20 @@ already be running.
 You can view running containers (including static Pods) by running (on the node):
 ```shell
 # Run this command on the node where the kubelet is running
-docker ps
+crictl ps
 ```

 The output might be something like:

+```console
+CONTAINER       IMAGE                                 CREATED           STATE      NAME    ATTEMPT    POD ID
+129fd7d382018   docker.io/library/nginx@sha256:...    11 minutes ago    Running    web     0          34533c6729106
 ```
-CONTAINER ID IMAGE         COMMAND  CREATED        STATUS         PORTS     NAMES
-f6d05272b57e nginx:latest  "nginx"  8 minutes ago  Up 8 minutes             k8s_web.6f802af4_static-web-fk-node1_default_67e24ed9466ba55986d120c867395f3c_378e5f3c
-```
+
+{{< note >}}
+`crictl` outputs the image URI and SHA-256 checksum. `NAME` will look more like:
+`docker.io/library/nginx@sha256:0d17b565c37bcbd895e9d92315a05c1c3c9a29f762b011a10c54a66cd53c9b31`.
+{{< /note >}}

 You can see the mirror Pod on the API server:

@@ -172,8 +177,8 @@ You can see the mirror Pod on the API server:
 kubectl get pods
 ```
 ```
-NAME                       READY     STATUS    RESTARTS   AGE
-static-web-my-node1        1/1       Running   0          2m
+NAME         READY   STATUS    RESTARTS        AGE
+static-web   1/1     Running   0               2m
 ```

 {{< note >}}
@@ -181,7 +186,6 @@ Make sure the kubelet has permission to create the mirror Pod in the API server.
 [PodSecurityPolicy](/docs/concepts/policy/pod-security-policy/).
 {{< /note >}}

-
 {{< glossary_tooltip term_id="label" text="Labels" >}} from the static Pod are
 propagated into the mirror Pod. You can use those labels as normal via
 {{< glossary_tooltip term_id="selector" text="selectors" >}}, etc.
@@ -190,34 +194,33 @@ If you try to use `kubectl` to delete the mirror Pod from the API server,
 the kubelet _doesn't_ remove the static Pod:

 ```shell
-kubectl delete pod static-web-my-node1
+kubectl delete pod static-web
 ```
 ```
-pod "static-web-my-node1" deleted
+pod "static-web" deleted
 ```
 You can see that the Pod is still running:
 ```shell
 kubectl get pods
 ```
 ```
-NAME                       READY     STATUS    RESTARTS   AGE
-static-web-my-node1        1/1       Running   0          12s
+NAME         READY   STATUS    RESTARTS   AGE
+static-web   1/1     Running   0          4s
 ```

-Back on your node where the kubelet is running, you can try to stop the Docker
-container manually.
+Back on your node where the kubelet is running, you can try to stop the container manually.
 You'll see that, after a time, the kubelet will notice and will restart the Pod
 automatically:

 ```shell
 # Run these commands on the node where the kubelet is running
-docker stop f6d05272b57e # replace with the ID of your container
+crictl stop 129fd7d382018 # replace with the ID of your container
 sleep 20
-docker ps
+crictl ps
 ```
-```
-CONTAINER ID        IMAGE         COMMAND                CREATED       ...
-5b920cbaf8b1        nginx:latest  "nginx -g 'daemon of   2 seconds ago ...
+```console
+CONTAINER       IMAGE                                 CREATED           STATE      NAME    ATTEMPT    POD ID
+89db4553e1eeb   docker.io/library/nginx@sha256:...    19 seconds ago    Running    web     1          34533c6729106
 ```

 ## Dynamic addition and removal of static pods
@@ -230,14 +233,13 @@ The running kubelet periodically scans the configured directory (`/etc/kubelet.d
 #
 mv /etc/kubelet.d/static-web.yaml /tmp
 sleep 20
-docker ps
+crictl ps
 # You see that no nginx container is running
 mv /tmp/static-web.yaml  /etc/kubelet.d/
 sleep 20
-docker ps
+crictl ps
 ```
+```console
+CONTAINER       IMAGE                                 CREATED           STATE      NAME    ATTEMPT    POD ID
+f427638871c35   docker.io/library/nginx@sha256:...    19 seconds ago    Running    web     1          34533c6729106
 ```
-CONTAINER ID        IMAGE         COMMAND                CREATED           ...
-e7a62e3427f1        nginx:latest  "nginx -g 'daemon of   27 seconds ago
-```
-
@@ -39,7 +39,7 @@ You may want to set
 (default to 1),
 [`.spec.minReadySeconds`](/docs/concepts/workloads/controllers/deployment/#min-ready-seconds) 
 (default to 0) and 
-[`.spec.maxSurge`](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#max-surge) 
+[`.spec.updateStrategy.rollingUpdate.maxSurge`](/docs/reference/kubernetes-api/workload-resources/daemon-set-v1/#DaemonSetSpec)
 (a beta feature and defaults to 0) as well.

 ### Creating a DaemonSet with `RollingUpdate` update strategy
@@ -70,14 +70,9 @@ spec:

 Patch your Deployment:

-{{< tabs name="kubectl_patch_example" >}}
-{{{< tab name="Bash" codelang="bash" >}}
-kubectl patch deployment patch-demo --patch "$(cat patch-file.yaml)"
-{{< /tab >}}
-{{< tab name="PowerShell" codelang="posh" >}}
-kubectl patch deployment patch-demo --patch $(Get-Content patch-file.yaml -Raw)
-{{< /tab >}}}
-{{< /tabs >}}
+```shell
+kubectl patch deployment patch-demo --patch-file patch-file.yaml
+```

 View the patched Deployment:

@@ -183,7 +178,7 @@ spec:
 Patch your Deployment:

 ```shell
-kubectl patch deployment patch-demo --patch "$(cat patch-file-tolerations.yaml)"
+kubectl patch deployment patch-demo --patch-file patch-file-tolerations.yaml
 ```

 View the patched Deployment:
@@ -249,7 +244,7 @@ spec:
 In your patch command, set `type` to `merge`:

 ```shell
-kubectl patch deployment patch-demo --type merge --patch "$(cat patch-file-2.yaml)"
+kubectl patch deployment patch-demo --type merge --patch-file patch-file-2.yaml
 ```

 View the patched Deployment:
@@ -308,14 +303,9 @@ spec:

 Patch your Deployment:

-{{< tabs name="kubectl_retainkeys_example" >}}
-{{{< tab name="Bash" codelang="bash" >}}
-kubectl patch deployment retainkeys-demo --type merge --patch "$(cat patch-file-no-retainkeys.yaml)"
-{{< /tab >}}
-{{< tab name="PowerShell" codelang="posh" >}}
-kubectl patch deployment retainkeys-demo --type merge --patch $(Get-Content patch-file-no-retainkeys.yaml -Raw)
-{{< /tab >}}}
-{{< /tabs >}}
+```shell
+kubectl patch deployment retainkeys-demo --type merge --patch-file patch-file-no-retainkeys.yaml
+```

 In the output, you can see that it is not possible to set `type` as `Recreate` when a value is defined for `spec.strategy.rollingUpdate`:

@@ -339,14 +329,9 @@ With this patch, we indicate that we want to retain only the `type` key of the `

 Patch your Deployment again with this new patch:

-{{< tabs name="kubectl_retainkeys2_example" >}}
-{{{< tab name="Bash" codelang="bash" >}}
-kubectl patch deployment retainkeys-demo --type merge --patch "$(cat patch-file-retainkeys.yaml)"
-{{< /tab >}}
-{{< tab name="PowerShell" codelang="posh" >}}
-kubectl patch deployment retainkeys-demo --type merge --patch $(Get-Content patch-file-retainkeys.yaml -Raw)
-{{< /tab >}}}
-{{< /tabs >}}
+```shell
+kubectl patch deployment retainkeys-demo --type merge --patch-file patch-file-retainkeys.yaml
+```

 Examine the content of the Deployment:

@@ -425,10 +410,10 @@ The following commands are equivalent:


 ```shell
-kubectl patch deployment patch-demo --patch "$(cat patch-file.yaml)"
+kubectl patch deployment patch-demo --patch-file patch-file.yaml
 kubectl patch deployment patch-demo --patch 'spec:\n template:\n  spec:\n   containers:\n   - name: patch-demo-ctr-2\n     image: redis'

-kubectl patch deployment patch-demo --patch "$(cat patch-file.json)"
+kubectl patch deployment patch-demo --patch-file patch-file.json
 kubectl patch deployment patch-demo --patch '{"spec": {"template": {"spec": {"containers": [{"name": "patch-demo-ctr-2","image": "redis"}]}}}}'
 ```

@@ -173,7 +173,10 @@ automatically responds to changes in the number of replicas of the corresponding

 ## Create the PDB object

-You can create or update the PDB object with a command like `kubectl apply -f mypdb.yaml`.
+You can create or update the PDB object using kubectl.
+```shell
+kubectl apply -f mypdb.yaml
+```

 ## Check the status of the PDB

@@ -85,7 +85,7 @@ For example, to download version {{< param "fullversion" >}} on Linux, type:
   chmod +x kubectl
   mkdir -p ~/.local/bin/kubectl
   mv ./kubectl ~/.local/bin/kubectl
-   # and then add ~/.local/bin/kubectl to $PATH
+   # and then append (or prepend) ~/.local/bin to $PATH
   ```

   {{< /note >}}
@@ -59,7 +59,7 @@ The following methods exist for installing kubectl on Windows:
     $($(CertUtil -hashfile .\kubectl.exe SHA256)[1] -replace " ", "") -eq $(type .\kubectl.exe.sha256)
     ```

-1. Add the binary in to your `PATH`.
+1. Append or prepend the kubectl binary folder to your `PATH` environment variable.

 1. Test to ensure the version of `kubectl` is the same as downloaded:

@@ -172,7 +172,7 @@ Below are the procedures to set up autocompletion for PowerShell.
     $($(CertUtil -hashfile .\kubectl-convert.exe SHA256)[1] -replace " ", "") -eq $(type .\kubectl-convert.exe.sha256)
     ```

-1. Add the binary in to your `PATH`.
+1. Append or prepend the kubectl binary folder to your `PATH` environment variable.

 1. Verify plugin is successfully installed

@@ -174,6 +174,15 @@ of security defaults while preserving the functionality of the workload. It is
 possible that the default profiles differ between container runtimes and their
 release versions, for example when comparing those from CRI-O and containerd.

+{{< note >}}
+Enabling the feature will neither change the Kubernetes
+`securityContext.seccompProfile` API field nor add the deprecated annotations of
+the workload. This provides users the possibility to rollback anytime without
+actually changing the workload configuration. Tools like
+[`crictl inspect`](https://github.com/kubernetes-sigs/cri-tools) can be used to
+verify which seccomp profile is being used by a container.
+{{< /note >}}
+
 Some workloads may require a lower amount of syscall restrictions than others.
 This means that they can fail during runtime even with the `RuntimeDefault`
 profile. To mitigate such a failure, you can:
@@ -203,6 +212,51 @@ kind: Cluster
 apiVersion: kind.x-k8s.io/v1alpha4
 featureGates:
  SeccompDefault: true
+nodes:
+  - role: control-plane
+    image: kindest/node:v1.23.0@sha256:49824ab1727c04e56a21a5d8372a402fcd32ea51ac96a2706a12af38934f81ac
+    kubeadmConfigPatches:
+      - |
+        kind: JoinConfiguration
+        nodeRegistration:
+          kubeletExtraArgs:
+            seccomp-default: "true"
+  - role: worker
+    image: kindest/node:v1.23.0@sha256:49824ab1727c04e56a21a5d8372a402fcd32ea51ac96a2706a12af38934f81ac
+    kubeadmConfigPatches:
+      - |
+        kind: JoinConfiguration
+        nodeRegistration:
+          kubeletExtraArgs:
+            feature-gates: SeccompDefault=true
+            seccomp-default: "true"
+```
+
+If the cluster is ready, then running a pod:
+
+```shell
+kubectl run --rm -it --restart=Never --image=alpine alpine -- sh
+```
+
+Should now have the default seccomp profile attached. This can be verified by
+using `docker exec` to run `crictl inspect` for the container on the kind
+worker:
+
+```shell
+docker exec -it kind-worker bash -c \
+    'crictl inspect $(crictl ps --name=alpine -q) | jq .info.runtimeSpec.linux.seccomp'
+```
+
+```json
+{
+  "defaultAction": "SCMP_ACT_ERRNO",
+  "architectures": ["SCMP_ARCH_X86_64", "SCMP_ARCH_X86", "SCMP_ARCH_X32"],
+  "syscalls": [
+    {
+      "names": ["..."]
+    }
+  ]
+}
 ```

 ## Create a Pod with a seccomp profile for syscall auditing
@@ -0,0 +1,10 @@
+apiVersion: networking.k8s.io/v1
+kind: IngressClass
+metadata:
+  labels:
+    app.kubernetes.io/component: controller
+  name: nginx-example
+  annotations:
+    ingressclass.kubernetes.io/is-default-class: "true"
+spec:
+  controller: k8s.io/ingress-nginx
@@ -5,6 +5,7 @@ metadata:
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
 spec:
+  ingressClassName: nginx-example
  rules:
  - http:
      paths:
@@ -1,4 +1,4 @@
-You need to either have a dynamic PersistentVolume provisioner with a default
+You need to either have a [dynamic PersistentVolume provisioner](/docs/concepts/storage/dynamic-provisioning/) with a default
 [StorageClass](/docs/concepts/storage/storage-classes/),
 or [statically provision PersistentVolumes](/docs/user-guide/persistent-volumes/#provisioning)
 yourself to satisfy the [PersistentVolumeClaims](/docs/user-guide/persistent-volumes/#persistentvolumeclaims)
@@ -92,6 +92,7 @@ End of Life for **1.23** is **2023-02-28**.

 | Patch Release | Cherry Pick Deadline | Target Date | Note |
 |---------------|----------------------|-------------|------|
+| 1.23.3        | 2022-01-24           | 2022-01-25  | [Out-of-Band Release](https://groups.google.com/u/2/a/kubernetes.io/g/dev/c/Xl1sm-CItaY) |
 | 1.23.2        | 2022-01-14           | 2022-01-19  |      |
 | 1.23.1        | 2021-12-14           | 2021-12-16  |      |

@@ -169,9 +169,9 @@ of each minor (1.Y) and patch (1.Y.Z) release
 GitHub team: [@kubernetes/build-admins](https://github.com/orgs/kubernetes/teams/build-admins)

 - Aaron Crickenberger ([@spiffxp](https://github.com/spiffxp))
- Amit Watve ([@amwat](https://github.com/amwat))
 - Benjamin Elder ([@BenTheElder](https://github.com/BenTheElder))
 - Grant McCloskey ([@MushuEE](https://github.com/MushuEE))
+- Juan Escobar ([@juanfescobar](https://github.com/juanfescobar))

 ## SIG Release Leads