Merge master into dev-1.20 to keep in sync

2020-10-22 17:44:02 +02:00
parent a9b21d2f4f 167a19e0b8
commit 45ec60bed5
268 changed files with 11321 additions and 55429 deletions
@@ -49,7 +49,7 @@ To verify this connection, use the `--kubelet-certificate-authority` flag to pro
 If that is not possible, use [SSH tunneling](#ssh-tunnels) between the apiserver and kubelet if required to avoid connecting over an
 untrusted or public network.

-Finally, [Kubelet authentication and/or authorization](/docs/admin/kubelet-authentication-authorization/) should be enabled to secure the kubelet API.
+Finally, [Kubelet authentication and/or authorization](/docs/reference/command-line-tools-reference/kubelet-authentication-authorization/) should be enabled to secure the kubelet API.

 ### apiserver to nodes, pods, and services

@@ -96,6 +96,16 @@ the current state closer in line.
 nodes in your cluster. See
 [Cluster autoscaling](/docs/tasks/administer-cluster/cluster-management/#cluster-autoscaling)).

+The important point here is that the controller makes some change to bring about
+your desired state, and then reports current state back to your cluster's API server.
+Other control loops can observe that reported data and take their own actions.
+
+In the thermostat example, if the room is very cold then a different controller
+might also turn on a frost protection heater. With Kubernetes clusters, the control
+plane indirectly works with IP address management tools, storage services,
+cloud provider APIS, and other services by
+[extending Kubernetes](/docs/concepts/extend-kubernetes/) to implement that.
+
 ## Desired versus current state {#desired-vs-current}

 Kubernetes takes a cloud-native view of systems, and is able to handle
@@ -51,7 +51,7 @@ Before choosing a guide, here are some considerations:

 * [Kubernetes Container Environment](/docs/concepts/containers/container-environment/) describes the environment for Kubelet managed containers on a Kubernetes node.

-* [Controlling Access to the Kubernetes API](/docs/reference/access-authn-authz/controlling-access/) describes how to set up permissions for users and service accounts.
+* [Controlling Access to the Kubernetes API](/docs/concepts/security/controlling-access) describes how Kubernetes implements access control for its own API.

 * [Authenticating](/docs/reference/access-authn-authz/authentication/) explains authentication in Kubernetes, including the various authentication options.

@@ -24,8 +24,8 @@ the description of how logs are stored and handled on the node to be useful.

 In this section, you can see an example of basic logging in Kubernetes that
 outputs data to the standard output stream. This demonstration uses
-a [pod specification](/examples/debug/counter-pod.yaml) with
-a container that writes some text to standard output once per second.
+a pod specification with a container that writes some text to standard output
+once per second.

 {{< codenew file="debug/counter-pod.yaml" >}}

@@ -104,7 +104,7 @@ The kubelet collects accelerator metrics through cAdvisor. To collect these metr

 The responsibility for collecting accelerator metrics now belongs to the vendor rather than the kubelet. Vendors must provide a container that collects metrics and exposes them to the metrics service (for example, Prometheus).

-The [`DisableAcceleratorUsageMetrics` feature gate](/docs/references/command-line-tools-reference/feature-gate.md#feature-gates-for-alpha-or-beta-features:~:text= DisableAcceleratorUsageMetrics,-false) disables metrics collected by the kubelet, with a [timeline for enabling this feature by default](https://github.com/kubernetes/enhancements/tree/411e51027db842355bd489691af897afc1a41a5e/keps/sig-node/1867-disable-accelerator-usage-metrics#graduation-criteria).
+The [`DisableAcceleratorUsageMetrics` feature gate](/docs/reference/command-line-tools-reference/feature-gates/#feature-gates-for-alpha-or-beta-features:~:text= DisableAcceleratorUsageMetrics,-false) disables metrics collected by the kubelet, with a [timeline for enabling this feature by default](https://github.com/kubernetes/enhancements/tree/411e51027db842355bd489691af897afc1a41a5e/keps/sig-node/1867-disable-accelerator-usage-metrics#graduation-criteria).

 ## Component metrics

@@ -11,7 +11,7 @@ weight: 70

 {{< feature-state for_k8s_version="v1.14" state="stable" >}}

-[Pods](/docs/concepts/workloads/pods/pod/) can have _priority_. Priority indicates the
+[Pods](/docs/concepts/workloads/pods/) can have _priority_. Priority indicates the
 importance of a Pod relative to other Pods. If a Pod cannot be scheduled, the
 scheduler tries to preempt (evict) lower priority Pods to make scheduling of the
 pending Pod possible.
@@ -183,7 +183,7 @@ are accounted for in Kubernetes.

 - [RuntimeClass Design](https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/585-runtime-class/README.md)
 - [RuntimeClass Scheduling Design](https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/585-runtime-class/README.md#runtimeclass-scheduling)
- Read about the [Pod Overhead](/docs/concepts/configuration/pod-overhead/) concept
+- Read about the [Pod Overhead](/docs/concepts/scheduling-eviction/pod-overhead/) concept
 - [PodOverhead Feature Design](https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/20190226-pod-overhead.md)


@@ -130,7 +130,7 @@ Adding an API does not directly let you affect the behavior of existing APIs (e.

 ### API Access Extensions

-When a request reaches the Kubernetes API Server, it is first Authenticated, then Authorized, then subject to various types of Admission Control. See [Controlling Access to the Kubernetes API](/docs/reference/access-authn-authz/controlling-access/) for more on this flow.
+When a request reaches the Kubernetes API Server, it is first Authenticated, then Authorized, then subject to various types of Admission Control. See [Controlling Access to the Kubernetes API](/docs/concepts/security/controlling-access/) for more on this flow.

 Each of these steps offers extension points.

@@ -16,7 +16,7 @@ methods for adding custom resources and how to choose between them.
 <!-- body -->
 ## Custom resources

-A *resource* is an endpoint in the [Kubernetes API](/docs/reference/using-api/api-overview/) that stores a collection of
+A *resource* is an endpoint in the [Kubernetes API](/docs/concepts/overview/kubernetes-api/) that stores a collection of
 [API objects](/docs/concepts/overview/working-with-objects/kubernetes-objects/) of a certain kind; for example, the built-in *pods* resource contains a collection of Pod objects.

 A *custom resource* is an extension of the Kubernetes API that is not necessarily available in a default
@@ -37,7 +37,7 @@ During the registration, the device plugin needs to send:
  * The name of its Unix socket.
  * The Device Plugin API version against which it was built.
  * The `ResourceName` it wants to advertise. Here `ResourceName` needs to follow the
-    [extended resource naming scheme](/docs/concepts/configuration/manage-resources-container/#extended-resources)
+    [extended resource naming scheme](/docs/concepts/configuration/manage-resources-containers/#extended-resources)
    as `vendor-domain/resourcetype`.
    (For example, an NVIDIA GPU is advertised as `nvidia.com/gpu`.)

@@ -210,7 +210,7 @@ Support for the "PodResources service" requires `KubeletPodResources` [feature g

 {{< feature-state for_k8s_version="v1.18" state="beta" >}}

-The Topology Manager is a Kubelet component that allows resources to be co-ordintated in a Topology aligned manner. In order to do this, the Device Plugin API was extended to include a `TopologyInfo` struct.
+The Topology Manager is a Kubelet component that allows resources to be co-ordinated in a Topology aligned manner. In order to do this, the Device Plugin API was extended to include a `TopologyInfo` struct.


 ```gRPC
@@ -131,7 +131,7 @@ Adding an API does not directly let you affect the behavior of existing APIs (e.

 ### API Access Extensions

-When a request reaches the Kubernetes API Server, it is first Authenticated, then Authorized, then subject to various types of Admission Control. See [Controlling Access to the Kubernetes API](/docs/reference/access-authn-authz/controlling-access/) for more on this flow.
+When a request reaches the Kubernetes API Server, it is first Authenticated, then Authorized, then subject to various types of Admission Control. See [Controlling Access to the Kubernetes API](/docs/concepts/security/controlling-access/) for more on this flow.

 Each of these steps offers extension points.

@@ -35,7 +35,7 @@ Kubernetes' {{< glossary_tooltip text="controllers" term_id="controller" >}}
 concept lets you extend the cluster's behaviour without modifying the code
 of Kubernetes itself.
 Operators are clients of the Kubernetes API that act as controllers for
-a [Custom Resource](/docs/concepts/api-extension/custom-resources/).
+a [Custom Resource](/docs/concepts/extend-kubernetes/api-extension/custom-resources/).

 ## An example Operator {#example}

@@ -229,7 +229,7 @@ The following example describes how to map secret values into application enviro

 * If you are familiar with {{< glossary_tooltip text="Helm Charts" term_id="helm-chart" >}}, [install Service Catalog using Helm](/docs/tasks/service-catalog/install-service-catalog-using-helm/) into your Kubernetes cluster. Alternatively, you can [install Service Catalog using the SC tool](/docs/tasks/service-catalog/install-service-catalog-using-sc/).
 * View [sample service brokers](https://github.com/openservicebrokerapi/servicebroker/blob/master/gettingStarted.md#sample-service-brokers).
-* Explore the [kubernetes-incubator/service-catalog](https://github.com/kubernetes-incubator/service-catalog) project.
+* Explore the [kubernetes-sigs/service-catalog](https://github.com/kubernetes-sigs/service-catalog) project.
 * View [svc-cat.io](https://svc-cat.io/docs/).


@@ -25,7 +25,7 @@ The Kubernetes API lets you query and manipulate the state of objects in the Kub
 Most operations can be performed through the
 [kubectl](/docs/reference/kubectl/overview/) command-line interface or other
 command-line tools, such as
-[kubeadm](/docs/reference/setup-tools/kubeadm/kubeadm/), which in turn use the
+[kubeadm](/docs/reference/setup-tools/kubeadm/), which in turn use the
 API. However, you can also access the API directly using REST calls.

 Consider using one of the [client libraries](/docs/reference/using-api/client-libraries/)
@@ -103,8 +103,8 @@ and behavior, and to enable controlling access to end-of-life and/or
 experimental APIs.

 To make it easier to evolve and to extend its API, Kubernetes implements
-[API groups](/docs/reference/using-api/api-overview/#api-groups) that can be
-[enabled or disabled](/docs/reference/using-api/api-overview/#enabling-or-disabling).
+[API groups](/docs/reference/using-api/#api-groups) that can be
+[enabled or disabled](/docs/reference/using-api/#enabling-or-disabling).

 API resources are distinguished by their API group, resource type, namespace
 (for namespaced resources), and name. The API server may serve the same
@@ -115,7 +115,7 @@ versions `v1` and `v1beta1` for the same resource. An object created by the
 `v1beta1` version can then be read, updated, and deleted by either the
 `v1beta1` or the `v1` versions.

-Refer to [API versions reference](/docs/reference/using-api/api-overview/#api-versioning)
+Refer to [API versions reference](/docs/reference/using-api/#api-versioning)
 for more details on the API version level definitions.

 ## API Extension
@@ -131,7 +131,7 @@ The Kubernetes API can be extended in one of two ways:

 - Learn how to extend the Kubernetes API by adding your own
  [CustomResourceDefinition](/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/).
- [Controlling API Access](/docs/reference/access-authn-authz/controlling-access/) describes
+- [Controlling Access To The Kubernetes API](/docs/concepts/security/controlling-access/) describes
  how the cluster manages authentication and authorization for API access.
 - Learn about API endpoints, resource types and samples by reading
  [API Reference](/docs/reference/kubernetes-api/).
@@ -38,7 +38,7 @@ Virtualization allows better utilization of resources in a physical server and a

 Each VM is a full machine running all the components, including its own operating system, on top of the virtualized hardware.

-**Container deployment era:** Containers are similar to VMs, but they have relaxed isolation properties to share the Operating System (OS) among the applications. Therefore, containers are considered lightweight. Similar to a VM, a container has its own filesystem, CPU, memory, process space, and more. As they are decoupled from the underlying infrastructure, they are portable across clouds and OS distributions.
+**Container deployment era:** Containers are similar to VMs, but they have relaxed isolation properties to share the Operating System (OS) among the applications. Therefore, containers are considered lightweight. Similar to a VM, a container has its own filesystem, share of CPU, memory, process space, and more. As they are decoupled from the underlying infrastructure, they are portable across clouds and OS distributions.

 Containers have become popular because they provide extra benefits, such as:

@@ -94,4 +94,3 @@ Kubernetes:

 *   Take a look at the [Kubernetes Components](/docs/concepts/overview/components/)
 *   Ready to [Get Started](/docs/setup/)?
-
@@ -91,9 +91,9 @@ and the `spec` format for a Deployment can be found in

 ## {{% heading "whatsnext" %}}

-* [Kubernetes API overview](/docs/reference/using-api/api-overview/) explains some more API concepts
 * Learn about the most important basic Kubernetes objects, such as [Pod](/docs/concepts/workloads/pods/).
-* Learn about [controllers](/docs/concepts/architecture/controller/) in Kubernetes
+* Learn about [controllers](/docs/concepts/architecture/controller/) in Kubernetes.
+* [Using the Kubernetes API](/docs/reference/using-api/) explains some more API concepts.



@@ -188,7 +188,7 @@ kubectl get pods -l 'environment,environment notin (frontend)'
 Some Kubernetes objects, such as [`services`](/docs/concepts/services-networking/service/)
 and [`replicationcontrollers`](/docs/concepts/workloads/controllers/replicationcontroller/),
 also use label selectors to specify sets of other resources, such as
-[pods](/docs/concepts/workloads/pods/pod/).
+[pods](/docs/concepts/workloads/pods/).

 #### Service and ReplicationController

@@ -140,13 +140,17 @@ For a complete example of authorizing a PodSecurityPolicy, see

 ### Troubleshooting

- The [Controller Manager](/docs/reference/command-line-tools-reference/kube-controller-manager/) must be run
-against [the secured API port](/docs/reference/access-authn-authz/controlling-access/),
-and must not have superuser permissions. Otherwise requests would bypass
-authentication and authorization modules, all PodSecurityPolicy objects would be
-allowed, and users would be able to create privileged containers. For more details
-on configuring Controller Manager authorization, see
-[Controller Roles](/docs/reference/access-authn-authz/rbac/#controller-roles).
+- The [controller manager](/docs/reference/command-line-tools-reference/kube-controller-manager/)
+  must be run against the secured API port and must not have superuser permissions. See
+  [Controlling Access to the Kubernetes API](/docs/concepts/security/controlling-access)
+  to learn about API server access controls.  
+  If the controller manager connected through the trusted API port (also known as the
+  `localhost` listener), requests would bypass authentication and authorization modules;
+  all PodSecurityPolicy objects would be allowed, and users would be able to create grant
+  themselves the ability to create privileged containers.
+
+  For more details on configuring controller manager authorization, see
+  [Controller Roles](/docs/reference/access-authn-authz/rbac/#controller-roles).

 ## Policy Order

@@ -18,17 +18,17 @@ Resource quotas are a tool for administrators to address this concern.
 A resource quota, defined by a `ResourceQuota` object, provides constraints that limit
 aggregate resource consumption per namespace.  It can limit the quantity of objects that can
 be created in a namespace by type, as well as the total amount of compute resources that may
-be consumed by resources in that project.
+be consumed by resources in that namespace.

 Resource quotas work like this:

 - Different teams work in different namespaces.  Currently this is voluntary, but
  support for making this mandatory via ACLs is planned.

- The administrator creates one `ResourceQuota` for each namespace.
+- The administrator creates one ResourceQuota for each namespace.

 - Users create resources (pods, services, etc.) in the namespace, and the quota system
-  tracks usage to ensure it does not exceed hard resource limits defined in a `ResourceQuota`.
+  tracks usage to ensure it does not exceed hard resource limits defined in a ResourceQuota.

 - If creating or updating a resource violates a quota constraint, the request will fail with HTTP
  status code `403 FORBIDDEN` with a message explaining the constraint that would have been violated.
@@ -40,7 +40,7 @@ Resource quotas work like this:
  See the [walkthrough](/docs/tasks/administer-cluster/manage-resources/quota-memory-cpu-namespace/)
  for an example of how to avoid this problem.

-The name of a `ResourceQuota` object must be a valid
+The name of a ResourceQuota object must be a valid
 [DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names).

 Examples of policies that could be created using namespaces and quotas are:
@@ -58,11 +58,11 @@ Neither contention nor changes to quota will affect already created resources.
 ## Enabling Resource Quota

 Resource Quota support is enabled by default for many Kubernetes distributions.  It is
-enabled when the apiserver `--enable-admission-plugins=` flag has `ResourceQuota` as
+enabled when the API server `--enable-admission-plugins=` flag has `ResourceQuota` as
 one of its arguments.

 A resource quota is enforced in a particular namespace when there is a
-`ResourceQuota` in that namespace.
+ResourceQuota in that namespace.

 ## Compute Resource Quota

@@ -76,6 +76,10 @@ The following resource types are supported:
 | `limits.memory` | Across all pods in a non-terminal state, the sum of memory limits cannot exceed this value. |
 | `requests.cpu` | Across all pods in a non-terminal state, the sum of CPU requests cannot exceed this value. |
 | `requests.memory` | Across all pods in a non-terminal state, the sum of memory requests cannot exceed this value. |
+| `hugepages-<size>` | Across all pods in a non-terminal state, the number of
+huge page requests of the specified size cannot exceed this value. |
+| `cpu` | Same as `requests.cpu` |
+| `memory` | Same as `requests.memory` |

 ### Resource Quota For Extended Resources

@@ -103,8 +107,8 @@ In addition, you can limit consumption of storage resources based on associated
 | Resource Name | Description |
 | --------------------- | ----------------------------------------------------------- |
 | `requests.storage` | Across all persistent volume claims, the sum of storage requests cannot exceed this value. |
-| `persistentvolumeclaims` | The total number of [persistent volume claims](/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims) that can exist in the namespace. |
-| `<storage-class-name>.storageclass.storage.k8s.io/requests.storage` | Across all persistent volume claims associated with the storage-class-name, the sum of storage requests cannot exceed this value. |
+| `persistentvolumeclaims` | The total number of [PersistentVolumeClaims](/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims) that can exist in the namespace. |
+| `<storage-class-name>.storageclass.storage.k8s.io/requests.storage` | Across all persistent volume claims associated with the `<storage-class-name>`, the sum of storage requests cannot exceed this value. |
 | `<storage-class-name>.storageclass.storage.k8s.io/persistentvolumeclaims` | Across all persistent volume claims associated with the storage-class-name, the total number of [persistent volume claims](/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims) that can exist in the namespace. |

 For example, if an operator wants to quota storage with `gold` storage class separate from `bronze` storage class, the operator can
@@ -119,12 +123,15 @@ In release 1.8, quota support for local ephemeral storage is added as an alpha f
 | ------------------------------- |----------------------------------------------------------- |
 | `requests.ephemeral-storage` | Across all pods in the namespace, the sum of local ephemeral storage requests cannot exceed this value. |
 | `limits.ephemeral-storage` | Across all pods in the namespace, the sum of local ephemeral storage limits cannot exceed this value. |
+| `ephemeral-storage` | Same as `requests.ephemeral-storage`. |

 ## Object Count Quota

-The 1.9 release added support to quota all standard namespaced resource types using the following syntax:
+You can set quota for the total number of certain resources of all standard,
+namespaced resource types using the following syntax:

-* `count/<resource>.<group>`
+* `count/<resource>.<group>` for resources from non-core groups
+* `count/<resource>` for resources from the core group

 Here is an example set of resources users may want to put under object count quota:

@@ -138,33 +145,30 @@ Here is an example set of resources users may want to put under object count quo
 * `count/statefulsets.apps`
 * `count/jobs.batch`
 * `count/cronjobs.batch`
-* `count/deployments.extensions`

-The 1.15 release added support for custom resources using the same syntax.
+The same syntax can be used for custom resources.
 For example, to create a quota on a `widgets` custom resource in the `example.com` API group, use `count/widgets.example.com`.

 When using `count/*` resource quota, an object is charged against the quota if it exists in server storage.
 These types of quotas are useful to protect against exhaustion of storage resources.  For example, you may
-want to quota the number of secrets in a server given their large size.  Too many secrets in a cluster can
-actually prevent servers and controllers from starting!  You may choose to quota jobs to protect against
-a poorly configured cronjob creating too many jobs in a namespace causing a denial of service.
-
-Prior to the 1.9 release, it was possible to do generic object count quota on a limited set of resources.
-In addition, it is possible to further constrain quota for particular resources by their type.
+want to limit the number of Secrets in a server given their large size. Too many Secrets in a cluster can
+actually prevent servers and controllers from starting. You can set a quota for Jobs to protect against
+a poorly configured CronJob. CronJobs that create too many Jobs in a namespace can lead to a denial of service.

+It is also possible to do generic object count quota on a limited set of resources.
 The following types are supported:

 | Resource Name | Description |
 | ------------------------------- | ------------------------------------------------- |
-| `configmaps` | The total number of config maps that can exist in the namespace. |
-| `persistentvolumeclaims` | The total number of [persistent volume claims](/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims) that can exist in the namespace. |
-| `pods` | The total number of pods in a non-terminal state that can exist in the namespace.  A pod is in a terminal state if `.status.phase in (Failed, Succeeded)` is true.  |
-| `replicationcontrollers` | The total number of replication controllers that can exist in the namespace. |
-| `resourcequotas` | The total number of [resource quotas](/docs/reference/access-authn-authz/admission-controllers/#resourcequota) that can exist in the namespace. |
-| `services` | The total number of services that can exist in the namespace. |
-| `services.loadbalancers` | The total number of services of type load balancer that can exist in the namespace. |
-| `services.nodeports` | The total number of services of type node port that can exist in the namespace. |
-| `secrets` | The total number of secrets that can exist in the namespace. |
+| `configmaps` | The total number of ConfigMaps that can exist in the namespace. |
+| `persistentvolumeclaims` | The total number of [PersistentVolumeClaims](/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims) that can exist in the namespace. |
+| `pods` | The total number of Pods in a non-terminal state that can exist in the namespace.  A pod is in a terminal state if `.status.phase in (Failed, Succeeded)` is true.  |
+| `replicationcontrollers` | The total number of ReplicationControllers that can exist in the namespace. |
+| `resourcequotas` | The total number of ResourceQuotas that can exist in the namespace. |
+| `services` | The total number of Services that can exist in the namespace. |
+| `services.loadbalancers` | The total number of Services of type `LoadBalancer` that can exist in the namespace. |
+| `services.nodeports` | The total number of Services of type `NodePort` that can exist in the namespace. |
+| `secrets` | The total number of Secrets that can exist in the namespace. |

 For example, `pods` quota counts and enforces a maximum on the number of `pods`
 created in a single namespace that are not terminal. You might want to set a `pods`
@@ -173,7 +177,7 @@ exhausts the cluster's supply of Pod IPs.

 ## Quota Scopes

-Each quota can have an associated set of scopes.  A quota will only measure usage for a resource if it matches
+Each quota can have an associated set of `scopes`. A quota will only measure usage for a resource if it matches
 the intersection of enumerated scopes.

 When a scope is added to the quota, it limits the number of resources it supports to those that pertain to the scope.
@@ -185,22 +189,60 @@ Resources specified on the quota outside of the allowed set results in a validat
 | `NotTerminating` | Match pods where `.spec.activeDeadlineSeconds is nil` |
 | `BestEffort` | Match pods that have best effort quality of service. |
 | `NotBestEffort` | Match pods that do not have best effort quality of service. |
+| `PriorityClass` | Match pods that references the specified [priority class](/docs/concepts/configuration/pod-priority-preemption). |

-The `BestEffort` scope restricts a quota to tracking the following resource: `pods`
+The `BestEffort` scope restricts a quota to tracking the following resource:

-The `Terminating`, `NotTerminating`, and `NotBestEffort` scopes restrict a quota to tracking the following resources:
-
-* `cpu`
-* `limits.cpu`
-* `limits.memory`
-* `memory`
 * `pods`
+
+The `Terminating`, `NotTerminating`, `NotBestEffort` and `PriorityClass`
+scopes restrict a quota to tracking the following resources:
+
+* `pods`
+* `cpu`
+* `memory`
 * `requests.cpu`
 * `requests.memory`
+* `limits.cpu`
+* `limits.memory`
+
+Note that you cannot specify both the `Terminating` and the `NotTerminating`
+scopes in the same quota, and you cannot specify both the `BestEffort` and
+`NotBestEffort` scopes in the same quota either.
+
+The `scopeSelector` supports the following values in the `operator` field:
+
+* `In`
+* `NotIn`
+* `Exists`
+* `DoesNotExist`
+
+When using one of the following values as the `scopeName` when defining the
+`scopeSelector`, the `operator` must be `Exists`. 
+
+* `Terminating`
+* `NotTerminating`
+* `BestEffort`
+* `NotBestEffort`
+
+If the `operator` is `In` or `NotIn`, the `values` field must have at least
+one value. For example:
+
+```yaml
+  scopeSelector:
+    matchExpressions:
+      - scopeName: PriorityClass
+        operator: In
+        values:
+          - middle
+```
+
+If the `operator` is `Exists` or `DoesNotExist`, the `values field must *NOT* be
+specified.

 ### Resource Quota Per PriorityClass

-{{< feature-state for_k8s_version="v1.12" state="beta" >}}
+{{< feature-state for_k8s_version="v1.17" state="stable" >}}

 Pods can be created at a specific [priority](/docs/concepts/configuration/pod-priority-preemption/#pod-priority).
 You can control a pod's consumption of system resources based on a pod's priority, by using the `scopeSelector`
@@ -283,7 +325,7 @@ Apply the YAML using `kubectl create`.
 kubectl create -f ./quota.yml
 ```

-```shell
+```
 resourcequota/pods-high created
 resourcequota/pods-medium created
 resourcequota/pods-low created
@@ -295,7 +337,7 @@ Verify that `Used` quota is `0` using `kubectl describe quota`.
 kubectl describe quota
 ```

-```shell
+```
 Name:       pods-high
 Namespace:  default
 Resource    Used  Hard
@@ -360,7 +402,7 @@ the other two quotas are unchanged.
 kubectl describe quota
 ```

-```shell
+```
 Name:       pods-high
 Namespace:  default
 Resource    Used  Hard
@@ -388,13 +430,6 @@ memory      0     20Gi
 pods        0     10
 ```

-`scopeSelector` supports the following values in the `operator` field:
-
-* `In`
-* `NotIn`
-* `Exist`
-* `DoesNotExist`
-
 ## Requests compared to Limits {#requests-vs-limits}

 When allocating compute resources, each container may specify a request and a limit value for either CPU or memory.
@@ -458,7 +493,7 @@ kubectl create -f ./object-counts.yaml --namespace=myspace
 kubectl get quota --namespace=myspace
 ```

-```shell
+```
 NAME                    AGE
 compute-resources       30s
 object-counts           32s
@@ -468,7 +503,7 @@ object-counts           32s
 kubectl describe quota compute-resources --namespace=myspace
 ```

-```shell
+```
 Name:                    compute-resources
 Namespace:               myspace
 Resource                 Used  Hard
@@ -484,7 +519,7 @@ requests.nvidia.com/gpu  0     4
 kubectl describe quota object-counts --namespace=myspace
 ```

-```shell
+```
 Name:                   object-counts
 Namespace:              myspace
 Resource                Used    Hard
@@ -506,7 +541,7 @@ kubectl create namespace myspace
 ```

 ```shell
-kubectl create quota test --hard=count/deployments.extensions=2,count/replicasets.extensions=4,count/pods=3,count/secrets=4 --namespace=myspace
+kubectl create quota test --hard=count/deployments.apps=2,count/replicasets.apps=4,count/pods=3,count/secrets=4 --namespace=myspace
 ```

 ```shell
@@ -517,29 +552,29 @@ kubectl create deployment nginx --image=nginx --namespace=myspace --replicas=2
 kubectl describe quota --namespace=myspace
 ```

-```shell
+```
 Name:                         test
 Namespace:                    myspace
 Resource                      Used  Hard
 --------                      ----  ----
-count/deployments.extensions  1     2
+count/deployments.apps        1     2
 count/pods                    2     3
-count/replicasets.extensions  1     4
+count/replicasets.apps        1     4
 count/secrets                 1     4
 ```

 ## Quota and Cluster Capacity

-`ResourceQuotas` are independent of the cluster capacity. They are
+ResourceQuotas are independent of the cluster capacity. They are
 expressed in absolute units.  So, if you add nodes to your cluster, this does *not*
 automatically give each namespace the ability to consume more resources.

 Sometimes more complex policies may be desired, such as:

-  - Proportionally divide total cluster resources among several teams.
-  - Allow each tenant to grow resource usage as needed, but have a generous
-    limit to prevent accidental resource exhaustion.
-  - Detect demand from one namespace, add nodes, and increase quota.
+- Proportionally divide total cluster resources among several teams.
+- Allow each tenant to grow resource usage as needed, but have a generous
+  limit to prevent accidental resource exhaustion.
+- Detect demand from one namespace, add nodes, and increase quota.

 Such policies could be implemented using `ResourceQuotas` as building blocks, by
 writing a "controller" that watches the quota usage and adjusts the quota
@@ -550,14 +585,16 @@ restrictions around nodes: pods from several namespaces may run on the same node

 ## Limit Priority Class consumption by default

-It may be desired that pods at a particular priority, eg. "cluster-services", should be allowed in a namespace, if and only if, a matching quota object exists.
+It may be desired that pods at a particular priority, eg. "cluster-services", 
+should be allowed in a namespace, if and only if, a matching quota object exists.

-With this mechanism, operators will be able to restrict usage of certain high priority classes to a limited number of namespaces and not every namespace will be able to consume these priority classes by default.
+With this mechanism, operators are able to restrict usage of certain high
+priority classes to a limited number of namespaces and not every namespace
+will be able to consume these priority classes by default.

-To enforce this, kube-apiserver flag `--admission-control-config-file` should be used to pass path to the following configuration file:
+To enforce this, `kube-apiserver` flag `--admission-control-config-file` should be
+used to pass path to the following configuration file:

-{{< tabs name="example1" >}}
-{{% tab name="apiserver.config.k8s.io/v1" %}}
 ```yaml
 apiVersion: apiserver.config.k8s.io/v1
 kind: AdmissionConfiguration
@@ -573,30 +610,10 @@ plugins:
        operator: In
        values: ["cluster-services"]
 ```
-{{% /tab %}}
-{{% tab name="apiserver.k8s.io/v1alpha1" %}}
-```yaml
-# Deprecated in v1.17 in favor of apiserver.config.k8s.io/v1
-apiVersion: apiserver.k8s.io/v1alpha1
-kind: AdmissionConfiguration
-plugins:
- name: "ResourceQuota"
-  configuration:
-    # Deprecated in v1.17 in favor of apiserver.config.k8s.io/v1, ResourceQuotaConfiguration
-    apiVersion: resourcequota.admission.k8s.io/v1beta1
-    kind: Configuration
-    limitedResources:
-    - resource: pods
-      matchScopes:
-      - scopeName: PriorityClass
-        operator: In
-        values: ["cluster-services"]
-```
-{{% /tab %}}
-{{< /tabs >}}

 Now, "cluster-services" pods will be allowed in only those namespaces where a quota object with a matching `scopeSelector` is present.
 For example:
+
 ```yaml
    scopeSelector:
      matchExpressions:
@@ -605,13 +622,9 @@ For example:
        values: ["cluster-services"]
 ```

-See [LimitedResources](https://github.com/kubernetes/kubernetes/pull/36765) and [Quota support for priority class design doc](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/scheduling/pod-priority-resourcequota.md) for more information.
-
-## Example
-
-See a [detailed example for how to use resource quota](/docs/tasks/administer-cluster/quota-api-object/).
-
 ## {{% heading "whatsnext" %}}

 - See [ResourceQuota design doc](https://git.k8s.io/community/contributors/design-proposals/resource-management/admission_control_resource_quota.md) for more information.
-
+- See a [detailed example for how to use resource quota](/docs/tasks/administer-cluster/quota-api-object/).
+- Read [Quota support for priority class design doc](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/scheduling/pod-priority-resourcequota.md).
+- See [LimitedResources](https://github.com/kubernetes/kubernetes/pull/36765)
@@ -79,7 +79,7 @@ of the scheduler:


 1. [Scheduling Policies](/docs/reference/scheduling/policies) allow you to configure _Predicates_ for filtering and _Priorities_ for scoring.
-1. [Scheduling Profiles](/docs/reference/scheduling/profiles) allow you to configure Plugins that implement different scheduling stages, including: `QueueSort`, `Filter`, `Score`, `Bind`, `Reserve`, `Permit`, and others. You can also configure the kube-scheduler to run different profiles.
+1. [Scheduling Profiles](/docs/reference/scheduling/config/#profiles) allow you to configure Plugins that implement different scheduling stages, including: `QueueSort`, `Filter`, `Score`, `Bind`, `Reserve`, `Permit`, and others. You can also configure the kube-scheduler to run different profiles.


 ## {{% heading "whatsnext" %}}
@@ -89,7 +89,7 @@ of the scheduler:
 * Read the [reference documentation](/docs/reference/command-line-tools-reference/kube-scheduler/) for kube-scheduler
 * Learn about [configuring multiple schedulers](/docs/tasks/extend-kubernetes/configure-multiple-schedulers/)
 * Learn about [topology management policies](/docs/tasks/administer-cluster/topology-manager/)
-* Learn about [Pod Overhead](/docs/concepts/configuration/pod-overhead/)
+* Learn about [Pod Overhead](/docs/concepts/scheduling-eviction/pod-overhead/)
 * Learn about scheduling of Pods that use volumes in:
  * [Volume Topology Support](/docs/concepts/storage/storage-classes/#volume-binding-mode)
  * [Storage Capacity Tracking](/docs/concepts/storage/storage-capacity/)
@@ -18,7 +18,7 @@ scheduling "core" simple and maintainable. Refer to the [design proposal of the
 scheduling framework][kep] for more technical information on the design of the
 framework.

-[kep]: https://github.com/kubernetes/enhancements/blob/master/keps/sig-scheduling/20180409-scheduling-framework.md
+[kep]: https://github.com/kubernetes/enhancements/blob/master/keps/sig-scheduling/624-scheduling-framework/README.md



@@ -4,7 +4,6 @@ reviewers:
 - lavalamp
 title: Controlling Access to the Kubernetes API
 content_type: concept
-weight: 5
 ---

 <!-- overview -->
@@ -12,7 +11,7 @@ This page provides an overview of controlling access to the Kubernetes API.


 <!-- body -->
-Users [access the API](/docs/tasks/access-application-cluster/access-cluster/) using `kubectl`,
+Users access the [Kubernetes API](/docs/concepts/overview/kubernetes-api/) using `kubectl`,
 client libraries, or by making REST requests.  Both human users and
 [Kubernetes service accounts](/docs/tasks/configure-pod-container/configure-service-account/) can be
 authorized for API access.
@@ -21,45 +20,46 @@ following diagram:

 ![Diagram of request handling steps for Kubernetes API request](/images/docs/admin/access-control-overview.svg)

-## Transport Security
+## Transport security

-In a typical Kubernetes cluster, the API serves on port 443.
-The API server presents a certificate. This certificate is
-often self-signed, so `$USER/.kube/config` on the user's machine typically
-contains the root certificate for the API server's certificate, which when specified
-is used in place of the system default root certificate.  This certificate is typically
-automatically written into your `$USER/.kube/config` when you create a cluster yourself
-using `kube-up.sh`.  If the cluster has multiple users, then the creator needs to share
-the certificate with other users.
+In a typical Kubernetes cluster, the API serves on port 443, protected by TLS.
+The API server presents a certificate. This certificate may be signed using
+a private certificate authority (CA), or based on a public key infrastructure linked
+to a generally recognized CA.
+
+If your cluster uses a private certificate authority, you need a copy of that CA
+certifcate configured into your `~/.kube/config` on the client, so that you can
+trust the connection and be confident it was not intercepted.
+
+Your client can present a TLS client certificate at this stage.

 ## Authentication

 Once TLS is established, the HTTP request moves to the Authentication step.
 This is shown as step **1** in the diagram.
 The cluster creation script or cluster admin configures the API server to run
-one or more Authenticator Modules.
-Authenticators are described in more detail [here](/docs/reference/access-authn-authz/authentication/).
+one or more Authenticator modules.
+Authenticators are described in more detail in
+[Authentication](/docs/reference/access-authn-authz/authentication/).

-The input to the authentication step is the entire HTTP request, however, it typically
+The input to the authentication step is the entire HTTP request; however, it typically
 just examines the headers and/or client certificate.

-Authentication modules include Client Certificates, Password, and Plain Tokens,
-Bootstrap Tokens, and JWT Tokens (used for service accounts).
+Authentication modules include client certificates, password, and plain tokens,
+bootstrap tokens, and JSON Web Tokens (used for service accounts).

 Multiple authentication modules can be specified, in which case each one is tried in sequence,
 until one of them succeeds.

-On GCE, Client Certificates, Password, Plain Tokens, and JWT Tokens are all enabled.
-
 If the request cannot be authenticated, it is rejected with HTTP status code 401.
 Otherwise, the user is authenticated as a specific `username`, and the user name
 is available to subsequent steps to use in their decisions.  Some authenticators
 also provide the group memberships of the user, while other authenticators
 do not.

-While Kubernetes uses `usernames` for access control decisions and in request logging,
-it does not have a `user` object nor does it store usernames or other information about
-users in its object store.
+While Kubernetes uses usernames for access control decisions and in request logging,
+it does not have a `User` object nor does it store usernames or other information about
+users in its API.

 ## Authorization

@@ -101,16 +101,16 @@ If Bob makes a request to write (`create` or `update`) to the objects in the `pr

 Kubernetes authorization requires that you use common REST attributes to interact with existing organization-wide or cloud-provider-wide access control systems. It is important to use REST formatting because these control systems might interact with other APIs besides the Kubernetes API.

-Kubernetes supports multiple authorization modules, such as ABAC mode, RBAC Mode, and Webhook mode. When an administrator creates a cluster, they configured the authorization modules that should be used in the API server. If more than one authorization modules are configured, Kubernetes checks each module, and if any module authorizes the request, then the request can proceed. If all of the modules deny the request, then the request is denied (HTTP status code 403).
+Kubernetes supports multiple authorization modules, such as ABAC mode, RBAC Mode, and Webhook mode. When an administrator creates a cluster, they configure the authorization modules that should be used in the API server. If more than one authorization modules are configured, Kubernetes checks each module, and if any module authorizes the request, then the request can proceed. If all of the modules deny the request, then the request is denied (HTTP status code 403).

-To learn more about Kubernetes authorization, including details about creating policies using the supported authorization modules, see [Authorization Overview](/docs/reference/access-authn-authz/authorization/).
+To learn more about Kubernetes authorization, including details about creating policies using the supported authorization modules, see [Authorization](/docs/reference/access-authn-authz/authorization/).


-## Admission Control
+## Admission control

-Admission Control Modules are software modules that can modify or reject requests.
-In addition to the attributes available to Authorization Modules, Admission
-Control Modules can access the contents of the object that is being created or modified.
+Admission Control modules are software modules that can modify or reject requests.
+In addition to the attributes available to Authorization modules, Admission
+Control modules can access the contents of the object that is being created or modified.

 Admission controllers act on requests that create, modify, delete, or connect to (proxy) an object.
 Admission controllers do not act on requests that merely read objects.
@@ -118,26 +118,26 @@ When multiple admission controllers are configured, they are called in order.

 This is shown as step **3** in the diagram.

-Unlike Authentication and Authorization Modules, if any admission controller module
+Unlike Authentication and Authorization modules, if any admission controller module
 rejects, then the request is immediately rejected.

 In addition to rejecting objects, admission controllers can also set complex defaults for
 fields.

-The available Admission Control Modules are described [here](/docs/reference/access-authn-authz/admission-controllers/).
+The available Admission Control modules are described in [Admission Controllers](/docs/reference/access-authn-authz/admission-controllers/).

 Once a request passes all admission controllers, it is validated using the validation routines
 for the corresponding API object, and then written to the object store (shown as step **4**).


-## API Server Ports and IPs
+## API server ports and IPs

 The previous discussion applies to requests sent to the secure port of the API server
 (the typical case).  The API server can actually serve on 2 ports:

 By default the Kubernetes API server serves HTTP on 2 ports:

-  1. `Localhost Port`:
+  1. `localhost` port:

      - is intended for testing and bootstrap, and for other components of the master node
        (scheduler, controller-manager) to talk to the API
@@ -148,7 +148,7 @@ By default the Kubernetes API server serves HTTP on 2 ports:
      - request handled by admission control module(s).
      - protected by need to have host access

-  2. `Secure Port`:
+  2. “Secure port”:

      - use whenever possible
      - uses TLS.  Set cert with `--tls-cert-file` and key with `--tls-private-key-file` flag.
@@ -158,8 +158,27 @@ By default the Kubernetes API server serves HTTP on 2 ports:
      - request handled by admission control module(s).
      - authentication and authorization modules run.

-When the cluster is created by `kube-up.sh`, on Google Compute Engine (GCE),
-and on several other cloud providers, the API server serves on port 443.  On
-GCE, a firewall rule is configured on the project to allow external HTTPS
-access to the API. Other cluster setup methods vary.
+## {{% heading "whatsnext" %}}

+Read more documentation on authentication, authorization and API access control:
+
+- [Authenticating](/docs/reference/access-authn-authz/authentication/)
+   - [Authenticating with Bootstrap Tokens](/docs/reference/access-authn-authz/bootstrap-tokens/)
+- [Admission Controllers](/docs/reference/access-authn-authz/admission-controllers/)
+   - [Dynamic Admission Control](/docs/reference/access-authn-authz/extensible-admission-controllers/)
+- [Authorization](/docs/reference/access-authn-authz/authorization/)
+   - [Role Based Access Control](/docs/reference/access-authn-authz/rbac/)
+   - [Attribute Based Access Control](/docs/reference/access-authn-authz/abac/)
+   - [Node Authorization](/docs/reference/access-authn-authz/node/)
+   - [Webhook Authorization](/docs/reference/access-authn-authz/webhook/)
+- [Certificate Signing Requests](/docs/reference/access-authn-authz/certificate-signing-requests/)
+   - including [CSR approval](/docs/reference/access-authn-authz/certificate-signing-requests/#approval-rejection)
+     and [certificate signing](/docs/reference/access-authn-authz/certificate-signing-requests/#signing)
+- Service accounts
+  - [Developer guide](/docs/tasks/configure-pod-container/configure-service-account/)
+  - [Administration](/docs/reference/access-authn-authz/service-accounts-admin/)
+
+You can learn about:
+- how Pods can use
+  [Secrets](/docs/concepts/configuration/secret/#service-accounts-automatically-create-and-attach-secrets-with-api-credentials)
+  to obtain API credentials.
@@ -103,7 +103,7 @@ areas of security concerns and recommendations for securing workloads running in
 Area of Concern for Workload Security | Recommendation |
 ------------------------------ | --------------------- |
 RBAC Authorization (Access to the Kubernetes API) | https://kubernetes.io/docs/reference/access-authn-authz/rbac/
-Authentication | https://kubernetes.io/docs/reference/access-authn-authz/controlling-access/
+Authentication | https://kubernetes.io/docs/concepts/security/controlling-access/
 Application secrets management (and encrypting them in etcd at rest) | https://kubernetes.io/docs/concepts/configuration/secret/ <br> https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/
 Pod Security Policies | https://kubernetes.io/docs/concepts/policy/pod-security-policy/
 Quality of Service (and Cluster resource management) | https://kubernetes.io/docs/tasks/configure-pod-container/quality-service-pod/
@@ -147,8 +147,8 @@ Learn about related Kubernetes security topics:

 * [Pod security standards](/docs/concepts/security/pod-security-standards/)
 * [Network policies for Pods](/docs/concepts/services-networking/network-policies/)
+* [Controlling Access to the Kubernetes API](/docs/concepts/security/controlling-access)
 * [Securing your cluster](/docs/tasks/administer-cluster/securing-a-cluster/)
-* [API access control](/docs/reference/access-authn-authz/controlling-access/)
 * [Data encryption in transit](/docs/tasks/tls/managing-tls-in-a-cluster/) for the control plane
 * [Data encryption at rest](/docs/tasks/administer-cluster/encrypt-data/)
 * [Secrets in Kubernetes](/docs/concepts/configuration/secret/)
@@ -98,7 +98,7 @@ On cloud providers which support IPv6 enabled external load balancers, setting t

 ## Egress Traffic

-The use of publicly routable and non-publicly routable IPv6 address blocks is acceptable provided the underlying {{< glossary_tooltip text="CNI" term_id="cni" >}} provider is able to implement the transport. If you have a Pod that uses non-publicly routable IPv6 and want that Pod to reach off-cluster destinations (eg. the public Internet), you must set up IP masquerading for the egress traffic and any replies. The [ip-masq-agent](https://github.com/kubernetes-incubator/ip-masq-agent) is dual-stack aware, so you can use ip-masq-agent for IP masquerading on dual-stack clusters.
+The use of publicly routable and non-publicly routable IPv6 address blocks is acceptable provided the underlying {{< glossary_tooltip text="CNI" term_id="cni" >}} provider is able to implement the transport. If you have a Pod that uses non-publicly routable IPv6 and want that Pod to reach off-cluster destinations (eg. the public Internet), you must set up IP masquerading for the egress traffic and any replies. The [ip-masq-agent](https://github.com/kubernetes-sigs/ip-masq-agent) is dual-stack aware, so you can use ip-masq-agent for IP masquerading on dual-stack clusters.

 ## Known Issues

@@ -413,6 +413,13 @@ secure the channel from the client to the load balancer using TLS. You need to m
 sure the TLS secret you created came from a certificate that contains a Common
 Name (CN), also known as a Fully Qualified Domain Name (FQDN) for `https-example.foo.com`.

+{{< note >}}
+Keep in mind that TLS will not work on the default rule because the
+certificates would have to be issued for all the possible sub-domains. Therefore,
+`hosts` in the `tls` section need to explicitly match the `host` in the `rules`
+section.
+{{< /note >}}
+
 {{< codenew file="service/networking/tls-example-ingress.yaml" >}}

 {{< note >}}
@@ -46,10 +46,9 @@ different purposes:
  [downwardAPI](/docs/concepts/storage/volumes/#downwardapi),
  [secret](/docs/concepts/storage/volumes/#secret): inject different
  kinds of Kubernetes data into a Pod
- [CSI ephemeral
-  volumes](docs/concepts/storage/volumes/#csi-ephemeral-volumes):
-  similar to the previous volume kinds, but provided by special [CSI
-  drivers](https://github.com/container-storage-interface/spec/blob/master/spec.md)
+- [CSI ephemeral volumes](#csi-ephemeral-volume):
+  similar to the previous volume kinds, but provided by special
+  [CSI drivers](https://github.com/container-storage-interface/spec/blob/master/spec.md)
  which specifically [support this feature](https://kubernetes-csi.github.io/docs/drivers.html)
 - [generic ephemeral volumes](#generic-ephemeral-volumes), which
  can be provided by all storage drivers that also support persistent volumes
@@ -181,8 +180,8 @@ spec:

 ### Lifecycle and PersistentVolumeClaim

-The key design idea is that the [parameters for a
-volume claim](/docs/reference/generated/kubernetes-api/#ephemeralvolumesource-v1alpha1-core)
+The key design idea is that the
+[parameters for a volume claim](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#ephemeralvolumesource-v1alpha1-core)
 are allowed inside a volume source of the Pod. Labels, annotations and
 the whole set of fields for a PersistentVolumeClaim are supported. When such a Pod gets
 created, the ephemeral volume controller then creates an actual PersistentVolumeClaim
@@ -183,27 +183,29 @@ By specifying a PersistentVolume in a PersistentVolumeClaim, you declare a bindi
 If the PersistentVolume exists and has not reserved PersistentVolumeClaims through its `claimRef` field, then the PersistentVolume and PersistentVolumeClaim will be bound.

 The binding happens regardless of some volume matching criteria, including node affinity.
-The control plane still checks that [storage class](https://kubernetes.io/docs/concepts/storage/storage-classes/), access modes, and requested storage size are valid.
+The control plane still checks that [storage class](/docs/concepts/storage/storage-classes/), access modes, and requested storage size are valid.

-```
+```yaml
 apiVersion: v1
 kind: PersistentVolumeClaim
 metadata:
  name: foo-pvc
  namespace: foo
 spec:
+  storageClassName: "" # Empty string must be explicitly set otherwise default StorageClass will be set
  volumeName: foo-pv
  ...
 ```

 This method does not guarantee any binding privileges to the PersistentVolume. If other PersistentVolumeClaims could use the PV that you specify, you first need to reserve that storage volume. Specify the relevant PersistentVolumeClaim in the `claimRef` field of the PV so that other PVCs can not bind to it.

-```
+```yaml
 apiVersion: v1
 kind: PersistentVolume
 metadata:
  name: foo-pv
 spec:
+  storageClassName: ""
  claimRef:
    name: foo-pvc
    namespace: foo
@@ -94,7 +94,7 @@ run, what volume plugin it uses (including Flex), etc. The repository
 [kubernetes-sigs/sig-storage-lib-external-provisioner](https://github.com/kubernetes-sigs/sig-storage-lib-external-provisioner)
 houses a library for writing external provisioners that implements the bulk of
 the specification. Some external provisioners are listed under the repository
-[kubernetes-incubator/external-storage](https://github.com/kubernetes-incubator/external-storage).
+[kubernetes-sigs/external-storage](https://github.com/kubernetes-sigs/external-dns).

 For example, NFS doesn't provide an internal provisioner, but an external
 provisioner can be used. There are also cases when 3rd party storage
@@ -18,7 +18,7 @@ Container starts with a clean state.  Second, when running Containers together
 in a `Pod` it is often necessary to share files between those Containers.  The
 Kubernetes `Volume` abstraction solves both of these problems.

-Familiarity with [Pods](/docs/concepts/workloads/pods/pod/) is suggested.
+Familiarity with [Pods](/docs/concepts/workloads/pods/) is suggested.

 <!-- body -->

@@ -52,8 +52,7 @@ field) and where to mount those into Containers (the
 field).

 A process in a container sees a filesystem view composed from their Docker
-image and volumes.  The [Docker
-image](https://docs.docker.com/userguide/dockerimages/) is at the root of the
+image and volumes. The [Docker image](https://docs.docker.com/userguide/dockerimages/) is at the root of the
 filesystem hierarchy, and any volumes are mounted at the specified paths within
 the image.  Volumes can not mount onto other volumes or have hard links to
 other volumes.  Each Container in the Pod must independently specify where to
@@ -3,5 +3,55 @@ title: "Workloads"
 weight: 50
 description: >
  Understand Pods, the smallest deployable compute object in Kubernetes, and the higher-level abstractions that help you to run them.
+no_list: true
 ---

+{{< glossary_definition term_id="workload" length="short" >}}
+Whether your workload is a single component or several that work together, on Kubernetes you run
+it inside a set of [Pods](/docs/concepts/workloads/pods).
+In Kubernetes, a Pod represents a set of running {{< glossary_tooltip text="containers" term_id="container" >}}
+on your cluster.
+
+A Pod has a defined lifecycle. For example, once a Pod is running in your cluster then
+a critical failure on the {{< glossary_tooltip text="node" term_id="node" >}} where that
+Pod is running means that all the Pods on that node fail. Kubernetes treats that level
+of failure as final: you would need to create a new Pod even if the node later recovers.
+
+However, to make life considerably easier, you don't need to manage each Pod directly.
+Instead, you can use _workload resources_ that manage a set of Pods on your behalf.
+These resources configure {{< glossary_tooltip term_id="controller" text="controllers" >}}
+that make sure the right number of the right kind of Pod are running, to match the state
+you specified.
+
+Those workload resources include:
+
+* [Deployment](/docs/concepts/workloads/controllers/deployment/) and [ReplicaSet](/docs/concepts/workloads/controllers/replicaset/)
+  (replacing the legacy resource {{< glossary_tooltip text="ReplicationController" term_id="replication-controller" >}});
+* [StatefulSet](/docs/concepts/workloads/controllers/statefulset/);
+* [DaemonSet](/docs/concepts/workloads/controllers/daemonset/) for running Pods that provide
+  node-local facilities, such as a storage driver or network plugin;
+* [Job](/docs/concepts/workloads/controllers/job/) and
+  [CronJob](/docs/concepts/workloads/controllers/cron-jobs/)
+  for tasks that run to completion.
+
+There are also two supporting concepts that you might find relevant:
+* [Garbage collection](/docs/concepts/workloads/controllers/garbage-collection/) tidies up objects
+  from your cluster after their _owning resource_ has been removed.
+* The [_time-to-live after finished_ controller](/docs/concepts/workloads/controllers/ttlafterfinished/)
+  removes Jobs once a defined time has passed since they completed.
+
+## {{% heading "whatsnext" %}}
+
+As well as reading about each resource, you can learn about specific tasks that relate to them:
+
+* [Run a stateless application using a Deployment](/docs/tasks/run-application/run-stateless-application-deployment/)
+* Run a stateful application either as a [single instance](/docs/tasks/run-application/run-single-instance-stateful-application/)
+  or as a [replicated set](/docs/tasks/run-application/run-replicated-stateful-application/)
+* [Run Automated Tasks with a CronJob](/docs/tasks/job/automated-tasks-with-cron-jobs/)
+
+Once your application is running, you might want to make it available on the internet as
+a [Service](/docs/concepts/services-networking/service/) or, for web application only,
+using an [Ingress](/docs/concepts/services-networking/ingress).
+
+You can also visit [Configuration](/docs/concepts/configuration/) to learn about Kubernetes'
+mechanisms for separating code from configuration.
@@ -1,5 +1,5 @@
 ---
-title: "Controllers"
+title: "Workload Resources"
 weight: 20
 ---

@@ -752,7 +752,7 @@ apply multiple fixes in between pausing and resuming without triggering unnecess
    REVISION  CHANGE-CAUSE
    1   <none>
    ```
-* Get the rollout status to ensure that the Deployment is updates successfully:
+* Get the rollout status to ensure that the Deployment is updated successfully:
    ```shell
    kubectl get rs
    ```
@@ -233,7 +233,7 @@ A ReplicaSet also needs a [`.spec` section](https://git.k8s.io/community/contrib

 ### Pod Template

-The `.spec.template` is a [pod template](/docs/concepts/workloads/Pods/pod-overview/#pod-templates) which is also
+The `.spec.template` is a [pod template](/docs/concepts/workloads/pods/#pod-templates) which is also
 required to have labels in place. In our `frontend.yaml` example we had one label: `tier: frontend`.
 Be careful not to overlap with the selectors of other controllers, lest they try to adopt this Pod.

@@ -351,7 +351,7 @@ Unlike the case where a user directly created Pods, a ReplicaSet replaces Pods t

 ### Job

-Use a [`Job`](/docs/concepts/jobs/run-to-completion-finite-workloads/) instead of a ReplicaSet for Pods that are expected to terminate on their own
+Use a [`Job`](/docs/concepts/workloads/controllers/job/) instead of a ReplicaSet for Pods that are expected to terminate on their own
 (that is, batch jobs).

 ### DaemonSet
@@ -145,7 +145,7 @@ from a _pod template_ and manage those Pods on your behalf.

 PodTemplates are specifications for creating Pods, and are included in workload resources such as
 [Deployments](/docs/concepts/workloads/controllers/deployment/),
-[Jobs](/docs/concepts/jobs/run-to-completion-finite-workloads/), and
+[Jobs](/docs/concepts/workloads/controllers/job/), and
 [DaemonSets](/docs/concepts/workloads/controllers/daemonset/).

 Each controller for a workload resource uses the `PodTemplate` inside the workload
@@ -264,7 +264,7 @@ but cannot be controlled from there.
 * [The Distributed System Toolkit: Patterns for Composite Containers](https://kubernetes.io/blog/2015/06/the-distributed-system-toolkit-patterns) explains common layouts for Pods with more than one container.

 To understand the context for why Kubernetes wraps a common Pod API in other resources (such as {{< glossary_tooltip text="StatefulSets" term_id="statefulset" >}} or {{< glossary_tooltip text="Deployments" term_id="deployment" >}}, you can read about the prior art, including:
-  * [Aurora](http://aurora.apache.org/documentation/latest/reference/configuration/#job-schema)
+  * [Aurora](https://aurora.apache.org/documentation/latest/reference/configuration/#job-schema)
  * [Borg](https://research.google.com/pubs/pub43438.html)
  * [Marathon](https://mesosphere.github.io/marathon/docs/rest-api.html)
  * [Omega](https://research.google/pubs/pub41684/)
@@ -77,13 +77,13 @@ have a given `phase` value.

 Here are the possible values for `phase`:

-Value | Description
-:-----|:-----------
-`Pending` | The Pod has been accepted by the Kubernetes cluster, but one or more of the containers has not been set up and made ready to run. This includes time a Pod spends waiting to be scheduled as well as the time spent downloading container images over the network.
-`Running` | The Pod has been bound to a node, and all of the containers have been created. At least one container is still running, or is in the process of starting or restarting.
+Value       | Description
+:-----------|:-----------
+`Pending`   | The Pod has been accepted by the Kubernetes cluster, but one or more of the containers has not been set up and made ready to run. This includes time a Pod spends waiting to be scheduled as well as the time spent downloading container images over the network.
+`Running`   | The Pod has been bound to a node, and all of the containers have been created. At least one container is still running, or is in the process of starting or restarting.
 `Succeeded` | All containers in the Pod have terminated in success, and will not be restarted.
-`Failed` | All containers in the Pod have terminated, and at least one container has terminated in failure. That is, the container either exited with non-zero status or was terminated by the system.
-`Unknown` | For some reason the state of the Pod could not be obtained. This phase typically occurs due to an error in communicating with the node where the Pod should be running.
+`Failed`    | All containers in the Pod have terminated, and at least one container has terminated in failure. That is, the container either exited with non-zero status or was terminated by the system.
+`Unknown`   | For some reason the state of the Pod could not be obtained. This phase typically occurs due to an error in communicating with the node where the Pod should be running.

 If a node dies or is disconnected from the rest of the cluster, Kubernetes
 applies a policy for setting the `phase` of all Pods on the lost node to Failed.
@@ -228,7 +228,8 @@ When a Pod's containers are Ready but at least one custom condition is missing o
 ## Container probes

 A [Probe](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#probe-v1-core) is a diagnostic
-performed periodically by the [kubelet](/docs/admin/kubelet/)
+performed periodically by the
+[kubelet](/docs/reference/command-line-tools-reference/kubelet/)
 on a Container. To perform a diagnostic,
 the kubelet calls a
 [Handler](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#handler-v1-core) implemented by
@@ -32,7 +32,7 @@ First, consult the [ISO 639-1 standard](https://www.loc.gov/standards/iso639-2/p

 ### Fork and clone the repo

-First, [create your own fork](/docs/contribute/new-content/new-content/#fork-the-repo) of the [kubernetes/website](https://github.com/kubernetes/website) repository.
+First, [create your own fork](/docs/contribute/new-content/open-a-pr/#fork-the-repo) of the [kubernetes/website](https://github.com/kubernetes/website) repository.

 Then, clone your fork and `cd` into it:

@@ -25,7 +25,7 @@ Anyone can write a blog post and submit it for review.
 ### Guidelines and expectations

 - Blog posts should not be vendor pitches. 
-  - Articles must contain content that applies broadly to the Kubernetes community. For example, a submission should focus on upstream Kubernetes as opposed to vendor-specific configurations. Check the [Documentation style guide](https://kubernetes.io/docs/contribute/style/content-guide/#what-s-allowed) for what is typically allowed on Kubernetes properties. 
+  - Articles must contain content that applies broadly to the Kubernetes community. For example, a submission should focus on upstream Kubernetes as opposed to vendor-specific configurations. Check the [Documentation style guide](/docs/contribute/style/content-guide/#what-s-allowed) for what is typically allowed on Kubernetes properties. 
  - Links should primarily be to the official Kubernetes documentation. When using external references, links should be diverse - For example a submission shouldn't contain only links back to a single company's blog.
  - Sometimes this is a delicate balance. The [blog team](https://kubernetes.slack.com/messages/sig-docs-blog/) is there to give guidance on whether a post is appropriate for the Kubernetes blog, so don't hesitate to reach out. 
 - Blog posts are not published on specific dates.
@@ -56,7 +56,7 @@ The SIG Docs [blog subproject](https://github.com/kubernetes/community/tree/mast

 To submit a blog post follow these directions:

- [Open a pull request](/docs/contribute/new-content/new-content/#fork-the-repo) with a new blog post. New blog posts go under the [`content/en/blog/_posts`](https://github.com/kubernetes/website/tree/master/content/en/blog/_posts) directory.
+- [Open a pull request](/docs/contribute/new-content/open-a-pr/#fork-the-repo) with a new blog post. New blog posts go under the [`content/en/blog/_posts`](https://github.com/kubernetes/website/tree/master/content/en/blog/_posts) directory.

 - Ensure that your blog post follows the correct naming conventions and the following frontmatter (metadata) information:

@@ -98,7 +98,8 @@ deadlines.
 1. Open a pull request against the
 `dev-{{< skew nextMinorVersion >}}` branch in the `kubernetes/website` repository, with a small
 commit that you will amend later.
-2. Edit the pull request description to include links to `k/k` PR(s) and `k/enhancement` issue(s).
+2. Edit the pull request description to include links to [kubernetes/kubernetes](https://github.com/kubernetes/kubernetes)
+PR(s) and [kubernetes/enhancements](https://github.com/kubernetes/enhancements) issue(s).
 3. Use the Prow command `/milestone {{< skew nextMinorVersion >}}` to
 assign the PR to the relevant milestone. This alerts the docs person managing
 this release that the feature docs are coming.
@@ -115,7 +116,9 @@ When ready, populate your placeholder PR with feature documentation.

 Do your best to describe your feature and how to use it. If you need help structuring your documentation, ask in the `#sig-docs` slack channel.

-When you complete your content, the documentation person assigned to your feature reviews it. Use their suggestions to get the content to a release ready state.
+When you complete your content, the documentation person assigned to your feature reviews it.
+To ensure technical accuracy, the content may also require a technical review from corresponding SIG(s).
+Use their suggestions to get the content to a release ready state.

 If your feature needs documentation and the first draft
 content is not received, the feature may be removed from the milestone.
@@ -6,7 +6,7 @@ weight: 20

 <!-- overview -->

-SIG Docs [approvers](/docs/contribute/participating/roles-and-responsibilites/#approvers) take week-long shifts [managing pull requests](https://github.com/kubernetes/website/wiki/PR-Wranglers) for the repository.
+SIG Docs [approvers](/docs/contribute/participate/roles-and-responsibilities/#approvers) take week-long shifts [managing pull requests](https://github.com/kubernetes/website/wiki/PR-Wranglers) for the repository.

 This section covers the duties of a PR wrangler. For more information on giving good reviews, see [Reviewing changes](/docs/contribute/review/).

@@ -18,8 +18,8 @@ This section of the Kubernetes documentation contains references.

 ## API Reference

-* [Kubernetes API Overview](/docs/reference/using-api/api-overview/) - Overview of the API for Kubernetes.
 * [Kubernetes API Reference {{< latest-version >}}](/docs/reference/generated/kubernetes-api/{{< latest-version >}}/)
+* [Using The Kubernetes API](/docs/reference/using-api/) - overview of the API for Kubernetes.

 ## API Client Libraries

@@ -36,7 +36,7 @@ client libraries:

 * [kubectl](/docs/reference/kubectl/overview/) - Main CLI tool for running commands and managing Kubernetes clusters.
    * [JSONPath](/docs/reference/kubectl/jsonpath/) - Syntax guide for using [JSONPath expressions](https://goessner.net/articles/JsonPath/) with kubectl.
-* [kubeadm](/docs/reference/setup-tools/kubeadm/kubeadm/) - CLI tool to easily provision a secure Kubernetes cluster.
+* [kubeadm](/docs/reference/setup-tools/kubeadm/) - CLI tool to easily provision a secure Kubernetes cluster.

 ## Components Reference

@@ -1,4 +1,26 @@
 ---
-title: Accessing the API
+title: API Access Control
 weight: 20
---
+no_list: true
+---
+
+For an introduction to how Kubernetes implements and controls API access,
+read [Controlling Access to the Kubernetes API](/docs/concepts/security/controlling-access/).
+
+Reference documentation:
+
+- [Authenticating](/docs/reference/access-authn-authz/authentication/)
+   - [Authenticating with Bootstrap Tokens](/docs/reference/access-authn-authz/bootstrap-tokens/)
+- [Admission Controllers](/docs/reference/access-authn-authz/admission-controllers/)
+   - [Dynamic Admission Control](/docs/reference/access-authn-authz/extensible-admission-controllers/)
+- [Authorization](/docs/reference/access-authn-authz/authorization/)
+   - [Role Based Access Control](/docs/reference/access-authn-authz/rbac/)
+   - [Attribute Based Access Control](/docs/reference/access-authn-authz/abac/)
+   - [Node Authorization](/docs/reference/access-authn-authz/node/)
+   - [Webhook Authorization](/docs/reference/access-authn-authz/webhook/)
+- [Certificate Signing Requests](/docs/reference/access-authn-authz/certificate-signing-requests/)
+   - including [CSR approval](/docs/reference/access-authn-authz/certificate-signing-requests/#approval-rejection)
+     and [certificate signing](/docs/reference/access-authn-authz/certificate-signing-requests/#signing)
+- Service accounts
+  - [Developer guide](/docs/tasks/configure-pod-container/configure-service-account/)
+  - [Administration](/docs/reference/access-authn-authz/service-accounts-admin/)
@@ -730,7 +730,7 @@ See the [resourceQuota design doc](https://git.k8s.io/community/contributors/des
 For [RuntimeClass](/docs/concepts/containers/runtime-class/) definitions which describe an overhead associated with running a pod,
 this admission controller will set the pod.Spec.Overhead field accordingly.

-See also [Pod Overhead](/docs/concepts/configuration/pod-overhead/)
+See also [Pod Overhead](/docs/concepts/scheduling-eviction/pod-overhead/)
 for more information.

 ### SecurityContextDeny {#securitycontextdeny}
@@ -29,7 +29,7 @@ It is assumed that a cluster-independent service manages normal users in the fol
 In this regard, _Kubernetes does not have objects which represent normal user
 accounts._ Normal users cannot be added to a cluster through an API call.

-Even though normal user cannot be added via an API call, but any user that
+Even though a normal user cannot be added via an API call, any user that
 presents a valid certificate signed by the cluster's certificate authority
 (CA) is considered authenticated. In this configuration, Kubernetes determines
 the username from the common name field in the 'subject' of the cert (e.g.,
@@ -17,7 +17,7 @@ policies using the supported authorization modules.
 <!-- body -->
 In Kubernetes, you must be authenticated (logged in) before your request can be
 authorized (granted permission to access). For information about authentication,
-see [Controlling Access to the Kubernetes API](/docs/reference/access-authn-authz/controlling-access/).
+see [Controlling Access to the Kubernetes API](/docs/concepts/security/controlling-access/).

 Kubernetes expects attributes that are common to REST API requests. This means
 that Kubernetes authorization works with existing organization-wide or
@@ -52,7 +52,7 @@ Kubernetes reviews only the following API request attributes:
 * **Resource** - The ID or name of the resource that is being accessed (for resource requests only) -- For resource requests using `get`, `update`, `patch`, and `delete` verbs, you must provide the resource name.
 * **Subresource** - The subresource that is being accessed (for resource requests only).
 * **Namespace** - The namespace of the object that is being accessed (for namespaced resource requests only).
- * **API group** - The {{< glossary_tooltip text="API Group" term_id="api-group" >}} being accessed (for resource requests only). An empty string designates the [core API group](/docs/reference/using-api/api-overview/#api-groups).
+ * **API group** - The {{< glossary_tooltip text="API Group" term_id="api-group" >}} being accessed (for resource requests only). An empty string designates the _core_ [API group](/docs/reference/using-api/#api-groups).

 ## Determine the Request Verb

@@ -202,6 +202,6 @@ action the account could take. This applies regardless of authorization mode.

 ## {{% heading "whatsnext" %}}

-* To learn more about Authentication, see **Authentication** in [Controlling Access to the Kubernetes API](/docs/reference/access-authn-authz/controlling-access/).
+* To learn more about Authentication, see **Authentication** in [Controlling Access to the Kubernetes API](/docs/concepts/security/controlling-access/).
 * To learn more about Admission Control, see [Using Admission Controllers](/docs/reference/access-authn-authz/admission-controllers/).

@@ -12,10 +12,10 @@ weight: 20

 Bootstrap tokens are a simple bearer token that is meant to be used when
 creating new clusters or joining new nodes to an existing cluster.  It was built
-to support [kubeadm](/docs/reference/setup-tools/kubeadm/kubeadm/), but can be used in other contexts
+to support [kubeadm](/docs/reference/setup-tools/kubeadm/), but can be used in other contexts
 for users that wish to start clusters without `kubeadm`. It is also built to
-work, via RBAC policy, with the [Kubelet TLS
-Bootstrapping](/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping/) system.
+work, via RBAC policy, with the
+[Kubelet TLS Bootstrapping](/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping/) system.


 <!-- body -->
@@ -21,17 +21,16 @@ A CertificateSigningRequest (CSR) resource is used to request that a certificate
 by a denoted signer, after which the request may be approved or denied before
 finally being signed.

-
-
 <!-- body -->
+
 ## Request signing process

-The _CertificateSigningRequest_ resource type allows a client to ask for an X.509 certificate
+The CertificateSigningRequest resource type allows a client to ask for an X.509 certificate
 be issued, based on a signing request.
 The CertificateSigningRequest object includes a PEM-encoded PKCS#10 signing request in
-the `spec.request` field. The CertificateSigningRequest denotes the _signer_ (the
+the `spec.request` field. The CertificateSigningRequest denotes the signer (the
 recipient that the request is being made to) using the `spec.signerName` field.
-Note that `spec.signerName` is a required key after api version `certificates.k8s.io/v1`.
+Note that `spec.signerName` is a required key after API version `certificates.k8s.io/v1`.

 Once created, a CertificateSigningRequest must be approved before it can be signed.
 Depending on the signer selected, a CertificateSigningRequest may be automatically approved
@@ -68,54 +67,80 @@ This includes:
 1. **Permitted subjects**: any restrictions on and behavior when a disallowed subject is requested.
 1. **Permitted x509 extensions**: including IP subjectAltNames, DNS subjectAltNames, Email subjectAltNames, URI subjectAltNames etc, and behavior when a disallowed extension is requested.
 1. **Permitted key usages / extended key usages**: any restrictions on and behavior when usages different than the signer-determined usages are specified in the CSR.
-1. **Expiration/certificate lifetime**: whether it is fixed by the signer, configurable by the admin, determined by the CSR object etc and the behavior when an expiration is different than the signer-determined expiration that is specified in the CSR.
+1. **Expiration/certificate lifetime**: whether it is fixed by the signer, configurable by the admin, determined by the CSR object etc
+   and the behavior when an expiration is different than the signer-determined expiration that is specified in the CSR.
 1. **CA bit allowed/disallowed**: and behavior if a CSR contains a request a for a CA certificate when the signer does not permit it.

-Commonly, the `status.certificate` field contains a single PEM-encoded X.509 certificate once the CSR is approved and the certificate is issued. Some signers store multiple certificates into the `status.certificate` field. In that case, the documentation for the signer should specify the meaning of additional certificates; for example, this might be the certificate plus intermediates to be presented during TLS handshakes.
+Commonly, the `status.certificate` field contains a single PEM-encoded X.509
+certificate once the CSR is approved and the certificate is issued. Some
+signers store multiple certificates into the `status.certificate` field. In
+that case, the documentation for the signer should specify the meaning of
+additional certificates; for example, this might be the certificate plus
+intermediates to be presented during TLS handshakes.
+
+The PKCS#10 signing request format doesn't allow to specify a certificate
+expiration or lifetime. The expiration or lifetime therefore has to be set
+through e.g. an annotation on the CSR object. While it's theoretically
+possible for a signer to use that expiration date, there is currently no
+known implementation that does. (The built-in signers all use the same
+`ClusterSigningDuration` configuration option, which defaults to 1 year,
+and can be changed with the `--cluster-signing-duration` command-line
+flag of the kube-controller-manager.)
+

 ### Kubernetes signers

 Kubernetes provides built-in signers that each have a well-known `signerName`:

-1. `kubernetes.io/kube-apiserver-client`: signs certificates that will be honored as client-certs by the kube-apiserver.
-  Never auto-approved by {{< glossary_tooltip term_id="kube-controller-manager" >}}.
-    1. Trust distribution: signed certificates must be honored as client-certificates by the kube-apiserver. The CA bundle is not distributed by any other means.
-    1. Permitted subjects - no subject restrictions, but approvers and signers may choose not to approve or sign. Certain subjects like cluster-admin level users or groups vary between distributions and installations, but deserve additional scrutiny before approval and signing. The `CertificateSubjectRestriction` admission plugin is available and enabled by default to restrict `system:masters`, but it is often not the only cluster-admin subject in a cluster.
+1. `kubernetes.io/kube-apiserver-client`: signs certificates that will be honored as client certificates by the API server.
+   Never auto-approved by {{< glossary_tooltip term_id="kube-controller-manager" >}}.
+    1. Trust distribution: signed certificates must be honored as client certificates by the API server. The CA bundle is not distributed by any other means.
+    1. Permitted subjects - no subject restrictions, but approvers and signers may choose not to approve or sign.
+       Certain subjects like cluster-admin level users or groups vary between distributions and installations,
+       but deserve additional scrutiny before approval and signing.
+       The `CertificateSubjectRestriction` admission plugin is enabled by default to restrict `system:masters`,
+       but it is often not the only cluster-admin subject in a cluster.
    1. Permitted x509 extensions - honors subjectAltName and key usage extensions and discards other extensions.
-    1. Permitted key usages - must include []string{"client auth"}. Must not include key usages beyond []string{"digital signature", "key encipherment", "client auth"}
-    1. Expiration/certificate lifetime - minimum of CSR signer or request. The signer is responsible for checking that the certificate lifetime is valid and permissible.
+    1. Permitted key usages - must include `["client auth"]`. Must not include key usages beyond `["digital signature", "key encipherment", "client auth"]`.
+    1. Expiration/certificate lifetime - set by the `--cluster-signing-duration` option for the
+       kube-controller-manager implementation of this signer.
    1. CA bit allowed/disallowed - not allowed.

-1. `kubernetes.io/kube-apiserver-client-kubelet`: signs client certificates that will be honored as client-certs by the
-  kube-apiserver.
-  May be auto-approved by {{< glossary_tooltip term_id="kube-controller-manager" >}}.
-    1. Trust distribution: signed certificates must be honored as client-certificates by the kube-apiserver. The CA bundle
+1. `kubernetes.io/kube-apiserver-client-kubelet`: signs client certificates that will be honored as client certificates by the
+   API server.
+   May be auto-approved by {{< glossary_tooltip term_id="kube-controller-manager" >}}.
+    1. Trust distribution: signed certificates must be honored as client certificates by the API server. The CA bundle
       is not distributed by any other means.
-    1. Permitted subjects - organizations are exactly `[]string{"system:nodes"}`, common name starts with `"system:node:"`
+    1. Permitted subjects - organizations are exactly `["system:nodes"]`, common name starts with "`system:node:`".
    1. Permitted x509 extensions - honors key usage extensions, forbids subjectAltName extensions and drops other extensions.
-    1. Permitted key usages - exactly `[]string{"key encipherment", "digital signature", "client auth"}`
-    1. Expiration/certificate lifetime - minimum of CSR signer or request. The signer is responsible for checking that the certificate lifetime is valid and permissible.
+    1. Permitted key usages - exactly `["key encipherment", "digital signature", "client auth"]`.
+    1. Expiration/certificate lifetime - set by the `--cluster-signing-duration` option for the
+       kube-controller-manager implementation of this signer.
    1. CA bit allowed/disallowed - not allowed.

 1. `kubernetes.io/kubelet-serving`: signs serving certificates that are honored as a valid kubelet serving certificate
-  by the kube-apiserver, but has no other guarantees.
-  Never auto-approved by {{< glossary_tooltip term_id="kube-controller-manager" >}}.
-    1. Trust distribution: signed certificates must be honored by the kube-apiserver as valid to terminate connections to a kubelet. The CA bundle is not distributed by any other means.
-    1. Permitted subjects - organizations are exactly `[]string{"system:nodes"}`, common name starts with `"system:node:"`
-    1. Permitted x509 extensions - honors key usage and DNSName/IPAddress subjectAltName extensions, forbids EmailAddress and URI subjectAltName extensions, drops other extensions. At least one DNS or IP subjectAltName must be present.
-    1. Permitted key usages - exactly `[]string{"key encipherment", "digital signature", "server auth"}`
-    1. Expiration/certificate lifetime - minimum of CSR signer or request.
+   by the API server, but has no other guarantees.
+   Never auto-approved by {{< glossary_tooltip term_id="kube-controller-manager" >}}.
+    1. Trust distribution: signed certificates must be honored by the API server as valid to terminate connections to a kubelet.
+       The CA bundle is not distributed by any other means.
+    1. Permitted subjects - organizations are exactly `["system:nodes"]`, common name starts with "`system:node:`".
+    1. Permitted x509 extensions - honors key usage and DNSName/IPAddress subjectAltName extensions, forbids EmailAddress and
+       URI subjectAltName extensions, drops other extensions. At least one DNS or IP subjectAltName must be present.
+    1. Permitted key usages - exactly `["key encipherment", "digital signature", "server auth"]`.
+    1. Expiration/certificate lifetime - set by the `--cluster-signing-duration` option for the
+       kube-controller-manager implementation of this signer.
    1. CA bit allowed/disallowed - not allowed.

 1. `kubernetes.io/legacy-unknown`:  has no guarantees for trust at all. Some third-party distributions of Kubernetes
-  may honor client certificates signed by it. The stable CertificateSigningRequest API (version `certificates.k8s.io/v1` and later)
-  does not allow to set the `signerName` as `kubernetes.io/legacy-unknown`.
-  Never auto-approved by {{< glossary_tooltip term_id="kube-controller-manager" >}}.
-    1. Trust distribution: None.  There is no standard trust or distribution for this signer in a Kubernetes cluster.
+   may honor client certificates signed by it. The stable CertificateSigningRequest API (version `certificates.k8s.io/v1` and later)
+   does not allow to set the `signerName` as `kubernetes.io/legacy-unknown`.
+   Never auto-approved by {{< glossary_tooltip term_id="kube-controller-manager" >}}.
+    1. Trust distribution: None. There is no standard trust or distribution for this signer in a Kubernetes cluster.
    1. Permitted subjects - any
    1. Permitted x509 extensions - honors subjectAltName and key usage extensions and discards other extensions.
    1. Permitted key usages - any
-    1. Expiration/certificate lifetime - minimum of CSR signer or request. The signer is responsible for checking that the certificate lifetime is valid and permissible.
+    1. Expiration/certificate lifetime - set by the `--cluster-signing-duration` option for the
+       kube-controller-manager implementation of this signer.
    1. CA bit allowed/disallowed - not allowed.

 {{< note >}}
@@ -126,7 +151,7 @@ Distribution of trust happens out of band for these signers.  Any trust outside
 coincidental. For instance, some distributions may honor `kubernetes.io/legacy-unknown` as client certificates for the
 kube-apiserver, but this is not a standard.
 None of these usages are related to ServiceAccount token secrets `.data[ca.crt]` in any way.  That CA bundle is only
-guaranteed to verify a connection to the kube-apiserver using the default service (`kubernetes.default.svc`).
+guaranteed to verify a connection to the API server using the default service (`kubernetes.default.svc`).

 ## Authorization

@@ -136,22 +161,7 @@ To allow creating a CertificateSigningRequest and retrieving any CertificateSign

 For example:

-```yaml
-apiVersion: rbac.authorization.k8s.io/v1
-kind: ClusterRole
-metadata:
-  name: csr-creator
-rules:
- apiGroups:
-  - certificates.k8s.io
-  resources:
-  - certificatesigningrequests
-  verbs:
-  - create
-  - get
-  - list
-  - watch
-```
+{{< codenew file="access/certificate-signing-request/clusterrole-create.yaml" >}}

 To allow approving a CertificateSigningRequest:

@@ -161,35 +171,7 @@ To allow approving a CertificateSigningRequest:

 For example:

-```yaml
-apiVersion: rbac.authorization.k8s.io/v1
-kind: ClusterRole
-metadata:
-  name: csr-approver
-rules:
- apiGroups:
-  - certificates.k8s.io
-  resources:
-  - certificatesigningrequests
-  verbs:
-  - get
-  - list
-  - watch
- apiGroups:
-  - certificates.k8s.io
-  resources:
-  - certificatesigningrequests/approval
-  verbs:
-  - update
- apiGroups:
-  - certificates.k8s.io
-  resources:
-  - signers
-  resourceNames:
-  - example.com/my-signer-name # example.com/* can be used to authorize for all signers in the 'example.com' domain
-  verbs:
-  - approve
-```
+{{< codenew file="access/certificate-signing-request/clusterrole-approve.yaml" >}}

 To allow signing a CertificateSigningRequest:

@@ -197,54 +179,32 @@ To allow signing a CertificateSigningRequest:
 * Verbs: `update`, group: `certificates.k8s.io`, resource: `certificatesigningrequests/status`
 * Verbs: `sign`, group: `certificates.k8s.io`, resource: `signers`, resourceName: `<signerNameDomain>/<signerNamePath>` or `<signerNameDomain>/*`

-```yaml
-apiVersion: rbac.authorization.k8s.io/v1
-kind: ClusterRole
-metadata:
-  name: csr-signer
-rules:
- apiGroups:
-  - certificates.k8s.io
-  resources:
-  - certificatesigningrequests
-  verbs:
-  - get
-  - list
-  - watch
- apiGroups:
-  - certificates.k8s.io
-  resources:
-  - certificatesigningrequests/status
-  verbs:
-  - update
- apiGroups:
-  - certificates.k8s.io
-  resources:
-  - signers
-  resourceName:
-  - example.com/my-signer-name # example.com/* can be used to authorize for all signers in the 'example.com' domain
-  verbs:
-  - sign
-```
+{{< codenew file="access/certificate-signing-request/clusterrole-sign.yaml" >}}

-## Normal User
+## Normal user

-A few steps are required in order to get normal user to be able to authenticate and invoke an API. First, this user must have certificate issued by the Kubernetes Cluster, and then present that Certificate to the API call as the Certificate Header or through the kubectl.
+A few steps are required in order to get a normal user to be able to
+authenticate and invoke an API. First, this user must have certificate issued
+by the Kubernetes cluster, and then present that Certificate to the API call
+as the Certificate Header or through the kubectl.

-### Create Private Key
+### Create private key

-The following scripts show how to generate PKI private key and CSR. It is important to set CN and O attribute of the CSR. CN is the name of the user and O is the group that this user will belong to. You can refer to [RBAC](/docs/reference/access-authn-authz/rbac/) for standard groups.
+The following scripts show how to generate PKI private key and CSR. It is
+important to set CN and O attribute of the CSR. CN is the name of the user and
+O is the group that this user will belong to. You can refer to
+[RBAC](/docs/reference/access-authn-authz/rbac/) for standard groups.

-```
+```shell
 openssl genrsa -out john.key 2048
 openssl req -new -key john.key -out john.csr
 ```

-### Create Certificate Request Kubernetes Object
+### Create CertificateSigningRequest

 Create a CertificateSigningRequest and submit it to a Kubernetes Cluster via kubectl. Below is a script to generate the CertificateSigningRequest.

-```
+```shell
 cat <<EOF | kubectl apply -f -
 apiVersion: certificates.k8s.io/v1
 kind: CertificateSigningRequest
@@ -262,68 +222,78 @@ EOF

 Some points to note:

- usage has to be 'client auth'
- request is the base64 encoded value of the CSR file content. You can use this command to get that ```cat john.csr | base64 | tr -d "\n"```
+- `usages` has to be '`client auth`'
+- `request` is the base64 encoded value of the CSR file content.
+  You can get the content using this command: ```cat john.csr | base64 | tr -d "\n"```

-### Approve Certificate Request
+### Approve certificate signing request

-Use kubeadmin to create a CSR and approve it.
+Use kubectl to create a CSR and approve it.

-Get the list of CSRs
-```
+Get the list of CSRs:
+
+```shell
 kubectl get csr
 ```

-Approve the CSR
-```
+Approve the CSR:
+
+```shell
 kubectl certificate approve john
 ```

-### Get the Certificate
+### Get the certificate

-Retrieve the Certificate from the CSR.
+Retrieve the certificate from the CSR:

-```
+```shell
 kubectl get csr/john -o yaml
 ```

-The Certificate value is in Base64-encoded format under status.certificate.
+The certificate value is in Base64-encoded format under `status.certificate`.

-### Create Role and Role Binding
+### Create Role and RoleBinding

-You get the Certificate already. Now it is time to define the Role and Role Binding for this user to access Kubernetes Cluster resources.
+With the certificate created. it is time to define the Role and RoleBinding for
+this user to access Kubernetes cluster resources.

-This is a sample script to create role for this new user
-```
+This is a sample script to create a Role for this new user:
+
+```shell
 kubectl create role developer --verb=create --verb=get --verb=list --verb=update --verb=delete --resource=pods
 ```

-This is a sample script to create role binding for this new user
-```
+This is a sample command to create a RoleBinding for this new user:
+
+```shell
 kubectl create rolebinding developer-binding-john --role=developer --user=john
 ```

-### Add to KubeConfig
+### Add to kubeconfig

-The last step is to add this user into the KubeConfig. We assume the key and crt files are located here "/home/vagrant/work/".
+The last step is to add this user into the kubeconfig file.
+This example assumes the key and certificate files are located at "/home/vagrant/work/".
+
+First, you need to add new credentials:

-First, we need to add new credentials
 ```
 kubectl config set-credentials john --client-key=/home/vagrant/work/john.key --client-certificate=/home/vagrant/work/john.crt --embed-certs=true

 ```

-Then, we need to add the context
+Then, you need to add the context:
+
 ```
 kubectl config set-context john --cluster=kubernetes --user=john
 ```

-To test it, change kubecontext to john
+To test it, change the context to `john`:
+
 ```
 kubectl config use-context john
 ```

-## Approval & rejection
+## Approval or rejection  {#approval-rejection}

 ### Control plane automated approval {#approval-rejection-control-plane}

@@ -333,7 +303,7 @@ permissions on CSRs for node credentials to authorization.
 The kube-controller-manager POSTs SubjectAccessReview resources to the API server
 in order to check authorization for certificate approval.

-### Approval & rejection using `kubectl` {#approval-rejection-kubectl}
+### Approval or rejection using `kubectl` {#approval-rejection-kubectl}

 A Kubernetes administrator (with appropriate permissions) can manually approve
 (or deny) CertificateSigningRequests by using the `kubectl certificate
@@ -341,17 +311,17 @@ approve` and `kubectl certificate deny` commands.

 To approve a CSR with kubectl:

-```bash
+```shell
 kubectl certificate approve <certificate-signing-request-name>
 ```

 Likewise, to deny a CSR:

-```bash
+```shell
 kubectl certificate deny <certificate-signing-request-name>
 ```

-### Approval & rejection using the Kubernetes API {#approval-rejection-api-client}
+### Approval or rejection using the Kubernetes API {#approval-rejection-api-client}

 Users of the REST API can approve CSRs by submitting an UPDATE request to the `approval`
 subresource of the CSR to be approved. For example, you could write an
@@ -400,7 +370,8 @@ you like. If you want to add a note just for human consumption, use the

 ### Control plane signer {#signer-control-plane}

-The Kubernetes control plane implements each of the [Kubernetes signers](/docs/reference/access-authn-authz/certificate-signing-requests/#kubernetes-signers),
+The Kubernetes control plane implements each of the
+[Kubernetes signers](/docs/reference/access-authn-authz/certificate-signing-requests/#kubernetes-signers),
 as part of the kube-controller-manager.

 {{< note >}}
@@ -460,13 +431,11 @@ status:
  certificate: "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JS..."
 ```

-
-
 ## {{% heading "whatsnext" %}}

-
 * Read [Manage TLS Certificates in a Cluster](/docs/tasks/tls/managing-tls-in-a-cluster/)
 * View the source code for the kube-controller-manager built in [signer](https://github.com/kubernetes/kubernetes/blob/32ec6c212ec9415f604ffc1f4c1f29b782968ff1/pkg/controller/certificates/signer/cfssl_signer.go)
 * View the source code for the kube-controller-manager built in [approver](https://github.com/kubernetes/kubernetes/blob/32ec6c212ec9415f604ffc1f4c1f29b782968ff1/pkg/controller/certificates/approver/sarapprove.go)
 * For details of X.509 itself, refer to [RFC 5280](https://tools.ietf.org/html/rfc5280#section-3.1) section 3.1
 * For information on the syntax of PKCS#10 certificate signing requests, refer to [RFC 2986](https://tools.ietf.org/html/rfc2986)
+
@@ -433,7 +433,7 @@ Each feature gate is designed for enabling/disabling a specific feature:
 - `DevicePlugins`: Enable the [device-plugins](/docs/concepts/cluster-administration/device-plugins/)
  based resource provisioning on nodes.
 - `DefaultPodTopologySpread`: Enables the use of `PodTopologySpread` scheduling plugin to do
-  [default spreading](/docs/concepts/workloads/pods/pod/pod-topology-spread-constraints/#internal-default-constraints).
+  [default spreading](/docs/concepts/workloads/pods/pod-topology-spread-constraints/#internal-default-constraints).
 - `DryRun`: Enable server-side [dry run](/docs/reference/using-api/api-concepts/#dry-run) requests
  so that validation, merging, and mutation can be tested without committing.
 - `DynamicAuditing`(*deprecated*): Used to enable dynamic auditing before v1.19.
@@ -479,8 +479,8 @@ Each feature gate is designed for enabling/disabling a specific feature:
 - `KubeletPodResources`: Enable the kubelet's pod resources grpc endpoint.
   See [Support Device Monitoring](https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/compute-device-assignment.md) for more details.
 - `LegacyNodeRoleBehavior`: When disabled, legacy behavior in service load balancers and node disruption will ignore the `node-role.kubernetes.io/master` label in favor of the feature-specific labels provided by `NodeDisruptionExclusion` and `ServiceNodeExclusion`.
- `LocalStorageCapacityIsolation`: Enable the consumption of [local ephemeral storage](/docs/concepts/configuration/manage-compute-resources-container/) and also the `sizeLimit` property of an [emptyDir volume](/docs/concepts/storage/volumes/#emptydir).
- `LocalStorageCapacityIsolationFSQuotaMonitoring`: When `LocalStorageCapacityIsolation` is enabled for [local ephemeral storage](/docs/concepts/configuration/manage-compute-resources-container/) and the backing filesystem for [emptyDir volumes](/docs/concepts/storage/volumes/#emptydir) supports project quotas and they are enabled, use project quotas to monitor [emptyDir volume](/docs/concepts/storage/volumes/#emptydir) storage consumption rather than filesystem walk for better performance and accuracy.
+- `LocalStorageCapacityIsolation`: Enable the consumption of [local ephemeral storage](/docs/concepts/configuration/manage-resources-containers/) and also the `sizeLimit` property of an [emptyDir volume](/docs/concepts/storage/volumes/#emptydir).
+- `LocalStorageCapacityIsolationFSQuotaMonitoring`: When `LocalStorageCapacityIsolation` is enabled for [local ephemeral storage](/docs/concepts/configuration/manage-resources-containers/) and the backing filesystem for [emptyDir volumes](/docs/concepts/storage/volumes/#emptydir) supports project quotas and they are enabled, use project quotas to monitor [emptyDir volume](/docs/concepts/storage/volumes/#emptydir) storage consumption rather than filesystem walk for better performance and accuracy.
 - `MountContainers`: Enable using utility containers on host as the volume mounter.
 - `MountPropagation`: Enable sharing volume mounted by one container to other containers or pods.
  For more details, please see [mount propagation](/docs/concepts/storage/volumes/#mount-propagation).
@@ -490,7 +490,7 @@ Each feature gate is designed for enabling/disabling a specific feature:
 - `PersistentLocalVolumes`: Enable the usage of `local` volume type in Pods.
  Pod affinity has to be specified if requesting a `local` volume.
 - `PodDisruptionBudget`: Enable the [PodDisruptionBudget](/docs/tasks/run-application/configure-pdb/) feature.
- `PodOverhead`: Enable the [PodOverhead](/docs/concepts/configuration/pod-overhead/) feature to account for pod overheads.
+- `PodOverhead`: Enable the [PodOverhead](/docs/concepts/scheduling-eviction/pod-overhead/) feature to account for pod overheads.
 - `PodPriority`: Enable the descheduling and preemption of Pods based on their [priorities](/docs/concepts/configuration/pod-priority-preemption/).
 - `PodReadinessGates`: Enable the setting of `PodReadinessGate` field for extending
  Pod readiness evaluation.  See [Pod readiness gate](/docs/concepts/workloads/pods/pod-lifecycle/#pod-readiness-gate)
@@ -516,7 +516,7 @@ Each feature gate is designed for enabling/disabling a specific feature:
 - `RuntimeClass`: Enable the [RuntimeClass](/docs/concepts/containers/runtime-class/) feature for selecting container runtime configurations.
 - `ScheduleDaemonSetPods`: Enable DaemonSet Pods to be scheduled by the default scheduler instead of the DaemonSet controller.
 - `SCTPSupport`: Enables the _SCTP_ `protocol` value in Pod, Service, Endpoints, EndpointSlice, and NetworkPolicy definitions.
- `ServerSideApply`: Enables the [Sever Side Apply (SSA)](/docs/reference/using-api/api-concepts/#server-side-apply) path at the API Server.
+- `ServerSideApply`: Enables the [Sever Side Apply (SSA)](/docs/reference/using-api/server-side-apply/) path at the API Server.
 - `ServiceAccountIssuerDiscovery`: Enable OIDC discovery endpoints (issuer and JWKS URLs) for the service account issuer in the API server. See [Configure Service Accounts for Pods](/docs/tasks/configure-pod-container/configure-service-account/#service-account-issuer-discovery) for more details.
 - `ServiceAppProtocol`: Enables the `AppProtocol` field on Services and Endpoints.
 - `ServiceLoadBalancerFinalizer`: Enable finalizer protection for Service load balancers.
@@ -2,7 +2,7 @@
 title: Container
 id: container
 date: 2018-04-12
-full_link: /docs/concepts/overview/what-is-kubernetes/#why-containers
+full_link: /docs/concepts/containers/
 short_description: >
  A lightweight and portable executable image that contains software and all of its dependencies.

@@ -20,7 +20,7 @@ Kubernetes contains several built-in tools to help you work with the Kubernetes

 ## Minikube

-[`minikube`](/docs/tasks/tools/install-minikube/) is a tool that makes it
+[`minikube`](https://minikube.sigs.k8s.io/docs/) is a tool that makes it
 easy to run a single-node Kubernetes cluster locally on your workstation for
 development and testing purposes.

@@ -44,7 +44,7 @@ Use Helm to:

 ## Kompose

-[`Kompose`](https://github.com/kubernetes-incubator/kompose) is a tool to help Docker Compose users move to Kubernetes.
+[`Kompose`](https://github.com/kubernetes/kompose) is a tool to help Docker Compose users move to Kubernetes.

 Use Kompose to:

@@ -1,4 +1,132 @@
 ---
-title: Using the Kubernetes API
+title: Kubernetes API Overview
+reviewers:
+- erictune
+- lavalamp
+- jbeda
+content_type: concept
 weight: 10
---
+card:
+  name: reference
+  weight: 50
+  title: Overview of API
+---
+
+<!-- overview -->
+
+This section provides reference information for the Kubernetes API.
+
+The REST API is the fundamental fabric of Kubernetes. All operations and
+communications between components, and external user commands are REST API
+calls that the API Server handles. Consequently, everything in the Kubernetes
+platform is treated as an API object and has a corresponding entry in the
+[API](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/).
+
+The [Kubernetes API reference](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/)
+lists the API for Kubernetes version {{< param "version" >}}.
+
+For general background information, read
+[The Kubernetes API](/docs/concepts/overview/kubernetes-api/).
+[Controlling Access to the Kubernetes API](/docs/concepts/security/controlling-access/)
+describes how clients can authenticate to the Kubernetes API server, and how their
+requests are authorized.
+
+<!-- body -->
+
+The REST API is the fundamental fabric of Kubernetes. All operations and
+communications between components, and external user commands are REST API
+calls that the API Server handles. Consequently, everything in the Kubernetes
+platform is treated as an API object and has a corresponding entry in the
+API.
+
+## API versioning
+
+The JSON and Protobuf serialization schemas follow the same guidelines for
+schema changes. The following descriptions cover both formats.
+
+The API versioning and software versioning are indirectly related.
+The [API and release versioning proposal](https://git.k8s.io/community/contributors/design-proposals/release/versioning.md)
+describes the relationship between API versioning and software versioning.
+
+Different API versions indicate different levels of stability and support. You
+can find more information about the criteria for each level in the
+[API Changes documentation](https://git.k8s.io/community/contributors/devel/sig-architecture/api_changes.md#alpha-beta-and-stable-versions).
+
+Here's a summary of each level:
+
+- Alpha:
+  - The version names contain `alpha` (for example, `v1alpha1`).
+  - The software may contain bugs. Enabling a feature may expose bugs. A
+    feature may be disabled by default.
+  - The support for a feature may be dropped at any time without notice.
+  - The API may change in incompatible ways in a later software release without notice.
+  - The software is recommended for use only in short-lived testing clusters,
+    due to increased risk of bugs and lack of long-term support.
+
+- Beta:
+  - The version names contain `beta` (for example, `v2beta3`).
+  - The software is well tested. Enabling a feature is considered safe.
+    Features are enabled by default.
+  - The support for a feature will not be dropped, though the details may change.
+
+  - The schema and/or semantics of objects may change in incompatible ways in
+    a subsequent beta or stable release. When this happens, migration
+    instructions are provided. Schema changes may require deleting, editing, and
+    re-creating API objects. The editing process may not be straightforward.
+    The migration may require downtime for applications that rely on the feature.
+  - The software is not recommended for production uses. Subsequent releases
+    may introduce incompatible changes. If you have multiple clusters which
+    can be upgraded independently, you may be able to relax this restriction.
+
+  {{< note >}}
+  Please try beta features and provide feedback. After the features exit beta, it
+  may not be practical to make more changes.
+  {{< /note >}}
+
+- Stable:
+  - The version name is `vX` where `X` is an integer.
+  - The stable versions of features appear in released software for many subsequent versions.
+
+## API groups
+
+[API groups](https://git.k8s.io/community/contributors/design-proposals/api-machinery/api-group.md)
+make it easier to extend the Kubernetes API.
+The API group is specified in a REST path and in the `apiVersion` field of a
+serialized object.
+
+There are several API groups in Kubernetes:
+
+*  The *core* (also called *legacy*) group is found at REST path `/api/v1`.
+   The core group is not specified as part of the `apiVersion` field, for
+   example, `apiVersion: v1`.
+*  The named groups are at REST path `/apis/$GROUP_NAME/$VERSION` and use
+   `apiVersion: $GROUP_NAME/$VERSION` (for example, `apiVersion: batch/v1`).
+   You can find the full list of supported API groups in
+   [Kubernetes API reference](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#-strong-api-groups-strong-).
+
+## Enabling or disabling API groups   {#enabling-or-disabling}
+
+Certain resources and API groups are enabled by default. You can enable or
+disable them by setting `--runtime-config` on the API server.  The
+`--runtime-config` flag accepts comma separated `<key>[=<value>]` pairs
+describing the runtime configuration of the API server. If the `=<value>`
+part is omitted, it is treated as if `=true` is specified. For example:
+
+ - to disable `batch/v1`, set `--runtime-config=batch/v1=false`
+ - to enable `batch/v2alpha1`, set `--runtime-config=batch/v2alpha1`
+
+{{< note >}}
+When you enable or disable groups or resources, you need to restart the API
+server and controller manager to pick up the `--runtime-config` changes.
+{{< /note >}}
+
+## Persistence
+
+Kubernetes stores its serialized state in terms of the API resources by writing them into
+{{< glossary_tooltip term_id="etcd" >}}.
+
+## {{% heading "whatsnext" %}}
+
+- Learn more about [API conventions](https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#api-conventions)
+- Read the design documentation for
+  [aggregator](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/aggregated-api-servers.md)
@@ -431,444 +431,14 @@ Some values of an object are typically generated before the object is persisted.

 ## Server Side Apply

-{{< feature-state for_k8s_version="v1.16" state="beta" >}}
-
-Starting from Kubernetes v1.18, if you have Server Side Apply enabled then the control plane tracks managed fields for all newly created objects.
-
-### Introduction
-
-Server Side Apply helps users and controllers manage their resources via
-declarative configurations. It allows them to create and/or modify their
-objects declaratively, simply by sending their fully specified intent.
-
-A fully specified intent is a partial object that only includes the fields and
-values for which the user has an opinion. That intent either creates a new
-object or is [combined](#merge-strategy), by the server, with the existing object.
-
-The system supports multiple appliers collaborating on a single object.
-
-Changes to an object's fields are tracked through a "[field management](#field-management)"
-mechanism. When a field's value changes, ownership moves from its current
-manager to the manager making the change. When trying to apply an object, fields
-that have a different value and are owned by another manager will result in a
-[conflict](#conflicts). This is done in order to signal that the operation might undo another
-collaborator's changes. Conflicts can be forced, in which case the value will be
-overridden, and the ownership will be transferred.
-
-If you remove a field from a configuration and apply the configuration, server side apply checks
-if there are any other field managers that also own the field.  If the field is
-not owned by any other field managers, it is either deleted from the live
-object or reset to its default value, if it has one. The same rule applies to associative list or
-map items.
-
-Server side apply is meant both as a replacement for the original `kubectl
-apply` and as a simpler mechanism for controllers to enact their changes.
-
-### Field Management
-
-Compared to the `last-applied` annotation managed by `kubectl`, Server Side
-Apply uses a more declarative approach, which tracks a user's field management,
-rather than a user's last applied state. This means that as a side effect of
-using Server Side Apply, information about which field manager manages each
-field in an object also becomes available.
-
-For a user to manage a field, in the Server Side Apply sense, means that the
-user relies on and expects the value of the field not to change. The user who
-last made an assertion about the value of a field will be recorded as the
-current field manager. This can be done either by changing the value with
-`POST`, `PUT`, or non-apply `PATCH`, or by including the field in a config sent
-to the Server Side Apply endpoint. When using Server-Side Apply, trying to
-change a field which is managed by someone else will result in a rejected
-request (if not forced, see [Conflicts](#conflicts)).
-
-When two or more appliers set a field to the same value, they share ownership of
-that field. Any subsequent attempt to change the value of the shared field, by any of
-the appliers, results in a conflict. Shared field owners may give up ownership
-of a field by removing it from their configuration.
-
-Field management is stored in a`managedFields` field that is part of an object's
-[`metadata`](/docs/reference/generated/kubernetes-api/{{< latest-version >}}/#objectmeta-v1-meta).
-
-A simple example of an object created by Server Side Apply could look like this:
-
-```yaml
-apiVersion: v1
-kind: ConfigMap
-metadata:
-  name: test-cm
-  namespace: default
-  labels:
-    test-label: test
-  managedFields:
-  - manager: kubectl
-    operation: Apply
-    apiVersion: v1
-    time: "2010-10-10T0:00:00Z"
-    fieldsType: FieldsV1
-    fieldsV1:
-      f:metadata:
-        f:labels:
-          f:test-label: {}
-      f:data:
-        f:key: {}
-data:
-  key: some value
-```
-
-The above object contains a single manager in `metadata.managedFields`. The
-manager consists of basic information about the managing entity itself, like
-operation type, API version, and the fields managed by it.
-
-{{< note >}}
-This field is managed by the  API server and should not be changed by
-the user.
-{{< /note >}}
-
-Nevertheless it is possible to change `metadata.managedFields` through an
-`Update` operation. Doing so is highly discouraged, but might be a reasonable
-option to try if, for example, the `managedFields` get into an inconsistent
-state (which clearly should not happen).
-
-The format of the `managedFields` is described in the [API](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#fieldsv1-v1-meta).
-
-### Conflicts
-
-A conflict is a special status error that occurs when an `Apply` operation tries
-to change a field, which another user also claims to manage. This prevents an
-applier from unintentionally overwriting the value set by another user. When
-this occurs, the applier has 3 options to resolve the conflicts:
-
-* **Overwrite value, become sole manager:** If overwriting the value was
-  intentional (or if the applier is an automated process like a controller) the
-  applier should set the `force` query parameter to true and make the request
-  again. This forces the operation to succeed, changes the value of the field,
-  and removes the field from all other managers' entries in managedFields.
-* **Don't overwrite value, give up management claim:** If the applier doesn't
-  care about the value of the field anymore, they can remove it from their
-  config and make the request again. This leaves the value unchanged, and causes
-  the field to be removed from the applier's entry in managedFields.
-* **Don't overwrite value, become shared manager:** If the applier still cares
-  about the value of the field, but doesn't want to overwrite it, they can
-  change the value of the field in their config to match the value of the object
-  on the server, and make the request again. This leaves the value unchanged,
-  and causes the field's management to be shared by the applier and all other
-  field managers that already claimed to manage it.
-
-
-### Managers
-
-Managers identify distinct workflows that are modifying the object (especially
-useful on conflicts!), and can be specified through the `fieldManager` query
-parameter as part of a modifying request. It is required for the apply endpoint,
-though kubectl will default it to `kubectl`. For other updates, its default is
-computed from the user-agent.
-
-### Apply and Update
-
-The two operation types considered by this feature are `Apply` (`PATCH` with
-content type `application/apply-patch+yaml`) and `Update` (all other operations
-which modify the object). Both operations update the `managedFields`, but behave
-a little differently.
-
-{{< note >}}
-Whether you are submitting JSON data or YAML data, use `application/apply-patch+yaml` as the
-`Content-Type` header value.
-
-All JSON documents are valid YAML.
-{{< /note >}}
-
-For instance, only the apply operation fails on conflicts while update does
-not. Also, apply operations are required to identify themselves by providing a
-`fieldManager` query parameter, while the query parameter is optional for update
-operations. Finally, when using the apply operation you cannot have `managedFields` in the object that is being applied.
-
-An example object with multiple managers could look like this:
-
-```yaml
-apiVersion: v1
-kind: ConfigMap
-metadata:
-  name: test-cm
-  namespace: default
-  labels:
-    test-label: test
-  managedFields:
-  - manager: kubectl
-    operation: Apply
-    apiVersion: v1
-    fields:
-      f:metadata:
-        f:labels:
-          f:test-label: {}
-  - manager: kube-controller-manager
-    operation: Update
-    apiVersion: v1
-    time: '2019-03-30T16:00:00.000Z'
-    fields:
-      f:data:
-        f:key: {}
-data:
-  key: new value
-```
-
-In this example, a second operation was run as an `Update` by the manager called
-`kube-controller-manager`. The update changed a value in the data field which
-caused the field's management to change to the `kube-controller-manager`.
-
-If this update would have been an `Apply` operation, the operation
-would have failed due to conflicting ownership.
-
-### Merge strategy
-
-The merging strategy, implemented with Server Side Apply, provides a generally
-more stable object lifecycle. Server Side Apply tries to merge fields based on
-the fact who manages them instead of overruling just based on values. This way
-it is intended to make it easier and more stable for multiple actors updating
-the same object by causing less unexpected interference.
-
-When a user sends a "fully-specified intent" object to the Server Side Apply
-endpoint, the server merges it with the live object favoring the value in the
-applied config if it is specified in both places. If the set of items present in
-the applied config is not a superset of the items applied by the same user last
-time, each missing item not managed by any other appliers is removed. For
-more information about how an object's schema is used to make decisions when
-merging, see
-[sigs.k8s.io/structured-merge-diff](https://sigs.k8s.io/structured-merge-diff).
-
-A number of markers were added in Kubernetes 1.16 and 1.17, to allow API
-developers to describe the merge strategy supported by lists, maps, and
-structs. These markers can be applied to objects of the respective type,
-in Go files or in the
-[OpenAPI schema definition of the CRD](/docs/reference/generated/kubernetes-api/{{< param "version" >}}#jsonschemaprops-v1-apiextensions-k8s-io):
-
-| Golang marker | OpenAPI extension | Accepted values | Description | Introduced in |
-|---|---|---|---|---|
-| `//+listType` | `x-kubernetes-list-type` | `atomic`/`set`/`map` | Applicable to lists. `atomic` and `set` apply to lists with scalar elements only. `map` applies to lists of nested types only. If configured as `atomic`, the entire list is replaced during merge; a single manager manages the list as a whole at any one time. If `granular`, different managers can manage entries separately. | 1.16          |
-| `//+listMapKey` | `x-kubernetes-list-map-keys` | Slice of map keys that uniquely identify entries for example `["port", "protocol"]` | Only applicable when `+listType=map`. A slice of strings whose values in combination must uniquely identify list entries. While there can be multiple keys, `listMapKey` is singular because keys need to be specified individually in the Go type. | 1.16 |
-| `//+mapType` | `x-kubernetes-map-type` | `atomic`/`granular` | Applicable to maps. `atomic` means that the map can only be entirely replaced by a single manager. `granular` means that the map supports separate managers updating individual fields. | 1.17 |
-| `//+structType` | `x-kubernetes-map-type` | `atomic`/`granular` | Applicable to structs; otherwise same usage and OpenAPI annotation as `//+mapType`.| 1.17 |
-
-### Custom Resources
-
-By default, Server Side Apply treats custom resources as unstructured data. All
-keys are treated the same as struct fields, and all lists are considered atomic.
-
-If the Custom Resource Definition defines a
-[schema](/docs/reference/generated/kubernetes-api/{{< param "version" >}}#jsonschemaprops-v1-apiextensions-k8s-io)
-that contains annotations as defined in the previous "Merge Strategy"
-section, these annotations will be used when merging objects of this
-type.
-
-### Using Server-Side Apply in a controller
-
-As a developer of a controller, you can use server-side apply as a way to
-simplify the update logic of your controller. The main differences with a
-read-modify-write and/or patch are the following:
-
-* the applied object must contain all the fields that the controller cares about.
-* there are no way to remove fields that haven't been applied by the controller
-  before (controller can still send a PATCH/UPDATE for these use-cases).
-* the object doesn't have to be read beforehand, `resourceVersion` doesn't have
-  to be specified.
-
-It is strongly recommended for controllers to always "force" conflicts, since they
-might not be able to resolve or act on these conflicts.
-
-### Transferring Ownership
-
-In addition to the concurrency controls provided by [conflict resolution](#conflicts),
-Server Side Apply provides ways to perform coordinated
-field ownership transfers from users to controllers.
-
-This is best explained by example. Let's look at how to safely transfer
-ownership of the `replicas` field from a user to a controller while enabling
-automatic horizontal scaling for a Deployment, using the HorizontalPodAutoscaler
-resource and its accompanying controller.
-
-Say a user has defined deployment with `replicas` set to the desired value:
-
-{{< codenew file="application/ssa/nginx-deployment.yaml" >}}
-
-And the user has created the deployment using server side apply like so:
-
-```shell
-kubectl apply -f https://k8s.io/examples/application/ssa/nginx-deployment.yaml --server-side
-```
-
-Then later, HPA is enabled for the deployment, e.g.:
-
-```shell
-kubectl autoscale deployment nginx-deployment --cpu-percent=50 --min=1 --max=10
-```
-
-Now, the user would like to remove `replicas` from their configuration, so they
-don't accidentally fight with the HPA controller. However, there is a race: it
-might take some time before HPA feels the need to adjust `replicas`, and if
-the user removes `replicas` before the HPA writes to the field and becomes
-its owner, then apiserver will set `replicas` to 1, its default value. This
-is not what the user wants to happen, even temporarily.
-
-There are two solutions:
-
- (easy) Leave `replicas` in the configuration; when HPA eventually writes to that
-  field, the system gives the user a conflict over it. At that point, it is safe
-  to remove from the configuration.
-
- (more advanced) If, however, the user doesn't want to wait, for example
-  because they want to keep the cluster legible to coworkers, then they can take
-  the following steps to make it safe to remove `replicas` from their
-  configuration:
-
-First, the user defines a new configuration containing only the `replicas` field:
-
-{{< codenew file="application/ssa/nginx-deployment-replicas-only.yaml" >}}
-
-The user applies that configuration using the field manager name `handover-to-hpa`:
-
-```shell
-kubectl apply -f https://k8s.io/examples/application/ssa/nginx-deployment-replicas-only.yaml --server-side --field-manager=handover-to-hpa --validate=false
-```
-
-If the apply results in a conflict with the HPA controller, then do nothing. The
-conflict just indicates the controller has claimed the field earlier in the
-process than it sometimes does.
-
-At this point the user may remove the `replicas` field from their configuration.
-
-{{< codenew file="application/ssa/nginx-deployment-no-replicas.yaml" >}}
-
-Note that whenever the HPA controller sets the `replicas` field to a new value,
-the temporary field manager will no longer own any fields and will be
-automatically deleted. No clean up is required.
-
-### Transferring Ownership Between Users
-
-Users can transfer ownership of a field between each other by setting the field
-to the same value in both of their applied configs, causing them to share
-ownership of the field. Once the users share ownership of the field, one of them
-can remove the field from their applied configuration to give up ownership and
-complete the transfer to the other user.
-
-### Comparison with Client Side Apply
-
-A consequence of the conflict detection and resolution implemented by Server
-Side Apply is that an applier always has up to date field values in their local
-state. If they don't, they get a conflict the next time they apply. Any of the
-three options to resolve conflicts results in the applied configuration being an
-up to date subset of the object on the server's fields.
-
-This is different from Client Side Apply, where outdated values which have been
-overwritten by other users are left in an applier's local config. These values
-only become accurate when the user updates that specific field, if ever, and an
-applier has no way of knowing whether their next apply will overwrite other
-users' changes.
-
-Another difference is that an applier using Client Side Apply is unable to
-change the API version they are using, but Server Side Apply supports this use
-case.
-
-### Upgrading from client-side apply to server-side apply
-
-Client-side apply users who manage a resource with `kubectl apply` can start
-using server-side apply with the following flag.
-
-```shell
-kubectl apply --server-side [--dry-run=server]
-```
-
-By default, field management of the object transfers from client-side apply
-to kubectl server-side apply without encountering conflicts.
-
-{{< caution >}}
-Keep the `last-applied-configuration` annotation up to date.
-The annotation infers client-side apply's managed fields.
-Any fields not managed by client-side apply raise conflicts.
-
-For example, if you used `kubectl scale` to update the replicas field after client-side apply,
-then this field is not owned by client-side apply and creates conflicts on `kubectl apply --server-side`.
-{{< /caution >}}
-
-This behavior applies to server-side apply with the `kubectl` field manager. As
-an exception, you can opt-out of this behavior by specifying a different,
-non-default field manager, as seen in the following example. The default field manager for kubectl
-server-side apply is `kubectl`.
-
-```shell
-kubectl apply --server-side --field-manager=my-manager [--dry-run=server]
-```
-
-### Downgrading from server-side apply to client-side apply
-
-If you manage a resource with `kubectl apply --server-side`,
-you can downgrade to client-side apply directly with `kubectl apply`.
-
-Downgrading works because kubectl server-side apply keeps the
-`last-applied-configuration` annotation up-to-date if you use
-`kubectl apply`.
-
-This behavior applies to server-side apply with the `kubectl` field manager. As
-an exception, you can opt-out of this behavior by specifying a different,
-non-default field manager, as seen in the following example. The default field manager for kubectl
-server-side apply is `kubectl`.
-
-```shell
-kubectl apply --server-side --field-manager=my-manager [--dry-run=server]
-```
-
-### API Endpoint
-
-With the Server Side Apply feature enabled, the `PATCH` endpoint accepts the
-additional `application/apply-patch+yaml` content type. Users of Server Side
-Apply can send partially specified objects as YAML to this endpoint.
-When applying a configuration, one should always include all the fields
-that they have an opinion about.
-
-### Clearing ManagedFields
-
-It is possible to strip all managedFields from an object by overwriting them
-using `MergePatch`, `StrategicMergePatch`, `JSONPatch` or `Update`, so every
-non-apply operation. This can be done by overwriting the managedFields field
-with an empty entry. Two examples are:
-
-```console
-PATCH /api/v1/namespaces/default/configmaps/example-cm
-Content-Type: application/merge-patch+json
-Accept: application/json
-Data: {"metadata":{"managedFields": [{}]}}
-```
-
-```console
-PATCH /api/v1/namespaces/default/configmaps/example-cm
-Content-Type: application/json-patch+json
-Accept: application/json
-Data: [{"op": "replace", "path": "/metadata/managedFields", "value": [{}]}]
-```
-
-This will overwrite the managedFields with a list containing a single empty
-entry that then results in the managedFields being stripped entirely from the
-object. Note that just setting the managedFields to an empty list will not reset
-the field. This is on purpose, so managedFields never get stripped by clients
-not aware of the field.
-
-In cases where the reset operation is combined with changes to other fields than
-the managedFields, this will result in the managedFields being reset first and
-the other changes being processed afterwards. As a result the applier takes
-ownership of any fields updated in the same request.
-
-{{< caution >}}
-Server Side Apply does not correctly track ownership on
-sub-resources that don't receive the resource object type. If you are
-using Server Side Apply with such a sub-resource, the changed fields
-won't be tracked.
-{{< /caution >}}
-
-### Disabling the feature
-
-Server Side Apply is a beta feature, so it is enabled by default. To turn this
-[feature gate](/docs/reference/command-line-tools-reference/feature-gates) off,
-you need to include the `--feature-gates ServerSideApply=false` flag when
-starting `kube-apiserver`. If you have multiple `kube-apiserver` replicas, all
-should have the same flag setting.
+Starting from Kubernetes v1.18, you can enable the
+[Server Side Apply](/docs/reference/using-api/server-side-apply/)
+feature so that the control plane tracks managed fields for all newly created objects.
+Server Side Apply provides a clear pattern for managing field conflicts,
+offers server-side `Apply` and `Update` operations, and replaces the
+client-side functionality of `kubectl apply`. For more details about this
+feature, see the section on
+[Server Side Apply](/docs/reference/using-api/server-side-apply/).

 ## Resource Versions

@@ -1,117 +0,0 @@
---
-title: Kubernetes API Overview
-reviewers:
- erictune
- lavalamp
- jbeda
-content_type: concept
-weight: 10
-card:
-  name: reference
-  weight: 50
-  title: Overview of API
---
-
-<!-- overview -->
-
-This page provides an overview of the Kubernetes API.
-
-<!-- body -->
-
-The REST API is the fundamental fabric of Kubernetes. All operations and
-communications between components, and external user commands are REST API
-calls that the API Server handles. Consequently, everything in the Kubernetes
-platform is treated as an API object and has a corresponding entry in the
-[API](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/).
-
-## API versioning
-
-The JSON and Protobuf serialization schemas follow the same guidelines for
-schema changes. The following descriptions cover both formats.
-
-The API versioning and software versioning are indirectly related.
-The [API and release versioning proposal](https://git.k8s.io/community/contributors/design-proposals/release/versioning.md)
-describes the relationship between API versioning and software versioning.
-
-Different API versions indicate different levels of stability and support. You
-can find more information about the criteria for each level in the
-[API Changes documentation](https://git.k8s.io/community/contributors/devel/sig-architecture/api_changes.md#alpha-beta-and-stable-versions).
-
-Here's a summary of each level:
-
- Alpha:
-  - The version names contain `alpha` (for example, `v1alpha1`).
-  - The software may contain bugs. Enabling a feature may expose bugs. A
-    feature may be disabled by default.
-  - The support for a feature may be dropped at any time without notice.
-  - The API may change in incompatible ways in a later software release without notice.
-  - The software is recommended for use only in short-lived testing clusters,
-    due to increased risk of bugs and lack of long-term support.
-  
- Beta:
-  - The version names contain `beta` (for example, `v2beta3`).
-  - The software is well tested. Enabling a feature is considered safe.
-    Features are enabled by default.
-  - The support for a feature will not be dropped, though the details may change.
-
-  - The schema and/or semantics of objects may change in incompatible ways in
-    a subsequent beta or stable release. When this happens, migration
-    instructions are provided. Schema changes may require deleting, editing, and
-    re-creating API objects. The editing process may not be straightforward.
-    The migration may require downtime for applications that rely on the feature.
-  - The software is not recommended for production uses. Subsequent releases
-    may introduce incompatible changes. If you have multiple clusters which
-    can be upgraded independently, you may be able to relax this restriction.
-  
-  {{< note >}}
-  Try the beta features and provide feedback. After the features exit beta, it
-  may not be practical to make more changes.
-  {{< /note >}}
-  
- Stable:
-  - The version name is `vX` where `X` is an integer.
-  - The stable versions of features appear in released software for many subsequent versions.
-
-## API groups
-
-[API groups](https://git.k8s.io/community/contributors/design-proposals/api-machinery/api-group.md)
-make it easier to extend the Kubernetes API.
-The API group is specified in a REST path and in the `apiVersion` field of a
-serialized object.
-
-Currently, there are several API groups in use:
-
-*  The *core* (also called *legacy*) group is found at REST path `/api/v1`.
-   The core group is not specified as part of the `apiVersion` field, for
-   example, `apiVersion: v1`.
-*  The named groups are at REST path `/apis/$GROUP_NAME/$VERSION` and use
-   `apiVersion: $GROUP_NAME/$VERSION` (for example, `apiVersion: batch/v1`).
-   You can find the full list of supported API groups in
-   [Kubernetes API reference](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/).
-
-## Enabling or disabling API groups   {#enabling-or-disabling}
-
-Certain resources and API groups are enabled by default. You can enable or
-disable them by setting `--runtime-config` on the API server.  The
-`--runtime-config` flag accepts comma separated `<key>=<value>` pairs
-describing the runtime configuration of the API server. For example:
-
- - to disable `batch/v1`, set `--runtime-config=batch/v1=false`
- - to enable `batch/v2alpha1`, set `--runtime-config=batch/v2alpha1`
-
-{{< note >}}
-When you enable or disable groups or resources, you need to restart the API
-server and controller manager to pick up the `--runtime-config` changes.
-{{< /note >}}
-
-## Persistence
-
-Kubernetes stores its serialized state in terms of the API resources by writing them into
-{{< glossary_tooltip term_id="etcd" >}}.
-
-## {{% heading "whatsnext" %}}
-
- Learn more about [API conventions](https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#api-conventions)
- Read the design documentation for
-  [aggregator](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/aggregated-api-servers.md)
-
@@ -12,7 +12,7 @@ API from various programming languages.


 <!-- body -->
-To write applications using the [Kubernetes REST API](/docs/reference/using-api/api-overview/),
+To write applications using the [Kubernetes REST API](/docs/reference/using-api/),
 you do not need to implement the API calls and request/response types yourself.
 You can use a client library for the programming language you are using.

@@ -24,7 +24,7 @@ a deprecation policy for aspects of the system that are slated to be removed.
 Since Kubernetes is an API-driven system, the API has evolved over time to
 reflect the evolving understanding of the problem space. The Kubernetes API is
 actually a set of APIs, called "API groups", and each API group is
-independently versioned.  [API versions](/docs/reference/using-api/api-overview/#api-versioning) fall
+independently versioned.  [API versions](/docs/reference/using-api/#api-versioning) fall
 into 3 main tracks, each of which has different policies for deprecation:

 | Example  | Track                            |
@@ -0,0 +1,462 @@
+---
+title: Server-Side Apply
+reviewers:
+- smarterclayton
+- apelisse
+- lavalamp
+- liggitt
+content_type: concept
+weight: 25
+min-kubernetes-server-version: 1.16
+---
+
+<!-- overview -->
+
+{{< feature-state for_k8s_version="v1.16" state="beta" >}}
+
+## Introduction
+
+Server Side Apply helps users and controllers manage their resources via
+declarative configurations. It allows them to create and/or modify their
+[objects](/docs/concepts/overview/working-with-objects/kubernetes-objects/)
+declaratively, simply by sending their fully specified intent.
+
+A fully specified intent is a partial object that only includes the fields and
+values for which the user has an opinion. That intent either creates a new
+object or is [combined](#merge-strategy), by the server, with the existing object.
+
+The system supports multiple appliers collaborating on a single object.
+
+Changes to an object's fields are tracked through a "[field management](#field-management)"
+mechanism. When a field's value changes, ownership moves from its current
+manager to the manager making the change. When trying to apply an object,
+fields that have a different value and are owned by another manager will
+result in a [conflict](#conflicts). This is done in order to signal that the
+operation might undo another collaborator's changes. Conflicts can be forced,
+in which case the value will be overridden, and the ownership will be
+transferred.
+
+If you remove a field from a configuration and apply the configuration, server
+side apply checks if there are any other field managers that also own the
+field.  If the field is not owned by any other field managers, it is either
+deleted from the live object or reset to its default value, if it has one. The
+same rule applies to associative list or map items.
+
+Server side apply is meant both as a replacement for the original `kubectl
+apply` and as a simpler mechanism for controllers to enact their changes.
+
+If you have Server Side Apply enabled, the control plane tracks managed fields
+for all newlly created objects.
+
+## Field Management
+
+Compared to the `last-applied` annotation managed by `kubectl`, Server Side
+Apply uses a more declarative approach, which tracks a user's field management,
+rather than a user's last applied state. This means that as a side effect of
+using Server Side Apply, information about which field manager manages each
+field in an object also becomes available.
+
+For a user to manage a field, in the Server Side Apply sense, means that the
+user relies on and expects the value of the field not to change. The user who
+last made an assertion about the value of a field will be recorded as the
+current field manager. This can be done either by changing the value with
+`POST`, `PUT`, or non-apply `PATCH`, or by including the field in a config sent
+to the Server Side Apply endpoint. When using Server-Side Apply, trying to
+change a field which is managed by someone else will result in a rejected
+request (if not forced, see [Conflicts](#conflicts)).
+
+When two or more appliers set a field to the same value, they share ownership of
+that field. Any subsequent attempt to change the value of the shared field, by any of
+the appliers, results in a conflict. Shared field owners may give up ownership
+of a field by removing it from their configuration.
+
+Field management is stored in a`managedFields` field that is part of an object's
+[`metadata`](/docs/reference/generated/kubernetes-api/{{< latest-version >}}/#objectmeta-v1-meta).
+
+A simple example of an object created by Server Side Apply could look like this:
+
+```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: test-cm
+  namespace: default
+  labels:
+    test-label: test
+  managedFields:
+  - manager: kubectl
+    operation: Apply
+    apiVersion: v1
+    time: "2010-10-10T0:00:00Z"
+    fieldsType: FieldsV1
+    fieldsV1:
+      f:metadata:
+        f:labels:
+          f:test-label: {}
+      f:data:
+        f:key: {}
+data:
+  key: some value
+```
+
+The above object contains a single manager in `metadata.managedFields`. The
+manager consists of basic information about the managing entity itself, like
+operation type, API version, and the fields managed by it.
+
+{{< note >}}
+This field is managed by the  API server and should not be changed by
+the user.
+{{< /note >}}
+
+Nevertheless it is possible to change `metadata.managedFields` through an
+`Update` operation. Doing so is highly discouraged, but might be a reasonable
+option to try if, for example, the `managedFields` get into an inconsistent
+state (which clearly should not happen).
+
+The format of the `managedFields` is described in the
+[API](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#fieldsv1-v1-meta).
+
+## Conflicts
+
+A conflict is a special status error that occurs when an `Apply` operation tries
+to change a field, which another user also claims to manage. This prevents an
+applier from unintentionally overwriting the value set by another user. When
+this occurs, the applier has 3 options to resolve the conflicts:
+
+* **Overwrite value, become sole manager:** If overwriting the value was
+  intentional (or if the applier is an automated process like a controller) the
+  applier should set the `force` query parameter to true and make the request
+  again. This forces the operation to succeed, changes the value of the field,
+  and removes the field from all other managers' entries in managedFields.
+
+* **Don't overwrite value, give up management claim:** If the applier doesn't
+  care about the value of the field anymore, they can remove it from their
+  config and make the request again. This leaves the value unchanged, and causes
+  the field to be removed from the applier's entry in managedFields.
+
+* **Don't overwrite value, become shared manager:** If the applier still cares
+  about the value of the field, but doesn't want to overwrite it, they can
+  change the value of the field in their config to match the value of the object
+  on the server, and make the request again. This leaves the value unchanged,
+  and causes the field's management to be shared by the applier and all other
+  field managers that already claimed to manage it.
+
+## Managers
+
+Managers identify distinct workflows that are modifying the object (especially
+useful on conflicts!), and can be specified through the `fieldManager` query
+parameter as part of a modifying request. It is required for the apply endpoint,
+though kubectl will default it to `kubectl`. For other updates, its default is
+computed from the user-agent.
+
+## Apply and Update
+
+The two operation types considered by this feature are `Apply` (`PATCH` with
+content type `application/apply-patch+yaml`) and `Update` (all other operations
+which modify the object). Both operations update the `managedFields`, but behave
+a little differently.
+
+{{< note >}}
+Whether you are submitting JSON data or YAML data, use
+`application/apply-patch+yaml` as the `Content-Type` header value.
+
+All JSON documents are valid YAML.
+{{< /note >}}
+
+For instance, only the apply operation fails on conflicts while update does
+not. Also, apply operations are required to identify themselves by providing a
+`fieldManager` query parameter, while the query parameter is optional for update
+operations. Finally, when using the apply operation you cannot have
+`managedFields` in the object that is being applied.
+
+An example object with multiple managers could look like this:
+
+```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: test-cm
+  namespace: default
+  labels:
+    test-label: test
+  managedFields:
+  - manager: kubectl
+    operation: Apply
+    apiVersion: v1
+    fields:
+      f:metadata:
+        f:labels:
+          f:test-label: {}
+  - manager: kube-controller-manager
+    operation: Update
+    apiVersion: v1
+    time: '2019-03-30T16:00:00.000Z'
+    fields:
+      f:data:
+        f:key: {}
+data:
+  key: new value
+```
+
+In this example, a second operation was run as an `Update` by the manager called
+`kube-controller-manager`. The update changed a value in the data field which
+caused the field's management to change to the `kube-controller-manager`.
+
+If this update would have been an `Apply` operation, the operation
+would have failed due to conflicting ownership.
+
+## Merge strategy
+
+The merging strategy, implemented with Server Side Apply, provides a generally
+more stable object lifecycle. Server Side Apply tries to merge fields based on
+the fact who manages them instead of overruling just based on values. This way
+it is intended to make it easier and more stable for multiple actors updating
+the same object by causing less unexpected interference.
+
+When a user sends a "fully-specified intent" object to the Server Side Apply
+endpoint, the server merges it with the live object favoring the value in the
+applied config if it is specified in both places. If the set of items present in
+the applied config is not a superset of the items applied by the same user last
+time, each missing item not managed by any other appliers is removed. For
+more information about how an object's schema is used to make decisions when
+merging, see
+[sigs.k8s.io/structured-merge-diff](https://sigs.k8s.io/structured-merge-diff).
+
+A number of markers were added in Kubernetes 1.16 and 1.17, to allow API
+developers to describe the merge strategy supported by lists, maps, and
+structs. These markers can be applied to objects of the respective type,
+in Go files or in the OpenAPI schema definition of the 
+[CRD](/docs/reference/generated/kubernetes-api/{{< param "version" >}}#jsonschemaprops-v1-apiextensions-k8s-io):
+
+| Golang marker | OpenAPI extension | Accepted values | Description | Introduced in |
+|---|---|---|---|---|
+| `//+listType` | `x-kubernetes-list-type` | `atomic`/`set`/`map` | Applicable to lists. `atomic` and `set` apply to lists with scalar elements only. `map` applies to lists of nested types only. If configured as `atomic`, the entire list is replaced during merge; a single manager manages the list as a whole at any one time. If `granular`, different managers can manage entries separately. | 1.16          |
+| `//+listMapKey` | `x-kubernetes-list-map-keys` | Slice of map keys that uniquely identify entries for example `["port", "protocol"]` | Only applicable when `+listType=map`. A slice of strings whose values in combination must uniquely identify list entries. While there can be multiple keys, `listMapKey` is singular because keys need to be specified individually in the Go type. | 1.16 |
+| `//+mapType` | `x-kubernetes-map-type` | `atomic`/`granular` | Applicable to maps. `atomic` means that the map can only be entirely replaced by a single manager. `granular` means that the map supports separate managers updating individual fields. | 1.17 |
+| `//+structType` | `x-kubernetes-map-type` | `atomic`/`granular` | Applicable to structs; otherwise same usage and OpenAPI annotation as `//+mapType`.| 1.17 |
+
+### Custom Resources
+
+By default, Server Side Apply treats custom resources as unstructured data. All
+keys are treated the same as struct fields, and all lists are considered atomic.
+
+If the Custom Resource Definition defines a
+[schema](/docs/reference/generated/kubernetes-api/{{< param "version" >}}#jsonschemaprops-v1-apiextensions-k8s-io)
+that contains annotations as defined in the previous "Merge Strategy"
+section, these annotations will be used when merging objects of this
+type.
+
+### Using Server-Side Apply in a controller
+
+As a developer of a controller, you can use server-side apply as a way to
+simplify the update logic of your controller. The main differences with a
+read-modify-write and/or patch are the following:
+
+* the applied object must contain all the fields that the controller cares about.
+* there are no way to remove fields that haven't been applied by the controller
+  before (controller can still send a PATCH/UPDATE for these use-cases).
+* the object doesn't have to be read beforehand, `resourceVersion` doesn't have
+  to be specified.
+
+It is strongly recommended for controllers to always "force" conflicts, since they
+might not be able to resolve or act on these conflicts.
+
+### Transferring Ownership
+
+In addition to the concurrency controls provided by [conflict resolution](#conflicts),
+Server Side Apply provides ways to perform coordinated
+field ownership transfers from users to controllers.
+
+This is best explained by example. Let's look at how to safely transfer
+ownership of the `replicas` field from a user to a controller while enabling
+automatic horizontal scaling for a Deployment, using the HorizontalPodAutoscaler
+resource and its accompanying controller.
+
+Say a user has defined deployment with `replicas` set to the desired value:
+
+{{< codenew file="application/ssa/nginx-deployment.yaml" >}}
+
+And the user has created the deployment using server side apply like so:
+
+```shell
+kubectl apply -f https://k8s.io/examples/application/ssa/nginx-deployment.yaml --server-side
+```
+
+Then later, HPA is enabled for the deployment, e.g.:
+
+```shell
+kubectl autoscale deployment nginx-deployment --cpu-percent=50 --min=1 --max=10
+```
+
+Now, the user would like to remove `replicas` from their configuration, so they
+don't accidentally fight with the HPA controller. However, there is a race: it
+might take some time before HPA feels the need to adjust `replicas`, and if
+the user removes `replicas` before the HPA writes to the field and becomes
+its owner, then apiserver will set `replicas` to 1, its default value. This
+is not what the user wants to happen, even temporarily.
+
+There are two solutions:
+
+- (easy) Leave `replicas` in the configuration; when HPA eventually writes to that
+  field, the system gives the user a conflict over it. At that point, it is safe
+  to remove from the configuration.
+
+- (more advanced) If, however, the user doesn't want to wait, for example
+  because they want to keep the cluster legible to coworkers, then they can take
+  the following steps to make it safe to remove `replicas` from their
+  configuration:
+
+First, the user defines a new configuration containing only the `replicas` field:
+
+{{< codenew file="application/ssa/nginx-deployment-replicas-only.yaml" >}}
+
+The user applies that configuration using the field manager name `handover-to-hpa`:
+
+```shell
+kubectl apply -f https://k8s.io/examples/application/ssa/nginx-deployment-replicas-only.yaml \
+  --server-side --field-manager=handover-to-hpa \
+  --validate=false
+```
+
+If the apply results in a conflict with the HPA controller, then do nothing. The
+conflict just indicates the controller has claimed the field earlier in the
+process than it sometimes does.
+
+At this point the user may remove the `replicas` field from their configuration.
+
+{{< codenew file="application/ssa/nginx-deployment-no-replicas.yaml" >}}
+
+Note that whenever the HPA controller sets the `replicas` field to a new value,
+the temporary field manager will no longer own any fields and will be
+automatically deleted. No clean up is required.
+
+## Transferring Ownership Between Users
+
+Users can transfer ownership of a field between each other by setting the field
+to the same value in both of their applied configs, causing them to share
+ownership of the field. Once the users share ownership of the field, one of them
+can remove the field from their applied configuration to give up ownership and
+complete the transfer to the other user.
+
+## Comparison with Client Side Apply
+
+A consequence of the conflict detection and resolution implemented by Server
+Side Apply is that an applier always has up to date field values in their local
+state. If they don't, they get a conflict the next time they apply. Any of the
+three options to resolve conflicts results in the applied configuration being an
+up to date subset of the object on the server's fields.
+
+This is different from Client Side Apply, where outdated values which have been
+overwritten by other users are left in an applier's local config. These values
+only become accurate when the user updates that specific field, if ever, and an
+applier has no way of knowing whether their next apply will overwrite other
+users' changes.
+
+Another difference is that an applier using Client Side Apply is unable to
+change the API version they are using, but Server Side Apply supports this use
+case.
+
+## Upgrading from client-side apply to server-side apply
+
+Client-side apply users who manage a resource with `kubectl apply` can start
+using server-side apply with the following flag.
+
+```shell
+kubectl apply --server-side [--dry-run=server]
+```
+
+By default, field management of the object transfers from client-side apply to
+kubectl server-side apply without encountering conflicts.
+
+{{< caution >}}
+Keep the `last-applied-configuration` annotation up to date.
+The annotation infers client-side apply's managed fields.
+Any fields not managed by client-side apply raise conflicts.
+
+For example, if you used `kubectl scale` to update the replicas field after
+client-side apply, then this field is not owned by client-side apply and
+creates conflicts on `kubectl apply --server-side`.
+{{< /caution >}}
+
+This behavior applies to server-side apply with the `kubectl` field manager.
+As an exception, you can opt-out of this behavior by specifying a different,
+non-default field manager, as seen in the following example. The default field
+manager for kubectl server-side apply is `kubectl`.
+
+```shell
+kubectl apply --server-side --field-manager=my-manager [--dry-run=server]
+```
+
+## Downgrading from server-side apply to client-side apply
+
+If you manage a resource with `kubectl apply --server-side`,
+you can downgrade to client-side apply directly with `kubectl apply`.
+
+Downgrading works because kubectl server-side apply keeps the
+`last-applied-configuration` annotation up-to-date if you use
+`kubectl apply`.
+
+This behavior applies to server-side apply with the `kubectl` field manager.
+As an exception, you can opt-out of this behavior by specifying a different,
+non-default field manager, as seen in the following example. The default field
+manager for kubectl server-side apply is `kubectl`.
+
+```shell
+kubectl apply --server-side --field-manager=my-manager [--dry-run=server]
+```
+
+## API Endpoint
+
+With the Server Side Apply feature enabled, the `PATCH` endpoint accepts the
+additional `application/apply-patch+yaml` content type. Users of Server Side
+Apply can send partially specified objects as YAML to this endpoint.  When
+applying a configuration, one should always include all the fields that they
+have an opinion about.
+
+## Clearing ManagedFields
+
+It is possible to strip all managedFields from an object by overwriting them
+using `MergePatch`, `StrategicMergePatch`, `JSONPatch` or `Update`, so every
+non-apply operation. This can be done by overwriting the managedFields field
+with an empty entry. Two examples are:
+
+```console
+PATCH /api/v1/namespaces/default/configmaps/example-cm
+Content-Type: application/merge-patch+json
+Accept: application/json
+Data: {"metadata":{"managedFields": [{}]}}
+```
+
+```console
+PATCH /api/v1/namespaces/default/configmaps/example-cm
+Content-Type: application/json-patch+json
+Accept: application/json
+Data: [{"op": "replace", "path": "/metadata/managedFields", "value": [{}]}]
+```
+
+This will overwrite the managedFields with a list containing a single empty
+entry that then results in the managedFields being stripped entirely from the
+object. Note that just setting the managedFields to an empty list will not
+reset the field. This is on purpose, so managedFields never get stripped by
+clients not aware of the field.
+
+In cases where the reset operation is combined with changes to other fields
+than the managedFields, this will result in the managedFields being reset
+first and the other changes being processed afterwards. As a result the
+applier takes ownership of any fields updated in the same request.
+
+{{< caution >}}
+Server Side Apply does not correctly track ownership on
+sub-resources that don't receive the resource object type. If you are
+using Server Side Apply with such a sub-resource, the changed fields
+won't be tracked.
+{{< /caution >}}
+
+## Disabling the feature
+
+Server Side Apply is a beta feature, so it is enabled by default. To turn this
+[feature gate](/docs/reference/command-line-tools-reference/feature-gates) off,
+you need to include the `--feature-gates ServerSideApply=false` flag when
+starting `kube-apiserver`. If you have multiple `kube-apiserver` replicas, all
+should have the same flag setting.
+
@@ -2,3 +2,32 @@
 title: Learning environment
 weight: 20
 ---
+
+<!--
+{{/* There is a Netlify redirect from this page to /docs/tasks/tools/ */}}
+{{/* This page content only exists to provide a navigation stub */}}
+{{/* and to protect in case that redirect is one day removed. */}}
+
+{{/* If you're localizing this page, you only need to copy the front matter */}}
+{{/* and add a redirect into "/static/_redirects", for YOUR localization. */}}
+-->
+
+## kind
+
+[`kind`](https://kind.sigs.k8s.io/docs/) lets you run Kubernetes on
+your local computer. This tool requires that you have
+[Docker](https://docs.docker.com/get-docker/) installed and configured.
+
+The kind [Quick Start](https://kind.sigs.k8s.io/docs/user/quick-start/) page
+shows you what you need to do to get up and running with kind.
+
+## minikube
+
+Like `kind`, [`minikube`](https://minikube.sigs.k8s.io/) is a tool that lets you run Kubernetes
+locally. `minikube` runs a single-node Kubernetes cluster on your personal
+computer (including Windows, macOS and Linux PCs) so that you can try out
+Kubernetes, or for daily development work.
+
+You can follow the official
+[Get Started!](https://minikube.sigs.k8s.io/docs/start/) guide if your focus is
+on getting the tool installed.
@@ -1,23 +0,0 @@
---
-title: Installing Kubernetes with Kind
-weight: 40
-content_type: concept
---
-
-<!-- overview -->
-
-Kind is a tool for running local Kubernetes clusters using Docker container "nodes".
-
-
-
-<!-- body -->
-
-## Installation
-
-See [Installing Kind](https://kind.sigs.k8s.io/docs/user/quick-start/).
-
-
-
-
-
-
@@ -1,519 +0,0 @@
---
-reviewers:
- dlorenc
- balopat
- aaron-prindle
-title: Installing Kubernetes with Minikube
-weight: 30
-content_type: concept
---
-
-<!-- overview -->
-
-Minikube is a tool that makes it easy to run Kubernetes locally. Minikube runs a single-node Kubernetes cluster inside a Virtual Machine (VM) on your laptop for users looking to try out Kubernetes or develop with it day-to-day.
-
-
-
-<!-- body -->
-
-## Minikube Features
-
-Minikube supports the following Kubernetes features:
-
-* DNS
-* NodePorts
-* ConfigMaps and Secrets
-* Dashboards
-* Container Runtime: [Docker](https://www.docker.com/), [CRI-O](https://cri-o.io/), and [containerd](https://github.com/containerd/containerd)
-* Enabling CNI (Container Network Interface)
-* Ingress
-
-## Installation
-
-See [Installing Minikube](/docs/tasks/tools/install-minikube/).
-
-## Quickstart
-
-This brief demo guides you on how to start, use, and delete Minikube locally. Follow the steps given below to start and explore Minikube.
-
-1. Start Minikube and create a cluster:
-
-   ```shell
-    minikube start
-   ```
-
-   The output is similar to this:
-
-   ```
-   Starting local Kubernetes cluster...
-   Running pre-create checks...
-   Creating machine...
-   Starting local Kubernetes cluster...
-   ```
-
-   For more information on starting your cluster on a specific Kubernetes version, VM, or container runtime, see [Starting a Cluster](#starting-a-cluster).
-
-2. Now, you can interact with your cluster using kubectl. For more information, see [Interacting with Your Cluster](#interacting-with-your-cluster).
-
-   Let's create a Kubernetes Deployment using an existing image named `echoserver`, which is a simple HTTP server and expose it on port 8080 using `--port`.
-
-   ```shell
-   kubectl create deployment hello-minikube --image=k8s.gcr.io/echoserver:1.10
-   ```
-
-   The output is similar to this:
-
-   ```
-   deployment.apps/hello-minikube created
-   ```
-3. To access the `hello-minikube` Deployment, expose it as a Service:
-
-   ```shell
-   kubectl expose deployment hello-minikube --type=NodePort --port=8080
-   ```
-
-   The option `--type=NodePort` specifies the type of the Service.
-
-   The output is similar to this:
-
-   ```
-   service/hello-minikube exposed
-   ```
-
-4. The `hello-minikube` Pod is now launched but you have to wait until the Pod is up before accessing it via the exposed Service.
-
-   Check if the Pod is up and running:
-
-   ```shell
-   kubectl get pod
-   ```
-
-   If the output shows the `STATUS` as `ContainerCreating`, the Pod is still being created:
-
-   ```
-   NAME                              READY     STATUS              RESTARTS   AGE
-   hello-minikube-3383150820-vctvh   0/1       ContainerCreating   0          3s
-   ```
-
-   If the output shows the `STATUS` as `Running`, the Pod is now up and running:
-
-   ```
-   NAME                              READY     STATUS    RESTARTS   AGE
-   hello-minikube-3383150820-vctvh   1/1       Running   0          13s
-   ```
-
-5. Get the URL of the exposed Service to view the Service details:
-
-   ```shell
-   minikube service hello-minikube --url
-   ```
-
-6. To view the details of your local cluster, copy and paste the URL you got as the output, on your browser.
-
-   The output is similar to this:
-
-   ```
-   Hostname: hello-minikube-7c77b68cff-8wdzq
-
-   Pod Information:
-    -no pod information available-
-
-   Server values:
-    server_version=nginx: 1.13.3 - lua: 10008
-
-   Request Information:
-    client_address=172.17.0.1
-    method=GET
-    real path=/
-    query=
-    request_version=1.1
-    request_scheme=http
-    request_uri=http://192.168.99.100:8080/
-
-   Request Headers:
-        accept=*/*
-        host=192.168.99.100:30674
-        user-agent=curl/7.47.0
-
-   Request Body:
-        -no body in request-
-   ```
-
-   If you no longer want the Service and cluster to run, you can delete them.
-
-7. Delete the `hello-minikube` Service:
-
-   ```shell
-   kubectl delete services hello-minikube
-   ```
-
-   The output is similar to this:
-
-   ```
-   service "hello-minikube" deleted
-   ```
-
-8. Delete the `hello-minikube` Deployment:
-
-   ```shell
-   kubectl delete deployment hello-minikube
-   ```
-
-   The output is similar to this:
-
-   ```
-   deployment.extensions "hello-minikube" deleted
-   ```
-
-9. Stop the local Minikube cluster:
-
-   ```shell
-   minikube stop
-   ```
-
-   The output is similar to this:
-
-   ```
-   Stopping "minikube"...
-   "minikube" stopped.
-   ```
-
-   For more information, see [Stopping a Cluster](#stopping-a-cluster).
-
-10. Delete the local Minikube cluster:
-
-    ```shell
-    minikube delete
-    ```
-    The output is similar to this:
-    ```
-    Deleting "minikube" ...
-    The "minikube" cluster has been deleted.
-    ```
-	For more information, see [Deleting a cluster](#deleting-a-cluster).
-
-## Managing your Cluster
-
-### Starting a Cluster
-
-The `minikube start` command can be used to start your cluster.
-This command creates and configures a Virtual Machine that runs a single-node Kubernetes cluster.
-This command also configures your [kubectl](/docs/reference/kubectl/overview/) installation to communicate with this cluster.
-
-{{< note >}}
-If you are behind a web proxy, you need to pass this information to the `minikube start` command:
-
-```shell
-https_proxy=<my proxy> minikube start --docker-env http_proxy=<my proxy> --docker-env https_proxy=<my proxy> --docker-env no_proxy=192.168.99.0/24
-```
-Unfortunately, setting the environment variables alone does not work.
-
-Minikube also creates a "minikube" context, and sets it to default in kubectl.
-To switch back to this context, run this command: `kubectl config use-context minikube`.
-{{< /note >}}
-
-#### Specifying the Kubernetes version
-
-You can specify the version of Kubernetes for Minikube to use by
-adding the `--kubernetes-version` string to the `minikube start` command. For
-example, to run version {{< param "fullversion" >}}, you would run the following:
-
-```
-minikube start --kubernetes-version {{< param "fullversion" >}}
-```
-#### Specifying the VM driver
-You can change the VM driver by adding the `--driver=<enter_driver_name>` flag to `minikube start`.
-For example the command would be.
-```shell
-minikube start --driver=<driver_name>
-```
-Minikube supports the following drivers:
-{{< note >}}
-See [DRIVERS](https://minikube.sigs.k8s.io/docs/reference/drivers/) for details on supported drivers and how to install
-plugins.
-{{< /note >}}
-
-* docker ([driver installation](https://minikube.sigs.k8s.io/docs/drivers/docker/))
-* virtualbox ([driver installation](https://minikube.sigs.k8s.io/docs/drivers/virtualbox/))
-* podman ([driver installation](https://minikube.sigs.k8s.io/docs/drivers/podman/)) (EXPERIMENTAL)
-* vmwarefusion
-* kvm2 ([driver installation](https://minikube.sigs.k8s.io/docs/reference/drivers/kvm2/))
-* hyperkit ([driver installation](https://minikube.sigs.k8s.io/docs/reference/drivers/hyperkit/))
-* hyperv ([driver installation](https://minikube.sigs.k8s.io/docs/reference/drivers/hyperv/))
-Note that the IP below is dynamic and can change. It can be retrieved with `minikube ip`.
-* vmware ([driver installation](https://minikube.sigs.k8s.io/docs/reference/drivers/vmware/)) (VMware unified driver)
-* parallels ([driver installation](https://minikube.sigs.k8s.io/docs/reference/drivers/parallels/))
-* none (Runs the Kubernetes components on the host and not in a virtual machine. You need to be running Linux and to have {{< glossary_tooltip term_id="docker" >}} installed.)
-
-{{< caution >}}
-If you use the `none` driver, some Kubernetes components run as privileged containers that have side effects outside of the Minikube environment. Those side effects mean that the `none` driver is not recommended for personal workstations.
-{{< /caution >}}
-
-#### Starting a cluster on alternative container runtimes
-You can start Minikube on the following container runtimes.
-{{< tabs name="container_runtimes" >}}
-{{% tab name="containerd" %}}
-To use [containerd](https://github.com/containerd/containerd) as the container runtime, run:
-```bash
-minikube start \
-    --network-plugin=cni \
-    --enable-default-cni \
-    --container-runtime=containerd \
-    --bootstrapper=kubeadm
-```
-
-Or you can use the extended version:
-
-```bash
-minikube start \
-    --network-plugin=cni \
-    --enable-default-cni \
-    --extra-config=kubelet.container-runtime=remote \
-    --extra-config=kubelet.container-runtime-endpoint=unix:///run/containerd/containerd.sock \
-    --extra-config=kubelet.image-service-endpoint=unix:///run/containerd/containerd.sock \
-    --bootstrapper=kubeadm
-```
-{{% /tab %}}
-{{% tab name="CRI-O" %}}
-To use [CRI-O](https://cri-o.io/) as the container runtime, run:
-```bash
-minikube start \
-    --network-plugin=cni \
-    --enable-default-cni \
-    --container-runtime=cri-o \
-    --bootstrapper=kubeadm
-```
-Or you can use the extended version:
-
-```bash
-minikube start \
-    --network-plugin=cni \
-    --enable-default-cni \
-    --extra-config=kubelet.container-runtime=remote \
-    --extra-config=kubelet.container-runtime-endpoint=/var/run/crio.sock \
-    --extra-config=kubelet.image-service-endpoint=/var/run/crio.sock \
-    --bootstrapper=kubeadm
-```
-{{% /tab %}}
-{{< /tabs >}}
-
-#### Use local images by re-using the Docker daemon
-
-When using a single VM for Kubernetes, it's useful to reuse Minikube's built-in Docker daemon. Reusing the built-in daemon means you don't have to build a Docker registry on your host machine and push the image into it. Instead, you can build inside the same Docker daemon as Minikube, which speeds up local experiments.
-
-{{< note >}}
-Be sure to tag your Docker image with something other than latest and use that tag to pull the image. Because `:latest` is the default value, with a corresponding default image pull policy of `Always`, an image pull error (`ErrImagePull`) eventually results if you do not have the Docker image in the default Docker registry (usually DockerHub).
-{{< /note >}}
-
-To work with the Docker daemon on your Mac/Linux host, run the last line from `minikube docker-env`.
-
-You can now use Docker at the command line of your host Mac/Linux machine to communicate with the Docker daemon inside the Minikube VM:
-
-```shell
-docker ps
-```
-
-{{< note >}}
-On Centos 7, Docker may report the following error:
-
-```
-Could not read CA certificate "/etc/docker/ca.pem": open /etc/docker/ca.pem: no such file or directory
-```
-
-You can fix this by updating /etc/sysconfig/docker to ensure that Minikube's environment changes are respected:
-
-```shell
-< DOCKER_CERT_PATH=/etc/docker
---
-> if [ -z "${DOCKER_CERT_PATH}" ]; then
->   DOCKER_CERT_PATH=/etc/docker
-> fi
-```
-{{< /note >}}
-
-### Configuring Kubernetes
-
-Minikube has a "configurator" feature that allows users to configure the Kubernetes components with arbitrary values.
-To use this feature, you can use the `--extra-config` flag on the `minikube start` command.
-
-This flag is repeated, so you can pass it several times with several different values to set multiple options.
-
-This flag takes a string of the form `component.key=value`, where `component` is one of the strings from the below list, `key` is a value on the
-configuration struct and `value` is the value to set.
-
-Valid keys can be found by examining the documentation for the Kubernetes `componentconfigs` for each component.
-Here is the documentation for each supported configuration:
-
-* [kubelet](https://godoc.org/k8s.io/kubernetes/pkg/kubelet/apis/config#KubeletConfiguration)
-* [apiserver](https://godoc.org/k8s.io/kubernetes/cmd/kube-apiserver/app/options#ServerRunOptions)
-* [proxy](https://godoc.org/k8s.io/kubernetes/pkg/proxy/apis/config#KubeProxyConfiguration)
-* [controller-manager](https://godoc.org/k8s.io/kubernetes/pkg/controller/apis/config#KubeControllerManagerConfiguration)
-* [etcd](https://godoc.org/github.com/coreos/etcd/etcdserver#ServerConfig)
-* [scheduler](https://godoc.org/k8s.io/kubernetes/pkg/scheduler/apis/config#KubeSchedulerConfiguration)
-
-#### Examples
-
-To change the `MaxPods` setting to 5 on the Kubelet, pass this flag: `--extra-config=kubelet.MaxPods=5`.
-
-This feature also supports nested structs. To change the `LeaderElection.LeaderElect` setting to `true` on the scheduler, pass this flag: `--extra-config=scheduler.LeaderElection.LeaderElect=true`.
-
-To set the `AuthorizationMode` on the `apiserver` to `RBAC`, you can use: `--extra-config=apiserver.authorization-mode=RBAC`.
-
-### Stopping a Cluster
-The `minikube stop` command can be used to stop your cluster.
-This command shuts down the Minikube Virtual Machine, but preserves all cluster state and data.
-Starting the cluster again will restore it to its previous state.
-
-### Deleting a Cluster
-The `minikube delete` command can be used to delete your cluster.
-This command shuts down and deletes the Minikube Virtual Machine. No data or state is preserved.
-
-### Upgrading Minikube
-If you are using macOS and [Brew Package Manager](https://brew.sh/) is installed run:
-
-```shell
-brew update
-brew upgrade minikube
-```
-
-## Interacting with Your Cluster
-
-### Kubectl
-
-The `minikube start` command creates a [kubectl context](/docs/reference/generated/kubectl/kubectl-commands#-em-set-context-em-) called "minikube".
-This context contains the configuration to communicate with your Minikube cluster.
-
-Minikube sets this context to default automatically, but if you need to switch back to it in the future, run:
-
-`kubectl config use-context minikube`
-
-Or pass the context on each command like this:
-
-`kubectl get pods --context=minikube`
-
-### Dashboard
-
-To access the [Kubernetes Dashboard](/docs/tasks/access-application-cluster/web-ui-dashboard/), run this command in a shell after starting Minikube to get the address:
-
-```shell
-minikube dashboard
-```
-
-### Services
-
-To access a Service exposed via a node port, run this command in a shell after starting Minikube to get the address:
-
-```shell
-minikube service [-n NAMESPACE] [--url] NAME
-```
-
-## Networking
-
-The Minikube VM is exposed to the host system via a host-only IP address, that can be obtained with the `minikube ip` command.
-Any services of type `NodePort` can be accessed over that IP address, on the NodePort.
-
-To determine the NodePort for your service, you can use a `kubectl` command like this:
-
-`kubectl get service $SERVICE --output='jsonpath="{.spec.ports[0].nodePort}"'`
-
-## Persistent Volumes
-Minikube supports [PersistentVolumes](/docs/concepts/storage/persistent-volumes/) of type `hostPath`.
-These PersistentVolumes are mapped to a directory inside the Minikube VM.
-
-The Minikube VM boots into a tmpfs, so most directories will not be persisted across reboots (`minikube stop`).
-However, Minikube is configured to persist files stored under the following host directories:
-
-* `/data`
-* `/var/lib/minikube`
-* `/var/lib/docker`
-
-Here is an example PersistentVolume config to persist data in the `/data` directory:
-
-```yaml
-apiVersion: v1
-kind: PersistentVolume
-metadata:
-  name: pv0001
-spec:
-  accessModes:
-    - ReadWriteOnce
-  capacity:
-    storage: 5Gi
-  hostPath:
-    path: /data/pv0001/
-```
-
-## Mounted Host Folders
-Some drivers will mount a host folder within the VM so that you can easily share files between the VM and host.  These are not configurable at the moment and different for the driver and OS you are using.
-
-{{< note >}}
-Host folder sharing is not implemented in the KVM driver yet.
-{{< /note >}}
-
-| Driver | OS | HostFolder | VM |
-| --- | --- | --- | --- |
-| VirtualBox | Linux | `/home` | `/hosthome` |
-| VirtualBox | macOS | `/Users` | `/Users` |
-| VirtualBox | Windows | `C://Users` | `/c/Users` |
-| VMware Fusion | macOS | `/Users` | `/mnt/hgfs/Users` |
-| Xhyve | macOS | `/Users` | `/Users` |
-
-## Private Container Registries
-
-To access a private container registry, follow the steps on [this page](/docs/concepts/containers/images/).
-
-We recommend you use `ImagePullSecrets`, but if you would like to configure access on the Minikube VM you can place the `.dockercfg` in the `/home/docker` directory or the `config.json` in the `/home/docker/.docker` directory.
-
-## Add-ons
-
-In order to have Minikube properly start or restart custom addons,
-place the addons you wish to be launched with Minikube in the `~/.minikube/addons`
-directory. Addons in this folder will be moved to the Minikube VM and
-launched each time Minikube is started or restarted.
-
-## Using Minikube with an HTTP Proxy
-
-Minikube creates a Virtual Machine that includes Kubernetes and a Docker daemon.
-When Kubernetes attempts to schedule containers using Docker, the Docker daemon may require external network access to pull containers.
-
-If you are behind an HTTP proxy, you may need to supply Docker with the proxy settings.
-To do this, pass the required environment variables as flags during `minikube start`.
-
-For example:
-
-```shell
-minikube start --docker-env http_proxy=http://$YOURPROXY:PORT \
-                 --docker-env https_proxy=https://$YOURPROXY:PORT
-```
-
-If your Virtual Machine address is 192.168.99.100, then chances are your proxy settings will prevent `kubectl` from directly reaching it.
-To by-pass proxy configuration for this IP address, you should modify your no_proxy settings. You can do so with:
-
-```shell
-export no_proxy=$no_proxy,$(minikube ip)
-```
-
-## Known Issues
-
-Features that require multiple nodes will not work in Minikube.
-
-## Design
-
-Minikube uses [libmachine](https://github.com/docker/machine/tree/master/libmachine) for provisioning VMs, and [kubeadm](https://github.com/kubernetes/kubeadm) to provision a Kubernetes cluster.
-
-For more information about Minikube, see the [proposal](https://git.k8s.io/community/contributors/design-proposals/cluster-lifecycle/local-cluster-ux.md).
-
-## Additional Links
-
-* **Goals and Non-Goals**: For the goals and non-goals of the Minikube project, please see our [roadmap](https://minikube.sigs.k8s.io/docs/contrib/roadmap/).
-* **Development Guide**: See [Contributing](https://minikube.sigs.k8s.io/docs/contrib/) for an overview of how to send pull requests.
-* **Building Minikube**: For instructions on how to build/test Minikube from source, see the [build guide](https://minikube.sigs.k8s.io/docs/contrib/building/).
-* **Adding a New Dependency**: For instructions on how to add a new dependency to Minikube, see the [adding dependencies guide](https://minikube.sigs.k8s.io/docs/contrib/drivers/).
-* **Adding a New Addon**: For instructions on how to add a new addon for Minikube, see the [adding an addon guide](https://minikube.sigs.k8s.io/docs/contrib/addons/).
-* **MicroK8s**: Linux users wishing to avoid running a virtual machine may consider [MicroK8s](https://microk8s.io/) as an alternative.
-
-## Community
-
-Contributions, questions, and comments are all welcomed and encouraged! Minikube developers hang out on [Slack](https://kubernetes.slack.com) in the `#minikube` channel (get an invitation [here](https://slack.kubernetes.io/)). We also have the [kubernetes-dev Google Groups mailing list](https://groups.google.com/forum/#!forum/kubernetes-dev). If you are posting to the list please prefix your subject with "minikube: ".
-
-
@@ -7,62 +7,368 @@ content_type: concept
 weight: 10
 ---
 <!-- overview -->
-{{< feature-state for_k8s_version="v1.6" state="stable" >}}
-To run containers in Pods, Kubernetes uses a container runtime. Here are
-the installation instructions for various runtimes.
-

+You need to install a
+{{< glossary_tooltip text="container runtime" term_id="container-runtime" >}}
+into each node in the cluster so that Pods can run there. This page outlines
+what is involved and describes related tasks for setting up nodes.

 <!-- body -->

+This page lists details for using several common container runtimes with
+Kubernetes, on Linux:

-{{< caution >}}
-A flaw was found in the way runc handled system file descriptors when running containers.
-A malicious container could use this flaw to overwrite contents of the runc binary and
-consequently run arbitrary commands on the container host system.
-
-Please refer to [CVE-2019-5736](https://access.redhat.com/security/cve/cve-2019-5736) for more
-information about the issue.
-{{< /caution >}}
-
-### Applicability
+- [containerd](#containerd)
+- [CRI-O](#cri-o)
+- [Docker](#docker)

 {{< note >}}
-This document is written for users installing CRI onto Linux. For other operating
-systems, look for documentation specific to your platform.
+For other operating systems, look for documentation specific to your platform.
 {{< /note >}}

-### Cgroup drivers
-
-When systemd is chosen as the init system for a Linux distribution, the init process generates
-and consumes a root control group (`cgroup`) and acts as a cgroup manager. Systemd has a tight
-integration with cgroups and will allocate cgroups per process. It's possible to configure your
-container runtime and the kubelet to use `cgroupfs`. Using `cgroupfs` alongside systemd means
-that there will be two different cgroup managers.
+## Cgroup drivers

 Control groups are used to constrain resources that are allocated to processes.
-A single cgroup manager will simplify the view of what resources are being allocated
-and will by default have a more consistent view of the available and in-use resources. When we have
-two managers we end up with two views of those resources. We have seen cases in the field
-where nodes that are configured to use `cgroupfs` for the kubelet and Docker, and `systemd`
-for the rest of the processes running on the node becomes unstable under resource pressure.
+
+When [systemd](https://www.freedesktop.org/wiki/Software/systemd/) is chosen as the init
+system for a Linux distribution, the init process generates and consumes a root control group
+(`cgroup`) and acts as a cgroup manager.
+Systemd has a tight integration with cgroups and allocates a cgroup per systemd unit. It's possible
+to configure your container runtime and the kubelet to use `cgroupfs`. Using `cgroupfs` alongside
+systemd means that there will be two different cgroup managers.
+
+A single cgroup manager simplifies the view of what resources are being allocated
+and will by default have a more consistent view of the available and in-use resources.
+When there are two cgroup managers on a system, you end up with two views of those resources.
+In the field, people have reported cases where nodes that are configured to use `cgroupfs`
+for the kubelet and Docker, but `systemd` for the rest of the processes, become unstable under
+resource pressure.

 Changing the settings such that your container runtime and kubelet use `systemd` as the cgroup driver
-stabilized the system. Please note the `native.cgroupdriver=systemd` option in the Docker setup below.
+stabilized the system. To configure this for Docker, set `native.cgroupdriver=systemd`.

 {{< caution >}}
-Changing the cgroup driver of a Node that has joined a cluster is highly unrecommended.
+Changing the cgroup driver of a Node that has joined a cluster is strongly *not* recommended.  
 If the kubelet has created Pods using the semantics of one cgroup driver, changing the container
-runtime to another cgroup driver can cause errors when trying to re-create the PodSandbox
-for such existing Pods. Restarting the kubelet may not solve such errors. The recommendation
-is to drain the Node from its workloads, remove it from the cluster and re-join it.
+runtime to another cgroup driver can cause errors when trying to re-create the Pod sandbox
+for such existing Pods. Restarting the kubelet may not solve such errors.
+
+If you have automation that makes it feasible, replace the node with another using the updated
+configuration, or reinstall it using automation.
 {{< /caution >}}

-## Docker
+## Container runtimes

-On each of your machines, install Docker.
-Version 19.03.11 is recommended, but 1.13.1, 17.03, 17.06, 17.09, 18.06 and 18.09 are known to work as well.
-Keep track of the latest verified Docker version in the Kubernetes release notes.
+{{% thirdparty-content %}}
+
+### containerd
+
+This section contains the necessary steps to use `containerd` as CRI runtime.
+
+Use the following commands to install Containerd on your system:
+
+Install and configure prerequisites:
+
+```shell
+cat <<EOF | sudo tee /etc/modules-load.d/containerd.conf
+overlay
+br_netfilter
+EOF
+
+sudo modprobe overlay
+sudo modprobe br_netfilter
+
+# Setup required sysctl params, these persist across reboots.
+cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
+net.bridge.bridge-nf-call-iptables  = 1
+net.ipv4.ip_forward                 = 1
+net.bridge.bridge-nf-call-ip6tables = 1
+EOF
+
+sudo sysctl --system
+```
+
+Install containerd:
+
+{{< tabs name="tab-cri-containerd-installation" >}}
+{{% tab name="Ubuntu 16.04" %}}
+
+```shell
+# (Install containerd)
+## Set up the repository
+### Install packages to allow apt to use a repository over HTTPS
+sudo apt-get update && sudo apt-get install -y apt-transport-https ca-certificates curl software-properties-common
+```
+
+```shell
+## Add Docker's official GPG key
+curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add --keyring /etc/apt/trusted.gpg.d/docker.gpg -
+```
+
+```shell
+## Add Docker apt repository.
+sudo add-apt-repository \
+    "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
+    $(lsb_release -cs) \
+    stable"
+```
+
+```shell
+## Install containerd
+sudo apt-get update && sudo apt-get install -y containerd.io
+```
+
+```shell
+# Configure containerd
+sudo mkdir -p /etc/containerd
+sudo containerd config default > /etc/containerd/config.toml
+```
+
+```shell
+# Restart containerd
+sudo systemctl restart containerd
+```
+{{% /tab %}}
+{{% tab name="CentOS/RHEL 7.4+" %}}
+
+```shell
+# (Install containerd)
+## Set up the repository
+### Install required packages
+sudo yum install -y yum-utils device-mapper-persistent-data lvm2
+```
+
+```shell
+## Add docker repository
+sudo yum-config-manager \
+    --add-repo \
+    https://download.docker.com/linux/centos/docker-ce.repo
+```
+
+```shell
+## Install containerd
+sudo yum update -y && sudo yum install -y containerd.io
+```
+
+```shell
+## Configure containerd
+sudo mkdir -p /etc/containerd
+sudo containerd config default > /etc/containerd/config.toml
+```
+
+```shell
+# Restart containerd
+sudo systemctl restart containerd
+```
+{{% /tab %}}
+{{% tab name="Windows (PowerShell)" %}}
+```powershell
+# (Install containerd)
+# download containerd
+cmd /c curl -OL https://github.com/containerd/containerd/releases/download/v1.4.0-beta.2/containerd-1.4.0-beta.2-windows-amd64.tar.gz
+cmd /c tar xvf .\containerd-1.4.0-beta.2-windows-amd64.tar.gz
+```
+
+```powershell
+# extract and configure
+Copy-Item -Path ".\bin\" -Destination "$Env:ProgramFiles\containerd" -Recurse -Force
+cd $Env:ProgramFiles\containerd\
+.\containerd.exe config default | Out-File config.toml -Encoding ascii
+
+# review the configuration. depending on setup you may want to adjust:
+# - the sandbox_image (kubernetes pause image)
+# - cni bin_dir and conf_dir locations
+Get-Content config.toml
+```
+
+```powershell
+# start containerd
+.\containerd.exe --register-service
+Start-Service containerd
+```
+{{% /tab %}}
+{{< /tabs >}}
+
+#### systemd
+
+To use the `systemd` cgroup driver in `/etc/containerd/config.toml` with `runc`, set
+
+```
+[plugins.cri]
+systemd_cgroup = true
+```
+
+When using kubeadm, manually configure the
+[cgroup driver for kubelet](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#configure-cgroup-driver-used-by-kubelet-on-control-plane-node).
+
+### CRI-O
+
+This section contains the necessary steps to install CRI-O as a container runtime.
+
+Use the following commands to install CRI-O on your system:
+
+{{< note >}}
+The CRI-O major and minor versions must match the Kubernetes major and minor versions.
+For more information, see the [CRI-O compatibility matrix](https://github.com/cri-o/cri-o).
+{{< /note >}}
+
+Install and configure prerequisites:
+
+```shell
+sudo modprobe overlay
+sudo modprobe br_netfilter
+
+# Set up required sysctl params, these persist across reboots.
+cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
+net.bridge.bridge-nf-call-iptables  = 1
+net.ipv4.ip_forward                 = 1
+net.bridge.bridge-nf-call-ip6tables = 1
+EOF
+
+sudo sysctl --system
+```
+
+{{< tabs name="tab-cri-cri-o-installation" >}}
+{{% tab name="Debian" %}}
+
+To install CRI-O on the following operating systems, set the environment variable `OS`
+to the appropriate value from the following table:
+
+| Operating system | `$OS`             |
+| ---------------- | ----------------- |
+| Debian Unstable  | `Debian_Unstable` |
+| Debian Testing   | `Debian_Testing`  |
+
+<br />
+Then, set `$VERSION` to the CRI-O version that matches your Kubernetes version.
+For instance, if you want to install CRI-O 1.18, set `VERSION=1.18`.
+You can pin your installation to a specific release.
+To install version 1.18.3, set `VERSION=1.18:1.18.3`.
+<br />
+
+Then run
+```shell
+cat <<EOF | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list
+deb https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/ /
+EOF
+cat <<EOF | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:stable:cri-o:$VERSION.list
+deb http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/$VERSION/$OS/ /
+EOF
+
+curl -L https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable:cri-o:$VERSION/$OS/Release.key | sudo apt-key add --keyring /etc/apt/trusted.gpg.d/libcontainers.gpg -
+curl -L https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/Release.key | sudo apt-key add --keyring /etc/apt/trusted.gpg.d/libcontainers.gpg -
+
+sudo apt-get update
+sudo apt-get install cri-o cri-o-runc
+```
+
+{{% /tab %}}
+
+{{% tab name="Ubuntu" %}}
+
+To install on the following operating systems, set the environment variable `OS` to the appropriate field in the following table:
+
+| Operating system | `$OS`             |
+| ---------------- | ----------------- |
+| Ubuntu 20.04     | `xUbuntu_20.04`   |
+| Ubuntu 19.10     | `xUbuntu_19.10`   |
+| Ubuntu 19.04     | `xUbuntu_19.04`   |
+| Ubuntu 18.04     | `xUbuntu_18.04`   |
+
+<br />
+Then, set `$VERSION` to the CRI-O version that matches your Kubernetes version.
+For instance, if you want to install CRI-O 1.18, set `VERSION=1.18`.
+You can pin your installation to a specific release.
+To install version 1.18.3, set `VERSION=1.18:1.18.3`.
+<br />
+
+Then run
+```shell
+cat <<EOF | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list
+deb https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/ /
+EOF
+cat <<EOF | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:stable:cri-o:$VERSION.list
+deb http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/$VERSION/$OS/ /
+EOF
+
+curl -L https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/Release.key | sudo apt-key add --keyring /etc/apt/trusted.gpg.d/libcontainers.gpg
+curl -L https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable:cri-o:$VERSION/$OS/Release.key | sudo apt-key add --keyring /etc/apt/trusted.gpg.d/libcontainers-cri-o.gpg -
+
+sudo apt-get update
+sudo apt-get install cri-o cri-o-runc
+```
+{{% /tab %}}
+
+{{% tab name="CentOS" %}}
+
+To install on the following operating systems, set the environment variable `OS` to the appropriate field in the following table:
+
+| Operating system | `$OS`             |
+| ---------------- | ----------------- |
+| Centos 8         | `CentOS_8`        |
+| Centos 8 Stream  | `CentOS_8_Stream` |
+| Centos 7         | `CentOS_7`        |
+
+<br />
+Then, set `$VERSION` to the CRI-O version that matches your Kubernetes version.
+For instance, if you want to install CRI-O 1.18, set `VERSION=1.18`.
+You can pin your installation to a specific release.
+To install version 1.18.3, set `VERSION=1.18:1.18.3`.
+<br />
+
+Then run
+```shell
+sudo curl -L -o /etc/yum.repos.d/devel:kubic:libcontainers:stable.repo https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/devel:kubic:libcontainers:stable.repo
+sudo curl -L -o /etc/yum.repos.d/devel:kubic:libcontainers:stable:cri-o:$VERSION.repo https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable:cri-o:$VERSION/$OS/devel:kubic:libcontainers:stable:cri-o:$VERSION.repo
+sudo yum install cri-o
+```
+
+{{% /tab %}}
+
+{{% tab name="openSUSE Tumbleweed" %}}
+
+```shell
+sudo zypper install cri-o
+```
+{{% /tab %}}
+{{% tab name="Fedora" %}}
+
+Set `$VERSION` to the CRI-O version that matches your Kubernetes version.
+For instance, if you want to install CRI-O 1.18, `VERSION=1.18`.
+
+You can find available versions with:
+```shell
+sudo dnf module list cri-o
+```
+CRI-O does not support pinning to specific releases on Fedora.
+
+Then run
+```shell
+sudo dnf module enable cri-o:$VERSION
+sudo dnf install cri-o
+```
+
+{{% /tab %}}
+{{< /tabs >}}
+
+Start CRI-O:
+
+```shell
+sudo systemctl daemon-reload
+sudo systemctl start crio
+```
+
+Refer to the [CRI-O installation guide](https://github.com/kubernetes-sigs/cri-o#getting-started)
+for more information.
+
+
+
+### Docker
+
+On each of your nodes, install Docker CE.
+
+The Kubernetes release notes list which versions of Docker are compatible
+with that version of Kubernetes.

 Use the following commands to install Docker on your system:

@@ -79,7 +385,7 @@ sudo apt-get update && sudo apt-get install -y \

 ```shell
 # Add Docker's official GPG key:
-curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
+curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add --keyring /etc/apt/trusted.gpg.d/docker.gpg -
 ```

 ```shell
@@ -179,7 +485,7 @@ sudo systemctl restart docker
 {{% /tab %}}
 {{< /tabs >}}

-If you want the docker service to start on boot, run the following command:
+If you want the `docker` service to start on boot, run the following command:

 ```shell
 sudo systemctl enable docker
@@ -188,301 +494,3 @@ sudo systemctl enable docker
 Refer to the [official Docker installation guides](https://docs.docker.com/engine/installation/)
 for more information.

-## CRI-O
-
-This section contains the necessary steps to install `CRI-O` as CRI runtime.
-
-Use the following commands to install CRI-O on your system:
-
-{{< note >}}
-The CRI-O major and minor versions must match the Kubernetes major and minor versions.
-For more information, see the [CRI-O compatibility matrix](https://github.com/cri-o/cri-o).
-{{< /note >}}
-
-### Prerequisites
-
-```shell
-sudo modprobe overlay
-sudo modprobe br_netfilter
-
-# Set up required sysctl params, these persist across reboots.
-cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
-net.bridge.bridge-nf-call-iptables  = 1
-net.ipv4.ip_forward                 = 1
-net.bridge.bridge-nf-call-ip6tables = 1
-EOF
-
-sudo sysctl --system
-```
-
-{{< tabs name="tab-cri-cri-o-installation" >}}
-{{% tab name="Debian" %}}
-
-To install CRI-O on the following operating systems, set the environment variable $OS to the appropriate field in the following table:
-
-| Operating system | $OS               |
-| ---------------- | ----------------- |
-| Debian Unstable  | `Debian_Unstable` |
-| Debian Testing   | `Debian_Testing`  |
-
-<br />
-Then, set `$VERSION` to the CRI-O version that matches your Kubernetes version.
-For instance, if you want to install CRI-O 1.18, set `VERSION=1.18`.
-You can pin your installation to a specific release.
-To install version 1.18.3, set `VERSION=1.18:1.18.3`.
-<br />
-
-Then run
-```shell
-cat <<EOF | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list
-"deb https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/ /"
-EOF
-cat <<EOF | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:stable:cri-o:$VERSION.list
-"deb http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/$VERSION/$OS/ /"
-EOF
-
-curl -L https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable:cri-o:$VERSION/$OS/Release.key | sudo apt-key add -
-curl -L https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/Release.key | sudo apt-key add -
-
-sudo apt-get update
-sudo apt-get install cri-o cri-o-runc
-```
-
-{{% /tab %}}
-
-{{% tab name="Ubuntu" %}}
-
-To install on the following operating systems, set the environment variable $OS to the appropriate field in the following table:
-
-| Operating system | $OS               |
-| ---------------- | ----------------- |
-| Ubuntu 20.04     | `xUbuntu_20.04`   |
-| Ubuntu 19.10     | `xUbuntu_19.10`   |
-| Ubuntu 19.04     | `xUbuntu_19.04`   |
-| Ubuntu 18.04     | `xUbuntu_18.04`   |
-
-<br />
-Then, set `$VERSION` to the CRI-O version that matches your Kubernetes version.
-For instance, if you want to install CRI-O 1.18, set `VERSION=1.18`.
-You can pin your installation to a specific release.
-To install version 1.18.3, set `VERSION=1.18:1.18.3`.
-<br />
-
-Then run
-```shell
-sudo echo "deb https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/ /" > /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list
-sudo echo "deb http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/$VERSION/$OS/ /" > /etc/apt/sources.list.d/devel:kubic:libcontainers:stable:cri-o:$VERSION.list
-
-curl -L https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable:cri-o:$VERSION/$OS/Release.key | sudo apt-key add -
-curl -L https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/Release.key | sudo apt-key add -
-
-sudo apt-get update
-sudo apt-get install cri-o cri-o-runc
-```
-{{% /tab %}}
-
-{{% tab name="CentOS" %}}
-
-To install on the following operating systems, set the environment variable $OS to the appropriate field in the following table:
-
-| Operating system | $OS               |
-| ---------------- | ----------------- |
-| Centos 8         | `CentOS_8`        |
-| Centos 8 Stream  | `CentOS_8_Stream` |
-| Centos 7         | `CentOS_7`        |
-
-<br />
-Then, set `$VERSION` to the CRI-O version that matches your Kubernetes version.
-For instance, if you want to install CRI-O 1.18, set `VERSION=1.18`.
-You can pin your installation to a specific release.
-To install version 1.18.3, set `VERSION=1.18:1.18.3`.
-<br />
-
-Then run
-```shell
-sudo curl -L -o /etc/yum.repos.d/devel:kubic:libcontainers:stable.repo https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/devel:kubic:libcontainers:stable.repo
-sudo curl -L -o /etc/yum.repos.d/devel:kubic:libcontainers:stable:cri-o:$VERSION.repo https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable:cri-o:$VERSION/$OS/devel:kubic:libcontainers:stable:cri-o:$VERSION.repo
-sudo yum install cri-o
-```
-
-{{% /tab %}}
-
-{{% tab name="openSUSE Tumbleweed" %}}
-
-```shell
-sudo zypper install cri-o
-```
-{{% /tab %}}
-{{% tab name="Fedora" %}}
-
-Set `$VERSION` to the CRI-O version that matches your Kubernetes version.
-For instance, if you want to install CRI-O 1.18, `VERSION=1.18`                                                                                                                                                                               
-You can find available versions with:
-```shell
-sudo dnf module list cri-o
-```
-CRI-O does not support pinning to specific releases on Fedora.
-
-Then run
-```shell
-sudo dnf module enable cri-o:$VERSION
-sudo dnf install cri-o
-```
-
-{{% /tab %}}
-{{< /tabs >}}
-
-### Start CRI-O
-
-```shell
-sudo systemctl daemon-reload
-sudo systemctl start crio
-```
-
-Refer to the [CRI-O installation guide](https://github.com/kubernetes-sigs/cri-o#getting-started)
-for more information.
-
-## Containerd
-
-This section contains the necessary steps to use `containerd` as CRI runtime.
-
-Use the following commands to install Containerd on your system:
-
-### Prerequisites
-
-```shell
-cat <<EOF | sudo tee /etc/modules-load.d/containerd.conf
-overlay
-br_netfilter
-EOF
-
-sudo modprobe overlay
-sudo modprobe br_netfilter
-
-# Setup required sysctl params, these persist across reboots.
-cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
-net.bridge.bridge-nf-call-iptables  = 1
-net.ipv4.ip_forward                 = 1
-net.bridge.bridge-nf-call-ip6tables = 1
-EOF
-
-sudo sysctl --system
-```
-
-### Install containerd
-
-{{< tabs name="tab-cri-containerd-installation" >}}
-{{% tab name="Ubuntu 16.04" %}}
-
-```shell
-# (Install containerd)
-## Set up the repository
-### Install packages to allow apt to use a repository over HTTPS
-sudo apt-get update && sudo apt-get install -y apt-transport-https ca-certificates curl software-properties-common
-```
-
-```shell
-## Add Docker's official GPG key
-curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
-```
-
-```shell
-## Add Docker apt repository.
-sudo add-apt-repository \
-    "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
-    $(lsb_release -cs) \
-    stable"
-```
-
-```shell
-## Install containerd
-sudo apt-get update && sudo apt-get install -y containerd.io
-```
-
-```shell
-# Configure containerd
-sudo mkdir -p /etc/containerd
-sudo containerd config default > /etc/containerd/config.toml
-```
-
-```shell
-# Restart containerd
-sudo systemctl restart containerd
-```
-{{% /tab %}}
-{{% tab name="CentOS/RHEL 7.4+" %}}
-
-```shell
-# (Install containerd)
-## Set up the repository
-### Install required packages
-sudo yum install -y yum-utils device-mapper-persistent-data lvm2
-```
-
-```shell
-## Add docker repository
-sudo yum-config-manager \
-    --add-repo \
-    https://download.docker.com/linux/centos/docker-ce.repo
-```
-
-```shell
-## Install containerd
-sudo yum update -y && sudo yum install -y containerd.io
-```
-
-```shell
-## Configure containerd
-sudo mkdir -p /etc/containerd
-sudo containerd config default > /etc/containerd/config.toml
-```
-
-```shell
-# Restart containerd
-sudo systemctl restart containerd
-```
-{{% /tab %}}
-{{% tab name="Windows (PowerShell)" %}}
-```powershell
-# (Install containerd)
-# download containerd
-cmd /c curl -OL https://github.com/containerd/containerd/releases/download/v1.4.0-beta.2/containerd-1.4.0-beta.2-windows-amd64.tar.gz
-cmd /c tar xvf .\containerd-1.4.0-beta.2-windows-amd64.tar.gz
-```
-
-```powershell
-# extract and configure
-Copy-Item -Path ".\bin\" -Destination "$Env:ProgramFiles\containerd" -Recurse -Force
-cd $Env:ProgramFiles\containerd\
-.\containerd.exe config default | Out-File config.toml -Encoding ascii
-
-# review the configuration. depending on setup you may want to adjust:
-# - the sandbox_image (kubernetes pause image)
-# - cni bin_dir and conf_dir locations
-Get-Content config.toml
-```
-
-```powershell
-# start containerd
-.\containerd.exe --register-service
-Start-Service containerd
-```
-{{% /tab %}}
-{{< /tabs >}}
-
-### systemd
-
-To use the `systemd` cgroup driver in `/etc/containerd/config.toml` set
-
-```
-[plugins.cri]
-systemd_cgroup = true
-```
-When using kubeadm, manually configure the
-[cgroup driver for kubelet](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#configure-cgroup-driver-used-by-kubelet-on-control-plane-node)
-
-## Other CRI runtimes: frakti
-
-Refer to the [Frakti QuickStart guide](https://github.com/kubernetes/frakti#quickstart) for more information.
-
-
@@ -200,6 +200,6 @@ The DEB and RPM packages shipped with the Kubernetes releases are:
 | `kubeadm`    | Installs the `/usr/bin/kubeadm` CLI tool and the [kubelet drop-in file](#the-kubelet-drop-in-file-for-systemd) for the kubelet. |
 | `kubelet`    | Installs the kubelet binary in `/usr/bin` and CNI binaries in `/opt/cni/bin`. |
 | `kubectl`    | Installs the `/usr/bin/kubectl` binary. |
-| `cri-tools` | Installs the `/usr/bin/crictl` binary from the [cri-tools git repository](https://github.com/kubernetes-incubator/cri-tools). |
+| `cri-tools` | Installs the `/usr/bin/crictl` binary from the [cri-tools git repository](https://github.com/kubernetes-sigs/cri-tools). |


@@ -23,7 +23,7 @@ To create a Kubernetes cluster on AWS, you will need an Access Key ID and a Secr

 * [Kubernetes Operations](https://github.com/kubernetes/kops) - Production Grade K8s Installation, Upgrades, and Management. Supports running Debian, Ubuntu, CentOS, and RHEL in AWS.

-* [kube-aws](https://github.com/kubernetes-incubator/kube-aws), creates and manages Kubernetes clusters with [Flatcar Linux](https://www.flatcar-linux.org/) nodes, using AWS tools: EC2, CloudFormation and Autoscaling.
+* [kube-aws](https://github.com/kubernetes-retired/kube-aws), creates and manages Kubernetes clusters with [Flatcar Linux](https://www.flatcar-linux.org/) nodes, using AWS tools: EC2, CloudFormation and Autoscaling.

 * [KubeOne](https://github.com/kubermatic/kubeone) is an open source cluster lifecycle management tool that creates, upgrades and manages Kubernetes Highly-Available clusters.

@@ -149,9 +149,8 @@ certificate.

 On some clusters, the apiserver does not require authentication; it may serve
 on localhost, or be protected by a firewall.  There is not a standard
-for this.  [Configuring Access to the API](/docs/reference/access-authn-authz/controlling-access/)
-describes how a cluster admin can configure this.  Such approaches may conflict
-with future high-availability support.
+for this.  [Controlling Access to the API](/docs/concepts/security/controlling-access)
+describes how a cluster admin can configure this.

 ## Programmatic access to the API

@@ -156,7 +156,7 @@ users:
 ```

 The `fake-ca-file`, `fake-cert-file` and `fake-key-file` above are the placeholders
-for the pathnames of the certificate files. You need change these to the actual pathnames
+for the pathnames of the certificate files. You need to change these to the actual pathnames
 of certificate files in your environment.

 Sometimes you may want to use Base64-encoded data embedded here instead of separate
@@ -148,9 +148,8 @@ certificate.

 On some clusters, the API server does not require authentication; it may serve
 on localhost, or be protected by a firewall. There is not a standard
-for this. [Configuring Access to the API](/docs/reference/access-authn-authz/controlling-access/)
-describes how a cluster admin can configure this. Such approaches may conflict
-with future high-availability support.
+for this. [Controlling Access to the Kubernetes API](/docs/concepts/security/controlling-access)
+describes how you can configure this as a cluster administrator.

 ### Programmatic access to the API

@@ -73,7 +73,7 @@ Oracle creates and manages a set of master nodes in the Oracle control plane on
 Different providers, and tools, will manage upgrades differently.  It is recommended that you consult their main documentation regarding upgrades.

 * [kops](https://github.com/kubernetes/kops)
-* [kubespray](https://github.com/kubernetes-incubator/kubespray)
+* [kubespray](https://github.com/kubernetes-sigs/kubespray)
 * [CoreOS Tectonic](https://coreos.com/tectonic/docs/latest/admin/upgrade.html)
 * [Digital Rebar](https://provision.readthedocs.io/en/tip/doc/content-packages/krib.html)
 * ...
@@ -143,7 +143,7 @@ The idea is that when a cluster is using nodes that have many cores,
 cores, `nodesPerReplica` dominates.

 There are other supported scaling patterns. For details, see
-[cluster-proportional-autoscaler](https://github.com/kubernetes-incubator/cluster-proportional-autoscaler).
+[cluster-proportional-autoscaler](https://github.com/kubernetes-sigs/cluster-proportional-autoscaler).

 ## Disable DNS horizontal autoscaling

@@ -233,5 +233,5 @@ patterns: *linear* and *ladder*.

 * Read about [Guaranteed Scheduling For Critical Add-On Pods](/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods/).
 * Learn more about the
-[implementation of cluster-proportional-autoscaler](https://github.com/kubernetes-incubator/cluster-proportional-autoscaler).
+[implementation of cluster-proportional-autoscaler](https://github.com/kubernetes-sigs/cluster-proportional-autoscaler).

@@ -62,7 +62,7 @@ By default, in GCE/Google Kubernetes Engine starting with Kubernetes version 1.7
 To create an ip-masq-agent, run the following kubectl command:

 `
-kubectl apply -f https://raw.githubusercontent.com/kubernetes-incubator/ip-masq-agent/master/ip-masq-agent.yaml
+kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/ip-masq-agent/master/ip-masq-agent.yaml
 `

 You must also apply the appropriate node label to any nodes in your cluster that you want the agent to run on.
@@ -71,7 +71,7 @@ You must also apply the appropriate node label to any nodes in your cluster that
 kubectl label nodes my-node beta.kubernetes.io/masq-agent-ds-ready=true
 `

-More information can be found in the ip-masq-agent documentation [here](https://github.com/kubernetes-incubator/ip-masq-agent)
+More information can be found in the ip-masq-agent documentation [here](https://github.com/kubernetes-sigs/ip-masq-agent)

 In most cases, the default set of rules should be sufficient; however, if this is not the case for your cluster, you can create and apply a [ConfigMap](/docs/tasks/configure-pod-container/configure-pod-configmap/) to customize the IP ranges that are affected.  For example, to allow only 10.0.0.0/8 to be considered by the ip-masq-agent, you can create the following [ConfigMap](/docs/tasks/configure-pod-container/configure-pod-configmap/) in a file called "config".

@@ -10,7 +10,7 @@ weight: 10

 {{< feature-state for_k8s_version="v1.15" state="stable" >}}

-Client certificates generated by [kubeadm](/docs/reference/setup-tools/kubeadm/kubeadm/) expire after 1 year. This page explains how to manage certificate renewals with kubeadm.
+Client certificates generated by [kubeadm](/docs/reference/setup-tools/kubeadm/) expire after 1 year. This page explains how to manage certificate renewals with kubeadm.

 ## {{% heading "prerequisites" %}}

@@ -12,7 +12,7 @@ This page shows how to use Romana for NetworkPolicy.

 ## {{% heading "prerequisites" %}}

-Complete steps 1, 2, and 3 of the [kubeadm getting started guide](/docs/reference/setup-tools/kubeadm/kubeadm/).
+Complete steps 1, 2, and 3 of the [kubeadm getting started guide](/docs/reference/setup-tools/kubeadm/).

 <!-- steps -->

@@ -13,7 +13,7 @@ This page shows how to use Weave Net for NetworkPolicy.
 ## {{% heading "prerequisites" %}}

 You need to have a Kubernetes cluster. Follow the
-[kubeadm getting started guide](/docs/reference/setup-tools/kubeadm/kubeadm/) to bootstrap one.
+[kubeadm getting started guide](/docs/reference/setup-tools/kubeadm/) to bootstrap one.

 <!-- steps -->

@@ -22,7 +22,7 @@ allocated as much CPU as it requests.
 Your cluster must have at least 1 CPU available for use to run the task examples.

 A few of the steps on this page require you to run the
-[metrics-server](https://github.com/kubernetes-incubator/metrics-server)
+[metrics-server](https://github.com/kubernetes-sigs/metrics-server)
 service in your cluster. If you have the metrics-server
 running, you can skip those steps.

@@ -21,7 +21,7 @@ but is not allowed to use more memory than its limit.
 Each node in your cluster must have at least 300 MiB of memory.

 A few of the steps on this page require you to run the
-[metrics-server](https://github.com/kubernetes-incubator/metrics-server)
+[metrics-server](https://github.com/kubernetes-sigs/metrics-server)
 service in your cluster. If you have the metrics-server
 running, you can skip those steps.

@@ -29,7 +29,7 @@ PersistentVolume.
 {{< glossary_tooltip text="kubectl" term_id="kubectl" >}}
 command-line tool must be configured to communicate with your cluster. If you
 do not already have a single-node cluster, you can create one by using
-[Minikube](/docs/setup/learning-environment/minikube/).
+[Minikube](https://minikube.sigs.k8s.io/docs/).

 * Familiarize yourself with the material in
 [Persistent Volumes](/docs/concepts/storage/persistent-volumes/).
@@ -37,7 +37,7 @@ shared Volume at `/work-dir`, and the application container mounts the shared
 Volume at `/usr/share/nginx/html`. The init container runs the following command
 and then terminates:

-    wget -O /work-dir/index.html http://kubernetes.io
+    wget -O /work-dir/index.html http://info.cern.ch

 Notice that the init container writes the `index.html` file in the root directory
 of the nginx server.
@@ -67,16 +67,13 @@ In your shell, send a GET request to the nginx server:

 The output shows that nginx is serving the web page that was written by the init container:

-    <!Doctype html>
-    <html id="home">
+    <html><head></head><body><header>
+    <title>http://info.cern.ch</title>
+    </header>

-    <head>
-    ...
-    "url": "http://kubernetes.io/"}</script>
-    </head>
-    <body>
+    <h1>http://info.cern.ch - home of the first website</h1>
      ...
-      <p>Kubernetes is open source giving you the freedom to take advantage ...</p>
+      <li><a href="http://info.cern.ch/hypertext/WWW/TheProject.html">Browse the first website</a></li>
      ...


@@ -15,7 +15,7 @@ content_type: task
 `crictl` is a command-line interface for CRI-compatible container runtimes.
 You can use it to inspect and debug container runtimes and applications on a
 Kubernetes node. `crictl` and its source are hosted in the
-[cri-tools](https://github.com/kubernetes-incubator/cri-tools) repository.
+[cri-tools](https://github.com/kubernetes-sigs/cri-tools) repository.



@@ -31,7 +31,7 @@ Kubernetes node. `crictl` and its source are hosted in the
 ## Installing crictl

 You can download a compressed archive `crictl` from the cri-tools [release
-page](https://github.com/kubernetes-incubator/cri-tools/releases), for several
+page](https://github.com/kubernetes-sigs/cri-tools/releases), for several
 different architectures. Download the version that corresponds to your version
 of Kubernetes. Extract it and move it to a location on your system path, such as
 `/usr/local/bin/`.
@@ -353,7 +353,7 @@ CONTAINER ID        IMAGE               CREATED              STATE

 <!-- discussion -->

-See [kubernetes-incubator/cri-tools](https://github.com/kubernetes-incubator/cri-tools)
+See [kubernetes-sigs/cri-tools](https://github.com/kubernetes-sigs/cri-tools)
 for more information.


@@ -54,7 +54,7 @@ The metric typically also includes some cached (file-backed) memory, because the

 ## Metrics Server

-[Metrics Server](https://github.com/kubernetes-incubator/metrics-server) is a cluster-wide aggregator of resource usage data.
+[Metrics Server](https://github.com/kubernetes-sigs/metrics-server) is a cluster-wide aggregator of resource usage data.
 By default, it is deployed in clusters created by `kube-up.sh` script
 as a Deployment object. If you use a different Kubernetes setup mechanism, you can deploy it using the provided
 [deployment components.yaml](https://github.com/kubernetes-sigs/metrics-server/releases) file.
@@ -30,7 +30,7 @@ cluster components such as the
 [Horizontal Pod Autoscaler](/docs/tasks/run-application/horizontal-pod-autoscale/)
 controller, as well as the `kubectl top` utility.
 These  metrics are collected by the lightweight, short-term, in-memory 
-[metrics-server](https://github.com/kubernetes-incubator/metrics-server) and
+[metrics-server](https://github.com/kubernetes-sigs/metrics-server) and
 are exposed via the `metrics.k8s.io` API. 

 metrics-server discovers all nodes on the cluster and 
@@ -954,15 +954,15 @@ the `/scale` subresource will return an error on GET.

  - It is a required value.
  - Only JSONPaths under `.status` and with the dot notation are allowed.
-  - If there is no value under the `StatusReplicasPath` in the custom resource,
+  - If there is no value under the `statusReplicasPath` in the custom resource,
 the status replica value in the `/scale` subresource will default to 0.

- `LabelSelectorPath` defines the JSONPath inside of a custom resource that corresponds to `Scale.Status.Selector`.
+- `labelSelectorPath` defines the JSONPath inside of a custom resource that corresponds to `Scale.Status.Selector`.

  - It is an optional value.
  - It must be set to work with HPA.
  - Only JSONPaths under `.status` or `.spec` and with the dot notation are allowed.
-  - If there is no value under the `LabelSelectorPath` in the custom resource,
+  - If there is no value under the `labelSelectorPath` in the custom resource,
 the status selector value in the `/scale` subresource will default to the empty string.
  - The field pointed by this JSON path must be a string field (not a complex selector struct) which contains a serialized label selector in string form.

@@ -18,7 +18,7 @@ This page shows how to use PodPreset objects to inject information like {{< glos
 ## {{% heading "prerequisites" %}}


-You need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one using [Minikube](/docs/setup/learning-environment/minikube/).
+You need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one using [Minikube](https://minikube.sigs.k8s.io/docs/).
 Make sure that you have [enabled PodPreset](/docs/concepts/workloads/pods/podpreset/#enable-pod-preset) in your cluster.


@@ -23,8 +23,8 @@ For more information on how Horizontal Pod Autoscaler behaves, see the


 This example requires a running Kubernetes cluster and kubectl, version 1.2 or later.
-[metrics-server](https://github.com/kubernetes-incubator/metrics-server/) monitoring needs to be deployed in the cluster
-to provide metrics via the resource metrics API, as Horizontal Pod Autoscaler uses this API to collect metrics. The instructions for deploying this are on the GitHub repository of [metrics-server](https://github.com/kubernetes-incubator/metrics-server/), if you followed [getting started on GCE guide](/docs/setup/production-environment/turnkey/gce/),
+[metrics-server](https://github.com/kubernetes-sigs/metrics-server) monitoring needs to be deployed in the cluster
+to provide metrics via the resource metrics API, as Horizontal Pod Autoscaler uses this API to collect metrics. The instructions for deploying this are on the GitHub repository of [metrics-server](https://github.com/kubernetes-sigs/metrics-server), if you followed [getting started on GCE guide](/docs/setup/production-environment/turnkey/gce/),
 metrics-server monitoring will be turned-on by default.

 To specify multiple resource metrics for a Horizontal Pod Autoscaler, you must have a Kubernetes cluster
@@ -265,12 +265,12 @@ APIs, cluster administrators must ensure that:

 * The corresponding APIs are registered:

-   * For resource metrics, this is the `metrics.k8s.io` API, generally provided by [metrics-server](https://github.com/kubernetes-incubator/metrics-server).
+   * For resource metrics, this is the `metrics.k8s.io` API, generally provided by [metrics-server](https://github.com/kubernetes-sigs/metrics-server).
     It can be launched as a cluster addon.

   * For custom metrics, this is the `custom.metrics.k8s.io` API.  It's provided by "adapter" API servers provided by metrics solution vendors.
     Check with your metrics pipeline, or the [list of known solutions](https://github.com/kubernetes/metrics/blob/master/IMPLEMENTATIONS.md#custom-metrics-api).
-     If you would like to write your own, check out the [boilerplate](https://github.com/kubernetes-incubator/custom-metrics-apiserver) to get started.
+     If you would like to write your own, check out the [boilerplate](https://github.com/kubernetes-sigs/custom-metrics-apiserver) to get started.

   * For external metrics, this is the `external.metrics.k8s.io` API.  It may be provided by the custom metrics adapters provided above.

@@ -78,6 +78,6 @@ sc uninstall
 ## {{% heading "whatsnext" %}}

 * View [sample service brokers](https://github.com/openservicebrokerapi/servicebroker/blob/master/gettingStarted.md#sample-service-brokers).
-* Explore the [kubernetes-incubator/service-catalog](https://github.com/kubernetes-incubator/service-catalog) project.
+* Explore the [kubernetes-sigs/service-catalog](https://github.com/kubernetes-sigs/service-catalog) project.


@@ -20,9 +20,20 @@ accessing your cluster.
 You can also read the
 [`kubectl` reference documentation](/docs/reference/kubectl/).

+## kind
+
+[`kind`](https://kind.sigs.k8s.io/docs/) lets you run Kubernetes on
+your local computer. This tool it requires that you have
+[Docker](https://docs.docker.com/get-docker/) installed and configured.
+
+The kind [Quick Start](https://kind.sigs.k8s.io/docs/user/quick-start/) page
+shows you what you need to do to get up and running with kind.
+
+<a class="btn btn-primary" href="https://kind.sigs.k8s.io/docs/user/quick-start/" role="button" aria-label="View kind Quick Start Guide">View kind Quick Start Guide</a>
+
 ## minikube

-[`minikube`](https://minikube.sigs.k8s.io/) is a tool that lets you run Kubernetes
+Like `kind`, [`minikube`](https://minikube.sigs.k8s.io/) is a tool that lets you run Kubernetes
 locally. `minikube` runs a single-node Kubernetes cluster on your personal
 computer (including Windows, macOS and Linux PCs) so that you can try out
 Kubernetes, or for daily development work.
@@ -36,14 +47,12 @@ on getting the tool installed.
 Once you have `minikube` working, you can use it to
 [run a sample application](/docs/tutorials/hello-minikube/).

-## kind
+## kubeadm

-Like `minikube`, [`kind`](https://kind.sigs.k8s.io/docs/) lets you run Kubernetes on
-your local computer. Unlike `minikube`, `kind` only works with a single container
-runtime: it requires that you have [Docker](https://docs.docker.com/get-docker/)
-installed and configured.
+You can use the {{< glossary_tooltip term_id="kubeadm" text="kubeadm" >}} tool to create and manage Kubernetes clusters.
+It performs the actions necessary to get a minimum viable, secure cluster up and running in a user friendly way.

-[Quick Start](https://kind.sigs.k8s.io/docs/user/quick-start/) shows you what
-you need to do to get up and running with `kind`.
+[Installing kubeadm](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/) shows you how to install kubeadm.
+Once installed, you can use it to [create a cluster](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/).

-<a class="btn btn-primary" href="https://kind.sigs.k8s.io/docs/user/quick-start/" role="button" aria-label="View kind Quick Start Guide">View kind Quick Start Guide</a>
+<a class="btn btn-primary" href="/docs/setup/production-environment/tools/kubeadm/install-kubeadm/" role="button" aria-label="View kubeadm Install Guide">View kubeadm Install Guide</a>
@@ -525,7 +525,7 @@ compinit

 ## {{% heading "whatsnext" %}}

-* [Install Minikube](/docs/tasks/tools/install-minikube/)
+* [Install Minikube](https://minikube.sigs.k8s.io/docs/start/)
 * See the [getting started guides](/docs/setup/) for more about creating clusters.
 * [Learn how to launch and expose your application.](/docs/tasks/access-application-cluster/service-access-application-cluster/)
 * If you need access to a cluster you didn't create, see the
@@ -37,7 +37,7 @@ profiles that give only the necessary privileges to your container processes.

 In order to complete all steps in this tutorial, you must install
 [kind](https://kind.sigs.k8s.io/docs/user/quick-start/) and
-[kubectl](/doc/tasks/tools/install-kubectl/). This tutorial will show examples
+[kubectl](/docs/tasks/tools/install-kubectl/). This tutorial will show examples
 with both alpha (pre-v1.19) and generally available seccomp functionality, so
 make sure that your cluster is [configured
 correctly](https://kind.sigs.k8s.io/docs/user/quick-start/#setting-kubernetes-version)
@@ -365,4 +365,4 @@ The default seccomp profile should provide adequate access for most workloads.
 Additional resources:

 * [A Seccomp Overview](https://lwn.net/Articles/656307/)
-* [Seccomp Security Profiles for Docker](https://docs.docker.com/engine/security/seccomp/)
+* [Seccomp Security Profiles for Docker](https://docs.docker.com/engine/security/seccomp/)
@@ -14,9 +14,9 @@ In this tutorial you will learn how and why to externalize your microservice’s
 ### Creating Kubernetes ConfigMaps & Secrets
 There are several ways to set environment variables for a Docker container in Kubernetes, including: Dockerfile, kubernetes.yml, Kubernetes ConfigMaps, and Kubernetes Secrets.  In the tutorial, you will learn how to use the latter two for setting your environment variables whose values will be injected into your microservices.  One of the benefits for using ConfigMaps and Secrets is that they can be re-used across multiple containers, including being assigned to different environment variables for the different containers.

-ConfigMaps are API Objects that store non-confidential key-value pairs.  In the Interactive Tutorial you will learn how to use a ConfigMap to store the application's name.  For more information regarding ConfigMaps, you can find the documentation [here](https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/).
+ConfigMaps are API Objects that store non-confidential key-value pairs.  In the Interactive Tutorial you will learn how to use a ConfigMap to store the application's name.  For more information regarding ConfigMaps, you can find the documentation [here](/docs/tasks/configure-pod-container/configure-pod-configmap/).

-Although Secrets are also used to store key-value pairs, they differ from ConfigMaps in that they're intended for confidential/sensitive information and are stored using Base64 encoding.  This makes secrets the appropriate choice for storing such things as credentials, keys, and tokens, the former of which you'll do in the Interactive Tutorial.  For more information on Secrets, you can find the documentation [here](https://kubernetes.io/docs/concepts/configuration/secret/).
+Although Secrets are also used to store key-value pairs, they differ from ConfigMaps in that they're intended for confidential/sensitive information and are stored using Base64 encoding.  This makes secrets the appropriate choice for storing such things as credentials, keys, and tokens, the former of which you'll do in the Interactive Tutorial.  For more information on Secrets, you can find the documentation [here](/docs/concepts/configuration/secret/).


 ### Externalizing Config from Code
@@ -36,4 +36,4 @@ Many open source frameworks and runtimes implement and support MicroProfile Conf
 <!-- lessoncontent -->

 ## Example: Externalizing config using MicroProfile, ConfigMaps and Secrets
-### [Start Interactive Tutorial](/docs/tutorials/configuration/configure-java-microservice/configure-java-microservice-interactive/) 
+### [Start Interactive Tutorial](/docs/tutorials/configuration/configure-java-microservice/configure-java-microservice-interactive/) 
@@ -16,24 +16,20 @@ card:
 <!-- overview -->

 This tutorial shows you how to run a sample app
-on Kubernetes using [Minikube](/docs/setup/learning-environment/minikube) and Katacoda.
+on Kubernetes using minikube and Katacoda.
 Katacoda provides a free, in-browser Kubernetes environment.

 {{< note >}}
-You can also follow this tutorial if you've installed [Minikube locally](/docs/tasks/tools/install-minikube/).
+You can also follow this tutorial if you've installed minikube locally.
+See [minikube start](https://minikube.sigs.k8s.io/docs/start/) for installation instructions.
 {{< /note >}}

-
-
 ## {{% heading "objectives" %}}

-
-* Deploy a sample application to Minikube.
+* Deploy a sample application to minikube.
 * Run the app.
 * View application logs.

-
-
 ## {{% heading "prerequisites" %}}


@@ -43,14 +39,14 @@ This tutorial provides a container image that uses NGINX to echo back all the re

 <!-- lessoncontent -->

-## Create a Minikube cluster
+## Create a minikube cluster

 1. Click **Launch Terminal**

    {{< kat-button >}}

 {{< note >}}
-    If you installed Minikube locally, run `minikube start`.
+If you installed minikube locally, run `minikube start`.
 {{< /note >}}

 2. Open the Kubernetes dashboard in a browser:
@@ -155,7 +151,7 @@ Kubernetes [*Service*](/docs/concepts/services-networking/service/).
    ```

    On cloud providers that support load balancers,
-    an external IP address would be provisioned to access the Service. On Minikube,
+    an external IP address would be provisioned to access the Service. On minikube,
    the `LoadBalancer` type makes the Service accessible through the `minikube service`
    command.

@@ -173,7 +169,7 @@ Kubernetes [*Service*](/docs/concepts/services-networking/service/).

 ## Enable addons

-Minikube has a set of built-in {{< glossary_tooltip text="addons" term_id="addons" >}} that can be enabled, disabled and opened in the local Kubernetes environment.
+The minikube tool includes a set of built-in {{< glossary_tooltip text="addons" term_id="addons" >}} that can be enabled, disabled and opened in the local Kubernetes environment.

 1. List the currently supported addons:

@@ -50,7 +50,7 @@ To complete this tutorial, you should already have a basic familiarity with
 ### Additional Minikube setup instructions

 {{< caution >}}
-[Minikube](/docs/setup/learning-environment/minikube/) defaults to 1024MiB of memory and 1 CPU.
+[Minikube](https://minikube.sigs.k8s.io/docs/) defaults to 1024MiB of memory and 1 CPU.
 Running Minikube with the default resource configuration results in insufficient resource
 errors during this tutorial. To avoid these errors, start Minikube with the following settings:

@@ -19,17 +19,14 @@ This tutorial builds upon the [PHP Guestbook with Redis](/docs/tutorials/statele
 * Metricbeat
 * Packetbeat

-
-
 ## {{% heading "objectives" %}}

 * Start up the PHP Guestbook with Redis.
 * Install kube-state-metrics.
-* Create a Kubernetes secret.
+* Create a Kubernetes Secret.
 * Deploy the Beats.
 * View dashboards of your logs and metrics.

-
 ## {{% heading "prerequisites" %}}


@@ -40,16 +37,18 @@ Additionally you need:

 * A running deployment of the [PHP Guestbook with Redis](/docs/tutorials/stateless-application/guestbook) tutorial.

-* A running Elasticsearch and Kibana deployment.  You can use [Elasticsearch Service in Elastic Cloud](https://cloud.elastic.co), run the [download files](https://www.elastic.co/guide/en/elastic-stack-get-started/current/get-started-elastic-stack.html) on your workstation or servers, or the [Elastic Helm Charts](https://github.com/elastic/helm-charts).
-
-
+* A running Elasticsearch and Kibana deployment. You can use [Elasticsearch Service in Elastic Cloud](https://cloud.elastic.co),
+  run the [downloaded files](https://www.elastic.co/guide/en/elastic-stack-get-started/current/get-started-elastic-stack.html)
+  on your workstation or servers, or the [Elastic Helm Charts](https://github.com/elastic/helm-charts).

 <!-- lessoncontent -->

 ## Start up the  PHP Guestbook with Redis
+
 This tutorial builds on the [PHP Guestbook with Redis](/docs/tutorials/stateless-application/guestbook) tutorial.  If you have the guestbook application running, then you can monitor that.  If you do not have it running then follow the instructions to deploy the guestbook and do not perform the **Cleanup** steps.  Come back to this page when you have the guestbook running.

 ## Add a Cluster role binding
+
 Create a [cluster level role binding](/docs/reference/access-authn-authz/rbac/#rolebinding-and-clusterrolebinding) so that you can deploy kube-state-metrics and the Beats at the cluster level (in kube-system).

 ```shell
@@ -60,31 +59,39 @@ kubectl create clusterrolebinding cluster-admin-binding \
 ## Install kube-state-metrics

 Kubernetes [*kube-state-metrics*](https://github.com/kubernetes/kube-state-metrics) is a simple service that listens to the Kubernetes API server and generates metrics about the state of the objects.  Metricbeat reports these metrics.  Add kube-state-metrics to the Kubernetes cluster that the guestbook is running in.
+
 ```shell
 git clone https://github.com/kubernetes/kube-state-metrics.git kube-state-metrics
 kubectl apply -f kube-state-metrics/examples/standard
 ```

 ### Check to see if kube-state-metrics is running
+
 ```shell
 kubectl get pods --namespace=kube-system -l app.kubernetes.io/name=kube-state-metrics
 ```
+
 Output:
-```shell
+
+```
 NAME                                 READY   STATUS    RESTARTS   AGE
 kube-state-metrics-89d656bf8-vdthm   1/1     Running     0          21s
 ```
+
 ## Clone the Elastic examples GitHub repo
+
 ```shell
 git clone https://github.com/elastic/examples.git
 ```

 The rest of the commands will reference files in the `examples/beats-k8s-send-anywhere` directory, so change dir there:
+
 ```shell
 cd examples/beats-k8s-send-anywhere
 ```

 ## Create a Kubernetes Secret
+
 A Kubernetes {{< glossary_tooltip text="Secret" term_id="secret" >}} is an object that contains a small amount of sensitive data such as a password, a token, or a key. Such information might otherwise be put in a Pod specification or in an image; putting it in a Secret object allows for more control over how it is used, and reduces the risk of accidental exposure.

 {{< note >}}
@@ -95,55 +102,70 @@ There are two sets of steps here, one for *self managed* Elasticsearch and Kiban
 {{% tab name="Self Managed" %}}

 ### Self managed
+
 Switch to the **Managed service** tab if you are connecting to Elasticsearch Service in Elastic Cloud.

 ### Set the credentials
+
 There are four files to edit to create a k8s secret when you are connecting to self managed Elasticsearch and Kibana (self managed is effectively anything other than the managed Elasticsearch Service in Elastic Cloud).  The files are:

-1. ELASTICSEARCH_HOSTS
-1. ELASTICSEARCH_PASSWORD
-1. ELASTICSEARCH_USERNAME
-1. KIBANA_HOST
+1. `ELASTICSEARCH_HOSTS`
+1. `ELASTICSEARCH_PASSWORD`
+1. `ELASTICSEARCH_USERNAME`
+1. `KIBANA_HOST`

 Set these with the information for your Elasticsearch cluster and your Kibana host.  Here are some examples (also see [*this configuration*](https://stackoverflow.com/questions/59892896/how-to-connect-from-minikube-to-elasticsearch-installed-on-host-local-developme/59892897#59892897))

 #### `ELASTICSEARCH_HOSTS`
+
 1. A nodeGroup from the Elastic Elasticsearch Helm Chart:

-    ```shell
-    ["http://elasticsearch-master.default.svc.cluster.local:9200"]
-    ```
+   ```
+   ["http://elasticsearch-master.default.svc.cluster.local:9200"]
+   ```
+
 1. A single Elasticsearch node running on a Mac where your Beats are running in Docker for Mac:

-    ```shell
-    ["http://host.docker.internal:9200"]
-    ```
+   ```
+   ["http://host.docker.internal:9200"]
+   ```
+
 1. Two Elasticsearch nodes running in VMs or on physical hardware:

-    ```shell
-    ["http://host1.example.com:9200", "http://host2.example.com:9200"]
-    ```
-Edit `ELASTICSEARCH_HOSTS`
+   ```
+   ["http://host1.example.com:9200", "http://host2.example.com:9200"]
+   ```
+
+Edit `ELASTICSEARCH_HOSTS`:
+
 ```shell
 vi ELASTICSEARCH_HOSTS
 ```

 #### `ELASTICSEARCH_PASSWORD`
-Just the password; no whitespace, quotes, or <>:

-    <yoursecretpassword>
+Just the password; no whitespace, quotes, `<` or `>`:
+
+```
+<yoursecretpassword>
+```
+
+Edit `ELASTICSEARCH_PASSWORD`:

-Edit `ELASTICSEARCH_PASSWORD`
 ```shell
 vi ELASTICSEARCH_PASSWORD
 ```

 #### `ELASTICSEARCH_USERNAME`
-Just the username; no whitespace, quotes, or <>:

-    <your ingest username for Elasticsearch>
+Just the username; no whitespace, quotes, `<` or `>`:
+
+```
+<your ingest username for Elasticsearch>
+```
+
+Edit `ELASTICSEARCH_USERNAME`:

-Edit `ELASTICSEARCH_USERNAME`
 ```shell
 vi ELASTICSEARCH_USERNAME
 ```
@@ -152,78 +174,98 @@ vi ELASTICSEARCH_USERNAME

 1. The Kibana instance from the Elastic Kibana Helm Chart.  The subdomain `default` refers to the default namespace.  If you have deployed the Helm Chart using a different namespace, then your subdomain will be different:

-    ```shell
-    "kibana-kibana.default.svc.cluster.local:5601"
-    ```
+   ```
+   "kibana-kibana.default.svc.cluster.local:5601"
+   ```
+
 1. A Kibana instance running on a Mac where your Beats are running in Docker for Mac:

-    ```shell
-    "host.docker.internal:5601"
-    ```
+   ```
+   "host.docker.internal:5601"
+   ```
 1. Two Elasticsearch nodes running in VMs or on physical hardware:

-    ```shell
-    "host1.example.com:5601"
-    ```
-Edit `KIBANA_HOST`
+   ```
+   "host1.example.com:5601"
+   ```
+
+Edit `KIBANA_HOST`:
+
 ```shell
 vi KIBANA_HOST
 ```

-### Create a Kubernetes secret
-This command creates a secret in the Kubernetes system level namespace (kube-system) based on the files you just edited:
+### Create a Kubernetes Secret

-    kubectl create secret generic dynamic-logging \
-      --from-file=./ELASTICSEARCH_HOSTS \
-      --from-file=./ELASTICSEARCH_PASSWORD \
-      --from-file=./ELASTICSEARCH_USERNAME \
-      --from-file=./KIBANA_HOST \
-      --namespace=kube-system
+This command creates a Secret in the Kubernetes system level namespace (`kube-system`) based on the files you just edited:
+
+```shell
+kubectl create secret generic dynamic-logging \
+  --from-file=./ELASTICSEARCH_HOSTS \
+  --from-file=./ELASTICSEARCH_PASSWORD \
+  --from-file=./ELASTICSEARCH_USERNAME \
+  --from-file=./KIBANA_HOST \
+  --namespace=kube-system
+```

 {{% /tab %}}
 {{% tab name="Managed service" %}}

 ## Managed service
-This tab is for Elasticsearch Service in Elastic Cloud only, if you have already created a secret for a self managed Elasticsearch and Kibana deployment, then continue with [Deploy the Beats](#deploy-the-beats).
-### Set the credentials
-There are two files to edit to create a k8s secret when you are connecting to the managed Elasticsearch Service in Elastic Cloud.  The files are:

-1. ELASTIC_CLOUD_AUTH
-1. ELASTIC_CLOUD_ID
+This tab is for Elasticsearch Service in Elastic Cloud only, if you have already created a secret for a self managed Elasticsearch and Kibana deployment, then continue with [Deploy the Beats](#deploy-the-beats).
+
+### Set the credentials
+
+There are two files to edit to create a Kubernetes Secret when you are connecting to the managed Elasticsearch Service in Elastic Cloud.  The files are:
+
+1. `ELASTIC_CLOUD_AUTH`
+1. `ELASTIC_CLOUD_ID`

 Set these with the information provided to you from the Elasticsearch Service console when you created the deployment.  Here are some examples:

-#### ELASTIC_CLOUD_ID
-```shell
+#### `ELASTIC_CLOUD_ID`
+
+```
 devk8s:ABC123def456ghi789jkl123mno456pqr789stu123vwx456yza789bcd012efg345hijj678klm901nop345zEwOTJjMTc5YWQ0YzQ5OThlN2U5MjAwYTg4NTIzZQ==
 ```

-#### ELASTIC_CLOUD_AUTH
+#### `ELASTIC_CLOUD_AUTH`
+
 Just the username, a colon (`:`), and the password, no whitespace or quotes:
-```shell
+
+```
 elastic:VFxJJf9Tjwer90wnfTghsn8w
 ```

 ### Edit the required files:
+
 ```shell
 vi ELASTIC_CLOUD_ID
 vi ELASTIC_CLOUD_AUTH
 ```
-### Create a Kubernetes secret
-This command creates a secret in the Kubernetes system level namespace (kube-system) based on the files you just edited:

-    kubectl create secret generic dynamic-logging \
-      --from-file=./ELASTIC_CLOUD_ID \
-      --from-file=./ELASTIC_CLOUD_AUTH \
-      --namespace=kube-system
+### Create a Kubernetes Secret
+
+This command creates a Secret in the Kubernetes system level namespace (`kube-system`) based on the files you just edited:
+
+```shell
+kubectl create secret generic dynamic-logging \
+  --from-file=./ELASTIC_CLOUD_ID \
+  --from-file=./ELASTIC_CLOUD_AUTH \
+  --namespace=kube-system
+```
+
+{{% /tab %}}

-  {{% /tab %}}
 {{< /tabs >}}

 ## Deploy the Beats
+
 Manifest files are provided for each Beat.  These manifest files use the secret created earlier to configure the Beats to connect to your Elasticsearch and Kibana servers.

 ### About Filebeat
+
 Filebeat will collect logs from the Kubernetes nodes and the containers running in each pod running on those nodes.  Filebeat is deployed as a {{< glossary_tooltip text="DaemonSet" term_id="daemonset" >}}.  Filebeat can autodiscover applications running in your Kubernetes cluster. At startup Filebeat scans existing containers and launches the proper configurations for them, then it will watch for new start/stop events.

 Here is the autodiscover configuration that enables Filebeat to locate and parse Redis logs from the Redis containers deployed with the guestbook application.  This configuration is in the file `filebeat-kubernetes.yaml`:
@@ -242,20 +284,25 @@ Here is the autodiscover configuration that enables Filebeat to locate and parse
        enabled: true
        var.hosts: ["${data.host}:${data.port}"]
 ```
+
 This configures Filebeat to apply the Filebeat module `redis` when a container is detected with a label `app` containing the string `redis`.  The redis module has the ability to collect the `log` stream from the container by using the docker input type (reading the file on the Kubernetes node associated with the STDOUT stream from this Redis container).  Additionally, the module has the ability to collect Redis `slowlog` entries by connecting to the proper pod host and port, which is provided in the container metadata.

 ### Deploy Filebeat:
+
 ```shell
 kubectl create -f filebeat-kubernetes.yaml
 ```

 #### Verify
+
 ```shell
 kubectl get pods -n kube-system -l k8s-app=filebeat-dynamic
 ```

 ### About Metricbeat
+
 Metricbeat autodiscover is configured in the same way as Filebeat.  Here is the Metricbeat autodiscover configuration for the Redis containers.  This configuration is in the file `metricbeat-kubernetes.yaml`:
+
 ```yaml
 - condition.equals:
    kubernetes.labels.tier: backend
@@ -267,22 +314,27 @@ Metricbeat autodiscover is configured in the same way as Filebeat.  Here is the
      # Redis hosts
      hosts: ["${data.host}:${data.port}"]
 ```
+
 This configures Metricbeat to apply the Metricbeat module `redis` when a container is detected with a label `tier` equal to the string `backend`.  The `redis` module has the ability to collect the `info` and `keyspace` metrics from the container by connecting to the proper pod host and port, which is provided in the container metadata.

 ### Deploy Metricbeat
+
 ```shell
 kubectl create -f metricbeat-kubernetes.yaml
 ```
+
 #### Verify
+
 ```shell
 kubectl get pods -n kube-system -l k8s-app=metricbeat
 ```

 ### About Packetbeat
+
 Packetbeat configuration is different than Filebeat and Metricbeat.  Rather than specify patterns to match against container labels the configuration is based on the protocols and port numbers involved.  Shown below is a subset of the port numbers.

 {{< note >}}
-If you are running a service on a non-standard port add that port number to the appropriate type in `filebeat.yaml` and delete / create the Packetbeat DaemonSet.
+If you are running a service on a non-standard port add that port number to the appropriate type in `filebeat.yaml` and delete/create the Packetbeat DaemonSet.
 {{< /note >}}

 ```yaml
@@ -309,11 +361,13 @@ packetbeat.flows:
 ```

 #### Deploy Packetbeat
+
 ```shell
 kubectl create -f packetbeat-kubernetes.yaml
 ```

 #### Verify
+
 ```shell
 kubectl get pods -n kube-system -l k8s-app=packetbeat-dynamic
 ```
@@ -328,15 +382,17 @@ Similarly, view dashboards for Apache and Redis.  You will see dashboards for lo

 To enable Metricbeat to retrieve the Apache metrics, enable server-status by adding a ConfigMap including a mod-status configuration file and re-deploy the guestbook.

+## Scale your Deployments and see new pods being monitored
+
+List the existing Deployments:

-## Scale your deployments and see new pods being monitored
-List the existing deployments:
 ```shell
 kubectl get deployments
 ```

 The output:
-```shell
+
+```
 NAME            READY   UP-TO-DATE   AVAILABLE   AGE
 frontend        3/3     3            3           3h27m
 redis-master    1/1     1            1           3h27m
@@ -344,54 +400,56 @@ redis-slave     2/2     2            2           3h27m
 ```

 Scale the frontend down to two pods:
+
 ```shell
 kubectl scale --replicas=2 deployment/frontend
 ```
+
 The output:
-```shell
+
+```
 deployment.extensions/frontend scaled
 ```
+
 Scale the frontend back up to three pods:
+
 ```shell
 kubectl scale --replicas=3 deployment/frontend
 ```

 ## View the changes in Kibana
+
 See the screenshot, add the indicated filters and then add the columns to the view.  You can see the ScalingReplicaSet entry that is marked, following from there to the top of the list of events shows the image being pulled, the volumes mounted, the pod starting, etc.
 ![Kibana Discover](https://raw.githubusercontent.com/elastic/examples/master/beats-k8s-send-anywhere/scaling-up.png)

-
-
 ## {{% heading "cleanup" %}}

 Deleting the Deployments and Services also deletes any running Pods. Use labels to delete multiple resources with one command.

 1. Run the following commands to delete all Pods, Deployments, and Services.

-      ```shell
-      kubectl delete deployment -l app=redis
-      kubectl delete service -l app=redis
-      kubectl delete deployment -l app=guestbook
-      kubectl delete service -l app=guestbook
-      kubectl delete -f filebeat-kubernetes.yaml
-      kubectl delete -f metricbeat-kubernetes.yaml
-      kubectl delete -f packetbeat-kubernetes.yaml
-      kubectl delete secret dynamic-logging -n kube-system
-      ```
+   ```shell
+   kubectl delete deployment -l app=redis
+   kubectl delete service -l app=redis
+   kubectl delete deployment -l app=guestbook
+   kubectl delete service -l app=guestbook
+   kubectl delete -f filebeat-kubernetes.yaml
+   kubectl delete -f metricbeat-kubernetes.yaml
+   kubectl delete -f packetbeat-kubernetes.yaml
+   kubectl delete secret dynamic-logging -n kube-system
+   ```

 1. Query the list of Pods to verify that no Pods are running:

-      ```shell
-      kubectl get pods
-      ```
-
-      The response should be this:
-
-      ```
-      No resources found.
-      ```
+   ```shell
+   kubectl get pods
+   ```

+   The response should be this:

+   ```
+   No resources found.
+   ```

 ## {{% heading "whatsnext" %}}

@@ -399,3 +457,4 @@ Deleting the Deployments and Services also deletes any running Pods. Use labels
 * Read more about [logging architecture](/docs/concepts/cluster-administration/logging/)
 * Read more about [application introspection and debugging](/docs/tasks/debug-application-cluster/)
 * Read more about [troubleshoot applications](/docs/tasks/debug-application-cluster/resource-usage-monitoring/)
+