Merge branch 'master' of git://github.com/kubernetes/website into release-1.12

* 'master' of git://github.com/kubernetes/website: (222 commits) Add temporary owners for 1.13 release (#11453) fix Minikube 404 error. (#11461) Resolve conflicts against dev-1.13 for /ko contents (#11439) replace `run` with `create deployment` (#11392) Updated list all pods with -o wide comment (#11394) fix broken link for KubeletConfiguration (#11423) Update on pod-priority-preemption.md (#11418) Add guidelines for working with localized content (#11415) Update what-is-kubernetes.md (#11399) Remove redundant close tags and little bit formatting (#11389) Add SysEleven MetaKube as hosted solution (#11393) Add rui to sig-docs-zh team (#11391) fix Improper translation (#11384) Add pigletfly(WangBing) as a sig-docs-zh-reviewer (#11370) update link to CloudProvider Interface (#11228) Fix the "my-scheduler-as-kube-scheduler" ClusterRoleBinding. (#11112) fix non-existing "CloudProvider Interface" link (#10953) Updated ingress.md (#11213) Further updates to TLS Bootstrapping (#11258) Updated 'exec' description (#11365) ...
2018-12-03 17:11:43 -05:00
parent 4fa6d715ec b1dde5578c
commit 6e094fa640
399 changed files with 3635 additions and 14434 deletions
@@ -35,6 +35,8 @@ client libraries:

 - [Kubernetes Go client library](https://github.com/kubernetes/client-go/)
 - [Kubernetes Python client library](https://github.com/kubernetes-client/python)
+- [Kubernetes Java client library](https://github.com/kubernetes-client/java)
+- [Kubernetes JavaScript client library](https://github.com/kubernetes-client/javascript)

 ## CLI Reference

@@ -49,12 +49,14 @@ properties:
          - `/foo/*` matches all subpaths of `/foo/`.
    - `readonly`, type boolean, when true, means that the Resource-matching policy only applies to get, list, and watch operations, Non-resource-matching policy only applies to get operation.

-**NOTES:** An unset property is the same as a property set to the zero value for its type
+{{< note >}}
+An unset property is the same as a property set to the zero value for its type
 (e.g. empty string, 0, false). However, unset should be preferred for
 readability.

 In the future, policies may be expressed in a JSON format, and managed via a
 REST interface.
+{{< /note >}}

 ## Authorization Algorithm

@@ -66,11 +66,11 @@ kube-apiserver --enable-admission-plugins=NamespaceLifecycle,LimitRanger ...
 ```

 {{< note >}}
-**Note**: Depending on the way your Kubernetes cluster is deployed and how the
-API server is started, you may need to apply the settings in different ways.
-For example, you may have to modify the systemd unit file if the API server is
-deployed as a systemd service, you may modify the manifest file for the API
-server if Kubernetes is deployed in a self-hosted way.
+Depending on the way your Kubernetes cluster is deployed and how the API server is
+started, you may need to apply the settings in different ways. For example, you may
+have to modify the systemd unit file if the API server is deployed as a systemd
+service, you may modify the manifest file for the API server if Kubernetes is deployed
+in a self-hosted way.
 {{< /note >}}

 ## How do I turn off an admission controller?
@@ -489,7 +489,7 @@ plugin configuration file as the node selector.
 Conflicts result in rejection.

 {{< note >}}
-**Note:** PodNodeSelector allows forcing pods to run on specifically labeled nodes. Also see the PodTolerationRestriction 
+PodNodeSelector allows forcing pods to run on specifically labeled nodes. Also see the PodTolerationRestriction 
 admission plugin, which allows preventing pods from running on specifically tainted nodes.
 {{< /note >}}

@@ -498,7 +498,7 @@ admission plugin, which allows preventing pods from running on specifically tain
 This admission controller implements additional validations for checking incoming `PersistentVolumeClaim` resize requests.

 {{< note >}}
-**Note:** Support for volume resizing is available as an alpha feature. Admins must set the feature gate `ExpandPersistentVolumes`
+Support for volume resizing is available as an alpha feature. Admins must set the feature gate `ExpandPersistentVolumes`
 to `true` to enable resizing.
 {{< /note >}}

@@ -610,7 +610,7 @@ Yes.
 For Kubernetes version 1.10 and later, we recommend running the following set of admission controllers using the `--enable-admission-plugins` flag (**order doesn't matter**).

 {{< note >}}
-**Note:** `--admission-control` was deprecated in 1.10 and replaced with `--enable-admission-plugins`.
+`--admission-control` was deprecated in 1.10 and replaced with `--enable-admission-plugins`.
 {{< /note >}}

 ```shell
@@ -97,7 +97,7 @@ The token file is a csv file with a minimum of 3 columns: token, user name, user
 followed by optional group names.

 {{< note >}}
-**Note:** If you have more than one group the column must be double quoted e.g.
+If you have more than one group the column must be double quoted e.g.

 ```conf
 token,user,uid,"group1,group2,group3"
@@ -137,7 +137,7 @@ Authorization: Bearer 781292.db7bc3a58fc5f07e
 ```

 You must enable the Bootstrap Token Authenticator with the
-`--experimental-bootstrap-token-auth` flag on the API Server.  You must enable
+`--enable-bootstrap-token-auth` flag on the API Server.  You must enable
 the TokenCleaner controller via the `--controllers` flag on the Controller
 Manager.  This is done with something like `--controllers=*,tokencleaner`.
 `kubeadm` will do this for you if you are using it to bootstrap a cluster.
@@ -190,7 +190,7 @@ talk to the API server. Accounts may be explicitly associated with pods using th
 `serviceAccountName` field of a `PodSpec`.

 {{< note >}}
-**Note:** `serviceAccountName` is usually omitted because this is done automatically.
+`serviceAccountName` is usually omitted because this is done automatically.
 {{< /note >}}

 ```yaml
@@ -246,7 +246,7 @@ type: kubernetes.io/service-account-token
 ```

 {{< note >}}
-**Note:** Values are base64 encoded because secrets are always base64 encoded.
+Values are base64 encoded because secrets are always base64 encoded.
 {{< /note >}}

 The signed JWT can be used as a bearer token to authenticate as the given service
@@ -510,8 +510,9 @@ It is designed for use in combination with an authenticating proxy, which sets t
 * `--requestheader-username-headers` Required, case-insensitive. Header names to check, in order, for the user identity. The first header containing a value is used as the username.
 * `--requestheader-group-headers` 1.6+. Optional, case-insensitive. "X-Remote-Group" is suggested. Header names to check, in order, for the user's groups. All values in all specified headers are used as group names.
 * `--requestheader-extra-headers-prefix` 1.6+. Optional, case-insensitive. "X-Remote-Extra-" is suggested. Header prefixes to look for to determine extra information about the user (typically used by the configured authorization plugin). Any headers beginning with any of the specified prefixes have the prefix removed. The remainder of the header name is lowercased and [percent-decoded](https://tools.ietf.org/html/rfc3986#section-2.1) and becomes the extra key, and the header value is the extra value.
+
 {{< note >}}
-**Note:** Prior to 1.11.3 (and 1.10.7, 1.9.11), the extra key could only contain characters which were [legal in HTTP header labels](https://tools.ietf.org/html/rfc7230#section-3.2.6).
+Prior to 1.11.3 (and 1.10.7, 1.9.11), the extra key could only contain characters which were [legal in HTTP header labels](https://tools.ietf.org/html/rfc7230#section-3.2.6).
 {{< /note >}}

 For example, with this configuration:
@@ -601,7 +602,7 @@ The following HTTP headers can be used to performing an impersonation request:
 * `Impersonate-Extra-( extra name )`: A dynamic header used to associate extra fields with the user. Optional. Requires "Impersonate-User". In order to be preserved consistently, `( extra name )` should be lower-case, and any characters which aren't [legal in HTTP header labels](https://tools.ietf.org/html/rfc7230#section-3.2.6) MUST be utf8 and [percent-encoded](https://tools.ietf.org/html/rfc3986#section-2.1).

 {{< note >}}
-**Note:** Prior to 1.11.3 (and 1.10.7, 1.9.11), `( extra name )` could only contain characters which were [legal in HTTP header labels](https://tools.ietf.org/html/rfc7230#section-3.2.6).
+Prior to 1.11.3 (and 1.10.7, 1.9.11), `( extra name )` could only contain characters which were [legal in HTTP header labels](https://tools.ietf.org/html/rfc7230#section-3.2.6).
 {{< /note >}}

 An example set of headers:
@@ -168,12 +168,11 @@ secrets the user cannot themselves read, or that run under a service account
 with different/greater permissions.

 {{< caution >}}
-**Caution:** System administrators, use care when granting access to pod
-creation.  A user granted permission to create pods (or controllers that create
-pods) in the namespace can: read all secrets in the namespace; read all config
-maps in the namespace; and impersonate any service account in the namespace and
-take any action the account could take. This applies regardless of authorization
-mode.
+System administrators, use care when granting access to pod creation. A user
+granted permission to create pods (or controllers that create pods) in the
+namespace can: read all secrets in the namespace; read all config maps in the
+namespace; and impersonate any service account in the namespace and take any
+action the account could take. This applies regardless of authorization mode.
 {{< /caution >}}
 {{% /capture %}}

@@ -179,7 +179,7 @@ verify the JWS using the `HS256` scheme (HMAC-SHA256) with the full token (e.g.
 is used.

 {{< warning >}}
-**Warning:** Any party with a bootstrapping token can create a valid signature for that
+Any party with a bootstrapping token can create a valid signature for that
 token. When using ConfigMap signing it's discouraged to share the same token with
 many clients, since a compromised client can potentially man-in-the middle another
 client relying on the signature to bootstrap TLS trust.
@@ -120,7 +120,7 @@ webhooks:
 ```

 {{< note >}}
-**Note:** When using `clientConfig.service`, the server cert must be valid for
+When using `clientConfig.service`, the server cert must be valid for
 `<svc_name>.<svc_namespace>.svc`.
 {{< /note >}}

@@ -132,7 +132,7 @@ After you create the webhook configuration, the system will take a few seconds
 to honor the new configuration.

 {{< note >}}
-**Note:** When the webhook plugin is deployed into the Kubernetes cluster as a
+When the webhook plugin is deployed into the Kubernetes cluster as a
 service, it has to expose its service on the 443 port. The communication
 between the API server and the webhook service may fail if a different port
 is used.
@@ -228,7 +228,7 @@ admission controller), or to inject defaults (e.g., the
 admission controller), etc.

 {{< note >}}
-**Note:** If your use case does not involve mutating objects, consider using
+If your use case does not involve mutating objects, consider using
 external admission webhooks, as they have better performance.
 {{< /note >}}

@@ -807,7 +807,7 @@ In order from most secure to least secure, the approaches are:

    If an application does not specify a `serviceAccountName`, it uses the "default" service account.

-    {{< note >}}**Note:** Permissions given to the "default" service
+    {{< note >}}Permissions given to the "default" service
    account are available to any pod in the namespace that does not
    specify a `serviceAccountName`.{{< /note >}}

@@ -823,7 +823,7 @@ In order from most secure to least secure, the approaches are:
    Many [add-ons](/docs/concepts/cluster-administration/addons/) currently run as the "default" service account in the `kube-system` namespace.
    To allow those add-ons to run with super-user access, grant cluster-admin permissions to the "default" service account in the `kube-system` namespace.

-    {{< note >}}**Note:** Enabling this means the `kube-system`
+    {{< note >}}Enabling this means the `kube-system`
    namespace contains secrets that grant super-user access to the
    API.{{< /note >}}

@@ -863,9 +863,11 @@ In order from most secure to least secure, the approaches are:

    If you don't care about partitioning permissions at all, you can grant super-user access to all service accounts.

-    {{< warning >}}**Warning:** This allows any user with read access
+    {{< warning >}}
+    This allows any user with read access
    to secrets or the ability to create a pod to access super-user
-    credentials.{{< /warning >}}
+    credentials.
+    {{< /warning >}}

    ```shell
    kubectl create clusterrolebinding serviceaccounts-cluster-admin \
@@ -909,7 +911,7 @@ in the server logs, you can remove the ABAC authorizer.
 You can replicate a permissive policy using RBAC role bindings.

 {{< warning >}}
-**Warning:** The following policy allows **ALL** service accounts to act as cluster administrators.
+The following policy allows **ALL** service accounts to act as cluster administrators.
 Any application running in a container receives service account credentials automatically,
 and could perform any action against the API, including viewing secrets and modifying permissions.
 This is not a recommended policy.
@@ -23,14 +23,14 @@ incomplete features are referred to in order to better describe service accounts
 Kubernetes distinguishes between the concept of a user account and a service account
 for a number of reasons:

-  - User accounts are for humans.  Service accounts are for processes, which
+  - User accounts are for humans. Service accounts are for processes, which
    run in pods.
  - User accounts are intended to be global. Names must be unique across all
    namespaces of a cluster, future user resource will not be namespaced.
    Service accounts are namespaced.
  - Typically, a cluster's User accounts might be synced from a corporate
    database, where new user account creation requires special privileges and
-    is tied to complex business  processes.  Service account creation is intended
+    is tied to complex business processes. Service account creation is intended
    to be more lightweight, allowing cluster users to create service accounts for
    specific tasks (i.e. principle of least privilege).
  - Auditing considerations for humans and service accounts may differ.
@@ -38,13 +38,17 @@ different Kubernetes components.
 | `APIListChunking` | `true` | Beta | 1.9 | |
 | `APIResponseCompression` | `false` | Alpha | 1.7 | |
 | `AppArmor` | `true` | Beta | 1.4 | |
-| `AttachVolumeLimit` | `true` | Alpha | 1.11 | |
+| `AttachVolumeLimit` | `true` | Alpha | 1.11 | 1.11 |
+| `AttachVolumeLimit` | `true` | Beta | 1.12 | |
 | `BlockVolume` | `false` | Alpha | 1.9 | |
+| `CPUCFSQuotaPeriod` | `false` | Alpha | 1.12 | |
 | `CPUManager` | `false` | Alpha | 1.8 | 1.9 |
 | `CPUManager` | `true` | Beta | 1.10 | |
 | `CRIContainerLogRotation` | `false` | Alpha | 1.10 | 1.10 |
 | `CRIContainerLogRotation` | `true` | Beta| 1.11 | |
-| `CSIBlockVolume` | `false` | Alpha | 1.11 | 1.11 |
+| `CSIBlockVolume` | `false` | Alpha | 1.11 | |
+| `CSIDriverRegistry` | `false` | Alpha | 1.12 | |
+| `CSINodeInfo` | `false` | Alpha | 1.12 | |
 | `CSIPersistentVolume` | `false` | Alpha | 1.9 | 1.9 |
 | `CSIPersistentVolume` | `true` | Beta | 1.10 | |
 | `CustomPodDNS` | `false` | Alpha | 1.9 | 1.9 |
@@ -55,6 +59,7 @@ different Kubernetes components.
 | `DebugContainers` | `false` | Alpha | 1.10 | |
 | `DevicePlugins` | `false` | Alpha | 1.8 | 1.9 |
 | `DevicePlugins` | `true` | Beta | 1.10 | |
+| `DynamicAuditing` | `false` | Alpha | 1.13 | |
 | `DynamicKubeletConfig` | `false` | Alpha | 1.4 | 1.10 |
 | `DynamicKubeletConfig` | `true` | Beta | 1.11 | |
 | `DynamicProvisioningScheduling` | `false` | Alpha | 1.11 | 1.11 |
@@ -80,6 +85,7 @@ different Kubernetes components.
 | `MountPropagation` | `false` | Alpha | 1.8 | 1.9 |
 | `MountPropagation` | `true` | Beta | 1.10 | 1.11 |
 | `MountPropagation` | `true` | GA | 1.12 | |
+| `NodeLease` | `false` | Alpha | 1.12 | |
 | `PersistentLocalVolumes` | `false` | Alpha | 1.7 | 1.9 |
 | `PersistentLocalVolumes` | `true` | Beta | 1.10 | |
 | `PodPriority` | `false` | Alpha | 1.8 | |
@@ -87,11 +93,14 @@ different Kubernetes components.
 | `PodReadinessGates` | `true` | Beta | 1.12 | |
 | `PodShareProcessNamespace` | `false` | Alpha | 1.10 | |
 | `PodShareProcessNamespace` | `true` | Beta | 1.12 | |
+| `ProcMountType` | `false` | Alpha | 1.12 | |
 | `PVCProtection` | `false` | Alpha | 1.9 | 1.9 |
-| `ReadOnlyAPIDataVolumes` | `true` | Deprecated | 1.10 | |
 | `ResourceLimitsPriorityFunction` | `false` | Alpha | 1.9 | |
+| `ResourceQuotaScopeSelectors` | `false` | Alpha | 1.11 | 1.11 |
+| `ResourceQuotaScopeSelectors` | `true` | Beta | 1.12 | |
 | `RotateKubeletClientCertificate` | `true` | Beta | 1.7 | |
-| `RotateKubeletServerCertificate` | `false` | Alpha | 1.7 | |
+| `RotateKubeletServerCertificate` | `false` | Alpha | 1.7 | 1.11 |
+| `RotateKubeletServerCertificate` | `true` | Beta | 1.12 | |
 | `RunAsGroup` | `false` | Alpha | 1.10 | |
 | `RuntimeClass` | `false` | Alpha | 1.12 | |
 | `SCTPSupport` | `false` | Alpha | 1.12 | |
@@ -116,6 +125,7 @@ different Kubernetes components.
 | `VolumeScheduling` | `false` | Alpha | 1.9 | 1.9 |
 | `VolumeScheduling` | `true` | Beta | 1.10 | |
 | `VolumeSubpathEnvExpansion` | `false` | Alpha | 1.11 | |
+| `VolumeSnapshotDataSource` | `false` | Alpha | 1.12 | |
 | `ScheduleDaemonSetPods` | `true` | Beta | 1.12 | |

 ## Using a Feature
@@ -147,7 +157,7 @@ A *Beta* feature means:
  that can be upgraded independently, you may be able to relax this restriction.

 {{< note >}}
-**Note:** Please do try *Beta* features and give feedback on them!
+Please do try *Beta* features and give feedback on them!
 After they exit beta, it may not be practical for us to make more changes.
 {{< /note >}}

@@ -174,9 +184,12 @@ Each feature gate is designed for enabling/disabling a specific feature:
 - `BlockVolume`: Enable the definition and consumption of raw block devices in Pods.
   See [Raw Block Volume Support](/docs/concepts/storage/persistent-volumes/#raw-block-volume-support)
   for more details.
+- `CPUCFSQuotaPeriod`: Enable nodes to change CPUCFSQuotaPeriod.
 - `CPUManager`: Enable container level CPU affinity support, see [CPU Management Policies](/docs/tasks/administer-cluster/cpu-management-policies/).
 - `CRIContainerLogRotation`: Enable container log rotation for cri container runtime.
 - `CSIBlockVolume`: Enable external CSI volume drivers to support block storage. See the [`csi` raw block volume support](/docs/concepts/storage/volumes/#csi-raw-block-volume-support) documentation for more details.
+- `CSIDriverRegistry`: Enable all logic related to the CSIDriver API object in csi.storage.k8s.io.
+- `CSINodeInfo`: Enable all logic related to the CSINodeInfo API object in csi.storage.k8s.io.
 - `CSIPersistentVolume`: Enable discovering and mounting volumes provisioned through a
  [CSI (Container Storage Interface)](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/container-storage-interface.md)
  compatible volume plugin.
@@ -192,6 +205,7 @@ Each feature gate is designed for enabling/disabling a specific feature:
  troubleshoot a running Pod.
 - `DevicePlugins`: Enable the [device-plugins](/docs/concepts/cluster-administration/device-plugins/)
  based resource provisioning on nodes.
+- `DynamicAuditing`: Enable [dynamic auditing](/docs/tasks/debug-application-cluster/audit/#dynamic-backend)
 - `DynamicKubeletConfig`: Enable the dynamic configuration of kubelet. See [Reconfigure kubelet](/docs/tasks/administer-cluster/reconfigure-kubelet/).
 - `DynamicProvisioningScheduling`: Extend the default scheduler to be aware of volume topology and handle PV provisioning.
  This feature is superceded by the `VolumeScheduling` feature completely in v1.12.
@@ -219,21 +233,22 @@ Each feature gate is designed for enabling/disabling a specific feature:
 - `MountContainers`: Enable using utility containers on host as the volume mounter.
 - `MountPropagation`: Enable sharing volume mounted by one container to other containers or pods.
  For more details, please see [mount propagation](/docs/concepts/storage/volumes/#mount-propagation).
+- `NodeLease`: Enable the new Lease API to report node heartbeats, which could be used as a node health signal.
 - `PersistentLocalVolumes`: Enable the usage of `local` volume type in Pods.
  Pod affinity has to be specified if requesting a `local` volume.
 - `PodPriority`: Enable the descheduling and preemption of Pods based on their [priorities](/docs/concepts/configuration/pod-priority-preemption/).
 - `PodReadinessGates`: Enable the setting of `PodReadinessGate` field for extending
  Pod readiness evaluation.
  For more details, please see [Pod readiness gate](/docs/concepts/workloads/pods/pod-lifecycle/#pod-readiness-gate).
+- `ProcMountType`: Enables control over ProcMountType for containers.
 - `PVCProtection`: Enable the prevention of a PersistentVolumeClaim (PVC) from
  being deleted when it is still used by any Pod.
  More details can be found [here](/docs/tasks/administer-cluster/storage-object-in-use-protection/).
- `ReadOnlyAPIDataVolumes`: Set Secret, ConfigMap, DownwardAPI and projected volumes to be mounted in read-only mode.
-  This gate exists only for backward compatibility. It will be removed in 1.11 release.
 - `ResourceLimitsPriorityFunction`: Enable a scheduler priority function that
  assigns a lowest possible score of 1 to a node that satisfies at least one of
  the input Pod's cpu and memory limits. The intent is to break ties between
  nodes with same scores.
+- `ResourceQuotaScopeSelectors`: Enable resource quota scope selectors.
 - `RotateKubeletClientCertificate`: Enable the rotation of the client TLS certificate on the kubelet.
  See [kubelet configuration](/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping/#kubelet-configuration) for more details.
 - `RotateKubeletServerCertificate`: Enable the rotation of the server TLS certificate on the kubelet.
@@ -265,4 +280,6 @@ Each feature gate is designed for enabling/disabling a specific feature:
  PersistentVolumeClaim (PVC) binding aware of scheduling decisions. It also
  enables the usage of [`local`](/docs/concepts/storage/volumes/#local) volume
  type when used together with the `PersistentLocalVolumes` feature gate.
+- `VolumeSnapshotDataSource`: Enable volume snapshot data source support.
+
 {{% /capture %}}
@@ -36,6 +36,12 @@ kube-scheduler [flags]
    <tr>
      <td></td><td style="line-height: 130%; word-wrap: break-word;">DEPRECATED: the IP address on which to listen for the --port port (set to 0.0.0.0 for all IPv4 interfaces and :: for all IPv6 interfaces). See --bind-address instead.</td>
    </tr>
+    <tr>
+      <td colspan="2">--bind-address string&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Default: "0.0.0.0"</td>
+    </tr>
+    <tr>
+      <td></td><td style="line-height: 130%; word-wrap: break-word;">The IP address on which to listen for the --secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If blank, all interfaces will be used (0.0.0.0 for all IPv4 interfaces and :: for all IPv6 interfaces).</td>
+    </tr>

    <tr>
      <td colspan="2">--algorithm-provider string</td>
@@ -198,6 +204,13 @@ kube-scheduler [flags]
      <td></td><td style="line-height: 130%; word-wrap: break-word;">DEPRECATED: the port on which to serve HTTP insecurely without authentication and authorization. If 0, don't serve HTTPS at all. See --secure-port instead.</td>
    </tr>

+    <tr>
+      <td colspan="2">--secure-port int&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Default: 10259</td>
+    </tr>
+    <tr>
+      <td></td><td style="line-height: 130%; word-wrap: break-word;">The port on which to serve HTTPS with authentication and authorization. If 0, don't serve HTTPS at all.</td>
+    </tr>
+
    <tr>
      <td colspan="2">--profiling</td>
    </tr>
@@ -10,45 +10,160 @@ content_template: templates/concept

 {{% capture overview %}}

-This document describes how to set up TLS client certificate bootstrapping for
-kubelets. Kubernetes 1.4 introduced an API for requesting certificates from a
-cluster-level Certificate Authority (CA). The original intent of this API is to
-enable provisioning of TLS client certificates for kubelets. The proposal can be
+In a Kubernetes cluster, the components on the worker nodes - kubelet and kube-proxy - need to communicate with Kubernetes master components, specifically kube-apiserver.
+In order to ensure that communication is kept private, not interfered with, and ensure that each component of the cluster is talking to another trusted component, we strongly
+recommend using client TLS certificates on nodes.
+
+The normal process of bootstrapping these components, especially worker nodes that need certificates so they can communicate safely with kube-apiserver, 
+can be a challenging process as it is often outside of the scope of Kubernetes and requires significant additional work. 
+This in turn, can make it challenging to initialize or scale a cluster.
+
+In order to simplify the process, beginning in version 1.4, Kubernetes introduced a certificate request and signing API to simplify the process. The proposal can be
 found [here](https://github.com/kubernetes/kubernetes/pull/20439).

+This document describes the process of node initialization, how to set up TLS client certificate bootstrapping for
+kubelets, and how it works. 
+
 {{% /capture %}}

 {{% capture body %}}

+## Initialization Process
+When a worker node starts up, the kubelet does the following:
+
+1. Look for its `kubeconfig` file
+2. Retrieve the URL of the API server and credentials, normally a TLS key and signed certificate from the `kubeconfig` file
+3. Attempt to communicate with the API server using the credentials.
+
+Assuming that the kube-apiserver successfully validates the kubelet's credentials, it will treat the kubelet as a valid node, and begin to assign pods to it.
+
+Note that the above process depends upon:
+
+* Existence of a key and certificate on the local host in the `kubeconfig`
+* The certificate having been signed by a Certificate Authority (CA) trusted by the kube-apiserver
+
+All of the following are responsibilities of whoever sets up and manages the cluster:
+
+1. Creating the CA key and certificate
+2. Distributing the CA certificate to the master nodes, where kube-apiserver is running
+3. Creating a key and certificate for each kubelet; strongly recommended to have a unique one, with a unique CN, for each kubelet
+4. Signing the kubelet certificate using the CA key
+5. Distributing the kubelet key and signed certificate to the specific node on which the kubelet is running
+
+The TLS Bootstrapping described in this document is intended to simplify, and partially or even completely automate, steps 3 onwards, as these are the most common when initializing or scaling
+a cluster.
+
+### Bootstrap Initialization
+In the bootstrap initialization process, the following occurs:
+
+1. kubelet begins
+2. kubelet sees that it does _not_ have a `kubeconfig` file
+3. kubelet searches for and finds a `bootstrap-kubeconfig` file
+4. kubelet reads its bootstrap file, retrieving the URL of the API server and a limited usage "token"
+5. kubelet connects to the API server, authenticates using the token
+6. kubelet now has limited credentials to create and retrieve a certificate signing request (CSR)
+7. kubelet creates a CSR for itself
+8. CSR is approved in one of two ways:
+  * If configured, kube-controller-manager automatically approves the CSR
+  * If configured, an outside process, possibly a person, approves the CSR using the Kubernetes API or via `kubectl`
+9. Certificate is created for the kubelet
+10. Certificate is issued to the kubelet
+11. kubelet retrieves the certificate
+12. kubelet creates a proper `kubeconfig` with the key and signed certificate
+13. kubelet begins normal operation
+14. Optional: if configured, kubelet automatically requests renewal of the certificate when it is close to expiry
+15. The renewed certificate is approved and issued, either automatically or manually, depending on configuration.
+
+The rest of this document describes the necessary steps to configure TLS Bootstrapping, and its limitations.
+
+## Configuration
+To configure for TLS bootstrapping and optional automatic approval, you must configure options on the following components:
+
+* kube-apiserver
+* kube-controller-manager
+* kubelet
+* in-cluster resources: `ClusterRoleBinding` and potentially `ClusterRole`
+
+In addition, you need your Kubernetes Certificate Authority (CA).
+
+## Certificate Authority
+As without bootstrapping, you will need a Certificate Authority (CA) key and certificate. As without bootstrapping, these will be used
+to sign the kubelet certificate. As before, it is your responsibility to distribute them to master nodes.
+
+For the purposes of this document, we will assume these have been distributed to master nodes at `/var/lib/kubernetes/ca.pem` (certificate) and `/var/lib/kubernetes/ca-key.pem` (key). 
+We will refer to these as "Kubernetes CA certificate and key".
+
+All Kubernetes components that use these certificates - kubelet, kube-apiserver, kube-controller-manager - assume the key and certificate to be PEM-encoded.
+
 ## kube-apiserver configuration
+The kube-apiserver has several requirements to enable TLS bootstrapping:

-The API server should be configured with an
-[authenticator](/docs/reference/access-authn-authz/authentication/) that can
-authenticate tokens as a user in the `system:bootstrappers` group.
+* Recognizing CA that signs the client certificate 
+* Authenticating the bootstrapping kubelet to the `system:bootstrappers` group
+* Authorize the bootstrapping kubelet to create a certificate signing request (CSR)

-This group will later be used in the controller-manager configuration to scope
-approvals in the default approval controller. As this feature matures, you
-should ensure tokens are bound to a Role Based Access Control (RBAC) policy
-which limits requests (using the [bootstrap
-token](/docs/reference/access-authn-authz/bootstrap-tokens/)) strictly to client
-requests related to certificate provisioning. With RBAC in place, scoping the
-tokens to a group allows for great flexibility (e.g. you could disable a
-particular bootstrap group's access when you are done provisioning the nodes).
+### Recognizing client certificates
+This is normal for all client certificate authentication.
+If not already set, add the `--client-ca-file=FILENAME` flag to the kube-apiserver command to enable
+client certificate authentication, referencing a certificate authority bundle
+containing the signing certificate, for example
+`--client-ca-file=/var/lib/kubernetes/ca.pem`.
+
+### Initial bootstrap authentication
+In order for the bootstrapping kubelet to connect to kube-apiserver and request a certificate, it must first authenticate to the server.
+You can use any [authenticator](https://kubernetes.io/docs/reference/access-authn-authz/authentication/) that can authenticate the kubelet.

 While any authentication strategy can be used for the kubelet's initial
 bootstrap credentials, the following two authenticators are recommended for ease
 of provisioning.

-1. [Bootstrap Tokens](/docs/reference/access-authn-authz/bootstrap-tokens/) - __beta__
+1. [Bootstrap Tokens](#bootstrap-tokens) - __beta__
 2. [Token authentication file](#token-authentication-file)

-Using bootstrap tokens is currently __beta__ and will simplify the management of
-bootstrap token management especially in a HA scenario.
+Bootstrap tokens are a simpler and more easily managed method to authenticate kubelets, and do not require any additional flags when starting kube-apiserver. 
+Using bootstrap tokens is currently __beta__ as of Kubernetes version 1.12.

-### Token authentication file
+Whichever method you choose, the requirement is that the kubelet be able to authenticate as a user with the rights to:

-Tokens are arbitrary but should represent at least 128 bits of entropy derived
-from a secure random number generator (such as /dev/urandom on most modern Linux
+1. create and retrieve CSRs
+2. be automatically approved to request node client certificates, if automatic approval is enabled. 
+
+A kubelet authenticating using bootstrap tokens is authenticated as a user in the group `system:bootstrappers`, which is the standard method to use.
+
+As this feature matures, you
+should ensure tokens are bound to a Role Based Access Control (RBAC) policy
+which limits requests (using the [bootstrap
+token](/docs/reference/access-authn-authz/bootstrap-tokens/)) strictly to client
+requests related to certificate provisioning. With RBAC in place, scoping the
+tokens to a group allows for great flexibility. For example, you could disable a
+particular bootstrap group's access when you are done provisioning the nodes.
+
+#### Bootstrap tokens
+Bootstrap tokens are described in detail [here](/docs/reference/access-authn-authz/bootstrap-tokens/). These are tokens that are stored as secrets in the Kubernetes cluster,
+and then issued to the individual kubelet. You can use a single token for an entire cluster, or issue one per worker node.
+
+The process is two-fold:
+
+1. Create a Kubernetes secret with the token ID, secret and scope(s).
+2. Issue the token to the kubelet
+
+From the kubelet's perspective, one token is like another and has no special meaning.
+From the kube-apiserver's perspective, however, the bootstrap token is special. Due to its `Type`, `namespace` and `name`, kube-apiserver recognizes it as a special token,
+and grants anyone authenticating with that token special bootstrap rights, notably treating them as a member of the `system:bootstrappers` group. This fulfills a basic requirement
+for TLS bootstrapping.
+
+The details for creating the secret are available [here](/docs/reference/access-authn-authz/bootstrap-tokens/). 
+
+If you want to use bootstrap tokens, you must enable it on kube-apiserver with the flag:
+
+```
+--enable-bootstrap-token-auth=true
+```
+
+#### Token authentication file
+kube-apiserver has an ability to accept tokens as authentication. 
+These tokens are arbitrary but should represent at least 128 bits of entropy derived
+from a secure random number generator (such as `/dev/urandom` on most modern Linux
 systems). There are multiple ways you can generate a token. For example:

 ```
@@ -69,99 +184,78 @@ systemd unit file perhaps) to enable the token file.  See docs
 [here](/docs/reference/access-authn-authz/authentication/#static-token-file) for
 further details.

-### Client certificate CA bundle
+### Authorize kubelet to create CSR
+Now that the bootstrapping node is _authenticated_ as part of the `system:bootstrappers` group, it needs to be _authorized_ to create a certificate signing request (CSR) as well as retrieve it when done. Fortunately, Kubernetes ships with a `ClusterRole` with precisely these (and just these) permissions, `system:node-bootstrapper`.

-Add the `--client-ca-file=FILENAME` flag to the kube-apiserver command to enable
-client certificate authentication, referencing a certificate authority bundle
-containing the signing certificate (e.g.
-`--client-ca-file=/var/lib/kubernetes/ca.pem`).
+To do this, you just need to create a `ClusterRoleBinding` that binds the `system:bootstrappers` group to the cluster role `system:node-bootstrapper`.
+
+```
+# enable bootstrapping nodes to create CSR
+kind: ClusterRoleBinding
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+  name: create-csrs-for-bootstrapping
+subjects:
+- kind: Group
+  name: system:bootstrappers
+  apiGroup: rbac.authorization.k8s.io
+roleRef:
+  kind: ClusterRole
+  name: system:node-bootstrapper
+  apiGroup: rbac.authorization.k8s.io
+```

 ## kube-controller-manager configuration
+While the apiserver receives the requests for certificates from the kubelet and authenticates those requests,
+the controller-manager is responsible for issuing actual signed certificates.

-The API for requesting certificates adds a certificate-issuing control loop to
-the Kubernetes Controller Manager. This takes the form of a
+The controller-manager performs this function via a certificate-issuing control loop.
+This takes the form of a
 [cfssl](https://blog.cloudflare.com/introducing-cfssl/) local signer using
 assets on disk. Currently, all certificates issued have one year validity and a
 default set of key usages.

-### Signing assets
+In order for the controller-manager to sign certificates, it needs the following:

-You must provide a Certificate Authority in order to provide the cryptographic
-materials necessary to issue certificates.  This CA should be trusted by
-kube-apiserver for authentication with the `--client-ca-file=FILENAME` flag. The
-management of the CA is beyond the scope of this document but it is recommended
-that you generate a dedicated CA for Kubernetes.  Both certificate and key are
-assumed to be PEM-encoded.
+* access to the "kuberetes CA key and certificate" that you created and distributed
+* enabling CSR signing

-The kube-controller-manager flags are:
+### Access to key and certificate
+As described earlier, you need to create a Kubernetes CA key and certificate, and distribute it to the master nodes.
+These will be used by the controller-manager to sign the kubelet certificates.
+
+Since these signed certificates will, in turn, be used by the kubelet to authenticate as a regular kubelet to kube-apiserver, it is important that the CA
+provided to the controller-manager at this stage also be trusted by kube-apiserver for authentication. This is provided to kube-apiserver
+with the flag `--client-ca-file=FILENAME` (for example, `--client-ca-file=/var/lib/kubernetes/ca.pem`), as described in the kube-apiserver configuration section.
+
+To provide the Kubernetes CA key and certificate to kube-controller-manager, use the following flags:

 ```
 --cluster-signing-cert-file="/etc/path/to/kubernetes/ca/ca.crt" --cluster-signing-key-file="/etc/path/to/kubernetes/ca/ca.key"
 ```

+for example:
+
+```
+--cluster-signing-cert-file="/var/lib/kubernetes/ca.pem" --cluster-signing-key-file="/var/lib/kubernetes/ca-key.pem"
+```
+
 The validity duration of signed certificates can be configured with flag:

 ```
 --experimental-cluster-signing-duration
 ```

-### SubjectAccessReview Approval Controller
+### Approval
+In order to approve CSRs, you need to tell the controller-manager that it is acceptable to approve them. This is done by granting
+RBAC permissions to the correct group.

-The `csrapproving` controller that ships as part of
-[kube-controller-manager](/docs/admin/kube-controller-manager/) and is enabled
-by default. The controller uses the [`SubjectAccessReview`
-API](/docs/reference/access-authn-authz/authorization/#checking-api-access) to
-determine if a given user is authorized to request a CSR, then approves based on
-the authorization outcome. To prevent conflicts with other approvers, the
-builtin approver doesn't explicitly deny CSRs. It only ignores unauthorized
-requests. The controller also prunes expired certificates as part of garbage
-collection.
+There are two distinct sets of permissions:

-The controller categorizes CSRs into three subresources:
+* `nodeclient`: If a node is creating a new certificate for a node, then it does not have a certificate yet. It is authenticating using one of the tokens listed above, and thus is part of the group `system:bootstrappers`. 
+* `selfnodeclient`: If a node is renewing its certificate, then it already has a certificate (by definition), which it uses continuously to authenticate as part of the group `system:nodes`.  

-1. `nodeclient` - a request by a user for a client certificate with `O=system:nodes` and `CN=system:node:(node name)`.
-2. `selfnodeclient` - a node renewing a client certificate with the same `O` and `CN`. A node can use its existing client certificate to authenticate this request.
-
-The following RBAC `ClusterRoles` represent the `nodeclient` and
-`selfnodeclient`, capabilities.
-
-```yml
-# A ClusterRole which instructs the CSR approver to approve a user requesting
-# node client credentials.
-kind: ClusterRole
-apiVersion: rbac.authorization.k8s.io/v1
-metadata:
-  name: approve-node-client-csr
-rules:
- apiGroups: ["certificates.k8s.io"]
-  resources: ["certificatesigningrequests/nodeclient"]
-  verbs: ["create"]
---
-# A ClusterRole which instructs the CSR approver to approve a node renewing its
-# own client credentials.
-kind: ClusterRole
-apiVersion: rbac.authorization.k8s.io/v1
-metadata:
-  name: approve-node-client-renewal-csr
-rules:
- apiGroups: ["certificates.k8s.io"]
-  resources: ["certificatesigningrequests/selfnodeclient"]
-  verbs: ["create"]
-```
-
-As of 1.8, equivalent roles to the ones listed above are automatically created
-as part of the default RBAC roles. For 1.8 clusters admins are recommended to
-bind node bootstrap identities to the following roles instead of creating their
-own:
-
-* `system:certificates.k8s.io:certificatesigningrequests:nodeclient`
-    - Automatically approve CSRs for client certs bound to this role.
-* `system:certificates.k8s.io:certificatesigningrequests:selfnodeclient`
-    - Automatically approve CSRs when a client bound to its role renews its own certificate.
-
-For example, to grant these permissions to identities attached to bootstrap
-tokens, an admin would create a `ClusterRoleBinding` targeting the
-`system:bootstrappers` group:
+To enable the kubelet to request and receive a new certificate, create a `ClusterRoleBinding` that binds the group in which the bootstrapping node is a member `system:bootstrappers` to the `ClusterRole` that grants it permission, `system:certificates.k8s.io:certificatesigningrequests:nodeclient`:

 ```yml
 # Approve all CSRs for the group "system:bootstrappers"
@@ -175,41 +269,126 @@ subjects:
  apiGroup: rbac.authorization.k8s.io
 roleRef:
  kind: ClusterRole
-  name: approve-node-client-csr
+  name: system:certificates.k8s.io:certificatesigningrequests:nodeclient
  apiGroup: rbac.authorization.k8s.io
 ```

-To let all nodes renew their own credentials, an admin can create a
-`ClusterRoleBinding` targeting node identities:
+To enable the kubelet to renew its own client certificate, create a `ClusterRoleBinding` that binds the group in which the fully functioning node is a member `system:nodes` to the `ClusterRole` that 
+grants it permission, `system:certificates.k8s.io:certificatesigningrequests:selfnodeclient`:

 ```yml
+# Approve renewal CSRs for the group "system:nodes"
 kind: ClusterRoleBinding
 apiVersion: rbac.authorization.k8s.io/v1
 metadata:
-  name: node1-client-cert-renewal
+  name: auto-approve-renewals-for-nodes
 subjects:
 - kind: Group
  name: system:nodes
  apiGroup: rbac.authorization.k8s.io
 roleRef:
  kind: ClusterRole
-  name: approve-node-client-renewal-csr
+  name: system:certificates.k8s.io:certificatesigningrequests:selfnodeclient
  apiGroup: rbac.authorization.k8s.io
 ```

+**Note: Kubernetes Below 1.8**: If you are running an earlier version of Kubernetes, notably a version below 1.8, then the cluster roles referenced above do not ship by default. You will have to create them yourself _in addition to_ the `ClusterRoleBindings` listed.
+
+To create the `ClusterRole`s:
+
+```yml
+# A ClusterRole which instructs the CSR approver to approve a user requesting
+# node client credentials.
+kind: ClusterRole
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+  name: system:certificates.k8s.io:certificatesigningrequests:nodeclient
+rules:
+- apiGroups: ["certificates.k8s.io"]
+  resources: ["certificatesigningrequests/nodeclient"]
+  verbs: ["create"]
+---
+# A ClusterRole which instructs the CSR approver to approve a node renewing its
+# own client credentials.
+kind: ClusterRole
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+  name: system:certificates.k8s.io:certificatesigningrequests:selfnodeclient
+rules:
+- apiGroups: ["certificates.k8s.io"]
+  resources: ["certificatesigningrequests/selfnodeclient"]
+  verbs: ["create"]
+```
+
+
+The `csrapproving` controller that ships as part of
+[kube-controller-manager](/docs/admin/kube-controller-manager/) and is enabled
+by default. The controller uses the [`SubjectAccessReview`
+API](/docs/reference/access-authn-authz/authorization/#checking-api-access) to
+determine if a given user is authorized to request a CSR, then approves based on
+the authorization outcome. To prevent conflicts with other approvers, the
+builtin approver doesn't explicitly deny CSRs. It only ignores unauthorized
+requests. The controller also prunes expired certificates as part of garbage
+collection.
+
+
 ## kubelet configuration
+Finally, with the master nodes properly set up and all of the necessary authentication and authorization in place, we can configure the kubelet.

-To request a client certificate from kube-apiserver, the kubelet first needs a
-path to a kubeconfig file that contains the credentials for the identity that it
-will use to bootstrap its individual node identity.
+The kubelet requires the following configuration to bootstrap:

-If you are using a bootstrap token, you can use `kubectl config set-cluster`,
-`set-credentials`, and `set-context` to build this kubeconfig.  Provide the name
-`kubelet-bootstrap` to `kubectl config set-credentials` and include
-`--token=<token-value>` as follows:
+* A path to store the key and certificate it generates (optional, can use default)
+* A path to a `kubeconfig` file that does not yet exist; it will place the bootstrapped config file here
+* A path to a bootstrap `kubeconfig` file to provide the URL for the server and bootstrap credentials, e.g. a bootstrap token
+* Optional: instructions to rotate certificates
+
+The bootstrap `kubeconfig` should be in a path available to the kubelet, for example `/var/lib/kubelet/bootstrap-kubeconfig`.
+
+Its format is identical to a normal `kubeconfig` file. A sample file might look as follows:
+
+```yml
+apiVersion: v1
+clusters:
+- cluster:
+    certificate-authority: /var/lib/kubernetes/ca.pem
+    server: https://my.server.example.com:6443
+  name: bootstrap
+contexts:
+- context:
+    cluster: bootstrap
+    user: kubelet-bootstrap
+  name: bootstrap
+current-context: bootstrap
+kind: Config
+preferences: {}
+users:
+- name: kubelet-bootstrap
+  user:
+    token: 07401b.f395accd246ae52d
+```
+
+The important elements to note are:
+
+* `certificate-authority`: path to a CA file, used to validate the server certificate presented by kube-apiserver
+* `server`: URL to kube-apiserver
+* `token`: the token to use
+
+The format of the token does not matter, as long as it matches what kube-apiserver expects. In the above example, we used a bootstrap token.
+As stated earlier, _any_ valid authentication method can be used, not just tokens.
+
+Because the bootstrap `kubeconfig` _is_ a standard `kubeconfig`, you can use `kubectl` to generate it. To create the above example file:

 ```
-kubectl config set-credentials kubelet-bootstrap --token=${BOOTSTRAP_TOKEN} --kubeconfig=bootstrap.kubeconfig
+kubectl config -kubeconfig=/var/lib/kubelet/bootstrap-kubeconfig set-cluster bootstrap --server='https://my.server.example.com:6443' --certificate-authority=/var/lib/kubernetes/ca.pem
+kubectl config -kubeconfig=/var/lib/kubelet/bootstrap-kubeconfig set-credentials kubelet-bootstrap --token=07401b.f395accd246ae52d
+kubectl config -kubeconfig=/var/lib/kubelet/bootstrap-kubeconfig set-context bootstrap --user=kubelet-bootstrap --cluster=bootstrap
+kubectl config -kubeconfig=/var/lib/kubelet/bootstrap-kubeconfig use-context bootstrap
+```
+
+To indicate to the kubelet to use the bootstrap `kubeconfig`, use the following kubelet flag:
+
+```
+--bootstrap-kubeconfig="/var/lib/kubelet/bootstrap-kubeconfig" --kubeconfig="/var/lib/kubelet/kubeconfig"
 ```

 When starting the kubelet, if the file specified via `--kubeconfig` does not
@@ -220,16 +399,25 @@ referencing the generated key and obtained certificate is written to the path
 specified by `--kubeconfig`. The certificate and key file will be placed in the
 directory specified by `--cert-dir`.

-{{< note >}}
-**Note:** The following flags are required to enable this bootstrapping when starting the kubelet:
+### Client and Serving Certificates
+All of the above relate to kubelet _client_ certificates, specifically, the certificates a kubelet
+uses to authenticate to kube-apiserver.

-```
--bootstrap-kubeconfig="/path/to/bootstrap/kubeconfig"
-```
-{{< /note >}}
+A kubelet also can use _serving_ certificates. The kubelet itself exposes an https endpoint for certain features.
+To secure these, the kubelet can do one of:

-Additionally, in 1.7 the kubelet implements __beta__ features for enabling
-rotation of both its client and/or serving certs.  These can be enabled through
+* use provided key and certificate, via the `--tls-private-key-file` and `--tls-cert-file` flags
+* create self-signed key and certificate, if a key and certificate are not provided
+* request serving certificates from the cluster server, via the CSR API
+
+The client certificate provided by TLS bootstrapping is signed, by default, for `client auth` only, and thus cannot
+be used as serving certificates, or `server auth`. 
+
+However, you _can_ enable its server certificate, at least partially, via certificate rotation.
+
+### Certificate Rotation
+Kubernetes v1.7 and higher kubelet implements __beta__ features for enabling
+rotation of its client and/or serving certficates.  These can be enabled through
 the respective `RotateKubeletClientCertificate` and
 `RotateKubeletServerCertificate` feature flags on the kubelet and are enabled by
 default.
@@ -242,8 +430,8 @@ this feature pass the following flag to the kubelet:
 --rotate-certificates
 ```

-`RotateKubeletServerCertificate` causes the kubelet to both request a serving
-certificate after bootstrapping its client credentials and to rotate that
+`RotateKubeletServerCertificate` causes the kubelet **both** to request a serving
+certificate after bootstrapping its client credentials **and** to rotate that
 certificate. To enable this feature pass the following flag to the kubelet:

 ```
@@ -251,13 +439,25 @@ certificate. To enable this feature pass the following flag to the kubelet:
 ```

 {{< note >}}
-**Note:** The CSR approving controllers implemented in core Kubernetes do not
-approve node serving certificates for [security
+The CSR approving controllers implemented in core Kubernetes do not
+approve node _serving_ certificates for [security
 reasons](https://github.com/kubernetes/community/pull/1982). To use
 `RotateKubeletServerCertificate` operators need to run a custom approving
 controller, or manually approve the serving certificate requests.
 {{< /note >}}

+## Other authenticating components
+All of TLS bootstrapping described in this document relates to the kubelet. However,
+other components may need to communicate directly with kube-apiserver. Notable is kube-proxy, which
+is part of the Kubernetes control plane and runs on every node, but may also include other components such as monitoring or networking.
+
+Like the kubelet, these other components also require a method of authenticating to kube-apiserver.
+You have several options for generating these credentials:
+
+* The old way: Create and distribute certificates the same way you did for kubelet before TLS bootstrapping
+* DaemonSet: Since the kubelet itself is loaded on each node, and is sufficient to start base services, you can run kube-proxy and other node-specific services not as a standalone process, but rather as a daemonset in the `kube-system` namespace. Since it will be in-cluster, you can give it a proper service account with appropriate permissions to perform its activities. This may be the simplest way to configure such services.
+
+
 ## kubectl approval

 CSRs can be approved outside of the approval flows builtin to the controller
@@ -273,4 +473,14 @@ list CSRs with `kubectl get csr` and describe one in detail with `kubectl
 describe csr <name>`. An administrator can approve or deny a CSR with `kubectl
 certificate approve <name>` and `kubectl certificate deny <name>`.

+
+## Limits
+Although Kubernetes supports running control plane master components like kube-apiserver and kube-controller-manager in containers, and even as `Pod`s in a kubelet, as of this writing, you cannot both TLS Bootstrap a kubelet and run master plane components on it.
+
+The reason for this limitation is that the kubelet attempts to bootstrap communication with kube-apiserver _before_ starting any pods, even static ones define on disk and referenced via the kubelet option `--pod-manifest-path=<PATH>`. Trying to do both TLS Bootstrapping and master components in kubelet leads to a race condition: kubelet needs to communicate to kube-apiserver to request certificates, yet requires those certificates to be available to start kube-apiserver.
+
+An issue is open referencing this [here](https://github.com/kubernetes/kubernetes/issues/68686).
+
+
+
 {{% /capture %}}
@@ -244,7 +244,7 @@ kubelet [flags]
       <td colspan="2">--container-runtime string</td> 
    </tr>
    <tr>
-      <td></td><td style="line-height: 130%; word-wrap: break-word;">The container runtime to use. Possible values: 'docker', 'rkt'. (default "docker")</td>
+      <td></td><td style="line-height: 130%; word-wrap: break-word;">The container runtime to use. Possible values: 'docker', 'remote', 'rkt(deprecated)'. (default "docker")</td>
    </tr>

     <tr>
@@ -286,7 +286,7 @@ kubelet [flags]
       <td colspan="2">--cpu-manager-policy string</td> 
    </tr>
    <tr>
-      <td></td><td style="line-height: 130%; word-wrap: break-word;"><Warning: Alpha feature> CPU Manager policy to use. Possible values: 'none', 'static'. Default: 'none' (default "none")</td>
+      <td></td><td style="line-height: 130%; word-wrap: break-word;"><Warning: Alpha feature> CPU Manager policy to use. Possible values: 'none', 'static'. (default "none")</td>
    </tr>

     <tr>
@@ -945,7 +945,7 @@ kubelet [flags]
       <td colspan="2">--registry-burst int32</td> 
    </tr>
    <tr>
-      <td></td><td style="line-height: 130%; word-wrap: break-word;">Maximum size of a bursty pulls, temporarily allows pulls to burst to this number, while still not exceeding registry-qps. Only used if --registry-qps > 0 (default 10)</td>
+      <td></td><td style="line-height: 130%; word-wrap: break-word;">Maximum size of bursty pulls, temporarily allows pulls to burst to this number, while still not exceeding registry-qps. Only used if --registry-qps > 0 (default 10)</td>
    </tr>

     <tr>
@@ -1113,7 +1113,7 @@ kubelet [flags]
      <td colspan="2">--tls-cipher-suites stringSlice</td>
    </tr>
    <tr>
-      <td></td><td style="line-height: 130%; word-wrap: break-word;">Comma-separated list of cipher suites for the server. If omitted, the default Go cipher suites will be use.  Possible values: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_RC4_128_SHA,TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_RC4_128_SHA,TLS_RSA_WITH_3DES_EDE_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA256,TLS_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_RC4_128_SHA</td>
+      <td></td><td style="line-height: 130%; word-wrap: break-word;">Comma-separated list of cipher suites for the server. If omitted, the default Go cipher suites will be used.  Possible values: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_RC4_128_SHA,TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_RC4_128_SHA,TLS_RSA_WITH_3DES_EDE_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA256,TLS_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_RC4_128_SHA</td>
    </tr>
    
     <tr>
@@ -16,5 +16,5 @@ tags:

 <!--more--> 

-Kubernetes v1.6 contains a new binary called cloud-controller-manager. cloud-controller-manager is a daemon that embeds cloud-specific control loops.  These cloud-specific control loops were originally in the kube-controller-manager. Since cloud providers develop and release at a different pace compared to the Kubernetes  project, abstracting the provider-specific code to the cloud-controller-manager binary allows cloud vendors to evolve independently from the core Kubernetes code.
+Kubernetes v1.6 contains a new binary called cloud-controller-manager. cloud-controller-manager is a daemon that embeds cloud-specific control loops. These cloud-specific control loops were originally in the kube-controller-manager. Since cloud providers develop and release at a different pace compared to the Kubernetes project, abstracting the provider-specific code to the cloud-controller-manager binary allows cloud vendors to evolve independently from the core Kubernetes code.

@@ -16,5 +16,7 @@ tags:

 Their primary responsibility is keeping a cluster up and running, which may involve periodic maintenance activities or upgrades.<br>

-**NOTE:** Cluster operators are different from the [Operator pattern](https://coreos.com/operators) that extends the Kubernetes API.
+{{< note >}}
+Cluster operators are different from the [Operator pattern](https://coreos.com/operators) that extends the Kubernetes API.
+{{< /note >}}

@@ -0,0 +1,18 @@
+---
+title: Container Lifecycle Hooks
+id: container-lifecycle-hooks
+date: 2018-10-08
+full_link: /docs/concepts/containers/container-lifecycle-hooks/
+short_description: >
+  The lifecycle hooks expose events in the container management lifecycle and let the user run code when the events occur.
+
+aka: 
+tags:
+- extension
+---
+  The lifecycle hooks expose events in the {{< glossary_tooltip text="Container" term_id="container" >}}container management lifecycle and let the user run code when the events occur.
+
+<!--more--> 
+
+Two hooks are exposed to Containers: PostStart which executes immediately after a container is created and PreStop which is blocking and is called immediately before a container is terminated.
+
@@ -110,8 +110,8 @@ metadata:
  name: mysecret
 type: Opaque
 data:
-  password: $(echo -n "s33msi4" | base64)
-  username: $(echo -n "jane" | base64)
+  password: $(echo -n "s33msi4" | base64 -w0)
+  username: $(echo -n "jane" | base64 -w0)
 EOF

 ```
@@ -163,12 +163,20 @@ kubectl get events --sort-by=.metadata.creationTimestamp

 ## Updating Resources

+As of version 1.11 `rolling-update` have been deprecated (see [CHANGELOG-1.11.md](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.11.md)), use `rollout` instead.
+
 ```bash
-kubectl rolling-update frontend-v1 -f frontend-v2.json           # Rolling update pods of frontend-v1
-kubectl rolling-update frontend-v1 frontend-v2 --image=image:v2  # Change the name of the resource and update the image
-kubectl rolling-update frontend --image=image:v2                 # Update the pods image of frontend
-kubectl rolling-update frontend-v1 frontend-v2 --rollback        # Abort existing rollout in progress
-cat pod.json | kubectl replace -f -                              # Replace a pod based on the JSON passed into stdin
+kubectl set image deployment/frontend www=image:v2               # Rolling update "www" containers of "frontend" deployment, updating the image
+kubectl rollout undo deployment/frontend                         # Rollback to the previous deployment
+kubectl rollout status -w deployment/frontend                    # Watch rolling update status of "frontend" deployment until completion
+
+# deprecated starting version 1.11
+kubectl rolling-update frontend-v1 -f frontend-v2.json           # (deprecated) Rolling update pods of frontend-v1
+kubectl rolling-update frontend-v1 frontend-v2 --image=image:v2  # (deprecated) Change the name of the resource and update the image
+kubectl rolling-update frontend --image=image:v2                 # (deprecated) Update the pods image of frontend
+kubectl rolling-update frontend-v1 frontend-v2 --rollback        # (deprecated) Abort existing rollout in progress
+
+cat pod.json | kubectl replace -f -                              # Replace a pod based on the JSON passed into std

 # Force replace, delete and then re-create the resource. Will cause a service outage.
 kubectl replace --force -f ./pod.json
@@ -3,17 +3,22 @@ title: kubectl Usage Conventions
 reviewers:
 - bgrant0607
 - janetkuo
+content_template: templates/concept
 ---

-{{< toc >}}
+{{% capture overview %}}
+Recommended usage conventions for `kubectl`.
+{{% /capture %}}
+
+{{% capture body %}}

 ## Using `kubectl` in Reusable Scripts

 For a stable output in a script:

-* Request one of the machine-oriented output forms, such as `-o name`, `-o json`, `-o yaml`, `-o go-template`, or `-o jsonpath`
+* Request one of the machine-oriented output forms, such as `-o name`, `-o json`, `-o yaml`, `-o go-template`, or `-o jsonpath`.
 * Fully-qualify the version. For example, `jobs.v1.batch/myjob`. This will ensure that kubectl does not use its default version that can change over time.
-* Specify `--generator` to pin to a specific behavior forever when you use generator-based commands such as `kubectl run` or `kubectl expose`
+* Specify the `--generator` flag to pin to a specific behavior when you use generator-based commands such as `kubectl run` or `kubectl expose`.
 * Don't rely on context, preferences, or other implicit states.

 ## Best Practices
@@ -30,15 +35,21 @@ For `kubectl run` to satisfy infrastructure as code:

 #### Generators

-You can generate the following resources in `kubectl run`using `--generator` flag:
+You can create the following resources using `kubectl run` with the `--generator` flag:

-* Pod - use `run-pod/v1`.
-* Replication controller - use `run/v1`.
-* Deployment - use `extensions/v1beta1` and for an endpoint - use `deployment/v1beta1` (default).
-* Deployment - use `apps/v1beta1` and for an endpoint - use `deployment/apps.v1beta1` (recommended).
-* Job - use `job/v1`.
-* CronJob - use `batch/v1beta1`and for an endpoint - use `cronjob/v1beta1`(default).
-* CronJob - use`batch/v2alpha1` and for an endpoint - use `cronjob/v2alpha1` (deprecated).
+| Resource                        | kubectl command                                   |
+|---------------------------------|---------------------------------------------------|
+| Pod                             | `kubectl run --generator=run-pod/v1`              |
+| Replication controller          | `kubectl run --generator=run/v1`                  |
+| Deployment                      | `kubectl run --generator=extensions/v1beta1`      |
+|  -for an endpoint (default)     | `kubectl run --generator=deployment/v1beta1`      |
+| Deployment                      | `kubectl run --generator=apps/v1beta1`            |
+|  -for an endpoint (recommended) | `kubectl run --generator=deployment/apps.v1beta1` |
+| Job                             | `kubectl run --generator=job/v1`                  |
+| CronJob                         | `kubectl run --generator=batch/v1beta1`           |
+|  -for an endpoint (default)     | `kubectl run --generator=cronjob/v1beta1`         |
+| CronJob                         | `kubectl run --generator=batch/v2alpha1`          |
+|  -for an endpoint (deprecated)  | `kubectl run --generator=cronjob/v2alpha1`        |

 If you do not specify a generator flag, other flags prompt you to use a specific generator. The following table lists the flags that force you to use specific generators, depending on the version of the cluster:

@@ -50,19 +61,22 @@ If you do not specify a generator flag, other flags prompt you to use a specific
 | Job                    | `--restart=OnFailure`  | `--restart=OnFailure` | `--restart=OnFailure` OR `--restart=Never` | N/A                                        |
 | Cron Job               | `--schedule=<cron>`    | N/A                   | N/A                                        | N/A                                        |

-**Note:** These flags use a default generator only when you have not specified
-any flag.  This means that when you combine `--generator` with other flags the generator that you specified later does not change. For example, in a cluster v1.4, if you initially specify
+{{< note >}}
+These flags use a default generator only when you have not specified any flag.
+This means that when you combine `--generator` with other flags the generator that you specified later does not change. For example, in a cluster v1.4, if you initially specify
 `--restart=Always`, a Deployment is created; if you later specify `--restart=Always`
 and `--generator=run/v1`, a Replication Controller is created.
 This enables you to pin to a specific behavior with the generator,
 even when the default generator is changed later.
+{{< /note >}}

-The flags set the generator in the following order: first the schedule flag then restart policy and finally the generator.
+The flags set the generator in the following order: first the `--schedule` flag, then the `--restart` policy flag, and finally the `--generator` flag.

-To check the final resource that was created, use `--dry-run`
+To check the final resource that was created, use the `--dry-run`
 flag, which provides the object to be submitted to the cluster.

-
 ### `kubectl apply`

 * You can use `kubectl apply` to create or update resources. However, to update a resource you should have created the resource by using `kubectl apply` or `kubectl create --save-config`. For more information about using kubectl apply to update resources, see [Managing Resources](/docs/concepts/cluster-administration/manage-deployment/#kubectl-apply).
+
+{{% /capture %}}
@@ -36,7 +36,7 @@ deployment "nginx-app" created
 ```

 {{< note >}}
-**Note:** `kubectl` commands print the type and name of the resource created or mutated, which can then be used in subsequent commands. You can expose a new Service after a Deployment is created.
+`kubectl` commands print the type and name of the resource created or mutated, which can then be used in subsequent commands. You can expose a new Service after a Deployment is created.
 {{< /note >}}

 ```shell
@@ -75,7 +75,7 @@ CONTAINER ID        IMAGE               COMMAND                  CREATED
 kubectl:

 ```shell
-$ kubectl get po -a
+$ kubectl get po
 NAME                        READY     STATUS      RESTARTS   AGE
 nginx-app-8df569cb7-4gd89   1/1       Running     0          3m
 ubuntu                      0/1       Completed   0          20s
@@ -222,7 +222,7 @@ $ kubectl get po -l run=nginx-app
 ```

 {{< note >}}
-**Note:** When you use kubectl, you don't delete the pod directly.You have to first delete the Deployment that owns the pod. If you delete the pod directly, the Deployment recreates the pod.
+When you use kubectl, you don't delete the pod directly.You have to first delete the Deployment that owns the pod. If you delete the pod directly, the Deployment recreates the pod.
 {{< /note >}}

 ## docker login
@@ -294,4 +294,4 @@ Grafana is running at https://108.59.85.141/api/v1/namespaces/kube-system/servic
 Heapster is running at https://108.59.85.141/api/v1/namespaces/kube-system/services/monitoring-heapster/proxy
 InfluxDB is running at https://108.59.85.141/api/v1/namespaces/kube-system/services/monitoring-influxdb/proxy
 ```
-{{% /capture %}}
+{{% /capture %}}
@@ -1,18 +1,32 @@
 ---
 title: JSONPath Support
+content_template: templates/concept
+weight: 25
 ---

-JSONPath template is composed of JSONPath expressions enclosed by {}.
-And we add three functions in addition to the original JSONPath syntax:
+{{% capture overview %}}
+Kubectl supports JSONPath template.
+{{% /capture %}}

-1. The `$` operator is optional since the expression always starts from the root object by default.
-2. We can use `""` to quote text inside JSONPath expressions.
-3. We can use `range` operator to iterate lists.
-4. We can use negative slice indices to step backwards through a list. Negative indices do not "wrap around" a list. They are valid as long as `-index + listLength >= 0`.
+{{% capture body %}}

-The result object is printed as its String() function.
+JSONPath template is composed of JSONPath expressions enclosed by curly braces {}.
+Kubectl uses JSONPath expressions to filter on specific fields in the JSON object and format the output.
+In addition to the original JSONPath template syntax, the following functions and syntax are valid:

-Given the input:
+1. Use double quotes to quote text inside JSONPath expressions.
+2. Use the `range`, `end` operators to iterate lists.
+3. Use negative slice indices to step backwards through a list. Negative indices do not "wrap around" a list and are valid as long as `-index + listLength >= 0`.
+
+{{< note >}}
+
+- The `$` operator is optional since the expression always starts from the root object by default.
+
+- The result object is printed as its String() function.
+
+{{< /note >}}
+
+Given the JSON input:

 ```json
 {
@@ -51,20 +65,20 @@ Given the input:
 }
 ```

-Function | Description        | Example            | Result
---------|--------------------|--------------------|------------------
-text     | the plain text     | kind is {.kind}    | kind is List
-@        | the current object | {@}                | the same as input
-. or []  | child operator     | {.kind} or {['kind']}| List
-..       | recursive descent  | {..name}           | 127.0.0.1 127.0.0.2 myself e2e
-\*        | wildcard. Get all objects| {.items[*].metadata.name} | [127.0.0.1 127.0.0.2]
-[start:end :step] | subscript operator | {.users[0].name}| myself
-[,]      | union operator     | {.items[*]['metadata.name', 'status.capacity']} | 127.0.0.1 127.0.0.2 map[cpu:4] map[cpu:8]
-?()      | filter             | {.users[?(@.name=="e2e")].user.password} | secret
-range, end | iterate list | {range .items[*]}[{.metadata.name}, {.status.capacity}] {end} | [127.0.0.1, map[cpu:4]] [127.0.0.2, map[cpu:8]]
-''       | quote interpreted string | {range .items[*]}{.metadata.name}{'\t'}{end} | 127.0.0.1    127.0.0.2
+Function          | Description               | Example                                                       | Result
+------------------|---------------------------|---------------------------------------------------------------|------------------
+text              | the plain text            | kind is {.kind}                                               | kind is List
+@                 | the current object        | {@}                                                           | the same as input
+. or []           | child operator            | {.kind} or {['kind']}                                         | List
+..                | recursive descent         | {..name}                                                      | 127.0.0.1 127.0.0.2 myself e2e
+\*                | wildcard. Get all objects | {.items[*].metadata.name}                                     | [127.0.0.1 127.0.0.2]
+[start:end :step] | subscript operator        | {.users[0].name}                                              | myself
+[,]               | union operator            | {.items[*]['metadata.name', 'status.capacity']}               | 127.0.0.1 127.0.0.2 map[cpu:4] map[cpu:8]
+?()               | filter                    | {.users[?(@.name=="e2e")].user.password}                      | secret
+range, end        | iterate list              | {range .items[*]}[{.metadata.name}, {.status.capacity}] {end} | [127.0.0.1, map[cpu:4]] [127.0.0.2, map[cpu:8]]
+''                | quote interpreted string  | {range .items[*]}{.metadata.name}{'\t'}{end}                  | 127.0.0.1    127.0.0.2

-Below are some examples using jsonpath:
+Examples using `kubectl` and JSONPath expressions:

 ```shell
 $ kubectl get pods -o json
@@ -80,3 +94,5 @@ On Windows, you must _double_ quote any JSONPath template that contains spaces (
 C:\> kubectl get pods -o=jsonpath="{range .items[*]}{.metadata.name}{'\t'}{.status.startTime}{'\n'}{end}"
 C:\> kubectl get pods -o=jsonpath="{range .items[*]}{.metadata.name}{\"\t\"}{.status.startTime}{\"\n\"}{end}"
 ```
+
+{{% /capture %}}
@@ -3,9 +3,17 @@ reviewers:
 - bgrant0607
 - hw-qiaolei
 title: Overview of kubectl
+content_template: templates/concept
+weight: 20
 ---

-`kubectl` is a command line interface for running commands against Kubernetes clusters. This overview covers `kubectl` syntax, describes the command operations, and provides common examples. For details about each command, including all the supported flags and subcommands, see the [kubectl](/docs/reference/generated/kubectl/kubectl-commands/) reference documentation. For installation instructions see [installing kubectl](/docs/tasks/kubectl/install/).
+{{% capture overview %}}
+Kubectl is a command line interface for running commands against Kubernetes clusters.
+This overview covers `kubectl` syntax, describes the command operations, and provides common examples. For details about each command, including all the supported flags and subcommands, see the [kubectl](/docs/reference/generated/kubectl/kubectl-commands/) reference documentation. For installation instructions see [installing kubectl](/docs/tasks/kubectl/install/).
+
+{{% /capture %}}
+
+{{% capture body %}}

 ## Syntax

@@ -45,7 +53,10 @@ where `command`, `TYPE`, `NAME`, and `flags` are:
     Example: `$ kubectl get pod -f ./pod.yaml`

 * `flags`: Specifies optional flags. For example, you can use the `-s` or `--server` flags to specify the address and port of the Kubernetes API server.<br/>
-**Important**: Flags that you specify from the command line override default values and any corresponding environment variables.
+
+{{< caution >}}
+Flags that you specify from the command line override default values and any corresponding environment variables.
+{{< /caution >}}

 If you need help, just run `kubectl help` from the terminal window.

@@ -66,7 +77,7 @@ Operation       | Syntax    |       Description
 `delete`        | `kubectl delete (-f FILENAME \| TYPE [NAME \| /NAME \| -l label \| --all]) [flags]` | Delete resources either from a file, stdin, or specifying label selectors, names, resource selectors, or resources.
 `describe`    | `kubectl describe (-f FILENAME \| TYPE [NAME_PREFIX \| /NAME \| -l label]) [flags]` | Display the detailed state of one or more resources.
 `edit`        | `kubectl edit (-f FILENAME \| TYPE NAME \| TYPE/NAME) [flags]` | Edit and update the definition of one or more resources on the server by using the default editor.
-`exec`        | `kubectl exec POD [-c CONTAINER] [-i] [-t] [flags] [-- COMMAND [args...]]` | Execute a command against a container in a pod,
+`exec`        | `kubectl exec POD [-c CONTAINER] [-i] [-t] [flags] [-- COMMAND [args...]]` | Execute a command against a container in a pod.
 `explain`    | `kubectl explain [--include-extended-apis=true] [--recursive=false] [flags]` | Get documentation of various resources. For instance pods, nodes, services, etc.
 `expose`        | `kubectl expose (-f FILENAME \| TYPE NAME \| TYPE/NAME) [--port=port] [--protocol=TCP\|UDP] [--target-port=number-or-name] [--name=name] [--external-ip=external-ip-of-service] [--type=type] [flags]` | Expose a replication controller, service, or pod as a new Kubernetes service.
 `get`        | `kubectl get (-f FILENAME \| TYPE [NAME \| /NAME \| -l label]) [--watch] [--sort-by=FIELD] [[-o \| --output]=OUTPUT_FORMAT] [flags]` | List one or more resources.
@@ -135,7 +146,7 @@ Use the following sections for information about how you can format or sort the

 ### Formatting output

-The default output format for all `kubectl` commands is the human readable plain-text format. To output details to your terminal window in a specific format, you can add either the `-o` or `-output` flags to a supported `kubectl` command.
+The default output format for all `kubectl` commands is the human readable plain-text format. To output details to your terminal window in a specific format, you can add either the `-o` or `--output` flags to a supported `kubectl` command.

 #### Syntax

@@ -262,7 +273,7 @@ $ kubectl create -f <directory>
 // List all pods in plain-text output format.
 $ kubectl get pods

-// List all pods in plain-text output format and includes additional information (such as node name).
+// List all pods in plain-text output format and include additional information (such as node name).
 $ kubectl get pods -o wide

 // List the replication controller with the specified name in plain-text output format. Tip: You can shorten and replace the 'replicationcontroller' resource type with the alias 'rc'.
@@ -299,7 +310,7 @@ $ kubectl describe pods --include-uninitialized=false
 ```

 {{< note >}}
-**Note:** The `kubectl get` command is usually used for retrieving one or more
+The `kubectl get` command is usually used for retrieving one or more
 resources of the same resource type. It features a rich set of flags that allows
 you to customize the output format using the `-o` or `--output` flag, for example.
 You can specify the `-w` or `--watch` flag to start watching updates to a particular
@@ -431,6 +442,10 @@ Current user: plugins-user

 To find out more about plugins, take a look at the [example cli plugin](https://github.com/kubernetes/sample-cli-plugin).

-## Next steps
+{{% /capture %}}
+
+{{% capture whatsnext %}}

 Start using the [kubectl](/docs/reference/generated/kubectl/kubectl-commands/) commands.
+
+{{% /capture %}}
@@ -86,7 +86,7 @@ In any case the user can skip specific preflight checks (or eventually all prefl
    - [error] if not Kernel 3.10+ or 4+ with specific KernelSpec
    - [error] if required cgroups subsystem aren't in set up
  - if using docker:
-    - [warning/error] if Docker service does not  exist, if it is disabled, if it is not active.
+    - [warning/error] if Docker service does not exist, if it is disabled, if it is not active.
    - [error] if Docker endpoint does not exist or does not work
    - [warning] if docker version >17.03
  - If using other cri engine:
@@ -106,9 +106,9 @@ In any case the user can skip specific preflight checks (or eventually all prefl
 - [Error] if `ip`, `iptables`,  `mount`, `nsenter` commands are not present in the command path
 - [warning] if `ebtables`, `ethtool`, `socat`, `tc`, `touch`, `crictl` commands are not present in the command path
 - [warning] if extra arg flags for API server, controller manager,  scheduler contains some invalid options
- [warning] if connection to https://API.AdvertiseAddress:API.BindPort goes thought proxy
- [warning] if connection to services subnet goes thought proxy (only first address checked)
- [warning] if connection to Pods subnet goes thought proxy (only first address checked)  
+- [warning] if connection to https://API.AdvertiseAddress:API.BindPort goes through proxy
+- [warning] if connection to services subnet goes through proxy (only first address checked)
+- [warning] if connection to Pods subnet goes through proxy (only first address checked)  
 - If external etcd is provided:
  - [Error] if etcd version less than 3.0.14
  - [Error] if etcd certificates or keys are specified, but not provided
@@ -200,7 +200,7 @@ Static Pod manifest share a set of common properties:
  of using Pod Priority and Preemption when ready)
 - `hostNetwork: true` is set on all static Pods to allow control plane startup before a network is configured; as a consequence:
  * The `address` that the controller-manager and the scheduler use to refer the API server is `127.0.0.1`
-  * If using a local etcd server, `etcd-servers` address  will be set to `127.0.0.1:2379`
+  * If using a local etcd server, `etcd-servers` address will be set to `127.0.0.1:2379`
 - Leader election is enabled for both the controller-manager and the scheduler
 - Controller-manager and the scheduler will reference kubeconfig files with their respective, unique identities
 - All static Pods gets any extra flags specified by the user as described in [passing custom arguments to control plane components](/docs/reference/setup-tools/kubeadm/kubeadm-init/#custom-args)
@@ -224,7 +224,7 @@ The static Pod manifest for the API server is affected by following parameters p
 - The `service-cluster-ip-range` to use for services
 - If an external etcd server is specified, the `etcd-servers` address and related TLS settings (`etcd-cafile`, `etcd-certfile`, `etcd-keyfile`);
   if an external etcd server is not be provided, a local etcd will be used (via host network)
- - If a cloud provider is specified, the corresponding `--cloud-provider` is configured, together with  the  `--cloud-config` path
+ - If a cloud provider is specified, the corresponding `--cloud-provider` is configured, together with the  `--cloud-config` path
   if such file exists (this is experimental, alpha and will be removed in a future version)
 - If kubeadm is invoked with `--feature-gates=HighAvailability`, the flag `--endpoint-reconciler-type=lease` is set, thus enabling
   automatic reconciliation of endpoints for the internal API server VIP
@@ -277,7 +277,7 @@ The static Pod manifest for the API server is affected by following parameters p
   setting:
   - `--allocate-node-cidrs=true`
   - `--cluster-cidr` and `--node-cidr-mask-size` flags according to the given CIDR
- - If a cloud provider is specified, the corresponding `--cloud-provider` is specified, together with  the  `--cloud-config` path
+ - If a cloud provider is specified, the corresponding `--cloud-provider` is specified, together with the  `--cloud-config` path
   if such configuration file exists (this is experimental, alpha and will be removed in a future version)

 Other flags that are set unconditionally are:
@@ -318,7 +318,7 @@ into `/var/lib/kubelet/config/init/kubelet` file.

 The init configuration is used for starting the kubelet on this specific node, providing an alternative for the kubelet drop-in file;
 such configuration will be replaced by the kubelet base configuration as described in following steps.
-See [set Kubelet parameters via a config file](/docs/tasks/administer-cluster/kubelet-config-file.md) for additional info.
+See [set Kubelet parameters via a config file](/docs/tasks/administer-cluster/kubelet-config-file) for additional info.

 Please note that:

@@ -145,7 +145,7 @@ You can install all the available addons with the `all` subcommand, or
 install them selectively.

 {{< note >}}
-**Note:** If `kubeadm` is invoked with `--feature-gates=CoreDNS=false`, kube-dns is installed.
+If `kubeadm` is invoked with `--feature-gates=CoreDNS=false`, kube-dns is installed.
 {{< /note >}}

 {{< tabs name="tab-addon" >}}
@@ -158,7 +158,7 @@ install them selectively.
 ## kubeadm alpha phase self-hosting {#cmd-phase-self-hosting}

 {{< caution >}}
-**Caution:** Self-hosting is an alpha feature. See [kubeadm init](/docs/reference/setup-tools/kubeadm/kubeadm-init/) documentation for self-hosting limitations.
+Self-hosting is an alpha feature. See [kubeadm init](/docs/reference/setup-tools/kubeadm/kubeadm-init/) documentation for self-hosting limitations.
 {{< /caution >}}

 {{< tabs name="tab-self-hosting" >}}
@@ -92,8 +92,7 @@ following steps:
 ### Using kubeadm init with a configuration file {#config-file}

 {{< caution >}}
-**Caution:** The config file is
-still considered alpha and may change in future versions.
+The config file is still considered alpha and may change in future versions.
 {{< /caution >}}

 It's possible to configure `kubeadm init` with a configuration file instead of command
@@ -284,14 +283,10 @@ configured in the kubelet via static files.
 To create a self-hosted cluster, pass the flag `--feature-gates=SelfHosting=true` to `kubeadm init`.

 {{< caution >}}
-**Caution:** `SelfHosting` is an alpha feature. It is deprecated in 1.12
+`SelfHosting` is an alpha feature. It is deprecated in 1.12
 and will be removed in 1.13.
 {{< /caution >}}

-{{< warning >}}
-**Warning:** See self-hosted caveats and limitations.
-{{< /warning >}}
-
 #### Caveats

 Self-hosting in 1.8 has some important limitations. In particular, a
@@ -309,7 +304,7 @@ authorization configuration for your cluster, and may not be appropriate for
 your environment.

 {{< caution >}}
-**Caution:** `StoreCertsInSecrets` is an alpha feature. It is deprecated in 1.12
+`StoreCertsInSecrets` is an alpha feature. It is deprecated in 1.12
 and will be removed in 1.13.
 {{< /caution >}}

@@ -204,8 +204,7 @@ These commands should be run after `kubeadm init` but before `kubeadm join`.
 ### Using kubeadm join with a configuration file {#config-file}

 {{< caution >}}
-**Caution:** The config file is
-still considered alpha and may change in future versions.
+The config file is still considered alpha and may change in future versions.
 {{< /caution >}}

 It's possible to configure `kubeadm join` with a configuration file instead of command
@@ -40,7 +40,7 @@ The version is set at the API level rather than at the resource or field level t
 The JSON and Protobuf serialization schemas follow the same guidelines for schema changes. The following descriptions cover both formats.

 {{< note >}}
-**Note:** The API versioning and software versioning are indirectly related.  The [API and release
+The API versioning and software versioning are indirectly related.  The [API and release
 versioning proposal](https://git.k8s.io/community/contributors/design-proposals/release/versioning.md) describes the relationship between API versioning and software versioning.
 {{< /note >}}

@@ -64,7 +64,7 @@ Here's a summary of each level:
  - The software is recommended for only non-business-critical uses because of potential for incompatible changes in subsequent releases. If you have multiple clusters which can be upgraded independently, you may be able to relax this restriction.
  
 {{< note >}}
-**Note:** Try the beta features and provide feedback. After the features exit beta, it may not be practical to make more changes.
+Try the beta features and provide feedback. After the features exit beta, it may not be practical to make more changes.
 {{< /note >}}
  
 - Stable:
@@ -97,7 +97,7 @@ on the apiserver. `--runtime-config` accepts comma separated values. For example
 The flag accepts comma separated set of key=value pairs describing runtime configuration of the apiserver.

 {{< note >}}
-**Note:** When you enable or disable groups or resources, you need to restart the apiserver and controller-manager
+When you enable or disable groups or resources, you need to restart the apiserver and controller-manager
 to pick up the `--runtime-config` changes.
 {{< /note >}}

@@ -57,7 +57,7 @@ can not be removed from that version or have its behavior significantly
 changed, regardless of track.

 {{< note >}}
-**Note:** For historical reasons, there are 2 "monolithic" API groups - "core" (no
+For historical reasons, there are 2 "monolithic" API groups - "core" (no
 group name) and "extensions".  Resources will incrementally be moved from these
 legacy API groups into more domain-specific API groups.
 {{< /note >}}
@@ -90,7 +90,7 @@ no less than:**
 This covers the maximum supported version skew of 2 releases.

 {{< note >}}
-**Note:** Until [#52185](https://github.com/kubernetes/kubernetes/issues/52185) is
+Until [#52185](https://github.com/kubernetes/kubernetes/issues/52185) is
 resolved, no API versions that have been persisted to storage may be removed. 
 Serving REST endpoints for those versions may be disabled (subject to the 
 deprecation timelines in this document), but the API server must remain capable 
@@ -365,6 +365,56 @@ This applies only to significant, user-visible behaviors which impact the
 correctness of applications running on Kubernetes or that impact the
 administration of Kubernetes clusters, and which are being removed entirely.

+An exception to the above rule is _feature gates_. Feature gates are key=value 
+pairs that allow for users to enable/disable experimental features.
+
+Feature gates are intended to cover the development life cycle of a feature - they 
+are not intended to be long-term APIs. As such, they are expected to be deprecated 
+and removed after a feature becomes GA or is dropped. 
+
+As a feature moves through the stages, the associated feature gate evolves. 
+The feature life cycle matched to its corresponding feature gate is:
+
+  * Alpha: the feature gate is disabled by default and can be enabled by the user.
+  * Beta: the feature gate is enabled by default and can be disabled by the user.
+  * GA: the feature gate is deprecated (see ["Deprecation"](#deprecation)) and becomes 
+  non-operational.
+  * GA, deprecation window complete: the feature gate is removed and calls to it are 
+  no longer accepted.
+
+### Deprecation
+
+Features can be removed at any point in the life cycle prior to GA. When features are 
+removed prior to GA, their associated feature gates are also deprecated.
+
+When an invocation tries to disable a non-operational feature gate, the call fails in order 
+to avoid unsupported scenarios that might otherwise run silently.
+
+In some cases, removing pre-GA features requires considerable time. Feature gates can remain 
+operational until their associated feature is fully removed, at which point the feature gate 
+itself can be deprecated. 
+
+When removing a feature gate for a GA feature also requires considerable time, calls to 
+feature gates may remain operational if the feature gate has no effect on the feature, 
+and if the feature gate causes no errors.
+
+Features intended to be disabled by users should include a mechanism for disabling the 
+feature in the associated feature gate.
+
+Versioning for feature gates is different from the previously discussed components,
+therefore the rules for deprecation are as follows:
+
+**Rule #8: Feature gates must be deprecated when the corresponding feature they control 
+transitions a lifecycle stage as follows. Feature gates must function for no less than:**
+
+   * **Beta feature to GA: 6 months or 2 releases (whichever is longer)**
+   * **Beta feature to EOL: 3 months or 1 release (whichever is longer)**
+   * **Alpha feature to EOL: 0 releases**
+
+**Rule #9: Deprecated feature gates must respond with a warning when used. When a feature gate 
+is deprecated it must be documented in both in the release notes and the corresponding CLI help. 
+Both warnings and documentation must indicate whether a feature gate is non-operational.**
+
 ## Exceptions

 No policy can cover every possible situation.  This policy is a living