Merge pull request #4208 from ericchiang/tls-boostrapping-17

Update TLS bootstrapping with 1.7 features

docs/admin/kubelet-tls-bootstrapping.md

@@ -1,6 +1,8 @@
 ---
 assignees:
+- ericchiang
 - mikedanese
+- jcbsmpsn
 title: TLS bootstrapping
 ---
 
@@ -59,18 +61,114 @@ The kube-controller-manager flags are:
 --cluster-signing-cert-file="/etc/path/to/kubernetes/ca/ca.crt" --cluster-signing-key-file="/etc/path/to/kubernetes/ca/ca.key"
 ```
 
-### Automatic approval
+### Approval controller
-To ease deployment and testing, there is an experimental flag in the certificate bootstrapping API to approve all certificate
-requests made by users in a certain group. The intended use of this is to whitelist only the group corresponding to the bootstrap
-token in the token file above. Use of this flag circumvents the approval process described below and is not recommended
-for production use.
 
-The flag is:
+In 1.7 the experimental "group auto approver" controller is dropped in favor of the new `csrapproving` controller
+that ships as part of [kube-controller-manager](/docs/admin/kube-controller-manager/) and is enabled by default.
+The controller uses the [`SubjectAccessReview` API](/docs/admin/authorization/#checking-api-access) to determine
+if a given user is authorized to request a CSR, then approves based on the authorization outcome. To prevent
+conflicts with other approvers, the builtin approver doesn't explicitly deny CSRs, only ignoring unauthorized requests.
+
+The controller categorizes CSRs into three subresources:
+
+1. `nodeclient` - a request by a user for a client certificate with `O=system:nodes` and `CN=system:node:(node name)`.
+2. `selfnodeclient` - a node renewing a client certificate with the same `O` and `CN`.
+3. `selfnodeserver` - a node renewing a serving certificate. (ALPHA, requires feature gate)
+
+The checks to determine if a CSR is a `selfnodeserver` request is currently tied to the kubelet's credential rotation
+implementation, an __alpha__ feature. As such, the definition of `selfnodeserver` will likely change in a future and
+requires the `RotateKubeletServerCertificate` feature gate on the controller manager. The feature progress can be
+tracked at [kubernetes/features#267](https://github.com/kubernetes/features/issues/267).
 
 ```
---insecure-experimental-approve-all-kubelet-csrs-for-group="system:kubelet-bootstrap"
+--feature-gates=RotateKubeletServerCertificate=true
 ```
 
+The following RBAC `ClusterRoles` represent the `nodeclient`, `selfnodeclient`, and `selfnodeserver` capabilities. Similar roles
+may be automatically created in future releases.
+
+```yml
+# A ClusterRole which instructs the CSR approver to approve a user requesting
+# node client credentials.
+kind: ClusterRole
+apiVersion: rbac.authorization.k8s.io/v1beta1
+metadata:
+  name: approve-node-client-csr
+rules:
+- apiGroups: ["certificates.k8s.io"]
+  resources: ["certificatesigningrequests/nodeclient"]
+  verbs: ["create"]
+---
+# A ClusterRole which instructs the CSR approver to approve a node renewing its
+# own client credentials.
+kind: ClusterRole
+apiVersion: rbac.authorization.k8s.io/v1beta1
+metadata:
+  name: approve-node-client-renewal-csr
+rules:
+- apiGroups: ["certificates.k8s.io"]
+  resources: ["certificatesigningrequests/selfnodeclient"]
+  verbs: ["create"]
+---
+# A ClusterRole which instructs the CSR approver to approve a node requesting a
+# serving cert matching its client cert.
+kind: ClusterRole
+apiVersion: rbac.authorization.k8s.io/v1beta1
+metadata:
+  name: approve-node-server-renewal-csr
+rules:
+- apiGroups: ["certificates.k8s.io"]
+  resources: ["certificatesigningrequests/selfnodeserver"]
+  verbs: ["create"]
+```
+
+These powers can be granted to credentials, such as bootstrapping tokens. For example, to replicate the behavior
+provided by the removed auto-approval flag, of approving all CSRs by a single group:
+
+```
+# REMOVED: This flag no longer works as of 1.7.
+--insecure-experimental-approve-all-kubelet-csrs-for-group="kubelet-bootstrap-token"
+```
+
+An admin would create a `ClusterRoleBinding` targeting that group.
+
+```yml
+# Approve all CSRs for the group "kubelet-bootstrap-token"
+kind: ClusterRoleBinding
+apiVersion: rbac.authorization.k8s.io/v1beta1
+metadata:
+  name: auto-approve-csrs-for-group
+subjects:
+- kind: Group
+  name: kubelet-bootstrap-token
+  apiGroup: rbac.authorization.k8s.io
+roleRef:
+  kind: ClusterRole
+  name: approve-node-client-csr
+  apiGroup: rbac.authorization.k8s.io
+```
+
+To let a node renew its own credentials, an admin can construct a `ClusterRoleBinding` targeting
+that node's credentials:
+
+```yml
+kind: ClusterRoleBinding
+apiVersion: rbac.authorization.k8s.io/v1beta1
+metadata:
+  name: node1-client-cert-renewal
+subjects:
+- kind: User
+  name: system:node:node-1 # Let "node-1" renew its client certificate.
+  apiGroup: rbac.authorization.k8s.io
+roleRef:
+  kind: ClusterRole
+  name: approve-node-client-renewal-csr
+  apiGroup: rbac.authorization.k8s.io
+```
+
+Deleting the binding will prevent the node from renewing its client credentials, effectively
+removing it from the cluster once its certificate expires.
+
 ## kubelet configuration
 To request a client certificate from kube-apiserver, the kubelet first needs a path to a kubeconfig file that contains the
 bootstrap authentication token. You can use `kubectl config set-cluster`, `set-credentials`, and `set-context` to build this kubeconfig. Provide the name `kubelet-bootstrap` to `kubectl config set-credentials` and include `--token=<token-value>` as follows:
@@ -87,6 +185,19 @@ The flag to enable this bootstrapping when starting the kubelet is:
 --experimental-bootstrap-kubeconfig="/path/to/bootstrap/kubeconfig"
 ```
 
+Additionally, in 1.7 the kubelet implements __alpha__ features for enabling rotation of both its client and/or serving certs.
+These can be enabled through the respective `RotateKubeletClientCertificate` and `RotateKubeletServerCertificate` feature
+flags on the kubelet, but may change in backward incompatible ways in future releases.
+
+```
+--feature-gates=RotateKubeletClientCertificate=true,RotateKubeletServerCertificate=true
+```
+
+`RotateKubeletClientCertificate` causes the kubelet to rotate its client certificates by creating new CSRs as its existing
+credentials expire. `RotateKubeletServerCertificate` causes the kubelet to both request a serving certificate after
+bootstrapping its client credentials and rotate the certificate. The serving cert currently does not request DNS or IP
+SANs.
+
 ## kubectl approval
 The signing controller does not immediately sign all certificate requests. Instead, it waits until they have been flagged with an
 "Approved" status by an appropriately-privileged user. This is intended to eventually be an automated process handled by an external