Merge pull request #1684 from liggitt/kubelet-authnz

Update kubelet authentication/authorization docs
2016-12-04 11:58:09 -08:00
parent fdd0ff5d84 2940a24692
commit 8b0b0c82fd
7 changed files with 232 additions and 98 deletions
@@ -178,7 +178,15 @@ toc:
  - title: kube-scheduler
    path: /docs/admin/kube-scheduler/
  - title: kubelet
-    path: /docs/admin/kubelet/
+    section:
    - title: Overview 
      path: /docs/admin/kubelet/
    - title: Master-Node communication
      path: /docs/admin/master-node-communication/
    - title: TLS bootstrapping
      path: /docs/admin/kubelet-tls-bootstrapping/
    - title: Kubelet authentication/authorization
      path: /docs/admin/kubelet-authentication-authorization/ 
 - title: Glossary
  section:
@@ -4,6 +4,7 @@ assignees:
 - lavalamp
 - ericchiang
 - deads2k
 - liggitt
 ---
 * TOC
@@ -2,6 +2,8 @@
 assignees:
 - erictune
 - lavalamp
 - deads2k
 - liggitt
 ---
@@ -565,10 +567,10 @@ Access to non-resource paths are sent as:
 Non-resource paths include: `/api`, `/apis`, `/metrics`, `/resetMetrics`,
 `/logs`, `/debug`, `/healthz`, `/swagger-ui/`, `/swaggerapi/`, `/ui`, and
-`/version.` Clients require access to `/api`, `/api/*/`, `/apis/`, `/apis/*`,
+`/version.` Clients require access to `/api`, `/api/*`, `/apis`, `/apis/*`,
-`/apis/*/*`, and `/version` to discover what resources and versions are present
+and `/version` to discover what resources and versions are present on the server.
-on the server. Access to other non-resource paths can be disallowed without
+Access to other non-resource paths can be disallowed without restricting access
-restricting access to the REST api.
+to the REST api.
 For further documentation refer to the authorization.v1beta1 API objects and
 plugin/pkg/auth/authorizer/webhook/webhook.go.
@@ -84,3 +84,8 @@ project](/docs/admin/salt).
 * **Sysctls** [sysctls](/docs/admin/sysctls.md)
 * **Audit** [audit](/docs/admin/audit)
 * **Securing the kubelet**
  * [Master-Node communication](/docs/admin/master-node-communication/)
  * [TLS bootstrapping](/docs/admin/kubelet-tls-bootstrapping/)
  * [Kubelet authentication/authorization](/docs/admin/kubelet-authentication-authorization/)
@@ -0,0 +1,81 @@
 ---
 assignees:
 - liggitt
 ---
 * TOC
 {:toc}
 ## Overview
 A kubelet's HTTPS endpoint exposes APIs which give access to data of varying sensitivity,
 and allow you to perform operations with varying levels of power on the node and within containers.
 This document describes how to authenticate and authorize access to the kubelet's HTTPS endpoint.
 ## Kubelet authentication
 By default, requests to the kubelet's HTTPS endpoint that are not rejected by other configured
 authentication methods are treated as anonymous requests, and given a username of `system:anonymous` 
 and a group of `system:unauthenticated`.
 To disable anonymous access and send `401 Unauthorized` responses to unauthenticated requests:
 * start the kubelet with the `--anonymous-auth=false` flag
 To enable X509 client certificate authentication to the kubelet's HTTPS endpoint:
 * start the kubelet with the `--client-ca-file` flag, providing a CA bundle to verify client certificates with 
 * start the apiserver with `--kubelet-client-certificate` and `--kubelet-client-key` flags
 * see the [apiserver authentication documentation](/docs/admin/authentication/#x509-client-certs) for more details
 To enable API bearer tokens (including service account tokens) to be used to authenticate to the kubelet's HTTPS endpoint:
 * ensure the `authentication.k8s.io/v1beta1` API group is enabled in the API server
 * start the kubelet with the `--authentication-token-webhook`, `--kubeconfig`, and `--require-kubeconfig` flags
 * the kubelet calls the `TokenReview` API on the configured API server to determine user information from bearer tokens 
 ## Kubelet authorization
 Any request that is successfully authenticated (including an anonymous request) is then authorized. The default authorization mode is `AlwaysAllow`, which allows all requests.
 There are many possible reasons to subdivide access to the kubelet API:
 * anonymous auth is enabled, but anonymous users' ability to call the kubelet API should be limited
 * bearer token auth is enabled, but arbitrary API users' (like service accounts) ability to call the kubelet API should be limited
 * client certificate auth is enabled, but only some of the client certificates signed by the configured CA should be allowed to use the kubelet API
 To subdivide access to the kubelet API, delegate authorization to the API server:
 * ensure the `authorization.k8s.io/v1beta1` API group is enabled in the API server
 * start the kubelet with the `--authorization-mode=Webhook`, `--kubeconfig`, and `--require-kubeconfig` flags
 * the kubelet calls the `SubjectAccessReview` API on the configured API server to determine whether each request is authorized 
 The kubelet authorizes API requests using the same [request attributes](/docs/admin/authorization/#request-attributes) approach as the apiserver.
 The verb is determined from the incoming request's HTTP verb:
 HTTP verb | request verb
 ----------|---------------
 POST      | create
 GET, HEAD | get
 PUT       | update
 PATCH     | patch
 DELETE    | delete
 The resource and subresource is determined from the incoming request's path:
 Kubelet API  | resource | subresource
 -------------|----------|------------
 /stats/*     | nodes    | stats
 /metrics/*   | nodes    | metrics
 /logs/*      | nodes    | log
 /spec/*      | nodes    | spec
 *all others* | nodes    | proxy
 The namespace and API group attributes are always an empty string, and 
 the resource name is always the name of the kubelet's `Node` API object.
 When running in this mode, ensure the user identified by the `--kubelet-client-certificate` and `--kubelet-client-key` 
 flags passed to the apiserver is authorized for the following attributes:
 * verb=*, resource=nodes, subresource=proxy
 * verb=*, resource=nodes, subresource=stats
 * verb=*, resource=nodes, subresource=log
 * verb=*, resource=nodes, subresource=spec
 * verb=*, resource=nodes, subresource=metrics 
@@ -0,0 +1,96 @@
 ---
 assignees:
 - mikedanese
 ---
 * TOC
 {:toc}
 ## Overview
 This document describes how to set up TLS client certificate boostrapping for kubelets.
 Kubernetes 1.4 introduces an experimental API for requesting certificates from a cluster-level 
 Certificate Authority (CA). The first supported use of this API is the provisioning of TLS client 
 certificates for kubelets. The proposal can be found [here](https://github.com/kubernetes/kubernetes/pull/20439)
 and progress on the feature is being tracked as [feature #43](https://github.com/kubernetes/features/issues/43). 
 ## apiserver configuration
 You must provide a token file which specifies at least one "bootstrap token" assigned to a kubelet boostrap-specific group.
 This group will later be used in the controller-manager configuration to scope approvals in the default approval
 controller. As this feature matures, you should ensure tokens are bound to an RBAC policy which limits requests
 using the bootstrap token to only be able to make requests related to certificate provisioning. When RBAC policy
 is in place, scoping the tokens to a group will allow great flexibility (e.g. you could disable a particular
 bootstrap group's access when you are done provisioning the nodes). 
 ### Token auth file 
 Tokens are arbitrary but should represent at least 128 bits of entropy derived from a secure random number 
 generator (such as /dev/urandom on most modern systems). There are multiple ways you can generate a token. For example: 
 `head -c 16 /dev/urandom | od -An -t x | tr -d ' '` 
 will generate tokens that look like `02b50b05283e98dd0fd71db496ef01e8` 
 The token file will look like the following example, where the first three values can be anything and the quoted group 
 name should be as depicted: 
 ```
 02b50b05283e98dd0fd71db496ef01e8,kubelet-bootstrap,10001,"system:kubelet-bootstrap"
 ``` 
 Add the `--token-auth-file=FILENAME` flag to the apiserver command to enable the token file. 
 See docs at http://kubernetes.io/docs/admin/authentication/#static-token-file for further details.
 ### Client certificate CA bundle
 Add the `--client-ca-file=FILENAME` flag to the apiserver command to enable client certificate authentication,
 referencing a certificate authority bundle containing the signing certificate.
 ## controller-manager configuration
 The API for requesting certificates adds a certificate-issuing control loop to the KCM. This takes the form of a 
 [cfssl](https://blog.cloudflare.com/introducing-cfssl/) local signer using assets on disk. 
 Currently, all certificates issued have one year validity and a default set of key usages. 
 ### Signing assets 
 You must provide a Certificate Authority in order to provide the cryptographic materials necessary to issue certificates. 
 This CA should be trusted by the apiserver for authentication with the `--client-ca-file=SOMEFILE` flag. The management
 of the CA is beyond the scope of this document but it is recommended that you generate a dedicated CA for Kubernetes.
 Both certificate and key are assumed to be PEM-encoded. 
 The new controller-manager flags are: 
 ```
 --cluster-signing-cert-file="/etc/path/to/kubernetes/ca/ca.crt" --cluster-signing-key-file="/etc/path/to/kubernetes/ca/ca.key"
 ``` 
 ### Auto-approval 
 To ease deployment and testing, the alpha version of the certificate request API includes a flag to approve all certificate 
 requests made by users in a certain group. The intended use of this is to whitelist only the group corresponding to the bootstrap
 token in the token file above. Use of this flag circumvents makes the "approval" process described below and is not recommended
 for production use. 
 The flag is: 
 ```
 --insecure-experimental-approve-all-kubelet-csrs-for-group="system:kubelet-bootstrap"
 ``` 
 ## kubelet configuration 
 To use request a client cert from the certificate request API, the kubelet needs a path to a kubeconfig file that contains the
 bootstrap auth token. If the file specified by `--kubeconfig` does not exist, the bootstrap kubeconfig is used to request a
 client certificate from the API server. On success, a kubeconfig file referencing the generated key and obtained certificate
 is written to the path specified by `--kubeconfig`. The certificate and key file will be stored in the directory pointed 
 by `--cert-dir`.  The new flag is: 
 ```
 --experimental-bootstrap-kubeconfig="/path/to/bootstrap/kubeconfig"
 ``` 
 ## kubectl approval 
 The signing controller does not immediately sign all certificate requests. Instead, it waits until they have been flagged with an 
 "Approved" status by an appropriately-privileged user. This is intended to eventually be an automated process handled by an external
 approval controller, but for the alpha version of the API it can be done manually by a cluster administrator using kubectl. 
 An administrator can list CSRs with `kubectl get csr`, describe one in detail with `kubectl describe <name>`. There are 
 [currently no direct approve/deny commands](https://github.com/kubernetes/kubernetes/issues/30163) so an approver will need to update
 the Status field directly. A rough example of how to do this in bash which should only be used until the porcelain merges is available
 at [https://github.com/gtank/csrctl](https://github.com/gtank/csrctl).
@@ -2,13 +2,14 @@
 assignees:
 - dchen1107
 - roberthbailey
 - liggitt
 ---
 * TOC
 {:toc}
-## Summary
+## Overview
 This document catalogs the communication paths between the master (really the
 apiserver) and the Kubernetes cluster. The intent is to allow users to
@@ -22,14 +23,21 @@ All communication paths from the cluster to the master terminate at the
 apiserver (none of the other master components are designed to expose remote
 services). In a typical deployment, the apiserver is configured to listen for
 remote connections on a secure HTTPS port (443) with one or more forms of
-client [authentication](/docs/admin/authentication/) enabled.
+client [authentication](/docs/admin/authentication/) enabled. One or more forms
 of [authorization](/docs/admin/authorization/) should be enabled, especially
 if [anonymous requests](/docs/admin/authentication/#anonymous-requests) or 
 [service account tokens](/docs/admin/authentication/#service-account-tokens) 
 are allowed.
 Nodes should be provisioned with the public root certificate for the cluster
 such that they can connect securely to the apiserver along with valid client
 credentials. For example, on a default GCE deployment, the client credentials
-provided to the kubelet are in the form of a client certificate. Pods that
+provided to the kubelet are in the form of a client certificate. See 
-wish to connect to the apiserver can do so securely by leveraging a service
+[kubelet TLS bootstrapping](/docs/admin/kubelet-tls-bootstrapping/) for 
-account so that Kubernetes will automatically inject the public root
+automated provisioning of kubelet client certificates. 
 Pods that wish to connect to the apiserver can do so securely by leveraging a 
 service account so that Kubernetes will automatically inject the public root
 certificate and a valid bearer token into the pod when it is instantiated.
 The `kubernetes` service (in all namespaces) is configured with a virtual IP
 address that is redirected (via kube-proxy) to the HTTPS endpoint on the
@@ -54,16 +62,29 @@ cluster. The first is from the apiserver to the kubelet process which runs on
 each node in the cluster. The second is from the apiserver to any node, pod,
 or service through the apiserver's proxy functionality.
 ### apiserver -> kubelet
 The connections from the apiserver to the kubelet are used for fetching logs
 for pods, attaching (through kubectl) to running pods, and using the kubelet's
-port-forwarding functionality. These connections terminate at the kubelet's
+port-forwarding functionality. These connections terminate at the kubelet's 
-HTTPS endpoint, which is typically using a self-signed certificate, and
+HTTPS endpoint.
-ignore the certificate presented by the kubelet (although you can override this
+
-behavior by specifying the `--kubelet-certificate-authority`,
+By default, the apiserver does not verify the kubelet's serving certificate,
-`--kubelet-client-certificate`, and `--kubelet-client-key` flags when starting
+which makes the connection subject to man-in-the-middle attacks, and 
-the cluster apiserver). By default, these connections **are not currently safe**
+**unsafe** to run over untrusted and/or public networks.
-to run over untrusted and/or public networks as they are subject to
+
-man-in-the-middle attacks.
+To verify this connection, use the `--kubelet-certificate-authority` flag to 
 provide the apiserver with a root certificates bundle to use to verify the 
 kubelet's serving certificate.
 If that is not possible, use [SSH tunneling](/docs/admin/master-node-communication/#ssh-tunnels)
 between the apiserver and kubelet if required to avoid connecting over an 
 untrusted or public network.
 Finally, [Kubelet authentication and/or authorization](/docs/admin/kubelet-authentication-authorization/)
 should be enabled to secure the kubelet API. 
 ### apiserver -> nodes, pods, and services
 The connections from the apiserver to a node, pod, or service default to plain
 HTTP connections and are therefore neither authenticated nor encrypted. They
@@ -83,83 +104,3 @@ cluster (connecting to the ssh server listening on port 22) and passes all
 traffic destined for a kubelet, node, pod, or service through the tunnel.
 This tunnel ensures that the traffic is not exposed outside of the private
 GCE network in which the cluster is running.
 ### Kubelet TLS Bootstrap 
 Kubernetes 1.4 introduces an experimental API for requesting certificates from a cluster-level 
 Certificate Authority (CA). The first supported use of this API is the provisioning of TLS client 
 certificates for kubelets. The proposal can be found [here](https://github.com/kubernetes/kubernetes/pull/20439)
 and progress on the feature is being tracked as [feature #43](https://github.com/kubernetes/features/issues/43). 
 ##### apiserver configuration
 You must provide a token file which specifies at least one "bootstrap token" assigned to a kubelet boostrap-specific group.
 This group will later be used in the controller-manager configuration to scope approvals in the default approval
 controller. As this feature matures, you should ensure tokens are bound to an RBAC policy which limits requests
 using the bootstrap token to only be able to make requests related to certificate provisioning. When RBAC policy
 is in place, scoping the tokens to a group will allow great flexibility (e.g. you could disable a particular
 bootstrap group's access when you are done provisioning the nodes). 
 ##### Token auth file 
 Tokens are arbitrary but should represent at least 128 bits of entropy derived from a secure random number 
 generator (such as /dev/urandom on most modern systems). There are multiple ways you can generate a token. For example: 
 `head -c 16 /dev/urandom | od -An -t x | tr -d ' '` 
 will generate tokens that look like `02b50b05283e98dd0fd71db496ef01e8` 
 The token file will look like the following example, where the first three values can be anything and the quoted group 
 name should be as depicted: 
 ```
 02b50b05283e98dd0fd71db496ef01e8,kubelet-bootstrap,10001,"system:kubelet-bootstrap"
 ``` 
 Add the `--token-auth-file=FILENAME` flag to the apiserver command to enable the token file. 
 See docs at http://kubernetes.io/docs/admin/authentication/#static-token-file for further details.
 #### controller-manager configuration
 The API for requesting certificates adds a certificate-issuing control loop to the KCM. This takes the form of a 
 [cfssl](https://blog.cloudflare.com/introducing-cfssl/) local signer using assets on disk. 
 Currently, all certificates issued have one year validity and a default set of key usages. 
 ##### Signing assets 
 You must provide a Certificate Authority in order to provide the cryptographic materials necessary to issue certificates. 
 This CA should be trusted by the apiserver for authentication with the `--client-ca-file=SOMEFILE` flag. The management
 of the CA is beyond the scope of this document but it is recommended that you generate a dedicated CA for Kubernetes.
 Both certificate and key are assumed to be PEM-encoded. 
 The new controller-manager flags are: 
 ```
 --cluster-signing-cert-file="/etc/path/to/kubernetes/ca/ca.crt" --cluster-signing-key-file="/etc/path/to/kubernetes/ca/ca.key"
 ``` 
 ##### Auto-approval 
 To ease deployment and testing, the alpha version of the certificate request API includes a flag to approve all certificate 
 requests made by users in a certain group. The intended use of this is to whitelist only the group corresponding to the bootstrap
 token in the token file above. Use of this flag circumvents makes the "approval" process described below and is not recommended
 for production use. 
 The flag is: 
 ```
 --insecure-experimental-approve-all-kubelet-csrs-for-group="system:kubelet-bootstrap"
 ``` 
 #### kubelet configuration 
 To use request a client cert from the certificate request API, the kubelet needs a path to a kubeconfig file that contains the
 bootstrap auth token. If the file specified by `--kubeconfig` does not exist, the bootstrap kubeconfig is used to request a
 client certificate from the API server. On success, a kubeconfig file referencing the generated key and obtained certificate
 is written to the path specified by `--kubeconfig`. The certificate and key file will be stored in the directory pointed 
 by `--cert-dir`.  The new flag is: 
 ```
 --experimental-bootstrap-kubeconfig="/path/to/bootstrap/kubeconfig"
 ``` 
 #### kubectl approval 
 The signing controller does not immediately sign all certificate requests. Instead, it waits until they have been flagged with an 
 "Approved" status by an appropriately-privileged user. This is intended to eventually be an automated process handled by an external
 approval controller, but for the alpha version of the API it can be done manually by a cluster administrator using kubectl. 
 An administrator can list CSRs with `kubectl get csr`, describe one in detail with `kubectl describe <name>`. There are 
 [currently no direct approve/deny commands](https://github.com/kubernetes/kubernetes/issues/30163) so an approver will need to update
 the Status field directly. A rough example of how to do this in bash which should only be used until the porcelain merges is available
 at https://github.com/gtank/csrctl.