Release 1.7 (#4094)
* Minor fixes in the Deployment doc Signed-off-by: Michail Kargakis <mkargaki@redhat.com> * add NodeRestriction to admission-controllers (#3842) * Admins Can Configure Zones in Storage Class The PR #38505 (https://github.com/kubernetes/kubernetes/pull/38505) added zones optional parameter to Storage Class for AWS and GCE provisioners. That's why documentation needs to be updated accordingly. * document custom resource definitions * add host paths to psp (#3971) * add host paths to psp * add italics * Update ConfigMap doc to explain TTL-based cache updates (#3989) * Update ConfigMap doc to explain TTL-based cache updates * swap word order Change "When a ConfigMap being already consumed..." to "When a ConfigMap already being consumed..." * Update NetworkPolicy docs for v1 * StorageOS Volume plugin * Update GPU docs * docs: HPA autoscaling/v2alpha1 status conditions This commit documents the new status conditions feature for HPA autoscaling/v2alpha1. It demonstrates how to get the status conditions using `kubectl describe`, and how to interpret them. * Update description about NodeRestriction kubelet node can alse create mirror pods for their own static pods. * adding storage as a supported resource to node allocatable Signed-off-by: Vishnu kannan <vishnuk@google.com> * Add documentation for podpreset opt-out annotation This adds the annotation for having the podpreset admission controller to skip (opt-out) manipulating the pod spec. Also, the annotation format for what presets have acted on a pod has been modified to add a prefix of "podpreset-". The new naming makes it such that there is no chance of collision with the newly introduced opt-out annotation (or future ones yet to be added). Opt-out annotation PR: kubernetes/kubernetes#44965 * Update PDB documentation to explain new field (#3885) * update-docs-pdb * Addressed erictune@'s comments * Fix title and add a TOC to the logging concept page * Patch #4118 for typos * Describe setting coredns server in nameserver resolv chain * Address comments in PR #3997. Comment is in https://github.com/kubernetes/kubernetes.github.io/pull/3997/files/f6eb59c67e28efc298c87b1ef49a96bc6adacd1e#diff-7a14981f3dd8eb203f897ce6c11d9828 * Update task for DaemonSet history and rollback (#4098) * Update task for DaemonSet history and rollback Also remove mentions of templateGeneration field because it's deprecated * Address comments * removed lt and gt as operators (#4152) * removed lt and gt as operators * replace lt and gt for node-affinfity * updated based on bsalamat review * Initial draft of upgrade guide for kubeadm clusters. In-place upgrades are supported between 1.6 and 1.7 releases. Rollback instructions to come in a separate commit. Fixes https://github.com/kubernetes/kubeadm/issues/278 * Add local volume documentation (#4050) * Add local volume documentation * Add PV local volume example * Patch PR #3999 * Add documentation for Stackdriver event exporter * Add documentation about controller metrics * Federation: Add task for setting up placement policies (#4075) * Add task for setting up placement policies * Update version of management sidecar in policy engine deployment * Address @nikhiljindal's comments - Lower case filenames - Comments in policy - Typo fixes - Removed type LoadBalancer from OPA Service * Add example that sets cluster selector Per-@nikhiljindal's suggestion * Fix wording and templating per @chenopis * PodDisruptionBudget documentation Improvements (#4140) * Changes from #3885 Title: Update PDB documentation to explain new field Author: foxish * Added Placeholder Disruptions Concept Guide New file: docs/concepts/workloads/pods/disruptions.md Intented contents: concept for Pod Disruption Budget, cross reference to Eviction and Preemption docs. Linked from: concepts > workloads > pods * Added placeholder Configuring PDB Task New file: docs/tasks/run-application/configure-pdb.md Intented contents: task for writing a Pod Disruption Budget. Linked from: tasks > configuring-applications > configure pdb. * Add refs to the "drain a node" task. * Refactor PDB docs. Move the "Requesting an eviction" section from: docs/tasks/administer-cluster/configure-pod-disruption-budget.md -- which is going away -- to: docs/tasks/administer-cluster/safely-drain-node.md The move is verbatim, except for an introductory sentence. Also added assignees. * Refactor of PDB docs Moved the section: Specifying a PodDisruptionBudget from: docs/tasks/administer-cluster/configure-pod-disruption-budget.md to: docs/tasks/run-application/configure-pdb.md because that former file is going away. Move is verbatim. * Explain how Eviction tools should handle failures * Refactor PDB docs Move text from: docs/tasks/administer-cluster/configure-pod-disruption-budget.md to: docs/concepts/workloads/pods/disruptions.md Delete the now empty: docs/tasks/administer-cluster/configure-pod-disruption-budget.md Added a redirects_from section to the new doc, containing the path of the now-deleted doc, plus all the redirects from the deleted doc. * Expand PDB Concept guide Building on a little content from the old task, greatly expanded the Disruptions concept guide, including an abstract example. * Update creating a pdb Task. * Address review comments. * Fixed for all cody-clark's review comments * Address review comments from mml * Address review comments from maisem * Fix missing backtick * Api and Kubectl reference docs updates for 1.7 (#4193) * Fix includes groups * Generated kubectl docs for 1.7 * Generated references docs for 1.7 api * Document node authorization mode * API Aggregator (#4173) * API Aggregator * Additional bullet points * incorporated feedback for apiserver-aggregation.md * split setup-api-aggregator.md into two docs and address feedback * fix link * addressed docs feedback * incorporate feedback * integrate feedback * Add documentation for DNS stub domains (#4063) * Add documentation for DNS stub domains * add additional prereq * fix image path * review feedback * minor grammar and style nits * documentation for using hostAliases to manage hosts file (#4080) * documentation for using hostAliases to manage hosts file * add to table of contents * review comments * update the right command to see hosts file * reformat doc based on suggestion and change some wording * Fix typo for #4080 * Patch PR #4063 * Fix wording in placement policy task introduction * Add update to statefulset concepts and basic tutorial (#4174) * Add update to statefulset concpets and basic tutorial * Address tech comments. * Update ESIPP docs for new added API fields * Custom resource docs * update audit document with advanced audit features added in 1.7 * kubeadm v1.7 documentation updates (#4018) * v1.7 updates for kubeadm * Address review comments * Address Luke's comments * Encrypting secrets at rest and cluster security guide * Edits for Custom DNS Documentation (#4207) * reorganize custom dns doc * format fixes * Update version numbers to 1.7 * Patch PR #4140 (#4215) * Patch PR #4140 * fix link and typos * Update PR template * Update TLS bootstrapping with 1.7 features This includes documenting the new CSR approver built into the controller manager and the kubelet alpha features for certificate rotation. Since the CSR approver changed over the 1.7 release cycle we need to call out the migration steps for those using the alpha feature. This document as a whole could probably use some updates, but the main focus of this PR is just to get these features minimally documented before the release. * Federated ClusterSelector formatting updates from review * complete PR #4181 (#4223) * complete PR #4181 * fix security link * Extensible admission controller (#4092) * extensible-admission-controllers * Update extensible-admission-controllers.md * more on initializers * fixes * Expand external admission webhooks documentation * wrap at 80 chars * more * add reference * Use correct apigroup for network policy * Docs changes to PR #4092 (#4224) * Docs changes to PR #4092 * address feedback * add doc for --as-group in cli Add doc for this pr: https://github.com/kubernetes/kubernetes/pull/43696
This commit is contained in:
zhangchaolei
@@ -298,6 +298,13 @@ extensions group (`--runtime-config=extensions/v1beta1/podsecuritypolicy=true`).
|
||||
See also [Pod Security Policy documentation](/docs/concepts/policy/pod-security-policy/)
|
||||
for more information.
|
||||
|
||||
### NodeRestriction
|
||||
|
||||
This plug-in limits the `Node` and `Pod` objects a kubelet can modify. In order to be limited by this admission plugin,
|
||||
kubelets must use credentials in the `system:nodes` group, with a username in the form `system:node:<nodeName>`.
|
||||
Such kubelets will only be allowed to modify their own `Node` API object, and only modify `Pod` API objects that are bound to their node.
|
||||
Future versions may add additional restrictions to ensure kubelets have the minimal set of permissions required to operate correctly.
|
||||
|
||||
## Is there a recommended set of plug-ins to use?
|
||||
|
||||
Yes.
|
||||
|
||||
@@ -614,13 +614,13 @@ Impersonate-Extra-scopes: development
|
||||
```
|
||||
|
||||
When using `kubectl` set the `--as` flag to configure the `Impersonate-User`
|
||||
header.
|
||||
header, set the `--as-group` flag to configure the `Impersonate-Group` header.
|
||||
|
||||
```shell
|
||||
$ kubectl drain mynode
|
||||
Error from server (Forbidden): User "clark" cannot get nodes at the cluster scope. (get nodes mynode)
|
||||
|
||||
$ kubectl drain mynode --as=superman
|
||||
$ kubectl drain mynode --as=superman --as-group=system:masters
|
||||
node "mynode" cordoned
|
||||
node "mynode" drained
|
||||
```
|
||||
|
||||
@@ -58,12 +58,13 @@ of the `bind` verb on `roles` and `clusterroles` resources in the `rbac.authoriz
|
||||
* [Authentication](/docs/admin/authentication/) layer checks for authorization of the `impersonate` verb on `users`, `groups`, and `serviceaccounts` in the core API group, and the `userextras` in the `authentication.k8s.io` API group.
|
||||
|
||||
## Authorization Modules
|
||||
* **ABAC Mode** - Attribute-based access control (ABAC) defines an access control paradigm whereby access rights are granted to users through the use of policies which combine attributes together. The policies can use any type of attributes (user attributes, resource attributes, object, environment attributes etc). To learn more about using the ABAC mode, see [ABAC Mode](/docs/admin/authorization/abac/)
|
||||
* **RBAC Mode** - Role-based access control (RBAC) is a method of regulating access to computer or network resources based on the roles of individual users within an enterprise. In this context, access is the ability of an individual user to perform a specific task, such as view, create, or modify a file. To learn more about using the RBAC mode, see [RBAC Mode](/docs/admin/authorization/rbac/)
|
||||
* **Node** - A special-purpose authorizer that grants permissions to kubelets based on the pods they are scheduled to run. To learn more about using the Node authorization mode, see [Node Authorization](/docs/admin/authorization/node/)
|
||||
* **ABAC** - Attribute-based access control (ABAC) defines an access control paradigm whereby access rights are granted to users through the use of policies which combine attributes together. The policies can use any type of attributes (user attributes, resource attributes, object, environment attributes etc). To learn more about using the ABAC mode, see [ABAC Mode](/docs/admin/authorization/abac/)
|
||||
* **RBAC** - Role-based access control (RBAC) is a method of regulating access to computer or network resources based on the roles of individual users within an enterprise. In this context, access is the ability of an individual user to perform a specific task, such as view, create, or modify a file. To learn more about using the RBAC mode, see [RBAC Mode](/docs/admin/authorization/rbac/)
|
||||
..* When specified "RBAC" (Role-Based Access Control) uses the "rbac.authorization.k8s.io" API group to drive authorization decisions, allowing admins to dynamically configure permission policies through the Kubernetes API.
|
||||
..* As of 1.6 RBAC mode is in beta.
|
||||
..* To enable RBAC, start the apiserver with `--authorization-mode=RBAC`.
|
||||
* **Webhook Mode** - A WebHook is an HTTP callback: an HTTP POST that occurs when something happens; a simple event-notification via HTTP POST. A web application implementing WebHooks will POST a message to a URL when certain things happen. To learn more about using the Webhook mode, see [Webhook Mode](/docs/admin/authorization/webhook/)
|
||||
* **Webhook** - A WebHook is an HTTP callback: an HTTP POST that occurs when something happens; a simple event-notification via HTTP POST. A web application implementing WebHooks will POST a message to a URL when certain things happen. To learn more about using the Webhook mode, see [Webhook Mode](/docs/admin/authorization/webhook/)
|
||||
* **Custom Modules** - You can create custom modules for using with Kubernetes. To learn more, see **Custom Modules** below.
|
||||
|
||||
### Custom Modules
|
||||
|
||||
@@ -0,0 +1,93 @@
|
||||
---
|
||||
assignees:
|
||||
- timstclair
|
||||
- deads2k
|
||||
- liggitt
|
||||
- ericchiang
|
||||
title: Using Node Authorization
|
||||
---
|
||||
|
||||
* TOC
|
||||
{:toc}
|
||||
|
||||
Node authorization is a special-purpose authorization mode that specifically authorizes API requests made by kubelets.
|
||||
|
||||
## Overview
|
||||
|
||||
The Node authorizer allows a kubelet to perform API operations. This includes:
|
||||
|
||||
Read operations:
|
||||
|
||||
* services
|
||||
* endpoints
|
||||
* nodes
|
||||
* pods
|
||||
* secrets, configmaps, persistent volume claims and persistent volumes related to pods bound to the kubelet's node
|
||||
|
||||
Write operations:
|
||||
|
||||
* nodes and node status (enable the `NodeRestriction` admission plugin to limit a kubelet to modify its own node)
|
||||
* pods and pod status (enable the `NodeRestriction` admission plugin to limit a kubelet to modify pods bound to itself)
|
||||
* events
|
||||
|
||||
Auth-related operations:
|
||||
|
||||
* read/write access to the certificationsigningrequests API for TLS bootstrapping
|
||||
* the ability to create tokenreviews and subjectaccessreviews for delegated authentication/authorization checks
|
||||
|
||||
In future releases, the node authorizer may add or remove permissions to ensure kubelets
|
||||
have the minimal set of permissions required to operate correctly.
|
||||
|
||||
In order to be authorized by the Node authorizer, kubelets must use a credential that identifies them as
|
||||
being in the `system:nodes` group, with a username of `system:node:<nodeName>`.
|
||||
This group and user name format match the identity created for each kubelet as part of
|
||||
[kubelet TLS bootstrapping](/docs/admin/kubelet-tls-bootstrapping/).
|
||||
|
||||
To enable the Node authorizer, start the apiserver with `--authorization-mode=Node`.
|
||||
|
||||
To limit the API objects kubelets are able to write, enable the [NodeRestriction](/docs/admin/admission-controllers#NodeRestriction) admission plugin by starting the apiserver with `--admission-control=...,NodeRestriction,...`
|
||||
|
||||
## Migration considerations
|
||||
|
||||
### Kubelets outside the `system:nodes` group
|
||||
|
||||
Kubelets outside the `system:nodes` group would not be authorized by the `Node` authorization mode,
|
||||
and would need to continue to be authorized via whatever mechanism currently authorizes them.
|
||||
The node admission plugin would not restrict requests from these kubelets.
|
||||
|
||||
### Kubelets with undifferentiated usernames
|
||||
|
||||
In some deployments, kubelets have credentials that place them in the `system:nodes` group,
|
||||
but do not identify the particular node they are associated with,
|
||||
because they do not have a username in the `system:node:...` format.
|
||||
These kubelets would not be authorized by the `Node` authorization mode,
|
||||
and would need to continue to be authorized via whatever mechanism currently authorizes them.
|
||||
|
||||
The `NodeRestriction` admission plugin would ignore requests from these kubelets,
|
||||
since the default node identifier implementation would not consider that a node identity.
|
||||
|
||||
### Upgrades from previous versions using RBAC
|
||||
|
||||
Upgraded pre-1.7 clusters using [RBAC](/docs/admin/authorization/rbac/) will continue functioning as-is because the `system:nodes` group binding will already exist.
|
||||
|
||||
If a cluster admin wishes to start using the `Node` authorizer and `NodeRestriction` admission plugin
|
||||
to limit node access to the API, that can be done non-disruptively:
|
||||
|
||||
1. Enable the `Node` authorization mode (`--authorization-mode=Node,RBAC`) and the `NodeRestriction` admission plugin
|
||||
2. Ensure all kubelets' credentials conform to the group/username requirements
|
||||
3. Audit apiserver logs to ensure the `Node` authorizer is not rejecting requests from kubelets (no persistent `NODE DENY` messages logged)
|
||||
4. Delete the `system:node` cluster role binding
|
||||
|
||||
### RBAC Node Permissions
|
||||
|
||||
In 1.6, the `system:node` cluster role was automatically bound to the `system:nodes` group when using the [RBAC Authorization mode](/docs/admin/authorization/rbac/).
|
||||
|
||||
In 1.7, the automatic binding of the `system:nodes` group to the `system:node` role is deprecated
|
||||
because the node authorizer accomplishes the same purpose with the benefit of additional restrictions
|
||||
on secret and configmap access. If the `Node` and `RBAC` authorization modes are both enabled,
|
||||
the automatic binding of the `system:nodes` group to the `system:node` role is not created in 1.7.
|
||||
|
||||
In 1.8, the binding will not be created at all.
|
||||
|
||||
When using RBAC, the `system:node` cluster role will continue to be created,
|
||||
for compatibility with deployment methods that bind other users or groups to that role.
|
||||
@@ -461,11 +461,12 @@ The permissions required by individual control loops are contained in the <a hre
|
||||
</tr>
|
||||
<tr>
|
||||
<td><b>system:node</b></td>
|
||||
<td><b>system:nodes</b> group</td>
|
||||
<td>Allows access to resources required by the kubelet component, <b>including read access to secrets, and write access to pods</b>.
|
||||
In the future, read access to secrets and write access to pods will be restricted to objects scheduled to the node.
|
||||
To maintain permissions in the future, Kubelets must identify themselves with the group <b>system:nodes</b> and a username in the form <b>system:node:<node-name></b>.
|
||||
See <a href="https://pr.k8s.io/40476">https://pr.k8s.io/40476</a> for details.
|
||||
<td><b>system:nodes</b> group (deprecated in 1.7)</td>
|
||||
<td>Allows access to resources required by the kubelet component, <b>including read access to all secrets, and write access to all pods</b>.
|
||||
As of 1.7, use of the [Node authorizer](/docs/admin/authorization/node/)
|
||||
and [NodeRestriction admission plugin](/docs/admin/admission-controllers#NodeRestriction)
|
||||
is recommended instead of this role, and allow granting API access to kubelets based on the pods scheduled to run on them.
|
||||
As of 1.7, when the `Node` authorization mode is enabled, the automatic binding to the `system:nodes` group is not created.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
|
||||
@@ -0,0 +1,284 @@
|
||||
---
|
||||
assignees:
|
||||
- smarterclayton
|
||||
- lavalamp
|
||||
- whitlockjc
|
||||
- caesrxuchao
|
||||
title: Dynamic Admission Control
|
||||
---
|
||||
|
||||
* TOC
|
||||
{:toc}
|
||||
|
||||
## Overview
|
||||
|
||||
The [admission controllers documentation](/docs/admin/admission-controllers/)
|
||||
introduces how to use standard, plugin-style admission controllers. However,
|
||||
plugin admission controllers are not flexible enough for all use cases, due to
|
||||
the following:
|
||||
|
||||
* They need to be compiled into kube-apiserver.
|
||||
* They are only configurable when the apiserver starts up.
|
||||
|
||||
1.7 introduces two alpha features, *Initializers* and *External Admission
|
||||
Webhooks*, that address these limitations. These features allow admission
|
||||
controllers to be developed out-of-tree and configured at runtime.
|
||||
|
||||
This page describes how to use Initializers and External Admission Webhooks.
|
||||
|
||||
## Initializers
|
||||
|
||||
### What are initializers?
|
||||
|
||||
*Initializer* has two meanings:
|
||||
|
||||
* A list of pending pre-initialization tasks, stored in every object's metadata
|
||||
(e.g., "AddMyCorporatePolicySidecar").
|
||||
|
||||
* A user customized controller, which actually perform those tasks. The name of the task
|
||||
corresponds to the controller which performs the task. For clarity, we call
|
||||
them *initializer controllers* in this page.
|
||||
|
||||
Once the controller has performed its assigned task, it removes its name from
|
||||
the list. For example, it may send a PATCH that inserts a container in a pod and
|
||||
also removes its name from `metadata.initializers`. Initializers may make
|
||||
mutations to objects.
|
||||
|
||||
Objects which have a non-empty initializer list are considered uninitialized,
|
||||
and are not visible in the API unless specifically requested by using the query parameter,
|
||||
`?includeUninitialized=true`.
|
||||
|
||||
### When to use initializers?
|
||||
|
||||
Initializers are useful for admins to force policies (e.g., the
|
||||
[AlwaysPullImages](/docs/admin/admission-controllers/#alwayspullimages)
|
||||
admission controller), or to inject defaults (e.g., the
|
||||
[DefaultStorageClass](/docs/admin/admission-controllers/#defaultstorageclass)
|
||||
admission controller), etc.
|
||||
|
||||
**Note:** If your use case does not involve mutating objects, consider using
|
||||
external admission webhooks, as they have better performance.
|
||||
|
||||
### How are initializers triggered?
|
||||
|
||||
When an object is POSTed, it is checked against all existing
|
||||
`initializerConfiguration` objects (explained below). For all that it matches,
|
||||
all `spec.initializers[].name`s are appended to the new object's
|
||||
`metadata.initializers` field.
|
||||
|
||||
An initializer controller should list and watch for uninitialized objects, by
|
||||
using the query parameter `?includeUninitialized=true`. If using client-go, just
|
||||
set
|
||||
[listOptions.includeUninitialized](https://github.com/kubernetes/kubernetes/blob/v1.7.0-rc.1/staging/src/k8s.io/apimachinery/pkg/apis/meta/v1/types.go#L315)
|
||||
to true.
|
||||
|
||||
For the observed uninitialized objects, an initializer controller should first
|
||||
check if its name matches `metadata.initializers[0]`. If so, it should then
|
||||
perform its assigned task and remove its name from the list.
|
||||
|
||||
### Enable initializers alpha feature
|
||||
|
||||
*Initializers* is an alpha feature, so it is disabled by default. To turn it on,
|
||||
you need to:
|
||||
|
||||
* Include "Initializer" in the `--admission-control` flag when starting
|
||||
`kube-apiserver`. If you have multiple `kube-apiserver` replicas, all should
|
||||
have the same flag setting.
|
||||
|
||||
* Enable the dynamic admission controller registration API by adding
|
||||
`admissionregistration.k8s.io/v1alpha1` to the `--runtime-config` flag passed
|
||||
to `kube-apiserver`, e.g.
|
||||
`--runtime-config=admissionregistration.k8s.io/v1alpha1`. Again, all replicas
|
||||
should have the same flag setting.
|
||||
|
||||
### Deploy an initializer controller
|
||||
|
||||
You should deploy an initializer controller via the [deployment
|
||||
API](/docs/api-reference/v1.6/#deployment-v1beta1-apps).
|
||||
|
||||
### Configure initializers on the fly
|
||||
|
||||
You can configure what initializers are enabled and what resources are subject
|
||||
to the initializers by creating `initializerconfigurations`.
|
||||
|
||||
You should first deploy the initializer controller and make sure that it is
|
||||
working properly before creating the `initializerconfigurations`. Otherwise, any
|
||||
newly created resources will be stuck in an uninitialized state.
|
||||
|
||||
The following is an example `initiallizerConfiguration`.
|
||||
|
||||
```yaml
|
||||
apiVersion: admissionregistration.k8s.io/v1alpha1
|
||||
kind: InitializerConfiguration
|
||||
metadata:
|
||||
name: example-config
|
||||
spec:
|
||||
initializers:
|
||||
# the name needs to be fully qualified, i.e., containing at least two "."
|
||||
- name: podimage.example.com
|
||||
rules:
|
||||
# apiGroups, apiVersion, resources all support wildcard "*".
|
||||
# "*" cannot be mixed with non-wildcard.
|
||||
- apiGroups:
|
||||
- ""
|
||||
apiVersions:
|
||||
- v1
|
||||
resources:
|
||||
- pods
|
||||
```
|
||||
|
||||
Make sure that all expansions of the `<apiGroup, apiVersions, resources>` tuple
|
||||
in a `rule` are valid. If they are not, separate them in different `rules`.
|
||||
|
||||
After you create the `initializerConfiguration`, the system will take a few
|
||||
seconds to honor the new configuration.
|
||||
|
||||
## External Admission Webhooks
|
||||
|
||||
### What are external admission webhooks?
|
||||
|
||||
External admission webhooks are HTTP callbacks that are intended to receive
|
||||
admission requests and do something with them. What an external admission
|
||||
webhook does is up to you, but there is an
|
||||
[interface](https://github.com/kubernetes/kubernetes/blob/v1.7.0-rc.1/pkg/apis/admission/v1alpha1/types.go)
|
||||
that it must adhere to so that it responds with whether or not the
|
||||
admission request should be allowed.
|
||||
|
||||
Unlike initializers or the plugin-style admission controllers, external
|
||||
admission webhooks are not allowed to mutate the admission request in any way.
|
||||
|
||||
Because admission is a high security operation, the external admission webhooks
|
||||
must support TLS.
|
||||
|
||||
### When to use admission webhooks?
|
||||
|
||||
A simple example use case for an external admission webhook is to do semantic validation
|
||||
of Kubernetes resources. Suppose that your infrastructure requires that all `Pod`
|
||||
resources have a common set of labels, and you do not want any `Pod` to be
|
||||
persisted to Kubernetes if those needs are not met. You could write your
|
||||
external admission webhook to do this validation and respond accordingly.
|
||||
|
||||
### How are external admission webhooks triggered?
|
||||
|
||||
Whenever a request comes in, the `GenericAdmissionWebhook` admission plugin will
|
||||
get the list of interested external admission webhooks from
|
||||
`externalAdmissionHookConfiguration` objects (explained below) and call them in
|
||||
parallel. If **all** of the external admission webhooks approve the admission
|
||||
request, the admission chain continues. If **any** of the external admission
|
||||
webhooks deny the admission request, the admission request will be denied, and
|
||||
the reason for doing so will be based on the _first_ external admission webhook
|
||||
denial reason. _This means if there is more than one external admission webhook
|
||||
that denied the admission request, only the first will be returned to the
|
||||
user._ If there is an error encountered when calling an external admission
|
||||
webhook, that request is ignored and will not be used to approve/deny the
|
||||
admission request.
|
||||
|
||||
**Note:** The admission chain depends solely on the order of the
|
||||
`--admission-control` option passed to `kube-apiserver`.
|
||||
|
||||
### Enable external admission webhooks
|
||||
|
||||
*External Admission Webhooks* is an alpha feature, so it is disabled by default.
|
||||
To turn it on, you need to
|
||||
|
||||
* Include "GenericAdmissionWebhook" in the `--admission-control` flag when
|
||||
starting the apiserver. If you have multiple `kube-apiserver` replicas, all
|
||||
should have the same flag setting.
|
||||
|
||||
* Enable the dynamic admission controller registration API by adding
|
||||
`admissionregistration.k8s.io/v1alpha1` to the `--runtime-config` flag passed
|
||||
to `kube-apiserver`, e.g.
|
||||
`--runtime-config=admissionregistration.k8s.io/v1alpha1`. Again, all replicas
|
||||
should have the same flag setting.
|
||||
|
||||
### Write a webhook admission controller
|
||||
|
||||
See [caesarxuchao/example-webhook-admission-controller](https://github.com/caesarxuchao/example-webhook-admission-controller)
|
||||
for an example webhook admission controller.
|
||||
|
||||
The communication between the webhook admission controller and the apiserver, or
|
||||
more precisely, the GenericAdmissionWebhook admission controller, needs to be
|
||||
TLS secured. You need to generate a CA cert and use it to sign the server cert
|
||||
used by your webhook admission controller. The pem formatted CA cert is supplied
|
||||
to the apiserver via the dynamic registration API
|
||||
`externaladmissionhookconfigurations.clientConfig.caBundle`.
|
||||
|
||||
For each request received by the apiserver, the GenericAdmissionWebhook
|
||||
admission controller sends an
|
||||
[admissionReview](https://github.com/kubernetes/kubernetes/blob/v1.7.0-rc.1/pkg/apis/admission/v1alpha1/types.go#L27)
|
||||
to the relevant webhook admission controller. The webhook admission controller
|
||||
gathers information like `object`, `oldobject`, and `userInfo`, from
|
||||
`admissionReview.spec`, sends back a response with the body also being the
|
||||
`admissionReview`, whose `status` field is filled with the admission decision.
|
||||
|
||||
### Deploy the webhook admission controller
|
||||
|
||||
See [caesarxuchao/example-webhook-admission-controller deployment](https://github.com/caesarxuchao/example-webhook-admission-controller/tree/master/deployment)
|
||||
for an example deployment.
|
||||
|
||||
The webhook admission controller should be deployed via the
|
||||
[deployment API](/docs/api-reference/v1.6/#deployment-v1beta1-apps).
|
||||
You also need to create a
|
||||
[service](/docs/api-reference/v1.6/#service-v1-core) as the
|
||||
front-end of the deployment.
|
||||
|
||||
### Configure webhook admission controller on the fly
|
||||
|
||||
You can configure what webhook admission controllers are enabled and what
|
||||
resources are subject to the admission controller via creating
|
||||
externaladmissionhookconfigurations.
|
||||
|
||||
We suggest that you first deploy the webhook admission controller and make sure
|
||||
it is working properly before creating the externaladmissionhookconfigurations.
|
||||
Otherwise, depending whether the webhook is configured as fail open or fail
|
||||
closed, operations will be unconditionally accepted or rejected.
|
||||
|
||||
The following is an example externaladmissionhookconfiguration.
|
||||
|
||||
```yaml
|
||||
apiVersion: admissionregistration.k8s.io/v1alpha1
|
||||
kind: ExternalAdmissionHookConfiguration
|
||||
metadata:
|
||||
name: example-config
|
||||
externalAdmissionHooks:
|
||||
- name: pod-image.k8s.io
|
||||
rules:
|
||||
- apiGroups:
|
||||
- ""
|
||||
apiVersions:
|
||||
- v1
|
||||
operations:
|
||||
- CREATE
|
||||
resources:
|
||||
- pods
|
||||
failurePolicy: Ignore
|
||||
clientConfig:
|
||||
caBundle: <pem encoded ca cert that signs the server cert used by the webhook>
|
||||
service:
|
||||
name: <name of the front-end service>
|
||||
namespace: <namespace of the front-end service>
|
||||
```
|
||||
|
||||
For a request received by the apiserver, if the request matches any of the
|
||||
`rules` of an `externalAdmissionHook`, the `GenericAdmissionWebhook` admission
|
||||
controller will send an `admissionReview` request to the `externalAdmissionHook`
|
||||
to ask for admission decision.
|
||||
|
||||
The `rule` is similar to the `rule` in `initializerConfiguration`, with two
|
||||
differences:
|
||||
|
||||
* The addition of the `operations` field, specifying what operations the webhook
|
||||
is interested in;
|
||||
|
||||
* The `resources` field accepts subresources in the form or resource/subresource.
|
||||
|
||||
Make sure that all expansions of the `<apiGroup, apiVersions,resources>` tuple
|
||||
in a `rule` are valid. If they are not, separate them to different `rules`.
|
||||
|
||||
You can also specify the `failurePolicy`. In 1.7, the system supports `Ignore`
|
||||
and `Fail` policies, meaning that upon a communication error with the webhook
|
||||
admission controller, the `GenericAdmissionWebhook` can admit or reject the
|
||||
operation based on the configured policy.
|
||||
|
||||
After you create the `initializerConfiguration`, the system will take a few
|
||||
seconds to honor the new configuration.
|
||||
+191
-17
@@ -4,7 +4,7 @@ assignees:
|
||||
- luxas
|
||||
- errordeveloper
|
||||
- jbeda
|
||||
title: kubeadm Setup Tool
|
||||
title: kubeadm Setup Tool Reference Guide
|
||||
---
|
||||
|
||||
This document provides information on how to use kubeadm's advanced options.
|
||||
@@ -30,7 +30,7 @@ following steps:
|
||||
1. Outputting a kubeconfig file for the kubelet to use to connect to the API
|
||||
server, as well as an additional kubeconfig file for administration.
|
||||
|
||||
1. kubeadm generates Kubernetes Static Pod manifests for the API server,
|
||||
1. kubeadm generates Kubernetes static Pod manifests for the API server,
|
||||
controller manager and scheduler. It places them in
|
||||
`/etc/kubernetes/manifests`. The kubelet watches this directory for Pods to
|
||||
create on startup. These are the core components of Kubernetes. Once they are
|
||||
@@ -51,7 +51,7 @@ steps:
|
||||
|
||||
1. kubeadm creates a local key pair. It prepares a certificate signing request
|
||||
(CSR) and sends that off to the API server for signing. The bootstrap token
|
||||
is used to authenticate. The API server is configured to sign this
|
||||
is used to authenticate. The control plane will sign this CSR requested
|
||||
automatically.
|
||||
|
||||
1. kubeadm configures the local kubelet to connect to the API server
|
||||
@@ -318,8 +318,8 @@ schedulerExtraArgs:
|
||||
<argument>: <value|string>
|
||||
<argument>: <value|string>
|
||||
apiServerCertSANs:
|
||||
- <name1|string>
|
||||
- <name2|string>
|
||||
- <name1|string>
|
||||
- <name2|string>
|
||||
certificatesDir: <string>
|
||||
```
|
||||
In addition, if authorizationMode is set to `ABAC`, you should write the config to `/etc/kubernetes/abac_policy.json`.
|
||||
@@ -333,16 +333,79 @@ kind: NodeConfiguration
|
||||
caCertPath: <path|string>
|
||||
discoveryFile: <path|string>
|
||||
discoveryToken: <string>
|
||||
|
||||
# Currently only the first server is used as a target for the cluster
|
||||
# bootstrap flow.
|
||||
discoveryTokenAPIServers:
|
||||
- <address|string>
|
||||
- <address|string>
|
||||
|
||||
- <address|string>
|
||||
- <address|string>
|
||||
tlsBootstrapToken: <string>
|
||||
```
|
||||
|
||||
## Securing your installation even more
|
||||
|
||||
The defaults for kubeadm may not work for everyone. This section documents how to tighten up a kubeadm install
|
||||
at the cost of some usability.
|
||||
|
||||
### Turning off auto-approval of Node Client Certificates
|
||||
|
||||
By default, there is an CSR auto-approver enabled that basically approves any client certificate request
|
||||
for a kubelet when a Bootstrap Token was used when authenticating. If you don't want the cluster to
|
||||
automatically approve kubelet client certs, you can turn it off by executing this command:
|
||||
|
||||
```console
|
||||
$ kubectl delete clusterrole kubeadm:node-autoapprove-bootstrap
|
||||
```
|
||||
|
||||
After that, `kubeadm join` will block until the admin has manually approved the CSR in flight:
|
||||
|
||||
```console
|
||||
$ kubectl get csr
|
||||
NAME AGE REQUESTOR CONDITION
|
||||
node-csr-c69HXe7aYcqkS1bKmH4faEnHAWxn6i2bHZ2mD04jZyQ 18s system:bootstrap:878f07 Pending
|
||||
|
||||
$ kubectl certificate approve node-csr-c69HXe7aYcqkS1bKmH4faEnHAWxn6i2bHZ2mD04jZyQ
|
||||
certificatesigningrequest "node-csr-c69HXe7aYcqkS1bKmH4faEnHAWxn6i2bHZ2mD04jZyQ" approved
|
||||
|
||||
$ kubectl get csr
|
||||
NAME AGE REQUESTOR CONDITION
|
||||
node-csr-c69HXe7aYcqkS1bKmH4faEnHAWxn6i2bHZ2mD04jZyQ 1m system:bootstrap:878f07 Approved,Issued
|
||||
```
|
||||
|
||||
Only after `kubectl certificate approve` has been run, `kubeadm join` can proceed.
|
||||
|
||||
### Turning off public access to the cluster-info ConfigMap
|
||||
|
||||
In order to achieve the joining flow using the token as the only piece of validation information, a
|
||||
public ConfigMap with some data needed for validation of the master's identity is exposed publicly by
|
||||
default. While there is no private data in this ConfigMap, some users are sensitive and wish to turn
|
||||
it off regardless. Doing so will disable the ability to use the `--discovery-token` flag of the
|
||||
`kubeadm join` flow. Here are the steps to do so:
|
||||
|
||||
Fetch the `cluster-info` file from the API Server:
|
||||
|
||||
```console
|
||||
$ kubectl -n kube-public get cm cluster-info -oyaml | grep "kubeconfig:" -A11 | grep "apiVersion" -A10 | sed "s/ //" | tee cluster-info.yaml
|
||||
apiVersion: v1
|
||||
clusters:
|
||||
- cluster:
|
||||
certificate-authority-data: <ca-cert>
|
||||
server: https://<ip>:<port>
|
||||
name: ""
|
||||
contexts: []
|
||||
current-context: ""
|
||||
kind: Config
|
||||
preferences: {}
|
||||
users: []
|
||||
```
|
||||
|
||||
You can then use the `cluster-info.yaml` file as an argument to `kubeadm join --discovery-file`.
|
||||
|
||||
Turning of public access to the `cluster-info` ConfigMap:
|
||||
|
||||
```console
|
||||
$ kubectl -n kube-public delete rolebinding kubeadm:bootstrap-signer-clusterinfo
|
||||
```
|
||||
|
||||
These command should be run after `kubeadm init` but before `kubeadm join`.
|
||||
|
||||
## Managing Tokens {#manage-tokens}
|
||||
|
||||
You can use the `kubeadm` tool to manage tokens on a running cluster. It will
|
||||
@@ -380,13 +443,13 @@ parallelize the token distribution for easier automation. To implement this
|
||||
automation, you must know the IP address that the master will have after it is
|
||||
started.
|
||||
|
||||
1. Generate a token. This token must have the form `<6 character string>.<16
|
||||
character string>`. More formally, it must match the regex
|
||||
1. Generate a token. This token must have the form `<6 character string>.<16
|
||||
character string>`. More formally, it must match the regex:
|
||||
`[a-z0-9]{6}\.[a-z0-9]{16}`.
|
||||
|
||||
Kubeadm can generate a token for you:
|
||||
kubeadm can generate a token for you:
|
||||
|
||||
``` bash
|
||||
```bash
|
||||
kubeadm token generate
|
||||
```
|
||||
|
||||
@@ -397,6 +460,112 @@ started.
|
||||
Once the cluster is up, you can grab the admin credentials from the master node
|
||||
at `/etc/kubernetes/admin.conf` and use that to talk to the cluster.
|
||||
|
||||
## Use Kubeadm with other CRI runtimes
|
||||
|
||||
Since [Kubernetes 1.6 release](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG.md#node-components-1), Kubernetes container runtimes have been transferred to using CRI by default. Currently, the build-in container runtime is Docker which is enabled by build-in `dockershim` in `kubelet`.
|
||||
|
||||
Using other CRI based runtimes with kubeadm is very simple, and currently supported runtimes are:
|
||||
|
||||
- [cri-o](https://github.com/kubernetes-incubator/cri-o)
|
||||
- [frakti](https://github.com/kubernetes/frakti)
|
||||
- [rkt](https://github.com/kubernetes-incubator/rktlet)
|
||||
|
||||
After you have successfully installed `kubeadm` and `kubelet`, please follow these two steps:
|
||||
|
||||
1. Install runtime shim on every node. You will need to follow the installation document in the runtime shim project listing above.
|
||||
|
||||
2. Configure kubelet to use remote CRI runtime. Please remember to change `RUNTIME_ENDPOINT` to your own value like `/var/run/{your_runtime}.sock`:
|
||||
|
||||
```shell
|
||||
$ cat > /etc/systemd/system/kubelet.service.d/20-cri.conf <<EOF
|
||||
Environment="KUBELET_EXTRA_ARGS=--container-runtime=remote --container-runtime-endpoint=$RUNTIME_ENDPOINT --feature-gates=AllAlpha=true"
|
||||
EOF
|
||||
$ systemctl daemon-reload
|
||||
```
|
||||
|
||||
Now `kubelet` is ready to use the specified CRI runtime, and you can continue with `kubeadm init` and `kubeadm join` workflow to deploy Kubernetes cluster.
|
||||
|
||||
## Using custom certificates
|
||||
|
||||
By default kubeadm will generate all the certificates needed for a cluster to run.
|
||||
You can override this behaviour by providing your own certificates.
|
||||
|
||||
To do so, you must place them in whatever directory is specified by the
|
||||
`--cert-dir` flag or `CertificatesDir` configuration file key. By default this
|
||||
is `/etc/kubernetes/pki`.
|
||||
|
||||
If a given certificate and private key pair both exist, kubeadm will skip the
|
||||
generation step and those files will be validated and used for the prescribed
|
||||
use-case.
|
||||
|
||||
This means you can, for example, prepopulate `/etc/kubernetes/pki/ca.crt`
|
||||
and `/etc/kubernetes/pki/ca.key` with an existing CA, which then will be used
|
||||
for signing the rest of the certs.
|
||||
|
||||
## Running kubeadm without an internet connection
|
||||
|
||||
All of the control plane components run in Pods started by the kubelet and
|
||||
the following images are required for the cluster works will be automatically
|
||||
pulled by the kubelet if they don't exist locally while `kubeadm init` is initializing
|
||||
your master:
|
||||
|
||||
| Image Name | v1.6 release branch version | v1.7 release branch version
|
||||
|---|---|---|
|
||||
| gcr.io/google_containers/kube-apiserver-${ARCH} | v1.6.x | v1.7.x
|
||||
| gcr.io/google_containers/kube-controller-manager-${ARCH} | v1.6.x | v1.7.x
|
||||
| gcr.io/google_containers/kube-scheduler-${ARCH} | v1.6.x | v1.7.x
|
||||
| gcr.io/google_containers/kube-proxy-${ARCH} | v1.6.x | v1.7.x
|
||||
| gcr.io/google_containers/etcd-${ARCH} | 3.0.17 | 3.0.17
|
||||
| gcr.io/google_containers/pause-${ARCH} | 3.0 | 3.0
|
||||
| gcr.io/google_containers/k8s-dns-sidecar-${ARCH} | 1.14.1 | 1.14.4
|
||||
| gcr.io/google_containers/k8s-dns-kube-dns-${ARCH} | 1.14.1 | 1.14.4
|
||||
| gcr.io/google_containers/k8s-dns-dnsmasq-nanny-${ARCH} | 1.14.1 | 1.14.4
|
||||
|
||||
Here `v1.7.x` means the "latest patch release of the v1.7 branch".
|
||||
|
||||
`${ARCH}` can be one of: `amd64`, `arm`, `arm64`, `ppc64le` or `s390x`.
|
||||
|
||||
|
||||
## Cloudprovider integrations (experimental)
|
||||
|
||||
Enabling specific cloud providers is a common request. This currently requires
|
||||
manual configuration and is therefore not yet fully supported. If you wish to do
|
||||
so, edit the kubeadm dropin for the kubelet service
|
||||
(`/etc/systemd/system/kubelet.service.d/10-kubeadm.conf`) on all nodes,
|
||||
including the master. If your cloud provider requires any extra packages
|
||||
installed on the host, for example for volume mounting/unmounting, install those
|
||||
packages.
|
||||
|
||||
Specify the `--cloud-provider` flag for the kubelet and set it to the cloud of your
|
||||
choice. If your cloudprovider requires a configuration file, create the file
|
||||
`/etc/kubernetes/cloud-config` on every node. The exact format and content of
|
||||
that file depends on the requirements imposed by your cloud provider. If you use
|
||||
the `/etc/kubernetes/cloud-config` file, you must append it to the kubelet
|
||||
arguments as follows: `--cloud-config=/etc/kubernetes/cloud-config`
|
||||
|
||||
Note that there is most likely other per-provider configuration that may be needed
|
||||
(IAM roles for AWS) that is currently underdocumented.
|
||||
|
||||
Next, specify the cloud provider in the kubeadm config file. Create a file called
|
||||
`kubeadm.conf` with the following contents:
|
||||
|
||||
``` yaml
|
||||
kind: MasterConfiguration
|
||||
apiVersion: kubeadm.k8s.io/v1alpha1
|
||||
cloudProvider: <cloud provider>
|
||||
```
|
||||
|
||||
Lastly, run `kubeadm init --config=kubeadm.conf` to bootstrap your cluster with
|
||||
the cloud provider.
|
||||
|
||||
This workflow is not yet fully supported, however we hope to make it extremely
|
||||
easy to spin up clusters with cloud providers in the future. (See [this
|
||||
proposal](https://github.com/kubernetes/community/pull/128) for more
|
||||
information) The [Kubelet Dynamic
|
||||
Settings](https://github.com/kubernetes/kubernetes/pull/29459) feature may also
|
||||
help to fully automate this process in the future.
|
||||
|
||||
|
||||
## Environment variables
|
||||
|
||||
There are some environment variables that modify the way that kubeadm works.
|
||||
@@ -404,6 +573,8 @@ Most users will have no need to set these. These environment variables are a
|
||||
short-term solution, eventually they will be integrated in the kubeadm
|
||||
configuration file.
|
||||
|
||||
**Note:** These environment variables are deprecated and will stop functioning in v1.8!
|
||||
|
||||
| Variable | Default | Description |
|
||||
| --- | --- | --- |
|
||||
| `KUBE_KUBERNETES_DIR` | `/etc/kubernetes` | Where most configuration files are written to and read from |
|
||||
@@ -414,8 +585,11 @@ configuration file.
|
||||
If `KUBE_KUBERNETES_DIR` is specified, you may need to rewrite the arguments of the kubelet.
|
||||
(e.g. --kubeconfig, --pod-manifest-path)
|
||||
|
||||
If `KUBE_REPO_PREFIX` is specified, you may need to set the kubelet flag `--pod-infra-container-image` which specifies which pause image to use.
|
||||
Defaults to `gcr.io/google_containers/pause-${ARCH}:3.0` where `${ARCH}` can be one of `amd64`, `arm`, `arm64`, `ppc64le` or `s390x`.
|
||||
If `KUBE_REPO_PREFIX` is specified, you may need to set the kubelet flag
|
||||
`--pod-infra-container-image` which specifies which pause image to use.
|
||||
|
||||
Defaults to `gcr.io/google_containers/pause-${ARCH}:3.0` where `${ARCH}`
|
||||
can be one of `amd64`, `arm`, `arm64`, `ppc64le` or `s390x`.
|
||||
|
||||
```bash
|
||||
cat > /etc/systemd/system/kubelet.service.d/20-pod-infra-image.conf <<EOF
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
---
|
||||
assignees:
|
||||
- ericchiang
|
||||
- mikedanese
|
||||
- jcbsmpsn
|
||||
title: TLS bootstrapping
|
||||
---
|
||||
|
||||
@@ -59,18 +61,114 @@ The kube-controller-manager flags are:
|
||||
--cluster-signing-cert-file="/etc/path/to/kubernetes/ca/ca.crt" --cluster-signing-key-file="/etc/path/to/kubernetes/ca/ca.key"
|
||||
```
|
||||
|
||||
### Automatic approval
|
||||
To ease deployment and testing, there is an experimental flag in the certificate bootstrapping API to approve all certificate
|
||||
requests made by users in a certain group. The intended use of this is to whitelist only the group corresponding to the bootstrap
|
||||
token in the token file above. Use of this flag circumvents the approval process described below and is not recommended
|
||||
for production use.
|
||||
### Approval controller
|
||||
|
||||
The flag is:
|
||||
In 1.7 the experimental "group auto approver" controller is dropped in favor of the new `csrapproving` controller
|
||||
that ships as part of [kube-controller-manager](/docs/admin/kube-controller-manager/) and is enabled by default.
|
||||
The controller uses the [`SubjectAccessReview` API](/docs/admin/authorization/#checking-api-access) to determine
|
||||
if a given user is authorized to request a CSR, then approves based on the authorization outcome. To prevent
|
||||
conflicts with other approvers, the builtin approver doesn't explicitly deny CSRs, only ignoring unauthorized requests.
|
||||
|
||||
The controller categorizes CSRs into three subresources:
|
||||
|
||||
1. `nodeclient` - a request by a user for a client certificate with `O=system:nodes` and `CN=system:node:(node name)`.
|
||||
2. `selfnodeclient` - a node renewing a client certificate with the same `O` and `CN`.
|
||||
3. `selfnodeserver` - a node renewing a serving certificate. (ALPHA, requires feature gate)
|
||||
|
||||
The checks to determine if a CSR is a `selfnodeserver` request is currently tied to the kubelet's credential rotation
|
||||
implementation, an __alpha__ feature. As such, the definition of `selfnodeserver` will likely change in a future and
|
||||
requires the `RotateKubeletServerCertificate` feature gate on the controller manager. The feature progress can be
|
||||
tracked at [kubernetes/features#267](https://github.com/kubernetes/features/issues/267).
|
||||
|
||||
```
|
||||
--insecure-experimental-approve-all-kubelet-csrs-for-group="system:kubelet-bootstrap"
|
||||
--feature-gates=RotateKubeletServerCertificate=true
|
||||
```
|
||||
|
||||
The following RBAC `ClusterRoles` represent the `nodeclient`, `selfnodeclient`, and `selfnodeserver` capabilities. Similar roles
|
||||
may be automatically created in future releases.
|
||||
|
||||
```yml
|
||||
# A ClusterRole which instructs the CSR approver to approve a user requesting
|
||||
# node client credentials.
|
||||
kind: ClusterRole
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
metadata:
|
||||
name: approve-node-client-csr
|
||||
rules:
|
||||
- apiGroups: ["certificates.k8s.io"]
|
||||
resources: ["certificatesigningrequests/nodeclient"]
|
||||
verbs: ["create"]
|
||||
---
|
||||
# A ClusterRole which instructs the CSR approver to approve a node renewing its
|
||||
# own client credentials.
|
||||
kind: ClusterRole
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
metadata:
|
||||
name: approve-node-client-renewal-csr
|
||||
rules:
|
||||
- apiGroups: ["certificates.k8s.io"]
|
||||
resources: ["certificatesigningrequests/selfnodeclient"]
|
||||
verbs: ["create"]
|
||||
---
|
||||
# A ClusterRole which instructs the CSR approver to approve a node requesting a
|
||||
# serving cert matching its client cert.
|
||||
kind: ClusterRole
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
metadata:
|
||||
name: approve-node-server-renewal-csr
|
||||
rules:
|
||||
- apiGroups: ["certificates.k8s.io"]
|
||||
resources: ["certificatesigningrequests/selfnodeserver"]
|
||||
verbs: ["create"]
|
||||
```
|
||||
|
||||
These powers can be granted to credentials, such as bootstrapping tokens. For example, to replicate the behavior
|
||||
provided by the removed auto-approval flag, of approving all CSRs by a single group:
|
||||
|
||||
```
|
||||
# REMOVED: This flag no longer works as of 1.7.
|
||||
--insecure-experimental-approve-all-kubelet-csrs-for-group="kubelet-bootstrap-token"
|
||||
```
|
||||
|
||||
An admin would create a `ClusterRoleBinding` targeting that group.
|
||||
|
||||
```yml
|
||||
# Approve all CSRs for the group "kubelet-bootstrap-token"
|
||||
kind: ClusterRoleBinding
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
metadata:
|
||||
name: auto-approve-csrs-for-group
|
||||
subjects:
|
||||
- kind: Group
|
||||
name: kubelet-bootstrap-token
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
roleRef:
|
||||
kind: ClusterRole
|
||||
name: approve-node-client-csr
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
```
|
||||
|
||||
To let a node renew its own credentials, an admin can construct a `ClusterRoleBinding` targeting
|
||||
that node's credentials:
|
||||
|
||||
```yml
|
||||
kind: ClusterRoleBinding
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
metadata:
|
||||
name: node1-client-cert-renewal
|
||||
subjects:
|
||||
- kind: User
|
||||
name: system:node:node-1 # Let "node-1" renew its client certificate.
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
roleRef:
|
||||
kind: ClusterRole
|
||||
name: approve-node-client-renewal-csr
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
```
|
||||
|
||||
Deleting the binding will prevent the node from renewing its client credentials, effectively
|
||||
removing it from the cluster once its certificate expires.
|
||||
|
||||
## kubelet configuration
|
||||
To request a client certificate from kube-apiserver, the kubelet first needs a path to a kubeconfig file that contains the
|
||||
bootstrap authentication token. You can use `kubectl config set-cluster`, `set-credentials`, and `set-context` to build this kubeconfig. Provide the name `kubelet-bootstrap` to `kubectl config set-credentials` and include `--token=<token-value>` as follows:
|
||||
@@ -87,6 +185,19 @@ The flag to enable this bootstrapping when starting the kubelet is:
|
||||
--experimental-bootstrap-kubeconfig="/path/to/bootstrap/kubeconfig"
|
||||
```
|
||||
|
||||
Additionally, in 1.7 the kubelet implements __alpha__ features for enabling rotation of both its client and/or serving certs.
|
||||
These can be enabled through the respective `RotateKubeletClientCertificate` and `RotateKubeletServerCertificate` feature
|
||||
flags on the kubelet, but may change in backward incompatible ways in future releases.
|
||||
|
||||
```
|
||||
--feature-gates=RotateKubeletClientCertificate=true,RotateKubeletServerCertificate=true
|
||||
```
|
||||
|
||||
`RotateKubeletClientCertificate` causes the kubelet to rotate its client certificates by creating new CSRs as its existing
|
||||
credentials expire. `RotateKubeletServerCertificate` causes the kubelet to both request a serving certificate after
|
||||
bootstrapping its client credentials and rotate the certificate. The serving cert currently does not request DNS or IP
|
||||
SANs.
|
||||
|
||||
## kubectl approval
|
||||
The signing controller does not immediately sign all certificate requests. Instead, it waits until they have been flagged with an
|
||||
"Approved" status by an appropriately-privileged user. This is intended to eventually be an automated process handled by an external
|
||||
|
||||
Reference in New Issue
Block a user