From 49a81e1a69a0d7d0b8f98bf7119583b025517792 Mon Sep 17 00:00:00 2001 From: Chao Xu Date: Wed, 14 Mar 2018 18:40:25 -0700 Subject: [PATCH] Update the doc on admission webhooks (#7733) --- .../admin/extensible-admission-controllers.md | 317 +++++++++--------- 1 file changed, 163 insertions(+), 154 deletions(-) diff --git a/docs/admin/extensible-admission-controllers.md b/docs/admin/extensible-admission-controllers.md index 7e67c2d252..afe5d58308 100644 --- a/docs/admin/extensible-admission-controllers.md +++ b/docs/admin/extensible-admission-controllers.md @@ -4,6 +4,7 @@ reviewers: - lavalamp - whitlockjc - caesarxuchao +- deads2k title: Dynamic Admission Control --- @@ -20,11 +21,169 @@ the following: * They need to be compiled into kube-apiserver. * They are only configurable when the apiserver starts up. -1.7 introduces two alpha features, *Initializers* and *External Admission -Webhooks*, that address these limitations. These features allow admission -controllers to be developed out-of-tree and configured at runtime. +Two features, *Admission Webhooks* (beta in 1.9) and *Initializers* (alpha), +address these limitations. They allow admission controllers to be developed +out-of-tree and configured at runtime. -This page describes how to use Initializers and External Admission Webhooks. +This page describes how to use Admission Webhooks and Initializers. + +## Admission Webhooks + +### What are admission webhooks? + +Admission webhooks are HTTP callbacks that receive admission requests and do +something with them. You can define two types of admission webhooks, +[ValidatingAdmissionWebhooks](/docs/admin/admission-controllers.md#validatingadmissionwebhook-alpha-in-18-beta-in-19) +and +[MutatingAdmissionWebhooks](/docs/admin/admission-controllers.md#mutatingadmissionwebhook-beta-in-19). +With `ValidatingAdmissionWebhooks`, you may reject requests to enforce custom +admission policies. With `MutatingAdmissionWebhooks`, you may change requests to +enforce custom defaults. + +### Experimenting with admission webhooks + +Admission webhooks are essentially part of the cluster control-plane. You should +write and deploy them with great caution. Please read the [user +guides](https://github.com/kubernetes/website/pull/6836/files)(WIP) for +instructions if you intend to write/deploy production-grade admission webhooks. +In the following, we describe how to quickly experiment with admission webhooks. + +### Prerequisites + +* Ensure that the Kubernetes cluster is at least as new as v1.9. + +* Ensure that MutatingAdmissionWebhook and ValidatingAdmissionWebhook + admission controllers are enabled. + [Here](/docs/admin/admission-controllers.md#is-there-a-recommended-set-of-admission-controllers-to-use) + is a recommended set of admission controllers to enable in general. + +* Ensure that the admissionregistration.k8s.io/v1beta1 API is enabled. + +### Write an admission webhook server + +Please refer to the implementation of the [admission webhook +server](https://github.com/kubernetes/kubernetes/blob/v1.10.0-beta.1/test/images/webhook/main.go) +that is validated in a Kubernetes e2e test. The webhook handles the +`admissionReview` requests sent by the apiservers, and sends back its decision +wrapped in `admissionResponse`. + +The example admission webhook server leaves the `ClientAuth` field +[empty](https://github.com/kubernetes/kubernetes/blob/v1.10.0-beta.1/test/images/webhook/config.go#L48-L49), +which defaults to `NoClientCert`. This means that the webhook server does not +authenticate the identity of the clients, supposedly apiservers. If you need +mutual TLS or other ways to authenticate the clients, see +how to [authenticate apiservers](#authenticate-apiservers). + +### Deploy the admission webhook service + +The webhook server in the e2e test is deployed in the Kubernetes cluster, via +the [deployment API](/docs/api-reference/{{page.version}}/#deployment-v1beta1-apps). +The test also creates a [service](/docs/api-reference/{{page.version}}/#service-v1-core) +as the front-end of the webhook server. See +[code](https://github.com/kubernetes/kubernetes/blob/v1.10.0-beta.1/test/e2e/apimachinery/webhook.go#L196). + +You may also deploy your webhooks outside of the cluster. You will need to update +your [webhook client configurations](https://github.com/kubernetes/kubernetes/blob/v1.10.0-beta.1/staging/src/k8s.io/api/admissionregistration/v1beta1/types.go#L218) accordingly. + +### Configure admission webhooks on the fly + +You can dynamically configure what resources are subject to what admission +webhooks via +[ValidatingWebhookConfiguration](https://github.com/kubernetes/kubernetes/blob/v1.10.0-beta.1/staging/src/k8s.io/api/admissionregistration/v1beta1/types.go#L68) +or +[MutatingWebhookConifuration](https://github.com/kubernetes/kubernetes/blob/v1.10.0-beta.1/staging/src/k8s.io/api/admissionregistration/v1beta1/types.go#L98). + +The following is an example `validatingWebhookConfiguration`, a mutating webhook +configuration is similar. + +```yaml +apiVersion: admissionregistration.k8s.io/v1beta1 +kind: ValidatingWebhookConfiguration +metadata: + name: +webhooks: +- name: + rules: + - apiGroups: + - "" + apiVersions: + - v1 + operations: + - CREATE + resources: + - pods + clientConfig: + service: + namespace: + name: + caBundle: +``` + +When an apiserver receives a request that matches one of the `rules`, the +apiserver sends an `admissionReview` request to webhook as specified in the +`clientConfig`. + +After you create the webhook configuration, the system will take a few seconds +to honor the new configuration. + +### Authenticate apiservers + +If your admission webhooks require authentication, you can configure the +apiservers to use basic auth, bearer token, or a cert to authenticate itself to +the webhooks. There are three steps to complete the configuration. + +* When starting the apiserver, specify the location of the admission control + configuration file via the `--admission-control-config-file` flag. + +* In the admission control configuration file, specify where the + MutatingAdmissionWebhook controller and ValidatingAdmissionWebhook controller + should read the credentials. The credentials are stored in kubeConfig files + (yes, the same schema that's used by kubectl), so the field name is + `kubeConfigFile`. Here is an example admission control configuration file: + +```yaml +apiVersion: apiserver.k8s.io/v1alpha1 +kind: AdmissionConfiguration +plugins: +- name: ValidatingAdmissionWebhook + configuration: + apiVersion: apiserver.config.k8s.io/v1alpha1 + kind: WebhookAdmission + kubeConfigFile: +- name: MutatingAdmissionWebhook + configuration: + apiVersion: apiserver.config.k8s.io/v1alpha1 + kind: WebhookAdmission + kubeConfigFile: +``` + +The schema of `admissionConfiguration` is defined +[here](https://github.com/kubernetes/kubernetes/blob/v1.10.0-beta.0/staging/src/k8s.io/apiserver/pkg/apis/apiserver/v1alpha1/types.go#L27). + +* In the kubeConfig file, provide the credentials: + +```yaml +apiVersion: v1 +kind: Config +users: +# DNS name of webhook service, i.e., ..svc, or the URL +# of the webhook server. +- name: 'webhook1.ns1.svc' + user: + client-certificate-data: + client-key-data: +# The `name` supports using * to wildmatch prefixing segments. +- name: '*.webhook-company.org' + user: + password: + username: +# '*' is the default match. +- name: '*' + user: + token: +``` + +Of course you need to set up the webhook server to handle these authentications. ## Initializers @@ -135,153 +294,3 @@ the pods will be stuck in an uninitialized state. Make sure that all expansions of the `` tuple in a `rule` are valid. If they are not, separate them in different `rules`. - -## External Admission Webhooks - -### What are external admission webhooks? - -External admission webhooks are HTTP callbacks that are intended to receive -admission requests and do something with them. What an external admission -webhook does is up to you, but there is an -[interface](https://github.com/kubernetes/kubernetes/blob/v1.7.0-rc.1/pkg/apis/admission/v1alpha1/types.go) -that it must adhere to so that it responds with whether or not the -admission request should be allowed. - -Unlike initializers or the plugin-style admission controllers, external -admission webhooks are not allowed to mutate the admission request in any way. - -Because admission is a high security operation, the external admission webhooks -must support TLS. - -### When to use admission webhooks? - -A simple example use case for an external admission webhook is to do semantic validation -of Kubernetes resources. Suppose that your infrastructure requires that all `Pod` -resources have a common set of labels, and you do not want any `Pod` to be -persisted to Kubernetes if those needs are not met. You could write your -external admission webhook to do this validation and respond accordingly. - -### How are external admission webhooks triggered? - -Whenever a request comes in, the `GenericAdmissionWebhook` admission plugin will -get the list of interested external admission webhooks from -`externalAdmissionHookConfiguration` objects (explained below) and call them in -parallel. If **all** of the external admission webhooks approve the admission -request, the admission chain continues. If **any** of the external admission -webhooks deny the admission request, the admission request will be denied, and -the reason for doing so will be based on the _first_ external admission webhook -denial reason. _This means if there is more than one external admission webhook -that denied the admission request, only the first will be returned to the -user._ If there is an error encountered when calling an external admission -webhook, that request is ignored and will not be used to approve/deny the -admission request. - -**Note:** The admission chain depends solely on the order of the -`--admission-control` option passed to `kube-apiserver`. - -### Enable external admission webhooks - -*External Admission Webhooks* is an alpha feature, so it is disabled by default. -To turn it on, you need to - -* Include "GenericAdmissionWebhook" in the `--admission-control` flag when - starting the apiserver. If you have multiple `kube-apiserver` replicas, all - should have the same flag setting. - -* Enable the dynamic admission controller registration API by adding - `admissionregistration.k8s.io/v1alpha1` to the `--runtime-config` flag passed - to `kube-apiserver`, e.g. - `--runtime-config=admissionregistration.k8s.io/v1alpha1`. Again, all replicas - should have the same flag setting. - -### Write a webhook admission controller - -See [caesarxuchao/example-webhook-admission-controller](https://github.com/caesarxuchao/example-webhook-admission-controller) -for an example webhook admission controller. - -The communication between the webhook admission controller and the apiserver, or -more precisely, the GenericAdmissionWebhook admission controller, needs to be -TLS secured. You need to generate a CA cert and use it to sign the server cert -used by your webhook admission controller. The pem formatted CA cert is supplied -to the apiserver via the dynamic registration API -`externaladmissionhookconfigurations.clientConfig.caBundle`. - -For each request received by the apiserver, the GenericAdmissionWebhook -admission controller sends an -[admissionReview](https://github.com/kubernetes/kubernetes/blob/v1.7.0-rc.1/pkg/apis/admission/v1alpha1/types.go#L27) -to the relevant webhook admission controller. The webhook admission controller -gathers information like `object`, `oldobject`, and `userInfo`, from -`admissionReview.spec`, sends back a response with the body also being the -`admissionReview`, whose `status` field is filled with the admission decision. - -### Deploy the webhook admission controller - -See [caesarxuchao/example-webhook-admission-controller deployment](https://github.com/caesarxuchao/example-webhook-admission-controller/tree/master/deployment) -for an example deployment. - -The webhook admission controller should be deployed via the -[deployment API](/docs/api-reference/{{page.version}}/#deployment-v1beta1-apps). -You also need to create a -[service](/docs/api-reference/{{page.version}}/#service-v1-core) as the -front-end of the deployment. - -### Configure webhook admission controller on the fly - -You can configure what webhook admission controllers are enabled and what -resources are subject to the admission controller via creating -externaladmissionhookconfigurations. - -We suggest that you first deploy the webhook admission controller and make sure -it is working properly before creating the externaladmissionhookconfigurations. -Otherwise, depending whether the webhook is configured as fail open or fail -closed, operations will be unconditionally accepted or rejected. - -The following is an example `externaladmissionhookconfiguration`: - -```yaml -apiVersion: admissionregistration.k8s.io/v1alpha1 -kind: ExternalAdmissionHookConfiguration -metadata: - name: example-config -externalAdmissionHooks: -- name: pod-image.k8s.io - rules: - - apiGroups: - - "" - apiVersions: - - v1 - operations: - - CREATE - resources: - - pods - failurePolicy: Ignore - clientConfig: - caBundle: - service: - name: - namespace: -``` - -For a request received by the apiserver, if the request matches any of the -`rules` of an `externalAdmissionHook`, the `GenericAdmissionWebhook` admission -controller will send an `admissionReview` request to the `externalAdmissionHook` -to ask for admission decision. - -The `rule` is similar to the `rule` in `initializerConfiguration`, with two -differences: - -* The addition of the `operations` field, specifying what operations the webhook - is interested in; - -* The `resources` field accepts subresources in the form or resource/subresource. - -Make sure that all expansions of the `` tuple -in a `rule` are valid. If they are not, separate them to different `rules`. - -You can also specify the `failurePolicy`. In 1.7, the system supports `Ignore` -and `Fail` policies, meaning that upon a communication error with the webhook -admission controller, the `GenericAdmissionWebhook` can admit or reject the -operation based on the configured policy. - -After you create the `externalAdmissionHookConfiguration`, the system will take a few -seconds to honor the new configuration.