Official 1.13 Release Docs (#11401)
* Update metadata.generation behaviour for custom resources (#10705) * update docs promoting plugins to beta (#10796) * docs update to promote TaintBasedEvictions to beta (#10765) * First Korean l10n work for dev-1.13 (#10719) * Update outdated l10n(ko) contents (#10689) fixes #10686 * Translate concepts/overview/what-is-kubernetes in Korean (#10690) * Translate concepts/overview/what-is-kubernetes in Korean * Feedback from ClaudiaJKang * Translate concepts/overview/components in Korean (#10882) * Translate concepts/overview/components in Korean #10717 * Translate concepts/overview/components in Korean * Translate concepts/overview/components in Korean * Apply Korean glossary: 서비스 어카운트 * Translate concepts/overview/kubernetes-api in Korean (#10773) * Translate concepts/overview/kubernetes-api in Korean * Applied feedback from ianychoi * kubeadm: update the configuration docs to v1beta1 (#10959) * kubeadm: add small v1beta1 related updates (#10988) * ADD content/zh/docs/reference/setup-tools/kubeadm/kubeadm.md (#11031) * ADD content/zh/docs/reference/setup-tools/kubeadm/kubeadm.md * ADD content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_init.md * Update content/zh/docs/reference/setup-tools/kubeadm/kubeadm.md Accepted Co-Authored-By: YouthLab <tsui@highyouth.com> * do not change 'master' or 'worker' nodes to '主从' * Doc updates for volume scheduling GA (#10743) * Doc updates for volume scheduling GA * Make trivial change to kick build * Document nodelease feature (#10699) * advanced audit doc for ModeBlockingStrict (#10203) * Rename EncryptionConfig to EncryptionConfiguration (#11080) EncryptionConfig was renamed to EncryptedConfiguration and added to the `apiserver.config.k8s.io` API group in Kubernetes 1.13. The feature was previously in alpha and was not handling versions properly, which lead to an originally unnoticed `v1` in the docs. * content/zh/docs/reference/setup-tools/kubeadm/kubeadm-init.md * trsanlate create-cluster-kubeadm.md to chinese (#11041) * trsanlate create-cluster-kubeadm.md to chinese * Update create-cluster-kubeadm.md * update the feature stage in v1.13 (#11307) * update new feature gates to document (#11295) * refresh controller role list on rbac description page (#11290) * node labeling restriction docs (#10944) * Update 1.13 docs for CSI GA (#10893) * dynamic audit documentation (#9947) * adds dynamic audit documentation * Copyedit for clarity See also inline question/s * Fix feature state shortcode * Update feature state * changes wording for dynamic audit flag behavior * Minor copyedit * fix dynamic audit yaml * adds api enablement command to dynamic audit docs * change ordering dynamic audit appears in * add references to dynamic audit in webhook backend * reword dynamic audit reference * updates stages field for audit sink object * changes audit sink api definition; rewords policy * kubeadm: remove kube-proxy workaround (#11162) * zh-trans content/en/docs/setup/independent/install-kubeadm.md (#11338) * zh-trans content/en/docs/setup/independent/install-kubeadm.md * Update install-kubeadm.md * Update dry run feature to beta (#11140) * vSphere volume raw block support doc update (#10932) * Add docs for Windows DNS configurations (#10036) * Update docs for fields allowed at root of CRD schema (#9973) * Add docs for Windows DNS configurations * add device monitoring documentation (#9945) * kubeadm: adds upgrade instructions for 1.13 (#11138) * kubeadm: adds upgrade instructions for 1.13 Signed-off-by: Chuck Ha <ha.chuck@gmail.com> * add minor copyedits Addressed a couple of copyedit comments a bit more cleanly. * kubeadm: add improvements to HA docs (#11094) * kubeadm: add information and diagrams for HA topologies * kubeadm: update HA doc with simplified steps * kubeadm: update HA doc with simplified steps * edit ha, add new topology topic, reorder by weight * troubleshoot markdown * fix more markdown, fix links * more markdown * more markdown * more markdown * changes after reviewer comments * add steps about Weave * update note about stacked topology * kubeadm external etcd HA upgrade 1.13 (#11364) * kubeadm external etcd HA upgrade 1.13 Signed-off-by: Ruben Orduz <rubenoz@gmail.com> * Update stacked controlplane steps * kubeadm cert documentation (#11093) * kubeadm certificate API and CSR documentation * copyedits * fix typo * PR for diff docs (#10789) * Empty commit against dev-1.13 for diff documentation * Complete Declarative maangement with diff commands * Second Korean l10n work for dev-1.13. (#11030) * Update outdated l10n(ko) contents (#10915) * Translate main menu for l10n(ko) docs (#10916) * Translate tasks/run-application/horizontal-pod-autoscale-walkthrough (#10980) * Translate content/ko/docs/concepts/overview/working-with-objects/kubernetes-object in Korean #11104 (#11332) * Pick-right-solution page translates into Korean. (#11340) * ko-trans: add jd/..., sap/..., ebay/..., homeoffice/... (#11336) * Translate concept/workloads/pods/pod-overview.md (#11092) Co-authored-by: June Yi <june.yi@samsung.com> Co-authored-by: Jesang Myung <jesang.myung@gmail.com> Co-authored-by: zerobig <38598117+zer0big@users.noreply.github.com> Co-authored-by: Claudia J.Kang <claudiajkang@gmail.com> Co-authored-by: lIuDuI <1693291525@qq.com> Co-authored-by: Woojin Na(Eddie) <cheapluv@gmail.com> * Rename encryption-at-rest related objects (#11059) EncryptionConfig was renamed to EncryptedConfiguration and added to the `apiserver.config.k8s.io` API group in Kubernetes 1.13. The feature was previously in alpha and was not handling versions properly, which lead to an originally unnoticed `v1` in the docs. Also, the `--experimental-encryption-provider-config` flag is now called just `--encryption-provider-config`. * Documenting FlexVolume Resize alpha feature. (#10097) * CR webhook conversion documentation (#10986) * CR Conversion * Addressing comments * Addressing more comments * Addressing even more comments * Addressing even^2 more comments * Remove references to etcd2 in v1.13 since support has been removed (#11414) * Remove etcd2 references as etcd2 is deprecated Link back to the v1.12 version of the etcd3 doc for the etcd2->etcd3 migration instructions. I updated the kube-apiserver reference manually, unsure if that is auto-generated somehow. The federation-apiserver can still potentially support etcd2 so I didn't touch that. * Remove outdated {master,node}.yaml files There are master/node yaml files that reference etcd2.service that are likely highly out of date. I couldn't find any docs that actually reference these templates so I removed them * Address review comments * Final Korean l10n work for dev-1.13 (#11440) * Update outdated l10n(ko) contents (#11425) fixes #11424 * Remove references to etcd2 in content/ko (#11416) * Resolve conflicts against master for /ko contents (#11438) * Fix unopened caution shortcode * kubeadm: update the reference docs for 1.13 (#10960) * docs update to promote TaintBasedEvictions to beta (#10765) * First Korean l10n work for dev-1.13 (#10719) * Update outdated l10n(ko) contents (#10689) fixes #10686 * Translate concepts/overview/what-is-kubernetes in Korean (#10690) * Translate concepts/overview/what-is-kubernetes in Korean * Feedback from ClaudiaJKang * Translate concepts/overview/components in Korean (#10882) * Translate concepts/overview/components in Korean #10717 * Translate concepts/overview/components in Korean * Translate concepts/overview/components in Korean * Apply Korean glossary: 서비스 어카운트 * Translate concepts/overview/kubernetes-api in Korean (#10773) * Translate concepts/overview/kubernetes-api in Korean * Applied feedback from ianychoi * kubeadm: update the configuration docs to v1beta1 (#10959) * kubeadm: add small v1beta1 related updates (#10988) * update new feature gates to document (#11295) * Update dry run feature to beta (#11140) * kubeadm: add improvements to HA docs (#11094) * kubeadm: add information and diagrams for HA topologies * kubeadm: update HA doc with simplified steps * kubeadm: update HA doc with simplified steps * edit ha, add new topology topic, reorder by weight * troubleshoot markdown * fix more markdown, fix links * more markdown * more markdown * more markdown * changes after reviewer comments * add steps about Weave * update note about stacked topology * kubeadm: update reference docs - add section about working with phases under kubeadm-init.md - update GA / beta status of features - kubeadm alpha phase was moved to kubeadm init phase - new commands were added under kubeadm alpha - included new CoreDNS usage examples * Generate components and tools reference * Add generated federation API Reference (#11491) * Add generated federation API Reference * Add front matter to federation reference * Remove whitespace from federation front matter * Remove more whitespace from federation front matter * Remove superfluous kubefed reference * Add frontmatter to generated kubefed reference * Fix kubefed reference page frontmatter * Generate kubectl reference docs 1.13 (#11487) * Generate kubectl reference docs 1.13 * Fix links in kubectl reference * Add 1.13 API reference (#11489) * Update config.toml (#11486) * Update config.toml Preparing for 1.13 release, updating the config.toml and dropping the 1.8 docs reference. * update dot releases and docsbranch typo * adding .Site. to Params.currentUrl (#11503) see https://github.com/kubernetes/website/pull/11502 for context * Add 1.13 Release notes (#11499)
This commit is contained in:
Jim Angel
committed by
Kubernetes Prow Robot
Kubernetes Prow Robot
parent
b1dde5578c
commit
27b7b453a9
+140
-13
@@ -11,13 +11,7 @@ weight: 30
|
||||
{{% capture overview %}}
|
||||
This page explains how to add versioning information to
|
||||
[CustomResourceDefinitions](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#customresourcedefinition-v1beta1-apiextensions), to indicate the stability
|
||||
level of your CustomResourceDefinitions. It also describes how to upgrade an
|
||||
object from one version to another.
|
||||
|
||||
{{< note >}}
|
||||
All specified versions must use the same schema. There is no schema conversion
|
||||
between versions.
|
||||
{{< /note >}}
|
||||
level of your CustomResourceDefinitions or advance your API to a new version with conversion between API representations. It also describes how to upgrade an object from one version to another.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
@@ -36,10 +30,10 @@ between versions.
|
||||
## Overview
|
||||
|
||||
The CustomResourceDefinition API supports a `versions` field that you can use to
|
||||
support multiple versions of custom resources that you have developed, and
|
||||
indicate the stability of a given custom resource. All versions must currently
|
||||
use the same schema, so if you need to add a field, you must add it to all
|
||||
versions.
|
||||
support multiple versions of custom resources that you have developed. Versions
|
||||
can have different schemas with a conversion webhook to convert custom resources between versions.
|
||||
Webhook conversions should follow the [Kubernetes API conventions](https://github.com/kubernetes/community/blob/master/contributors/devel/api-conventions.md) wherever applicable.
|
||||
Specifically, See the [API change documentation](https://github.com/kubernetes/community/blob/master/contributors/devel/api_changes.md) for a set of useful gotchas and suggestions.
|
||||
|
||||
{{< note >}}
|
||||
Earlier iterations included a `version` field instead of `versions`. The
|
||||
@@ -49,8 +43,9 @@ match the first item in the `versions` field.
|
||||
|
||||
## Specify multiple versions
|
||||
|
||||
This example shows a CustomResourceDefinition with two versions. The comments in
|
||||
the YAML provide more context.
|
||||
This example shows a CustomResourceDefinition with two versions. For the first
|
||||
example, the assumption is all versions share the same schema with no conversion
|
||||
between them. The comments in the YAML provide more context.
|
||||
|
||||
```yaml
|
||||
apiVersion: apiextensions.k8s.io/v1beta1
|
||||
@@ -71,6 +66,12 @@ spec:
|
||||
- name: v1
|
||||
served: true
|
||||
storage: false
|
||||
# The conversion section is introduced in Kubernetes 1.13+ with a default value of
|
||||
# None conversion (strategy sub-field set to None).
|
||||
conversion:
|
||||
# None conversion assumes the same schema for all versions and only sets the apiVersion
|
||||
# field of custom resources to the proper value
|
||||
strategy: None
|
||||
# either Namespaced or Cluster
|
||||
scope: Namespaced
|
||||
names:
|
||||
@@ -144,6 +145,132 @@ version sort order is `v1`, followed by `v1beta1`. This causes the kubectl
|
||||
command to use `v1` as the default version unless the provided object specifies
|
||||
the version.
|
||||
|
||||
## Webhook conversion
|
||||
|
||||
{{< note >}}
|
||||
Webhook conversion is introduced in Kubernetes 1.13 as an alpha feature. To use it, the
|
||||
`CustomResourceWebhookConversion` feature should be enabled. Please refer to the [feature gate](https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/) documentation for more information.
|
||||
{{< /note >}}
|
||||
|
||||
The above example has a None conversion between versions which only sets the `apiVersion` field
|
||||
on conversion and does not change the rest of the object. The API server also supports webhook
|
||||
conversions that call an external service in case a conversion is required. For example when:
|
||||
|
||||
* custom resource is requested in a different version than stored version.
|
||||
* Watch is created in one version but the changed object is stored in another version.
|
||||
* custom resource PUT request is in a different version than storage version.
|
||||
|
||||
To cover all of these cases and to optimize conversion by the API server, the conversion requests may contain multiple objects in order to minimize the external calls. The webhook should perform these conversions independently.
|
||||
|
||||
### Write a conversion webhook server
|
||||
|
||||
Please refer to the implementation of the [custom resource conversion webhook
|
||||
server](https://github.com/kubernetes/kubernetes/tree/v1.13.0/test/images/crd-conversion-webhook/main.go)
|
||||
that is validated in a Kubernetes e2e test. The webhook handles the
|
||||
`ConversionReview` requests sent by the API servers, and sends back conversion
|
||||
results wrapped in `ConversionResponse`. Note that the request
|
||||
contains a list of custom resources that need to be converted independently without
|
||||
changing the order of objects.
|
||||
The example server is organized in a way to be reused for other conversions. Most of the common code are located in the [framework file]((https://github.com/kubernetes/kubernetes/tree/v1.13.0/test/images/crd-conversion-webhook/converter/framework.go)) that leaves only [one function]((https://github.com/kubernetes/kubernetes/tree/v1.13.0/test/images/crd-conversion-webhook/converter/example-converter.go#L29-L80)) to be implemented for different conversions.
|
||||
|
||||
{{< note >}}
|
||||
The example conversion webhook server leaves the `ClientAuth` field
|
||||
[empty](https://github.com/kubernetes/kubernetes/tree/v1.13.0/test/images/crd-conversion-webhook/config.go#L47-L48),
|
||||
which defaults to `NoClientCert`. This means that the webhook server does not
|
||||
authenticate the identity of the clients, supposedly API servers. If you need
|
||||
mutual TLS or other ways to authenticate the clients, see
|
||||
how to [authenticate API servers](/docs/reference/access-authn-authz/extensible-admission-controllers/#authenticate-apiservers).
|
||||
{{< /note >}}
|
||||
|
||||
### Deploy the conversion webhook service
|
||||
|
||||
Documentation for deploying the conversion webhook is the same as for the [admission webhook example service](/docs/reference/access-authn-authz/extensible-admission-controllers/#deploy_the_admission_webhook_service).
|
||||
The assumption for next sections is that the conversion webhook server is deployed to a service named `example-conversion-webhook-server` in `default` namespace.
|
||||
|
||||
{{< note >}}
|
||||
When the webhook server is deployed into the Kubernetes cluster as a
|
||||
service, it has to be exposed via a service on port 443 (The server
|
||||
itself can have an arbitrary port but the service object should map it to port 443).
|
||||
The communication between the API server and the webhook service may fail
|
||||
if a different port is used for the service.
|
||||
{{< /note >}}
|
||||
|
||||
### Configure CustomResourceDefinition to use conversion webhooks
|
||||
|
||||
The `None` conversion example can be extended to use the conversion webhook by modifying `conversion`
|
||||
section of the `spec`:
|
||||
|
||||
```yaml
|
||||
apiVersion: apiextensions.k8s.io/v1beta1
|
||||
kind: CustomResourceDefinition
|
||||
metadata:
|
||||
# name must match the spec fields below, and be in the form: <plural>.<group>
|
||||
name: crontabs.example.com
|
||||
spec:
|
||||
# group name to use for REST API: /apis/<group>/<version>
|
||||
group: example.com
|
||||
# list of versions supported by this CustomResourceDefinition
|
||||
versions:
|
||||
- name: v1beta1
|
||||
# Each version can be enabled/disabled by Served flag.
|
||||
served: true
|
||||
# One and only one version must be marked as the storage version.
|
||||
storage: true
|
||||
# Each version can define it's own schema when there is no top-level
|
||||
# schema is defined.
|
||||
schema:
|
||||
openAPIV3Schema:
|
||||
properties:
|
||||
hostPort:
|
||||
type: string
|
||||
- name: v1
|
||||
served: true
|
||||
storage: false
|
||||
schema:
|
||||
openAPIV3Schema:
|
||||
properties:
|
||||
host:
|
||||
type: string
|
||||
port:
|
||||
type: string
|
||||
conversion:
|
||||
# a Webhook strategy instruct API server to call an external webhook for any conversion between custom resources.
|
||||
strategy: Webhook
|
||||
# webhookClientConfig is required when strategy is `Webhook` and it configure the webhook endpoint to be
|
||||
# called by API server.
|
||||
webhookClientConfig:
|
||||
service:
|
||||
namespace: default
|
||||
name: example-conversion-webhook-server
|
||||
caBundle: <pem encoded ca cert that signs the server cert used by the webhook>
|
||||
# either Namespaced or Cluster
|
||||
scope: Namespaced
|
||||
names:
|
||||
# plural name to be used in the URL: /apis/<group>/<version>/<plural>
|
||||
plural: crontabs
|
||||
# singular name to be used as an alias on the CLI and for display
|
||||
singular: crontab
|
||||
# kind is normally the CamelCased singular type. Your resource manifests use this.
|
||||
kind: CronTab
|
||||
# shortNames allow shorter string to match your resource on the CLI
|
||||
shortNames:
|
||||
- ct
|
||||
```
|
||||
|
||||
{{< note >}}
|
||||
When using `clientConfig.service`, the server cert must be valid for
|
||||
`<svc_name>.<svc_namespace>.svc`.
|
||||
{{< /note >}}
|
||||
|
||||
You can save the CustomResourceDefinition in a YAML file, then use
|
||||
`kubectl apply` to apply it.
|
||||
|
||||
```shell
|
||||
kubectl apply -f my-versioned-crontab-with-conversion.yaml
|
||||
```
|
||||
|
||||
Make sure the conversion service is up and running before applying new changes.
|
||||
|
||||
## Writing, reading, and updating versioned CustomResourceDefinition objects
|
||||
|
||||
When an object is written, it is persisted at the version designated as the
|
||||
|
||||
+3
-5
@@ -146,10 +146,8 @@ items:
|
||||
- apiVersion: stable.example.com/v1
|
||||
kind: CronTab
|
||||
metadata:
|
||||
clusterName: ""
|
||||
creationTimestamp: 2017-05-31T12:56:35Z
|
||||
deletionGracePeriodSeconds: null
|
||||
deletionTimestamp: null
|
||||
generation: 1
|
||||
name: my-new-cron-object
|
||||
namespace: default
|
||||
resourceVersion: "285"
|
||||
@@ -175,7 +173,7 @@ kubectl get crontabs
|
||||
```
|
||||
|
||||
```console
|
||||
Error from server (NotFound): Unable to list "crontabs": the server could not find the requested resource (get crontabs.stable.example.com)
|
||||
Error from server (NotFound): Unable to list {"stable.example.com" "v1" "crontabs"}: the server could not find the requested resource (get crontabs.stable.example.com)
|
||||
```
|
||||
|
||||
If you later recreate the same CustomResourceDefinition, it will start out empty.
|
||||
@@ -470,7 +468,7 @@ When the status subresource is enabled, the `/status` subresource for the custom
|
||||
- `PUT` requests to the `/status` subresource take a custom resource object and ignore changes to anything except the status stanza.
|
||||
- `PUT` requests to the `/status` subresource only validate the status stanza of the custom resource.
|
||||
- `PUT`/`POST`/`PATCH` requests to the custom resource ignore changes to the status stanza.
|
||||
- Any changes to the spec stanza increments the value at `.metadata.generation`.
|
||||
- The `.metadata.generation` value is incremented for all changes, except for changes to `.metadata` or `.status`.
|
||||
- Only the following constructs are allowed at the root of the CRD OpenAPI validation schema:
|
||||
|
||||
- Description
|
||||
|
||||
@@ -201,223 +201,29 @@ If the majority of etcd members have permanently failed, the etcd cluster is con
|
||||
|
||||
## Upgrading and rolling back etcd clusters
|
||||
|
||||
### Important assumptions
|
||||
As of Kubernetes v1.13.0, etcd2 is no longer supported as a storage backend for
|
||||
new or existing Kubernetes clusters. The timeline for Kubernetes support for
|
||||
etcd2 and etcd3 is as follows:
|
||||
|
||||
The upgrade procedure described in this document assumes that either:
|
||||
- Kubernetes v1.0: etcd2 only
|
||||
- Kubernetes v1.5.1: etcd3 support added, new clusters still default to etcd2
|
||||
- Kubernetes v1.6.0: new clusters created with `kube-up.sh` default to etcd3,
|
||||
and `kube-apiserver` defaults to etcd3
|
||||
- Kubernetes v1.9.0: deprecation of etcd2 storage backend announced
|
||||
- Kubernetes v1.13.0: etcd2 storage backend removed, `kube-apiserver` will
|
||||
refuse to start with `--storage-backend=etcd2`, with the
|
||||
message `etcd2 is no longer a supported storage backend`
|
||||
|
||||
1. The etcd cluster has only a single node.
|
||||
2. The etcd cluster has multiple nodes.
|
||||
Before upgrading a v1.12.x kube-apiserver using `--storage-backend=etcd2` to
|
||||
v1.13.x, etcd v2 data MUST by migrated to the v3 storage backend, and
|
||||
kube-apiserver invocations changed to use `--storage-backend=etcd3`.
|
||||
|
||||
In this case, the upgrade procedure requires shutting down the
|
||||
etcd cluster. During the time the etcd cluster is shut down, the Kubernetes API Server will be read only.
|
||||
The process for migrating from etcd2 to etcd3 is highly dependent on how the
|
||||
etcd cluster was deployed and configured, as well as how the Kubernetes
|
||||
cluster was deployed and configured. We recommend that you consult your cluster
|
||||
provider's documentation to see if there is a predefined solution.
|
||||
|
||||
{{< warning >}}
|
||||
Deviations from the assumptions are untested by continuous
|
||||
integration, and deviations might create undesirable consequences. Additional information about operating an etcd cluster is available [from the etcd maintainers](https://github.com/coreos/etcd/tree/master/Documentation).
|
||||
{{< /warning >}}
|
||||
|
||||
### Background
|
||||
|
||||
As of Kubernetes version 1.5.1, we are still using etcd from the 2.2.1 release with
|
||||
the v2 API. Also, we have no pre-existing process for updating etcd, as we have
|
||||
never updated etcd by either minor or major version.
|
||||
|
||||
Note that we need to migrate both the etcd versions that we are using (from 2.2.1
|
||||
to at least 3.0.x) as well as the version of the etcd API that Kubernetes talks to. The etcd 3.0.x
|
||||
binaries support both the v2 and v3 API.
|
||||
|
||||
This document describes how to do this migration. If you want to skip the
|
||||
background and cut right to the procedure, see [Upgrade
|
||||
Procedure](#upgrade-procedure).
|
||||
|
||||
### etcd upgrade requirements
|
||||
|
||||
There are requirements on how an etcd cluster upgrade can be performed. The primary considerations are:
|
||||
- Upgrade between one minor release at a time
|
||||
- Rollback supported through additional tooling
|
||||
|
||||
#### One minor release at a time
|
||||
|
||||
Upgrade only one minor release at a time. For example, we cannot upgrade directly from 2.1.x to 2.3.x.
|
||||
Within patch releases it is possible to upgrade and downgrade between arbitrary versions. Starting a cluster for
|
||||
any intermediate minor release, waiting until the cluster is healthy, and then
|
||||
shutting down the cluster will perform the migration. For example, to upgrade from version 2.1.x to 2.3.y,
|
||||
it is enough to start etcd in 2.2.z version, wait until it is healthy, stop it, and then start the
|
||||
2.3.y version.
|
||||
|
||||
#### Rollback via additional tooling
|
||||
|
||||
Versions 3.0+ of etcd do not support general rollback. That is,
|
||||
after migrating from M.N to M.N+1, there is no way to go back to M.N.
|
||||
The etcd team has provided a [custom rollback tool](https://git.k8s.io/kubernetes/cluster/images/etcd/rollback)
|
||||
but the rollback tool has these limitations:
|
||||
|
||||
* This custom rollback tool is not part of the etcd repo and does not receive the same
|
||||
testing as the rest of etcd. We are testing it in a couple of end-to-end tests.
|
||||
There is only community support here.
|
||||
|
||||
* The rollback can be done only from the 3.0.x version (that is using the v3 API) to the
|
||||
2.2.1 version (that is using the v2 API).
|
||||
|
||||
* The tool only works if the data is stored in `application/json` format.
|
||||
|
||||
* Rollback doesn’t preserve resource versions of objects stored in etcd.
|
||||
|
||||
{{< warning >}}
|
||||
If the data is not kept in `application/json` format (see [Upgrade
|
||||
Procedure](#upgrade-procedure)), you will lose the option to roll back to etcd
|
||||
2.2.
|
||||
{{< /warning >}}
|
||||
|
||||
The last bullet means that any component or user that has some logic
|
||||
depending on resource versions may require restart after etcd rollback. This
|
||||
includes that all clients using the watch API, which depends on
|
||||
resource versions. Since both the kubelet and kube-proxy use the watch API, a
|
||||
rollback might require restarting all Kubernetes components on all nodes.
|
||||
|
||||
{{< note >}}
|
||||
At the time of writing, both Kubelet and KubeProxy are using “resource
|
||||
version” only for watching (i.e. are not using resource versions for anything
|
||||
else). And both are using reflector and/or informer frameworks for watching
|
||||
(i.e. they don’t send watch requests themselves). Both those frameworks if they
|
||||
can’t renew watch, they will start from “current version” by doing “list + watch
|
||||
from the resource version returned by list”. That means that if the apiserver
|
||||
will be down for the period of rollback, all of node components should basically
|
||||
restart their watches and start from “now” when apiserver is back. And it will
|
||||
be back with new resource version. That would mean that restarting node
|
||||
components is not needed. But the assumptions here may not hold forever.
|
||||
{{< /note >}}
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture discussion %}}
|
||||
|
||||
### Design
|
||||
|
||||
This section describes how we are going to do the migration, given the
|
||||
[etcd upgrade requirements](#etcd-upgrade-requirements).
|
||||
|
||||
Note that because the code changes in Kubernetes code needed
|
||||
to support the etcd v3 API are local and straightforward, we do not
|
||||
focus on them at all. We focus only on the upgrade/rollback here.
|
||||
|
||||
### New etcd Docker image
|
||||
|
||||
We decided to completely change the content of the etcd image and the way it works.
|
||||
So far, the Docker image for etcd in version X has contained only the etcd and
|
||||
etcdctl binaries.
|
||||
|
||||
Going forward, the Docker image for etcd in version X will contain multiple
|
||||
versions of etcd. For example, the 3.0.17 image will contain the 2.2.1, 2.3.7, and
|
||||
3.0.17 binaries of etcd and etcdctl. This will allow running etcd in multiple
|
||||
different versions using the same Docker image.
|
||||
|
||||
Additionally, the image will contain a custom script, written by the Kubernetes team,
|
||||
for doing migration between versions. The image will also contain the rollback tool
|
||||
provided by the etcd team.
|
||||
|
||||
### Migration script
|
||||
The migration script that will be part of the etcd Docker image is a bash
|
||||
script that works as follows:
|
||||
|
||||
1. Detect which version of etcd we were previously running.
|
||||
For that purpose, we have added a dedicated file, `version.txt`, that
|
||||
holds that information and is stored in the etcd-data-specific directory,
|
||||
next to the etcd data. If the file doesn’t exist, we default it to version 2.2.1.
|
||||
1. If we are in version 2.2.1 and are supposed to upgrade, backup
|
||||
data.
|
||||
1. Based on the detected previous etcd version and the desired one
|
||||
(communicated via environment variable), do the upgrade steps as
|
||||
needed. This means that for every minor etcd release greater than the detected one and
|
||||
less than or equal to the desired one:
|
||||
1. Start etcd in that version.
|
||||
1. Wait until it is healthy. Healthy means that you can write some data to it.
|
||||
1. Stop this etcd. Note that this etcd will not listen on the default
|
||||
etcd port. It is hard coded to listen on ports that the API server is not
|
||||
configured to connect to, which means that API server won’t be able to connect
|
||||
to it. Assuming no other client goes out of its way to try to
|
||||
connect and write to this obscure port, no new data will be written during
|
||||
this period.
|
||||
1. If the desired API version is v3 and the detected version is v2, do the offline
|
||||
migration from the v2 to v3 data format. For that we use two tools:
|
||||
* ./etcdctl migrate: This is the official tool for migration provided by the etcd team.
|
||||
* A custom script that is attaching TTLs to events in the etcd. Note that etcdctl
|
||||
migrate doesn’t support TTLs.
|
||||
1. After every successful step, update contents of the version file.
|
||||
This will protect us from the situation where something crashes in the
|
||||
meantime ,and the version file gets completely unsynchronized with the
|
||||
real data. Note that it is safe if the script crashes after the step is
|
||||
done and before the file is updated. This will only result in redoing one
|
||||
step in the next try.
|
||||
|
||||
All the previous steps are for the case where the detected version is less than or
|
||||
equal to the desired version. In the opposite case, that is for a rollback, the
|
||||
script works as follows:
|
||||
|
||||
1. Verify that the detected version is 3.0.x with the v3 API, and the
|
||||
desired version is 2.2.1 with the v2 API. We don’t support any other rollback.
|
||||
1. If so, we run the custom tool provided by etcd team to do the offline
|
||||
rollback. This tool reads the v3 formatted data and writes it back to disk
|
||||
in v2 format.
|
||||
1. Finally update the contents of the version file.
|
||||
|
||||
### Upgrade procedure
|
||||
Simply modify the command line in the etcd manifest to:
|
||||
|
||||
1. Run the migration script. If the previously run version is already in the
|
||||
desired version, this will be no-op.
|
||||
1. Start etcd in the desired version.
|
||||
|
||||
Starting in Kubernetes version 1.6, this has been done in the manifests for new
|
||||
Google Compute Engine clusters. You should also specify these environment
|
||||
variables. In particular, you must keep `STORAGE_MEDIA_TYPE` set to
|
||||
`application/json` if you wish to preserve the option to roll back.
|
||||
|
||||
```
|
||||
TARGET_STORAGE=etcd3
|
||||
ETCD_IMAGE=3.0.17
|
||||
TARGET_VERSION=3.0.17
|
||||
STORAGE_MEDIA_TYPE=application/json
|
||||
```
|
||||
|
||||
To roll back, use these:
|
||||
|
||||
```
|
||||
TARGET_STORAGE=etcd2
|
||||
ETCD_IMAGE=3.0.17
|
||||
TARGET_VERSION=2.2.1
|
||||
STORAGE_MEDIA_TYPE=application/json
|
||||
```
|
||||
|
||||
{{< note >}}
|
||||
This procedure upgrades from 2.x to 3.x. Version `3.0.17` is not recommended for running in production (see [prerequisites](#prerequisites) for minimum recommended etcd versions).
|
||||
{{< /note >}}
|
||||
|
||||
## Notes for etcd Version 2.2.1
|
||||
|
||||
### Default configuration
|
||||
|
||||
The default setup scripts use kubelet's file-based static pods feature to run etcd in a
|
||||
[pod](http://releases.k8s.io/{{< param "githubbranch" >}}/cluster/gce/manifests/etcd.manifest). This manifest should only
|
||||
be run on master VMs. The default location that kubelet scans for manifests is
|
||||
`/etc/kubernetes/manifests/`.
|
||||
|
||||
### Kubernetes's usage of etcd
|
||||
|
||||
By default, Kubernetes objects are stored under the `/registry` key in etcd.
|
||||
This path can be prefixed by using the [kube-apiserver](/docs/admin/kube-apiserver) flag
|
||||
`--etcd-prefix="/foo"`.
|
||||
|
||||
`etcd` is the only place that Kubernetes keeps state.
|
||||
|
||||
### Troubleshooting
|
||||
|
||||
To test whether `etcd` is running correctly, you can try writing a value to a
|
||||
test key. On your master VM (or somewhere with firewalls configured such that
|
||||
you can talk to your cluster's etcd), try:
|
||||
|
||||
```shell
|
||||
curl -X PUT "http://${host}:${port}/v2/keys/_test"
|
||||
```
|
||||
If your cluster was created via `kube-up.sh` and is still using etcd2 as its
|
||||
storage backend, please consult the [Kubernetes v1.12 etcd cluster upgrade docs](https://v1-12.docs.kubernetes.io/docs/tasks/administer-cluster/configure-upgrade-etcd/#upgrading-and-rolling-back-etcd-clusters)
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
@@ -42,6 +42,10 @@ during an upgrade. For example, here is what a `v1.11.0` upgrade would look like
|
||||
kubeadm upgrade apply v1.11.0 --feature-gates=CoreDNS=true
|
||||
```
|
||||
|
||||
In Kubernetes version 1.13 and later the `CoreDNS` feature gate is removed and CoreDNS
|
||||
is used by default. Follow the guide outlined [here](/docs/reference/setup-tools/kubeadm/kubeadm-init-phase#cmd-phase-addon) if you want
|
||||
your upgraded cluster to use kube-dns.
|
||||
|
||||
In versions prior to 1.11 the Corefile will be **overwritten** by the one created during upgrade.
|
||||
**You should save your existing ConfigMap if you have customized it.** You may re-apply your
|
||||
customizations after the new ConfigMap is up and running.
|
||||
@@ -56,13 +60,15 @@ In Kubernetes 1.11, CoreDNS has graduated to General Availability (GA)
|
||||
and is installed by default.
|
||||
{{< /note >}}
|
||||
|
||||
To install kube-dns instead, set the `CoreDNS` feature gate
|
||||
To install kube-dns on versions prior to 1.13, set the `CoreDNS` feature gate
|
||||
value to `false`:
|
||||
|
||||
```
|
||||
kubeadm init --feature-gates=CoreDNS=false
|
||||
```
|
||||
|
||||
For versions 1.13 and later, follow the guide outlined [here](/docs/reference/setup-tools/kubeadm/kubeadm-init-phase#cmd-phase-addon).
|
||||
|
||||
## Tuning CoreDNS
|
||||
|
||||
When resource utilisation is a concern, it may be useful to tune the configuration of CoreDNS. For more details, check out the
|
||||
|
||||
@@ -13,7 +13,7 @@ This page shows how to enable and configure encryption of secret data at rest.
|
||||
|
||||
* {{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
|
||||
|
||||
* Kubernetes version 1.7.0 or later is required
|
||||
* Kubernetes version 1.13.0 or later is required
|
||||
|
||||
* etcd v3 or later is required
|
||||
|
||||
@@ -23,15 +23,18 @@ This page shows how to enable and configure encryption of secret data at rest.
|
||||
|
||||
## Configuration and determining whether encryption at rest is already enabled
|
||||
|
||||
The `kube-apiserver` process accepts an argument `--experimental-encryption-provider-config`
|
||||
The `kube-apiserver` process accepts an argument `--encryption-provider-config`
|
||||
that controls how API data is encrypted in etcd. An example configuration
|
||||
is provided below.
|
||||
|
||||
Note:
|
||||
The alpha version of the encryption feature prior to 1.13 used the `--experimental-encryption-provider-config` flag.
|
||||
|
||||
## Understanding the encryption at rest configuration.
|
||||
|
||||
```yaml
|
||||
kind: EncryptionConfig
|
||||
apiVersion: v1
|
||||
kind: EncryptionConfiguration
|
||||
apiVersion: apiserver.config.k8s.io/v1
|
||||
resources:
|
||||
- resources:
|
||||
- secrets
|
||||
@@ -66,8 +69,12 @@ resources from storage each provider that matches the stored data attempts to de
|
||||
order. If no provider can read the stored data due to a mismatch in format or secret key, an error
|
||||
is returned which prevents clients from accessing that resource.
|
||||
|
||||
Note:
|
||||
The alpha version of the encryption feature prior to 1.13 required to be configured with
|
||||
`kind: EncryptionConfig` and `apiVersion: v1`.
|
||||
|
||||
{{< caution >}}
|
||||
If any resource is not readable via the encryption config (because keys were changed),
|
||||
**IMPORTANT:** If any resource is not readable via the encryption config (because keys were changed),
|
||||
the only recourse is to delete that key from the underlying etcd directly. Calls that attempt to
|
||||
read that resource will fail until it is deleted or a valid decryption key is provided.
|
||||
{{< /caution >}}
|
||||
@@ -90,8 +97,8 @@ is the first provider, the first key is used for encryption.
|
||||
Create a new encryption config file:
|
||||
|
||||
```yaml
|
||||
kind: EncryptionConfig
|
||||
apiVersion: v1
|
||||
kind: EncryptionConfiguration
|
||||
apiVersion: apiserver.config.k8s.io/v1
|
||||
resources:
|
||||
- resources:
|
||||
- secrets
|
||||
@@ -112,7 +119,7 @@ To create a new secret perform the following steps:
|
||||
```
|
||||
|
||||
2. Place that value in the secret field.
|
||||
3. Set the `--experimental-encryption-provider-config` flag on the `kube-apiserver` to point to the location of the config file.
|
||||
3. Set the `--encryption-provider-config` flag on the `kube-apiserver` to point to the location of the config file.
|
||||
4. Restart your API server.
|
||||
|
||||
{{< caution >}}
|
||||
@@ -183,8 +190,8 @@ With a single `kube-apiserver`, step 2 may be skipped.
|
||||
To disable encryption at rest place the `identity` provider as the first entry in the config:
|
||||
|
||||
```yaml
|
||||
kind: EncryptionConfig
|
||||
apiVersion: v1
|
||||
kind: EncryptionConfiguration
|
||||
apiVersion: apiserver.config.k8s.io/v1
|
||||
resources:
|
||||
- resources:
|
||||
- secrets
|
||||
|
||||
@@ -79,8 +79,8 @@ To encrypt the data:
|
||||
1. Create a new encryption configuration file using the appropriate properties for the `kms` provider:
|
||||
|
||||
```yaml
|
||||
kind: EncryptionConfig
|
||||
apiVersion: v1
|
||||
kind: EncryptionConfiguration
|
||||
apiVersion: apiserver.config.k8s.io/v1
|
||||
resources:
|
||||
- resources:
|
||||
- secrets
|
||||
@@ -92,9 +92,13 @@ resources:
|
||||
- identity: {}
|
||||
```
|
||||
|
||||
2. Set the `--experimental-encryption-provider-config` flag on the kube-apiserver to point to the location of the configuration file.
|
||||
2. Set the `--encryption-provider-config` flag on the kube-apiserver to point to the location of the configuration file.
|
||||
3. Restart your API server.
|
||||
|
||||
Note:
|
||||
The alpha version of the encryption feature prior to 1.13 required a config file with
|
||||
`kind: EncryptionConfig` and `apiVersion: v1`, and used the `--experimental-encryption-provider-config` flag.
|
||||
|
||||
## Verifying that the data is encrypted
|
||||
Data is encrypted when written to etcd. After restarting your kube-apiserver, any newly created or updated secret should be encrypted when stored. To verify, you can use the etcdctl command line program to retrieve the contents of your secret.
|
||||
|
||||
@@ -130,8 +134,8 @@ To switch from a local encryption provider to the `kms` provider and re-encrypt
|
||||
1. Add the `kms` provider as the first entry in the configuration file as shown in the following example.
|
||||
|
||||
```yaml
|
||||
kind: EncryptionConfig
|
||||
apiVersion: v1
|
||||
kind: EncryptionConfiguration
|
||||
apiVersion: apiserver.config.k8s.io/v1
|
||||
resources:
|
||||
- resources:
|
||||
- secrets
|
||||
@@ -160,8 +164,8 @@ To disable encryption at rest:
|
||||
1. Place the `identity` provider as the first entry in the configuration file:
|
||||
|
||||
```yaml
|
||||
kind: EncryptionConfig
|
||||
apiVersion: v1
|
||||
kind: EncryptionConfiguration
|
||||
apiVersion: apiserver.config.k8s.io/v1
|
||||
resources:
|
||||
- resources:
|
||||
- secrets
|
||||
|
||||
@@ -0,0 +1,124 @@
|
||||
---
|
||||
reviewers:
|
||||
- sig-cluster-lifecycle
|
||||
title: Certificate Management with kubeadm
|
||||
content_template: templates/task
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
This page explains how to manage certificates manually with kubeadm.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture prerequisites %}}
|
||||
|
||||
These are advanced topics for users who need to integrate their organization's certificate infrastructure into a kubeadm-built cluster. If kubeadm with the default configuration satisfies your needs, you should let kubeadm manage certificates instead.
|
||||
|
||||
You should be familiar with [PKI certificates and requirements in Kubernetes](/docs/setup/certificates/).
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture steps %}}
|
||||
|
||||
## Renew certificates with the certificates API
|
||||
|
||||
Kubeadm can renew certificates with the `kubeadm alpha certs renew` commands.
|
||||
|
||||
Typically this is done by loading on-disk CA certificates and keys and using them to issue new certificates.
|
||||
This approach works well if your certificate tree is self-contained. However, if your certificates are externally
|
||||
managed, you might need a different approach.
|
||||
|
||||
As an alternative, Kubernetes provides its own [API for managing certificates][manage-tls].
|
||||
With kubeadm, you can use this API by running `kubeadm alpha certs renew --use-api`.
|
||||
|
||||
## Set up a signer
|
||||
|
||||
The Kubernetes Certificate Authority does not work out of the box.
|
||||
You can configure an external signer such as [cert-manager][cert-manager-issuer], or you can use the build-in signer.
|
||||
The built-in signer is part of [`kube-controller-manager`][kcm].
|
||||
To activate the build-in signer, you pass the `--cluster-signing-cert-file` and `--cluster-signing-key-file` arguments.
|
||||
|
||||
You pass these arguments in any of the following ways:
|
||||
|
||||
* Edit `/etc/kubernetes/manifests/kube-controller-manager.yaml` to add the arguments to the command.
|
||||
Remember that your changes could be overwritten when you upgrade.
|
||||
|
||||
* If you're creating a new cluster, you can use a kubeadm [configuration file][config]:
|
||||
|
||||
```yaml
|
||||
apiVersion: kubeadm.k8s.io/v1beta1
|
||||
kind: ClusterConfiguration
|
||||
controllerManager:
|
||||
extraArgs:
|
||||
cluster-signing-cert-file: /etc/kubernetes/pki/ca.crt
|
||||
cluster-signing-key-file: /etc/kubernetes/pki/ca.key
|
||||
```
|
||||
|
||||
* You can also upload a config file using [`kubeadm config upload from-files`][config-upload]
|
||||
|
||||
[cert-manager-issuer]: https://cert-manager.readthedocs.io/en/latest/tutorials/ca/creating-ca-issuer.html
|
||||
[kcm]: https://kubernetes.io/docs/reference/command-line-tools-reference/kube-controller-manager/
|
||||
[config]: https://godoc.org/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta1
|
||||
[config-upload]: https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-config/#cmd-config-from-file
|
||||
|
||||
### Approve requests
|
||||
|
||||
If you set up an external signer such as [cert-manager][cert-manager], certificate signing requests (CSRs) are automatically approved.
|
||||
Otherwise, you must manually approve certificates with the [`kubectl certificates`][certs] command.
|
||||
The following kubeadm command outputs the name of the certificate to approve, then blocks and waits for approval to occur:
|
||||
|
||||
```shell
|
||||
$ sudo kubeadm alpha certs renew apiserver --use-api &
|
||||
[1] 2890
|
||||
[certs] certificate request "kubeadm-cert-kube-apiserver-ld526" created
|
||||
$ kubectl certificate approve kubeadm-cert-kube-apiserver-ld526
|
||||
certificatesigningrequest.certificates.k8s.io/kubeadm-cert-kube-apiserver-ld526 approved
|
||||
[1]+ Done sudo kubeadm alpha certs renew apiserver --use-api
|
||||
```
|
||||
|
||||
You can view a list of pending certificates with `kubectl get csr`.
|
||||
|
||||
[manage-tls]: https://kubernetes.io/docs/tasks/tls/managing-tls-in-a-cluster/
|
||||
[cert-manager]: https://github.com/jetstack/cert-manager
|
||||
[certs]: https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#certificate
|
||||
|
||||
## Certificate requests with kubeadm
|
||||
|
||||
To better integrate with external CAs, kubeadm can also produce certificate signing requests (CSRs).
|
||||
A CSR represents a request to a CA for a signed certificate for a client.
|
||||
In kubeadm terms, any certificate that would normally be signed by an on-disk CA can be produced as a CSR instead. A CA, however, cannot be produced as a CSR.
|
||||
|
||||
You can create an individual CSR with `kubeadm init phase certs apiserver --use-csr`.
|
||||
The `--use-csr` flag can be applied only to individual phases. After [all certificates are in place][certs], you can run `kubeadm init --external-ca`.
|
||||
|
||||
You can pass in a directory with `--csr-dir` to output the CSRs to the specified location.
|
||||
If `--csr-dire` is not specified, the default certificate directory (`/etc/kubernetes/pki`) is used.
|
||||
Both the CSR and the accompanying private key are given in the output. After a certificate is signed, the certificate and the private key must be copied to the PKI directory (by default `/etc/kubernetes/pki`).
|
||||
|
||||
### Renew certificates
|
||||
|
||||
Certificates can be renewed with `kubeadm alpha certs renew --use-csr`.
|
||||
As with `kubeadm init`, an output directory can be specified with the `--csr-dir` flag.
|
||||
To use the new certificates, copy the signed certificate and private key into the PKI directory (by default `/etc/kubernetes/pki`)
|
||||
|
||||
## Cert usage
|
||||
|
||||
A CSR contains a certificate's name, domains, and IPs, but it does not specify usages.
|
||||
It is the responsibility of the CA to specify [the correct cert usages][cert-table] when issuing a certificate.
|
||||
|
||||
* In `openssl` this is done with the [`openssl ca` command][openssl-ca].
|
||||
* In `cfssl` you specify [usages in the config file][cfssl-usages]
|
||||
|
||||
## CA selection
|
||||
|
||||
Kubeadm sets up [three CAs][cert-cas] by default. Make sure to sign the CSRs with a corresponding CA.
|
||||
|
||||
[openssl-ca]: https://superuser.com/questions/738612/openssl-ca-keyusage-extension
|
||||
[cfssl-usages]: https://github.com/cloudflare/cfssl/blob/master/doc/cmd/cfssl.txt#L170
|
||||
[certs]: https://kubernetes.io/docs/setup/certificates
|
||||
[cert-cas]: https://kubernetes.io/docs/setup/certificates/#single-root-ca
|
||||
[cert-table]: https://kubernetes.io/docs/setup/certificates/#all-certificates
|
||||
|
||||
{{% /capture %}}
|
||||
https://prow.k8s.io/?pull=71212
|
||||
@@ -0,0 +1,278 @@
|
||||
---
|
||||
reviewers:
|
||||
- sig-cluster-lifecycle
|
||||
title: Upgrading kubeadm clusters from v1.12 to v1.13
|
||||
content_template: templates/task
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
This page explains how to upgrade a Kubernetes cluster created with `kubeadm` from version 1.12.x to version 1.13.x, and from version 1.13.x to 1.13.y, where `y > x`.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture prerequisites %}}
|
||||
|
||||
- You need to have a `kubeadm` Kubernetes cluster running version 1.12.0 or later.
|
||||
[Swap must be disabled][swap].
|
||||
The cluster should use a static control plane and etcd pods.
|
||||
- Make sure you read the [release notes](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.13.md) carefully.
|
||||
- Make sure to back up any important components, such as app-level state stored in a database.
|
||||
`kubeadm upgrade` does not touch your workloads, only components internal to Kubernetes, but backups are always a best practice.
|
||||
|
||||
|
||||
[swap]: https://serverfault.com/questions/684771/best-way-to-disable-swap-in-linux
|
||||
### Additional information
|
||||
|
||||
- All containers are restarted after upgrade, because the container spec hash value is changed.
|
||||
- You can upgrade only from one minor version to the next minor version.
|
||||
That is, you cannot skip versions when you upgrade.
|
||||
For example, you can upgrade only from 1.10 to 1.11, not from 1.9 to 1.11.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture steps %}}
|
||||
|
||||
## Upgrade the control plane
|
||||
|
||||
1. On your master node, upgrade kubeadm:
|
||||
|
||||
{{< tabs name="k8s_install" >}}
|
||||
{{% tab name="Ubuntu, Debian or HypriotOS" %}}
|
||||
apt-get update
|
||||
apt-get upgrade -y kubelet kubeadm
|
||||
{{% /tab %}}
|
||||
{{% tab name="CentOS, RHEL or Fedora" %}}
|
||||
yum upgrade -y kubeadm --disableexcludes=kubernetes
|
||||
{{% /tab %}}
|
||||
{{< /tabs >}}
|
||||
|
||||
1. Verify that the download works and has the expected version:
|
||||
|
||||
```shell
|
||||
kubeadm version
|
||||
```
|
||||
|
||||
1. On the master node, run:
|
||||
|
||||
```shell
|
||||
kubeadm upgrade plan
|
||||
```
|
||||
|
||||
You should see output similar to this:
|
||||
|
||||
```shell
|
||||
[preflight] Running pre-flight checks.
|
||||
[upgrade] Making sure the cluster is healthy:
|
||||
[upgrade/config] Making sure the configuration is correct:
|
||||
[upgrade/config] Reading configuration from the cluster...
|
||||
[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
|
||||
[upgrade] Fetching available versions to upgrade to
|
||||
[upgrade/versions] Cluster version: v1.12.2
|
||||
[upgrade/versions] kubeadm version: v1.13.0
|
||||
|
||||
Components that must be upgraded manually after you have upgraded the control plane with 'kubeadm upgrade apply':
|
||||
COMPONENT CURRENT AVAILABLE
|
||||
Kubelet 2 x v1.12.2 v1.13.0
|
||||
|
||||
Upgrade to the latest version in the v1.12 series:
|
||||
|
||||
COMPONENT CURRENT AVAILABLE
|
||||
API Server v1.12.2 v1.13.0
|
||||
Controller Manager v1.12.2 v1.13.0
|
||||
Scheduler v1.12.2 v1.13.0
|
||||
Kube Proxy v1.12.2 v1.13.0
|
||||
CoreDNS 1.2.2 1.2.6
|
||||
Etcd 3.2.24 3.2.24
|
||||
|
||||
You can now apply the upgrade by executing the following command:
|
||||
|
||||
kubeadm upgrade apply v1.13.0
|
||||
|
||||
_____________________________________________________________________
|
||||
```
|
||||
|
||||
This command checks that your cluster can be upgraded, and fetches the versions you can upgrade to.
|
||||
|
||||
1. Choose a version to upgrade to, and run the appropriate command. For example:
|
||||
|
||||
```shell
|
||||
kubeadm upgrade apply v1.13.0
|
||||
```
|
||||
|
||||
You should see output similar to this:
|
||||
|
||||
<!-- TODO: output from stable -->
|
||||
|
||||
```shell
|
||||
[preflight] Running pre-flight checks.
|
||||
[upgrade] Making sure the cluster is healthy:
|
||||
[upgrade/config] Making sure the configuration is correct:
|
||||
[upgrade/config] Reading configuration from the cluster...
|
||||
[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
|
||||
[upgrade/apply] Respecting the --cri-socket flag that is set with higher priority than the config file.
|
||||
[upgrade/version] You have chosen to change the cluster version to "v1.13.0"
|
||||
[upgrade/versions] Cluster version: v1.12.2
|
||||
[upgrade/versions] kubeadm version: v1.13.0
|
||||
[upgrade/confirm] Are you sure you want to proceed with the upgrade? [y/N]: y
|
||||
[upgrade/prepull] Will prepull images for components [kube-apiserver kube-controller-manager kube-scheduler etcd]
|
||||
[upgrade/prepull] Prepulling image for component etcd.
|
||||
[upgrade/prepull] Prepulling image for component kube-controller-manager.
|
||||
[upgrade/prepull] Prepulling image for component kube-scheduler.
|
||||
[upgrade/prepull] Prepulling image for component kube-apiserver.
|
||||
[apiclient] Found 0 Pods for label selector k8s-app=upgrade-prepull-kube-controller-manager
|
||||
[apiclient] Found 0 Pods for label selector k8s-app=upgrade-prepull-etcd
|
||||
[apiclient] Found 0 Pods for label selector k8s-app=upgrade-prepull-kube-scheduler
|
||||
[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-apiserver
|
||||
[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-controller-manager
|
||||
[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-etcd
|
||||
[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-scheduler
|
||||
[upgrade/prepull] Prepulled image for component etcd.
|
||||
[upgrade/prepull] Prepulled image for component kube-apiserver.
|
||||
[upgrade/prepull] Prepulled image for component kube-scheduler.
|
||||
[upgrade/prepull] Prepulled image for component kube-controller-manager.
|
||||
[upgrade/prepull] Successfully prepulled the images for all the control plane components
|
||||
[upgrade/apply] Upgrading your Static Pod-hosted control plane to version "v1.13.0"...
|
||||
Static pod: kube-apiserver-ip-10-0-0-7 hash: 4af3463d6ace12615f1795e40811c1a1
|
||||
Static pod: kube-controller-manager-ip-10-0-0-7 hash: a640b0098f5bddc701786e007c96e220
|
||||
Static pod: kube-scheduler-ip-10-0-0-7 hash: ee7b1077c61516320f4273309e9b4690
|
||||
map[localhost:2379:3.2.24]
|
||||
[upgrade/staticpods] Writing new Static Pod manifests to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests969681047"
|
||||
[upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-apiserver.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests-2018-11-20-18-30-42/kube-apiserver.yaml"
|
||||
[upgrade/staticpods] Waiting for the kubelet to restart the component
|
||||
[upgrade/staticpods] This might take a minute or longer depending on the component/version gap (timeout 5m0s)
|
||||
Static pod: kube-apiserver-ip-10-0-0-7 hash: 4af3463d6ace12615f1795e40811c1a1
|
||||
Static pod: kube-apiserver-ip-10-0-0-7 hash: bf5b045d2be93e73654f3eb7027a4ef8
|
||||
[apiclient] Found 1 Pods for label selector component=kube-apiserver
|
||||
[upgrade/staticpods] Component "kube-apiserver" upgraded successfully!
|
||||
[upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-controller-manager.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests-2018-11-20-18-30-42/kube-controller-manager.yaml"
|
||||
[upgrade/staticpods] Waiting for the kubelet to restart the component
|
||||
[upgrade/staticpods] This might take a minute or longer depending on the component/version gap (timeout 5m0s)
|
||||
Static pod: kube-controller-manager-ip-10-0-0-7 hash: a640b0098f5bddc701786e007c96e220
|
||||
Static pod: kube-controller-manager-ip-10-0-0-7 hash: 1e0eea23b3d971460ac032c18ab7daac
|
||||
[apiclient] Found 1 Pods for label selector component=kube-controller-manager
|
||||
[upgrade/staticpods] Component "kube-controller-manager" upgraded successfully!
|
||||
[upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-scheduler.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests-2018-11-20-18-30-42/kube-scheduler.yaml"
|
||||
[upgrade/staticpods] Waiting for the kubelet to restart the component
|
||||
[upgrade/staticpods] This might take a minute or longer depending on the component/version gap (timeout 5m0s)
|
||||
Static pod: kube-scheduler-ip-10-0-0-7 hash: ee7b1077c61516320f4273309e9b4690
|
||||
Static pod: kube-scheduler-ip-10-0-0-7 hash: 7f7d929b61a2cc5bcdf36609f75927ec
|
||||
[apiclient] Found 1 Pods for label selector component=kube-scheduler
|
||||
[upgrade/staticpods] Component "kube-scheduler" upgraded successfully!
|
||||
[uploadconfig] storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
|
||||
[kubelet] Creating a ConfigMap "kubelet-config-1.13" in namespace kube-system with the configuration for the kubelets in the cluster
|
||||
[kubelet] Downloading configuration for the kubelet from the "kubelet-config-1.13" ConfigMap in the kube-system namespace
|
||||
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
|
||||
[patchnode] Uploading the CRI Socket information "/var/run/dockershim.sock" to the Node API object "ip-10-0-0-7" as an annotation
|
||||
[bootstraptoken] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
|
||||
[bootstraptoken] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
|
||||
[bootstraptoken] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
|
||||
[addons] Applied essential addon: CoreDNS
|
||||
[addons] Applied essential addon: kube-proxy
|
||||
|
||||
[upgrade/successful] SUCCESS! Your cluster was upgraded to "v1.13.0". Enjoy!
|
||||
|
||||
[upgrade/kubelet] Now that your control plane is upgraded, please proceed with upgrading your kubelets if you haven't already done so.
|
||||
```
|
||||
|
||||
1. Manually upgrade your Software Defined Network (SDN).
|
||||
|
||||
Your Container Network Interface (CNI) provider may have its own upgrade instructions to follow.
|
||||
Check the [addons](/docs/concepts/cluster-administration/addons/) page to
|
||||
find your CNI provider and see whether additional upgrade steps are required.
|
||||
|
||||
## Upgrade master and node packages
|
||||
|
||||
1. Prepare each node for maintenance by marking it unschedulable and evicting the workloads. Run:
|
||||
|
||||
```shell
|
||||
kubectl drain $NODE --ignore-daemonsets
|
||||
```
|
||||
|
||||
On the master node, you must add `--ignore-daemonsets`:
|
||||
|
||||
```shell
|
||||
kubectl drain ip-172-31-85-18
|
||||
node "ip-172-31-85-18" cordoned
|
||||
error: unable to drain node "ip-172-31-85-18", aborting command...
|
||||
|
||||
There are pending nodes to be drained:
|
||||
ip-172-31-85-18
|
||||
error: DaemonSet-managed pods (use --ignore-daemonsets to ignore): calico-node-5798d, kube-proxy-thjp9
|
||||
```
|
||||
|
||||
```
|
||||
kubectl drain ip-172-31-85-18 --ignore-daemonsets
|
||||
node "ip-172-31-85-18" already cordoned
|
||||
WARNING: Ignoring DaemonSet-managed pods: calico-node-5798d, kube-proxy-thjp9
|
||||
node "ip-172-31-85-18" drained
|
||||
```
|
||||
|
||||
1. Upgrade the Kubernetes package version on each `$NODE` node by running the Linux package manager for your distribution:
|
||||
|
||||
{{< tabs name="k8s_install" >}}
|
||||
{{% tab name="Ubuntu, Debian or HypriotOS" %}}
|
||||
apt-get update
|
||||
apt-get upgrade -y kubelet kubeadm
|
||||
{{% /tab %}}
|
||||
{{% tab name="CentOS, RHEL or Fedora" %}}
|
||||
yum upgrade -y kubelet kubeadm --disableexcludes=kubernetes
|
||||
{{% /tab %}}
|
||||
{{< /tabs >}}
|
||||
|
||||
## Upgrade kubelet on each node
|
||||
|
||||
1. On each node except the master node, upgrade the kubelet config:
|
||||
|
||||
```shell
|
||||
kubeadm upgrade node config --kubelet-version $(kubelet --version | cut -d ' ' -f 2)
|
||||
```
|
||||
|
||||
1. Restart the kubelet process:
|
||||
|
||||
```shell
|
||||
systemctl restart kubelet
|
||||
```
|
||||
|
||||
1. Verify that the new version of the `kubelet` is running on the node:
|
||||
|
||||
```shell
|
||||
systemctl status kubelet
|
||||
```
|
||||
|
||||
1. Bring the node back online by marking it schedulable:
|
||||
|
||||
```shell
|
||||
kubectl uncordon $NODE
|
||||
```
|
||||
|
||||
1. After the kubelet is upgraded on all nodes, verify that all nodes are available again by running the following command from anywhere kubectl can access the cluster:
|
||||
|
||||
```shell
|
||||
kubectl get nodes
|
||||
```
|
||||
|
||||
The `STATUS` column should show `Ready` for all your nodes, and the version number should be updated.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
## Recovering from a failure state
|
||||
|
||||
If `kubeadm upgrade` fails and does not roll back, for example because of an unexpected shutdown during execution, you can run `kubeadm upgrade` again.
|
||||
This command is idempotent and eventually makes sure that the actual state is the desired state you declare.
|
||||
|
||||
To recover from a bad state, you can also run `kubeadm upgrade --force` without changing the version that your cluster is running.
|
||||
|
||||
## How it works
|
||||
|
||||
`kubeadm upgrade apply` does the following:
|
||||
|
||||
- Checks that your cluster is in an upgradeable state:
|
||||
- The API server is reachable
|
||||
- All nodes are in the `Ready` state
|
||||
- The control plane is healthy
|
||||
- Enforces the version skew policies.
|
||||
- Makes sure the control plane images are available or available to pull to the machine.
|
||||
- Upgrades the control plane components or rollbacks if any of them fails to come up.
|
||||
- Applies the new `kube-dns` and `kube-proxy` manifests and makes sure that all necessary RBAC rules are created.
|
||||
- Creates new certificate and key files of the API server and backs up old files if they're about to expire in 180 days.
|
||||
@@ -113,7 +113,7 @@ You should see something like the following:
|
||||
|
||||
[upgrade/successful] SUCCESS! Your cluster was upgraded to "v1.12.0". Enjoy!
|
||||
|
||||
The `kubeadm-config` ConfigMap is now updated from `v1alpha2` version to `v1alpha3`.
|
||||
The `kubeadm-config` ConfigMap is now updated from `v1alpha3` version to `v1beta1`.
|
||||
|
||||
### Upgrading additional control plane nodes
|
||||
|
||||
|
||||
@@ -0,0 +1,168 @@
|
||||
---
|
||||
reviewers:
|
||||
- luxas
|
||||
- timothysc
|
||||
- jbeda
|
||||
title: Upgrading kubeadm HA clusters from v1.12 to v1.13
|
||||
content_template: templates/task
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
This page explains how to upgrade a highly available (HA) Kubernetes cluster created with `kubeadm` from version 1.12.x to version 1.13.y. In addition to upgrading, you must also follow the instructions in [Creating HA clusters with kubeadm](/docs/setup/independent/high-availability/).
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture prerequisites %}}
|
||||
|
||||
Before proceeding:
|
||||
|
||||
- You need to have a `kubeadm` HA cluster running version 1.12 or higher.
|
||||
- Make sure you read the [release notes](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.13.md) carefully.
|
||||
- Make sure to back up any important components, such as app-level state stored in a database. `kubeadm upgrade` does not touch your workloads, only components internal to Kubernetes, but backups are always a best practice.
|
||||
- Check the prerequisites for [Upgrading/downgrading kubeadm clusters between v1.12 to v1.13](/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade-1-13/).
|
||||
|
||||
{{< note >}}
|
||||
All commands on any control plane or etcd node should be run as root.
|
||||
{{< /note >}}
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture steps %}}
|
||||
|
||||
## Prepare for both methods
|
||||
|
||||
Upgrade `kubeadm` to the version that matches the version of Kubernetes that you are upgrading to:
|
||||
|
||||
```shell
|
||||
apt-mark unhold kubeadm && \
|
||||
apt-get update && apt-get install -y kubeadm && \
|
||||
apt-mark hold kubeadm
|
||||
```
|
||||
|
||||
Check prerequisites and determine the upgrade versions:
|
||||
|
||||
```shell
|
||||
kubeadm upgrade plan
|
||||
```
|
||||
|
||||
You should see something like the following:
|
||||
|
||||
```
|
||||
Upgrade to the latest version in the v1.13 series:
|
||||
|
||||
COMPONENT CURRENT AVAILABLE
|
||||
API Server v1.12.2 v1.13.0
|
||||
Controller Manager v1.12.2 v1.13.0
|
||||
Scheduler v1.12.2 v1.13.0
|
||||
Kube Proxy v1.12.2 v1.13.0
|
||||
CoreDNS 1.2.2 1.2.6
|
||||
```
|
||||
|
||||
## Stacked control plane nodes
|
||||
|
||||
### Upgrade the first control plane node
|
||||
|
||||
Modify `configmap/kubeadm-config` for this control plane node:
|
||||
|
||||
```shell
|
||||
kubectl edit configmap -n kube-system kubeadm-config
|
||||
```
|
||||
|
||||
Make the following modifications to the ClusterConfiguration key:
|
||||
|
||||
- `etcd`
|
||||
|
||||
Remove the etcd section completely
|
||||
|
||||
Make the following modifications to the ClusterStatus key:
|
||||
|
||||
- `apiEndpoints`
|
||||
|
||||
Add an entry for each of the additional control plane hosts
|
||||
|
||||
Start the upgrade:
|
||||
|
||||
```shell
|
||||
kubeadm upgrade apply v<YOUR-CHOSEN-VERSION-HERE>
|
||||
```
|
||||
|
||||
You should see something like the following:
|
||||
|
||||
[upgrade/successful] SUCCESS! Your cluster was upgraded to "v1.13.0". Enjoy!
|
||||
|
||||
The `kubeadm-config` ConfigMap is now updated from `v1alpha3` version to `v1beta1`.
|
||||
|
||||
### Upgrading additional control plane nodes
|
||||
|
||||
Start the upgrade:
|
||||
|
||||
```shell
|
||||
kubeadm upgrade node experimental-control-plane
|
||||
```
|
||||
|
||||
## External etcd
|
||||
|
||||
### Upgrade the first control plane
|
||||
|
||||
Run the upgrade:
|
||||
|
||||
```
|
||||
kubeadm upgrade apply v1.13.0
|
||||
```
|
||||
|
||||
### Upgrade the other control plane nodes
|
||||
|
||||
For other control plane nodes in the cluster, run the following command:
|
||||
|
||||
```
|
||||
kubeadm upgrade node experimental-control-plane
|
||||
```
|
||||
|
||||
## Next steps
|
||||
|
||||
### Manually upgrade your CNI provider
|
||||
|
||||
Your Container Network Interface (CNI) provider might have its own upgrade instructions to follow. Check the [addons](/docs/concepts/cluster-administration/addons/) page to find your CNI provider and see whether you need to take additional upgrade steps.
|
||||
|
||||
### Update kubelet and kubectl packages
|
||||
|
||||
Upgrade the kubelet and kubectl by running the following on each node:
|
||||
|
||||
```shell
|
||||
# use your distro's package manager, e.g. 'apt-get' on Debian-based systems
|
||||
# for the versions stick to kubeadm's output (see above)
|
||||
apt-mark unhold kubelet kubectl && \
|
||||
apt-get update && \
|
||||
apt-get install kubelet=<NEW-K8S-VERSION> kubectl=<NEW-K8S-VERSION> && \
|
||||
apt-mark hold kubelet kubectl && \
|
||||
systemctl restart kubelet
|
||||
```
|
||||
|
||||
In this example a _deb_-based system is assumed and `apt-get` is used for installing the upgraded software. On rpm-based systems the command is `yum install <PACKAGE>=<NEW-K8S-VERSION>` for all packages.
|
||||
|
||||
Verify that the new version of the kubelet is running:
|
||||
|
||||
```shell
|
||||
systemctl status kubelet
|
||||
```
|
||||
|
||||
Verify that the upgraded node is available again by running the following command from wherever you run `kubectl`:
|
||||
|
||||
```shell
|
||||
kubectl get nodes
|
||||
```
|
||||
|
||||
If the `STATUS` column shows `Ready` for the upgraded host, you can continue. You might need to repeat the command until the node shows `Ready`.
|
||||
|
||||
## If something goes wrong
|
||||
|
||||
If the upgrade fails, see whether one of the following scenarios applies:
|
||||
|
||||
- If `kubeadm upgrade apply` failed to upgrade the cluster, it will try to perform a rollback. If this is the case on the first master, the cluster is probably still intact.
|
||||
|
||||
You can run `kubeadm upgrade apply` again, because it is idempotent and should eventually make sure the actual state is the desired state you are declaring. You can run `kubeadm upgrade apply` to change a running cluster with `x.x.x --> x.x.x` with `--force` to recover from a bad state.
|
||||
|
||||
- If `kubeadm upgrade apply` on one of the secondary masters failed, the cluster is upgraded and working, but the secondary masters are in an undefined state. You need to investigate further and join the secondaries manually.
|
||||
|
||||
{{% /capture %}}
|
||||
@@ -96,6 +96,7 @@ Audit backends persist audit events to an external storage.
|
||||
|
||||
- Log backend, which writes events to a disk
|
||||
- Webhook backend, which sends events to an external API
|
||||
- Dynamic backend, which configures webhook backends through an AuditSink API object.
|
||||
|
||||
In both cases, audit events structure is defined by the API in the
|
||||
`audit.k8s.io` API group. The current version of the API is
|
||||
@@ -146,6 +147,8 @@ audit backend using the following kube-apiserver flags:
|
||||
The webhook config file uses the kubeconfig format to specify the remote address of
|
||||
the service and credentials used to connect to it.
|
||||
|
||||
In v1.13 webhook backends can be configured [dynamically](#dynamic-backend).
|
||||
|
||||
### Batching
|
||||
|
||||
Both log and webhook backends support batching. Using webhook as an example, here's the list of
|
||||
@@ -156,6 +159,7 @@ throttling is enabled in `webhook` and disabled in `log`.
|
||||
- `--audit-webhook-mode` defines the buffering strategy. One of the following:
|
||||
- `batch` - buffer events and asynchronously process them in batches. This is the default.
|
||||
- `blocking` - block API server responses on processing each individual event.
|
||||
- `blocking-strict` - Same as blocking, but when there is a failure during audit logging at RequestReceived stage, the whole request to apiserver will fail.
|
||||
|
||||
The following flags are used only in the `batch` mode.
|
||||
|
||||
@@ -199,6 +203,56 @@ available for the log backend:
|
||||
|
||||
By default truncate is disabled in both `webhook` and `log`, a cluster administrator should set `audit-log-truncate-enabled` or `audit-webhook-truncate-enabled` to enable the feature.
|
||||
|
||||
### Dynamic backend
|
||||
|
||||
{{< feature-state for_k8s_version="v1.13" state="alpha" >}}
|
||||
|
||||
In Kubernetes version 1.13, you can configure dynamic audit webhook backends AuditSink API objects.
|
||||
|
||||
To enable dynamic auditing you must set the following apiserver flags:
|
||||
|
||||
- `--audit-dynamic-configuration`: the primary switch. When the feature is at GA, the only required flag.
|
||||
- `--feature-gates=DynamicAuditing=true`: feature gate at alpha and beta.
|
||||
- `--runtime-config=auditregistration.k8s.io/v1alpha1=true`: enable API.
|
||||
|
||||
When enabled, an AuditSink object can be provisioned:
|
||||
|
||||
```yaml
|
||||
apiVersion: auditregistration.k8s.io/v1alpha1
|
||||
kind: AuditSink
|
||||
metadata:
|
||||
name: mysink
|
||||
spec:
|
||||
policy:
|
||||
level: Metadata
|
||||
stages:
|
||||
- ResponseComplete
|
||||
webhook:
|
||||
throttle:
|
||||
qps: 10
|
||||
burst: 15
|
||||
clientConfig:
|
||||
url: "https://audit.app"
|
||||
```
|
||||
|
||||
For the complete API definition, see [AuditSink](/docs/reference/generated/kubernetes-api/v1.13/#auditsink-v1alpha1-auditregistration-k8s-io). Multiple objects will exist as independent solutions.
|
||||
|
||||
Existing static backends that you configure with runtime flags are not affected by this feature. However, the dynamic backends share the truncate options of the static webhook. If webhook truncate options are set with runtime flags, they are applied to all dynamic backends.
|
||||
|
||||
#### Policy
|
||||
|
||||
The AuditSink policy differs from the legacy audit runtime policy. This is because the API object serves different use cases. The policy will continue to evolve to serve more use cases.
|
||||
|
||||
The `level` field applies the given audit level to all requests. The `stages` field is now a whitelist of stages to record.
|
||||
|
||||
#### Security
|
||||
|
||||
Administrators should be aware that allowing write access to this feature grants read access to all cluster data. Access should be treated as a `cluster-admin` level privilege.
|
||||
|
||||
#### Performance
|
||||
|
||||
Currently, this feature has performance implications for the apiserver in the form of increased cpu and memory usage. This should be nominal for a small number of sinks, and performance impact testing will be done to understand its scope before the API progresses to beta.
|
||||
|
||||
## Multi-cluster setup
|
||||
|
||||
If you're extending the Kubernetes API with the [aggregation layer][kube-aggregator], you can also
|
||||
|
||||
@@ -9,7 +9,7 @@ content_template: templates/task
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
{{< feature-state state="alpha" >}}
|
||||
{{< feature-state state="beta" >}}
|
||||
|
||||
This guide demonstrates how to install and write extensions for [kubectl](/docs/reference/kubectl/kubectl/). By thinking of core `kubectl` commands as essential building blocks for interacting with a Kubernetes cluster, a cluster administrator can think
|
||||
of plugins as a means of utilizing these building blocks to create more complex behavior. Plugins extend `kubectl` with new sub-commands, allowing for new and custom features not included in the main distribution of `kubectl`.
|
||||
@@ -279,7 +279,7 @@ See the [Sample CLI Plugin](https://github.com/kubernetes/sample-cli-plugin) for
|
||||
|
||||
* Check the Sample CLI Plugin repository for [a detailed example](https://github.com/kubernetes/sample-cli-plugin) of a plugin written in Go.
|
||||
* In case of any questions, feel free to reach out to the [CLI SIG team](https://github.com/kubernetes/community/tree/master/sig-cli).
|
||||
* Binary plugins is still an alpha feature, so this is the time to contribute ideas and improvements to the codebase. We're also excited to hear about what you're planning to implement with plugins, so [let us know](https://github.com/kubernetes/community/tree/master/sig-cli)!
|
||||
* Binary plugins are a beta feature, so this is the time to contribute ideas and improvements to the codebase. We're also excited to hear about what you're planning to implement with plugins, so [let us know](https://github.com/kubernetes/community/tree/master/sig-cli)!
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user