Official 1.14 Release Docs (#13174)

* Official documentation on Poseidon/Firmament, a new multi-scheduler support for K8S. (#11752)

* Added documentation about Poseidon-Firmament scheduler

* Fixed some style issues.

* Udpated the document as per the review comments.

* Fixed some typos and updated the document

* Updated the document as per the review comments.

* Document timeout attribute for kms-plugin. (#12158)

See 72540.

* Official documentation on Poseidon/Firmament, a new multi-scheduler  (#12343)

* Removed the old version of the Poseidon documentation. Incorrect location.

* Official documentation on Poseidon/Firmament, a new multi-scheduler support for K8S (#12069)

* Official documentation on Poseidon/Firmament, a new multi-scheduler support for K8S. (#11752)

* Added documentation about Poseidon-Firmament scheduler

* Fixed some style issues.

* Udpated the document as per the review comments.

* Fixed some typos and updated the document

* Updated the document as per the review comments.

* Updated the document as per review comments. Added config details.

* Updated the document as per the latest review comments. Fixed nits

* Made changes as per latest suggestions.

* Some more changes added.

* Updated as per suggestions.

* Changed the release process section.

* SIG Docs edits

Small edits to match style guidelines.

* add plus to feature state

* capitalization

* revert feature state shortcode

since this is a Kubernetes extension, not a direct feature, it shouldn't use the regular feature state tagging.

(cherry picked from commit 7730c1540b)

* Remove initializers from doc. It will be removed in 1.14 (#12331)

* kubeadm: Document CRI auto detection functionality (#12462)

Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>

* Minor doc change for GAing Pod DNS Config (#12514)

* Graduate ExpandInUsePersistentVolumes feature to beta (#10574)

* Rename 2018-11-07-grpc-load-balancing-with-linkerd.md.md file (#12594)

* Add dynamic percentage of node scoring to user docs (#12235)

* Add dynamic percentage of node scoring to user docs

* addressed review comments

* delete special symbol (#12445)

* Update documentation for VolumeSubpathEnvExpansion (#11843)

* Update documentation for VolumeSubpathEnvExpansion

* Address comments - improve descriptions

* Graduate Pod Priority and Preemption to GA (#12428)

* Added Instana links to the documentation (#12977)

* Added link to the Instana Kubernetes integration

* Added Instana link for services section

Added Instana and a link to the Kubernetes integration to the analytics services section and broadened the scope to APM, monitoring and analytics.

* Oxford comma /flex

* More Oxford commas, because they matter

* Update kubectl plugins to stable (#12847)

* documentation for CSI topology beta (#12889)

* Document changes to default RBAC discovery ClusterRole(Binding)s (#12888)

* Document changes to default RBAC discovery ClusterRole(Binding)s

Documentation for https://github.com/kubernetes/enhancements/issues/789 and https://github.com/kubernetes/kubernetes/pull/73807

* documentation review feedback

* CSI raw block to beta (#12931)

* Change incorrect string raw to block (#12926)

Fixes #12925

* Update documentation on node OS/arch labels (#12976)

These labels have been promoted to GA:
https://github.com/kubernetes/enhancements/issues/793

* local pv GA doc updates (#12915)

* Publish CRD OpenAPI Documentation (#12910)

* add documentation for CustomResourcePublishOpenAPI

* address comments

fix links, ordered lists, style and typo

* kubeadm: add document for upgrading from 1.13 to 1.14 (single CP and HA) (#13189)

* kubeadm: add document for upgrading from 1.13 to 1.14

- remove doc for upgrading 1.10 -> 1.11

* kubeadm: apply amends to upgrade-1.14 doc

* kubeadm: apply amends to upgrade-1.14 doc (part2)

* kubeadm: apply amends to upgrade-1.14 doc (part3)

* kubeadm: add note about "upgrade node experimental-control-plane"

+ add comment about `upgrade plan`

* kubeadm: add missing "You should see output similar to this"

* fix bullet indentation (#13214)

* mark PodReadinessGate GA (#12800)

* Update RuntimeClass documentation for beta (#13043)

* Update RuntimeClass documentation for beta

* Update feature gate & add upgrade section

* formatting fixes

* Highlight upgrade action required

* Address feedback

* CSI ephemeral volume alpha documentation (#10934)

* update kubectl documentation (#12867)

* update kubectl documentation

* add document for Secret/ConfigMap generators

* replace `kubectl create -f` by `kubectl apply -f`

* Add page for kustomization support in kubectl

* fix spelling errors and address comments

* Documentation for Windows GMSA feature (#12936)

* Documentation for Windows GMSA feature

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Enhancements to GMSA docs

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Fix links

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Fix GMSA link

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Add GMSA feature flag in feature flag list

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Relocate GMSA to container configuration

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Add example for container spec

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Remove changes in Windows index

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Update configure-gmsa.md

* Update configure-gmsa.md

* Update configure-gmsa.md

* Update configure-gmsa.md

* Rearrange the steps into two sections and other edits

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Fix links

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Add reference to script to generate GMSA YAMLs

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Some more clarifications for GMSA

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* HugePages graduated to GA (#13004)

* HugePages graduated to GA

* fixing nit for build

* Docs for node PID limiting (https://github.com/kubernetes/kubernetes/pull/73651) (#12932)

* kubeadm: update the reference documentation for 1.14 (#12911)

* kubeadm: update list of generated files for 1.14

NOTE: PLACEHOLDERS! these files are generated by SIG Docs each
release, but we need them to pass the k/website PR CI.

- add join_phase* (new sub phases of join)
- add init_phase_upload-certs.md (new upload certs phase for init)
- remove alpha-preflight (now both init and join have this)

* kubeadm: update reference docs includes for 1.14

- remove includes from alpha.md
- add upload-certs to init-phase.md
- add join-phase.md and it's phases

* kubeadm: update the editorial content of join and init

- cleanup master->control-plane node
- add some notes about phases and join
- remove table about pre-pulling images
- remove outdated info about self-hosting

* kubeadm: update target release for v1alpha3 removal

1.14 -> 1.15

* kubeadm: copy edits for 1.14 reference docs (part1)

* kubeadm: use "shell" for code blocks

* kubeadm: update the 1.14 HA guide (#13191)

* kubeadm: update the 1.14 HA guide

* kubeadm: try to fix note/caution indent in HA page

* kubeadm: fix missing sudo and minor amends in HA doc

* kubeadm: apply latest amends to the HA doc for 1.14

* fixed a few missed merge conflicts

* Admission Webhook new features doc (#12938)

- kubernetes/kubernetes#74998
- kubernetes/kubernetes#74477
- kubernetes/kubernetes#74562

* Clarifications and fixes in GMSA doc (#13226)

* Clarifications and fixes in GMSA doc

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Update configure-gmsa.md

* Reformat to align headings and pre-reqs better

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Reformat to align headings and pre-reqs better

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Reformat to fix bullets

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Reword application of sample gmsa

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Update configure-gmsa.md

* Address feedback to use active voice

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Address feedback to use active voice

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* RunAsGroup documentation for Progressing this to Beta (#12297)

* start serverside-apply documentation (#13077)

* start serverside-apply documentation

* add more concept info on server side apply

* Update api concepts

* Update api-concepts.md

* fix style issues

* Document CSI update (#12928)

* Document CSI update

* Finish CSI documentation

Also fix mistake with  ExpandInUsePersistentVolumes documented as beta

* Overall docs for CSI Migration feature (#12935)

* Placeholder docs for CSI Migration feature

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Address CR comments and update feature gates

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Add mappings for CSI plugins

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Add sections for AWS and GCE PD migration

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Add docs for Cinder and CSI Migration info

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Clarify scope to volumes with file system

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Change the format of EBS and Cinder CSI Migration sections to follow the GCE template

Signed-off-by: Deep Debroy <ddebroy@docker.com>

* Windows documentation updates for 1.14 (#12929)

* Updated the note to indicate doc work for 1.14

* first attempt at md export from gdoc

* simplifyig

* big attempt

* moving DRAFT windows content to PR for review

* moving content to PR in markdown for review

* updated note tags

* Delete windows-contributing.md

deleting this file as it is already ported to the github contributor guide

* fixed formatting in intro and cluster setup guide

* updating formatting for running containers guide

* rejiggered end of troubleshooting

* fixed minor typos

* Clarified the windows binary download step

* Update _index.md

making updates based on feedback

* Update _index.md

updating ovn-kubernetes docs

* Update _index.md

* Update _index.md

* updating relative docs links

updating all the links to be relative links to /docs

* Update _index.md

* Update _index.md

updates for windows services and ovn-kubernetes

* formatted for correct step numbering

* fix typos

* Update _index.md

updates for flannel PR in troubleshooting

* Update _index.md

* Update _index.md

updating a few sections like roadmap, services, troubleshooting/filing tickets

* Update _index.md

* Update _index.md

* Update _index.md

* Fixed a few whitespace issues

* Update _index.md

* Update _index.md

* Update _index.md

* add section on upgrading CoreDNS (#12909)

* documentation for kubelet resource metrics endpoint (#12934)

* windows docs updates for 1.14 (#13279)

* Delete sample-l2bridge-wincni-config.json

this file is not used anywhere

* Update _index.md

* Update _index.md

* Update _index.md

* Update _index.md

* Update _index.md

* Rename content/en/docs/getting-started-guides/windows/_index.md to content/en/docs/setup/windows/_index.md

moving to new location

* Delete flannel-master-kubectl-get-ds.png

* Delete flannel-master-kubeclt-get-pods.png

* Delete windows-docker-error.png

* Add files via upload

* Rename _index.md to add-windows-nodes.md

* Create _index.md

* Update _index.md

* Update add-windows-nodes.md

* Update add-windows-nodes.md

* Create user-guide-windows-nodes.md

* Create user-guide-windows-containers.md

* Update and rename add-windows-nodes.md to intro-windows-nodes.md

* Update user-guide-windows-containers.md

* Rename intro-windows-nodes.md to intro-windows-in-kubernetes.md

* Update user-guide-windows-nodes.md

* Update user-guide-windows-containers.md

* Update user-guide-windows-containers.md

* Update user-guide-windows-nodes.md

* Update user-guide-windows-containers.md

* Update _index.md

* Update intro-windows-in-kubernetes.md

* Update intro-windows-in-kubernetes.md

fixing the pause image

* Update intro-windows-in-kubernetes.md

changing tables from html to MD

* Update user-guide-windows-nodes.md

converting tables from HTML to MD

* Update intro-windows-in-kubernetes.md

* Update user-guide-windows-nodes.md

* Update user-guide-windows-nodes.md

* Update user-guide-windows-nodes.md

updating the numbering , even though it messes up the notes a little bit. Jim will file a ticket to follow up

* Update user-guide-windows-nodes.md

* update to windows docs for 1.14 (#13322)

* Update intro-windows-in-kubernetes.md

* Update intro-windows-in-kubernetes.md

* Update intro-windows-in-kubernetes.md

* Update intro-windows-in-kubernetes.md

* Update intro-windows-in-kubernetes.md

* Update user-guide-windows-containers.md

* Update user-guide-windows-nodes.md

* Update intro-windows-in-kubernetes.md (#13344)

* server side apply followup (#13321)

* change some parts of serverside apply docs in response to comments

* fix typos and wording

* Update config.toml (#13365)
This commit is contained in:
Jim Angel
2019-03-25 17:06:16 -05:00
committed by Kubernetes Prow Robot
parent 28dd4d5154
commit 851ef58fa8
160 changed files with 4319 additions and 1812 deletions
@@ -429,9 +429,9 @@ This label prefix is reserved for administrators to label their `Node` objects f
and kubelets will not be allowed to modify labels with that prefix.
* **Allows** kubelets to add/remove/update these labels and label prefixes:
* `kubernetes.io/hostname`
* `beta.kubernetes.io/arch`
* `kubernetes.io/arch`
* `kubernetes.io/os`
* `beta.kubernetes.io/instance-type`
* `beta.kubernetes.io/os`
* `failure-domain.beta.kubernetes.io/region`
* `failure-domain.beta.kubernetes.io/zone`
* `kubelet.kubernetes.io/`-prefixed labels
@@ -5,6 +5,8 @@ reviewers:
- whitlockjc
- caesarxuchao
- deads2k
- liggitt
- mbohlool
title: Dynamic Admission Control
content_template: templates/concept
weight: 40
@@ -19,16 +21,14 @@ the following:
* They need to be compiled into kube-apiserver.
* They are only configurable when the apiserver starts up.
Two features, *Admission Webhooks* (beta in 1.9) and *Initializers* (alpha),
address these limitations. They allow admission controllers to be developed
out-of-tree and configured at runtime.
*Admission Webhooks* (beta in 1.9) addresses these limitations. It allows
admission controllers to be developed out-of-tree and configured at runtime.
This page describes how to use Admission Webhooks.
This page describes how to use Admission Webhooks and Initializers.
{{% /capture %}}
{{% capture body %}}
## Admission Webhooks
### What are admission webhooks?
Admission webhooks are HTTP callbacks that receive admission requests and do
@@ -67,6 +67,13 @@ that is validated in a Kubernetes e2e test. The webhook handles the
`admissionReview` requests sent by the apiservers, and sends back its decision
wrapped in `admissionResponse`.
the `admissionReview` request can have different versions (e.g. v1beta1 or `v1` in a future version).
The webhook can define what version they accept using `admissionReviewVersions` field. API server
will try to use first version in the list which it supports. If none of the versions specified
in this list supported by API server, validation will fail for this object. If the webhook
configuration has already been persisted, calls to the webhook will fail and be
subject to the failure policy.
The example admission webhook server leaves the `ClientAuth` field
[empty](https://github.com/kubernetes/kubernetes/blob/v1.13.0/test/images/webhook/config.go#L47-L48),
which defaults to `NoClientCert`. This means that the webhook server does not
@@ -112,18 +119,32 @@ webhooks:
- CREATE
resources:
- pods
scope: "Namespaced"
clientConfig:
service:
namespace: <namespace of the front-end service>
name: <name of the front-end service>
caBundle: <pem encoded ca cert that signs the server cert used by the webhook>
admissionReviewVersions:
- v1beta1
timeoutSeconds: 1
```
The scope field specifies if only cluster-scoped resources ("Cluster") or namespace-scoped
resources ("Namespaced") will match this rule. "*" means that there are no scope restrictions.
{{< note >}}
When using `clientConfig.service`, the server cert must be valid for
`<svc_name>.<svc_namespace>.svc`.
{{< /note >}}
{{< note >}}
Default timeout for a webhook call is 30 seconds but starting in kubernetes 1.14 you
can set the timeout and it is encouraged to use a very small timeout for webhooks.
If the webhook call times out, the request is handled according to the webhook's
failure policy.
{{< /note >}}
When an apiserver receives a request that matches one of the `rules`, the
apiserver sends an `admissionReview` request to webhook as specified in the
`clientConfig`.
@@ -131,13 +152,6 @@ apiserver sends an `admissionReview` request to webhook as specified in the
After you create the webhook configuration, the system will take a few seconds
to honor the new configuration.
{{< note >}}
When the webhook plugin is deployed into the Kubernetes cluster as a
service, it has to expose its service on the 443 port. The communication
between the API server and the webhook service may fail if a different port
is used.
{{< /note >}}
### Authenticate apiservers
If your admission webhooks require authentication, you can configure the
@@ -196,116 +210,4 @@ users:
```
Of course you need to set up the webhook server to handle these authentications.
## Initializers
### What are initializers?
*Initializer* has two meanings:
* A list of pending pre-initialization tasks, stored in every object's metadata
(e.g., "AddMyCorporatePolicySidecar").
* A user customized controller, which actually performs those tasks. The name of the task
corresponds to the controller which performs the task. For clarity, we call
them *initializer controllers* in this page.
Once the controller has performed its assigned task, it removes its name from
the list. For example, it may send a PATCH that inserts a container in a pod and
also removes its name from `metadata.initializers.pending`. Initializers may make
mutations to objects.
Objects which have a non-empty initializer list are considered uninitialized,
and are not visible in the API unless specifically requested by using the query parameter,
`?includeUninitialized=true`.
### When to use initializers?
Initializers are useful for admins to force policies (e.g., the
[AlwaysPullImages](/docs/reference/access-authn-authz/admission-controllers/#alwayspullimages)
admission controller), or to inject defaults (e.g., the
[DefaultStorageClass](/docs/reference/access-authn-authz/admission-controllers/#defaultstorageclass)
admission controller), etc.
{{< note >}}
If your use case does not involve mutating objects, consider using
external admission webhooks, as they have better performance.
{{< /note >}}
### How are initializers triggered?
When an object is POSTed, it is checked against all existing
`initializerConfiguration` objects (explained below). For all that it matches,
all `spec.initializers[].name`s are appended to the new object's
`metadata.initializers.pending` field.
An initializer controller should list and watch for uninitialized objects, by
using the query parameter `?includeUninitialized=true`. If using client-go, just
set
[listOptions.includeUninitialized](https://github.com/kubernetes/kubernetes/blob/v1.13.0/staging/src/k8s.io/apimachinery/pkg/apis/meta/v1/types.go#L332)
to true.
For the observed uninitialized objects, an initializer controller should first
check if its name matches `metadata.initializers.pending[0]`. If so, it should then
perform its assigned task and remove its name from the list.
### Enable initializers alpha feature
*Initializers* is an alpha feature, so it is disabled by default. To turn it on,
you need to:
* Include "Initializers" in the `--enable-admission-plugins` flag when starting
`kube-apiserver`. If you have multiple `kube-apiserver` replicas, all should
have the same flag setting.
* Enable the dynamic admission controller registration API by adding
`admissionregistration.k8s.io/v1alpha1` to the `--runtime-config` flag passed
to `kube-apiserver`, e.g.
`--runtime-config=admissionregistration.k8s.io/v1alpha1`. Again, all replicas
should have the same flag setting.
### Deploy an initializer controller
You should deploy an initializer controller via the [deployment
API](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#deployment-v1beta1-apps).
### Configure initializers on the fly
You can configure what initializers are enabled and what resources are subject
to the initializers by creating `initializerConfiguration` resources.
You should first deploy the initializer controller and make sure that it is
working properly before creating the `initializerConfiguration`. Otherwise, any
newly created resources will be stuck in an uninitialized state.
The following is an example `initializerConfiguration`:
```yaml
apiVersion: admissionregistration.k8s.io/v1alpha1
kind: InitializerConfiguration
metadata:
name: example-config
initializers:
# the name needs to be fully qualified, i.e., containing at least two "."
- name: podimage.example.com
rules:
# apiGroups, apiVersion, resources all support wildcard "*".
# "*" cannot be mixed with non-wildcard.
- apiGroups:
- ""
apiVersions:
- v1
resources:
- pods
```
After you create the `initializerConfiguration`, the system will take a few
seconds to honor the new configuration. Then, `"podimage.example.com"` will be
appended to the `metadata.initializers.pending` field of newly created pods. You
should already have a ready "podimage" initializer controller that handles pods
whose `metadata.initializers.pending[0].name="podimage.example.com"`. Otherwise
the pods will be stuck in an uninitialized state.
Make sure that all expansions of the `<apiGroup, apiVersions, resources>` tuple
in a `rule` are valid. If they are not, separate them in different `rules`.
{{% /capture %}}
@@ -26,7 +26,7 @@ To enable RBAC, start the apiserver with `--authorization-mode=RBAC`.
The RBAC API declares four top-level types which will be covered in this
section. Users can interact with these resources as they would with any other
API resource (via `kubectl`, API calls, etc.). For instance,
`kubectl create -f (resource).yml` can be used with any of these examples,
`kubectl apply -f (resource).yml` can be used with any of these examples,
though readers who wish to follow along should review the section on
bootstrapping first.
@@ -489,13 +489,18 @@ NOTE: editing the role is not recommended as changes will be overwritten on API
</tr>
<tr>
<td><b>system:basic-user</b></td>
<td><b>system:authenticated</b> and <b>system:unauthenticated</b> groups</td>
<td>Allows a user read-only access to basic information about themselves.</td>
<td><b>system:authenticated</b> group</td>
<td>Allows a user read-only access to basic information about themselves. Prior to 1.14, this role was also bound to `system:unauthenticated` by default.</td>
</tr>
<tr>
<td><b>system:discovery</b></td>
<td><b>system:authenticated</b> group</td>
<td>Allows read-only access to API discovery endpoints needed to discover and negotiate an API level. Prior to 1.14, this role was also bound to `system:unauthenticated` by default.</td>
</tr>
<tr>
<td><b>system:public-info-viewer</b></td>
<td><b>system:authenticated</b> and <b>system:unauthenticated</b> groups</td>
<td>Allows read-only access to API discovery endpoints needed to discover and negotiate an API level.</td>
<td>Allows read-only access to non-sensitive information about the cluster. Introduced in 1.14.</td>
</tr>
</table>
@@ -55,9 +55,17 @@ different Kubernetes components.
| `CPUManager` | `true` | Beta | 1.10 | |
| `CRIContainerLogRotation` | `false` | Alpha | 1.10 | 1.10 |
| `CRIContainerLogRotation` | `true` | Beta| 1.11 | |
| `CSIBlockVolume` | `false` | Alpha | 1.11 | |
| `CSIDriverRegistry` | `false` | Alpha | 1.12 | |
| `CSINodeInfo` | `false` | Alpha | 1.12 | |
| `CSIBlockVolume` | `false` | Alpha | 1.11 | 1.13 |
| `CSIBlockVolume` | `true` | Beta | 1.14 | |
| `CSIDriverRegistry` | `false` | Alpha | 1.12 | 1.13 |
| `CSIDriverRegistry` | `true` | Beta | 1.14 | |
| `CSIInlineVolume` | `false` | Alpha | 1.14 | - |
| `CSIMigration` | `false` | Alpha | 1.14 | |
| `CSIMigrationAWS` | `false` | Alpha | 1.14 | |
| `CSIMigrationGCE` | `false` | Alpha | 1.14 | |
| `CSIMigrationOpenStack` | `false` | Alpha | 1.14 | |
| `CSINodeInfo` | `false` | Alpha | 1.12 | 1.13 |
| `CSINodeInfo` | `true` | Beta | 1.14 | |
| `CSIPersistentVolume` | `false` | Alpha | 1.9 | 1.9 |
| `CSIPersistentVolume` | `true` | Beta | 1.10 | 1.12 |
| `CSIPersistentVolume` | `true` | GA | 1.13 | - |
@@ -79,7 +87,8 @@ different Kubernetes components.
| `DynamicVolumeProvisioning` | `true` | Alpha | 1.3 | 1.7 |
| `DynamicVolumeProvisioning` | `true` | GA | 1.8 | |
| `EnableEquivalenceClassCache` | `false` | Alpha | 1.8 | |
| `ExpandInUsePersistentVolumes` | `false` | Alpha | 1.11 | |
| `ExpandCSIVolumes` | `false` | Alpha | 1.14 | | |
| `ExpandInUsePersistentVolumes` | `false` | Alpha | 1.11 | 1.13 | |
| `ExpandPersistentVolumes` | `false` | Alpha | 1.8 | 1.10 |
| `ExpandPersistentVolumes` | `true` | Beta | 1.11 | |
| `ExperimentalCriticalPodAnnotation` | `false` | Alpha | 1.5 | |
@@ -103,7 +112,8 @@ different Kubernetes components.
| `MountPropagation` | `true` | GA | 1.12 | |
| `NodeLease` | `false` | Alpha | 1.12 | |
| `PersistentLocalVolumes` | `false` | Alpha | 1.7 | 1.9 |
| `PersistentLocalVolumes` | `true` | Beta | 1.10 | |
| `PersistentLocalVolumes` | `true` | Beta | 1.10 | 1.13 |
| `PersistentLocalVolumes` | `true` | GA | 1.14 | |
| `PodPriority` | `false` | Alpha | 1.8 | 1.10 |
| `PodPriority` | `true` | Beta | 1.11 | |
| `PodReadinessGates` | `false` | Alpha | 1.11 | |
@@ -118,9 +128,10 @@ different Kubernetes components.
| `RotateKubeletClientCertificate` | `true` | Beta | 1.7 | |
| `RotateKubeletServerCertificate` | `false` | Alpha | 1.7 | 1.11 |
| `RotateKubeletServerCertificate` | `true` | Beta | 1.12 | |
| `RunAsGroup` | `false` | Alpha | 1.10 | |
| `RuntimeClass` | `false` | Alpha | 1.12 | |
| `RunAsGroup` | `true` | Beta | 1.14 | |
| `RuntimeClass` | `true` | Beta | 1.14 | |
| `SCTPSupport` | `false` | Alpha | 1.12 | |
| `ServerSideApply` | `false` | Alpha | 1.14 | |
| `ServiceNodeExclusion` | `false` | Alpha | 1.8 | |
| `StorageObjectInUseProtection` | `true` | Beta | 1.10 | 1.10 |
| `StorageObjectInUseProtection` | `true` | GA | 1.11 | |
@@ -143,10 +154,11 @@ different Kubernetes components.
| `VolumeScheduling` | `false` | Alpha | 1.9 | 1.9 |
| `VolumeScheduling` | `true` | Beta | 1.10 | 1.12 |
| `VolumeScheduling` | `true` | GA | 1.13 | |
| `VolumeSubpathEnvExpansion` | `false` | Alpha | 1.11 | |
| `VolumeSubpathEnvExpansion` | `false` | Alpha | 1.14 | |
| `VolumeSnapshotDataSource` | `false` | Alpha | 1.12 | - |
| `ScheduleDaemonSetPods` | `false` | Alpha | 1.11 | 1.11 |
| `ScheduleDaemonSetPods` | `true` | Beta | 1.12 | |
| `WindowsGMSA` | `false` | Alpha | 1.14 | |
## Using a Feature
@@ -213,11 +225,16 @@ Each feature gate is designed for enabling/disabling a specific feature:
- `CRIContainerLogRotation`: Enable container log rotation for cri container runtime.
- `CSIBlockVolume`: Enable external CSI volume drivers to support block storage. See the [`csi` raw block volume support](/docs/concepts/storage/volumes/#csi-raw-block-volume-support) documentation for more details.
- `CSIDriverRegistry`: Enable all logic related to the CSIDriver API object in csi.storage.k8s.io.
- `CSIMigration`: Enables shims and translation logic to route volume operations from in-tree plugins to corresponding pre-installed CSI plugins
- `CSIMigrationAWS`: Enables shims and translation logic to route volume operations from the AWS-EBS in-tree plugin to EBS CSI plugin
- `CSIMigrationGCE`: Enables shims and translation logic to route volume operations from the GCE-PD in-tree plugin to PD CSI plugin
- `CSIMigrationOpenStack`: Enables shims and translation logic to route volume operations from the Cinder in-tree plugin to Cinder CSI plugin
- `CSINodeInfo`: Enable all logic related to the CSINodeInfo API object in csi.storage.k8s.io.
- `CSIPersistentVolume`: Enable discovering and mounting volumes provisioned through a
[CSI (Container Storage Interface)](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/container-storage-interface.md)
compatible volume plugin.
Check the [`csi` volume type](/docs/concepts/storage/volumes/#csi) documentation for more details.
- `CSIInlineVolume`: Enable CSI volumes to be directly embedded in Pod specifications instead of a PersistentVolume.
- `CustomPodDNS`: Enable customizing the DNS settings for a Pod using its `dnsConfig` property.
Check [Pod's DNS Config](/docs/concepts/services-networking/dns-pod-service/#pods-dns-config)
for more details.
@@ -286,6 +303,7 @@ Each feature gate is designed for enabling/disabling a specific feature:
- `RuntimeClass`: Enable the [RuntimeClass](/docs/concepts/containers/runtime-class/) feature for selecting container runtime configurations.
- `ScheduleDaemonSetPods`: Enable DaemonSet Pods to be scheduled by the default scheduler instead of the DaemonSet controller.
- `SCTPSupport`: Enables the usage of SCTP as `protocol` value in `Service`, `Endpoint`, `NetworkPolicy` and `Pod` definitions
- `ServerSideApply`: Enables the [Sever Side Apply (SSA)](/docs/reference/using-api/api-concepts/#server-side-apply) path at the API Server.
- `ServiceNodeExclusion`: Enable the exclusion of nodes from load balancers created by a cloud provider.
A node is eligible for exclusion if annotated with "`alpha.service-controller.kubernetes.io/exclude-balancer`" key.
- `StorageObjectInUseProtection`: Postpone the deletion of PersistentVolume or
@@ -310,5 +328,7 @@ Each feature gate is designed for enabling/disabling a specific feature:
enables the usage of [`local`](/docs/concepts/storage/volumes/#local) volume
type when used together with the `PersistentLocalVolumes` feature gate.
- `VolumeSnapshotDataSource`: Enable volume snapshot data source support.
- `VolumeSubpathEnvExpansion`: Enable `subPathExpr` field for expanding environment variables into a `subPath`.
- `WindowsGMSA`: Enables passing of GMSA credential specs from pods to container runtimes.
{{% /capture %}}
@@ -707,7 +707,7 @@ kubelet [flags]
<td colspan="2">--kube-reserved mapStringString</td>
</tr>
<tr>
<td></td><td style="line-height: 130%; word-wrap: break-word;">A set of ResourceName=ResourceQuantity (e.g. cpu=200m,memory=500Mi,ephemeral-storage=1Gi) pairs that describe resources reserved for kubernetes system components. Currently cpu, memory and local ephemeral storage for root file system are supported. See /docs/user-guide/compute-resources for more detail. [default=none]</td>
<td></td><td style="line-height: 130%; word-wrap: break-word;">A set of ResourceName=ResourceQuantity (e.g. cpu=200m,memory=500Mi,ephemeral-storage=1Gi,pid=1000) pairs that describe resources reserved for kubernetes system components. Currently cpu, memory, pid, and local ephemeral storage for root file system are supported. See http://kubernetes.io/docs/user-guide/compute-resources for more detail. [default=none]</td>
</tr>
<tr>
@@ -1092,7 +1092,7 @@ kubelet [flags]
<td colspan="2">--system-reserved mapStringString</td>
</tr>
<tr>
<td></td><td style="line-height: 130%; word-wrap: break-word;">A set of ResourceName=ResourceQuantity (e.g. cpu=200m,memory=500Mi,ephemeral-storage=1Gi) pairs that describe resources reserved for non-kubernetes components. Currently only cpu and memory are supported. See /docs/user-guide/compute-resources for more detail. [default=none]</td>
<td></td><td style="line-height: 130%; word-wrap: break-word;">A set of ResourceName=ResourceQuantity (e.g. cpu=200m,memory=500Mi,ephemeral-storage=1Gi,pid=1000) pairs that describe resources reserved for non-kubernetes components. Currently only cpu, memory, and pid are supported. See http://kubernetes.io/docs/user-guide/compute-resources for more detail. [default=none]</td>
</tr>
<tr>
@@ -14,5 +14,4 @@ tags:
<!--more-->
The securityContext field in a {{< glossary_tooltip term_id="pod" >}} (applying to all containers) or container is used to set the user (runAsUser) and group (fsGroup), capabilities, privilege settings, and security policies (SELinux/AppArmor/Seccomp) that container processes use.
The securityContext field in a {{< glossary_tooltip term_id="pod" >}} (applying to all containers) or container is used to set the user, groups, capabilities, privilege settings, and security policies (SELinux/AppArmor/Seccomp) and more that container processes use.
@@ -75,21 +75,24 @@ kubectl config set-context gce --user=cluster-admin --namespace=foo \
kubectl config unset users.foo # delete user foo
```
## Apply
`apply` manages applications through files defining Kubernetes resources. It creates and updates resources in a cluster through running `kubectl apply`. This is the recommended way of managing Kubernetes applications on production. See [Kubectl Book](https://kubectl.docs.kubernetes.io).
## Creating Objects
Kubernetes manifests can be defined in json or yaml. The file extension `.yaml`,
`.yml`, and `.json` can be used.
```bash
kubectl create -f ./my-manifest.yaml # create resource(s)
kubectl create -f ./my1.yaml -f ./my2.yaml # create from multiple files
kubectl create -f ./dir # create resource(s) in all manifest files in dir
kubectl create -f https://git.io/vPieo # create resource(s) from url
kubectl apply -f ./my-manifest.yaml # create resource(s)
kubectl apply -f ./my1.yaml -f ./my2.yaml # create from multiple files
kubectl apply -f ./dir # create resource(s) in all manifest files in dir
kubectl apply -f https://git.io/vPieo # create resource(s) from url
kubectl create deployment nginx --image=nginx # start a single instance of nginx
kubectl explain pods,svc # get the documentation for pod and svc manifests
# Create multiple YAML objects from stdin
cat <<EOF | kubectl create -f -
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
@@ -116,7 +119,7 @@ spec:
EOF
# Create a secret with several keys
cat <<EOF | kubectl create -f -
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Secret
metadata:
@@ -77,6 +77,6 @@ flag, which provides the object to be submitted to the cluster.
### `kubectl apply`
* You can use `kubectl apply` to create or update resources. However, to update a resource you should have created the resource by using `kubectl apply` or `kubectl create --save-config`. For more information about using kubectl apply to update resources, see [Managing Resources](/docs/concepts/cluster-administration/manage-deployment/#kubectl-apply).
* You can use `kubectl apply` to create or update resources. For more information about using kubectl apply to update resources, see [Kubectl Book](https://kubectl.docs.kubernetes.io).
{{% /capture %}}
@@ -268,17 +268,17 @@ kubectl get pods --sort-by=.metadata.name
Use the following set of examples to help you familiarize yourself with running the commonly used `kubectl` operations:
`kubectl create` - Create a resource from a file or stdin.
`kubectl apply` - Apply or Update a resource from a file or stdin.
```shell
# Create a service using the definition in example-service.yaml.
kubectl create -f example-service.yaml
kubectl apply -f example-service.yaml
# Create a replication controller using the definition in example-controller.yaml.
kubectl create -f example-controller.yaml
kubectl apply -f example-controller.yaml
# Create the objects that are defined in any .yaml, .yml, or .json file within the <directory> directory.
kubectl create -f <directory>
kubectl apply -f <directory>
```
`kubectl get` - List one or more resources.
@@ -11,23 +11,32 @@ This document serves both as a reference to the values, and as a coordination po
{{% /capture %}}
{{% capture body %}}
## beta.kubernetes.io/arch
## kubernetes.io/arch
Example: `beta.kubernetes.io/arch=amd64`
Example: `kubernetes.io/arch=amd64`
Used on: Node
Kubelet populates this with `runtime.GOARCH` as defined by Go. This can be handy if you are mixing arm and x86 nodes,
for example.
## beta.kubernetes.io/os
## kubernetes.io/os
Example: `beta.kubernetes.io/os=linux`
Example: `kubernetes.io/os=linux`
Used on: Node
Kubelet populates this with `runtime.GOOS` as defined by Go. This can be handy if you are mixing operating systems
in your cluster (although currently Linux is the only OS supported by Kubernetes).
in your cluster (e.g., mixing Linux and Windows nodes).
## beta.kubernetes.io/arch (deprecated)
This label has been deprecated. Please use `kubernetes.io/arch` instead.
## beta.kubernetes.io/os (deprecated)
This label has been deprecated. Please use `kubernetes.io/os` instead.
## kubernetes.io/hostname
@@ -1,50 +0,0 @@
Commands related to pre-flight checks
### Synopsis
This command is not meant to be run on its own. See list of available subcommands.
### Options
<table style="width: 100%; table-layout: fixed;">
<colgroup>
<col span="1" style="width: 10px;" />
<col span="1" />
</colgroup>
<tbody>
<tr>
<td colspan="2">-h, --help</td>
</tr>
<tr>
<td></td><td style="line-height: 130%; word-wrap: break-word;">help for preflight</td>
</tr>
</tbody>
</table>
### Options inherited from parent commands
<table style="width: 100%; table-layout: fixed;">
<colgroup>
<col span="1" style="width: 10px;" />
<col span="1" />
</colgroup>
<tbody>
<tr>
<td colspan="2">--rootfs string</td>
</tr>
<tr>
<td></td><td style="line-height: 130%; word-wrap: break-word;">[EXPERIMENTAL] The path to the 'real' host root filesystem.</td>
</tr>
</tbody>
</table>
@@ -1,77 +0,0 @@
Run node pre-flight checks
### Synopsis
Run node pre-flight checks, functionally equivalent to what implemented by kubeadm join.
Alpha Disclaimer: this command is currently alpha.
```
kubeadm alpha preflight node [flags]
```
### Examples
```
# Run node pre-flight checks.
kubeadm alpha preflight node
```
### Options
<table style="width: 100%; table-layout: fixed;">
<colgroup>
<col span="1" style="width: 10px;" />
<col span="1" />
</colgroup>
<tbody>
<tr>
<td colspan="2">--config string</td>
</tr>
<tr>
<td></td><td style="line-height: 130%; word-wrap: break-word;">Path to a kubeadm configuration file.</td>
</tr>
<tr>
<td colspan="2">-h, --help</td>
</tr>
<tr>
<td></td><td style="line-height: 130%; word-wrap: break-word;">help for node</td>
</tr>
<tr>
<td colspan="2">--ignore-preflight-errors stringSlice</td>
</tr>
<tr>
<td></td><td style="line-height: 130%; word-wrap: break-word;">A list of checks whose errors will be shown as warnings. Example: 'IsPrivilegedUser,Swap'. Value 'all' ignores errors from all checks.</td>
</tr>
</tbody>
</table>
### Options inherited from parent commands
<table style="width: 100%; table-layout: fixed;">
<colgroup>
<col span="1" style="width: 10px;" />
<col span="1" />
</colgroup>
<tbody>
<tr>
<td colspan="2">--rootfs string</td>
</tr>
<tr>
<td></td><td style="line-height: 130%; word-wrap: break-word;">[EXPERIMENTAL] The path to the 'real' host root filesystem.</td>
</tr>
</tbody>
</table>
@@ -0,0 +1,27 @@
Upload certificates to kubeadm-certs
### Synopsis
This command is not meant to be run on its own. See list of available subcommands.
```
kubeadm init phase upload-certs [flags]
```
### Options
```
--certificate-key string Key used to encrypt the control-plane certificates in the kubeadm-certs Secret.
--config string Path to a kubeadm configuration file.
--experimental-upload-certs Upload control-plane certificates to the kubeadm-certs Secret.
-h, --help help for upload-certs
--skip-certificate-key-print Don't print the key used to encrypt the control-plane certificates.
```
### Options inherited from parent commands
```
--rootfs string [EXPERIMENTAL] The path to the 'real' host root filesystem.
```
@@ -0,0 +1,19 @@
use this command to invoke single phase of the join workflow
### Synopsis
use this command to invoke single phase of the join workflow
### Options
```
-h, --help help for phase
```
### Options inherited from parent commands
```
--rootfs string [EXPERIMENTAL] The path to the 'real' host root filesystem.
```
@@ -0,0 +1,30 @@
Joins a machine as a control plane instance
### Synopsis
Joins a machine as a control plane instance
```
kubeadm join phase control-plane-join [flags]
```
### Examples
```
# Joins a machine as a control plane instance
kubeadm join phase control-plane-join all
```
### Options
```
-h, --help help for control-plane-join
```
### Options inherited from parent commands
```
--rootfs string [EXPERIMENTAL] The path to the 'real' host root filesystem.
```
@@ -0,0 +1,27 @@
Joins a machine as a control plane instance
### Synopsis
Joins a machine as a control plane instance
```
kubeadm join phase control-plane-join all [flags]
```
### Options
```
--apiserver-advertise-address string If the node should host a new control plane instance, the IP address the API Server will advertise it's listening on. If not set the default network interface will be used.
--config string Path to kubeadm config file.
--experimental-control-plane Create a new control plane instance on this node
-h, --help help for all
--node-name string Specify the node name.
```
### Options inherited from parent commands
```
--rootfs string [EXPERIMENTAL] The path to the 'real' host root filesystem.
```
@@ -0,0 +1,27 @@
Add a new local etcd member
### Synopsis
Add a new local etcd member
```
kubeadm join phase control-plane-join etcd [flags]
```
### Options
```
--apiserver-advertise-address string If the node should host a new control plane instance, the IP address the API Server will advertise it's listening on. If not set the default network interface will be used.
--config string Path to kubeadm config file.
--experimental-control-plane Create a new control plane instance on this node
-h, --help help for etcd
--node-name string Specify the node name.
```
### Options inherited from parent commands
```
--rootfs string [EXPERIMENTAL] The path to the 'real' host root filesystem.
```
@@ -0,0 +1,26 @@
Mark a node as a control-plane
### Synopsis
Mark a node as a control-plane
```
kubeadm join phase control-plane-join mark-control-plane [flags]
```
### Options
```
--config string Path to kubeadm config file.
--experimental-control-plane Create a new control plane instance on this node
-h, --help help for mark-control-plane
--node-name string Specify the node name.
```
### Options inherited from parent commands
```
--rootfs string [EXPERIMENTAL] The path to the 'real' host root filesystem.
```
@@ -0,0 +1,27 @@
Register the new control-plane node into the ClusterStatus maintained in the kubeadm-config ConfigMap
### Synopsis
Register the new control-plane node into the ClusterStatus maintained in the kubeadm-config ConfigMap
```
kubeadm join phase control-plane-join update-status [flags]
```
### Options
```
--apiserver-advertise-address string If the node should host a new control plane instance, the IP address the API Server will advertise it's listening on. If not set the default network interface will be used.
--config string Path to kubeadm config file.
--experimental-control-plane Create a new control plane instance on this node
-h, --help help for update-status
--node-name string Specify the node name.
```
### Options inherited from parent commands
```
--rootfs string [EXPERIMENTAL] The path to the 'real' host root filesystem.
```
@@ -0,0 +1,30 @@
Prepares the machine for serving a control plane.
### Synopsis
Prepares the machine for serving a control plane.
```
kubeadm join phase control-plane-prepare [flags]
```
### Examples
```
# Prepares the machine for serving a control plane
kubeadm join phase control-plane-prepare all
```
### Options
```
-h, --help help for control-plane-prepare
```
### Options inherited from parent commands
```
--rootfs string [EXPERIMENTAL] The path to the 'real' host root filesystem.
```
@@ -0,0 +1,35 @@
Prepares the machine for serving a control plane.
### Synopsis
Prepares the machine for serving a control plane.
```
kubeadm join phase control-plane-prepare all [api-server-endpoint] [flags]
```
### Options
```
--apiserver-advertise-address string If the node should host a new control plane instance, the IP address the API Server will advertise it's listening on. If not set the default network interface will be used.
--apiserver-bind-port int32 If the node should host a new control plane instance, the port for the API Server to bind to. (default 6443)
--certificate-key string Use this key to decrypt the certificate secrets uploaded by init.
--config string Path to kubeadm config file.
--discovery-file string For file-based discovery, a file or URL from which to load cluster information.
--discovery-token string For token-based discovery, the token used to validate cluster information fetched from the API server.
--discovery-token-ca-cert-hash strings For token-based discovery, validate that the root CA public key matches this hash (format: "<type>:<value>").
--discovery-token-unsafe-skip-ca-verification For token-based discovery, allow joining without --discovery-token-ca-cert-hash pinning.
--experimental-control-plane Create a new control plane instance on this node
-h, --help help for all
--node-name string Specify the node name.
--tls-bootstrap-token string Specify the token used to temporarily authenticate with the Kubernetes Control Plane while joining the node.
--token string Use this token for both discovery-token and tls-bootstrap-token when those values are not provided.
```
### Options inherited from parent commands
```
--rootfs string [EXPERIMENTAL] The path to the 'real' host root filesystem.
```
@@ -0,0 +1,33 @@
Generates the certificates for the new control plane components
### Synopsis
Generates the certificates for the new control plane components
```
kubeadm join phase control-plane-prepare certs [api-server-endpoint] [flags]
```
### Options
```
--apiserver-advertise-address string If the node should host a new control plane instance, the IP address the API Server will advertise it's listening on. If not set the default network interface will be used.
--config string Path to kubeadm config file.
--discovery-file string For file-based discovery, a file or URL from which to load cluster information.
--discovery-token string For token-based discovery, the token used to validate cluster information fetched from the API server.
--discovery-token-ca-cert-hash strings For token-based discovery, validate that the root CA public key matches this hash (format: "<type>:<value>").
--discovery-token-unsafe-skip-ca-verification For token-based discovery, allow joining without --discovery-token-ca-cert-hash pinning.
--experimental-control-plane Create a new control plane instance on this node
-h, --help help for certs
--node-name string Specify the node name.
--tls-bootstrap-token string Specify the token used to temporarily authenticate with the Kubernetes Control Plane while joining the node.
--token string Use this token for both discovery-token and tls-bootstrap-token when those values are not provided.
```
### Options inherited from parent commands
```
--rootfs string [EXPERIMENTAL] The path to the 'real' host root filesystem.
```
@@ -0,0 +1,27 @@
Generates the manifests for the new control plane components
### Synopsis
Generates the manifests for the new control plane components
```
kubeadm join phase control-plane-prepare control-plane [flags]
```
### Options
```
--apiserver-advertise-address string If the node should host a new control plane instance, the IP address the API Server will advertise it's listening on. If not set the default network interface will be used.
--apiserver-bind-port int32 If the node should host a new control plane instance, the port for the API Server to bind to. (default 6443)
--config string Path to kubeadm config file.
--experimental-control-plane Create a new control plane instance on this node
-h, --help help for control-plane
```
### Options inherited from parent commands
```
--rootfs string [EXPERIMENTAL] The path to the 'real' host root filesystem.
```
@@ -0,0 +1,32 @@
[EXPERIMENTAL] Downloads certificates shared among control-plane nodes from the kubeadm-certs Secret
### Synopsis
[EXPERIMENTAL] Downloads certificates shared among control-plane nodes from the kubeadm-certs Secret
```
kubeadm join phase control-plane-prepare download-certs [api-server-endpoint] [flags]
```
### Options
```
--certificate-key string Use this key to decrypt the certificate secrets uploaded by init.
--config string Path to kubeadm config file.
--discovery-file string For file-based discovery, a file or URL from which to load cluster information.
--discovery-token string For token-based discovery, the token used to validate cluster information fetched from the API server.
--discovery-token-ca-cert-hash strings For token-based discovery, validate that the root CA public key matches this hash (format: "<type>:<value>").
--discovery-token-unsafe-skip-ca-verification For token-based discovery, allow joining without --discovery-token-ca-cert-hash pinning.
--experimental-control-plane Create a new control plane instance on this node
-h, --help help for download-certs
--tls-bootstrap-token string Specify the token used to temporarily authenticate with the Kubernetes Control Plane while joining the node.
--token string Use this token for both discovery-token and tls-bootstrap-token when those values are not provided.
```
### Options inherited from parent commands
```
--rootfs string [EXPERIMENTAL] The path to the 'real' host root filesystem.
```
@@ -0,0 +1,32 @@
Generates the kubeconfig for the new control plane components
### Synopsis
Generates the kubeconfig for the new control plane components
```
kubeadm join phase control-plane-prepare kubeconfig [api-server-endpoint] [flags]
```
### Options
```
--certificate-key string Use this key to decrypt the certificate secrets uploaded by init.
--config string Path to kubeadm config file.
--discovery-file string For file-based discovery, a file or URL from which to load cluster information.
--discovery-token string For token-based discovery, the token used to validate cluster information fetched from the API server.
--discovery-token-ca-cert-hash strings For token-based discovery, validate that the root CA public key matches this hash (format: "<type>:<value>").
--discovery-token-unsafe-skip-ca-verification For token-based discovery, allow joining without --discovery-token-ca-cert-hash pinning.
--experimental-control-plane Create a new control plane instance on this node
-h, --help help for kubeconfig
--tls-bootstrap-token string Specify the token used to temporarily authenticate with the Kubernetes Control Plane while joining the node.
--token string Use this token for both discovery-token and tls-bootstrap-token when those values are not provided.
```
### Options inherited from parent commands
```
--rootfs string [EXPERIMENTAL] The path to the 'real' host root filesystem.
```
@@ -0,0 +1,32 @@
Writes kubelet settings, certificates and (re)starts the kubelet
### Synopsis
Writes a file with KubeletConfiguration and an environment file with node specific kubelet settings, and then (re)starts kubelet.
```
kubeadm join phase kubelet-start [api-server-endpoint] [flags]
```
### Options
```
--config string Path to kubeadm config file.
--cri-socket string Path to the CRI socket to connect. If empty kubeadm will try to auto-detect this value; use this option only if you have more than one CRI installed or if you have non-standard CRI socket.
--discovery-file string For file-based discovery, a file or URL from which to load cluster information.
--discovery-token string For token-based discovery, the token used to validate cluster information fetched from the API server.
--discovery-token-ca-cert-hash strings For token-based discovery, validate that the root CA public key matches this hash (format: "<type>:<value>").
--discovery-token-unsafe-skip-ca-verification For token-based discovery, allow joining without --discovery-token-ca-cert-hash pinning.
-h, --help help for kubelet-start
--node-name string Specify the node name.
--tls-bootstrap-token string Specify the token used to temporarily authenticate with the Kubernetes Control Plane while joining the node.
--token string Use this token for both discovery-token and tls-bootstrap-token when those values are not provided.
```
### Options inherited from parent commands
```
--rootfs string [EXPERIMENTAL] The path to the 'real' host root filesystem.
```
@@ -0,0 +1,44 @@
Run join pre-flight checks
### Synopsis
Run pre-flight checks for kubeadm join.
```
kubeadm join phase preflight [api-server-endpoint] [flags]
```
### Examples
```
# Run join pre-flight checks using a config file.
kubeadm join phase preflight --config kubeadm-config.yml
```
### Options
```
--apiserver-advertise-address string If the node should host a new control plane instance, the IP address the API Server will advertise it's listening on. If not set the default network interface will be used.
--apiserver-bind-port int32 If the node should host a new control plane instance, the port for the API Server to bind to. (default 6443)
--certificate-key string Use this key to decrypt the certificate secrets uploaded by init.
--config string Path to kubeadm config file.
--cri-socket string Path to the CRI socket to connect. If empty kubeadm will try to auto-detect this value; use this option only if you have more than one CRI installed or if you have non-standard CRI socket.
--discovery-file string For file-based discovery, a file or URL from which to load cluster information.
--discovery-token string For token-based discovery, the token used to validate cluster information fetched from the API server.
--discovery-token-ca-cert-hash strings For token-based discovery, validate that the root CA public key matches this hash (format: "<type>:<value>").
--discovery-token-unsafe-skip-ca-verification For token-based discovery, allow joining without --discovery-token-ca-cert-hash pinning.
--experimental-control-plane Create a new control plane instance on this node
-h, --help help for preflight
--ignore-preflight-errors strings A list of checks whose errors will be shown as warnings. Example: 'IsPrivilegedUser,Swap'. Value 'all' ignores errors from all checks.
--node-name string Specify the node name.
--tls-bootstrap-token string Specify the token used to temporarily authenticate with the Kubernetes Control Plane while joining the node.
--token string Use this token for both discovery-token and tls-bootstrap-token when those values are not provided.
```
### Options inherited from parent commands
```
--rootfs string [EXPERIMENTAL] The path to the 'real' host root filesystem.
```
@@ -48,15 +48,6 @@ to enable the DynamicKubeletConfiguration feature.
{{< tab name="enable-dynamic" include="generated/kubeadm_alpha_kubelet_config_download.md" />}}
{{< /tabs >}}
## kubeadm alpha preflight node {#cmd-phase-preflight}
You can use the `node` sub command to run preflight checks on a worker node.
{{< tabs name="tab-preflight" >}}
{{< tab name="preflight" include="generated/kubeadm_alpha_preflight.md" />}}
{{< tab name="node" include="generated/kubeadm_alpha_preflight_node.md" />}}
{{< /tabs >}}
## kubeadm alpha selfhosting pivot {#cmd-selfhosting}
@@ -79,7 +79,17 @@ Use the following phase to create a local etcd instance based on a static Pod fi
{{< /tabs >}}
## kubeadm init phase mark-control-plane {#cmd-phase-control-plane}
## kubeadm init phase upload-certs {#cmd-phase-upload-certs}
Use the following phase to upload control-plane certificates to the cluster.
By default the certs and encryption key expire after two hours.
{{< tabs name="tab-upload-certs" >}}
{{< tab name="upload-certs" include="generated/kubeadm_init_phase_upload-certs.md" />}}
{{< /tabs >}}
## kubeadm init phase mark-control-plane {#cmd-phase-mark-control-plane}
Use the following phase to label and taint the node with the `node-role.kubernetes.io/master=""` key-value pair.
@@ -49,7 +49,7 @@ following steps:
run there.
1. Generates the token that additional nodes can use to register
themselves with the master in the future. Optionally, the user can provide a
themselves with a control-plane in the future. Optionally, the user can provide a
token via `--token`, as described in the
[kubeadm token](/docs/reference/setup-tools/kubeadm/kubeadm-token/) docs.
@@ -82,13 +82,13 @@ Note that by calling `kubeadm init` all of the phases and sub-phases will be exe
Some phases have unique flags, so if you want to have a look at the list of available options add `--help`, for example:
```bash
```shell
sudo kubeadm init phase control-plane controller-manager --help
```
You can also use `--help` to see the list of sub-phases for a certain parent phase:
```bash
```shell
sudo kubeadm init phase control-plane --help
```
@@ -96,7 +96,7 @@ sudo kubeadm init phase control-plane --help
An example:
```bash
```shell
sudo kubeadm init phase control-plane all --config=configfile.yaml
sudo kubeadm init phase etcd local --config=configfile.yaml
# you can now modify the control plane and etcd manifest files
@@ -117,9 +117,10 @@ configuration file options. This file is passed in the `--config` option.
In Kubernetes 1.11 and later, the default configuration can be printed out using the
[kubeadm config print](/docs/reference/setup-tools/kubeadm/kubeadm-config/) command.
It is **recommended** that you migrate your old `v1alpha3` configuration to `v1beta1` using
the [kubeadm config migrate](/docs/reference/setup-tools/kubeadm/kubeadm-config/) command,
because `v1alpha3` will be removed in Kubernetes 1.14.
because `v1alpha3` will be removed in Kubernetes 1.15.
For more details on each field in the `v1beta1` configuration you can navigate to our
[API reference pages](https://godoc.org/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta1).
@@ -266,20 +267,6 @@ with the `kubeadm init` and `kubeadm join` workflow to deploy Kubernetes cluster
You may also want to set `--cri-socket` to `kubeadm init` and `kubeadm reset` when
using an external CRI implementation.
### Using internal IPs in your cluster
In order to set up a cluster where the master and worker nodes communicate with internal IP addresses (instead of public ones), execute following steps.
1. When running init, you must make sure you specify an internal IP for the API server's bind address, like so:
`kubeadm init --apiserver-advertise-address=<private-master-ip>`
2. When a master or worker node has been provisioned, add a flag to `/etc/systemd/system/kubelet.service.d/10-kubeadm.conf` that specifies the private IP of the worker node:
`--node-ip=<private-node-ip>`
3. Finally, when you run `kubeadm join`, make sure you provide the private IP of the API server addressed as defined in step 1.
### Setting the node name
By default, `kubeadm` assigns a node name based on a machine's host address. You can override this setting with the `--node-name`flag.
@@ -296,27 +283,23 @@ manager, and scheduler run as [DaemonSet pods](/docs/concepts/workloads/controll
configured via the Kubernetes API instead of [static pods](/docs/tasks/administer-cluster/static-pod/)
configured in the kubelet via static files.
To create a self-hosted cluster, pass the flag `--feature-gates=SelfHosting=true` to `kubeadm init`.
{{< caution >}}
`SelfHosting` is an alpha feature. It is deprecated in 1.12
and will be removed in 1.13.
{{< /caution >}}
To create a self-hosted cluster see the `kubeadm alpha selfhosting` command.
#### Caveats
Self-hosting in 1.8 and later has some important limitations. In particular, a
self-hosted cluster _cannot recover from a reboot of the control-plane node_
without manual intervention. This and other limitations are expected to be
resolved before self-hosting graduates from alpha.
1. Self-hosting in 1.8 and later has some important limitations. In particular, a
self-hosted cluster _cannot recover from a reboot of the control-plane node_
without manual intervention.
By default, self-hosted control plane Pods rely on credentials loaded from
[`hostPath`](/docs/concepts/storage/volumes/#hostpath)
volumes. Except for initial creation, these credentials are not managed by
kubeadm.
1. A self-hosted cluster is not upgradeable using `kubeadm upgrade`.
In kubeadm 1.8, the self-hosted portion of the control plane does not include etcd,
which still runs as a static Pod.
1. By default, self-hosted control plane Pods rely on credentials loaded from
[`hostPath`](https://kubernetes.io/docs/concepts/storage/volumes/#hostpath)
volumes. Except for initial creation, these credentials are not managed by
kubeadm.
1. The self-hosted portion of the control plane does not include etcd,
which still runs as a static Pod.
#### Process
@@ -345,35 +328,16 @@ In summary, `kubeadm alpha selfhosting` works as follows:
### Running kubeadm without an internet connection
For running kubeadm without an internet connection you have to pre-pull the required master images for the version of choice:
| Image Name | v1.10 release branch version |
|--------------------------------------------|------------------------------|
| k8s.gcr.io/kube-apiserver-${ARCH} | v1.10.x |
| k8s.gcr.io/kube-controller-manager-${ARCH} | v1.10.x |
| k8s.gcr.io/kube-scheduler-${ARCH} | v1.10.x |
| k8s.gcr.io/kube-proxy-${ARCH} | v1.10.x |
| k8s.gcr.io/etcd-${ARCH} | 3.1.12 |
| k8s.gcr.io/pause-${ARCH} | 3.1 |
| k8s.gcr.io/k8s-dns-sidecar-${ARCH} | 1.14.8 |
| k8s.gcr.io/k8s-dns-kube-dns-${ARCH} | 1.14.8 |
| k8s.gcr.io/k8s-dns-dnsmasq-nanny-${ARCH} | 1.14.8 |
| coredns/coredns | 1.0.6 |
Here `v1.10.x` means the "latest patch release of the v1.10 branch".
`${ARCH}` can be one of: `amd64`, `arm`, `arm64`, `ppc64le` or `s390x`.
If you run Kubernetes version 1.10 or earlier, and if you set `--feature-gates=CoreDNS=true`,
you must also use the `coredns/coredns` image, instead of the three `k8s-dns-*` images.
For running kubeadm without an internet connection you have to pre-pull the required control-plane images.
In Kubernetes 1.11 and later, you can list and pull the images using the `kubeadm config images` sub-command:
```
```shell
kubeadm config images list
kubeadm config images pull
```
Starting with Kubernetes 1.12, the `k8s.gcr.io/kube-*`, `k8s.gcr.io/etcd` and `k8s.gcr.io/pause` images
In Kubernetes 1.12 and later, the `k8s.gcr.io/kube-*`, `k8s.gcr.io/etcd` and `k8s.gcr.io/pause` images
don't require an `-${ARCH}` suffix.
### Automating kubeadm
@@ -381,7 +345,7 @@ don't require an `-${ARCH}` suffix.
Rather than copying the token you obtained from `kubeadm init` to each node, as
in the [basic kubeadm tutorial](/docs/setup/independent/create-cluster-kubeadm/), you can parallelize the
token distribution for easier automation. To implement this automation, you must
know the IP address that the master will have after it is started.
know the IP address that the control-plane node will have after it is started.
1. Generate a token. This token must have the form `<6 character string>.<16
character string>`. More formally, it must match the regex:
@@ -389,7 +353,7 @@ know the IP address that the master will have after it is started.
kubeadm can generate a token for you:
```bash
```shell
kubeadm token generate
```
@@ -0,0 +1,62 @@
---
title: kubeadm join phase
weight: 90
---
In v1.14.0, kubeadm introduces the `kubeadm join phase` command with the aim of making kubeadm more modular. This modularity enables you to invoke atomic sub-steps of the join process.
Hence, you can let kubeadm do some parts and fill in yourself where you need customizations.
`kubeadm join phase` is consistent with the [kubeadm join workflow](/docs/reference/setup-tools/kubeadm/kubeadm-join/#join-workflow),
and behind the scene both use the same code.
## kubeadm join phase {#cmd-join-phase}
{{< tabs name="tab-phase" >}}
{{< tab name="phase" include="generated/kubeadm_join_phase.md" />}}
{{< /tabs >}}
## kubeadm join phase preflight {#cmd-join-phase-preflight}
Using this phase you can execute preflight checks on a joining node.
{{< tabs name="tab-preflight" >}}
{{< tab name="preflight" include="generated/kubeadm_join_phase_preflight.md" />}}
{{< /tabs >}}
## kubeadm join phase control-plane-prepare {#cmd-join-phase-control-plane-prepare}
Using this phase you can prepare a node for serving a control-plane.
{{< tabs name="tab-control-plane-prepare" >}}
{{< tab name="control-plane-prepare" include="generated/kubeadm_join_phase_control-plane-prepare.md" />}}
{{< tab name="all" include="generated/kubeadm_join_phase_control-plane-prepare_all.md" />}}
{{< tab name="download-certs" include="generated/kubeadm_join_phase_control-plane-prepare_download-certs.md" />}}
{{< tab name="certs" include="generated/kubeadm_join_phase_control-plane-prepare_certs.md" />}}
{{< tab name="kubeconfig" include="generated/kubeadm_join_phase_control-plane-prepare_kubeconfig.md" />}}
{{< tab name="control-plane" include="generated/kubeadm_join_phase_control-plane-prepare_control-plane.md" />}}
{{< /tabs >}}
## kubeadm join phase kubelet-start {#cmd-join-phase-kubelet-start}
Using this phase you can write the kubelet settings, certificates and (re)start the kubelet.
{{< tabs name="tab-kubelet-start" >}}
{{< tab name="kubelet-start" include="generated/kubeadm_join_phase_kubelet-start.md" />}}
{{< /tabs >}}
## kubeadm join phase control-plane-join {#cmd-join-phase-control-plane-join}
Using this phase you can join a node as a control-plane instance.
{{< tabs name="tab-control-plane-join" >}}
{{< tab name="control-plane-join" include="generated/kubeadm_join_phase_control-plane-join.md" />}}
{{< tab name="all" include="generated/kubeadm_join_phase_control-plane-join_all.md" />}}
{{< tab name="etcd" include="generated/kubeadm_join_phase_control-plane-join_etcd.md" />}}
{{< tab name="update-status" include="generated/kubeadm_join_phase_control-plane-join_update-status.md" />}}
{{< tab name="mark-control-plane" include="generated/kubeadm_join_phase_control-plane-join_mark-control-plane.md" />}}
{{< /tabs >}}
## What's next
* [kubeadm init](/docs/reference/setup-tools/kubeadm/kubeadm-init/) to bootstrap a Kubernetes control-plane node
* [kubeadm join](/docs/reference/setup-tools/kubeadm/kubeadm-join/) to connect a node to the cluster
* [kubeadm reset](/docs/reference/setup-tools/kubeadm/kubeadm-reset/) to revert any changes made to this host by `kubeadm init` or `kubeadm join`
* [kubeadm alpha](/docs/reference/setup-tools/kubeadm/kubeadm-alpha/) to try experimental functionality
@@ -14,23 +14,16 @@ This command initializes a Kubernetes worker node and joins it to the cluster.
{{% capture body %}}
{{< include "generated/kubeadm_join.md" >}}
### The joining workflow
### The join workflow {#join-workflow}
`kubeadm join` bootstraps a Kubernetes worker node and joins it to the cluster.
This action consists of the following steps:
`kubeadm join` bootstraps a Kubernetes worker node or a control-plane node and adds it to the cluster.
This action consists of the following steps for worker nodes:
1. kubeadm downloads necessary cluster information from the API server.
By default, it uses the bootstrap token and the CA key hash to verify the
authenticity of that data. The root CA can also be discovered directly via a
file or URL.
1. If kubeadm is invoked with `--feature-gates=DynamicKubeletConfig` enabled,
it first retrieves the kubelet init configuration from the master and writes it to
the disk. When kubelet starts up, kubeadm updates the node `Node.spec.configSource` property of the node.
See [Set Kubelet parameters via a config file](/docs/tasks/administer-cluster/kubelet-config-file/)
and [Reconfigure a Node's Kubelet in a Live Cluster](/docs/tasks/administer-cluster/reconfigure-kubelet/)
for more information about Dynamic Kubelet Configuration.
1. Once the cluster information is known, kubelet can start the TLS bootstrapping
process.
@@ -41,6 +34,40 @@ This action consists of the following steps:
1. Finally, kubeadm configures the local kubelet to connect to the API
server with the definitive identity assigned to the node.
For control-plane nodes additional steps are performed:
1. Downloading certificates shared among control-plane nodes from the cluster
(if explicitly requested by the user).
1. Generating control-plane component manifests, certificates and kubeconfig.
1. Adding new local etcd member.
1. Adding this node to the ClusterStatus of the kubeadm cluster.
### Using join phases with kubeadm {#join-phases}
Kubeadm allows you join a node to the cluster in phases. The `kubeadm join phase` command was added in v1.14.0.
To view the ordered list of phases and sub-phases you can call `kubeadm join --help`. The list will be located
at the top of the help screen and each phase will have a description next to it.
Note that by calling `kubeadm join` all of the phases and sub-phases will be executed in this exact order.
Some phases have unique flags, so if you want to have a look at the list of available options add `--help`, for example:
```shell
kubeadm join phase kubelet-start --help
```
Similar to the [kubeadm init phase](/docs/reference/setup-tools/kubeadm/kubeadm-init/#init-phases)
command, `kubadm join phase` allows you to skip a list of phases using the `--skip-phases` flag.
For example:
```shell
sudo kubeadm join --skip-phases=preflight --config=config.yaml
```
### Discovering what cluster CA to trust
The kubeadm discovery has several options, each with security tradeoffs.
@@ -56,27 +83,35 @@ that the API server certificate is valid under the root CA.
The CA key hash has the format `sha256:<hex_encoded_hash>`. By default, the hash value is returned in the `kubeadm join` command printed at the end of `kubeadm init` or in the output of `kubeadm token create --print-join-command`. It is in a standard format (see [RFC7469](https://tools.ietf.org/html/rfc7469#section-2.4)) and can also be calculated by 3rd party tools or provisioning systems. For example, using the OpenSSL CLI:
```bash
```shell
openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'
```
**Example `kubeadm join` command:**
**Example `kubeadm join` commands:**
```bash
For worker nodes:
```shell
kubeadm join --discovery-token abcdef.1234567890abcdef --discovery-token-ca-cert-hash sha256:1234..cdef 1.2.3.4:6443
```
For control-plane nodes:
```shell
kubeadm join --discovery-token abcdef.1234567890abcdef --discovery-token-ca-cert-hash sha256:1234..cdef --experimental-control-plane 1.2.3.4:6443
```
**Advantages:**
- Allows bootstrapping nodes to securely discover a root of trust for the
master even if other worker nodes or the network are compromised.
control-plane node even if other worker nodes or the network are compromised.
- Convenient to execute manually since all of the information required fits
into a single `kubeadm join` command that is easy to copy and paste.
**Disadvantages:**
- The CA hash is not normally known until the master has been provisioned,
- The CA hash is not normally known until the control-plane node has been provisioned,
which can make it more difficult to build automated provisioning tools that
use kubeadm. By generating your CA in beforehand, you may workaround this
limitation though.
@@ -86,13 +121,13 @@ kubeadm join --discovery-token abcdef.1234567890abcdef --discovery-token-ca-cert
_This was the default in Kubernetes 1.7 and earlier_, but comes with some
important caveats. This mode relies only on the symmetric token to sign
(HMAC-SHA256) the discovery information that establishes the root of trust for
the master. It's still possible in Kubernetes 1.8 and above using the
the control-plane. It's still possible in Kubernetes 1.8 and above using the
`--discovery-token-unsafe-skip-ca-verification` flag, but you should consider
using one of the other modes if possible.
**Example `kubeadm join` command:**
```
```shell
kubeadm join --token abcdef.1234567890abcdef --discovery-token-unsafe-skip-ca-verification 1.2.3.4:6443`
```
@@ -100,7 +135,7 @@ kubeadm join --token abcdef.1234567890abcdef --discovery-token-unsafe-skip-ca-ve
- Still protects against many network-level attacks.
- The token can be generated ahead of time and shared with the master and
- The token can be generated ahead of time and shared with the control-plane node and
worker nodes, which can then bootstrap in parallel without coordination. This
allows it to be used in many provisioning scenarios.
@@ -108,11 +143,11 @@ kubeadm join --token abcdef.1234567890abcdef --discovery-token-unsafe-skip-ca-ve
- If an attacker is able to steal a bootstrap token via some vulnerability,
they can use that token (along with network-level access) to impersonate the
master to other bootstrapping nodes. This may or may not be an appropriate
control-plane node to other bootstrapping nodes. This may or may not be an appropriate
tradeoff in your environment.
#### File or HTTPS-based discovery
This provides an out-of-band way to establish a root of trust between the master
This provides an out-of-band way to establish a root of trust between the control-plane node
and bootstrapping nodes. Consider using this mode if you are building automated provisioning
using kubeadm.
@@ -125,12 +160,12 @@ using kubeadm.
**Advantages:**
- Allows bootstrapping nodes to securely discover a root of trust for the
master even if the network or other worker nodes are compromised.
control-plane node even if the network or other worker nodes are compromised.
**Disadvantages:**
- Requires that you have some way to carry the discovery information from
the master to the bootstrapping nodes. This might be possible, for example,
the control-plane node to the bootstrapping nodes. This might be possible, for example,
via your cloud provider or provisioning tool. The information in this file is
not secret, but HTTPS or equivalent is required to ensure its integrity.
@@ -145,21 +180,21 @@ By default, there is a CSR auto-approver enabled that basically approves any cli
for a kubelet when a Bootstrap Token was used when authenticating. If you don't want the cluster to
automatically approve kubelet client certs, you can turn it off by executing this command:
```console
```shell
$ kubectl delete clusterrolebinding kubeadm:node-autoapprove-bootstrap
```
After that, `kubeadm join` will block until the admin has manually approved the CSR in flight:
```console
$ kubectl get csr
```shell
kubectl get csr
NAME AGE REQUESTOR CONDITION
node-csr-c69HXe7aYcqkS1bKmH4faEnHAWxn6i2bHZ2mD04jZyQ 18s system:bootstrap:878f07 Pending
$ kubectl certificate approve node-csr-c69HXe7aYcqkS1bKmH4faEnHAWxn6i2bHZ2mD04jZyQ
kubectl certificate approve node-csr-c69HXe7aYcqkS1bKmH4faEnHAWxn6i2bHZ2mD04jZyQ
certificatesigningrequest "node-csr-c69HXe7aYcqkS1bKmH4faEnHAWxn6i2bHZ2mD04jZyQ" approved
$ kubectl get csr
kubectl get csr
NAME AGE REQUESTOR CONDITION
node-csr-c69HXe7aYcqkS1bKmH4faEnHAWxn6i2bHZ2mD04jZyQ 1m system:bootstrap:878f07 Approved,Issued
```
@@ -169,15 +204,15 @@ Only after `kubectl certificate approve` has been run, `kubeadm join` can procee
#### Turning off public access to the cluster-info ConfigMap
In order to achieve the joining flow using the token as the only piece of validation information, a
ConfigMap with some data needed for validation of the master's identity is exposed publicly by
ConfigMap with some data needed for validation of the control-plane node's identity is exposed publicly by
default. While there is no private data in this ConfigMap, some users might wish to turn
it off regardless. Doing so will disable the ability to use the `--discovery-token` flag of the
`kubeadm join` flow. Here are the steps to do so:
* Fetch the `cluster-info` file from the API Server:
```console
$ kubectl -n kube-public get cm cluster-info -o yaml | grep "kubeconfig:" -A11 | grep "apiVersion" -A10 | sed "s/ //" | tee cluster-info.yaml
```shell
kubectl -n kube-public get cm cluster-info -o yaml | grep "kubeconfig:" -A11 | grep "apiVersion" -A10 | sed "s/ //" | tee cluster-info.yaml
apiVersion: v1
clusters:
- cluster:
@@ -195,8 +230,8 @@ users: []
* Turn off public access to the `cluster-info` ConfigMap:
```console
$ kubectl -n kube-public delete rolebinding kubeadm:bootstrap-signer-clusterinfo
```shell
kubectl -n kube-public delete rolebinding kubeadm:bootstrap-signer-clusterinfo
```
These commands should be run after `kubeadm init` but before `kubeadm join`.
@@ -214,7 +249,7 @@ contain a `JoinConfiguration` structure.
To print the default values of `JoinConfiguration` run the following command:
```bash
```shell
kubeadm config print-default --api-objects=JoinConfiguration
```
@@ -317,6 +317,116 @@ Some values of an object are typically generated before the object is persisted.
* Any field set by a mutating admission controller
* For the `Service` resource: Ports or IPs that kube-apiserver assigns to v1.Service objects
{{% /capture %}}
## Server Side Apply
{{< feature-state for_k8s_version="v1.14" state="alpha" >}} Server Side Apply allows clients other than kubectl to perform the Apply operation, and will eventually fully replace the complicated Client Side Apply logic that only exists in kubectl. If the Server Side Apply feature is enabled, the `PATCH` endpoint accepts the additional `application/apply-patch+yaml` content type. Users of Server Side Apply can send partially specified objects to this endpoint. An applied config should always include every field that the applier has an opinion about.
### Enable the Server Side Apply alpha feature
Server Side Apply is an alpha feature, so it is disabled by default. To turn this [feature gate](/docs/reference/command-line-tools-reference/feature-gates) on,
you need to include the `--feature-gates ServerSideApply=true` flag when starting `kube-apiserver`.
If you have multiple `kube-apiserver` replicas, all should have the same flag setting.
### Field Management
Compared to the `last-applied` annotation managed by `kubectl`, Server Side Apply uses a more declarative approach, which tracks a user's field management, rather than a user's last applied state. This means that as a side effect of using Server Side Apply, information about which field manager manages each field in an object also becomes available.
For a user to manage a field, in the Server Side Apply sense, means that the user relies on and expects the value of the field not to change. The user who last made an assertion about the value of a field will be recorded as the current field manager. This can be done either by changing the value with `POST`, `PUT`, or non-apply `PATCH`, or by including the field in a config sent to the Server Side Apply endpoint. Any applier that tries to change the field which is managed by someone else will get its request rejected (if not forced, see the Conflicts section below).
Field management is stored in a newly introduced `managedFields` field that is part of an object's [`metadata`](/docs/reference/generated/kubernetes-api/v1.14/#objectmeta-v1-meta).
A simple example of an object created by Server Side Apply could look like this:
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: test-cm
namespace: default
labels:
test-label: test
managedFields:
- manager: kubectl
operation: Apply
apiVersion: v1
fields:
f:metadata:
f:labels:
f:test-label: {}
f:data:
f:key: {}
data:
key: some value
```
The above object contains a single manager in `metadata.managedFields`. The manager consists of basic information about the managing entity itself, like operation type, api version, and the fields managed by it.
{{< note >}} This field is managed by the apiserver and should not be changed by the user. {{< /note >}}
Nevertheless it is possible to change `metadata.managedFields` through an `Update` operation. Doing so is highly discouraged, but might be a reasonable option to try if, for example, the `managedFields` get into an inconsistent state (which clearly should not happen).
### Operations
The two operation types considered by this feature are `Apply` (`PATCH` with content type `application/apply-patch+yaml`) and `Update` (all other operations which modify the object). Both operations update the `managedFields`, but behave a little differently.
For instance, only the apply operation fails on conflicts while update does not. Also, apply operations are required to identify themselves by providing a `fieldManager` query parameter, while the query parameter is optional for update operations.
An example object with multiple managers could look like this:
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: test-cm
namespace: default
labels:
test-label: test
managedFields:
- manager: kubectl
operation: Apply
apiVersion: v1
fields:
f:metadata:
f:labels:
f:test-label: {}
- manager: kube-controller-manager
operation: Update
apiVersion: v1
time: '2019-03-30T16:00:00.000Z'
fields:
f:data:
f:key: {}
data:
key: new value
```
In this example, a second operation was run as an `Update` by the manager called `kube-controller-manager`. The update changed a value in the data field which caused the field's management to change to the `kube-controller-manager`.
{{< note >}}If this update would have been an `Apply` operation, the operation would have failed due to conflicting ownership.{{< /note >}}
### Merge Strategy
The merging strategy, implemented with Server Side Apply, provides a generally more stable object lifecycle.
Server Side Apply tries to merge fields based on the fact who manages them instead of overruling just based on values.
This way it is intended to make it easier and more stable for multiple actors updating the same object by causing less unexpected interference.
When a user sends a partially specified object to the Server Side Apply endpoint, the server merges it with the live object favoring the value in the applied config if it is specified in both places. If the set of items present in the applied config is not a superset of the items applied by the same user last time, each missing item not managed by any other field manager is removed. For more information about how an object's schema is used to make decisions when merging, see [sigs.k8s.io/structured-merge-diff](https://sigs.k8s.io/structured-merge-diff).
### Conflicts
A conflict is a special status error that occurs when an `Apply` operation tries to change a field, which another user also claims to manage. This prevents an applier from unintentionally overwriting the value set by another user. When this occurs, the applier has 3 options to resolve the conflicts:
* **Overwrite value, become sole manager:** If overwriting the value was intentional (or if the applier is an automated process like a controller) the applier should set the `force` query parameter to true and make the request again. This forces the operation to succeed, changes the value of the field, and removes the field from all other managers' entries in managedFields.
* **Don't overwrite value, give up management claim:** If the applier doesn't care about the value of the field anymore, they can remove it from their config and make the request again. This leaves the value unchanged, and causes the field to be removed from the applier's entry in managedFields.
* **Don't overwrite value, become shared manager:** If the applier still cares about the value of the field, but doesn't want to overwrite it, they can change the value of the field in their config to match the value of the object on the server, and make the request again. This leaves the value unchanged, and causes the field's management to be shared by the applier and all other field managers that already claimed to manage it.
### Comparison with Client Side Apply
A consequence of the conflict detection and resolution implemented by Server Side Apply is that an applier always has up to date field values in their local state. If they don't, they get a conflict the next time they apply. Any of the three options to resolve conflicts results in the applied config being an up to date subset of the object on the server's fields.
This is different from Client Side Apply, where outdated values which have been overwritten by other users are left in an applier's local config. These values only become accurate when the user updates that specific field, if ever, and an applier has no way of knowing whether their next apply will overwrite other users' changes.
Another difference is that an applier using Client Side Apply is unable to change the API version they are using, but Server Side Apply supports this use case.
### Custom Resources
Server Side Apply currently treats all custom resources as unstructured data. All keys are treated the same as struct fields, and all lists are considered atomic. In the future, it will use the validation field in Custom Resource Definitions to allow Custom Resource authors to define how to how to merge their own objects.