From a1f811d8524e6e9dd8682927244252ed0e1a064b Mon Sep 17 00:00:00 2001 From: Marek Counts Date: Tue, 7 May 2019 14:30:41 -0400 Subject: [PATCH] Updated self-hosting documentation (#13866) * updated self-hosting documentation * removed more referances * updated wording * updated self-hosting warning to be more clear about the effects of pivot has on your cluster * Update content/en/docs/setup/independent/self-hosting.md Co-Authored-By: Klaven --- .../kubeadm/implementation-details.md | 20 ------ .../setup-tools/kubeadm/kubeadm-alpha.md | 4 +- .../setup-tools/kubeadm/kubeadm-init.md | 51 --------------- .../independent/create-cluster-kubeadm.md | 1 - .../en/docs/setup/independent/self-hosting.md | 63 +++++++++++++++++++ 5 files changed, 66 insertions(+), 73 deletions(-) create mode 100644 content/en/docs/setup/independent/self-hosting.md diff --git a/content/en/docs/reference/setup-tools/kubeadm/implementation-details.md b/content/en/docs/reference/setup-tools/kubeadm/implementation-details.md index 8a5fe68a8c..c2d73023ec 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/implementation-details.md +++ b/content/en/docs/reference/setup-tools/kubeadm/implementation-details.md @@ -466,26 +466,6 @@ Deploy the `kube-dns` Deployment and Service: - It's the upstream CoreDNS deployment relatively unmodified - The `kube-dns` ServiceAccount is bound to the privileges in the `system:kube-dns` ClusterRole -### Optional self-hosting - -To enable self hosting on an existing static Pod control-plane use `kubeadm alpha selfhosting pivot`. - -Self hosting basically replaces static Pods for control plane components with DaemonSets; this is achieved by executing -following procedure for API server, scheduler and controller manager static Pods: - -- Load the static Pod specification from disk -- Extract the PodSpec from static Pod manifest file -- Mutate the PodSpec to be compatible with self-hosting, and more in detail: - - Add node selector attribute targeting nodes with `node-role.kubernetes.io/master=""` label, - - Add a toleration for `node-role.kubernetes.io/master:NoSchedule` taint, - - Set `spec.DNSPolicy` to `ClusterFirstWithHostNet` -- Build a new DaemonSet object for the self-hosted component in question. Use the above mentioned PodSpec -- Create the DaemonSet resource in `kube-system` namespace. Wait until the Pods are running. -- Remove the static Pod manifest file. The kubelet will stop the original static Pod-hosted component that was running - -Please note that self hosting is not yet resilient to node restarts; this can be fixed with external checkpointing or with kubelet checkpointing - for the control plane Pods. See [self-hosting](/docs/reference/setup-tools/kubeadm/kubeadm-init/#self-hosting) for more details. - ## kubeadm join phases internal design Similarly to `kubeadm init`, also `kubeadm join` internal workflow consists of a sequence of atomic work tasks to perform. diff --git a/content/en/docs/reference/setup-tools/kubeadm/kubeadm-alpha.md b/content/en/docs/reference/setup-tools/kubeadm/kubeadm-alpha.md index ad59320140..b5114e0861 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/kubeadm-alpha.md +++ b/content/en/docs/reference/setup-tools/kubeadm/kubeadm-alpha.md @@ -51,7 +51,9 @@ to enable the DynamicKubeletConfiguration feature. ## kubeadm alpha selfhosting pivot {#cmd-selfhosting} -The subcommand `pivot` can be used to conver a static Pod-hosted control plane into a self-hosted one. +The subcommand `pivot` can be used to convert a static Pod-hosted control plane into a self-hosted one. + +[Documentation](/docs/setup/independent/self-hosting) {{< tabs name="selfhosting" >}} {{< tab name="selfhosting" include="generated/kubeadm_alpha_selfhosting.md" />}} diff --git a/content/en/docs/reference/setup-tools/kubeadm/kubeadm-init.md b/content/en/docs/reference/setup-tools/kubeadm/kubeadm-init.md index 3c2300fab4..57d456b907 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/kubeadm-init.md +++ b/content/en/docs/reference/setup-tools/kubeadm/kubeadm-init.md @@ -275,57 +275,6 @@ to the kubelet. Be aware that overriding the hostname can [interfere with cloud providers](https://github.com/kubernetes/website/pull/8873). -### Self-hosting the Kubernetes control plane {#self-hosting} - -As of 1.8, you can experimentally create a _self-hosted_ Kubernetes control -plane. This means that key components such as the API server, controller -manager, and scheduler run as [DaemonSet pods](/docs/concepts/workloads/controllers/daemonset/) -configured via the Kubernetes API instead of [static pods](/docs/tasks/administer-cluster/static-pod/) -configured in the kubelet via static files. - -To create a self-hosted cluster see the `kubeadm alpha selfhosting` command. - -#### Caveats - -1. Self-hosting in 1.8 and later has some important limitations. In particular, a - self-hosted cluster _cannot recover from a reboot of the control-plane node_ - without manual intervention. - -1. A self-hosted cluster is not upgradeable using `kubeadm upgrade`. - -1. By default, self-hosted control plane Pods rely on credentials loaded from - [`hostPath`](https://kubernetes.io/docs/concepts/storage/volumes/#hostpath) - volumes. Except for initial creation, these credentials are not managed by - kubeadm. - -1. The self-hosted portion of the control plane does not include etcd, - which still runs as a static Pod. - -#### Process - -The self-hosting bootstrap process is documented in the [kubeadm design -document](https://github.com/kubernetes/kubeadm/blob/master/docs/design/design_v1.9.md#optional-self-hosting). - -In summary, `kubeadm alpha selfhosting` works as follows: - - 1. Waits for this bootstrap static control plane to be running and - healthy. This is identical to the `kubeadm init` process without self-hosting. - - 1. Uses the static control plane Pod manifests to construct a set of - DaemonSet manifests that will run the self-hosted control plane. - It also modifies these manifests where necessary, for example adding new volumes - for secrets. - - 1. Creates DaemonSets in the `kube-system` namespace and waits for the - resulting Pods to be running. - - 1. Once self-hosted Pods are operational, their associated static Pods are deleted - and kubeadm moves on to install the next component. This triggers kubelet to - stop those static Pods. - - 1. When the original static control plane stops, the new self-hosted control - plane is able to bind to listening ports and become active. - ### Running kubeadm without an internet connection For running kubeadm without an internet connection you have to pre-pull the required control-plane images. diff --git a/content/en/docs/setup/independent/create-cluster-kubeadm.md b/content/en/docs/setup/independent/create-cluster-kubeadm.md index a91c7e25b1..cd73f6482b 100644 --- a/content/en/docs/setup/independent/create-cluster-kubeadm.md +++ b/content/en/docs/setup/independent/create-cluster-kubeadm.md @@ -44,7 +44,6 @@ but you may also build them from source for other OSes. | kubeadm alpha subcommands | alpha | | High availability | alpha | | DynamicKubeletConfig | alpha | -| Self-hosting | alpha | kubeadm's overall feature state is **GA**. Some sub-features, like the configuration diff --git a/content/en/docs/setup/independent/self-hosting.md b/content/en/docs/setup/independent/self-hosting.md new file mode 100644 index 0000000000..090b5efd5b --- /dev/null +++ b/content/en/docs/setup/independent/self-hosting.md @@ -0,0 +1,63 @@ +--- +reviewers: +- sig-cluster-lifecycle +title: Configuring your kubernetes cluster to self-host the control plane +content_template: templates/concept +weight: 100 +--- + +{{% capture overview %}} + +### Self-hosting the Kubernetes control plane {#self-hosting} + +As of 1.8, you can experimentally create a _self-hosted_ Kubernetes control +plane. This means that key components such as the API server, controller +manager, and scheduler run as [DaemonSet pods](/docs/concepts/workloads/controllers/daemonset/) +configured via the Kubernetes API instead of [static pods](/docs/tasks/administer-cluster/static-pod/) +configured in the kubelet via static files. + +To create a self-hosted cluster see the `kubeadm alpha selfhosting pivot` command. + +#### Caveats + +{{< caution >}} +This feature pivots your cluster into an unsupported state, rendering kubeadm unable +to manage you cluster any longer. This includes `kubeadm upagrade`. +{{< /caution >}} + +1. Self-hosting in 1.8 and later has some important limitations. In particular, a + self-hosted cluster _cannot recover from a reboot of the control-plane node_ + without manual intervention. + +1. By default, self-hosted control plane Pods rely on credentials loaded from + [`hostPath`](https://kubernetes.io/docs/concepts/storage/volumes/#hostpath) + volumes. Except for initial creation, these credentials are not managed by + kubeadm. + +1. The self-hosted portion of the control plane does not include etcd, + which still runs as a static Pod. + +#### Process + +The self-hosting bootstrap process is documented in the [kubeadm design +document](https://github.com/kubernetes/kubeadm/blob/master/docs/design/design_v1.9.md#optional-self-hosting). + +In summary, `kubeadm alpha selfhosting` works as follows: + + 1. Waits for this bootstrap static control plane to be running and + healthy. This is identical to the `kubeadm init` process without self-hosting. + + 1. Uses the static control plane Pod manifests to construct a set of + DaemonSet manifests that will run the self-hosted control plane. + It also modifies these manifests where necessary, for example adding new volumes + for secrets. + + 1. Creates DaemonSets in the `kube-system` namespace and waits for the + resulting Pods to be running. + + 1. Once self-hosted Pods are operational, their associated static Pods are deleted + and kubeadm moves on to install the next component. This triggers kubelet to + stop those static Pods. + + 1. When the original static control plane stops, the new self-hosted control + plane is able to bind to listening ports and become active.