From f4ce844fcbec289eac601abc08046407b7e18612 Mon Sep 17 00:00:00 2001 From: Michael Hausenblas Date: Sat, 16 Sep 2017 11:27:12 +0200 Subject: [PATCH] addresses `kubeadm upgrade` review comments CC: @luxas --- ...-upgrade-cmd.md => kubeadm-upgrade-1-8.md} | 155 ++++++++++++------ 1 file changed, 103 insertions(+), 52 deletions(-) rename docs/tasks/administer-cluster/{kubeadm-upgrade-cmd.md => kubeadm-upgrade-1-8.md} (56%) diff --git a/docs/tasks/administer-cluster/kubeadm-upgrade-cmd.md b/docs/tasks/administer-cluster/kubeadm-upgrade-1-8.md similarity index 56% rename from docs/tasks/administer-cluster/kubeadm-upgrade-cmd.md rename to docs/tasks/administer-cluster/kubeadm-upgrade-1-8.md index 5c9edc2fcb..d1ef1628bd 100644 --- a/docs/tasks/administer-cluster/kubeadm-upgrade-cmd.md +++ b/docs/tasks/administer-cluster/kubeadm-upgrade-1-8.md @@ -4,12 +4,12 @@ approvers: - luxas - roberthbailey - jbeda -title: Upgrading kubeadm clusters +title: Upgrading kubeadm clusters from 1.7 to 1.8 --- {% capture overview %} -This guide is for upgrading `kubeadm` clusters from version 1.7.x to 1.8.x. +This guide is for upgrading `kubeadm` clusters from version 1.7.x to 1.8.x, as well as 1.7.x to 1.7.y and 1.8.x to 1.8.y where `y > x`. See also [upgrading kubeadm clusters from 1.6 to 1.7](/docs/tasks/administer-cluster/kubeadm-upgrade-1-7/) if you're on a 1.6 cluster currently. {% endcapture %} @@ -41,95 +41,140 @@ $ export ARCH=amd64 # or: arm, arm64, ppc64le, s390x $ curl -sSL https://dl.k8s.io/release/${VERSION}/bin/linux/${ARCH}/kubeadm > /usr/bin/kubeadm ``` -1. On the master node, run the following: +2. If this the first time you use `kubeadm upgrade`, in order to preserve the configuration for future upgrades, do: + +Note that for below you will need to recall what CLI args you passed to `kubeadm init` the first time. + +If you used flags, do: + +```shell +$ kubeadm config upload from-flags [flags] +``` + +Where `flags` can be empty. + +If you used a config file, do: + +```shell +$ kubeadm config upload from-file --config [config] +``` + +Where the `config` is mandatory. + +3. On the master node, run the following: ```shell $ kubeadm upgrade plan +[preflight] Running pre-flight checks [upgrade] Making sure the cluster is healthy: [upgrade/health] Checking API Server health: Healthy [upgrade/health] Checking Node health: All Nodes are healthy -[upgrade/health] Checking if control plane is Static Pod-hosted or Self-Hosted: Static Pod-hosted. -[upgrade/health] NOTE: kubeadm will upgrade your Static Pod-hosted control plane to a Self-Hosted one when upgrading if --feature-gates=SelfHosting=true is set (which is the default) -[upgrade/health] If you strictly want to continue using a Static Pod-hosted control plane, set --feature-gates=SelfHosting=true when running 'kubeadm upgrade apply' -[upgrade/health] Checking Static Pod manifests exists on disk: All required Static Pod manifests exist on disk -[upgrade] Making sure the configuration is correct: -[upgrade/config] Reading configuration from the cluster (you can get this with 'kubectl -n kube-system get cm kubeadm-config -oyaml') +[upgrade/health] Checking Static Pod manifests exists on disk: All manifests exist on disk +[upgrade/config] Making sure the configuration is correct: +[upgrade/config] Reading configuration from the cluster... +[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml' [upgrade] Fetching available versions to upgrade to: [upgrade/versions] Cluster version: v1.7.1 [upgrade/versions] kubeadm version: v1.8.0 -[upgrade/versions] Latest stable version: v1.7.3 -[upgrade/versions] Latest version in the v1.7 series: v1.7.3 +[upgrade/versions] Latest stable version: v1.8.0 +[upgrade/versions] Latest version in the v1.7 series: v1.7.6 -Components that must be upgraded manually after you've upgraded the control plane with `kubeadm upgrade apply`: +Components that must be upgraded manually after you've upgraded the control plane with 'kubeadm upgrade apply': COMPONENT CURRENT AVAILABLE -Kubelet 1 x v1.7.0 v1.7.3 +Kubelet 1 x v1.7.1 v1.7.6 Upgrade to the latest version in the v1.7 series: COMPONENT CURRENT AVAILABLE -API Server v1.7.1 v1.7.3 -Controller Manager v1.7.1 v1.7.3 -Scheduler v1.7.1 v1.7.3 -Kube Proxy v1.7.1 v1.7.3 +API Server v1.7.1 v1.7.6 +Controller Manager v1.7.1 v1.7.6 +Scheduler v1.7.1 v1.7.6 +Kube Proxy v1.7.1 v1.7.6 Kube DNS 1.14.4 1.14.4 You can now apply the upgrade by executing the following command: - kubeadm upgrade apply --version v1.7.3 + kubeadm upgrade apply v1.7.6 + +_____________________________________________________________________ + +Components that must be upgraded manually after you've upgraded the control plane with 'kubeadm upgrade apply': +COMPONENT CURRENT AVAILABLE +Kubelet 1 x v1.7.1 v1.8.0 + +Upgrade to the latest experimental version: + +COMPONENT CURRENT AVAILABLE +API Server v1.7.1 v1.8.0 +Controller Manager v1.7.1 v1.8.0 +Scheduler v1.7.1 v1.8.0 +Kube Proxy v1.7.1 v1.8.0 +Kube DNS 1.14.4 1.14.4 + +You can now apply the upgrade by executing the following command: + + kubeadm upgrade apply v1.8.0 + +Note: Before you do can perform this upgrade, you have to update kubeadm to v1.8.0 + +_____________________________________________________________________ ``` The `kubeadm upgrade plan` checks that your cluster is in an upgradeable state and fetches the versions available to upgrade to in an user-friendly way. -1. Pick a version to upgrade to and run, for example, `kubeadm upgrade apply` as follows: +4. Pick a version to upgrade to and run, for example, `kubeadm upgrade apply` as follows: ```shell -$ kubeadm upgrade apply --version v1.8.0 +$ kubeadm upgrade apply v1.8.0 +[preflight] Running pre-flight checks [upgrade] Making sure the cluster is healthy: [upgrade/health] Checking API Server health: Healthy [upgrade/health] Checking Node health: All Nodes are healthy -[upgrade/health] Checking if control plane is Static Pod-hosted or Self-Hosted: Static Pod-hosted. -[upgrade/health] NOTE: kubeadm will upgrade your Static Pod-hosted control plane to a Self-Hosted one when upgrading if --feature-gates=SelfHosting=true is set (which is the default) -[upgrade/health] If you strictly want to continue using a Static Pod-hosted control plane, set --feature-gates=SelfHosting=true when running 'kubeadm upgrade apply' -[upgrade/health] Checking Static Pod manifests exists on disk: All required Static Pod manifests exist on disk -[upgrade] Making sure the configuration is correct: -[upgrade/config] Reading configuration from the cluster (you can get this with 'kubectl -n kube-system get cm kubeadm-config -oyaml') +[upgrade/health] Checking Static Pod manifests exists on disk: All manifests exist on disk +[upgrade/config] Making sure the configuration is correct: +[upgrade/config] Reading configuration from the cluster... +[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml' [upgrade/version] You have chosen to upgrade to version "v1.8.0" [upgrade/versions] Cluster version: v1.7.1 [upgrade/versions] kubeadm version: v1.8.0 -[upgrade/confirm] Are you sure you want to proceed with the upgrade? [y/N]: Y [upgrade/prepull] Will prepull images for components [kube-apiserver kube-controller-manager kube-scheduler] [upgrade/prepull] Prepulling image for component kube-scheduler. [upgrade/prepull] Prepulling image for component kube-apiserver. [upgrade/prepull] Prepulling image for component kube-controller-manager. -[upgrade/prepull] Prepulled image for component kube-scheduler. +[apiclient] Found 0 Pods for label selector k8s-app=upgrade-prepull-kube-scheduler +[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-scheduler +[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-apiserver +[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-controller-manager [upgrade/prepull] Prepulled image for component kube-apiserver. [upgrade/prepull] Prepulled image for component kube-controller-manager. +[upgrade/prepull] Prepulled image for component kube-scheduler. [upgrade/prepull] Successfully prepulled the images for all the control plane components [upgrade/apply] Upgrading your Static Pod-hosted control plane to version "v1.8.0"... -[upgrade/staticpods] Wrote upgraded Static Pod manifests to "/tmp/kubeadm-upgrade830923296" -[upgrade/staticpods] Moved upgraded manifest to "/etc/kubernetes/manifests/kube-apiserver.yaml" and backuped old manifest to "/tmp/kubeadm-upgrade830923296/old-manifests/kube-apiserver.yaml" +[upgrade/staticpods] Writing upgraded Static Pod manifests to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests432902769" +[controlplane] Wrote Static Pod manifest for component kube-apiserver to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests432902769/kube-apiserver.yaml" +[controlplane] Wrote Static Pod manifest for component kube-controller-manager to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests432902769/kube-controller-manager.yaml" +[controlplane] Wrote Static Pod manifest for component kube-scheduler to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests432902769/kube-scheduler.yaml" +[upgrade/staticpods] Moved upgraded manifest to "/etc/kubernetes/manifests/kube-apiserver.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests155856668/kube-apiserver.yaml" [upgrade/staticpods] Waiting for the kubelet to restart the component [apiclient] Found 1 Pods for label selector component=kube-apiserver [upgrade/staticpods] Component "kube-apiserver" upgraded successfully! -[upgrade/staticpods] Moved upgraded manifest to "/etc/kubernetes/manifests/kube-controller-manager.yaml" and backuped old manifest to "/tmp/kubeadm-upgrade830923296/old-manifests/kube-controller-manager.yaml" +[upgrade/staticpods] Moved upgraded manifest to "/etc/kubernetes/manifests/kube-controller-manager.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests155856668/kube-controller-manager.yaml" [upgrade/staticpods] Waiting for the kubelet to restart the component [apiclient] Found 1 Pods for label selector component=kube-controller-manager [upgrade/staticpods] Component "kube-controller-manager" upgraded successfully! -[upgrade/staticpods] Moved upgraded manifest to "/etc/kubernetes/manifests/kube-scheduler.yaml" and backuped old manifest to "/tmp/kubeadm-upgrade830923296/old-manifests/kube-scheduler.yaml" +[upgrade/staticpods] Moved upgraded manifest to "/etc/kubernetes/manifests/kube-scheduler.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests155856668/kube-scheduler.yaml" [upgrade/staticpods] Waiting for the kubelet to restart the component [apiclient] Found 1 Pods for label selector component=kube-scheduler -[apiclient] Found 0 Pods for label selector component=kube-scheduler -[apiclient] Found 1 Pods for label selector component=kube-scheduler [upgrade/staticpods] Component "kube-scheduler" upgraded successfully! -[apiclient] Found 0 Pods for label selector k8s-app=self-hosted-kube-apiserver -[apiclient] Found 1 Pods for label selector k8s-app=self-hosted-kube-apiserver -[apiclient] Found 0 Pods for label selector k8s-app=self-hosted-kube-controller-manager -[apiclient] Found 1 Pods for label selector k8s-app=self-hosted-kube-controller-manager -[apiclient] Found 0 Pods for label selector k8s-app=self-hosted-kube-scheduler -[apiclient] Found 1 Pods for label selector k8s-app=self-hosted-kube-scheduler -[apiconfig] Created RBAC rules -[addons] Applied essential addon: kube-proxy +[uploadconfig] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace +[bootstraptoken] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials +[bootstraptoken] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token [addons] Applied essential addon: kube-dns +[addons] Applied essential addon: kube-proxy + +[upgrade/successful] SUCCESS! Your cluster was upgraded to "v1.8.0". Enjoy! + +[upgrade/kubelet] Now that your control plane is upgraded, please proceed with upgrading your kubelets in turn. ``` `kubeadm upgrade apply` does the following: @@ -143,7 +188,7 @@ $ kubeadm upgrade apply --version v1.8.0 - It upgrades the control plane components or rollbacks if any of them fails to come up. - It applies the new `kube-dns` and `kube-proxy` manifests and enforces that all necessary RBAC rules are created. -1. Manually upgrade your Software Defined Network (SDN). +5. Manually upgrade your Software Defined Network (SDN). Your Container Network Interface (CNI) provider might have its own upgrade instructions to follow now. Check the [addons](/docs/concepts/cluster-administration/addons/) page to @@ -161,29 +206,35 @@ $ kubectl cordon $WORKER $ kubectl drain $WORKER ``` -1. Upgrade the `kubelet` version on the `$WORKER` node, either by using a Linux distribution-specific package manager such as `apt-get` or `yum` or manually as described in the following: +2. Upgrade the `kubelet` version on the `$WORKER` node by using a Linux distribution-specific package manager: + +If the node is running a Debian-based distro such as Ubuntu, run: ```shell -$ sudo systemctl stop kubelet -$ curl -s -L -o kubelet \ - https://storage.googleapis.com/kubernetes-release/release/v1.8.0/bin/linux/amd64/kubelet -$ chmod +x kubectl && sudo mv kubelet /usr/local/bin/ -$ sudo systemctl start kubelet +$ apt-get update +$ apt-get install -y kubelet ``` -Now, the new version of the `kubelet` should be running on the `$WORKER` node. Verify this using the following command: +If the node is running CentOS or the like, run: + +```shell +$ yum update +$ yum install -y kubelet +``` + +Now the new version of the `kubelet` should be running on the `$WORKER` node. Verify this using the following command: ```shell $ systemctl status kubelet ``` -1. Bring the `$WORKER` node back online by marking it schedulable: +3. Bring the `$WORKER` node back online by marking it schedulable: ```shell $ kubectl uncordon $WORKER ``` -1. After upgrading `kubelet` on each worker node in your cluster, verify that all nodes are available again by executing the following (from anywhere, for example, from outside the cluster): +4. After upgrading `kubelet` on each worker node in your cluster, verify that all nodes are available again by executing the following (from anywhere, for example, from outside the cluster): ```shell $ kubectl get nodes