From c4a0363943f61f4dc6ee0cdd52a4b72b36056dd6 Mon Sep 17 00:00:00 2001 From: Stewart-YU Date: Sun, 12 Nov 2017 21:15:59 +0800 Subject: [PATCH 1/4] Update troubleshooting-kubeadm.md Update troubleshooting docs about apiclient blocking while run `kubeadm init`. --- .../independent/troubleshooting-kubeadm.md | 41 ++++++++++++++++--- 1 file changed, 35 insertions(+), 6 deletions(-) diff --git a/docs/setup/independent/troubleshooting-kubeadm.md b/docs/setup/independent/troubleshooting-kubeadm.md index 884514b320..0a939b5a5f 100644 --- a/docs/setup/independent/troubleshooting-kubeadm.md +++ b/docs/setup/independent/troubleshooting-kubeadm.md @@ -34,13 +34,42 @@ If you see the following warnings while running `kubeadm init` Then you may be missing ebtables and ethtool on your Linux machine. You can install them with the following commands: -``` -# For ubuntu/debian users, try -apt install ebtables ethtool +- For ubuntu/debian users, try `apt install ebtables ethtool`. +- For CentOS/Fedora users, try `yum install ebtables ethtool`. + +#### kubeadm blocks waiting for `control plane` during installation + +If you see the following blocks while running `kubeadm init` -# For CentOS/Fedora users, try -yum install ebtables ethtool ``` +[kubeadm] WARNING: kubeadm is in beta, please do not use it for production clusters. +[init] Using Kubernetes version: v1.8.0 +[init] Using Authorization modes: [Node RBAC] +[preflight] Skipping pre-flight checks +[kubeadm] WARNING: starting in 1.8, tokens expire after 24 hours by default (if you require a non-expiring token use --token-ttl 0) +[certificates] Using the existing CA certificate and key. +[certificates] Using the existing API Server certificate and key. +[certificates] Using the existing API Server kubelet client certificate and key. +[certificates] Using the existing service account token signing key. +[certificates] Using the existing front-proxy CA certificate and key. +[certificates] Using the existing front-proxy client certificate and key. +[certificates] Valid certificates and keys now exist in "/etc/kubernetes/pki" +[kubeconfig] Using existing up-to-date KubeConfig file: "/etc/kubernetes/controller-manager.conf" +[kubeconfig] Using existing up-to-date KubeConfig file: "/etc/kubernetes/scheduler.conf" +[kubeconfig] Using existing up-to-date KubeConfig file: "/etc/kubernetes/admin.conf" +[kubeconfig] Using existing up-to-date KubeConfig file: "/etc/kubernetes/kubelet.conf" +[apiclient] Created API client, waiting for the control plane to become ready +``` + +Then there may can not connect to net or parameter `Cgroup Driver` diff bettween docker and kubelet in your linux machine. + +- Ensure your machine can connect to net. +- If you find some log in `var/log/message`, look like: +``` +error: failed to run Kubelet: failed to create kubelet: +misconfiguration: kubelet cgroup driver: "systemd" is different from docker cgroup driver: "cgroupfs" +``` +then modify startup parameter `KUBELET_CGROUP_ARGS=--cgroup-driver=` in `/etc/systemd/system/kubelet.service.d/10-kubeadm.conf`(Ubuntu). #### Pods in `RunContainerError`, `CrashLoopBackOff` or `Error` state @@ -141,4 +170,4 @@ The `kubectl describe pod` or `kubectl logs` commands can help you diagnose erro kubectl -n ${NAMESPACE} describe pod ${POD_NAME} kubectl -n ${NAMESPACE} logs ${POD_NAME} -c ${CONTAINER_NAME} -``` \ No newline at end of file +``` From d7278dbd9d04d42d713720d04eca8d47ceeada97 Mon Sep 17 00:00:00 2001 From: Qiming Date: Tue, 14 Nov 2017 14:34:18 +0800 Subject: [PATCH 2/4] Update troubleshooting-kubeadm.md --- .../independent/troubleshooting-kubeadm.md | 31 ++++++------------- 1 file changed, 10 insertions(+), 21 deletions(-) diff --git a/docs/setup/independent/troubleshooting-kubeadm.md b/docs/setup/independent/troubleshooting-kubeadm.md index 0a939b5a5f..8f0ca79f54 100644 --- a/docs/setup/independent/troubleshooting-kubeadm.md +++ b/docs/setup/independent/troubleshooting-kubeadm.md @@ -39,37 +39,26 @@ Then you may be missing ebtables and ethtool on your Linux machine. You can inst #### kubeadm blocks waiting for `control plane` during installation -If you see the following blocks while running `kubeadm init` +If you notice that `kubeadm init` hangs after printing out the following line ``` -[kubeadm] WARNING: kubeadm is in beta, please do not use it for production clusters. -[init] Using Kubernetes version: v1.8.0 -[init] Using Authorization modes: [Node RBAC] -[preflight] Skipping pre-flight checks -[kubeadm] WARNING: starting in 1.8, tokens expire after 24 hours by default (if you require a non-expiring token use --token-ttl 0) -[certificates] Using the existing CA certificate and key. -[certificates] Using the existing API Server certificate and key. -[certificates] Using the existing API Server kubelet client certificate and key. -[certificates] Using the existing service account token signing key. -[certificates] Using the existing front-proxy CA certificate and key. -[certificates] Using the existing front-proxy client certificate and key. -[certificates] Valid certificates and keys now exist in "/etc/kubernetes/pki" -[kubeconfig] Using existing up-to-date KubeConfig file: "/etc/kubernetes/controller-manager.conf" -[kubeconfig] Using existing up-to-date KubeConfig file: "/etc/kubernetes/scheduler.conf" -[kubeconfig] Using existing up-to-date KubeConfig file: "/etc/kubernetes/admin.conf" -[kubeconfig] Using existing up-to-date KubeConfig file: "/etc/kubernetes/kubelet.conf" [apiclient] Created API client, waiting for the control plane to become ready ``` -Then there may can not connect to net or parameter `Cgroup Driver` diff bettween docker and kubelet in your linux machine. +You may want to first check if your node has network connection problem. +Another reason that `kubeadm init` hands is that the default CGroup driver configuration +for the kubelet differs from that used by Docker. + +Check the system log file (e.g. `var/log/message`) or examine the output from `journalctl -u kubelet`. +If you see something like the following -- Ensure your machine can connect to net. -- If you find some log in `var/log/message`, look like: ``` error: failed to run Kubelet: failed to create kubelet: misconfiguration: kubelet cgroup driver: "systemd" is different from docker cgroup driver: "cgroupfs" ``` -then modify startup parameter `KUBELET_CGROUP_ARGS=--cgroup-driver=` in `/etc/systemd/system/kubelet.service.d/10-kubeadm.conf`(Ubuntu). + +you will need to fix the cgroup driver problem by following intstructions +[here](/docs/setup/indenpendent/install-kubeadm/#installing-docker). #### Pods in `RunContainerError`, `CrashLoopBackOff` or `Error` state From 8d20b9e72ab1779a6cda7965cef0c8024c34c7dd Mon Sep 17 00:00:00 2001 From: Qiming Date: Wed, 15 Nov 2017 14:06:13 +0800 Subject: [PATCH 3/4] Update troubleshooting-kubeadm.md --- docs/setup/independent/troubleshooting-kubeadm.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/setup/independent/troubleshooting-kubeadm.md b/docs/setup/independent/troubleshooting-kubeadm.md index 8f0ca79f54..d9af4adf1c 100644 --- a/docs/setup/independent/troubleshooting-kubeadm.md +++ b/docs/setup/independent/troubleshooting-kubeadm.md @@ -46,7 +46,7 @@ If you notice that `kubeadm init` hangs after printing out the following line ``` You may want to first check if your node has network connection problem. -Another reason that `kubeadm init` hands is that the default CGroup driver configuration +Another reason that `kubeadm init` hangs could be that the default cgroup driver configuration for the kubelet differs from that used by Docker. Check the system log file (e.g. `var/log/message`) or examine the output from `journalctl -u kubelet`. From 4c017787a42e55748f51aadea3caee3bae2fd6aa Mon Sep 17 00:00:00 2001 From: stewart-yu Date: Wed, 6 Dec 2017 09:36:54 +0800 Subject: [PATCH 4/4] Update troubleshooting-kubeadm.md --- .../independent/troubleshooting-kubeadm.md | 31 ++++++++++--------- 1 file changed, 16 insertions(+), 15 deletions(-) diff --git a/docs/setup/independent/troubleshooting-kubeadm.md b/docs/setup/independent/troubleshooting-kubeadm.md index d9af4adf1c..d62650115f 100644 --- a/docs/setup/independent/troubleshooting-kubeadm.md +++ b/docs/setup/independent/troubleshooting-kubeadm.md @@ -34,31 +34,32 @@ If you see the following warnings while running `kubeadm init` Then you may be missing ebtables and ethtool on your Linux machine. You can install them with the following commands: -- For ubuntu/debian users, try `apt install ebtables ethtool`. -- For CentOS/Fedora users, try `yum install ebtables ethtool`. +- For ubuntu/debian users, run `apt install ebtables ethtool`. +- For CentOS/Fedora users, run `yum install ebtables ethtool`. -#### kubeadm blocks waiting for `control plane` during installation +#### kubeadm blocks waiting for control plane during installation -If you notice that `kubeadm init` hangs after printing out the following line +If you notice that `kubeadm init` hangs after printing out the following line: ``` [apiclient] Created API client, waiting for the control plane to become ready ``` -You may want to first check if your node has network connection problem. -Another reason that `kubeadm init` hangs could be that the default cgroup driver configuration -for the kubelet differs from that used by Docker. +This may be caused by a number of problems. The most common are: -Check the system log file (e.g. `var/log/message`) or examine the output from `journalctl -u kubelet`. -If you see something like the following +- network connection problems. Check that your machine has full network connectivity before continuing. +- the default cgroup driver configuration for the kubelet differs from that used by Docker. + Check the system log file (e.g. `/var/log/message`) or examine the output from `journalctl -u kubelet`. If you see something like the following: -``` -error: failed to run Kubelet: failed to create kubelet: -misconfiguration: kubelet cgroup driver: "systemd" is different from docker cgroup driver: "cgroupfs" -``` + ```shell + error: failed to run Kubelet: failed to create kubelet: + misconfiguration: kubelet cgroup driver: "systemd" is different from docker cgroup driver: "cgroupfs" + ``` + + you will need to fix the cgroup driver problem by following intstructions + [here](/docs/setup/indenpendent/install-kubeadm/#installing-docker). +- control plane Docker containers are crashlooping or hanging. You can check this by running `docker ps` and investigating each container by running `docker logs`. -you will need to fix the cgroup driver problem by following intstructions -[here](/docs/setup/indenpendent/install-kubeadm/#installing-docker). #### Pods in `RunContainerError`, `CrashLoopBackOff` or `Error` state