whentimeslows/website
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add a section about routing errors (#7078)

Browse Source 
A common source of beffudlement when using local hypervisors or cloud
providers with peculiar interface setups, IP adressing, or network
policies for which kubelet cannot guess the right IP to use.

`awk` is being used so that the example works in CoreOS Container Linux
too.

Requested in kubernetes/kubeadm#203.
This commit is contained in:
Loic Nageleisen
2018-03-03 21:13:54 +01:00
committed by k8s-ci-robot
parent 2a49808049
commit bf661f9f44
1 changed files with 34 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/setup/independent/troubleshooting-kubeadm.md
+34
View File
@@ -181,3 +181,37 @@ If you're using flannel as the pod network inside vagrant, then you will have to
Vagrant typically assigns two interfaces to all VMs. The first, for which all hosts are assigned the IP address `10.0.2.15`, is for external traffic that gets NATed.
This may lead to problems with flannel. By default, flannel selects the first interface on a host. This leads to all hosts thinking they have the same public IP address. To prevent this issue, pass the `--iface eth1` flag to flannel so that the second interface is chosen.
### Routing errors
In some situations `kubectl logs` and `kubectl run` commands may return with the following errors despite an otherwise apparently correctly working cluster:
```
Error from server: Get https://10.19.0.41:10250/containerLogs/default/mysql-ddc65b868-glc5m/mysql: dial tcp 10.19.0.41:10250: getsockopt: no route to host
```
This is due to Kubernetes using an IP that can not communicate with other IPs on the seemingly same subnet, possibly by policy of the machine provider. As an example, Digital Ocean assigns a public IP to `eth0` as well as a private one to be used internally as anchor for their floating IP feature, yet `kubelet` will pick the latter as the node's `InternalIP` instead of the public one.
Use `ip addr show` to check for this scenario instead of `ifconfig` because `ifconfig` will not display the offending alias IP address. Alternatively an API endpoint specific to Digital Ocean allows to query for the anchor IP from the droplet:
```
curl http://169.254.169.254/metadata/v1/interfaces/public/0/anchor_ipv4/address
```
The workaround is to tell `kubelet` which IP to use using `--node-ip`. When using Digital Ocean, it can be the public one (assigned to `eth0`) or the private one (assigned to `eth1`) should you want to use the optional private network. For example:
```
IFACE=eth0 # change to eth1 for DO's private network
DROPLET_IP_ADDRESS=$(ip addr show dev $IFACE | awk 'match($0,/inet (([0-9]|\.)+).* scope global/,a) { print a[1]; exit }')
echo $DROPLET_IP_ADDRESS # check this, just in case
echo "Environment=\"KUBELET_EXTRA_ARGS=--node-ip=$DROPLET_IP_ADDRESS\"" >> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
```
Please note that this assumes `KUBELET_EXTRA_ARGS` hasn't already been set in the unit file.
Then restart `kubelet`:
```
systemctl daemon-reload
systemctl restart kubelet
```