whentimeslows/website
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

kubeadm v1.7 documentation updates (#4018)

Browse Source 
* v1.7 updates for kubeadm

* Address review comments

* Address Luke's comments
This commit is contained in:
Lucas Käldström
2017-06-27 21:32:54 +03:00
committed by Andrew Chen
parent 171a5aeddf
commit 0391995573
3 changed files with 283 additions and 335 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/admin/kubeadm.md
+193 -70
View File
@@ -4,7 +4,7 @@ assignees:
- luxas
- errordeveloper
- jbeda
title: kubeadm Setup Tool
title: kubeadm Setup Tool Reference Guide
---
This document provides information on how to use kubeadm's advanced options.
@@ -30,7 +30,7 @@ following steps:
1. Outputting a kubeconfig file for the kubelet to use to connect to the API
server, as well as an additional kubeconfig file for administration.
1. kubeadm generates Kubernetes Static Pod manifests for the API server,
1. kubeadm generates Kubernetes static Pod manifests for the API server,
controller manager and scheduler. It places them in
`/etc/kubernetes/manifests`. The kubelet watches this directory for Pods to
create on startup. These are the core components of Kubernetes. Once they are
@@ -51,7 +51,7 @@ steps:
1. kubeadm creates a local key pair. It prepares a certificate signing request
(CSR) and sends that off to the API server for signing. The bootstrap token
is used to authenticate. The API server is configured to sign this
is used to authenticate. The control plane will sign this CSR requested
automatically.
1. kubeadm configures the local kubelet to connect to the API server
@@ -318,8 +318,8 @@ schedulerExtraArgs:
<argument>: <value|string>
<argument>: <value|string>
apiServerCertSANs:
- <name1|string>
- <name2|string>
- <name1|string>
- <name2|string>
certificatesDir: <string>
```
In addition, if authorizationMode is set to `ABAC`, you should write the config to `/etc/kubernetes/abac_policy.json`.
@@ -333,16 +333,79 @@ kind: NodeConfiguration
caCertPath: <path|string>
discoveryFile: <path|string>
discoveryToken: <string>
# Currently only the first server is used as a target for the cluster
# bootstrap flow.
discoveryTokenAPIServers:
- <address|string>
- <address|string>
- <address|string>
- <address|string>
tlsBootstrapToken: <string>
```
## Securing your installation even more
The defaults for kubeadm may not work for everyone. This section documents how to tighten up a kubeadm install
at the cost of some usability.
### Turning off auto-approval of Node Client Certificates
By default, there is an CSR auto-approver enabled that basically approves any client certificate request
for a kubelet when a Bootstrap Token was used when authenticating. If you don't want the cluster to
automatically approve kubelet client certs, you can turn it off by executing this command:
```console
$ kubectl delete clusterrole kubeadm:node-autoapprove-bootstrap
```
After that, `kubeadm join` will block until the admin has manually approved the CSR in flight:
```console
$ kubectl get csr
NAME AGE REQUESTOR CONDITION
node-csr-c69HXe7aYcqkS1bKmH4faEnHAWxn6i2bHZ2mD04jZyQ 18s system:bootstrap:878f07 Pending
$ kubectl certificate approve node-csr-c69HXe7aYcqkS1bKmH4faEnHAWxn6i2bHZ2mD04jZyQ
certificatesigningrequest "node-csr-c69HXe7aYcqkS1bKmH4faEnHAWxn6i2bHZ2mD04jZyQ" approved
$ kubectl get csr
NAME AGE REQUESTOR CONDITION
node-csr-c69HXe7aYcqkS1bKmH4faEnHAWxn6i2bHZ2mD04jZyQ 1m system:bootstrap:878f07 Approved,Issued
```
Only after `kubectl certificate approve` has been run, `kubeadm join` can proceed.
### Turning off public access to the cluster-info ConfigMap
In order to achieve the joining flow using the token as the only piece of validation information, a
public ConfigMap with some data needed for validation of the master's identity is exposed publicly by
default. While there is no private data in this ConfigMap, some users are sensitive and wish to turn
it off regardless. Doing so will disable the ability to use the `--discovery-token` flag of the
`kubeadm join` flow. Here are the steps to do so:
Fetch the `cluster-info` file from the API Server:
```console
$ kubectl -n kube-public get cm cluster-info -oyaml | grep "kubeconfig:" -A11 | grep "apiVersion" -A10 | sed "s/ //" | tee cluster-info.yaml
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: <ca-cert>
server: https://<ip>:<port>
name: ""
contexts: []
current-context: ""
kind: Config
preferences: {}
users: []
```
You can then use the `cluster-info.yaml` file as an argument to `kubeadm join --discovery-file`.
Turning of public access to the `cluster-info` ConfigMap:
```console
$ kubectl -n kube-public delete rolebinding kubeadm:bootstrap-signer-clusterinfo
```
These command should be run after `kubeadm init` but before `kubeadm join`.
## Managing Tokens {#manage-tokens}
You can use the `kubeadm` tool to manage tokens on a running cluster. It will
@@ -380,13 +443,13 @@ parallelize the token distribution for easier automation. To implement this
automation, you must know the IP address that the master will have after it is
started.
1. Generate a token. This token must have the form `<6 character string>.<16
character string>`. More formally, it must match the regex
1. Generate a token. This token must have the form `<6 character string>.<16
character string>`. More formally, it must match the regex:
`[a-z0-9]{6}\.[a-z0-9]{16}`.
Kubeadm can generate a token for you:
kubeadm can generate a token for you:
``` bash
```bash
kubeadm token generate
```
@@ -397,55 +460,6 @@ started.
Once the cluster is up, you can grab the admin credentials from the master node
at `/etc/kubernetes/admin.conf` and use that to talk to the cluster.
## Environment variables
There are some environment variables that modify the way that kubeadm works.
Most users will have no need to set these. These environment variables are a
short-term solution, eventually they will be integrated in the kubeadm
configuration file.
| Variable | Default | Description |
| --- | --- | --- |
| `KUBE_KUBERNETES_DIR` | `/etc/kubernetes` | Where most configuration files are written to and read from |
| `KUBE_HYPERKUBE_IMAGE` | | If set, use a single hyperkube image with this name. If not set, individual images per server component will be used. |
| `KUBE_ETCD_IMAGE` | `gcr.io/google_containers/etcd-<arch>:3.0.17` | The etcd container image to use. |
| `KUBE_REPO_PREFIX` | `gcr.io/google_containers` | The image prefix for all images that are used. |
If `KUBE_KUBERNETES_DIR` is specified, you may need to rewrite the arguments of the kubelet.
(e.g. --kubeconfig, --pod-manifest-path)
If `KUBE_REPO_PREFIX` is specified, you may need to set the kubelet flag `--pod-infra-container-image` which specifies which pause image to use.
Defaults to `gcr.io/google_containers/pause-${ARCH}:3.0` where `${ARCH}` can be one of `amd64`, `arm`, `arm64`, `ppc64le` or `s390x`.
```bash
cat > /etc/systemd/system/kubelet.service.d/20-pod-infra-image.conf <<EOF
[Service]
Environment="KUBELET_EXTRA_ARGS=--pod-infra-container-image=<your-image>"
EOF
systemctl daemon-reload
systemctl restart kubelet
```
If you want to use kubeadm with an http proxy, you may need to configure it to
support http_proxy, https_proxy, or no_proxy.
For example, if your kube master node IP address is 10.18.17.16 and you have a
proxy which supports both http/https on 10.18.17.16 port 8080, you can use the
following command:
```bash
export PROXY_PORT=8080
export PROXY_IP=10.18.17.16
export http_proxy=http://$PROXY_IP:$PROXY_PORT
export HTTP_PROXY=$http_proxy
export https_proxy=$http_proxy
export HTTPS_PROXY=$http_proxy
export no_proxy="localhost,127.0.0.1,localaddress,.localdomain.com,example.com,10.18.17.16"
```
Remember to change `proxy_ip` and add a kube master node IP address to
`no_proxy`.
## Use Kubeadm with other CRI runtimes
Since [Kubernetes 1.6 release](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG.md#node-components-1), Kubernetes container runtimes have been transferred to using CRI by default. Currently, the build-in container runtime is Docker which is enabled by build-in `dockershim` in `kubelet`.
@@ -488,11 +502,120 @@ This means you can, for example, prepopulate `/etc/kubernetes/pki/ca.crt`
and `/etc/kubernetes/pki/ca.key` with an existing CA, which then will be used
for signing the rest of the certs.
## Releases and release notes
## Running kubeadm without an internet connection
If you already have kubeadm installed and want to upgrade, run `apt-get update
&& apt-get upgrade` or `yum update` to get the latest version of kubeadm.
All of the control plane components run in Pods started by the kubelet and
the following images are required for the cluster works will be automatically
pulled by the kubelet if they don't exist locally while `kubeadm init` is initializing
your master:
Refer to the
[CHANGELOG.md](https://github.com/kubernetes/kubeadm/blob/master/CHANGELOG.md)
for more information.
| Image Name | v1.6 release branch version | v1.7 release branch version
|---|---|---|
| gcr.io/google_containers/kube-apiserver-${ARCH} | v1.6.x | v1.7.x
| gcr.io/google_containers/kube-controller-manager-${ARCH} | v1.6.x | v1.7.x
| gcr.io/google_containers/kube-scheduler-${ARCH} | v1.6.x | v1.7.x
| gcr.io/google_containers/kube-proxy-${ARCH} | v1.6.x | v1.7.x
| gcr.io/google_containers/etcd-${ARCH} | 3.0.17 | 3.0.17
| gcr.io/google_containers/pause-${ARCH} | 3.0 | 3.0
| gcr.io/google_containers/k8s-dns-sidecar-${ARCH} | 1.14.1 | 1.14.4
| gcr.io/google_containers/k8s-dns-kube-dns-${ARCH} | 1.14.1 | 1.14.4
| gcr.io/google_containers/k8s-dns-dnsmasq-nanny-${ARCH} | 1.14.1 | 1.14.4
Here `v1.7.x` means the "latest patch release of the v1.7 branch".
`${ARCH}` can be one of: `amd64`, `arm`, `arm64`, `ppc64le` or `s390x`.
## Cloudprovider integrations (experimental)
Enabling specific cloud providers is a common request. This currently requires
manual configuration and is therefore not yet fully supported. If you wish to do
so, edit the kubeadm dropin for the kubelet service
(`/etc/systemd/system/kubelet.service.d/10-kubeadm.conf`) on all nodes,
including the master. If your cloud provider requires any extra packages
installed on the host, for example for volume mounting/unmounting, install those
packages.
Specify the `--cloud-provider` flag for the kubelet and set it to the cloud of your
choice. If your cloudprovider requires a configuration file, create the file
`/etc/kubernetes/cloud-config` on every node. The exact format and content of
that file depends on the requirements imposed by your cloud provider. If you use
the `/etc/kubernetes/cloud-config` file, you must append it to the kubelet
arguments as follows: `--cloud-config=/etc/kubernetes/cloud-config`
Note that there is most likely other per-provider configuration that may be needed
(IAM roles for AWS) that is currently underdocumented.
Next, specify the cloud provider in the kubeadm config file. Create a file called
`kubeadm.conf` with the following contents:
``` yaml
kind: MasterConfiguration
apiVersion: kubeadm.k8s.io/v1alpha1
cloudProvider: <cloud provider>
```
Lastly, run `kubeadm init --config=kubeadm.conf` to bootstrap your cluster with
the cloud provider.
This workflow is not yet fully supported, however we hope to make it extremely
easy to spin up clusters with cloud providers in the future. (See [this
proposal](https://github.com/kubernetes/community/pull/128) for more
information) The [Kubelet Dynamic
Settings](https://github.com/kubernetes/kubernetes/pull/29459) feature may also
help to fully automate this process in the future.
## Environment variables
There are some environment variables that modify the way that kubeadm works.
Most users will have no need to set these. These environment variables are a
short-term solution, eventually they will be integrated in the kubeadm
configuration file.
**Note:** These environment variables are deprecated and will stop functioning in v1.8!
| Variable | Default | Description |
| --- | --- | --- |
| `KUBE_KUBERNETES_DIR` | `/etc/kubernetes` | Where most configuration files are written to and read from |
| `KUBE_HYPERKUBE_IMAGE` | | If set, use a single hyperkube image with this name. If not set, individual images per server component will be used. |
| `KUBE_ETCD_IMAGE` | `gcr.io/google_containers/etcd-<arch>:3.0.17` | The etcd container image to use. |
| `KUBE_REPO_PREFIX` | `gcr.io/google_containers` | The image prefix for all images that are used. |
If `KUBE_KUBERNETES_DIR` is specified, you may need to rewrite the arguments of the kubelet.
(e.g. --kubeconfig, --pod-manifest-path)
If `KUBE_REPO_PREFIX` is specified, you may need to set the kubelet flag
`--pod-infra-container-image` which specifies which pause image to use.
Defaults to `gcr.io/google_containers/pause-${ARCH}:3.0` where `${ARCH}`
can be one of `amd64`, `arm`, `arm64`, `ppc64le` or `s390x`.
```bash
cat > /etc/systemd/system/kubelet.service.d/20-pod-infra-image.conf <<EOF
[Service]
Environment="KUBELET_EXTRA_ARGS=--pod-infra-container-image=<your-image>"
EOF
systemctl daemon-reload
systemctl restart kubelet
```
If you want to use kubeadm with an http proxy, you may need to configure it to
support http_proxy, https_proxy, or no_proxy.
For example, if your kube master node IP address is 10.18.17.16 and you have a
proxy which supports both http/https on 10.18.17.16 port 8080, you can use the
following command:
```bash
export PROXY_PORT=8080
export PROXY_IP=10.18.17.16
export http_proxy=http://$PROXY_IP:$PROXY_PORT
export HTTP_PROXY=$http_proxy
export https_proxy=$http_proxy
export HTTPS_PROXY=$http_proxy
export no_proxy="localhost,127.0.0.1,localaddress,.localdomain.com,example.com,10.18.17.16"
```
Remember to change `proxy_ip` and add a kube master node IP address to
`no_proxy`.
docs/setup/independent/create-cluster-kubeadm.md
+84 -252
View File
@@ -13,7 +13,7 @@ redirect_from:
{% capture overview %}
This quickstart shows you how to easily install a Kubernetes cluster on machines
running Ubuntu 16.04, CentOS 7 or HypriotOS v1.0.1+. The installation uses a
running Ubuntu 16.04+, CentOS 7 or HypriotOS v1.0.1+. The installation uses a
tool called _kubeadm_ which is part of Kubernetes. As of v1.6, kubeadm aims to
create a secure cluster out of the box via mechanisms such as RBAC.
@@ -86,9 +86,16 @@ particularly easy.
See [Installing kubeadm](/docs/setup/independent/install-kubeadm/)
**Note:** If you already have kubeadm installed, you should do a `apt-get update &&
apt-get upgrade` or `yum update` to get the latest version of kubeadm.
The kubelet is now restarting every few seconds, as it waits in a crashloop for
kubeadm to tell it what to do.
### (2/4) Initializing your master
The master is the machine where the "control plane" components run, including
The master is the machine where the control plane components run, including
etcd (the cluster database) and the API server (which the kubectl CLI
communicates with).
@@ -108,25 +115,26 @@ There are pod network implementations where the master also plays a role in
allocating a set of network address space for each node. When using
[flannel](https://github.com/coreos/flannel) as the [pod network](#pod-network)
(described in step 3), specify `--pod-network-cidr=10.244.0.0/16`. _This is not
required for any other networks besides Flannel._
required for any other networks besides flannel._
Please refer to the [kubeadm reference doc](/docs/admin/kubeadm/) if you want to
read more about the flags `kubeadm init` provides.
`kubeadm init` will first run a series of prechecks to ensure that the machine
is ready to run Kubernetes. It will expose warnings and exit on errors. It
will then download and install the cluster database and "control plane"
is ready to run Kubernetes. It will expose warnings and exit on errors. It
will then download and install the cluster database and control plane
components. This may take several minutes.
You can't run `kubeadm init` twice without tearing down the cluster in between,
You can't run `kubeadm init` twice without tearing down the cluster in between
([unless you're upgrading from v1.6 to v1.7](/docs/tasks/administer-cluster/kubeadm-upgrade-1-7.md)),
see [Tear Down](#tear-down).
The output should look like:
```
[kubeadm] WARNING: kubeadm is in beta, please do not use it for production clusters.
[init] Using Kubernetes version: v1.6.0
[init] Using Authorization mode: RBAC
[init] Using Kubernetes version: v1.7.0
[init] Using Authorization modes: [Node RBAC]
[preflight] Running pre-flight checks
[preflight] Starting the kubelet service
[certificates] Generated CA certificate and key.
@@ -142,22 +150,19 @@ The output should look like:
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/controller-manager.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/scheduler.conf"
[apiclient] Created API client, waiting for the control plane to become ready
[apiclient] All control plane components are healthy after 16.772251 seconds
[apiclient] Waiting for at least one node to register and become ready
[apiclient] First node is ready after 5.002536 seconds
[apiclient] Test deployment succeeded
[apiclient] All control plane components are healthy after 16.502136 seconds
[token] Using token: <token>
[apiconfig] Created RBAC rules
[addons] Created essential addon: kube-proxy
[addons] Created essential addon: kube-dns
[addons] Applied essential addon: kube-proxy
[addons] Applied essential addon: kube-dns
Your Kubernetes master has initialized successfully!
To start using your cluster, you need to run (as a regular user):
sudo cp /etc/kubernetes/admin.conf $HOME/
sudo chown $(id -u):$(id -g) $HOME/admin.conf
export KUBECONFIG=$HOME/admin.conf
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
@@ -178,28 +183,10 @@ token can add authenticated nodes to your cluster. These tokens can be listed,
created and deleted with the `kubeadm token` command. See the [reference
guide](/docs/admin/kubeadm/#manage-tokens).
#### Master Images
All of these components run in pods started by kubelet and the following images
are required and will be automatically pulled by kubelet if they are absent
while `kubeadm init` is initializing your master:
| Image Name | Version |
|---|---|
| gcr.io/google_containers/kube-apiserver-amd64 | v1.6.0
| gcr.io/google_containers/kube-controller-manager-amd64 | v1.6.0
| gcr.io/google_containers/kube-scheduler-amd64 | v1.6.0
| gcr.io/google_containers/kube-proxy-amd64 | v1.6.0
| gcr.io/google_containers/etcd-amd64 | 3.0.17
| gcr.io/google_containers/pause-amd64 | 3.0
| gcr.io/google_containers/k8s-dns-sidecar-amd64 | 1.14.1
| gcr.io/google_containers/k8s-dns-kube-dns-amd64 | 1.14.1
| gcr.io/google_containers/k8s-dns-dnsmasq-nanny-amd64 | 1.14.1
#### Master Isolation
By default, your cluster will not schedule pods on the master for security
reasons. If you want to be able to schedule pods on the master, e.g. a
reasons. If you want to be able to schedule pods on the master, e.g. for a
single-machine Kubernetes cluster for development, run:
``` bash
@@ -242,22 +229,20 @@ You can install a pod network add-on with the following command:
kubectl apply -f <add-on.yaml>
```
Please refer to the specific add-on installation guide for exact details. You
should only install one pod network per cluster.
Please refer to the specific add-on installation guide for exact details.
**NOTE:** You can install **only one** pod network per cluster.
If you are on another architecture than amd64, you should use the
flannel or Weave Net overlay networks as described in [the
multi-platform section](#multi-platform)
NOTE: You can install **only one** pod network per cluster.
Once a pod network has been installed, you can confirm that it is working by
checking that the kube-dns pod is Running in the output of `kubectl get pods
--all-namespaces`. And once the kube-dns pod is up and running, you can continue by joining your
nodes.
checking that the kube-dns pod is Running in the output of `kubectl get pods --all-namespaces`.
And once the kube-dns pod is up and running, you can continue by joining your nodes.
If your network is not working or kube-dns is not in the Running state, check
out the [troubleshooting section](#pod-network-trouble) below.
out the [troubleshooting section](#troubleshooting) below.
### (4/4) Joining your nodes
@@ -280,7 +265,7 @@ The output should look something like:
[discovery] Created cluster-info discovery client, requesting info from "https://10.138.0.4:6443"
[discovery] Cluster info signature and contents are valid, will use API Server "https://10.138.0.4:6443"
[discovery] Successfully established connection with API Server "10.138.0.4:6443"
[bootstrap] Detected server version: v1.6.0-beta.3
[bootstrap] Detected server version: v1.7.0
[bootstrap] The server supports the Certificates API (certificates.k8s.io/v1beta1)
[csr] Created API client to obtain unique certificate for this node, generating keys and certificate signing request
[csr] Received signed certificate from the API server, generating KubeConfig...
@@ -300,20 +285,20 @@ nodes` when run on the master.
### (Optional) Controlling your cluster from machines other than the master
In order to get a kubectl on some other computer (e.g. laptop) to talk to your
cluster, you need to copy the kubeconfig file from your master to your
workstation like this:
cluster, you need to copy the administrator kubeconfig file from your master
to your workstation like this:
``` bash
scp root@<master ip>:/etc/kubernetes/admin.conf .
kubectl --kubeconfig ./admin.conf get nodes
```
**Note:** If you are using GCE, instances, by default, disable ssh access for
root. First log in to the machine, copy the file someplace that can be accessed
and then use [`gcloud compute
copy-files`](https://cloud.google.com/sdk/gcloud/reference/compute/copy-files)
**Note:** If you are using GCE, instances disable ssh access for root by default.
If that's the case you can log in to the machine, copy the file someplace that
can be accessed and then use
[`gcloud compute copy-files`](https://cloud.google.com/sdk/gcloud/reference/compute/copy-files)
### (Optional) Connecting to the API Server
### (Optional) Proxying API Server to localhost
If you want to connect to the API Server from outside the cluster you can use
`kubectl proxy`:
@@ -347,7 +332,7 @@ running:
kubectl -n sock-shop get svc front-end
```
Output:
Sample output:
```
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
@@ -409,68 +394,33 @@ control of your Kubernetes cluster.
## Feedback
* Slack Channel:
* kubeadm support Slack Channel:
[#kubeadm](https://kubernetes.slack.com/messages/kubeadm/)
* General SIG Cluster Lifecycle Development Slack Channel:
[#sig-cluster-lifecycle](https://kubernetes.slack.com/messages/sig-cluster-lifecycle/)
* Mailing List:
[kubernetes-sig-cluster-lifecycle](https://groups.google.com/forum/#!forum/kubernetes-sig-cluster-lifecycle)
* [GitHub Issues in the kubeadm
repository](https://github.com/kubernetes/kubeadm/issues)
## Version skew policy
The kubeadm CLI tool of version vX.Y may deploy clusters with a control plane of version vX.Y or vX.(Y-1).
kubeadm CLI vX.Y can also upgrade an existing kubeadm-created cluster of version vX.(Y-1).
Due to that we can't see into the future, kubeadm CLI vX.Y may or may not be able to deploy vX.(Y+1) clusters.
Example: kubeadm v1.7 can deploy both v1.6 and v1.7 clusters and upgrade v1.6 kubeadm-created clusters to
v1.7.
## kubeadm is multi-platform {#multi-platform}
kubeadm deb/rpm packages and binaries are built for amd64, arm64, armhfp,
ppc64el, and s390x following the [multi-platform
kubeadm deb/rpm packages and binaries are built for amd64, arm (32-bit), arm64, ppc64le, and s390x
following the [multi-platform
proposal](https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/multi-platform.md).
Currently, only the pod networks flannel and Weave Net work on multiple architectures.
For Weave Net just use its [standard install](https://www.weave.works/docs/net/latest/kube-addon/).
Flannel requires special installation instructions:
``` bash
export ARCH=amd64
curl -sSL "https://github.com/coreos/flannel/blob/master/Documentation/kube-flannel.yml?raw=true" | sed "s/amd64/${ARCH}/g" | kubectl create -f -
```
Replace `ARCH=amd64` with `ARCH=arm` or `ARCH=arm64` depending on the platform
you're running on. Note that the Raspberry Pi 3 is in ARM 32-bit mode, so for
RPi 3 you should set `ARCH` to `arm`, not `arm64`.
## Cloudprovider integrations (experimental)
Enabling specific cloud providers is a common request. This currently requires
manual configuration and is therefore not yet fully supported. If you wish to do
so, edit the kubeadm dropin for the kubelet service
(`/etc/systemd/system/kubelet.service.d/10-kubeadm.conf`) on all nodes,
including the master. If your cloud provider requires any extra packages
installed on host, for example for volume mounting/unmounting, install those
packages.
Specify the `--cloud-provider` flag to kubelet and set it to the cloud of your
choice. If your cloudprovider requires a configuration file, create the file
`/etc/kubernetes/cloud-config` on every node. The exact format and content of
that file depends on the requirements imposed by your cloud provider. If you use
the `/etc/kubernetes/cloud-config` file, you must append it to the kubelet
arguments as follows: `--cloud-config=/etc/kubernetes/cloud-config`
Next, specify the cloud provider in the kubeadm config file. Create a file called
`kubeadm.conf` with the following contents:
``` yaml
kind: MasterConfiguration
apiVersion: kubeadm.k8s.io/v1alpha1
cloudProvider: <cloud provider>
```
Lastly, run `kubeadm init --config=kubeadm.conf` to bootstrap your cluster with
the cloud provider.
This workflow is not yet fully supported, however we hope to make it extremely
easy to spin up clusters with cloud providers in the future. (See [this
proposal](https://github.com/kubernetes/community/pull/128) for more
information) The [Kubelet Dynamic
Settings](https://github.com/kubernetes/kubernetes/pull/29459) feature may also
help to fully automate this process in the future.
Currently, only the pod networks [flannel](https://github.com/coreos/flannel) and [Weave Net](https://www.weave.works/docs/net/latest/kube-addon/) work on multiple architectures.
## Limitations
@@ -487,31 +437,36 @@ addressed in due course.
etcd](https://coreos.com/etcd/docs/latest/admin_guide.html). The etcd data
directory configured by kubeadm is at `/var/lib/etcd` on the master.
1. The `HostPort` and `HostIP` functionality does not work with kubeadm due to
that CNI networking is used, see issue
[#31307](https://github.com/kubernetes/kubernetes/issues/31307).
Workaround: use the [NodePort feature of
services](/docs/concepts/services-networking/service/#type-nodeport) instead, or use
HostNetwork.
1. Some users on RHEL/CentOS 7 have reported issues with traffic being routed
incorrectly due to iptables being bypassed. You should ensure
`net.bridge.bridge-nf-call-iptables` is set to 1 in your sysctl config, eg.
## Troubleshooting {#troubleshooting}
``` bash
cat /etc/sysctl.d/k8s.conf
```
You may have trouble in the configuration if you see Pod statuses like `RunContainerError`,
`CrashLoopBackOff` or `Error`.
Should have:
1. **There are Pods in the `RunContainerError`, `CrashLoopBackOff` or `Error` state**
Right after `kubeadm init` there should not be any such Pods. If there are Pods in
such a state _right after_ `kubeadm init`, please open an issue in the kubeadm repo.
`kube-dns` should be in the `Pending` state until you have deployed the network solution.
However, if you see Pods in the `RunContainerError`, `CrashLoopBackOff` or `Error` state
after deploying the network solution and nothing happens to `kube-dns`, it's very
likely that the Pod Network solution that you installed is somehow broken. You
might have to grant it more RBAC privileges or use a newer version. Please file
an issue in the Pod Network providers' issue tracker and get the issue triaged there.
```
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
```
1. **The `kube-dns` Pod is stuck in the `Pending` state forever**
This is expected and part of the design. kubeadm is network provider-agnostic, so the admin
should [install the pod network solution](/docs/concepts/cluster-administration/addons/)
of choice. You have to install a Pod Network
before `kube-dns` may deployed fully. Hence the `Pending` state before the network is set up.
1. Users can list, create and delete tokens using the `kubeadm token` command.
See the [reference guide](/docs/admin/kubeadm/#manage-tokens) for details.
1. **I tried to set `HostPort` on one workload, but it didn't have any effect**
The `HostPort` and `HostIP` functionality is available depending on your Pod Network
provider. Please contact the author of the Pod Network solution to find out whether
`HostPort` and `HostIP` functionality are available.
If not, you may still use the [NodePort feature of
services](/docs/concepts/services-networking/service/#type-nodeport) or use `HostNetwork=true`.
1. If you are using VirtualBox (directly or via Vagrant), you will need to
ensure that `hostname -i` returns a routable IP address (i.e. one on the
@@ -520,138 +475,15 @@ addressed in due course.
usually NATed. Workaround: Modify `/etc/hosts`, take a look at this
[`Vagrantfile`][ubuntu-vagrantfile] for how this can be achieved.
[ubuntu-vagrantfile]: https://github.com/errordeveloper/k8s-playground/blob/22dd39dfc06111235620e6c4404a96ae146f26fd/Vagrantfile#L11),
[ubuntu-vagrantfile]: https://github.com/errordeveloper/k8s-playground/blob/22dd39dfc06111235620e6c4404a96ae146f26fd/Vagrantfile#L11)
## Troubleshooting {#troubleshooting}
### Pod Network Troubleshooting {#pod-network-trouble}
You may have trouble in the configuration if you see the following statuses.
This example is for canal but there may be similar errors for other pod network
systems.
```
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system canal-node-f0lqp 2/3 RunContainerError 2 48s
kube-system canal-node-77d0h 2/3 CrashLoopBackOff 3 3m
kube-system kube-dns-2924299975-7q1vq 0/4 ContainerCreating 0 15m
```
The three statuses RunContainerError and CrashLoopBackOff and ContainerCreating
are very common.
To help diagnose what happened, you can use the following command to check what
is in the logs:
``` bash
kubectl describe -n kube-system po {YOUR_POD_NAME}
```
Do not use kubectl logs as they only work with Pods that have started. If you run:
``` bash
kubectl logs -n kube-system canal-node-f0lqp
```
You will got the following error:
```
Error from server (BadRequest): the server rejected our request for an unknown reason (get pods canal-node-f0lqp)
```
The `kubectl describe` comand gives you more details about what went wrong.
``` bash
kubectl describe -n kube-system po kube-dns-2924299975-1l2t7
```
The events should show something like this:
```
2m 2m 1 {kubelet nac} spec.containers{flannel} Warning Failed Failed to start container with docker id 927e7ccdc32b with error: Error response from daemon: {"message":"chown /etc/resolv.conf: operation not permitted"}
```
Or this:
```
6m 1m 191 {kubelet nac} Warning FailedSync Error syncing pod, skipping: failed to "SetupNetwork" for "kube-dns-2924299975-1l2t7_kube-system" with SetupNetworkError: "Failed to setup network for pod \"kube-dns-2924299975-1l2t7_kube-system(dee8ef21-fbcb-11e6-ba19-38d547e0006a)\" using network plugins \"cni\": open /run/flannel/subnet.env: no such file or directory; Skipping pod"
```
A web search on the error message may help narrow down the issue. Or
communicate the errors you are seeing to the community/company that provides the
pod network implementation you are using.
### Installing kubeadm 1.5 {#old-kubeadm}
This section covers the previous version, kubeadm 1.5. It is still available but is a little tricky to get to. Also note that the command line options and other configuration parameters have changed.
As root, run the following. This is very similar to the regular instructions except for pinning the versions. Note that, due to some unfortunate version strings, kubeadm isn't indexed in the repos and the packages must be downloaded and installed directly.
#### Ubuntu
As with all Kubernetes troubleshooting, normal commands you can take advantage of to help diagnose
what happened are `kubectl describe pod` or `kubectl logs`. Example usage:
```bash
apt-get update && apt-get install -y apt-transport-https
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb http://apt.kubernetes.io/ kubernetes-xenial main
EOF
apt-get update
kubectl -n ${NAMESPACE} describe pod ${POD_NAME}
# 1.5.6 does exist in the repo but it has a hard depenedency on a newer kubernetes-cni. Use 1.5.3 instead.
sudo apt-get -y install kubectl=1.5.3-00 kubelet=1.5.3-00 kubernetes-cni=0.3.0.1-07a8a2-00
# Versioning strangeness for how we packaged kubeadm pre-1.6 means that the version number
# says 1.6.0-alpha even though it is the 1.5 version of kubeadm. Because of how this sorts,
# we cannot keep this deb in the repo. Download it manually and install it.
curl -Lo /tmp/old-kubeadm.deb https://apt.k8s.io/pool/kubeadm_1.6.0-alpha.0.2074-a092d8e0f95f52-00_amd64_0206dba536f698b5777c7d210444a8ace18f48e045ab78687327631c6c694f42.deb
sudo dpkg -i /tmp/old-kubeadm.deb
sudo apt-get install -f
# Hold these packages back so that we don't accidentally upgrade them.
sudo apt-mark hold kubeadm kubectl kubelet kubernetes-cni
```
#### CentOS
```bash
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=http://yum.kubernetes.io/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg
https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF
setenforce 0
# 1.5.4 is the latest previous version in the repo. Because of messed up
# versioning in the 1.5 release, kubeadm is no longer indexed in the repos
# so we have to refer to the RPM directly.
sudo yum -y install \
yum-versionlock \
docker \
kubectl-1.5.4-0 \
kubelet-1.5.4-0 \
kubernetes-cni-0.3.0.1-0.07a8a2 \
http://yum.kubernetes.io/pool/082436e6e6cad1852864438b8f98ee6fa3b86b597554720b631876db39b8ef04-kubeadm-1.6.0-0.alpha.0.2074.a092d8e0f95f52.x86_64.rpm
# Lock the version of these packages so that we don't upgrade them accidentally.
sudo yum versionlock add kubectl kubelet kubernetes-cni kubeadm
# Enable and start up docker and the kubelet
systemctl enable docker && systemctl start docker
systemctl enable kubelet && systemctl start kubelet
```
#### Running `kubeadm init`
Finally, when running `kubeadm init` you must specify the `--use-kubernetes-version` flag:
```bash
kubeadm init --use-kubernetes-version=v1.5.6
kubectl -n ${NAMESPACE} logs ${POD_NAME} -c ${CONTAINER_NAME}
```
{% endcapture %}
docs/setup/independent/install-kubeadm.md
+6 -13
View File
@@ -31,15 +31,15 @@ For installation instructions, see
On each of your machines,
[install kubectl](/docs/tasks/tools/install-kubectl/).
You only need kubectl on the master, but it can be useful to have on the other
nodes as well.
You only need kubectl on the master and/or your workstation, but it can be
useful to have on the other nodes as well.
## Installing kubelet and kubeadm
You will install these packages on all of your machines:
* `kubelet`: the most core component of Kubernetes. It runs on all of the
machines in your cluster and does things like starting pods and containers.
* `kubelet`: the component that runs on all of the machines in your cluster
and does things like starting pods and containers.
* `kubeadm`: the command to bootstrap the cluster.
@@ -62,9 +62,7 @@ For each machine:
deb http://apt.kubernetes.io/ kubernetes-xenial main
EOF
apt-get update
# Install docker if you don't have it already.
apt-get install -y docker-engine
apt-get install -y kubelet kubeadm kubernetes-cni
apt-get install -y kubelet kubeadm
```
* If the machine is running CentOS, run:
@@ -81,8 +79,7 @@ For each machine:
https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF
setenforce 0
yum install -y docker kubelet kubeadm kubernetes-cni
systemctl enable docker && systemctl start docker
yum install -y kubelet kubeadm
systemctl enable kubelet && systemctl start kubelet
```
@@ -93,10 +90,6 @@ For each machine:
containers to access the host filesystem, which is required by pod networks for
example. You have to do this until SELinux support is improved in the kubelet.
While this guide is correct for kubeadm 1.6, the previous version is still
available but can be a bit tricky to install. [See below](#old-kubeadm) for
details.
{% endcapture %}
{% capture whatsnext %}