From ade8225f0f193ae5bdbb583d506c1c41070aea83 Mon Sep 17 00:00:00 2001 From: Joe Beda Date: Sun, 12 Mar 2017 14:25:44 -0700 Subject: [PATCH 1/2] Reformat kubeadm guide * Word wrap paragraphs * Switch to fenced code blocks for all code samples * Render programs as plain text * Separate out program invocation from program output Much of this is to follow the style guide: https://kubernetes.io/docs/contribute/style-guide/ Signed-off-by: Joe Beda --- docs/getting-started-guides/kubeadm.md | 628 ++++++++++++++++--------- 1 file changed, 396 insertions(+), 232 deletions(-) diff --git a/docs/getting-started-guides/kubeadm.md b/docs/getting-started-guides/kubeadm.md index 03cf546eaa..d9f292b73e 100644 --- a/docs/getting-started-guides/kubeadm.md +++ b/docs/getting-started-guides/kubeadm.md @@ -13,39 +13,50 @@ li>.highlighter-rouge {position:relative; top:3px;} ## Overview -This quickstart shows you how to easily install a secure Kubernetes cluster on machines running Ubuntu 16.04, CentOS 7 or HypriotOS v1.0.1+. -The installation uses a tool called `kubeadm` which is part of Kubernetes. +This quickstart shows you how to easily install a secure Kubernetes cluster on +machines running Ubuntu 16.04, CentOS 7 or HypriotOS v1.0.1+. The installation +uses a tool called _kubeadm_ which is part of Kubernetes. -This process works with local VMs, physical servers and/or cloud servers. -It is simple enough that you can easily integrate its use into your own automation (Terraform, Chef, Puppet, etc). +This process works with local VMs, physical servers and/or cloud servers. It is +simple enough that you can easily integrate its use into your own automation +(Terraform, Chef, Puppet, etc). -See the full `kubeadm` [reference](/docs/admin/kubeadm) for information on all `kubeadm` command-line flags and for advice on automating `kubeadm` itself. +See the full [kubeadm reference](/docs/admin/kubeadm) for information on all +kubeadm command-line flags and for advice on automating kubeadm itself. -**The `kubeadm` tool is currently in alpha but please try it out and give us [feedback](/docs/getting-started-guides/kubeadm/#feedback)! -Be sure to read the [limitations](#limitations); in particular note that kubeadm doesn't have great support for -automatically configuring cloud providers. Please refer to the specific cloud provider documentation or -use another provisioning system.** +**The kubeadm tool is currently in alpha but please try it out and give us +[feedback](/docs/getting-started-guides/kubeadm/#feedback)! Be sure to read the +[limitations](#limitations); in particular note that kubeadm doesn't have great +support for automatically configuring cloud providers. Please refer to the +specific cloud provider documentation or use another provisioning system.** -kubeadm assumes you have a set of machines (virtual or real) that are up and running. It is designed -to be part of a large provisioning system - or just for easy manual provisioning. kubeadm is a great -choice where you have your own infrastructure (e.g. bare metal), or where you have an existing -orchestration system (e.g. Puppet) that you have to integrate with. +kubeadm assumes you have a set of machines (virtual or real) that are up and +running. It is designed to be part of a large provisioning system - or just for +easy manual provisioning. kubeadm is a great choice where you have your own +infrastructure (e.g. bare metal), or where you have an existing orchestration +system (e.g. Puppet) that you have to integrate with. -If you are not constrained, there are some other tools built to give you complete clusters: +If you are not constrained, there are some other tools built to give you +complete clusters: -* On GCE, [Google Container Engine](https://cloud.google.com/container-engine/) gives you one-click Kubernetes clusters -* On AWS, [kops](https://github.com/kubernetes/kops) makes cluster installation and management easy (and supports high availability) +* On GCE, [Google Container Engine](https://cloud.google.com/container-engine/) + gives you one-click Kubernetes clusters +* On AWS, [kops](https://github.com/kubernetes/kops) makes cluster installation + and management easy (and supports high availability) ## Prerequisites 1. One or more machines running Ubuntu 16.04+, CentOS 7 or HypriotOS v1.0.1+ -1. 1GB or more of RAM per machine (any less will leave little room for your apps) -1. Full network connectivity between all machines in the cluster (public or private network is fine) +1. 1GB or more of RAM per machine (any less will leave little room for your + apps) +1. Full network connectivity between all machines in the cluster (public or + private network is fine) ## Objectives * Install a secure Kubernetes cluster on your machines -* Install a pod network on the cluster so that application components (pods) can talk to each other +* Install a pod network on the cluster so that application components (pods) can + talk to each other * Install a sample microservices application (a socks shop) on the cluster ## Instructions @@ -54,155 +65,209 @@ If you are not constrained, there are some other tools built to give you complet You will install the following packages on all the machines: -* `docker`: the container runtime, which Kubernetes depends on. v1.11.2 is recommended, but v1.10.3 and v1.12.1 are known to work as well. -* `kubelet`: the most core component of Kubernetes. - It runs on all of the machines in your cluster and does things like starting pods and containers. -* `kubectl`: the command to control the cluster once it's running. - You will only need this on the master, but it can be useful to have on the other nodes as well. +* `docker`: the container runtime, which Kubernetes depends on. v1.11.2 is + recommended, but v1.10.3 and v1.12.1 are known to work as well. +* `kubelet`: the most core component of Kubernetes. It runs on all of the + machines in your cluster and does things like starting pods and containers. +* `kubectl`: the command to control the cluster once it's running. You will only + need this on the master, but it can be useful to have on the other nodes as + well. * `kubeadm`: the command to bootstrap the cluster. -NOTE: If you already have kubeadm installed, you should do a `apt-get update && apt-get upgrade` or `yum update` to get the latest version of kubeadm. -See the reference doc if you want to read about the different [kubeadm releases](https://github.com/kubernetes/kubeadm/blob/master/CHANGELOG.md) +NOTE: If you already have kubeadm installed, you should do a `apt-get update && +apt-get upgrade` or `yum update` to get the latest version of kubeadm. See the +reference doc if you want to read about the different [kubeadm +releases](https://github.com/kubernetes/kubeadm/blob/master/CHANGELOG.md) For each host in turn: -* SSH into the machine and become `root` if you are not already (for example, run `sudo su -`). +* SSH into the machine and become root if you are not already (for example, + run `sudo su -`). * If the machine is running Ubuntu or HypriotOS, run: - apt-get update && apt-get install -y apt-transport-https - curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - - cat < /etc/apt/sources.list.d/kubernetes.list - deb http://apt.kubernetes.io/ kubernetes-xenial main - EOF - apt-get update - # Install docker if you don't have it already. - apt-get install -y docker.io - apt-get install -y kubelet kubeadm kubectl kubernetes-cni + ``` bash + apt-get update && apt-get install -y apt-transport-https + curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - + cat < /etc/apt/sources.list.d/kubernetes.list + deb http://apt.kubernetes.io/ kubernetes-xenial main + EOF + apt-get update + # Install docker if you don't have it already. + apt-get install -y docker.io + apt-get install -y kubelet kubeadm kubectl kubernetes-cni + ``` - If the machine is running CentOS, run: + If the machine is running CentOS, run: - cat < /etc/yum.repos.d/kubernetes.repo - [kubernetes] - name=Kubernetes - baseurl=http://yum.kubernetes.io/repos/kubernetes-el7-x86_64 - enabled=1 - gpgcheck=1 - repo_gpgcheck=1 - gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg - https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg - EOF - setenforce 0 - yum install -y docker kubelet kubeadm kubectl kubernetes-cni - systemctl enable docker && systemctl start docker - systemctl enable kubelet && systemctl start kubelet + ``` bash + cat < /etc/yum.repos.d/kubernetes.repo + [kubernetes] + name=Kubernetes + baseurl=http://yum.kubernetes.io/repos/kubernetes-el7-x86_64 + enabled=1 + gpgcheck=1 + repo_gpgcheck=1 + gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg + https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg + EOF + setenforce 0 + yum install -y docker kubelet kubeadm kubectl kubernetes-cni + systemctl enable docker && systemctl start docker + systemctl enable kubelet && systemctl start kubelet + ``` -The kubelet is now restarting every few seconds, as it waits in a crashloop for `kubeadm` to tell it what to do. +The kubelet is now restarting every few seconds, as it waits in a crashloop for +kubeadm to tell it what to do. -Note: Disabling SELinux by running `setenforce 0` is required in order to allow containers to access the host filesystem, which is required by pod networks for example. You have to do this until kubelet can handle SELinux better. +Note: Disabling SELinux by running `setenforce 0` is required in order to allow +containers to access the host filesystem, which is required by pod networks for +example. You have to do this until kubelet can handle SELinux better. ### (2/4) Initializing your master -The master is the machine where the "control plane" components run, including `etcd` (the cluster database) and the API server (which the `kubectl` CLI communicates with). -All of these components run in pods started by `kubelet` and the following images are required and will be automatically pulled by `kubelet` if they are absent while `kubeadm init` is initializing your master: +The master is the machine where the "control plane" components run, including +etcd (the cluster database) and the API server (which the kubectl CLI +communicates with). All of these components run in pods started by kubelet and +the following images are required and will be automatically pulled by kubelet +if they are absent while `kubeadm init` is initializing your master: - gcr.io/google_containers/kube-proxy-amd64 v1.5.3 - gcr.io/google_containers/kube-controller-manager-amd64 v1.5.3 - gcr.io/google_containers/kube-scheduler-amd64 v1.5.3 - gcr.io/google_containers/kube-apiserver-amd64 v1.5.3 - gcr.io/google_containers/etcd-amd64 3.0.14-kubeadm - gcr.io/google_containers/kube-discovery-amd64 1.0 - gcr.io/google_containers/pause-amd64 3.0 +| Image Name | Version | +|---|---| +| gcr.io/google_containers/kube-proxy-amd64 | v1.5.3 +| gcr.io/google_containers/kube-controller-manager-amd64 | v1.5.3 +| gcr.io/google_containers/kube-scheduler-amd64 | v1.5.3 +| gcr.io/google_containers/kube-apiserver-amd64 | v1.5.3 +| gcr.io/google_containers/etcd-amd64 | 3.0.14-kubeadm +| gcr.io/google_containers/kube-discovery-amd64 | 1.0 +| gcr.io/google_containers/pause-amd64 | 3.0 -Right now you can't run `kubeadm init` twice without tearing down the cluster in between, see [Tear down](#tear-down). +Right now you can't run `kubeadm init` twice without tearing down the cluster in +between, see [Tear down](#tear-down). -If you try to run `kubeadm init` and your machine is in a state that is incompatible with starting a Kubernetes cluster, `kubeadm` will warn you about things that might not work or it will error out for unsatisfied mandatory requirements. +If you try to run `kubeadm init` and your machine is in a state that is +incompatible with starting a Kubernetes cluster, kubeadm will warn you about +things that might not work or it will error out for unsatisfied mandatory +requirements. -To initialize the master, pick one of the machines you previously installed `kubelet` and `kubeadm` on, and run: +To initialize the master, pick one of the machines you previously installed +kubelet and kubeadm on, and run: - # kubeadm init +``` bash +kubeadm init +``` -**Note:** this will autodetect the network interface to advertise the master on as the interface with the default gateway. -If you want to use a different interface, specify `--api-advertise-addresses ` argument to `kubeadm init`. +**Note:** this will autodetect the network interface to advertise the master on +as the interface with the default gateway. If you want to use a different +interface, specify `--api-advertise-addresses ` argument to `kubeadm +init`. -If you want to use [flannel](https://github.com/coreos/flannel) as the pod network, specify `--pod-network-cidr 10.244.0.0/16` if you're using the daemonset manifest below. _However, please note that this is not required for any other networks besides Flannel._ +If you want to use [flannel](https://github.com/coreos/flannel) as the pod +network, specify `--pod-network-cidr 10.244.0.0/16` if you're using the +daemonset manifest below. _However, please note that this is not required for +any other networks besides Flannel._ -Please refer to the [kubeadm reference doc](/docs/admin/kubeadm/) if you want to read more about the flags `kubeadm init` provides. +Please refer to the [kubeadm reference doc](/docs/admin/kubeadm/) if you want to +read more about the flags `kubeadm init` provides. -This will download and install the cluster database and "control plane" components. -This may take several minutes. +This will download and install the cluster database and "control plane" +components. This may take several minutes. The output should look like: - [kubeadm] WARNING: kubeadm is in alpha, please do not use it for production clusters. - [preflight] Running pre-flight checks - [init] Using Kubernetes version: v1.5.1 - [tokens] Generated token: "064158.548b9ddb1d3fad3e" - [certificates] Generated Certificate Authority key and certificate. - [certificates] Generated API Server key and certificate - [certificates] Generated Service Account signing keys - [certificates] Created keys and certificates in "/etc/kubernetes/pki" - [kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/kubelet.conf" - [kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/admin.conf" - [apiclient] Created API client, waiting for the control plane to become ready - [apiclient] All control plane components are healthy after 61.317580 seconds - [apiclient] Waiting for at least one node to register and become ready - [apiclient] First node is ready after 6.556101 seconds - [apiclient] Creating a test deployment - [apiclient] Test deployment succeeded - [token-discovery] Created the kube-discovery deployment, waiting for it to become ready - [token-discovery] kube-discovery is ready after 6.020980 seconds - [addons] Created essential addon: kube-proxy - [addons] Created essential addon: kube-dns +``` +[kubeadm] WARNING: kubeadm is in alpha, please do not use it for production clusters. +[preflight] Running pre-flight checks +[init] Using Kubernetes version: v1.5.1 +[tokens] Generated token: "064158.548b9ddb1d3fad3e" +[certificates] Generated Certificate Authority key and certificate. +[certificates] Generated API Server key and certificate +[certificates] Generated Service Account signing keys +[certificates] Created keys and certificates in "/etc/kubernetes/pki" +[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/kubelet.conf" +[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/admin.conf" +[apiclient] Created API client, waiting for the control plane to become ready +[apiclient] All control plane components are healthy after 61.317580 seconds +[apiclient] Waiting for at least one node to register and become ready +[apiclient] First node is ready after 6.556101 seconds +[apiclient] Creating a test deployment +[apiclient] Test deployment succeeded +[token-discovery] Created the kube-discovery deployment, waiting for it to become ready +[token-discovery] kube-discovery is ready after 6.020980 seconds +[addons] Created essential addon: kube-proxy +[addons] Created essential addon: kube-dns - Your Kubernetes master has initialized successfully! +Your Kubernetes master has initialized successfully! - You should now deploy a pod network to the cluster. - Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: - http://kubernetes.io/docs/admin/addons/ +You should now deploy a pod network to the cluster. +Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: + http://kubernetes.io/docs/admin/addons/ - You can now join any number of machines by running the following on each node: +You can now join any number of machines by running the following on each node: - kubeadm join --token= +kubeadm join --token= +``` -Make a record of the `kubeadm join` command that `kubeadm init` outputs. -You will need this in a moment. -The key included here is secret, keep it safe — anyone with this key can add authenticated nodes to your cluster. +Make a record of the `kubeadm join` command that `kubeadm init` outputs. You +will need this in a moment. The key included here is secret, keep it safe +— anyone with this key can add authenticated nodes to your cluster. The key is used for mutual authentication between the master and the joining nodes. -By default, your cluster will not schedule pods on the master for security reasons. -If you want to be able to schedule pods on the master, for example if you want a single-machine Kubernetes cluster for development, run: +By default, your cluster will not schedule pods on the master for security +reasons. If you want to be able to schedule pods on the master, for example if +you want a single-machine Kubernetes cluster for development, run: - # kubectl taint nodes --all dedicated- - node "test-01" tainted - taint key="dedicated" and effect="" not found. - taint key="dedicated" and effect="" not found. +``` bash +kubectl taint nodes --all dedicated- +``` -This will remove the "dedicated" taint from any nodes that have it, including the master node, meaning that the scheduler will then be able to schedule pods everywhere. +With output looking something like: + +``` +node "test-01" tainted +taint key="dedicated" and effect="" not found. +taint key="dedicated" and effect="" not found. +``` + +This will remove the "dedicated" taint from any nodes that have it, including +the master node, meaning that the scheduler will then be able to schedule pods +everywhere. ### (3/4) Installing a pod network -You must install a pod network add-on so that your pods can communicate with each other. +You must install a pod network add-on so that your pods can communicate with +each other. -**It is necessary to do this before you try to deploy any applications to your cluster, and before `kube-dns` will start up. Note also that `kubeadm` only supports CNI based networks and therefore kubenet based networks will not work.** +**It is necessary to do this before you try to deploy any applications to your +cluster. Also, kube-dns, a helper service, will no start up before a network is +installed. Note also that kubeadm only supports CNI based networks and therefore +kubenet based networks will not work.** -Several projects provide Kubernetes pod networks using CNI, some of which -also support [Network Policy](/docs/user-guide/networkpolicies/). See the [add-ons page](/docs/admin/addons/) for a complete list of available network add-ons. +Several projects provide Kubernetes pod networks using CNI, some of which also +support [Network Policy](/docs/user-guide/networkpolicies/). See the [add-ons +page](/docs/admin/addons/) for a complete list of available network add-ons. -You can install a pod network add-on with the following command: +You can install a pod network add-on with the following command: - # kubectl apply -f +``` bash +kubectl apply -f +``` -Please refer to the specific add-on installation guide for exact details. You should only install one pod network per cluster. +Please refer to the specific add-on installation guide for exact details. You +should only install one pod network per cluster. -If you are on another architecture than amd64, you should use the flannel overlay network as described in [the multi-platform section](#kubeadm-is-multi-platform) +If you are on another architecture than amd64, you should use the flannel +overlay network as described in [the multi-platform +section](#kubeadm-is-multi-platform) NOTE: You can install **only one** pod network per cluster. -Once a pod network has been installed, you can confirm that it is working by checking that the `kube-dns` pod is `Running` in the output of `kubectl get pods --all-namespaces`. - -And once the `kube-dns` pod is up and running, you can continue by joining your nodes. +Once a pod network has been installed, you can confirm that it is working by +checking that the kube-dns pod is `Running` in the output of `kubectl get pods +--all-namespaces`. +And once the kube-dns pod is up and running, you can continue by joining your +nodes. You may have trouble in the configuration if you see the following statuses @@ -213,199 +278,298 @@ kube-system canal-node-77d0h 2/3 CrashLoopBackOff 3 kube-system kube-dns-2924299975-7q1vq 0/4 ContainerCreating 0 15m ``` -The three statuses ```RunContainerError``` and ```CrashLoopBackOff``` and ```ContainerCreating``` are very common. +The three statuses ```RunContainerError``` and ```CrashLoopBackOff``` and +```ContainerCreating``` are very common. -To help diagnose what happened, you can use the following command to check what is in the logs: +To help diagnose what happened, you can use the following command to check what +is in the logs: -```bash +``` bash kubectl describe -n kube-system po {YOUR_POD_NAME} ``` -Do not using kubectl logs. You will got the following error: +Do not using kubectl logs. If you run: + +``` bash +kubectl logs -n kube-system canal-node-f0lqp +``` + +You will got the following error: ``` -# kubectl logs -n kube-system canal-node-f0lqp Error from server (BadRequest): the server rejected our request for an unknown reason (get pods canal-node-f0lqp) ``` -The ```kubectl describe``` comand gives you more details about the logs +The ```kubectl describe``` comand gives you more details about what went wrong. + +``` bash +kubectl describe -n kube-system po kube-dns-2924299975-1l2t7 +``` + +The events should show something like this: ``` -# kubectl describe -n kube-system po kube-dns-2924299975-1l2t7 2m 2m 1 {kubelet nac} spec.containers{flannel} Warning Failed Failed to start container with docker id 927e7ccdc32b with error: Error response from daemon: {"message":"chown /etc/resolv.conf: operation not permitted"} - ``` -Or +Or this: + ``` 6m 1m 191 {kubelet nac} Warning FailedSync Error syncing pod, skipping: failed to "SetupNetwork" for "kube-dns-2924299975-1l2t7_kube-system" with SetupNetworkError: "Failed to setup network for pod \"kube-dns-2924299975-1l2t7_kube-system(dee8ef21-fbcb-11e6-ba19-38d547e0006a)\" using network plugins \"cni\": open /run/flannel/subnet.env: no such file or directory; Skipping pod" ``` -You can then do some Google searches on the error messages, which may help you to find some solutions. +You can then do some Google searches on the error messages, which may help you +to find some solutions. ### (4/4) Joining your nodes -The nodes are where your workloads (containers and pods, etc) run. -If you want to add any new machines as nodes to your cluster, for each machine: SSH to that machine, become root (e.g. `sudo su -`) and run the command that was output by `kubeadm init`. -For example: +The nodes are where your workloads (containers and pods, etc) run. If you want +to add any new machines as nodes to your cluster, for each machine: SSH to that +machine, become root (e.g. `sudo su -`) and run the command that was output by +`kubeadm init`. For example: - # kubeadm join --token - [kubeadm] WARNING: kubeadm is in alpha, please do not use it for production clusters. - [preflight] Running pre-flight checks - [preflight] Starting the kubelet service - [tokens] Validating provided token - [discovery] Created cluster info discovery client, requesting info from "http://192.168.x.y:9898/cluster-info/v1/?token-id=f11877" - [discovery] Cluster info object received, verifying signature using given token - [discovery] Cluster info signature and contents are valid, will use API endpoints [https://192.168.x.y:6443] - [bootstrap] Trying to connect to endpoint https://192.168.x.y:6443 - [bootstrap] Detected server version: v1.5.1 - [bootstrap] Successfully established connection with endpoint "https://192.168.x.y:6443" - [csr] Created API client to obtain unique certificate for this node, generating keys and certificate signing request - [csr] Received signed certificate from the API server: - Issuer: CN=kubernetes | Subject: CN=system:node:yournode | CA: false - Not before: 2016-12-15 19:44:00 +0000 UTC Not After: 2017-12-15 19:44:00 +0000 UTC - [csr] Generating kubelet configuration - [kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/kubelet.conf" +``` bash +kubeadm join --token +``` - Node join complete: - * Certificate signing request sent to master and response - received. - * Kubelet informed of new secure connection details. +The output should look something like: - Run 'kubectl get nodes' on the master to see this machine join. +``` +[kubeadm] WARNING: kubeadm is in alpha, please do not use it for production clusters. +[preflight] Running pre-flight checks +[preflight] Starting the kubelet service +[tokens] Validating provided token +[discovery] Created cluster info discovery client, requesting info from "http://192.168.x.y:9898/cluster-info/v1/?token-id=f11877" +[discovery] Cluster info object received, verifying signature using given token +[discovery] Cluster info signature and contents are valid, will use API endpoints [https://192.168.x.y:6443] +[bootstrap] Trying to connect to endpoint https://192.168.x.y:6443 +[bootstrap] Detected server version: v1.5.1 +[bootstrap] Successfully established connection with endpoint "https://192.168.x.y:6443" +[csr] Created API client to obtain unique certificate for this node, generating keys and certificate signing request +[csr] Received signed certificate from the API server: +Issuer: CN=kubernetes | Subject: CN=system:node:yournode | CA: false +Not before: 2016-12-15 19:44:00 +0000 UTC Not After: 2017-12-15 19:44:00 +0000 UTC +[csr] Generating kubelet configuration +[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/kubelet.conf" -A few seconds later, you should notice that running `kubectl get nodes` on the master shows a cluster with as many machines as you created. +Node join complete: +* Certificate signing request sent to master and response + received. +* Kubelet informed of new secure connection details. + +Run 'kubectl get nodes' on the master to see this machine join. +``` + +A few seconds later, you should notice that running `kubectl get nodes` on the +master shows a cluster with as many machines as you created. ### (Optional) Controlling your cluster from machines other than the master -In order to get a kubectl on your laptop for example to talk to your cluster, you need to copy the `KubeConfig` file from your master to your laptop like this: +In order to get a kubectl on your laptop for example to talk to your cluster, +you need to copy the `KubeConfig` file from your master to your laptop like +this: - # scp root@:/etc/kubernetes/admin.conf . - # kubectl --kubeconfig ./admin.conf get nodes +``` bash +scp root@:/etc/kubernetes/admin.conf . +kubectl --kubeconfig ./admin.conf get nodes +``` ### (Optional) Connecting to the API Server -If you want to connect to the API Server for viewing the dashboard (note: the dashboard isn't deployed by default) from outside the cluster for example, you can use `kubectl proxy`: +If you want to connect to the API Server for viewing the dashboard (note: the +dashboard isn't deployed by default) from outside the cluster for example, you +can use `kubectl proxy`: - # scp root@:/etc/kubernetes/admin.conf . - # kubectl --kubeconfig ./admin.conf proxy +``` bash +scp root@:/etc/kubernetes/admin.conf . +kubectl --kubeconfig ./admin.conf proxy +``` You can now access the API Server locally at `http://localhost:8001/api/v1` ### (Optional) Installing a sample application -As an example, install a sample microservices application, a socks shop, to put your cluster through its paces. Note that this demo does only work on `amd64`. -To learn more about the sample microservices app, see the [GitHub README](https://github.com/microservices-demo/microservices-demo). +As an example, install a sample microservices application, a socks shop, to put +your cluster through its paces. Note that this demo does only work on `amd64`. +To learn more about the sample microservices app, see the [GitHub +README](https://github.com/microservices-demo/microservices-demo). - # kubectl create namespace sock-shop - # kubectl apply -n sock-shop -f "https://github.com/microservices-demo/microservices-demo/blob/master/deploy/kubernetes/complete-demo.yaml?raw=true" +``` bash +kubectl create namespace sock-shop +kubectl apply -n sock-shop -f "https://github.com/microservices-demo/microservices-demo/blob/master/deploy/kubernetes/complete-demo.yaml?raw=true" +``` -You can then find out the port that the [NodePort feature of services](/docs/user-guide/services/) allocated for the front-end service by running: +You can then find out the port that the [NodePort feature of +services](/docs/user-guide/services/) allocated for the front-end service by +running: - # kubectl describe svc front-end -n sock-shop - Name: front-end - Namespace: sock-shop - Labels: name=front-end - Selector: name=front-end - Type: NodePort - IP: 100.66.88.176 - Port: 80/TCP - NodePort: 31869/TCP - Endpoints: - Session Affinity: None +``` bash +kubectl describe svc front-end -n sock-shop +``` -It takes several minutes to download and start all the containers, watch the output of `kubectl get pods -n sock-shop` to see when they're all up and running. +Output: -Then go to the IP address of your cluster's master node in your browser, and specify the given port. -So for example, `http://:`. -In the example above, this was `30001`, but it is a different port for you. +``` +Name: front-end +Namespace: sock-shop +Labels: name=front-end +Selector: name=front-end +Type: NodePort +IP: 100.66.88.176 +Port: 80/TCP +NodePort: 31869/TCP +Endpoints: +Session Affinity: None +``` -If there is a firewall, make sure it exposes this port to the internet before you try to access it. +It takes several minutes to download and start all the containers, watch the +output of `kubectl get pods -n sock-shop` to see when they're all up and +running. + +Then go to the IP address of your cluster's master node in your browser, and +specify the given port. So for example, `http://:`. In the +example above, this was `30001`, but it is a different port for you. + +If there is a firewall, make sure it exposes this port to the internet before +you try to access it. ## Tear down -* To uninstall the socks shop, run `kubectl delete namespace sock-shop` on the master. +* To uninstall the socks shop, run `kubectl delete namespace sock-shop` on the + master. +* To undo what kubeadm did, simply run: -* To undo what `kubeadm` did, simply run: + ``` bash + kubeadm reset + ``` - # kubeadm reset - - If you wish to start over, run `systemctl start kubelet` followed by `kubeadm init` or `kubeadm join`. + If you wish to start over, run `systemctl start kubelet` followed by `kubeadm + init` or `kubeadm join`. ## Explore other add-ons -See the [list of add-ons](/docs/admin/addons/) to explore other add-ons, including tools for logging, monitoring, network policy, visualization & control of your Kubernetes cluster. +See the [list of add-ons](/docs/admin/addons/) to explore other add-ons, +including tools for logging, monitoring, network policy, visualization & +control of your Kubernetes cluster. ## What's next -* Learn about `kubeadm`'s advanced usage on the [advanced reference doc](/docs/admin/kubeadm/) -* Learn more about [Kubernetes concepts and kubectl in Kubernetes 101](/docs/user-guide/walkthrough/). +* Learn about kubeadm's advanced usage on the [advanced reference + doc](/docs/admin/kubeadm/) +* Learn more about [Kubernetes concepts and kubectl in Kubernetes + 101](/docs/user-guide/walkthrough/). ## Feedback -* Slack Channel: [#sig-cluster-lifecycle](https://kubernetes.slack.com/messages/sig-cluster-lifecycle/) -* Mailing List: [kubernetes-sig-cluster-lifecycle](https://groups.google.com/forum/#!forum/kubernetes-sig-cluster-lifecycle) -* [GitHub Issues in the kubeadm repository](https://github.com/kubernetes/kubeadm/issues) +* Slack Channel: + [#sig-cluster-lifecycle](https://kubernetes.slack.com/messages/sig-cluster-lifecycle/) +* Mailing List: + [kubernetes-sig-cluster-lifecycle](https://groups.google.com/forum/#!forum/kubernetes-sig-cluster-lifecycle) +* [GitHub Issues in the kubeadm + repository](https://github.com/kubernetes/kubeadm/issues) ## kubeadm is multi-platform -kubeadm deb packages and binaries are built for amd64, arm and arm64, following the [multi-platform proposal](https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/multi-platform.md). +kubeadm deb packages and binaries are built for amd64, arm and arm64, following +the [multi-platform +proposal](https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/multi-platform.md). -deb-packages are released for ARM and ARM 64-bit, but not RPMs (yet, reach out if there's interest). +deb-packages are released for ARM and ARM 64-bit, but not RPMs (yet, reach out +if there's interest). -Currently, only the pod network flannel is working on multiple architectures. You can install it this way: +Currently, only the pod network flannel is working on multiple architectures. +You can install it this way: - # export ARCH=amd64 - # curl -sSL "https://github.com/coreos/flannel/blob/master/Documentation/kube-flannel.yml?raw=true" | sed "s/amd64/${ARCH}/g" | kubectl create -f - +``` bash +export ARCH=amd64 +curl -sSL "https://github.com/coreos/flannel/blob/master/Documentation/kube-flannel.yml?raw=true" | sed "s/amd64/${ARCH}/g" | kubectl create -f - +``` -Replace `ARCH=amd64` with `ARCH=arm` or `ARCH=arm64` depending on the platform you're running on. -Note that the Raspberry Pi 3 is in ARM 32-bit mode, so for RPi 3 you should set `ARCH` to `arm`, not `arm64`. +Replace `ARCH=amd64` with `ARCH=arm` or `ARCH=arm64` depending on the platform +you're running on. Note that the Raspberry Pi 3 is in ARM 32-bit mode, so for +RPi 3 you should set `ARCH` to `arm`, not `arm64`. ## Cloudprovider integrations (experimental) -Enabling specific cloud providers is a common request, this currently requires manual configuration and is therefore not yet supported. If you wish to do so, -edit the `kubeadm` dropin for the `kubelet` service (`/etc/systemd/system/kubelet.service.d/10-kubeadm.conf`) on all nodes, including the master. -If your cloud provider requires any extra packages installed on host, for example for volume mounting/unmounting, install those packages. +Enabling specific cloud providers is a common request, this currently requires +manual configuration and is therefore not yet supported. If you wish to do so, +edit the kubeadm dropin for the kubelet service +(`/etc/systemd/system/kubelet.service.d/10-kubeadm.conf`) on all nodes, +including the master. If your cloud provider requires any extra packages +installed on host, for example for volume mounting/unmounting, install those +packages. -Specify the `--cloud-provider` flag to kubelet and set it to the cloud of your choice. If your cloudprovider requires a configuration -file, create the file `/etc/kubernetes/cloud-config` on every node. The exact format and content of that file depends on the requirements imposed by your cloud provider. -If you use the `/etc/kubernetes/cloud-config` file, you must append it to the `kubelet` arguments as follows: -`--cloud-config=/etc/kubernetes/cloud-config` +Specify the `--cloud-provider` flag to kubelet and set it to the cloud of your +choice. If your cloudprovider requires a configuration file, create the file +`/etc/kubernetes/cloud-config` on every node. The exact format and content of +that file depends on the requirements imposed by your cloud provider. If you use +the `/etc/kubernetes/cloud-config` file, you must append it to the kubelet +arguments as follows: `--cloud-config=/etc/kubernetes/cloud-config` -Lastly, run `kubeadm init --cloud-provider=xxx` to bootstrap your cluster with cloud provider features. +Lastly, run `kubeadm init --cloud-provider=xxx` to bootstrap your cluster with +cloud provider features. -This workflow is not yet fully supported, however we hope to make it extremely easy to spin up clusters with cloud providers in the future. -(See [this proposal](https://github.com/kubernetes/community/pull/128) for more information) The [Kubelet Dynamic Settings](https://github.com/kubernetes/kubernetes/pull/29459) feature may also help to fully automate this process in the future. +This workflow is not yet fully supported, however we hope to make it extremely +easy to spin up clusters with cloud providers in the future. (See [this +proposal](https://github.com/kubernetes/community/pull/128) for more +information) The [Kubelet Dynamic +Settings](https://github.com/kubernetes/kubernetes/pull/29459) feature may also +help to fully automate this process in the future. ## Limitations -Please note: `kubeadm` is a work in progress and these limitations will be addressed in due course. - -1. The cluster created here has a single master, with a single `etcd` database running on it. - This means that if the master fails, your cluster loses its configuration data and will need to be recreated from scratch. - Adding HA support (multiple `etcd` servers, multiple API servers, etc) to `kubeadm` is still a work-in-progress. +Please note: kubeadm is a work in progress and these limitations will be +addressed in due course. - Workaround: regularly [back up etcd](https://coreos.com/etcd/docs/latest/admin_guide.html). - The `etcd` data directory configured by `kubeadm` is at `/var/lib/etcd` on the master. -1. The `HostPort` and `HostIP` functionality does not work with kubeadm due to that CNI networking is used, see issue [#31307](https://github.com/kubernetes/kubernetes/issues/31307). +1. The cluster created here has a single master, with a single etcd database + running on it. This means that if the master fails, your cluster loses its + configuration data and will need to be recreated from scratch. Adding HA + support (multiple etcd servers, multiple API servers, etc) to kubeadm is + still a work-in-progress. - Workaround: use the [NodePort feature of services](/docs/user-guide/services/#type-nodeport) instead, or use HostNetwork. -1. Some users on RHEL/CentOS 7 have reported issues with traffic being routed incorrectly due to iptables being bypassed. You should ensure `net.bridge.bridge-nf-call-iptables` is set to 1 in your sysctl config, eg. + Workaround: regularly [back up + etcd](https://coreos.com/etcd/docs/latest/admin_guide.html). The etcd data + directory configured by kubeadm is at `/var/lib/etcd` on the master. +1. The `HostPort` and `HostIP` functionality does not work with kubeadm due to + that CNI networking is used, see issue + [#31307](https://github.com/kubernetes/kubernetes/issues/31307). - ```console - # cat /etc/sysctl.d/k8s.conf - net.bridge.bridge-nf-call-ip6tables = 1 - net.bridge.bridge-nf-call-iptables = 1 - ``` + Workaround: use the [NodePort feature of + services](/docs/user-guide/services/#type-nodeport) instead, or use + HostNetwork. +1. Some users on RHEL/CentOS 7 have reported issues with traffic being routed + incorrectly due to iptables being bypassed. You should ensure + `net.bridge.bridge-nf-call-iptables` is set to 1 in your sysctl config, eg. -1. There is no built-in way of fetching the token easily once the cluster is up and running, but here is a `kubectl` command you can copy and paste that will print out the token for you: + ``` bash + cat /etc/sysctl.d/k8s.conf + ``` - ```console - # kubectl -n kube-system get secret clusterinfo -o yaml | grep token-map | awk '{print $2}' | base64 --decode | sed "s|{||g;s|}||g;s|:|.|g;s/\"//g;" | xargs echo - ``` + Should have: -1. If you are using VirtualBox (directly or via Vagrant), you will need to ensure that `hostname -i` returns a routable IP address (i.e. one on the second network interface, not the first one). - By default, it doesn't do this and kubelet ends-up using first non-loopback network interface, which is usually NATed. - Workaround: Modify `/etc/hosts`, take a look at this [`Vagrantfile`][ubuntu-vagrantfile] for how this can be achieved. + ``` + net.bridge.bridge-nf-call-ip6tables = 1 + net.bridge.bridge-nf-call-iptables = 1 + ``` + +1. There is no built-in way of fetching the token easily once the cluster is up + and running, but here is a kubectl command you can copy and paste that will + print out the token for you: + + ``` bash + kubectl -n kube-system get secret clusterinfo -o yaml | \ + grep token-map | \ + awk '{print $2}' | \ + base64 --decode | \ + sed "s|{||g;s|}||g;s|:|.|g;s/\"//g;" | \ + xargs echo + ``` + +1. If you are using VirtualBox (directly or via Vagrant), you will need to + ensure that `hostname -i` returns a routable IP address (i.e. one on the + second network interface, not the first one). By default, it doesn't do this + and kubelet ends-up using first non-loopback network interface, which is + usually NATed. Workaround: Modify `/etc/hosts`, take a look at this + [`Vagrantfile`][ubuntu-vagrantfile] for how this can be achieved. [ubuntu-vagrantfile]: https://github.com/errordeveloper/k8s-playground/blob/22dd39dfc06111235620e6c4404a96ae146f26fd/Vagrantfile#L11), From 28c1fc5df863a5cedfa1c7e4e528ab5ce15bde32 Mon Sep 17 00:00:00 2001 From: Joe Beda Date: Mon, 13 Mar 2017 16:18:17 -0700 Subject: [PATCH 2/2] Update kubeadm guide and reference pages for 1.6 Signed-off-by: Joe Beda --- docs/admin/kubeadm.md | 425 ++++++++++++++-------- docs/getting-started-guides/kubeadm.md | 465 ++++++++++++++----------- docs/index.md | 2 +- docs/tools/index.md | 16 +- 4 files changed, 543 insertions(+), 365 deletions(-) diff --git a/docs/admin/kubeadm.md b/docs/admin/kubeadm.md index 8abdf0d6d7..514b5ad89a 100644 --- a/docs/admin/kubeadm.md +++ b/docs/admin/kubeadm.md @@ -13,127 +13,137 @@ Running `kubeadm init` bootstraps a Kubernetes cluster. This consists of the following steps: 1. kubeadm runs a series of pre-flight checks to validate the system state -before making changes. Some checks only trigger warnings, others are -considered errors and will exit kubeadm until the problem is corrected or -the user specifies `--skip-preflight-checks`. + before making changes. Some checks only trigger warnings, others are + considered errors and will exit kubeadm until the problem is corrected or the + user specifies `--skip-preflight-checks`. 1. kubeadm generates a token that additional nodes can use to register -themselves with the master in future. Optionally, the user can provide a token. + themselves with the master in future. Optionally, the user can provide a + token. -1. kubeadm generates a self-signed CA using openssl to provision identities -for each node in the cluster, and for the API server to secure communication -with clients. +1. kubeadm generates a self-signed CA to provision identities for each component + (including nodes) in the cluster. It also generates client certificates to + be used by various components. 1. Outputting a kubeconfig file for the kubelet to use to connect to the API -server, as well as an additional kubeconfig file for administration. + server, as well as an additional kubeconfig file for administration. -1. kubeadm generates Kubernetes resource manifests for the API server, -controller manager and scheduler, and placing them in -`/etc/kubernetes/manifests`. The kubelet watches this directory for static -resources to create on startup. These are the core components of Kubernetes, and -once they are up and running we can use `kubectl` to set up or manage any -additional components. +1. kubeadm generates Kubernetes Static Pod manifests for the API server, + controller manager and scheduler. It places them in + `/etc/kubernetes/manifests`. The kubelet watches this directory for Pods to + create on startup. These are the core components of Kubernetes. Once they are + up and running kubeadm can set up and manage any additional components. -1. kubeadm installs some add-on components, such as DNS or discovery, via the API -server. +1. kubeadm "taints" the master node so that only control plane components will + run there. It also sets up the RBAC authorization system and writes a + special ConfigMap that is used to bootstrap trust with the kubelets. -Running `kubeadm join` on each node in the cluster consists of the following steps: +1. kubeadm installs installs add-on components via the API server. Right now + this is the internal DNS server and the kube-proxy DaemonSet. -1. Use the token to talk to the API server and securely get the root CA -certificate. +Running `kubeadm join` on each node in the cluster consists of the following +steps: -1. Creates a local key pair. Prepares a certificate signing request (CSR) and -sends that off to the API server for signing. +1. kubeadm downloads root CA information from the API server. It uses the token + to verify the authenticity of that data. -1. Configures the local kubelet to connect to the API server +1. kubeadm creates a local key pair. It prepares a certificate signing request + (CSR) and sends that off to the API server for signing. The bootstrap token + is used to authenticate. The API server is configured to sign this + automatically. + +1. kubeadm configures the local kubelet to connect to the API server ## Usage -Fields that support multiple values do so either with comma separation, or by specifying -the flag multiple times. +Fields that support multiple values do so either with comma separation, or by +specifying the flag multiple times. + +The kubeadm command line interface is currently in **beta**. We are aiming to +not break any scripted use of the main `kubeadm init` and `kubeadm join`. The +single exception here is the format of the kubeadm config file as detailed +below. That format is still considered alpha and may change. ### `kubeadm init` -It is usually sufficient to run `kubeadm init` without any flags, -but in some cases you might like to override the default behaviour. -Here we specify all the flags that can be used to customise the Kubernetes -installation. +It is usually sufficient to run `kubeadm init` without any flags, but in some +cases you might like to override the default behaviour. Here we specify all the +flags that can be used to customise the Kubernetes installation. -- `--api-advertise-addresses` (multiple values are allowed) -- `--api-external-dns-names` (multiple values are allowed) +- `--api-advertise-address` -By default, `kubeadm init` automatically detects IP addresses and uses -these to generate certificates for the API server. This uses the IP address -of the default network interface. If you would like to access the API server -through a different IP address, or through a hostname, you can override these -defaults with `--api-advertise-addresses` and `--api-external-dns-names`. -For example, to generate certificates that verify the API server at addresses -`10.100.245.1` and `100.123.121.1`, you could use -`--api-advertise-addresses=10.100.245.1,100.123.121.1`. To allow it to be accessed -with a hostname, `--api-external-dns-names=kubernetes.example.com,kube.example.com` -Specifying `--api-advertise-addresses` disables auto detection of IP addresses. +This is the address the API Server will advertise to other members of the +cluster. This is also the address used to construct the suggested `kubeadm +join` line at the end of the init process. If not set (or set to 0.0.0.0) then +IP for the default interface will be used. -- `--cloud-provider` +This address is also added to the certifcate that the API Server uses. -Currently, `kubeadm init` does not provide autodetection of cloud provider. -This means that load balancing and persistent volumes are not supported out -of the box. You can specify a cloud provider using `--cloud-provider`. -Valid values are the ones supported by `controller-manager`, namely `"aws"`, -`"azure"`, `"cloudstack"`, `"gce"`, `"mesos"`, `"openstack"`, `"ovirt"`, -`"rackspace"`, `"vsphere"`. In order to provide additional configuration for -the cloud provider, you should create a `/etc/kubernetes/cloud-config` -file manually, before running `kubeadm init`. `kubeadm` automatically -picks those settings up and ensures other nodes are configured correctly. -The exact format and content of the file `/etc/kubernetes/cloud-config` depends -on the type you specified for `--cloud-provider`; see the appropriate documentation -for your cloud provider for details. -You must also set the `--cloud-provider` and `--cloud-config` parameters -yourself by editing the `/etc/systemd/system/kubelet.service.d/10-kubeadm.conf` -file appropriately. +- `--apiserver-bind-port` -- `--external-etcd-cafile` etcd certificate authority file -- `--external-etcd-endpoints` (multiple values are allowed) -- `--external-etcd-certfile` etcd client certificate file -- `--external-etcd-keyfile` etcd client key file +The port that the API server will bind on. This defaults to 6443. -By default, `kubeadm` deploys a single node etcd cluster on the master -to store Kubernetes state. This means that any failure on the master node -requires you to rebuild your cluster from scratch. Currently `kubeadm init` -does not support automatic deployment of a highly available etcd cluster. -If you would like to use your own etcd cluster, you can override this -behaviour with `--external-etcd-endpoints`. `kubeadm` supports etcd client -authentication using the `--external-etcd-cafile`, `--external-etcd-certfile` -and `--external-etcd-keyfile` flags. +- `--apiserver-cert-extra-sans` + +Additional hostnames or IP addresses that should be added to the Subject +Alternate Name section for the certificate that the API Server will use. If you +expose the API Server through a load balancer and public DNS you could specify +this with + +``` +--apiserver-cert-extra-sans=kubernetes.example.com,kube.example.com,10.100.245.1 +``` + +- `--cert-dir` + +The path where to save and store the certificates. The default is +"/etc/kubernetes/pki". + +- `--config` + +A kubeadm specific [config file](#config-file). This can be used to specify an +extended set of options including passing arbitrary command line flags to the +control plane components. + +- `--kubernetes-version` (default 'latest') the kubernetes version to initialise + +The **v1.6** version of kubeadm only supports building clusters that are at +least **v1.6.0**. There are many reasons for this including kubeadm's use of +RBAC, the Bootstrap Token system, and enhancements to the Certificates API. With +this flag you can try any future version of Kubernetes. Check [releases +page](https://github.com/kubernetes/kubernetes/releases) for a full list of +available versions. - `--pod-network-cidr` For certain networking solutions the Kubernetes master can also play a role in -allocating network ranges (CIDRs) to each node. This includes many cloud providers -and flannel. You can specify a subnet range that will be broken down and handed out -to each node with the `--pod-network-cidr` flag. This should be a minimum of a /16 so -controller-manager is able to assign /24 subnets to each node in the cluster. -If you are using flannel with [this manifest](https://github.com/coreos/flannel/blob/master/Documentation/kube-flannel.yml) -you should use `--pod-network-cidr=10.244.0.0/16`. Most CNI based networking solutions -do not require this flag. +allocating network ranges (CIDRs) to each node. This includes many cloud +providers and flannel. You can specify a subnet range that will be broken down +and handed out to each node with the `--pod-network-cidr` flag. This should be a +minimum of a /16 so controller-manager is able to assign /24 subnets to each +node in the cluster. If you are using flannel with [this +manifest](https://github.com/coreos/flannel/blob/master/Documentation/kube-flannel.yml) +you should use `--pod-network-cidr=10.244.0.0/16`. Most CNI based networking +solutions do not require this flag. - `--service-cidr` (default '10.96.0.0/12') You can use the `--service-cidr` flag to override the subnet Kubernetes uses to assign pods IP addresses. If you do, you will also need to update the -`/etc/systemd/system/kubelet.service.d/10-kubeadm.conf` file to reflect this change -else DNS will not function correctly. +`/etc/systemd/system/kubelet.service.d/10-kubeadm.conf` file to reflect this +change else DNS will not function correctly. - `--service-dns-domain` (default 'cluster.local') -By default, `kubeadm init` deploys a cluster that assigns services with DNS names -`..svc.cluster.local`. You can use the `--service-dns-domain` -to change the DNS name suffix. Again, you will need to update the -`/etc/systemd/system/kubelet.service.d/10-kubeadm.conf` file accordingly else DNS will -not function correctly. +By default, `kubeadm init` deploys a cluster that assigns services with DNS +names `..svc.cluster.local`. You can use the +`--service-dns-domain` to change the DNS name suffix. Again, you will need to +update the `/etc/systemd/system/kubelet.service.d/10-kubeadm.conf` file +accordingly else DNS will not function correctly. - `--skip-preflight-checks` -By default, `kubeadm` runs a series of preflight checks to validate the system +By default, kubeadm runs a series of preflight checks to validate the system before making any changes. Advanced users can use this flag to bypass these if necessary. @@ -141,43 +151,110 @@ necessary. By default, `kubeadm init` automatically generates the token used to initialise each new node. If you would like to manually specify this token, you can use the -`--token` flag. The token must be of the format `<6 character string>.<16 character string>`. +`--token` flag. The token must be of the format `[a-z0-9]{6}\.[a-z0-9]{16}`. A +compatible random token can be generated `kubeadm token generate`. Tokens can +be managed through the API after the cluster is created. See the [section on +managing tokens](#manage-tokens) below. -- `--kubernetes-version` (default 'latest') the kubernetes version to initialise +- `--token-ttl` -`kubeadm` was originally built for Kubernetes version **v1.4.0**, older versions are not -supported. The current version of `kubeadm` requires at least **v1.6.0-alpha.3** due to RBAC being enabled by default. -With this flag you can try any future version, e.g. **v1.6.0-beta.1** -whenever it comes out (check [releases page](https://github.com/kubernetes/kubernetes/releases) -for a full list of available versions). +This sets an expiration time for the token. This is specified as a duration +from the current time. After this time the token will no longer be valid and +will be removed. A value of 0 specifies that the token never expires. 0 is the +default. See the [section on managing tokens](#manage-tokens) below. ### `kubeadm join` -When you use kubeadm join, you must supply the token used to secure cluster -boostrap as a mandatory flag, and the master IP address as a mandatory argument. +When joining a kubeadm initialized cluster, we need to establish bidirectional +trust. This is split into discovery (having the Node trust the Kubernetes +master) and TLS bootstrap (having the Kubernetes master trust the Node). + +There are 2 main schemes for discovery. The first is to use a shared token along +with the IP address of the API server. The second is to provide a file (a subset +of the standard kubeconfig file). This file can be a local file or downloaded +via an HTTPS URL. The forms are `kubeadm join --discovery-token +abcdef.1234567890abcdef 1.2.3.4:6443`, `kubeadm join --discovery-file +path/to/file.conf` or `kubeadm join --discovery-file https://url/file.conf`. +Only one form can be used. If the discovery information is loaded from a URL, +HTTPS must be used and the host installed CA bundle is used to verify the +connection. + +The TLS bootstrap mechanism is also driven via a shared token. This is used to +temporarily authenticate with the Kubernetes master to submit a certificate +signing request (CSR) for a locally created key pair. By default kubeadm will +set up the Kubernetes master to automatically approve these signing requests. +This token is passed in with the `--tls-bootstrap-token abcdef.1234567890abcdef` +flag. + +Often times the same token is use for both parts. In this case, the `--token` flag +can be used instead of specifying the each token individually. Here's an example on how to use it: -`kubeadm join --token=the_secret_token 192.168.1.1` +`kubeadm join --token=abcdef.1234567890abcdef 192.168.1.1:6443` + +Specific options: + +- `--config` + +Extended options a specified in the [kubeadm specific config file](#config-file). - `--skip-preflight-checks` -By default, `kubeadm` runs a series of preflight checks to validate the system +By default, kubeadm runs a series of preflight checks to validate the system before making any changes. Advanced users can use this flag to bypass these if necessary. +- `--discovery-file` + +A local file path or HTTPS URL. The file specified must be a kubeconfig file +with nothing but an unnamed cluster entry. This is used to find both the +location of the API server to join along with a root CA bundle to use when +talking to that server. + +This might look something like this: + +``` yaml +apiVersion: v1 +clusters: +- cluster: + certificate-authority-data: + server: https://10.138.0.2:6443 + name: "" +contexts: [] +current-context: "" +kind: Config +preferences: {} +users: [] +``` + +- `--discovery-token` + +The discovery token is used along with the address of the API server (as an +unnamed argument) to download and verify information about the cluster. The +most critical part of the cluster information is the root CA bundle used to +verify the identity of the server during subsequent TLS connections. + +- `--tls-bootstrap-token` + +The token used to authenticate to the API server for the purposes of TLS +bootstrapping. + - `--token=` -By default, when `kubeadm init` runs, a token is generated and revealed in the output. -That's the token you should use here. +Often times the same token is used for both `--discovery-token` and +`--tls-bootstrap-token`. This option specifies the same token for both. Other +flags override this flag if present. +## Using kubeadm with a configuration file {#config-file} -## Using kubeadm with a configuration file +**WARNING:** While kubeadm command line interface is in beta, the config file is +still considered alpha and may change in future versions. -WARNING: kubeadm is in alpha and the configuration API syntax will likely change before GA. - -It's possible to configure kubeadm with a configuration file instead of command line flags, and some more advanced features may only be -available as configuration file options. +It's possible to configure kubeadm with a configuration file instead of command +line flags, and some more advanced features may only be available as +configuration file options. This file is passed in to the `--config` option on +both `kubeadm init` and `kubeadm join`. ### Sample Master Configuration @@ -185,16 +262,7 @@ available as configuration file options. apiVersion: kubeadm.k8s.io/v1alpha1 kind: MasterConfiguration api: - advertiseAddresses: - - - - - bindPort: - externalDNSNames: - - - - -authorizationMode: -cloudProvider: -discovery: + advertiseAddress: bindPort: etcd: endpoints: @@ -203,25 +271,29 @@ etcd: caFile: certFile: keyFile: -kubernetesVersion: networking: dnsDomain: serviceSubnet: podSubnet: -secrets: - givenToken: -apiServerExtraArgs: { - : , - : , -} -controllerManagerExtraArgs: { - : , - : , -} -schedulerExtraArgs: { - : , - : , -} +kubernetesVersion: +cloudProvider: +authorizationMode: +token: +tokenTTL: