Convert site to Hugo (#8316)
This commit converts content and layout to use Hugo.
This commit is contained in:
Bjørn Erik Pedersen
committed by
k8s-ci-robot
k8s-ci-robot
parent
7745f0e0c5
commit
7f3b633aa0
@@ -0,0 +1,14 @@
|
||||
---
|
||||
reviewers:
|
||||
- brendandburns
|
||||
- erictune
|
||||
- mikedanese
|
||||
no_issue: true
|
||||
title: Setup
|
||||
main_menu: true
|
||||
weight: 30
|
||||
---
|
||||
|
||||
This section provides instructions for installing Kubernetes and setting
|
||||
up a Kubernetes cluster. For an overview of the different options, see
|
||||
[Picking the Right Solution](/docs/setup/pick-right-solution/).
|
||||
@@ -0,0 +1,24 @@
|
||||
---
|
||||
reviewers:
|
||||
- david-mcmahon
|
||||
- jbeda
|
||||
title: Building from Source
|
||||
---
|
||||
|
||||
You can either build a release from source or download a pre-built release. If you do not plan on developing Kubernetes itself, we suggest using a pre-built version of the current release, which can be found in the [Release Notes](/docs/imported/release/notes/).
|
||||
|
||||
The Kubernetes source code can be downloaded from the [kubernetes/kubernetes repo](https://github.com/kubernetes/kubernetes).
|
||||
|
||||
### Building from source
|
||||
|
||||
If you are simply building a release from source there is no need to set up a full golang environment as all building happens in a Docker container.
|
||||
|
||||
Building a release is simple.
|
||||
|
||||
```shell
|
||||
git clone https://github.com/kubernetes/kubernetes.git
|
||||
cd kubernetes
|
||||
make release
|
||||
```
|
||||
|
||||
For more details on the release process see the kubernetes/kubernetes [`build`](http://releases.k8s.io/{{< param "githubbranch" >}}/build/) directory.
|
||||
Executable
+5
@@ -0,0 +1,5 @@
|
||||
---
|
||||
title: "Bootstrapping Clusters with kubeadm"
|
||||
weight: 20
|
||||
---
|
||||
|
||||
@@ -0,0 +1,546 @@
|
||||
---
|
||||
reviewers:
|
||||
- mikedanese
|
||||
- luxas
|
||||
- errordeveloper
|
||||
- jbeda
|
||||
title: Using kubeadm to Create a Cluster
|
||||
content_template: templates/task
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cncf/artwork/master/kubernetes/certified-kubernetes/versionless/color/certified-kubernetes-color.png" align="right" width="150px">**kubeadm** is a toolkit that helps you bootstrap a best-practice Kubernetes
|
||||
cluster in an easy, reasonably secure and extensible way. It also supports
|
||||
managing [Bootstrap Tokens](/docs/admin/bootstrap-tokens/) for you and upgrading/downgrading clusters.
|
||||
|
||||
kubeadm aims to set up a minimum viable cluster that pass the
|
||||
[Kubernetes Conformance tests](https://kubernetes.io/blog/2017/10/software-conformance-certification), but installing other addons than
|
||||
really necessary for a functional cluster is out of scope.
|
||||
|
||||
It by design does not install a networking solution for you, which means you
|
||||
have to install a third-party CNI-compliant networking solution yourself
|
||||
using `kubectl apply`.
|
||||
|
||||
kubeadm expects the user to bring a machine to execute on, the type doesn't
|
||||
matter, can be a Linux laptop, virtual machine, physical/cloud server or
|
||||
Raspberry Pi. This makes kubeadm well suited to integrate with provisioning
|
||||
systems of different kinds (e.g. Terraform, Ansible, etc.).
|
||||
|
||||
kubeadm is designed to be a simple way for new users to start trying
|
||||
Kubernetes out, possibly for the first time, a way for existing users to
|
||||
test their application on and stitch together a cluster easily, and also to be
|
||||
a building block in other ecosystem and/or installer tool with a larger
|
||||
scope.
|
||||
|
||||
You can install _kubeadm_ very easily on operating systems that support
|
||||
installing deb or rpm packages. The responsible SIG for kubeadm,
|
||||
[SIG Cluster Lifecycle](https://github.com/kubernetes/community/tree/master/sig-cluster-lifecycle), provides these packages pre-built for you,
|
||||
but you may also on other OSes.
|
||||
|
||||
|
||||
### kubeadm Maturity
|
||||
|
||||
| Area | Maturity Level |
|
||||
|---------------------------|--------------- |
|
||||
| Command line UX | beta |
|
||||
| Implementation | beta |
|
||||
| Config file API | alpha |
|
||||
| Self-hosting | alpha |
|
||||
| kubeadm alpha subcommands | alpha |
|
||||
| CoreDNS | alpha |
|
||||
| DynamicKubeletConfig | alpha |
|
||||
|
||||
|
||||
kubeadm's overall feature state is **Beta** and will soon be graduated to
|
||||
**General Availability (GA)** during 2018. Some sub-features, like self-hosting
|
||||
or the configuration file API are still under active development. The
|
||||
implementation of creating the cluster may change slightly as the tool evolves,
|
||||
but the overall implementation should be pretty stable. Any commands under
|
||||
`kubeadm alpha` are by definition, supported on an alpha level.
|
||||
|
||||
|
||||
### Support timeframes
|
||||
|
||||
Kubernetes releases are generally supported for nine months, and during that
|
||||
period a patch release may be issued from the release branch if a severe bug or
|
||||
security issue is found. Here are the latest Kubernetes releases and the support
|
||||
timeframe; which also applies to `kubeadm`.
|
||||
|
||||
| Kubernetes version | Release month | End-of-life-month |
|
||||
|--------------------|----------------|-------------------|
|
||||
| v1.6.x | March 2017 | December 2017 |
|
||||
| v1.7.x | June 2017 | March 2018 |
|
||||
| v1.8.x | September 2017 | June 2018 |
|
||||
| v1.9.x | December 2017 | September 2018 |
|
||||
| v1.10.x | March 2018 | December 2018 |
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture prerequisites %}}
|
||||
|
||||
1. One or more machines running a deb/rpm-compatible OS, e.g. Ubuntu or CentOS
|
||||
1. 2 GB or more of RAM per machine (any less will leave little room for your
|
||||
apps)
|
||||
1. 2 CPUs or more on the master
|
||||
1. Full network connectivity between all machines in the cluster (public or
|
||||
private network is fine)
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture steps %}}
|
||||
|
||||
## Objectives
|
||||
|
||||
* Install a secure Kubernetes cluster on your machines
|
||||
* Install a Pod network on the cluster so that your Pods can
|
||||
talk to each other
|
||||
|
||||
## Instructions
|
||||
|
||||
### (1/4) Installing kubeadm on your hosts
|
||||
|
||||
See [Installing kubeadm](/docs/setup/independent/install-kubeadm/).
|
||||
|
||||
**Note:** If you already have kubeadm installed, you should do a `apt-get update &&
|
||||
apt-get upgrade` or `yum update` to get the latest version of kubeadm.
|
||||
|
||||
|
||||
The kubelet is now restarting every few seconds, as it waits in a crashloop for
|
||||
kubeadm to tell it what to do. This crashloop is expected and normal, please
|
||||
proceed with the next step and the kubelet will start running normally.
|
||||
|
||||
### (2/4) Initializing your master
|
||||
|
||||
The master is the machine where the control plane components run, including
|
||||
etcd (the cluster database) and the API server (which the kubectl CLI
|
||||
communicates with).
|
||||
|
||||
To initialize the master, pick one of the machines you previously installed
|
||||
kubeadm on, and run:
|
||||
|
||||
```bash
|
||||
kubeadm init
|
||||
```
|
||||
|
||||
**Notes:**
|
||||
|
||||
- Please refer to the [kubeadm reference guide](/docs/reference/setup-tools/kubeadm/kubeadm/) if you want to
|
||||
read more about the flags `kubeadm init` provides. You can also specify a
|
||||
[configuration file](/docs/reference/setup-tools/kubeadm/kubeadm-init/#config-file) instead of using flags.
|
||||
- You need to choose a Pod Network Plugin in the next step. Depending on what
|
||||
third-party provider you choose, you might have to set the `--pod-network-cidr` to
|
||||
something provider-specific. The tabs below will contain a notice about what flags
|
||||
on `kubeadm init` are required.
|
||||
- Unless otherwise specified, kubeadm uses the default gateway's network interface
|
||||
to advertise the master's IP. If you want to use a different network interface, specify
|
||||
`--apiserver-advertise-address=<ip-address>` argument to `kubeadm init`. To deploy an IPv6 Kubernetes cluster using IPv6 addressing, you must specify an IPv6, e.g. `--apiserver-advertise-address=fd00::101`
|
||||
- If you would like to customise control plane components including optional IPv6 assignment to liveness probe for control plane components and etcd server, you can do so by providing extra args to each one, as documented [here](/docs/admin/kubeadm#custom-args).
|
||||
- `kubeadm init` will first run a series of prechecks to ensure that the machine
|
||||
is ready to run Kubernetes. It will expose warnings and exit on errors. It
|
||||
will then download and install the cluster database and control plane
|
||||
components. This may take several minutes.
|
||||
- You can't run `kubeadm init` twice without tearing down the cluster in between
|
||||
([unless you're upgrading from v1.6 to v1.7](/docs/tasks/administer-cluster/kubeadm-upgrade-1-7/)),
|
||||
see [Tear Down](#tear-down).
|
||||
|
||||
The output should look like:
|
||||
|
||||
```
|
||||
[init] Using Kubernetes version: v1.8.0
|
||||
[init] Using Authorization modes: [Node RBAC]
|
||||
[preflight] Running pre-flight checks
|
||||
[kubeadm] WARNING: starting in 1.8, tokens expire after 24 hours by default (if you require a non-expiring token use --token-ttl 0)
|
||||
[certificates] Generated ca certificate and key.
|
||||
[certificates] Generated apiserver certificate and key.
|
||||
[certificates] apiserver serving cert is signed for DNS names [kubeadm-master kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 10.138.0.4]
|
||||
[certificates] Generated apiserver-kubelet-client certificate and key.
|
||||
[certificates] Generated sa key and public key.
|
||||
[certificates] Generated front-proxy-ca certificate and key.
|
||||
[certificates] Generated front-proxy-client certificate and key.
|
||||
[certificates] Valid certificates and keys now exist in "/etc/kubernetes/pki"
|
||||
[kubeconfig] Wrote KubeConfig file to disk: "admin.conf"
|
||||
[kubeconfig] Wrote KubeConfig file to disk: "kubelet.conf"
|
||||
[kubeconfig] Wrote KubeConfig file to disk: "controller-manager.conf"
|
||||
[kubeconfig] Wrote KubeConfig file to disk: "scheduler.conf"
|
||||
[controlplane] Wrote Static Pod manifest for component kube-apiserver to "/etc/kubernetes/manifests/kube-apiserver.yaml"
|
||||
[controlplane] Wrote Static Pod manifest for component kube-controller-manager to "/etc/kubernetes/manifests/kube-controller-manager.yaml"
|
||||
[controlplane] Wrote Static Pod manifest for component kube-scheduler to "/etc/kubernetes/manifests/kube-scheduler.yaml"
|
||||
[etcd] Wrote Static Pod manifest for a local etcd instance to "/etc/kubernetes/manifests/etcd.yaml"
|
||||
[init] Waiting for the kubelet to boot up the control plane as Static Pods from directory "/etc/kubernetes/manifests"
|
||||
[init] This often takes around a minute; or longer if the control plane images have to be pulled.
|
||||
[apiclient] All control plane components are healthy after 39.511972 seconds
|
||||
[uploadconfig] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
|
||||
[markmaster] Will mark node master as master by adding a label and a taint
|
||||
[markmaster] Master master tainted and labelled with key/value: node-role.kubernetes.io/master=""
|
||||
[bootstraptoken] Using token: <token>
|
||||
[bootstraptoken] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
|
||||
[bootstraptoken] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
|
||||
[bootstraptoken] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
|
||||
[addons] Applied essential addon: kube-dns
|
||||
[addons] Applied essential addon: kube-proxy
|
||||
|
||||
Your Kubernetes master has initialized successfully!
|
||||
|
||||
To start using your cluster, you need to run (as a regular user):
|
||||
|
||||
mkdir -p $HOME/.kube
|
||||
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
|
||||
sudo chown $(id -u):$(id -g) $HOME/.kube/config
|
||||
|
||||
You should now deploy a pod network to the cluster.
|
||||
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
|
||||
http://kubernetes.io/docs/admin/addons/
|
||||
|
||||
You can now join any number of machines by running the following on each node
|
||||
as root:
|
||||
|
||||
kubeadm join --token <token> <master-ip>:<master-port> --discovery-token-ca-cert-hash sha256:<hash>
|
||||
```
|
||||
|
||||
To make kubectl work for your non-root user, you might want to run these commands (which is also a part of the `kubeadm init` output):
|
||||
|
||||
```bash
|
||||
mkdir -p $HOME/.kube
|
||||
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
|
||||
sudo chown $(id -u):$(id -g) $HOME/.kube/config
|
||||
```
|
||||
|
||||
Alternatively, if you are the `root` user, you could run this:
|
||||
|
||||
```bash
|
||||
export KUBECONFIG=/etc/kubernetes/admin.conf
|
||||
```
|
||||
|
||||
Make a record of the `kubeadm join` command that `kubeadm init` outputs. You
|
||||
will need this in a moment.
|
||||
|
||||
The token is used for mutual authentication between the master and the joining
|
||||
nodes. The token included here is secret, keep it safe as anyone with this
|
||||
token can add authenticated nodes to your cluster. These tokens can be listed,
|
||||
created and deleted with the `kubeadm token` command. See the
|
||||
[reference guide](/docs/reference/setup-tools/kubeadm/kubeadm-token/) for more details.
|
||||
|
||||
### (3/4) Installing a pod network {#pod-network}
|
||||
|
||||
You **MUST** install a pod network add-on so that your pods can communicate with
|
||||
each other.
|
||||
|
||||
**The network must be deployed before any applications. Also, kube-dns, an
|
||||
internal helper service, will not start up before a network is installed. kubeadm only
|
||||
supports Container Network Interface (CNI) based networks (and does not support kubenet).**
|
||||
|
||||
Several projects provide Kubernetes pod networks using CNI, some of which also
|
||||
support [Network Policy](/docs/concepts/services-networking/networkpolicies/). See the [add-ons page](/docs/concepts/cluster-administration/addons/) for a complete list of available network add-ons. IPv6 support was added in [CNI v0.6.0](https://github.com/containernetworking/cni/releases/tag/v0.6.0). [CNI bridge](https://github.com/containernetworking/plugins/blob/master/plugins/main/bridge/README.md) and [local-ipam](https://github.com/containernetworking/plugins/blob/master/plugins/ipam/host-local/README.md) are the only supported IPv6 network plugins in 1.9.
|
||||
|
||||
|
||||
**Note:** kubeadm sets up a more secure cluster by default and enforces use of [RBAC](#TODO).
|
||||
Please make sure that the network manifest of choice supports RBAC.
|
||||
|
||||
You can install a pod network add-on with the following command:
|
||||
|
||||
```bash
|
||||
kubectl apply -f <add-on.yaml>
|
||||
```
|
||||
|
||||
**NOTE:** You can install **only one** pod network per cluster.
|
||||
|
||||
{{< tabs name="tabs-pod-install" >}}
|
||||
{{% tab name="Choose one..." %}}
|
||||
Please select one of the tabs to see installation instructions for the respective third-party Pod Network Provider.
|
||||
{{% /tab %}}
|
||||
{{% tab name="Calico" %}}
|
||||
Refer to the Calico documentation for a [kubeadm quickstart](https://docs.projectcalico.org/latest/getting-started/kubernetes/), a [kubeadm installation guide](http://docs.projectcalico.org/latest/getting-started/kubernetes/installation/hosted/kubeadm/), and other resources.
|
||||
|
||||
**Note:**
|
||||
|
||||
- In order for Network Policy to work correctly, you need to pass `--pod-network-cidr=192.168.0.0/16` to `kubeadm init`.
|
||||
- Calico works on `amd64` only.
|
||||
|
||||
```shell
|
||||
kubectl apply -f https://docs.projectcalico.org/v3.0/getting-started/kubernetes/installation/hosted/kubeadm/1.7/calico.yaml
|
||||
```
|
||||
{{% /tab %}}
|
||||
{{% tab name="Canal" %}}
|
||||
The official Canal set-up guide is [here](https://github.com/projectcalico/canal/tree/master/k8s-install).
|
||||
|
||||
**Note:**
|
||||
|
||||
- For Canal to work correctly, `--pod-network-cidr=10.244.0.0/16` has to be passed to `kubeadm init`.
|
||||
- Canal works on `amd64` only.
|
||||
|
||||
```shell
|
||||
kubectl apply -f https://raw.githubusercontent.com/projectcalico/canal/master/k8s-install/1.7/rbac.yaml
|
||||
kubectl apply -f https://raw.githubusercontent.com/projectcalico/canal/master/k8s-install/1.7/canal.yaml
|
||||
```
|
||||
{{% /tab %}}
|
||||
{{% tab name="Flannel" %}}
|
||||
**Note:**
|
||||
|
||||
- For `flannel` to work correctly, `--pod-network-cidr=10.244.0.0/16` has to be passed to `kubeadm init`.
|
||||
- `flannel` works on `amd64`, `arm`, `arm64` and `ppc64le`, but for it to work on a platform other than
|
||||
`amd64` you have to manually download the manifest and replace `amd64` occurrences with your chosen platform.
|
||||
- Set `/proc/sys/net/bridge/bridge-nf-call-iptables` to `1` by running `sysctl net.bridge.bridge-nf-call-iptables=1`
|
||||
to pass bridged IPv4 traffic to iptables' chains. This is a requirement for some CNI plugins to work, for more information
|
||||
please see [here](https://kubernetes.io/docs/concepts/cluster-administration/network-plugins/#network-plugin-requirements).
|
||||
|
||||
```shell
|
||||
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/v0.9.1/Documentation/kube-flannel.yml
|
||||
```
|
||||
|
||||
- For more information about `flannel`, please see [here](https://github.com/coreos/flannel).
|
||||
{{% /tab %}}
|
||||
{{% tab name="Kube-router" %}}
|
||||
Set `/proc/sys/net/bridge/bridge-nf-call-iptables` to `1` by running `sysctl net.bridge.bridge-nf-call-iptables=1`
|
||||
to pass bridged IPv4 traffic to iptables' chains. This is a requirement for some CNI plugins to work, for more information
|
||||
please see [here](https://kubernetes.io/docs/concepts/cluster-administration/network-plugins/#network-plugin-requirements).
|
||||
|
||||
Kube-router relies on kube-controller-manager to allocate pod CIDR for the nodes. Therefore, use `kubeadm init` with the `--pod-network-cidr` flag.
|
||||
|
||||
Kube-router provides pod networking, network policy, and high-performing IP Virtual Server(IPVS)/Linux Virtual Server(LVS) based service proxy.
|
||||
|
||||
For information on setting up Kubernetes cluster with Kube-router using kubeadm, please see official [setup guide](https://github.com/cloudnativelabs/kube-router/blob/master/Documentation/kubeadm.md).
|
||||
{{% /tab %}}
|
||||
{{% tab name="Romana" %}}
|
||||
Set `/proc/sys/net/bridge/bridge-nf-call-iptables` to `1` by running `sysctl net.bridge.bridge-nf-call-iptables=1`
|
||||
to pass bridged IPv4 traffic to iptables' chains. This is a requirement for some CNI plugins to work, for more information
|
||||
please see [here](https://kubernetes.io/docs/concepts/cluster-administration/network-plugins/#network-plugin-requirements).
|
||||
|
||||
The official Romana set-up guide is [here](https://github.com/romana/romana/tree/master/containerize#using-kubeadm).
|
||||
|
||||
**Note:** Romana works on `amd64` only.
|
||||
|
||||
```shell
|
||||
kubectl apply -f https://raw.githubusercontent.com/romana/romana/master/containerize/specs/romana-kubeadm.yml
|
||||
```
|
||||
{{% /tab %}}
|
||||
{{% tab name="Weave Net" %}}
|
||||
Set `/proc/sys/net/bridge/bridge-nf-call-iptables` to `1` by running `sysctl net.bridge.bridge-nf-call-iptables=1`
|
||||
to pass bridged IPv4 traffic to iptables' chains. This is a requirement for some CNI plugins to work, for more information
|
||||
please see [here](https://kubernetes.io/docs/concepts/cluster-administration/network-plugins/#network-plugin-requirements).
|
||||
|
||||
The official Weave Net set-up guide is [here](https://www.weave.works/docs/net/latest/kube-addon/).
|
||||
|
||||
**Note:** Weave Net works on `amd64`, `arm`, `arm64` and `ppc64le` without any extra action required.
|
||||
Weave Net sets hairpin mode by default. This allows Pods to access themselves via their Service IP address
|
||||
if they don't know their PodIP.
|
||||
|
||||
```shell
|
||||
export kubever=$(kubectl version | base64 | tr -d '
|
||||
')
|
||||
kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version="
|
||||
```
|
||||
{{% /tab %}}
|
||||
{{< /tabs >}}
|
||||
|
||||
|
||||
Once a pod network has been installed, you can confirm that it is working by
|
||||
checking that the kube-dns pod is Running in the output of `kubectl get pods --all-namespaces`.
|
||||
And once the kube-dns pod is up and running, you can continue by joining your nodes.
|
||||
|
||||
If your network is not working or kube-dns is not in the Running state, check
|
||||
out our [troubleshooting docs](/docs/setup/independent/troubleshooting-kubeadm/).
|
||||
|
||||
#### Master Isolation
|
||||
|
||||
By default, your cluster will not schedule pods on the master for security
|
||||
reasons. If you want to be able to schedule pods on the master, e.g. for a
|
||||
single-machine Kubernetes cluster for development, run:
|
||||
|
||||
```bash
|
||||
kubectl taint nodes --all node-role.kubernetes.io/master-
|
||||
```
|
||||
|
||||
With output looking something like:
|
||||
|
||||
```
|
||||
node "test-01" untainted
|
||||
taint key="dedicated" and effect="" not found.
|
||||
taint key="dedicated" and effect="" not found.
|
||||
```
|
||||
|
||||
This will remove the `node-role.kubernetes.io/master` taint from any nodes that
|
||||
have it, including the master node, meaning that the scheduler will then be able
|
||||
to schedule pods everywhere.
|
||||
|
||||
### (4/4) Joining your nodes
|
||||
|
||||
The nodes are where your workloads (containers and pods, etc) run. To add new nodes to your cluster do the following for each machine:
|
||||
|
||||
* SSH to the machine
|
||||
* Become root (e.g. `sudo su -`)
|
||||
* Run the command that was output by `kubeadm init`. For example:
|
||||
|
||||
``` bash
|
||||
kubeadm join --token <token> <master-ip>:<master-port> --discovery-token-ca-cert-hash sha256:<hash>
|
||||
```
|
||||
|
||||
{{< note >}}
|
||||
**Note:** To specify an IPv6 tuple for `<master-ip>:<master-port>`, IPv6 address must be enclosed in square brackets, for example: `[fd00::101]:2073`.
|
||||
{{< /note >}}
|
||||
|
||||
The output should look something like:
|
||||
|
||||
```
|
||||
[preflight] Running pre-flight checks
|
||||
[discovery] Trying to connect to API Server "10.138.0.4:6443"
|
||||
[discovery] Created cluster-info discovery client, requesting info from "https://10.138.0.4:6443"
|
||||
[discovery] Requesting info from "https://10.138.0.4:6443" again to validate TLS against the pinned public key
|
||||
[discovery] Cluster info signature and contents are valid and TLS certificate validates against pinned roots, will use API Server "10.138.0.4:6443"
|
||||
[discovery] Successfully established connection with API Server "10.138.0.4:6443"
|
||||
[bootstrap] Detected server version: v1.8.0
|
||||
[bootstrap] The server supports the Certificates API (certificates.k8s.io/v1beta1)
|
||||
[csr] Created API client to obtain unique certificate for this node, generating keys and certificate signing request
|
||||
[csr] Received signed certificate from the API server, generating KubeConfig...
|
||||
|
||||
Node join complete:
|
||||
* Certificate signing request sent to master and response
|
||||
received.
|
||||
* Kubelet informed of new secure connection details.
|
||||
|
||||
Run 'kubectl get nodes' on the master to see this machine join.
|
||||
```
|
||||
|
||||
A few seconds later, you should notice this node in the output from `kubectl get
|
||||
nodes` when run on the master.
|
||||
|
||||
### (Optional) Controlling your cluster from machines other than the master
|
||||
|
||||
In order to get a kubectl on some other computer (e.g. laptop) to talk to your
|
||||
cluster, you need to copy the administrator kubeconfig file from your master
|
||||
to your workstation like this:
|
||||
|
||||
``` bash
|
||||
scp root@<master ip>:/etc/kubernetes/admin.conf .
|
||||
kubectl --kubeconfig ./admin.conf get nodes
|
||||
```
|
||||
|
||||
**Note:**
|
||||
- The example above assumes SSH access is enabled for root. If that is not the
|
||||
case, you can copy the `admin.conf` file to be accessible by some other user
|
||||
and `scp` using that other user instead.
|
||||
- The `admin.conf` file gives the user _superuser_ privileges over the cluster.
|
||||
This file should be used sparingly. For normal users, it's recommended to
|
||||
generate an unique credential to which you whitelist privileges. You can do
|
||||
this with the `kubeadm alpha phase kubeconfig user --client-name <CN>`
|
||||
command. That command will print out a KubeConfig file to STDOUT which you
|
||||
should save to a file and distribute to your user. After that, whitelist
|
||||
privileges by using `kubectl create (cluster)rolebinding`.
|
||||
|
||||
### (Optional) Proxying API Server to localhost
|
||||
|
||||
If you want to connect to the API Server from outside the cluster you can use
|
||||
`kubectl proxy`:
|
||||
|
||||
```bash
|
||||
scp root@<master ip>:/etc/kubernetes/admin.conf .
|
||||
kubectl --kubeconfig ./admin.conf proxy
|
||||
```
|
||||
|
||||
You can now access the API Server locally at `http://localhost:8001/api/v1`
|
||||
|
||||
## Tear down {#tear-down}
|
||||
|
||||
To undo what kubeadm did, you should first [drain the
|
||||
node](/docs/reference/generated/kubectl/kubectl-commands#drain) and make
|
||||
sure that the node is empty before shutting it down.
|
||||
|
||||
Talking to the master with the appropriate credentials, run:
|
||||
|
||||
```bash
|
||||
kubectl drain <node name> --delete-local-data --force --ignore-daemonsets
|
||||
kubectl delete node <node name>
|
||||
```
|
||||
|
||||
Then, on the node being removed, reset all kubeadm installed state:
|
||||
|
||||
```bash
|
||||
kubeadm reset
|
||||
```
|
||||
|
||||
If you wish to start over simply run `kubeadm init` or `kubeadm join` with the
|
||||
appropriate arguments.
|
||||
|
||||
More options and information about the
|
||||
[`kubeadm reset command`](/docs/reference/setup-tools/kubeadm/kubeadm-reset/).
|
||||
|
||||
## Upgrading a kubeadm cluster {#upgrades}
|
||||
|
||||
Instructions for upgrading kubeadm clusters are available for:
|
||||
|
||||
* [1.6 to 1.7 upgrades](/docs/tasks/administer-cluster/kubeadm-upgrade-1-7/)
|
||||
* [1.7.x to 1.7.y upgrades](/docs/tasks/administer-cluster/kubeadm-upgrade-1-8/)
|
||||
* [1.7 to 1.8 upgrades](/docs/tasks/administer-cluster/kubeadm-upgrade-1-8/)
|
||||
* [1.8.x to 1.8.y upgrades](/docs/tasks/administer-cluster/kubeadm-upgrade-1-8/)
|
||||
* [1.8 to 1.9 upgrades/downgrades](/docs/tasks/administer-cluster/kubeadm-upgrade-1-9/)
|
||||
* [1.9.x to 1.9.y upgrades](/docs/tasks/administer-cluster/kubeadm-upgrade-1-9/)
|
||||
* [1.9.x to 1.9.y HA cluster upgrades](/docs/tasks/administer-cluster/upgrade-downgrade/kubeadm-upgrade-ha/)
|
||||
|
||||
## Explore other add-ons {#other-addons}
|
||||
|
||||
See the [list of add-ons](/docs/concepts/cluster-administration/addons/) to explore other add-ons,
|
||||
including tools for logging, monitoring, network policy, visualization &
|
||||
control of your Kubernetes cluster.
|
||||
|
||||
## What's next {#whats-next}
|
||||
|
||||
* Learn about kubeadm's advanced usage in the [kubeadm reference documentation](/docs/reference/setup-tools/kubeadm/kubeadm)
|
||||
* Learn more about Kubernetes [concepts](/docs/concepts/) and [`kubectl`](/docs/user-guide/kubectl-overview/).
|
||||
* Configure log rotation. You can use **logrotate** for that. When using Docker, you can specify log rotation options for Docker daemon, for example `--log-driver=json-file --log-opt=max-size=10m --log-opt=max-file=5`. See [Configure and troubleshoot the Docker daemon](https://docs.docker.com/engine/admin/) for more details.
|
||||
|
||||
## Feedback {#feedback}
|
||||
|
||||
* kubeadm support Slack Channel:
|
||||
[#kubeadm](https://kubernetes.slack.com/messages/kubeadm/)
|
||||
* General SIG Cluster Lifecycle Development Slack Channel:
|
||||
[#sig-cluster-lifecycle](https://kubernetes.slack.com/messages/sig-cluster-lifecycle/)
|
||||
* SIG Cluster Lifecycle [SIG information](#TODO)
|
||||
* SIG Cluster Lifecycle Mailing List:
|
||||
[kubernetes-sig-cluster-lifecycle](https://groups.google.com/forum/#!forum/kubernetes-sig-cluster-lifecycle)
|
||||
* [kubeadm Github issue tracker](https://github.com/kubernetes/kubeadm/issues)
|
||||
|
||||
|
||||
## Version skew policy {#version-skew-policy}
|
||||
|
||||
The kubeadm CLI tool of version vX.Y may deploy clusters with a control plane of version vX.Y or vX.(Y-1).
|
||||
kubeadm CLI vX.Y can also upgrade an existing kubeadm-created cluster of version vX.(Y-1).
|
||||
|
||||
Due to that we can't see into the future, kubeadm CLI vX.Y may or may not be able to deploy vX.(Y+1) clusters.
|
||||
|
||||
Example: kubeadm v1.8 can deploy both v1.7 and v1.8 clusters and upgrade v1.7 kubeadm-created clusters to
|
||||
v1.8.
|
||||
|
||||
Please also check our [installation guide](/docs/setup/independent/install-kubeadm/#installing-kubeadm-kubelet-and-kubectl)
|
||||
for more information on the version skew between kubelets and the control plane.
|
||||
|
||||
## kubeadm works on multiple platforms {#multi-platform}
|
||||
|
||||
kubeadm deb/rpm packages and binaries are built for amd64, arm (32-bit), arm64, ppc64le, and s390x
|
||||
following the [multi-platform
|
||||
proposal](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/multi-platform.md).
|
||||
|
||||
Only some of the network providers offer solutions for all platforms. Please consult the list of
|
||||
network providers above or the documentation from each provider to figure out whether the provider
|
||||
supports your chosen platform.
|
||||
|
||||
## Limitations {#limitations}
|
||||
|
||||
Please note: kubeadm is a work in progress and these limitations will be
|
||||
addressed in due course.
|
||||
|
||||
1. The cluster created here has a single master, with a single etcd database
|
||||
running on it. This means that if the master fails, your cluster may lose
|
||||
data and may need to be recreated from scratch. Adding HA support
|
||||
(multiple etcd servers, multiple API servers, etc) to kubeadm is
|
||||
still a work-in-progress.
|
||||
|
||||
Workaround: regularly
|
||||
[back up etcd](https://coreos.com/etcd/docs/latest/admin_guide.html). The
|
||||
etcd data directory configured by kubeadm is at `/var/lib/etcd` on the master.
|
||||
|
||||
## Troubleshooting {#troubleshooting}
|
||||
|
||||
If you are running into difficulties with kubeadm, please consult our [troubleshooting docs](/docs/setup/independent/troubleshooting-kubeadm/).
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,546 @@
|
||||
---
|
||||
reviewers:
|
||||
- mikedanese
|
||||
- luxas
|
||||
- errordeveloper
|
||||
- jbeda
|
||||
title: Creating HA clusters with kubeadm
|
||||
content_template: templates/task
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
This guide shows you how to install and set up a highly available Kubernetes cluster using kubeadm.
|
||||
|
||||
This document shows you how to perform setup tasks that kubeadm doesn't perform: provision hardware; configure multiple systems; and load balancing.
|
||||
|
||||
{{< note >}}
|
||||
**Note:** This guide is only one potential solution, and there are many ways to configure a highly available cluster. If a better solution works for you, please use it. If you find a better solution that can be adopted by the community, feel free to contribute it back.
|
||||
{{< /note >}}
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture prerequisites %}}
|
||||
|
||||
- Three machines that meet [kubeadm's minimum requirements](/docs/setup/independent/install-kubeadm/#before-you-begin) for the masters
|
||||
- Three machines that meet [kubeadm's minimum requirements](/docs/setup/independent/install-kubeadm/#before-you-begin) for the workers
|
||||
- **Optional:** At least three machines that meet [kubeadm's minimum requirements](/docs/setup/independent/install-kubeadm/#before-you-begin)
|
||||
if you intend to host etcd on dedicated nodes (see information below)
|
||||
- 1GB or more of RAM per machine (any less will leave little room for your apps)
|
||||
- Full network connectivity between all machines in the cluster (public or
|
||||
private network is fine)
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture steps %}}
|
||||
|
||||
## Installing prerequisites on masters
|
||||
|
||||
For each master that has been provisioned, follow the [installation guide](/docs/setup/independent/install-kubeadm/) on how to install kubeadm and its dependencies. At the end of this step, you should have all the dependencies installed on each master.
|
||||
|
||||
## Setting up an HA etcd cluster
|
||||
|
||||
For highly available setups, you will need to decide how to host your etcd cluster. A cluster is composed of at least 3 members. We recommend one of the following models:
|
||||
|
||||
1. Hosting etcd cluster on separate compute nodes (Virtual Machines), or
|
||||
2. Hosting etcd cluster on the master nodes.
|
||||
|
||||
While the first option provides more performance and better hardware isolation, it is also more expensive and requires an additional support burden.
|
||||
|
||||
For **Option 1**: create 3 virtual machines that follow [CoreOS's hardware recommendations](https://coreos.com/etcd/docs/latest/op-guide/hardware.html). For the sake of simplicity, we will refer to them as `etcd0`, `etcd1` and `etcd2`.
|
||||
|
||||
For **Option 2**: you can skip to the next step. Any reference to `etcd0`, `etcd1` and `etcd2` throughout this guide should be replaced with `master0`, `master1` and `master2` accordingly, since your master nodes host etcd.
|
||||
|
||||
### Create etcd CA certs
|
||||
|
||||
1. Install `cfssl` and `cfssljson`:
|
||||
|
||||
```shell
|
||||
curl -o /usr/local/bin/cfssl https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
|
||||
curl -o /usr/local/bin/cfssljson https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
|
||||
chmod +x /usr/local/bin/cfssl*
|
||||
```
|
||||
|
||||
1. SSH into `etcd0` and run the following:
|
||||
|
||||
```shell
|
||||
mkdir -p /etc/kubernetes/pki/etcd
|
||||
cd /etc/kubernetes/pki/etcd
|
||||
```
|
||||
```shell
|
||||
cat >ca-config.json <<EOF
|
||||
{
|
||||
"signing": {
|
||||
"default": {
|
||||
"expiry": "43800h"
|
||||
},
|
||||
"profiles": {
|
||||
"server": {
|
||||
"expiry": "43800h",
|
||||
"usages": [
|
||||
"signing",
|
||||
"key encipherment",
|
||||
"server auth",
|
||||
"client auth"
|
||||
]
|
||||
},
|
||||
"client": {
|
||||
"expiry": "43800h",
|
||||
"usages": [
|
||||
"signing",
|
||||
"key encipherment",
|
||||
"client auth"
|
||||
]
|
||||
},
|
||||
"peer": {
|
||||
"expiry": "43800h",
|
||||
"usages": [
|
||||
"signing",
|
||||
"key encipherment",
|
||||
"server auth",
|
||||
"client auth"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
EOF
|
||||
```
|
||||
```shell
|
||||
cat >ca-csr.json <<EOF
|
||||
{
|
||||
"CN": "etcd",
|
||||
"key": {
|
||||
"algo": "rsa",
|
||||
"size": 2048
|
||||
}
|
||||
}
|
||||
EOF
|
||||
```
|
||||
|
||||
{{< note >}}
|
||||
**Optional:** You can modify `ca-csr.json` to add a section for `names`.
|
||||
See [the CFSSL wiki](https://github.com/cloudflare/cfssl/wiki/Creating-a-new-CSR) for an example.
|
||||
{{< /note >}}
|
||||
|
||||
1. Next, generate the CA certs like so:
|
||||
|
||||
```shell
|
||||
cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
|
||||
```
|
||||
|
||||
### Generate etcd client certs
|
||||
|
||||
1. Generate the client certificates.
|
||||
|
||||
While on `etcd0`, run the following:
|
||||
|
||||
```shell
|
||||
cat >client.json <<EOF
|
||||
{
|
||||
"CN": "client",
|
||||
"key": {
|
||||
"algo": "ecdsa",
|
||||
"size": 256
|
||||
}
|
||||
}
|
||||
EOF
|
||||
```
|
||||
```shell
|
||||
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=client client.json | cfssljson -bare client
|
||||
```
|
||||
|
||||
This should result in `client.pem` and `client-key.pem` being created.
|
||||
|
||||
### Create SSH access
|
||||
|
||||
In order to copy certs between machines, you must enable SSH access for `scp`.
|
||||
|
||||
1. First, open new tabs in your shell for `etcd1` and `etcd2`. Ensure you are SSHed into all three machines and then run the following (it will be a lot quicker if you use tmux syncing - to do this in iTerm enter `cmd+shift+i`):
|
||||
|
||||
```shell
|
||||
export PEER_NAME=$(hostname)
|
||||
export PRIVATE_IP=$(ip addr show eth1 | grep -Po 'inet \K[\d.]+')
|
||||
```
|
||||
|
||||
Make sure that `eth1` corresponds to the network interface for the IPv4 address of the private network. This might vary depending on your networking setup, so please check by running `echo $PRIVATE_IP` before continuing.
|
||||
|
||||
1. Next, generate some SSH keys for the boxes:
|
||||
|
||||
```shell
|
||||
ssh-keygen -t rsa -b 4096 -C "<email>"
|
||||
```
|
||||
|
||||
Make sure to replace `<email>` with your email, a placeholder, or an empty string. Keep hitting enter until files exist in `~/.ssh`.
|
||||
|
||||
1. Output the contents of the public key file for `etcd1` and `etcd2`, like so:
|
||||
|
||||
```shell
|
||||
cat ~/.ssh/id_rsa.pub
|
||||
```
|
||||
|
||||
1. Finally, copy the output for each and paste them into `etcd0`'s `~/.ssh/authorized_keys` file. This will permit `etcd1` and `etcd2` to SSH in to the machine.
|
||||
|
||||
### Generate etcd server and peer certs
|
||||
|
||||
1. In order to generate certs, each etcd machine needs the root CA generated by `etcd0`. On `etcd1` and `etcd2`, run the following:
|
||||
|
||||
```shell
|
||||
mkdir -p /etc/kubernetes/pki/etcd
|
||||
cd /etc/kubernetes/pki/etcd
|
||||
scp root@<etcd0-ip-address>:/etc/kubernetes/pki/etcd/ca.pem .
|
||||
scp root@<etcd0-ip-address>:/etc/kubernetes/pki/etcd/ca-key.pem .
|
||||
scp root@<etcd0-ip-address>:/etc/kubernetes/pki/etcd/client.pem .
|
||||
scp root@<etcd0-ip-address>:/etc/kubernetes/pki/etcd/client-key.pem .
|
||||
scp root@<etcd0-ip-address>:/etc/kubernetes/pki/etcd/ca-config.json .
|
||||
```
|
||||
|
||||
Where `<etcd0-ip-address>` corresponds to the public or private IPv4 of `etcd0`.
|
||||
|
||||
1. Once this is done, run the following on all etcd machines:
|
||||
|
||||
```shell
|
||||
cfssl print-defaults csr > config.json
|
||||
sed -i '0,/CN/{s/example\.net/'"$PEER_NAME"'/}' config.json
|
||||
sed -i 's/www\.example\.net/'"$PRIVATE_IP"'/' config.json
|
||||
sed -i 's/example\.net/'"$PEER_NAME"'/' config.json
|
||||
|
||||
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=server config.json | cfssljson -bare server
|
||||
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=peer config.json | cfssljson -bare peer
|
||||
```
|
||||
|
||||
The above will replace the default configuration with your machine's hostname as the peer name, and its IP addresses. Make sure
|
||||
these are correct before generating the certs. If you found an error, reconfigure `config.json` and re-run the `cfssl` commands.
|
||||
|
||||
This will result in the following files: `peer.pem`, `peer-key.pem`, `server.pem`, `server-key.pem`.
|
||||
|
||||
### {{< tabs name="etcd_mode" >}}
|
||||
{{% tab name="Choose one..." %}}
|
||||
Please select one of the tabs to see installation instructions for the respective way to run etcd.
|
||||
{{% /tab %}}
|
||||
{{% tab name="systemd" %}}
|
||||
1. First you will install etcd binaries like so:
|
||||
|
||||
```shell
|
||||
export ETCD_VERSION=v3.1.10
|
||||
curl -sSL https://github.com/coreos/etcd/releases/download//etcd--linux-amd64.tar.gz | tar -xzv --strip-components=1 -C /usr/local/bin/
|
||||
rm -rf etcd--linux-amd64*
|
||||
```
|
||||
|
||||
It is worth noting that etcd v3.1.10 is the preferred version for Kubernetes v1.9. For other versions of Kubernetes please consult [the changelog](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG.md).
|
||||
|
||||
Also, please realise that most distributions of Linux already have a version of etcd installed, so you will be replacing the system default.
|
||||
|
||||
1. Next, generate the environment file that systemd will use:
|
||||
|
||||
```
|
||||
touch /etc/etcd.env
|
||||
echo "PEER_NAME=" >> /etc/etcd.env
|
||||
echo "PRIVATE_IP=" >> /etc/etcd.env
|
||||
```
|
||||
|
||||
1. Now copy the systemd unit file like so:
|
||||
|
||||
```shell
|
||||
cat >/etc/systemd/system/etcd.service <<EOF
|
||||
[Unit]
|
||||
Description=etcd
|
||||
Documentation=https://github.com/coreos/etcd
|
||||
Conflicts=etcd.service
|
||||
Conflicts=etcd2.service
|
||||
|
||||
[Service]
|
||||
EnvironmentFile=/etc/etcd.env
|
||||
Type=notify
|
||||
Restart=always
|
||||
RestartSec=5s
|
||||
LimitNOFILE=40000
|
||||
TimeoutStartSec=0
|
||||
|
||||
ExecStart=/usr/local/bin/etcd --name --data-dir /var/lib/etcd --listen-client-urls https://:2379 --advertise-client-urls https://:2379 --listen-peer-urls https://:2380 --initial-advertise-peer-urls https://:2380 --cert-file=/etc/kubernetes/pki/etcd/server.pem --key-file=/etc/kubernetes/pki/etcd/server-key.pem --client-cert-auth --trusted-ca-file=/etc/kubernetes/pki/etcd/ca.pem --peer-cert-file=/etc/kubernetes/pki/etcd/peer.pem --peer-key-file=/etc/kubernetes/pki/etcd/peer-key.pem --peer-client-cert-auth --peer-trusted-ca-file=/etc/kubernetes/pki/etcd/ca.pem --initial-cluster <etcd0>=https://<etcd0-ip-address>:2380,<etcd1>=https://<etcd1-ip-address>:2380,<etcd2>=https://<etcd2-ip-address>:2380 --initial-cluster-token my-etcd-token --initial-cluster-state new
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
EOF
|
||||
```
|
||||
|
||||
Make sure you replace `<etcd0-ip-address>`, `<etcd1-ip-address>` and `<etcd2-ip-address>` with the appropriate IPv4 addresses. Also, make sure that you replace `<etcd0>`, `<etcd1>` and `<etcd2>` with real hostnames of each machine. These machines must be able to reach every other using DNS or make sure that records are added to `/etc/hosts`.
|
||||
|
||||
1. Finally, launch etcd like so:
|
||||
|
||||
```shell
|
||||
systemctl daemon-reload
|
||||
systemctl start etcd
|
||||
```
|
||||
|
||||
1. Check that it launched successfully:
|
||||
|
||||
```shell
|
||||
systemctl status etcd
|
||||
```
|
||||
{{% /tab %}}
|
||||
{{% tab name="Static Pods" %}}
|
||||
**Note**: This is only supported on nodes that have the all dependencies for the kubelet installed. If you are hosting etcd on the master nodes, this has already been set up. If you are hosting etcd on dedicated nodes, you should either use systemd or run the [installation guide](/docs/setup/independent/install-kubeadm/) on each dedicated etcd machine.
|
||||
|
||||
1. The first step is to run the following to generate the manifest file:
|
||||
|
||||
```shell
|
||||
cat >/etc/kubernetes/manifests/etcd.yaml <<EOF
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
labels:
|
||||
component: etcd
|
||||
tier: control-plane
|
||||
name: <podname>
|
||||
namespace: kube-system
|
||||
spec:
|
||||
containers:
|
||||
- command:
|
||||
- etcd --name - --data-dir /var/lib/etcd - --listen-client-urls https://:2379 - --advertise-client-urls https://:2379 - --listen-peer-urls https://:2380 - --initial-advertise-peer-urls https://:2380 - --cert-file=/certs/server.pem - --key-file=/certs/server-key.pem - --client-cert-auth - --trusted-ca-file=/certs/ca.pem - --peer-cert-file=/certs/peer.pem - --peer-key-file=/certs/peer-key.pem - --peer-client-cert-auth - --peer-trusted-ca-file=/certs/ca.pem - --initial-cluster etcd0=https://<etcd0-ip-address>:2380,etcd1=https://<etcd1-ip-address>:2380,etcd2=https://<etcd2-ip-address>:2380 - --initial-cluster-token my-etcd-token - --initial-cluster-state new
|
||||
image: k8s.gcr.io/etcd-amd64:3.1.10
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /health
|
||||
port: 2379
|
||||
scheme: HTTP
|
||||
initialDelaySeconds: 15
|
||||
timeoutSeconds: 15
|
||||
name: etcd
|
||||
env:
|
||||
- name: PUBLIC_IP
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: status.hostIP
|
||||
- name: PRIVATE_IP
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: status.podIP
|
||||
- name: PEER_NAME
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: metadata.name
|
||||
volumeMounts:
|
||||
- mountPath: /var/lib/etcd
|
||||
name: etcd
|
||||
- mountPath: /certs
|
||||
name: certs
|
||||
hostNetwork: true
|
||||
volumes:
|
||||
- hostPath:
|
||||
path: /var/lib/etcd
|
||||
type: DirectoryOrCreate
|
||||
name: etcd
|
||||
- hostPath:
|
||||
path: /etc/kubernetes/pki/etcd
|
||||
name: certs
|
||||
EOF
|
||||
```
|
||||
|
||||
Make sure you replace:
|
||||
* `<podname>` with the name of the node you're running on (e.g. `etcd0`, `etcd1` or `etcd2`)
|
||||
* `<etcd0-ip-address>`, `<etcd1-ip-address>` and `<etcd2-ip-address>` with the public IPv4s of the other machines that host etcd.
|
||||
{{% /tab %}}
|
||||
{{< /tabs >}}
|
||||
|
||||
|
||||
## {{< tabs name="lb_mode" >}}
|
||||
{{% tab name="Choose one..." %}}
|
||||
Please select one of the tabs to see installation instructions for the respective way to run etcd.
|
||||
{{% /tab %}}
|
||||
{{% tab name="Cloud" %}}
|
||||
Some examples of cloud provider solutions are:
|
||||
|
||||
* [AWS Elastic Load Balancer](https://aws.amazon.com/elasticloadbalancing/)
|
||||
* [GCE Load Balancing](https://cloud.google.com/compute/docs/load-balancing/)
|
||||
* [Azure](https://docs.microsoft.com/en-us/azure/load-balancer/load-balancer-overview)
|
||||
|
||||
You will need to ensure that the load balancer routes to **just `master0` on port 6443**. This is because kubeadm will perform health checks using the load balancer IP. Since `master0` is set up individually first, the other masters will not have running apiservers, which will result in kubeadm hanging indefinitely.
|
||||
|
||||
If possible, use a smart load balancing algorithm like "least connections", and use health checks so unhealthy nodes can be removed from circulation. Most providers will provide these features.
|
||||
{{% /tab %}}
|
||||
{{% tab name="On-Site" %}}
|
||||
In an on-site environment there may not be a physical load balancer available. Instead, a virtual IP pointing to a healthy master node can be used. There are a number of solutions for this including keepalived, Pacemaker and probably many others, some with and some without load balancing.
|
||||
|
||||
As an example we outline a simple setup based on keepalived. Depending on environment and requirements people may prefer different solutions. The configuration shown here provides an _active/passive_ failover without load balancing. If required, load balancing can by added quite easily by setting up HAProxy, NGINX or similar on the master nodes (not covered in this guide).
|
||||
|
||||
1. Install keepalived, e.g. using your distribution's package manager. The configuration shown here works with version `1.3.5` but is expected to work with may other versions. Make sure to have it enabled (chkconfig, systemd, ...) so that it starts automatically when the respective node comes up.
|
||||
|
||||
2. Create the following configuration file _/etc/keepalived/keepalived.conf_ on all master nodes:
|
||||
|
||||
```shell
|
||||
! Configuration File for keepalived
|
||||
global_defs {
|
||||
router_id LVS_DEVEL
|
||||
}
|
||||
|
||||
vrrp_script check_apiserver {
|
||||
script "/etc/keepalived/check_apiserver.sh"
|
||||
interval 3
|
||||
weight -2
|
||||
fall 10
|
||||
rise 2
|
||||
}
|
||||
|
||||
vrrp_instance VI_1 {
|
||||
state <STATE>
|
||||
interface <INTERFACE>
|
||||
virtual_router_id 51
|
||||
priority <PRIORITY>
|
||||
authentication {
|
||||
auth_type PASS
|
||||
auth_pass 4be37dc3b4c90194d1600c483e10ad1d
|
||||
}
|
||||
virtual_ipaddress {
|
||||
<VIRTUAL-IP>
|
||||
}
|
||||
track_script {
|
||||
check_apiserver
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
In the section `vrrp_instance VI_1`, change few lines depending on your setup:
|
||||
|
||||
* `state` is either `MASTER` (on the first master nodes) or `BACKUP` (the other master nodes).
|
||||
* `interface` is the name of an existing public interface to bind the virtual IP to (usually the primary interface).
|
||||
* `priority` should be higher for the first master node, e.g. 101, and lower for the others, e.g. 100.
|
||||
* `auth_pass` use any random string here.
|
||||
* `virtual_ipaddresses` should contain the virtual IP for the master nodes.
|
||||
|
||||
3. Install the following health check script to _/etc/keepalived/check_apiserver.sh_ on all master nodes:
|
||||
|
||||
```shell
|
||||
#!/bin/sh
|
||||
|
||||
errorExit() {
|
||||
echo "*** $*" 1>&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
curl --silent --max-time 2 --insecure https://localhost:6443/ -o /dev/null || errorExit "Error GET https://localhost:6443/"
|
||||
if ip addr | grep -q <VIRTUAL-IP>; then
|
||||
curl --silent --max-time 2 --insecure https://<VIRTUAL-IP>:6443/ -o /dev/null || errorExit "Error GET https://<VIRTUAL-IP>:6443/"
|
||||
fi
|
||||
```
|
||||
|
||||
Replace the `<VIRTUAL-IP>` by your chosen virtual IP.
|
||||
|
||||
4. Restart keepalived. While no Kubernetes services are up yet it will log health check fails on all master nodes. This will stop as soon as the first master node has been bootstrapped.
|
||||
{{% /tab %}}
|
||||
{{< /tabs >}}
|
||||
|
||||
|
||||
## Acquire etcd certs
|
||||
|
||||
Only follow this step if your etcd is hosted on dedicated nodes (**Option 1**). If you are hosting etcd on the masters (**Option 2**), you can skip this step since you've already generated the etcd certificates on the masters.
|
||||
|
||||
1. Generate SSH keys for each of the master nodes by following the steps in the [create ssh access](#create-ssh-access) section. After doing this, each master will have an SSH key in `~/.ssh/id_rsa.pub` and an entry in `etcd0`'s `~/.ssh/authorized_keys` file.
|
||||
|
||||
1. Run the following:
|
||||
|
||||
```shell
|
||||
mkdir -p /etc/kubernetes/pki/etcd
|
||||
scp root@<etcd0-ip-address>:/etc/kubernetes/pki/etcd/ca.pem /etc/kubernetes/pki/etcd
|
||||
scp root@<etcd0-ip-address>:/etc/kubernetes/pki/etcd/client.pem /etc/kubernetes/pki/etcd
|
||||
scp root@<etcd0-ip-address>:/etc/kubernetes/pki/etcd/client-key.pem /etc/kubernetes/pki/etcd
|
||||
```
|
||||
|
||||
## Run `kubeadm init` on `master0` {#kubeadm-init-master0}
|
||||
|
||||
1. In order for kubeadm to run, you first need to write a configuration file:
|
||||
|
||||
```shell
|
||||
cat >config.yaml <<EOF
|
||||
apiVersion: kubeadm.k8s.io/v1alpha1
|
||||
kind: MasterConfiguration
|
||||
api:
|
||||
advertiseAddress: <private-ip>
|
||||
etcd:
|
||||
endpoints:
|
||||
- https://<etcd0-ip-address>:2379
|
||||
- https://<etcd1-ip-address>:2379
|
||||
- https://<etcd2-ip-address>:2379
|
||||
caFile: /etc/kubernetes/pki/etcd/ca.pem
|
||||
certFile: /etc/kubernetes/pki/etcd/client.pem
|
||||
keyFile: /etc/kubernetes/pki/etcd/client-key.pem
|
||||
networking:
|
||||
podSubnet: <podCIDR>
|
||||
apiServerCertSANs:
|
||||
- <load-balancer-ip>
|
||||
apiServerExtraArgs:
|
||||
apiserver-count: "3"
|
||||
EOF
|
||||
```
|
||||
|
||||
Ensure that the following placeholders are replaced:
|
||||
|
||||
- `<private-ip>` with the private IPv4 of the master server.
|
||||
- `<etcd0-ip>`, `<etcd1-ip>` and `<etcd2-ip>` with the IP addresses of your three etcd nodes
|
||||
- `<podCIDR>` with your Pod CIDR. Please read the [CNI network section](/docs/setup/independent/create-cluster-kubeadm/#pod-network) of the docs for more information. Some CNI providers do not require a value to be set.
|
||||
- `<load-balancer-ip>` with the virtual IP set up in the load balancer. Please read [setting up a master load balancer](/docs/setup/independent/high-availability/#set-up-master-load-balancer) section of the docs for more information.
|
||||
|
||||
**Note:** If you are using Kubernetes 1.9+, you can replace the `apiserver-count: 3` extra argument with `endpoint-reconciler-type: lease`. For more information, see [the documentation](/docs/admin/high-availability/#endpoint-reconciler).
|
||||
|
||||
1. When this is done, run kubeadm like so:
|
||||
|
||||
```shell
|
||||
kubeadm init --config=config.yaml
|
||||
```
|
||||
|
||||
## Run `kubeadm init` on `master1` and `master2`
|
||||
|
||||
Before running kubeadm on the other masters, you need to first copy the K8s CA cert from `master0`. To do this, you have two options:
|
||||
|
||||
#### Option 1: Copy with scp
|
||||
|
||||
1. Follow the steps in the [create ssh access](#create-ssh-access) section, but instead of adding to `etcd0`'s `authorized_keys` file, add them to `master0`.
|
||||
1. Once you've done this, run:
|
||||
|
||||
```shell
|
||||
scp root@<master0-ip-address>:/etc/kubernetes/pki/* /etc/kubernetes/pki
|
||||
rm apiserver.*
|
||||
```
|
||||
|
||||
#### Option 2: Copy paste
|
||||
|
||||
1. Copy the contents of `/etc/kubernetes/pki/ca.crt`, `/etc/kubernetes/pki/ca.key`, `/etc/kubernetes/pki/sa.key` and `/etc/kubernetes/pki/sa.pub` and create these files manually on `master1` and `master2`.
|
||||
|
||||
When this is done, you can follow the [previous step](#kubeadm-init-master0) to install the control plane with kubeadm.
|
||||
|
||||
## Add `master1` and `master2` to load balancer
|
||||
|
||||
Once kubeadm has provisioned the other masters, you can add them to the load balancer pool.
|
||||
|
||||
## Install CNI network
|
||||
|
||||
Follow the instructions [here](/docs/setup/independent/create-cluster-kubeadm/#pod-network) to install the pod network. Make sure this corresponds to whichever pod CIDR you provided in the master configuration file.
|
||||
|
||||
## Install workers
|
||||
|
||||
Next provision and set up the worker nodes. To do this, you will need to provision at least 3 Virtual Machines.
|
||||
|
||||
1. To configure the worker nodes, [follow the same steps](/docs/setup/independent/create-cluster-kubeadm/#44-joining-your-nodes) as non-HA workloads.
|
||||
|
||||
## Configure workers
|
||||
|
||||
1. Reconfigure kube-proxy to access kube-apiserver via the load balancer:
|
||||
|
||||
```shell
|
||||
kubectl get configmap -n kube-system kube-proxy -o yaml > kube-proxy-cm.yaml
|
||||
sed -i 's#server:.*#server: https://<masterLoadBalancerFQDN>:6443#g' kube-proxy-cm.yaml
|
||||
kubectl apply -f kube-proxy-cm.yaml --force
|
||||
# restart all kube-proxy pods to ensure that they load the new configmap
|
||||
kubectl delete pod -n kube-system -l k8s-app=kube-proxy
|
||||
```
|
||||
|
||||
1. Reconfigure the kubelet to access kube-apiserver via the load balancer:
|
||||
|
||||
```shell
|
||||
sudo sed -i 's#server:.*#server: https://<masterLoadBalancerFQDN>:6443#g' /etc/kubernetes/kubelet.conf
|
||||
sudo systemctl restart kubelet
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,270 @@
|
||||
---
|
||||
title: Installing kubeadm
|
||||
content_template: templates/task
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cncf/artwork/master/kubernetes/certified-kubernetes/versionless/color/certified-kubernetes-color.png" align="right" width="150px">This page shows how to install the `kubeadm` toolbox.
|
||||
For information how to create a cluster with kubeadm once you have performed this installation process,
|
||||
see the [Using kubeadm to Create a Cluster](/docs/setup/independent/create-cluster-kubeadm/) page.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture prerequisites %}}
|
||||
|
||||
* One or more machines running one of:
|
||||
- Ubuntu 16.04+
|
||||
- Debian 9
|
||||
- CentOS 7
|
||||
- RHEL 7
|
||||
- Fedora 25/26 (best-effort)
|
||||
- HypriotOS v1.0.1+
|
||||
- Container Linux (tested with 1576.4.0)
|
||||
* 2 GB or more of RAM per machine (any less will leave little room for your apps)
|
||||
* 2 CPUs or more
|
||||
* Full network connectivity between all machines in the cluster (public or private network is fine)
|
||||
* Unique hostname, MAC address, and product_uuid for every node. See [here](https://kubernetes.io/docs/setup/independent/install-kubeadm/#verify-the-mac-address-and-product_uuid-are-unique-for-every-node) for more details.
|
||||
* Certain ports are open on your machines. See [here](/docs/setup/independent/install-kubeadm/#check-required-ports) for more details.
|
||||
* Swap disabled. You **MUST** disable swap in order for the kubelet to work properly.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture steps %}}
|
||||
|
||||
## Verify the MAC address and product_uuid are unique for every node
|
||||
|
||||
* You can get the MAC address of the network interfaces using the command `ip link` or `ifconfig -a`
|
||||
* The product_uuid can be checked by using the command `sudo cat /sys/class/dmi/id/product_uuid`
|
||||
|
||||
It is very likely that hardware devices will have unique addresses, although some virtual machines may have
|
||||
identical values. Kubernetes uses these values to uniquely identify the nodes in the cluster.
|
||||
If these values are not unique to each node, the installation process
|
||||
may [fail](https://github.com/kubernetes/kubeadm/issues/31).
|
||||
|
||||
## Check network adapters
|
||||
|
||||
If you have more than one network adapter, and your Kubernetes components are not reachable on the default
|
||||
route, we recommend you add IP route(s) so Kubernetes cluster addresses go via the appropriate adapter.
|
||||
|
||||
## Check required ports
|
||||
|
||||
### Master node(s)
|
||||
|
||||
| Protocol | Direction | Port Range | Purpose |
|
||||
|----------|-----------|------------|-------------------------|
|
||||
| TCP | Inbound | 6443* | Kubernetes API server |
|
||||
| TCP | Inbound | 2379-2380 | etcd server client API |
|
||||
| TCP | Inbound | 10250 | Kubelet API |
|
||||
| TCP | Inbound | 10251 | kube-scheduler |
|
||||
| TCP | Inbound | 10252 | kube-controller-manager |
|
||||
| TCP | Inbound | 10255 | Read-only Kubelet API |
|
||||
|
||||
### Worker node(s)
|
||||
|
||||
| Protocol | Direction | Port Range | Purpose |
|
||||
|----------|-----------|-------------|-----------------------|
|
||||
| TCP | Inbound | 10250 | Kubelet API |
|
||||
| TCP | Inbound | 10255 | Read-only Kubelet API |
|
||||
| TCP | Inbound | 30000-32767 | NodePort Services** |
|
||||
|
||||
** Default port range for [NodePort Services](/docs/concepts/services-networking/service/).
|
||||
|
||||
Any port numbers marked with * are overridable, so you will need to ensure any
|
||||
custom ports you provide are also open.
|
||||
|
||||
Although etcd ports are included in master nodes, you can also host your own
|
||||
etcd cluster externally or on custom ports.
|
||||
|
||||
The pod network plugin you use (see below) may also require certain ports to be
|
||||
open. Since this differs with each pod network plugin, please see the
|
||||
documentation for the plugins about what port(s) those need.
|
||||
|
||||
## Installing Docker
|
||||
|
||||
On each of your machines, install Docker.
|
||||
Version v1.12 is recommended, but v1.11, v1.13 and 17.03 are known to work as well.
|
||||
Versions 17.06+ _might work_, but have not yet been tested and verified by the Kubernetes node team.
|
||||
|
||||
Please proceed with executing the following commands based on your OS as root. You may become the root user by executing `sudo -i` after SSH-ing to each host.
|
||||
|
||||
If you already have the required versions of the Docker installed, you can move on to next section.
|
||||
If not, you can use the following commands to install Docker on your system:
|
||||
|
||||
{{< tabs name="docker_install" >}}
|
||||
{{% tab name="Ubuntu, Debian or HypriotOS" %}}
|
||||
Install Docker from Ubuntu's repositories:
|
||||
|
||||
```bash
|
||||
apt-get update
|
||||
apt-get install -y docker.io
|
||||
```
|
||||
|
||||
or install Docker CE 17.03 from Docker's repositories for Ubuntu or Debian:
|
||||
|
||||
```bash
|
||||
apt-get update
|
||||
apt-get install -y apt-transport-https ca-certificates curl software-properties-common
|
||||
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
|
||||
add-apt-repository "deb https://download.docker.com/linux/$(. /etc/os-release; echo "") $(lsb_release -cs) stable"
|
||||
apt-get update && apt-get install -y docker-ce=$(apt-cache madison docker-ce | grep 17.03 | head -1 | awk '{print }')
|
||||
```
|
||||
{{% /tab %}}
|
||||
{{% tab name="CentOS, RHEL or Fedora" %}}
|
||||
Install Docker using your operating system's bundled package:
|
||||
|
||||
```bash
|
||||
yum install -y docker
|
||||
systemctl enable docker && systemctl start docker
|
||||
```
|
||||
{{% /tab %}}
|
||||
{{% tab name="Container Linux" %}}
|
||||
Enable and start Docker:
|
||||
|
||||
```bash
|
||||
systemctl enable docker && systemctl start docker
|
||||
```
|
||||
{{% /tab %}}
|
||||
{{< /tabs >}}
|
||||
|
||||
|
||||
Refer to the [official Docker installation guides](https://docs.docker.com/engine/installation/)
|
||||
for more information.
|
||||
|
||||
## Installing kubeadm, kubelet and kubectl
|
||||
|
||||
You will install these packages on all of your machines:
|
||||
|
||||
* `kubeadm`: the command to bootstrap the cluster.
|
||||
|
||||
* `kubelet`: the component that runs on all of the machines in your cluster
|
||||
and does things like starting pods and containers.
|
||||
|
||||
* `kubectl`: the command line util to talk to your cluster.
|
||||
|
||||
kubeadm **will not** install or manage `kubelet` or `kubectl` for you, so you will
|
||||
need to ensure they match the version of the Kubernetes control panel you want
|
||||
kubeadm to install for you. If you do not, there is a risk of a version skew occurring that
|
||||
can lead to unexpected, buggy behaviour. However, _one_ minor version skew between the
|
||||
kubelet and the control plane is supported, but the kubelet version may never exceed the API
|
||||
server version. For example, kubelets running 1.7.0 should be fully compatible with a 1.8.0 API server,
|
||||
but not vice versa.
|
||||
|
||||
For more information on version skews, please read our
|
||||
[version skew policy](/docs/setup/independent/create-cluster-kubeadm/#version-skew-policy).
|
||||
|
||||
{{< tabs name="k8s_install" >}}
|
||||
{{< tab name="Ubuntu, Debian or HypriotOS" codelang="bash" >}}
|
||||
apt-get update && apt-get install -y apt-transport-https curl
|
||||
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
|
||||
cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
|
||||
deb http://apt.kubernetes.io/ kubernetes-xenial main
|
||||
EOF
|
||||
apt-get update
|
||||
apt-get install -y kubelet kubeadm kubectl
|
||||
{{< /tab >}}
|
||||
{{% tab name="CentOS, RHEL or Fedora" %}}
|
||||
``
|
||||
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
|
||||
[kubernetes]
|
||||
name=Kubernetes
|
||||
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-\
|
||||
enabled=1
|
||||
gpgcheck=1
|
||||
repo_gpgcheck=1
|
||||
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
|
||||
EOF
|
||||
setenforce 0
|
||||
yum install -y kubelet kubeadm kubectl
|
||||
systemctl enable kubelet && systemctl start kubelet
|
||||
```
|
||||
|
||||
**Note:**
|
||||
|
||||
- Disabling SELinux by running `setenforce 0` is required to allow containers to access the host filesystem, which is required by pod networks for example.
|
||||
You have to do this until SELinux support is improved in the kubelet.
|
||||
- Some users on RHEL/CentOS 7 have reported issues with traffic being routed incorrectly due to iptables being bypassed. You should ensure
|
||||
`net.bridge.bridge-nf-call-iptables` is set to 1 in your `sysctl` config, e.g.
|
||||
|
||||
``` bash
|
||||
cat <<EOF > /etc/sysctl.d/k8s.conf
|
||||
net.bridge.bridge-nf-call-ip6tables = 1
|
||||
net.bridge.bridge-nf-call-iptables = 1
|
||||
EOF
|
||||
sysctl --system
|
||||
```
|
||||
{{% /tab %}}
|
||||
{{% tab name="Container Linux" %}}
|
||||
Install CNI plugins (required for most pod network):
|
||||
|
||||
```bash
|
||||
CNI_VERSION="v0.6.0"
|
||||
mkdir -p /opt/cni/bin
|
||||
curl -L "https://github.com/containernetworking/plugins/releases/download//cni-plugins-amd64-.tgz" | tar -C /opt/cni/bin -xz
|
||||
```
|
||||
|
||||
Install `kubeadm`, `kubelet`, `kubectl` and add a `kubelet` systemd service:
|
||||
|
||||
```bash
|
||||
RELEASE="$(curl -sSL https://dl.k8s.io/release/stable.txt)"
|
||||
|
||||
mkdir -p /opt/bin
|
||||
cd /opt/bin
|
||||
curl -L --remote-name-all https://storage.googleapis.com/kubernetes-release/release//bin/linux/amd64/{kubeadm,kubelet,kubectl}
|
||||
chmod +x {kubeadm,kubelet,kubectl}
|
||||
|
||||
curl -sSL "https://raw.githubusercontent.com/kubernetes/kubernetes//build/debs/kubelet.service" | sed "s:/usr/bin:/opt/bin:g" > /etc/systemd/system/kubelet.service
|
||||
mkdir -p /etc/systemd/system/kubelet.service.d
|
||||
curl -sSL "https://raw.githubusercontent.com/kubernetes/kubernetes//build/debs/10-kubeadm.conf" | sed "s:/usr/bin:/opt/bin:g" > /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
|
||||
```
|
||||
|
||||
Enable and start `kubelet`:
|
||||
|
||||
```bash
|
||||
systemctl enable kubelet && systemctl start kubelet
|
||||
```
|
||||
{{% /tab %}}
|
||||
{{< /tabs >}}
|
||||
|
||||
|
||||
The kubelet is now restarting every few seconds, as it waits in a crashloop for
|
||||
kubeadm to tell it what to do.
|
||||
|
||||
## Configure cgroup driver used by kubelet on Master Node
|
||||
|
||||
Make sure that the cgroup driver used by kubelet is the same as the one used by Docker. Verify that your Docker cgroup driver matches the kubelet config:
|
||||
|
||||
```bash
|
||||
docker info | grep -i cgroup
|
||||
cat /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
|
||||
```
|
||||
|
||||
If the Docker cgroup driver and the kubelet config don't match, change the kubelet config to match the Docker cgroup driver. The
|
||||
flag you need to change is `--cgroup-driver`. If it's already set, you can update like so:
|
||||
|
||||
```bash
|
||||
sed -i "s/cgroup-driver=systemd/cgroup-driver=cgroupfs/g" /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
|
||||
```
|
||||
|
||||
Otherwise, you will need to open the systemd file and add the flag to an existing environment line.
|
||||
|
||||
Then restart kubelet:
|
||||
|
||||
```bash
|
||||
systemctl daemon-reload
|
||||
systemctl restart kubelet
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
If you are running into difficulties with kubeadm, please consult our [troubleshooting docs](/docs/setup/independent/troubleshooting-kubeadm/).
|
||||
|
||||
{{% capture whatsnext %}}
|
||||
|
||||
* [Using kubeadm to Create a Cluster](/docs/setup/independent/create-cluster-kubeadm/)
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,190 @@
|
||||
---
|
||||
title: Troubleshooting kubeadm
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
As with any program, you might run into an error using or operating it. Below we have listed
|
||||
common failure scenarios and have provided steps that will help you to understand and hopefully
|
||||
fix the problem.
|
||||
|
||||
If your problem is not listed below, please follow the following steps:
|
||||
|
||||
- If you think your problem is a bug with kubeadm:
|
||||
- Go to [github.com/kubernetes/kubeadm](https://github.com/kubernetes/kubeadm/issues) and search for existing issues.
|
||||
- If no issue exists, please [open one](https://github.com/kubernetes/kubeadm/issues/new) and follow the issue template.
|
||||
|
||||
- If you are unsure about how kubeadm or kubernetes works, and would like to receive
|
||||
support about your question, please ask on Slack in #kubeadm, or open a question on StackOverflow. Please include
|
||||
relevant tags like `#kubernetes` and `#kubeadm` so folks can help you.
|
||||
|
||||
If your cluster is in an error state, you may have trouble in the configuration if you see Pod statuses like `RunContainerError`,
|
||||
`CrashLoopBackOff` or `Error`. If this is the case, please read below.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
#### `ebtables` or some similar executable not found during installation
|
||||
|
||||
If you see the following warnings while running `kubeadm init`
|
||||
|
||||
```
|
||||
[preflight] WARNING: ebtables not found in system path
|
||||
[preflight] WARNING: ethtool not found in system path
|
||||
```
|
||||
|
||||
Then you may be missing `ebtables`, `ethtool` or a similar executable on your Linux machine. You can install them with the following commands:
|
||||
|
||||
- For ubuntu/debian users, run `apt install ebtables ethtool`.
|
||||
- For CentOS/Fedora users, run `yum install ebtables ethtool`.
|
||||
|
||||
#### kubeadm blocks waiting for control plane during installation
|
||||
|
||||
If you notice that `kubeadm init` hangs after printing out the following line:
|
||||
|
||||
```
|
||||
[apiclient] Created API client, waiting for the control plane to become ready
|
||||
```
|
||||
|
||||
This may be caused by a number of problems. The most common are:
|
||||
|
||||
- network connection problems. Check that your machine has full network connectivity before continuing.
|
||||
- the default cgroup driver configuration for the kubelet differs from that used by Docker.
|
||||
Check the system log file (e.g. `/var/log/message`) or examine the output from `journalctl -u kubelet`. If you see something like the following:
|
||||
|
||||
```shell
|
||||
error: failed to run Kubelet: failed to create kubelet:
|
||||
misconfiguration: kubelet cgroup driver: "systemd" is different from docker cgroup driver: "cgroupfs"
|
||||
```
|
||||
|
||||
There are two common ways to fix the cgroup driver problem:
|
||||
|
||||
1. Install docker again following instructions
|
||||
[here](/docs/setup/independent/install-kubeadm/#installing-docker).
|
||||
1. Change the kubelet config to match the Docker cgroup driver manually, you can refer to
|
||||
[Configure cgroup driver used by kubelet on Master Node](/docs/setup/independent/install-kubeadm/#configure-cgroup-driver-used-by-kubelet-on-master-node)
|
||||
for detailed instructions.
|
||||
The `kubectl describe pod` or `kubectl logs` commands can help you diagnose errors. For example:
|
||||
|
||||
```bash
|
||||
kubectl -n ${NAMESPACE} describe pod ${POD_NAME}
|
||||
|
||||
kubectl -n ${NAMESPACE} logs ${POD_NAME} -c ${CONTAINER_NAME}
|
||||
```
|
||||
|
||||
- control plane Docker containers are crashlooping or hanging. You can check this by running `docker ps` and investigating each container by running `docker logs`.
|
||||
|
||||
|
||||
#### Pods in `RunContainerError`, `CrashLoopBackOff` or `Error` state
|
||||
|
||||
Right after `kubeadm init` there should not be any such Pods. If there are Pods in
|
||||
such a state _right after_ `kubeadm init`, please open an issue in the kubeadm repo.
|
||||
`kube-dns` should be in the `Pending` state until you have deployed the network solution.
|
||||
However, if you see Pods in the `RunContainerError`, `CrashLoopBackOff` or `Error` state
|
||||
after deploying the network solution and nothing happens to `kube-dns`, it's very
|
||||
likely that the Pod Network solution that you installed is somehow broken. You
|
||||
might have to grant it more RBAC privileges or use a newer version. Please file
|
||||
an issue in the Pod Network providers' issue tracker and get the issue triaged there.
|
||||
|
||||
#### `kube-dns` is stuck in the `Pending` state
|
||||
|
||||
This is **expected** and part of the design. kubeadm is network provider-agnostic, so the admin
|
||||
should [install the pod network solution](/docs/concepts/cluster-administration/addons/)
|
||||
of choice. You have to install a Pod Network
|
||||
before `kube-dns` may deployed fully. Hence the `Pending` state before the network is set up.
|
||||
|
||||
#### `HostPort` services do not work
|
||||
|
||||
The `HostPort` and `HostIP` functionality is available depending on your Pod Network
|
||||
provider. Please contact the author of the Pod Network solution to find out whether
|
||||
`HostPort` and `HostIP` functionality are available.
|
||||
|
||||
Verified HostPort CNI providers:
|
||||
- Calico
|
||||
- Canal
|
||||
- Flannel
|
||||
|
||||
For more information, read the [CNI portmap documentation](https://github.com/containernetworking/plugins/blob/master/plugins/meta/portmap/README.md).
|
||||
|
||||
If your network provider does not support the portmap CNI plugin, you may need to use the [NodePort feature of
|
||||
services](/docs/concepts/services-networking/service/#type-nodeport) or use `HostNetwork=true`.
|
||||
|
||||
#### Pods are not accessible via their Service IP
|
||||
|
||||
Many network add-ons do not yet enable [hairpin mode](https://kubernetes.io/docs/tasks/debug-application-cluster/debug-service/#a-pod-cannot-reach-itself-via-service-ip)
|
||||
which allows pods to access themselves via their Service IP if they don't know about their podIP. This is an issue
|
||||
related to [CNI](https://github.com/containernetworking/cni/issues/476). Please contact the providers of the network
|
||||
add-on providers to get timely information about whether they support hairpin mode.
|
||||
|
||||
If you are using VirtualBox (directly or via Vagrant), you will need to
|
||||
ensure that `hostname -i` returns a routable IP address (i.e. one on the
|
||||
second network interface, not the first one). By default, it doesn't do this
|
||||
and kubelet ends-up using first non-loopback network interface, which is
|
||||
usually NATed. Workaround: Modify `/etc/hosts`, take a look at this
|
||||
`Vagrantfile`[ubuntu-vagrantfile](https://github.com/errordeveloper/k8s-playground/blob/22dd39dfc06111235620e6c4404a96ae146f26fd/Vagrantfile#L11) for how this can be achieved.
|
||||
|
||||
#### TLS certificate errors
|
||||
|
||||
The following error indicates a possible certificate mismatch.
|
||||
|
||||
```
|
||||
# kubectl get po
|
||||
Unable to connect to the server: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubernetes")
|
||||
```
|
||||
|
||||
Verify that the `$HOME/.kube/config` file contains a valid certificate, and regenerate a certificate if necessary.
|
||||
Another workaround is to overwrite the default `kubeconfig` for the "admin" user:
|
||||
|
||||
```
|
||||
mv $HOME/.kube $HOME/.kube.bak
|
||||
mkdir -p $HOME/.kube
|
||||
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
|
||||
sudo chown $(id -u):$(id -g) $HOME/.kube/config
|
||||
```
|
||||
|
||||
### Default NIC When using flannel as the pod network in Vagrant
|
||||
|
||||
The following error might indicate that something was wrong in the pod network:
|
||||
|
||||
```
|
||||
Error from server (NotFound): the server could not find the requested resource
|
||||
```
|
||||
|
||||
If you're using flannel as the pod network inside vagrant, then you will have to specify the default interface name for flannel.
|
||||
|
||||
Vagrant typically assigns two interfaces to all VMs. The first, for which all hosts are assigned the IP address `10.0.2.15`, is for external traffic that gets NATed.
|
||||
|
||||
This may lead to problems with flannel. By default, flannel selects the first interface on a host. This leads to all hosts thinking they have the same public IP address. To prevent this issue, pass the `--iface eth1` flag to flannel so that the second interface is chosen.
|
||||
|
||||
### Routing errors
|
||||
|
||||
In some situations `kubectl logs` and `kubectl run` commands may return with the following errors despite an otherwise apparently correctly working cluster:
|
||||
|
||||
```
|
||||
Error from server: Get https://10.19.0.41:10250/containerLogs/default/mysql-ddc65b868-glc5m/mysql: dial tcp 10.19.0.41:10250: getsockopt: no route to host
|
||||
```
|
||||
|
||||
This is due to Kubernetes using an IP that can not communicate with other IPs on the seemingly same subnet, possibly by policy of the machine provider. As an example, Digital Ocean assigns a public IP to `eth0` as well as a private one to be used internally as anchor for their floating IP feature, yet `kubelet` will pick the latter as the node's `InternalIP` instead of the public one.
|
||||
|
||||
Use `ip addr show` to check for this scenario instead of `ifconfig` because `ifconfig` will not display the offending alias IP address. Alternatively an API endpoint specific to Digital Ocean allows to query for the anchor IP from the droplet:
|
||||
|
||||
```
|
||||
curl http://169.254.169.254/metadata/v1/interfaces/public/0/anchor_ipv4/address
|
||||
```
|
||||
|
||||
The workaround is to tell `kubelet` which IP to use using `--node-ip`. When using Digital Ocean, it can be the public one (assigned to `eth0`) or the private one (assigned to `eth1`) should you want to use the optional private network. For example:
|
||||
|
||||
```
|
||||
IFACE=eth0 # change to eth1 for DO's private network
|
||||
DROPLET_IP_ADDRESS=$(ip addr show dev $IFACE | awk 'match($0,/inet (([0-9]|\.)+).* scope global/,a) { print a[1]; exit }')
|
||||
echo $DROPLET_IP_ADDRESS # check this, just in case
|
||||
echo "Environment=\"KUBELET_EXTRA_ARGS=--node-ip=$DROPLET_IP_ADDRESS\"" >> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
|
||||
```
|
||||
|
||||
Please note that this assumes `KUBELET_EXTRA_ARGS` hasn't already been set in the unit file.
|
||||
|
||||
Then restart `kubelet`:
|
||||
|
||||
```
|
||||
systemctl daemon-reload
|
||||
systemctl restart kubelet
|
||||
```
|
||||
@@ -0,0 +1,214 @@
|
||||
---
|
||||
reviewers:
|
||||
- brendandburns
|
||||
- erictune
|
||||
- mikedanese
|
||||
title: Picking the Right Solution
|
||||
---
|
||||
|
||||
Kubernetes can run on various platforms: from your laptop, to VMs on a cloud provider, to a rack of
|
||||
bare metal servers. The effort required to set up a cluster varies from running a single command to
|
||||
crafting your own customized cluster. Use this guide to choose a solution that fits your needs.
|
||||
|
||||
If you just want to "kick the tires" on Kubernetes, use the [local Docker-based solutions](#local-machine-solutions).
|
||||
|
||||
When you are ready to scale up to more machines and higher availability, a [hosted solution](#hosted-solutions) is the easiest to create and maintain.
|
||||
|
||||
[Turnkey cloud solutions](#turnkey-cloud-solutions) require only a few commands to create
|
||||
and cover a wide range of cloud providers. [On-Premises turnkey cloud solutions](#on-premises-turnkey-cloud-solutions) have the simplicity of the turnkey cloud solution combined with the security of your own private network.
|
||||
|
||||
If you already have a way to configure hosting resources, use [kubeadm](/docs/setup/independent/create-cluster-kubeadm/) to easily bring up a cluster with a single command per machine.
|
||||
|
||||
[Custom solutions](#custom-solutions) vary from step-by-step instructions to general advice for setting up
|
||||
a Kubernetes cluster from scratch.
|
||||
|
||||
{{< toc >}}
|
||||
|
||||
# Local-machine Solutions
|
||||
|
||||
* [Minikube](/docs/getting-started-guides/minikube/) is the recommended method for creating a local, single-node Kubernetes cluster for development and testing. Setup is completely automated and doesn't require a cloud provider account.
|
||||
|
||||
* [Kubeadm-dind](https://github.com/Mirantis/kubeadm-dind-cluster) is a multi-node (while minikube is single-node) Kubernetes cluster which only requires a docker daemon. It uses docker-in-docker technique to spawn the Kubernetes cluster.
|
||||
|
||||
* [Ubuntu on LXD](/docs/getting-started-guides/ubuntu/local/) supports a nine-instance deployment on localhost.
|
||||
|
||||
* [IBM Cloud Private-CE (Community Edition)](https://github.com/IBM/deploy-ibm-cloud-private) can use VirtualBox on your machine to deploy Kubernetes to one or more VMs for development and test scenarios. Scales to full multi-node cluster.
|
||||
|
||||
* [IBM Cloud Private-CE (Community Edition) on Linux Containers](https://github.com/HSBawa/icp-ce-on-linux-containers) is a Terraform/Packer/BASH based Infrastructure as Code (IaC) scripts to create a seven node (1 Boot, 1 Master, 1 Management, 1 Proxy and 3 Workers) LXD cluster on Linux Host.
|
||||
|
||||
# Hosted Solutions
|
||||
|
||||
* [Google Kubernetes Engine](https://cloud.google.com/kubernetes-engine/) offers managed Kubernetes clusters.
|
||||
|
||||
* [Amazon Elastic Container Service for Kubernetes](https://aws.amazon.com/eks/) offers managed Kubernetes service.
|
||||
|
||||
* [Azure Container Service](https://azure.microsoft.com/services/container-service/) offers managed Kubernetes clusters.
|
||||
|
||||
* [Stackpoint.io](https://stackpoint.io) provides Kubernetes infrastructure automation and management for multiple public clouds.
|
||||
|
||||
* [AppsCode.com](https://appscode.com/products/cloud-deployment/) provides managed Kubernetes clusters for various public clouds, including AWS and Google Cloud Platform.
|
||||
|
||||
* [Madcore.Ai](https://madcore.ai) is devops-focused CLI tool for deploying Kubernetes infrastructure in AWS. Master, auto-scaling group nodes with spot-instances, ingress-ssl-lego, Heapster, and Grafana.
|
||||
|
||||
* [Platform9](https://platform9.com/products/kubernetes/) offers managed Kubernetes on-premises or on any public cloud, and provides 24/7 health monitoring and alerting. (Kube2go, a web-UI driven Kubernetes cluster deployment service Platform9 released, has been integrated to Platform9 Sandbox.)
|
||||
|
||||
* [OpenShift Dedicated](https://www.openshift.com/dedicated/) offers managed Kubernetes clusters powered by OpenShift.
|
||||
|
||||
* [OpenShift Online](https://www.openshift.com/features/) provides free hosted access for Kubernetes applications.
|
||||
|
||||
* [IBM Cloud Container Service](https://console.bluemix.net/docs/containers/container_index.html) offers managed Kubernetes clusters with isolation choice, operational tools, integrated security insight into images and containers, and integration with Watson, IoT, and data.
|
||||
|
||||
* [Giant Swarm](https://giantswarm.io/product/) offers managed Kubernetes clusters in their own datacenter, on-premises, or on public clouds.
|
||||
|
||||
* [Kubermatic](https://www.loodse.com) provides managed Kubernetes clusters for various public clouds, including AWS and Digital Ocean, as well as on-premises with OpenStack integration.
|
||||
|
||||
* [Pivotal Container Service](https://pivotal.io/platform/pivotal-container-service) provides enterprise-grade Kubernetes for both on-premises and public clouds. PKS enables on-demand provisioning of Kubernetes clusters, multi-tenancy and fully automated day-2 operations.
|
||||
|
||||
# Turnkey Cloud Solutions
|
||||
|
||||
These solutions allow you to create Kubernetes clusters on a range of Cloud IaaS providers with only a
|
||||
few commands. These solutions are actively developed and have active community support.
|
||||
|
||||
* [Conjure-up Kubernetes with Ubuntu on AWS, Azure, Google Cloud, Oracle Cloud](/docs/getting-started-guides/ubuntu/)
|
||||
* [Google Compute Engine (GCE)](/docs/getting-started-guides/gce/)
|
||||
* [AWS](/docs/getting-started-guides/aws/)
|
||||
* [Azure](/docs/getting-started-guides/azure/)
|
||||
* [Tectonic by CoreOS](https://coreos.com/tectonic)
|
||||
* [CenturyLink Cloud](/docs/getting-started-guides/clc/)
|
||||
* [IBM Cloud](https://github.com/patrocinio/kubernetes-softlayer)
|
||||
* [Stackpoint.io](/docs/getting-started-guides/stackpoint/)
|
||||
* [Madcore.Ai](https://madcore.ai/)
|
||||
* [Kubermatic](https://cloud.kubermatic.io)
|
||||
|
||||
# On-Premises turnkey cloud solutions
|
||||
These solutions allow you to create Kubernetes clusters on your internal, secure, cloud network with only a
|
||||
few commands.
|
||||
|
||||
* [IBM Cloud Private](https://www.ibm.com/cloud-computing/products/ibm-cloud-private/)
|
||||
* [Kubermatic](https://www.loodse.com)
|
||||
* [SUSE CaaS Platform](https://www.suse.com/products/caas-platform)
|
||||
* [SUSE Cloud Application Platform](https://www.suse.com/products/cloud-application-platform/)
|
||||
|
||||
# Custom Solutions
|
||||
|
||||
Kubernetes can run on a wide range of Cloud providers and bare-metal environments, and with many
|
||||
base operating systems.
|
||||
|
||||
If you can find a guide below that matches your needs, use it. It may be a little out of date, but
|
||||
it will be easier than starting from scratch. If you do want to start from scratch, either because you
|
||||
have special requirements, or just because you want to understand what is underneath a Kubernetes
|
||||
cluster, try the [Getting Started from Scratch](/docs/getting-started-guides/scratch/) guide.
|
||||
|
||||
If you are interested in supporting Kubernetes on a new platform, see
|
||||
[Writing a Getting Started Guide](https://git.k8s.io/community/contributors/devel/writing-a-getting-started-guide.md).
|
||||
|
||||
## Universal
|
||||
|
||||
If you already have a way to configure hosting resources, use
|
||||
[kubeadm](/docs/setup/independent/create-cluster-kubeadm/) to easily bring up a cluster
|
||||
with a single command per machine.
|
||||
|
||||
## Cloud
|
||||
|
||||
These solutions are combinations of cloud providers and operating systems not covered by the above solutions.
|
||||
|
||||
* [CoreOS on AWS or GCE](/docs/getting-started-guides/coreos/)
|
||||
* [Kubernetes on Ubuntu](/docs/getting-started-guides/ubuntu/)
|
||||
* [Kubespray](/docs/getting-started-guides/kubespray/)
|
||||
|
||||
## On-Premises VMs
|
||||
|
||||
* [Vagrant](/docs/getting-started-guides/coreos/) (uses CoreOS and flannel)
|
||||
* [CloudStack](/docs/getting-started-guides/cloudstack/) (uses Ansible, CoreOS and flannel)
|
||||
* [VMware vSphere](https://vmware.github.io/vsphere-storage-for-kubernetes/documentation/)
|
||||
* [VMware vSphere, OpenStack, or Bare Metal](/docs/getting-started-guides/ubuntu/) (uses Juju, Ubuntu and flannel)
|
||||
* [VMware](/docs/getting-started-guides/coreos/) (uses CoreOS and flannel)
|
||||
* [oVirt](/docs/getting-started-guides/ovirt/)
|
||||
* [Fedora (Multi Node)](/docs/getting-started-guides/fedora/flannel_multi_node_cluster/) (uses Fedora and flannel)
|
||||
|
||||
## Bare Metal
|
||||
|
||||
* [Fedora (Single Node)](/docs/getting-started-guides/fedora/fedora_manual_config/)
|
||||
* [Fedora (Multi Node)](/docs/getting-started-guides/fedora/flannel_multi_node_cluster/)
|
||||
* [Kubernetes on Ubuntu](/docs/getting-started-guides/ubuntu/)
|
||||
* [CoreOS](/docs/getting-started-guides/coreos/)
|
||||
|
||||
## Integrations
|
||||
|
||||
These solutions provide integration with third-party schedulers, resource managers, and/or lower level platforms.
|
||||
|
||||
* [DCOS](/docs/getting-started-guides/dcos/)
|
||||
* Community Edition DCOS uses AWS
|
||||
* Enterprise Edition DCOS supports cloud hosting, on-premises VMs, and bare metal
|
||||
|
||||
# Table of Solutions
|
||||
|
||||
Below is a table of all of the solutions listed above.
|
||||
|
||||
IaaS Provider | Config. Mgmt. | OS | Networking | Docs | Support Level
|
||||
-------------------- | ------------ | ------ | ---------- | --------------------------------------------- | ----------------------------
|
||||
any | any | multi-support | any CNI | [docs](/docs/setup/independent/create-cluster-kubeadm/) | Project ([SIG-cluster-lifecycle](https://git.k8s.io/community/sig-cluster-lifecycle))
|
||||
Google Kubernetes Engine | | | GCE | [docs](https://cloud.google.com/kubernetes-engine/docs/) | Commercial
|
||||
Stackpoint.io | | multi-support | multi-support | [docs](https://stackpoint.io/) | Commercial
|
||||
AppsCode.com | Saltstack | Debian | multi-support | [docs](https://appscode.com/products/cloud-deployment/) | Commercial
|
||||
Madcore.Ai | Jenkins DSL | Ubuntu | flannel | [docs](https://madcore.ai) | Community ([@madcore-ai](https://github.com/madcore-ai))
|
||||
Platform9 | | multi-support | multi-support | [docs](https://platform9.com/managed-kubernetes/) | Commercial
|
||||
Kubermatic | | multi-support | multi-support | [docs](http://docs.kubermatic.io/) | Commercial
|
||||
Giant Swarm | | CoreOS | flannel and/or Calico | [docs](https://docs.giantswarm.io/) | Commercial
|
||||
GCE | Saltstack | Debian | GCE | [docs](/docs/getting-started-guides/gce/) | Project
|
||||
Azure Container Service | | Ubuntu | Azure | [docs](https://azure.microsoft.com/en-us/services/container-service/) | Commercial
|
||||
Azure (IaaS) | | Ubuntu | Azure | [docs](/docs/getting-started-guides/azure/) | [Community (Microsoft)](https://github.com/Azure/acs-engine)
|
||||
Bare-metal | custom | Fedora | _none_ | [docs](/docs/getting-started-guides/fedora/fedora_manual_config/) | Project
|
||||
Bare-metal | custom | Fedora | flannel | [docs](/docs/getting-started-guides/fedora/flannel_multi_node_cluster/) | Community ([@aveshagarwal](https://github.com/aveshagarwal))
|
||||
libvirt | custom | Fedora | flannel | [docs](/docs/getting-started-guides/fedora/flannel_multi_node_cluster/) | Community ([@aveshagarwal](https://github.com/aveshagarwal))
|
||||
KVM | custom | Fedora | flannel | [docs](/docs/getting-started-guides/fedora/flannel_multi_node_cluster/) | Community ([@aveshagarwal](https://github.com/aveshagarwal))
|
||||
DCOS | Marathon | CoreOS/Alpine | custom | [docs](/docs/getting-started-guides/dcos/) | Community ([Kubernetes-Mesos Authors](https://github.com/mesosphere/kubernetes-mesos/blob/master/AUTHORS.md))
|
||||
AWS | CoreOS | CoreOS | flannel | [docs](/docs/getting-started-guides/aws/) | Community
|
||||
GCE | CoreOS | CoreOS | flannel | [docs](/docs/getting-started-guides/coreos/) | Community ([@pires](https://github.com/pires))
|
||||
Vagrant | CoreOS | CoreOS | flannel | [docs](/docs/getting-started-guides/coreos/) | Community ([@pires](https://github.com/pires), [@AntonioMeireles](https://github.com/AntonioMeireles))
|
||||
CloudStack | Ansible | CoreOS | flannel | [docs](/docs/getting-started-guides/cloudstack/) | Community ([@sebgoa](https://github.com/sebgoa))
|
||||
VMware vSphere | any | multi-support | multi-support | [docs](https://vmware.github.io/vsphere-storage-for-kubernetes/documentation/) | [Community](https://vmware.github.io/vsphere-storage-for-kubernetes/documentation/contactus.html)
|
||||
Bare-metal | custom | CentOS | flannel | [docs](/docs/getting-started-guides/centos/centos_manual_config/) | Community ([@coolsvap](https://github.com/coolsvap))
|
||||
lxd | Juju | Ubuntu | flannel/canal | [docs](/docs/getting-started-guides/ubuntu/local/) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes)
|
||||
AWS | Juju | Ubuntu | flannel/calico/canal | [docs](/docs/getting-started-guides/ubuntu/) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes)
|
||||
Azure | Juju | Ubuntu | flannel/calico/canal | [docs](/docs/getting-started-guides/ubuntu/) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes)
|
||||
GCE | Juju | Ubuntu | flannel/calico/canal | [docs](/docs/getting-started-guides/ubuntu/) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes)
|
||||
Oracle Cloud | Juju | Ubuntu | flannel/calico/canal | [docs](/docs/getting-started-guides/ubuntu/) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes)
|
||||
Rackspace | Juju | Ubuntu | flannel/calico/canal | [docs](/docs/getting-started-guides/ubuntu/) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes)
|
||||
VMware vSphere | Juju | Ubuntu | flannel/calico/canal | [docs](/docs/getting-started-guides/ubuntu/) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes)
|
||||
Bare Metal | Juju | Ubuntu | flannel/calico/canal | [docs](/docs/getting-started-guides/ubuntu/) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes)
|
||||
AWS | Saltstack | Debian | AWS | [docs](/docs/getting-started-guides/aws/) | Community ([@justinsb](https://github.com/justinsb))
|
||||
AWS | kops | Debian | AWS | [docs](https://github.com/kubernetes/kops/) | Community ([@justinsb](https://github.com/justinsb))
|
||||
Bare-metal | custom | Ubuntu | flannel | [docs](/docs/getting-started-guides/ubuntu/) | Community ([@resouer](https://github.com/resouer), [@WIZARD-CXY](https://github.com/WIZARD-CXY))
|
||||
oVirt | | | | [docs](/docs/getting-started-guides/ovirt/) | Community ([@simon3z](https://github.com/simon3z))
|
||||
any | any | any | any | [docs](/docs/getting-started-guides/scratch/) | Community ([@erictune](https://github.com/erictune))
|
||||
any | any | any | any | [docs](http://docs.projectcalico.org/v2.2/getting-started/kubernetes/installation/) | Commercial and Community
|
||||
|
||||
**Note**: The above table is ordered by version test/used in nodes, followed by support level.
|
||||
|
||||
## Definition of columns
|
||||
|
||||
* **IaaS Provider** is the product or organization which provides the virtual or physical machines (nodes) that Kubernetes runs on.
|
||||
* **OS** is the base operating system of the nodes.
|
||||
* **Config. Mgmt.** is the configuration management system that helps install and maintain Kubernetes on the
|
||||
nodes.
|
||||
* **Networking** is what implements the [networking model](/docs/concepts/cluster-administration/networking/). Those with networking type
|
||||
_none_ may not support more than a single node, or may support multiple VM nodes in a single physical node.
|
||||
* **Conformance** indicates whether a cluster created with this configuration has passed the project's conformance
|
||||
tests for supporting the API and base features of Kubernetes v1.0.0.
|
||||
* **Support Levels**
|
||||
* **Project**: Kubernetes committers regularly use this configuration, so it usually works with the latest release
|
||||
of Kubernetes.
|
||||
* **Commercial**: A commercial offering with its own support arrangements.
|
||||
* **Community**: Actively supported by community contributions. May not work with recent releases of Kubernetes.
|
||||
* **Inactive**: Not actively maintained. Not recommended for first-time Kubernetes users, and may be removed.
|
||||
* **Notes** has other relevant information, such as the version of Kubernetes used.
|
||||
|
||||
|
||||
<!-- reference style links below here -->
|
||||
<!-- GCE conformance test result -->
|
||||
[1]: https://gist.github.com/erictune/4cabc010906afbcc5061
|
||||
<!-- Vagrant conformance test result -->
|
||||
[2]: https://gist.github.com/derekwaynecarr/505e56036cdf010bf6b6
|
||||
<!-- Google Kubernetes Engine conformance test result -->
|
||||
[3]: https://gist.github.com/erictune/2f39b22f72565365e59b
|
||||
Reference in New Issue
Block a user