whentimeslows/website
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

merge master to 1.10, with fixes (#7682)

Browse Source
This commit is contained in:
Jennifer Rondeau
2018-03-08 14:03:55 -05:00
committed by k8s-ci-robot
parent bb8c59a640
commit 44b51d6056
548 changed files with 11634 additions and 318622 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/setup/building-from-source.md
+1 -1
View File
@@ -1,5 +1,5 @@
---
approvers:
reviewers:
- david-mcmahon
- jbeda
title: Building from Source
docs/setup/hosted-solutions/overview.md
+20
View File
@@ -0,0 +1,20 @@
---
title: Hosted Kubernetes Solutions
---
{% capture overview %}
TODO
{% endcapture %}
{% capture body %}
TODO
{% endcapture %}
{% include templates/concept.md %}
docs/setup/independent/create-cluster-kubeadm.md
+4 -6
View File
@@ -1,5 +1,5 @@
---
approvers:
reviewers:
- mikedanese
- luxas
- errordeveloper
@@ -9,7 +9,7 @@ title: Using kubeadm to Create a Cluster
{% capture overview %}
<img src="https://raw.githubusercontent.com/cncf/artwork/master/kubernetes/certified-kubernetes/versionless/color/certified_kubernetes_color.png" align="right" width="150px">**kubeadm** is a toolkit that helps you bootstrap a best-practice Kubernetes
<img src="https://raw.githubusercontent.com/cncf/artwork/master/kubernetes/certified-kubernetes/versionless/color/certified-kubernetes-color.png" align="right" width="150px">**kubeadm** is a toolkit that helps you bootstrap a best-practice Kubernetes
cluster in an easy, reasonably secure and extensible way. It also supports
managing [Bootstrap Tokens](/docs/admin/bootstrap-tokens/) for you and upgrading/downgrading clusters.
@@ -28,7 +28,7 @@ systems of different kinds (e.g. Terraform, Ansible, etc.).
kubeadm is designed to be a simple way for new users to start trying
Kubernetes out, possibly for the first time, a way for existing users to
test their application on and stich together a cluster easily, and also to be
test their application on and stitch together a cluster easily, and also to be
a building block in other ecosystem and/or installer tool with a larger
scope.
@@ -144,7 +144,6 @@ see [Tear Down](#tear-down).
The output should look like:
```
[kubeadm] WARNING: kubeadm is in beta, please do not use it for production clusters.
[init] Using Kubernetes version: v1.8.0
[init] Using Authorization modes: [Node RBAC]
[preflight] Running pre-flight checks
@@ -257,7 +256,7 @@ Refer to the Calico documentation for a [kubeadm quickstart](https://docs.projec
- Calico works on `amd64` only.
```shell
kubectl apply -f https://docs.projectcalico.org/v2.6/getting-started/kubernetes/installation/hosted/kubeadm/1.6/calico.yaml
kubectl apply -f https://docs.projectcalico.org/v3.0/getting-started/kubernetes/installation/hosted/kubeadm/1.7/calico.yaml
```
{% endcapture %}
@@ -394,7 +393,6 @@ kubeadm join --token <token> <master-ip>:<master-port> --discovery-token-ca-cert
The output should look something like:
```
[kubeadm] WARNING: kubeadm is in beta, please do not use it for production clusters.
[preflight] Running pre-flight checks
[discovery] Trying to connect to API Server "10.138.0.4:6443"
[discovery] Created cluster-info discovery client, requesting info from "https://10.138.0.4:6443"
docs/setup/independent/high-availability.md
+384 -297
View File
@@ -1,5 +1,5 @@
---
approvers:
reviewers:
- mikedanese
- luxas
- errordeveloper
@@ -11,7 +11,7 @@ title: Creating HA clusters with kubeadm
This guide shows you how to install and set up a highly available Kubernetes cluster using kubeadm.
This document shows you how to perform setup tasks that kubeadm doesn't perform: provision hardware; configure multiple systems; and load balancing.
This document shows you how to perform setup tasks that kubeadm doesn't perform: provision hardware; configure multiple systems; and load balancing.
**Note:** This guide is only one potential solution, and there are many ways to configure a highly available cluster. If a better solution works for you, please use it. If you find a better solution that can be adopted by the community, feel free to contribute it back.
{: .note}
@@ -22,7 +22,7 @@ This document shows you how to perform setup tasks that kubeadm doesn't perform:
- Three machines that meet [kubeadm's minimum requirements](https://kubernetes.io/docs/setup/independent/install-kubeadm/#before-you-begin) for the masters
- Three machines that meet [kubeadm's minimum requirements](https://kubernetes.io/docs/setup/independent/install-kubeadm/#before-you-begin) for the workers
- **Optional:** At least three machines that meet [kubeadm's minimum requirements](https://kubernetes.io/docs/setup/independent/install-kubeadm/#before-you-begin)
- **Optional:** At least three machines that meet [kubeadm's minimum requirements](https://kubernetes.io/docs/setup/independent/install-kubeadm/#before-you-begin)
if you intend to host etcd on dedicated nodes (see information below)
- 1GB or more of RAM per machine (any less will leave little room for your apps)
- Full network connectivity between all machines in the cluster (public or
@@ -45,8 +45,7 @@ For highly available setups, you will need to decide how to host your etcd clust
While the first option provides more performance and better hardware isolation, it is also more expensive and requires an additional support burden.
For **Option 1**: create 3 virtual machines that follow [CoreOS's hardware recommendations](https://coreos.com/etcd/docs/latest/op-guide/hardware.html). For the sake of simplicity, we
will refer to them as `etcd0`, `etcd1` and `etcd2`.
For **Option 1**: create 3 virtual machines that follow [CoreOS's hardware recommendations](https://coreos.com/etcd/docs/latest/op-guide/hardware.html). For the sake of simplicity, we will refer to them as `etcd0`, `etcd1` and `etcd2`.
For **Option 2**: you can skip to the next step. Any reference to `etcd0`, `etcd1` and `etcd2` throughout this guide should be replaced with `master0`, `master1` and `master2` accordingly, since your master nodes host etcd.
@@ -54,97 +53,99 @@ For **Option 2**: you can skip to the next step. Any reference to `etcd0`, `etcd
1. Install `cfssl` and `cfssljson`:
```shell
curl -o /usr/local/bin/cfssl https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
curl -o /usr/local/bin/cfssljson https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
chmod +x /usr/local/bin/cfssl*
```
```shell
curl -o /usr/local/bin/cfssl https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
curl -o /usr/local/bin/cfssljson https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
chmod +x /usr/local/bin/cfssl*
```
1. SSH into `etcd0` and run the following:
```shell
mkdir -p /etc/kubernetes/pki/etcd
cd /etc/kubernetes/pki/etcd
```
```shell
cat >ca-config.json <<EOL
{
"signing": {
"default": {
"expiry": "43800h"
},
"profiles": {
"server": {
"expiry": "43800h",
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
]
},
"client": {
"expiry": "43800h",
"usages": [
"signing",
"key encipherment",
"client auth"
]
},
"peer": {
"expiry": "43800h",
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
]
}
}
}
}
EOL
```
```shell
cat >ca-csr.json <<EOL
{
"CN": "etcd",
"key": {
"algo": "rsa",
"size": 2048
}
}
EOL
```
```shell
mkdir -p /etc/kubernetes/pki/etcd
cd /etc/kubernetes/pki/etcd
```
```shell
cat >ca-config.json <<EOF
{
"signing": {
"default": {
"expiry": "43800h"
},
"profiles": {
"server": {
"expiry": "43800h",
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
]
},
"client": {
"expiry": "43800h",
"usages": [
"signing",
"key encipherment",
"client auth"
]
},
"peer": {
"expiry": "43800h",
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
]
}
}
}
}
EOF
```
```shell
cat >ca-csr.json <<EOF
{
"CN": "etcd",
"key": {
"algo": "rsa",
"size": 2048
}
}
EOF
```
Ensure that the `names` section in `ca-csr.json` matches your own company or personal address, or that you use a suitable default.
**Optional:** You can modify `ca-csr.json` to add a section for `names`.
See [the CFSSL wiki](https://github.com/cloudflare/cfssl/wiki/Creating-a-new-CSR) for an example.
{: .note}
1. Next, generate the CA certs like so:
```shell
cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
```
```shell
cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
```
### Generate etcd client certs
1. Generate the client certificates.
1. Generate the client certificates.
While on `etcd0`, run the following:
```shell
cat >client.json <<EOL
{
"CN": "client",
"key": {
"algo": "ecdsa",
"size": 256
}
}
EOL
```
```shell
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=client client.json | cfssljson -bare client
```
```shell
cat >client.json <<EOF
{
"CN": "client",
"key": {
"algo": "ecdsa",
"size": 256
}
}
EOF
```
```shell
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=client client.json | cfssljson -bare client
```
This should result in `client.pem` and `client-key.pem` being created.
@@ -152,28 +153,28 @@ This should result in `client.pem` and `client-key.pem` being created.
In order to copy certs between machines, you must enable SSH access for `scp`.
1. First, open new new tabs in your shell for `etcd1` and `etcd2`. Ensure you are SSHed into all three machines and then run the following (it will be a lot quicker if you use tmux syncing - to do this in iTerm enter `cmd+shift+i`):
1. First, open new tabs in your shell for `etcd1` and `etcd2`. Ensure you are SSHed into all three machines and then run the following (it will be a lot quicker if you use tmux syncing - to do this in iTerm enter `cmd+shift+i`):
```shell
export PEER_NAME=$(hostname)
export PRIVATE_IP=$(ip addr show eth1 | grep -Po 'inet \K[\d.]+')
```
```shell
export PEER_NAME=$(hostname)
export PRIVATE_IP=$(ip addr show eth1 | grep -Po 'inet \K[\d.]+')
```
Make sure that `eth1` corresponds to the network interface for the IPv4 address of the private network. This might vary depending on your networking setup, so please check by running `echo $PRIVATE_IP` before continuing.
Make sure that `eth1` corresponds to the network interface for the IPv4 address of the private network. This might vary depending on your networking setup, so please check by running `echo $PRIVATE_IP` before continuing.
1. Next, generate some SSH keys for the boxes:
```shell
ssh-keygen -t rsa -b 4096 -C "<email>"
```
```shell
ssh-keygen -t rsa -b 4096 -C "<email>"
```
Make sure to replace `<email>` with your email, a placeholder, or an empty string. Keep hitting enter until files exist in `~/.ssh`.
Make sure to replace `<email>` with your email, a placeholder, or an empty string. Keep hitting enter until files exist in `~/.ssh`.
1. Output the contents of the public key file for `etcd1` and `etcd2`, like so:
```shell
cat ~/.ssh/id_rsa.pub
```
```shell
cat ~/.ssh/id_rsa.pub
```
1. Finally, copy the output for each and paste them into `etcd0`'s `~/.ssh/authorized_keys` file. This will permit `etcd1` and `etcd2` to SSH in to the machine.
@@ -181,38 +182,38 @@ In order to copy certs between machines, you must enable SSH access for `scp`.
1. In order to generate certs, each etcd machine needs the root CA generated by `etcd0`. On `etcd1` and `etcd2`, run the following:
```shell
mkdir -p /etc/kubernetes/pki/etcd
cd /etc/kubernetes/pki/etcd
scp root@<etcd0-ip-address>:/etc/kubernetes/pki/etcd/ca.pem .
scp root@<etcd0-ip-address>:/etc/kubernetes/pki/etcd/ca-key.pem .
scp root@<etcd0-ip-address>:/etc/kubernetes/pki/etcd/client.pem .
scp root@<etcd0-ip-address>:/etc/kubernetes/pki/etcd/client-key.pem .
scp root@<etcd0-ip-address>:/etc/kubernetes/pki/etcd/ca-config.json .
```
```shell
mkdir -p /etc/kubernetes/pki/etcd
cd /etc/kubernetes/pki/etcd
scp root@<etcd0-ip-address>:/etc/kubernetes/pki/etcd/ca.pem .
scp root@<etcd0-ip-address>:/etc/kubernetes/pki/etcd/ca-key.pem .
scp root@<etcd0-ip-address>:/etc/kubernetes/pki/etcd/client.pem .
scp root@<etcd0-ip-address>:/etc/kubernetes/pki/etcd/client-key.pem .
scp root@<etcd0-ip-address>:/etc/kubernetes/pki/etcd/ca-config.json .
```
Where `<etcd0-ip-address>` corresponds to the public or private IPv4 of `etcd0`.
Where `<etcd0-ip-address>` corresponds to the public or private IPv4 of `etcd0`.
1. Once this is done, run the following on all etcd machines:
```shell
cfssl print-defaults csr > config.json
sed -i '0,/CN/{s/example\.net/'"$PEER_NAME"'/}' config.json
sed -i 's/www\.example\.net/'"$PRIVATE_IP"'/' config.json
sed -i 's/example\.net/'"$PUBLIC_IP"'/' config.json
```shell
cfssl print-defaults csr > config.json
sed -i '0,/CN/{s/example\.net/'"$PEER_NAME"'/}' config.json
sed -i 's/www\.example\.net/'"$PRIVATE_IP"'/' config.json
sed -i 's/example\.net/'"$PEER_NAME"'/' config.json
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=server config.json | cfssljson -bare server
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=peer config.json | cfssljson -bare peer
```
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=server config.json | cfssljson -bare server
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=peer config.json | cfssljson -bare peer
```
The above will replace the default configuration with your machine's hostname as the peer name, and its IP addresses. Make sure
these are correct before generating the certs. If you found an error, reconfigure `config.json` and re-run the `cfssl` commands.
The above will replace the default configuration with your machine's hostname as the peer name, and its IP addresses. Make sure
these are correct before generating the certs. If you found an error, reconfigure `config.json` and re-run the `cfssl` commands.
This will result in the following files: `peer.pem`, `peer-key.pem`, `server.pem`, `server-key.pem`.
### Run etcd
Now that all the certificates have been generated, you will now install and set up etcd on each machine.
Now that all the certificates have been generated, you will now install and set up etcd on each machine.
{% capture choose %}
Please select one of the tabs to see installation instructions for the respective way to run etcd.
@@ -222,79 +223,79 @@ Please select one of the tabs to see installation instructions for the respectiv
1. First you will install etcd binaries like so:
```shell
export ETCD_VERSION=v3.1.10
curl -sSL https://github.com/coreos/etcd/releases/download/${ETCD_VERSION}/etcd-${ETCD_VERSION}-linux-amd64.tar.gz | tar -xzv --strip-components=1 -C /usr/local/bin/
rm -rf etcd-$ETCD_VERSION-linux-amd64*
```
```shell
export ETCD_VERSION=v3.1.10
curl -sSL https://github.com/coreos/etcd/releases/download/${ETCD_VERSION}/etcd-${ETCD_VERSION}-linux-amd64.tar.gz | tar -xzv --strip-components=1 -C /usr/local/bin/
rm -rf etcd-$ETCD_VERSION-linux-amd64*
```
It is worth noting that etcd v3.1.10 is the preferred version for Kubernetes v1.9. For other versions of Kubernetes please consult [the changelog](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG.md).
It is worth noting that etcd v3.1.10 is the preferred version for Kubernetes v1.9. For other versions of Kubernetes please consult [the changelog](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG.md).
Also, please realise that most distributions of Linux already have a version of etcd installed, so you will be replacing the system default.
Also, please realise that most distributions of Linux already have a version of etcd installed, so you will be replacing the system default.
1. Next, generate the environment file that systemd will use:
```
touch /etc/etcd.env
echo "PEER_NAME=$PEER_NAME" >> /etc/etcd.env
echo "PRIVATE_IP=$PRIVATE_IP" >> /etc/etcd.env
```
```
touch /etc/etcd.env
echo "PEER_NAME=$PEER_NAME" >> /etc/etcd.env
echo "PRIVATE_IP=$PRIVATE_IP" >> /etc/etcd.env
```
1. Now copy the systemd unit file like so:
```shell
cat >/etc/systemd/system/etcd.service <<EOL
[Unit]
Description=etcd
Documentation=https://github.com/coreos/etcd
Conflicts=etcd.service
Conflicts=etcd2.service
```shell
cat >/etc/systemd/system/etcd.service <<EOF
[Unit]
Description=etcd
Documentation=https://github.com/coreos/etcd
Conflicts=etcd.service
Conflicts=etcd2.service
[Service]
EnvironmentFile=/etc/etcd.env
Type=notify
Restart=always
RestartSec=5s
LimitNOFILE=40000
TimeoutStartSec=0
[Service]
EnvironmentFile=/etc/etcd.env
Type=notify
Restart=always
RestartSec=5s
LimitNOFILE=40000
TimeoutStartSec=0
ExecStart=/usr/local/bin/etcd --name ${PEER_NAME} \
--data-dir /var/lib/etcd \
--listen-client-urls https://${PRIVATE_IP}:2379 \
--advertise-client-urls https://${PRIVATE_IP}:2379 \
--listen-peer-urls https://${PRIVATE_IP}:2380 \
--initial-advertise-peer-urls https://${PRIVATE_IP}:2380 \
--cert-file=/etc/kubernetes/pki/etcd/server.pem \
--key-file=/etc/kubernetes/pki/etcd/server-key.pem \
--client-cert-auth \
--trusted-ca-file=/etc/kubernetes/pki/etcd/ca.pem \
--peer-cert-file=/etc/kubernetes/pki/etcd/peer.pem \
--peer-key-file=/etc/kubernetes/pki/etcd/peer-key.pem \
--peer-client-cert-auth \
--peer-trusted-ca-file=/etc/kubernetes/pki/etcd/ca.pem \
--initial-cluster etcd0=https://<etcd0-ip-address>:2380,etcd1=https://<etcd1-ip-address>:2380,etcd2=https://<etcd2-ip-address>:2380 \
--initial-cluster-token my-etcd-token \
--initial-cluster-state new
ExecStart=/usr/local/bin/etcd --name ${PEER_NAME} \
--data-dir /var/lib/etcd \
--listen-client-urls https://${PRIVATE_IP}:2379 \
--advertise-client-urls https://${PRIVATE_IP}:2379 \
--listen-peer-urls https://${PRIVATE_IP}:2380 \
--initial-advertise-peer-urls https://${PRIVATE_IP}:2380 \
--cert-file=/etc/kubernetes/pki/etcd/server.pem \
--key-file=/etc/kubernetes/pki/etcd/server-key.pem \
--client-cert-auth \
--trusted-ca-file=/etc/kubernetes/pki/etcd/ca.pem \
--peer-cert-file=/etc/kubernetes/pki/etcd/peer.pem \
--peer-key-file=/etc/kubernetes/pki/etcd/peer-key.pem \
--peer-client-cert-auth \
--peer-trusted-ca-file=/etc/kubernetes/pki/etcd/ca.pem \
--initial-cluster etcd0=https://<etcd0-ip-address>:2380,etcd1=https://<etcd1-ip-address>:2380,etcd2=https://<etcd2-ip-address>:2380 \
--initial-cluster-token my-etcd-token \
--initial-cluster-state new
[Install]
WantedBy=multi-user.target
EOL
```
[Install]
WantedBy=multi-user.target
EOF
```
Make sure you replace `<etcd0-ip-address>`, `<etcd1-ip-address>` and `<etcd2-ip-address>` with the appropriate IPv4 addresses.
Make sure you replace `<etcd0-ip-address>`, `<etcd1-ip-address>` and `<etcd2-ip-address>` with the appropriate IPv4 addresses.
1. Finally, launch etcd like so:
```shell
systemctl daemon-reload
systemctl start etcd
```
```shell
systemctl daemon-reload
systemctl start etcd
```
1. Check that it launched successfully:
```shell
systemctl status etcd
```
```shell
systemctl status etcd
```
{% endcapture %}
{% capture static_pods %}
@@ -303,81 +304,82 @@ Please select one of the tabs to see installation instructions for the respectiv
1. The first step is to run the following to generate the manifest file:
```shell
cat >/etc/kubernetes/manifests/etcd.yaml <<EOL
apiVersion: v1
kind: Pod
metadata:
labels:
component: etcd
tier: control-plane
name: <podname>
namespace: kube-system
spec:
containers:
- command:
- etcd --name ${PEER_NAME} \
- --data-dir /var/lib/etcd \
- --listen-client-urls https://${PRIVATE_IP}:2379 \
- --advertise-client-urls https://${PRIVATE_IP}:2379 \
- --listen-peer-urls https://${PRIVATE_IP}:2380 \
- --initial-advertise-peer-urls https://${PRIVATE_IP}:2380 \
- --cert-file=/certs/server.pem \
- --key-file=/certs/server-key.pem \
- --client-cert-auth \
- --trusted-ca-file=/certs/ca.pem \
- --peer-cert-file=/certs/peer.pem \
- --peer-key-file=/certs/peer-key.pem \
- --peer-client-cert-auth \
- --peer-trusted-ca-file=/certs/ca.pem \
- --initial-cluster etcd0=https://<etcd0-ip-address>:2380,etcd1=https://<etcd1-ip-address>:2380,etcd1=https://<etcd2-ip-address>:2380 \
- --initial-cluster-token my-etcd-token \
- --initial-cluster-state new
image: gcr.io/google_containers/etcd-amd64:3.1.0
livenessProbe:
httpGet:
path: /health
port: 2379
scheme: HTTP
initialDelaySeconds: 15
timeoutSeconds: 15
name: etcd
env:
- name: PUBLIC_IP
valueFrom:
fieldRef:
fieldPath: status.hostIP
- name: PRIVATE_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: PEER_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
volumeMounts:
- mountPath: /var/lib/etcd
name: etcd
- mountPath: /certs
name: certs
hostNetwork: true
volumes:
- hostPath:
path: /var/lib/etcd
type: DirectoryOrCreate
name: etcd
- hostPath:
path: /etc/kubernetes/pki/etcd
name: certs
EOL
```
```shell
cat >/etc/kubernetes/manifests/etcd.yaml <<EOF
apiVersion: v1
kind: Pod
metadata:
labels:
component: etcd
tier: control-plane
name: <podname>
namespace: kube-system
spec:
containers:
- command:
- etcd --name ${PEER_NAME} \
- --data-dir /var/lib/etcd \
- --listen-client-urls https://${PRIVATE_IP}:2379 \
- --advertise-client-urls https://${PRIVATE_IP}:2379 \
- --listen-peer-urls https://${PRIVATE_IP}:2380 \
- --initial-advertise-peer-urls https://${PRIVATE_IP}:2380 \
- --cert-file=/certs/server.pem \
- --key-file=/certs/server-key.pem \
- --client-cert-auth \
- --trusted-ca-file=/certs/ca.pem \
- --peer-cert-file=/certs/peer.pem \
- --peer-key-file=/certs/peer-key.pem \
- --peer-client-cert-auth \
- --peer-trusted-ca-file=/certs/ca.pem \
- --initial-cluster etcd0=https://<etcd0-ip-address>:2380,etcd1=https://<etcd1-ip-address>:2380,etcd2=https://<etcd2-ip-address>:2380 \
- --initial-cluster-token my-etcd-token \
- --initial-cluster-state new
image: gcr.io/google_containers/etcd-amd64:3.1.0
livenessProbe:
httpGet:
path: /health
port: 2379
scheme: HTTP
initialDelaySeconds: 15
timeoutSeconds: 15
name: etcd
env:
- name: PUBLIC_IP
valueFrom:
fieldRef:
fieldPath: status.hostIP
- name: PRIVATE_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: PEER_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
volumeMounts:
- mountPath: /var/lib/etcd
name: etcd
- mountPath: /certs
name: certs
hostNetwork: true
volumes:
- hostPath:
path: /var/lib/etcd
type: DirectoryOrCreate
name: etcd
- hostPath:
path: /etc/kubernetes/pki/etcd
name: certs
EOF
```
Make sure you replace:
* `<podname>` with the name of the node you're running on (e.g. `etcd0`, `etcd1` or `etcd2`)
* `<etcd0-ip-address>`, `<etcd1-ip-address>` and `<etcd2-ip-address>` with the public IPv4s of the other machines that host etcd.
Make sure you replace:
* `<podname>` with the name of the node you're running on (e.g. `etcd0`, `etcd1` or `etcd2`)
* `<etcd0-ip-address>`, `<etcd1-ip-address>` and `<etcd2-ip-address>` with the public IPv4s of the other machines that host etcd.
{% endcapture %}
{% assign tab_set_name = "etcd_mode" %}
{% assign tab_names = "Choose one...,systemd,Static Pods" | split: ',' | compact %}
{% assign tab_contents = site.emptyArray | push: choose | push: systemd | push: static_pods %}
@@ -385,7 +387,14 @@ Please select one of the tabs to see installation instructions for the respectiv
## Set up master Load Balancer
The next step is to create a Load Balancer that sits in front of your master nodes. How you do this depends on your environment; you could, for example, leverage a cloud provider Load Balancer, or set up your own using nginx, keepalived, or HAproxy. Some examples of cloud provider solutions are:
The next step is to create a Load Balancer that sits in front of your master nodes. How you do this depends on your environment; you could, for example, leverage a cloud provider Load Balancer, or set up your own using NGINX, keepalived, or HAproxy.
{% capture choose %}
Please select one of the tabs to see installation instructions for information on load balancing in the respective environment.
{% endcapture %}
{% capture cloud %}
Some examples of cloud provider solutions are:
* [AWS Elastic Load Balancer](https://aws.amazon.com/elasticloadbalancing/)
* [GCE Load Balancing](https://cloud.google.com/compute/docs/load-balancing/)
@@ -394,6 +403,84 @@ The next step is to create a Load Balancer that sits in front of your master nod
You will need to ensure that the load balancer routes to **just `master0` on port 6443**. This is because kubeadm will perform health checks using the load balancer IP. Since `master0` is set up individually first, the other masters will not have running apiservers, which will result in kubeadm hanging indefinitely.
If possible, use a smart load balancing algorithm like "least connections", and use health checks so unhealthy nodes can be removed from circulation. Most providers will provide these features.
{% endcapture %}
{% capture onsite %}
In an on-site environment there may not be a physical load balancer available. Instead, a virtual IP pointing to a healthy master node can be used. There are a number of solutions for this including keepalived, Pacemaker and probably many others, some with and some without load balancing.
As an example we outline a simple setup based on keepalived. Depending on environment and requirements people may prefer different solutions. The configuration shown here provides an _active/passive_ failover without load balancing. If required, load balancing can by added quite easily by setting up HAProxy, NGINX or similar on the master nodes (not covered in this guide).
1. Install keepalived, e.g. using your distribution's package manager. The configuration shown here works with version `1.3.5` but is expected to work with may other versions. Make sure to have it enabled (chkconfig, systemd, ...) so that it starts automatically when the respective node comes up.
2. Create the following configuration file _/etc/keepalived/keepalived.conf_ on all master nodes:
```shell
! Configuration File for keepalived
global_defs {
router_id LVS_DEVEL
}
vrrp_script check_apiserver {
script "/etc/keepalived/check_apiserver.sh"
interval 3
weight -2
fall 10
rise 2
}
vrrp_instance VI_1 {
state <STATE>
interface <INTERFACE>
virtual_router_id 51
priority <PRIORITY>
authentication {
auth_type PASS
auth_pass 4be37dc3b4c90194d1600c483e10ad1d
}
virtual_ipaddress {
<VIRTUAL-IP>
}
track_script {
check_apiserver
}
}
```
In the section `vrrp_instance VI_1`, change few lines depending on your setup:
* `state` is either `MASTER` (on the first master nodes) or `BACKUP` (the other master nodes).
* `interface` is the name of an existing public interface to bind the virtual IP to (usually the primary interface).
* `priority` should be higher for the first master node, e.g. 101, and lower for the others, e.g. 100.
* `auth_pass` use any random string here.
* `virtual_ipaddresses` should contain the virtual IP for the master nodes.
3. Install the following health check script to _/etc/keepalived/check_apiserver.sh_ on all master nodes:
```shell
#!/bin/sh
errorExit() {
echo "*** $*" 1>&2
exit 1
}
curl --silent --max-time 2 --insecure https://localhost:6443/ -o /dev/null || errorExit "Error GET https://localhost:6443/"
if ip addr | grep -q <VIRTUAL-IP>; then
curl --silent --max-time 2 --insecure https://<VIRTUAL-IP>:6443/ -o /dev/null || errorExit "Error GET https://<VIRTUAL-IP>:6443/"
fi
```
Replace the `<VIRTUAL-IP>` by your chosen virtual IP.
4. Restart keepalived. While no Kubernetes services are up yet it will log health check fails on all master nodes. This will stop as soon as the first master node has been bootstrapped.
{% endcapture %}
{% assign tab_set_name = "lb_mode" %}
{% assign tab_names = "Choose one...,Cloud,On-Site" | split: ',' | compact %}
{% assign tab_contents = site.emptyArray | push: choose | push: cloud | push: onsite %}
{% include tabs.md %}
## Acquire etcd certs
@@ -403,53 +490,53 @@ Only follow this step if your etcd is hosted on dedicated nodes (**Option 1**).
1. Run the following:
```shell
mkdir -p /etc/kubernetes/pki/etcd
scp root@<etcd0-ip-address>:/etc/kubernetes/pki/etcd/ca.pem /etc/kubernetes/pki/etcd
scp root@<etcd0-ip-address>:/etc/kubernetes/pki/etcd/client.pem /etc/kubernetes/pki/etcd
scp root@<etcd0-ip-address>:/etc/kubernetes/pki/etcd/client-key.pem /etc/kubernetes/pki/etcd
```
```shell
mkdir -p /etc/kubernetes/pki/etcd
scp root@<etcd0-ip-address>:/etc/kubernetes/pki/etcd/ca.pem /etc/kubernetes/pki/etcd
scp root@<etcd0-ip-address>:/etc/kubernetes/pki/etcd/client.pem /etc/kubernetes/pki/etcd
scp root@<etcd0-ip-address>:/etc/kubernetes/pki/etcd/client-key.pem /etc/kubernetes/pki/etcd
```
## Run `kubeadm init` on `master0` {#kubeadm-init-master0}
1. In order for kubeadm to run, you first need to write a configuration file:
```shell
cat >config.yaml <<EOL
apiVersion: kubeadm.k8s.io/v1alpha1
kind: MasterConfiguration
api:
advertiseAddress: <private-ip>
etcd:
endpoints:
- https://<etcd0-ip-address>:2379
- https://<etcd1-ip-address>:2379
- https://<etcd2-ip-address>:2379
caFile: /etc/kubernetes/pki/etcd/ca.pem
certFile: /etc/kubernetes/pki/etcd/client.pem
keyFile: /etc/kubernetes/pki/etcd/client-key.pem
networking:
podSubnet: <podCIDR>
apiServerCertSANs:
- <load-balancer-ip>
apiServerExtraArgs:
apiserver-count: 3
EOL
```
```shell
cat >config.yaml <<EOF
apiVersion: kubeadm.k8s.io/v1alpha1
kind: MasterConfiguration
api:
advertiseAddress: <private-ip>
etcd:
endpoints:
- https://<etcd0-ip-address>:2379
- https://<etcd1-ip-address>:2379
- https://<etcd2-ip-address>:2379
caFile: /etc/kubernetes/pki/etcd/ca.pem
certFile: /etc/kubernetes/pki/etcd/client.pem
keyFile: /etc/kubernetes/pki/etcd/client-key.pem
networking:
podSubnet: <podCIDR>
apiServerCertSANs:
- <load-balancer-ip>
apiServerExtraArgs:
apiserver-count: "3"
EOF
```
Ensure that the following placeholders are replaced:
Ensure that the following placeholders are replaced:
- `<private-ip>` with the private IPv4 of the master server.
- `<etcd0-ip>`, `<etcd1-ip>` and `<etcd2-ip>` with the IP addresses of your three etcd nodes
- `<podCIDR>` with your Pod CIDR. Please read the [CNI network section](https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#pod-network) of the docs for more information. Some CNI providers do not require a value to be set.
- `<private-ip>` with the private IPv4 of the master server.
- `<etcd0-ip>`, `<etcd1-ip>` and `<etcd2-ip>` with the IP addresses of your three etcd nodes
- `<podCIDR>` with your Pod CIDR. Please read the [CNI network section](https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#pod-network) of the docs for more information. Some CNI providers do not require a value to be set.
**Note:** If you are using Kubernetes 1.9+, you can replace the `apiserver-count: 3` extra argument with `endpoint-reconciler-type=lease`. For more information, see [the documentation](https://kubernetes.io/docs/admin/high-availability/#endpoint-reconciler).
**Note:** If you are using Kubernetes 1.9+, you can replace the `apiserver-count: 3` extra argument with `endpoint-reconciler-type: lease`. For more information, see [the documentation](https://kubernetes.io/docs/admin/high-availability/#endpoint-reconciler).
1. When this is done, run kubeadm like so:
```shell
kubeadm init --config=config.yaml
```
```shell
kubeadm init --config=config.yaml
```
## Run `kubeadm init` on `master1` and `master2`
@@ -457,17 +544,17 @@ Before running kubeadm on the other masters, you need to first copy the K8s CA c
#### Option 1: Copy with scp
1. Follow the steps in the [create ssh access](#create-ssh-access) section, but instead of adding to `etcd0`'s `authorized_keys` file, add them to `master0`.
1. Follow the steps in the [create ssh access](#create-ssh-access) section, but instead of adding to `etcd0`'s `authorized_keys` file, add them to `master0`.
1. Once you've done this, run:
```shell
scp root@<master0-ip-address>:/etc/kubernetes/pki/* /etc/kubernetes/pki
rm apiserver.crt
```
```shell
scp root@<master0-ip-address>:/etc/kubernetes/pki/* /etc/kubernetes/pki
rm apiserver.crt
```
#### Option 2: Copy paste
1. Copy the contents of `/etc/kubernetes/pki/ca.crt` and `/etc/kubernetes/pki/ca.key` and create these files manually on `master1` and `master2`.
1. Copy the contents of `/etc/kubernetes/pki/ca.crt`, `/etc/kubernetes/pki/ca.key`, `/etc/kubernetes/pki/sa.key` and `/etc/kubernetes/pki/sa.pub` and create these files manually on `master1` and `master2`.
When this is done, you can follow the [previous step](#kubeadm-init-master0) to install the control plane with kubeadm.
@@ -489,20 +576,20 @@ Next provision and set up the worker nodes. To do this, you will need to provisi
1. Reconfigure kube-proxy to access kube-apiserver via the load balancer:
```shell
kubectl get configmap -n kube-system kube-proxy -o yaml > kube-proxy.yaml
sudo sed -i 's#server:.*#server: https://<masterLoadBalancerFQDN>:6443#g' kube-proxy.cm
kubectl apply -f kube-proxy.cm --force
# restart all kube-proxy pods to ensure that they load the new configmap
kubectl delete pod -n kube-system -l k8s-app=kube-proxy
```
```shell
kubectl get configmap -n kube-system kube-proxy -o yaml > kube-proxy-сm.yaml
sed -i 's#server:.*#server: https://<masterLoadBalancerFQDN>:6443#g' kube-proxy-cm.yaml
kubectl apply -f kube-proxy-cm.yaml --force
# restart all kube-proxy pods to ensure that they load the new configmap
kubectl delete pod -n kube-system -l k8s-app=kube-proxy
```
1. Reconfigure the kubelet to access kube-apiserver via the load balancer:
```shell
sudo sed -i 's#server:.*#server: https://<masterLoadBalancerFQDN>:6443#g' /etc/kubernetes/kubelet.conf
sudo systemctl restart kubelet
```
```shell
sudo sed -i 's#server:.*#server: https://<masterLoadBalancerFQDN>:6443#g' /etc/kubernetes/kubelet.conf
sudo systemctl restart kubelet
```
{% endcapture %}
docs/setup/independent/install-kubeadm.md
+31 -19
View File
@@ -4,7 +4,7 @@ title: Installing kubeadm
{% capture overview %}
<img src="https://raw.githubusercontent.com/cncf/artwork/master/kubernetes/certified-kubernetes/versionless/color/certified_kubernetes_color.png" align="right" width="150px">This page shows how to install the `kubeadm` toolbox.
<img src="https://raw.githubusercontent.com/cncf/artwork/master/kubernetes/certified-kubernetes/versionless/color/certified-kubernetes-color.png" align="right" width="150px">This page shows how to install the `kubeadm` toolbox.
For information how to create a cluster with kubeadm once you have performed this installation process,
see the [Using kubeadm to Create a Cluster](/docs/setup/independent/create-cluster-kubeadm/) page.
@@ -23,8 +23,8 @@ see the [Using kubeadm to Create a Cluster](/docs/setup/independent/create-clust
* 2 GB or more of RAM per machine (any less will leave little room for your apps)
* 2 CPUs or more
* Full network connectivity between all machines in the cluster (public or private network is fine)
* Unique hostname, MAC address, and product_uuid for every node
* Certain ports are open on your machines. See the section below for more details
* Unique hostname, MAC address, and product_uuid for every node. See [here](https://kubernetes.io/docs/setup/independent/install-kubeadm/#verify-the-mac-address-and-product_uuid-are-unique-for-every-node) for more details.
* Certain ports are open on your machines. See [here](/docs/setup/independent/install-kubeadm/#check-required-ports) for more details.
* Swap disabled. You **MUST** disable swap in order for the kubelet to work properly.
{% endcapture %}
@@ -39,7 +39,7 @@ see the [Using kubeadm to Create a Cluster](/docs/setup/independent/create-clust
It is very likely that hardware devices will have unique addresses, although some virtual machines may have
identical values. Kubernetes uses these values to uniquely identify the nodes in the cluster.
If these values are not unique to each node, the installation process
[may fail](https://github.com/kubernetes/kubeadm/issues/31).
may [fail](https://github.com/kubernetes/kubeadm/issues/31).
## Check network adapters
@@ -87,7 +87,8 @@ Versions 17.06+ _might work_, but have not yet been tested and verified by the K
Please proceed with executing the following commands based on your OS as root. You may become the root user by executing `sudo -i` after SSH-ing to each host.
You can use the following commands to install Docker on your system:
If you already have the required versions of the Docker installed, you can move on to next section.
If not, you can use the following commands to install Docker on your system:
{% capture docker_ubuntu %}
@@ -138,20 +139,6 @@ systemctl enable docker && systemctl start docker
{% endcapture %}
**Note**: Make sure that the cgroup driver used by kubelet is the same as the one used by
Docker. To ensure compatibility you can either update Docker, like so:
```bash
cat << EOF > /etc/docker/daemon.json
{
"exec-opts": ["native.cgroupdriver=systemd"]
}
EOF
```
and restart Docker. Or ensure the `--cgroup-driver` kubelet flag is set to the same value
as Docker (e.g. `cgroupfs`).
{% assign tab_set_name = "docker_install" %}
{% assign tab_names = "Ubuntu, Debian or HypriotOS;CentOS, RHEL or Fedora; Container Linux" | split: ';' | compact %}
{% assign tab_contents = site.emptyArray | push: docker_ubuntu | push: docker_centos | push: docker_coreos %}
@@ -273,6 +260,31 @@ systemctl enable kubelet && systemctl start kubelet
The kubelet is now restarting every few seconds, as it waits in a crashloop for
kubeadm to tell it what to do.
## Configure cgroup driver used by kubelet on Master Node
Make sure that the cgroup driver used by kubelet is the same as the one used by Docker. Verify that your Docker cgroup driver matches the kubelet config:
```bash
docker info | grep -i cgroup
cat /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
```
If the Docker cgroup driver and the kubelet config don't match, change the kubelet config to match the Docker cgroup driver. The
flag you need to change is `--cgroup-driver`. If it's already set, you can update like so:
```bash
sed -i "s/cgroup-driver=systemd/cgroup-driver=cgroupfs/g" /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
```
Otherwise, you will need to open the systemd file and add the flag to an existing environment line.
Then restart kubelet:
```bash
systemctl daemon-reload
systemctl restart kubelet
```
## Troubleshooting
If you are running into difficulties with kubeadm, please consult our [troubleshooting docs](/docs/setup/independent/troubleshooting-kubeadm/).
docs/setup/independent/troubleshooting-kubeadm.md
+44 -37
View File
@@ -23,7 +23,7 @@ If your cluster is in an error state, you may have trouble in the configuration
{% endcapture %}
#### `ebtables` or executable not found during installation
#### `ebtables` or some similar executable not found during installation
If you see the following warnings while running `kubeadm init`
@@ -58,11 +58,18 @@ This may be caused by a number of problems. The most common are:
There are two common ways to fix the cgroup driver problem:
1. Install docker again following intstructions
1. Install docker again following instructions
[here](/docs/setup/independent/install-kubeadm/#installing-docker).
1. Change the kubelet config to match the Docker cgroup driver manually, you can refer to
[Errors on CentOS when setting up masters](#errors-on-centos-when-setting-up-masters)
[Configure cgroup driver used by kubelet on Master Node](/docs/setup/independent/install-kubeadm/#configure-cgroup-driver-used-by-kubelet-on-master-node)
for detailed instructions.
The `kubectl describe pod` or `kubectl logs` commands can help you diagnose errors. For example:
```bash
kubectl -n ${NAMESPACE} describe pod ${POD_NAME}
kubectl -n ${NAMESPACE} logs ${POD_NAME} -c ${CONTAINER_NAME}
```
- control plane Docker containers are crashlooping or hanging. You can check this by running `docker ps` and investigating each container by running `docker logs`.
@@ -134,40 +141,6 @@ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
```
#### Errors on CentOS when setting up masters
If you are using CentOS and encounter difficulty while setting up the master node，
verify that your Docker cgroup driver matches the kubelet config:
```bash
docker info | grep -i cgroup
cat /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
```
If the Docker cgroup driver and the kubelet config don't match, change the kubelet config to match the Docker cgroup driver. The
flag you need to change is `--cgroup-driver`. If it's already set, you can update like so:
```bash
sed -i "s/cgroup-driver=systemd/cgroup-driver=cgroupfs/g /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
```
Otherwise, you will need to open the systemd file and add the flag to an existing environment line.
Then restart kubelet:
```bash
systemctl daemon-reload
systemctl restart kubelet
```
The `kubectl describe pod` or `kubectl logs` commands can help you diagnose errors. For example:
```bash
kubectl -n ${NAMESPACE} describe pod ${POD_NAME}
kubectl -n ${NAMESPACE} logs ${POD_NAME} -c ${CONTAINER_NAME}
```
### Default NIC When using flannel as the pod network in Vagrant
The following error might indicate that something was wrong in the pod network:
@@ -181,3 +154,37 @@ If you're using flannel as the pod network inside vagrant, then you will have to
Vagrant typically assigns two interfaces to all VMs. The first, for which all hosts are assigned the IP address `10.0.2.15`, is for external traffic that gets NATed.
This may lead to problems with flannel. By default, flannel selects the first interface on a host. This leads to all hosts thinking they have the same public IP address. To prevent this issue, pass the `--iface eth1` flag to flannel so that the second interface is chosen.
### Routing errors
In some situations `kubectl logs` and `kubectl run` commands may return with the following errors despite an otherwise apparently correctly working cluster:
```
Error from server: Get https://10.19.0.41:10250/containerLogs/default/mysql-ddc65b868-glc5m/mysql: dial tcp 10.19.0.41:10250: getsockopt: no route to host
```
This is due to Kubernetes using an IP that can not communicate with other IPs on the seemingly same subnet, possibly by policy of the machine provider. As an example, Digital Ocean assigns a public IP to `eth0` as well as a private one to be used internally as anchor for their floating IP feature, yet `kubelet` will pick the latter as the node's `InternalIP` instead of the public one.
Use `ip addr show` to check for this scenario instead of `ifconfig` because `ifconfig` will not display the offending alias IP address. Alternatively an API endpoint specific to Digital Ocean allows to query for the anchor IP from the droplet:
```
curl http://169.254.169.254/metadata/v1/interfaces/public/0/anchor_ipv4/address
```
The workaround is to tell `kubelet` which IP to use using `--node-ip`. When using Digital Ocean, it can be the public one (assigned to `eth0`) or the private one (assigned to `eth1`) should you want to use the optional private network. For example:
```
IFACE=eth0 # change to eth1 for DO's private network
DROPLET_IP_ADDRESS=$(ip addr show dev $IFACE | awk 'match($0,/inet (([0-9]|\.)+).* scope global/,a) { print a[1]; exit }')
echo $DROPLET_IP_ADDRESS # check this, just in case
echo "Environment=\"KUBELET_EXTRA_ARGS=--node-ip=$DROPLET_IP_ADDRESS\"" >> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
```
Please note that this assumes `KUBELET_EXTRA_ARGS` hasn't already been set in the unit file.
Then restart `kubelet`:
```
systemctl daemon-reload
systemctl restart kubelet
```
docs/setup/index.md
+1 -1
View File
@@ -1,5 +1,5 @@
---
approvers:
reviewers:
- brendandburns
- erictune
- mikedanese
docs/setup/pick-right-solution.md
+18 -23
View File
@@ -1,5 +1,5 @@
---
approvers:
reviewers:
- brendandburns
- erictune
- mikedanese
@@ -10,7 +10,7 @@ Kubernetes can run on various platforms: from your laptop, to VMs on a cloud pro
bare metal servers. The effort required to set up a cluster varies from running a single command to
crafting your own customized cluster. Use this guide to choose a solution that fits your needs.
If you just want to "kick the tires" on Kubernetes, use the [local Docker-based solution using MiniKube](#local-machine-solutions).
If you just want to "kick the tires" on Kubernetes, use the [local Docker-based solutions](#local-machine-solutions).
When you are ready to scale up to more machines and higher availability, a [hosted solution](#hosted-solutions) is the easiest to create and maintain.
@@ -33,7 +33,7 @@ a Kubernetes cluster from scratch.
* [Ubuntu on LXD](/docs/getting-started-guides/ubuntu/local/) supports a nine-instance deployment on localhost.
* [IBM Cloud Private-CE (Community Edition)](https://github.com/IBM/deploy-ibm-cloud-private) can use VirtualBox on your machine to deploy Kubernetes to one or more VMs for development and test scenarios. Scales to full multi-node cluster.
* [IBM Cloud Private-CE (Community Edition)](https://github.com/IBM/deploy-ibm-cloud-private) can use VirtualBox on your machine to deploy Kubernetes to one or more VMs for development and test scenarios. Scales to full multi-node cluster.
# Hosted Solutions
@@ -41,17 +41,15 @@ a Kubernetes cluster from scratch.
* [Amazon Elastic Container Service for Kubernetes](https://aws.amazon.com/eks/) offers managed Kubernetes service.
* [Azure Container Service](https://azure.microsoft.com/en-us/services/container-service/) can easily deploy Kubernetes clusters.
* [Azure Container Service](https://azure.microsoft.com/services/container-service/) offers managed Kubernetes clusters.
* [Stackpoint.io](https://stackpoint.io) provides Kubernetes infrastructure automation and management for multiple public clouds.
* [AppsCode.com](https://appscode.com/products/cloud-deployment/) provides managed Kubernetes clusters for various public clouds, including AWS and Google Cloud Platform.
* [KUBE2GO.io](https://kube2go.io) get started with highly available Kubernetes clusters on multiple public clouds along with useful tools for development, debugging, monitoring.
* [Madcore.Ai](https://madcore.ai) is devops-focused CLI tool for deploying Kubernetes infrastructure in AWS. Master, auto-scaling group nodes with spot-instances, ingress-ssl-lego, Heapster, and Grafana.
* [Platform9](https://platform9.com/products/kubernetes/) offers managed Kubernetes on-premises or on any public cloud, and provides 24/7 health monitoring and alerting.
* [Platform9](https://platform9.com/products/kubernetes/) offers managed Kubernetes on-premises or on any public cloud, and provides 24/7 health monitoring and alerting. (Kube2go, a web-UI driven Kubernetes cluster deployment service Platform9 released, has been integrated to Platform9 Sandbox.)
* [OpenShift Dedicated](https://www.openshift.com/dedicated/) offers managed Kubernetes clusters powered by OpenShift.
@@ -61,8 +59,10 @@ a Kubernetes cluster from scratch.
* [Giant Swarm](https://giantswarm.io/product/) offers managed Kubernetes clusters in their own datacenter, on-premises, or on public clouds.
* [Kubermatic](https://kubermatic.io) provides managed Kubernetes clusters for various public clouds, including AWS and Digital Ocean, as well as on-premises with OpenStack integration.
* [Kubermatic](https://www.loodse.com) provides managed Kubernetes clusters for various public clouds, including AWS and Digital Ocean, as well as on-premises with OpenStack integration.
* [Pivotal Container Service](https://pivotal.io/platform/pivotal-container-service) provides enterprise-grade Kubernetes for both on-premises and public clouds. PKS enables on-demand provisioning of Kubernetes clusters, multi-tenancy and fully automated day-2 operations.
`
# Turnkey Cloud Solutions
These solutions allow you to create Kubernetes clusters on a range of Cloud IaaS providers with only a
@@ -76,16 +76,15 @@ few commands. These solutions are actively developed and have active community s
* [CenturyLink Cloud](/docs/getting-started-guides/clc/)
* [IBM Cloud](https://github.com/patrocinio/kubernetes-softlayer)
* [Stackpoint.io](/docs/getting-started-guides/stackpoint/)
* [KUBE2GO.io](https://kube2go.io/)
* [Madcore.Ai](https://madcore.ai/)
* [Kubermatic](https://kubermatic.io/)
* [Kubermatic](https://cloud.kubermatic.io)
# On-Premises turnkey cloud solutions
These solutions allow you to create Kubernetes clusters on your internal, secure, cloud network with only a
few commands.
* [IBM Cloud Private](https://www.ibm.com/cloud-computing/products/ibm-cloud-private/)
* [Kubermatic](https://kubermatic.io/)
* [Kubermatic](https://www.loodse.com)
# Custom Solutions
@@ -118,19 +117,17 @@ These solutions are combinations of cloud providers and operating systems not co
* [Vagrant](/docs/getting-started-guides/coreos/) (uses CoreOS and flannel)
* [CloudStack](/docs/getting-started-guides/cloudstack/) (uses Ansible, CoreOS and flannel)
* [Vmware vSphere](/docs/getting-started-guides/vsphere/) (uses Debian)
* [Vmware vSphere, OpenStack, or Bare Metal](/docs/getting-started-guides/ubuntu/) (uses Juju, Ubuntu and flannel)
* [Vmware](/docs/getting-started-guides/coreos/) (uses CoreOS and flannel)
* [VMware vSphere](https://vmware.github.io/vsphere-storage-for-kubernetes/documentation/)
* [VMware vSphere, OpenStack, or Bare Metal](/docs/getting-started-guides/ubuntu/) (uses Juju, Ubuntu and flannel)
* [VMware](/docs/getting-started-guides/coreos/) (uses CoreOS and flannel)
* [oVirt](/docs/getting-started-guides/ovirt/)
* [Fedora (Multi Node)](/docs/getting-started-guides/fedora/flannel_multi_node_cluster/) (uses Fedora and flannel)
## Bare Metal
* [Offline](/docs/getting-started-guides/coreos/bare_metal_offline/) (no internet required. Uses CoreOS and Flannel)
* [Fedora via Ansible](/docs/getting-started-guides/fedora/fedora_ansible_config/)
* [Fedora (Single Node)](/docs/getting-started-guides/fedora/fedora_manual_config/)
* [Fedora (Multi Node)](/docs/getting-started-guides/fedora/flannel_multi_node_cluster/)
* [CentOS](/docs/getting-started-guides/centos/centos_manual_config/)
* [Kubernetes on Ubuntu](/docs/getting-started-guides/ubuntu/)
* [CoreOS on AWS or GCE](/docs/getting-started-guides/coreos/)
@@ -138,8 +135,6 @@ These solutions are combinations of cloud providers and operating systems not co
These solutions provide integration with third-party schedulers, resource managers, and/or lower level platforms.
* [Kubernetes on Mesos](/docs/getting-started-guides/mesos/)
* Instructions specify GCE, but are generic enough to be adapted to most existing Mesos clusters
* [DCOS](/docs/getting-started-guides/dcos/)
* Community Edition DCOS uses AWS
* Enterprise Edition DCOS supports cloud hosting, on-premises VMs, and bare metal
@@ -154,7 +149,6 @@ any | any | multi-support | any CNI | [docs](/docs/set
Google Kubernetes Engine | | | GCE | [docs](https://cloud.google.com/kubernetes-engine/docs/) | Commercial
Stackpoint.io | | multi-support | multi-support | [docs](https://stackpoint.io/) | Commercial
AppsCode.com | Saltstack | Debian | multi-support | [docs](https://appscode.com/products/cloud-deployment/) | Commercial
KUBE2GO.io | | multi-support | multi-support | [docs](https://kube2go.io) | Commercial
Madcore.Ai | Jenkins DSL | Ubuntu | flannel | [docs](https://madcore.ai) | Community ([@madcore-ai](https://github.com/madcore-ai))
Platform9 | | multi-support | multi-support | [docs](https://platform9.com/managed-kubernetes/) | Commercial
Kubermatic | | multi-support | multi-support | [docs](http://docs.kubermatic.io/) | Commercial
@@ -162,28 +156,29 @@ Giant Swarm | | CoreOS | flannel and/or Calico | [docs](http
GCE | Saltstack | Debian | GCE | [docs](/docs/getting-started-guides/gce/) | Project
Azure Container Service | | Ubuntu | Azure | [docs](https://azure.microsoft.com/en-us/services/container-service/) | Commercial
Azure (IaaS) | | Ubuntu | Azure | [docs](/docs/getting-started-guides/azure/) | [Community (Microsoft)](https://github.com/Azure/acs-engine)
Bare-metal | Ansible | Fedora | flannel | [docs](/docs/getting-started-guides/fedora/fedora_ansible_config/) | Project
Bare-metal | custom | Fedora | _none_ | [docs](/docs/getting-started-guides/fedora/fedora_manual_config/) | Project
Bare-metal | custom | Fedora | flannel | [docs](/docs/getting-started-guides/fedora/flannel_multi_node_cluster/) | Community ([@aveshagarwal](https://github.com/aveshagarwal))
libvirt | custom | Fedora | flannel | [docs](/docs/getting-started-guides/fedora/flannel_multi_node_cluster/) | Community ([@aveshagarwal](https://github.com/aveshagarwal))
KVM | custom | Fedora | flannel | [docs](/docs/getting-started-guides/fedora/flannel_multi_node_cluster/) | Community ([@aveshagarwal](https://github.com/aveshagarwal))
Mesos/Docker | custom | Ubuntu | Docker | [docs](/docs/getting-started-guides/mesos-docker/) | Community ([Kubernetes-Mesos Authors](https://github.com/mesosphere/kubernetes-mesos/blob/master/AUTHORS.md))
Mesos/GCE | | | | [docs](/docs/getting-started-guides/mesos/) | Community ([Kubernetes-Mesos Authors](https://github.com/mesosphere/kubernetes-mesos/blob/master/AUTHORS.md))
DCOS | Marathon | CoreOS/Alpine | custom | [docs](/docs/getting-started-guides/dcos/) | Community ([Kubernetes-Mesos Authors](https://github.com/mesosphere/kubernetes-mesos/blob/master/AUTHORS.md))
AWS | CoreOS | CoreOS | flannel | [docs](/docs/getting-started-guides/aws/) | Community
GCE | CoreOS | CoreOS | flannel | [docs](/docs/getting-started-guides/coreos/) | Community ([@pires](https://github.com/pires))
Vagrant | CoreOS | CoreOS | flannel | [docs](/docs/getting-started-guides/coreos/) | Community ([@pires](https://github.com/pires), [@AntonioMeireles](https://github.com/AntonioMeireles))
Bare-metal (Offline) | CoreOS | CoreOS | flannel | [docs](/docs/getting-started-guides/coreos/bare_metal_offline/) | Community ([@jeffbean](https://github.com/jeffbean))
CloudStack | Ansible | CoreOS | flannel | [docs](/docs/getting-started-guides/cloudstack/) | Community ([@sebgoa](https://github.com/sebgoa))
<<<<<<< HEAD
VMware vSphere | any | multi-support | multi-support | [docs](https://vmware.github.io/vsphere-storage-for-kubernetes/documentation/) | [Community](https://vmware.github.io/vsphere-storage-for-kubernetes/documentation/contactus.html)
=======
Vmware vSphere | Saltstack | Debian | OVS | [docs](/docs/getting-started-guides/vsphere/) | Community ([@imkin](https://github.com/imkin))
Bare-metal | custom | CentOS | flannel | [docs](/docs/getting-started-guides/centos/centos_manual_config/) | Community ([@coolsvap](https://github.com/coolsvap))
>>>>>>> bb8c59a640337f352e57a094ddac20bd6bbd4715
lxd | Juju | Ubuntu | flannel/canal | [docs](/docs/getting-started-guides/ubuntu/local/) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes)
AWS | Juju | Ubuntu | flannel/calico/canal | [docs](/docs/getting-started-guides/ubuntu/) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes)
Azure | Juju | Ubuntu | flannel/calico/canal | [docs](/docs/getting-started-guides/ubuntu/) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes)
GCE | Juju | Ubuntu | flannel/calico/canal | [docs](/docs/getting-started-guides/ubuntu/) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes)
Oracle Cloud | Juju | Ubuntu | flannel/calico/canal | [docs](/docs/getting-started-guides/ubuntu/) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes)
Rackspace | Juju | Ubuntu | flannel/calico/canal | [docs](/docs/getting-started-guides/ubuntu/) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes)
Vmware vSphere | Juju | Ubuntu | flannel/calico/canal | [docs](/docs/getting-started-guides/ubuntu/) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes)
VMware vSphere | Juju | Ubuntu | flannel/calico/canal | [docs](/docs/getting-started-guides/ubuntu/) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes)
Bare Metal | Juju | Ubuntu | flannel/calico/canal | [docs](/docs/getting-started-guides/ubuntu/) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes)
AWS | Saltstack | Debian | AWS | [docs](/docs/getting-started-guides/aws/) | Community ([@justinsb](https://github.com/justinsb))
AWS | kops | Debian | AWS | [docs](https://github.com/kubernetes/kops/) | Community ([@justinsb](https://github.com/justinsb))