Merge pull request #1144 from adieu/format
Some small style tweaks to make pandoc working
This commit is contained in:
devin-donnelly
@@ -1,182 +1,182 @@
|
||||
---
|
||||
assignees:
|
||||
- lavalamp
|
||||
- thockin
|
||||
|
||||
---
|
||||
|
||||
* TOC
|
||||
{:toc}
|
||||
|
||||
## Prerequisites
|
||||
|
||||
You need two machines with CentOS installed on them.
|
||||
|
||||
## Starting a cluster
|
||||
|
||||
This is a getting started guide for CentOS. It is a manual configuration so you understand all the underlying packages / services / ports, etc...
|
||||
|
||||
This guide will only get ONE node working. Multiple nodes requires a functional [networking configuration](/docs/admin/networking) done outside of kubernetes. Although the additional Kubernetes configuration requirements should be obvious.
|
||||
|
||||
The Kubernetes package provides a few services: kube-apiserver, kube-scheduler, kube-controller-manager, kubelet, kube-proxy. These services are managed by systemd and the configuration resides in a central location: /etc/kubernetes. We will break the services up between the hosts. The first host, centos-master, will be the Kubernetes master. This host will run the kube-apiserver, kube-controller-manager, and kube-scheduler. In addition, the master will also run _etcd_. The remaining host, centos-minion will be the node and run kubelet, proxy, cadvisor and docker.
|
||||
|
||||
**System Information:**
|
||||
|
||||
Hosts:
|
||||
|
||||
Please replace host IP with your environment.
|
||||
|
||||
```conf
|
||||
centos-master = 192.168.121.9
|
||||
centos-minion = 192.168.121.65
|
||||
```
|
||||
|
||||
**Prepare the hosts:**
|
||||
|
||||
* Create a /etc/yum.repos.d/virt7-docker-common-release.repo on all hosts - centos-{master,minion} with following information.
|
||||
|
||||
```conf
|
||||
[virt7-docker-common-release]
|
||||
name=virt7-docker-common-release
|
||||
baseurl=http://cbs.centos.org/repos/virt7-docker-common-release/x86_64/os/
|
||||
gpgcheck=0
|
||||
```
|
||||
|
||||
* Install Kubernetes and etcd on all hosts - centos-{master,minion}. This will also pull in docker and cadvisor.
|
||||
|
||||
```shell
|
||||
yum -y install --enablerepo=virt7-docker-common-release kubernetes etcd
|
||||
```
|
||||
|
||||
* Add master and node to /etc/hosts on all machines (not needed if hostnames already in DNS)
|
||||
|
||||
```shell
|
||||
echo "192.168.121.9 centos-master
|
||||
192.168.121.65 centos-minion" >> /etc/hosts
|
||||
```
|
||||
|
||||
* Edit /etc/kubernetes/config which will be the same on all hosts to contain:
|
||||
|
||||
```shell
|
||||
# Comma separated list of nodes in the etcd cluster
|
||||
KUBE_ETCD_SERVERS="--etcd-servers=http://centos-master:2379"
|
||||
|
||||
# logging to stderr means we get it in the systemd journal
|
||||
KUBE_LOGTOSTDERR="--logtostderr=true"
|
||||
|
||||
# journal message level, 0 is debug
|
||||
KUBE_LOG_LEVEL="--v=0"
|
||||
|
||||
# Should this cluster be allowed to run privileged docker containers
|
||||
KUBE_ALLOW_PRIV="--allow-privileged=false"
|
||||
|
||||
# How the replication controller and scheduler find the kube-apiserver
|
||||
KUBE_MASTER="--master=http://centos-master:8080"
|
||||
```
|
||||
|
||||
* Disable the firewall on both the master and node, as docker does not play well with other firewall rule managers
|
||||
|
||||
```shell
|
||||
systemctl disable iptables-services firewalld
|
||||
systemctl stop iptables-services firewalld
|
||||
```
|
||||
|
||||
**Configure the Kubernetes services on the master.**
|
||||
|
||||
* Edit /etc/etcd/etcd.conf to appear as such:
|
||||
|
||||
```shell
|
||||
# [member]
|
||||
ETCD_NAME=default
|
||||
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
|
||||
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379"
|
||||
|
||||
#[cluster]
|
||||
ETCD_ADVERTISE_CLIENT_URLS="http://0.0.0.0:2379"
|
||||
```
|
||||
|
||||
* Edit /etc/kubernetes/apiserver to appear as such:
|
||||
|
||||
```shell
|
||||
# The address on the local server to listen to.
|
||||
KUBE_API_ADDRESS="--address=0.0.0.0"
|
||||
|
||||
# The port on the local server to listen on.
|
||||
KUBE_API_PORT="--port=8080"
|
||||
|
||||
# Port kubelets listen on
|
||||
KUBELET_PORT="--kubelet-port=10250"
|
||||
|
||||
# Address range to use for services
|
||||
KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16"
|
||||
|
||||
# Add your own!
|
||||
KUBE_API_ARGS=""
|
||||
```
|
||||
|
||||
* Start the appropriate services on master:
|
||||
|
||||
```shell
|
||||
for SERVICES in etcd kube-apiserver kube-controller-manager kube-scheduler; do
|
||||
systemctl restart $SERVICES
|
||||
systemctl enable $SERVICES
|
||||
systemctl status $SERVICES
|
||||
done
|
||||
```
|
||||
|
||||
**Configure the Kubernetes services on the node.**
|
||||
|
||||
***We need to configure the kubelet and start the kubelet and proxy***
|
||||
|
||||
* Edit /etc/kubernetes/kubelet to appear as such:
|
||||
|
||||
```shell
|
||||
# The address for the info server to serve on
|
||||
KUBELET_ADDRESS="--address=0.0.0.0"
|
||||
|
||||
# The port for the info server to serve on
|
||||
KUBELET_PORT="--port=10250"
|
||||
|
||||
# You may leave this blank to use the actual hostname
|
||||
KUBELET_HOSTNAME="--hostname-override=centos-minion"
|
||||
|
||||
# Location of the api-server
|
||||
KUBELET_API_SERVER="--api-servers=http://centos-master:8080"
|
||||
|
||||
# Add your own!
|
||||
KUBELET_ARGS=""
|
||||
```
|
||||
|
||||
* Start the appropriate services on node (centos-minion).
|
||||
|
||||
```shell
|
||||
for SERVICES in kube-proxy kubelet docker; do
|
||||
systemctl restart $SERVICES
|
||||
systemctl enable $SERVICES
|
||||
systemctl status $SERVICES
|
||||
done
|
||||
```
|
||||
|
||||
*You should be finished!*
|
||||
|
||||
* Check to make sure the cluster can see the node (on centos-master)
|
||||
|
||||
```shell
|
||||
$ kubectl get nodes
|
||||
NAME LABELS STATUS
|
||||
centos-minion <none> Ready
|
||||
```
|
||||
|
||||
**The cluster should be running! Launch a test pod.**
|
||||
|
||||
You should have a functional cluster, check out [101](/docs/user-guide/walkthrough/)!
|
||||
|
||||
## Support Level
|
||||
|
||||
|
||||
IaaS Provider | Config. Mgmt | OS | Networking | Docs | Conforms | Support Level
|
||||
-------------------- | ------------ | ------ | ---------- | --------------------------------------------- | ---------| ----------------------------
|
||||
Bare-metal | custom | CentOS | _none_ | [docs](/docs/getting-started-guides/centos/centos_manual_config) | | Community ([@coolsvap](https://github.com/coolsvap))
|
||||
|
||||
For support level information on all solutions, see the [Table of solutions](/docs/getting-started-guides/#table-of-solutions) chart.
|
||||
|
||||
---
|
||||
assignees:
|
||||
- lavalamp
|
||||
- thockin
|
||||
|
||||
---
|
||||
|
||||
* TOC
|
||||
{:toc}
|
||||
|
||||
## Prerequisites
|
||||
|
||||
You need two machines with CentOS installed on them.
|
||||
|
||||
## Starting a cluster
|
||||
|
||||
This is a getting started guide for CentOS. It is a manual configuration so you understand all the underlying packages / services / ports, etc...
|
||||
|
||||
This guide will only get ONE node working. Multiple nodes requires a functional [networking configuration](/docs/admin/networking) done outside of kubernetes. Although the additional Kubernetes configuration requirements should be obvious.
|
||||
|
||||
The Kubernetes package provides a few services: kube-apiserver, kube-scheduler, kube-controller-manager, kubelet, kube-proxy. These services are managed by systemd and the configuration resides in a central location: /etc/kubernetes. We will break the services up between the hosts. The first host, centos-master, will be the Kubernetes master. This host will run the kube-apiserver, kube-controller-manager, and kube-scheduler. In addition, the master will also run _etcd_. The remaining host, centos-minion will be the node and run kubelet, proxy, cadvisor and docker.
|
||||
|
||||
**System Information:**
|
||||
|
||||
Hosts:
|
||||
|
||||
Please replace host IP with your environment.
|
||||
|
||||
```conf
|
||||
centos-master = 192.168.121.9
|
||||
centos-minion = 192.168.121.65
|
||||
```
|
||||
|
||||
**Prepare the hosts:**
|
||||
|
||||
* Create a /etc/yum.repos.d/virt7-docker-common-release.repo on all hosts - centos-{master,minion} with following information.
|
||||
|
||||
```conf
|
||||
[virt7-docker-common-release]
|
||||
name=virt7-docker-common-release
|
||||
baseurl=http://cbs.centos.org/repos/virt7-docker-common-release/x86_64/os/
|
||||
gpgcheck=0
|
||||
```
|
||||
|
||||
* Install Kubernetes and etcd on all hosts - centos-{master,minion}. This will also pull in docker and cadvisor.
|
||||
|
||||
```shell
|
||||
yum -y install --enablerepo=virt7-docker-common-release kubernetes etcd
|
||||
```
|
||||
|
||||
* Add master and node to /etc/hosts on all machines (not needed if hostnames already in DNS)
|
||||
|
||||
```shell
|
||||
echo "192.168.121.9 centos-master
|
||||
192.168.121.65 centos-minion" >> /etc/hosts
|
||||
```
|
||||
|
||||
* Edit /etc/kubernetes/config which will be the same on all hosts to contain:
|
||||
|
||||
```shell
|
||||
# Comma separated list of nodes in the etcd cluster
|
||||
KUBE_ETCD_SERVERS="--etcd-servers=http://centos-master:2379"
|
||||
|
||||
# logging to stderr means we get it in the systemd journal
|
||||
KUBE_LOGTOSTDERR="--logtostderr=true"
|
||||
|
||||
# journal message level, 0 is debug
|
||||
KUBE_LOG_LEVEL="--v=0"
|
||||
|
||||
# Should this cluster be allowed to run privileged docker containers
|
||||
KUBE_ALLOW_PRIV="--allow-privileged=false"
|
||||
|
||||
# How the replication controller and scheduler find the kube-apiserver
|
||||
KUBE_MASTER="--master=http://centos-master:8080"
|
||||
```
|
||||
|
||||
* Disable the firewall on both the master and node, as docker does not play well with other firewall rule managers
|
||||
|
||||
```shell
|
||||
systemctl disable iptables-services firewalld
|
||||
systemctl stop iptables-services firewalld
|
||||
```
|
||||
|
||||
**Configure the Kubernetes services on the master.**
|
||||
|
||||
* Edit /etc/etcd/etcd.conf to appear as such:
|
||||
|
||||
```shell
|
||||
# [member]
|
||||
ETCD_NAME=default
|
||||
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
|
||||
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379"
|
||||
|
||||
#[cluster]
|
||||
ETCD_ADVERTISE_CLIENT_URLS="http://0.0.0.0:2379"
|
||||
```
|
||||
|
||||
* Edit /etc/kubernetes/apiserver to appear as such:
|
||||
|
||||
```shell
|
||||
# The address on the local server to listen to.
|
||||
KUBE_API_ADDRESS="--address=0.0.0.0"
|
||||
|
||||
# The port on the local server to listen on.
|
||||
KUBE_API_PORT="--port=8080"
|
||||
|
||||
# Port kubelets listen on
|
||||
KUBELET_PORT="--kubelet-port=10250"
|
||||
|
||||
# Address range to use for services
|
||||
KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16"
|
||||
|
||||
# Add your own!
|
||||
KUBE_API_ARGS=""
|
||||
```
|
||||
|
||||
* Start the appropriate services on master:
|
||||
|
||||
```shell
|
||||
for SERVICES in etcd kube-apiserver kube-controller-manager kube-scheduler; do
|
||||
systemctl restart $SERVICES
|
||||
systemctl enable $SERVICES
|
||||
systemctl status $SERVICES
|
||||
done
|
||||
```
|
||||
|
||||
**Configure the Kubernetes services on the node.**
|
||||
|
||||
***We need to configure the kubelet and start the kubelet and proxy***
|
||||
|
||||
* Edit /etc/kubernetes/kubelet to appear as such:
|
||||
|
||||
```shell
|
||||
# The address for the info server to serve on
|
||||
KUBELET_ADDRESS="--address=0.0.0.0"
|
||||
|
||||
# The port for the info server to serve on
|
||||
KUBELET_PORT="--port=10250"
|
||||
|
||||
# You may leave this blank to use the actual hostname
|
||||
KUBELET_HOSTNAME="--hostname-override=centos-minion"
|
||||
|
||||
# Location of the api-server
|
||||
KUBELET_API_SERVER="--api-servers=http://centos-master:8080"
|
||||
|
||||
# Add your own!
|
||||
KUBELET_ARGS=""
|
||||
```
|
||||
|
||||
* Start the appropriate services on node (centos-minion).
|
||||
|
||||
```shell
|
||||
for SERVICES in kube-proxy kubelet docker; do
|
||||
systemctl restart $SERVICES
|
||||
systemctl enable $SERVICES
|
||||
systemctl status $SERVICES
|
||||
done
|
||||
```
|
||||
|
||||
*You should be finished!*
|
||||
|
||||
* Check to make sure the cluster can see the node (on centos-master)
|
||||
|
||||
```shell
|
||||
$ kubectl get nodes
|
||||
NAME LABELS STATUS
|
||||
centos-minion <none> Ready
|
||||
```
|
||||
|
||||
**The cluster should be running! Launch a test pod.**
|
||||
|
||||
You should have a functional cluster, check out [101](/docs/user-guide/walkthrough/)!
|
||||
|
||||
## Support Level
|
||||
|
||||
|
||||
IaaS Provider | Config. Mgmt | OS | Networking | Docs | Conforms | Support Level
|
||||
-------------------- | ------------ | ------ | ---------- | --------------------------------------------- | ---------| ----------------------------
|
||||
Bare-metal | custom | CentOS | _none_ | [docs](/docs/getting-started-guides/centos/centos_manual_config) | | Community ([@coolsvap](https://github.com/coolsvap))
|
||||
|
||||
For support level information on all solutions, see the [Table of solutions](/docs/getting-started-guides/#table-of-solutions) chart.
|
||||
|
||||
|
||||
@@ -1,209 +1,209 @@
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
This document describes how to deploy Kubernetes with Calico networking on _bare metal_ CoreOS. For more information on Project Calico, visit [projectcalico.org](http://projectcalico.org) and the [calico-containers repository](https://github.com/projectcalico/calico-containers).
|
||||
|
||||
To install Calico on an existing Kubernetes cluster, or for more information on deploying Calico with Kubernetes in a number of other environments take a look at our supported [deployment guides](https://github.com/projectcalico/calico-containers/tree/master/docs/cni/kubernetes).
|
||||
|
||||
Specifically, this guide will have you do the following:
|
||||
|
||||
- Deploy a Kubernetes master node on CoreOS using cloud-config.
|
||||
- Deploy two Kubernetes compute nodes with Calico Networking using cloud-config.
|
||||
- Configure `kubectl` to access your cluster.
|
||||
|
||||
The resulting cluster will use SSL between Kubernetes components. It will run the SkyDNS service and kube-ui, and be fully conformant with the Kubernetes v1.1 conformance tests.
|
||||
|
||||
## Prerequisites and Assumptions
|
||||
|
||||
- At least three bare-metal machines (or VMs) to work with. This guide will configure them as follows:
|
||||
- 1 Kubernetes Master
|
||||
- 2 Kubernetes Nodes
|
||||
- Your nodes should have IP connectivity to each other and the internet.
|
||||
- This guide assumes a DHCP server on your network to assign server IPs.
|
||||
- This guide uses `192.168.0.0/16` as the subnet from which pod IP addresses are assigned. If this overlaps with your host subnet, you will need to configure Calico to use a different [IP pool](https://github.com/projectcalico/calico-containers/blob/master/docs/calicoctl/pool.md#calicoctl-pool-commands).
|
||||
|
||||
## Cloud-config
|
||||
|
||||
This guide will use [cloud-config](https://coreos.com/docs/cluster-management/setup/cloudinit-cloud-config/) to configure each of the nodes in our Kubernetes cluster.
|
||||
|
||||
We'll use two cloud-config files:
|
||||
- `master-config.yaml`: cloud-config for the Kubernetes master
|
||||
- `node-config.yaml`: cloud-config for each Kubernetes node
|
||||
|
||||
## Download CoreOS
|
||||
|
||||
Download the stable CoreOS bootable ISO from the [CoreOS website](https://coreos.com/docs/running-coreos/platforms/iso/).
|
||||
|
||||
## Configure the Kubernetes Master
|
||||
|
||||
1. Once you've downloaded the ISO image, burn the ISO to a CD/DVD/USB key and boot from it (if using a virtual machine you can boot directly from the ISO). Once booted, you should be automatically logged in as the `core` user at the terminal. At this point CoreOS is running from the ISO and it hasn't been installed yet.
|
||||
|
||||
2. *On another machine*, download the [master cloud-config template](https://raw.githubusercontent.com/projectcalico/calico-cni/k8s-1.1-docs/samples/kubernetes/cloud-config/master-config-template.yaml) and save it as `master-config.yaml`.
|
||||
|
||||
3. Replace the following variables in the `master-config.yaml` file.
|
||||
|
||||
- `<SSH_PUBLIC_KEY>`: The public key you will use for SSH access to this server. See [generating ssh keys](https://help.github.com/articles/generating-ssh-keys/)
|
||||
|
||||
4. Copy the edited `master-config.yaml` to your Kubernetes master machine (using a USB stick, for example).
|
||||
|
||||
5. The CoreOS bootable ISO comes with a tool called `coreos-install` which will allow us to install CoreOS and configure the machine using a cloud-config file. The following command will download and install stable CoreOS using the `master-config.yaml` file we just created for configuration. Run this on the Kubernetes master.
|
||||
|
||||
> **Warning:** this is a destructive operation that erases disk `sda` on your server.
|
||||
|
||||
```shell
|
||||
sudo coreos-install -d /dev/sda -C stable -c master-config.yaml
|
||||
```
|
||||
|
||||
6. Once complete, restart the server and boot from `/dev/sda` (you may need to remove the ISO image). When it comes back up, you should have SSH access as the `core` user using the public key provided in the `master-config.yaml` file.
|
||||
|
||||
### Configure TLS
|
||||
|
||||
The master requires the CA certificate, `ca.pem`; its own certificate, `apiserver.pem` and its private key, `apiserver-key.pem`. This [CoreOS guide](https://coreos.com/kubernetes/docs/latest/openssl.html) explains how to generate these.
|
||||
|
||||
1. Generate the necessary certificates for the master. This [guide for generating Kubernetes TLS Assets](https://coreos.com/kubernetes/docs/latest/openssl.html) explains how to use OpenSSL to generate the required assets.
|
||||
|
||||
2. Send the three files to your master host (using `scp` for example).
|
||||
|
||||
3. Move them to the `/etc/kubernetes/ssl` folder and ensure that only the root user can read the key:
|
||||
|
||||
```shell
|
||||
# Move keys
|
||||
sudo mkdir -p /etc/kubernetes/ssl/
|
||||
sudo mv -t /etc/kubernetes/ssl/ ca.pem apiserver.pem apiserver-key.pem
|
||||
|
||||
# Set Permissions
|
||||
sudo chmod 600 /etc/kubernetes/ssl/apiserver-key.pem
|
||||
sudo chown root:root /etc/kubernetes/ssl/apiserver-key.pem
|
||||
```
|
||||
|
||||
4. Restart the kubelet to pick up the changes:
|
||||
|
||||
```shell
|
||||
sudo systemctl restart kubelet
|
||||
```
|
||||
|
||||
## Configure the compute nodes
|
||||
|
||||
The following steps will set up a single Kubernetes node for use as a compute host. Run these steps to deploy each Kubernetes node in your cluster.
|
||||
|
||||
1. Boot up the node machine using the bootable ISO we downloaded earlier. You should be automatically logged in as the `core` user.
|
||||
|
||||
2. Make a copy of the [node cloud-config template](https://raw.githubusercontent.com/projectcalico/calico-cni/k8s-1.1-docs/samples/kubernetes/cloud-config/node-config-template.yaml) for this machine.
|
||||
|
||||
3. Replace the following placeholders in the `node-config.yaml` file to match your deployment.
|
||||
|
||||
- `<HOSTNAME>`: Hostname for this node (e.g. kube-node1, kube-node2)
|
||||
- `<SSH_PUBLIC_KEY>`: The public key you will use for SSH access to this server.
|
||||
- `<KUBERNETES_MASTER>`: The IPv4 address of the Kubernetes master.
|
||||
|
||||
4. Replace the following placeholders with the contents of their respective files.
|
||||
|
||||
- `<CA_CERT>`: Complete contents of `ca.pem`
|
||||
- `<CA_KEY_CERT>`: Complete contents of `ca-key.pem`
|
||||
|
||||
> **Important:** in a production deployment, embedding the secret key in cloud-config is a bad idea! In production you should use an appropriate secret manager.
|
||||
|
||||
> **Important:** Make sure you indent the entire file to match the indentation of the placeholder. For example:
|
||||
>
|
||||
> ```shell
|
||||
> - path: /etc/kubernetes/ssl/ca.pem
|
||||
> owner: core
|
||||
> permissions: 0644
|
||||
> content: |
|
||||
> <CA_CERT>
|
||||
> ```
|
||||
>
|
||||
> should look like this once the certificate is in place:
|
||||
>
|
||||
> ```shell
|
||||
> - path: /etc/kubernetes/ssl/ca.pem
|
||||
> owner: core
|
||||
> permissions: 0644
|
||||
> content: |
|
||||
> -----BEGIN CERTIFICATE-----
|
||||
> MIIC9zCCAd+gAwIBAgIJAJMnVnhVhy5pMA0GCSqGSIb3DQEBCwUAMBIxEDAOBgNV
|
||||
> ...<snip>...
|
||||
> QHwi1rNc8eBLNrd4BM/A1ZeDVh/Q9KxN+ZG/hHIXhmWKgN5wQx6/81FIFg==
|
||||
> -----END CERTIFICATE-----
|
||||
> ```
|
||||
|
||||
5. Move the modified `node-config.yaml` to your Kubernetes node machine and install and configure CoreOS on the node using the following command.
|
||||
|
||||
> **Warning:** this is a destructive operation that erases disk `sda` on your server.
|
||||
|
||||
```shell
|
||||
sudo coreos-install -d /dev/sda -C stable -c node-config.yaml
|
||||
```
|
||||
|
||||
6. Once complete, restart the server and boot into `/dev/sda`. When it comes back up, you should have SSH access as the `core` user using the public key provided in the `node-config.yaml` file. It will take some time for the node to be fully configured.
|
||||
|
||||
## Configure Kubeconfig
|
||||
|
||||
To administer your cluster from a separate host, you will need the client and admin certificates generated earlier (`ca.pem`, `admin.pem`, `admin-key.pem`). With certificates in place, run the following commands with the appropriate filepaths.
|
||||
|
||||
```shell
|
||||
kubectl config set-cluster calico-cluster --server=https://<KUBERNETES_MASTER> --certificate-authority=<CA_CERT_PATH>
|
||||
kubectl config set-credentials calico-admin --certificate-authority=<CA_CERT_PATH> --client-key=<ADMIN_KEY_PATH> --client-certificate=<ADMIN_CERT_PATH>
|
||||
kubectl config set-context calico --cluster=calico-cluster --user=calico-admin
|
||||
kubectl config use-context calico
|
||||
```
|
||||
|
||||
Check your work with `kubectl get nodes`.
|
||||
|
||||
## Install the DNS Addon
|
||||
|
||||
Most Kubernetes deployments will require the DNS addon for service discovery. To install DNS, create the skydns service and replication controller provided.
|
||||
|
||||
```shell
|
||||
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico-cni/k8s-1.1-docs/samples/kubernetes/master/dns/skydns.yaml
|
||||
```
|
||||
|
||||
## Install the Kubernetes UI Addon (Optional)
|
||||
|
||||
The Kubernetes UI can be installed using `kubectl` to run the following manifest file.
|
||||
|
||||
```shell
|
||||
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico-cni/k8s-1.1-docs/samples/kubernetes/master/kube-ui/kube-ui.yaml
|
||||
```
|
||||
|
||||
## Launch other Services With Calico-Kubernetes
|
||||
|
||||
At this point, you have a fully functioning cluster running on Kubernetes with a master and two nodes networked with Calico. You can now follow any of the [standard documentation](https://github.com/kubernetes/kubernetes/tree/{{page.version}}.0/examples/) to set up other services on your cluster.
|
||||
|
||||
## Connectivity to outside the cluster
|
||||
|
||||
Because containers in this guide have private `192.168.0.0/16` IPs, you will need NAT to allow connectivity between containers and the internet. However, in a production data center deployment, NAT is not always necessary, since Calico can peer with the data center's border routers over BGP.
|
||||
|
||||
### NAT on the nodes
|
||||
|
||||
The simplest method for enabling connectivity from containers to the internet is to use outgoing NAT on your Kubernetes nodes.
|
||||
|
||||
Calico can provide outgoing NAT for containers. To enable it, use the following `calicoctl` command:
|
||||
|
||||
```shell
|
||||
ETCD_AUTHORITY=<master_ip:6666> calicoctl pool add <CONTAINER_SUBNET> --nat-outgoing
|
||||
```
|
||||
|
||||
By default, `<CONTAINER_SUBNET>` will be `192.168.0.0/16`. You can find out which pools have been configured with the following command:
|
||||
|
||||
```shell
|
||||
ETCD_AUTHORITY=<master_ip:6666> calicoctl pool show
|
||||
```
|
||||
|
||||
### NAT at the border router
|
||||
|
||||
In a data center environment, it is recommended to configure Calico to peer with the border routers over BGP. This means that the container IPs will be routable anywhere in the data center, and so NAT is not needed on the nodes (though it may be enabled at the data center edge to allow outbound-only internet connectivity).
|
||||
|
||||
The Calico documentation contains more information on how to configure Calico to [peer with existing infrastructure](https://github.com/projectcalico/calico-containers/blob/master/docs/ExternalConnectivity.md).
|
||||
|
||||
## Support Level
|
||||
|
||||
|
||||
IaaS Provider | Config. Mgmt | OS | Networking | Docs | Conforms | Support Level
|
||||
-------------------- | ------------ | ------ | ---------- | --------------------------------------------- | ---------| ----------------------------
|
||||
Bare-metal | CoreOS | CoreOS | Calico | [docs](/docs/getting-started-guides/coreos/bare_metal_calico) | | Community ([@caseydavenport](https://github.com/caseydavenport))
|
||||
|
||||
|
||||
For support level information on all solutions, see the [Table of solutions](/docs/getting-started-guides/#table-of-solutions) chart.
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
This document describes how to deploy Kubernetes with Calico networking on _bare metal_ CoreOS. For more information on Project Calico, visit [projectcalico.org](http://projectcalico.org) and the [calico-containers repository](https://github.com/projectcalico/calico-containers).
|
||||
|
||||
To install Calico on an existing Kubernetes cluster, or for more information on deploying Calico with Kubernetes in a number of other environments take a look at our supported [deployment guides](https://github.com/projectcalico/calico-containers/tree/master/docs/cni/kubernetes).
|
||||
|
||||
Specifically, this guide will have you do the following:
|
||||
|
||||
- Deploy a Kubernetes master node on CoreOS using cloud-config.
|
||||
- Deploy two Kubernetes compute nodes with Calico Networking using cloud-config.
|
||||
- Configure `kubectl` to access your cluster.
|
||||
|
||||
The resulting cluster will use SSL between Kubernetes components. It will run the SkyDNS service and kube-ui, and be fully conformant with the Kubernetes v1.1 conformance tests.
|
||||
|
||||
## Prerequisites and Assumptions
|
||||
|
||||
- At least three bare-metal machines (or VMs) to work with. This guide will configure them as follows:
|
||||
- 1 Kubernetes Master
|
||||
- 2 Kubernetes Nodes
|
||||
- Your nodes should have IP connectivity to each other and the internet.
|
||||
- This guide assumes a DHCP server on your network to assign server IPs.
|
||||
- This guide uses `192.168.0.0/16` as the subnet from which pod IP addresses are assigned. If this overlaps with your host subnet, you will need to configure Calico to use a different [IP pool](https://github.com/projectcalico/calico-containers/blob/master/docs/calicoctl/pool.md#calicoctl-pool-commands).
|
||||
|
||||
## Cloud-config
|
||||
|
||||
This guide will use [cloud-config](https://coreos.com/docs/cluster-management/setup/cloudinit-cloud-config/) to configure each of the nodes in our Kubernetes cluster.
|
||||
|
||||
We'll use two cloud-config files:
|
||||
- `master-config.yaml`: cloud-config for the Kubernetes master
|
||||
- `node-config.yaml`: cloud-config for each Kubernetes node
|
||||
|
||||
## Download CoreOS
|
||||
|
||||
Download the stable CoreOS bootable ISO from the [CoreOS website](https://coreos.com/docs/running-coreos/platforms/iso/).
|
||||
|
||||
## Configure the Kubernetes Master
|
||||
|
||||
1. Once you've downloaded the ISO image, burn the ISO to a CD/DVD/USB key and boot from it (if using a virtual machine you can boot directly from the ISO). Once booted, you should be automatically logged in as the `core` user at the terminal. At this point CoreOS is running from the ISO and it hasn't been installed yet.
|
||||
|
||||
2. *On another machine*, download the [master cloud-config template](https://raw.githubusercontent.com/projectcalico/calico-cni/k8s-1.1-docs/samples/kubernetes/cloud-config/master-config-template.yaml) and save it as `master-config.yaml`.
|
||||
|
||||
3. Replace the following variables in the `master-config.yaml` file.
|
||||
|
||||
- `<SSH_PUBLIC_KEY>`: The public key you will use for SSH access to this server. See [generating ssh keys](https://help.github.com/articles/generating-ssh-keys/)
|
||||
|
||||
4. Copy the edited `master-config.yaml` to your Kubernetes master machine (using a USB stick, for example).
|
||||
|
||||
5. The CoreOS bootable ISO comes with a tool called `coreos-install` which will allow us to install CoreOS and configure the machine using a cloud-config file. The following command will download and install stable CoreOS using the `master-config.yaml` file we just created for configuration. Run this on the Kubernetes master.
|
||||
|
||||
> **Warning:** this is a destructive operation that erases disk `sda` on your server.
|
||||
|
||||
```shell
|
||||
sudo coreos-install -d /dev/sda -C stable -c master-config.yaml
|
||||
```
|
||||
|
||||
6. Once complete, restart the server and boot from `/dev/sda` (you may need to remove the ISO image). When it comes back up, you should have SSH access as the `core` user using the public key provided in the `master-config.yaml` file.
|
||||
|
||||
### Configure TLS
|
||||
|
||||
The master requires the CA certificate, `ca.pem`; its own certificate, `apiserver.pem` and its private key, `apiserver-key.pem`. This [CoreOS guide](https://coreos.com/kubernetes/docs/latest/openssl.html) explains how to generate these.
|
||||
|
||||
1. Generate the necessary certificates for the master. This [guide for generating Kubernetes TLS Assets](https://coreos.com/kubernetes/docs/latest/openssl.html) explains how to use OpenSSL to generate the required assets.
|
||||
|
||||
2. Send the three files to your master host (using `scp` for example).
|
||||
|
||||
3. Move them to the `/etc/kubernetes/ssl` folder and ensure that only the root user can read the key:
|
||||
|
||||
```shell
|
||||
# Move keys
|
||||
sudo mkdir -p /etc/kubernetes/ssl/
|
||||
sudo mv -t /etc/kubernetes/ssl/ ca.pem apiserver.pem apiserver-key.pem
|
||||
|
||||
# Set Permissions
|
||||
sudo chmod 600 /etc/kubernetes/ssl/apiserver-key.pem
|
||||
sudo chown root:root /etc/kubernetes/ssl/apiserver-key.pem
|
||||
```
|
||||
|
||||
4. Restart the kubelet to pick up the changes:
|
||||
|
||||
```shell
|
||||
sudo systemctl restart kubelet
|
||||
```
|
||||
|
||||
## Configure the compute nodes
|
||||
|
||||
The following steps will set up a single Kubernetes node for use as a compute host. Run these steps to deploy each Kubernetes node in your cluster.
|
||||
|
||||
1. Boot up the node machine using the bootable ISO we downloaded earlier. You should be automatically logged in as the `core` user.
|
||||
|
||||
2. Make a copy of the [node cloud-config template](https://raw.githubusercontent.com/projectcalico/calico-cni/k8s-1.1-docs/samples/kubernetes/cloud-config/node-config-template.yaml) for this machine.
|
||||
|
||||
3. Replace the following placeholders in the `node-config.yaml` file to match your deployment.
|
||||
|
||||
- `<HOSTNAME>`: Hostname for this node (e.g. kube-node1, kube-node2)
|
||||
- `<SSH_PUBLIC_KEY>`: The public key you will use for SSH access to this server.
|
||||
- `<KUBERNETES_MASTER>`: The IPv4 address of the Kubernetes master.
|
||||
|
||||
4. Replace the following placeholders with the contents of their respective files.
|
||||
|
||||
- `<CA_CERT>`: Complete contents of `ca.pem`
|
||||
- `<CA_KEY_CERT>`: Complete contents of `ca-key.pem`
|
||||
|
||||
> **Important:** in a production deployment, embedding the secret key in cloud-config is a bad idea! In production you should use an appropriate secret manager.
|
||||
|
||||
> **Important:** Make sure you indent the entire file to match the indentation of the placeholder. For example:
|
||||
>
|
||||
> ```shell
|
||||
> - path: /etc/kubernetes/ssl/ca.pem
|
||||
> owner: core
|
||||
> permissions: 0644
|
||||
> content: |
|
||||
> <CA_CERT>
|
||||
> ```
|
||||
>
|
||||
> should look like this once the certificate is in place:
|
||||
>
|
||||
> ```shell
|
||||
> - path: /etc/kubernetes/ssl/ca.pem
|
||||
> owner: core
|
||||
> permissions: 0644
|
||||
> content: |
|
||||
> -----BEGIN CERTIFICATE-----
|
||||
> MIIC9zCCAd+gAwIBAgIJAJMnVnhVhy5pMA0GCSqGSIb3DQEBCwUAMBIxEDAOBgNV
|
||||
> ...<snip>...
|
||||
> QHwi1rNc8eBLNrd4BM/A1ZeDVh/Q9KxN+ZG/hHIXhmWKgN5wQx6/81FIFg==
|
||||
> -----END CERTIFICATE-----
|
||||
> ```
|
||||
|
||||
5. Move the modified `node-config.yaml` to your Kubernetes node machine and install and configure CoreOS on the node using the following command.
|
||||
|
||||
> **Warning:** this is a destructive operation that erases disk `sda` on your server.
|
||||
|
||||
```shell
|
||||
sudo coreos-install -d /dev/sda -C stable -c node-config.yaml
|
||||
```
|
||||
|
||||
6. Once complete, restart the server and boot into `/dev/sda`. When it comes back up, you should have SSH access as the `core` user using the public key provided in the `node-config.yaml` file. It will take some time for the node to be fully configured.
|
||||
|
||||
## Configure Kubeconfig
|
||||
|
||||
To administer your cluster from a separate host, you will need the client and admin certificates generated earlier (`ca.pem`, `admin.pem`, `admin-key.pem`). With certificates in place, run the following commands with the appropriate filepaths.
|
||||
|
||||
```shell
|
||||
kubectl config set-cluster calico-cluster --server=https://<KUBERNETES_MASTER> --certificate-authority=<CA_CERT_PATH>
|
||||
kubectl config set-credentials calico-admin --certificate-authority=<CA_CERT_PATH> --client-key=<ADMIN_KEY_PATH> --client-certificate=<ADMIN_CERT_PATH>
|
||||
kubectl config set-context calico --cluster=calico-cluster --user=calico-admin
|
||||
kubectl config use-context calico
|
||||
```
|
||||
|
||||
Check your work with `kubectl get nodes`.
|
||||
|
||||
## Install the DNS Addon
|
||||
|
||||
Most Kubernetes deployments will require the DNS addon for service discovery. To install DNS, create the skydns service and replication controller provided.
|
||||
|
||||
```shell
|
||||
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico-cni/k8s-1.1-docs/samples/kubernetes/master/dns/skydns.yaml
|
||||
```
|
||||
|
||||
## Install the Kubernetes UI Addon (Optional)
|
||||
|
||||
The Kubernetes UI can be installed using `kubectl` to run the following manifest file.
|
||||
|
||||
```shell
|
||||
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico-cni/k8s-1.1-docs/samples/kubernetes/master/kube-ui/kube-ui.yaml
|
||||
```
|
||||
|
||||
## Launch other Services With Calico-Kubernetes
|
||||
|
||||
At this point, you have a fully functioning cluster running on Kubernetes with a master and two nodes networked with Calico. You can now follow any of the [standard documentation](https://github.com/kubernetes/kubernetes/tree/{{page.version}}.0/examples/) to set up other services on your cluster.
|
||||
|
||||
## Connectivity to outside the cluster
|
||||
|
||||
Because containers in this guide have private `192.168.0.0/16` IPs, you will need NAT to allow connectivity between containers and the internet. However, in a production data center deployment, NAT is not always necessary, since Calico can peer with the data center's border routers over BGP.
|
||||
|
||||
### NAT on the nodes
|
||||
|
||||
The simplest method for enabling connectivity from containers to the internet is to use outgoing NAT on your Kubernetes nodes.
|
||||
|
||||
Calico can provide outgoing NAT for containers. To enable it, use the following `calicoctl` command:
|
||||
|
||||
```shell
|
||||
ETCD_AUTHORITY=<master_ip:6666> calicoctl pool add <CONTAINER_SUBNET> --nat-outgoing
|
||||
```
|
||||
|
||||
By default, `<CONTAINER_SUBNET>` will be `192.168.0.0/16`. You can find out which pools have been configured with the following command:
|
||||
|
||||
```shell
|
||||
ETCD_AUTHORITY=<master_ip:6666> calicoctl pool show
|
||||
```
|
||||
|
||||
### NAT at the border router
|
||||
|
||||
In a data center environment, it is recommended to configure Calico to peer with the border routers over BGP. This means that the container IPs will be routable anywhere in the data center, and so NAT is not needed on the nodes (though it may be enabled at the data center edge to allow outbound-only internet connectivity).
|
||||
|
||||
The Calico documentation contains more information on how to configure Calico to [peer with existing infrastructure](https://github.com/projectcalico/calico-containers/blob/master/docs/ExternalConnectivity.md).
|
||||
|
||||
## Support Level
|
||||
|
||||
|
||||
IaaS Provider | Config. Mgmt | OS | Networking | Docs | Conforms | Support Level
|
||||
-------------------- | ------------ | ------ | ---------- | --------------------------------------------- | ---------| ----------------------------
|
||||
Bare-metal | CoreOS | CoreOS | Calico | [docs](/docs/getting-started-guides/coreos/bare_metal_calico) | | Community ([@caseydavenport](https://github.com/caseydavenport))
|
||||
|
||||
|
||||
For support level information on all solutions, see the [Table of solutions](/docs/getting-started-guides/#table-of-solutions) chart.
|
||||
|
||||
|
||||
@@ -16,7 +16,7 @@ and a _worker_ node which receives work from the master. You can repeat the proc
|
||||
times to create larger clusters.
|
||||
|
||||
Here's a diagram of what the final result will look like:
|
||||
|
||||
|
||||
|
||||
### Bootstrap Docker
|
||||
|
||||
|
||||
@@ -1,241 +1,241 @@
|
||||
---
|
||||
assignees:
|
||||
- aveshagarwal
|
||||
- erictune
|
||||
|
||||
---
|
||||
|
||||
Configuring Kubernetes on Fedora via Ansible offers a simple way to quickly create a clustered environment with little effort.
|
||||
|
||||
* TOC
|
||||
{:toc}
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. Host able to run ansible and able to clone the following repo: [kubernetes](https://github.com/kubernetes/kubernetes.git)
|
||||
2. A Fedora 21+ host to act as cluster master
|
||||
3. As many Fedora 21+ hosts as you would like, that act as cluster nodes
|
||||
|
||||
The hosts can be virtual or bare metal. Ansible will take care of the rest of the configuration for you - configuring networking, installing packages, handling the firewall, etc. This example will use one master and two nodes.
|
||||
|
||||
## Architecture of the cluster
|
||||
|
||||
A Kubernetes cluster requires etcd, a master, and n nodes, so we will create a cluster with three hosts, for example:
|
||||
|
||||
```shell
|
||||
master,etcd = kube-master.example.com
|
||||
node1 = kube-node-01.example.com
|
||||
node2 = kube-node-02.example.com
|
||||
```
|
||||
|
||||
**Make sure your local machine has**
|
||||
|
||||
- ansible (must be 1.9.0+)
|
||||
- git
|
||||
- python-netaddr
|
||||
|
||||
If not
|
||||
|
||||
```shell
|
||||
yum install -y ansible git python-netaddr
|
||||
```
|
||||
|
||||
**Now clone down the Kubernetes repository**
|
||||
|
||||
```shell
|
||||
git clone https://github.com/kubernetes/contrib.git
|
||||
cd contrib/ansible
|
||||
```
|
||||
|
||||
**Tell ansible about each machine and its role in your cluster**
|
||||
|
||||
Get the IP addresses from the master and nodes. Add those to the `~/contrib/ansible/inventory` file on the host running Ansible.
|
||||
|
||||
```shell
|
||||
[masters]
|
||||
kube-master.example.com
|
||||
|
||||
[etcd]
|
||||
kube-master.example.com
|
||||
|
||||
[nodes]
|
||||
kube-node-01.example.com
|
||||
kube-node-02.example.com
|
||||
```
|
||||
|
||||
## Setting up ansible access to your nodes
|
||||
|
||||
If you already are running on a machine which has passwordless ssh access to the kube-master and kube-node-{01,02} nodes, and 'sudo' privileges, simply set the value of `ansible_ssh_user` in `~/contrib/ansible/group_vars/all.yml` to the username which you use to ssh to the nodes (i.e. `fedora`), and proceed to the next step...
|
||||
|
||||
*Otherwise* setup ssh on the machines like so (you will need to know the root password to all machines in the cluster).
|
||||
|
||||
edit: ~/contrib/ansible/group_vars/all.yml
|
||||
|
||||
```yaml
|
||||
ansible_ssh_user: root
|
||||
```
|
||||
|
||||
**Configuring ssh access to the cluster**
|
||||
|
||||
If you already have ssh access to every machine using ssh public keys you may skip to [setting up the cluster](#setting-up-the-cluster)
|
||||
|
||||
Make sure your local machine (root) has an ssh key pair if not
|
||||
|
||||
```shell
|
||||
ssh-keygen
|
||||
```
|
||||
|
||||
Copy the ssh public key to **all** nodes in the cluster
|
||||
|
||||
```shell
|
||||
for node in kube-master.example.com kube-node-01.example.com kube-node-02.example.com; do
|
||||
ssh-copy-id ${node}
|
||||
done
|
||||
```
|
||||
|
||||
## Setting up the cluster
|
||||
|
||||
Although the default value of variables in `~/contrib/ansible/group_vars/all.yml` should be good enough, if not, change them as needed.
|
||||
|
||||
```conf
|
||||
edit: ~/contrib/ansible/group_vars/all.yml
|
||||
```
|
||||
|
||||
**Configure access to kubernetes packages**
|
||||
|
||||
Modify `source_type` as below to access kubernetes packages through the package manager.
|
||||
|
||||
```yaml
|
||||
source_type: packageManager
|
||||
```
|
||||
|
||||
**Configure the IP addresses used for services**
|
||||
|
||||
Each Kubernetes service gets its own IP address. These are not real IPs. You need only select a range of IPs which are not in use elsewhere in your environment.
|
||||
|
||||
```yaml
|
||||
kube_service_addresses: 10.254.0.0/16
|
||||
```
|
||||
|
||||
**Managing flannel**
|
||||
|
||||
Modify `flannel_subnet`, `flannel_prefix` and `flannel_host_prefix` only if defaults are not appropriate for your cluster.
|
||||
|
||||
|
||||
**Managing add on services in your cluster**
|
||||
|
||||
Set `cluster_logging` to false or true (default) to disable or enable logging with elasticsearch.
|
||||
|
||||
```yaml
|
||||
cluster_logging: true
|
||||
```
|
||||
|
||||
Turn `cluster_monitoring` to true (default) or false to enable or disable cluster monitoring with heapster and influxdb.
|
||||
|
||||
```yaml
|
||||
cluster_monitoring: true
|
||||
```
|
||||
|
||||
Turn `dns_setup` to true (recommended) or false to enable or disable whole DNS configuration.
|
||||
|
||||
```yaml
|
||||
dns_setup: true
|
||||
```
|
||||
|
||||
**Tell ansible to get to work!**
|
||||
|
||||
This will finally setup your whole Kubernetes cluster for you.
|
||||
|
||||
```shell
|
||||
cd ~/contrib/ansible/
|
||||
|
||||
./setup.sh
|
||||
```
|
||||
|
||||
## Testing and using your new cluster
|
||||
|
||||
That's all there is to it. It's really that easy. At this point you should have a functioning Kubernetes cluster.
|
||||
|
||||
**Show kubernetes nodes**
|
||||
|
||||
Run the following on the kube-master:
|
||||
|
||||
```shell
|
||||
kubectl get nodes
|
||||
```
|
||||
|
||||
**Show services running on masters and nodes**
|
||||
|
||||
```shell
|
||||
systemctl | grep -i kube
|
||||
```
|
||||
|
||||
**Show firewall rules on the masters and nodes**
|
||||
|
||||
```shell
|
||||
iptables -nvL
|
||||
|
||||
```
|
||||
|
||||
**Create /tmp/apache.json on the master with the following contents and deploy pod**
|
||||
|
||||
```json
|
||||
{
|
||||
"kind": "Pod",
|
||||
"apiVersion": "v1",
|
||||
"metadata": {
|
||||
"name": "fedoraapache",
|
||||
"labels": {
|
||||
"name": "fedoraapache"
|
||||
}
|
||||
},
|
||||
"spec": {
|
||||
"containers": [
|
||||
{
|
||||
"name": "fedoraapache",
|
||||
"image": "fedora/apache",
|
||||
"ports": [
|
||||
{
|
||||
"hostPort": 80,
|
||||
"containerPort": 80
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
```shell
|
||||
kubectl create -f /tmp/apache.json
|
||||
```
|
||||
|
||||
**Check where the pod was created**
|
||||
|
||||
```shell
|
||||
kubectl get pods
|
||||
```
|
||||
|
||||
**Check Docker status on nodes**
|
||||
|
||||
```shell
|
||||
docker ps
|
||||
docker images
|
||||
```
|
||||
|
||||
**After the pod is 'Running' Check web server access on the node**
|
||||
|
||||
```shell
|
||||
curl http://localhost
|
||||
```
|
||||
|
||||
That's it !
|
||||
|
||||
## Support Level
|
||||
|
||||
|
||||
IaaS Provider | Config. Mgmt | OS | Networking | Docs | Conforms | Support Level
|
||||
-------------------- | ------------ | ------ | ---------- | --------------------------------------------- | ---------| ----------------------------
|
||||
Bare-metal | Ansible | Fedora | flannel | [docs](/docs/getting-started-guides/fedora/fedora_ansible_config) | | Project
|
||||
|
||||
For support level information on all solutions, see the [Table of solutions](/docs/getting-started-guides/#table-of-solutions) chart.
|
||||
---
|
||||
assignees:
|
||||
- aveshagarwal
|
||||
- erictune
|
||||
|
||||
---
|
||||
|
||||
Configuring Kubernetes on Fedora via Ansible offers a simple way to quickly create a clustered environment with little effort.
|
||||
|
||||
* TOC
|
||||
{:toc}
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. Host able to run ansible and able to clone the following repo: [kubernetes](https://github.com/kubernetes/kubernetes.git)
|
||||
2. A Fedora 21+ host to act as cluster master
|
||||
3. As many Fedora 21+ hosts as you would like, that act as cluster nodes
|
||||
|
||||
The hosts can be virtual or bare metal. Ansible will take care of the rest of the configuration for you - configuring networking, installing packages, handling the firewall, etc. This example will use one master and two nodes.
|
||||
|
||||
## Architecture of the cluster
|
||||
|
||||
A Kubernetes cluster requires etcd, a master, and n nodes, so we will create a cluster with three hosts, for example:
|
||||
|
||||
```shell
|
||||
master,etcd = kube-master.example.com
|
||||
node1 = kube-node-01.example.com
|
||||
node2 = kube-node-02.example.com
|
||||
```
|
||||
|
||||
**Make sure your local machine has**
|
||||
|
||||
- ansible (must be 1.9.0+)
|
||||
- git
|
||||
- python-netaddr
|
||||
|
||||
If not
|
||||
|
||||
```shell
|
||||
yum install -y ansible git python-netaddr
|
||||
```
|
||||
|
||||
**Now clone down the Kubernetes repository**
|
||||
|
||||
```shell
|
||||
git clone https://github.com/kubernetes/contrib.git
|
||||
cd contrib/ansible
|
||||
```
|
||||
|
||||
**Tell ansible about each machine and its role in your cluster**
|
||||
|
||||
Get the IP addresses from the master and nodes. Add those to the `~/contrib/ansible/inventory` file on the host running Ansible.
|
||||
|
||||
```shell
|
||||
[masters]
|
||||
kube-master.example.com
|
||||
|
||||
[etcd]
|
||||
kube-master.example.com
|
||||
|
||||
[nodes]
|
||||
kube-node-01.example.com
|
||||
kube-node-02.example.com
|
||||
```
|
||||
|
||||
## Setting up ansible access to your nodes
|
||||
|
||||
If you already are running on a machine which has passwordless ssh access to the kube-master and kube-node-{01,02} nodes, and 'sudo' privileges, simply set the value of `ansible_ssh_user` in `~/contrib/ansible/group_vars/all.yml` to the username which you use to ssh to the nodes (i.e. `fedora`), and proceed to the next step...
|
||||
|
||||
*Otherwise* setup ssh on the machines like so (you will need to know the root password to all machines in the cluster).
|
||||
|
||||
edit: ~/contrib/ansible/group_vars/all.yml
|
||||
|
||||
```yaml
|
||||
ansible_ssh_user: root
|
||||
```
|
||||
|
||||
**Configuring ssh access to the cluster**
|
||||
|
||||
If you already have ssh access to every machine using ssh public keys you may skip to [setting up the cluster](#setting-up-the-cluster)
|
||||
|
||||
Make sure your local machine (root) has an ssh key pair if not
|
||||
|
||||
```shell
|
||||
ssh-keygen
|
||||
```
|
||||
|
||||
Copy the ssh public key to **all** nodes in the cluster
|
||||
|
||||
```shell
|
||||
for node in kube-master.example.com kube-node-01.example.com kube-node-02.example.com; do
|
||||
ssh-copy-id ${node}
|
||||
done
|
||||
```
|
||||
|
||||
## Setting up the cluster
|
||||
|
||||
Although the default value of variables in `~/contrib/ansible/group_vars/all.yml` should be good enough, if not, change them as needed.
|
||||
|
||||
```conf
|
||||
edit: ~/contrib/ansible/group_vars/all.yml
|
||||
```
|
||||
|
||||
**Configure access to kubernetes packages**
|
||||
|
||||
Modify `source_type` as below to access kubernetes packages through the package manager.
|
||||
|
||||
```yaml
|
||||
source_type: packageManager
|
||||
```
|
||||
|
||||
**Configure the IP addresses used for services**
|
||||
|
||||
Each Kubernetes service gets its own IP address. These are not real IPs. You need only select a range of IPs which are not in use elsewhere in your environment.
|
||||
|
||||
```yaml
|
||||
kube_service_addresses: 10.254.0.0/16
|
||||
```
|
||||
|
||||
**Managing flannel**
|
||||
|
||||
Modify `flannel_subnet`, `flannel_prefix` and `flannel_host_prefix` only if defaults are not appropriate for your cluster.
|
||||
|
||||
|
||||
**Managing add on services in your cluster**
|
||||
|
||||
Set `cluster_logging` to false or true (default) to disable or enable logging with elasticsearch.
|
||||
|
||||
```yaml
|
||||
cluster_logging: true
|
||||
```
|
||||
|
||||
Turn `cluster_monitoring` to true (default) or false to enable or disable cluster monitoring with heapster and influxdb.
|
||||
|
||||
```yaml
|
||||
cluster_monitoring: true
|
||||
```
|
||||
|
||||
Turn `dns_setup` to true (recommended) or false to enable or disable whole DNS configuration.
|
||||
|
||||
```yaml
|
||||
dns_setup: true
|
||||
```
|
||||
|
||||
**Tell ansible to get to work!**
|
||||
|
||||
This will finally setup your whole Kubernetes cluster for you.
|
||||
|
||||
```shell
|
||||
cd ~/contrib/ansible/
|
||||
|
||||
./setup.sh
|
||||
```
|
||||
|
||||
## Testing and using your new cluster
|
||||
|
||||
That's all there is to it. It's really that easy. At this point you should have a functioning Kubernetes cluster.
|
||||
|
||||
**Show kubernetes nodes**
|
||||
|
||||
Run the following on the kube-master:
|
||||
|
||||
```shell
|
||||
kubectl get nodes
|
||||
```
|
||||
|
||||
**Show services running on masters and nodes**
|
||||
|
||||
```shell
|
||||
systemctl | grep -i kube
|
||||
```
|
||||
|
||||
**Show firewall rules on the masters and nodes**
|
||||
|
||||
```shell
|
||||
iptables -nvL
|
||||
|
||||
```
|
||||
|
||||
**Create /tmp/apache.json on the master with the following contents and deploy pod**
|
||||
|
||||
```json
|
||||
{
|
||||
"kind": "Pod",
|
||||
"apiVersion": "v1",
|
||||
"metadata": {
|
||||
"name": "fedoraapache",
|
||||
"labels": {
|
||||
"name": "fedoraapache"
|
||||
}
|
||||
},
|
||||
"spec": {
|
||||
"containers": [
|
||||
{
|
||||
"name": "fedoraapache",
|
||||
"image": "fedora/apache",
|
||||
"ports": [
|
||||
{
|
||||
"hostPort": 80,
|
||||
"containerPort": 80
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
```shell
|
||||
kubectl create -f /tmp/apache.json
|
||||
```
|
||||
|
||||
**Check where the pod was created**
|
||||
|
||||
```shell
|
||||
kubectl get pods
|
||||
```
|
||||
|
||||
**Check Docker status on nodes**
|
||||
|
||||
```shell
|
||||
docker ps
|
||||
docker images
|
||||
```
|
||||
|
||||
**After the pod is 'Running' Check web server access on the node**
|
||||
|
||||
```shell
|
||||
curl http://localhost
|
||||
```
|
||||
|
||||
That's it !
|
||||
|
||||
## Support Level
|
||||
|
||||
|
||||
IaaS Provider | Config. Mgmt | OS | Networking | Docs | Conforms | Support Level
|
||||
-------------------- | ------------ | ------ | ---------- | --------------------------------------------- | ---------| ----------------------------
|
||||
Bare-metal | Ansible | Fedora | flannel | [docs](/docs/getting-started-guides/fedora/fedora_ansible_config) | | Project
|
||||
|
||||
For support level information on all solutions, see the [Table of solutions](/docs/getting-started-guides/#table-of-solutions) chart.
|
||||
|
||||
@@ -1,219 +1,219 @@
|
||||
---
|
||||
assignees:
|
||||
- aveshagarwal
|
||||
- eparis
|
||||
- thockin
|
||||
|
||||
---
|
||||
|
||||
* TOC
|
||||
{:toc}
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. You need 2 or more machines with Fedora installed.
|
||||
|
||||
## Instructions
|
||||
|
||||
This is a getting started guide for Fedora. It is a manual configuration so you understand all the underlying packages / services / ports, etc...
|
||||
|
||||
This guide will only get ONE node (previously minion) working. Multiple nodes require a functional [networking configuration](/docs/admin/networking/) done outside of Kubernetes. Although the additional Kubernetes configuration requirements should be obvious.
|
||||
|
||||
The Kubernetes package provides a few services: kube-apiserver, kube-scheduler, kube-controller-manager, kubelet, kube-proxy. These services are managed by systemd and the configuration resides in a central location: /etc/kubernetes. We will break the services up between the hosts. The first host, fed-master, will be the Kubernetes master. This host will run the kube-apiserver, kube-controller-manager, and kube-scheduler. In addition, the master will also run _etcd_ (not needed if _etcd_ runs on a different host but this guide assumes that _etcd_ and Kubernetes master run on the same host). The remaining host, fed-node will be the node and run kubelet, proxy and docker.
|
||||
|
||||
**System Information:**
|
||||
|
||||
Hosts:
|
||||
|
||||
```conf
|
||||
fed-master = 192.168.121.9
|
||||
fed-node = 192.168.121.65
|
||||
```
|
||||
|
||||
**Prepare the hosts:**
|
||||
|
||||
* Install Kubernetes on all hosts - fed-{master,node}. This will also pull in docker. Also install etcd on fed-master. This guide has been tested with kubernetes-0.18 and beyond.
|
||||
* The [--enablerepo=updates-testing](https://fedoraproject.org/wiki/QA:Updates_Testing) directive in the yum command below will ensure that the most recent Kubernetes version that is scheduled for pre-release will be installed. This should be a more recent version than the Fedora "stable" release for Kubernetes that you would get without adding the directive.
|
||||
* If you want the very latest Kubernetes release [you can download and yum install the RPM directly from Fedora Koji](http://koji.fedoraproject.org/koji/packageinfo?packageID=19202) instead of using the yum install command below.
|
||||
* Running on AWS EC2 with RHEL 7.2, you need to enable "extras" repository for yum by editing `/etc/yum.repos.d/redhat-rhui.repo` and changing the changing the `enable=0` to `enable=1` for extras.
|
||||
|
||||
```shell
|
||||
yum -y install --enablerepo=updates-testing kubernetes
|
||||
```
|
||||
|
||||
* Install etcd and iptables
|
||||
|
||||
```shell
|
||||
yum -y install etcd iptables
|
||||
```
|
||||
|
||||
* Add master and node to /etc/hosts on all machines (not needed if hostnames already in DNS). Make sure that communication works between fed-master and fed-node by using a utility such as ping.
|
||||
|
||||
```shell
|
||||
echo "192.168.121.9 fed-master
|
||||
192.168.121.65 fed-node" >> /etc/hosts
|
||||
```
|
||||
|
||||
* Edit /etc/kubernetes/config which will be the same on all hosts (master and node) to contain:
|
||||
|
||||
```shell
|
||||
# Comma separated list of nodes in the etcd cluster
|
||||
KUBE_MASTER="--master=http://fed-master:8080"
|
||||
|
||||
# logging to stderr means we get it in the systemd journal
|
||||
KUBE_LOGTOSTDERR="--logtostderr=true"
|
||||
|
||||
# journal message level, 0 is debug
|
||||
KUBE_LOG_LEVEL="--v=0"
|
||||
|
||||
# Should this cluster be allowed to run privileged docker containers
|
||||
KUBE_ALLOW_PRIV="--allow-privileged=false"
|
||||
```
|
||||
|
||||
* Disable the firewall on both the master and node, as docker does not play well with other firewall rule managers. Please note that iptables-services does not exist on default fedora server install.
|
||||
|
||||
```shell
|
||||
systemctl disable iptables-services firewalld
|
||||
systemctl stop iptables-services firewalld
|
||||
```
|
||||
|
||||
**Configure the Kubernetes services on the master.**
|
||||
|
||||
* Edit /etc/kubernetes/apiserver to appear as such. The service-cluster-ip-range IP addresses must be an unused block of addresses, not used anywhere else. They do not need to be routed or assigned to anything.
|
||||
|
||||
```shell
|
||||
# The address on the local server to listen to.
|
||||
KUBE_API_ADDRESS="--address=0.0.0.0"
|
||||
|
||||
# Comma separated list of nodes in the etcd cluster
|
||||
KUBE_ETCD_SERVERS="--etcd-servers=http://127.0.0.1:4001"
|
||||
|
||||
# Address range to use for services
|
||||
KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16"
|
||||
|
||||
# Add your own!
|
||||
KUBE_API_ARGS=""
|
||||
```
|
||||
|
||||
* Edit /etc/etcd/etcd.conf,let the etcd to listen all the ip instead of 127.0.0.1, if not, you will get the error like "connection refused". Note that Fedora 22 uses etcd 2.0, One of the changes in etcd 2.0 is that now uses port 2379 and 2380 (as opposed to etcd 0.46 which userd 4001 and 7001).
|
||||
|
||||
```shell
|
||||
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:4001"
|
||||
```
|
||||
|
||||
* Create /var/run/kubernetes on master:
|
||||
|
||||
```shell
|
||||
mkdir /var/run/kubernetes
|
||||
chown kube:kube /var/run/kubernetes
|
||||
chmod 750 /var/run/kubernetes
|
||||
```
|
||||
|
||||
* Start the appropriate services on master:
|
||||
|
||||
```shell
|
||||
for SERVICES in etcd kube-apiserver kube-controller-manager kube-scheduler; do
|
||||
systemctl restart $SERVICES
|
||||
systemctl enable $SERVICES
|
||||
systemctl status $SERVICES
|
||||
done
|
||||
```
|
||||
|
||||
* Addition of nodes:
|
||||
|
||||
* Create following node.json file on Kubernetes master node:
|
||||
|
||||
```json
|
||||
{
|
||||
"apiVersion": "v1",
|
||||
"kind": "Node",
|
||||
"metadata": {
|
||||
"name": "fed-node",
|
||||
"labels":{ "name": "fed-node-label"}
|
||||
},
|
||||
"spec": {
|
||||
"externalID": "fed-node"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Now create a node object internally in your Kubernetes cluster by running:
|
||||
|
||||
```shell
|
||||
$ kubectl create -f ./node.json
|
||||
|
||||
$ kubectl get nodes
|
||||
NAME LABELS STATUS
|
||||
fed-node name=fed-node-label Unknown
|
||||
```
|
||||
|
||||
Please note that in the above, it only creates a representation for the node
|
||||
_fed-node_ internally. It does not provision the actual _fed-node_. Also, it
|
||||
is assumed that _fed-node_ (as specified in `name`) can be resolved and is
|
||||
reachable from Kubernetes master node. This guide will discuss how to provision
|
||||
a Kubernetes node (fed-node) below.
|
||||
|
||||
**Configure the Kubernetes services on the node.**
|
||||
|
||||
***We need to configure the kubelet on the node.***
|
||||
|
||||
* Edit /etc/kubernetes/kubelet to appear as such:
|
||||
|
||||
```shell
|
||||
###
|
||||
# Kubernetes kubelet (node) config
|
||||
|
||||
# The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces)
|
||||
KUBELET_ADDRESS="--address=0.0.0.0"
|
||||
|
||||
# You may leave this blank to use the actual hostname
|
||||
KUBELET_HOSTNAME="--hostname-override=fed-node"
|
||||
|
||||
# location of the api-server
|
||||
KUBELET_API_SERVER="--api-servers=http://fed-master:8080"
|
||||
|
||||
# Add your own!
|
||||
#KUBELET_ARGS=""
|
||||
```
|
||||
|
||||
* Start the appropriate services on the node (fed-node).
|
||||
|
||||
```shell
|
||||
for SERVICES in kube-proxy kubelet docker; do
|
||||
systemctl restart $SERVICES
|
||||
systemctl enable $SERVICES
|
||||
systemctl status $SERVICES
|
||||
done
|
||||
```
|
||||
|
||||
* Check to make sure now the cluster can see the fed-node on fed-master, and its status changes to _Ready_.
|
||||
|
||||
```shell
|
||||
kubectl get nodes
|
||||
NAME LABELS STATUS
|
||||
fed-node name=fed-node-label Ready
|
||||
```
|
||||
|
||||
* Deletion of nodes:
|
||||
|
||||
To delete _fed-node_ from your Kubernetes cluster, one should run the following on fed-master (Please do not do it, it is just for information):
|
||||
|
||||
```shell
|
||||
kubectl delete -f ./node.json
|
||||
```
|
||||
|
||||
*You should be finished!*
|
||||
|
||||
**The cluster should be running! Launch a test pod.**
|
||||
|
||||
You should have a functional cluster, check out [101](/docs/user-guide/walkthrough/)!
|
||||
|
||||
## Support Level
|
||||
|
||||
|
||||
IaaS Provider | Config. Mgmt | OS | Networking | Docs | Conforms | Support Level
|
||||
-------------------- | ------------ | ------ | ---------- | --------------------------------------------- | ---------| ----------------------------
|
||||
Bare-metal | custom | Fedora | _none_ | [docs](/docs/getting-started-guides/fedora/fedora_manual_config) | | Project
|
||||
|
||||
For support level information on all solutions, see the [Table of solutions](/docs/getting-started-guides/#table-of-solutions) chart.
|
||||
|
||||
---
|
||||
assignees:
|
||||
- aveshagarwal
|
||||
- eparis
|
||||
- thockin
|
||||
|
||||
---
|
||||
|
||||
* TOC
|
||||
{:toc}
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. You need 2 or more machines with Fedora installed.
|
||||
|
||||
## Instructions
|
||||
|
||||
This is a getting started guide for Fedora. It is a manual configuration so you understand all the underlying packages / services / ports, etc...
|
||||
|
||||
This guide will only get ONE node (previously minion) working. Multiple nodes require a functional [networking configuration](/docs/admin/networking/) done outside of Kubernetes. Although the additional Kubernetes configuration requirements should be obvious.
|
||||
|
||||
The Kubernetes package provides a few services: kube-apiserver, kube-scheduler, kube-controller-manager, kubelet, kube-proxy. These services are managed by systemd and the configuration resides in a central location: /etc/kubernetes. We will break the services up between the hosts. The first host, fed-master, will be the Kubernetes master. This host will run the kube-apiserver, kube-controller-manager, and kube-scheduler. In addition, the master will also run _etcd_ (not needed if _etcd_ runs on a different host but this guide assumes that _etcd_ and Kubernetes master run on the same host). The remaining host, fed-node will be the node and run kubelet, proxy and docker.
|
||||
|
||||
**System Information:**
|
||||
|
||||
Hosts:
|
||||
|
||||
```conf
|
||||
fed-master = 192.168.121.9
|
||||
fed-node = 192.168.121.65
|
||||
```
|
||||
|
||||
**Prepare the hosts:**
|
||||
|
||||
* Install Kubernetes on all hosts - fed-{master,node}. This will also pull in docker. Also install etcd on fed-master. This guide has been tested with kubernetes-0.18 and beyond.
|
||||
* The [--enablerepo=updates-testing](https://fedoraproject.org/wiki/QA:Updates_Testing) directive in the yum command below will ensure that the most recent Kubernetes version that is scheduled for pre-release will be installed. This should be a more recent version than the Fedora "stable" release for Kubernetes that you would get without adding the directive.
|
||||
* If you want the very latest Kubernetes release [you can download and yum install the RPM directly from Fedora Koji](http://koji.fedoraproject.org/koji/packageinfo?packageID=19202) instead of using the yum install command below.
|
||||
* Running on AWS EC2 with RHEL 7.2, you need to enable "extras" repository for yum by editing `/etc/yum.repos.d/redhat-rhui.repo` and changing the changing the `enable=0` to `enable=1` for extras.
|
||||
|
||||
```shell
|
||||
yum -y install --enablerepo=updates-testing kubernetes
|
||||
```
|
||||
|
||||
* Install etcd and iptables
|
||||
|
||||
```shell
|
||||
yum -y install etcd iptables
|
||||
```
|
||||
|
||||
* Add master and node to /etc/hosts on all machines (not needed if hostnames already in DNS). Make sure that communication works between fed-master and fed-node by using a utility such as ping.
|
||||
|
||||
```shell
|
||||
echo "192.168.121.9 fed-master
|
||||
192.168.121.65 fed-node" >> /etc/hosts
|
||||
```
|
||||
|
||||
* Edit /etc/kubernetes/config which will be the same on all hosts (master and node) to contain:
|
||||
|
||||
```shell
|
||||
# Comma separated list of nodes in the etcd cluster
|
||||
KUBE_MASTER="--master=http://fed-master:8080"
|
||||
|
||||
# logging to stderr means we get it in the systemd journal
|
||||
KUBE_LOGTOSTDERR="--logtostderr=true"
|
||||
|
||||
# journal message level, 0 is debug
|
||||
KUBE_LOG_LEVEL="--v=0"
|
||||
|
||||
# Should this cluster be allowed to run privileged docker containers
|
||||
KUBE_ALLOW_PRIV="--allow-privileged=false"
|
||||
```
|
||||
|
||||
* Disable the firewall on both the master and node, as docker does not play well with other firewall rule managers. Please note that iptables-services does not exist on default fedora server install.
|
||||
|
||||
```shell
|
||||
systemctl disable iptables-services firewalld
|
||||
systemctl stop iptables-services firewalld
|
||||
```
|
||||
|
||||
**Configure the Kubernetes services on the master.**
|
||||
|
||||
* Edit /etc/kubernetes/apiserver to appear as such. The service-cluster-ip-range IP addresses must be an unused block of addresses, not used anywhere else. They do not need to be routed or assigned to anything.
|
||||
|
||||
```shell
|
||||
# The address on the local server to listen to.
|
||||
KUBE_API_ADDRESS="--address=0.0.0.0"
|
||||
|
||||
# Comma separated list of nodes in the etcd cluster
|
||||
KUBE_ETCD_SERVERS="--etcd-servers=http://127.0.0.1:4001"
|
||||
|
||||
# Address range to use for services
|
||||
KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16"
|
||||
|
||||
# Add your own!
|
||||
KUBE_API_ARGS=""
|
||||
```
|
||||
|
||||
* Edit /etc/etcd/etcd.conf,let the etcd to listen all the ip instead of 127.0.0.1, if not, you will get the error like "connection refused". Note that Fedora 22 uses etcd 2.0, One of the changes in etcd 2.0 is that now uses port 2379 and 2380 (as opposed to etcd 0.46 which userd 4001 and 7001).
|
||||
|
||||
```shell
|
||||
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:4001"
|
||||
```
|
||||
|
||||
* Create /var/run/kubernetes on master:
|
||||
|
||||
```shell
|
||||
mkdir /var/run/kubernetes
|
||||
chown kube:kube /var/run/kubernetes
|
||||
chmod 750 /var/run/kubernetes
|
||||
```
|
||||
|
||||
* Start the appropriate services on master:
|
||||
|
||||
```shell
|
||||
for SERVICES in etcd kube-apiserver kube-controller-manager kube-scheduler; do
|
||||
systemctl restart $SERVICES
|
||||
systemctl enable $SERVICES
|
||||
systemctl status $SERVICES
|
||||
done
|
||||
```
|
||||
|
||||
* Addition of nodes:
|
||||
|
||||
* Create following node.json file on Kubernetes master node:
|
||||
|
||||
```json
|
||||
{
|
||||
"apiVersion": "v1",
|
||||
"kind": "Node",
|
||||
"metadata": {
|
||||
"name": "fed-node",
|
||||
"labels":{ "name": "fed-node-label"}
|
||||
},
|
||||
"spec": {
|
||||
"externalID": "fed-node"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Now create a node object internally in your Kubernetes cluster by running:
|
||||
|
||||
```shell
|
||||
$ kubectl create -f ./node.json
|
||||
|
||||
$ kubectl get nodes
|
||||
NAME LABELS STATUS
|
||||
fed-node name=fed-node-label Unknown
|
||||
```
|
||||
|
||||
Please note that in the above, it only creates a representation for the node
|
||||
_fed-node_ internally. It does not provision the actual _fed-node_. Also, it
|
||||
is assumed that _fed-node_ (as specified in `name`) can be resolved and is
|
||||
reachable from Kubernetes master node. This guide will discuss how to provision
|
||||
a Kubernetes node (fed-node) below.
|
||||
|
||||
**Configure the Kubernetes services on the node.**
|
||||
|
||||
***We need to configure the kubelet on the node.***
|
||||
|
||||
* Edit /etc/kubernetes/kubelet to appear as such:
|
||||
|
||||
```shell
|
||||
###
|
||||
# Kubernetes kubelet (node) config
|
||||
|
||||
# The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces)
|
||||
KUBELET_ADDRESS="--address=0.0.0.0"
|
||||
|
||||
# You may leave this blank to use the actual hostname
|
||||
KUBELET_HOSTNAME="--hostname-override=fed-node"
|
||||
|
||||
# location of the api-server
|
||||
KUBELET_API_SERVER="--api-servers=http://fed-master:8080"
|
||||
|
||||
# Add your own!
|
||||
#KUBELET_ARGS=""
|
||||
```
|
||||
|
||||
* Start the appropriate services on the node (fed-node).
|
||||
|
||||
```shell
|
||||
for SERVICES in kube-proxy kubelet docker; do
|
||||
systemctl restart $SERVICES
|
||||
systemctl enable $SERVICES
|
||||
systemctl status $SERVICES
|
||||
done
|
||||
```
|
||||
|
||||
* Check to make sure now the cluster can see the fed-node on fed-master, and its status changes to _Ready_.
|
||||
|
||||
```shell
|
||||
kubectl get nodes
|
||||
NAME LABELS STATUS
|
||||
fed-node name=fed-node-label Ready
|
||||
```
|
||||
|
||||
* Deletion of nodes:
|
||||
|
||||
To delete _fed-node_ from your Kubernetes cluster, one should run the following on fed-master (Please do not do it, it is just for information):
|
||||
|
||||
```shell
|
||||
kubectl delete -f ./node.json
|
||||
```
|
||||
|
||||
*You should be finished!*
|
||||
|
||||
**The cluster should be running! Launch a test pod.**
|
||||
|
||||
You should have a functional cluster, check out [101](/docs/user-guide/walkthrough/)!
|
||||
|
||||
## Support Level
|
||||
|
||||
|
||||
IaaS Provider | Config. Mgmt | OS | Networking | Docs | Conforms | Support Level
|
||||
-------------------- | ------------ | ------ | ---------- | --------------------------------------------- | ---------| ----------------------------
|
||||
Bare-metal | custom | Fedora | _none_ | [docs](/docs/getting-started-guides/fedora/fedora_manual_config) | | Project
|
||||
|
||||
For support level information on all solutions, see the [Table of solutions](/docs/getting-started-guides/#table-of-solutions) chart.
|
||||
|
||||
|
||||
@@ -1,191 +1,191 @@
|
||||
---
|
||||
assignees:
|
||||
- dchen1107
|
||||
- erictune
|
||||
- thockin
|
||||
|
||||
---
|
||||
* TOC
|
||||
{:toc}
|
||||
|
||||
This document describes how to deploy Kubernetes on multiple hosts to set up a multi-node cluster and networking with flannel. Follow fedora [getting started guide](/docs/getting-started-guides/fedora/fedora_manual_config/) to setup 1 master (fed-master) and 2 or more nodes. Make sure that all nodes have different names (fed-node1, fed-node2 and so on) and labels (fed-node1-label, fed-node2-label, and so on) to avoid any conflict. Also make sure that the Kubernetes master host is running etcd, kube-controller-manager, kube-scheduler, and kube-apiserver services, and the nodes are running docker, kube-proxy and kubelet services. Now install flannel on Kubernetes nodes. flannel on each node configures an overlay network that docker uses. flannel runs on each node to setup a unique class-C container network.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
You need 2 or more machines with Fedora installed.
|
||||
|
||||
## Master Setup
|
||||
|
||||
**Perform following commands on the Kubernetes master**
|
||||
|
||||
* Configure flannel by creating a `flannel-config.json` in your current directory on fed-master. Flannel provides udp and vxlan among other overlay networking backend options. In this guide, we choose kernel based vxlan backend. The contents of the json are:
|
||||
|
||||
```json
|
||||
{
|
||||
"Network": "18.16.0.0/16",
|
||||
"SubnetLen": 24,
|
||||
"Backend": {
|
||||
"Type": "vxlan",
|
||||
"VNI": 1
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**NOTE:** Choose an IP range that is *NOT* part of the public IP address range.
|
||||
|
||||
Add the configuration to the etcd server on fed-master.
|
||||
|
||||
```shell
|
||||
etcdctl set /coreos.com/network/config < flannel-config.json
|
||||
```
|
||||
|
||||
* Verify the key exists in the etcd server on fed-master.
|
||||
|
||||
```shell
|
||||
etcdctl get /coreos.com/network/config
|
||||
```
|
||||
|
||||
## Node Setup
|
||||
|
||||
**Perform following commands on all Kubernetes nodes**
|
||||
|
||||
Edit the flannel configuration file /etc/sysconfig/flanneld as follows:
|
||||
|
||||
```shell
|
||||
# Flanneld configuration options
|
||||
|
||||
# etcd url location. Point this to the server where etcd runs
|
||||
FLANNEL_ETCD="http://fed-master:4001"
|
||||
|
||||
# etcd config key. This is the configuration key that flannel queries
|
||||
# For address range assignment
|
||||
FLANNEL_ETCD_KEY="/coreos.com/network"
|
||||
|
||||
# Any additional options that you want to pass
|
||||
FLANNEL_OPTIONS=""
|
||||
```
|
||||
|
||||
**Note:** By default, flannel uses the interface for the default route. If you have multiple interfaces and would like to use an interface other than the default route one, you could add "-iface=" to FLANNEL_OPTIONS. For additional options, run `flanneld --help` on command line.
|
||||
|
||||
Enable the flannel service.
|
||||
|
||||
```shell
|
||||
systemctl enable flanneld
|
||||
```
|
||||
|
||||
If docker is not running, then starting flannel service is enough and skip the next step.
|
||||
|
||||
```shell
|
||||
systemctl start flanneld
|
||||
```
|
||||
|
||||
If docker is already running, then stop docker, delete docker bridge (docker0), start flanneld and restart docker as follows. Another alternative is to just reboot the system (`systemctl reboot`).
|
||||
|
||||
```shell
|
||||
systemctl stop docker
|
||||
ip link delete docker0
|
||||
systemctl start flanneld
|
||||
systemctl start docker
|
||||
```
|
||||
|
||||
|
||||
## **Test the cluster and flannel configuration**
|
||||
|
||||
Now check the interfaces on the nodes. Notice there is now a flannel.1 interface, and the ip addresses of docker0 and flannel.1 interfaces are in the same network. You will notice that docker0 is assigned a subnet (18.16.29.0/24 as shown below) on each Kubernetes node out of the IP range configured above. A working output should look like this:
|
||||
|
||||
```shell
|
||||
# ip -4 a|grep inet
|
||||
inet 127.0.0.1/8 scope host lo
|
||||
inet 192.168.122.77/24 brd 192.168.122.255 scope global dynamic eth0
|
||||
inet 18.16.29.0/16 scope global flannel.1
|
||||
inet 18.16.29.1/24 scope global docker0
|
||||
```
|
||||
|
||||
From any node in the cluster, check the cluster members by issuing a query to etcd server via curl (only partial output is shown using `grep -E "\{|\}|key|value"`). If you set up a 1 master and 3 nodes cluster, you should see one block for each node showing the subnets they have been assigned. You can associate those subnets to each node by the MAC address (VtepMAC) and IP address (Public IP) that is listed in the output.
|
||||
|
||||
```shell
|
||||
curl -s http://fed-master:4001/v2/keys/coreos.com/network/subnets | python -mjson.tool
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"node": {
|
||||
"key": "/coreos.com/network/subnets",
|
||||
{
|
||||
"key": "/coreos.com/network/subnets/18.16.29.0-24",
|
||||
"value": "{\"PublicIP\":\"192.168.122.77\",\"BackendType\":\"vxlan\",\"BackendData\":{\"VtepMAC\":\"46:f1:d0:18:d0:65\"}}"
|
||||
},
|
||||
{
|
||||
"key": "/coreos.com/network/subnets/18.16.83.0-24",
|
||||
"value": "{\"PublicIP\":\"192.168.122.36\",\"BackendType\":\"vxlan\",\"BackendData\":{\"VtepMAC\":\"ca:38:78:fc:72:29\"}}"
|
||||
},
|
||||
{
|
||||
"key": "/coreos.com/network/subnets/18.16.90.0-24",
|
||||
"value": "{\"PublicIP\":\"192.168.122.127\",\"BackendType\":\"vxlan\",\"BackendData\":{\"VtepMAC\":\"92:e2:80:ba:2d:4d\"}}"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
From all nodes, review the `/run/flannel/subnet.env` file. This file was generated automatically by flannel.
|
||||
|
||||
```shell
|
||||
# cat /run/flannel/subnet.env
|
||||
FLANNEL_SUBNET=18.16.29.1/24
|
||||
FLANNEL_MTU=1450
|
||||
FLANNEL_IPMASQ=false
|
||||
```
|
||||
|
||||
At this point, we have etcd running on the Kubernetes master, and flannel / docker running on Kubernetes nodes. Next steps are for testing cross-host container communication which will confirm that docker and flannel are configured properly.
|
||||
|
||||
Issue the following commands on any 2 nodes:
|
||||
|
||||
```shell
|
||||
# docker run -it fedora:latest bash
|
||||
bash-4.3#
|
||||
```
|
||||
|
||||
This will place you inside the container. Install iproute and iputils packages to install ip and ping utilities. Due to a [bug](https://bugzilla.redhat.com/show_bug.cgi?id=1142311), it is required to modify capabilities of ping binary to work around "Operation not permitted" error.
|
||||
|
||||
```shell
|
||||
bash-4.3# yum -y install iproute iputils
|
||||
bash-4.3# setcap cap_net_raw-ep /usr/bin/ping
|
||||
```
|
||||
|
||||
Now note the IP address on the first node:
|
||||
|
||||
```shell
|
||||
bash-4.3# ip -4 a l eth0 | grep inet
|
||||
inet 18.16.29.4/24 scope global eth0
|
||||
```
|
||||
|
||||
And also note the IP address on the other node:
|
||||
|
||||
```shell
|
||||
bash-4.3# ip a l eth0 | grep inet
|
||||
inet 18.16.90.4/24 scope global eth0
|
||||
```
|
||||
Now ping from the first node to the other node:
|
||||
|
||||
```shell
|
||||
bash-4.3# ping 18.16.90.4
|
||||
PING 18.16.90.4 (18.16.90.4) 56(84) bytes of data.
|
||||
64 bytes from 18.16.90.4: icmp_seq=1 ttl=62 time=0.275 ms
|
||||
64 bytes from 18.16.90.4: icmp_seq=2 ttl=62 time=0.372 ms
|
||||
```
|
||||
|
||||
Now Kubernetes multi-node cluster is set up with overlay networking set up by flannel.
|
||||
|
||||
## Support Level
|
||||
|
||||
|
||||
IaaS Provider | Config. Mgmt | OS | Networking | Docs | Conforms | Support Level
|
||||
-------------------- | ------------ | ------ | ---------- | --------------------------------------------- | ---------| ----------------------------
|
||||
Bare-metal | custom | Fedora | flannel | [docs](/docs/getting-started-guides/fedora/flannel_multi_node_cluster) | | Community ([@aveshagarwal](https://github.com/aveshagarwal))
|
||||
libvirt | custom | Fedora | flannel | [docs](/docs/getting-started-guides/fedora/flannel_multi_node_cluster) | | Community ([@aveshagarwal](https://github.com/aveshagarwal))
|
||||
KVM | custom | Fedora | flannel | [docs](/docs/getting-started-guides/fedora/flannel_multi_node_cluster) | | Community ([@aveshagarwal](https://github.com/aveshagarwal))
|
||||
|
||||
|
||||
|
||||
For support level information on all solutions, see the [Table of solutions](/docs/getting-started-guides/#table-of-solutions) chart.
|
||||
|
||||
---
|
||||
assignees:
|
||||
- dchen1107
|
||||
- erictune
|
||||
- thockin
|
||||
|
||||
---
|
||||
* TOC
|
||||
{:toc}
|
||||
|
||||
This document describes how to deploy Kubernetes on multiple hosts to set up a multi-node cluster and networking with flannel. Follow fedora [getting started guide](/docs/getting-started-guides/fedora/fedora_manual_config/) to setup 1 master (fed-master) and 2 or more nodes. Make sure that all nodes have different names (fed-node1, fed-node2 and so on) and labels (fed-node1-label, fed-node2-label, and so on) to avoid any conflict. Also make sure that the Kubernetes master host is running etcd, kube-controller-manager, kube-scheduler, and kube-apiserver services, and the nodes are running docker, kube-proxy and kubelet services. Now install flannel on Kubernetes nodes. flannel on each node configures an overlay network that docker uses. flannel runs on each node to setup a unique class-C container network.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
You need 2 or more machines with Fedora installed.
|
||||
|
||||
## Master Setup
|
||||
|
||||
**Perform following commands on the Kubernetes master**
|
||||
|
||||
* Configure flannel by creating a `flannel-config.json` in your current directory on fed-master. Flannel provides udp and vxlan among other overlay networking backend options. In this guide, we choose kernel based vxlan backend. The contents of the json are:
|
||||
|
||||
```json
|
||||
{
|
||||
"Network": "18.16.0.0/16",
|
||||
"SubnetLen": 24,
|
||||
"Backend": {
|
||||
"Type": "vxlan",
|
||||
"VNI": 1
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**NOTE:** Choose an IP range that is *NOT* part of the public IP address range.
|
||||
|
||||
Add the configuration to the etcd server on fed-master.
|
||||
|
||||
```shell
|
||||
etcdctl set /coreos.com/network/config < flannel-config.json
|
||||
```
|
||||
|
||||
* Verify the key exists in the etcd server on fed-master.
|
||||
|
||||
```shell
|
||||
etcdctl get /coreos.com/network/config
|
||||
```
|
||||
|
||||
## Node Setup
|
||||
|
||||
**Perform following commands on all Kubernetes nodes**
|
||||
|
||||
Edit the flannel configuration file /etc/sysconfig/flanneld as follows:
|
||||
|
||||
```shell
|
||||
# Flanneld configuration options
|
||||
|
||||
# etcd url location. Point this to the server where etcd runs
|
||||
FLANNEL_ETCD="http://fed-master:4001"
|
||||
|
||||
# etcd config key. This is the configuration key that flannel queries
|
||||
# For address range assignment
|
||||
FLANNEL_ETCD_KEY="/coreos.com/network"
|
||||
|
||||
# Any additional options that you want to pass
|
||||
FLANNEL_OPTIONS=""
|
||||
```
|
||||
|
||||
**Note:** By default, flannel uses the interface for the default route. If you have multiple interfaces and would like to use an interface other than the default route one, you could add "-iface=" to FLANNEL_OPTIONS. For additional options, run `flanneld --help` on command line.
|
||||
|
||||
Enable the flannel service.
|
||||
|
||||
```shell
|
||||
systemctl enable flanneld
|
||||
```
|
||||
|
||||
If docker is not running, then starting flannel service is enough and skip the next step.
|
||||
|
||||
```shell
|
||||
systemctl start flanneld
|
||||
```
|
||||
|
||||
If docker is already running, then stop docker, delete docker bridge (docker0), start flanneld and restart docker as follows. Another alternative is to just reboot the system (`systemctl reboot`).
|
||||
|
||||
```shell
|
||||
systemctl stop docker
|
||||
ip link delete docker0
|
||||
systemctl start flanneld
|
||||
systemctl start docker
|
||||
```
|
||||
|
||||
|
||||
## **Test the cluster and flannel configuration**
|
||||
|
||||
Now check the interfaces on the nodes. Notice there is now a flannel.1 interface, and the ip addresses of docker0 and flannel.1 interfaces are in the same network. You will notice that docker0 is assigned a subnet (18.16.29.0/24 as shown below) on each Kubernetes node out of the IP range configured above. A working output should look like this:
|
||||
|
||||
```shell
|
||||
# ip -4 a|grep inet
|
||||
inet 127.0.0.1/8 scope host lo
|
||||
inet 192.168.122.77/24 brd 192.168.122.255 scope global dynamic eth0
|
||||
inet 18.16.29.0/16 scope global flannel.1
|
||||
inet 18.16.29.1/24 scope global docker0
|
||||
```
|
||||
|
||||
From any node in the cluster, check the cluster members by issuing a query to etcd server via curl (only partial output is shown using `grep -E "\{|\}|key|value"`). If you set up a 1 master and 3 nodes cluster, you should see one block for each node showing the subnets they have been assigned. You can associate those subnets to each node by the MAC address (VtepMAC) and IP address (Public IP) that is listed in the output.
|
||||
|
||||
```shell
|
||||
curl -s http://fed-master:4001/v2/keys/coreos.com/network/subnets | python -mjson.tool
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"node": {
|
||||
"key": "/coreos.com/network/subnets",
|
||||
{
|
||||
"key": "/coreos.com/network/subnets/18.16.29.0-24",
|
||||
"value": "{\"PublicIP\":\"192.168.122.77\",\"BackendType\":\"vxlan\",\"BackendData\":{\"VtepMAC\":\"46:f1:d0:18:d0:65\"}}"
|
||||
},
|
||||
{
|
||||
"key": "/coreos.com/network/subnets/18.16.83.0-24",
|
||||
"value": "{\"PublicIP\":\"192.168.122.36\",\"BackendType\":\"vxlan\",\"BackendData\":{\"VtepMAC\":\"ca:38:78:fc:72:29\"}}"
|
||||
},
|
||||
{
|
||||
"key": "/coreos.com/network/subnets/18.16.90.0-24",
|
||||
"value": "{\"PublicIP\":\"192.168.122.127\",\"BackendType\":\"vxlan\",\"BackendData\":{\"VtepMAC\":\"92:e2:80:ba:2d:4d\"}}"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
From all nodes, review the `/run/flannel/subnet.env` file. This file was generated automatically by flannel.
|
||||
|
||||
```shell
|
||||
# cat /run/flannel/subnet.env
|
||||
FLANNEL_SUBNET=18.16.29.1/24
|
||||
FLANNEL_MTU=1450
|
||||
FLANNEL_IPMASQ=false
|
||||
```
|
||||
|
||||
At this point, we have etcd running on the Kubernetes master, and flannel / docker running on Kubernetes nodes. Next steps are for testing cross-host container communication which will confirm that docker and flannel are configured properly.
|
||||
|
||||
Issue the following commands on any 2 nodes:
|
||||
|
||||
```shell
|
||||
# docker run -it fedora:latest bash
|
||||
bash-4.3#
|
||||
```
|
||||
|
||||
This will place you inside the container. Install iproute and iputils packages to install ip and ping utilities. Due to a [bug](https://bugzilla.redhat.com/show_bug.cgi?id=1142311), it is required to modify capabilities of ping binary to work around "Operation not permitted" error.
|
||||
|
||||
```shell
|
||||
bash-4.3# yum -y install iproute iputils
|
||||
bash-4.3# setcap cap_net_raw-ep /usr/bin/ping
|
||||
```
|
||||
|
||||
Now note the IP address on the first node:
|
||||
|
||||
```shell
|
||||
bash-4.3# ip -4 a l eth0 | grep inet
|
||||
inet 18.16.29.4/24 scope global eth0
|
||||
```
|
||||
|
||||
And also note the IP address on the other node:
|
||||
|
||||
```shell
|
||||
bash-4.3# ip a l eth0 | grep inet
|
||||
inet 18.16.90.4/24 scope global eth0
|
||||
```
|
||||
Now ping from the first node to the other node:
|
||||
|
||||
```shell
|
||||
bash-4.3# ping 18.16.90.4
|
||||
PING 18.16.90.4 (18.16.90.4) 56(84) bytes of data.
|
||||
64 bytes from 18.16.90.4: icmp_seq=1 ttl=62 time=0.275 ms
|
||||
64 bytes from 18.16.90.4: icmp_seq=2 ttl=62 time=0.372 ms
|
||||
```
|
||||
|
||||
Now Kubernetes multi-node cluster is set up with overlay networking set up by flannel.
|
||||
|
||||
## Support Level
|
||||
|
||||
|
||||
IaaS Provider | Config. Mgmt | OS | Networking | Docs | Conforms | Support Level
|
||||
-------------------- | ------------ | ------ | ---------- | --------------------------------------------- | ---------| ----------------------------
|
||||
Bare-metal | custom | Fedora | flannel | [docs](/docs/getting-started-guides/fedora/flannel_multi_node_cluster) | | Community ([@aveshagarwal](https://github.com/aveshagarwal))
|
||||
libvirt | custom | Fedora | flannel | [docs](/docs/getting-started-guides/fedora/flannel_multi_node_cluster) | | Community ([@aveshagarwal](https://github.com/aveshagarwal))
|
||||
KVM | custom | Fedora | flannel | [docs](/docs/getting-started-guides/fedora/flannel_multi_node_cluster) | | Community ([@aveshagarwal](https://github.com/aveshagarwal))
|
||||
|
||||
|
||||
|
||||
For support level information on all solutions, see the [Table of solutions](/docs/getting-started-guides/#table-of-solutions) chart.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user