Initial checkin of v1.1 -- does not build
This commit is contained in:
John Mulhausen
@@ -0,0 +1,386 @@
|
||||
---
|
||||
layout: docwithnav
|
||||
title: "Running Kubernetes with Calico Networking on a Digital Ocean Fedora Host"
|
||||
---
|
||||
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
|
||||
|
||||
|
||||
<!-- END MUNGE: UNVERSIONED_WARNING -->
|
||||
|
||||
Running Kubernetes with [Calico Networking](http://projectcalico.org) on a [Digital Ocean](http://digitalocean.com) [Fedora Host](http://fedoraproject.org)
|
||||
-----------------------------------------------------
|
||||
|
||||
## Table of Contents
|
||||
|
||||
* [Prerequisites](#prerequisites)
|
||||
* [Overview](#overview)
|
||||
* [Setup Communication Between Hosts](#setup-communication-between-hosts)
|
||||
* [Setup Master](#setup-master)
|
||||
* [Install etcd](#install-etcd)
|
||||
* [Install Kubernetes](#install-kubernetes)
|
||||
* [Install Calico](#install-calico)
|
||||
* [Setup Node](#setup-node)
|
||||
* [Configure the Virtual Interface - cbr0](#configure-the-virtual-interface---cbr0)
|
||||
* [Install Docker](#install-docker)
|
||||
* [Install Calico](#install-calico-1)
|
||||
* [Install Kubernetes](#install-kubernetes-1)
|
||||
* [Check Running Cluster](#check-running-cluster)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
You need two or more Fedora 22 droplets on Digital Ocean with [Private Networking](https://www.digitalocean.com/community/tutorials/how-to-set-up-and-use-digitalocean-private-networking) enabled.
|
||||
|
||||
## Overview
|
||||
|
||||
This guide will walk you through the process of getting a Kubernetes Fedora cluster running on Digital Ocean with networking powered by Calico networking. It will cover the installation and configuration of the following systemd processes on the following hosts:
|
||||
|
||||
Kubernetes Master:
|
||||
- `kube-apiserver`
|
||||
- `kube-controller-manager`
|
||||
- `kube-scheduler`
|
||||
- `etcd`
|
||||
- `docker`
|
||||
- `calico-node`
|
||||
|
||||
Kubernetes Node:
|
||||
- `kubelet`
|
||||
- `kube-proxy`
|
||||
- `docker`
|
||||
- `calico-node`
|
||||
|
||||
For this demo, we will be setting up one Master and one Node with the following information:
|
||||
|
||||
| Hostname | IP |
|
||||
|-------------|-------------|
|
||||
| kube-master |10.134.251.56|
|
||||
| kube-node-1 |10.134.251.55|
|
||||
|
||||
This guide is scalable to multiple nodes provided you [configure interface-cbr0 with its own subnet on each Node](#configure-the-virtual-interface---cbr0) and [add an entry to /etc/hosts for each host](#setup-communication-between-hosts).
|
||||
|
||||
Ensure you substitute the IP Addresses and Hostnames used in this guide with ones in your own setup.
|
||||
|
||||
## Setup Communication Between Hosts
|
||||
|
||||
Digital Ocean private networking configures a private network on eth1 for each host. To simplify communication between the hosts, we will add an entry to /etc/hosts so that all hosts in the cluster can hostname-resolve one another to this interface. **It is important that the hostname resolves to this interface instead of eth0, as all Kubernetes and Calico services will be running on it.**
|
||||
|
||||
```
|
||||
{% raw %}
|
||||
echo "10.134.251.56 kube-master" >> /etc/hosts
|
||||
echo "10.134.251.55 kube-node-1" >> /etc/hosts
|
||||
{% endraw %}
|
||||
```
|
||||
|
||||
>Make sure that communication works between kube-master and each kube-node by using a utility such as ping.
|
||||
|
||||
## Setup Master
|
||||
|
||||
### Install etcd
|
||||
|
||||
* Both Calico and Kubernetes use etcd as their datastore. We will run etcd on Master and point all Kubernetes and Calico services at it.
|
||||
|
||||
```
|
||||
{% raw %}
|
||||
yum -y install etcd
|
||||
{% endraw %}
|
||||
```
|
||||
|
||||
* Edit `/etc/etcd/etcd.conf`
|
||||
|
||||
```
|
||||
{% raw %}
|
||||
ETCD_LISTEN_CLIENT_URLS="http://kube-master:4001"
|
||||
|
||||
ETCD_ADVERTISE_CLIENT_URLS="http://kube-master:4001"
|
||||
{% endraw %}
|
||||
```
|
||||
|
||||
### Install Kubernetes
|
||||
|
||||
* Run the following command on Master to install the latest Kubernetes (as well as docker):
|
||||
|
||||
```
|
||||
{% raw %}
|
||||
yum -y install kubernetes
|
||||
{% endraw %}
|
||||
```
|
||||
|
||||
* Edit `/etc/kubernetes/config `
|
||||
|
||||
```
|
||||
{% raw %}
|
||||
# How the controller-manager, scheduler, and proxy find the apiserver
|
||||
KUBE_MASTER="--master=http://kube-master:8080"
|
||||
{% endraw %}
|
||||
```
|
||||
|
||||
* Edit `/etc/kubernetes/apiserver`
|
||||
|
||||
```
|
||||
{% raw %}
|
||||
# The address on the local server to listen to.
|
||||
KUBE_API_ADDRESS="--insecure-bind-address=0.0.0.0"
|
||||
|
||||
KUBE_ETCD_SERVERS="--etcd-servers=http://kube-master:4001"
|
||||
|
||||
# Remove ServiceAccount from this line to run without API Tokens
|
||||
KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ResourceQuota"
|
||||
{% endraw %}
|
||||
```
|
||||
|
||||
* Create /var/run/kubernetes on master:
|
||||
|
||||
```
|
||||
{% raw %}
|
||||
mkdir /var/run/kubernetes
|
||||
chown kube:kube /var/run/kubernetes
|
||||
chmod 750 /var/run/kubernetes
|
||||
{% endraw %}
|
||||
```
|
||||
|
||||
* Start the appropriate services on master:
|
||||
|
||||
```
|
||||
{% raw %}
|
||||
for SERVICE in etcd kube-apiserver kube-controller-manager kube-scheduler; do
|
||||
systemctl restart $SERVICE
|
||||
systemctl enable $SERVICE
|
||||
systemctl status $SERVICE
|
||||
done
|
||||
{% endraw %}
|
||||
```
|
||||
|
||||
### Install Calico
|
||||
|
||||
Next, we'll launch Calico on Master to allow communication between Pods and any services running on the Master.
|
||||
* Install calicoctl, the calico configuration tool.
|
||||
|
||||
```
|
||||
{% raw %}
|
||||
wget https://github.com/Metaswitch/calico-docker/releases/download/v0.5.5/calicoctl
|
||||
chmod +x ./calicoctl
|
||||
sudo mv ./calicoctl /usr/bin
|
||||
{% endraw %}
|
||||
```
|
||||
|
||||
* Create `/etc/systemd/system/calico-node.service`
|
||||
|
||||
```
|
||||
{% raw %}
|
||||
[Unit]
|
||||
Description=calicoctl node
|
||||
Requires=docker.service
|
||||
After=docker.service
|
||||
|
||||
[Service]
|
||||
User=root
|
||||
Environment="ETCD_AUTHORITY=kube-master:4001"
|
||||
PermissionsStartOnly=true
|
||||
ExecStartPre=/usr/bin/calicoctl checksystem --fix
|
||||
ExecStart=/usr/bin/calicoctl node --ip=10.134.251.56 --detach=false
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
{% endraw %}
|
||||
```
|
||||
|
||||
>Be sure to substitute `--ip=10.134.251.56` with your Master's eth1 IP Address.
|
||||
|
||||
* Start Calico
|
||||
|
||||
```
|
||||
{% raw %}
|
||||
systemctl enable calico-node.service
|
||||
systemctl start calico-node.service
|
||||
{% endraw %}
|
||||
```
|
||||
|
||||
>Starting calico for the first time may take a few minutes as the calico-node docker image is downloaded.
|
||||
|
||||
## Setup Node
|
||||
|
||||
### Configure the Virtual Interface - cbr0
|
||||
|
||||
By default, docker will create and run on a virtual interface called `docker0`. This interface is automatically assigned the address range 172.17.42.1/16. In order to set our own address range, we will create a new virtual interface called `cbr0` and then start docker on it.
|
||||
|
||||
* Add a virtual interface by creating `/etc/sysconfig/network-scripts/ifcfg-cbr0`:
|
||||
|
||||
```
|
||||
{% raw %}
|
||||
DEVICE=cbr0
|
||||
TYPE=Bridge
|
||||
IPADDR=192.168.1.1
|
||||
NETMASK=255.255.255.0
|
||||
ONBOOT=yes
|
||||
BOOTPROTO=static
|
||||
{% endraw %}
|
||||
```
|
||||
|
||||
>**Note for Multi-Node Clusters:** Each node should be assigned an IP address on a unique subnet. In this example, node-1 is using 192.168.1.1/24, so node-2 should be assigned another pool on the 192.168.x.0/24 subnet, e.g. 192.168.2.1/24.
|
||||
|
||||
* Ensure that your system has bridge-utils installed. Then, restart the networking daemon to activate the new interface
|
||||
|
||||
```
|
||||
{% raw %}
|
||||
systemctl restart network.service
|
||||
{% endraw %}
|
||||
```
|
||||
|
||||
### Install Docker
|
||||
|
||||
* Install Docker
|
||||
|
||||
```
|
||||
{% raw %}
|
||||
yum -y install docker
|
||||
{% endraw %}
|
||||
```
|
||||
|
||||
* Configure docker to run on `cbr0` by editing `/etc/sysconfig/docker-network`:
|
||||
|
||||
```
|
||||
{% raw %}
|
||||
DOCKER_NETWORK_OPTIONS="--bridge=cbr0 --iptables=false --ip-masq=false"
|
||||
{% endraw %}
|
||||
```
|
||||
|
||||
* Start docker
|
||||
|
||||
```
|
||||
{% raw %}
|
||||
systemctl start docker
|
||||
{% endraw %}
|
||||
```
|
||||
|
||||
### Install Calico
|
||||
|
||||
* Install calicoctl, the calico configuration tool.
|
||||
|
||||
```
|
||||
{% raw %}
|
||||
wget https://github.com/Metaswitch/calico-docker/releases/download/v0.5.5/calicoctl
|
||||
chmod +x ./calicoctl
|
||||
sudo mv ./calicoctl /usr/bin
|
||||
{% endraw %}
|
||||
```
|
||||
|
||||
* Create `/etc/systemd/system/calico-node.service`
|
||||
|
||||
```
|
||||
{% raw %}
|
||||
[Unit]
|
||||
Description=calicoctl node
|
||||
Requires=docker.service
|
||||
After=docker.service
|
||||
|
||||
[Service]
|
||||
User=root
|
||||
Environment="ETCD_AUTHORITY=kube-master:4001"
|
||||
PermissionsStartOnly=true
|
||||
ExecStartPre=/usr/bin/calicoctl checksystem --fix
|
||||
ExecStart=/usr/bin/calicoctl node --ip=10.134.251.55 --detach=false --kubernetes
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
{% endraw %}
|
||||
```
|
||||
|
||||
> Note: You must replace the IP address with your node's eth1 IP Address!
|
||||
|
||||
* Start Calico
|
||||
|
||||
```
|
||||
{% raw %}
|
||||
systemctl enable calico-node.service
|
||||
systemctl start calico-node.service
|
||||
{% endraw %}
|
||||
```
|
||||
|
||||
* Configure the IP Address Pool
|
||||
|
||||
Most Kubernetes application deployments will require communication between Pods and the kube-apiserver on Master. On a standard Digital Ocean Private Network, requests sent from Pods to the kube-apiserver will not be returned as the networking fabric will drop response packets destined for any 192.168.0.0/16 address. To resolve this, you can have calicoctl add a masquerade rule to all outgoing traffic on the node:
|
||||
|
||||
```
|
||||
{% raw %}
|
||||
ETCD_AUTHORITY=kube-master:4001 calicoctl pool add 192.168.0.0/16 --nat-outgoing
|
||||
{% endraw %}
|
||||
```
|
||||
|
||||
### Install Kubernetes
|
||||
|
||||
* First, install Kubernetes.
|
||||
|
||||
```
|
||||
{% raw %}
|
||||
yum -y install kubernetes
|
||||
{% endraw %}
|
||||
```
|
||||
|
||||
* Edit `/etc/kubernetes/config`
|
||||
|
||||
```
|
||||
{% raw %}
|
||||
# How the controller-manager, scheduler, and proxy find the apiserver
|
||||
KUBE_MASTER="--master=http://kube-master:8080"
|
||||
{% endraw %}
|
||||
```
|
||||
|
||||
* Edit `/etc/kubernetes/kubelet`
|
||||
|
||||
We'll pass in an extra parameter - `--network-plugin=calico` to tell the Kubelet to use the Calico networking plugin. Additionally, we'll add two environment variables that will be used by the Calico networking plugin.
|
||||
|
||||
```
|
||||
{% raw %}
|
||||
# The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces)
|
||||
KUBELET_ADDRESS="--address=0.0.0.0"
|
||||
|
||||
# You may leave this blank to use the actual hostname
|
||||
# KUBELET_HOSTNAME="--hostname-override=127.0.0.1"
|
||||
|
||||
# location of the api-server
|
||||
KUBELET_API_SERVER="--api-servers=http://kube-master:8080"
|
||||
|
||||
# Add your own!
|
||||
KUBELET_ARGS="--network-plugin=calico"
|
||||
|
||||
# The following are variables which the kubelet will pass to the calico-networking plugin
|
||||
ETCD_AUTHORITY="kube-master:4001"
|
||||
KUBE_API_ROOT="http://kube-master:8080/api/v1"
|
||||
{% endraw %}
|
||||
```
|
||||
|
||||
* Start Kubernetes on the node.
|
||||
|
||||
```
|
||||
{% raw %}
|
||||
for SERVICE in kube-proxy kubelet; do
|
||||
systemctl restart $SERVICE
|
||||
systemctl enable $SERVICE
|
||||
systemctl status $SERVICE
|
||||
done
|
||||
{% endraw %}
|
||||
```
|
||||
|
||||
## Check Running Cluster
|
||||
|
||||
The cluster should be running! Check that your nodes are reporting as such:
|
||||
|
||||
```
|
||||
{% raw %}
|
||||
kubectl get nodes
|
||||
NAME LABELS STATUS
|
||||
kube-node-1 kubernetes.io/hostname=kube-node-1 Ready
|
||||
{% endraw %}
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
<!-- BEGIN MUNGE: IS_VERSIONED -->
|
||||
<!-- TAG IS_VERSIONED -->
|
||||
<!-- END MUNGE: IS_VERSIONED -->
|
||||
|
||||
|
||||
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
|
||||
[]()
|
||||
<!-- END MUNGE: GENERATED_ANALYTICS -->
|
||||
|
||||
@@ -0,0 +1,293 @@
|
||||
---
|
||||
layout: docwithnav
|
||||
title: "Configuring Kubernetes on Fedora via Ansible"
|
||||
---
|
||||
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
|
||||
|
||||
|
||||
<!-- END MUNGE: UNVERSIONED_WARNING -->
|
||||
Configuring Kubernetes on [Fedora](http://fedoraproject.org) via [Ansible](http://www.ansible.com/home)
|
||||
-------------------------------------------------------------------------------------------------------
|
||||
|
||||
Configuring Kubernetes on Fedora via Ansible offers a simple way to quickly create a clustered environment with little effort.
|
||||
|
||||
**Table of Contents**
|
||||
|
||||
- [Prerequisites](#prerequisites)
|
||||
- [Architecture of the cluster](#architecture-of-the-cluster)
|
||||
- [Setting up ansible access to your nodes](#setting-up-ansible-access-to-your-nodes)
|
||||
- [Setting up the cluster](#setting-up-the-cluster)
|
||||
- [Testing and using your new cluster](#testing-and-using-your-new-cluster)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. Host able to run ansible and able to clone the following repo: [kubernetes](https://github.com/kubernetes/kubernetes.git)
|
||||
2. A Fedora 21+ host to act as cluster master
|
||||
3. As many Fedora 21+ hosts as you would like, that act as cluster nodes
|
||||
|
||||
The hosts can be virtual or bare metal. Ansible will take care of the rest of the configuration for you - configuring networking, installing packages, handling the firewall, etc. This example will use one master and two nodes.
|
||||
|
||||
## Architecture of the cluster
|
||||
|
||||
A Kubernetes cluster requires etcd, a master, and n nodes, so we will create a cluster with three hosts, for example:
|
||||
|
||||
{% highlight console %}
|
||||
{% raw %}
|
||||
master,etcd = kube-master.example.com
|
||||
node1 = kube-node-01.example.com
|
||||
node2 = kube-node-02.example.com
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
**Make sure your local machine has**
|
||||
|
||||
- ansible (must be 1.9.0+)
|
||||
- git
|
||||
- python-netaddr
|
||||
|
||||
If not
|
||||
|
||||
{% highlight sh %}
|
||||
{% raw %}
|
||||
yum install -y ansible git python-netaddr
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
**Now clone down the Kubernetes repository**
|
||||
|
||||
{% highlight sh %}
|
||||
{% raw %}
|
||||
git clone https://github.com/kubernetes/contrib.git
|
||||
cd contrib/ansible
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
**Tell ansible about each machine and its role in your cluster**
|
||||
|
||||
Get the IP addresses from the master and nodes. Add those to the `~/contrib/ansible/inventory` file on the host running Ansible.
|
||||
|
||||
{% highlight console %}
|
||||
{% raw %}
|
||||
[masters]
|
||||
kube-master.example.com
|
||||
|
||||
[etcd]
|
||||
kube-master.example.com
|
||||
|
||||
[nodes]
|
||||
kube-node-01.example.com
|
||||
kube-node-02.example.com
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
## Setting up ansible access to your nodes
|
||||
|
||||
If you already are running on a machine which has passwordless ssh access to the kube-master and kube-node-{01,02} nodes, and 'sudo' privileges, simply set the value of `ansible_ssh_user` in `~/contrib/ansible/group_vars/all.yaml` to the username which you use to ssh to the nodes (i.e. `fedora`), and proceed to the next step...
|
||||
|
||||
*Otherwise* setup ssh on the machines like so (you will need to know the root password to all machines in the cluster).
|
||||
|
||||
edit: ~/contrib/ansible/group_vars/all.yml
|
||||
|
||||
{% highlight yaml %}
|
||||
{% raw %}
|
||||
ansible_ssh_user: root
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
**Configuring ssh access to the cluster**
|
||||
|
||||
If you already have ssh access to every machine using ssh public keys you may skip to [setting up the cluster](#setting-up-the-cluster)
|
||||
|
||||
Make sure your local machine (root) has an ssh key pair if not
|
||||
|
||||
{% highlight sh %}
|
||||
{% raw %}
|
||||
ssh-keygen
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
Copy the ssh public key to **all** nodes in the cluster
|
||||
|
||||
{% highlight sh %}
|
||||
{% raw %}
|
||||
for node in kube-master.example.com kube-node-01.example.com kube-node-02.example.com; do
|
||||
ssh-copy-id ${node}
|
||||
done
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
## Setting up the cluster
|
||||
|
||||
Although the default value of variables in `~/contrib/ansible/group_vars/all.yml` should be good enough, if not, change them as needed.
|
||||
|
||||
edit: ~/contrib/ansible/group_vars/all.yml
|
||||
|
||||
**Configure access to kubernetes packages**
|
||||
|
||||
Modify `source_type` as below to access kubernetes packages through the package manager.
|
||||
|
||||
{% highlight yaml %}
|
||||
{% raw %}
|
||||
source_type: packageManager
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
**Configure the IP addresses used for services**
|
||||
|
||||
Each Kubernetes service gets its own IP address. These are not real IPs. You need only select a range of IPs which are not in use elsewhere in your environment.
|
||||
|
||||
{% highlight yaml %}
|
||||
{% raw %}
|
||||
kube_service_addresses: 10.254.0.0/16
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
**Managing flannel**
|
||||
|
||||
Modify `flannel_subnet`, `flannel_prefix` and `flannel_host_prefix` only if defaults are not appropriate for your cluster.
|
||||
|
||||
|
||||
**Managing add on services in your cluster**
|
||||
|
||||
Set `cluster_logging` to false or true (default) to disable or enable logging with elasticsearch.
|
||||
|
||||
{% highlight yaml %}
|
||||
{% raw %}
|
||||
cluster_logging: true
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
Turn `cluster_monitoring` to true (default) or false to enable or disable cluster monitoring with heapster and influxdb.
|
||||
|
||||
{% highlight yaml %}
|
||||
{% raw %}
|
||||
cluster_monitoring: true
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
Turn `dns_setup` to true (recommended) or false to enable or disable whole DNS configuration.
|
||||
|
||||
{% highlight yaml %}
|
||||
{% raw %}
|
||||
dns_setup: true
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
**Tell ansible to get to work!**
|
||||
|
||||
This will finally setup your whole Kubernetes cluster for you.
|
||||
|
||||
{% highlight sh %}
|
||||
{% raw %}
|
||||
cd ~/contrib/ansible/
|
||||
|
||||
./setup.sh
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
## Testing and using your new cluster
|
||||
|
||||
That's all there is to it. It's really that easy. At this point you should have a functioning Kubernetes cluster.
|
||||
|
||||
**Show kubernetes nodes**
|
||||
|
||||
Run the following on the kube-master:
|
||||
|
||||
{% highlight sh %}
|
||||
{% raw %}
|
||||
kubectl get nodes
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
**Show services running on masters and nodes**
|
||||
|
||||
{% highlight sh %}
|
||||
{% raw %}
|
||||
systemctl | grep -i kube
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
**Show firewall rules on the masters and nodes**
|
||||
|
||||
{% highlight sh %}
|
||||
{% raw %}
|
||||
iptables -nvL
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
**Create /tmp/apache.json on the master with the following contents and deploy pod**
|
||||
|
||||
{% highlight json %}
|
||||
{% raw %}
|
||||
{
|
||||
"kind": "Pod",
|
||||
"apiVersion": "v1",
|
||||
"metadata": {
|
||||
"name": "fedoraapache",
|
||||
"labels": {
|
||||
"name": "fedoraapache"
|
||||
}
|
||||
},
|
||||
"spec": {
|
||||
"containers": [
|
||||
{
|
||||
"name": "fedoraapache",
|
||||
"image": "fedora/apache",
|
||||
"ports": [
|
||||
{
|
||||
"hostPort": 80,
|
||||
"containerPort": 80
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
{% highlight sh %}
|
||||
{% raw %}
|
||||
kubectl create -f /tmp/apache.json
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
**Check where the pod was created**
|
||||
|
||||
{% highlight sh %}
|
||||
{% raw %}
|
||||
kubectl get pods
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
**Check Docker status on nodes**
|
||||
|
||||
{% highlight sh %}
|
||||
{% raw %}
|
||||
docker ps
|
||||
docker images
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
**After the pod is 'Running' Check web server access on the node**
|
||||
|
||||
{% highlight sh %}
|
||||
{% raw %}
|
||||
curl http://localhost
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
That's it !
|
||||
|
||||
|
||||
|
||||
|
||||
<!-- BEGIN MUNGE: IS_VERSIONED -->
|
||||
<!-- TAG IS_VERSIONED -->
|
||||
<!-- END MUNGE: IS_VERSIONED -->
|
||||
|
||||
|
||||
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
|
||||
[]()
|
||||
<!-- END MUNGE: GENERATED_ANALYTICS -->
|
||||
|
||||
@@ -0,0 +1,258 @@
|
||||
---
|
||||
layout: docwithnav
|
||||
title: "Getting started on Fedora"
|
||||
---
|
||||
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
|
||||
|
||||
|
||||
<!-- END MUNGE: UNVERSIONED_WARNING -->
|
||||
Getting started on [Fedora](http://fedoraproject.org)
|
||||
-----------------------------------------------------
|
||||
|
||||
**Table of Contents**
|
||||
|
||||
- [Prerequisites](#prerequisites)
|
||||
- [Instructions](#instructions)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. You need 2 or more machines with Fedora installed.
|
||||
|
||||
## Instructions
|
||||
|
||||
This is a getting started guide for Fedora. It is a manual configuration so you understand all the underlying packages / services / ports, etc...
|
||||
|
||||
This guide will only get ONE node (previously minion) working. Multiple nodes require a functional [networking configuration](../../admin/networking.html) done outside of Kubernetes. Although the additional Kubernetes configuration requirements should be obvious.
|
||||
|
||||
The Kubernetes package provides a few services: kube-apiserver, kube-scheduler, kube-controller-manager, kubelet, kube-proxy. These services are managed by systemd and the configuration resides in a central location: /etc/kubernetes. We will break the services up between the hosts. The first host, fed-master, will be the Kubernetes master. This host will run the kube-apiserver, kube-controller-manager, and kube-scheduler. In addition, the master will also run _etcd_ (not needed if _etcd_ runs on a different host but this guide assumes that _etcd_ and Kubernetes master run on the same host). The remaining host, fed-node will be the node and run kubelet, proxy and docker.
|
||||
|
||||
**System Information:**
|
||||
|
||||
Hosts:
|
||||
|
||||
```
|
||||
{% raw %}
|
||||
fed-master = 192.168.121.9
|
||||
fed-node = 192.168.121.65
|
||||
{% endraw %}
|
||||
```
|
||||
|
||||
**Prepare the hosts:**
|
||||
|
||||
* Install Kubernetes on all hosts - fed-{master,node}. This will also pull in docker. Also install etcd on fed-master. This guide has been tested with kubernetes-0.18 and beyond.
|
||||
* The [--enablerepo=updates-testing](https://fedoraproject.org/wiki/QA:Updates_Testing) directive in the yum command below will ensure that the most recent Kubernetes version that is scheduled for pre-release will be installed. This should be a more recent version than the Fedora "stable" release for Kubernetes that you would get without adding the directive.
|
||||
* If you want the very latest Kubernetes release [you can download and yum install the RPM directly from Fedora Koji](http://koji.fedoraproject.org/koji/packageinfo?packageID=19202) instead of using the yum install command below.
|
||||
|
||||
{% highlight sh %}
|
||||
{% raw %}
|
||||
yum -y install --enablerepo=updates-testing kubernetes
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
* Install etcd and iptables
|
||||
|
||||
{% highlight sh %}
|
||||
{% raw %}
|
||||
yum -y install etcd iptables
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
* Add master and node to /etc/hosts on all machines (not needed if hostnames already in DNS). Make sure that communication works between fed-master and fed-node by using a utility such as ping.
|
||||
|
||||
{% highlight sh %}
|
||||
{% raw %}
|
||||
echo "192.168.121.9 fed-master
|
||||
192.168.121.65 fed-node" >> /etc/hosts
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
* Edit /etc/kubernetes/config which will be the same on all hosts (master and node) to contain:
|
||||
|
||||
{% highlight sh %}
|
||||
{% raw %}
|
||||
# Comma separated list of nodes in the etcd cluster
|
||||
KUBE_MASTER="--master=http://fed-master:8080"
|
||||
|
||||
# logging to stderr means we get it in the systemd journal
|
||||
KUBE_LOGTOSTDERR="--logtostderr=true"
|
||||
|
||||
# journal message level, 0 is debug
|
||||
KUBE_LOG_LEVEL="--v=0"
|
||||
|
||||
# Should this cluster be allowed to run privileged docker containers
|
||||
KUBE_ALLOW_PRIV="--allow-privileged=false"
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
* Disable the firewall on both the master and node, as docker does not play well with other firewall rule managers. Please note that iptables-services does not exist on default fedora server install.
|
||||
|
||||
{% highlight sh %}
|
||||
{% raw %}
|
||||
systemctl disable iptables-services firewalld
|
||||
systemctl stop iptables-services firewalld
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
**Configure the Kubernetes services on the master.**
|
||||
|
||||
* Edit /etc/kubernetes/apiserver to appear as such. The service-cluster-ip-range IP addresses must be an unused block of addresses, not used anywhere else. They do not need to be routed or assigned to anything.
|
||||
|
||||
{% highlight sh %}
|
||||
{% raw %}
|
||||
# The address on the local server to listen to.
|
||||
KUBE_API_ADDRESS="--address=0.0.0.0"
|
||||
|
||||
# Comma separated list of nodes in the etcd cluster
|
||||
KUBE_ETCD_SERVERS="--etcd-servers=http://127.0.0.1:4001"
|
||||
|
||||
# Address range to use for services
|
||||
KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16"
|
||||
|
||||
# Add your own!
|
||||
KUBE_API_ARGS=""
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
* Edit /etc/etcd/etcd.conf,let the etcd to listen all the ip instead of 127.0.0.1, if not, you will get the error like "connection refused". Note that Fedora 22 uses etcd 2.0, One of the changes in etcd 2.0 is that now uses port 2379 and 2380 (as opposed to etcd 0.46 which userd 4001 and 7001).
|
||||
|
||||
{% highlight sh %}
|
||||
{% raw %}
|
||||
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:4001"
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
* Create /var/run/kubernetes on master:
|
||||
|
||||
{% highlight sh %}
|
||||
{% raw %}
|
||||
mkdir /var/run/kubernetes
|
||||
chown kube:kube /var/run/kubernetes
|
||||
chmod 750 /var/run/kubernetes
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
* Start the appropriate services on master:
|
||||
|
||||
{% highlight sh %}
|
||||
{% raw %}
|
||||
for SERVICES in etcd kube-apiserver kube-controller-manager kube-scheduler; do
|
||||
systemctl restart $SERVICES
|
||||
systemctl enable $SERVICES
|
||||
systemctl status $SERVICES
|
||||
done
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
* Addition of nodes:
|
||||
|
||||
* Create following node.json file on Kubernetes master node:
|
||||
|
||||
{% highlight json %}
|
||||
{% raw %}
|
||||
{
|
||||
"apiVersion": "v1",
|
||||
"kind": "Node",
|
||||
"metadata": {
|
||||
"name": "fed-node",
|
||||
"labels":{ "name": "fed-node-label"}
|
||||
},
|
||||
"spec": {
|
||||
"externalID": "fed-node"
|
||||
}
|
||||
}
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
Now create a node object internally in your Kubernetes cluster by running:
|
||||
|
||||
{% highlight console %}
|
||||
{% raw %}
|
||||
$ kubectl create -f ./node.json
|
||||
|
||||
$ kubectl get nodes
|
||||
NAME LABELS STATUS
|
||||
fed-node name=fed-node-label Unknown
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
Please note that in the above, it only creates a representation for the node
|
||||
_fed-node_ internally. It does not provision the actual _fed-node_. Also, it
|
||||
is assumed that _fed-node_ (as specified in `name`) can be resolved and is
|
||||
reachable from Kubernetes master node. This guide will discuss how to provision
|
||||
a Kubernetes node (fed-node) below.
|
||||
|
||||
**Configure the Kubernetes services on the node.**
|
||||
|
||||
***We need to configure the kubelet on the node.***
|
||||
|
||||
* Edit /etc/kubernetes/kubelet to appear as such:
|
||||
|
||||
{% highlight sh %}
|
||||
{% raw %}
|
||||
###
|
||||
# Kubernetes kubelet (node) config
|
||||
|
||||
# The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces)
|
||||
KUBELET_ADDRESS="--address=0.0.0.0"
|
||||
|
||||
# You may leave this blank to use the actual hostname
|
||||
KUBELET_HOSTNAME="--hostname-override=fed-node"
|
||||
|
||||
# location of the api-server
|
||||
KUBELET_API_SERVER="--api-servers=http://fed-master:8080"
|
||||
|
||||
# Add your own!
|
||||
#KUBELET_ARGS=""
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
* Start the appropriate services on the node (fed-node).
|
||||
|
||||
{% highlight sh %}
|
||||
{% raw %}
|
||||
for SERVICES in kube-proxy kubelet docker; do
|
||||
systemctl restart $SERVICES
|
||||
systemctl enable $SERVICES
|
||||
systemctl status $SERVICES
|
||||
done
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
* Check to make sure now the cluster can see the fed-node on fed-master, and its status changes to _Ready_.
|
||||
|
||||
{% highlight console %}
|
||||
{% raw %}
|
||||
kubectl get nodes
|
||||
NAME LABELS STATUS
|
||||
fed-node name=fed-node-label Ready
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
* Deletion of nodes:
|
||||
|
||||
To delete _fed-node_ from your Kubernetes cluster, one should run the following on fed-master (Please do not do it, it is just for information):
|
||||
|
||||
{% highlight sh %}
|
||||
{% raw %}
|
||||
kubectl delete -f ./node.json
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
*You should be finished!*
|
||||
|
||||
**The cluster should be running! Launch a test pod.**
|
||||
|
||||
You should have a functional cluster, check out [101](../../../docs/user-guide/walkthrough/README.html)!
|
||||
|
||||
|
||||
|
||||
|
||||
<!-- BEGIN MUNGE: IS_VERSIONED -->
|
||||
<!-- TAG IS_VERSIONED -->
|
||||
<!-- END MUNGE: IS_VERSIONED -->
|
||||
|
||||
|
||||
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
|
||||
[]()
|
||||
<!-- END MUNGE: GENERATED_ANALYTICS -->
|
||||
|
||||
@@ -0,0 +1,235 @@
|
||||
---
|
||||
layout: docwithnav
|
||||
title: "Kubernetes multiple nodes cluster with flannel on Fedora"
|
||||
---
|
||||
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
|
||||
|
||||
|
||||
<!-- END MUNGE: UNVERSIONED_WARNING -->
|
||||
Kubernetes multiple nodes cluster with flannel on Fedora
|
||||
--------------------------------------------------------
|
||||
|
||||
**Table of Contents**
|
||||
|
||||
- [Introduction](#introduction)
|
||||
- [Prerequisites](#prerequisites)
|
||||
- [Master Setup](#master-setup)
|
||||
- [Node Setup](#node-setup)
|
||||
- [**Test the cluster and flannel configuration**](#test-the-cluster-and-flannel-configuration)
|
||||
|
||||
## Introduction
|
||||
|
||||
This document describes how to deploy Kubernetes on multiple hosts to set up a multi-node cluster and networking with flannel. Follow fedora [getting started guide](fedora_manual_config.html) to setup 1 master (fed-master) and 2 or more nodes. Make sure that all nodes have different names (fed-node1, fed-node2 and so on) and labels (fed-node1-label, fed-node2-label, and so on) to avoid any conflict. Also make sure that the Kubernetes master host is running etcd, kube-controller-manager, kube-scheduler, and kube-apiserver services, and the nodes are running docker, kube-proxy and kubelet services. Now install flannel on Kubernetes nodes. flannel on each node configures an overlay network that docker uses. flannel runs on each node to setup a unique class-C container network.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. You need 2 or more machines with Fedora installed.
|
||||
|
||||
## Master Setup
|
||||
|
||||
**Perform following commands on the Kubernetes master**
|
||||
|
||||
* Configure flannel by creating a `flannel-config.json` in your current directory on fed-master. flannel provides udp and vxlan among other overlay networking backend options. In this guide, we choose kernel based vxlan backend. The contents of the json are:
|
||||
|
||||
{% highlight json %}
|
||||
{% raw %}
|
||||
{
|
||||
"Network": "18.16.0.0/16",
|
||||
"SubnetLen": 24,
|
||||
"Backend": {
|
||||
"Type": "vxlan",
|
||||
"VNI": 1
|
||||
}
|
||||
}
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
**NOTE:** Choose an IP range that is *NOT* part of the public IP address range.
|
||||
|
||||
* Add the configuration to the etcd server on fed-master.
|
||||
|
||||
{% highlight sh %}
|
||||
{% raw %}
|
||||
etcdctl set /coreos.com/network/config < flannel-config.json
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
* Verify the key exists in the etcd server on fed-master.
|
||||
|
||||
{% highlight sh %}
|
||||
{% raw %}
|
||||
etcdctl get /coreos.com/network/config
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
## Node Setup
|
||||
|
||||
**Perform following commands on all Kubernetes nodes**
|
||||
|
||||
* Edit the flannel configuration file /etc/sysconfig/flanneld as follows:
|
||||
|
||||
{% highlight sh %}
|
||||
{% raw %}
|
||||
# Flanneld configuration options
|
||||
|
||||
# etcd url location. Point this to the server where etcd runs
|
||||
FLANNEL_ETCD="http://fed-master:4001"
|
||||
|
||||
# etcd config key. This is the configuration key that flannel queries
|
||||
# For address range assignment
|
||||
FLANNEL_ETCD_KEY="/coreos.com/network"
|
||||
|
||||
# Any additional options that you want to pass
|
||||
FLANNEL_OPTIONS=""
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
**Note:** By default, flannel uses the interface for the default route. If you have multiple interfaces and would like to use an interface other than the default route one, you could add "-iface=" to FLANNEL_OPTIONS. For additional options, run `flanneld --help` on command line.
|
||||
|
||||
* Enable the flannel service.
|
||||
|
||||
{% highlight sh %}
|
||||
{% raw %}
|
||||
systemctl enable flanneld
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
* If docker is not running, then starting flannel service is enough and skip the next step.
|
||||
|
||||
{% highlight sh %}
|
||||
{% raw %}
|
||||
systemctl start flanneld
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
* If docker is already running, then stop docker, delete docker bridge (docker0), start flanneld and restart docker as follows. Another alternative is to just reboot the system (`systemctl reboot`).
|
||||
|
||||
{% highlight sh %}
|
||||
{% raw %}
|
||||
systemctl stop docker
|
||||
ip link delete docker0
|
||||
systemctl start flanneld
|
||||
systemctl start docker
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
***
|
||||
|
||||
## **Test the cluster and flannel configuration**
|
||||
|
||||
* Now check the interfaces on the nodes. Notice there is now a flannel.1 interface, and the ip addresses of docker0 and flannel.1 interfaces are in the same network. You will notice that docker0 is assigned a subnet (18.16.29.0/24 as shown below) on each Kubernetes node out of the IP range configured above. A working output should look like this:
|
||||
|
||||
{% highlight console %}
|
||||
{% raw %}
|
||||
# ip -4 a|grep inet
|
||||
inet 127.0.0.1/8 scope host lo
|
||||
inet 192.168.122.77/24 brd 192.168.122.255 scope global dynamic eth0
|
||||
inet 18.16.29.0/16 scope global flannel.1
|
||||
inet 18.16.29.1/24 scope global docker0
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
* From any node in the cluster, check the cluster members by issuing a query to etcd server via curl (only partial output is shown using `grep -E "\{|\}|key|value"`). If you set up a 1 master and 3 nodes cluster, you should see one block for each node showing the subnets they have been assigned. You can associate those subnets to each node by the MAC address (VtepMAC) and IP address (Public IP) that is listed in the output.
|
||||
|
||||
{% highlight sh %}
|
||||
{% raw %}
|
||||
curl -s http://fed-master:4001/v2/keys/coreos.com/network/subnets | python -mjson.tool
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
{% highlight json %}
|
||||
{% raw %}
|
||||
{
|
||||
"node": {
|
||||
"key": "/coreos.com/network/subnets",
|
||||
{
|
||||
"key": "/coreos.com/network/subnets/18.16.29.0-24",
|
||||
"value": "{\"PublicIP\":\"192.168.122.77\",\"BackendType\":\"vxlan\",\"BackendData\":{\"VtepMAC\":\"46:f1:d0:18:d0:65\"}}"
|
||||
},
|
||||
{
|
||||
"key": "/coreos.com/network/subnets/18.16.83.0-24",
|
||||
"value": "{\"PublicIP\":\"192.168.122.36\",\"BackendType\":\"vxlan\",\"BackendData\":{\"VtepMAC\":\"ca:38:78:fc:72:29\"}}"
|
||||
},
|
||||
{
|
||||
"key": "/coreos.com/network/subnets/18.16.90.0-24",
|
||||
"value": "{\"PublicIP\":\"192.168.122.127\",\"BackendType\":\"vxlan\",\"BackendData\":{\"VtepMAC\":\"92:e2:80:ba:2d:4d\"}}"
|
||||
}
|
||||
}
|
||||
}
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
* From all nodes, review the `/run/flannel/subnet.env` file. This file was generated automatically by flannel.
|
||||
|
||||
{% highlight console %}
|
||||
{% raw %}
|
||||
# cat /run/flannel/subnet.env
|
||||
FLANNEL_SUBNET=18.16.29.1/24
|
||||
FLANNEL_MTU=1450
|
||||
FLANNEL_IPMASQ=false
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
* At this point, we have etcd running on the Kubernetes master, and flannel / docker running on Kubernetes nodes. Next steps are for testing cross-host container communication which will confirm that docker and flannel are configured properly.
|
||||
|
||||
* Issue the following commands on any 2 nodes:
|
||||
|
||||
{% highlight console %}
|
||||
{% raw %}
|
||||
# docker run -it fedora:latest bash
|
||||
bash-4.3#
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
* This will place you inside the container. Install iproute and iputils packages to install ip and ping utilities. Due to a [bug](https://bugzilla.redhat.com/show_bug.cgi?id=1142311), it is required to modify capabilities of ping binary to work around "Operation not permitted" error.
|
||||
|
||||
{% highlight console %}
|
||||
{% raw %}
|
||||
bash-4.3# yum -y install iproute iputils
|
||||
bash-4.3# setcap cap_net_raw-ep /usr/bin/ping
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
* Now note the IP address on the first node:
|
||||
|
||||
{% highlight console %}
|
||||
{% raw %}
|
||||
bash-4.3# ip -4 a l eth0 | grep inet
|
||||
inet 18.16.29.4/24 scope global eth0
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
* And also note the IP address on the other node:
|
||||
|
||||
{% highlight console %}
|
||||
{% raw %}
|
||||
bash-4.3# ip a l eth0 | grep inet
|
||||
inet 18.16.90.4/24 scope global eth0
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
* Now ping from the first node to the other node:
|
||||
|
||||
{% highlight console %}
|
||||
{% raw %}
|
||||
bash-4.3# ping 18.16.90.4
|
||||
PING 18.16.90.4 (18.16.90.4) 56(84) bytes of data.
|
||||
64 bytes from 18.16.90.4: icmp_seq=1 ttl=62 time=0.275 ms
|
||||
64 bytes from 18.16.90.4: icmp_seq=2 ttl=62 time=0.372 ms
|
||||
{% endraw %}
|
||||
{% endhighlight %}
|
||||
|
||||
* Now Kubernetes multi-node cluster is set up with overlay networking set up by flannel.
|
||||
|
||||
|
||||
|
||||
|
||||
<!-- BEGIN MUNGE: IS_VERSIONED -->
|
||||
<!-- TAG IS_VERSIONED -->
|
||||
<!-- END MUNGE: IS_VERSIONED -->
|
||||
|
||||
|
||||
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
|
||||
[]()
|
||||
<!-- END MUNGE: GENERATED_ANALYTICS -->
|
||||
|
||||
Reference in New Issue
Block a user