diff --git a/_layouts/docwithnav.html b/_layouts/docwithnav.html index ef7ace3c24..e84481ad6f 100755 --- a/_layouts/docwithnav.html +++ b/_layouts/docwithnav.html @@ -72,6 +72,5 @@ ---> \ No newline at end of file diff --git a/docs/getting-started-guides/aws.md b/docs/getting-started-guides/aws.md index 49d070066d..6ac90d5829 100644 --- a/docs/getting-started-guides/aws.md +++ b/docs/getting-started-guides/aws.md @@ -1,53 +1,53 @@ --- --- - + * TOC -{:toc} - -## Prerequisites - -1. You need an AWS account. Visit [http://aws.amazon.com](http://aws.amazon.com) to get started -2. Install and configure [AWS Command Line Interface](http://aws.amazon.com/cli) -3. You need an AWS [instance profile and role](http://docs.aws.amazon.com/IAM/latest/UserGuide/instance-profiles.html) with EC2 full access. - -NOTE: This script use the 'default' AWS profile by default. -You may explicitly set AWS profile to use using the `AWS_DEFAULT_PROFILE` environment variable: - +{:toc} + +## Prerequisites + +1. You need an AWS account. Visit [http://aws.amazon.com](http://aws.amazon.com) to get started +2. Install and configure [AWS Command Line Interface](http://aws.amazon.com/cli) +3. You need an AWS [instance profile and role](http://docs.aws.amazon.com/IAM/latest/UserGuide/instance-profiles.html) with EC2 full access. + +NOTE: This script use the 'default' AWS profile by default. +You may explicitly set AWS profile to use using the `AWS_DEFAULT_PROFILE` environment variable: + ```shell -export AWS_DEFAULT_PROFILE=myawsprofile +export AWS_DEFAULT_PROFILE=myawsprofile ``` -## Cluster turnup - -### Supported procedure: `get-kube` - +## Cluster turnup + +### Supported procedure: `get-kube` + ```shell -#Using wget -export KUBERNETES_PROVIDER=aws; wget -q -O - https://get.k8s.io | bash -#Using cURL -export KUBERNETES_PROVIDER=aws; curl -sS https://get.k8s.io | bash +#Using wget +export KUBERNETES_PROVIDER=aws; wget -q -O - https://get.k8s.io | bash +#Using cURL +export KUBERNETES_PROVIDER=aws; curl -sS https://get.k8s.io | bash ``` -NOTE: This script calls [cluster/kube-up.sh](http://releases.k8s.io/{{page.githubbranch}}/cluster/kube-up.sh) -which in turn calls [cluster/aws/util.sh](http://releases.k8s.io/{{page.githubbranch}}/cluster/aws/util.sh) -using [cluster/aws/config-default.sh](http://releases.k8s.io/{{page.githubbranch}}/cluster/aws/config-default.sh). - -This process takes about 5 to 10 minutes. Once the cluster is up, the IP addresses of your master and node(s) will be printed, -as well as information about the default services running in the cluster (monitoring, logging, dns). User credentials and security -tokens are written in `~/.kube/config`, they will be necessary to use the CLI or the HTTP Basic Auth. - -By default, the script will provision a new VPC and a 4 node k8s cluster in us-west-2a (Oregon) with EC2 instances running on Ubuntu. -You can override the variables defined in [config-default.sh](http://releases.k8s.io/{{page.githubbranch}}/cluster/aws/config-default.sh) to change this behavior as follows: - +NOTE: This script calls [cluster/kube-up.sh](http://releases.k8s.io/{{page.githubbranch}}/cluster/kube-up.sh) +which in turn calls [cluster/aws/util.sh](http://releases.k8s.io/{{page.githubbranch}}/cluster/aws/util.sh) +using [cluster/aws/config-default.sh](http://releases.k8s.io/{{page.githubbranch}}/cluster/aws/config-default.sh). + +This process takes about 5 to 10 minutes. Once the cluster is up, the IP addresses of your master and node(s) will be printed, +as well as information about the default services running in the cluster (monitoring, logging, dns). User credentials and security +tokens are written in `~/.kube/config`, they will be necessary to use the CLI or the HTTP Basic Auth. + +By default, the script will provision a new VPC and a 4 node k8s cluster in us-west-2a (Oregon) with EC2 instances running on Ubuntu. +You can override the variables defined in [config-default.sh](http://releases.k8s.io/{{page.githubbranch}}/cluster/aws/config-default.sh) to change this behavior as follows: + ```shell -export KUBE_AWS_ZONE=eu-west-1c -export NUM_NODES=2 +export KUBE_AWS_ZONE=eu-west-1c +export NUM_NODES=2 export MASTER_SIZE=m3.medium -export NODE_SIZE=m3.medium -export AWS_S3_REGION=eu-west-1 -export AWS_S3_BUCKET=mycompany-kubernetes-artifacts -export INSTANCE_PREFIX=k8s -... +export NODE_SIZE=m3.medium +export AWS_S3_REGION=eu-west-1 +export AWS_S3_BUCKET=mycompany-kubernetes-artifacts +export INSTANCE_PREFIX=k8s +... ``` If you don't specify master and minion sizes, the scripts will attempt to guess @@ -88,55 +88,55 @@ instance storage, you may want to increase the `NODE_ROOT_DISK_SIZE` value, although the default value of 32 is probably sufficient for the smaller instance types in the m4 family. -The script will also try to create or reuse a keypair called "kubernetes", and IAM profiles called "kubernetes-master" and "kubernetes-minion". -If these already exist, make sure you want them to be used here. - -NOTE: If using an existing keypair named "kubernetes" then you must set the `AWS_SSH_KEY` key to point to your private key. - -### Alternatives - -CoreOS maintains [a CLI tool](https://coreos.com/kubernetes/docs/latest/kubernetes-on-aws.html), `kube-aws` that will create and manage a Kubernetes cluster based on [CoreOS](http://www.coreos.com), using AWS tools: EC2, CloudFormation and Autoscaling. - -## Getting started with your cluster - -### Command line administration tool: `kubectl` - -The cluster startup script will leave you with a `kubernetes` directory on your workstation. -Alternately, you can download the latest Kubernetes release from [this page](https://github.com/kubernetes/kubernetes/releases). - -Next, add the appropriate binary folder to your `PATH` to access kubectl: - -```shell -# OS X -export PATH=/platforms/darwin/amd64:$PATH +The script will also try to create or reuse a keypair called "kubernetes", and IAM profiles called "kubernetes-master" and "kubernetes-minion". +If these already exist, make sure you want them to be used here. -# Linux -export PATH=/platforms/linux/amd64:$PATH +NOTE: If using an existing keypair named "kubernetes" then you must set the `AWS_SSH_KEY` key to point to your private key. + +### Alternatives + +CoreOS maintains [a CLI tool](https://coreos.com/kubernetes/docs/latest/kubernetes-on-aws.html), `kube-aws` that will create and manage a Kubernetes cluster based on [CoreOS](http://www.coreos.com), using AWS tools: EC2, CloudFormation and Autoscaling. + +## Getting started with your cluster + +### Command line administration tool: `kubectl` + +The cluster startup script will leave you with a `kubernetes` directory on your workstation. +Alternately, you can download the latest Kubernetes release from [this page](https://github.com/kubernetes/kubernetes/releases). + +Next, add the appropriate binary folder to your `PATH` to access kubectl: + +```shell +# OS X +export PATH=/platforms/darwin/amd64:$PATH + +# Linux +export PATH=/platforms/linux/amd64:$PATH ``` -An up-to-date documentation page for this tool is available here: [kubectl manual](/docs/user-guide/kubectl/kubectl) - -By default, `kubectl` will use the `kubeconfig` file generated during the cluster startup for authenticating against the API. -For more information, please read [kubeconfig files](/docs/user-guide/kubeconfig-file) - -### Examples - -See [a simple nginx example](/docs/user-guide/simple-nginx) to try out your new cluster. - -The "Guestbook" application is another popular example to get started with Kubernetes: [guestbook example](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/guestbook/) - -For more complete applications, please look in the [examples directory](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/) - -## Tearing down the cluster - -Make sure the environment variables you used to provision your cluster are still exported, then call the following script inside the -`kubernetes` directory: - +An up-to-date documentation page for this tool is available here: [kubectl manual](/docs/user-guide/kubectl/kubectl) + +By default, `kubectl` will use the `kubeconfig` file generated during the cluster startup for authenticating against the API. +For more information, please read [kubeconfig files](/docs/user-guide/kubeconfig-file) + +### Examples + +See [a simple nginx example](/docs/user-guide/simple-nginx) to try out your new cluster. + +The "Guestbook" application is another popular example to get started with Kubernetes: [guestbook example](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/guestbook/) + +For more complete applications, please look in the [examples directory](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/) + +## Tearing down the cluster + +Make sure the environment variables you used to provision your cluster are still exported, then call the following script inside the +`kubernetes` directory: + ```shell -cluster/kube-down.sh +cluster/kube-down.sh ``` -## Further reading - -Please see the [Kubernetes docs](/docs/) for more details on administering +## Further reading + +Please see the [Kubernetes docs](/docs/) for more details on administering and using a Kubernetes cluster. \ No newline at end of file diff --git a/docs/getting-started-guides/dcos.md b/docs/getting-started-guides/dcos.md index d46cb1fb65..310b5fff29 100644 --- a/docs/getting-started-guides/dcos.md +++ b/docs/getting-started-guides/dcos.md @@ -1,130 +1,130 @@ --- --- - -This guide will walk you through installing [Kubernetes-Mesos](https://github.com/mesosphere/kubernetes-mesos) on [Datacenter Operating System (DCOS)](https://mesosphere.com/product/) with the [DCOS CLI](https://github.com/mesosphere/dcos-cli) and operating Kubernetes with the [DCOS Kubectl plugin](https://github.com/mesosphere/dcos-kubectl). - + +This guide will walk you through installing [Kubernetes-Mesos](https://github.com/mesosphere/kubernetes-mesos) on [Datacenter Operating System (DCOS)](https://mesosphere.com/product/) with the [DCOS CLI](https://github.com/mesosphere/dcos-cli) and operating Kubernetes with the [DCOS Kubectl plugin](https://github.com/mesosphere/dcos-kubectl). + * TOC -{:toc} - - -## About Kubernetes on DCOS - -DCOS is system software that manages computer cluster hardware and software resources and provides common services for distributed applications. Among other services, it provides [Apache Mesos](http://mesos.apache.org/) as its cluster kernel and [Marathon](https://mesosphere.github.io/marathon/) as its init system. With DCOS CLI, Mesos frameworks like [Kubernetes-Mesos](https://github.com/mesosphere/kubernetes-mesos) can be installed with a single command. - -Another feature of the DCOS CLI is that it allows plugins like the [DCOS Kubectl plugin](https://github.com/mesosphere/dcos-kubectl). This allows for easy access to a version-compatible Kubectl without having to manually download or install. - -Further information about the benefits of installing Kubernetes on DCOS can be found in the [Kubernetes-Mesos documentation](https://releases.k8s.io/{{page.githubbranch}}/contrib/mesos/README.md). - -For more details about the Kubernetes DCOS packaging, see the [Kubernetes-Mesos project](https://github.com/mesosphere/kubernetes-mesos). - -Since Kubernetes-Mesos is still alpha, it is a good idea to familiarize yourself with the [current known issues](https://releases.k8s.io/{{page.githubbranch}}/contrib/mesos/docs/issues.md) which may limit or modify the behavior of Kubernetes on DCOS. - -If you have problems completing the steps below, please [file an issue against the kubernetes-mesos project](https://github.com/mesosphere/kubernetes-mesos/issues). - - -## Resources - -Explore the following resources for more information about Kubernetes, Kubernetes on Mesos/DCOS, and DCOS itself. - -- [DCOS Documentation](https://docs.mesosphere.com/) -- [Managing DCOS Services](https://docs.mesosphere.com/services/kubernetes/) -- [Kubernetes Examples](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/) -- [Kubernetes on Mesos Documentation](https://releases.k8s.io/{{page.githubbranch}}/contrib/mesos/README.md) -- [Kubernetes on Mesos Release Notes](https://github.com/mesosphere/kubernetes-mesos/releases) -- [Kubernetes on DCOS Package Source](https://github.com/mesosphere/kubernetes-mesos) - - -## Prerequisites - -- A running [DCOS cluster](https://mesosphere.com/product/) - - [DCOS Community Edition](https://docs.mesosphere.com/install/) is currently available on [AWS](https://mesosphere.com/amazon/). - - [DCOS Enterprise Edition](https://mesosphere.com/product/) can be deployed on virtual or bare metal machines. Contact sales@mesosphere.com for more info and to set up an engagement. -- [DCOS CLI](https://docs.mesosphere.com/install/cli/) installed locally - - -## Install - -1. Configure and validate the [Mesosphere Multiverse](https://github.com/mesosphere/multiverse) as a package source repository - +{:toc} + + +## About Kubernetes on DCOS + +DCOS is system software that manages computer cluster hardware and software resources and provides common services for distributed applications. Among other services, it provides [Apache Mesos](http://mesos.apache.org/) as its cluster kernel and [Marathon](https://mesosphere.github.io/marathon/) as its init system. With DCOS CLI, Mesos frameworks like [Kubernetes-Mesos](https://github.com/mesosphere/kubernetes-mesos) can be installed with a single command. + +Another feature of the DCOS CLI is that it allows plugins like the [DCOS Kubectl plugin](https://github.com/mesosphere/dcos-kubectl). This allows for easy access to a version-compatible Kubectl without having to manually download or install. + +Further information about the benefits of installing Kubernetes on DCOS can be found in the [Kubernetes-Mesos documentation](https://releases.k8s.io/{{page.githubbranch}}/contrib/mesos/README.md). + +For more details about the Kubernetes DCOS packaging, see the [Kubernetes-Mesos project](https://github.com/mesosphere/kubernetes-mesos). + +Since Kubernetes-Mesos is still alpha, it is a good idea to familiarize yourself with the [current known issues](https://releases.k8s.io/{{page.githubbranch}}/contrib/mesos/docs/issues.md) which may limit or modify the behavior of Kubernetes on DCOS. + +If you have problems completing the steps below, please [file an issue against the kubernetes-mesos project](https://github.com/mesosphere/kubernetes-mesos/issues). + + +## Resources + +Explore the following resources for more information about Kubernetes, Kubernetes on Mesos/DCOS, and DCOS itself. + +- [DCOS Documentation](https://docs.mesosphere.com/) +- [Managing DCOS Services](https://docs.mesosphere.com/services/kubernetes/) +- [Kubernetes Examples](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/) +- [Kubernetes on Mesos Documentation](https://releases.k8s.io/{{page.githubbranch}}/contrib/mesos/README.md) +- [Kubernetes on Mesos Release Notes](https://github.com/mesosphere/kubernetes-mesos/releases) +- [Kubernetes on DCOS Package Source](https://github.com/mesosphere/kubernetes-mesos) + + +## Prerequisites + +- A running [DCOS cluster](https://mesosphere.com/product/) + - [DCOS Community Edition](https://docs.mesosphere.com/install/) is currently available on [AWS](https://mesosphere.com/amazon/). + - [DCOS Enterprise Edition](https://mesosphere.com/product/) can be deployed on virtual or bare metal machines. Contact sales@mesosphere.com for more info and to set up an engagement. +- [DCOS CLI](https://docs.mesosphere.com/install/cli/) installed locally + + +## Install + +1. Configure and validate the [Mesosphere Multiverse](https://github.com/mesosphere/multiverse) as a package source repository + ```shell -$ dcos config prepend package.sources https://github.com/mesosphere/multiverse/archive/version-1.x.zip - $ dcos package update --validate +$ dcos config prepend package.sources https://github.com/mesosphere/multiverse/archive/version-1.x.zip + $ dcos package update --validate ``` -2. Install etcd - - By default, the Kubernetes DCOS package starts a single-node etcd. In order to avoid state loss in the event of Kubernetes component container failure, install an HA [etcd-mesos](https://github.com/mesosphere/etcd-mesos) cluster on DCOS. - +2. Install etcd + + By default, the Kubernetes DCOS package starts a single-node etcd. In order to avoid state loss in the event of Kubernetes component container failure, install an HA [etcd-mesos](https://github.com/mesosphere/etcd-mesos) cluster on DCOS. + ```shell -$ dcos package install etcd +$ dcos package install etcd ``` -3. Verify that etcd is installed and healthy - - The etcd cluster takes a short while to deploy. Verify that `/etcd` is healthy before going on to the next step. - +3. Verify that etcd is installed and healthy + + The etcd cluster takes a short while to deploy. Verify that `/etcd` is healthy before going on to the next step. + ```shell -$ dcos marathon app list - ID MEM CPUS TASKS HEALTH DEPLOYMENT CONTAINER CMD - /etcd 128 0.2 1/1 1/1 --- DOCKER None +$ dcos marathon app list + ID MEM CPUS TASKS HEALTH DEPLOYMENT CONTAINER CMD + /etcd 128 0.2 1/1 1/1 --- DOCKER None ``` -4. Create Kubernetes installation configuration - - Configure Kubernetes to use the HA etcd installed on DCOS. - +4. Create Kubernetes installation configuration + + Configure Kubernetes to use the HA etcd installed on DCOS. + ```shell -$ cat >/tmp/options.json </tmp/options.json < +This guide will set up a 2-node Kubernetes cluster, consisting of a _master_ node which hosts the API server and orchestrates work +and a _worker_ node which receives work from the master. You can repeat the process of adding worker nodes an arbitrary number of +times to create larger clusters. + +Here's a diagram of what the final result will look like: + +![Kubernetes Single Node on Docker](/images/docs/k8s-docker.png) + +### Bootstrap Docker + +This guide also uses a pattern of running two instances of the Docker daemon + 1) A _bootstrap_ Docker instance which is used to start system daemons like `flanneld` and `etcd` + 2) A _main_ Docker instance which is used for the Kubernetes infrastructure and user's scheduled containers + +This pattern is necessary because the `flannel` daemon is responsible for setting up and managing the network that interconnects +all of the Docker containers created by Kubernetes. To achieve this, it must run outside of the _main_ Docker daemon. However, +it is still useful to use containers for deployment and management, so we create a simpler _bootstrap_ daemon to achieve this. + +You can specify the version on every node before install: + +```shell +export K8S_VERSION= export ETCD_VERSION= export FLANNEL_VERSION= export FLANNEL_IFACE= export FLANNEL_IPMASQ= -``` - -Otherwise, we'll use latest `hyperkube` image as default k8s version. - -## Master Node - -The first step in the process is to initialize the master node. - +``` + +Otherwise, we'll use latest `hyperkube` image as default k8s version. + +## Master Node + +The first step in the process is to initialize the master node. + The MASTER_IP step here is optional, it defaults to the first value of `hostname -I`. -Clone the Kubernetes repo, and run [master.sh](/docs/getting-started-guides/docker-multinode/master.sh) on the master machine _with root_: - -```shell +Clone the Kubernetes repo, and run [master.sh](/docs/getting-started-guides/docker-multinode/master.sh) on the master machine _with root_: + +```shell $ export MASTER_IP= -$ cd kubernetes/docs/getting-started-guides/docker-multinode/ -$ ./master.sh +$ cd kubernetes/docs/getting-started-guides/docker-multinode/ +$ ./master.sh ``` - -`Master done!` - -See [here](/docs/getting-started-guides/docker-multinode/master) for detailed instructions explanation. - -## Adding a worker node - -Once your master is up and running you can add one or more workers on different machines. - -Clone the Kubernetes repo, and run [worker.sh](/docs/getting-started-guides/docker-multinode/worker.sh) on the worker machine _with root_: - -```shell -$ export MASTER_IP= -$ cd kubernetes/docs/getting-started-guides/docker-multinode/ -$ ./worker.sh + +`Master done!` + +See [here](/docs/getting-started-guides/docker-multinode/master) for detailed instructions explanation. + +## Adding a worker node + +Once your master is up and running you can add one or more workers on different machines. + +Clone the Kubernetes repo, and run [worker.sh](/docs/getting-started-guides/docker-multinode/worker.sh) on the worker machine _with root_: + +```shell +$ export MASTER_IP= +$ cd kubernetes/docs/getting-started-guides/docker-multinode/ +$ ./worker.sh ``` - -`Worker done!` - -See [here](/docs/getting-started-guides/docker-multinode/worker) for a detailed explanation. - -## Deploy a DNS - -See [here](/docs/getting-started-guides/docker-multinode/deployDNS) for instructions. - -## Testing your cluster - -Once your cluster has been created you can [test it out](/docs/getting-started-guides/docker-multinode/testing) - -For more complete applications, please look in the [examples directory](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/) + +`Worker done!` + +See [here](/docs/getting-started-guides/docker-multinode/worker) for a detailed explanation. + +## Deploy a DNS + +See [here](/docs/getting-started-guides/docker-multinode/deployDNS) for instructions. + +## Testing your cluster + +Once your cluster has been created you can [test it out](/docs/getting-started-guides/docker-multinode/testing) + +For more complete applications, please look in the [examples directory](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/) diff --git a/docs/getting-started-guides/docker.md b/docs/getting-started-guides/docker.md index e5cb648a34..3abbf9cf8c 100644 --- a/docs/getting-started-guides/docker.md +++ b/docs/getting-started-guides/docker.md @@ -1,36 +1,36 @@ --- --- - + The following instructions show you how to set up a simple, single node Kubernetes cluster using Docker. - -Here's a diagram of what the final result will look like: - + +Here's a diagram of what the final result will look like: + ![Kubernetes Single Node on Docker](/images/docs/k8s-singlenode-docker.png) * TOC -{:toc} - -## Prerequisites - -1. You need to have docker installed on one machine. -2. Decide what Kubernetes version to use. Set the `${K8S_VERSION}` variable to - a released version of Kubernetes >= "1.2.0-alpha.7" - -### Run it - -```shell -docker run \ - --volume=/:/rootfs:ro \ - --volume=/sys:/sys:ro \ - --volume=/var/lib/docker/:/var/lib/docker:rw \ - --volume=/var/lib/kubelet/:/var/lib/kubelet:rw \ - --volume=/var/run:/var/run:rw \ - --net=host \ - --pid=host \ - --privileged=true \ - -d \ - gcr.io/google_containers/hyperkube-amd64:v${K8S_VERSION} \ - /hyperkube kubelet \ +{:toc} + +## Prerequisites + +1. You need to have docker installed on one machine. +2. Decide what Kubernetes version to use. Set the `${K8S_VERSION}` variable to + a released version of Kubernetes >= "1.2.0-alpha.7" + +### Run it + +```shell +docker run \ + --volume=/:/rootfs:ro \ + --volume=/sys:/sys:ro \ + --volume=/var/lib/docker/:/var/lib/docker:rw \ + --volume=/var/lib/kubelet/:/var/lib/kubelet:rw \ + --volume=/var/run:/var/run:rw \ + --net=host \ + --pid=host \ + --privileged=true \ + -d \ + gcr.io/google_containers/hyperkube-amd64:v${K8S_VERSION} \ + /hyperkube kubelet \ --containerized \ --hostname-override="127.0.0.1" \ --address="0.0.0.0" \ @@ -39,17 +39,17 @@ docker run \ --cluster-dns=10.0.0.10 \ --cluster-domain=cluster.local \ --allow-privileged=true --v=2 -``` - +``` + > Note that `--cluster-dns` and `--cluster-domain` is used to deploy dns, feel free to discard them if dns is not needed. > If you would like to mount an external device as a volume, add `--volume=/dev:/dev` to the command above. It may however, cause some problems described in [#18230](https://github.com/kubernetes/kubernetes/issues/18230) -This actually runs the kubelet, which in turn runs a [pod](/docs/user-guide/pods/) that contains the other master components. - +This actually runs the kubelet, which in turn runs a [pod](/docs/user-guide/pods/) that contains the other master components. + ### Download `kubectl` - -At this point you should have a running Kubernetes cluster. You can test this + +At this point you should have a running Kubernetes cluster. You can test this by downloading the kubectl binary for `${K8S_VERSION}` (look at the URL in the following links) and make it available by editing your PATH environment variable. @@ -58,94 +58,94 @@ variable. ([linux/amd64](http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0-alpha.7/bin/linux/amd64/kubectl)) ([linux/386](http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0-alpha.7/bin/linux/386/kubectl)) ([linux/arm](http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0-alpha.7/bin/linux/arm/kubectl)) - + For example, OS X: - + ```shell $ wget http://storage.googleapis.com/kubernetes-release/release/v${K8S_VERSION}/bin/darwin/amd64/kubectl $ chmod 755 kubectl $ PATH=$PATH:`pwd` -``` - +``` + Linux: - -```shell -$ wget http://storage.googleapis.com/kubernetes-release/release/v${K8S_VERSION}/bin/linux/amd64/kubectl -$ chmod 755 kubectl -$ PATH=$PATH:`pwd` -``` - + +```shell +$ wget http://storage.googleapis.com/kubernetes-release/release/v${K8S_VERSION}/bin/linux/amd64/kubectl +$ chmod 755 kubectl +$ PATH=$PATH:`pwd` +``` + Create configuration: - -```shell -$ kubectl config set-cluster test-doc --server=http://localhost:8080 + +```shell +$ kubectl config set-cluster test-doc --server=http://localhost:8080 $ kubectl config set-context test-doc --cluster=test-doc $ kubectl config use-context test-doc -``` - +``` + For Max OS X users instead of `localhost` you will have to use IP address of your docker machine, which you can find by running `docker-machine env ` (see [documentation](https://docs.docker.com/machine/reference/env/) for details). ### Test it out -List the nodes in your cluster by running: - -```shell -kubectl get nodes -``` - -This should print: - -```shell -NAME LABELS STATUS -127.0.0.1 kubernetes.io/hostname=127.0.0.1 Ready -``` - -### Run an application - -```shell -kubectl run nginx --image=nginx --port=80 -``` - -Now run `docker ps` you should see nginx running. You may need to wait a few minutes for the image to get pulled. - -### Expose it as a service - -```shell -kubectl expose rc nginx --port=80 -``` - -Run the following command to obtain the IP of this service we just created. There are two IPs, the first one is internal (CLUSTER_IP), and the second one is the external load-balanced IP (if a LoadBalancer is configured) - -```shell -kubectl get svc nginx -``` - -Alternatively, you can obtain only the first IP (CLUSTER_IP) by running: - -```shell -kubectl get svc nginx --template={{.spec.clusterIP}} -``` - -Hit the webserver with the first IP (CLUSTER_IP): - -```shell -curl -``` - -Note that you will need run this curl command on your boot2docker VM if you are running on OS X. - +List the nodes in your cluster by running: + +```shell +kubectl get nodes +``` + +This should print: + +```shell +NAME LABELS STATUS +127.0.0.1 kubernetes.io/hostname=127.0.0.1 Ready +``` + +### Run an application + +```shell +kubectl run nginx --image=nginx --port=80 +``` + +Now run `docker ps` you should see nginx running. You may need to wait a few minutes for the image to get pulled. + +### Expose it as a service + +```shell +kubectl expose rc nginx --port=80 +``` + +Run the following command to obtain the IP of this service we just created. There are two IPs, the first one is internal (CLUSTER_IP), and the second one is the external load-balanced IP (if a LoadBalancer is configured) + +```shell +kubectl get svc nginx +``` + +Alternatively, you can obtain only the first IP (CLUSTER_IP) by running: + +```shell +kubectl get svc nginx --template={{.spec.clusterIP}} +``` + +Hit the webserver with the first IP (CLUSTER_IP): + +```shell +curl +``` + +Note that you will need run this curl command on your boot2docker VM if you are running on OS X. + ## Deploy a DNS See [here](/docs/getting-started-guides/docker-multinode/deployDNS/) for instructions. -### A note on turning down your cluster - -Many of these containers run under the management of the `kubelet` binary, which attempts to keep containers running, even if they fail. So, in order to turn down -the cluster, you need to first kill the kubelet container, and then any other containers. - -You may use `docker kill $(docker ps -aq)`, note this removes _all_ containers running under Docker, so use with caution. +### A note on turning down your cluster + +Many of these containers run under the management of the `kubelet` binary, which attempts to keep containers running, even if they fail. So, in order to turn down +the cluster, you need to first kill the kubelet container, and then any other containers. + +You may use `docker kill $(docker ps -aq)`, note this removes _all_ containers running under Docker, so use with caution. ### Troubleshooting diff --git a/docs/getting-started-guides/gce.md b/docs/getting-started-guides/gce.md index ffead1c97f..e486022afc 100644 --- a/docs/getting-started-guides/gce.md +++ b/docs/getting-started-guides/gce.md @@ -1,215 +1,215 @@ --- --- - - -The example below creates a Kubernetes cluster with 4 worker node Virtual Machines and a master Virtual Machine (i.e. 5 VMs in your cluster). This cluster is set up and controlled from your workstation (or wherever you find convenient). - + + +The example below creates a Kubernetes cluster with 4 worker node Virtual Machines and a master Virtual Machine (i.e. 5 VMs in your cluster). This cluster is set up and controlled from your workstation (or wherever you find convenient). + * TOC -{:toc} - -### Before you start - -If you want a simplified getting started experience and GUI for managing clusters, please consider trying [Google Container Engine](https://cloud.google.com/container-engine/) (GKE) for hosted cluster installation and management. - -If you want to use custom binaries or pure open source Kubernetes, please continue with the instructions below. - -### Prerequisites - -1. You need a Google Cloud Platform account with billing enabled. Visit the [Google Developers Console](http://cloud.google.com/console) for more details. -1. Install `gcloud` as necessary. `gcloud` can be installed as a part of the [Google Cloud SDK](https://cloud.google.com/sdk/). -1. Enable the [Compute Engine Instance Group Manager API](https://developers.google.com/console/help/new/#activatingapis) in the [Google Cloud developers console](https://console.developers.google.com). -1. Make sure that gcloud is set to use the Google Cloud Platform project you want. You can check the current project using `gcloud config list project` and change it via `gcloud config set project `. -1. Make sure you have credentials for GCloud by running ` gcloud auth login`. -1. Make sure you can start up a GCE VM from the command line. At least make sure you can do the [Create an instance](https://cloud.google.com/compute/docs/instances/#startinstancegcloud) part of the GCE Quickstart. -1. Make sure you can ssh into the VM without interactive prompts. See the [Log in to the instance](https://cloud.google.com/compute/docs/instances/#sshing) part of the GCE Quickstart. - -### Starting a cluster - -You can install a client and start a cluster with either one of these commands (we list both in case only one is installed on your machine): - - +{:toc} + +### Before you start + +If you want a simplified getting started experience and GUI for managing clusters, please consider trying [Google Container Engine](https://cloud.google.com/container-engine/) (GKE) for hosted cluster installation and management. + +If you want to use custom binaries or pure open source Kubernetes, please continue with the instructions below. + +### Prerequisites + +1. You need a Google Cloud Platform account with billing enabled. Visit the [Google Developers Console](http://cloud.google.com/console) for more details. +1. Install `gcloud` as necessary. `gcloud` can be installed as a part of the [Google Cloud SDK](https://cloud.google.com/sdk/). +1. Enable the [Compute Engine Instance Group Manager API](https://developers.google.com/console/help/new/#activatingapis) in the [Google Cloud developers console](https://console.developers.google.com). +1. Make sure that gcloud is set to use the Google Cloud Platform project you want. You can check the current project using `gcloud config list project` and change it via `gcloud config set project `. +1. Make sure you have credentials for GCloud by running ` gcloud auth login`. +1. Make sure you can start up a GCE VM from the command line. At least make sure you can do the [Create an instance](https://cloud.google.com/compute/docs/instances/#startinstancegcloud) part of the GCE Quickstart. +1. Make sure you can ssh into the VM without interactive prompts. See the [Log in to the instance](https://cloud.google.com/compute/docs/instances/#sshing) part of the GCE Quickstart. + +### Starting a cluster + +You can install a client and start a cluster with either one of these commands (we list both in case only one is installed on your machine): + + ```shell -curl -sS https://get.k8s.io | bash +curl -sS https://get.k8s.io | bash ``` -or - +or + ```shell -wget -q -O - https://get.k8s.io | bash +wget -q -O - https://get.k8s.io | bash ``` -Once this command completes, you will have a master VM and four worker VMs, running as a Kubernetes cluster. - -By default, some containers will already be running on your cluster. Containers like `kibana` and `elasticsearch` provide [logging](/docs/getting-started-guides/logging), while `heapster` provides [monitoring](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/cluster-monitoring/README.md) services. - -The script run by the commands above creates a cluster with the name/prefix "kubernetes". It defines one specific cluster config, so you can't run it more than once. - -Alternately, you can download and install the latest Kubernetes release from [this page](https://github.com/kubernetes/kubernetes/releases), then run the `/cluster/kube-up.sh` script to start the cluster: - +Once this command completes, you will have a master VM and four worker VMs, running as a Kubernetes cluster. + +By default, some containers will already be running on your cluster. Containers like `kibana` and `elasticsearch` provide [logging](/docs/getting-started-guides/logging), while `heapster` provides [monitoring](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/cluster-monitoring/README.md) services. + +The script run by the commands above creates a cluster with the name/prefix "kubernetes". It defines one specific cluster config, so you can't run it more than once. + +Alternately, you can download and install the latest Kubernetes release from [this page](https://github.com/kubernetes/kubernetes/releases), then run the `/cluster/kube-up.sh` script to start the cluster: + ```shell -cd kubernetes -cluster/kube-up.sh +cd kubernetes +cluster/kube-up.sh ``` -If you want more than one cluster running in your project, want to use a different name, or want a different number of worker nodes, see the `/cluster/gce/config-default.sh` file for more fine-grained configuration before you start up your cluster. - -If you run into trouble, please see the section on [troubleshooting](/docs/getting-started-guides/gce/#troubleshooting), post to the -[google-containers group](https://groups.google.com/forum/#!forum/google-containers), or come ask questions on [Slack](/docs/troubleshooting/#slack). - -The next few steps will show you: - -1. how to set up the command line client on your workstation to manage the cluster -1. examples of how to use the cluster -1. how to delete the cluster -1. how to start clusters with non-default options (like larger clusters) - -### Installing the Kubernetes command line tools on your workstation - -The cluster startup script will leave you with a running cluster and a `kubernetes` directory on your workstation. -The next step is to make sure the `kubectl` tool is in your path. - -The [kubectl](/docs/user-guide/kubectl/kubectl) tool controls the Kubernetes cluster manager. It lets you inspect your cluster resources, create, delete, and update components, and much more. -You will use it to look at your new cluster and bring up example apps. - -Add the appropriate binary folder to your `PATH` to access kubectl: - +If you want more than one cluster running in your project, want to use a different name, or want a different number of worker nodes, see the `/cluster/gce/config-default.sh` file for more fine-grained configuration before you start up your cluster. + +If you run into trouble, please see the section on [troubleshooting](/docs/getting-started-guides/gce/#troubleshooting), post to the +[google-containers group](https://groups.google.com/forum/#!forum/google-containers), or come ask questions on [Slack](/docs/troubleshooting/#slack). + +The next few steps will show you: + +1. how to set up the command line client on your workstation to manage the cluster +1. examples of how to use the cluster +1. how to delete the cluster +1. how to start clusters with non-default options (like larger clusters) + +### Installing the Kubernetes command line tools on your workstation + +The cluster startup script will leave you with a running cluster and a `kubernetes` directory on your workstation. +The next step is to make sure the `kubectl` tool is in your path. + +The [kubectl](/docs/user-guide/kubectl/kubectl) tool controls the Kubernetes cluster manager. It lets you inspect your cluster resources, create, delete, and update components, and much more. +You will use it to look at your new cluster and bring up example apps. + +Add the appropriate binary folder to your `PATH` to access kubectl: + ```shell -# OS X -export PATH=/platforms/darwin/amd64:$PATH -# Linux -export PATH=/platforms/linux/amd64:$PATH +# OS X +export PATH=/platforms/darwin/amd64:$PATH +# Linux +export PATH=/platforms/linux/amd64:$PATH ``` -**Note**: gcloud also ships with `kubectl`, which by default is added to your path. -However the gcloud bundled kubectl version may be older than the one downloaded by the -get.k8s.io install script. We recommend you use the downloaded binary to avoid -potential issues with client/server version skew. - -#### Enabling bash completion of the Kubernetes command line tools - -You may find it useful to enable `kubectl` bash completion: - +**Note**: gcloud also ships with `kubectl`, which by default is added to your path. +However the gcloud bundled kubectl version may be older than the one downloaded by the +get.k8s.io install script. We recommend you use the downloaded binary to avoid +potential issues with client/server version skew. + +#### Enabling bash completion of the Kubernetes command line tools + +You may find it useful to enable `kubectl` bash completion: + ``` -$ source ./contrib/completions/bash/kubectl +$ source ./contrib/completions/bash/kubectl ``` -**Note**: This will last for the duration of your bash session. If you want to make this permanent you need to add this line in your bash profile. - -Alternatively, on most linux distributions you can also move the completions file to your bash_completions.d like this: - +**Note**: This will last for the duration of your bash session. If you want to make this permanent you need to add this line in your bash profile. + +Alternatively, on most linux distributions you can also move the completions file to your bash_completions.d like this: + ``` -$ cp ./contrib/completions/bash/kubectl /etc/bash_completion.d/ +$ cp ./contrib/completions/bash/kubectl /etc/bash_completion.d/ ``` -but then you have to update it when you update kubectl. - -### Getting started with your cluster - -#### Inspect your cluster - -Once `kubectl` is in your path, you can use it to look at your cluster. E.g., running: - +but then you have to update it when you update kubectl. + +### Getting started with your cluster + +#### Inspect your cluster + +Once `kubectl` is in your path, you can use it to look at your cluster. E.g., running: + ```shell -$ kubectl get --all-namespaces services +$ kubectl get --all-namespaces services ``` -should show a set of [services](/docs/user-guide/services) that look something like this: - +should show a set of [services](/docs/user-guide/services) that look something like this: + ```shell -NAMESPACE NAME CLUSTER_IP EXTERNAL_IP PORT(S) SELECTOR AGE -default kubernetes 10.0.0.1 443/TCP 1d -kube-system kube-dns 10.0.0.2 53/TCP,53/UDP k8s-app=kube-dns 1d -kube-system kube-ui 10.0.0.3 80/TCP k8s-app=kube-ui 1d -... +NAMESPACE NAME CLUSTER_IP EXTERNAL_IP PORT(S) SELECTOR AGE +default kubernetes 10.0.0.1 443/TCP 1d +kube-system kube-dns 10.0.0.2 53/TCP,53/UDP k8s-app=kube-dns 1d +kube-system kube-ui 10.0.0.3 80/TCP k8s-app=kube-ui 1d +... ``` -Similarly, you can take a look at the set of [pods](/docs/user-guide/pods) that were created during cluster startup. -You can do this via the - +Similarly, you can take a look at the set of [pods](/docs/user-guide/pods) that were created during cluster startup. +You can do this via the + ```shell -$ kubectl get --all-namespaces pods +$ kubectl get --all-namespaces pods ``` -command. - -You'll see a list of pods that looks something like this (the name specifics will be different): - +command. + +You'll see a list of pods that looks something like this (the name specifics will be different): + ```shell -NAMESPACE NAME READY STATUS RESTARTS AGE -kube-system fluentd-cloud-logging-kubernetes-minion-63uo 1/1 Running 0 14m -kube-system fluentd-cloud-logging-kubernetes-minion-c1n9 1/1 Running 0 14m -kube-system fluentd-cloud-logging-kubernetes-minion-c4og 1/1 Running 0 14m -kube-system fluentd-cloud-logging-kubernetes-minion-ngua 1/1 Running 0 14m -kube-system kube-dns-v5-7ztia 3/3 Running 0 15m -kube-system kube-ui-v1-curt1 1/1 Running 0 15m -kube-system monitoring-heapster-v5-ex4u3 1/1 Running 1 15m -kube-system monitoring-influx-grafana-v1-piled 2/2 Running 0 15m +NAMESPACE NAME READY STATUS RESTARTS AGE +kube-system fluentd-cloud-logging-kubernetes-minion-63uo 1/1 Running 0 14m +kube-system fluentd-cloud-logging-kubernetes-minion-c1n9 1/1 Running 0 14m +kube-system fluentd-cloud-logging-kubernetes-minion-c4og 1/1 Running 0 14m +kube-system fluentd-cloud-logging-kubernetes-minion-ngua 1/1 Running 0 14m +kube-system kube-dns-v5-7ztia 3/3 Running 0 15m +kube-system kube-ui-v1-curt1 1/1 Running 0 15m +kube-system monitoring-heapster-v5-ex4u3 1/1 Running 1 15m +kube-system monitoring-influx-grafana-v1-piled 2/2 Running 0 15m ``` -Some of the pods may take a few seconds to start up (during this time they'll show `Pending`), but check that they all show as `Running` after a short period. - -#### Run some examples - -Then, see [a simple nginx example](/docs/user-guide/simple-nginx) to try out your new cluster. - -For more complete applications, please look in the [examples directory](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/). The [guestbook example](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/guestbook/) is a good "getting started" walkthrough. - -### Tearing down the cluster - -To remove/delete/teardown the cluster, use the `kube-down.sh` script. - +Some of the pods may take a few seconds to start up (during this time they'll show `Pending`), but check that they all show as `Running` after a short period. + +#### Run some examples + +Then, see [a simple nginx example](/docs/user-guide/simple-nginx) to try out your new cluster. + +For more complete applications, please look in the [examples directory](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/). The [guestbook example](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/guestbook/) is a good "getting started" walkthrough. + +### Tearing down the cluster + +To remove/delete/teardown the cluster, use the `kube-down.sh` script. + ```shell -cd kubernetes -cluster/kube-down.sh +cd kubernetes +cluster/kube-down.sh ``` -Likewise, the `kube-up.sh` in the same directory will bring it back up. You do not need to rerun the `curl` or `wget` command: everything needed to setup the Kubernetes cluster is now on your workstation. - -### Customizing - -The script above relies on Google Storage to stage the Kubernetes release. It -then will start (by default) a single master VM along with 4 worker VMs. You -can tweak some of these parameters by editing `kubernetes/cluster/gce/config-default.sh` -You can view a transcript of a successful cluster creation -[here](https://gist.github.com/satnam6502/fc689d1b46db9772adea). - -### Troubleshooting - -#### Project settings - -You need to have the Google Cloud Storage API, and the Google Cloud Storage -JSON API enabled. It is activated by default for new projects. Otherwise, it -can be done in the Google Cloud Console. See the [Google Cloud Storage JSON -API Overview](https://cloud.google.com/storage/docs/json_api/) for more -details. - -Also ensure that-- as listed in the [Prerequsites section](#prerequisites)-- you've enabled the `Compute Engine Instance Group Manager API`, and can start up a GCE VM from the command line as in the [GCE Quickstart](https://cloud.google.com/compute/docs/quickstart) instructions. - -#### Cluster initialization hang - -If the Kubernetes startup script hangs waiting for the API to be reachable, you can troubleshoot by SSHing into the master and node VMs and looking at logs such as `/var/log/startupscript.log`. - -**Once you fix the issue, you should run `kube-down.sh` to cleanup** after the partial cluster creation, before running `kube-up.sh` to try again. - -#### SSH - -If you're having trouble SSHing into your instances, ensure the GCE firewall -isn't blocking port 22 to your VMs. By default, this should work but if you -have edited firewall rules or created a new non-default network, you'll need to -expose it: `gcloud compute firewall-rules create default-ssh --network= ---description "SSH allowed from anywhere" --allow tcp:22` - -Additionally, your GCE SSH key must either have no passcode or you need to be -using `ssh-agent`. - -#### Networking - -The instances must be able to connect to each other using their private IP. The -script uses the "default" network which should have a firewall rule called -"default-allow-internal" which allows traffic on any port on the private IPs. -If this rule is missing from the default network or if you change the network -being used in `cluster/config-default.sh` create a new rule with the following -field values: - -* Source Ranges: `10.0.0.0/8` +Likewise, the `kube-up.sh` in the same directory will bring it back up. You do not need to rerun the `curl` or `wget` command: everything needed to setup the Kubernetes cluster is now on your workstation. + +### Customizing + +The script above relies on Google Storage to stage the Kubernetes release. It +then will start (by default) a single master VM along with 4 worker VMs. You +can tweak some of these parameters by editing `kubernetes/cluster/gce/config-default.sh` +You can view a transcript of a successful cluster creation +[here](https://gist.github.com/satnam6502/fc689d1b46db9772adea). + +### Troubleshooting + +#### Project settings + +You need to have the Google Cloud Storage API, and the Google Cloud Storage +JSON API enabled. It is activated by default for new projects. Otherwise, it +can be done in the Google Cloud Console. See the [Google Cloud Storage JSON +API Overview](https://cloud.google.com/storage/docs/json_api/) for more +details. + +Also ensure that-- as listed in the [Prerequsites section](#prerequisites)-- you've enabled the `Compute Engine Instance Group Manager API`, and can start up a GCE VM from the command line as in the [GCE Quickstart](https://cloud.google.com/compute/docs/quickstart) instructions. + +#### Cluster initialization hang + +If the Kubernetes startup script hangs waiting for the API to be reachable, you can troubleshoot by SSHing into the master and node VMs and looking at logs such as `/var/log/startupscript.log`. + +**Once you fix the issue, you should run `kube-down.sh` to cleanup** after the partial cluster creation, before running `kube-up.sh` to try again. + +#### SSH + +If you're having trouble SSHing into your instances, ensure the GCE firewall +isn't blocking port 22 to your VMs. By default, this should work but if you +have edited firewall rules or created a new non-default network, you'll need to +expose it: `gcloud compute firewall-rules create default-ssh --network= +--description "SSH allowed from anywhere" --allow tcp:22` + +Additionally, your GCE SSH key must either have no passcode or you need to be +using `ssh-agent`. + +#### Networking + +The instances must be able to connect to each other using their private IP. The +script uses the "default" network which should have a firewall rule called +"default-allow-internal" which allows traffic on any port on the private IPs. +If this rule is missing from the default network or if you change the network +being used in `cluster/config-default.sh` create a new rule with the following +field values: + +* Source Ranges: `10.0.0.0/8` * Allowed Protocols and Port: `tcp:1-65535;udp:1-65535;icmp` \ No newline at end of file diff --git a/docs/getting-started-guides/juju.md b/docs/getting-started-guides/juju.md index aa6a6efa32..a2ec08615d 100644 --- a/docs/getting-started-guides/juju.md +++ b/docs/getting-started-guides/juju.md @@ -1,248 +1,248 @@ --- --- - -[Juju](https://jujucharms.com/docs/stable/about-juju) makes it easy to deploy -Kubernetes by provisioning, installing and configuring all the systems in -the cluster. Once deployed the cluster can easily scale up with one command -to increase the cluster size. - + +[Juju](https://jujucharms.com/docs/stable/about-juju) makes it easy to deploy +Kubernetes by provisioning, installing and configuring all the systems in +the cluster. Once deployed the cluster can easily scale up with one command +to increase the cluster size. + * TOC -{:toc} - -## Prerequisites - -> Note: If you're running kube-up, on Ubuntu - all of the dependencies -> will be handled for you. You may safely skip to the section: -> [Launch Kubernetes Cluster](#launch-kubernetes-cluster) - -### On Ubuntu - -[Install the Juju client](https://jujucharms.com/get-started) on your -local Ubuntu system: +{:toc} -```shell -sudo add-apt-repository ppa:juju/stable -sudo apt-get update -sudo apt-get install juju-core juju-quickstart -``` - -### With Docker - -If you are not using Ubuntu or prefer the isolation of Docker, you may -run the following: +## Prerequisites -```shell -mkdir ~/.juju +> Note: If you're running kube-up, on Ubuntu - all of the dependencies +> will be handled for you. You may safely skip to the section: +> [Launch Kubernetes Cluster](#launch-kubernetes-cluster) + +### On Ubuntu + +[Install the Juju client](https://jujucharms.com/get-started) on your +local Ubuntu system: + +```shell +sudo add-apt-repository ppa:juju/stable +sudo apt-get update +sudo apt-get install juju-core juju-quickstart +``` + +### With Docker + +If you are not using Ubuntu or prefer the isolation of Docker, you may +run the following: + +```shell +mkdir ~/.juju sudo docker run -v ~/.juju:/home/ubuntu/.juju -ti jujusolutions/jujubox:latest -``` - -At this point from either path you will have access to the `juju -quickstart` command. - -To set up the credentials for your chosen cloud run: +``` -```shell +At this point from either path you will have access to the `juju +quickstart` command. + +To set up the credentials for your chosen cloud run: + +```shell juju quickstart --constraints="mem=3.75G" -i -``` - -> The `constraints` flag is optional, it changes the size of virtual machines -> that Juju will generate when it requests a new machine. Larger machines -> will run faster but cost more money than smaller machines. - -Follow the dialogue and choose `save` and `use`. Quickstart will now -bootstrap the juju root node and setup the juju web based user -interface. - - -## Launch Kubernetes cluster - -You will need to export the `KUBERNETES_PROVIDER` environment variable before -bringing up the cluster. +``` -```shell -export KUBERNETES_PROVIDER=juju +> The `constraints` flag is optional, it changes the size of virtual machines +> that Juju will generate when it requests a new machine. Larger machines +> will run faster but cost more money than smaller machines. + +Follow the dialogue and choose `save` and `use`. Quickstart will now +bootstrap the juju root node and setup the juju web based user +interface. + + +## Launch Kubernetes cluster + +You will need to export the `KUBERNETES_PROVIDER` environment variable before +bringing up the cluster. + +```shell +export KUBERNETES_PROVIDER=juju cluster/kube-up.sh -``` - -If this is your first time running the `kube-up.sh` script, it will install -the required dependencies to get started with Juju, additionally it will -launch a curses based configuration utility allowing you to select your cloud -provider and enter the proper access credentials. - -Next it will deploy the kubernetes master, etcd, 2 nodes with flannel based -Software Defined Networking (SDN) so containers on different hosts can -communicate with each other. - - -## Exploring the cluster - -The `juju status` command provides information about each unit in the cluster: - -```shell -$ juju status --format=oneline -- docker/0: 52.4.92.78 (started) - - flannel-docker/0: 52.4.92.78 (started) - - kubernetes/0: 52.4.92.78 (started) -- docker/1: 52.6.104.142 (started) - - flannel-docker/1: 52.6.104.142 (started) - - kubernetes/1: 52.6.104.142 (started) -- etcd/0: 52.5.216.210 (started) 4001/tcp -- juju-gui/0: 52.5.205.174 (started) 80/tcp, 443/tcp -- kubernetes-master/0: 52.6.19.238 (started) 8080/tcp ``` - -You can use `juju ssh` to access any of the units: -```shell -juju ssh kubernetes-master/0 -``` - -## Run some containers! - -`kubectl` is available on the Kubernetes master node. We'll ssh in to -launch some containers, but one could use `kubectl` locally by setting -`KUBERNETES_MASTER` to point at the ip address of "kubernetes-master/0". - -No pods will be available before starting a container: +If this is your first time running the `kube-up.sh` script, it will install +the required dependencies to get started with Juju, additionally it will +launch a curses based configuration utility allowing you to select your cloud +provider and enter the proper access credentials. -```shell -kubectl get pods -NAME READY STATUSRESTARTS AGE - -kubectl get replicationcontrollers +Next it will deploy the kubernetes master, etcd, 2 nodes with flannel based +Software Defined Networking (SDN) so containers on different hosts can +communicate with each other. + + +## Exploring the cluster + +The `juju status` command provides information about each unit in the cluster: + +```shell +$ juju status --format=oneline +- docker/0: 52.4.92.78 (started) + - flannel-docker/0: 52.4.92.78 (started) + - kubernetes/0: 52.4.92.78 (started) +- docker/1: 52.6.104.142 (started) + - flannel-docker/1: 52.6.104.142 (started) + - kubernetes/1: 52.6.104.142 (started) +- etcd/0: 52.5.216.210 (started) 4001/tcp +- juju-gui/0: 52.5.205.174 (started) 80/tcp, 443/tcp +- kubernetes-master/0: 52.6.19.238 (started) 8080/tcp +``` + +You can use `juju ssh` to access any of the units: + +```shell +juju ssh kubernetes-master/0 +``` + +## Run some containers! + +`kubectl` is available on the Kubernetes master node. We'll ssh in to +launch some containers, but one could use `kubectl` locally by setting +`KUBERNETES_MASTER` to point at the ip address of "kubernetes-master/0". + +No pods will be available before starting a container: + +```shell +kubectl get pods +NAME READY STATUSRESTARTS AGE + +kubectl get replicationcontrollers CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS -``` - -We'll follow the aws-coreos example. Create a pod manifest: `pod.json` - +``` + +We'll follow the aws-coreos example. Create a pod manifest: `pod.json` + ```json -{ - "apiVersion": "v1", - "kind": "Pod", - "metadata": { -"name": "hello", -"labels": { - "name": "hello", - "environment": "testing" -} - }, - "spec": { -"containers": [{ - "name": "hello", - "image": "quay.io/kelseyhightower/hello", - "ports": [{ -"containerPort": 80, -"hostPort": 80 - }] -}] - } -} +{ + "apiVersion": "v1", + "kind": "Pod", + "metadata": { +"name": "hello", +"labels": { + "name": "hello", + "environment": "testing" +} + }, + "spec": { +"containers": [{ + "name": "hello", + "image": "quay.io/kelseyhightower/hello", + "ports": [{ +"containerPort": 80, +"hostPort": 80 + }] +}] + } +} ``` -Create the pod with kubectl: +Create the pod with kubectl: -```shell -kubectl create -f pod.json -``` - -Get info on the pod: - -```shell -kubectl get pods -``` - -To test the hello app, we need to locate which node is hosting -the container. Better tooling for using Juju to introspect container -is in the works but we can use `juju run` and `juju status` to find -our hello app. - -Exit out of our ssh session and run: - -```shell -juju run --unit kubernetes/0 "docker ps -n=1" -... -juju run --unit kubernetes/1 "docker ps -n=1" -CONTAINER IDIMAGE COMMAND CREATED STATUS PORTS NAMES -02beb61339d8quay.io/kelseyhightower/hello:latest /hello About an hour ago Up About an hourk8s_hello.... -``` - -We see "kubernetes/1" has our container, we can open port 80: - -```shell -juju run --unit kubernetes/1 "open-port 80" -juju expose kubernetes -sudo apt-get install curl -curl $(juju status --format=oneline kubernetes/1 | cut -d' ' -f3) +```shell +kubectl create -f pod.json ``` - -Finally delete the pod: -```shell -juju ssh kubernetes-master/0 -kubectl delete pods hello -``` - -## Scale out cluster - -We can add node units like so: +Get info on the pod: -```shell +```shell +kubectl get pods +``` + +To test the hello app, we need to locate which node is hosting +the container. Better tooling for using Juju to introspect container +is in the works but we can use `juju run` and `juju status` to find +our hello app. + +Exit out of our ssh session and run: + +```shell +juju run --unit kubernetes/0 "docker ps -n=1" +... +juju run --unit kubernetes/1 "docker ps -n=1" +CONTAINER IDIMAGE COMMAND CREATED STATUS PORTS NAMES +02beb61339d8quay.io/kelseyhightower/hello:latest /hello About an hour ago Up About an hourk8s_hello.... +``` + +We see "kubernetes/1" has our container, we can open port 80: + +```shell +juju run --unit kubernetes/1 "open-port 80" +juju expose kubernetes +sudo apt-get install curl +curl $(juju status --format=oneline kubernetes/1 | cut -d' ' -f3) +``` + +Finally delete the pod: + +```shell +juju ssh kubernetes-master/0 +kubectl delete pods hello +``` + +## Scale out cluster + +We can add node units like so: + +```shell juju add-unit docker # creates unit docker/2, kubernetes/2, docker-flannel/2 -``` - -## Launch the "k8petstore" example app - -The [k8petstore example](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/k8petstore/) is available as a -[juju action](https://jujucharms.com/docs/devel/actions). +``` -```shell +## Launch the "k8petstore" example app + +The [k8petstore example](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/k8petstore/) is available as a +[juju action](https://jujucharms.com/docs/devel/actions). + +```shell juju action do kubernetes-master/0 -``` - -> Note: this example includes curl statements to exercise the app, which -> automatically generates "petstore" transactions written to redis, and allows -> you to visualize the throughput in your browser. - -## Tear down cluster +``` -```shell +> Note: this example includes curl statements to exercise the app, which +> automatically generates "petstore" transactions written to redis, and allows +> you to visualize the throughput in your browser. + +## Tear down cluster + +```shell ./kube-down.sh -``` - -or destroy your current Juju environment (using the `juju env` command): +``` -```shell +or destroy your current Juju environment (using the `juju env` command): + +```shell juju destroy-environment --force `juju env` -``` - - -## More Info - -The Kubernetes charms and bundles can be found in the `kubernetes` project on -github.com: - - - [Bundle Repository](http://releases.k8s.io/{{page.githubbranch}}/cluster/juju/bundles) - * [Kubernetes master charm](https://releases.k8s.io/{{page.githubbranch}}/cluster/juju/charms/trusty/kubernetes-master) - * [Kubernetes node charm](https://releases.k8s.io/{{page.githubbranch}}/cluster/juju/charms/trusty/kubernetes) - - [More about Juju](https://jujucharms.com) - - -### Cloud compatibility - -Juju runs natively against a variety of public cloud providers. Juju currently -works with [Amazon Web Service](https://jujucharms.com/docs/stable/config-aws), -[Windows Azure](https://jujucharms.com/docs/stable/config-azure), -[DigitalOcean](https://jujucharms.com/docs/stable/config-digitalocean), -[Google Compute Engine](https://jujucharms.com/docs/stable/config-gce), -[HP Public Cloud](https://jujucharms.com/docs/stable/config-hpcloud), -[Joyent](https://jujucharms.com/docs/stable/config-joyent), -[LXC](https://jujucharms.com/docs/stable/config-LXC), any -[OpenStack](https://jujucharms.com/docs/stable/config-openstack) deployment, -[Vagrant](https://jujucharms.com/docs/stable/config-vagrant), and -[Vmware vSphere](https://jujucharms.com/docs/stable/config-vmware). - -If you do not see your favorite cloud provider listed many clouds can be -configured for [manual provisioning](https://jujucharms.com/docs/stable/config-manual). - -The Kubernetes bundle has been tested on GCE and AWS and found to work with +``` + + +## More Info + +The Kubernetes charms and bundles can be found in the `kubernetes` project on +github.com: + + - [Bundle Repository](http://releases.k8s.io/{{page.githubbranch}}/cluster/juju/bundles) + * [Kubernetes master charm](https://releases.k8s.io/{{page.githubbranch}}/cluster/juju/charms/trusty/kubernetes-master) + * [Kubernetes node charm](https://releases.k8s.io/{{page.githubbranch}}/cluster/juju/charms/trusty/kubernetes) + - [More about Juju](https://jujucharms.com) + + +### Cloud compatibility + +Juju runs natively against a variety of public cloud providers. Juju currently +works with [Amazon Web Service](https://jujucharms.com/docs/stable/config-aws), +[Windows Azure](https://jujucharms.com/docs/stable/config-azure), +[DigitalOcean](https://jujucharms.com/docs/stable/config-digitalocean), +[Google Compute Engine](https://jujucharms.com/docs/stable/config-gce), +[HP Public Cloud](https://jujucharms.com/docs/stable/config-hpcloud), +[Joyent](https://jujucharms.com/docs/stable/config-joyent), +[LXC](https://jujucharms.com/docs/stable/config-LXC), any +[OpenStack](https://jujucharms.com/docs/stable/config-openstack) deployment, +[Vagrant](https://jujucharms.com/docs/stable/config-vagrant), and +[Vmware vSphere](https://jujucharms.com/docs/stable/config-vmware). + +If you do not see your favorite cloud provider listed many clouds can be +configured for [manual provisioning](https://jujucharms.com/docs/stable/config-manual). + +The Kubernetes bundle has been tested on GCE and AWS and found to work with version 1.0.0. \ No newline at end of file diff --git a/docs/getting-started-guides/libvirt-coreos.md b/docs/getting-started-guides/libvirt-coreos.md index 8d8a447209..4645a41c8d 100644 --- a/docs/getting-started-guides/libvirt-coreos.md +++ b/docs/getting-started-guides/libvirt-coreos.md @@ -1,294 +1,294 @@ --- --- - + * TOC -{:toc} - -### Highlights - -* Super-fast cluster boot-up (few seconds instead of several minutes for vagrant) -* Reduced disk usage thanks to [COW](https://en.wikibooks.org/wiki/QEMU/Images#Copy_on_write) -* Reduced memory footprint thanks to [KSM](https://www.kernel.org/doc/Documentation/vm/ksm.txt) - -### Warnings about `libvirt-coreos` use case - -The primary goal of the `libvirt-coreos` cluster provider is to deploy a multi-node Kubernetes cluster on local VMs as fast as possible and to be as light as possible in term of resources used. - -In order to achieve that goal, its deployment is very different from the "standard production deployment" method used on other providers. This was done on purpose in order to implement some optimizations made possible by the fact that we know that all VMs will be running on the same physical machine. - -The `libvirt-coreos` cluster provider doesn't aim at being production look-alike. - -Another difference is that no security is enforced on `libvirt-coreos` at all. For example, - -* Kube API server is reachable via a clear-text connection (no SSL); -* Kube API server requires no credentials; -* etcd access is not protected; -* Kubernetes secrets are not protected as securely as they are on production environments; -* etc. - -So, an k8s application developer should not validate its interaction with Kubernetes on `libvirt-coreos` because he might technically succeed in doing things that are prohibited on a production environment like: - -* un-authenticated access to Kube API server; -* Access to Kubernetes private data structures inside etcd; -* etc. - -On the other hand, `libvirt-coreos` might be useful for people investigating low level implementation of Kubernetes because debugging techniques like sniffing the network traffic or introspecting the etcd content are easier on `libvirt-coreos` than on a production deployment. - -### Prerequisites - -1. Install [dnsmasq](http://www.thekelleys.org.uk/dnsmasq/doc.html) -2. Install [ebtables](http://ebtables.netfilter.org/) -3. Install [qemu](http://wiki.qemu.org/Main_Page) -4. Install [libvirt](http://libvirt.org/) +{:toc} + +### Highlights + +* Super-fast cluster boot-up (few seconds instead of several minutes for vagrant) +* Reduced disk usage thanks to [COW](https://en.wikibooks.org/wiki/QEMU/Images#Copy_on_write) +* Reduced memory footprint thanks to [KSM](https://www.kernel.org/doc/Documentation/vm/ksm.txt) + +### Warnings about `libvirt-coreos` use case + +The primary goal of the `libvirt-coreos` cluster provider is to deploy a multi-node Kubernetes cluster on local VMs as fast as possible and to be as light as possible in term of resources used. + +In order to achieve that goal, its deployment is very different from the "standard production deployment" method used on other providers. This was done on purpose in order to implement some optimizations made possible by the fact that we know that all VMs will be running on the same physical machine. + +The `libvirt-coreos` cluster provider doesn't aim at being production look-alike. + +Another difference is that no security is enforced on `libvirt-coreos` at all. For example, + +* Kube API server is reachable via a clear-text connection (no SSL); +* Kube API server requires no credentials; +* etcd access is not protected; +* Kubernetes secrets are not protected as securely as they are on production environments; +* etc. + +So, an k8s application developer should not validate its interaction with Kubernetes on `libvirt-coreos` because he might technically succeed in doing things that are prohibited on a production environment like: + +* un-authenticated access to Kube API server; +* Access to Kubernetes private data structures inside etcd; +* etc. + +On the other hand, `libvirt-coreos` might be useful for people investigating low level implementation of Kubernetes because debugging techniques like sniffing the network traffic or introspecting the etcd content are easier on `libvirt-coreos` than on a production deployment. + +### Prerequisites + +1. Install [dnsmasq](http://www.thekelleys.org.uk/dnsmasq/doc.html) +2. Install [ebtables](http://ebtables.netfilter.org/) +3. Install [qemu](http://wiki.qemu.org/Main_Page) +4. Install [libvirt](http://libvirt.org/) 5. Install [openssl](http://openssl.org/) -6. Enable and start the libvirt daemon, e.g: - * ``systemctl enable libvirtd && systemctl start libvirtd`` # for systemd-based systems - * ``/etc/init.d/libvirt-bin start`` # for init.d-based systems -7. [Grant libvirt access to your user¹](https://libvirt.org/aclpolkit.html) -8. Check that your $HOME is accessible to the qemu user² - -#### ¹ Depending on your distribution, libvirt access may be denied by default or may require a password at each access. - -You can test it with the following command: - +6. Enable and start the libvirt daemon, e.g: + * ``systemctl enable libvirtd && systemctl start libvirtd`` # for systemd-based systems + * ``/etc/init.d/libvirt-bin start`` # for init.d-based systems +7. [Grant libvirt access to your user¹](https://libvirt.org/aclpolkit.html) +8. Check that your $HOME is accessible to the qemu user² + +#### ¹ Depending on your distribution, libvirt access may be denied by default or may require a password at each access. + +You can test it with the following command: + ```shell -virsh -c qemu:///system pool-list +virsh -c qemu:///system pool-list ``` -If you have access error messages, please read https://libvirt.org/acl.html and https://libvirt.org/aclpolkit.html . - -In short, if your libvirt has been compiled with Polkit support (ex: Arch, Fedora 21), you can create `/etc/polkit-1/rules.d/50-org.libvirt.unix.manage.rules` as follows to grant full access to libvirt to `$USER` - +If you have access error messages, please read https://libvirt.org/acl.html and https://libvirt.org/aclpolkit.html . + +In short, if your libvirt has been compiled with Polkit support (ex: Arch, Fedora 21), you can create `/etc/polkit-1/rules.d/50-org.libvirt.unix.manage.rules` as follows to grant full access to libvirt to `$USER` + ```shell sudo /bin/sh -c "cat - > /etc/polkit-1/rules.d/50-org.libvirt.unix.manage.rules" << EOF ``` -```conf -polkit.addRule(function(action, subject) { - if (action.id == "org.libvirt.unix.manage" && - subject.user == "$USER") { - return polkit.Result.YES; - polkit.log("action=" + action); - polkit.log("subject=" + subject); - } -}); -EOF +```conf +polkit.addRule(function(action, subject) { + if (action.id == "org.libvirt.unix.manage" && + subject.user == "$USER") { + return polkit.Result.YES; + polkit.log("action=" + action); + polkit.log("subject=" + subject); + } +}); +EOF ``` -If your libvirt has not been compiled with Polkit (ex: Ubuntu 14.04.1 LTS), check the permissions on the libvirt unix socket: - +If your libvirt has not been compiled with Polkit (ex: Ubuntu 14.04.1 LTS), check the permissions on the libvirt unix socket: + ```shell -$ ls -l /var/run/libvirt/libvirt-sock -srwxrwx--- 1 root libvirtd 0 févr. 12 16:03 /var/run/libvirt/libvirt-sock - -$ usermod -a -G libvirtd $USER -# $USER needs to logout/login to have the new group be taken into account +$ ls -l /var/run/libvirt/libvirt-sock +srwxrwx--- 1 root libvirtd 0 févr. 12 16:03 /var/run/libvirt/libvirt-sock + +$ usermod -a -G libvirtd $USER +# $USER needs to logout/login to have the new group be taken into account ``` - -(Replace `$USER` with your login name) - -#### ² Qemu will run with a specific user. It must have access to the VMs drives - -All the disk drive resources needed by the VM (CoreOS disk image, Kubernetes binaries, cloud-init files, etc.) are put inside `./cluster/libvirt-coreos/libvirt_storage_pool`. - -As we're using the `qemu:///system` instance of libvirt, qemu will run with a specific `user:group` distinct from your user. It is configured in `/etc/libvirt/qemu.conf`. That qemu user must have access to that libvirt storage pool. - -If your `$HOME` is world readable, everything is fine. If your $HOME is private, `cluster/kube-up.sh` will fail with an error message like: - + +(Replace `$USER` with your login name) + +#### ² Qemu will run with a specific user. It must have access to the VMs drives + +All the disk drive resources needed by the VM (CoreOS disk image, Kubernetes binaries, cloud-init files, etc.) are put inside `./cluster/libvirt-coreos/libvirt_storage_pool`. + +As we're using the `qemu:///system` instance of libvirt, qemu will run with a specific `user:group` distinct from your user. It is configured in `/etc/libvirt/qemu.conf`. That qemu user must have access to that libvirt storage pool. + +If your `$HOME` is world readable, everything is fine. If your $HOME is private, `cluster/kube-up.sh` will fail with an error message like: + ```shell -error: Cannot access storage file '$HOME/.../kubernetes/cluster/libvirt-coreos/libvirt_storage_pool/kubernetes_master.img' (as uid:99, gid:78): Permission denied +error: Cannot access storage file '$HOME/.../kubernetes/cluster/libvirt-coreos/libvirt_storage_pool/kubernetes_master.img' (as uid:99, gid:78): Permission denied ``` In order to fix that issue, you have several possibilities: - -* set `POOL_PATH` inside `cluster/libvirt-coreos/config-default.sh` to a directory: - * backed by a filesystem with a lot of free disk space - * writable by your user; - * accessible by the qemu user. -* Grant the qemu user access to the storage pool. - -On Arch: - + +* set `POOL_PATH` inside `cluster/libvirt-coreos/config-default.sh` to a directory: + * backed by a filesystem with a lot of free disk space + * writable by your user; + * accessible by the qemu user. +* Grant the qemu user access to the storage pool. + +On Arch: + ```shell -setfacl -m g:kvm:--x ~ +setfacl -m g:kvm:--x ~ ``` -### Setup - -By default, the libvirt-coreos setup will create a single Kubernetes master and 3 Kubernetes nodes. Because the VM drives use Copy-on-Write and because of memory ballooning and KSM, there is a lot of resource over-allocation. - -To start your local cluster, open a shell and run: - +### Setup + +By default, the libvirt-coreos setup will create a single Kubernetes master and 3 Kubernetes nodes. Because the VM drives use Copy-on-Write and because of memory ballooning and KSM, there is a lot of resource over-allocation. + +To start your local cluster, open a shell and run: + ```shell -cd kubernetes - -export KUBERNETES_PROVIDER=libvirt-coreos -cluster/kube-up.sh +cd kubernetes + +export KUBERNETES_PROVIDER=libvirt-coreos +cluster/kube-up.sh ``` -The `KUBERNETES_PROVIDER` environment variable tells all of the various cluster management scripts which variant to use. If you forget to set this, the assumption is you are running on Google Compute Engine. - -The `NUM_NODES` environment variable may be set to specify the number of nodes to start. If it is not set, the number of nodes defaults to 3. - -The `KUBE_PUSH` environment variable may be set to specify which Kubernetes binaries must be deployed on the cluster. Its possible values are: - -* `release` (default if `KUBE_PUSH` is not set) will deploy the binaries of `_output/release-tars/kubernetes-server-….tar.gz`. This is built with `make release` or `make release-skip-tests`. -* `local` will deploy the binaries of `_output/local/go/bin`. These are built with `make`. - -You can check that your machines are there and running with: - +The `KUBERNETES_PROVIDER` environment variable tells all of the various cluster management scripts which variant to use. If you forget to set this, the assumption is you are running on Google Compute Engine. + +The `NUM_NODES` environment variable may be set to specify the number of nodes to start. If it is not set, the number of nodes defaults to 3. + +The `KUBE_PUSH` environment variable may be set to specify which Kubernetes binaries must be deployed on the cluster. Its possible values are: + +* `release` (default if `KUBE_PUSH` is not set) will deploy the binaries of `_output/release-tars/kubernetes-server-….tar.gz`. This is built with `make release` or `make release-skip-tests`. +* `local` will deploy the binaries of `_output/local/go/bin`. These are built with `make`. + +You can check that your machines are there and running with: + ```shell -$ virsh -c qemu:///system list - Id Name State ----------------------------------------------------- - 15 kubernetes_master running - 16 kubernetes_node-01 running - 17 kubernetes_node-02 running - 18 kubernetes_node-03 running +$ virsh -c qemu:///system list + Id Name State +---------------------------------------------------- + 15 kubernetes_master running + 16 kubernetes_node-01 running + 17 kubernetes_node-02 running + 18 kubernetes_node-03 running ``` -You can check that the Kubernetes cluster is working with: - +You can check that the Kubernetes cluster is working with: + ```shell -$ kubectl get nodes -NAME LABELS STATUS -192.168.10.2 Ready -192.168.10.3 Ready -192.168.10.4 Ready +$ kubectl get nodes +NAME LABELS STATUS +192.168.10.2 Ready +192.168.10.3 Ready +192.168.10.4 Ready ``` -The VMs are running [CoreOS](https://coreos.com/). -Your ssh keys have already been pushed to the VM. (It looks for ~/.ssh/id_*.pub) -The user to use to connect to the VM is `core`. -The IP to connect to the master is 192.168.10.1. -The IPs to connect to the nodes are 192.168.10.2 and onwards. - -Connect to `kubernetes_master`: - +The VMs are running [CoreOS](https://coreos.com/). +Your ssh keys have already been pushed to the VM. (It looks for ~/.ssh/id_*.pub) +The user to use to connect to the VM is `core`. +The IP to connect to the master is 192.168.10.1. +The IPs to connect to the nodes are 192.168.10.2 and onwards. + +Connect to `kubernetes_master`: + ```shell -ssh core@192.168.10.1 +ssh core@192.168.10.1 ``` -Connect to `kubernetes_node-01`: - +Connect to `kubernetes_node-01`: + ```shell -ssh core@192.168.10.2 +ssh core@192.168.10.2 ``` -### Interacting with your Kubernetes cluster with the `kube-*` scripts. - -All of the following commands assume you have set `KUBERNETES_PROVIDER` appropriately: - +### Interacting with your Kubernetes cluster with the `kube-*` scripts. + +All of the following commands assume you have set `KUBERNETES_PROVIDER` appropriately: + ```shell -export KUBERNETES_PROVIDER=libvirt-coreos +export KUBERNETES_PROVIDER=libvirt-coreos ``` -Bring up a libvirt-CoreOS cluster of 5 nodes - +Bring up a libvirt-CoreOS cluster of 5 nodes + ```shell -NUM_NODES=5 cluster/kube-up.sh +NUM_NODES=5 cluster/kube-up.sh ``` -Destroy the libvirt-CoreOS cluster - +Destroy the libvirt-CoreOS cluster + ```shell -cluster/kube-down.sh +cluster/kube-down.sh ``` -Update the libvirt-CoreOS cluster with a new Kubernetes release produced by `make release` or `make release-skip-tests`: - +Update the libvirt-CoreOS cluster with a new Kubernetes release produced by `make release` or `make release-skip-tests`: + ```shell -cluster/kube-push.sh +cluster/kube-push.sh ``` -Update the libvirt-CoreOS cluster with the locally built Kubernetes binaries produced by `make`: - +Update the libvirt-CoreOS cluster with the locally built Kubernetes binaries produced by `make`: + ```shell -KUBE_PUSH=local cluster/kube-push.sh +KUBE_PUSH=local cluster/kube-push.sh ``` -Interact with the cluster - +Interact with the cluster + ```shell -kubectl ... +kubectl ... ``` -### Troubleshooting - -#### !!! Cannot find kubernetes-server-linux-amd64.tar.gz - -Build the release tarballs: - +### Troubleshooting + +#### !!! Cannot find kubernetes-server-linux-amd64.tar.gz + +Build the release tarballs: + ```shell -make release +make release ``` -#### Can't find virsh in PATH, please fix and retry. - -Install libvirt - -On Arch: - +#### Can't find virsh in PATH, please fix and retry. + +Install libvirt + +On Arch: + ```shell -pacman -S qemu libvirt +pacman -S qemu libvirt ``` -On Ubuntu 14.04.1: - +On Ubuntu 14.04.1: + ```shell -aptitude install qemu-system-x86 libvirt-bin +aptitude install qemu-system-x86 libvirt-bin ``` -On Fedora 21: - +On Fedora 21: + ```shell -yum install qemu libvirt +yum install qemu libvirt ``` -#### error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory - -Start the libvirt daemon - -On Arch: - +#### error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory + +Start the libvirt daemon + +On Arch: + ```shell -systemctl start libvirtd +systemctl start libvirtd ``` -On Ubuntu 14.04.1: - +On Ubuntu 14.04.1: + ```shell -service libvirt-bin start +service libvirt-bin start ``` -#### error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': Permission denied - -Fix libvirt access permission (Remember to adapt `$USER`) - -On Arch and Fedora 21: - +#### error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': Permission denied + +Fix libvirt access permission (Remember to adapt `$USER`) + +On Arch and Fedora 21: + ```shell cat > /etc/polkit-1/rules.d/50-org.libvirt.unix.manage.rules < +export MESOS_MASTER= ``` -Create a cloud config file `mesos-cloud.conf` in the current directory with the following contents: - +Create a cloud config file `mesos-cloud.conf` in the current directory with the following contents: + ```shell -$ cat <mesos-cloud.conf -[mesos-cloud] - mesos-master = ${MESOS_MASTER} -EOF +$ cat <mesos-cloud.conf +[mesos-cloud] + mesos-master = ${MESOS_MASTER} +EOF ``` -Now start the kubernetes-mesos API server, controller manager, and scheduler on the master node: - +Now start the kubernetes-mesos API server, controller manager, and scheduler on the master node: + ```shell -$ km apiserver \ - --address=${KUBERNETES_MASTER_IP} \ - --etcd-servers=http://${KUBERNETES_MASTER_IP}:4001 \ - --service-cluster-ip-range=10.10.10.0/24 \ - --port=8888 \ - --cloud-provider=mesos \ - --cloud-config=mesos-cloud.conf \ - --secure-port=0 \ - --v=1 >apiserver.log 2>&1 & - -$ km controller-manager \ - --master=${KUBERNETES_MASTER_IP}:8888 \ - --cloud-provider=mesos \ - --cloud-config=./mesos-cloud.conf \ - --v=1 >controller.log 2>&1 & - -$ km scheduler \ - --address=${KUBERNETES_MASTER_IP} \ - --mesos-master=${MESOS_MASTER} \ - --etcd-servers=http://${KUBERNETES_MASTER_IP}:4001 \ - --mesos-user=root \ - --api-servers=${KUBERNETES_MASTER_IP}:8888 \ - --cluster-dns=10.10.10.10 \ - --cluster-domain=cluster.local \ - --v=2 >scheduler.log 2>&1 & +$ km apiserver \ + --address=${KUBERNETES_MASTER_IP} \ + --etcd-servers=http://${KUBERNETES_MASTER_IP}:4001 \ + --service-cluster-ip-range=10.10.10.0/24 \ + --port=8888 \ + --cloud-provider=mesos \ + --cloud-config=mesos-cloud.conf \ + --secure-port=0 \ + --v=1 >apiserver.log 2>&1 & + +$ km controller-manager \ + --master=${KUBERNETES_MASTER_IP}:8888 \ + --cloud-provider=mesos \ + --cloud-config=./mesos-cloud.conf \ + --v=1 >controller.log 2>&1 & + +$ km scheduler \ + --address=${KUBERNETES_MASTER_IP} \ + --mesos-master=${MESOS_MASTER} \ + --etcd-servers=http://${KUBERNETES_MASTER_IP}:4001 \ + --mesos-user=root \ + --api-servers=${KUBERNETES_MASTER_IP}:8888 \ + --cluster-dns=10.10.10.10 \ + --cluster-domain=cluster.local \ + --v=2 >scheduler.log 2>&1 & ``` -Disown your background jobs so that they'll stay running if you log out. - +Disown your background jobs so that they'll stay running if you log out. + ```shell -disown -a +disown -a ``` -#### Validate KM Services - -Interact with the kubernetes-mesos framework via `kubectl`: - +#### Validate KM Services + +Interact with the kubernetes-mesos framework via `kubectl`: + ```shell -$ kubectl get pods -NAME READY STATUS RESTARTS AGE +$ kubectl get pods +NAME READY STATUS RESTARTS AGE ``` ```shell -# NOTE: your service IPs will likely differ -$ kubectl get services -NAME LABELS SELECTOR IP(S) PORT(S) -k8sm-scheduler component=scheduler,provider=k8sm 10.10.10.113 10251/TCP -kubernetes component=apiserver,provider=kubernetes 10.10.10.1 443/TCP +# NOTE: your service IPs will likely differ +$ kubectl get services +NAME LABELS SELECTOR IP(S) PORT(S) +k8sm-scheduler component=scheduler,provider=k8sm 10.10.10.113 10251/TCP +kubernetes component=apiserver,provider=kubernetes 10.10.10.1 443/TCP ``` -Lastly, look for Kubernetes in the Mesos web GUI by pointing your browser to -`http://`. Make sure you have an active VPN connection. -Go to the Frameworks tab, and look for an active framework named "Kubernetes". - -## Spin up a pod - -Write a JSON pod description to a local file: - +Lastly, look for Kubernetes in the Mesos web GUI by pointing your browser to +`http://`. Make sure you have an active VPN connection. +Go to the Frameworks tab, and look for an active framework named "Kubernetes". + +## Spin up a pod + +Write a JSON pod description to a local file: + ```shell -$ cat <nginx.yaml +$ cat <nginx.yaml ``` ```yaml -apiVersion: v1 -kind: Pod -metadata: - name: nginx -spec: - containers: - - name: nginx - image: nginx - ports: - - containerPort: 80 -EOPOD +apiVersion: v1 +kind: Pod +metadata: + name: nginx +spec: + containers: + - name: nginx + image: nginx + ports: + - containerPort: 80 +EOPOD ``` -Send the pod description to Kubernetes using the `kubectl` CLI: - +Send the pod description to Kubernetes using the `kubectl` CLI: + ```shell -$ kubectl create -f ./nginx.yaml -pods/nginx +$ kubectl create -f ./nginx.yaml +pods/nginx ``` -Wait a minute or two while `dockerd` downloads the image layers from the internet. -We can use the `kubectl` interface to monitor the status of our pod: - +Wait a minute or two while `dockerd` downloads the image layers from the internet. +We can use the `kubectl` interface to monitor the status of our pod: + ```shell -$ kubectl get pods -NAME READY STATUS RESTARTS AGE -nginx 1/1 Running 0 14s +$ kubectl get pods +NAME READY STATUS RESTARTS AGE +nginx 1/1 Running 0 14s ``` -Verify that the pod task is running in the Mesos web GUI. Click on the -Kubernetes framework. The next screen should show the running Mesos task that -started the Kubernetes pod. - -## Launching kube-dns - -Kube-dns is an addon for Kubernetes which adds DNS-based service discovery to the cluster. For a detailed explanation see [DNS in Kubernetes][4]. - -The kube-dns addon runs as a pod inside the cluster. The pod consists of three co-located containers: - -- a local etcd instance -- the [skydns][11] DNS server -- the kube2sky process to glue skydns to the state of the Kubernetes cluster. - -The skydns container offers DNS service via port 53 to the cluster. The etcd communication works via local 127.0.0.1 communication - -We assume that kube-dns will use - -- the service IP `10.10.10.10` -- and the `cluster.local` domain. - -Note that we have passed these two values already as parameter to the apiserver above. - -A template for an replication controller spinning up the pod with the 3 containers can be found at [cluster/addons/dns/skydns-rc.yaml.in][11] in the repository. The following steps are necessary in order to get a valid replication controller yaml file: - -- replace `{{ pillar['dns_replicas'] }}` with `1` -- replace `{{ pillar['dns_domain'] }}` with `cluster.local.` -- add `--kube_master_url=${KUBERNETES_MASTER}` parameter to the kube2sky container command. - -In addition the service template at [cluster/addons/dns/skydns-svc.yaml.in][12] needs the following replacement: - -- `{{ pillar['dns_server'] }}` with `10.10.10.10`. - -To do this automatically: - +Verify that the pod task is running in the Mesos web GUI. Click on the +Kubernetes framework. The next screen should show the running Mesos task that +started the Kubernetes pod. + +## Launching kube-dns + +Kube-dns is an addon for Kubernetes which adds DNS-based service discovery to the cluster. For a detailed explanation see [DNS in Kubernetes][4]. + +The kube-dns addon runs as a pod inside the cluster. The pod consists of three co-located containers: + +- a local etcd instance +- the [skydns][11] DNS server +- the kube2sky process to glue skydns to the state of the Kubernetes cluster. + +The skydns container offers DNS service via port 53 to the cluster. The etcd communication works via local 127.0.0.1 communication + +We assume that kube-dns will use + +- the service IP `10.10.10.10` +- and the `cluster.local` domain. + +Note that we have passed these two values already as parameter to the apiserver above. + +A template for an replication controller spinning up the pod with the 3 containers can be found at [cluster/addons/dns/skydns-rc.yaml.in][11] in the repository. The following steps are necessary in order to get a valid replication controller yaml file: + +- replace `{{ pillar['dns_replicas'] }}` with `1` +- replace `{{ pillar['dns_domain'] }}` with `cluster.local.` +- add `--kube_master_url=${KUBERNETES_MASTER}` parameter to the kube2sky container command. + +In addition the service template at [cluster/addons/dns/skydns-svc.yaml.in][12] needs the following replacement: + +- `{{ pillar['dns_server'] }}` with `10.10.10.10`. + +To do this automatically: + ```shell -sed -e "s/{{ pillar\['dns_replicas'\] }}/1/g;"\ -"s,\(command = \"/kube2sky\"\),\\1\\"$'\n'" - --kube_master_url=${KUBERNETES_MASTER},;"\ -"s/{{ pillar\['dns_domain'\] }}/cluster.local/g" \ - cluster/addons/dns/skydns-rc.yaml.in > skydns-rc.yaml -sed -e "s/{{ pillar\['dns_server'\] }}/10.10.10.10/g" \ - cluster/addons/dns/skydns-svc.yaml.in > skydns-svc.yaml +sed -e "s/{{ pillar\['dns_replicas'\] }}/1/g;"\ +"s,\(command = \"/kube2sky\"\),\\1\\"$'\n'" - --kube_master_url=${KUBERNETES_MASTER},;"\ +"s/{{ pillar\['dns_domain'\] }}/cluster.local/g" \ + cluster/addons/dns/skydns-rc.yaml.in > skydns-rc.yaml +sed -e "s/{{ pillar\['dns_server'\] }}/10.10.10.10/g" \ + cluster/addons/dns/skydns-svc.yaml.in > skydns-svc.yaml ``` -Now the kube-dns pod and service are ready to be launched: - +Now the kube-dns pod and service are ready to be launched: + ```shell -kubectl create -f ./skydns-rc.yaml -kubectl create -f ./skydns-svc.yaml +kubectl create -f ./skydns-rc.yaml +kubectl create -f ./skydns-svc.yaml ``` -Check with `kubectl get pods --namespace=kube-system` that 3/3 containers of the pods are eventually up and running. Note that the kube-dns pods run in the `kube-system` namespace, not in `default`. - -To check that the new DNS service in the cluster works, we start a busybox pod and use that to do a DNS lookup. First create the `busybox.yaml` pod spec: - +Check with `kubectl get pods --namespace=kube-system` that 3/3 containers of the pods are eventually up and running. Note that the kube-dns pods run in the `kube-system` namespace, not in `default`. + +To check that the new DNS service in the cluster works, we start a busybox pod and use that to do a DNS lookup. First create the `busybox.yaml` pod spec: + ```shell -cat <busybox.yaml +cat <busybox.yaml ``` ```yaml -apiVersion: v1 -kind: Pod -metadata: - name: busybox - namespace: default -spec: - containers: - - image: busybox - command: - - sleep - - "3600" - imagePullPolicy: IfNotPresent - name: busybox - restartPolicy: Always -EOF +apiVersion: v1 +kind: Pod +metadata: + name: busybox + namespace: default +spec: + containers: + - image: busybox + command: + - sleep + - "3600" + imagePullPolicy: IfNotPresent + name: busybox + restartPolicy: Always +EOF ``` -Then start the pod: - +Then start the pod: + ```shell -kubectl create -f ./busybox.yaml +kubectl create -f ./busybox.yaml ``` -When the pod is up and running, start a lookup for the Kubernetes master service, made available on 10.10.10.1 by default: - +When the pod is up and running, start a lookup for the Kubernetes master service, made available on 10.10.10.1 by default: + ```shell -kubectl exec busybox -- nslookup kubernetes +kubectl exec busybox -- nslookup kubernetes ``` -If everything works fine, you will get this output: - +If everything works fine, you will get this output: + ```shell -Server: 10.10.10.10 -Address 1: 10.10.10.10 - -Name: kubernetes -Address 1: 10.10.10.1 +Server: 10.10.10.10 +Address 1: 10.10.10.10 + +Name: kubernetes +Address 1: 10.10.10.1 ``` -## What next? - -Try out some of the standard [Kubernetes examples][9]. - -Read about Kubernetes on Mesos' architecture in the [contrib directory][13]. - -**NOTE:** Some examples require Kubernetes DNS to be installed on the cluster. -Future work will add instructions to this guide to enable support for Kubernetes DNS. - -**NOTE:** Please be aware that there are [known issues with the current Kubernetes-Mesos implementation][7]. - -[1]: http://mesosphere.com/docs/tutorials/run-hadoop-on-mesos-using-installer -[2]: http://mesosphere.com/docs/tutorials/run-spark-on-mesos -[3]: http://mesosphere.com/docs/tutorials/run-chronos-on-mesos -[4]: https://releases.k8s.io/{{page.githubbranch}}/cluster/addons/dns/README.md -[5]: http://open.mesosphere.com/getting-started/cloud/google/mesosphere/ -[6]: http://mesos.apache.org/ -[7]: https://releases.k8s.io/{{page.githubbranch}}/contrib/mesos/docs/issues.md -[8]: https://github.com/mesosphere/kubernetes-mesos/issues -[9]: https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples -[10]: http://open.mesosphere.com/getting-started/cloud/google/mesosphere/#vpn-setup -[11]: https://releases.k8s.io/{{page.githubbranch}}/cluster/addons/dns/skydns-rc.yaml.in -[12]: https://releases.k8s.io/{{page.githubbranch}}/cluster/addons/dns/skydns-svc.yaml.in +## What next? + +Try out some of the standard [Kubernetes examples][9]. + +Read about Kubernetes on Mesos' architecture in the [contrib directory][13]. + +**NOTE:** Some examples require Kubernetes DNS to be installed on the cluster. +Future work will add instructions to this guide to enable support for Kubernetes DNS. + +**NOTE:** Please be aware that there are [known issues with the current Kubernetes-Mesos implementation][7]. + +[1]: http://mesosphere.com/docs/tutorials/run-hadoop-on-mesos-using-installer +[2]: http://mesosphere.com/docs/tutorials/run-spark-on-mesos +[3]: http://mesosphere.com/docs/tutorials/run-chronos-on-mesos +[4]: https://releases.k8s.io/{{page.githubbranch}}/cluster/addons/dns/README.md +[5]: http://open.mesosphere.com/getting-started/cloud/google/mesosphere/ +[6]: http://mesos.apache.org/ +[7]: https://releases.k8s.io/{{page.githubbranch}}/contrib/mesos/docs/issues.md +[8]: https://github.com/mesosphere/kubernetes-mesos/issues +[9]: https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples +[10]: http://open.mesosphere.com/getting-started/cloud/google/mesosphere/#vpn-setup +[11]: https://releases.k8s.io/{{page.githubbranch}}/cluster/addons/dns/skydns-rc.yaml.in +[12]: https://releases.k8s.io/{{page.githubbranch}}/cluster/addons/dns/skydns-svc.yaml.in [13]: https://releases.k8s.io/{{page.githubbranch}}/contrib/mesos/README.md \ No newline at end of file diff --git a/docs/getting-started-guides/scratch.md b/docs/getting-started-guides/scratch.md index 5e82a7108c..98945817f2 100644 --- a/docs/getting-started-guides/scratch.md +++ b/docs/getting-started-guides/scratch.md @@ -1,799 +1,799 @@ --- --- - -This guide is for people who want to craft a custom Kubernetes cluster. If you -can find an existing Getting Started Guide that meets your needs on [this -list](/docs/getting-started-guides/), then we recommend using it, as you will be able to benefit -from the experience of others. However, if you have specific IaaS, networking, -configuration management, or operating system requirements not met by any of -those guides, then this guide will provide an outline of the steps you need to -take. Note that it requires considerably more effort than using one of the -pre-defined guides. - -This guide is also useful for those wanting to understand at a high level some of the -steps that existing cluster setup scripts are making. - + +This guide is for people who want to craft a custom Kubernetes cluster. If you +can find an existing Getting Started Guide that meets your needs on [this +list](/docs/getting-started-guides/), then we recommend using it, as you will be able to benefit +from the experience of others. However, if you have specific IaaS, networking, +configuration management, or operating system requirements not met by any of +those guides, then this guide will provide an outline of the steps you need to +take. Note that it requires considerably more effort than using one of the +pre-defined guides. + +This guide is also useful for those wanting to understand at a high level some of the +steps that existing cluster setup scripts are making. + * TOC -{:toc} - -## Designing and Preparing - -### Learning - - 1. You should be familiar with using Kubernetes already. We suggest you set - up a temporary cluster by following one of the other Getting Started Guides. - This will help you become familiar with the CLI ([kubectl](/docs/user-guide/kubectl/kubectl)) and concepts ([pods](/docs/user-guide/pods), [services](/docs/user-guide/services), etc.) first. - 1. You should have `kubectl` installed on your desktop. This will happen as a side - effect of completing one of the other Getting Started Guides. If not, follow the instructions - [here](/docs/user-guide/prereqs). - -### Cloud Provider - -Kubernetes has the concept of a Cloud Provider, which is a module which provides -an interface for managing TCP Load Balancers, Nodes (Instances) and Networking Routes. -The interface is defined in `pkg/cloudprovider/cloud.go`. It is possible to -create a custom cluster without implementing a cloud provider (for example if using -bare-metal), and not all parts of the interface need to be implemented, depending -on how flags are set on various components. - -### Nodes - -- You can use virtual or physical machines. -- While you can build a cluster with 1 machine, in order to run all the examples and tests you - need at least 4 nodes. -- Many Getting-started-guides make a distinction between the master node and regular nodes. This - is not strictly necessary. -- Nodes will need to run some version of Linux with the x86_64 architecture. It may be possible - to run on other OSes and Architectures, but this guide does not try to assist with that. -- Apiserver and etcd together are fine on a machine with 1 core and 1GB RAM for clusters with 10s of nodes. - Larger or more active clusters may benefit from more cores. -- Other nodes can have any reasonable amount of memory and any number of cores. They need not - have identical configurations. - -### Network - -Kubernetes has a distinctive [networking model](/docs/admin/networking). - -Kubernetes allocates an IP address to each pod. When creating a cluster, you -need to allocate a block of IPs for Kubernetes to use as Pod IPs. The simplest -approach is to allocate a different block of IPs to each node in the cluster as -the node is added. A process in one pod should be able to communicate with -another pod using the IP of the second pod. This connectivity can be +{:toc} + +## Designing and Preparing + +### Learning + + 1. You should be familiar with using Kubernetes already. We suggest you set + up a temporary cluster by following one of the other Getting Started Guides. + This will help you become familiar with the CLI ([kubectl](/docs/user-guide/kubectl/kubectl)) and concepts ([pods](/docs/user-guide/pods), [services](/docs/user-guide/services), etc.) first. + 1. You should have `kubectl` installed on your desktop. This will happen as a side + effect of completing one of the other Getting Started Guides. If not, follow the instructions + [here](/docs/user-guide/prereqs). + +### Cloud Provider + +Kubernetes has the concept of a Cloud Provider, which is a module which provides +an interface for managing TCP Load Balancers, Nodes (Instances) and Networking Routes. +The interface is defined in `pkg/cloudprovider/cloud.go`. It is possible to +create a custom cluster without implementing a cloud provider (for example if using +bare-metal), and not all parts of the interface need to be implemented, depending +on how flags are set on various components. + +### Nodes + +- You can use virtual or physical machines. +- While you can build a cluster with 1 machine, in order to run all the examples and tests you + need at least 4 nodes. +- Many Getting-started-guides make a distinction between the master node and regular nodes. This + is not strictly necessary. +- Nodes will need to run some version of Linux with the x86_64 architecture. It may be possible + to run on other OSes and Architectures, but this guide does not try to assist with that. +- Apiserver and etcd together are fine on a machine with 1 core and 1GB RAM for clusters with 10s of nodes. + Larger or more active clusters may benefit from more cores. +- Other nodes can have any reasonable amount of memory and any number of cores. They need not + have identical configurations. + +### Network + +Kubernetes has a distinctive [networking model](/docs/admin/networking). + +Kubernetes allocates an IP address to each pod. When creating a cluster, you +need to allocate a block of IPs for Kubernetes to use as Pod IPs. The simplest +approach is to allocate a different block of IPs to each node in the cluster as +the node is added. A process in one pod should be able to communicate with +another pod using the IP of the second pod. This connectivity can be accomplished in two ways: - -- Configure network to route Pod IPs - - Harder to setup from scratch. - - Google Compute Engine ([GCE](/docs/getting-started-guides/gce)) and [AWS](/docs/getting-started-guides/aws) guides use this approach. - - Need to make the Pod IPs routable by programming routers, switches, etc. - - Can be configured external to Kubernetes, or can implement in the "Routes" interface of a Cloud Provider module. - - Generally highest performance. -- Create an Overlay network - - Easier to setup - - Traffic is encapsulated, so per-pod IPs are routable. - - Examples: - - [Flannel](https://github.com/coreos/flannel) - - [Weave](http://weave.works/) - - [Open vSwitch (OVS)](http://openvswitch.org/) - - Does not require "Routes" portion of Cloud Provider module. - - Reduced performance (exactly how much depends on your solution). - + +- Configure network to route Pod IPs + - Harder to setup from scratch. + - Google Compute Engine ([GCE](/docs/getting-started-guides/gce)) and [AWS](/docs/getting-started-guides/aws) guides use this approach. + - Need to make the Pod IPs routable by programming routers, switches, etc. + - Can be configured external to Kubernetes, or can implement in the "Routes" interface of a Cloud Provider module. + - Generally highest performance. +- Create an Overlay network + - Easier to setup + - Traffic is encapsulated, so per-pod IPs are routable. + - Examples: + - [Flannel](https://github.com/coreos/flannel) + - [Weave](http://weave.works/) + - [Open vSwitch (OVS)](http://openvswitch.org/) + - Does not require "Routes" portion of Cloud Provider module. + - Reduced performance (exactly how much depends on your solution). + You need to select an address range for the Pod IPs. - -- Various approaches: - - GCE: each project has its own `10.0.0.0/8`. Carve off a `/16` for each - Kubernetes cluster from that space, which leaves room for several clusters. - Each node gets a further subdivision of this space. - - AWS: use one VPC for whole organization, carve off a chunk for each - cluster, or use different VPC for different clusters. - - IPv6 is not supported yet. -- Allocate one CIDR subnet for each node's PodIPs, or a single large CIDR - from which smaller CIDRs are automatically allocated to each node (if nodes - are dynamically added). - - You need max-pods-per-node * max-number-of-nodes IPs in total. A `/24` per - node supports 254 pods per machine and is a common choice. If IPs are - scarce, a `/26` (62 pods per machine) or even a `/27` (30 pods) may be sufficient. - - e.g. use `10.10.0.0/16` as the range for the cluster, with up to 256 nodes - using `10.10.0.0/24` through `10.10.255.0/24`, respectively. - - Need to make these routable or connect with overlay. - -Kubernetes also allocates an IP to each [service](/docs/user-guide/services). However, -service IPs do not necessarily need to be routable. The kube-proxy takes care -of translating Service IPs to Pod IPs before traffic leaves the node. You do -need to Allocate a block of IPs for services. Call this -`SERVICE_CLUSTER_IP_RANGE`. For example, you could set -`SERVICE_CLUSTER_IP_RANGE="10.0.0.0/16"`, allowing 65534 distinct services to -be active at once. Note that you can grow the end of this range, but you -cannot move it without disrupting the services and pods that already use it. - -Also, you need to pick a static IP for master node. -- Call this `MASTER_IP`. -- Open any firewalls to allow access to the apiserver ports 80 and/or 443. -- Enable ipv4 forwarding sysctl, `net.ipv4.ip_forward = 1` - -### Cluster Naming - -You should pick a name for your cluster. Pick a short name for each cluster + +- Various approaches: + - GCE: each project has its own `10.0.0.0/8`. Carve off a `/16` for each + Kubernetes cluster from that space, which leaves room for several clusters. + Each node gets a further subdivision of this space. + - AWS: use one VPC for whole organization, carve off a chunk for each + cluster, or use different VPC for different clusters. + - IPv6 is not supported yet. +- Allocate one CIDR subnet for each node's PodIPs, or a single large CIDR + from which smaller CIDRs are automatically allocated to each node (if nodes + are dynamically added). + - You need max-pods-per-node * max-number-of-nodes IPs in total. A `/24` per + node supports 254 pods per machine and is a common choice. If IPs are + scarce, a `/26` (62 pods per machine) or even a `/27` (30 pods) may be sufficient. + - e.g. use `10.10.0.0/16` as the range for the cluster, with up to 256 nodes + using `10.10.0.0/24` through `10.10.255.0/24`, respectively. + - Need to make these routable or connect with overlay. + +Kubernetes also allocates an IP to each [service](/docs/user-guide/services). However, +service IPs do not necessarily need to be routable. The kube-proxy takes care +of translating Service IPs to Pod IPs before traffic leaves the node. You do +need to Allocate a block of IPs for services. Call this +`SERVICE_CLUSTER_IP_RANGE`. For example, you could set +`SERVICE_CLUSTER_IP_RANGE="10.0.0.0/16"`, allowing 65534 distinct services to +be active at once. Note that you can grow the end of this range, but you +cannot move it without disrupting the services and pods that already use it. + +Also, you need to pick a static IP for master node. +- Call this `MASTER_IP`. +- Open any firewalls to allow access to the apiserver ports 80 and/or 443. +- Enable ipv4 forwarding sysctl, `net.ipv4.ip_forward = 1` + +### Cluster Naming + +You should pick a name for your cluster. Pick a short name for each cluster which is unique from future cluster names. This will be used in several ways: - - - by kubectl to distinguish between various clusters you have access to. You will probably want a - second one sometime later, such as for testing new Kubernetes releases, running in a different -region of the world, etc. - - Kubernetes clusters can create cloud provider resources (e.g. AWS ELBs) and different clusters - need to distinguish which resources each created. Call this `CLUSTERNAME`. - -### Software Binaries - + + - by kubectl to distinguish between various clusters you have access to. You will probably want a + second one sometime later, such as for testing new Kubernetes releases, running in a different +region of the world, etc. + - Kubernetes clusters can create cloud provider resources (e.g. AWS ELBs) and different clusters + need to distinguish which resources each created. Call this `CLUSTERNAME`. + +### Software Binaries + You will need binaries for: - - - etcd - - A container runner, one of: - - docker - - rkt - - Kubernetes - - kubelet - - kube-proxy - - kube-apiserver - - kube-controller-manager - - kube-scheduler - -#### Downloading and Extracting Kubernetes Binaries - -A Kubernetes binary release includes all the Kubernetes binaries as well as the supported release of etcd. -You can use a Kubernetes binary release (recommended) or build your Kubernetes binaries following the instructions in the -[Developer Documentation](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/devel/). Only using a binary release is covered in this guide. - -Download the [latest binary release](https://github.com/kubernetes/kubernetes/releases/latest) and unzip it. -Then locate `./kubernetes/server/kubernetes-server-linux-amd64.tar.gz` and unzip *that*. -Then, within the second set of unzipped files, locate `./kubernetes/server/bin`, which contains -all the necessary binaries. - -#### Selecting Images - -You will run docker, kubelet, and kube-proxy outside of a container, the same way you would run any system daemon, so -you just need the bare binaries. For etcd, kube-apiserver, kube-controller-manager, and kube-scheduler, -we recommend that you run these as containers, so you need an image to be built. - + + - etcd + - A container runner, one of: + - docker + - rkt + - Kubernetes + - kubelet + - kube-proxy + - kube-apiserver + - kube-controller-manager + - kube-scheduler + +#### Downloading and Extracting Kubernetes Binaries + +A Kubernetes binary release includes all the Kubernetes binaries as well as the supported release of etcd. +You can use a Kubernetes binary release (recommended) or build your Kubernetes binaries following the instructions in the +[Developer Documentation](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/devel/). Only using a binary release is covered in this guide. + +Download the [latest binary release](https://github.com/kubernetes/kubernetes/releases/latest) and unzip it. +Then locate `./kubernetes/server/kubernetes-server-linux-amd64.tar.gz` and unzip *that*. +Then, within the second set of unzipped files, locate `./kubernetes/server/bin`, which contains +all the necessary binaries. + +#### Selecting Images + +You will run docker, kubelet, and kube-proxy outside of a container, the same way you would run any system daemon, so +you just need the bare binaries. For etcd, kube-apiserver, kube-controller-manager, and kube-scheduler, +we recommend that you run these as containers, so you need an image to be built. + You have several choices for Kubernetes images: - -- Use images hosted on Google Container Registry (GCR): - - e.g `gcr.io/google_containers/hyperkube:$TAG`, where `TAG` is the latest - release tag, which can be found on the [latest releases page](https://github.com/kubernetes/kubernetes/releases/latest). - - Ensure $TAG is the same tag as the release tag you are using for kubelet and kube-proxy. - - The [hyperkube](https://releases.k8s.io/{{page.githubbranch}}/cmd/hyperkube) binary is an all in one binary - - `hyperkube kubelet ...` runs the kublet, `hyperkube apiserver ...` runs an apiserver, etc. -- Build your own images. - - Useful if you are using a private registry. - - The release contains files such as `./kubernetes/server/bin/kube-apiserver.tar` which - can be converted into docker images using a command like - `docker load -i kube-apiserver.tar` - - You can verify if the image is loaded successfully with the right repository and tag using - command like `docker images` - + +- Use images hosted on Google Container Registry (GCR): + - e.g `gcr.io/google_containers/hyperkube:$TAG`, where `TAG` is the latest + release tag, which can be found on the [latest releases page](https://github.com/kubernetes/kubernetes/releases/latest). + - Ensure $TAG is the same tag as the release tag you are using for kubelet and kube-proxy. + - The [hyperkube](https://releases.k8s.io/{{page.githubbranch}}/cmd/hyperkube) binary is an all in one binary + - `hyperkube kubelet ...` runs the kublet, `hyperkube apiserver ...` runs an apiserver, etc. +- Build your own images. + - Useful if you are using a private registry. + - The release contains files such as `./kubernetes/server/bin/kube-apiserver.tar` which + can be converted into docker images using a command like + `docker load -i kube-apiserver.tar` + - You can verify if the image is loaded successfully with the right repository and tag using + command like `docker images` + For etcd, you can: -- Use images hosted on Google Container Registry (GCR), such as `gcr.io/google_containers/etcd:2.2.1` -- Use images hosted on [Docker Hub](https://hub.docker.com/search/?q=etcd) or [Quay.io](https://quay.io/repository/coreos/etcd), such as `quay.io/coreos/etcd:v2.2.1` -- Use etcd binary included in your OS distro. -- Build your own image - - You can do: `cd kubernetes/cluster/images/etcd; make` - -We recommend that you use the etcd version which is provided in the Kubernetes binary distribution. The Kubernetes binaries in the release -were tested extensively with this version of etcd and not with any other version. -The recommended version number can also be found as the value of `ETCD_VERSION` in `kubernetes/cluster/images/etcd/Makefile`. - +- Use images hosted on Google Container Registry (GCR), such as `gcr.io/google_containers/etcd:2.2.1` +- Use images hosted on [Docker Hub](https://hub.docker.com/search/?q=etcd) or [Quay.io](https://quay.io/repository/coreos/etcd), such as `quay.io/coreos/etcd:v2.2.1` +- Use etcd binary included in your OS distro. +- Build your own image + - You can do: `cd kubernetes/cluster/images/etcd; make` + +We recommend that you use the etcd version which is provided in the Kubernetes binary distribution. The Kubernetes binaries in the release +were tested extensively with this version of etcd and not with any other version. +The recommended version number can also be found as the value of `ETCD_VERSION` in `kubernetes/cluster/images/etcd/Makefile`. + The remainder of the document assumes that the image identifiers have been chosen and stored in corresponding env vars. Examples (replace with latest tags and appropriate registry): - - - `HYPERKUBE_IMAGE==gcr.io/google_containers/hyperkube:$TAG` - - `ETCD_IMAGE=gcr.io/google_containers/etcd:$ETCD_VERSION` - -### Security Models - + + - `HYPERKUBE_IMAGE==gcr.io/google_containers/hyperkube:$TAG` + - `ETCD_IMAGE=gcr.io/google_containers/etcd:$ETCD_VERSION` + +### Security Models + There are two main options for security: - -- Access the apiserver using HTTP. - - Use a firewall for security. - - This is easier to setup. -- Access the apiserver using HTTPS - - Use https with certs, and credentials for user. - - This is the recommended approach. - - Configuring certs can be tricky. - -If following the HTTPS approach, you will need to prepare certs and credentials. - -#### Preparing Certs - + +- Access the apiserver using HTTP. + - Use a firewall for security. + - This is easier to setup. +- Access the apiserver using HTTPS + - Use https with certs, and credentials for user. + - This is the recommended approach. + - Configuring certs can be tricky. + +If following the HTTPS approach, you will need to prepare certs and credentials. + +#### Preparing Certs + You need to prepare several certs: - -- The master needs a cert to act as an HTTPS server. -- The kubelets optionally need certs to identify themselves as clients of the master, and when - serving its own API over HTTPS. - + +- The master needs a cert to act as an HTTPS server. +- The kubelets optionally need certs to identify themselves as clients of the master, and when + serving its own API over HTTPS. + Unless you plan to have a real CA generate your certs, you will need to generate a root cert and use that to sign the master, kubelet, and kubectl certs. -- see function `create-certs` in `cluster/gce/util.sh` -- see also `cluster/saltbase/salt/generate-cert/make-ca-cert.sh` and - `cluster/saltbase/salt/generate-cert/make-cert.sh` - +- see function `create-certs` in `cluster/gce/util.sh` +- see also `cluster/saltbase/salt/generate-cert/make-ca-cert.sh` and + `cluster/saltbase/salt/generate-cert/make-cert.sh` + You will end up with the following files (we will use these variables later on) -- `CA_CERT` - - put in on node where apiserver runs, in e.g. `/srv/kubernetes/ca.crt`. -- `MASTER_CERT` - - signed by CA_CERT - - put in on node where apiserver runs, in e.g. `/srv/kubernetes/server.crt` -- `MASTER_KEY ` - - put in on node where apiserver runs, in e.g. `/srv/kubernetes/server.key` -- `KUBELET_CERT` - - optional -- `KUBELET_KEY` - - optional - -#### Preparing Credentials - +- `CA_CERT` + - put in on node where apiserver runs, in e.g. `/srv/kubernetes/ca.crt`. +- `MASTER_CERT` + - signed by CA_CERT + - put in on node where apiserver runs, in e.g. `/srv/kubernetes/server.crt` +- `MASTER_KEY ` + - put in on node where apiserver runs, in e.g. `/srv/kubernetes/server.key` +- `KUBELET_CERT` + - optional +- `KUBELET_KEY` + - optional + +#### Preparing Credentials + The admin user (and any users) need: - - - a token or a password to identify them. - - tokens are just long alphanumeric strings, e.g. 32 chars. See - - `TOKEN=$(dd if=/dev/urandom bs=128 count=1 2>/dev/null | base64 | tr -d "=+/" | dd bs=32 count=1 2>/dev/null)` - -Your tokens and passwords need to be stored in a file for the apiserver -to read. This guide uses `/var/lib/kube-apiserver/known_tokens.csv`. -The format for this file is described in the [authentication documentation](/docs/admin/authentication). - -For distributing credentials to clients, the convention in Kubernetes is to put the credentials -into a [kubeconfig file](/docs/user-guide/kubeconfig-file). - + + - a token or a password to identify them. + - tokens are just long alphanumeric strings, e.g. 32 chars. See + - `TOKEN=$(dd if=/dev/urandom bs=128 count=1 2>/dev/null | base64 | tr -d "=+/" | dd bs=32 count=1 2>/dev/null)` + +Your tokens and passwords need to be stored in a file for the apiserver +to read. This guide uses `/var/lib/kube-apiserver/known_tokens.csv`. +The format for this file is described in the [authentication documentation](/docs/admin/authentication). + +For distributing credentials to clients, the convention in Kubernetes is to put the credentials +into a [kubeconfig file](/docs/user-guide/kubeconfig-file). + The kubeconfig file for the administrator can be created as follows: - - - If you have already used Kubernetes with a non-custom cluster (for example, used a Getting Started - Guide), you will already have a `$HOME/.kube/config` file. - - You need to add certs, keys, and the master IP to the kubeconfig file: - - If using the firewall-only security option, set the apiserver this way: - - `kubectl config set-cluster $CLUSTER_NAME --server=http://$MASTER_IP --insecure-skip-tls-verify=true` - - Otherwise, do this to set the apiserver ip, client certs, and user credentials. - - `kubectl config set-cluster $CLUSTER_NAME --certificate-authority=$CA_CERT --embed-certs=true --server=https://$MASTER_IP` - - `kubectl config set-credentials $USER --client-certificate=$CLI_CERT --client-key=$CLI_KEY --embed-certs=true --token=$TOKEN` - - Set your cluster as the default cluster to use: - - `kubectl config set-context $CONTEXT_NAME --cluster=$CLUSTER_NAME --user=$USER` - - `kubectl config use-context $CONTEXT_NAME` - -Next, make a kubeconfig file for the kubelets and kube-proxy. There are a couple of options for how + + - If you have already used Kubernetes with a non-custom cluster (for example, used a Getting Started + Guide), you will already have a `$HOME/.kube/config` file. + - You need to add certs, keys, and the master IP to the kubeconfig file: + - If using the firewall-only security option, set the apiserver this way: + - `kubectl config set-cluster $CLUSTER_NAME --server=http://$MASTER_IP --insecure-skip-tls-verify=true` + - Otherwise, do this to set the apiserver ip, client certs, and user credentials. + - `kubectl config set-cluster $CLUSTER_NAME --certificate-authority=$CA_CERT --embed-certs=true --server=https://$MASTER_IP` + - `kubectl config set-credentials $USER --client-certificate=$CLI_CERT --client-key=$CLI_KEY --embed-certs=true --token=$TOKEN` + - Set your cluster as the default cluster to use: + - `kubectl config set-context $CONTEXT_NAME --cluster=$CLUSTER_NAME --user=$USER` + - `kubectl config use-context $CONTEXT_NAME` + +Next, make a kubeconfig file for the kubelets and kube-proxy. There are a couple of options for how many distinct files to make: - - 1. Use the same credential as the admin - - This is simplest to setup. - 1. One token and kubeconfig file for all kubelets, one for all kube-proxy, one for admin. - - This mirrors what is done on GCE today - 1. Different credentials for every kubelet, etc. - - We are working on this but all the pieces are not ready yet. - -You can make the files by copying the `$HOME/.kube/config`, by following the code -in `cluster/gce/configure-vm.sh` or by using the following template: - + + 1. Use the same credential as the admin + - This is simplest to setup. + 1. One token and kubeconfig file for all kubelets, one for all kube-proxy, one for admin. + - This mirrors what is done on GCE today + 1. Different credentials for every kubelet, etc. + - We are working on this but all the pieces are not ready yet. + +You can make the files by copying the `$HOME/.kube/config`, by following the code +in `cluster/gce/configure-vm.sh` or by using the following template: + ```yaml -apiVersion: v1 -kind: Config -users: -- name: kubelet - user: - token: ${KUBELET_TOKEN} -clusters: -- name: local - cluster: - certificate-authority-data: ${CA_CERT_BASE64_ENCODED} -contexts: -- context: - cluster: local - user: kubelet - name: service-account-context -current-context: service-account-context +apiVersion: v1 +kind: Config +users: +- name: kubelet + user: + token: ${KUBELET_TOKEN} +clusters: +- name: local + cluster: + certificate-authority-data: ${CA_CERT_BASE64_ENCODED} +contexts: +- context: + cluster: local + user: kubelet + name: service-account-context +current-context: service-account-context ``` -Put the kubeconfig(s) on every node. The examples later in this -guide assume that there are kubeconfigs in `/var/lib/kube-proxy/kubeconfig` and -`/var/lib/kubelet/kubeconfig`. - -## Configuring and Installing Base Software on Nodes - -This section discusses how to configure machines to be Kubernetes nodes. - +Put the kubeconfig(s) on every node. The examples later in this +guide assume that there are kubeconfigs in `/var/lib/kube-proxy/kubeconfig` and +`/var/lib/kubelet/kubeconfig`. + +## Configuring and Installing Base Software on Nodes + +This section discusses how to configure machines to be Kubernetes nodes. + You should run three daemons on every node: - - - docker or rkt - - kubelet - - kube-proxy - -You will also need to do assorted other configuration on top of a -base OS install. - -Tip: One possible starting point is to setup a cluster using an existing Getting -Started Guide. After getting a cluster running, you can then copy the init.d scripts or systemd unit files from that -cluster, and then modify them for use on your custom cluster. - -### Docker - -The minimum required Docker version will vary as the kubelet version changes. The newest stable release is a good choice. Kubelet will log a warning and refuse to start pods if the version is too old, so pick a version and try it. - -If you previously had Docker installed on a node without setting Kubernetes-specific -options, you may have a Docker-created bridge and iptables rules. You may want to remove these -as follows before proceeding to configure Docker for Kubernetes. - + + - docker or rkt + - kubelet + - kube-proxy + +You will also need to do assorted other configuration on top of a +base OS install. + +Tip: One possible starting point is to setup a cluster using an existing Getting +Started Guide. After getting a cluster running, you can then copy the init.d scripts or systemd unit files from that +cluster, and then modify them for use on your custom cluster. + +### Docker + +The minimum required Docker version will vary as the kubelet version changes. The newest stable release is a good choice. Kubelet will log a warning and refuse to start pods if the version is too old, so pick a version and try it. + +If you previously had Docker installed on a node without setting Kubernetes-specific +options, you may have a Docker-created bridge and iptables rules. You may want to remove these +as follows before proceeding to configure Docker for Kubernetes. + ```shell -iptables -t nat -F -ifconfig docker0 down -brctl delbr docker0 +iptables -t nat -F +ifconfig docker0 down +brctl delbr docker0 ``` -The way you configure docker will depend in whether you have chosen the routable-vip or overlay-network approaches for your network. +The way you configure docker will depend in whether you have chosen the routable-vip or overlay-network approaches for your network. Some suggested docker options: - - - create your own bridge for the per-node CIDR ranges, call it cbr0, and set `--bridge=cbr0` option on docker. - - set `--iptables=false` so docker will not manipulate iptables for host-ports (too coarse on older docker versions, may be fixed in newer versions) -so that kube-proxy can manage iptables instead of docker. - - `--ip-masq=false` - - if you have setup PodIPs to be routable, then you want this false, otherwise, docker will - rewrite the PodIP source-address to a NodeIP. - - some environments (e.g. GCE) still need you to masquerade out-bound traffic when it leaves the cloud environment. This is very environment specific. - - if you are using an overlay network, consult those instructions. - - `--mtu=` - - may be required when using Flannel, because of the extra packet size due to udp encapsulation - - `--insecure-registry $CLUSTER_SUBNET` - - to connect to a private registry, if you set one up, without using SSL. - + + - create your own bridge for the per-node CIDR ranges, call it cbr0, and set `--bridge=cbr0` option on docker. + - set `--iptables=false` so docker will not manipulate iptables for host-ports (too coarse on older docker versions, may be fixed in newer versions) +so that kube-proxy can manage iptables instead of docker. + - `--ip-masq=false` + - if you have setup PodIPs to be routable, then you want this false, otherwise, docker will + rewrite the PodIP source-address to a NodeIP. + - some environments (e.g. GCE) still need you to masquerade out-bound traffic when it leaves the cloud environment. This is very environment specific. + - if you are using an overlay network, consult those instructions. + - `--mtu=` + - may be required when using Flannel, because of the extra packet size due to udp encapsulation + - `--insecure-registry $CLUSTER_SUBNET` + - to connect to a private registry, if you set one up, without using SSL. + You may want to increase the number of open files for docker: - - - `DOCKER_NOFILE=1000000` - -Where this config goes depends on your node OS. For example, GCE's Debian-based distro uses `/etc/default/docker`. - -Ensure docker is working correctly on your system before proceeding with the rest of the -installation, by following examples given in the Docker documentation. - -### rkt - -[rkt](https://github.com/coreos/rkt) is an alternative to Docker. You only need to install one of Docker or rkt. -The minimum version required is [v0.5.6](https://github.com/coreos/rkt/releases/tag/v0.5.6). - -[systemd](http://www.freedesktop.org/wiki/Software/systemd/) is required on your node to run rkt. The -minimum version required to match rkt v0.5.6 is -[systemd 215](http://lists.freedesktop.org/archives/systemd-devel/2014-July/020903.html). - -[rkt metadata service](https://github.com/coreos/rkt/blob/master/Documentation/networking.md) is also required -for rkt networking support. You can start rkt metadata service by using command like -`sudo systemd-run rkt metadata-service` - + + - `DOCKER_NOFILE=1000000` + +Where this config goes depends on your node OS. For example, GCE's Debian-based distro uses `/etc/default/docker`. + +Ensure docker is working correctly on your system before proceeding with the rest of the +installation, by following examples given in the Docker documentation. + +### rkt + +[rkt](https://github.com/coreos/rkt) is an alternative to Docker. You only need to install one of Docker or rkt. +The minimum version required is [v0.5.6](https://github.com/coreos/rkt/releases/tag/v0.5.6). + +[systemd](http://www.freedesktop.org/wiki/Software/systemd/) is required on your node to run rkt. The +minimum version required to match rkt v0.5.6 is +[systemd 215](http://lists.freedesktop.org/archives/systemd-devel/2014-July/020903.html). + +[rkt metadata service](https://github.com/coreos/rkt/blob/master/Documentation/networking.md) is also required +for rkt networking support. You can start rkt metadata service by using command like +`sudo systemd-run rkt metadata-service` + Then you need to configure your kubelet with flag: - - - `--container-runtime=rkt` - -### kubelet - -All nodes should run kubelet. See [Selecting Binaries](#selecting-binaries). - + + - `--container-runtime=rkt` + +### kubelet + +All nodes should run kubelet. See [Selecting Binaries](#selecting-binaries). + Arguments to consider: - - - If following the HTTPS security approach: - - `--api-servers=https://$MASTER_IP` - - `--kubeconfig=/var/lib/kubelet/kubeconfig` - - Otherwise, if taking the firewall-based security approach - - `--api-servers=http://$MASTER_IP` - - `--config=/etc/kubernetes/manifests` - - `--cluster-dns=` to the address of the DNS server you will setup (see [Starting Cluster Services](#starting-cluster-services).) - - `--cluster-domain=` to the dns domain prefix to use for cluster DNS addresses. - - `--docker-root=` - - `--root-dir=` - - `--configure-cbr0=` (described above) - - `--register-node` (described in [Node](/docs/admin/node) documentation.) - -### kube-proxy - -All nodes should run kube-proxy. (Running kube-proxy on a "master" node is not -strictly required, but being consistent is easier.) Obtain a binary as described for -kubelet. - + + - If following the HTTPS security approach: + - `--api-servers=https://$MASTER_IP` + - `--kubeconfig=/var/lib/kubelet/kubeconfig` + - Otherwise, if taking the firewall-based security approach + - `--api-servers=http://$MASTER_IP` + - `--config=/etc/kubernetes/manifests` + - `--cluster-dns=` to the address of the DNS server you will setup (see [Starting Cluster Services](#starting-cluster-services).) + - `--cluster-domain=` to the dns domain prefix to use for cluster DNS addresses. + - `--docker-root=` + - `--root-dir=` + - `--configure-cbr0=` (described above) + - `--register-node` (described in [Node](/docs/admin/node) documentation.) + +### kube-proxy + +All nodes should run kube-proxy. (Running kube-proxy on a "master" node is not +strictly required, but being consistent is easier.) Obtain a binary as described for +kubelet. + Arguments to consider: - - - If following the HTTPS security approach: - - `--api-servers=https://$MASTER_IP` - - `--kubeconfig=/var/lib/kube-proxy/kubeconfig` - - Otherwise, if taking the firewall-based security approach - - `--api-servers=http://$MASTER_IP` - -### Networking - -Each node needs to be allocated its own CIDR range for pod networking. -Call this `NODE_X_POD_CIDR`. - -A bridge called `cbr0` needs to be created on each node. The bridge is explained -further in the [networking documentation](/docs/admin/networking). The bridge itself -needs an address from `$NODE_X_POD_CIDR` - by convention the first IP. Call -this `NODE_X_BRIDGE_ADDR`. For example, if `NODE_X_POD_CIDR` is `10.0.0.0/16`, -then `NODE_X_BRIDGE_ADDR` is `10.0.0.1/16`. NOTE: this retains the `/16` suffix -because of how this is used later. - + + - If following the HTTPS security approach: + - `--api-servers=https://$MASTER_IP` + - `--kubeconfig=/var/lib/kube-proxy/kubeconfig` + - Otherwise, if taking the firewall-based security approach + - `--api-servers=http://$MASTER_IP` + +### Networking + +Each node needs to be allocated its own CIDR range for pod networking. +Call this `NODE_X_POD_CIDR`. + +A bridge called `cbr0` needs to be created on each node. The bridge is explained +further in the [networking documentation](/docs/admin/networking). The bridge itself +needs an address from `$NODE_X_POD_CIDR` - by convention the first IP. Call +this `NODE_X_BRIDGE_ADDR`. For example, if `NODE_X_POD_CIDR` is `10.0.0.0/16`, +then `NODE_X_BRIDGE_ADDR` is `10.0.0.1/16`. NOTE: this retains the `/16` suffix +because of how this is used later. + - Recommended, automatic approach: - 1. Set `--configure-cbr0=true` option in kubelet init script and restart kubelet service. Kubelet will configure cbr0 automatically. - It will wait to do this until the node controller has set Node.Spec.PodCIDR. Since you have not setup apiserver and node controller - yet, the bridge will not be setup immediately. + 1. Set `--configure-cbr0=true` option in kubelet init script and restart kubelet service. Kubelet will configure cbr0 automatically. + It will wait to do this until the node controller has set Node.Spec.PodCIDR. Since you have not setup apiserver and node controller + yet, the bridge will not be setup immediately. - Alternate, manual approach: - 1. Set `--configure-cbr0=false` on kubelet and restart. - 1. Create a bridge - - `brctl addbr cbr0`. - 1. Set appropriate MTU. NOTE: the actual value of MTU will depend on your network environment + 1. Set `--configure-cbr0=false` on kubelet and restart. + 1. Create a bridge + - `brctl addbr cbr0`. + 1. Set appropriate MTU. NOTE: the actual value of MTU will depend on your network environment - `ip link set dev cbr0 mtu 1460` - 1. Add the node's network to the bridge (docker will go on other side of bridge). - - `ip addr add $NODE_X_BRIDGE_ADDR dev cbr0` - 1. Turn it on - - `ip link set dev cbr0 up` - -If you have turned off Docker's IP masquerading to allow pods to talk to each -other, then you may need to do masquerading just for destination IPs outside -the cluster network. For example: - + 1. Add the node's network to the bridge (docker will go on other side of bridge). + - `ip addr add $NODE_X_BRIDGE_ADDR dev cbr0` + 1. Turn it on + - `ip link set dev cbr0 up` + +If you have turned off Docker's IP masquerading to allow pods to talk to each +other, then you may need to do masquerading just for destination IPs outside +the cluster network. For example: + ```shell -iptables -t nat -A POSTROUTING ! -d ${CLUSTER_SUBNET} -m addrtype ! --dst-type LOCAL -j MASQUERADE +iptables -t nat -A POSTROUTING ! -d ${CLUSTER_SUBNET} -m addrtype ! --dst-type LOCAL -j MASQUERADE ``` -This will rewrite the source address from -the PodIP to the Node IP for traffic bound outside the cluster, and kernel -[connection tracking](http://www.iptables.info/en/connection-state.html) -will ensure that responses destined to the node still reach -the pod. - -NOTE: This is environment specific. Some environments will not need -any masquerading at all. Others, such as GCE, will not allow pod IPs to send -traffic to the internet, but have no problem with them inside your GCE Project. - -### Other - -- Enable auto-upgrades for your OS package manager, if desired. -- Configure log rotation for all node components (e.g. using [logrotate](http://linux.die.net/man/8/logrotate)). -- Setup liveness-monitoring (e.g. using [supervisord](http://supervisord.org/)). -- Setup volume plugin support (optional) - - Install any client binaries for optional volume types, such as `glusterfs-client` for GlusterFS - volumes. - -### Using Configuration Management - -The previous steps all involved "conventional" system administration techniques for setting up -machines. You may want to use a Configuration Management system to automate the node configuration -process. There are examples of [Saltstack](/docs/admin/salt), Ansible, Juju, and CoreOS Cloud Config in the -various Getting Started Guides. - -## Bootstrapping the Cluster - -While the basic node services (kubelet, kube-proxy, docker) are typically started and managed using -traditional system administration/automation approaches, the remaining *master* components of Kubernetes are +This will rewrite the source address from +the PodIP to the Node IP for traffic bound outside the cluster, and kernel +[connection tracking](http://www.iptables.info/en/connection-state.html) +will ensure that responses destined to the node still reach +the pod. + +NOTE: This is environment specific. Some environments will not need +any masquerading at all. Others, such as GCE, will not allow pod IPs to send +traffic to the internet, but have no problem with them inside your GCE Project. + +### Other + +- Enable auto-upgrades for your OS package manager, if desired. +- Configure log rotation for all node components (e.g. using [logrotate](http://linux.die.net/man/8/logrotate)). +- Setup liveness-monitoring (e.g. using [supervisord](http://supervisord.org/)). +- Setup volume plugin support (optional) + - Install any client binaries for optional volume types, such as `glusterfs-client` for GlusterFS + volumes. + +### Using Configuration Management + +The previous steps all involved "conventional" system administration techniques for setting up +machines. You may want to use a Configuration Management system to automate the node configuration +process. There are examples of [Saltstack](/docs/admin/salt), Ansible, Juju, and CoreOS Cloud Config in the +various Getting Started Guides. + +## Bootstrapping the Cluster + +While the basic node services (kubelet, kube-proxy, docker) are typically started and managed using +traditional system administration/automation approaches, the remaining *master* components of Kubernetes are all configured and managed *by Kubernetes*: - - - their options are specified in a Pod spec (yaml or json) rather than an /etc/init.d file or - systemd unit. - - they are kept running by Kubernetes rather than by init. - -### etcd - + + - their options are specified in a Pod spec (yaml or json) rather than an /etc/init.d file or + systemd unit. + - they are kept running by Kubernetes rather than by init. + +### etcd + You will need to run one or more instances of etcd. - - - Recommended approach: run one etcd instance, with its log written to a directory backed - by durable storage (RAID, GCE PD) - - Alternative: run 3 or 5 etcd instances. - - Log can be written to non-durable storage because storage is replicated. + + - Recommended approach: run one etcd instance, with its log written to a directory backed + by durable storage (RAID, GCE PD) + - Alternative: run 3 or 5 etcd instances. + - Log can be written to non-durable storage because storage is replicated. - run a single apiserver which connects to one of the etc nodes. - -See [cluster-troubleshooting](/docs/admin/cluster-troubleshooting) for more discussion on factors affecting cluster -availability. - -To run an etcd instance: - -1. copy `cluster/saltbase/salt/etcd/etcd.manifest` -1. make any modifications needed -1. start the pod by putting it into the kubelet manifest directory - -### Apiserver, Controller Manager, and Scheduler - -The apiserver, controller manager, and scheduler will each run as a pod on the master node. - -For each of these components, the steps to start them running are similar: - -1. Start with a provided template for a pod. -1. Set the `HYPERKUBE_IMAGE` to the values chosen in [Selecting Images](#selecting-images). -1. Determine which flags are needed for your cluster, using the advice below each template. -1. Set the flags to be individual strings in the command array (e.g. $ARGN below) -1. Start the pod by putting the completed template into the kubelet manifest directory. -1. Verify that the pod is started. - -#### Apiserver pod template - + +See [cluster-troubleshooting](/docs/admin/cluster-troubleshooting) for more discussion on factors affecting cluster +availability. + +To run an etcd instance: + +1. copy `cluster/saltbase/salt/etcd/etcd.manifest` +1. make any modifications needed +1. start the pod by putting it into the kubelet manifest directory + +### Apiserver, Controller Manager, and Scheduler + +The apiserver, controller manager, and scheduler will each run as a pod on the master node. + +For each of these components, the steps to start them running are similar: + +1. Start with a provided template for a pod. +1. Set the `HYPERKUBE_IMAGE` to the values chosen in [Selecting Images](#selecting-images). +1. Determine which flags are needed for your cluster, using the advice below each template. +1. Set the flags to be individual strings in the command array (e.g. $ARGN below) +1. Start the pod by putting the completed template into the kubelet manifest directory. +1. Verify that the pod is started. + +#### Apiserver pod template + ```json -{ - "kind": "Pod", - "apiVersion": "v1", - "metadata": { - "name": "kube-apiserver" - }, - "spec": { - "hostNetwork": true, - "containers": [ - { - "name": "kube-apiserver", - "image": "${HYPERKUBE_IMAGE}", - "command": [ - "/hyperkube", - "apiserver", - "$ARG1", - "$ARG2", - ... - "$ARGN" - ], - "ports": [ - { - "name": "https", - "hostPort": 443, - "containerPort": 443 - }, - { - "name": "local", - "hostPort": 8080, - "containerPort": 8080 - } - ], - "volumeMounts": [ - { - "name": "srvkube", - "mountPath": "/srv/kubernetes", - "readOnly": true - }, - { - "name": "etcssl", - "mountPath": "/etc/ssl", - "readOnly": true - } - ], - "livenessProbe": { - "httpGet": { - "path": "/healthz", - "port": 8080 - }, - "initialDelaySeconds": 15, - "timeoutSeconds": 15 - } - } - ], - "volumes": [ - { - "name": "srvkube", - "hostPath": { - "path": "/srv/kubernetes" - } - }, - { - "name": "etcssl", - "hostPath": { - "path": "/etc/ssl" - } - } - ] - } -} +{ + "kind": "Pod", + "apiVersion": "v1", + "metadata": { + "name": "kube-apiserver" + }, + "spec": { + "hostNetwork": true, + "containers": [ + { + "name": "kube-apiserver", + "image": "${HYPERKUBE_IMAGE}", + "command": [ + "/hyperkube", + "apiserver", + "$ARG1", + "$ARG2", + ... + "$ARGN" + ], + "ports": [ + { + "name": "https", + "hostPort": 443, + "containerPort": 443 + }, + { + "name": "local", + "hostPort": 8080, + "containerPort": 8080 + } + ], + "volumeMounts": [ + { + "name": "srvkube", + "mountPath": "/srv/kubernetes", + "readOnly": true + }, + { + "name": "etcssl", + "mountPath": "/etc/ssl", + "readOnly": true + } + ], + "livenessProbe": { + "httpGet": { + "path": "/healthz", + "port": 8080 + }, + "initialDelaySeconds": 15, + "timeoutSeconds": 15 + } + } + ], + "volumes": [ + { + "name": "srvkube", + "hostPath": { + "path": "/srv/kubernetes" + } + }, + { + "name": "etcssl", + "hostPath": { + "path": "/etc/ssl" + } + } + ] + } +} ``` -Here are some apiserver flags you may need to set: - -- `--cloud-provider=` see [cloud providers](#cloud-providers) -- `--cloud-config=` see [cloud providers](#cloud-providers) -- `--address=${MASTER_IP}` *or* `--bind-address=127.0.0.1` and `--address=127.0.0.1` if you want to run a proxy on the master node. -- `--cluster-name=$CLUSTER_NAME` -- `--service-cluster-ip-range=$SERVICE_CLUSTER_IP_RANGE` -- `--etcd-servers=http://127.0.0.1:4001` -- `--tls-cert-file=/srv/kubernetes/server.cert` -- `--tls-private-key-file=/srv/kubernetes/server.key` -- `--admission-control=$RECOMMENDED_LIST` - - See [admission controllers](/docs/admin/admission-controllers) for recommended arguments. -- `--allow-privileged=true`, only if you trust your cluster user to run pods as root. - -If you are following the firewall-only security approach, then use these arguments: - -- `--token-auth-file=/dev/null` -- `--insecure-bind-address=$MASTER_IP` -- `--advertise-address=$MASTER_IP` - -If you are using the HTTPS approach, then set: -- `--client-ca-file=/srv/kubernetes/ca.crt` -- `--token-auth-file=/srv/kubernetes/known_tokens.csv` -- `--basic-auth-file=/srv/kubernetes/basic_auth.csv` - +Here are some apiserver flags you may need to set: + +- `--cloud-provider=` see [cloud providers](#cloud-providers) +- `--cloud-config=` see [cloud providers](#cloud-providers) +- `--address=${MASTER_IP}` *or* `--bind-address=127.0.0.1` and `--address=127.0.0.1` if you want to run a proxy on the master node. +- `--cluster-name=$CLUSTER_NAME` +- `--service-cluster-ip-range=$SERVICE_CLUSTER_IP_RANGE` +- `--etcd-servers=http://127.0.0.1:4001` +- `--tls-cert-file=/srv/kubernetes/server.cert` +- `--tls-private-key-file=/srv/kubernetes/server.key` +- `--admission-control=$RECOMMENDED_LIST` + - See [admission controllers](/docs/admin/admission-controllers) for recommended arguments. +- `--allow-privileged=true`, only if you trust your cluster user to run pods as root. + +If you are following the firewall-only security approach, then use these arguments: + +- `--token-auth-file=/dev/null` +- `--insecure-bind-address=$MASTER_IP` +- `--advertise-address=$MASTER_IP` + +If you are using the HTTPS approach, then set: +- `--client-ca-file=/srv/kubernetes/ca.crt` +- `--token-auth-file=/srv/kubernetes/known_tokens.csv` +- `--basic-auth-file=/srv/kubernetes/basic_auth.csv` + This pod mounts several node file system directories using the `hostPath` volumes. Their purposes are: - -- The `/etc/ssl` mount allows the apiserver to find the SSL root certs so it can - authenticate external services, such as a cloud provider. - - This is not required if you do not use a cloud provider (e.g. bare-metal). -- The `/srv/kubernetes` mount allows the apiserver to read certs and credentials stored on the - node disk. These could instead be stored on a persistent disk, such as a GCE PD, or baked into the image. -- Optionally, you may want to mount `/var/log` as well and redirect output there (not shown in template). - - Do this if you prefer your logs to be accessible from the root filesystem with tools like journalctl. - -*TODO* document proxy-ssh setup. - -##### Cloud Providers - -Apiserver supports several cloud providers. - -- options for `--cloud-provider` flag are `aws`, `gce`, `mesos`, `openshift`, `ovirt`, `rackspace`, `vagrant`, or unset. -- unset used for e.g. bare metal setups. -- support for new IaaS is added by contributing code [here](https://releases.k8s.io/{{page.githubbranch}}/pkg/cloudprovider/providers) - -Some cloud providers require a config file. If so, you need to put config file into apiserver image or mount through hostPath. - -- `--cloud-config=` set if cloud provider requires a config file. -- Used by `aws`, `gce`, `mesos`, `openshift`, `ovirt` and `rackspace`. -- You must put config file into apiserver image or mount through hostPath. -- Cloud config file syntax is [Gcfg](https://code.google.com/p/gcfg/). -- AWS format defined by type [AWSCloudConfig](https://releases.k8s.io/{{page.githubbranch}}/pkg/cloudprovider/providers/aws/aws.go) -- There is a similar type in the corresponding file for other cloud providers. -- GCE example: search for `gce.conf` in [this file](https://releases.k8s.io/{{page.githubbranch}}/cluster/gce/configure-vm.sh) - -#### Scheduler pod template - -Complete this template for the scheduler pod: - + +- The `/etc/ssl` mount allows the apiserver to find the SSL root certs so it can + authenticate external services, such as a cloud provider. + - This is not required if you do not use a cloud provider (e.g. bare-metal). +- The `/srv/kubernetes` mount allows the apiserver to read certs and credentials stored on the + node disk. These could instead be stored on a persistent disk, such as a GCE PD, or baked into the image. +- Optionally, you may want to mount `/var/log` as well and redirect output there (not shown in template). + - Do this if you prefer your logs to be accessible from the root filesystem with tools like journalctl. + +*TODO* document proxy-ssh setup. + +##### Cloud Providers + +Apiserver supports several cloud providers. + +- options for `--cloud-provider` flag are `aws`, `gce`, `mesos`, `openshift`, `ovirt`, `rackspace`, `vagrant`, or unset. +- unset used for e.g. bare metal setups. +- support for new IaaS is added by contributing code [here](https://releases.k8s.io/{{page.githubbranch}}/pkg/cloudprovider/providers) + +Some cloud providers require a config file. If so, you need to put config file into apiserver image or mount through hostPath. + +- `--cloud-config=` set if cloud provider requires a config file. +- Used by `aws`, `gce`, `mesos`, `openshift`, `ovirt` and `rackspace`. +- You must put config file into apiserver image or mount through hostPath. +- Cloud config file syntax is [Gcfg](https://code.google.com/p/gcfg/). +- AWS format defined by type [AWSCloudConfig](https://releases.k8s.io/{{page.githubbranch}}/pkg/cloudprovider/providers/aws/aws.go) +- There is a similar type in the corresponding file for other cloud providers. +- GCE example: search for `gce.conf` in [this file](https://releases.k8s.io/{{page.githubbranch}}/cluster/gce/configure-vm.sh) + +#### Scheduler pod template + +Complete this template for the scheduler pod: + ```json -{ - "kind": "Pod", - "apiVersion": "v1", - "metadata": { - "name": "kube-scheduler" - }, - "spec": { - "hostNetwork": true, - "containers": [ - { - "name": "kube-scheduler", - "image": "$HYBERKUBE_IMAGE", - "command": [ - "/hyperkube", - "scheduler", - "--master=127.0.0.1:8080", - "$SCHEDULER_FLAG1", - ... - "$SCHEDULER_FLAGN" - ], - "livenessProbe": { - "httpGet": { - "host" : "127.0.0.1", - "path": "/healthz", - "port": 10251 - }, - "initialDelaySeconds": 15, - "timeoutSeconds": 15 - } - } - ] - } -} +{ + "kind": "Pod", + "apiVersion": "v1", + "metadata": { + "name": "kube-scheduler" + }, + "spec": { + "hostNetwork": true, + "containers": [ + { + "name": "kube-scheduler", + "image": "$HYBERKUBE_IMAGE", + "command": [ + "/hyperkube", + "scheduler", + "--master=127.0.0.1:8080", + "$SCHEDULER_FLAG1", + ... + "$SCHEDULER_FLAGN" + ], + "livenessProbe": { + "httpGet": { + "host" : "127.0.0.1", + "path": "/healthz", + "port": 10251 + }, + "initialDelaySeconds": 15, + "timeoutSeconds": 15 + } + } + ] + } +} ``` -Typically, no additional flags are required for the scheduler. - -Optionally, you may want to mount `/var/log` as well and redirect output there. - -#### Controller Manager Template - -Template for controller manager pod: - +Typically, no additional flags are required for the scheduler. + +Optionally, you may want to mount `/var/log` as well and redirect output there. + +#### Controller Manager Template + +Template for controller manager pod: + ```json -{ - "kind": "Pod", - "apiVersion": "v1", - "metadata": { - "name": "kube-controller-manager" - }, - "spec": { - "hostNetwork": true, - "containers": [ - { - "name": "kube-controller-manager", - "image": "$HYPERKUBE_IMAGE", - "command": [ - "/hyperkube", - "controller-manager", - "$CNTRLMNGR_FLAG1", - ... - "$CNTRLMNGR_FLAGN" - ], - "volumeMounts": [ - { - "name": "srvkube", - "mountPath": "/srv/kubernetes", - "readOnly": true - }, - { - "name": "etcssl", - "mountPath": "/etc/ssl", - "readOnly": true - } - ], - "livenessProbe": { - "httpGet": { - "host": "127.0.0.1", - "path": "/healthz", - "port": 10252 - }, - "initialDelaySeconds": 15, - "timeoutSeconds": 15 - } - } - ], - "volumes": [ - { - "name": "srvkube", - "hostPath": { - "path": "/srv/kubernetes" - } - }, - { - "name": "etcssl", - "hostPath": { - "path": "/etc/ssl" - } - } - ] - } -} +{ + "kind": "Pod", + "apiVersion": "v1", + "metadata": { + "name": "kube-controller-manager" + }, + "spec": { + "hostNetwork": true, + "containers": [ + { + "name": "kube-controller-manager", + "image": "$HYPERKUBE_IMAGE", + "command": [ + "/hyperkube", + "controller-manager", + "$CNTRLMNGR_FLAG1", + ... + "$CNTRLMNGR_FLAGN" + ], + "volumeMounts": [ + { + "name": "srvkube", + "mountPath": "/srv/kubernetes", + "readOnly": true + }, + { + "name": "etcssl", + "mountPath": "/etc/ssl", + "readOnly": true + } + ], + "livenessProbe": { + "httpGet": { + "host": "127.0.0.1", + "path": "/healthz", + "port": 10252 + }, + "initialDelaySeconds": 15, + "timeoutSeconds": 15 + } + } + ], + "volumes": [ + { + "name": "srvkube", + "hostPath": { + "path": "/srv/kubernetes" + } + }, + { + "name": "etcssl", + "hostPath": { + "path": "/etc/ssl" + } + } + ] + } +} ``` Flags to consider using with controller manager: - - - `--cluster-name=$CLUSTER_NAME` - - `--cluster-cidr=` - - *TODO*: explain this flag. - - `--allocate-node-cidrs=` - - *TODO*: explain when you want controller to do this and when you want to do it another way. - - `--cloud-provider=` and `--cloud-config` as described in apiserver section. - - `--service-account-private-key-file=/srv/kubernetes/server.key`, used by the [service account](/docs/user-guide/service-accounts) feature. - - `--master=127.0.0.1:8080` - -#### Starting and Verifying Apiserver, Scheduler, and Controller Manager - -Place each completed pod template into the kubelet config dir -(whatever `--config=` argument of kubelet is set to, typically -`/etc/kubernetes/manifests`). The order does not matter: scheduler and -controller manager will retry reaching the apiserver until it is up. - -Use `ps` or `docker ps` to verify that each process has started. For example, verify that kubelet has started a container for the apiserver like this: - + + - `--cluster-name=$CLUSTER_NAME` + - `--cluster-cidr=` + - *TODO*: explain this flag. + - `--allocate-node-cidrs=` + - *TODO*: explain when you want controller to do this and when you want to do it another way. + - `--cloud-provider=` and `--cloud-config` as described in apiserver section. + - `--service-account-private-key-file=/srv/kubernetes/server.key`, used by the [service account](/docs/user-guide/service-accounts) feature. + - `--master=127.0.0.1:8080` + +#### Starting and Verifying Apiserver, Scheduler, and Controller Manager + +Place each completed pod template into the kubelet config dir +(whatever `--config=` argument of kubelet is set to, typically +`/etc/kubernetes/manifests`). The order does not matter: scheduler and +controller manager will retry reaching the apiserver until it is up. + +Use `ps` or `docker ps` to verify that each process has started. For example, verify that kubelet has started a container for the apiserver like this: + ```shell -$ sudo docker ps | grep apiserver: -5783290746d5 gcr.io/google_containers/kube-apiserver:e36bf367342b5a80d7467fd7611ad873 "/bin/sh -c '/usr/lo'" 10 seconds ago Up 9 seconds k8s_kube-apiserver.feb145e7_kube-apiserver-kubernetes-master_default_eaebc600cf80dae59902b44225f2fc0a_225a4695 +$ sudo docker ps | grep apiserver: +5783290746d5 gcr.io/google_containers/kube-apiserver:e36bf367342b5a80d7467fd7611ad873 "/bin/sh -c '/usr/lo'" 10 seconds ago Up 9 seconds k8s_kube-apiserver.feb145e7_kube-apiserver-kubernetes-master_default_eaebc600cf80dae59902b44225f2fc0a_225a4695 ``` -Then try to connect to the apiserver: - +Then try to connect to the apiserver: + ```shell -$ echo $(curl -s http://localhost:8080/healthz) -ok -$ curl -s http://localhost:8080/api -{ - "versions": [ - "v1" - ] -} +$ echo $(curl -s http://localhost:8080/healthz) +ok +$ curl -s http://localhost:8080/api +{ + "versions": [ + "v1" + ] +} ``` -If you have selected the `--register-node=true` option for kubelets, they will now begin self-registering with the apiserver. -You should soon be able to see all your nodes by running the `kubectl get nodes` command. -Otherwise, you will need to manually create node objects. - +If you have selected the `--register-node=true` option for kubelets, they will now begin self-registering with the apiserver. +You should soon be able to see all your nodes by running the `kubectl get nodes` command. +Otherwise, you will need to manually create node objects. + ### Starting Cluster Services - -You will want to complete your Kubernetes clusters by adding cluster-wide + +You will want to complete your Kubernetes clusters by adding cluster-wide services. These are sometimes called *addons*, and [an overview of their purpose is in the admin guide](/docs/admin/cluster-components/#addons). - -Notes for setting up each cluster service are given below: - -* Cluster DNS: + +Notes for setting up each cluster service are given below: + +* Cluster DNS: * required for many kubernetes examples * [Setup instructions](http://releases.k8s.io/release-1.2/cluster/addons/dns/) * [Admin Guide](../admin/dns.md) @@ -803,37 +803,37 @@ Notes for setting up each cluster service are given below: * [Google Cloud Logging Backend Setup Instructions](http://releases.k8s.io/release-1.2/cluster/addons/fluentd-gcp/). * Both require running fluentd on each node. * [User Guide](../user-guide/logging.md) -* Container Resource Monitoring - * [Setup instructions](http://releases.k8s.io/release-1.2/cluster/addons/cluster-monitoring/) -* GUI - * [Setup instructions](http://releases.k8s.io/release-1.2/cluster/addons/kube-ui/) - cluster. - -## Troubleshooting - -### Running validate-cluster - -**TODO** explain how to use `cluster/validate-cluster.sh` - -### Inspect pods and services - -Try to run through the "Inspect your cluster" section in one of the other Getting Started Guides, such as [GCE](/docs/getting-started-guides/gce/#inspect-your-cluster). -You should see some services. You should also see "mirror pods" for the apiserver, scheduler and controller-manager, plus any add-ons you started. - -### Try Examples - -At this point you should be able to run through one of the basic examples, such as the [nginx example](/examples/simple-nginx). - -### Running the Conformance Test - -You may want to try to run the [Conformance test](http://releases.k8s.io/{{page.githubbranch}}/hack/conformance-test.sh). Any failures may give a hint as to areas that need more attention. - -### Networking - -The nodes must be able to connect to each other using their private IP. Verify this by -pinging or SSH-ing from one node to another. - -### Getting Help - -If you run into trouble, please see the section on [troubleshooting](/docs/getting-started-guides/gce#troubleshooting), post to the +* Container Resource Monitoring + * [Setup instructions](http://releases.k8s.io/release-1.2/cluster/addons/cluster-monitoring/) +* GUI + * [Setup instructions](http://releases.k8s.io/release-1.2/cluster/addons/kube-ui/) + cluster. + +## Troubleshooting + +### Running validate-cluster + +**TODO** explain how to use `cluster/validate-cluster.sh` + +### Inspect pods and services + +Try to run through the "Inspect your cluster" section in one of the other Getting Started Guides, such as [GCE](/docs/getting-started-guides/gce/#inspect-your-cluster). +You should see some services. You should also see "mirror pods" for the apiserver, scheduler and controller-manager, plus any add-ons you started. + +### Try Examples + +At this point you should be able to run through one of the basic examples, such as the [nginx example](/examples/simple-nginx). + +### Running the Conformance Test + +You may want to try to run the [Conformance test](http://releases.k8s.io/{{page.githubbranch}}/hack/conformance-test.sh). Any failures may give a hint as to areas that need more attention. + +### Networking + +The nodes must be able to connect to each other using their private IP. Verify this by +pinging or SSH-ing from one node to another. + +### Getting Help + +If you run into trouble, please see the section on [troubleshooting](/docs/getting-started-guides/gce#troubleshooting), post to the [google-containers group](https://groups.google.com/forum/#!forum/google-containers), or come ask questions on [Slack](/docs/troubleshooting#slack). \ No newline at end of file diff --git a/docs/getting-started-guides/vagrant.md b/docs/getting-started-guides/vagrant.md index ce9a18e313..fde7c6fe26 100644 --- a/docs/getting-started-guides/vagrant.md +++ b/docs/getting-started-guides/vagrant.md @@ -1,272 +1,272 @@ --- --- - -Running Kubernetes with Vagrant (and VirtualBox) is an easy way to run/test/develop on your local machine (Linux, Mac OS X). - + +Running Kubernetes with Vagrant (and VirtualBox) is an easy way to run/test/develop on your local machine (Linux, Mac OS X). + * TOC -{:toc} - -### Prerequisites - -1. Install latest version >= 1.7.4 of vagrant from http://www.vagrantup.com/downloads.html -2. Install one of: - 1. The latest version of Virtual Box from https://www.virtualbox.org/wiki/Downloads - 2. [VMWare Fusion](https://www.vmware.com/products/fusion/) version 5 or greater as well as the appropriate [Vagrant VMWare Fusion provider](https://www.vagrantup.com/vmware) - 3. [VMWare Workstation](https://www.vmware.com/products/workstation/) version 9 or greater as well as the [Vagrant VMWare Workstation provider](https://www.vagrantup.com/vmware) - 4. [Parallels Desktop](https://www.parallels.com/products/desktop/) version 9 or greater as well as the [Vagrant Parallels provider](https://parallels.github.io/vagrant-parallels/) - 5. libvirt with KVM and enable support of hardware virtualisation. [Vagrant-libvirt](https://github.com/pradels/vagrant-libvirt). For fedora provided official rpm, and possible to use `yum install vagrant-libvirt` - -### Setup - -Setting up a cluster is as simple as running: - +{:toc} + +### Prerequisites + +1. Install latest version >= 1.7.4 of vagrant from http://www.vagrantup.com/downloads.html +2. Install one of: + 1. The latest version of Virtual Box from https://www.virtualbox.org/wiki/Downloads + 2. [VMWare Fusion](https://www.vmware.com/products/fusion/) version 5 or greater as well as the appropriate [Vagrant VMWare Fusion provider](https://www.vagrantup.com/vmware) + 3. [VMWare Workstation](https://www.vmware.com/products/workstation/) version 9 or greater as well as the [Vagrant VMWare Workstation provider](https://www.vagrantup.com/vmware) + 4. [Parallels Desktop](https://www.parallels.com/products/desktop/) version 9 or greater as well as the [Vagrant Parallels provider](https://parallels.github.io/vagrant-parallels/) + 5. libvirt with KVM and enable support of hardware virtualisation. [Vagrant-libvirt](https://github.com/pradels/vagrant-libvirt). For fedora provided official rpm, and possible to use `yum install vagrant-libvirt` + +### Setup + +Setting up a cluster is as simple as running: + ```shell -export KUBERNETES_PROVIDER=vagrant -curl -sS https://get.k8s.io | bash +export KUBERNETES_PROVIDER=vagrant +curl -sS https://get.k8s.io | bash ``` -Alternatively, you can download [Kubernetes release](https://github.com/kubernetes/kubernetes/releases) and extract the archive. To start your local cluster, open a shell and run: - +Alternatively, you can download [Kubernetes release](https://github.com/kubernetes/kubernetes/releases) and extract the archive. To start your local cluster, open a shell and run: + ```shell -cd kubernetes - -export KUBERNETES_PROVIDER=vagrant -./cluster/kube-up.sh +cd kubernetes + +export KUBERNETES_PROVIDER=vagrant +./cluster/kube-up.sh ``` -The `KUBERNETES_PROVIDER` environment variable tells all of the various cluster management scripts which variant to use. If you forget to set this, the assumption is you are running on Google Compute Engine. - -By default, the Vagrant setup will create a single master VM (called kubernetes-master) and one node (called kubernetes-node-1). Each VM will take 1 GB, so make sure you have at least 2GB to 4GB of free memory (plus appropriate free disk space). - -Vagrant will provision each machine in the cluster with all the necessary components to run Kubernetes. The initial setup can take a few minutes to complete on each machine. - -If you installed more than one Vagrant provider, Kubernetes will usually pick the appropriate one. However, you can override which one Kubernetes will use by setting the [`VAGRANT_DEFAULT_PROVIDER`](https://docs.vagrantup.com/v2/providers/default.html) environment variable: - +The `KUBERNETES_PROVIDER` environment variable tells all of the various cluster management scripts which variant to use. If you forget to set this, the assumption is you are running on Google Compute Engine. + +By default, the Vagrant setup will create a single master VM (called kubernetes-master) and one node (called kubernetes-node-1). Each VM will take 1 GB, so make sure you have at least 2GB to 4GB of free memory (plus appropriate free disk space). + +Vagrant will provision each machine in the cluster with all the necessary components to run Kubernetes. The initial setup can take a few minutes to complete on each machine. + +If you installed more than one Vagrant provider, Kubernetes will usually pick the appropriate one. However, you can override which one Kubernetes will use by setting the [`VAGRANT_DEFAULT_PROVIDER`](https://docs.vagrantup.com/v2/providers/default.html) environment variable: + ```shell -export VAGRANT_DEFAULT_PROVIDER=parallels -export KUBERNETES_PROVIDER=vagrant -./cluster/kube-up.sh +export VAGRANT_DEFAULT_PROVIDER=parallels +export KUBERNETES_PROVIDER=vagrant +./cluster/kube-up.sh ``` -By default, each VM in the cluster is running Fedora. - -To access the master or any node: - +By default, each VM in the cluster is running Fedora. + +To access the master or any node: + ```shell -vagrant ssh master -vagrant ssh node-1 +vagrant ssh master +vagrant ssh node-1 ``` -If you are running more than one node, you can access the others by: - +If you are running more than one node, you can access the others by: + ```shell -vagrant ssh node-2 -vagrant ssh node-3 +vagrant ssh node-2 +vagrant ssh node-3 ``` -Each node in the cluster installs the docker daemon and the kubelet. - -The master node instantiates the Kubernetes master components as pods on the machine. - -To view the service status and/or logs on the kubernetes-master: - +Each node in the cluster installs the docker daemon and the kubelet. + +The master node instantiates the Kubernetes master components as pods on the machine. + +To view the service status and/or logs on the kubernetes-master: + ```shell -[vagrant@kubernetes-master ~] $ vagrant ssh master -[vagrant@kubernetes-master ~] $ sudo su - -[root@kubernetes-master ~] $ systemctl status kubelet -[root@kubernetes-master ~] $ journalctl -ru kubelet - -[root@kubernetes-master ~] $ systemctl status docker -[root@kubernetes-master ~] $ journalctl -ru docker - -[root@kubernetes-master ~] $ tail -f /var/log/kube-apiserver.log -[root@kubernetes-master ~] $ tail -f /var/log/kube-controller-manager.log -[root@kubernetes-master ~] $ tail -f /var/log/kube-scheduler.log +[vagrant@kubernetes-master ~] $ vagrant ssh master +[vagrant@kubernetes-master ~] $ sudo su + +[root@kubernetes-master ~] $ systemctl status kubelet +[root@kubernetes-master ~] $ journalctl -ru kubelet + +[root@kubernetes-master ~] $ systemctl status docker +[root@kubernetes-master ~] $ journalctl -ru docker + +[root@kubernetes-master ~] $ tail -f /var/log/kube-apiserver.log +[root@kubernetes-master ~] $ tail -f /var/log/kube-controller-manager.log +[root@kubernetes-master ~] $ tail -f /var/log/kube-scheduler.log ``` -To view the services on any of the nodes: - +To view the services on any of the nodes: + ```shell -[vagrant@kubernetes-master ~] $ vagrant ssh node-1 -[vagrant@kubernetes-master ~] $ sudo su - -[root@kubernetes-master ~] $ systemctl status kubelet -[root@kubernetes-master ~] $ journalctl -ru kubelet - -[root@kubernetes-master ~] $ systemctl status docker -[root@kubernetes-master ~] $ journalctl -ru docker +[vagrant@kubernetes-master ~] $ vagrant ssh node-1 +[vagrant@kubernetes-master ~] $ sudo su + +[root@kubernetes-master ~] $ systemctl status kubelet +[root@kubernetes-master ~] $ journalctl -ru kubelet + +[root@kubernetes-master ~] $ systemctl status docker +[root@kubernetes-master ~] $ journalctl -ru docker ``` -### Interacting with your Kubernetes cluster with Vagrant. - -With your Kubernetes cluster up, you can manage the nodes in your cluster with the regular Vagrant commands. - -To push updates to new Kubernetes code after making source changes: - +### Interacting with your Kubernetes cluster with Vagrant. + +With your Kubernetes cluster up, you can manage the nodes in your cluster with the regular Vagrant commands. + +To push updates to new Kubernetes code after making source changes: + ```shell -./cluster/kube-push.sh +./cluster/kube-push.sh ``` -To stop and then restart the cluster: - +To stop and then restart the cluster: + ```shell -vagrant halt -./cluster/kube-up.sh +vagrant halt +./cluster/kube-up.sh ``` -To destroy the cluster: - +To destroy the cluster: + ```shell -vagrant destroy +vagrant destroy ``` -Once your Vagrant machines are up and provisioned, the first thing to do is to check that you can use the `kubectl.sh` script. - -You may need to build the binaries first, you can do this with `make` - +Once your Vagrant machines are up and provisioned, the first thing to do is to check that you can use the `kubectl.sh` script. + +You may need to build the binaries first, you can do this with `make` + ```shell -$ ./cluster/kubectl.sh get nodes - -NAME LABELS -10.245.1.4 -10.245.1.5 -10.245.1.3 +$ ./cluster/kubectl.sh get nodes + +NAME LABELS +10.245.1.4 +10.245.1.5 +10.245.1.3 ``` -### Authenticating with your master - -When using the vagrant provider in Kubernetes, the `cluster/kubectl.sh` script will cache your credentials in a `~/.kubernetes_vagrant_auth` file so you will not be prompted for them in the future. - +### Authenticating with your master + +When using the vagrant provider in Kubernetes, the `cluster/kubectl.sh` script will cache your credentials in a `~/.kubernetes_vagrant_auth` file so you will not be prompted for them in the future. + ```shell -cat ~/.kubernetes_vagrant_auth +cat ~/.kubernetes_vagrant_auth ``` ```json -{ "User": "vagrant", - "Password": "vagrant", - "CAFile": "/home/k8s_user/.kubernetes.vagrant.ca.crt", - "CertFile": "/home/k8s_user/.kubecfg.vagrant.crt", - "KeyFile": "/home/k8s_user/.kubecfg.vagrant.key" -} +{ "User": "vagrant", + "Password": "vagrant", + "CAFile": "/home/k8s_user/.kubernetes.vagrant.ca.crt", + "CertFile": "/home/k8s_user/.kubecfg.vagrant.crt", + "KeyFile": "/home/k8s_user/.kubecfg.vagrant.key" +} ``` -You should now be set to use the `cluster/kubectl.sh` script. For example try to list the nodes that you have started with: - +You should now be set to use the `cluster/kubectl.sh` script. For example try to list the nodes that you have started with: + ```shell -./cluster/kubectl.sh get nodes +./cluster/kubectl.sh get nodes ``` -### Running containers - -Your cluster is running, you can list the nodes in your cluster: - +### Running containers + +Your cluster is running, you can list the nodes in your cluster: + ```shell -$ ./cluster/kubectl.sh get nodes - -NAME LABELS -10.245.2.4 -10.245.2.3 -10.245.2.2 +$ ./cluster/kubectl.sh get nodes + +NAME LABELS +10.245.2.4 +10.245.2.3 +10.245.2.2 ``` -Now start running some containers! - -You can now use any of the `cluster/kube-*.sh` commands to interact with your VM machines. -Before starting a container there will be no pods, services and replication controllers. - +Now start running some containers! + +You can now use any of the `cluster/kube-*.sh` commands to interact with your VM machines. +Before starting a container there will be no pods, services and replication controllers. + ```shell -$ ./cluster/kubectl.sh get pods -NAME READY STATUS RESTARTS AGE - -$ ./cluster/kubectl.sh get services -NAME CLUSTER_IP EXTERNAL_IP PORT(S) SELECTOR AGE - -$ ./cluster/kubectl.sh get replicationcontrollers -CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS +$ ./cluster/kubectl.sh get pods +NAME READY STATUS RESTARTS AGE + +$ ./cluster/kubectl.sh get services +NAME CLUSTER_IP EXTERNAL_IP PORT(S) SELECTOR AGE + +$ ./cluster/kubectl.sh get replicationcontrollers +CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS ``` -Start a container running nginx with a replication controller and three replicas - +Start a container running nginx with a replication controller and three replicas + ```shell -$ ./cluster/kubectl.sh run my-nginx --image=nginx --replicas=3 --port=80 +$ ./cluster/kubectl.sh run my-nginx --image=nginx --replicas=3 --port=80 ``` -When listing the pods, you will see that three containers have been started and are in Waiting state: - +When listing the pods, you will see that three containers have been started and are in Waiting state: + ```shell -$ ./cluster/kubectl.sh get pods -NAME READY STATUS RESTARTS AGE -my-nginx-5kq0g 0/1 Pending 0 10s -my-nginx-gr3hh 0/1 Pending 0 10s -my-nginx-xql4j 0/1 Pending 0 10s +$ ./cluster/kubectl.sh get pods +NAME READY STATUS RESTARTS AGE +my-nginx-5kq0g 0/1 Pending 0 10s +my-nginx-gr3hh 0/1 Pending 0 10s +my-nginx-xql4j 0/1 Pending 0 10s ``` -You need to wait for the provisioning to complete, you can monitor the nodes by doing: - +You need to wait for the provisioning to complete, you can monitor the nodes by doing: + ```shell -$ vagrant ssh node-1 -c 'sudo docker images' -kubernetes-node-1: - REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE - 96864a7d2df3 26 hours ago 204.4 MB - google/cadvisor latest e0575e677c50 13 days ago 12.64 MB - kubernetes/pause latest 6c4579af347b 8 weeks ago 239.8 kB +$ vagrant ssh node-1 -c 'sudo docker images' +kubernetes-node-1: + REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE + 96864a7d2df3 26 hours ago 204.4 MB + google/cadvisor latest e0575e677c50 13 days ago 12.64 MB + kubernetes/pause latest 6c4579af347b 8 weeks ago 239.8 kB ``` -Once the docker image for nginx has been downloaded, the container will start and you can list it: - +Once the docker image for nginx has been downloaded, the container will start and you can list it: + ```shell -$ vagrant ssh node-1 -c 'sudo docker ps' -kubernetes-node-1: - CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES - dbe79bf6e25b nginx:latest "nginx" 21 seconds ago Up 19 seconds k8s--mynginx.8c5b8a3a--7813c8bd_-_3ffe_-_11e4_-_9036_-_0800279696e1.etcd--7813c8bd_-_3ffe_-_11e4_-_9036_-_0800279696e1--fcfa837f - fa0e29c94501 kubernetes/pause:latest "/pause" 8 minutes ago Up 8 minutes 0.0.0.0:8080->80/tcp k8s--net.a90e7ce4--7813c8bd_-_3ffe_-_11e4_-_9036_-_0800279696e1.etcd--7813c8bd_-_3ffe_-_11e4_-_9036_-_0800279696e1--baf5b21b - aa2ee3ed844a google/cadvisor:latest "/usr/bin/cadvisor" 38 minutes ago Up 38 minutes k8s--cadvisor.9e90d182--cadvisor_-_agent.file--4626b3a2 - 65a3a926f357 kubernetes/pause:latest "/pause" 39 minutes ago Up 39 minutes 0.0.0.0:4194->8080/tcp k8s--net.c5ba7f0e--cadvisor_-_agent.file--342fd561 +$ vagrant ssh node-1 -c 'sudo docker ps' +kubernetes-node-1: + CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES + dbe79bf6e25b nginx:latest "nginx" 21 seconds ago Up 19 seconds k8s--mynginx.8c5b8a3a--7813c8bd_-_3ffe_-_11e4_-_9036_-_0800279696e1.etcd--7813c8bd_-_3ffe_-_11e4_-_9036_-_0800279696e1--fcfa837f + fa0e29c94501 kubernetes/pause:latest "/pause" 8 minutes ago Up 8 minutes 0.0.0.0:8080->80/tcp k8s--net.a90e7ce4--7813c8bd_-_3ffe_-_11e4_-_9036_-_0800279696e1.etcd--7813c8bd_-_3ffe_-_11e4_-_9036_-_0800279696e1--baf5b21b + aa2ee3ed844a google/cadvisor:latest "/usr/bin/cadvisor" 38 minutes ago Up 38 minutes k8s--cadvisor.9e90d182--cadvisor_-_agent.file--4626b3a2 + 65a3a926f357 kubernetes/pause:latest "/pause" 39 minutes ago Up 39 minutes 0.0.0.0:4194->8080/tcp k8s--net.c5ba7f0e--cadvisor_-_agent.file--342fd561 ``` -Going back to listing the pods, services and replicationcontrollers, you now have: - +Going back to listing the pods, services and replicationcontrollers, you now have: + ```shell -$ ./cluster/kubectl.sh get pods -NAME READY STATUS RESTARTS AGE -my-nginx-5kq0g 1/1 Running 0 1m -my-nginx-gr3hh 1/1 Running 0 1m -my-nginx-xql4j 1/1 Running 0 1m - -$ ./cluster/kubectl.sh get services -NAME CLUSTER_IP EXTERNAL_IP PORT(S) SELECTOR AGE - +$ ./cluster/kubectl.sh get pods +NAME READY STATUS RESTARTS AGE +my-nginx-5kq0g 1/1 Running 0 1m +my-nginx-gr3hh 1/1 Running 0 1m +my-nginx-xql4j 1/1 Running 0 1m + +$ ./cluster/kubectl.sh get services +NAME CLUSTER_IP EXTERNAL_IP PORT(S) SELECTOR AGE + $ ./cluster/kubectl.sh get replicationcontrollers CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS AGE my-nginx my-nginx nginx run=my-nginx 3 1m ``` -We did not start any services, hence there are none listed. But we see three replicas displayed properly. -Check the [guestbook](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/guestbook/) application to learn how to create a service. -You can already play with scaling the replicas with: - +We did not start any services, hence there are none listed. But we see three replicas displayed properly. +Check the [guestbook](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/guestbook/) application to learn how to create a service. +You can already play with scaling the replicas with: + ```shell -$ ./cluster/kubectl.sh scale rc my-nginx --replicas=2 -$ ./cluster/kubectl.sh get pods -NAME READY STATUS RESTARTS AGE -my-nginx-5kq0g 1/1 Running 0 2m -my-nginx-gr3hh 1/1 Running 0 2m +$ ./cluster/kubectl.sh scale rc my-nginx --replicas=2 +$ ./cluster/kubectl.sh get pods +NAME READY STATUS RESTARTS AGE +my-nginx-5kq0g 1/1 Running 0 2m +my-nginx-gr3hh 1/1 Running 0 2m ``` -Congratulations! - -### Troubleshooting - -#### I keep downloading the same (large) box all the time! - -By default the Vagrantfile will download the box from S3. You can change this (and cache the box locally) by providing a name and an alternate URL when calling `kube-up.sh` - +Congratulations! + +### Troubleshooting + +#### I keep downloading the same (large) box all the time! + +By default the Vagrantfile will download the box from S3. You can change this (and cache the box locally) by providing a name and an alternate URL when calling `kube-up.sh` + ```shell -export KUBERNETES_BOX_NAME=choose_your_own_name_for_your_kuber_box -export KUBERNETES_BOX_URL=path_of_your_kuber_box -export KUBERNETES_PROVIDER=vagrant -./cluster/kube-up.sh +export KUBERNETES_BOX_NAME=choose_your_own_name_for_your_kuber_box +export KUBERNETES_BOX_URL=path_of_your_kuber_box +export KUBERNETES_PROVIDER=vagrant +./cluster/kube-up.sh ``` #### I am getting timeouts when trying to curl the master from my host! @@ -306,71 +306,89 @@ If you do not see a response on your host machine, you will most likely need to If you do see a network, but are still unable to ping the machine, check if your VPN is blocking the request. -#### I just created the cluster, but I am getting authorization errors! - -You probably have an incorrect ~/.kubernetes_vagrant_auth file for the cluster you are attempting to contact. - +#### I just created the cluster, but I am getting authorization errors! + +You probably have an incorrect ~/.kubernetes_vagrant_auth file for the cluster you are attempting to contact. + ```shell -rm ~/.kubernetes_vagrant_auth +rm ~/.kubernetes_vagrant_auth ``` -After using kubectl.sh make sure that the correct credentials are set: - +After using kubectl.sh make sure that the correct credentials are set: + ```shell -cat ~/.kubernetes_vagrant_auth +cat ~/.kubernetes_vagrant_auth ``` ```json -{ - "User": "vagrant", - "Password": "vagrant" -} +{ + "User": "vagrant", + "Password": "vagrant" +} ``` -#### I just created the cluster, but I do not see my container running! - -If this is your first time creating the cluster, the kubelet on each node schedules a number of docker pull requests to fetch prerequisite images. This can take some time and as a result may delay your initial pod getting provisioned. - -#### I want to make changes to Kubernetes code! - -To set up a vagrant cluster for hacking, follow the [vagrant developer guide](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/devel/developer-guides/vagrant.md). - -#### I have brought Vagrant up but the nodes cannot validate! - -Log on to one of the nodes (`vagrant ssh node-1`) and inspect the salt minion log (`sudo cat /var/log/salt/minion`). - -#### I want to change the number of nodes! - -You can control the number of nodes that are instantiated via the environment variable `NUM_NODES` on your host machine. If you plan to work with replicas, we strongly encourage you to work with enough nodes to satisfy your largest intended replica size. If you do not plan to work with replicas, you can save some system resources by running with a single node. You do this, by setting `NUM_NODES` to 1 like so: - +#### I just created the cluster, but I do not see my container running! + +If this is your first time creating the cluster, the kubelet on each node schedules a number of docker pull requests to fetch prerequisite images. This can take some time and as a result may delay your initial pod getting provisioned. + +#### I want to make changes to Kubernetes code! + +To set up a vagrant cluster for hacking, follow the [vagrant developer guide](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/devel/developer-guides/vagrant.md). + +#### I have brought Vagrant up but the nodes cannot validate! + +Log on to one of the nodes (`vagrant ssh node-1`) and inspect the salt minion log (`sudo cat /var/log/salt/minion`). + +#### I want to change the number of nodes! + +You can control the number of nodes that are instantiated via the environment variable `NUM_NODES` on your host machine. If you plan to work with replicas, we strongly encourage you to work with enough nodes to satisfy your largest intended replica size. If you do not plan to work with replicas, you can save some system resources by running with a single node. You do this, by setting `NUM_NODES` to 1 like so: + ```shell -export NUM_NODES=1 +export NUM_NODES=1 ``` -#### I want my VMs to have more memory! - -You can control the memory allotted to virtual machines with the `KUBERNETES_MEMORY` environment variable. -Just set it to the number of megabytes you would like the machines to have. For example: - +#### I want my VMs to have more memory! + +You can control the memory allotted to virtual machines with the `KUBERNETES_MEMORY` environment variable. +Just set it to the number of megabytes you would like the machines to have. For example: + ```shell -export KUBERNETES_MEMORY=2048 +export KUBERNETES_MEMORY=2048 ``` -If you need more granular control, you can set the amount of memory for the master and nodes independently. For example: - +If you need more granular control, you can set the amount of memory for the master and nodes independently. For example: + ```shell -export KUBERNETES_MASTER_MEMORY=1536 -export KUBERNETES_NODE_MEMORY=2048 +export KUBERNETES_MASTER_MEMORY=1536 +export KUBERNETES_NODE_MEMORY=2048 ``` -#### I ran vagrant suspend and nothing works! - -`vagrant suspend` seems to mess up the network. This is not supported at this time. - -#### I want vagrant to sync folders via nfs! - -You can ensure that vagrant uses nfs to sync folders with virtual machines by setting the KUBERNETES_VAGRANT_USE_NFS environment variable to 'true'. nfs is faster than virtualbox or vmware's 'shared folders' and does not require guest additions. See the [vagrant docs](http://docs.vagrantup.com/v2/synced-folders/nfs.html) for details on configuring nfs on the host. This setting will have no effect on the libvirt provider, which uses nfs by default. For example: - -```shell -export KUBERNETES_VAGRANT_USE_NFS=true +#### I want to set proxy settings for my Kubernetes cluster boot strapping! + +If you are behind a proxy, you need to install vagrant proxy plugin and set the proxy settings by + +```sh +vagrant plugin install vagrant-proxyconf +export KUBERNETES_HTTP_PROXY=http://username:password@proxyaddr:proxyport +export KUBERNETES_HTTPS_PROXY=https://username:password@proxyaddr:proxyport +``` + +Optionally you can specify addresses to not proxy, for example + +```sh +export KUBERNETES_NO_PROXY=127.0.0.1 +``` + +If you are using sudo to make kubernetes build for example make quick-release, you need run `sudo -E make quick-release` to pass the environment variables. + +#### I ran vagrant suspend and nothing works! + +`vagrant suspend` seems to mess up the network. This is not supported at this time. + +#### I want vagrant to sync folders via nfs! + +You can ensure that vagrant uses nfs to sync folders with virtual machines by setting the KUBERNETES_VAGRANT_USE_NFS environment variable to 'true'. nfs is faster than virtualbox or vmware's 'shared folders' and does not require guest additions. See the [vagrant docs](http://docs.vagrantup.com/v2/synced-folders/nfs.html) for details on configuring nfs on the host. This setting will have no effect on the libvirt provider, which uses nfs by default. For example: + +```shell +export KUBERNETES_VAGRANT_USE_NFS=true ``` diff --git a/docs/getting-started-guides/vsphere.md b/docs/getting-started-guides/vsphere.md index 905719b16c..02e06adf0d 100644 --- a/docs/getting-started-guides/vsphere.md +++ b/docs/getting-started-guides/vsphere.md @@ -1,96 +1,96 @@ --- --- - -The example below creates a Kubernetes cluster with 4 worker node Virtual -Machines and a master Virtual Machine (i.e. 5 VMs in your cluster). This -cluster is set up and controlled from your workstation (or wherever you find -convenient). - + +The example below creates a Kubernetes cluster with 4 worker node Virtual +Machines and a master Virtual Machine (i.e. 5 VMs in your cluster). This +cluster is set up and controlled from your workstation (or wherever you find +convenient). + * TOC -{:toc} - -### Prerequisites - -1. You need administrator credentials to an ESXi machine or vCenter instance. -2. You must have Go (see [here](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/devel/development.md#go-versions) for supported versions) installed: [www.golang.org](http://www.golang.org). -3. You must have your `GOPATH` set up and include `$GOPATH/bin` in your `PATH`. - +{:toc} + +### Prerequisites + +1. You need administrator credentials to an ESXi machine or vCenter instance. +2. You must have Go (see [here](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/devel/development.md#go-versions) for supported versions) installed: [www.golang.org](http://www.golang.org). +3. You must have your `GOPATH` set up and include `$GOPATH/bin` in your `PATH`. + ```shell -export GOPATH=$HOME/src/go -mkdir -p $GOPATH -export PATH=$PATH:$GOPATH/bin +export GOPATH=$HOME/src/go +mkdir -p $GOPATH +export PATH=$PATH:$GOPATH/bin ``` -4. Install the govc tool to interact with ESXi/vCenter: - +4. Install the govc tool to interact with ESXi/vCenter: + ```shell -go get github.com/vmware/govmomi/govc +go get github.com/vmware/govmomi/govc ``` -5. Get or build a [binary release](/docs/getting-started-guides/binary_release) - -### Setup - -Download a prebuilt Debian 8.2 VMDK that we'll use as a base image: - +5. Get or build a [binary release](/docs/getting-started-guides/binary_release) + +### Setup + +Download a prebuilt Debian 8.2 VMDK that we'll use as a base image: + ```shell -curl --remote-name-all https://storage.googleapis.com/govmomi/vmdk/2016-01-08/kube.vmdk.gz{,.md5} -md5sum -c kube.vmdk.gz.md5 -gzip -d kube.vmdk.gz +curl --remote-name-all https://storage.googleapis.com/govmomi/vmdk/2016-01-08/kube.vmdk.gz{,.md5} +md5sum -c kube.vmdk.gz.md5 +gzip -d kube.vmdk.gz ``` -Import this VMDK into your vSphere datastore: - +Import this VMDK into your vSphere datastore: + ```shell -export GOVC_URL='hostname' # hostname of the vc +export GOVC_URL='hostname' # hostname of the vc export GOVC_USERNAME='username' # username for logging into the vsphere. export GOVC_PASSWORD='password' # password for the above username export GOVC_NETWORK='Network Name' # Name of the network the vms should join. Many times it could be "VM Network" -export GOVC_INSECURE=1 # If the host above uses a self-signed cert -export GOVC_DATASTORE='target datastore' -export GOVC_RESOURCE_POOL='resource pool or cluster with access to datastore' - -govc import.vmdk kube.vmdk ./kube/ +export GOVC_INSECURE=1 # If the host above uses a self-signed cert +export GOVC_DATASTORE='target datastore' +export GOVC_RESOURCE_POOL='resource pool or cluster with access to datastore' + +govc import.vmdk kube.vmdk ./kube/ ``` -Verify that the VMDK was correctly uploaded and expanded to ~3GiB: - +Verify that the VMDK was correctly uploaded and expanded to ~3GiB: + ```shell -govc datastore.ls ./kube/ +govc datastore.ls ./kube/ ``` -Take a look at the file `cluster/vsphere/config-common.sh` fill in the required -parameters. The guest login for the image that you imported is `kube:kube`. - -### Starting a cluster - -Now, let's continue with deploying Kubernetes. -This process takes about ~20-30 minutes depending on your network. - +Take a look at the file `cluster/vsphere/config-common.sh` fill in the required +parameters. The guest login for the image that you imported is `kube:kube`. + +### Starting a cluster + +Now, let's continue with deploying Kubernetes. +This process takes about ~20-30 minutes depending on your network. + #### From extracted binary release ```shell cd kubernetes KUBERNETES_PROVIDER=vsphere cluster/kube-up.sh ``` - + #### Build from source - + ```shell -cd kubernetes +cd kubernetes make release KUBERNETES_PROVIDER=vsphere cluster/kube-up.sh ``` -Refer to the top level README and the getting started guide for Google Compute -Engine. Once you have successfully reached this point, your vSphere Kubernetes -deployment works just as any other one! - -**Enjoy!** - -### Extra: debugging deployment failure - -The output of `kube-up.sh` displays the IP addresses of the VMs it deploys. You -can log into any VM as the `kube` user to poke around and figure out what is -going on (find yourself authorized with your SSH key, or use the password +Refer to the top level README and the getting started guide for Google Compute +Engine. Once you have successfully reached this point, your vSphere Kubernetes +deployment works just as any other one! + +**Enjoy!** + +### Extra: debugging deployment failure + +The output of `kube-up.sh` displays the IP addresses of the VMs it deploys. You +can log into any VM as the `kube` user to poke around and figure out what is +going on (find yourself authorized with your SSH key, or use the password `kube` otherwise). diff --git a/docs/hellonode.md b/docs/hellonode.md index 2815ab5dc7..d512af329e 100755 --- a/docs/hellonode.md +++ b/docs/hellonode.md @@ -116,7 +116,7 @@ We recommend using the nano editor but vi and emacs are also available in Cloud From Cloud Shell simply exit the editor and save the server.js file. Since CloudShell has the node executable installed we can now run this simple command : ```shell -nano server.js +node server.js ``` and use the built-in Web [preview feature](https://cloud.google.com/cloud-shell/docs/features#web_preview) of CloudShell to open a new browser tab and proxy a request to the instance you just started on port 8080.