Merge branch 'master' into patch-1
This commit is contained in:
Joe Beda
@@ -33,7 +33,7 @@ cd kubernetes
|
||||
make release
|
||||
```
|
||||
|
||||
For more details on the release process see the [`build/`](http://releases.k8s.io/{{page.githubbranch}}/build/) directory
|
||||
For more details on the release process see the [`build-tools/`](http://releases.k8s.io/{{page.githubbranch}}/build-tools/) directory
|
||||
|
||||
### Download Kubernetes and automatically set up a default cluster
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# This config should be kept as similar as possible to the one at
|
||||
# cluster/addons/gci/fluentd-gcp.yaml
|
||||
# cluster/saltbase/salt/fluentd-gcp-gci/fluentd-gcp-gci.yaml
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
@@ -11,7 +11,7 @@ spec:
|
||||
dnsPolicy: Default
|
||||
containers:
|
||||
- name: fluentd-cloud-logging
|
||||
image: gcr.io/google_containers/fluentd-gcp:1.25
|
||||
image: gcr.io/google_containers/fluentd-gcp:1.28
|
||||
resources:
|
||||
limits:
|
||||
memory: 200Mi
|
||||
@@ -23,6 +23,14 @@ spec:
|
||||
env:
|
||||
- name: FLUENTD_ARGS
|
||||
value: -q
|
||||
# Jemalloc is a widely used way to decrease memory consumption
|
||||
# in Ruby world. It's a better implementation of malloc(3).
|
||||
- name: "LD_PRELOAD"
|
||||
value: "/opt/td-agent/embedded/lib/libjemalloc.so"
|
||||
# This is quite hacky, but forces Ruby GC to be ivoked more often
|
||||
# resulting in lower memory consumption, which is important for us.
|
||||
- name: "RUBY_GC_HEAP_OLDOBJECT_LIMIT_FACTOR"
|
||||
value: "0.9"
|
||||
volumeMounts:
|
||||
- name: varlog
|
||||
mountPath: /var/log
|
||||
|
||||
@@ -27,7 +27,7 @@ a Kubernetes cluster from scratch.
|
||||
|
||||
### Local-machine Solutions
|
||||
|
||||
[Minikube](/docs/getting-started-guides/minikube/) is the recommended method for you to create a single node kubernetes cluster locally for purposes of development and testing. Setup is completely automated and doesn't require a cloud provider account.
|
||||
[Minikube](/docs/getting-started-guides/minikube/) is the recommended method for you to create a single node kubernetes cluster locally for purposes of development and testing. Setup is completely automated and doesn't require a cloud provider account.
|
||||
|
||||
Use the [Minikube getting started guide](/docs/getting-started-guides/minikube/) to try it out.
|
||||
|
||||
@@ -45,6 +45,8 @@ clusters.
|
||||
|
||||
[Platform9](https://platform9.com/products/kubernetes/) offers managed Kubernetes on-premises or any public cloud, and provides 24/7 health monitoring and alerting.
|
||||
|
||||
[OpenShift Dedicated](https://www.openshift.com/dedicated/) offers managed Kubernetes clusters powered by OpenShift and [OpenShift Online](https://www.openshift.com/features/) provides free hosted access for Kubernetes applications.
|
||||
|
||||
### Turn-key Cloud Solutions
|
||||
|
||||
These solutions allow you to create Kubernetes clusters on a range of Cloud IaaS providers with only a
|
||||
|
||||
@@ -6,144 +6,314 @@ assignees:
|
||||
|
||||
---
|
||||
|
||||
Minikube is a tool that makes it easy to run Kubernetes locally. Minikube runs a single-node Kubernetes cluster inside a VM on your laptop for users looking to try out Kubernetes or develop with it day-to-day.
|
||||
|
||||
* TOC
|
||||
{:toc}
|
||||
|
||||
Minikube starts a single node kubernetes cluster locally for purposes of development and testing.
|
||||
Minikube packages and configures a Linux VM, Docker and all Kubernetes components, optimized for local development.
|
||||
Minikube supports Kubernetes features such as:
|
||||
### Minikube Features
|
||||
|
||||
* DNS
|
||||
* NodePorts
|
||||
* ConfigMaps and Secrets
|
||||
* Dashboards
|
||||
* Minikube supports Kubernetes features such as:
|
||||
* DNS
|
||||
* NodePorts
|
||||
* ConfigMaps and Secrets
|
||||
* Dashboards
|
||||
* Container Runtime: Docker, and [rkt](https://github.com/coreos/rkt)
|
||||
* Enabling CNI (Container Network Interface)
|
||||
* Ingress
|
||||
|
||||
Minikube does not yet support Cloud Provider specific features such as:
|
||||
|
||||
* LoadBalancers
|
||||
* PersistentVolumes
|
||||
* Ingress
|
||||
## Installation
|
||||
|
||||
### Requirements
|
||||
|
||||
Minikube requires that VT-x/AMD-v virtualization is enabled in BIOS on all platforms.
|
||||
* OS X
|
||||
* [xhyve driver](./DRIVERS.md#xhyve-driver), [VirtualBox](https://www.virtualbox.org/wiki/Downloads) or [VMware Fusion](https://www.vmware.com/products/fusion) installation
|
||||
* Linux
|
||||
* [VirtualBox](https://www.virtualbox.org/wiki/Downloads) or [KVM](http://www.linux-kvm.org/) installation,
|
||||
* VT-x/AMD-v virtualization must be enabled in BIOS
|
||||
* `kubectl` must be on your path. To install kubectl:
|
||||
|
||||
To check that this is enabled on Linux, run:
|
||||
**Kubectl for Linux/amd64**
|
||||
|
||||
```
|
||||
curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/
|
||||
```
|
||||
|
||||
**Kubectl for OS X/amd64**
|
||||
|
||||
```
|
||||
curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/darwin/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/
|
||||
```
|
||||
|
||||
### Instructions
|
||||
|
||||
See the installation instructions for the [latest release](https://github.com/kubernetes/minikube/releases).
|
||||
|
||||
## Quickstart
|
||||
|
||||
Here's a brief demo of minikube usage.
|
||||
If you want to change the VM driver add the appropriate `--vm-driver=xxx` flag to `minikube start`. Minikube Supports
|
||||
the following drivers:
|
||||
|
||||
* virtualbox
|
||||
* vmwarefusion
|
||||
* kvm ([driver installation](./DRIVERS.md#kvm-driver))
|
||||
* xhyve ([driver installation](./DRIVERS.md#xhyve-driver))
|
||||
|
||||
Note that the IP below is dynamic and can change. It can be retrieved with `minikube ip`.
|
||||
|
||||
```shell
|
||||
cat /proc/cpuinfo | grep 'vmx\|svm'
|
||||
```
|
||||
|
||||
This command should output something if the setting is enabled.
|
||||
|
||||
To check that this is enabled on OSX (most newer Macs have this enabled by default), run:
|
||||
|
||||
```shell
|
||||
sysctl -a | grep machdep.cpu.features | grep VMX
|
||||
|
||||
```
|
||||
|
||||
This command should output something if the setting is enabled.
|
||||
|
||||
#### Linux
|
||||
|
||||
Minikube requires the latest [Virtualbox](https://www.virtualbox.org/wiki/Downloads) to be installed on your system.
|
||||
|
||||
#### OSX
|
||||
|
||||
Minikube requires one of the following:
|
||||
|
||||
* The latest [Virtualbox](https://www.virtualbox.org/wiki/Downloads).
|
||||
* The latest version of [VMWare Fusion](https://www.vmware.com/products/fusion).
|
||||
|
||||
### Install `minikube`
|
||||
|
||||
See the [latest Minikube release](https://github.com/kubernetes/minikube/releases) for installation instructions.
|
||||
|
||||
### Install `kubectl`
|
||||
|
||||
You will need to download and install the kubectl client binary for `${K8S_VERSION}` (in this example: `{{page.version}}.0`)
|
||||
to run commands against the cluster.
|
||||
|
||||
```shell
|
||||
# linux/amd64
|
||||
curl -Lo kubectl https://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/
|
||||
# linux/386
|
||||
curl -Lo kubectl https://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/386/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/
|
||||
# linux/arm
|
||||
curl -Lo kubectl https://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/arm/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/
|
||||
# linux/arm64
|
||||
curl -Lo kubectl https://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/arm64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/
|
||||
#linux/ppc64le
|
||||
curl -Lo kubectl https://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/ppc64le/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/
|
||||
# OS X/amd64
|
||||
curl -Lo kubectl https://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/darwin/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/
|
||||
# OS X/386
|
||||
curl -Lo kubectl https://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/darwin/386/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/
|
||||
```
|
||||
|
||||
For Windows, download [kubectl.exe](http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/windows/amd64/kubectl.exe) and save it to a location on your PATH.
|
||||
|
||||
The generic download path is:
|
||||
```
|
||||
https://storage.googleapis.com/kubernetes-release/release/${K8S_VERSION}/bin/${GOOS}/${GOARCH}/${K8S_BINARY}
|
||||
```
|
||||
|
||||
### Starting the cluster
|
||||
|
||||
To start a cluster, run the command:
|
||||
|
||||
```shell
|
||||
minikube start
|
||||
$ minikube start
|
||||
Starting local Kubernetes cluster...
|
||||
Kubectl is now configured to use the cluster.
|
||||
Running pre-create checks...
|
||||
Creating machine...
|
||||
Starting local Kubernetes cluster...
|
||||
|
||||
$ kubectl run hello-minikube --image=gcr.io/google_containers/echoserver:1.4 --port=8080
|
||||
deployment "hello-minikube" created
|
||||
$ kubectl expose deployment hello-minikube --type=NodePort
|
||||
service "hello-minikube" exposed
|
||||
|
||||
# We have now launched an echoserver pod but we have to wait until the pod is up before curling/accessing it
|
||||
# via the exposed service.
|
||||
# To check whether the pod is up and running we can use the following:
|
||||
$ kubectl get pod
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
hello-minikube-3383150820-vctvh 1/1 ContainerCreating 0 3s
|
||||
# We can see that the pod is still being created from the ContainerCreating status
|
||||
$ kubectl get pod
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
hello-minikube-3383150820-vctvh 1/1 Running 0 13s
|
||||
# We can see that the pod is now Running and we will now be able to curl it:
|
||||
$ curl $(minikube service hello-minikube --url)
|
||||
CLIENT VALUES:
|
||||
client_address=192.168.99.1
|
||||
command=GET
|
||||
real path=/
|
||||
...
|
||||
$ minikube stop
|
||||
Stopping local Kubernetes cluster...
|
||||
Stopping "minikube"...
|
||||
```
|
||||
|
||||
This will build and start a lightweight local cluster, consisting of a master, etcd, Docker and a single node.
|
||||
### Using rkt container engine
|
||||
|
||||
Minikube will also create a "minikube" context, and set it to default in kubectl.
|
||||
To switch back to this context later, run this command: `kubectl config use-context minikube`.
|
||||
|
||||
Type `minikube stop` to shut the cluster down.
|
||||
|
||||
Minikube also includes the [Kubernetes dashboard](http://kubernetes.io/docs/user-guide/ui/). Run this command to see the included kube-system pods:
|
||||
To use [rkt](https://github.com/coreos/rkt) as the container runtime run:
|
||||
|
||||
```shell
|
||||
$ kubectl get pods --all-namespaces
|
||||
NAMESPACE NAME READY STATUS RESTARTS AGE
|
||||
kube-system kube-addon-manager-127.0.0.1 1/1 Running 0 35s
|
||||
kube-system kubernetes-dashboard-9brhv 1/1 Running 0 20s
|
||||
$ minikube start \
|
||||
--network-plugin=cni \
|
||||
--container-runtime=rkt \
|
||||
--iso-url=https://github.com/coreos/minikube-iso/releases/download/v0.0.5/minikube-v0.0.5.iso
|
||||
```
|
||||
|
||||
Run this command to open the Kubernetes dashboard:
|
||||
This will use an alternative minikube ISO image containing both rkt, and Docker, and enable CNI networking.
|
||||
|
||||
### Driver plugins
|
||||
|
||||
See [DRIVERS](./DRIVERS.md) for details on supported drivers and how to install
|
||||
plugins, if required.
|
||||
|
||||
### Reusing the Docker daemon
|
||||
|
||||
When using a single VM of kubernetes its really handy to reuse the Docker daemon inside the VM; as this means you don't have to build on your host machine and push the image into a docker registry - you can just build inside the same docker daemon as minikube which speeds up local experiments.
|
||||
|
||||
To be able to work with the docker daemon on your mac/linux host use the [docker-env command](./docs/minikube_docker-env.md) in your shell:
|
||||
|
||||
```
|
||||
eval $(minikube docker-env)
|
||||
```
|
||||
you should now be able to use docker on the command line on your host mac/linux machine talking to the docker daemon inside the minikube VM:
|
||||
```
|
||||
docker ps
|
||||
```
|
||||
|
||||
On Centos 7, docker may report the following error:
|
||||
|
||||
```
|
||||
Could not read CA certificate "/etc/docker/ca.pem": open /etc/docker/ca.pem: no such file or directory
|
||||
```
|
||||
|
||||
The fix is to update /etc/sysconfig/docker to ensure that minikube's environment changes are respected:
|
||||
|
||||
```
|
||||
< DOCKER_CERT_PATH=/etc/docker
|
||||
---
|
||||
> if [ -z "${DOCKER_CERT_PATH}" ]; then
|
||||
> DOCKER_CERT_PATH=/etc/docker
|
||||
> fi
|
||||
```
|
||||
|
||||
Remember to turn off the imagePullPolicy:Always, as otherwise kubernetes won't use images you built locally.
|
||||
|
||||
## Managing your Cluster
|
||||
|
||||
### Starting a Cluster
|
||||
|
||||
The [minikube start](./docs/minikube_start.md) command can be used to start your cluster.
|
||||
This command creates and configures a virtual machine that runs a single-node Kubernetes cluster.
|
||||
This command also configures your [kubectl](http://kubernetes.io/docs/user-guide/kubectl-overview/) installation to communicate with this cluster.
|
||||
|
||||
### Configuring Kubernetes
|
||||
|
||||
Minikube has a "configurator" feature that allows users to configure the Kubernetes components with arbitrary values.
|
||||
To use this feature, you can use the `--extra-config` flag on the `minikube start` command.
|
||||
|
||||
This flag is repeated, so you can pass it several times with several different values to set multiple options.
|
||||
|
||||
This flag takes a string of the form `component.key=value`, where `component` is one of the strings from the above list, `key` is a value on the
|
||||
configuration struct and `value` is the value to set.
|
||||
|
||||
Valid `key`s can be found by examining the documentation for the Kubernetes `componentconfigs` for each component.
|
||||
Here is the documentation for each supported configuration:
|
||||
|
||||
* [kubelet](https://godoc.org/k8s.io/kubernetes/pkg/apis/componentconfig#KubeletConfiguration)
|
||||
* [apiserver](https://godoc.org/k8s.io/kubernetes/cmd/kube-apiserver/app/options#APIServer)
|
||||
* [proxy](https://godoc.org/k8s.io/kubernetes/pkg/apis/componentconfig#KubeProxyConfiguration)
|
||||
* [controller-manager](https://godoc.org/k8s.io/kubernetes/pkg/apis/componentconfig#KubeControllerManagerConfiguration)
|
||||
* [etcd](https://godoc.org/github.com/coreos/etcd/etcdserver#ServerConfig)
|
||||
* [scheduler](https://godoc.org/k8s.io/kubernetes/pkg/apis/componentconfig#KubeSchedulerConfiguration)
|
||||
|
||||
#### Examples
|
||||
|
||||
To change the `MaxPods` setting to 5 on the Kubelet, pass this flag: `--extra-config=kubelet.MaxPods=5`.
|
||||
|
||||
This feature also supports nested structs. To change the `LeaderElection.LeaderElect` setting to `true` on the scheduler, pass this flag: `--extra-config=scheduler.LeaderElection.LeaderElect=true`.
|
||||
|
||||
To set the `AuthorizationMode` on the `apiserver` to `RBAC`, you can use: `--extra-config=apiserver.AuthorizationMode=RBAC`.
|
||||
|
||||
### Stopping a Cluster
|
||||
The [minikube stop](./docs/minikube_stop.md) command can be used to stop your cluster.
|
||||
This command shuts down the minikube virtual machine, but preserves all cluster state and data.
|
||||
Starting the cluster again will restore it to it's previous state.
|
||||
|
||||
### Deleting a Cluster
|
||||
The [minikube delete](./docs/minikube_delete.md) command can be used to delete your cluster.
|
||||
This command shuts down and deletes the minikube virtual machine. No data or state is preserved.
|
||||
|
||||
## Interacting With your Cluster
|
||||
|
||||
### Kubectl
|
||||
|
||||
The `minikube start` command creates a "[kubectl context](http://kubernetes.io/docs/user-guide/kubectl/kubectl_config_set-context/)" called "minikube".
|
||||
This context contains the configuration to communicate with your minikube cluster.
|
||||
|
||||
Minikube sets this context to default automatically, but if you need to switch back to it in the future, run:
|
||||
|
||||
`kubectl config use-context minikube`,
|
||||
|
||||
or pass the context on each command like this: `kubectl get pods --context=minikube`.
|
||||
|
||||
### Dashboard
|
||||
|
||||
To access the [Kubernetes Dashboard](http://kubernetes.io/docs/user-guide/ui/), run this command in a shell after starting minikube to get the address:
|
||||
```shell
|
||||
minikube dashboard
|
||||
```
|
||||
|
||||
### Test it out
|
||||
|
||||
List the nodes in your cluster by running:
|
||||
### Services
|
||||
|
||||
To access a service exposed via a node port, run this command in a shell after starting minikube to get the address:
|
||||
```shell
|
||||
kubectl get nodes
|
||||
minikube service [-n NAMESPACE] [--url] NAME
|
||||
```
|
||||
|
||||
Minikube contains a built-in Docker daemon for running containers.
|
||||
If you use another Docker daemon for building your containers, you will have to publish them to a registry before minikube can pull them.
|
||||
You can use minikube's built in Docker daemon to avoid this extra step of pushing your images.
|
||||
Use the built-in Docker daemon with:
|
||||
## Networking
|
||||
|
||||
The minikube VM is exposed to the host system via a host-only IP address, that can be obtained with the `minikube ip` command.
|
||||
Any services of type `NodePort` can be accessed over that IP address, on the NodePort.
|
||||
|
||||
To determine the NodePort for your service, you can use a `kubectl` command like this:
|
||||
|
||||
`kubectl get service $SERVICE --output='jsonpath="{.spec.ports[0].NodePort}"'`
|
||||
|
||||
## Persistent Volumes
|
||||
Minikube supports [PersistentVolumes](http://kubernetes.io/docs/user-guide/persistent-volumes/) of type `hostPath`.
|
||||
These PersistentVolumes are mapped to a directory inside the minikube VM.
|
||||
|
||||
The Minikube VM boots into a tmpfs, so most directories will not be persisted across reboots (`minikube stop`).
|
||||
However, Minikube is configured to persist files stored under the following host directories:
|
||||
|
||||
* `/data`
|
||||
* `/var/lib/localkube`
|
||||
* `/var/lib/docker`
|
||||
|
||||
Here is an example PersistentVolume config to persist data in the '/data' directory:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: PersistentVolume
|
||||
metadata:
|
||||
name: pv0001
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
capacity:
|
||||
storage: 5Gi
|
||||
hostPath:
|
||||
path: /data/pv0001/
|
||||
```
|
||||
|
||||
## Mounted Host Folders
|
||||
Some drivers will mount a host folder within the VM so that you can easily share files between the VM and host. These are not configurable at the moment and different for the driver and OS you are using. Note: Host folder sharing is not implemented on Linux yet.
|
||||
|
||||
| Driver | OS | HostFolder | VM |
|
||||
| --- | --- | --- | --- |
|
||||
| Virtualbox | OSX | /Users | /Users |
|
||||
| Virtualbox | Windows | C://Users | /c/Users |
|
||||
| VMWare Fusion | OSX | /Users | /Users |
|
||||
| Xhyve | OSX | /Users | /Users |
|
||||
|
||||
|
||||
## Private Container Registries
|
||||
|
||||
To access a private container registry, follow the steps on [this page](http://kubernetes.io/docs/user-guide/images/).
|
||||
|
||||
We recommend you use ImagePullSecrets, but if you would like to configure access on the minikube VM you can place the `.dockercfg` in the `/home/docker` directory or the `config.json` in the `/home/docker/.docker` directory.
|
||||
|
||||
## Add-ons
|
||||
|
||||
In order to have minikube properly start/restart custom addons, place the addon(s) you wish to be launched with minikube in the `.minikube/addons` directory. Addons in this folder will be moved to the minikubeVM and launched each time minikube is started/restarted.
|
||||
|
||||
## Documentation
|
||||
|
||||
For a list of minikube's available commands see the [full CLI docs](./docs/minikube.md).
|
||||
|
||||
## Using Minikube with an HTTP Proxy
|
||||
|
||||
Minikube creates a Virtual Machine that includes Kubernetes and a Docker daemon.
|
||||
When Kubernetes attempts to schedule containers using Docker, the Docker daemon may require external network access to pull containers.
|
||||
|
||||
If you are behind an HTTP proxy, you may need to supply Docker with the proxy settings.
|
||||
To do this, pass the required environment variables as flags during `minikube start`.
|
||||
|
||||
For example:
|
||||
|
||||
```shell
|
||||
eval $(minikube docker-env)
|
||||
$ minikube start --docker-env HTTP_PROXY=http://$YOURPROXY:PORT \
|
||||
--docker-env HTTPS_PROXY=https://$YOURPROXY:PORT
|
||||
```
|
||||
This command sets up the Docker environment variables so a Docker client can communicate with the minikube Docker daemon.
|
||||
|
||||
```shell
|
||||
docker ps
|
||||
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
|
||||
42c643fea98b gcr.io/google_containers/kubernetes-dashboard-amd64:v1.0.1 "/dashboard --port=90" 3 minutes ago Up 3 minutes k8s_kubernetes-dashboard.1d0d880_kubernetes-dashboard-9brhv_kube-system_5062dd0b-370b-11e6-84b6-5eab1f51187f_134cba4c
|
||||
475db7659edf gcr.io/google_containers/pause-amd64:3.0 "/pause" 3 minutes ago Up 3 minutes k8s_POD.2225036b_kubernetes-dashboard-9brhv_kube-system_5062dd0b-370b-11e6-84b6-5eab1f51187f_e76d8136
|
||||
e9096501addf gcr.io/google-containers/kube-addon-manager-amd64:v2 "/opt/kube-addons.sh" 3 minutes ago Up 3 minutes k8s_kube-addon-manager.a1c58ca2_kube-addon-manager-127.0.0.1_kube-system_48abed82af93bb0b941173334110923f_82655b7d
|
||||
64748893cf7c gcr.io/google_containers/pause-amd64:3.0 "/pause" 4 minutes ago Up 4 minutes k8s_POD.d8dbe16c_kube-addon-manager-127.0.0.1_kube-system_48abed82af93bb0b941173334110923f_c67701c3
|
||||
```
|
||||
|
||||
## Known Issues
|
||||
* Features that require a Cloud Provider will not work in Minikube. These include:
|
||||
* LoadBalancers
|
||||
* Features that require multiple nodes. These include:
|
||||
* Advanced scheduling policies
|
||||
|
||||
## Design
|
||||
|
||||
Minikube uses [libmachine](https://github.com/docker/machine/tree/master/libmachine) for provisioning VMs, and [localkube](https://github.com/kubernetes/minikube/tree/master/pkg/localkube) (originally written and donated to this project by [RedSpread](https://redspread.com/)) for running the cluster.
|
||||
|
||||
For more information about minikube, see the [proposal](https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/local-cluster-ux.md).
|
||||
|
||||
## Additional Links:
|
||||
* **Goals and Non-Goals**: For the goals and non-goals of the minikube project, please see our [roadmap](./ROADMAP.md).
|
||||
* **Development Guide**: See [CONTRIBUTING.md](./CONTRIBUTING.md) for an overview of how to send pull requests.
|
||||
* **Building Minikube**: For instructions on how to build/test minikube from source, see the [build guide](./BUILD_GUIDE.md)
|
||||
* **Adding a New Dependency**: For instructions on how to add a new dependency to minikube see the [adding dependencies guide](./ADD_DEPENDENCY.md)
|
||||
* **Updating Kubernetes**: For instructions on how to add a new dependency to minikube see the [updating kubernetes guide](./UPDATE_KUBERNETES.md)
|
||||
|
||||
## Community
|
||||
|
||||
Contributions, questions, and comments are all welcomed and encouraged! minkube developers hang out on [Slack](https://kubernetes.slack.com) in the #minikube channel (get an invitation [here](http://slack.kubernetes.io/)). We also have the [kubernetes-dev Google Groups mailing list](https://groups.google.com/forum/#!forum/kubernetes-dev). If you are posting to the list please prefix your subject with "minikube: ".
|
||||
@@ -0,0 +1,172 @@
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
Kubernetes version 1.5 introduces support for Windows Server Containers. In version 1.5, the Kubernetes control plane (API Server, Scheduler, Controller Manager, etc) continue to run on Linux, while the kubelet and kube-proxy can be run on Windows Server.
|
||||
|
||||
**Note:** Windows Server Containers on Kubernetes is an Alpha feature in Kubernetes 1.5.
|
||||
|
||||
## Prerequisites
|
||||
In Kubernetes version 1.5, Windows Server Containers for Kubernetes is supported using the following:
|
||||
|
||||
1. Kubernetes control plane running on existing Linux infrastructure (version 1.5 or later)
|
||||
2. Kubenet network plugin setup on the Linux nodes
|
||||
3. Windows Server 2016 (RTM version 10.0.14393 or later)
|
||||
4. Docker Version 1.12.2-cs2-ws-beta or later
|
||||
|
||||
## Networking
|
||||
Network is achieved using L3 routing. Because third-party networking plugins (e.g. flannel, calico, etc) don’t natively work on Windows Server, existing technology that is built into the Windows and Linux operating systems is relied on. In this L3 networking approach, a /16 subnet is chosen for the cluster nodes, and a /24 subnet is assigned to each worker node. All pods on a given worker node will be connected to the /24 subnet. This allows pods on the same node to communicate with each other. In order to enable networking between pods running on different nodes, routing features that are built into Windows Server 2016 and Linux are used.
|
||||
|
||||
### Linux
|
||||
The above networking approach is already supported on Linux using a bridge interface, which essentially creates a private network local to the node. Similar to the Windows side, routes to all other pod CIDRs must be created in order to send packets via the “public” NIC.
|
||||
|
||||
### Windows
|
||||
Each Window Server node should have the following configuration:
|
||||
|
||||
1. Two NICs (virtual networking adapters) are required on each Windows Server node - The two Windows container networking modes of interest (transparent and L2 bridge) use an external Hyper-V virtual switch. This means that one of the NICs is entirely allocated to the bridge, creating the need for the second NIC.
|
||||
2. Transparent container network created - This is a manual configuration step and is shown in **_Route Setup_** section below
|
||||
3. RRAS (Routing) Windows feature enabled - Allows routing between NICs on the box, and also “captures” packets that have the destination IP of a POD running on the node. To enable, open “Server Manager”. Click on “Roles”, “Add Roles”. Click “Next”. Select “Network Policy and Access Services”. Click on “Routing and Remote Access Service” and the underlying checkboxes
|
||||
4. Routes defined pointing to the other pod CIDRs via the “public” NIC - These routes are added to the built-in routing table as shown in **_Route Setup_** section below
|
||||
|
||||
The following diagram illustrates the Windows Server networking setup for Kubernetes Setup
|
||||
|
||||
|
||||
## Setting up Windows Server Containers on Kubernetes
|
||||
To run Windows Server Containers on Kubernetes, you'll need to set up both your host machines and the Kubernetes node components for Windows and setup Routes for Pod communication on different nodes
|
||||
### Host Setup
|
||||
**Windows Host Setup**
|
||||
|
||||
1. Windows Server container host running Windows Server 2016 and Docker v1.12. Follow the setup instructions outlined by this blog post: https://msdn.microsoft.com/en-us/virtualization/windowscontainers/quick_start/quick_start_windows_server
|
||||
2. DNS support for Windows recently got merged to docker master and is currently not supported in a stable docker release. To use DNS build docker from master or download the binary from [Docker master](https://master.dockerproject.org/)
|
||||
3. Pull the `apprenda/pause` image from `https://hub.docker.com/r/apprenda/pause`
|
||||
4. RRAS (Routing) Windows feature enabled
|
||||
|
||||
**Linux Host Setup**
|
||||
|
||||
1. Linux hosts should be setup according to their respective distro documentation and the requirements of the Kubernetes version you will be using.
|
||||
2. CNI network plugin installed.
|
||||
|
||||
### Component Setup
|
||||
|
||||
Requirements
|
||||
|
||||
* Git
|
||||
* Go 1.7.1+
|
||||
* make (if using Linux or MacOS)
|
||||
* Important notes and other dependencies are listed [here](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md#building-kubernetes-on-a-local-osshell-environment)
|
||||
|
||||
**kubelet**
|
||||
|
||||
To build the *kubelet*, run:
|
||||
|
||||
1. `cd $GOPATH/src/k8s.io/kubernetes`
|
||||
2. Build *kubelet*
|
||||
1. Linux/MacOS: `KUBE_BUILD_PLATFORMS=windows/amd64 make WHAT=cmd/kubelet`
|
||||
2. Windows: `go build cmd/kubelet/kubelet.go`
|
||||
|
||||
**kube-proxy**
|
||||
|
||||
To build *kube-proxy*, run:
|
||||
|
||||
1. `cd $GOPATH/src/k8s.io/kubernetes`
|
||||
2. Build *kube-proxy*
|
||||
1. Linux/MacOS: `KUBE_BUILD_PLATFORMS=windows/amd64 make WHAT=cmd/kube-proxy`
|
||||
2. Windows: `go build cmd/kube-proxy/proxy.go`
|
||||
|
||||
### Route Setup
|
||||
The below example setup assumes one Linux and two Windows Server 2016 nodes and a cluster CIDR 192.168.0.0/16
|
||||
|
||||
| Hostname | Routable IP address | Pod CIDR |
|
||||
| --- | --- | --- |
|
||||
| Lin01 | `<IP of Lin01 host>` | 192.168.0.0/24 |
|
||||
| Win01 | `<IP of Win01 host>` | 192.168.1.0/24 |
|
||||
| Win02 | `<IP of Win02 host>` | 192.168.2.0/24 |
|
||||
|
||||
**Lin01**
|
||||
|
||||
```
|
||||
ip route add 192.168.1.0/24 via <IP of Win01 host>
|
||||
ip route add 192.168.2.0/24 via <IP of Win02 host>
|
||||
```
|
||||
|
||||
**Win01**
|
||||
|
||||
```
|
||||
docker network create -d transparent --gateway 192.168.1.1 --subnet 192.168.1.0/24 <network name>
|
||||
# A bridge is created with Adapter name "vEthernet (HNSTransparent)". Set its IP address to transparent network gateway
|
||||
netsh interface ipv4 set address "vEthernet (HNSTransparent)" addr=192.168.1.1
|
||||
route add 192.168.0.0 mask 255.255.255.0 192.168.0.1 if <Interface Id of the Routable Ethernet Adapter> -p
|
||||
route add 192.168.2.0 mask 255.255.255.0 192.168.2.1 if <Interface Id of the Routable Ethernet Adapter> -p
|
||||
```
|
||||
|
||||
**Win02**
|
||||
|
||||
```
|
||||
docker network create -d transparent --gateway 192.168.2.1 --subnet 192.168.2.0/24 <network name>
|
||||
# A bridge is created with Adapter name "vEthernet (HNSTransparent)". Set its IP address to transparent network gateway
|
||||
netsh interface ipv4 set address "vEthernet (HNSTransparent)" addr=192.168.2.1
|
||||
route add 192.168.0.0 mask 255.255.255.0 192.168.0.1 if <Interface Id of the Routable Ethernet Adapter> -p
|
||||
route add 192.168.1.0 mask 255.255.255.0 192.168.1.1 if <Interface Id of the Routable Ethernet Adapter> -p
|
||||
```
|
||||
|
||||
## Starting the Cluster
|
||||
To start your cluster, you'll need to start both the Linux-based Kubernetes control plane, and the Windows Server-based Kubernetes node components.
|
||||
## Starting the Linux-based Control Plane
|
||||
Use your preferred method to start Kubernetes cluster on Linux. Please note that Cluster CIDR might need to be updated.
|
||||
## Starting the Windows Node Components
|
||||
To start kubelet on your Windows node:
|
||||
Run the following in a PowerShell window. Be aware that if the node reboots or the process exits, you will have to rerun the commands below to restart the kubelet
|
||||
|
||||
1. Set environment variable *CONTAINER_NETWORK* value to the docker container network to use
|
||||
`$env:CONTAINER_NETWORK = "<docker network>"`
|
||||
|
||||
2. Run *kubelet* executable using the below command
|
||||
`kubelet.exe --hostname-override=<ip address/hostname of the windows node> --pod-infra-container-image="apprenda/pause" --resolv-conf="" --api_servers=<api server location>`
|
||||
|
||||
To start kube-proxy on your Windows node:
|
||||
|
||||
Run the following in a PowerShell window with administrative privileges. Be aware that if the node reboots or the process exits, you will have to rerun the commands below to restart the kube-proxy.
|
||||
|
||||
1. Set environment variable *INTERFACE_TO_ADD_SERVICE_IP* value to a node only network interface. The interface created when docker is installed should work
|
||||
`$env:INTERFACE_TO_ADD_SERVICE_IP = "vEthernet (HNS Internal NIC)"`
|
||||
|
||||
2. Run *kube-proxy* executable using the below command
|
||||
`.\proxy.exe --v=3 --proxy-mode=userspace --hostname-override=<ip address/hostname of the windows node> --master=<api server location> --bind-address=<ip address of the windows node>`
|
||||
|
||||
## Scheduling Pods on Windows
|
||||
Because your cluster has both Linux and Windows nodes, you must explictly set the nodeSelector constraint to be able to schedule Pods to Windows nodes. You must set nodeSelector with the label beta.kubernetes.io/os to the value windows; see the following example:
|
||||
|
||||
```
|
||||
{
|
||||
"apiVersion": "v1",
|
||||
"kind": "Pod",
|
||||
"metadata": {
|
||||
"name": "iis",
|
||||
"labels": {
|
||||
"name": "iis"
|
||||
}
|
||||
},
|
||||
"spec": {
|
||||
"containers": [
|
||||
{
|
||||
"name": "iis",
|
||||
"image": "microsoft/iis",
|
||||
"ports": [
|
||||
{
|
||||
"containerPort": 80
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"nodeSelector": {
|
||||
"beta.kubernetes.io/os": "windows"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Known Limitations:
|
||||
1. There is no network namespace in Windows and as a result currently only one container per pod is supported
|
||||
2. Secrets currently do not work because of a bug in Windows Server Containers described [here](https://github.com/docker/docker/issues/28401)
|
||||
3. ConfigMaps have not been implemented yet.
|
||||
4. `kube-proxy` implementation uses `netsh portproxy` and as it only supports TCP, DNS currently works only if the client retries DNS query using TCP
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 50 KiB |
Reference in New Issue
Block a user