whentimeslows/website
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #699 from yifan-gu/rkt_docs

Browse Source 
docs/getting-started-guides/rkt: Update rktnetes documentation for 1.3
This commit is contained in:
johndmulhausen
2016-06-24 11:15:01 -07:00
committed by GitHub
parent ee396b4439 53e888aa9c
commit 6f609cf321
3 changed files with 153 additions and 199 deletions
Download Patch File Download Diff File Expand all files Collapse all files
_data/guides.yml
+2 -2
View File
@@ -192,9 +192,9 @@ toc:
path: /docs/getting-started-guides/fedora/fedora-calico/
- title: rkt
section:
- title: Running Kubernetes on rkt
- title: Running Kubernetes with rkt
path: /docs/getting-started-guides/rkt/
- title: Notes on Different UX with rkt Container Runtime
- title: Known Issues when Using rkt
path: /docs/getting-started-guides/rkt/notes/
- title: Kubernetes on Mesos
path: /docs/getting-started-guides/mesos/
docs/getting-started-guides/rkt/index.md
+89 -128
View File
@@ -2,108 +2,100 @@
---
This document describes how to run Kubernetes using [rkt](https://github.com/coreos/rkt) as a container runtime.
We still have [a bunch of work](http://issue.k8s.io/8262) to do to make the experience with rkt wonderful, please stay tuned!
### **Prerequisite**
- [systemd](http://www.freedesktop.org/wiki/Software/systemd/) should be installed on the machine and should be enabled. The minimum version required at this moment (2015/09/01) is 219
- [systemd](http://www.freedesktop.org/wiki/Software/systemd/) should be installed on the machine and should be enabled.
The minimum version required for Kubernetes 1.3 is `219`.
*(Note that systemd is not required by rkt itself, we are using it here to monitor and manage the pods launched by kubelet.)*
- Install the latest rkt release according to the instructions [here](https://github.com/coreos/rkt).
The minimum version required for now is [v0.8.0](https://github.com/coreos/rkt/releases/tag/v0.8.0).
The minimum version required is [v1.9.1](https://github.com/coreos/rkt/releases/tag/v1.9.1).
- Note that for rkt version later than v0.7.0, `metadata service` is not required for running pods in private networks. So now rkt pods will not register the metadata service be default.
- The [rkt API service](http://coreos.com/rkt/docs/latest/subcommands/api-service.html) must be running on the node.
- Since release [v1.2.0-alpha.5](https://github.com/kubernetes/kubernetes/releases/tag/v1.2.0-alpha.5),
the [rkt API service](https://github.com/coreos/rkt/blob/master/api/v1alpha/README.md)
must be running on the node.
### Setup network
### Network Setup
You can configure the Kubernetes networking using its own `kubenet` and `CNI` [network
plugins](http://kubernetes.io/docs/admin/network-plugins/) by setting the kubelet's `--network-plugin` and `--network-plugin-dir` flag.
In addition, rkt supports using rkt's [Contained Networking](https://coreos.com/rkt/docs/latest/networking.html#contained-mode).
rkt uses the [Container Network Interface (CNI)](https://github.com/appc/cni)
to manage container networking. By default, all pods attempt to join a network
called `rkt.kubernetes.io`, which is currently defined [in
`rkt.go`](https://github.com/kubernetes/kubernetes/blob/v1.2.0-alpha.6/pkg/kubelet/rkt/rkt.go#L91).
In order for pods to get correct IP addresses, the CNI config file must be
edited to add this `rkt.kubernetes.io` network:
##### Use rkt's Contained Networking
#### Using flannel
In this mode, rkt will attempt to join pods into a network named `rkt.kubernetes.io`.
To use rkt's contained networking, you can leave the `--network-plugin` to empty, and put a network config file under one of the rkt's [config directories](https://github.com/coreos/rkt/blob/master/Documentation/configuration.md#command-line-flags), for example:
In addition to the basic prerequisites above, each node must be running
a [flannel](https://github.com/coreos/flannel) daemon. This implies
that a flannel-supporting etcd service must be available to the cluster
as well, apart from the Kubernetes etcd, which will not yet be
available at flannel configuration time. Once it's running, flannel can
be set up with a CNI config like:
```shell
$ cat <<EOF >/etc/rkt/net.d/k8s_cluster.conf
```shell
$ cat <<EOF >/etc/rkt/net.d/k8s_network_example.conf
{
"name": "rkt.kubernetes.io",
"type": "flannel"
}
EOF
```
While `k8s_cluster.conf` is a rather arbitrary name for the config file itself,
and can be adjusted to suit local conventions, the keys and values should be exactly
as shown above. `name` must be `rkt.kubernetes.io` and `type` should be `flannel`.
More details about the flannel CNI plugin can be found
[in the CNI documentation](https://github.com/appc/cni/blob/master/Documentation/flannel.md).
#### On GCE
Each VM on GCE has an additional 256 IP addresses routed to it, so
it is possible to forego flannel in smaller clusters. This makes the
necessary CNI config file a bit more verbose:
```shell
$ cat <<EOF >/etc/rkt/net.d/k8s_cluster.conf
{
"name": "rkt.kubernetes.io",
"type": "bridge",
"bridge": "cbr0",
"isGateway": true,
"ipam": {
"type": "host-local",
"subnet": "10.255.228.1/24",
"gateway": "10.255.228.1"
},
"name": "rkt.kubernetes.io",
"type": "bridge",
"bridge": "mybridge",
"mtu": 1460,
"addIf": "true",
"isGateway": true,
"ipMasq": true,
"ipam": {
"type": "host-local",
"subnet": "10.22.0.0/16",
"gateway": "10.22.0.1",
"routes": [
{ "dst": "0.0.0.0/0" }
]
}
}
EOF
```
This example creates a `bridge` plugin configuration for the CNI network, specifying
the bridge name `cbr0`. It also specifies the CIDR, in the `ipam` field.
However, there are a small number of caveats you should be aware of when using rkt's networking:
Creating these files for any moderately-sized cluster is at best inconvenient.
Work is in progress to
[enable Kubernetes to use the CNI by default]
(https://github.com/kubernetes/kubernetes/pull/18795/files).
As that work matures, such manual CNI config munging will become unnecessary
for primary use cases. For early adopters, an initial example shows one way to
[automatically generate these CNI configurations]
(https://gist.github.com/yifan-gu/fbb911db83d785915543)
for rkt.
* You must create an appropriate CNI configuration file with a network name of `rkt.kubernetes.io`.
* The downwards API and environment variable substitution will not contain the pod IP.
* The `/etc/hosts` file will not contain your own hostname (though `/etc/hostname` is populated).
### Local cluster
##### Use flannel
While it's recommended that you configure flannel using kubernetes' CNI support, you can also configure it using rkt's contained networking.
An example flannel/CNI config file looks like this:
```shell
$ cat <<EOF >/etc/rkt/net.d/k8s_flannel_example.conf
{
"name": "rkt.kubernetes.io",
"type": "flannel",
"delegate": {
"isDefaultGateway": true
}
}
EOF
```
For more information on flannel configuration, please read [CNI/flannel README](https://github.com/containernetworking/cni/blob/master/Documentation/flannel.md).
##### Use Google Compute Engine (GCE) network
Each VM on GCE has an additional 256 IP addresses routed to it, so it is possible to forego flannel in smaller clusters.
This can most easily be done by using the builtin kubenet plugin.
This can be done by setting `--network-plugin=kubenet`.
### Launch a local cluster
To use rkt as the container runtime, we need to supply the following flags to kubelet:
- `--container-runtime=rkt` chooses the container runtime to use. Possible values: 'docker', 'rkt'. Default: 'docker'.
- `--rkt-path=$PATH_TO_RKT_BINARY` sets the path of rkt binary. Leave empty to use the first rkt in $PATH.
- `--rkt-stage1-image` sets the path of the stage1 image. Local paths and http/https URLs are supported. Leave empty to use the 'stage1.aci' that locates in the same directory as the rkt binary.
- `--container-runtime=rkt` chooses the container runtime to use.
- `--rkt-api-endpoint=HOST:PORT` sets the endpoint of the rkt API service.
Leave empty to use the default one (`localhost:15441`).
- `--rkt-path=$PATH_TO_RKT_BINARY` sets the path of rkt binary.
Leave empty to use the first rkt in $PATH.
- `--rkt-stage1-image` sets the name of the stage1 image, e.g. coreos.com/rkt/stage1-coreos.
Leave empty to use the default stage1 image in the rkt's configuration.
If you are using the [hack/local-up-cluster.sh](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/hack/local-up-cluster.sh) script to launch the local cluster, then you can edit the environment variable `CONTAINER_RUNTIME`, `RKT_PATH` and `RKT_STAGE1_IMAGE` to
set these flags:
If you are using the [hack/local-up-cluster.sh](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/hack/local-up-cluster.sh) script to launch the local cluster, then you can edit the environment variable `CONTAINER_RUNTIME`, `RKT_PATH` and `RKT_STAGE1_IMAGE` to set these flags, the `RKT_PATH` and `RKT_STAGE1_IMAGE` are optional if you have `rkt` in your `$PATH` with appropriate configuration.
```shell
$ export CONTAINER_RUNTIME=rkt
$ export RKT_PATH=$PATH_TO_RKT_BINARY
$ export RKT_STAGE1_IMAGE=PATH=$PATH_TO_STAGE1_IMAGE
$ export RKT_STAGE1_IMAGE=$NAME_OF_THE_STAGE1_IMAGE
```
Then we can launch the local cluster using the script:
@@ -112,21 +104,26 @@ Then we can launch the local cluster using the script:
$ hack/local-up-cluster.sh
```
### CoreOS cluster on Google Compute Engine (GCE)
We are also working on set up rkt as the container runtime for [minikube](https://github.com/kubernetes/minikube/issues/168).
To use rkt as the container runtime for your CoreOS cluster on GCE, you need to specify the OS distribution, project, image:
### Launch a CoreOS/rkt cluster on Google Compute Engine (GCE)
Here we provide instruction on how to use the `kube-up` script to launch a CoreOS/rkt cluster on GCE.
In order to do that, you need to specify the OS distribution, project, image:
```shell
$ export KUBE_OS_DISTRIBUTION=coreos
$ export KUBE_GCE_NODE_IMAGE=<image_id>
$ export KUBE_GCE_MASTER_PROJECT=coreos-cloud
$ export KUBE_GCE_MASTER_IMAGE=<image_id>
$ export KUBE_GCE_NODE_PROJECT=coreos-cloud
$ export KUBE_GCE_NODE_IMAGE=<image_id>
$ export KUBE_CONTAINER_RUNTIME=rkt
```
You can optionally choose the version of rkt used by setting `KUBE_RKT_VERSION`:
```shell
$ export KUBE_RKT_VERSION=0.15.0
$ export KUBE_RKT_VERSION=1.9.1
```
Then you can launch the cluster by:
@@ -135,52 +132,25 @@ Then you can launch the cluster by:
$ cluster/kube-up.sh
```
Note that we are still working on making all containerized the master components run smoothly in rkt. Before that we are not able to run the master node with rkt yet.
### Launch a CoreOS/rkt cluster on AWS
### CoreOS cluster on AWS
`kube-up` for AWS is currently unsupported.
Instead, we recommend you to refer the [Kubernetes on AWS guide](https://coreos.com/kubernetes/docs/latest/kubernetes-on-aws.html) to launch a CoreOS/rkt cluster on AWS.
To use rkt as the container runtime for your CoreOS cluster on AWS, you need to specify the provider and OS distribution:
### Deploy apps to your cluster
```shell
$ export KUBERNETES_PROVIDER=aws
$ export KUBE_OS_DISTRIBUTION=coreos
$ export KUBE_CONTAINER_RUNTIME=rkt
```
After you created the cluster, you can start deploying apps to the cluster. For example here is how you can [deploy a simgle nginx app](/docs/user-guide/simple-nginx).
More examples can be found in the [examples directory](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/).
You can optionally choose the version of rkt used by setting `KUBE_RKT_VERSION`:
### Known Issues and Differences
```shell
$ export KUBE_RKT_VERSION=0.8.0
```
You can optionally choose the CoreOS channel by setting `COREOS_CHANNEL`:
```shell
$ export COREOS_CHANNEL=stable
```
Then you can launch the cluster by:
```shell
$ kube-up.sh
```
Note: CoreOS is not supported as the master using the automated launch
scripts. The master node is always Ubuntu.
### Getting started with your cluster
See [a simple nginx example](/docs/user-guide/simple-nginx) to try out your new cluster.
For more complete applications, please look in the [examples directory](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/).
### Different UX with rkt container runtime
rkt and Docker have very different designs, as well as ACI and Docker image format. Users might experience some different experience when switching from one to the other. More information can be found [here](/docs/getting-started-guides/rkt/notes/).
rkt and Docker have very different designs, as well as ACI and Docker image format.
Users might experience some different experience when switching from one to the other.
More information can be found [here](/docs/getting-started-guides/rkt/notes/).
### Debugging
Here are several tips for you when you run into any issues.
Here are several tips in case you run into any issues.
##### Check logs
@@ -191,29 +161,20 @@ If the cluster is using salt, we can edit the [logging.sls](https://github.com/k
##### Check rkt pod status
To check the pods' status, we can use rkt command, such as `rkt list`, `rkt status`, `rkt image list`, etc.
More information about rkt command line can be found [here](https://github.com/coreos/rkt/blob/master/Documentation/commands.md)
More information about rkt command line can be found [here](https://github.com/coreos/rkt/blob/master/Documentation/commands.md).
##### Check journal logs
As we use systemd to launch rkt pods(by creating service files which will run `rkt run-prepared`, we can check the pods' log
using `journalctl`:
As we use systemd to launch/manage rkt pods, we can check the pods' log using `journalctl`:
- Check the running state of the systemd service:
```shell
$ sudo journalctl -u $SERVICE_FILE
$ sudo journalctl -u ${SERVICE_NAME}
```
where `$SERVICE_FILE` is the name of the service file created for the pod, you can find it in the kubelet logs.
##### Check the log of the container in the pod:
```shell
$ sudo journalctl -M rkt-$UUID -u $CONTAINER_NAME
```
where `$UUID` is the rkt pod's UUID, which you can find via `rkt list --full`, and `$CONTAINER_NAME` is the container's name.
where `${SERVICE_NAME}` is the name of the service file created for the pod, typically the format is `k8s_${RKT_UUID}`.
##### Check Kubernetes events, logs.
Besides above tricks, Kubernetes also provides us handy tools for debugging the pods. More information can be found [here](/docs/user-guide/application-troubleshooting)
Kubernetes also provides various tools for debugging. More information can be found [here](/docs/user-guide/application-troubleshooting).
docs/getting-started-guides/rkt/notes.md
+62 -69
View File
@@ -1,99 +1,92 @@
---
---
# Notes on Different UX with rkt container runtime
# Known issues
### Doesn't support ENTRYPOINT + CMD feature
The following features either are not supported or have large caveats when using the rkt container runtime.
To run a Docker image, rkt will convert it into [App Container Image (ACI) format](https://github.com/appc/spec/blob/master/SPEC.md) first.
However, during the conversion, the `ENTRYPOINT` and `CMD` are concatentated to construct ACI's `Exec` field.
This means after the conversion, we are not able to replace only `ENTRYPOINT` or `CMD` without touching the other part.
So for now, users are recommended to specify the **executable path** in `Command` and **arguments** in `Args`.
(This has the same effect if users specify the **executable path + arguments** in `Command` or `Args` alone).
##### Non-existent host volume paths
For example:
When mounting a host volume path that does not exist, rkt will error out. Under the Docker runtime, an empty directory will be created at the referenced path.
An example of a pod which will error out:
```yaml
apiVersion: v1
kind: Pod
metadata:
name: nginx
labels:
name: nginx
name: mount-dne
name: mount-dne
spec:
volumes:
- name: does-not-exist
hostPath:
path: /does/not/exist
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
- name: exit
image: busybox
command: ["sh", "-c", "ls /test; sleep 60"]
volumeMounts:
- mountPath: /test
name: does-not-exist
```
The above pod yaml file is valid as it's not specifying `Command` or `Args`, so the default `ENTRYPOINT` and `CMD` of the image will be used.
##### Kubectl attach
The `kubectl attach` command does not work under the rkt container runtime.
And because of this, some flags in `kubectl run` are not supported, including:
- --attach=true
- --leave-stdin-open=true
- --rm=true
##### Init containers
The alpha [init container](https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/container-init.md) feature is currently not supported.
##### Container restart back-off
Exponential restart back-off for a failing container is currently not supported.
##### Experimental NVIDIA GPU support
The `--experimental-nvidia-gpus` flag, and related [GPU features](https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/gpu-support.md) are not supported.
##### QoS Classes
Under rkt, QoS classes do not result in the `OOM Score` of containers being adjusted as occurs under Docker.
##### HostPID and HostIPC namespaces
Setting a the hostPID or hostIPC flag on a pod is not supported.
For example, the following pod will not run correctly:
```yaml
apiVersion: v1
kind: Pod
metadata:
name: busybox
labels:
name: busybox
name: host-ipc-pid
name: host-ipc-pid
spec:
hostIPC: true
hostPID: true
containers:
- name: busybox
image: busybox
command:
- /bin/sleep
- 1000
...
```
```yaml
apiVersion: v1
kind: Pod
metadata:
name: busybox
labels:
name: busybox
spec:
containers:
- name: busybox
image: busybox
command:
- /bin/sleep
args:
- 1000
```
On the other hand, when running the pod with [stage1-fly](https://coreos.com/rkt/docs/latest/running-fly-stage1.html), the pod will be run in the host namespace.
```yaml
apiVersion: v1
kind: Pod
metadata:
name: busybox
labels:
name: busybox
spec:
containers:
- name: busybox
image: busybox
args:
- /bin/sleep
- 1000
```
All the three examples above are valid as they contain both the executable path and the arguments.
##### Container image updates (patch)
```yaml
apiVersion: v1
kind: Pod
metadata:
name: busybox
labels:
name: busybox
spec:
containers:
- name: busybox
image: busybox
args:
- 1000
```
Patching a pod to change the image will result in the entire pod restarting, not just the container that was changed.
The last example is invalid, as we cannot override just the `CMD` of the image alone.
##### Volume mounts specifying a subPath
The [subPath](https://github.com/kubernetes/kubernetes/pull/22575) feature does not work correctly under rkt.
In addition, the above-issue of Non-existent host volume paths being invalid would make many common use-cases for subPaths fail in that way as well.
In some cases, this issue can be worked around by creating and using subdirectories from within the container rather than relying on Kubernetes to do so.