From c8abb1c5f84986c77a1c7d1b4f1db79751ed2846 Mon Sep 17 00:00:00 2001 From: Bryan Boreham Date: Wed, 28 Sep 2016 16:30:41 +0100 Subject: [PATCH 01/87] Clarify what kubenet does and does not do --- docs/admin/network-plugins.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/admin/network-plugins.md b/docs/admin/network-plugins.md index c0397016da..0d7f8981e2 100644 --- a/docs/admin/network-plugins.md +++ b/docs/admin/network-plugins.md @@ -40,7 +40,11 @@ The CNI plugin is selected by passing Kubelet the `--network-plugin=cni` command ### kubenet -The Linux-only kubenet plugin provides functionality similar to the `--configure-cbr0` kubelet command-line option. It creates a Linux bridge named `cbr0` and creates a veth pair for each pod with the host end of each pair connected to `cbr0`. The pod end of the pair is assigned an IP address allocated from a range assigned to the node either through configuration or by the controller-manager. `cbr0` is assigned an MTU matching the smallest MTU of an enabled normal interface on the host. The kubenet plugin is currently mutually exclusive with, and will eventually replace, the --configure-cbr0 option. It is also currently incompatible with the flannel experimental overlay. +Kubenet is intended to be a very basic, simple network plugin, on Linux only. It does not, of itself, implement more advanced features like cross-node networking or network policy. It is typically used together with a cloud provider that sets up routing rules for communication between nodes, or in single-node environments. + +Kubenet creates a Linux bridge named `cbr0` and creates a veth pair for each pod with the host end of each pair connected to `cbr0`. The pod end of the pair is assigned an IP address allocated from a range assigned to the node either through configuration or by the controller-manager. `cbr0` is assigned an MTU matching the smallest MTU of an enabled normal interface on the host. + +The kubenet plugin is currently mutually exclusive with, and will eventually replace, the --configure-cbr0 option. It is also currently incompatible with the flannel experimental overlay. The plugin requires a few things: From 8b4e7e8bc611fddc7250a65a8ad4a265a2ebaa33 Mon Sep 17 00:00:00 2001 From: Keith Wansbrough Date: Tue, 18 Oct 2016 15:22:13 +0100 Subject: [PATCH 02/87] Instructions for Windows download of kubectl. The binary name is kubectl.exe in this case, not kubectl, which is not immediately obvious. --- docs/getting-started-guides/minikube.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/getting-started-guides/minikube.md b/docs/getting-started-guides/minikube.md index 020fc5e257..5d3c69f6b1 100644 --- a/docs/getting-started-guides/minikube.md +++ b/docs/getting-started-guides/minikube.md @@ -80,6 +80,8 @@ curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/darwin/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ # OS X/386 curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/darwin/386/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ +# Windows +curl -Lo kubectl.exe http://storage.googleapis.com/kubernetes-release/release/v1.3.0/bin/windows/amd64/kubectl.exe ``` The generic download path is: From 0903054357418e6a93746eee498f12ca7413bda4 Mon Sep 17 00:00:00 2001 From: Keith Wansbrough Date: Wed, 19 Oct 2016 12:03:28 +0100 Subject: [PATCH 03/87] Don't hardcode kubectl version. --- docs/getting-started-guides/minikube.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting-started-guides/minikube.md b/docs/getting-started-guides/minikube.md index 5d3c69f6b1..e72a70299a 100644 --- a/docs/getting-started-guides/minikube.md +++ b/docs/getting-started-guides/minikube.md @@ -81,7 +81,7 @@ curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page # OS X/386 curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/darwin/386/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ # Windows -curl -Lo kubectl.exe http://storage.googleapis.com/kubernetes-release/release/v1.3.0/bin/windows/amd64/kubectl.exe +curl -Lo kubectl.exe http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/windows/amd64/kubectl.exe ``` The generic download path is: From eb5f7add4b1fa40e5a47f6825cef7c547ab08128 Mon Sep 17 00:00:00 2001 From: Keith Wansbrough Date: Wed, 19 Oct 2016 12:04:25 +0100 Subject: [PATCH 04/87] Suggest alternative to curl for Windows. --- docs/getting-started-guides/minikube.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting-started-guides/minikube.md b/docs/getting-started-guides/minikube.md index e72a70299a..e8b75b99d8 100644 --- a/docs/getting-started-guides/minikube.md +++ b/docs/getting-started-guides/minikube.md @@ -80,7 +80,7 @@ curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/darwin/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ # OS X/386 curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/darwin/386/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ -# Windows +# Windows (using curl; or use your browser "save link as...") curl -Lo kubectl.exe http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/windows/amd64/kubectl.exe ``` From 33cdc55903b43bd204bf4e0cdfbab184755714cf Mon Sep 17 00:00:00 2001 From: Keith Wansbrough Date: Wed, 19 Oct 2016 14:01:30 +0100 Subject: [PATCH 05/87] Just give a simple link for Windows kubectl. --- docs/getting-started-guides/minikube.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/getting-started-guides/minikube.md b/docs/getting-started-guides/minikube.md index e8b75b99d8..ee827bab55 100644 --- a/docs/getting-started-guides/minikube.md +++ b/docs/getting-started-guides/minikube.md @@ -80,10 +80,10 @@ curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/darwin/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ # OS X/386 curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/darwin/386/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ -# Windows (using curl; or use your browser "save link as...") -curl -Lo kubectl.exe http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/windows/amd64/kubectl.exe ``` +For Windows, download [kubectl.exe](http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/windows/amd64/kubectl.exe) and save it to a location on your PATH. + The generic download path is: ``` https://storage.googleapis.com/kubernetes-release/release/${K8S_VERSION}/bin/${GOOS}/${GOARCH}/${K8S_BINARY} From 441043a22a59d34b34cbdcdf8dc2437c552ef5c8 Mon Sep 17 00:00:00 2001 From: Justin Santa Barbara Date: Wed, 19 Oct 2016 13:08:21 -0400 Subject: [PATCH 06/87] Instructions for how to download kubectl --- _data/guides.yml | 2 + docs/getting-started-guides/kubectl.md | 63 ++++++++++++++++++++++++++ 2 files changed, 65 insertions(+) create mode 100644 docs/getting-started-guides/kubectl.md diff --git a/_data/guides.yml b/_data/guides.yml index 40d47b08d6..e77e8e1b01 100644 --- a/_data/guides.yml +++ b/_data/guides.yml @@ -12,6 +12,8 @@ toc: path: /docs/getting-started-guides/kubeadm/ - title: Hello World on Google Container Engine path: /docs/hellonode/ + - title: Downloading kubectl + path: /docs/getting-started-guides/kubectl/ - title: Downloading or Building Kubernetes path: /docs/getting-started-guides/binary_release/ - title: Online Training Course diff --git a/docs/getting-started-guides/kubectl.md b/docs/getting-started-guides/kubectl.md new file mode 100644 index 0000000000..6c5043e744 --- /dev/null +++ b/docs/getting-started-guides/kubectl.md @@ -0,0 +1,63 @@ +--- +--- + + + +## Overview + +kubectl is the command line tool you will use when interacting with kubernetes. + +You should use a version of kubectl that is newer than your server. `kubectl version` will print +the server and client versions. + +## Download a release + +Download kubectl from the [official Kubernetes releases](https://console.cloud.google.com/storage/browser/kubernetes-release/release/): + +On MacOS: + +``` +wget https://storage.googleapis.com/kubernetes-release/release/v1.4.3/bin/darwin/amd64/kubectl +chmod +x kubectl +mv kubectl /usr/local/bin/kubectl +``` + +On Linux: + +``` +wget https://storage.googleapis.com/kubernetes-release/release/v1.4.3/bin/linux/amd64/kubectl +chmod +x kubectl +mv kubectl /usr/local/bin/kubectl +``` + + +You may need to `sudo` the `mv`; you can put it anywhere in your `PATH` - some people prefer to install to `~/bin`. + + +## Alternatives + +### Download as part of the Google Cloud SDK + +kubectl can be installed as part of the Google Cloud SDK: + +First install the [Google Cloud SDK](https://cloud.google.com/sdk/). + +After Google Cloud SDK installs, run the following command to install `kubectl`: + +```shell +gcloud components install kubectl +``` + +Do check that the version is sufficiently up-to-date using `kubectl version --client`. + +### Install with brew + +If you are on MacOS and using brew, you can install with: + +```shell +brew install kubectl +``` + +Do check that the version is sufficiently up-to-date using `kubectl version --client`. From 4f0d6c160907044e6ce4c4940b3d69ee0e952c77 Mon Sep 17 00:00:00 2001 From: Bryan Boreham Date: Thu, 20 Oct 2016 09:48:17 +0100 Subject: [PATCH 07/87] Adjust wording per review comments --- docs/admin/network-plugins.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/admin/network-plugins.md b/docs/admin/network-plugins.md index 0d7f8981e2..b20e788b1a 100644 --- a/docs/admin/network-plugins.md +++ b/docs/admin/network-plugins.md @@ -40,11 +40,11 @@ The CNI plugin is selected by passing Kubelet the `--network-plugin=cni` command ### kubenet -Kubenet is intended to be a very basic, simple network plugin, on Linux only. It does not, of itself, implement more advanced features like cross-node networking or network policy. It is typically used together with a cloud provider that sets up routing rules for communication between nodes, or in single-node environments. +Kubenet is a very basic, simple network plugin, on Linux only. It does not, of itself, implement more advanced features like cross-node networking or network policy. It is typically used together with a cloud provider that sets up routing rules for communication between nodes, or in single-node environments. Kubenet creates a Linux bridge named `cbr0` and creates a veth pair for each pod with the host end of each pair connected to `cbr0`. The pod end of the pair is assigned an IP address allocated from a range assigned to the node either through configuration or by the controller-manager. `cbr0` is assigned an MTU matching the smallest MTU of an enabled normal interface on the host. -The kubenet plugin is currently mutually exclusive with, and will eventually replace, the --configure-cbr0 option. It is also currently incompatible with the flannel experimental overlay. +The kubenet plugin is mutually exclusive with the --configure-cbr0 option. The plugin requires a few things: From 98a4b521fd8fbf82426354705087ae0a78031cf7 Mon Sep 17 00:00:00 2001 From: Seth Jennings Date: Thu, 20 Oct 2016 10:46:51 -0500 Subject: [PATCH 08/87] AUTOSCALER not AUTOSCALING --- docs/admin/cluster-management.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/admin/cluster-management.md b/docs/admin/cluster-management.md index 7781368bc8..97362c4bab 100644 --- a/docs/admin/cluster-management.md +++ b/docs/admin/cluster-management.md @@ -95,13 +95,13 @@ If you are using GCE then you can either enable it while creating a cluster with To configure cluser autoscaler you have to set 3 environment variables: * `KUBE_ENABLE_CLUSTER_AUTOSCALER` - it enables cluster autoscaler if set to true. -* `KUBE_AUTOSCALING_MIN_NODES` - minimum number of nodes in the cluster. -* `KUBE_AUTOSCALING_MAX_NODES` - maximum number of nodes in the cluster. +* `KUBE_AUTOSCALER_MIN_NODES` - minimum number of nodes in the cluster. +* `KUBE_AUTOSCALER_MAX_NODES` - maximum number of nodes in the cluster. Example: ```shell -KUBE_ENABLE_CLUSTER_AUTOSCALER=true KUBE_AUTOSCALING_MIN_NODES=3 KUBE_AUTOSCALING_MAX_NODES=10 NUM_NODES=5 ./cluster/kube-up.sh +KUBE_ENABLE_CLUSTER_AUTOSCALER=true KUBE_AUTOSCALER_MIN_NODES=3 KUBE_AUTOSCALER_MAX_NODES=10 NUM_NODES=5 ./cluster/kube-up.sh ``` On GKE you configure cluster autoscaler either on cluster creation or update or when creating a particular node pool From 0d3a83f0d51c49be7f05c6afb2df94e9acc17780 Mon Sep 17 00:00:00 2001 From: roopakparikh Date: Fri, 21 Oct 2016 14:01:00 -0700 Subject: [PATCH 09/87] Added Platform9.com solutions at couple of Places Adding Platform9.com Kubernetes solution in the Hosted section as well as Table of solutions. --- docs/getting-started-guides/index.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/getting-started-guides/index.md b/docs/getting-started-guides/index.md index a728980e50..b2da179cf9 100644 --- a/docs/getting-started-guides/index.md +++ b/docs/getting-started-guides/index.md @@ -43,6 +43,8 @@ clusters. [KCluster.io](https://kcluster.io) provides highly available and scalable managed Kubernetes clusters for AWS. +[Platform9.com](https://platform9.com/products/kubernetes/) offers managed Kubernetes on-premises or any public cloud, and provides 24/7 health monitoring and alerting. + ### Turn-key Cloud Solutions These solutions allow you to create Kubernetes clusters on a range of Cloud IaaS providers with only a @@ -123,6 +125,7 @@ GKE | | | GCE | [docs](https://clou Stackpoint.io | | multi-support | multi-support | [docs](http://www.stackpointcloud.com) | | Commercial AppsCode.com | Saltstack | Debian | multi-support | [docs](https://appscode.com/products/cloud-deployment/) | | Commercial KCluster.io | | multi-support | multi-support | [docs](https://kcluster.io) | | Commercial +Platform9.com | | multi-support | multi-support | [docs](https://platform9.com/products/kubernetes/) | | Commercial GCE | Saltstack | Debian | GCE | [docs](/docs/getting-started-guides/gce) | ['œ“][1] | Project Azure | CoreOS | CoreOS | Weave | [docs](/docs/getting-started-guides/coreos/azure/) | | Community ([@errordeveloper](https://github.com/errordeveloper), [@squillace](https://github.com/squillace), [@chanezon](https://github.com/chanezon), [@crossorigin](https://github.com/crossorigin)) Azure | Ignition | Ubuntu | Azure | [docs](/docs/getting-started-guides/azure) | | Community (Microsoft: [@brendandburns](https://github.com/brendandburns), [@colemickens](https://github.com/colemickens)) From 0ebd20ccfbb56b1e3730ad5d559de5e7377d12a8 Mon Sep 17 00:00:00 2001 From: Justin Santa Barbara Date: Fri, 21 Oct 2016 21:05:23 -0400 Subject: [PATCH 10/87] Fixes per code review --- _data/guides.yml | 2 +- docs/getting-started-guides/kubectl.md | 21 ++++++++++++--------- 2 files changed, 13 insertions(+), 10 deletions(-) diff --git a/_data/guides.yml b/_data/guides.yml index e77e8e1b01..a661cbfb9a 100644 --- a/_data/guides.yml +++ b/_data/guides.yml @@ -12,7 +12,7 @@ toc: path: /docs/getting-started-guides/kubeadm/ - title: Hello World on Google Container Engine path: /docs/hellonode/ - - title: Downloading kubectl + - title: Installing kubectl path: /docs/getting-started-guides/kubectl/ - title: Downloading or Building Kubernetes path: /docs/getting-started-guides/binary_release/ diff --git a/docs/getting-started-guides/kubectl.md b/docs/getting-started-guides/kubectl.md index 6c5043e744..0498ceb646 100644 --- a/docs/getting-started-guides/kubectl.md +++ b/docs/getting-started-guides/kubectl.md @@ -7,10 +7,12 @@ li>.highlighter-rouge {position:relative; top:3px;} ## Overview -kubectl is the command line tool you will use when interacting with kubernetes. +kubectl is the command line tool you use to interact with Kubernetes clusters. -You should use a version of kubectl that is newer than your server. `kubectl version` will print -the server and client versions. +You should use a version of kubectl that is at least as new as your server. +`kubectl version` will print the server and client versions. Using the same version of kubectl +as your server naturally works; using a newer kubectl than your server also works; but if you use +an older kubectl with a newer server you may see odd validation errors . ## Download a release @@ -18,16 +20,16 @@ Download kubectl from the [official Kubernetes releases](https://console.cloud.g On MacOS: -``` -wget https://storage.googleapis.com/kubernetes-release/release/v1.4.3/bin/darwin/amd64/kubectl +```shell +wget https://storage.googleapis.com/kubernetes-release/release/v1.4.4/bin/darwin/amd64/kubectl chmod +x kubectl mv kubectl /usr/local/bin/kubectl ``` On Linux: -``` -wget https://storage.googleapis.com/kubernetes-release/release/v1.4.3/bin/linux/amd64/kubectl +```shell +wget https://storage.googleapis.com/kubernetes-release/release/v1.4.4/bin/linux/amd64/kubectl chmod +x kubectl mv kubectl /usr/local/bin/kubectl ``` @@ -50,7 +52,7 @@ After Google Cloud SDK installs, run the following command to install `kubectl`: gcloud components install kubectl ``` -Do check that the version is sufficiently up-to-date using `kubectl version --client`. +Do check that the version is sufficiently up-to-date using `kubectl version`. ### Install with brew @@ -60,4 +62,5 @@ If you are on MacOS and using brew, you can install with: brew install kubectl ``` -Do check that the version is sufficiently up-to-date using `kubectl version --client`. +The homebrew project is independent from kubernetes, so do check that the version is +sufficiently up-to-date using `kubectl version`. \ No newline at end of file From 689c4b02b48783991c2e8cdf2e6c2a7446b8546d Mon Sep 17 00:00:00 2001 From: Justin Santa Barbara Date: Fri, 21 Oct 2016 21:05:38 -0400 Subject: [PATCH 11/87] Add kubectl shell autocompletion section --- docs/getting-started-guides/kubectl.md | 43 +++++++++++++++++++++++++- 1 file changed, 42 insertions(+), 1 deletion(-) diff --git a/docs/getting-started-guides/kubectl.md b/docs/getting-started-guides/kubectl.md index 0498ceb646..b346b75c7d 100644 --- a/docs/getting-started-guides/kubectl.md +++ b/docs/getting-started-guides/kubectl.md @@ -63,4 +63,45 @@ brew install kubectl ``` The homebrew project is independent from kubernetes, so do check that the version is -sufficiently up-to-date using `kubectl version`. \ No newline at end of file +sufficiently up-to-date using `kubectl version`. + + +# Enabling shell autocompletion + +kubectl includes autocompletion support, which can save a lot of typing! + +The completion script itself is generated by kubectl, so you typically just need to invoke it from your profile. + +Common examples are provided here, but for more details please consult `kubectl completion -h` + +## On Linux, using bash + +To add it to your current shell: `source <(kubectl completion bash)` + +To add kubectl autocompletion to your profile (so it is automatically loaded in future shells): + +```shell +echo "source <(kubectl completion bash)" >> ~/.bashrc +``` + +## On MacOS, using bash + +On MacOS, you will need to install the bash-completion support first: + +```shell +brew install bash-completion +``` + +To add it to your current shell: + +```shell +source $(brew --prefix)/etc/bash_completion +source <(kubectl completion bash) +``` + +To add kubectl autocompletion to your profile (so it is automatically loaded in future shells): + +```shell +echo "source $(brew --prefix)/etc/bash_completion" >> ~/.bashrc +echo "source <(kubectl completion bash)" >> ~/.bashrc +``` From 500aec5eedf60f574d4022b65f162f6f6dd0344c Mon Sep 17 00:00:00 2001 From: Justin Santa Barbara Date: Fri, 21 Oct 2016 21:39:25 -0400 Subject: [PATCH 12/87] A few updates based on MacOS testing --- docs/getting-started-guides/kubectl.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/docs/getting-started-guides/kubectl.md b/docs/getting-started-guides/kubectl.md index b346b75c7d..bd2512707b 100644 --- a/docs/getting-started-guides/kubectl.md +++ b/docs/getting-started-guides/kubectl.md @@ -102,6 +102,9 @@ source <(kubectl completion bash) To add kubectl autocompletion to your profile (so it is automatically loaded in future shells): ```shell -echo "source $(brew --prefix)/etc/bash_completion" >> ~/.bashrc -echo "source <(kubectl completion bash)" >> ~/.bashrc +echo "source $(brew --prefix)/etc/bash_completion" >> ~/.bash_profile +echo "source <(kubectl completion bash)" >> ~/.bash_profile ``` + +Please note that this only appears to work currently if you install using `brew install kubectl`, +and not if you downloaded kubectl directly. \ No newline at end of file From 115d3cde97ee794bbb7fc073160b17b75df88b31 Mon Sep 17 00:00:00 2001 From: xiangpengzhao Date: Fri, 28 Oct 2016 00:43:49 -0400 Subject: [PATCH 13/87] Fix path in reference.yml --- _data/reference.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/_data/reference.yml b/_data/reference.yml index 5d4fe17f7b..62fdb0ca92 100644 --- a/_data/reference.yml +++ b/_data/reference.yml @@ -219,7 +219,7 @@ toc: - title: Replication Controller path: /docs/user-guide/replication-controller/ - title: Resource Quotas - path: /docs/admin/resource-quota/ + path: /docs/admin/resourcequota/ - title: Scheduled Jobs path: /docs/user-guide/scheduled-jobs/ - title: Secrets @@ -269,6 +269,6 @@ toc: - title: Federation Components section: - title: federation-apiserver - path: /docs/admin/federation-apiserver.md + path: /docs/admin/federation-apiserver - title : federation-controller-mananger - path: /docs/admin/federation-controller-manager.md + path: /docs/admin/federation-controller-manager From e34352c152198620da8d135ef5b8aaf19e7dd7ce Mon Sep 17 00:00:00 2001 From: Julien Vey Date: Fri, 28 Oct 2016 10:39:51 +0200 Subject: [PATCH 14/87] Remove line numbers in ingress resource snippet * To be consistent with the rest of the guide. * To simplify copy/paste of the example. --- docs/user-guide/ingress.md | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/docs/user-guide/ingress.md b/docs/user-guide/ingress.md index 6ce1eb915d..a8324bea24 100644 --- a/docs/user-guide/ingress.md +++ b/docs/user-guide/ingress.md @@ -53,18 +53,18 @@ Make sure you review the [beta limitations](https://github.com/kubernetes/contri A minimal Ingress might look like: ```yaml -01. apiVersion: extensions/v1beta1 -02. kind: Ingress -03. metadata: -04. name: test-ingress -05. spec: -06. rules: -07. - http: -08. paths: -09. - path: /testpath -10. backend: -11. serviceName: test -12. servicePort: 80 +apiVersion: extensions/v1beta1 +kind: Ingress +metadata: + name: test-ingress +spec: + rules: + - http: + paths: + - path: /testpath + backend: + serviceName: test + servicePort: 80 ``` *POSTing this to the API server will have no effect if you have not configured an [Ingress controller](#ingress-controllers).* From b72518e69c048db1e1f1b76a1001ca33a2eb49eb Mon Sep 17 00:00:00 2001 From: Zihong Zheng Date: Fri, 28 Oct 2016 09:39:32 -0700 Subject: [PATCH 15/87] Add troubleshooting section to kube-dns readme --- docs/admin/dns.md | 143 +++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 140 insertions(+), 3 deletions(-) diff --git a/docs/admin/dns.md b/docs/admin/dns.md index cc132201aa..b87d3196bc 100644 --- a/docs/admin/dns.md +++ b/docs/admin/dns.md @@ -124,7 +124,7 @@ With v1.3, the following annotations are deprecated: `pod.beta.kubernetes.io/hos ## How do I test if it is working? -### Create a simple Pod to use as a test environment. +### Create a simple Pod to use as a test environment Create a file named busybox.yaml with the following contents: @@ -152,7 +152,7 @@ Then create a pod using this file: kubectl create -f busybox.yaml ``` -### Wait for this pod to go into the running state. +### Wait for this pod to go into the running state You can get its status with: ``` @@ -165,7 +165,7 @@ NAME READY STATUS RESTARTS AGE busybox 1/1 Running 0 ``` -### Validate DNS works +### Validate that DNS is working Once that pod is running, you can exec nslookup in that environment: @@ -185,6 +185,115 @@ Address 1: 10.0.0.1 If you see that, DNS is working correctly. +### Troubleshooting Tips + +If the nslookup command fails, check the following: + +#### Check the local DNS configuration first +Take a look inside the resolv.conf file. (See "Inheriting DNS from the node" and "Known issues" below for more information) + +``` +cat /etc/resolv.conf +``` + +Verify that the search path and name server are set up like the following (note that seach path may vary for different cloud providers): + +``` +search default.svc.cluster.local svc.cluster.local cluster.local google.internal c.gce_project_id.internal +nameserver 10.0.0.10 +options ndots:5 +``` + +#### Quick diagnosis + +Errors such as the following indicate a problem with the kube-dns add-on or associated Services: + +``` +$ kubectl exec busybox -- nslookup kubernetes.default +Server: 10.0.0.10 +Address 1: 10.0.0.10 + +nslookup: can't resolve 'kubernetes.default' +``` + +or + +``` +$ kubectl exec busybox -- nslookup kubernetes.default +Server: 10.0.0.10 +Address 1: 10.0.0.10 kube-dns.kube-system.svc.cluster.local + +nslookup: can't resolve 'kubernetes.default' +``` + +#### Check if the DNS pod is running + +Use the kubectl get pods command to verify that the DNS pod is running. + +``` +kubectl get pods --namespace=kube-system -l k8s-app=kube-dns +``` + +You should see something like: + +``` +NAME READY STATUS RESTARTS AGE +... +kube-dns-v19-ezo1y 3/3 Running 0 1h +... +``` + +If you see that no pod is running or that the pod has failed/completed, the dns add-on may not be deployed by default in your current environment and you will have to deploy it manually. + +#### Check for Errors in the DNS pod + +Use `kubectl logs` command to see logs for the DNS daemons. + +``` +kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c kubedns +kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c dnsmasq +kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c healthz +``` + +See if there is any suspicious log. W, E, F letter at the beginning represent Warning, Error and Failure. Please search for entries that have these as the logging level and use [kubernetes issues](https://github.com/kubernetes/kubernetes/issues) to report unexpected errors. + +#### Is dns service up? + +Verify that the DNS service is up by using the `kubectl get service` command. + +``` +kubectl get svc --namespace=kube-system +``` + +You should see: + +``` +NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE +... +kube-dns 10.0.0.10 53/UDP,53/TCP 1h +... +``` + +If you have created the service or in the case it should be created by default but it does not appear, see this [debugging services page](http://kubernetes.io/docs/user-guide/debugging-services/) for more information. + +#### Are dns endpoints exposed? + +You can verify that dns endpoints are exposed by using the `kubectl get endpoints` command. + +``` +kubectl get ep kube-dns --namespace=kube-system +``` + +You should see something like: +``` +NAME ENDPOINTS AGE +kube-dns 10.180.3.17:53,10.180.3.17:53 1h +``` + +If you do not see the endpoints, see endpoints section in the [debugging services documentation](http://kubernetes.io/docs/user-guide/debugging-services/). + +For additional Kubernetes DNS examples, see the [cluster-dns examples](https://github.com/kubernetes/kubernetes/tree/master/examples/cluster-dns) in the Kubernetes GitHub repository. + ## Kubernetes Federation (Multiple Zone support) Release 1.3 introduced Cluster Federation support for multi-site @@ -213,6 +322,34 @@ the flag `--cluster-domain=` The Kubernetes cluster DNS server (based off the [SkyDNS](https://github.com/skynetservices/skydns) library) supports forward lookups (A records), service lookups (SRV records) and reverse IP address lookups (PTR records). +## Inheriting DNS from the node +When running a pod, kubelet will prepend the cluster DNS server and search +paths to the node's own DNS settings. If the node is able to resolve DNS names +specific to the larger environment, pods should be able to, also. See "Known +issues" below for a caveat. + +If you don't want this, or if you want a different DNS config for pods, you can +use the kubelet's `--resolv-conf` flag. Setting it to "" means that pods will +not inherit DNS. Setting it to a valid file path means that kubelet will use +this file instead of `/etc/resolv.conf` for DNS inheritance. + +## Known issues +Kubernetes installs do not configure the nodes' resolv.conf files to use the +cluster DNS by default, because that process is inherently distro-specific. +This should probably be implemented eventually. + +Linux's libc is impossibly stuck ([see this bug from +2005](https://bugzilla.redhat.com/show_bug.cgi?id=168253)) with limits of just +3 DNS `nameserver` records and 6 DNS `search` records. Kubernetes needs to +consume 1 `nameserver` record and 3 `search` records. This means that if a +local installation already uses 3 `nameserver`s or uses more than 3 `search`es, +some of those settings will be lost. As a partial workaround, the node can run +`dnsmasq` which will provide more `nameserver` entries, but not more `search` +entries. You can also use kubelet's `--resolv-conf` flag. + +If you are using Alpine version 3.3 or earlier as your base image, dns may not +work properly owing to a known issue with Alpine. Check [here](https://github.com/kubernetes/kubernetes/issues/30215) +for more information. ## References From f83b95234a70fc8bfec6697c0267b1120203e2be Mon Sep 17 00:00:00 2001 From: Dominic001 Date: Fri, 28 Oct 2016 22:49:17 +0200 Subject: [PATCH 16/87] more explicit etcdctl description assuming that kubernetes is almost always used with docker. In case other container solutions as lxc are integrated this section will need further revision. Just using the bare etcl command is a bit too less, at least there should be a hint that you log in into the container. --- docs/admin/high-availability/index.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/admin/high-availability/index.md b/docs/admin/high-availability/index.md index efe6ddac4b..62da7f7eb9 100644 --- a/docs/admin/high-availability/index.md +++ b/docs/admin/high-availability/index.md @@ -100,16 +100,16 @@ for `${NODE_IP}` on each machine. #### Validating your cluster -Once you copy this into all three nodes, you should have a clustered etcd set up. You can validate with +Once you copy this into all three nodes, you should have a clustered etcd set up. In case you run your pods on docker container you can validate with ```shell -etcdctl member list +docker exec -i -t < container_id > etcdctl member list ``` and ```shell -etcdctl cluster-health +docker exec -i -t < container_id > etcdctl cluster-health ``` You can also validate that this is working with `etcdctl set foo bar` on one node, and `etcdctl get foo` From f6f0da2f9af5b93e0e31bb286cc9705945df3062 Mon Sep 17 00:00:00 2001 From: Dominic001 Date: Sat, 29 Oct 2016 01:48:34 +0200 Subject: [PATCH 17/87] more generic patch use kubectl exec command so we dont have to worry about container type --- docs/admin/high-availability/index.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/docs/admin/high-availability/index.md b/docs/admin/high-availability/index.md index 62da7f7eb9..ad78270e4a 100644 --- a/docs/admin/high-availability/index.md +++ b/docs/admin/high-availability/index.md @@ -100,16 +100,15 @@ for `${NODE_IP}` on each machine. #### Validating your cluster -Once you copy this into all three nodes, you should have a clustered etcd set up. In case you run your pods on docker container you can validate with - +Once you copy this into all three nodes, you should have a clustered etcd set up. You can validate on master with ```shell -docker exec -i -t < container_id > etcdctl member list +kubectl exec < pod_name > etcdctl member list ``` and ```shell -docker exec -i -t < container_id > etcdctl cluster-health +kubectl exec < pod_name > etcdctl cluster-health ``` You can also validate that this is working with `etcdctl set foo bar` on one node, and `etcdctl get foo` From 68f8244507d3d2de51ac10a8e208df72e6d0330a Mon Sep 17 00:00:00 2001 From: Cao Shufeng Date: Tue, 1 Nov 2016 05:50:43 -0400 Subject: [PATCH 18/87] Add details about loopback cni plugin This change provide a href to users, so that users knowns where to download this loopback cin plugin. And a more accurate link about cni configuration file is provided. --- docs/admin/network-plugins.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/admin/network-plugins.md b/docs/admin/network-plugins.md index 8cfeb658c9..b89ddc88f7 100644 --- a/docs/admin/network-plugins.md +++ b/docs/admin/network-plugins.md @@ -36,11 +36,11 @@ Place plugins in `network-plugin-dir/plugin-name/plugin-name`, i.e if you have a ### CNI -The CNI plugin is selected by passing Kubelet the `--network-plugin=cni` command-line option. Kubelet reads a file from `--cni-conf-dir` (default `/etc/cni/net.d`) and uses the CNI configuration from that file to set up each pod's network. The CNI configuration file must match the [CNI specification](https://github.com/containernetworking/cni/blob/master/SPEC.md), and any required CNI plugins referenced by the configuration must be present in `--cni-bin-dir` (default `/opt/cni/bin`). +The CNI plugin is selected by passing Kubelet the `--network-plugin=cni` command-line option. Kubelet reads a file from `--cni-conf-dir` (default `/etc/cni/net.d`) and uses the CNI configuration from that file to set up each pod's network. The CNI configuration file must match the [CNI specification](https://github.com/containernetworking/cni/blob/master/SPEC.md#network-configuration), and any required CNI plugins referenced by the configuration must be present in `--cni-bin-dir` (default `/opt/cni/bin`). If there are multiple CNI configuration files in the directory, the first one in lexicographic order of file name is used. -In addition to the CNI plugin specified by the configuration file, Kubernetes requires the standard CNI `lo` plugin, at minimum version 0.2.0 +In addition to the CNI plugin specified by the configuration file, Kubernetes requires the standard CNI [`lo`](https://github.com/containernetworking/cni/blob/master/plugins/main/loopback/loopback.go) plugin, at minimum version 0.2.0 ### kubenet @@ -72,4 +72,4 @@ This option is provided to the network-plugin; currently **only kubenet supports * `--network-plugin=exec` specifies that we use the `exec` plugin, with executables located in `--network-plugin-dir`. * `--network-plugin=cni` specifies that we use the `cni` network plugin with actual CNI plugin binaries located in `--cni-bin-dir` (default `/opt/cni/bin`) and CNI plugin configuration located in `--cni-conf-dir` (default `/etc/cni/net.d`). * `--network-plugin=kubenet` specifies that we use the `kubenet` network plugin with CNI `bridge` and `host-local` plugins placed in `/opt/cni/bin` or `network-plugin-dir`. -* `--network-plugin-mtu=9001` specifies the MTU to use, currently only used by the `kubenet` network plugin. \ No newline at end of file +* `--network-plugin-mtu=9001` specifies the MTU to use, currently only used by the `kubenet` network plugin. From c01dcf1a946ba78eb233c1e59f102cfc093f0df7 Mon Sep 17 00:00:00 2001 From: deads2k Date: Tue, 1 Nov 2016 08:42:59 -0400 Subject: [PATCH 19/87] document authenticating proxy: --- docs/admin/authentication.md | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/docs/admin/authentication.md b/docs/admin/authentication.md index 6819677107..6e13c28929 100644 --- a/docs/admin/authentication.md +++ b/docs/admin/authentication.md @@ -33,7 +33,7 @@ or be treated as an anonymous user. ## Authentication strategies -Kubernetes uses client certificates, bearer tokens, or HTTP basic auth to +Kubernetes uses client certificates, bearer tokens, an authenticating proxy, or HTTP basic auth to authenticate API requests through authentication plugins. As HTTP request are made to the API server plugins attempts to associate the following attributes with the request: @@ -360,6 +360,20 @@ An unsuccessful request would return: HTTP status codes can be used to supply additional error context. + +### Authenticating Proxy + +The API server can be configured to identify users from request header values, such as `X-Remote-User`. +It is designed for use in combination with an authenticating proxy, which sets the request header value. +In order to prevent header spoofing, the authenticating proxy is required to present a valid client +certificate to the API server for validation against the specified CA before the request headers are +checked. + +* `--requestheader-username-headers` Required, case-insensitive. Header names to check, in order, for the user identity. The first header containing a value is used as the identity. +* `--requestheader-client-ca-file` Required. PEM-encoded certificate bundle. A valid client certificate must be presented and validated against the certificate authorities in the specified file before the request headers are checked for user names. +* `--requestheader-allowed-names` Optional. List of common names (cn). If set, a valid client certificate with a Common Name (cn) in the specified list must be presented before the request headers are checked for user names. If empty, any Common Name is allowed. + + ### Keystone Password Keystone authentication is enabled by passing the `--experimental-keystone-url=` From 07c0d9298491cbb7de9a15f838180fac5b1fb3e5 Mon Sep 17 00:00:00 2001 From: Matt Baldwin Date: Tue, 1 Nov 2016 15:55:23 -0700 Subject: [PATCH 20/87] Added StackPointCloud as a Technology and Services Partner Change-Id: I04c32a60e447f918aba2d9a27741938a4dcfb434 --- _includes/partner-script.js | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/_includes/partner-script.js b/_includes/partner-script.js index 00dc9f1ee1..3f45a47b4b 100644 --- a/_includes/partner-script.js +++ b/_includes/partner-script.js @@ -14,6 +14,13 @@ link: 'https://deis.com', blurb: 'Deis the creators of Helm, Workflow, and Steward, helps developers and operators build, deploy, manage and scale their applications on top of Kubernetes.' }, + { + type: 0, + name: 'StackPointCloud', + logo: 'stackpointio', + link: 'https://stackpoint.io', + blurb: 'StackPointCloud builds Stackpoint.io, the universal control plane for Kubernetes Anywhere -- compose and build your own infrastructure as easily as a DigitalOcean droplet at any public cloud provider.' + }, { type: 0, name: 'Sysdig Cloud', @@ -168,6 +175,13 @@ link: 'https://deis.com/services/', blurb: 'Deis provides professional services and 24x7 operational support for any Kubernetes cluster managed by our global cluster operations team.' }, + { + type: 1, + name: 'StackPointCloud', + logo: 'stackpoint', + link: 'https://stackpoint.io', + blurb: 'StackPointCloud offers a wide range of support plans for managed Kubernetes clusters built through its universal control plane for Kubernetes Anywhere.' + }, { type: 1, name: 'Samsung SDS', From 0a8408081c8a2805355227994efd16427d1761d4 Mon Sep 17 00:00:00 2001 From: Magnus Kulke Date: Thu, 3 Nov 2016 20:28:51 +0100 Subject: [PATCH 21/87] Updated the NodeHostname entry. --- docs/admin/node.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/admin/node.md b/docs/admin/node.md index e7dbbba987..08b5d92f6e 100644 --- a/docs/admin/node.md +++ b/docs/admin/node.md @@ -27,7 +27,7 @@ pieces of information: The usage of these fields varies depending on your cloud provider or bare metal configuration. -* HostName: Generally not used +* HostName: The hostname as reported by the node's kernel. Can be overriden via the kubelet `--hostname-override` parameter. * ExternalIP: Generally the IP address of the node that is externally routable (available from outside the cluster) From 47b59475cc70d25edb8b2f68d021f18e074f32f5 Mon Sep 17 00:00:00 2001 From: Fan Lin Date: Fri, 4 Nov 2016 09:03:48 +0800 Subject: [PATCH 22/87] Add limitation note of using host-port --- docs/admin/network-plugins.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/admin/network-plugins.md b/docs/admin/network-plugins.md index 8cfeb658c9..976e85bd1e 100644 --- a/docs/admin/network-plugins.md +++ b/docs/admin/network-plugins.md @@ -42,6 +42,8 @@ If there are multiple CNI configuration files in the directory, the first one in In addition to the CNI plugin specified by the configuration file, Kubernetes requires the standard CNI `lo` plugin, at minimum version 0.2.0 +Limitation: Due to [#31307](https://github.com/kubernetes/kubernetes/issues/31307), `HostPort` won't work with CNI networking plugin at the moment. That means all `hostPort` attribute in pod would be simply ignored. + ### kubenet The Linux-only kubenet plugin provides functionality similar to the `--configure-cbr0` kubelet command-line option. It creates a Linux bridge named `cbr0` and creates a veth pair for each pod with the host end of each pair connected to `cbr0`. The pod end of the pair is assigned an IP address allocated from a range assigned to the node either through configuration or by the controller-manager. `cbr0` is assigned an MTU matching the smallest MTU of an enabled normal interface on the host. The kubenet plugin is currently mutually exclusive with, and will eventually replace, the --configure-cbr0 option. It is also currently incompatible with the flannel experimental overlay. @@ -72,4 +74,4 @@ This option is provided to the network-plugin; currently **only kubenet supports * `--network-plugin=exec` specifies that we use the `exec` plugin, with executables located in `--network-plugin-dir`. * `--network-plugin=cni` specifies that we use the `cni` network plugin with actual CNI plugin binaries located in `--cni-bin-dir` (default `/opt/cni/bin`) and CNI plugin configuration located in `--cni-conf-dir` (default `/etc/cni/net.d`). * `--network-plugin=kubenet` specifies that we use the `kubenet` network plugin with CNI `bridge` and `host-local` plugins placed in `/opt/cni/bin` or `network-plugin-dir`. -* `--network-plugin-mtu=9001` specifies the MTU to use, currently only used by the `kubenet` network plugin. \ No newline at end of file +* `--network-plugin-mtu=9001` specifies the MTU to use, currently only used by the `kubenet` network plugin. From 38385006efd9954885e2c853ae6d04ed0cfddafa Mon Sep 17 00:00:00 2001 From: Magnus Kulke Date: Fri, 4 Nov 2016 13:56:13 +0100 Subject: [PATCH 23/87] Fixed spelling --- docs/admin/node.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/admin/node.md b/docs/admin/node.md index 08b5d92f6e..78ef3c4b2e 100644 --- a/docs/admin/node.md +++ b/docs/admin/node.md @@ -27,7 +27,7 @@ pieces of information: The usage of these fields varies depending on your cloud provider or bare metal configuration. -* HostName: The hostname as reported by the node's kernel. Can be overriden via the kubelet `--hostname-override` parameter. +* HostName: The hostname as reported by the node's kernel. Can be overridden via the kubelet `--hostname-override` parameter. * ExternalIP: Generally the IP address of the node that is externally routable (available from outside the cluster) From a516182f80e9d9df9df9d4a17b428f683b258791 Mon Sep 17 00:00:00 2001 From: Daniel Sachse Date: Fri, 4 Nov 2016 14:44:58 +0100 Subject: [PATCH 24/87] Updated documentation to use beta version of init-containers --- docs/user-guide/nginx-init-containers.yaml | 2 +- docs/user-guide/petset/bootstrapping/petset_peers.yaml | 2 +- docs/user-guide/petset/bootstrapping/petset_vm.yaml | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/user-guide/nginx-init-containers.yaml b/docs/user-guide/nginx-init-containers.yaml index 34c20fa66a..24124c7459 100644 --- a/docs/user-guide/nginx-init-containers.yaml +++ b/docs/user-guide/nginx-init-containers.yaml @@ -3,7 +3,7 @@ kind: Pod metadata: name: nginx annotations: - pod.alpha.kubernetes.io/init-containers: '[ + pod.beta.kubernetes.io/init-containers: '[ { "name": "install", "image": "busybox", diff --git a/docs/user-guide/petset/bootstrapping/petset_peers.yaml b/docs/user-guide/petset/bootstrapping/petset_peers.yaml index f8393b5c2c..4f992ead71 100644 --- a/docs/user-guide/petset/bootstrapping/petset_peers.yaml +++ b/docs/user-guide/petset/bootstrapping/petset_peers.yaml @@ -29,7 +29,7 @@ spec: app: nginx annotations: pod.alpha.kubernetes.io/initialized: "true" - pod.alpha.kubernetes.io/init-containers: '[ + pod.beta.kubernetes.io/init-containers: '[ { "name": "peerfinder", "image": "gcr.io/google_containers/peer-finder:0.1", diff --git a/docs/user-guide/petset/bootstrapping/petset_vm.yaml b/docs/user-guide/petset/bootstrapping/petset_vm.yaml index c506a1bac6..a3da5211a3 100644 --- a/docs/user-guide/petset/bootstrapping/petset_vm.yaml +++ b/docs/user-guide/petset/bootstrapping/petset_vm.yaml @@ -27,7 +27,7 @@ spec: app: ub annotations: pod.alpha.kubernetes.io/initialized: "true" - pod.alpha.kubernetes.io/init-containers: '[ + pod.beta.kubernetes.io/init-containers: '[ { "name": "rootfs", "image": "ubuntu:15.10", From bbceda0ba2459fe6f5e02e54642b9d0e94781327 Mon Sep 17 00:00:00 2001 From: blankenhaus Date: Sat, 5 Nov 2016 12:29:50 -0700 Subject: [PATCH 25/87] Layer 3 <=> layer 4 TCP/UDP are layer 4. --- docs/user-guide/services/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/services/index.md b/docs/user-guide/services/index.md index 94faabcd1c..fc7e339dbc 100644 --- a/docs/user-guide/services/index.md +++ b/docs/user-guide/services/index.md @@ -182,7 +182,7 @@ In Kubernetes v1.0 the proxy was purely in userspace. In Kubernetes v1.1 an iptables proxy was added, but was not the default operating mode. Since Kubernetes v1.2, the iptables proxy is the default. -As of Kubernetes v1.0, `Services` are a "layer 3" (TCP/UDP over IP) construct. +As of Kubernetes v1.0, `Services` are a "layer 4" (TCP/UDP over IP) construct. In Kubernetes v1.1 the `Ingress` API was added (beta) to represent "layer 7" (HTTP) services. From 3d7e0b128c67d30ea1f72f043a4ff69bcd4c6fd5 Mon Sep 17 00:00:00 2001 From: Tremaine Eto Date: Mon, 7 Nov 2016 10:16:15 -0800 Subject: [PATCH 26/87] Fixed typo Removed extra "e" in "specify" in the "Secret files permission" section. --- docs/user-guide/secrets/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/secrets/index.md b/docs/user-guide/secrets/index.md index f97c6185b8..c55868f1f1 100644 --- a/docs/user-guide/secrets/index.md +++ b/docs/user-guide/secrets/index.md @@ -265,7 +265,7 @@ All listed keys must exist in the corresponding secret. Otherwise, the volume is **Secret files permissions** You can also specify the permission mode bits files part of a secret will have. -If you don't specify any, `0644` is used by default. You can sepecify a default +If you don't specify any, `0644` is used by default. You can specify a default mode for the whole secret volume and override per key if needed. For example, you can specify a default mode like this: From 215dca8e8ba6764369e7cd30b3cbd650576cb230 Mon Sep 17 00:00:00 2001 From: TRAVIS ALLEN SALAS COX Date: Mon, 7 Nov 2016 13:27:27 -0600 Subject: [PATCH 27/87] Update dns.md This should at least fix a broken link referenced in [#1640](https://github.com/kubernetes/kubernetes.github.io/issues/1640). Looks like a lot of information was taking out of [this](https://github.com/kubernetes/kubernetes/tree/master/build-tools/kube-dns) README.md recently, if you'd like i can delete the reference section completely if it is out of date and/or no longer needed. --- docs/admin/dns.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/admin/dns.md b/docs/admin/dns.md index cc132201aa..82b3bab6c0 100644 --- a/docs/admin/dns.md +++ b/docs/admin/dns.md @@ -216,5 +216,5 @@ supports forward lookups (A records), service lookups (SRV records) and reverse ## References -- [Docs for the DNS cluster addon](http://releases.k8s.io/{{page.githubbranch}}/build/kube-dns/README.md) +- [Docs for the DNS cluster addon](http://releases.k8s.io/{{page.githubbranch}}/build-tools/kube-dns/README.md) From 3ca34c8e0478f503ec5135c53ab23c6206a75a59 Mon Sep 17 00:00:00 2001 From: Steffen Gebert Date: Tue, 8 Nov 2016 11:23:46 +0100 Subject: [PATCH 28/87] configuring-containers: Fix syntax highlight looks weird to me without that.. --- docs/user-guide/configuring-containers.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/configuring-containers.md b/docs/user-guide/configuring-containers.md index 6b7b447289..99dc823375 100644 --- a/docs/user-guide/configuring-containers.md +++ b/docs/user-guide/configuring-containers.md @@ -12,7 +12,7 @@ assignees: In addition to the imperative-style commands, such as `kubectl run` and `kubectl expose`, described [elsewhere](/docs/user-guide/quick-start), Kubernetes supports declarative configuration. Oftentimes, configuration files are preferable to imperative commands, since they can be checked into version control and changes to the files can be code reviewed, which is especially important for more complex configurations, producing a more robust, reliable and archival system. -In the declarative style, all configuration is stored in YAML or JSON configuration files using Kubernetes's API resource schemas as the configuration schemas. `kubectl` can create, update, delete, and get API resources. The `apiVersion` (currently 'v1'?), resource `kind`, and resource `name` are used by `kubectl` to construct the appropriate API path to invoke for the specified operation. +In the declarative style, all configuration is stored in YAML or JSON configuration files using Kubernetes's API resource schemas as the configuration schemas. `kubectl` can create, update, delete, and get API resources. The `apiVersion` (currently `v1`?), resource `kind`, and resource `name` are used by `kubectl` to construct the appropriate API path to invoke for the specified operation. ## Launching a container using a configuration file From a3f1880405bc64d1b717b4189b5ce26fdafb2a59 Mon Sep 17 00:00:00 2001 From: Jared Date: Tue, 8 Nov 2016 14:49:02 -0800 Subject: [PATCH 29/87] creating contributing.md Adding a contributing.md file with contributing guidelines for docs on kubernetes.io (https://github.com/blog/1184-contributing-guidelines). This should not be merged until the PR for the docs style guide is merged. (https://github.com/kubernetes/kubernetes.github.io/pull/1619) --- CONTRIBUTING.md | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100644 CONTRIBUTING.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000000..9dd8149a15 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,36 @@ +# Contributing to Kubernetes Documentation + +**First off, thanks for taking the time to contribute!** + +The following is a set of guidelines for contributing to Kubernetes documentation, hosted at [Kubernetes.io](http://kubernetes.io/). +These are just guidelines, not rules. Use your best judgment, and feel free to propose changes to this document in a pull request. + +## Before you get started + +### Code of Conduct + +Kubernetes follows the [Cloud Native Computing Foundation (CNCF) Code of Conduct](https://github.com/cncf/foundation/blob/master/code-of-conduct.md). By participating, you are expected to uphold this code. Please report unacceptable behavior to Sarah Novotny [sarahnovotny@google.com](mailto:sarahnovotny@google.com) and/or Dan Kohn [dan@linuxfoundation.org](mailto:dan@linuxfoundation.org). + +### Documentation and Site Decisions + +The [Kubernetes SIG Docs Discussion Group](https://groups.google.com/forum/#!forum/kubernetes-sig-docs) is the discussion group for doc releases, suggested site improvements, and improving the doc contribution experience. If you are planning to be a regular contributor, join this group to stay informed and involved. + +### Style Guides and Templates + +Before submitting a pull request to create new content, please review the [Kubernetes.io style guide](http://kubernetes.io/docs/contribute/style-guide/) and follow the [instructions for using page templates](http://kubernetes.io/docs/contribute/page-templates/). + + +## Contributing to Documentation + +### Reporting Documentation Issues + +Kubernetes.io uses github issues to track documentation issues and requests. If you see a documentation issue, submit an issue using the following steps: + +1. Check the [kubernetes.io issues list](https://github.com/kubernetes/kubernetes.github.io/issues) as you might find out the issue is a duplicate. +2. Use the [included template for every new issue](https://github.com/kubernetes/kubernetes.github.io/issues/new). When you create a bug report, include as many details as possible and include suggested fixes to the issue. + +Note that code issues should be filed against the main kubernetes repository, while documentation issues should go in the kubernetes.io repository. + +### Submitting Documentation Pull Requests + +If you’re fixing an issue in the existing documentation, you should submit a PR against the master branch. Follow [these instructions to create a documentation pull request against the kubernetes.io repository](http://kubernetes.io/docs/contribute/create-pull-request/). From 0bd5651d4f529f7bc707ce3de6ea4a475ed72199 Mon Sep 17 00:00:00 2001 From: David Kirstein Date: Thu, 10 Nov 2016 08:19:22 +0100 Subject: [PATCH 30/87] Fix typo in working-with-resources.md mode -> more --- docs/user-guide/working-with-resources.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/working-with-resources.md b/docs/user-guide/working-with-resources.md index 5b300ee6bd..b5b0f8f862 100644 --- a/docs/user-guide/working-with-resources.md +++ b/docs/user-guide/working-with-resources.md @@ -46,7 +46,7 @@ The system adds fields in several ways: - Some fields are added synchronously with creation of the resource and some are set asynchronously. - For example: `metadata.uid` is set synchronously. (Read more about [metadata](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/devel/api-conventions.md#metadata)). - - For example, `status.hostIP` is set only after the pod has been scheduled. This often happens fast, but you may notice pods which do not have this set yet. This is called Late Initialization. (Read mode about [status](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/devel/api-conventions.md#spec-and-status) and [late initialization](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/devel/api-conventions.md#late-initialization) ). + - For example, `status.hostIP` is set only after the pod has been scheduled. This often happens fast, but you may notice pods which do not have this set yet. This is called Late Initialization. (Read more about [status](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/devel/api-conventions.md#spec-and-status) and [late initialization](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/devel/api-conventions.md#late-initialization) ). - Some fields are set to default values. Some defaults vary by cluster and some are fixed for the API at a certain version. (Read more about [defaulting](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/devel/api-conventions.md#defaulting)). - For example, `spec.containers[0].imagePullPolicy` always defaults to `IfNotPresent` in api v1. - For example, `spec.containers[0].resources.limits.cpu` may be defaulted to `100m` on some clusters, to some other value on others, and not defaulted at all on others. From e745b0480157ae4ce17b534b107d21d7552da17c Mon Sep 17 00:00:00 2001 From: David Kirstein Date: Thu, 10 Nov 2016 08:27:09 +0100 Subject: [PATCH 31/87] remove extra space in working-with-resources.md --- docs/user-guide/working-with-resources.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/working-with-resources.md b/docs/user-guide/working-with-resources.md index b5b0f8f862..d2aeeb621e 100644 --- a/docs/user-guide/working-with-resources.md +++ b/docs/user-guide/working-with-resources.md @@ -46,7 +46,7 @@ The system adds fields in several ways: - Some fields are added synchronously with creation of the resource and some are set asynchronously. - For example: `metadata.uid` is set synchronously. (Read more about [metadata](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/devel/api-conventions.md#metadata)). - - For example, `status.hostIP` is set only after the pod has been scheduled. This often happens fast, but you may notice pods which do not have this set yet. This is called Late Initialization. (Read more about [status](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/devel/api-conventions.md#spec-and-status) and [late initialization](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/devel/api-conventions.md#late-initialization) ). + - For example, `status.hostIP` is set only after the pod has been scheduled. This often happens fast, but you may notice pods which do not have this set yet. This is called Late Initialization. (Read more about [status](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/devel/api-conventions.md#spec-and-status) and [late initialization](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/devel/api-conventions.md#late-initialization)). - Some fields are set to default values. Some defaults vary by cluster and some are fixed for the API at a certain version. (Read more about [defaulting](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/devel/api-conventions.md#defaulting)). - For example, `spec.containers[0].imagePullPolicy` always defaults to `IfNotPresent` in api v1. - For example, `spec.containers[0].resources.limits.cpu` may be defaulted to `100m` on some clusters, to some other value on others, and not defaulted at all on others. From 5dd3a00742b273b5a2010bb2aa522955a5f0d2a8 Mon Sep 17 00:00:00 2001 From: Luke Marsden Date: Thu, 27 Oct 2016 17:01:13 +0100 Subject: [PATCH 32/87] Be more vendor neutral. Address review feedback. --- docs/getting-started-guides/kubeadm.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting-started-guides/kubeadm.md b/docs/getting-started-guides/kubeadm.md index 08148bd7c3..8e82d52423 100644 --- a/docs/getting-started-guides/kubeadm.md +++ b/docs/getting-started-guides/kubeadm.md @@ -110,7 +110,7 @@ To initialize the master, pick one of the machines you previously installed `kub **Note:** this will autodetect the network interface to advertise the master on as the interface with the default gateway. If you want to use a different interface, specify `--api-advertise-addresses=` argument to `kubeadm init`. -If you want to use [flannel](https://github.com/coreos/flannel) as the pod network; specify `--pod-network-cidr=10.244.0.0/16` if you're using the daemonset manifest below. _However, please note that this is not required for any other networks, including Weave, which is the recommended pod network._ +If you want to use [flannel](https://github.com/coreos/flannel) as the pod network, specify `--pod-network-cidr=10.244.0.0/16` if you're using the daemonset manifest below. _However, please note that this is not required for any other networks besides Flannel._ Please refer to the [kubeadm reference doc](/docs/admin/kubeadm/) if you want to read more about the flags `kubeadm init` provides. From 337c6c054694ce3a75b3607a7a6a220dd3b0c230 Mon Sep 17 00:00:00 2001 From: Brandon DuRette Date: Fri, 11 Nov 2016 11:20:48 -0600 Subject: [PATCH 33/87] Typo fix --- docs/user-guide/thirdpartyresources.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/thirdpartyresources.md b/docs/user-guide/thirdpartyresources.md index d8f2bc5ba9..a1035d44c4 100644 --- a/docs/user-guide/thirdpartyresources.md +++ b/docs/user-guide/thirdpartyresources.md @@ -22,7 +22,7 @@ Each `ThirdPartyResource` has the following: * `description` - A free text description of the resource. * `versions` - A list of the versions of the resource. -The `kind` for a `ThirdPartyResource` takes the form `.`. You are expected to provide a unique kind and domain name in order to avoid conflicts with other `ThirdPartyResource` objects. Kind names will be converted to CamelCase when creating instances of the `ThirdPartyResource`. Hypens in the `kind` are assumed to be word breaks. For instance the kind `camel-case` would be converted to `CamelCase` but `camelcase` would be converted to `Camelcase`. +The `kind` for a `ThirdPartyResource` takes the form `.`. You are expected to provide a unique kind and domain name in order to avoid conflicts with other `ThirdPartyResource` objects. Kind names will be converted to CamelCase when creating instances of the `ThirdPartyResource`. Hyphens in the `kind` are assumed to be word breaks. For instance the kind `camel-case` would be converted to `CamelCase` but `camelcase` would be converted to `Camelcase`. Other fields on the `ThirdPartyResource` are treated as custom data fields. These fields can hold arbitrary JSON data and have any structure. From fbba8e49019ad61a809b5a3df781201cf13f631c Mon Sep 17 00:00:00 2001 From: Charlie Drage Date: Mon, 14 Nov 2016 10:54:34 -0500 Subject: [PATCH 34/87] Remove "stars" ratings from each page. This removes the stars rating that appears on the bottom of each page. Doing so also required the update the "a.issue" css that correctly aligns the "Create Issue" button and "Edit this Page" button on the bottom of the page. --- _layouts/docwithnav.html | 2 +- _sass/_base.sass | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/_layouts/docwithnav.html b/_layouts/docwithnav.html index 877ccc4e63..2153d358e2 100755 --- a/_layouts/docwithnav.html +++ b/_layouts/docwithnav.html @@ -41,7 +41,7 @@ {% if notitle != "true" %}

{{ title }}

{% endif %} {{ content }}

Analytics - {% if page.url != "/404.html" and page.url != "/docs/search/" %}

+ {% if page.url != "/404.html" and page.url != "/docs/search/" %} Create Issue - Edit This Page + window.location.pathname)" class="button issue">Create an Issue + Edit this Page {% endif %} From e3c87235a548a6a698f6dc73a40485f691fefd78 Mon Sep 17 00:00:00 2001 From: Janet Kuo Date: Mon, 14 Nov 2016 10:08:00 -0800 Subject: [PATCH 36/87] Fix example test failure: rename ScheduledJob to CronJob --- test/examples_test.go | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/test/examples_test.go b/test/examples_test.go index 1853cdaf0a..7e5660c4f9 100644 --- a/test/examples_test.go +++ b/test/examples_test.go @@ -127,11 +127,11 @@ func validateObject(obj runtime.Object) (errors field.ErrorList) { t.Namespace = api.NamespaceDefault } errors = expvalidation.ValidateDaemonSet(t) - case *batch.ScheduledJob: + case *batch.CronJob: if t.Namespace == "" { t.Namespace = api.NamespaceDefault } - errors = batch_validation.ValidateScheduledJob(t) + errors = batch_validation.ValidateCronJob(t) default: errors = field.ErrorList{} errors = append(errors, field.InternalError(field.NewPath(""), fmt.Errorf("no validation defined for %#v", obj))) @@ -242,7 +242,7 @@ func TestExampleObjectSchemas(t *testing.T) { "redis-resource-deployment": &extensions.Deployment{}, "redis-secret-deployment": &extensions.Deployment{}, "run-my-nginx": &extensions.Deployment{}, - "sj": &batch.ScheduledJob{}, + "sj": &batch.CronJob{}, }, "../docs/admin": { "daemon": &extensions.DaemonSet{}, @@ -272,7 +272,7 @@ func TestExampleObjectSchemas(t *testing.T) { "../docs/user-guide/node-selection": { "pod": &api.Pod{}, "pod-with-node-affinity": &api.Pod{}, - "pod-with-pod-affinity": &api.Pod{}, + "pod-with-pod-affinity": &api.Pod{}, }, "../docs/admin/resourcequota": { "best-effort": &api.ResourceQuota{}, From d124edabc34b7c891e6664a37c67e762e0b7a2ff Mon Sep 17 00:00:00 2001 From: Matt Baldwin Date: Mon, 14 Nov 2016 10:34:30 -0800 Subject: [PATCH 37/87] - updated partner-script.js with the correct logo name. - added the correct logo. Change-Id: I160ecf13028977fa491c3047727b12c85b05eae0 --- _includes/partner-script.js | 2 +- images/square-logos/stackpoint.png | Bin 0 -> 5189 bytes 2 files changed, 1 insertion(+), 1 deletion(-) create mode 100644 images/square-logos/stackpoint.png diff --git a/_includes/partner-script.js b/_includes/partner-script.js index 3f45a47b4b..7322b0af45 100644 --- a/_includes/partner-script.js +++ b/_includes/partner-script.js @@ -17,7 +17,7 @@ { type: 0, name: 'StackPointCloud', - logo: 'stackpointio', + logo: 'stackpoint', link: 'https://stackpoint.io', blurb: 'StackPointCloud builds Stackpoint.io, the universal control plane for Kubernetes Anywhere -- compose and build your own infrastructure as easily as a DigitalOcean droplet at any public cloud provider.' }, diff --git a/images/square-logos/stackpoint.png b/images/square-logos/stackpoint.png new file mode 100644 index 0000000000000000000000000000000000000000..dd2822d493d58fbc4a3f31daf5658fd7c1d91e7c GIT binary patch literal 5189 zcmeI0=QA7t)4&f=g2Y9Qa%$8laYEG7IVDc-y+)6AdT&93LkJEDhagBUdW|4DQKCd7 z%87pBPCw=Ke0%?g=lQU^vokw8JG(Qxzu6RhJvZH8B`YzqyPZjg_fGKaR|{~5fzfb1T?U})G6F9@6VLr91pcNfI8ES z01)f7F$u3ZbRIU+hnpi_zjpQ3rfEdyDQv2r4XJ3~1D1SAg;oicmU0W&>oX5yi3z{M zeg9lb2Pl3H-p7uQw|*Y=zs8>-!~Y`j*VhyIf6m78@dd4ucUYz9O^uBkG!FG|sl+Im zn3xD>>GC;o38XP9-}?vzLjDRerYg+?^$nA1H5%ma@s?+C22uzJs+#dya`Kk@|IhjV zJ-D=5XJ7upn`dp<61@Mrb@JJv+Li2mu{Ne?yef^tVX7PJbkbxeqfFu#-~xCAn@uwY*72*sbOaf>5PdvdDCmIlV<@uLG;IxQ|tqSodS zo8RAw@66T`+U0D=hd$GHe!`sThZX#JP9HB$i6Ou?VReyVdY;@kElN(NCpn71jB^oP z-)RC=D>TaG@y`n4>3Rq3uU4MIf&zH^Dyy_u)}xj`3ze`Eo&C;}-|@y+V+up{z)sYH zCdUOIl8`bhrl=S_Lqa3}apWFt^tYGH``^NUr5EO;D8Vf;$bfO?sb9aW)k8keZQz0( zqS=mjkkVd?qh}Vs9__Cth5Sq@Q3yXqg~(sM2Voz>G%Fw4oo=?53+rGvS!=5Z-k$QI z24>9b9tDryV(iPfFmSmRs?y6OyxAluiEwL! zf8`or$HKDn1rG5GY9Z2NdosCYJs>+((3)Ro-W?U$ZaRb5tJ^QkQGrrcCG6hv6WV*6 zMOD>eS?sfwu`V`tGd-X?V|Ix1@btD-dMkcHv3v7NEJ+?BiehO(=3sT&vNir5;XsyS zfp1$aT_)4_QSCabkFpn|J>FKXjZ; ztNSO6?2SuIOw{A>6@HzMN(?P&NGDRlg?jx}&b3<=OY}0VFL^qV%!(0C;kgXoiq8*R z&~<1X8Z1j~aFV&alaPZ|pQW)wVU7!OjqGcjR|AWQvUYsvYHTH8_IS0i1}$og<9byU7$(CQMzU7li*z3(=R zMG!CVQ*CDU!TerEzoWmCdMk78cNqJuX|FIQE$5T@A!9M=-TSV*(y5K*aa+MQCBK_L zi=Bn9s?vwDoy&goz4QFzfx&r}L36wk({jRN-22A6`Oo=1L}K6QZW=Wi$_?Q-xBVI# zk_GnFu_5BB=GGofN%`@b6eEX*4dvzPWabuf5eQsj>{>~h8qb__Exl$H7vBVvO6pEa z$3WF?k@2z3Y4GxspS3V5k#T>;ybN#V24s2~c(_|)iR__zaCY(`56nR+C@y;(yxNNF z=5!ct%0SC&=U*k%#J!&q*~u6$$8StT7Y+u(pG%lWnqu6hE%kPEo8;WPpn}Z1!lK=X zuY;(%aElLep&UnO`HKD0)jHD3OQ70|Pr&Pl2+x_N?&#}#hfnOrZ>ec%rRY}Z=ID$zu#|_h@j7mHjYt|8xwOG!tNz+1}o!JzOb}aN&F?^U$nr z_wYc`_jgBDIJUJ?HoJ>D=PtXvbdoN3%YTP(YOchHJXK^(^32V2l!5Sp;bYNgUXSF1DA7f zCOHNS)n94piX@D`%kp(3#ROz7{|vP18%xM!MJy2e@#`AIa2A!jpi zyUYlSbi;93-(JrjE*{@^*5SZ}GZVA*6}wo6-PVFqyp@o4>wd;73Od6%G4GWRh1&dFJC8^9+8j7MZ+Tzb+!jAP9 z#$0lV!CuwNds6V7U)cLQ72Vb^hA5|vaUm@aZNd}z8;4bY2U#2GBzbNR=WHD6(*&{< zTDCsyJ!m<8`Ce05C`j=&L3VK>J;0a~!}Nk6lAv>MN{fyIC-pIQ%?N3#JwFk9>1w&0 z<{lKyI}#t|b*q-n(ohe2)o%BI88Jqsx$cK;-54$&Q1*&Ctq+K8u*W^??!E1XTLAM# z&P2d(gGvOKbl9YtxSDfv&j)NV;g)<2s1Nl^(O`S}qsN@e81e;YC3hM2e(})yimt9G znOhgJ_|4pi+cX2&AQ>AFTFlTnlB3~Uv;7(f-8tTjh@n?qtbECzbxGXzw^&Lp=BkjO zF0zo-#M+z#S;94ZeT<_Jdr&7T0#TUH>(Q<*$t&LErRK_hi4$|6S#KkyV!Vj_QEgHm zI=OQ@|9ckt?W1UzQ`wANz2ekYnClNB zikxW5&RFN!Pdoz|DGX#fs>%77eh!=jTzvQg)F(-zR$8~?1ysD~K`Fy|>5TEBO^<)j+6hvz&*!-(N*#L@nc6~y|vUj-Pb zSTMWG_sdM8obYOAuKo|VPg4Dqb~?_Jr!eeYWWaNagTy3%^h(GGMW;zcplt$+8aW0_`loUER3=oig@2s3U`Fmn5IP({Faj zC?Xa6G_MM^`?T>jkBf=nU#R*?#fY)tamCMJ1Ue}U*HO8x)HKxZY5Q=~C+e(!JqU(g z`H@!q!2+L>lbzpD9Cx9A(K{;V!z|;E;XB#Td()Sjkv$xN(GY{HYaFZN)<3%=hgsyN zUrZLw+vd6Rh^`Ab?p;vNuGM6E2M;FK&hLt!b`h}Q-?W@nhqhj({^4`@D<~MI0wKW* zOL>6uz0Nybi@}9VB25}|kQ1z0>H5gbim>U+)XVvai8R}foh=>Hml=*`FTSar#v}D~ zC&-ESDobIp3D?<=P5rNki9}E;kH zs$&U^h)U^a;`Y2XyV%dWlckZYQ=&mT*-`*AO-s!ujtW~JyrfR2BKPDYdgRq9-rF?#!J`ws(%OUA8^ZcZ& zV6a4~fHKtV8{)vx%$XAt3LN=K2VmZCaH`b_78EJ|@VY+fao)7yG|(+7V5)q2MNo=m z(1}K5^>sSX&tTU!`TFS;wnEeVO$zqT=>}{i*@!ucs67>U2RKHHryWKwT8g z^JdhbT@)*!%)UNdz-7b8Qm@!~1n31%JPX%2PM1S!4_aRn5?W52KmB}Yr8oJyfhY9* zRWpK5u6%@<+JeJl4?62BKj#|6(L&^U-iWBnGeTjyyT_sAoXh{Lya^L75+#+zVIGf; zaen0BNX*QTPSe*EYEkV!`nJX+=RN5adHn(+u3AkmH|Pa< z`0Ps$fK1zWA3A;Y=1P2!9o`Ayx~_Q0vAF+{L#Hi5r?WROIBGOBcX6c?8MxpxP#E~>C?dkt;R8gzZkCo|7FfUey|ti zR+Vr@pC*6Y$!5Zxuw-JJ&^d^q7bItwxYD~4uuo>PM|1Uh)}XDflACJBbPJ_xQyKcV zhXI4<_p#GUV;KVnw3D&aOpYh4Tj$f2uf}@2e2O*Y z37lw3bkQ(doLy|T*0%RW^cvy*n~s;eLe8$4qS>2!KAfw;5=#=Sp!^ynWotDCxfb+` zM|!3d_vC>pFD=>}|JEPNmg!Ei#Q-<2KwYVtSJZu|1N4CgIw~Zk%Z}Ja)Y_0~OQ69a zCmX)sB7{?L4}|i}#@{E*9&%zDXe$gk8iIs|8ii~OZ0+Z-&X+^3Ya{s>EK{;wWGzS$ zRaF+!To@ame-_Y?TDg=7%A~?aQJmQ`#=TVVDo-nzLOlPUDJH$YC^`Fj8`0gnnEEP( zTh+QSzGVv~XQah|jJ{9AyjH_vbQRB{ntM#WM?ce%Su5VF^ z2a9)p&KQ8p+rVbAT$Pf;-(b0Uc^oLD|FrYvTekCby;%d3cnt~)LHtYM_h@b)fLMDN zpk$Kmdh^j4`)9NVxS>cgKy|@mRV6L=q!JJE8;S%2^nak3iM6#QE{x>>H@Ha@Q_VMa z6IifP`IG2BxCt Date: Mon, 14 Nov 2016 15:02:42 -0800 Subject: [PATCH 38/87] Fix broken kubectl link --- docs/user-guide/production-pods.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/production-pods.md b/docs/user-guide/production-pods.md index 440ac4619a..240aec4e8f 100644 --- a/docs/user-guide/production-pods.md +++ b/docs/user-guide/production-pods.md @@ -218,7 +218,7 @@ The specification of a pre-stop hook is similar to that of probes, but without t ## Termination message -In order to achieve a reasonably high level of availability, especially for actively developed applications, it's important to debug failures quickly. Kubernetes can speed debugging by surfacing causes of fatal errors in a way that can be display using [`kubectl`](/docs/user-guide/kubectl/kubectl) or the [UI](/docs/user-guide/ui), in addition to general [log collection](/docs/user-guide/logging). It is possible to specify a `terminationMessagePath` where a container will write its 'death rattle'?, such as assertion failure messages, stack traces, exceptions, and so on. The default path is `/dev/termination-log`. +In order to achieve a reasonably high level of availability, especially for actively developed applications, it's important to debug failures quickly. Kubernetes can speed debugging by surfacing causes of fatal errors in a way that can be display using [`kubectl`](/docs/user-guide/kubectl/) or the [UI](/docs/user-guide/ui), in addition to general [log collection](/docs/user-guide/logging). It is possible to specify a `terminationMessagePath` where a container will write its 'death rattle'?, such as assertion failure messages, stack traces, exceptions, and so on. The default path is `/dev/termination-log`. Here is a toy example: From c4914efbf5f032f5b46357141051d35e2a70b9a9 Mon Sep 17 00:00:00 2001 From: Justin Santa Barbara Date: Tue, 15 Nov 2016 12:41:02 -0500 Subject: [PATCH 39/87] Remove unmaintained coreos multinode cluster Fix #1688 --- _data/guides.yml | 2 - docs/getting-started-guides/cloudstack.md | 2 +- .../coreos/coreos_multinode_cluster.md | 197 ------------------ docs/getting-started-guides/coreos/index.md | 6 - 4 files changed, 1 insertion(+), 206 deletions(-) delete mode 100644 docs/getting-started-guides/coreos/coreos_multinode_cluster.md diff --git a/_data/guides.yml b/_data/guides.yml index 40d47b08d6..83c9c46cd9 100644 --- a/_data/guides.yml +++ b/_data/guides.yml @@ -203,8 +203,6 @@ toc: path: /docs/getting-started-guides/ovirt/ - title: OpenStack Heat path: /docs/getting-started-guides/openstack-heat/ - - title: CoreOS on Multinode Cluster - path: /docs/getting-started-guides/coreos/coreos_multinode_cluster/ - title: rkt section: - title: Running Kubernetes with rkt diff --git a/docs/getting-started-guides/cloudstack.md b/docs/getting-started-guides/cloudstack.md index 2361a4fd79..ac79275007 100644 --- a/docs/getting-started-guides/cloudstack.md +++ b/docs/getting-started-guides/cloudstack.md @@ -10,7 +10,7 @@ CloudStack is a software to build public and private clouds based on hardware vi [CoreOS](http://coreos.com) templates for CloudStack are built [nightly](http://stable.release.core-os.net/amd64-usr/current/). CloudStack operators need to [register](http://docs.cloudstack.apache.org/projects/cloudstack-administration/en/latest/templates.html) this template in their cloud before proceeding with these Kubernetes deployment instructions. This guide uses an [Ansible playbook](https://github.com/runseb/ansible-kubernetes). -This is a completely automated, a single playbook deploys Kubernetes based on the coreOS [instructions](/docs/getting-started-guides/coreos/coreos_multinode_cluster). +This is completely automated, a single playbook deploys Kubernetes. This [Ansible](http://ansibleworks.com) playbook deploys Kubernetes on a CloudStack based Cloud using CoreOS images. The playbook, creates an ssh key pair, creates a security group and associated rules and finally starts coreOS instances configured via cloud-init. diff --git a/docs/getting-started-guides/coreos/coreos_multinode_cluster.md b/docs/getting-started-guides/coreos/coreos_multinode_cluster.md deleted file mode 100644 index fe1a164df6..0000000000 --- a/docs/getting-started-guides/coreos/coreos_multinode_cluster.md +++ /dev/null @@ -1,197 +0,0 @@ ---- -assignees: -- dchen1107 - ---- - -Use the [master.yaml](/docs/getting-started-guides/coreos/cloud-configs/master.yaml) and [node.yaml](/docs/getting-started-guides/coreos/cloud-configs/node.yaml) cloud-configs to provision a multi-node Kubernetes cluster. - -> **Attention**: This requires at least CoreOS version **[695.0.0][coreos695]**, which includes `etcd2`. - -[coreos695]: https://coreos.com/releases/#695.0.0 - -* TOC -{:toc} - -### AWS - -*Attention:* Replace `` below for a [suitable version of CoreOS image for AWS](https://coreos.com/docs/running-coreos/cloud-providers/ec2/). - -#### Provision the Master - -```shell -aws ec2 create-security-group --group-name kubernetes --description "Kubernetes Security Group" -aws ec2 authorize-security-group-ingress --group-name kubernetes --protocol tcp --port 22 --cidr 0.0.0.0/0 -aws ec2 authorize-security-group-ingress --group-name kubernetes --protocol tcp --port 80 --cidr 0.0.0.0/0 -aws ec2 authorize-security-group-ingress --group-name kubernetes --source-security-group-name kubernetes -``` - -```shell -aws ec2 run-instances \ ---image-id \ ---key-name \ ---region us-west-2 \ ---security-groups kubernetes \ ---instance-type m3.medium \ ---user-data file://master.yaml -``` - -#### Capture the private IP address - -```shell -aws ec2 describe-instances --instance-id -``` - -#### Edit node.yaml - -Edit `node.yaml` and replace all instances of `` with the private IP address of the master node. - -#### Provision worker nodes - -```shell -aws ec2 run-instances \ ---count 1 \ ---image-id \ ---key-name \ ---region us-west-2 \ ---security-groups kubernetes \ ---instance-type m3.medium \ ---user-data file://node.yaml -``` - -### Google Compute Engine (GCE) - -*Attention:* Replace `` below for a [suitable version of CoreOS image for Google Compute Engine](https://coreos.com/docs/running-coreos/cloud-providers/google-compute-engine/). - -#### Provision the Master - -```shell -gcloud compute instances create master \ ---image-project coreos-cloud \ ---image \ ---boot-disk-size 200GB \ ---machine-type n1-standard-1 \ ---zone us-central1-a \ ---metadata-from-file user-data=master.yaml -``` - -#### Capture the private IP address - -```shell -gcloud compute instances list -``` - -#### Edit node.yaml - -Edit `node.yaml` and replace all instances of `` with the private IP address of the master node. - -#### Provision worker nodes - -```shell -gcloud compute instances create node1 \ ---image-project coreos-cloud \ ---image \ ---boot-disk-size 200GB \ ---machine-type n1-standard-1 \ ---zone us-central1-a \ ---metadata-from-file user-data=node.yaml -``` - -#### Establish network connectivity - -Next, setup an ssh tunnel to the master so you can run kubectl from your local host. -In one terminal, run `gcloud compute ssh master --ssh-flag="-L 8080:127.0.0.1:8080"` and in a second -run `gcloud compute ssh master --ssh-flag="-R 8080:127.0.0.1:8080"`. - -### OpenStack - -These instructions are for running on the command line. Most of this you can also do through the Horizon dashboard. -These instructions were tested on the Ice House release on a Metacloud distribution of OpenStack but should be similar if not the same across other versions/distributions of OpenStack. - -#### Make sure you can connect with OpenStack - -Make sure the environment variables are set for OpenStack such as: - -```shell -OS_TENANT_ID -OS_PASSWORD -OS_AUTH_URL -OS_USERNAME -OS_TENANT_NAME -``` - -Test this works with something like: - -```shell -nova list -``` - -#### Get a Suitable CoreOS Image - -You'll need a [suitable version of CoreOS image for OpenStack](https://coreos.com/os/docs/latest/booting-on-openstack.html) -Once you download that, upload it to glance. An example is shown below: - -```shell -glance image-create --name CoreOS723 \ ---container-format bare --disk-format qcow2 \ ---file coreos_production_openstack_image.img \ ---is-public True -``` - -#### Create security group - -```shell -nova secgroup-create kubernetes "Kubernetes Security Group" -nova secgroup-add-rule kubernetes tcp 22 22 0.0.0.0/0 -nova secgroup-add-rule kubernetes tcp 80 80 0.0.0.0/0 -``` - -#### Provision the Master - -```shell -nova boot \ ---image \ ---key-name \ ---flavor \ ---security-group kubernetes \ ---user-data files/master.yaml \ -kube-master -``` - -`` is the CoreOS image name. In our example we can use the image we created in the previous step and put in 'CoreOS723' - -`` is the keypair name that you already generated to access the instance. - -`` is the flavor ID you use to size the instance. Run `nova flavor-list` to get the IDs. 3 on the system this was tested with gives the m1.large size. - -The important part is to ensure you have the files/master.yml as this is what will do all the post boot configuration. This path is relevant so we are assuming in this example that you are running the nova command in a directory where there is a subdirectory called files that has the master.yml file in it. Absolute paths also work. - -Next, assign it a public IP address: - -```shell -nova floating-ip-list -``` - -Get an IP address that's free and run: - -```shell -nova floating-ip-associate kube-master -``` - -where `` is the IP address that was available from the `nova floating-ip-list` command. - -#### Provision Worker Nodes - -Edit `node.yaml` and replace all instances of `` with the private IP address of the master node. You can get this by running `nova show kube-master` assuming you named your instance kube master. This is not the floating IP address you just assigned it. - -```shell -nova boot \ ---image \ ---key-name \ ---flavor \ ---security-group kubernetes \ ---user-data files/node.yaml \ -minion01 -``` - -This is basically the same as the master nodes but with the node.yaml post-boot script instead of the master. \ No newline at end of file diff --git a/docs/getting-started-guides/coreos/index.md b/docs/getting-started-guides/coreos/index.md index c119477b19..80199a61c0 100644 --- a/docs/getting-started-guides/coreos/index.md +++ b/docs/getting-started-guides/coreos/index.md @@ -41,12 +41,6 @@ A generic guide to setting up an HA cluster on any cloud or bare metal, with ful These guides are maintained by community members, cover specific platforms and use cases, and experiment with different ways of configuring Kubernetes on CoreOS. -[**Multi-node Cluster**](/docs/getting-started-guides/coreos/coreos_multinode_cluster) - -Set up a single master, multi-worker cluster on your choice of platform: AWS, GCE, or VMware Fusion. - -
- [**Easy Multi-node Cluster on Google Compute Engine**](https://github.com/rimusz/coreos-multi-node-k8s-gce/blob/master/README.md) Scripted installation of a single master, multi-worker cluster on GCE. Kubernetes components are managed by [fleet](https://github.com/coreos/fleet). From 27840ef8fe81b16da93f01af1456ea250dfbd51a Mon Sep 17 00:00:00 2001 From: Quinton Hoole Date: Mon, 10 Oct 2016 16:27:54 -0700 Subject: [PATCH 40/87] Improvements to Federated Ingress User Guide. --- .../federation/federated-ingress.md | 74 ++++++++++++++++--- 1 file changed, 62 insertions(+), 12 deletions(-) diff --git a/docs/user-guide/federation/federated-ingress.md b/docs/user-guide/federation/federated-ingress.md index 87965a3fc7..6198de1817 100644 --- a/docs/user-guide/federation/federated-ingress.md +++ b/docs/user-guide/federation/federated-ingress.md @@ -64,12 +64,12 @@ healthy backend service endpoint at all times, even in the event of pod, cluster, availability zone or regional outages. -Note that in the - case of Google Cloud, the logical L7 load balancer is not a single physical device (which - would present both a single point of failure, and a single global - network routing choke point), but rather a [truly global, highly available - load balancing managed service](https://cloud.google.com/load-balancing/), - globally reachable via a single, static IP address. +Note that in the case of Google Cloud, the logical L7 load balancer is +not a single physical device (which would present both a single point +of failure, and a single global network routing choke point), but +rather a +[truly global, highly available load balancing managed service](https://cloud.google.com/load-balancing/), +globally reachable via a single, static IP address. Clients inside your federated Kubernetes clusters (i.e. Pods) will be automatically routed to the cluster-local shard of the Federated Service @@ -86,13 +86,13 @@ You can create a federated ingress in any of the usual ways, for example using k ``` shell kubectl --context=federation-cluster create -f myingress.yaml ``` - +For example ingress YAML configurations, see the [Ingress User Guide](/docs/user-guide/ingress/) The '--context=federation-cluster' flag tells kubectl to submit the request to the Federation API endpoint, with the appropriate credentials. If you have not yet configured such a context, visit the [federation admin guide](/docs/admin/federation/) or one of the [administration tutorials](https://github.com/kelseyhightower/kubernetes-cluster-federation) -to find out how to do so. TODO: Update links +to find out how to do so. As described above, the Federated Ingress will automatically create and maintain matching Kubernetes ingresses in all of the clusters @@ -147,17 +147,28 @@ Events: 2m 2m 1 {loadbalancer-controller } Normal CREATE ip: 130.211.5.194 ``` -Note the address of your Federated Ingress +Note that: + +1. the address of your Federated Ingress corresponds with the address of all of the underlying Kubernetes ingresses (once these have been allocated - this may take up to a few minutes). - -Note also that we have not yet provisioned any backend Pods to receive +2. we have not yet provisioned any backend Pods to receive the network traffic directed to this ingress (i.e. 'Service Endpoints' behind the service backing the Ingress), so the Federated Ingress does not yet consider these to be healthy shards and will not direct traffic to any of these clusters. +3. the federation control system will +automatically reconfigure the load balancer controllers in all of the +clusters in your federation to make them consistent, and allow +them to share global load balancers. But this reconfiguration can +only complete successfully if there are no pre-existing Ingresses in +those clusters (this is a safety feature to prevent accidental +breakage of existing ingresses). So to ensure that your federated +ingresses function correctly, either start with new, empty clusters, or make +sure that you delete (and recreate if necessary) all pre-existing +Ingresses in the clusters comprising your federation. -## Adding backend services and pods +#Adding backend services and pods To render the underlying ingress shards healthy, we need to add backend Pods behind the service upon which the Ingress is based. There are several ways to achieve this, but @@ -175,6 +186,16 @@ kubectl --context=federation-cluster create -f services/nginx.yaml kubectl --context=federation-cluster create -f myreplicaset.yaml ``` +Note that in order for your federated ingress to work correctly on +Google Cloud, the node ports of all of the underlying cluster-local +services need to be identical. If you're using a federated service +this is easy to do. Simply pick a node port that is not already +being used in any of your clusters, and add that to the spec of your +federated service. If you do not specify a node port for your +federated service, each cluster will choose it's own node port for +its cluster-local shard of the service, and these will probably end +up being different, which is not what you want. + You can verify this by checking in each of the underlying clusters, for example: ``` shell @@ -258,6 +279,35 @@ Check that: `service-controller` or `replicaset-controller`, errors in the output of `kubectl logs federation-controller-manager --namespace federation`). +#### I can create a federated ingress successfully, but request load is not correctly distributed across the underlying clusters + +Check that: + +1. the services underlying your federated ingress in each cluster have + identical node ports. See [above](#creating_a_federated_ingress) for further explanation. +2. the load balancer controllers in each of your clusters are of the + correct type ("GLBC") and have been correctly reconfigured by the + federation control plane to share a global GCE load balancer (this + should happen automatically). If they of the correct type, and + have been correctly reconfigured, the UID data item in the GLBC + configmap in each cluster will be identical across all clusters. + See + [the GLBC docs](https://github.com/kubernetes/contrib/blob/master/ingress/controllers/gce/BETA_LIMITATIONS.md#changing-the-cluster-uid) + for further details. + If this is not the case, check the logs of your federation + controller manager to determine why this automated reconfiguration + might be failing. +3. no ingresses have been manually created in any of your clusters before the above + reconfiguration of the load balancer controller completed + successfully. Ingresses created before the reconfiguration of + your GLBC will interfere with the behavior of your federated + ingresses created after the reconfiguration (see + [the GLBC docs](https://github.com/kubernetes/contrib/blob/master/ingress/controllers/gce/BETA_LIMITATIONS.md#changing-the-cluster-uid) + for further information. To remedy this, + delete any ingresses created before the cluster joined the + federation (and had it's GLBC reconfigured), and recreate them if + necessary. + #### This troubleshooting guide did not help me solve my problem Please use one of our [support channels](http://kubernetes.io/docs/troubleshooting/) to seek assistance. From 7f8a38b7432a059ee9b0116469fadeca6ca9b017 Mon Sep 17 00:00:00 2001 From: Steve Perry Date: Tue, 15 Nov 2016 12:26:42 -0800 Subject: [PATCH 41/87] Write style guide. (#1619) --- _data/support.yml | 2 + docs/contribute/style-guide.md | 203 +++++++++++++++++++++++++++++++++ 2 files changed, 205 insertions(+) create mode 100644 docs/contribute/style-guide.md diff --git a/_data/support.yml b/_data/support.yml index e280c827d7..1b8e80699a 100644 --- a/_data/support.yml +++ b/_data/support.yml @@ -14,6 +14,8 @@ toc: path: /docs/contribute/stage-documentation-changes/ - title: Using Page Templates path: /docs/contribute/page-templates/ + - title: Documentation Style Guide + path: /docs/contribute/style-guide/ - title: Troubleshooting section: diff --git a/docs/contribute/style-guide.md b/docs/contribute/style-guide.md new file mode 100644 index 0000000000..219d62560d --- /dev/null +++ b/docs/contribute/style-guide.md @@ -0,0 +1,203 @@ +--- +--- + +{% capture overview %} +This page gives writing style guidelines for the Kubernetes documentation. +These are guidelines, not rules. Use your best judgment, and feel free to +propose changes to this document in a pull request. + +For additional information on creating new content for the Kubernetes +docs, follow the instructions on +[using page templates](/docs/contribute/page-templates/) and +[creating a documentation pull request](/docs/contribute/create-pull-request/). +{% endcapture %} + +{% capture body %} + +## Documentation formatting standards + +### Capitalize API objects + +Capitalize the names of API objects. Refer to API objects without saying +"object." + + + + + +
DoDon't
The Pod has two Containers.The pod has two containers.
The Deployment is responsible for ...The Deployment object is responsible for ...
+ +### Use angle brackets for placeholders + +Use angle brackets for placeholders. Tell the reader what a placeholder +represents. + +1. Display information about a pod: + + kubectl describe pod + + where `` is the name of one of your pods. + +### Use bold for user interface elements + + + + + +
DoDon't
Click Fork.Click "Fork".
Select Other.Select 'Other'.
+ +### Use italics to define or introduce new terms + + + + + +
DoDon't
A cluster is a set of nodes ...A "cluster" is a set of nodes ...
These components form the control plane.These components form the control plane.
+ +### Use code style for filenames, directories, and paths + + + + + + +
DoDon't
Open the envars.yaml file.Open the envars.yaml file.
Go to the /docs/tutorials directory.Go to the /docs/tutorials directory.
Open the /_data/concepts.yaml file.Open the /_data/concepts.yaml file.
+ +## Code snippet formatting + +### Use code style for inline code and commands + +For inline code in an HTML document, use the `` tag. In a Markdown +document, use the backtick (`). + + + + + +
DoDon't
Set the value of the replicas field in the configuration file.Set the value of the "replicas" field in the configuration file.
The kubectl run command creates a Deployment.The "kubectl run" command creates a Deployment.
+ +### Don't include the command prompt + + + + +
DoDon't
kubectl get pods$ kubectl get pods
+ +### Separate commands from output + +Verify that the pod is running on your chosen node: + + kubectl get pods --output=wide + +The output is similar to this: + + NAME READY STATUS RESTARTS AGE IP NODE + nginx 1/1 Running 0 13s 10.200.0.4 worker0 + + +{% comment %}## Kubernetes.io word list + +A list of Kubernetes-specific terms and words to be used consistently across the site. + + + + +
TermUseage
TBDTBD
{% endcomment %} + + +## Content best practices + +This section contains suggested best practices for clear, concise, and consistent content. + +### Use present tense + + + + +
DoDon't
This command starts a proxy.This command will start a proxy.
+ +Exception: Use future or past tense if it is required to convey the correct +meaning. + +### Use active voice + + + + + +
DoDon't
You can explore the API using a browser.The API can be explored using a browser.
The YAML file specifies the replica count.The replica count is specified in the YAML file.
+ +Exception: Use passive voice if active voice leads to an awkward construction. + +### Use simple and direct language + +Use simple and direct language. Avoid using unnecessary phrases, such as saying "please." + + + + + + + +
DoDon't
To create a ReplicaSet, ...In order to create a ReplicaSet, ...
See the configuration file.Please see the configuration file.
View the Pods.With this next command, we'll view the Pods.
+ +### Address the reader as "you" + + + + + +
DoDon't
You can create a Deployment by ...We'll create a Deployment by ...
In the preceding output, you can see...In the preceding output, we can see ...
+ +## Patterns to avoid + +### Avoid using "we" + +Using "we" in a sentence can be confusing, because the reader might not know +whether they're part of the "we" you're describing. + + + + + + +
DoDon't
Version 1.4 includes ...In version 1.4, we have added ...
Kubernetes provides a new feature for ...We provide a new feature ...
This page teaches you how to use pods.In this page, we are going to learn about pods.
+ +### Avoid jargon and idioms + +Some readers speak English as a second language. Avoid jargon and idioms to help make their understanding easier. + + + + + +
DoDon't
Internally, ...Under the hood, ...
Create a new cluster.Turn up a new cluster.
+ +### Avoid statements about the future + +Avoid making promises or giving hints about the future. If you need to talk about +an alpha feature, put the text under a heading that identifies it as alpha +information. + +### Avoid statements that will soon be out of date + +Avoid words like "currently" and "new." A feature that is new today might not be +considered new in a few months. + + + + + +
DoDon't
In version 1.4, ...In the current version, ...
The Federation feature provides ...The new Federation feature provides ...
+ +{% endcapture %} + + +{% capture whatsnext %} +* Learn about [writing a new topic](/docs/contribute/write-new-topic/). +* Learn about [using page templates](/docs/contribute/page-templates/). +* Learn about [staging your changes](/docs/contribute/stage-documentation-changes/) +* Learn about [creating a pull request](/docs/contribute/create-pull-request/). +{% endcapture %} + +{% include templates/concept.md %} From 16065824dc1e62efbbd2e4364edd5ea58cdfeb89 Mon Sep 17 00:00:00 2001 From: "Jorge O. Castro" Date: Wed, 9 Nov 2016 12:38:54 -0500 Subject: [PATCH 42/87] Move the Ubuntu guides into their own directory and update for 16.04. --- docs/getting-started-guides/index.md | 18 +- docs/getting-started-guides/juju.md | 325 ------------------ .../ubuntu/automated.md | 288 ++++++++++++++++ .../{ubuntu-calico.md => ubuntu/calico.md} | 0 .../{ubuntu.md => ubuntu/manual.md} | 0 5 files changed, 298 insertions(+), 333 deletions(-) delete mode 100644 docs/getting-started-guides/juju.md create mode 100644 docs/getting-started-guides/ubuntu/automated.md rename docs/getting-started-guides/{ubuntu-calico.md => ubuntu/calico.md} (100%) rename docs/getting-started-guides/{ubuntu.md => ubuntu/manual.md} (100%) diff --git a/docs/getting-started-guides/index.md b/docs/getting-started-guides/index.md index a728980e50..7afa6d8301 100644 --- a/docs/getting-started-guides/index.md +++ b/docs/getting-started-guides/index.md @@ -74,8 +74,7 @@ These solutions are combinations of cloud provider and OS not covered by the abo - [AWS + CoreOS](/docs/getting-started-guides/coreos) - [GCE + CoreOS](/docs/getting-started-guides/coreos) -- [AWS + Ubuntu](/docs/getting-started-guides/juju) -- [Joyent + Ubuntu](/docs/getting-started-guides/juju) +- [AWS/GCE/Rackspace/Joyent + Ubuntu](/docs/getting-started-guides/ubuntu/automated) - [Rackspace + CoreOS](/docs/getting-started-guides/rackspace) #### On-Premises VMs @@ -84,7 +83,7 @@ These solutions are combinations of cloud provider and OS not covered by the abo - [CloudStack](/docs/getting-started-guides/cloudstack) (uses Ansible, CoreOS and flannel) - [Vmware vSphere](/docs/getting-started-guides/vsphere) (uses Debian) - [Vmware Photon Controller](/docs/getting-started-guides/photon-controller) (uses Debian) -- [juju.md](/docs/getting-started-guides/juju) (uses Juju, Ubuntu and flannel) +- [Vmware vSphere, OpenStack, or Bare Metal](/docs/getting-started-guides/ubuntu/automated) (uses Juju, Ubuntu and flannel) - [Vmware](/docs/getting-started-guides/coreos) (uses CoreOS and flannel) - [libvirt-coreos.md](/docs/getting-started-guides/libvirt-coreos) (uses CoreOS) - [oVirt](/docs/getting-started-guides/ovirt) @@ -99,7 +98,8 @@ These solutions are combinations of cloud provider and OS not covered by the abo - [Fedora single node](/docs/getting-started-guides/fedora/fedora_manual_config) - [Fedora multi node](/docs/getting-started-guides/fedora/flannel_multi_node_cluster) - [Centos](/docs/getting-started-guides/centos/centos_manual_config) -- [Ubuntu](/docs/getting-started-guides/ubuntu) +- [Bare Metal with Ubuntu](/docs/getting-started-guides/ubuntu/automated) +- [Ubuntu Manual](/docs/getting-started-guides/ubuntu/manual) - [Docker Multi Node](/docs/getting-started-guides/docker-multinode) - [CoreOS](/docs/getting-started-guides/coreos) @@ -145,9 +145,11 @@ CloudStack | Ansible | CoreOS | flannel | [docs](/docs/gettin Vmware vSphere | Saltstack | Debian | OVS | [docs](/docs/getting-started-guides/vsphere) | | Community ([@imkin](https://github.com/imkin)) Vmware Photon | Saltstack | Debian | OVS | [docs](/docs/getting-started-guides/photon-controller) | | Community ([@alainroy](https://github.com/alainroy)) Bare-metal | custom | CentOS | _none_ | [docs](/docs/getting-started-guides/centos/centos_manual_config) | | Community ([@coolsvap](https://github.com/coolsvap)) -AWS | Juju | Ubuntu | flannel | [docs](/docs/getting-started-guides/juju) | | [Community](https://github.com/whitmo/bundle-kubernetes) ( [@whit](https://github.com/whitmo), [@matt](https://github.com/mbruzek), [@chuck](https://github.com/chuckbutler) ) -OpenStack/HPCloud | Juju | Ubuntu | flannel | [docs](/docs/getting-started-guides/juju) | | [Community](https://github.com/whitmo/bundle-kubernetes) ( [@whit](https://github.com/whitmo), [@matt](https://github.com/mbruzek), [@chuck](https://github.com/chuckbutler) ) -Joyent | Juju | Ubuntu | flannel | [docs](/docs/getting-started-guides/juju) | | [Community](https://github.com/whitmo/bundle-kubernetes) ( [@whit](https://github.com/whitmo), [@matt](https://github.com/mbruzek), [@chuck](https://github.com/chuckbutler) ) +AWS | Juju | Ubuntu | flannel | [docs](/docs/getting-started-guides/ubuntu/automated) | | [Community](https://github.com/juju-solutions/bundle-canonical-kubernetes) ( [@matt](https://github.com/mbruzek), [@chuck]*(https://github.com/chuckbutler) ) +GCE | Juju | Ubuntu | flannel | [docs](/docs/getting-started-guides/ubuntu/automated) | | [Community](https://github.com/juju-solutions/bundle-canonical-kubernetes) ( [@matt](https://github.com/mbruzek), [@chuck]*(https://github.com/chuckbutler) ) +Bare Metal | Juju | Ubuntu | flannel | [docs](/docs/getting-started-guides/ubuntu/automated) | | [Community](https://github.com/juju-solutions/bundle-canonical-kubernetes) ( [@matt](https://github.com/mbruzek), [@chuck]*(https://github.com/chuckbutler) ) +Rackspace | Juju | Ubuntu | flannel | [docs](/docs/getting-started-guides/ubuntu/automated) | | [Community](https://github.com/juju-solutions/bundle-canonical-kubernetes) ( [@matt](https://github.com/mbruzek), [@chuck]*(https://github.com/chuckbutler) ) +Vmware vSphere | Juju | Ubuntu | flannel | [docs](/docs/getting-started-guides/ubuntu/automated) | | [Community](https://github.com/juju-solutions/bundle-canonical-kubernetes) ( [@matt](https://github.com/mbruzek), [@chuck]*(https://github.com/chuckbutler) ) AWS | Saltstack | Debian | AWS | [docs](/docs/getting-started-guides/aws) | | Community ([@justinsb](https://github.com/justinsb)) AWS | kops | Debian | AWS | [docs](https://github.com/kubernetes/kops) | | Community ([@justinsb](https://github.com/justinsb)) Bare-metal | custom | Ubuntu | Calico | [docs](/docs/getting-started-guides/ubuntu-calico) | | Community ([@djosborne](https://github.com/djosborne)) @@ -161,7 +163,7 @@ any | any | any | any | [docs](/docs/gettin *Note*: The above table is ordered by version test/used in notes followed by support level. -Definition of columns: +Definition of columns - **IaaS Provider** is who/what provides the virtual or physical machines (nodes) that Kubernetes runs on. - **OS** is the base operating system of the nodes. diff --git a/docs/getting-started-guides/juju.md b/docs/getting-started-guides/juju.md deleted file mode 100644 index 5982f152a2..0000000000 --- a/docs/getting-started-guides/juju.md +++ /dev/null @@ -1,325 +0,0 @@ ---- -assignees: -- caesarxuchao -- erictune - ---- - -[Juju](https://jujucharms.com/docs/2.0/about-juju) encapsulates the -operational knowledge of provisioning, installing, and securing a Kubernetes -cluster into one step. Juju allows you to deploy a Kubernetes cluster on -different cloud providers with a consistent, repeatable user experience. -Once deployed the cluster can easily scale up with one command. - -The Juju Kubernetes work is curated by a dedicated team of community members, -let us know how we are doing. If you find any problems please open an -[issue on the kubernetes project](https://github.com/kubernetes/kubernetes/issues) -and tag the issue with "juju" so we can find them. - - -* TOC -{:toc} - -## Prerequisites - -> Note: If you're running kube-up, on Ubuntu - all of the dependencies -> will be handled for you. You may safely skip to the section: -> [Launch a Kubernetes Cluster](#launch-a-kubernetes-cluster) - -### On Ubuntu - -[Install the Juju client](https://jujucharms.com/docs/2.0/getting-started-general) - -> This documentation focuses on the Juju 2.0 release which will be -> promoted to stable during the April 2016 release cycle. - -To paraphrase, on your local Ubuntu system: - -```shell -sudo add-apt-repository ppa:juju/devel -sudo apt-get update -sudo apt-get install juju -``` - -If you are using another distro/platform - please consult the -[getting started guide](https://jujucharms.com/docs/2.0/getting-started-general) -to install the Juju dependencies for your platform. - -### With Docker - -If you prefer the isolation of Docker, you can run the Juju client in a -container. Create a local directory to store the Juju configuration, then -volume mount the container: - -```shell -mkdir -p $HOME/.local/share/juju -docker run --rm -ti \ - -v $HOME/.local/share/juju:/home/ubuntu/.local/share/juju \ - jujusolutions/charmbox:devel -``` - -> While this is a common target, the charmbox flavors of images are -> unofficial, and should be treated as experimental. If you encounter any issues -> turning up the Kubernetes cluster with charmbox, please file a bug on the -> [charmbox issue tracker](https://github.com/juju-solutions/charmbox/issues). - -### Configure Juju to your favorite cloud provider - -At this point you have access to the Juju client. Before you can deploy a -cluster you have to configure Juju with the -[cloud credentials](https://jujucharms.com/docs/2.0/credentials) for each -cloud provider you would like to use. - -Juju [supports a wide variety of public clouds](#cloud-compatibility) to set -up the credentials for your chosen cloud see the -[cloud setup page](https://jujucharms.com/docs/devel/getting-started-general#2.-choose-a-cloud). - -After configuration is complete test your setup with a `juju bootstrap` -command: `juju bootstrap $controllername $cloudtype` you are ready to launch -the Kubernetes cluster. - -## Launch a Kubernetes cluster - -You can deploy a Kubernetes cluster with Juju from the `kubernetes` directory of -the [kubernetes github project](https://github.com/kubernetes/kubernetes.git). -Clone the repository on your local system. Export the `KUBERNETES_PROVIDER` -environment variable before bringing up the cluster. - -```shell -cd kubernetes -export KUBERNETES_PROVIDER=juju -cluster/kube-up.sh -``` - -If this is your first time running the `kube-up.sh` script, it will attempt to -install the required dependencies to get started with Juju. - -The script will deploy two nodes of Kubernetes, 1 unit of etcd, and network -the units so containers on different hosts can communicate with each other. - -## Exploring the cluster - -The `juju status` command provides information about each unit in the cluster: - -```shell -$ juju status -MODEL CONTROLLER CLOUD/REGION VERSION -default windows azure/centralus 2.0-beta13 - -APP VERSION STATUS EXPOSED ORIGIN CHARM REV OS -etcd active false jujucharms etcd 3 ubuntu -kubernetes active true jujucharms kubernetes 5 ubuntu - -RELATION PROVIDES CONSUMES TYPE -cluster etcd etcd peer -etcd etcd kubernetes regular -certificates kubernetes kubernetes peer - -UNIT WORKLOAD AGENT MACHINE PORTS PUBLIC-ADDRESS MESSAGE -etcd/0 active idle 0 2379/tcp 13.67.217.11 (leader) cluster is healthy -kubernetes/0 active idle 1 8088/tcp 13.67.219.76 Kubernetes running. -kubernetes/1 active idle 2 6443/tcp 13.67.219.182 (master) Kubernetes running. - -MACHINE STATE DNS INS-ID SERIES AZ -0 started 13.67.217.11 machine-0 trusty -1 started 13.67.219.76 machine-1 trusty -2 started 13.67.219.182 machine-2 trusty -``` - -## Run some containers! - -The `kubectl` file, and the TLS certificates along with the configuration are -all available on the Kubernetes master unit. Fetch the kubectl package so you -can run commands on the new Kuberntetes cluster. - -Use the `juju status` command to figure out which unit is the master. In the -example above the "kubernetes/1" unit is the master. Use the `juju scp` -command to copy the file from the unit: - -```shell -juju scp kubernetes/1:kubectl_package.tar.gz . -tar xvfz kubectl_package.tar.gz -./kubectl --kubeconfig kubeconfig get pods -``` - -If you are not on a Linux amd64 host system, you will need to find or build a -kubectl binary package for your architecture. - -Copy the `kubeconfig` file to the home directory so you don't have to specify -it on the command line each time. The default location is -`${HOME}/.kube/config`. - -No pods will be available before starting a container: - -```shell -kubectl get pods -NAME READY STATUSRESTARTS AGE - -kubectl get replicationcontrollers -CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS -``` - -We'll follow the aws-coreos example. Create a pod manifest: `pod.json` - -```json -{ - "apiVersion": "v1", - "kind": "Pod", - "metadata": { -"name": "hello", -"labels": { - "name": "hello", - "environment": "testing" -} - }, - "spec": { -"containers": [{ - "name": "hello", - "image": "quay.io/kelseyhightower/hello", - "ports": [{ -"containerPort": 80, -"hostPort": 80 - }] -}] - } -} -``` - -Create the pod with kubectl: - -```shell -kubectl create -f pod.json -``` - -Get info on the pod: - -```shell -kubectl get pods -``` - -To test the hello app, we need to locate which node is hosting -the container. We can use `juju run` and `juju status` commands to find -our hello app. - -Exit out of our ssh session and run: - -```shell -juju run --unit kubernetes/0 "docker ps -n=1" -... -juju run --unit kubernetes/1 "docker ps -n=1" -CONTAINER IDIMAGE COMMAND CREATED STATUS PORTS NAMES -02beb61339d8quay.io/kelseyhightower/hello:latest /hello About an hour ago Up About an hourk8s_hello.... -``` - -We see "kubernetes/1" has our container, expose the kubernetes charm and open -port 80: - -```shell -juju run --unit kubernetes/1 "open-port 80" -juju expose kubernetes -sudo apt-get install curl -curl $(juju status --format=oneline kubernetes/1 | cut -d' ' -f3) -``` - -Finally delete the pod: - -```shell -juju ssh kubernetes/0 -kubectl delete pods hello -``` - -## Scale up cluster - -Want larger Kubernetes nodes? It is easy to request different sizes of cloud -resources from Juju by using **constraints**. You can increase the amount of -CPU or memory (RAM) in any of the systems requested by Juju. This allows you -to fine tune th Kubernetes cluster to fit your workload. Use flags on the -bootstrap command or as a separate `juju constraints` command. Look to the -[Juju documentation for machine](https://jujucharms.com/docs/2.0/charms-constraints) -details. - -## Scale out cluster - -Need more workers? Juju makes it easy to add units of a charm: - -```shell -juju add-unit kubernetes -``` - -Or multiple units at one time: - -```shell -juju add-unit -n3 kubernetes -``` - -You can also scale the etcd charm for more fault tolerant key/value storage: - -```shell -juju add-unit -n2 etcd -``` - -## Tear down cluster - -We recommend that you use the `kube-down.sh` script when you are done using -the cluster, as it properly brings down the cloud and removes some of the -build directories. - -```shell -./cluster/kube-down.sh -``` - -Alternately if you want stop the servers you can destroy the Juju model or the -controller. Use the `juju switch` command to get the current controller name: - -```shell -juju switch -juju destroy-controller $controllername --destroy-all-models -``` - -## More Info - -Juju works with charms and bundles to deploy solutions. The code that stands up -a Kubernetes cluster is done in the charm code. The charm is built from using -a layered approach to keep the code smaller and more focused on the operations -of Kubernetes. - -The Kubernetes layer and bundles can be found in the `kubernetes` -project on github.com: - - - [Bundle location](https://github.com/kubernetes/kubernetes/tree/master/cluster/juju/bundles) - - [Kubernetes charm layer location](https://github.com/kubernetes/kubernetes/tree/master/cluster/juju/layers/kubernetes) - - [More about Juju](https://jujucharms.com) - - -### Cloud compatibility - -Juju is cloud agnostic and gives you a consistent experience across different -cloud providers. Juju supports a variety of public cloud providers: [Amazon Web Service](https://jujucharms.com/docs/2.0/help-aws), -[Microsoft Azure](https://jujucharms.com/docs/2.0/help-azure), -[Google Compute Engine](https://jujucharms.com/docs/2.0/help-google), -[Joyent](https://jujucharms.com/docs/2.0/help-joyent), -[Rackspace](https://jujucharms.com/docs/2.0/help-rackspace), any -[OpenStack cloud](https://jujucharms.com/docs/2.0/clouds#specifying-additional-clouds), -and -[Vmware vSphere](https://jujucharms.com/docs/2.0/config-vmware). - -If you do not see your favorite cloud provider listed many clouds with ssh -access can be configured for -[manual provisioning](https://jujucharms.com/docs/2.0/clouds-manual). - -To change to a different cloud you can use the `juju switch` command and set -up the credentials for that cloud provider and continue to use the `kubeup.sh` -script. - -## Support Level - - -IaaS Provider | Config. Mgmt | OS | Networking | Docs | Conforms | Support Level --------------------- | ------------ | ------ | ---------- | --------------------------------------------- | ---------| ---------------------------- -Amazon Web Services (AWS) | Juju | Ubuntu | flannel | [docs](/docs/getting-started-guides/juju) | | [Community](https://github.com/juju-solutions/bundle-kubernetes-core) ( [@mbruzek](https://github.com/mbruzek), [@chuckbutler](https://github.com/chuckbutler) ) -OpenStack | Juju | Ubuntu | flannel | [docs](/docs/getting-started-guides/juju) | | [Community](https://github.com/juju-solutions/bundle-kubernetes-core) ( [@mbruzek](https://github.com/mbruzek), [@chuckbutler](https://github.com/chuckbutler) ) -Microsoft Azure | Juju | Ubuntu | flannel | [docs](/docs/getting-started-guides/juju) | | [Community](https://github.com/juju-solutions/bundle-kubernetes-core) ( [@mbruzek](https://github.com/mbruzek), [@chuckbutler](https://github.com/chuckbutler) ) -Google Compute Engine (GCE) | Juju | Ubuntu | flannel | [docs](/docs/getting-started-guides/juju) | | [Community](https://github.com/juju-solutions/bundle-kubernetes-core) ( [@mbruzek](https://github.com/mbruzek), [@chuckbutler](https://github.com/chuckbutler) ) - - -For support level information on all solutions, see the [Table of solutions](/docs/getting-started-guides/#table-of-solutions) chart. diff --git a/docs/getting-started-guides/ubuntu/automated.md b/docs/getting-started-guides/ubuntu/automated.md new file mode 100644 index 0000000000..21c25e1c3c --- /dev/null +++ b/docs/getting-started-guides/ubuntu/automated.md @@ -0,0 +1,288 @@ +--- +assignees: +- caesarxuchao +- erictune + +--- + +Ubuntu 16.04 introduced the [Canonical Distribution of Kubernetes](https://jujucharms.com/canonical-kubernetes/), a pure upstream distribution of Kubernetes designed for production usage. Out of the box it comes with the following components on 12 machines: + +- Kubernetes (automated deployment, operations, and scaling) + - Three node Kubernetes cluster with one master and two worker nodes. + - TLS used for communication between units for security. + - Flannel Software Defined Network (SDN) plugin + - A load balancer for HA kubernetes-master (Experimental) + - Optional Ingress Controller (on worker) + - Optional Dashboard addon (on master) including Heapster for cluster monitoring +- EasyRSA + - Performs the role of a certificate authority serving self signed certificates + to the requesting units of the cluster. +- Etcd (distributed key value store) + - Three unit cluster for reliability. +- Elastic stack + - Two units for ElasticSearch + - One units for a Kibana dashboard + - Beats on every Kubernetes and Etcd units: + - Filebeat for forwarding logs to ElasticSearch + - Topbeat for inserting server monitoring data to ElasticSearch + + +The Juju Kubernetes work is curated by a dedicated team of community members, +let us know how we are doing. If you find any problems please open an +[issue on our tracker](https://github.com/juju-solutions/bundle-canonical-kubernetes) +so we can find them. + +* TOC +{:toc} + +## Prerequisites + +- A working [Juju client](https://jujucharms.com/docs/2.0/getting-started-general); this does not have to be a Linux machine, it can also be Windows or OSX. +- A [supported cloud](#cloud-compatibility). + +### On Ubuntu + +On your local Ubuntu system: + +```shell +sudo add-apt-repository ppa:juju/stable +sudo apt-get update +sudo apt-get install juju +``` + +If you are using another distro/platform - please consult the +[getting started guide](https://jujucharms.com/docs/2.0/getting-started-general) +to install the Juju dependencies for your platform. + +### Configure Juju to your favorite cloud provider + +Deployment of the cluster is [supported on a wide variety of public clouds](#cloud-compatibility), private OpenStack clouds, or raw bare metal clusters. + +After deciding which cloud to deploy to, follow the [cloud setup page](https://jujucharms.com/docs/devel/getting-started-general#2.-choose-a-cloud) to configure deploying to that cloud. + +Load your [cloud credentials](https://jujucharms.com/docs/2.0/credentials) for each +cloud provider you would like to use. + +In this example + +```shell +juju add-credential aws +credential name: my_credentials +select auth-type [userpass, oauth, etc]: userpass +enter username: jorge +enter password: ******* +``` + +You can also just auto load credentials for popular clouds with the `juju autoload-credentials` command, which will auto import your credentials from the default files and environment variables for each cloud. + +Next we need to bootstrap a controller to manage the cluster. You need to define the cloud you want to bootstrap on, the region, and then any name for your controller node: + +```shell +juju update-clouds # This command ensures all the latest regions are up to date on your client +juju bootstrap aws/us-east-2 +``` +or, another example, this time on Azure: + +```shell +juju bootstrap azure/centralus +``` + +You will need a controller node for each cloud or region you are deploying to. See the [controller documentation](https://jujucharms.com/docs/2.0/controllers) for more information. + +Note that each controller can host multiple Kubernetes clusters in a given cloud or region. + +## Launch a Kubernetes cluster + +The following command will deploy the intial 12-node starter cluster. The speed of execution is very dependent of the performance of the cloud you're deploying to, but + +```shell +juju deploy canonical-kubernetes +``` + +After this command executes we need to wait for the cloud to return back instances and for all the automated deployment tasks to execute. + +## Monitor deployment + +The `juju status` command provides information about each unit in the cluster. We recommend using the `watch -c juju status --color` command to get a real-time view of the cluster as it deploys. When all the states are green and "Idle", the cluster is ready to go. + + +```shell +$ juju status +Model Controller Cloud/Region Version +default aws-us-east-2 aws/us-east-2 2.0.1 + +App Version Status Scale Charm Store Rev OS Notes +easyrsa 3.0.1 active 1 easyrsa jujucharms 3 ubuntu +elasticsearch active 2 elasticsearch jujucharms 19 ubuntu +etcd 2.2.5 active 3 etcd jujucharms 14 ubuntu +filebeat active 4 filebeat jujucharms 5 ubuntu +flannel 0.6.1 maintenance 4 flannel jujucharms 5 ubuntu +kibana active 1 kibana jujucharms 15 ubuntu +kubeapi-load-balancer 1.10.0 active 1 kubeapi-load-balancer jujucharms 3 ubuntu exposed +kubernetes-master 1.4.5 active 1 kubernetes-master jujucharms 6 ubuntu +kubernetes-worker 1.4.5 active 3 kubernetes-worker jujucharms 8 ubuntu exposed +topbeat active 3 topbeat jujucharms 5 ubuntu + +Unit Workload Agent Machine Public address Ports Message +easyrsa/0* active idle 0 52.15.95.92 Certificate Authority connected. +elasticsearch/0* active idle 1 52.15.67.111 9200/tcp Ready +elasticsearch/1 active idle 2 52.15.109.132 9200/tcp Ready +etcd/0 active idle 3 52.15.79.127 2379/tcp Healthy with 3 known peers. +etcd/1* active idle 4 52.15.111.66 2379/tcp Healthy with 3 known peers. (leader) +etcd/2 active idle 5 52.15.144.25 2379/tcp Healthy with 3 known peers. +kibana/0* active idle 6 52.15.57.157 80/tcp,9200/tcp ready +kubeapi-load-balancer/0* active idle 7 52.15.84.179 443/tcp Loadbalancer ready. +kubernetes-master/0* active idle 8 52.15.106.225 6443/tcp Kubernetes master services ready. + filebeat/3 active idle 52.15.106.225 Filebeat ready. + flannel/3 maintenance idle 52.15.106.225 Installing flannel. +kubernetes-worker/0* active idle 9 52.15.153.246 Kubernetes worker running. + filebeat/2 active idle 52.15.153.246 Filebeat ready. + flannel/2 active idle 52.15.153.246 Flannel subnet 10.1.53.1/24 + topbeat/2 active idle 52.15.153.246 Topbeat ready. +kubernetes-worker/1 active idle 10 52.15.52.103 Kubernetes worker running. + filebeat/0* active idle 52.15.52.103 Filebeat ready. + flannel/0* active idle 52.15.52.103 Flannel subnet 10.1.31.1/24 + topbeat/0* active idle 52.15.52.103 Topbeat ready. +kubernetes-worker/2 active idle 11 52.15.104.181 Kubernetes worker running. + filebeat/1 active idle 52.15.104.181 Filebeat ready. + flannel/1 active idle 52.15.104.181 Flannel subnet 10.1.83.1/24 + topbeat/1 active idle 52.15.104.181 Topbeat ready. + +Machine State DNS Inst id Series AZ +0 started 52.15.95.92 i-06e66414008eca61c xenial us-east-2c +1 started 52.15.67.111 i-050cbd7eb35fa0fe6 trusty us-east-2a +2 started 52.15.109.132 i-069196660db07c2f6 trusty us-east-2b +3 started 52.15.79.127 i-0038186d2c5103739 xenial us-east-2b +4 started 52.15.111.66 i-0ac66c86a8ec93b18 xenial us-east-2a +5 started 52.15.144.25 i-078cfe79313d598c9 xenial us-east-2c +6 started 52.15.57.157 i-09fd16d9328105ec0 trusty us-east-2a +7 started 52.15.84.179 i-00fd70321a51b658b xenial us-east-2c +8 started 52.15.106.225 i-0109a5fc942c53ed7 xenial us-east-2b +9 started 52.15.153.246 i-0ab63e34959cace8d xenial us-east-2b +10 started 52.15.52.103 i-0108a8cc0978954b5 xenial us-east-2a +11 started 52.15.104.181 i-0f5562571c649f0f2 xenial us-east-2c +``` + +## Interacting with the cluster + +After the cluster is deployed you may assume control over the cluster from any kubernetes-master, or kubernetes-worker node. + +First we need to download the credentials and client application to your local workstation: + +Create the kubectl config directory. + +```shell +mkdir -p ~/.kube +``` +Copy the kubeconfig file to the default location. + +```shell +juju scp kubernetes-master/0:config ~/.kube/config +``` + +Fetch a binary for the architecture you have deployed. If your client is a +different architecture you will need to get the appropriate `kubectl` binary +through other means. + +```shell +juju scp kubernetes-master/0:kubectl ./kubectl +``` + +Query the cluster. + +```shell +./kubectl cluster-info +Kubernetes master is running at https://52.15.104.227:443 +Heapster is running at https://52.15.104.227:443/api/v1/proxy/namespaces/kube-system/services/heapster +KubeDNS is running at https://52.15.104.227:443/api/v1/proxy/namespaces/kube-system/services/kube-dns +Grafana is running at https://52.15.104.227:443/api/v1/proxy/namespaces/kube-system/services/monitoring-grafana +InfluxDB is running at https://52.15.104.227:443/api/v1/proxy/namespaces/kube-system/services/monitoring-influxdb +``` + +Congratulations, you've now set up a Kubernetes cluster! + +## Scale up cluster + +Want larger Kubernetes nodes? It is easy to request different sizes of cloud +resources from Juju by using **constraints**. You can increase the amount of +CPU or memory (RAM) in any of the systems requested by Juju. This allows you +to fine tune th Kubernetes cluster to fit your workload. Use flags on the +bootstrap command or as a separate `juju constraints` command. Look to the +[Juju documentation for machine](https://jujucharms.com/docs/2.0/charms-constraints) +details. + +## Scale out cluster + +Need more workers? We just add more units: + +```shell +juju add-unit kubernetes-worker +``` + +Or multiple units at one time: + +```shell +juju add-unit -n3 kubernetes-worker +``` +You can also ask for specific instance types or other machine-specific constraints. See the [constraints documentation](https://jujucharms.com/docs/stable/reference-constraints) for more information. Here are some examples, note that generic constraints such as `cores` and `mem` are more portable between clouds. In this case we'll ask for a specific instance type from AWS: + +```shell +juju set-constraints kubernetes-worker instance-type=c4.large +juju add-unit kubernetes-worker +``` + +You can also scale the etcd charm for more fault tolerant key/value storage: + +```shell +juju add-unit -n3 etcd +``` +It is strongly recommended to run an odd number of units for quorum. + +## Tear down cluster + +If you want stop the servers you can destroy the Juju model or the +controller. Use the `juju switch` command to get the current controller name: + +```shell +juju switch +juju destroy-controller $controllername --destroy-all-models +``` +This will shutdown and terminate all running instances on that cloud. + +## More Info + +We stand up Kubernetes with open-source operations, or operations as code, known as charms. These charms are assembled from layers which keeps the code smaller and more focused on the operations of just Kubernetes and its components. + +The Kubernetes layer and bundles can be found in the `kubernetes` +project on github.com: + + - [Bundle location](https://github.com/kubernetes/kubernetes/tree/master/cluster/juju/bundles) + - [Kubernetes charm layer location](https://github.com/kubernetes/kubernetes/tree/master/cluster/juju/layers/kubernetes) + - [Canonical Kubernetes home](https://jujucharms.com/canonical-kubernetes/) + +Feature requests, bug reports, pull requests or any feedback would be much appreciated. + +### Cloud compatibility + +This deployment methodology is continually tested on the following clouds: + +[Amazon Web Service](https://jujucharms.com/docs/2.0/help-aws), +[Microsoft Azure](https://jujucharms.com/docs/2.0/help-azure), +[Google Compute Engine](https://jujucharms.com/docs/2.0/help-google), +[Joyent](https://jujucharms.com/docs/2.0/help-joyent), +[Rackspace](https://jujucharms.com/docs/2.0/help-rackspace), any +[OpenStack cloud](https://jujucharms.com/docs/2.0/clouds#specifying-additional-clouds), +and +[Vmware vSphere](https://jujucharms.com/docs/2.0/config-vmware). + +## Support Level + + +IaaS Provider | Config. Mgmt | OS | Networking | Docs | Conforms | Support Level +-------------------- | ------------ | ------ | ---------- | --------------------------------------------- | ---------| ---------------------------- +Amazon Web Services (AWS) | Juju | Ubuntu | flannel | [docs](/docs/getting-started-guides/juju) | | [Community](https://github.com/juju-solutions/bundle-kubernetes-core) ( [@mbruzek](https://github.com/mbruzek), [@chuckbutler](https://github.com/chuckbutler) ) +OpenStack | Juju | Ubuntu | flannel | [docs](/docs/getting-started-guides/juju) | | [Community](https://github.com/juju-solutions/bundle-kubernetes-core) ( [@mbruzek](https://github.com/mbruzek), [@chuckbutler](https://github.com/chuckbutler) ) +Microsoft Azure | Juju | Ubuntu | flannel | [docs](/docs/getting-started-guides/juju) | | [Community](https://github.com/juju-solutions/bundle-kubernetes-core) ( [@mbruzek](https://github.com/mbruzek), [@chuckbutler](https://github.com/chuckbutler) ) +Google Compute Engine (GCE) | Juju | Ubuntu | flannel | [docs](/docs/getting-started-guides/juju) | | [Community](https://github.com/juju-solutions/bundle-kubernetes-core) ( [@mbruzek](https://github.com/mbruzek), [@chuckbutler](https://github.com/chuckbutler) ) + +For support level information on all solutions, see the [Table of solutions](/docs/getting-started-guides/#table-of-solutions) chart. diff --git a/docs/getting-started-guides/ubuntu-calico.md b/docs/getting-started-guides/ubuntu/calico.md similarity index 100% rename from docs/getting-started-guides/ubuntu-calico.md rename to docs/getting-started-guides/ubuntu/calico.md diff --git a/docs/getting-started-guides/ubuntu.md b/docs/getting-started-guides/ubuntu/manual.md similarity index 100% rename from docs/getting-started-guides/ubuntu.md rename to docs/getting-started-guides/ubuntu/manual.md From d10fed8566acd1c407ea869a9a33238695a20fd5 Mon Sep 17 00:00:00 2001 From: ankurshukla Date: Tue, 15 Nov 2016 13:03:34 -0800 Subject: [PATCH 43/87] Added new set of Tech and Services Partners --- README.md | 9 +-------- _includes/partner-script.js | 23 ++++++++++++++++++++++- images/square-logos/datadog.png | Bin 13224 -> 12214 bytes images/square-logos/endocode.png | Bin 0 -> 5231 bytes images/square-logos/giant_swarm.png | Bin 0 -> 9772 bytes images/square-logos/mirantis.png | Bin 0 -> 17311 bytes 6 files changed, 23 insertions(+), 9 deletions(-) create mode 100644 images/square-logos/endocode.png create mode 100644 images/square-logos/giant_swarm.png create mode 100644 images/square-logos/mirantis.png diff --git a/README.md b/README.md index 381cd46567..2801eaead0 100644 --- a/README.md +++ b/README.md @@ -174,14 +174,7 @@ example. If creating an image for a doc, follow the section on "Docker images" from the Kubernetes repository. ## Partners -Kubernetes partners refers to the companies who contribute to the Kubernetes core codebase and/or extend their platform to support Kubernetes. Partners can get their logos added to the partner section of the [community page](http://k8s.io/community) by following the below steps and meeting the below logo specifications. Partners will also need to have a URL that is specific to integrating with Kubernetes ready; this URL will be the destination when the logo is clicked. - -* The partner product logo should be a transparent png image centered in a 215x125 px frame. (look at the existing logos for reference) -* The logo must link to a URL that is specific to integrating with Kubernetes, hosted on the partner's site. -* The logo should be named *product-name*_logo.png and placed in the `/images/community_logos` folder. -* The image reference (including the link to the partner URL) should be added in `community.html` under `
...
`. -* Please do not change the order of the existing partner images. Append your logo to the end of the list. -* Once completed and tested the look and feel, submit the pull request. +Kubernetes partners refers to the companies who contribute to the Kubernetes core codebase, extend their platform to support Kubernetes or provide managed services to users centered around the Kubernetes platform. Partners can get their services and offerings added to the [partner page](https://k8s.io/partners) by completing and submitting the [partner request form](https://goo.gl/qcSnZF). Once the information and assets are verified, the partner product/services will be listed in the partner page. This would typically take 7-10 days. ## Thank you! diff --git a/_includes/partner-script.js b/_includes/partner-script.js index f7447c63b7..5763047efc 100644 --- a/_includes/partner-script.js +++ b/_includes/partner-script.js @@ -175,6 +175,20 @@ link: 'https://aporeto.com/trireme', blurb: 'Aporeto makes cloud-native applications secure by default without impacting developer velocity and works at any scale, on any cloud.' }, + { + type: 0, + name: 'Giant Swarm', + logo: 'giant_swarm', + link: 'https://giantswarm.io', + blurb: 'Giant Swarm provides fully-managed Kubernetes Clusters in your location of choice, so you can focus on your product.' + }, + { + type: 0, + name: 'Mirantis', + logo: 'mirantis', + link: 'https://content.mirantis.com/Containerizing-OpenStack-on-Kubernetes-Video-Landing-Page.html', + blurb: 'Mirantis builds and manages private clouds with open source software such as OpenStack, deployed as containers orchestrated by Kubernetes.' + }, { type: 1, name: 'Apprenda', @@ -237,7 +251,14 @@ logo: 'skippbox', link: 'http://www.skippbox.com/services/', blurb: 'Skippbox brings its Kubernetes expertise to help companies embrace Kubernetes on their way to digital transformation. Skippbox offers both professional services and expert training.' - } + }, + { + type: 1, + name: 'Endocode', + logo: 'endocode', + link: 'https://endocode.com/kubernetes/', + blurb: 'Endocode practices and teaches the open source way. Kernel to cluster - Dev to Ops. We offer Kubernetes trainings, services and support.' + } ] var isvContainer = document.getElementById('isvContainer') diff --git a/images/square-logos/datadog.png b/images/square-logos/datadog.png index 82d6d11aaf93168cddb9ba48dac2458cd077562a..aeab0f227f42eeb6fcacd2635bb3835ae2dc0451 100644 GIT binary patch literal 12214 zcmZvCbx<5n)a?=yG`MSUcXxs>?(Xis1h?Q4U>A1?u0evky9W#I*F#^(Id@Ksnu;763NZ=*06>!mN^8Kz64;f7gaEtu3;|bR1F{=X-xB~p z#rf}o17zh80sttg_EJ)6YId$5S5G@vHwt+vDGE0aR~vgLYXHFKcdn+bmL?Qm^ltN7 zQZYItMbT9Q4~arUGA;xcOvgltjHwV!S-6U)-iIY4^PXh1AR0a)AtWA8o#kUJ$_nB> z<@bcJqUhN1ryc(y$ED7@!>QM%WwE2`tK7yJ#9pM2X+Umu-jI(KlH^$1VIzaXJNs;( zLs4kl0GLRPR+OGE)Np|NAQ2H}s$Rry0G!VPDiWYqIlGsGH0llGuVkhXTu3-vF9g}u(a3%sIdaf=z|Xz03o^enf`D9!xTz9xB@VM z0>?U98ek_3sGiY@l>z8+0B~&-2L%ButN;#qT^k^vt_9FLjfvU-K>7&aP>+sa1ibeL zn2l0X`vAf-0l2c4x*`|MwU|dtu$9Ve5NV?klnFINVs%H<)ny}Rn3gAC#^p1AXP)($ zt=~73h%1x_WBY0n04MkrYa>&b)V8{Mu)n{eFeqtg zKCT=1YTIwxtNY^g5G3+^ceCBSOB2dv94hzze!GA2TB(F|27(-Jv3{5+_uPW|_Dr@w zH7IY`4z3qqOd zBuK9N2w8d&Nc%pl_n|(U@r6s$4E9T6N}^bWkb0OgR)3Xbh@7e;V=`m^^_h&PSKTf$ z*$thoU%M4u)D7d&94W7lp)(Zuy~HRIjydJ;XgG^lJ?aD`g3O3}%0A$G>^K^Vv1nX% zDn&3akW(X8i$WVnb}8!dPB6+ywjC@y3gG(s9N8(ynG)KdF8b?ZlT=$dKXJJ1chp%c z5NB2*enIxE$4_F>c)r4;*(y7X--%*UBOJYd2obplojBNr2PEqSSdfz~C+exHKUNJB z*XdNlcf{OS&eVIq#}1Y0Mdo|&ElFWcE&H$=0}DPSMR5%N-kIY zN!O0`Fi2>@#TlgsWT9TdPQ+oy^pBPra9nOL`|?U#vBcJuf@&qCpL1smSI5 z5>qFS{kC#!Kom@2p|T7Yxyzt2?w~+)fO)`kfMA`$ zC81rErLj?7sD;IwXdqcr*sSqGJ*pgs$16*4DhyvH@JHf!&?`k3XPxtc_H1 zmGoKN`3b1ppY%WQ*WMq|BJsjTqHP(7vI!9hZHc@Ie`F|?5-r3Ljc9T3a~@|LWnyHy zYU?pIGX>y*$5B!hQt47jnAo){DoZOjD^aztv?sI@D@-&4Yh|^$HTNozf2CDvSLUi6 zYm#W%RtT3zRoa%jYKa#I>A%Dv=np6js7e*97u1{Xbq-UYS`sz-uuwXpRYiJ*@($9~ zPpt-gVd}_uA?t_}t*j^%l;T$4)@{_dO0f|~+{Zj_;v z;VX*@-PE#GI+l z9VMNj@?Yh%EDhzhQpQqYC1ItJsmZDDQ%jk=*`l1wTu0eQ*~{5mEoKHj1}x3x&8^n* z4f_VA`e)6ZE)fQT`XzeW2F8uDHMupkOC2>GWmsixD#Lkq z=5KPd8?*aQ%h!IK@txV9MXl#g^5iq->2rI|4vb1|s~^mXqvoe_IBr!3sY`FmSNTP9QoT=PsLwxoio->lyJ z-sS*hA+&I|@G+qhp@#3C?)?HNdlJNj)~s(ER!=433)&>`Boe>Yd_Cx^p|l%4Nvff| zJIa^_PurtMs0IC5nng4}G>;ny`bZxU6`>m`p5PX(M?A-w&g(t5F6Fr;{V1I#y_s#x zsVew`Re_gDtdh+^YhJA z2K%V7n{VNC|8+zeegzvgi%{BUrDzp_lz$SKOy&6Tw3j&EvNRP1wnbI~hO zMoi=?mp~?EKgbEVlh>DKh`WU4pe>xfBB@geO=%-rC=WmD3xzY6s8APY7zCW;E?G&b zoZM{t*Y?on%S5$xM8oTlOI!;(c`D+VV`EbF2YptQX35e&S_TZuOo$D(*19%5Q!(R$ zX;pd5Q>T-S<7-L$Ip_Irr`Y}2JCNp*NJU*%GQGpDY6rmv5qIA;ls&u;%#k_>)grC2 z%}maVs#T}D>9s*!bbiW?p(cIfU;CQi9FI}8Q4S&kkJpz^jaBv7^|zYLA0u-~#PDKb zYjM{I((!^<7}k9bNTOWFdotHkepXZswpq>C%-cW@H(B!x(bFo^57W_I&{lu{MBCE& zd3tjzv&No2#jw^9)-2Ul)FaZ`XZ>;dx8J9w3sI-y3uvz=Xy7Ece91R5#&Mt zGCk3;>P>%E)2Z5Ndj0uSVpVD*YEj(qb$#W0J=mewyWnbWUG5GcB5dXre3OVF^&v4c zaUv2e@;FaGOi*xF+~WoMTqZSRHA9z2@cR9AqLu)kJ?MV%czJp*W3Dj$GCjG+IiTd_ zZ|^bDkG0|cVe;-VA8t;QYn`XlKJRsJvh$)X4AuPrieQE}yNCK4&DpZ2WTNEcxA*WF zSI?H*tZOj7Nnr`pPzC_}XaRtbFaY4;4K_jn051*z;IA0~Ad~?B5V%52hhzZ&njLv* z2`!)BC%N7kMz(?H;$=$ttE*fcRjdYHH7xoN)-i~4o37J32j9wjq{wYdws6dks^hf$ z>Fa|=5KN?Sh?!C&nB+@1iASoFlRjjz3T2gGSt z4Ff_f6$yb)YH*WyR#kvQfc57JpYwMqlxsb{bn?ENisRy+SKfm=znbBFPNguezc-~4 zW6#+ay@=i`hD~S`yB~f(`xn%x&}8C0mg1lz0Yz@8=Ree&O-JzMbCeY3G%GMoyXq=D zC7W7|9j{nxdfpcy{BH};ms|jYyif=A{c2_8iif^G8wHY(^`G{Hw7EH6GJSATsy%YZ zeFSc^)42X$kqqZnY)Yue5%dygkF0AO-TtZNWomw0VovYD7^Ej7Mbc zpwajXlENLA9fJ1NX^3u7NDkQMNetDB{HpOO|M?LOS9c%F>D=;3aisM+%|Yz(fLXt; zGu*cO4jpQRuqbBV2uhF{&m1BMT`O%*Rx+SY9xs3jHLzGVG6NexvZ_YQAQ0KvbE6_- z?I?Uf!QjLk8QMO9zg}}KJ?w(D>QO+rlfF7g#j%EFl`pARgl~2N}ke00-k9GJf3LsjllA>9z|?0RHFWPw@t6W&j<*EgNUm<0J zT1CS5h>J$206slW(&nr$NS6!8I!vr~v*4?ku23CJ7LY|-Sm_L>BmwJAAWXF#-gpru zo9e-a-33W4dBTIc+SUR8LcLO^*!QQ(?iP#|WSG*V>lw9l2pZ2Khm7jF9WdMQWXMxG zkW-!z2DRl~`T(xosIH-Lcc!9!sopicQ1WPlhq!ek?=6|z5zb@@#8+C-UgBFXoE zdV%NPpy5D>^x_BFNkg4}Jvx2!igY^xiGdiELpJ2P!`~6r-41Z{cf6YZb>I30#n`%S zDEXd#aYt2HMQTRFUik(VeRaC;NV|x@tfs4b~PD4=l@h%R7kJu z+Udp@J~B;zF8uyyf*jB_G~2(&1CM{RCCexvJXS{L#P6V0{+)u>kBV*he4~NK>@+1* zN;s`-$vIN?AL32<%lYf7La7q!)@&xcu!RsS62Tl)@8~Jm~hDF7%X?!e-1! zT*lhh$|F5(2%m8VT&G5tT3F1KKjC7`C(*>D*+$blt*gSJIe|RZFqj+M9$Iy-Kjzx~ zguhWkiK{()rd1(6KTRFn5;2@?9Eswsw6gz0;1jSSkDz$+kI_vbDxz;)o5ftE$CsW9 ze6p#o6s>z(Cl3B#3$!txtYz}8um#KW7V2yj&}Dw`3pRmiN3;)(ysaj?!Wp?w8w{Fc zHtcST7aYq1=Jv6Puh+KdiW&9=PdhqWVy^Jp373_2 zUUa%BK4UIy3($Aw`X%|I`gcxmev!N6Yqbuadd@Cj6?S;pKq!TSymQV2b_9d|EfV+~ zJy5gg8xQ}jB7i6rm$e^L#Vh#3&IYYfNb{VeWQ9)_L2x?`E4RCS(Xr5&UTnhfZ2ZUK zB0=fB3v`EDzN`Hnmp$*E^{L0~s&Cf^zrqiEIdyHDo1BSwb?m(>b5=G6t(6VDIYJ~9Kx zz(f2LP1KGkSIJ=D)VtyD_y?a$kL-V~f5#(Ed>#nfT?BY9fL}Lj8A2~b4357tXlAQx zS;&ujMyEhn3M;X*gRj|KUk|rig(s{)&CeG671B&)bra1;==D%TL;I?8i6$b*G_*Oi zcENyP(id}LNvG~I_QPPm`~28PXo_sBaO|2mYoibIf2d77LY8FVQ=c7t&o&eAMz%Zm ztDMPap~q?O-|Yen?XpTW)wujB3|r}km&)#_ndb{6(d7APHf{+L_Jii|Vmf-<1Q$sv z87ju<)pq1`tk=bk$~BqkLiF<;v}`FEx#JgNBz3Hu8Yb6f$ct}3GcT_PiLnZ>$ZPBwojM}#{y?(_ zH&)DRvi8$e2&nG@_Bb4nJl$7Aqn6)xTeYh>KD==4l9z}{Z}<=q?r#A*e}&c{r!T3} zED;vTZ7+R_jD{S3C#yEP>dq*Bvd8&w700r&jG4ea){NGL>>8cB`;Sec!7_|%d1;5SWaL3YJ_u5H8^qxz-s%Z24;412mnLZ-6x`As@oTEQT zNUI~*&Rw)~DR4i@?^4>rELqMye}03X^2m^FnzR&jjq%AZ)QV;C$1Z)yyLfx7#%B06rxRZj{y8*|aXzr{ zWTn>Gcaw7w<9}IE1lu{#& z3)lWfk&$u?qV7emjOCPgEi77cg<@B8n)k=t#>=pGwY+;I7hViEmPymQJk?BYzdHZ9 zmURTVInupV3kG(49bcq)!$7Wa@e2Htlxg;!R6Ud(87yV$flf$IeWV74p5^8*}_& z8b5Mwb(OeP^k79F%T!d(m6;MrKQnf7_k%pe;^PO)Wc~f(UKqV7a{o>O#QsIP+1x0dtPFB;#(A%H;q)OymO1luO=4kmR8rn%abYMgC*PN`oRr_8pj6g zfgth^x$jVIwe3J)&h0DnnV!VM*BE3NkXpq@cypo*&izs347|}TSi#DdX#tio}W|-wX;W4UXWmU zXXaVCkJmQFeygE(Smb0*sjV1iUwfM1^t#hgUb*`@E0A+yRbkwz+QWlZoQlo|^knaF z%hO<|E$Df8aP$*e(xSC@z2b9{zqu{!PrT}Lgu=I3AJBi`$ct#;3)2LD?EVxsv5nrQ zCtw{b&g*R3amLC-RIg&Sdbi?yQ- zOl4li-av>I*dr3NFIroryFuq2Q4CNu$J<1peCc1bHt!TAgAC?2~ckXs{tA-L{ygnww!(E1A6hSurCT0`zKH!f#)( zyRTn5d|t)3ESIb3eO@j^JMa3j(yA-W7V%Is42_jc3Q6?Avpl)oX7LMhYraoyQoqO? zq#%7~8jMU6MFoS31@o#y->DQ1AIF%l_t%93X?&hl(g&Q=_l8q;B>#iEVdzTC_i<25 z1k?#dy-k2$ju)2SaWwMu3r3H9*5U1zq+5%hnZV|6O9r>DRYQi`{igd+dh1qUg;rq{ ziyE@TtueXROD23%EwREVbEV{MGZN3r%ZrEH%7qf_Um)Cpr!Qt8!g`=@Bqyu$d_#$I zVJ%X;)VPxD(dv26z*yH5jRoVQpC!$A_{E)As$C)>X+H~Te?$j?#t8=cJ^Od5L1%A& z7g{*eLsIN!SSI>q(Vl|( zVm0_SgW$T1)nW1q#1k>goBO#fakuSZA5SS*-G?c2C-(>=S;Y!S(iIKz&ZDM|jqCiH zfd=EgB41&D|6FkfT=wb|yoI|Y@K2(NPFR?jH{BmL-M8tBODWU6IcrU&J-TT z`R>3M@?+MSo7dw9;C3pFgn7n7BQj%fa=37ohb(X>7uX=pKuDcAnR64D zG-lWN)DM3B{td>7LQXJz0)7J+Z5OYjSCz)Qp-8B-mW*uc0>LYf(M5&jlTlN(*vPce zH?%X4G6J1Np_)$yb%MAcW$IsKyp!?Ri4K218rg$AYzi!u;xanPGRL>sFFY1TmiIfq z4LA?v9?=lT;-KXUMtxapjPf~)I#2jrFu@_befJjF+A|eb3Y8AmH}_t_25K~ifOM?A zY#NovwBo&$tM`1)O^rDJDzs4doqBCw2-^#9C{y{_09O~W4;M%7Fr82}y<2nF#5zKU zsv@uc4sO~w-16QXjm$yWPt=?|B-O1+gqY?}vdAfCJTz38Sc^5#{N*Uv7&NHrcz# zT>NF57_&0AG`X_RZk*^3!UrTXe@52!MZGR+5m`$r8dvuFUD`%XlXW!n3|YjhETMXr41#Q!W=) zpvoFdKQFBm`g{QCt^2xG$iQ|NN`r+yJj)Zv<#66u zQ=5E&yD1V?Tke5fL^!~v<&!Hj&Yf6CaWJn=YsE7ZL2~52rCWQJpFu01^vP|H-2)t5 z`o{^`-Nzr^Ros^6+|+vjyY^v}2_;sJ?b8*a$Vx6#U>C9l7?(Uha2($E=0fzn;lN8= zlnMfb2KLig4dM_NPX;WIXm%wN;uphWk*nE|={e*~tHs5*`}-}UxT#-NMoo-ibk8## z1>sH0tfesR=cszJ*QD&f+Hr$>7vyVoE<#dua{LVo`Ylm$T$lfpnpmaIV!((AA2AZ| zZ{GS=f30r9UWNs7HVsj#Qr|67XspP(nCjbuN@5@d(UsqN(JQT9mE8(c0T`6(TL%J zDbUaI5&7fRL)9y4G6(YjUl|vZXK1bY^wj<*&D_m`of)4i<4VRiV;k@#i{G|^!GPiR z?isra>CaAJtzO{4JBvVhzL%?Pk*jmpJ!ZH1-T6v54ZqWSco*UXulUTx7&%RmyHDrQ z5N$-LbV&VgFa>X}-6(~%dBw8*crYTYPem(VVGZhhP+2l+@fjEHhVBt) z!fNaEuu1c?M_pnPy+x|%DRoqY14dK*X(s$iZqZsyER(7NB$1x65$_zmEK3s7P@)vk zco3A2QG7C3PWX-@d78sEh_b}9Y?Ga#XJr8IQ#+gtmXT3I4m_`{`Q{P>01otp+7 zj{WPC+?xEb=4dRBC4uC!*Uz6Su4U$$OkZf}m_0PY6BIZtQva(*t9m8wmOQ!NW1)tl zHxQC^%<_LY$BGRv5k)WgGZo|f@BVBIB-6Nxc!jiw)mSQwLDd$ZR&;B4VOmt*XykEq z4>Yx0M3PimL@gWdj(SjSQ5k0t<>=$V6?!SW_Zk;a$A_aEVf`R|JeV9N_3r_9tG#p0 zgYVEB+G&Pm8gOtyAiQYatuwhs^)#PP@~_aJELowNLPyz{tZHCupvv zC0l9nd?No=#Ioz4<<&d#p);)mE0%9olMWylHzz8dg21SGmSa1eNTNRrOhdt)JBcZo zat!0VHKc-=q;x%zWWX_zI0nW12?NknO-bjGC+uW{4Ebdc7tuE*11ZVsaYxO6c)my} z2kKg9MXlp@DoLihFr*^ZN(u$O1tXmmrS_4EkV3lnA){-LJ@RFB9%D>Xsy@h3Ls&%xCPM@Qm7Vc-Wl9j{BlpAPr)N^=^x zvMBQDit}k;aLE_@E-04yl7Zz)y`JIo!`F8r54D9As>VR|3aFO1$T}W)jYO*X(eiZ1 z%yDII4X-hxol1cT+$@~KH@3f#t`E6n@!jU@=&wpVl~K2LX??ROW_%=e`4I*|32t}+ z&5F?+LtyRa=IXXtjWyoM`Z=$z;CesfWv~^pOzOqGF>dMu?9H*tuerVLL%wr;>V9`O z8=r^{EHqVZeM%UYsJ1q5iL$8 z0_6rlZ9U6z(|j0m_DZZ={?oscRcyn_=fqit%MtL@yi1@V$y}_qu0maBfw|S?4Q!97 z4od}BU1{Z^1z^Me^qHx;RR^{AlNW3TjWmpX<)!Wd+Rb8JpX4E}$DSmzr&k%MMkY~G z(d$yZal3ms!0KbWzf%B(vGo9^G+fR4@}&Mt5hen}3~u&MuwHD-45!hf*9ez$Sy;WB z>SW-fom^*Av+x2zyq~`iwE|B{5=dFsO6FD(QhyR#4V*^CcX}UzvqJZ3_<@K!8a=*y zv$ha>0oq)DQxS-(duXq_ULP<|$ziisZRs zSgWvpGeNARrck#=dHgZ7qeFpDS^BF6T786fT>=2d5Fs2tRSHF0oOLal83?PV*1j! z-N;ch4Fzd`9F=$-Mh#b8p2aUh87IdvZK`Ari9eSSk-bMcyNt1Di4{f*CK_0dY)kUL zkE$$kTj0;+M)w5vGP3A)Aj5$X5)QKV}_U2q*wQ#0l#4KqYHjh4E( zeEgOYU8xzLN3oO6JTeC#LpzGXC|dtl&N$TpK%@5+D=eepBdo*3WEgSvsdq$XlFjxP zshu(}>A9BE7d*4XRi`98MjZ6vlovzCCueb)RNO_Ib3SnG0L>0@IL}OTE_A3IwaG0H z?4dL;Av~+)K>v@%?M=FPpE>b#38d`#bzNXk{|_LDbpqn#U0L>i#~}iWiCv6p$kPZE z{Agg4MDgfBSCcv(8(@rnp?Er!-<4mnx^{wVKd@z?q{Hpsl3!=X)0Omwi6Ce-RNH*k z+$~K0(6Qw#dK-k3rT(D`1FP2W(^Q z^!be7Jo|JKxCVt;*S_#}Yc&_xnIAwtH#nJxZrEtrxqd8xk*g9|EfU-V7h`3lMFTJZ zy!h5E8Y;#LgDgE-1f@K5M3tA94C}AfzUSWHhpkP)#4I8|s}gxjPK|OsN7VzpZ~^+vfpd z(G%dOIiM#D)W5-$9j`;6-b`m^gJa`3-D{)VS_?G{jhRdq82$Gy5@p+O;c3evJVG`M z@t&fnjL(v=eiGlk(?}i3h-RAJpOrU0e%3_EGtHwH+l(lTpmxS=&;Wx&wlV4|CZ%U| zhkt|c+g?v1_y_qyA41U3_7W>e1gjn0NFoQ*56_`HS&D- zHzc=@_sIXEIT&Rn1^+GgMiI8{MMPrtzJ2)TtOQ-c`p3B~A#pZmv)W`vL-M*Cpt}Vn zrR{6#PEotGRc}`6N4y9ULeIjqXDX|L}H<9VJC;5MR^pyUB{&`01u)upD5h%N3A2K-){! zyC<0NkRL9sH5kT)SdQSrDSu%YdXCa>+Z7~UC;!YB*Al|oi{(B|@U8{b1KtVns+g1C z_Tis*sA|^c9e^FFW|ksBpK17ubppoXPs#b4%x_v z6e52h)T2=6I}`ACppDkHUZY7o&i#YiroDI!Z{2w$r0Yj_N5|UsmC)eWODY2SYwz9gPehE~L@$L)7mqfa9h*I<* zc89C)MJLgv9|;`Jm9{R$U1Sm9-zVL>!oQAQKV0>`S?-U27x*WGpsSP_rz++`WyS1v zCxEVFBp{*#1ey_qH8NC11(>3&+j`*U!EN%%r!y4}m+kF5bV6T#gi8nYU%%Lv?Fi)d zYL?y|557J0zFRmFC$$xKH)qdso;w*ZYsLtk;$f$~kNT`24eWOUWnW++=*<7_?+viI zQhw^^S(##1HZP!~tiZfASn_QTgJz~QOr0{_Rd9u}A;(lVj>)LRZMy8Nah14XLD z<`Xt{|_$AtvJBi(6K6aQuA6or|NJ5SeBH@ zi!ZC63IpC4N1)}HQ^UKkfA_@v*Yu4PK6GN=;+sgWWsUD9& z6JytoH9S3I8W9UTYEx!{n?*n;^GL53q>p7eEdr8xIn^-S(YFx z$m~Q&S!j=yPrSRRY$uh~fNvIB9ouj#zBEc?!{rkXIJxnhQ zSvc8O9%&XEGDhp<*jCS+EpluXdVWsEf=58w9JPpraiarO4j3e29YEi$J^7&Td z3<<^g?nz~nzm`~ZAV&QB)P7DfvZ~rhF~v+BaMvQK5v0G)-$F5?U6g_R=0a6Zq>6HN zWWE639nuHq$DI?5RIC*5zD_6`I*etm`7&uG>^wN?RmS15t$O5EE&7$iJ7cstGxU3W z3N&XDeaU!NSQ!Ypn2iE=OyhrBj{y6tM)%B-=i$oIm8q0J=UWz&$_*?JK8RD~^nTbi zQ0RAY0&LYv@%r%nh{fu%-V$9-#l9 zALxHQK>vAz{x|-=JV5{Xfna8$|MdnZ!; zTS z?H3Gu6qA5{P}~kWr4sN4{q~-A0cJ$Nfa^^&QPV63W*kr-_Ui8123oIjd?;{1TqNOf zF&a)rKr%*Ob_uB(FsqN^q*}uHG0Gl?UQyC)#f?opL8n-v48fLsBUW5>Nq=_@Dh@|26mxm#;pE|{8sB~^Q9@ zb;*Av?wEs9f?J7XqMT4SGFUAL8CQBX#E};ONR$b}#_VO*5>EKZH3A9{90EXth8dqN z=<#n4Wu5-hnIK8OB}z$##ljS)@qaVGI+Q`hvVpr&?Mi|dgfJu$+KmtE2k{S%e?Y+T`%NK+f zoib8R-y+He0E{5E3cA!pkyuPp?LQ9|-k~kZycQ6H)N^fpJJIqPI zCbC2$#i$@qV~XuOwQ!6Hg&gFezsq}!2KG;X)bDOISS+AU6=fmn%k@9m%e`x{{Fa?Y zMVxiS#%btID2i0&yJ8qX#d3HkFH-sQ6e>fN5R37C!9kc%|;8Jx{#>BtrE z(WN9?s-W^}_N0MMLZS>!XQT$boW`)PX8W%mSQgIIf|6TX0K7oTy&{Bm7G#5p2;R_W zqIbo!%74Wqav=n2-McWFX(-Urd(SUPf!th#zE#UpXq@$384h^g({-?=PJO}D?6rsA z*Z5Cz+>nC`b*K?a4tpzB{xg9N;c$R5s?3MgyL+=}ac*V&~zRDnI%M!BefBx>~E zevw#zkj>?y?raYqI1}22QsI0jKxe*xhgRmq zk5#JQ+6l)JYkL`rkSpa13Kf+Nrgk#{+$O`RH>hf$adK*GclQ{=8|}_?TOkVOj~s9T zHx?1Vo`7sIRE5^#W7>cB;c$rXUFJ(kxKI-=Giz*Nd`p}Oh7Z9P8&QED{WmHov=)hg zeE=+Kr5P>JSWdRhD)nbRPFiWIPLe$5KAHO}QE7xgq8ED6W zau=tN2c}i$#C3ew5`A$ZPSH&Z-AKSvG*k(#d3#=eJk#dMJF^y%W~Twb3hNH&9 z2?paRGv8c$C8_TtJ*CL%-nH8n;2BS22iTLZywGYOs$IWS#eeSj=6P_~Lcf zY~xzWGE3md%aKH51XhkqOOC)~ZFFdzwD1WPh9634xJ_Fz*U3~lvH2E3BkGdo8)Rdu zAWt*FVHHnq`?=sW4z`1m(bgLdg|ArDfJ?#fe0}RRfgnBwy6uz1E(AXwQi+pc$K~eG zaGn;E{T*0AD4SPAoC1yaXKG!-Y$RU}+tHwWy-PR?XJ{$N1mo+yF^~BT2ZGCs3_(0; z%X2W?@{=)mW>y3sOr7@eSOUT_531&eYTA`;AO0@^K^Q8tw%|4*^uYZWz00N4ooS$s z%0g!`COY9=kwbPcA#Tft0Qtr1>lcls@G0+{3P}J-j*XOu1mQw7AZ>#mPr>Q;v(;uK z{q>=vg4uK$XUSB}(CZ1Y%fc(|>$_iel6AC6-`yl2NcLYM-ovEibYjFd@1FBqMltZN z9}2g=qQ%GaCs6run^(Jp`N+5(%1ZoJ;ovs$0r_%BP+~jEs<-)fy=J)%VW9=3(FM3F zS3hj9sA2jBUYyK1Y@m-6?98Tw307CkRFAb5jA}|e9nN7f%xGf{7B>P*80*FgWp@lG z7~a3zL@gkgWbeHY;_V*7Vie=*iJ=@yvA}|Of3_Zlzh{>z8YuW2C!NNHQJJyhqTG=> zDf1+HdL;~)9HSrWS70XiZXV<20mTO935_0~$P7H*a33Yc;9A~N-PA)?6ppc~@3+D? z&-$Nb5_=&8AkL(rMBx`^;mZX}oqz{amAak1!oLoQx??zF%y^Tcj*{^wclfu-KC`R5 zKQd|y{&gHJw~@+1iOW$MoDo@YlQg6@J4)=s8TdV*K0;sf{jMkx4l`4g0{4kc7%#f# zD~V7=%VBk1^x3Hbxogj(-!3Dhc5fI1t{!U4AWj%CxUv`ol!yi^s(-v43f)Rl*~_F@ zk`(L~^hU#~+Y362eoent@8EbC>yVk6n6HzX!%k2ha0y4#Ku2jJ)NaoXIK|RK^^$8{ zZechEn+xcCOo(|)d|{p(4jx#i*z+nL^u^8rI`j>mLun`bScjC5N*U@@t~<458Zw;T9+_bxC5ho4FKU(Ql{3 z#+VmJE46P~Ux2O<@rzXBaG>JV*pr%(HUN5rOyda%>KH6)e*QF z6|YPlQ9?Z9fFqWR(cma+GW`0?(VQ5x)$TRje_z@@3aA{k1K^Qh@MuIwa|OdCnBI-L}x=WM+C6 zC&$M_sDwqnw{ARhZ_U^?(vsKN0=^$B?!Uj@4L>EH(t1@x?Avt;f1>sbj3I%wi9f@p zNoo6re*-7Ub(YweVA{=OLYRThg%gt{3fKx7>Q#a7PcuS=XFudi4Y4JXotK`e1;rF! z{0gyi%!JQzEj0)|;GHnqh)NG=ArFsBP#KAgI1)1up$Yr>Fu{M&>}d zUhH(;1=yG2`i0KTU@bVM<3@ZI(UxYwiWpyUy(u+4&U?LW(m(Nwj}(X6nZQ?{1l%F>q9$hl9#S9uri@*1OTK?8#5s3#T<=m3wiQW>2MPadXE_X`mm%TjN`5D@5FA?yA zCn1R1zx^JMC#i|49_KMOf&<$wY;MsLC@{4^$DO|d!eOE8rCq7?1aZ_%iL2>ZiDHZe zYamFJml3s`LR>Plmj7-Z;K(ZxJ9wg;en-hu4faSJBua^ zCoP~-pDt(*P#$a`v?t0u*S%99{<&`?xqR+MU>rc$fJxD6`b4QhKHI=u2_{N0Y$_eE zCnr*sb^l3^PJy669GgW7jXN2c@e3KJ984!Ca z0;{Wen(M0)HXw|xDB=cbuAUe4O^ysoI~hb*NgrA|`Y(V9w`Ybt#up(0N_{wFl2G4X z=N#GxI)REez?v|$OSf@?x5Xk}B;D|2?drs-J7QxBsAwjZ2a66>PlMM-{29C)2(wuL z;u0A)_EeeTHaY>p!jfcci-?7=B|5jO;k>&9zk}ZSsvD&PPPBTfM_WlStsIh!h*|Lq8exS<=HoGk zS+b^e`SE*q+t(Fxb=P}%jYW?YraD==kFu)BmTQtfUUl&9r&>i%S>~_reb$L$Fx~G5 zE8HGy`hWn7*UAx{g45QeU$S|74%B|v1RX0@7Ap>On24j^`}u~;EoZ#QRQ!mPtv3*q zI*WzfI*7)CAOkgQoc?Uon6Di4;U~rS(AW6>eY6f}P?o}l^~qsy2XDwvTvESaKX}{- z9_XY=jpH}GN`j}8FjG1D*Dz2`z<+*c)Yal|Z0UPJAVV(Qn3xl~+(_#9^IM1Al($VK zTPU$Q-%oH=^thu-Z1QVZ(4gVPnUItr{8y^uqb{)Fzli-_H@O2#J=Kcb?mf^%xHOnN ziV@v*V$`|;2w{L(_&|i|n-WXizt-tI_nt!`{OA-ep(%cQUeajCSy)j=m3Z|Dyh(pR@woz{F&b)` zO2j=|$L>wO^u>g{Ib!-JAl`mR=w>UvCReaJ22emmtdlZ`VDWS~fncq6r^{_dkVjZ+ zanE3-Wsi3i<956G1wXzjWL^V@_01k50-+(P6S}ehZOsHkiU7U0{Sbwy5v>w}T|q@j zr|}2%d}9NTd7mp$YaSA#ynL~&9f2ZjDz!4l?Yfa`3J^3Z*S>dV{EL{#&tG^_B13dz zE$5SIFS%Riv5w^q1ooEpyr56${qr8SleC5Qp0=v)Yv@@HC;ym4$!M;{XFAV}XNa|> zxU_2d)ogA)&LMNz&=UxU#)2go0=4^>IKqtvm0ZsBW7zyMvg>jbxk6lco*9usm?6ky zCRR3{{AyHe(Dp6KG8o?tKKIR*!N)}3m|4g`5md|c&1dj4HbuUh<`XBB!i`QB_v(ZM zrFQg<`4vLl)_LESNk~HR_RhkEJ7=!KrKT?L;Z zOIMFnYnP>9uY$I6RQl-%?`p=&^BggJk0g z>c5}=fH&kaIppqy%0$ufku+?uW-uRAoQtq&E%>gl`g%Z)4Gl;rwe|3?4KTWs3IHpa$lGxOx6VcA2HzW!CF+p^5%yzRU)E zMEQ&u-=UCz+exFTGj(2=Z-zFzC^XKg5*|v395r%z)ilp(y0TUS^~{}@mqt-W=zq?$ z=thKtL)dm3Y&Gjy3F!CYPT_LDL$jGmwKap^F~!{|)zLenp#?CY-&`}sC!XMK$3PIuf`ZO(yaQhFmc zrdTt2o`Qv2ucDCfxbv9nGipMik2a?~W_d#{X~|3`8=5tatOe&1jpn*K8U)-M{(Y7| z;6}L8Z@&}i7BK;Zzywm+oz=A|=b3P;`rr5ipt}2XKNWf#kOPYEXb6}t9k-{98sVBt za!X<%MG#yQRL|APJh@_P1XS08`>G-*45(jmIo`PYRgza3o75BGyUd6NxS6Dx19lyUm zlOpfRxWiJz<-Wf&PU!LhZ zE+1mpx>s*k8@%ZcUjln@aM|TZd>M!trMp&KNbHBDw1yK~DIB1;8~U5a=ky=HUPZ-a zX$h}TdC_}aN{4A=6yW%|CD_dKs%b-%1Dx>iE3Jh~Z~Q{A3*rNSLh~p}9Qg(&j}`xL zBM!(`qZb|45g%!Q{4%LdHh(Z zLNOyGH2U=TjZxK^D$})`kFI9@G1q%C`mB!)K;&N>s2WKJS-JLvX0q*?%Zhcu>K>m1&Fm=OZzV zS?~0>>}|nKnI_ClUj4I?5PM2Z$gA9{W@QW~!Opnu)h#T6Jh)W48 zY)VBMq;VGdqvglxq+u<63Jjm%bxpQPqq{fsm$6vTIE_Kb0y{Ds9&7aq%nF@o(wKTi zDwT6a=)4IO0GE$qHO-QW!=O_6exKTqgY~6;4o5q+X#>U6$R{||Va+f`d%m%NA@|Zc z=wDKVT#DJ_&TbP{yI`CRsQ}ZNr zL><-3YHBsjz8!XmNfcN4KM{vRe6p7o7HcQcFtGOdI~aUTni8zd3;PBH{;5}oW4Z7( zW5H82EU@*I0bqKe1 zP2KsZR=B@X)2CY%BLe5dd}{%`CIK}Dn26-l#q-{qb&Un-m8W|-T70HbM^|zxIzF+M zh7H5bzXS-pgG0-rLJO3cYbghNq0$`lD~1!5zrPHp!1~aNvyp54<05HN!utLYs=WPz zAsEm4c57JXz>LD7cM>@$S@iN%I* zXIH+JjQHV-VJr6dDe$;OoqrSj^%KNWAKmb2>6~pj&q>dnl)3rP;fi5-0<7`B{U_wtmCM87 zAjZ;tN&e?5kvtGv)^DU!pkmg8Olki!kFva5&9UBmi}== zLRM>?Ejc#cGXK=>%jnnMf1D3>bhi=$RD^`0g8Oi4lD8_f<#?^L6ppwHhV(OX-aq|p1-Wg7m7-NSUSzOyZ>*qM zGzx1?{d4z|&IdTu`<|!6`3?My0{1@O#|2j=a5eEXDMrV#+wW!E!TVK49||C-yj6>% zp1G$aJ6<<>z4+ZyIXk$o9!RK5QgYV&4c%K>3=s+f#Y#t|dLmV}8f(&@c&sNoM_PGH9 zC2p$KObkE&14hnV+~iRdPgvdSl#TcEfxuPAM-!H64|#`|2W7U_XSo6dZQN4_+}hy3H^W=Cz6l=x%_>z8O-TILfUbn9+}0u?(N;Y*~D`3Xe$K<1Qw2 z07ie4ktB|higOlBhsMN-f$~yqsrg5elFv6QWNu}I*sCD5uq*rm>LNj{+%g010^r$k zzosy6GupWOST@h^ZRZuHeuA_}?}T1#M0q%rk0;jbtCfpql@ENB!KwL>p+6;uHA1c` zzE4;Fu&F`UxN&S+jD+Zb=f{WUlza2gsG;#XCGfWyHF(suh)F;zln7BA^5Et#V^; z;fk?pj1{e<>%-=98nK1IYJ`e}(P_o9fa+yR}VhM7n{{%^-M6-NLZ$!p3- z`7*JwDB$4Tf3+b8rtEx=Am*bGPzCUW04 z1@f1_HdSflBS_6hzc?!_=s!ex6M9_;3!*8Cj!~3313KC&CHaueQB%`m-+nuMKT7mT zkr(}ncE3Lmau#$DJQZyEIvzT`6MRwGsQphk7P4U`?;;32&8CY#Dd}c0-3S+bvGf(0 z=~MH2;OMGiKnQZX5o%yN$>Pl-7*d@B)(Q#_U4>gRy5bgx=+f_bXfc;>+;@d)J#=fd z_Xtd)CGMI%y(fDWX+t6!eA;yW@@CH1ik0L+V+0qfP^*?zJY+Wg;sGlZFfy4fLU1^e zWyWIJ8N3Jns??707>IB+%O%6b#;=h%af%HzM7E7u<{&lm zRZS#uA~q_`?x`G!%oTKINY;b$g2E8|f4Q1n zRy`W^UHi2{0l>5y;*(&wcS8-x#!-S?^d{9jYl$@;_RlNK4_oCP)-viBewq55yo}g+ z_>{U&Z^=Xg~XdX!}_m2HuUkPANG+KUNT3)X-OLUUDsp{gsouF+@orb4%rn0h2XUpv)4K4$M|ChjGy< zDXyFEdHCk(mjoD(v4Y31^5p}BAjoP2%RtvkW;j+M&%Z6w=xIdD5HUWYKR0MH+0dbN zk|x$SbCK-rj<)%Ffj0Xgk~XWIS!%T7igL}A<6r2ZJi9BX2(PHvj&|!;KWz!(saDjz zh5^D*FdZfK!`9ZK(IE{?ltI>*D?#kIekwf*bFL;l{lS*FoXcUa>1Q89!^5jH zR*zk$r$^|9W;P3~-eyIV@Xlxz$v)a_N)a`2Ggs74-9n5$eDi`Lmu-s>wL1@1y#><= zG2YJV={gl#MZPZ<_m@tzp%(KF!%sdPpL#J&Uhg+GK6_81sZitl0$4_Y9(JvjkLTT0 zvEJx?7Zak2B>0Qn?^isv6^U90=035->K?Jz8E z5K^xI*V&dgO1{z4Ye)2gK>`Mum^u7;idkYC1SUFedui zbw3!4K@d~i>adcW#Uu4+5SXic)Uy}r#o8oJ#N0r zW*2u2e;dcvc@r#UerACxi-DJCLmT6Rqb8I%Z}QnU(ChYB@1b6{rN$2Ys~8PY({FeT zz^-+^sL$iRvyZ_7*cqXX4>tIrgT6{#b=QV(oPl#A`pUOn7FM`wdXh-Uyc=A$pQK2;1Wq9fZmpP${$s z_{%4s8j%gTrZSMGujoGB} zOxR0(!^#eJEo7TlU!!B_*Jj^Q4_R}x#cwR=qnD{*P)_f;>{xG5@aY=k zCJk083gz0M0yJaGqyL_F(|I0r|&0sR*9YJnDpouhlWo>YWuL}O^~?Juxqi=pJ-o53mj3=i z<8)iQX_d{g`IT63;YdkL6RSx5N8fq#u7(u8y=oeZH~{P!<)Hh^oFgHf*JA@MewwAt z=eWfu_VV&q%fLnV;HhLHRvD^(3#M`RD)L0#V$`FPXc#Mrv( zxpfmJ{j>^zGD>reZ2qQZFRwr2MNCVoIN;^3k@9m2GoN0#-8b9)Bn~Qa5_T)6{XZ5l z22aFKkYt8CPjZ@nklZ_+>lJ&$g+ev@e{5;$8}a*IR6R_*umL!~##VP6)DQ%&G!rrG za#gtn9_R2F+3J%zUSd@GZk`76xC@cSJPUCt&HCrp$|%q&l}UA(LdYKi<3MI|MjMR# z?t4vOeGvh3Q!^n#_hFqOBGbIoWngi$&1&oRRA!=93v6y7H$*ZMxp{I2j9!v{y5fDB zrkyM3>{U;% zT)YkRI6S!+dzI{m8(%!D-83FAu?%o4{_blR(oCtlN)e% z^_Ht&U+{D#Iw+vSXdPMIVRuexv^CRyMrfhxxZ2+A-~nR9VC^tJ9~kYP7&)c!AUv8L zp#RBK)^u^)aWIqfu6vWNywYkS1<8GXBwzb*ycmZ3d2kf=9PdBT&smh`=1tIO&Nl%N zx)Fk$T;_Zu&3pJfi^wP4=uG=OSm<}r0&X}Qasze|SOY&U&)?zJmOKJp7oSI#nhJDp zZdpA8tCAQ-8weK8eU}Kdgg{QZ-ydRo{za&Xf&bZM)$aJa1+?#XoP0c-*sTdS?mgZ6 z5p1&W9f^J-XpP(MbNaj6lJOetK*5Q=QQLvSQb54p+X|Q}{X{XAm8d~~>r72F$Qvyp zC61@>!7KMdcTs9pLVjie4}~`*C1j5V8?_b{N5LucdGPOANfnki84T$WUzO7>XK6Bx zEc?(EmBgQWd%yobw?%B5V9XDoPx;3x!>y)E|wKKLdJ zIlxv1x(tKhC-Q!3JvC8}MVP=fYn_MXjp@tI!Rqd|z34TO_QObZ0ec0m6ndnDf13Z`q1iokd=ut;InF}MvMqR#^))@{7M{Km z@^|YHa8>Uz>NB|N_7TYzuCZ;=sx=|TCxnrQg8@oB-@g!i=g}#)K;hHmtMB^YAEMhn z%nAP4R(XsTA=I|;!1`*+ye*(2`zNEz9UFghIYe&G{F4K2F?j#Ttws9znb1Y`vf;tK z_4bbgM)O}8SRa#xps6!u4Bd);9{5<(%!xQNYYuIRUd48&ldr$&6U>)r~sg}SccSX@+Tf{Un%4X4t2+` zlM8Ne3pV+Dd);arx|`ljhI3o+kNwcSmzO?CgYb}g7h2shWFvs!(P8l$^48$Z(8LC@ zEtUQA{9zHe$vucm7f$nm;g-ue+wUQ~Gsd#Lrk@_j(Q!mCWlf@Z8X|sWMx2fV0F9%9 zazwBqyZ`N~T_qa=vQW3B-@t^OcEbXHE33vJqZ3T{^hP?xh?;FsK9$si^uTK`){l3u zkIro?k(b<)#YS?%&Sz|qzi!{4y*4eCo9*rBMd{ra1{VodohPf*d+ia@hF}4;4%uAO zF>7ckh0AYx2P}~rhsJoH4UIK{)0Mf)+<(5J;g)bI#*$q=52+pe*!HQ?Z{OVHEIc*_ ze5wRVZ#&r~X-FBNbwX{CFK#JDKHoR#81|g+s#3_y9Z=o9n$cMx-m}%&Kp}5;?djL0 z&j>1PmeIi)bLz+3Q^Ai?PIe4OOX*SEJ)?3_S--3>rtH`ftuqi zE7#8m94IOU*!#WY?mTh0Nbz;&P{S#%_ixp47HC58Hp8rW&TRh7ved&D3>M|by}M0A z&T8Er4aG=BChpnY&zzRRd7t`tTt!Jd+gx<(nx;#O@fk4BBW9^ zcEWkieZmtD=NlH`pQV`DIyOX!|4Plucw$RdTokj2q%;frtDdG7pPJ1j?xUWp`)ks_ z>#-KY4*eny-jva79?c-*-4QIEuj-e3xbrXM!mj!?y8Ye)}CEZGbfl3w% z=Z6`vV4Ta^Y-6Lv`cbZ0%)ELfU#x z{eLY>acr{482xU`mlNBG38=evQmPi8UA0wjVD=~;$!4FB$qJ_7;dIIh^q(W=9it??sy^cf zHST|&Cp$kD==4O6b((=nu;eU7x*e>i3}pXi{i>>oh2=kfeOYo6`7GCIHnBkE+KW2~ z&%Kw$d+*ht3vl=-s~q*tlB)eY&qlBE?R=W`ZU0s(I$l3lz;$!#&aj0c#=54{?_C1> z;_%&-YpaeZw|(@-d-nP72Z+Slb9&3!MQ+_$c*1n|4Gv286k%xbO0olAEsOpaT#f6? z!xQ=G|0$~&sT#jG+~@wu47glCL4!uAay0PGx5xI4d+Mrhx)3&bg)hQ4&!2ZH|6sG? zK!fW^f2f(Po4<&VL_Z1Z)P3blXLweZC=P;vCOS%Nz{f8K0h8oO=zu-<#*_1$cV{_| z?;m%U`yWncg)bZ0>g3<8K&K?T^iB1d;@)|5Bs5_rS7T|~7f)mH#>inm#nf9=N7x(| zuZLMt1i6f!Bu->TgM|l-=y8HRO`v1=k9_g} zuyG|-8&LJhstC8ys25ra{S#2bA zhZ5_U_4-I+w|0Ig2x|Q_W3}`nAU|=1h5Y5#&-aNIIzIba{onqIXcGK?j`RZvX%Q diff --git a/images/square-logos/endocode.png b/images/square-logos/endocode.png new file mode 100644 index 0000000000000000000000000000000000000000..a90189d6f99f6467accc7809ba304fdfb8a0437e GIT binary patch literal 5231 zcmai$WmMG7+sA)|bT+Gz%gwB})j>E!`#Eoze>u(j~c+E}aS@NdCF+ zbDs0RpMEj(Idjfj-mNGuhV;leg@Kse5bRSs!FjBEF9_F3_m8Az@yQrAB z0{|}RUjqfm%%%bW9BoH1`1x}Os0Y;D0qR1p3I@}=xIyh5U)uqI*K&>?L|<=@Lh54e zm%MsJK(abimmG^;S3Wv`EP<7i0h>rIf+2s2T&I^v7iMq38=TP76X(jNrkVuWLM11MgzxL5#OGYif~8-7o?FQ4%OB_IR^?vWs- zjuk)#$oj;}mjJR#C;>U?ta?BmI$$|!V~qr!a|4!a@3&`xfSk(=UlhPBnSmT7F9Dz@ zwTn;y93+9NNy8{bz?cspvsdpI2Nt;jK2;-o6`-yafKL$NHUL<70H01oCHxp;~3i^7MIU zPzwYK5pJA~0YF{?`NM0^ZoMW5Y9=N`Vp|9-xc7T7?isDEk#}2@l`gUX@Xf=2`tBKT z00H)@iLLBYUV6bX@M?(dT)lE<+Ppmnq4s0xy24;!ze*Kr6-av|X9 zGx$UxwJYm?;*EG9&bop;rZ+B$}CiID$-vQ9VIe zgkO;V zIwH?)ChKA7B!P->Y!P&rJiXNuITd3KW35G%GJ0BOA>w6hQEapjojx|kq%!p;)^_4Q z{Z!Th{Nctb&zR;(;z)Ukd?Ud9qRb?bU;(zacvRK;BK;YX8Ks#wx=ee|%Cm%29y28k zFWWjbV2UR{qv!e(x|y^2c~fwcW|QI63@680*0tzT?>oEh$SXDKEut-o$xYJp|_dhc!OsDYK6qC><^wYj-``iC0seXr)aV z&5Z6T@22gd{et1)hmi*lML^i8v#2nsAk;9b+H{Q)>e*=OA$>kk{)6$@Vgs$aL4sY!5B$9rn_+GykYHvJ;N)95f|D8g2Y` zFPth;(^8EqRVsBVl@Gc^YK}^CO1cy~Os)e7nFzkoM%wx(rpKc?_GtylSLgS7)8%{xrIHoOT& z)eCj4#juJCs}oA;-_mX$k)BI_|u;~@B`?|uFLaM@A7KC40yp~%{ zm{Yof=M`vKv|XrOW{}6;WNgkj$5{2XI;}VlXN}04*WK$kM`z*RbH3vueJRsL@6mm<@X-dR9OqKiK`F(KYnzJftBsRgnR z*|zr_t+ESkxEpvA-OBqU1#{S`F}~`N2{%rQe95l9DftoVVoPzDx)C>>@pf0$EAa-9t2Y7%N&bi{%n+jr^eL)^(|Jz2 z^2jKy#=nczN7E_rW$#E(Z!?P10j>egR;b^>%EFPkwlS~Cdb8D4SPqRed1O=#8GlaV(C6+y7e5<};&OY^|gV6E5lFvNN zRLauJyf(8oYu6r=o}S-d^SMz*JSEH)0-1QLo#_udF7E}+CP7XoV{%e*{QPJig5+-be39c1ljXIQ`^t+JM+1HTq||ItIPW*#r^zTuDH}{*O?FWV&LL@onqDBjqP?~v||az zc3j=5-D&Yl_DF6Cyc#|yV|KT)c(U^T6&#j#Hoc;Jfe{)!d6#e=M+kltmk~D_h97p2 zD<&;2{$0lHcJHPXJZU@0noD_Ca5q{*Nx|!XId`xyF`Yi0pLUv-)Z_fF_;w$DKwXO% z>>GU2J?tgO|MHjN^-(Wu1x9yLxK60O`HucQ`@O@T`g6Uh((5GZq@;UvwDhwZ8$oWw zgKyH?sOV|}fDa1*1Ox-XpZf>y0l-^60NA$#0Eu(}poAt`3@AOQX{M@zoW9rcVU};a zp&i|jjIWJRjZI!O^3ecao0y7~szWRv-)&K9CQ8)( zyYN5w{|Ef1p#L-ApEntBO7*b_ZkdJnV~DXSAHKB2|Dn9$vw@yRGJCuZSKBC_T3^E) zi)*Sxxx;<;$N@DK^+WE(du%TZFWO7esobAbC5!di7@2_*(QRmpB$V!AQ>J>Oip}_6 zjZ2xbamG+fOf&*+;Ubn?KbWU73ZT%s?i@$geI~zaK^L)a?|IR9e)pd=_)FOh{qd0R zn1)jZBHhK@>L7^1XhC)C6(DAW!;g0g?hDzp~~^}Gn@_$pKh zkn_6Lt@xyZhBc-O&5t9pzK_n7wF4`f)-*rpUE=7Rk|_=le`JH%CLV93rX$T!H6~q- z+XFs`c!Q|CoiTp=$@cd^i9TEDw&3O?g_XXGBSb^W_dNaGRJ*x5$}CwP2awW^WIrjSAR z1z_xv|DzTC?>Sdf3C4Pcx;R1vm$GVmoZ`h*rxccSd;;6eDezMFeh{WHVJD9jyc#2R zn7EbY*V)2%`;_FNjwfO*8+kRSCLO+a7R(Y)8#BrUuug%L<5bFJRlRy{gCj?2IV`@N*2Gx~&d+^g8FZ<#1RH3hCeL{%D$|OeJ3C&EMn5WCa zWxtOwEEVJtxZH-dOly8{!ozHCU@dFvO$dIfU+zKu;)q&#(r$sSG|J8LEbFMR5b(5B zSMKHVVL$2$l=3<;q-FN-I!mf?E+s!owsO2mm8Cp#RSx?|GuPHxTn?XdgJGis{T{~f zLoVr#4K7OOdS~bKGxm9QW56gKwRfX|;){hQpF4k>;rZQ=7QsMF&_W^krBf=dvHr-x zVNw688yDP&u8=u$R5v-~-N9y9{2~pVt_WfQUAc3}=Bja;VT*mkLwoF==qoW&jhRq^ z=a;SadV%m4d;$(-v{EmO+q$VqUSk=Egqn4Cr5 zshyrHdtOIW9V#^!E97UbJD?NqSMuIvBhnjNWPVQ{#f&euu5#3vK*P>1vcV(B9}DW{ z=Ln-Xg#!1*#wU;Q2)vHTY+M(J5G3|Oltbe4Z`0!)`yJGX*__suAR5dT9-Sf}fkyjc zq>73-+3xR^{VdQ}IGG$NZ7W?z5?$m4n&^iMP2Sg|mY0Um-Cj zUGZJw*JLH4%Gt}P!9q@J)1yH035My26I`Qe#2?5LMVd<33Qd*$nAZW?C62k2zEJxRx+Wn!28$C|4a$yUK z{&uLyM(o2z#;s{qeTnx~%WRPcizJd;D|o=oJdK+JdOse!HJjT|+!)Bd(n}R+$>_nHSLGQ)6 zWnF$=%cf|5Th{Ep+Re+hX#Cv_=4slg2Rb@-)zIKw)Ghg{P|B)L5#c|VLEjx*iP^2( z67+_;DV*o>B=UV9iPXGM@Fman9pO!!C5;qg4%SVmWq1d#m8RbQAJ?#heH+qVaKIqh zT-;SiZ3$C?U+M5z7{b}|Grf9SYImtrIJU?%#TFz1X>3?$JM&r)5k+$KJ34XlrzSNz z>u9DM`&W)i%;r}_>+Z0zqpMgvK~>u80L7HM**zQdkD?=^m(SOad5CtrH}qI?dnB=? zQ~BxfiogHi4TNBR^YcVGDS#Tm(&JENMu^?BSeQB(B`53Oo_`R(empTjxlQICe(nrW z-8u2!AVu>IoKZ2c-SH@d_&g;xK~`%KH%dc-=Iwt@zGap_LPrczENAj*ju{v-E~~jY zADYr&nd$X|Y$F%sH4f+x{y+8VY?+f_ao;}}DbIZsw#_|KE$ z@>9Q#l%5ZH$D)yzNTuq573W(oDwveOm_$(K;U9Eg*3zlgq+LMH*}xm9%el)W~sMWy*tN+AjcsQk#MH^EU8zVDZQ7QfketXnSZ&&zZKvg z?(omAKlE?;c#x6*A|Zbl{tp5F-D3XN0Ik4(ocXu#{BvtmVfH;fK(##2mrJfL_h58@ Ns-l)cjl5;hzW_Vd&9wjk literal 0 HcmV?d00001 diff --git a/images/square-logos/giant_swarm.png b/images/square-logos/giant_swarm.png new file mode 100644 index 0000000000000000000000000000000000000000..6434f98735d8106dd8c799ea4349d072a750c399 GIT binary patch literal 9772 zcmcIqWm6nXvt1x~@ZcIOxVr{-hs7nq-B}23!7aEhE{nTMNN@|XxVyXS<@s=b!rN6d z{b{C8cg>vc>gotpWf?RSA`}1sfF>s^ss65u-d73|!u#IcC%f>jKDdg@X(GLg50Y6p z06+ndlN8hR%sR>T^3;Sb^!T53f;e=M+bIN5#2RKJi5;B9V45eG-76^`H-O7}w)Rk; zQo{yd6wprXa|?|^EOO-PTH=*ZdVSa4 z@?FkJ4!qF+RQ$dUf|z^mVh;8Bp`nbJvlfUg%EcPcL?^1%s; zr#zw9O(lu>x_J+RYTDcjt>3Eo!4BszUk64ZmVC|cKnYHV-QX&DSM8q1D){>@RS+?p z!JkB%8fXAfPrP>eM5|#!&MOWWi%dg8PvHlo5Fy!;@#jt~JTb zU@ISb5VM|GWwaBC9%j_-()xW*@~7RJ>j9TLn#{x&>*8*odf;Mz#ViogyI7)re(kxR zoK4cdI#3RHgY|Z!nD=jgB4C_Gmexx#R%z-{%|#3l*TqO}botV#X1&@&k2C>Noxy0I zb1hv~@XOS;lZlT{Rg)zq;p zY4{-d(#MKJ*t5?rvQx+qgaB+wX}f8af&<%<#I3g3sPOApwp_ zx(~!pe8Efz5&J5(5lT+t&4P4Q58S?RDfR4&zLvS6!ly{efYP>OuW}O+ScyyV|jr`Jb(l)JrzB8^L0}ovMg4Fs7XqF)^)# z{-&l$##s;FLLxZ4>%>ut%AARR>!Rh z*`+$`mvPMs8L{V5zgb48u+HhS<`R&pow7YkeRhwNZ`~=Ss3XaX6Wq9?a#cG#jL`Xk zgREf~Pq=8HlQ^>XlwwEwW1FE=)e`?NN7l!$)k;UKiC!(tJo+VQ0=?WF0kS+0mD?`I z7yhNlyj_fA({zKdiSh_9MkjJMOCH3UfZxE)MZ{gmHv2x@wT&gjew~p9ct5^$ze)E!N-gaCDVEvI+w(mV#c8)@I*$lV&a5H%g#0zI&8!^tkRv}o zrxmhxHoWi6Y(S06>x$~60b3H)zSb*Io1)K3=v)YSQk&4VEwBl_d|TO4HPx-Ani80^ z6ug&?>EpE%rLH7i)RAzKU=N&l3!ViX;F10X$2wO3aY8r^Ei@0EVtEMZiR7I$(I}VE z(i&y=eSPS&P1xWTZSec-k%06Ss$^gtLRfnX+da}1D(e2jLLkCWoW9&|Vn|wN{--Ni zjpN#yQrcgK{$HEU#u<6?wUEiHo;HxMyXrDC~@on2{HHjQ;p{QN1tWk&;^d{jJmh6 zbi10}hL9_AzJ!XXx2aQ02Q)6#r)yHKPr?iIUYF!P*y(#aRAq!GkCncMDl^^Ktq8g$ zw_YYYa2YDik+j8tr^qS62G?MYKu{LN1U!k6IQ5Szt(%i`act<6X6DQEYtN(dkmi@m zPN4kQ7hl`pn#5)d9iQ2|yotL>>wCtj z3de+$RBl>T1Ks^avgS3~P#0cG*hix~0oRaag)I-uQq=h}IMPl;{}ua>va@fq zL~o)_8h-g=C1i8_jZY`LFzKF3eB!&eu-ICIW1C)ar~Aq^o0S{V9gaE z?7+*Z_>W(~utaVp36AlZ-r0A&skA)?=24~Xd8=1h<2^z}kRWpybBUWugPEMS$~m_u znv7f2=BuXbogP%?EKf4W z-?6s_S)1}vAFev(tbB&7(jdGE+xSqMlE`_H0~*nP>zFWQt=xWnwst22Ch=%o7&wwWOEe2H8qt^=86g#mP`9KGsx0*ZxnJlxM%N2-tA%ar@(woW4{ zoh$#ajn7qGj8&w~K3Uq+7-vsdZMn!?X)aG{f!$9AeM{)fp-NeCBe)*|Db|1G#R&bf zLI#d$`GPc*e5DQ7EX$NQnwFyLHY1Hsc`7RG?wV(mT4iGVYuf0kEq!Rqi?0;`T7oiA zacJ)$CtY3^ZvER(@tT3$i@qZdSUR5I;$cW=PvAe)W?9ZOK8e@=Xftf&Y9{q`$ zNhlxG(qfdU8q(pjo&z)Ihn7(Il>be$+#E0P-y3iZ820xppNIQj&9fmKR~x6 zM{f6TVl4;9*7KwPj}PPPok&%TtHnqj6DzWU&xPnwWMT8U%r3T~0ju>Rkaya}o{yEH z4kIWkwOb6J`my?S^H`~k#y^U7F8;3wz$aC*S-wBS5Yo&@Wc}5wdp1{Z{`tVe0qS^P zVeaf^IaS2HGh4o|+>@^I@51`)_>s{#eakJ-0MP(3+L56@HO%?3agg%f;)cNxq+abk zQtq~x^BQn5fw&fMV(?Xenlg%ka6`n1B3{*K?X=Q0`FD(Q{mm<(-TlK*(nL#ek2a0- z>vXJ8<4RZI+drJ1)eDeuu!C(!zfCr`T8TsZFoGpr;0%05+_Y(bTclAVNtmth#{hEF zt>f4uLE^1U5&4sCT1IAvJCS#;AlsCs?J;^$lAmT3hec_|) z`tvj1pN9`BYESZ?7iU6vW;8mTM4+rpCg-Xw=-~Yc{ZmjOLm7E7y>NM-R+H8 z!vlLwj7`(kqATNkg2OB=&cX!kG{ze$!ZukIC)o=ce7V<|S&X@!KSvGc077u88zaw+ zZNGj|{C!HAyKh^wwMaG2d}P*{&@8;boXKi5gl!N@9n(lLgg!~fGcu*kclwQ3?i_Ub zM_rcV^~pwy1ZK2$S|s5gHVD7yANm|&E#wK)`vR-a<*M5WdL8OhOw7$1 z3CEdPO-IJM^DeGtLuKK>h9N7$KSGywZO407lJ0He?$!6T&-FZqYc8ha9&RDGB_UZb zaD?aQoRxSGySC#GiqKiS%s7z$W;QrQd1#0ur89U<%v}Nu1!E*q2I*HHxYt1patG6w z&W%4q{Esh`77Tabyko^G{qvq{=c1?DL5i`!w!(*1mxXoq!(y;0w~lH1oM8E|Z!ijX znqO4g!2*$DfqkN1Y#X5rz^>|oG_U`(psFu6?ee#G9H2%M_;c^^BUguZlB!4ZcgEzn zQH!mj#Qaa#J@QfHwND7A3%ebK)BaX7Bu$0Xr9Rjc^pjU5O64N`nn20Fm0-A1Mk-K) z&rBnt17X|UZ-4{Ei4~5bXnnTj%t{L(DCva9(|N+=K}KXtbmqe}3v(uqr<;1ZLJ*5} zm%TfYuhLDcZaq(Oic!-5PALg#=S8mlhdT*C(D;Uqpiu%b`x7jqze7FPUURz{n(zk#Q5#6-%^jJwz_r$9Rr%ewG==*+f=X& zahANtC40F?YWtrH7{R7_7$UbRHX+W5GVMwLM#wYw~qD|ZB$ z{BEY}Cm#`d&GR0m-z-<@C(BHI)@EJA_mZ&a0gGnvjlq^|vip4LJAy2djE0b;em+F^ zhJQ@%5>&{hj>m+e&b5*qMWfA&PU@Q>oZ>Vlv!jQVGluhv;0681?2E| z_GbxdLjjets%Z_wuvh|Z-#mH>bAFpiLTRTenEkioOqFlc46BeE6m~WJ1$rOzuTJXhY`1h0dq=(E0cRg z{RfSPgv$u;-qEfnW#mf8L=&Hr!kM&NM?o781@?44?Ulw@lJThZgdb|UBbQ>$AQiLA zE|B}lOyiTjv8i1xCpP?@PAM}wr?&c4&|$HB$uV4PEK^g_!OfDY7c^@QAinC)DYGG; z=O5_P!F^#+NQG$;+e?cTVa0K4a%)*%%u`5Kk3Or6&B6QJ57InX2s>E?8L6Q$)U%0O z5(F>Ti=4$bth__-B6wOHgvTka`#DAIuk^CyNSodK*lbi%Wu@C5#SU9Orz(6_TeyW* zVfZKIs02&rPU|`!y|EZwr=_ChI_Xbc^rHVsg^L>1-B3PvOs2A^NejW=zgSUL@289) z`+r692MFdKVQU7B*~PgAaPevr%>9jBr-h3yD#(v-=xin%<^I}K%`z+*Z0=}DFV+G( zpf|#1wNoo&Z1(;S?Cy=4A_AFX-8s@uS1J2^1`lJW z6Rh4OxxbysRJT5!W^vTc79(7H0OsU7fzSl>*}6lX1=q#V8IP$Q#tGgm4>K6bM1fDT zjgxg|-78>bz1G*irs%yJ++vI5C|?3gm5FQ4q1|v5q6hcHf%;p|eCTC}r2qZ=b2Q@E zH_N|stFgQD5Z}H-NV8ij%sRSkBM=@n5M#U|7zvBn%JJta#K@8kZSxk zFL1OxF%#^Sq)&Y`wHK-s$y9aj@p>L{VmnDj+Nu*N--i3Mnx3MnyNpE~3_R@Hyb!5N zUYyjNnMn5EcvN;Pt!&mD2j=O$-MCmMR#i3J8&SF?1e}}`4(r4SiD?HraFvdfg=)ig z5Z*g0k#6#pUQ6q#cxxY2|GO>5^bLIjMRIX%3|eGWMgjU@dvrz$)SStsIQkOX4VPdH z65n>u7hoi1?4HY)(lXxNEnZA7QwX5?g|EUJ<+RjueVoqWxO(ddTtpCb-%5)m+*#qiBAjm_S3V ztd?d=L}$`#$YaJh8c=$UU>qMHReXi!DHw+v3gEMDrLlM22tVxr zN=pAd?1>TOG)TQ!G+4i^ry9S~rc=;tDgQuB2p0Gi9UM#2a@_~DZ>^pXHdomyPyzvx zxFw8q=Wo4EvPVHKc2T35dy7@vy1~G&hQvX$uv?2#hl^Lx#x94Zc6WUgWVZbsDX_^l zvs#oH+RuSzTuYt)oe{X;)Yl`8;2X)hB{BiCzj}O52MTA@_VC;rgTe8t=8VO){ke1>OG{+5A14pu$dCOH+inX3+l?A? z)L55j19O9RprQ9v{9I1vsvpxjr$MP$xu)TJ>#7UsjlAKA+p5y)#h#h;au}5~qnDuV zI+V3>SN_@(T2ZqlK zdv~5MNseH);MYRh;WK~28m$i~ct>JK;p5&%#vdLXCGkU(J7HMQ|1edJ{CB_6oYhvx z)?o1LuZNJu+yv?U7^vah?QCYeJNy4e`!ADr=`{)Ev>Dd1##o+xk)UgUPY{sEjE44p z7*Z7XS7~uKy4y)0f7)t+<;${uho^FW@C}Gwngq1*`@3stDLRmvxB+3((P?KzOChWS zVS34E;V2;<9nDRe)LV?zy`r5hE6hY+O@KKsuj3@1)9-Q`hS&UXG1XFPb(GC0o@^sK zUwRs(d4KjxZHtwl{JSU}rU0RMA;0aIcFcS{=WBW6&!I$Ze7uee(#W=vga&nVA!=47 zQrqchdNJg+YG5sWU7_AK2vY7F7&cPj2lS{Yv~_ z*{hN4Tn$T*{xBD@S@tG@aanU#4rF#;RONo;a8L=l>T#ef1R=@93HVLDR`TeT$QLmr zm!=S=f&^%7@F`}^uw3+`L!r9|y=uQnuJ9vtttuc8dG{>i1g?K4g4&NV@>ASJy>F84 zU$rtdcWk1C&K`-FmkJeeHh2?^3`U3)uMUv=LgD`iN^Ooyd>|!Rs04CHOdv&5?A6fg zLTGtiyj${Bc)0e>k0MbRq)B+KMMs)mncoAn@sx6R#x2KK6uK)Z{B?U{lg9Pb@j-Yy zL=IF!$S6+_uK~|y^Zf9FHa>=x+0BECKaRX+DE!=Fe657FXM>9_#YMVfi<=^F(}0s@lnxcG zfhWYC5A*XQHT{SHmF06G99DT-i{FV`WaK8g7>30m5?B(fvmUwJR#ln{YjQKyO1`U@ z_RP+;rVSUqKijF?Vij&1#2c=c2R(+2M5TUOUfQ%YJLT`}J9`tAtxM!w3f4&1idwT98hH?h^*i#198Q;Wa9~y`WKE>kc}Y2-ze{aEG&Eimdi zo|vA#$N#}PVP?quw2O~FP7iR~4ne)eO{O125!>qu9<_)ZbsS5iiyK1>DY+~Y>OF=^C z1d$tSQPVoAIHyDs3sE<*-Y>cFr#vPck51^+;*!hoXzQQ zDFS zuk)`e)EG#NMTHOkG=sPa7pMHri-xv%D@#b>Izn`mf5iJ_dR(%eKer~fmB8=VOG-EG ziUE|Bm69_99i@=SCre~8#d(``orDL)rVLd3X$AW{;$RNwVK=J6&1q3x)NFmt&@Ur6 zRYNMA%0ge2iQJz<2ViI*2CUe+y>;F5+c8;A8>~R)oGUAzp@Af4Z!$o}6epsfGx@Nh z+;;VDf``5!UYPW&DiVf^ddxDlY>?nSAmO#v6-c>3T(_BFEZN`0JwO9#jwQ*(@X>e;nqIVbvK-=tCa&fH%;RWtdx zZS7Bb(X(8jxSM~;Ng z(?Y~p5^kQz^s_lQtjAQ8zEs&W`%-;TduA<%WYIRu2^2P?0)V-_{-MHs1u=HhC{)xb zEvh!uF-@J@luerYR9se;c^t0W{J#9nUVe!v)V+^}x_l*Llb4bCUfhQ+Igk`u7kn(N z4J%Z#A;NA4>$k+>VmL0|Ka+P9L?F*nz6w6w%!5SN`6Q;04kt1#QC5~LeXbvFM)Jo- z_S^>A@A>BE@0T)2|IOOWpfV$8K~5}NdwQzyCg$j1o11jp~Yh* z%X0Xqu|*9uax=WD-|Tbc`05yep1CZ%HKdFzWiCEhX^STeEUJm23Z=Pp>MuRFvWiuY z7L3-R)=Vo$CHI??+H{b#u|erP$!F&%Lx`%kQPHipI$avOkethhm=hak;ToMVvxml5 zOAy;DryYOQI0{bKHw>nnCF92_-VS$i+azcZC8TAA?BYBi z=EelKp~apyi>Z3znm{+hMA0ed`uZ#iKlG?$SJ?e}dFRGwo3Z>b(OfUolb#Fwn`k>F zz(*H%=Z)n)wv~Az{}~tG;354^x6?3y{IM3p;^{F0b$JrDc4%ry1@|{7S$^w8dgc)Y z5!7-RP^isM0vMToJo$T|7YGaj^gA^sNi7?hvf>ePyF#pRDH?0cuN|61pRLi(5 zr{7&vv6&5EInW&v!U}#AKK^kWJ7Vb}9MPf%S*ZA`!?^+g06+KNEWoy0|0K?SuT!+O zaJuN`EFwUmvI{AZY7=5x%8LP~la^O&&HX(&Dl zImsmkGEvbMM=#X9Lz1w)4R>e#jQq>4%hU!}+>r#rsWlcIA0cw4Of#eT@eXa(@cVeF znrWzwd_F2-GTP)c8_^b79v{(DzU;9+fd)=LU3Pzm>Y|?#f1SVj+;` z27Grm$cL5KoFExjX?umA{rz*>oXougZOM|a;nno|D>!aF1-;n%Q8ILfuEsov*Qg0* zC``8`Wx+0)TordAu-O5 z%;d>w?BYv?sr!7IygYX?nZZ)s_u>XZlM_Ai`P`HTB1O{VOhuCgbnX>bTR{Yds1X@f zM01^7cT2lqHHWYDzFo=hLg*P;;O?NAjgX!OMrNd_EpB4c7afU!n?(f^=G(~S2|JE1QK=%Tg--glbq4=cLO?a|at8sqV zJIjYv;=pACDvNL9_o%5+b>`;jj_Q&c_#0u1rE8SlJMWc8Y_BbaA2L_A;*1X^0rXt1 z!gfSM-sJ}`JdRR9X`YvHs~;-1|54KLOM)nkAIiag)&)Kb?xe8T&;B7YH4#}mP(@6a z5wU4RxNO*8Mckh&IPOc)(11;&VwTl z8r38x@;&~R)GJ*=f}?F~+pzc7`gOilkGs+q5!-zUb}9mnG-wtIY2zTQMgiforeDn= zmFsNu2DMsc%9NJ&tB{xCLSFk)@rW%w+u(Y5E?m<*BqaE{HhWuXa^dZ%JJ?-55DpE~ zxEgUmjmOuw21Ub!S0gXjxl!PohbS+K`2d7+q*Oe;X$3I?Y8m+&nX11nvFk3aY*|9p6kZ0o_362a!4 z(0X{raAeBD^+)-69>jp9VWvkVh9iRApF1LHJ^_4NG+R(!?hN}c6=ZOJyPOClhF`#k z(uP9mu84_lSo|SE(LY7>=w|e!Q{f0Q11j4LtPjMk%(WnnQg+W{8>=$b4=q?u_K|wa z24Gny_-9wqO>%nyQdq-!Pyo7@LMg?w$sr*!54kpypvM? z8kVW-riF({sU?*VhMPvuOofE0^p&a%f~Pr%B`XU{GG6)>COJ7Q5l@p9H6D2dVV~+- za%B0}_{rCuz;dUh-p9k~kG5s8qx##T)>(uBMAU2rpeAn^YONGG)^_CB$mq^KyF@rL ztvdh{vDJpk>zxJ)@Dw5)c?Xa!}%jS<;A5cKug$r)!930QFXETJv)CD@1j z^N6^^c`&wb!2m#M8s2}qy?ytaL2sOy;Y)5uH)s3z3;vVZ%4+Z9V7AU(0sz?Y4w?U8 zNu%%rr&f6pEkQ&tgkR)TfBf~M=_|tG`1R3YnTUR222C* zf!#*{;HJlI@DC#*OqhM-&aCg}rTDviDIFjTq?qmj0GP?pvg=H?ijN@z05YZF%*~P{ zcY}ni1MsASaO;C8Zx(z}QnVvOQkYW6HesZm7EJXKQj9Uv&1B3L9DgLpcm_1>V=~;) z*@tvH(M8=cUMvxd2N`?Akzggq5pgW3R=+}7#T(EhBNF6AKT!=Tz+xxRQciru)udKV z<5l3&ir1mkRUo?&^@J9THIeI16CMX}N4&-K%5!Cgw`hv~L2Z-ns^KS&lKX}-XXDM4 zpMqanFz5M`STvEZ>}ame9%D5{OnQuS;Fu5rIO5F7J~}M*Lx2@2!+Pombvo1 zJxovBz4h!5Us&vL*#RUzSYIhhOL9pC0~G_c6@?l~Qd%CYRV02Sm?+I52I`C&<)8H3 zSkEJbR@_{%1`4b+OV}wm9GHP$rAPQ_vE!t<8M;!T6@OIdEMPClEx2mYoUqmw@F)<| zWK68uIJF=MX0lSUj7RSm?GNt*_eu7tZj6zOtRy@tp0sxvwI&^vhz>9hcn;ugbGapT z%k#B1YRYu5cvFm|8p_(Wnlxi;aCpAt3r6AYTR_hD?$w4O@gnD6zDQleNBwfeDAIic0Xd7L$o)lk*o1l zMTPDg*}pqgl9ctx`2KROzQ`@^Hw!CPajV|+?>l-S4d!^)d3SlKzL!TefNw>mg)|Fs#xb^btto9fEFoQKnPr)?6P4CAFH-(jE=dj}(P7kNk9N;x)Md1s9-f|x zURm{@>P5bmYF8Cgm9UDi%Gfk``rGtU9&dpt*E07}!BN3-!B&Tbk)IK3drf<%tzyf* zQI+9Ed#`J>k)UCvfv%Bht6W1-1KmN7&HS0d3-RFJciAT# zb_TPUvn#;8cAP1u5r-n}f<<{d0=q7+$lI?8amo=>>^WlW1tg;z<9+)|-uE7+I{^## zMFp(|L+3SXO&5F@4i~ZOC19QsreZ^&*WB>9^tR@~yf{ip7N^rzeTcfWdbnNN5eOwi0W6exX_yILkr|wGO@Hv5>GNEq9Nj7uz|{c{4BMB>hUOd zCT+$6Jz68=cxev7^3XD2I0Tg;IyPEAMm*X5s{!#mR}Qc5{JON)mduMxw#;UM9hbUb z6Ppq*v)FfbN0C+jZk}%OQFf3a_^&~cfugfAsUSOZBcH2t|JK|2ZR~E*F3MgYK`zI* zsk?vK%+OtQHGVBSHmgv!gvwVnfy^sOOy(N=M7kRsU%F&u{~@v=J8F)3zWJ{!(k9I0 zYBvhZssZU|X}!Guw7-FstOs3D47I7fDrhPj1wzI6`972`+@eB#T#@N%sUA|_sZ=xC z?f%&vf_$dxZKGR0e!C`gaFAyqOgOcs*25XHA-79a9qSk|E;A#v*xBlXeoe*q9v^w9^XI)$4JPDTv2K{=hG=pr9P2cNDlpPQ;1a2Rg2S-!*+k%-~N#W&)v z5#->7t}w3q9gxJjP5#PT&-__iH_~M@3t9l39B#4|8>44`&pFINcSVD&{zTi-`+0u< zAiKtqGtIcx6WK1^RsKt)v)8tIVrE)cT&F_kPUl63?_y!2&2st4b^d}Ru=j!I?C+X! z@8i4JME{yDl&(b8e186tAoYnm;P-AyveUoU=;F*82T&S>P$GSoc2iyX?mF-dT|SOgIwm6`nD<>GZy7Sv$bd@6^w45;AxB=lxIQBV$p0 ztf#Oizp=~2sp}%3uPqStctTRlKeKVy=YNg=x%9VKP;{g3Ho)!i_v8H*{zix=!~4us z55$+@qM=v4*ZfZ6ToNL^5&Ku%_+x$Lay`^>z_;{veqH_%K00#tBkeu~LmDn6FJ&qQ zE#|aXKul0@SKRacTNvv5L6o74pZ3o`?zQKtUo(g@GCpBpa&O

>52LK36D#}Re z_^qB5`S#lC`ak}fH441czUb-$lf_6<#=~RDXh8i4ha-s_vH#XDH(wJ+Y%QsJr}{(I znrAe7B!*rtkq#auLWz}EK3vgh_t)ctT3{zm! zyYcL8eKlK#3<4Rcf_o9uajEnqy3lcIOC&mz@c%DVoR$6`;eYys{$JsL)&4*8{IA;o zGtd7z^#5MyNi~A)eX4u@OD2hJ6sKKctOq?ky#c?FQl$xf&PuZb4o#$VCiQGz0UsA9 z=lHq_nW3DoF9v!nl%&<^lis9aM4oksWjHU*@Hi^cjR?&(SMtF68#E9FrJjgf(my$3 zr?=B?QEmw}X_p2j4wa5PnWt5{l{L^EsJiT{+Z|KM83jo#BbS_RCf0y|TX@CUhbQ7p z$W7ydSye(TV2fgl-xbz)YGdejxP4kl661z{c^Pj-J)$Od7+o3BTS~5%Mc$vBxRpq! zWrX#Fqi0|^snweZJKU#*aau~5&?>~F1v_77JJ0f;8OyL)q8hz+%tWSeB7xaphy2XlX7T4{r4hxU z*7d}hS*)b_kK#^#5n?1)Xr6xKI2RryoK3NVz%as9H+Y zQGhuF(io5QZ3id$C6O8M3sfvb%@=FV(M;yG8o8N0x%|rj4PD<#uanYC9`H4QNwJ^9 z1BdhV7dVjxO7Zm5!|7TsCI(#pqacqQ6E$kwdLa>!KZ0b7LP{@Oar<6d8$*PN&JdfS z+GqJJP9bIZ+7AIMFw_6_-IT~ZBhR!BlRg?oJ7-K6rfeAu59@@w1mzm1qXItr;c-SG z%q@^<*VXSd>WjGr>-vdCWhU;f;Z!e4oY?_I6xk4ZI$96EgmP!82lqF+hrHg+*M|G# zRG|bW*gBNMc_2|0yUB&Gizx~rGZO)ZV{jdquI24Lkrf~{XNG%)e2xdixl;wv0i?7cz{ot1U2@g+0l4;p~RxzJ&@52m4}bzLNrb#{f_+LSAf(<=BMsQ$}gPm zYfv%+Z3FWIngcX9AAKc5_qZ*(ZB(D}Zq4VSQ$#3o}CIbp4ja`gS*^>4gRTP4P*d!&R|kW!)j}65Ls-$&S0-g&(lU zc|~)rf{hpa-e33{EG>=`G@bWqBVjzihqms$_e@#U2J%O9uOrjTL?OM;0qbf3X=(V-c$ztT0b_AL@6LG(IvXfCZJ@f)*`);U}+^RVJ~$uBI1`{9;D*%?vGSd zP;zu~J>N~MDXHw`X4I)%VfvTBw{9bcuHc1H@KSAUK}<}H+obeUIPj6kg0e#7tHCVn zjQa|fM8g5Q&{CpxxIW_MWpijT*6?78*U4Dgoh4IYnqrd|Nd5fcUpMTL zS%$LRWi{v+PIz_uckQ=DP7<|@8mlQ^0p~1lF62*3kT};5!zxnbd$M)0->QfD%w#=9 zG{}M_ZWXVIfSUzs0b%qFnz&p{GDZKe7~^AuA|f0D`q$!%V&od%5X%t8 zsOH8xBVm&T8n9nQ9^JVUrbusQCb^lClH1hH8??JQLa^eua2$5n`c(29N_I&h1OE^2 z+l%O2;or#1ueJX$j+m~{hsE-$mvE0_NM)-VrUi`$YYR7EsLIyAH2<>0)BZraEi z;fg9F-+uP=$E4Ijb3le6*FnKM?VH8qEtJF#O7>OW zx=sQ|@MH9&F)Ue=Tz|EmA(A4HM9l$Z=D?#61ixe(H$=%#`(!T>N3|Ih977e~1{C0i zYJM<1e}8(++lVLWFF3!5iJ0lPU({qfy`fIUN6&X))TDr-FaJHwyAui1kIt#D6)Iba zmHhtK52Tw;oE=fAr*=~3SqA6+o6)f@ftgeO+eVZtaP(qC)QxKSCx>9+eF{IRctbBP1$iMect^fr05y%Z=GK>q){+eR&O!hfu|cFBy%Eo;5ld z#?pveXjFw;YuY`7hwBYWa6gj#k4A};Mmg##Z$gDyjWqlM`sRZ|x-tvH1Yn2rJQ8n% zk_RZPFE$+Vi``}{2_2TAM22N2coyIc5eW!I#`216K`H!~o8EKz^WkjP#^z2dy|jF@ zGdJU#A#$Xhtg)$BEY6){uly;49w-ZgXeauh3%~8p3OZkU1N|xS&h_U`!LM$_igFnEA`fM9Kh^9OOQ>(G)LhxYv9GF=ltn@UoKIqd*BBZiD}r7a@Z_X{ zpVg(tU!Td5L->N{w=pBx;(EAI$^w=Qx;u-i=7V1ouElm zP!4DQ14c@CWG|sE@{5b;Yy>0wk$Kjzw+o>9r4c!Rt>P?n6H`k~rpEHUQ1;I7#pc{* zCQMjK9GJ@mnTzx}7#Lpiq^6=dRkMr*?_1kvkM)@JtdE*RCNM$>keo1KjFHqd1kb@= z;G>{r+jY58lG%Rb%Xhj1!^1aWSWyIU;&@0x?iTmiftm@+iSp)p^#V^(v9Up^UfoXN z@U~puMsBRTPj;m`Gu;FUs+vTLQQN65)p1-Yf3ASIZ%lqnmh-$|N}6I2@?DWg)N($= znAIL8%GWYE#zK!;YlqU+B2gglFn_#nY4-+)q$b~~L}%e1Jy>SNpYV~Kb8&-O{SUQC z)zIwIjND`gNY<^R=DhlDli`;HqNTgH{S%4@?z`x#~?^O5z^OOKjTCj#Gw! zzjSRTDhw(*dKo|s+QEx_DCZWlI5YMtLS$bb0Vy#yuk|zj+wuLTAegB2)+&YAi-B zB*#6bX<2iB4eX}y;@Jlg=%~~3$Ge)u92ySn4iqaqfH0!;4ufwX5JQli9T6e1Toy7t z7Ojm>?d-X4e$|%fh)zJ1R}#;xdV}w>S*`11(3<|DTsQJlA@0nm-XX<$4p~b!B6=Hn z6e_Xa({^@0d&bix&(#Lc(GZUYFOA+n2Q`r8x2NJW?y%TFp|CecL5Y9j6s))x&WBOJ zO^>ZJ9|XT_0t9sHR*=~oCMVCP)q*SZQq$t-luRt6Oj7Wv0SXf@a($K0HFR(z;ojB+ z`Kalfb#I}s6N4FLHJz@*Bxw{Vpk}_TIHC55d49Msvsfj!u9hr|ivI42eKkZB%tk4( zWxQTQJNi6t9w&cvzcdF%s_`4RH8d(=Y?_`9CD zzL|b^ZJ_l{R;UTd53boCL)ARHG>^NWvl(u}jqG2|tkxl?c++R=NY`R0FTC(w&@(;R zSL}_#+}6_N6a4MTUL*Ou^-%qP#{JcxNhY2L&qR_SMCeAqN$?r;B2EVBPoGR4wn^S% z;97QbOtcjgeuY=3#+&jq>e?WaEgMrh$_28Ob^ec;0Z4F`2_xyniB8>#tD=MLOnhHx z32n!|)GvN9!Rz0RO6Le^--NHiVNjQm8mB5TgznsxNf?-tOClhxsHaAr6hHVMCCjiX ze(|)X{#0V=B?_lzs5>$7S{!*YC(~gvXKvk^-MLZeZoStqBkX;>6;UF0S?!qUu&`se znrJG^Iim~y(BbW4e)7B{eF$(^F zs58?y^R#8lU6+h5W_qVzYzxJgntiQ&h%wm;Qnff+DU3#iZB1ygj<`h(*H|L8Ft{%Og$UwExi^A z!ivkhn9j9^eL{hXUeUf!y3kb-ziI9Ek?^?*VPA7^$Ff;|75vl#23i~2swx)(4fgup zMSC}g=}s{Y6WXx1)JZBid`{c4JC!*DA3XWvq()ZScrVSE{l3Rhz#;M;4G4m}X1iBP zPWu7_D|NbwpR~6nKMST7r4T96NP6MMvT_dug%s!}?IwplE9caN zJ#RzUetzx3%)Zcb^UvL{lmG%pP@>RSSH|KxpV;8tKCJQB=vMI&1f!@+n#PObai+*^ zw<$dK!;QL}R*Kxj*Y}Y^PV)^drO!OP4sVDEq{4d9FCQz<#h~(#J~Bq3mmT_N8zy5B z=&Re~_~}Hza`mO`HnkY}t%3hBj>o2>@M1kll=U!AR48Gi2UT<$B&SnAYlOc(QMF^MxL?WLGzf?Bgtm^j^WOGhIXm8mHLUn}Is~LwlbuIJmh)|H|}| zYo@^qD%cT{dN(R)cRcVDTdMiwEmcjA_o{m;f}wX}c}K;gMVmDw8+z$Ld;K3mv5E#k>_lK)Do3GodLUt8mh#)WjYpb{P}4lWIxJkFX5*U7JYlny8}%+Z1m$hGbgYAv%FTuAW*zlF2@^*CGnV zgC>um;v~j#-%sI?{f3epZz4nQ)0p$3Q`op)j3n=eC>!)t37m;qI(xDOd_car1lY#V zu&!;xUmh`eTH=EuKwEhevXN) zZFBr}seT7$(W_7=-d}5it4noZn)wRh@XUDTDK$@&;pT&5`m6;L+JiKM7U}1@7NNCN zlTyL6da%u<_YB(kAvZiVMG@iiv5X=|)5wI5gZouG<`5mcelDbKfKZk{B|2-Rq5=El zw*{_juPdSezF$kBFa(Sa-R+Af-~qCs3KHWt+Z$?^IWkSh?lkYkSR4q#Jd|nLhWudj z7>)BE%+mzBjWtV<4O^DUWFA%=P!|ZRB~&SwQ*EOW&u|ibaY5--I#=?f$v)o5%3E~A z%AlV|L3`xOYv3Okx`}|&>ih{!LHzpH#r*x%(dB9`Ee+?=o{yuCkBxi#7s1FYcW#1J zcX#)y)w>Uz=xkxI;_-wuOr$t$un6pX-L}4qxI?WDwGUSeiZ)RD^K8%EBlL1$i(1sz zS9rBOp1-wh{FQ33ybnP=szTM`7f&Zyt8jhu4!u!|P4YLHa)_3i%R;*sixMuGZgn$HxYSSCYQhuLvT>KV8k1$3e+Jkt9A=59eq`izs zrUdzJxRYVIhs*6M&SAlSX(yUA!7#Hb+rdzW--eQ8-1G@VH+ZEG86|c&{$A!j@yrxM zKga>qe2>onC5TiiKFi%-rJ^F-$x(aEMsLTTMZJ0lw`I;{QCylXT+n2PBN$qOFC@{y z#C#Y_KdG&q(lQ%laOuq2P9bQ( z|5FJ_?z|GH65xPNfwVwCtD29ei1qu-;A*D7#n$=sk;OD?ajNpBpa|mDDqS}m997M@ zckVLQ^~m%1Dm_m5t|Qp=fTe$tz|XKsA;k_TCuEStYKy{D z0I~9vX?jm>eWJopq8;`=Bw%71@(<8}J_?+Z9C z{?^&0tpV5?ErQVm^B&C0WX^c!F^I6%Gxht)b9_Jq9+)>&mu9*He$N_=CIpZ8YXf=c>D@h~NM(7_qr~)iqSzP?Ocn|vykor{)CU~W zKZx!WEkFoaOCQ&*hUDzPMZ?YKoc6BF-Xck7=Xrf&t-J?Ozd6hfzL(%GD z0Cgc?yK-%j4`@0>3LNe~!}SuU@0zQAQ2k z@-ZS&kT(O;j37S#KV{;Me>7b~K-w>PUZ303rj9F9h9)^u=)DOw{P|I(=KIKYF`0|B z?*FDR##A_#b9_I)q4@EQe&%+yhL^vY z>3nfV&r3q;H!-s>Cas1f1kD}%AmZoE2QQ!UtGcY;b;aLadTQQf(JF=r<6^~C#tnkm zoEHfu1-9B{7{*V2k%_Def1a$o)^P?BxogaOtRPZZm>C?(jKU2H@DALjhCs7w1j+vu zd}JK{7z*;P1vg^&KSd3jg?W2d>WKOIxp)3&c^>ACQv_se`b-c6LZGIJ7)L*?o4g4mYcpnA&1ul)wor)0$p6#qhhmQH zKMR^H%_@S4=ur$2OT5D=1g?#GKXHJ80xQGe`ua+hENHK8iC%10EZFP>@zzq=!R2{X zTVKnc#!ZCghcELXwQOVee5!V&mhP~Gpb+6t+uilxy{PPQJh7YtF|&Mj(@I+NJPHf9 ze?5IYt}%s2vRIdlbbsio&?%*=6no`Bc2*Zkg`etRQCp(_4$4GM!CYlL8K#e*v#hc| z$6P@e!)tv6q z;YU5TT;W)ZXEX(I?2Eeb&w+md_JC?-LxL|WbB_+FkDiMgFu=c_tTFf;cYN6}XVnQ#RDcVD;mR2`b z>zb%@wsl>OD*O;jl~XO^B903zjaLG&e8l!j{+x7#Vu?JxA#skTVXrRLE;F_ zx33u4%JrugAg>S!gOrVd%L2%W|GxT+#T?wZY|jd6vR5p>OP9s>rq|u|p0yu;jTiI; z^4(C6;TVIk&SncCOrVFL$fKzz2bRxkrvY`aKj;t?N2A_l5;1@Q{SqE0KjjxQY&M=A zWj36gA<($=*YxR&bS_`3N20?pFQ?N_k%4nLJl=}}vvQIJ*3XCvH^cqVpiqZNdk<-v zYuKYFccv=PXq8AP;BVVKb#ev-WpP9t&RI0!3vZ9ip;T^ar-Yj?^zf3I&x1gP032i>=l zRNs}16YEx@ErCq%P2R(o$80~x!GbQiQFU57OsTqHn~t38JnBj$40Uli@$MD^`2 z5AmbiBx&xrS8|HAm==>JdiV>BwYZqUs5Doxy`B(E>z0-N+ObQmhIOfNn(KH`kA;Qv zZXo?Az5HvOaV%ccD~F~({u9UCGTzW4@T4;Bmcg=JEN;)5j!bxN?Nv>2ndvwP$PV+L zwHv&G!azY`{*KD(j>k8-Rgqt2=3id}0!oNI)rS3|ynG0oi(kLnciFcK(VP$r$T}3i z+ZD&wF)Q|(FpU1oXwly#3l1*CAJd`x_qgTz2pB^qCj%$*Lag`iw?$Qk3oFc@3hQjU z`EpI&Y?hGsmM0E?u&|*#Wz%4s(xLp2fTCLH|;GKx2Dnut3ob@KRH<%pcg@1)ix8>_Fe>L}j!B zVNAZ(dGS%MlzN@qbc0SeTAM$sS?GWBv<4j`G^2NRW3!vGYFtHE@BoAR)K~+vn0^ z_P*%lp;}cD=7b*C6}lh4 zLSU-(o{pl3Q9y{16t3Ru@P( zCKC1b?-_La(x~(WKaHa=4sWeSaUOpJ4uL*ht7V+#$eGKwoeXXb$LRfETs})^3~xN@ zzGmrG#r(Dlp@}1BcozjtKTN)q3W-W&RU~1`LJ-d8X}@EfGcFmeCyP)5Nwl=RCI7kZ zpMZA;+7AbxVse&rq*|Th?>=oUO}AM>0PStPZN1?SnrYwyW)Pj?5=~M*q-G?{vM(lp zi-o_H$D^T_|EQ37y6Q&6$vcbXRx%$71ZzyTirP+x@+>;D-fL|8WfzY&CDC?aEjFRZfbveQ^x?3&K#c(31lPJtbr`(2lLWTs&yDOK}NlrwC|hXjO@ zOSWCQf24YK<~e9hYGyqsKF@AOzR)H-CsXwHr~1U7U8Fo+pxFHQ6XI|G@Pt)GBQOJx zUGpSK%@R#t(MI9zih}mHB^b5d>P_c41B23@RoPC8bsvyi_7QkSZDDIM#YEF{7mO&} zc{Ien{w4kR^aPU4(QNY(UpOy?&kfAitC(TSa@LJBj<6RR zBUmhK0^8ihNJ6)Een?*YmTX1Bw;#PqSif$Qn-OvMp785P$Q^nz%)e2eE)^Q~~0Jp9oxqWN<}GxF6_-jYbL{(QMUHxvE% zhl}tgDOW4X{fdj;z6OD?IDOlzq9P-)cSv~}(3*OtH0>x+XpiX70hbRTJ=4jER>=En zK*fY!jZ~AIu>4DEk07(2_j(SIqVh{pTmNfIbS6V~K(oqImw%9lz^qx8nelsWc3!hH z`ZxQMxX$mv4?eesvV%oz+4%xT?LB`WU4r!0)#JCljIKT6K3fyWNP_fFJ<+J@YSLF4 zZiR;Nd2W9_+j@M0Ey^amPe*$=&CG9Pm-yC{>_q1d5|sOTT_PxdbXOvj(M~qL-~zJ+Qcrbu{71J^ zc06g+#9x}m$0&PEE>Klu^w`!kSYUj1z!u-=EXLG5*>(=z)mR+5FlK)4^^)17iJLZV z=M)zA{ft#p3!016`)aS5Vc1=#=lVL^^d6_a@@E~jKX4th{N;h zr;^MeDMFyNa+tC7jIo_|yVBiHT79Z4Xd=$0Z*50HmtDXbjq3#1pLEOgw9+%~NkZK6 z_+y>U%loa2X9s33%VMeLX4)(AlCpgZu6+z^)Vvk`YV4Y7=@Fz21k+0d@gdfgQ9M)E zX&gqc{#%Zix|sIhzTD>s|P`>=#W!i7w-rsDB?a}tvF1R4ula*f=6sqVyw&9z>0a4g6uUL{q6Gr*=sNKye zoi)vH+NW=Z{cI!mfIr>6!D+^qfW_hn`}H6QRC)jT0CXc)s7MU=g{f})h8*hRE+)*< za|Hd_aeq4&Hp{x#DvlshFB|_0rvGQba+Yz)a|IoaarH{YR*V-xq$%v1qk8ujeuxYC z^~civ(SV)q9n9+Rs2u|-d=ONPMPdeoMWMm|0@qM@j#v}dgbZq2(6>8pqlZMRpUBVu z)a)^ad-v}ETmrTQNil419Uh>I%#XLZ?4V++a%wbDo6ZcWP$^AM0Ac;>wM*hBtg9gx zlYZHo_@KaGbGdO(H^b4wvrAy5i#%}vGbjZ%b&)3L8S-EKQyQGZ`sz6`9Rz(4Uf>Tt zFu`cctkbtx!6bmmJ`!a|+Ja)el4qQ3^-D1*=e95$Vj1nWP>iK=xkg5{(RiQ&Cooul zKw#`1YOy!J1;vOZ#26jsv~lSMlR0&8-kk(_sFotLaXsm~FqG^0bMN4QlHn3W!qbWa zU~njH>*NAfRaJGgGC&15ZiKpyz80?es3hjm4JaN8D!L0;9#xrOBv#ohs$+M*?Nl)PdAq&ynIj7?}oyx?~3?q{b?ojYfeO?rZ_9}xjn^`34C+v8C z2ichsAy%J*3a5-pSlKKrf72b2pmo3Gf4j3D0HY(!*(hIFsyp(mS&BKwa%|3%vn0qcGpnm3w2q<+`Yz8k;(GbWwXv>?GnP6I#2kNqZs4 zKJ_E2_cq|RRaXwJYIlwXjxGEslvAAI1N424Kkbi5F5Y^{hrK(bUKq5re11JoXu`MK z$nhm-pN|vn^~62PJ)bpbHodaK07;FMvrUknDl>^81#Y%=bPIR(SgnixZf=+t238Mq zdmF^Wc-=9(7@Kq)F5qx`Ys8?VfMQ+pSAsT8jW2c^8+k#_ zqda!;Z!W$Zwv%jT-D(KB1TBI{13#A~)ZI zw{>3w`CSS^^5K1v_}Jp5qkm&=wygE+Th?|4_7(=ZpYE{jkJiz>+gt}sv(Hveu|>6* zC;j2AYid^hVfe-2ZDLRZI7N|u0LhvWPG^)mCTvz(n?YTeTe`kJ9|=-#kZUvAm`H|D zL}8k^`C>SdW#Z=bNLnG6&z8oTdLueVB?cWKfNXO_eB|@}Tg!hoq?|s)2jd=7?{ zHKOJY*-0XCS#Ipnoxn{YnlY-&Mt=8lsoQxpwA=N&WzKOx%TXj)`$6=S9%*E z&KEM1gC8OA0k01-EjkN@dH27?0{kxo#)fV#OmqiD3|4?tIoPeo{F!@&*3F?7hEY+} zvTkuK|K#ANQND8r3P@rgtz@wi-dJ+UF9?r)uesDM9a2bxYpB<4dXtHZjX;YPfFRcU zjtWIqZCArHz<7}J{M?)LuG6n&FmmWKl(uQ~_tQ!WMESP5F>}^llr;CEM#}d3H19Dl zkvVaa$D&ke(_+F*##Xzqy!hJ8I@Po}@uBd-|FutnNJ>7Yz`rLxTvZnP0|Kg4DJ2LgdPSTFcNf!#gg z_{MkYh~P6G3<|6O(K#Mq{xm(%t1d(HC-I2ubB(i_f9JOYoKtQC3R>Z@P3W<$>Df)@cRvIwvKHua#Uhy&=D%&^PHNY0y`-0AeAS zoIFwCCJd1VQ7$ZX()kv|3KOAjH998^IcA=FtmaWamjL{ z2J#=S)5qQ41X$K>C*5X?_!(4l4S%V~NB@xv>*?6iZ5;!V!h2|}lop~Wr3vFz$96e9 z@_d2l<0Zf_%~}<;e+i$@%{NT%*F%UlviZnM=RBGLkjY>b_xiN;FXPeazKTl>v(0mn zFlQ+yn&bYPO};kB$s;qA={OU*tY>#B%A||I-$M@+-tyf_RqFn-=DL!kIt}gx1v&mx zm0WKxFKN`P;}Q~@%uh=@mkV{<(o7FhX5Ku!Y2Gdi$IEatu=+vn!sX1rZkA-)+vRw6 zRu{o#6ZdkjlBK&V@vp@`6{8|*sRZA+-IlSHPSRmcqe-9b^v*_KRInqei$XlOmRKc?}KC% zX9tyOj6__qq5_6I!jj^|cjrs}|NbowLV3|w*ovXbEUfvl?)D8!@6QMg=Al8j4ZVHJ zI(Er!Lq?E)9IB!0bkRp3xV-KU)0e^Bc(gX^f>^2{F(E8wXn!Qr!G~8lpn5Z z_8stce!_o8Ayihq;gxixZ>?)XMOcn~^?2;;2b+TiLHBAsM$f~j2D%&_f||N_X`JialSLdn9m3Wd)i3D*omD8CxRb}$8NOKf?k(UR(3v| zhg$@mq6yTxuAhoTt>1XI%wiLadu9Z|S_DI?p5#ClhRsjS(BMOHD@OEX+^8LLebC1e`qDreclKF2V(t_qiOW>(G~MM zi5_38G%g*2WWia_gUIEoHV)!(D0P?c^ciN?v5LltM0zHyhGM4{<+N7SsGWzhO!c6+ zLO^O>U)6n`wmdvSGTMSI3-tkV{}8hf7xE`x3&#RZ&WPuta|w^#bWFLd6|rddM*EV5 zRs#kQpTb!RL6Eh4_5CK<2sy9{^T?w2Z({kEuv#|&itK5=2nOp4G)?5NjiH>Vss{-A zgZX;?cG{V+v`$YL-2jV8Kc9|AMc9UcU)i0|naf*w3f_T1+bo4+CUR9yse9n%L2wGx zm61oXe_<|R-QeV3gD3xdF)}bjj+y{7hLgRb$?MfS2mTlUo7sbA`C2Ex1u`6KCW%yc z*_^fMNaBV_V+>wTw-;?ps)|_bF#hqy9>W|tTPJPx^4eQ%Pmw?l*0P`*K4*t=%Ciph zQLA#0Gc!o*^PEDT-F~4SvCNk86s2+I6>(Ol*q;Zzeh_ssylNQx;OV*RE zD<~*9vA<8dkPe)WOu{mfpF2M?sJ#tBWNb6WwDTV&dtAr)?`!L3%KFKw^1~cSdpIAi|ubS^I}v-g*f=JO-_RUz|Vy zN#F-BIl|jLwh_fnMH}NB`BZWtA}c4t%Y_?&O5I~7U{$kDE4}%@LV?qzMp1}hhaX>n zt4R(?y^RcuPG02iA+Cj=f@}XF6^uiIk4|kFm#PK5-fr5^7$DdFEPP#<`wd@oZ|gEM zhN!DLf%qAwoXJpXB%mG)#NSeF|19?WrUz_c_D(dX!9-^&n|e!)DBjJd4N^_rTyX(=R>A)& zX$9B8o60N=(_q(!8CQcJ%J$Kqb&aVmmHoF~<;0IRr?k6M%put5fM2HW98_~)ZdSE_ zV(}UpyUqQTJGEhYEE5DkYv-UsQ2zIho>%d1=D*_Lo8H;~Z%00QoS*vf{o6E#%vnik zES>9uezLre{#SGS>EXl5y$cpKsGT)(3Usb5e)D(kxtz5kt8BNfQ`YjVxp?V)9N#Ae zr;i`%XP$^nOIt2~KrPka&59ep`-4@I&TN_Y{*kReu)!O<`R1G`U_b1|X1UzSR-e}K z?VG!#db6c@>J10AcW0a=iq8dl3h`W@(DJ#jbLGi{H&q?~H14ZgHT6``*XcRW>w)`P z-ik>-*xPQBnpgL4)6Jhh--l~Yecmdt=lI@r6PMl>nia6xx>cKR&->zYwu@eDd33|d zuk`puNNK12 z_uAZ&=enzD(V0(tA_bp6E97!beN`Sa$tJHX>(`;qm7GTdj~raQ*yGpCBmNCRvzE<% z<(-!#9{F6{`gm0c zlC-&a+*EwGoV;Ch>$~E*=_ai+a?NJj$6B3aJ{Oqx^`7D8d=D!bkER(+syZk4l-*wc zE%L!`A<-jhhhOE|Cv4HoTh65n6T^5~#Q#IHeT<5 k;M5AU=%Cz(|7;AGf6a`~S@v)la0euVr>mdKI;Vst0KdS;Z2$lO literal 0 HcmV?d00001 From 0bb20de3a46b9d358d82fa9129e5797c4aafc834 Mon Sep 17 00:00:00 2001 From: chrismarino Date: Tue, 15 Nov 2016 21:39:54 -0800 Subject: [PATCH 44/87] Fix broken link by removing invalid characters.... (#1605) * Fix broken link by removing invalid characters.... The link to this page is broken because of invalid characters introduced with PR #1549. * Update networking.md --- docs/admin/networking.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/admin/networking.md b/docs/admin/networking.md index 2acbf062d4..0cfd9d7989 100644 --- a/docs/admin/networking.md +++ b/docs/admin/networking.md @@ -83,7 +83,7 @@ talk to other VMs in your project. This is the same basic model. Until now this document has talked about containers. In reality, Kubernetes applies IP addresses at the `Pod` scope - containers within a `Pod` share their network namespaces - including their IP address. This means that containers -within a `Pod` can all reach each other’s ports on `localhost`. This does imply +within a `Pod` can all reach each other's ports on `localhost`. This does imply that containers within a `Pod` must coordinate port usage, but this is no different than processes in a VM. We call this the "IP-per-pod" model. This is implemented in Docker as a "pod container" which holds the network namespace @@ -163,7 +163,7 @@ Lars Kellogg-Stedman. [Weave Net](https://www.weave.works/products/weave-net/) is a resilient and simple to use network for Kubernetes and its hosted applications. Weave Net runs as a [CNI plug-in](https://www.weave.works/docs/net/latest/cni-plugin/) -or stand-alone. In either version, it doesn’t require any configuration or extra code +or stand-alone. In either version, it doesn't require any configuration or extra code to run, and in both cases, the network provides one IP address per pod - as is standard for Kubernetes. From 087f5fbbc5b34b587cc1b8db167ed586d32d81ef Mon Sep 17 00:00:00 2001 From: chrismarino Date: Wed, 16 Nov 2016 08:11:48 -0800 Subject: [PATCH 45/87] Invalid characters Invalid character prevented this page from rendering properly. --- docs/admin/networking.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/admin/networking.md b/docs/admin/networking.md index 0cfd9d7989..d3deabbb87 100644 --- a/docs/admin/networking.md +++ b/docs/admin/networking.md @@ -1,4 +1,4 @@ ---- +--- assignees: - lavalamp - thockin From 625774cc1b43aae3570572960fdff2baeb7f4e9a Mon Sep 17 00:00:00 2001 From: Eric Tune Date: Fri, 14 Oct 2016 10:38:46 -0700 Subject: [PATCH 46/87] Added init container documentation. Based loosely on the design proposal. --- docs/getting-started-guides/rkt/notes.md | 2 +- docs/user-guide/petset/bootstrapping/index.md | 2 +- .../petset/bootstrapping/petset_peers.yaml | 2 +- docs/user-guide/pods/init-container.md | 169 ++++++++++++++++++ docs/user-guide/production-pods.md | 2 + 5 files changed, 174 insertions(+), 3 deletions(-) create mode 100644 docs/user-guide/pods/init-container.md diff --git a/docs/getting-started-guides/rkt/notes.md b/docs/getting-started-guides/rkt/notes.md index 28a622d2ab..096beab3c5 100644 --- a/docs/getting-started-guides/rkt/notes.md +++ b/docs/getting-started-guides/rkt/notes.md @@ -59,7 +59,7 @@ Under rktnetes, `kubectl get logs` currently cannot get logs from applications t ## Init containers -The alpha [init container](https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/container-init.md) feature is currently not supported. +The beta [init container](/docs/user-guide/pods/init-containers.md) feature is currently not supported. ## Container restart back-off diff --git a/docs/user-guide/petset/bootstrapping/index.md b/docs/user-guide/petset/bootstrapping/index.md index 03ba721edc..9dc4f7e899 100644 --- a/docs/user-guide/petset/bootstrapping/index.md +++ b/docs/user-guide/petset/bootstrapping/index.md @@ -88,7 +88,7 @@ vm-1 # printf "GET / HTTP/1.0\r\n\r\n" | netcat vm-0.ub 80 It's worth exploring what just happened. Init containers run sequentially *before* the application container. In this example we used the init container to copy shared libraries from the rootfs, while preserving user installed packages across container restart. ```yaml -pod.alpha.kubernetes.io/init-containers: '[ +pod.beta.kubernetes.io/init-containers: '[ { "name": "rootfs", "image": "ubuntu:15.10", diff --git a/docs/user-guide/petset/bootstrapping/petset_peers.yaml b/docs/user-guide/petset/bootstrapping/petset_peers.yaml index f8393b5c2c..4f992ead71 100644 --- a/docs/user-guide/petset/bootstrapping/petset_peers.yaml +++ b/docs/user-guide/petset/bootstrapping/petset_peers.yaml @@ -29,7 +29,7 @@ spec: app: nginx annotations: pod.alpha.kubernetes.io/initialized: "true" - pod.alpha.kubernetes.io/init-containers: '[ + pod.beta.kubernetes.io/init-containers: '[ { "name": "peerfinder", "image": "gcr.io/google_containers/peer-finder:0.1", diff --git a/docs/user-guide/pods/init-container.md b/docs/user-guide/pods/init-container.md new file mode 100644 index 0000000000..75b6efcac3 --- /dev/null +++ b/docs/user-guide/pods/init-container.md @@ -0,0 +1,169 @@ +--- +assignees: +- erictune + +--- + +* TOC +{:toc} + +In addition to having one or more main containers (or **app containers**), a +pod can also have one or more **init containers** which run before the app +containers. Init containers allow you to reduce and reorganize setup scripts +and "glue code". + +## Overview + +An init container is exactly like a regular container, except that it always +runs to completion and each init container must complete successfully before +the next one is started. If the init container fails, Kubernetes will restart +the pod until the init container succeeds. If a pod is marked as `RestartNever`, +the pod will fail if the init container fails. + +You specify a container as an init container by adding an annotation +The annotation key is `pod.beta.kubernetes.io/init-containers`. The annotation +value is a JSON array of [objects of type `v1.Container` +](http://kubernetes.io/docs/api-reference/v1/definitions/#_v1_container) + +Once the feature exits beta, the init containers will be specified on the Pod +Spec alongside the app `containers` array. +The status of the init containers is returned as another annotation - +`pod.beta.kubernetes.io/init-container-statuses` -- as an array of the +container statuses (similar to the `status.containerStatuses` field). + +Init containers support all of the same features as normal containers, +including resource limits, volumes, and security settings. The resource +requests and limits for an init container are [handled slightly differently]( +#resources). Init containers do not support readiness probes since they will +run to completion before the pod can be ready. +An init container has all of the fields of an app container. + +If you specify multiple init containers for a pod, those containers run one at +a time in sequential order. Each must succeed before the next can run. Once all +init containers have run to completion, Kubernetes initializes the pod and runs +the application containers as usual. + +## What are Init Containers Good For? + +Because init containers have separate images from application containers, they +have some advantages for start-up related code. These include: + +* they can contain utilities that are not desirable to include in the app container + image for security reasons, +* they can contain utilities or custom code for setup that is not present in an app + image. (No need to make an image `FROM` another image just to use a tool like + `sed`, `awk`, `python`, `dig`, etc during setup). +* the application image builder and the deployer roles can work independently without + the need to jointly build a single app image. + +Because init containers have different filesystem view (Linux namespaces) from +app containers, they can be given access to Secrets that the app containers are +not able to access. + +Since init containers run to completion before any app containers start, and +since app containers run in parallel, they provide an easier way to block or +delay the startup of application containers until some precondition is met. + +Because init containers run in sequence and there can be multiple init containers, +they can be composed easily. + +Here are some ideas for how to use init containers: +- Wait for a service to be created with a shell command like: + `for i in {1..100}; do sleep 1; if dig myservice; then exit 0; fi; exit 1` +- Register this pod with a remote server with a command like: + `curl -X POST http://$MANAGEMENT_SERVICE_HOST:$MANAGEMENT_SERVICE_PORT/register -d 'instance=$(POD_NAME)&ip=$(POD_IP)'` + using `POD_NAME` and `POD_IP` from the downward API. +- Wait for some time before starting the app container with a command like `sleep 60`. +- Clone a git repository into a volume +- Place values like a POD_IP into a configuration file, and run a template tool (e.g. jinja) + to generate a configuration file to be consumed by the main app contianer. +``` + +Complete usage examples can be found in the [PetSets +guide](docs/user-guide/petset/bootstrapping/index.md) and the [Production Pods +guide](/docs/user-guide/production-pods.md#handling-initialization). + + +## Detailed Behavior + +Each pod may have 0..N init containers defined along with the existing +1..M app containers. + +On startup of the pod, after the network and volumes are initialized, the init +containers are started in order. Each container must exit successfully before +the next is invoked. If a container fails to start (due to the runtime) or +exits with failure, it is retried according to the pod RestartPolicy, except +when the pod restart policy is RestartPolicyAlways, in which case just the init +containers use RestartPolicyOnFailure. + +A pod cannot be ready until all init containers have succeeded. The ports on an +init container are not aggregated under a service. A pod that is being +initialized is in the `Pending` phase but should has a condition `Initializing` +set to `true`. + +If the pod is [restarted](#pod-restart-reasons) all init containers must +execute again. + +Changes to the init container spec are limited to the container image field. +Altering a init container image field is equivalent to restarting the pod. + +Because init containers can be restarted, retried, or reexecuted, init container +code should be idempotent. In particular, code that writes to files on EmptyDirs +should be prepared for the possibility that an output file already exists. + +An init container has all of the fields of an app container. The following +fields are prohibited from being used on init containers by validation: + +* `readinessProbe` - init containers must exit for pod startup to continue, + are not included in rotation, and so cannot define readiness distinct from + completion. + +Init container authors may use `activeDeadlineSeconds` on the pod and +`livenessProbe` on the container to prevent init containers from failing +forever. The active deadline includes init containers. + +The name of each app and init container in a pod must be unique - it is a +validation error for any container to share a name. + +### Resources + +Given the ordering and execution for init containers, the following rules +for resource usage apply: + +* The highest of any particular resource request or limit defined on all init + containers is the **effective init request/limit** +* The pod's **effective request/limit** for a resource is the higher of: + * sum of all app containers request/limit for a resource + * effective init request/limit for a resource +* Scheduling is done based on effective requests/limits, which means + init containers can reserve resources for initialization that are not used + during the life of the pod. +* QoS tier of the pod's **effective QoS tier** is the QoS tier for init containers + and app containers alike. + +Quota and limits are applied based on the effective pod request and +limit. + +Pod level cGroups are based on the effective pod request and limit, the +same as the scheduler. + + +## Pod Restart Reasons + +A Pod may "restart", causing reexecution of init containers, for the following +reasons: + +* An init container image is changed by a user updating the Pod Spec. + * App container image changes only restart the app container. +* The pod infrastructure container is restarted + * This is uncommon and would have to be done by someone with root access to nodes. +* All containers in a pod are terminated, requiring a restart (RestartPolicyAlways) AND the record of init container completion has been lost due to garbage collection. + +## Support and compatibilty + +A cluster with Kubelet and Apiserver version 1.4.0 or greater supports init +containers with the beta annotations. Support varies for other combinations of +Kubelet and Apiserver version; see the [release notes +](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG.md) for details. + + diff --git a/docs/user-guide/production-pods.md b/docs/user-guide/production-pods.md index 440ac4619a..1c3ac18142 100644 --- a/docs/user-guide/production-pods.md +++ b/docs/user-guide/production-pods.md @@ -204,6 +204,8 @@ The status of the init containers is returned as another annotation - `pod.beta. Init containers support all of the same features as normal containers, including resource limits, volumes, and security settings. The resource requests and limits for an init container are handled slightly different than normal containers since init containers are run one at a time instead of all at once - any limits or quotas will be applied based on the largest init container resource quantity, rather than as the sum of quantities. Init containers do not support readiness probes since they will run to completion before the pod can be ready. +[Complete Init Container Documentation](/docs/user-guide/pods/init-containers.md) + ## Lifecycle hooks and termination notice From d41078c48226729c85cdd17280a5b3b09b396b83 Mon Sep 17 00:00:00 2001 From: Eric Tune Date: Thu, 7 Jul 2016 10:53:13 -0700 Subject: [PATCH 47/87] Delete references to v1beta3, which was deleted --- docs/api.md | 43 ------------------------------------------- 1 file changed, 43 deletions(-) diff --git a/docs/api.md b/docs/api.md index aa9aea1d7f..9ea627721e 100644 --- a/docs/api.md +++ b/docs/api.md @@ -95,46 +95,3 @@ DaemonSets, Deployments, HorizontalPodAutoscalers, Ingress, Jobs and ReplicaSets Other extensions resources can be enabled by setting runtime-config on apiserver. runtime-config accepts comma separated values. For ex: to disable deployments and jobs, set `--runtime-config=extensions/v1beta1/deployments=false,extensions/v1beta1/jobs=false` - -## v1beta1, v1beta2, and v1beta3 are deprecated; please move to v1 ASAP - -As of June 4, 2015, the Kubernetes v1 API has been enabled by default. The v1beta1 and v1beta2 APIs were deleted on June 1, 2015. v1beta3 is planned to be deleted on July 6, 2015. - -### v1 conversion tips (from v1beta3) - -We're working to convert all documentation and examples to v1. Use `kubectl create --validate` in order to validate your json or yaml against our Swagger spec. - -Changes to services are the most significant difference between v1beta3 and v1. - -* The `service.spec.portalIP` property is renamed to `service.spec.clusterIP`. -* The `service.spec.createExternalLoadBalancer` property is removed. Specify `service.spec.type: "LoadBalancer"` to create an external load balancer instead. -* The `service.spec.publicIPs` property is deprecated and now called `service.spec.deprecatedPublicIPs`. This property will be removed entirely when v1beta3 is removed. The vast majority of users of this field were using it to expose services on ports on the node. Those users should specify `service.spec.type: "NodePort"` instead. Read [External Services](/docs/user-guide/services/#external-services) for more info. If this is not sufficient for your use case, please file an issue or contact @thockin. - -Some other difference between v1beta3 and v1: - -* The `pod.spec.containers[*].privileged` and `pod.spec.containers[*].capabilities` properties are now nested under the `pod.spec.containers[*].securityContext` property. See [Security Contexts](/docs/user-guide/security-context). -* The `pod.spec.host` property is renamed to `pod.spec.nodeName`. -* The `endpoints.subsets[*].addresses.IP` property is renamed to `endpoints.subsets[*].addresses.ip`. -* The `pod.status.containerStatuses[*].state.termination` and `pod.status.containerStatuses[*].lastState.termination` properties are renamed to `pod.status.containerStatuses[*].state.terminated` and `pod.status.containerStatuses[*].lastState.terminated` respectively. -* The `pod.status.Condition` property is renamed to `pod.status.conditions`. -* The `status.details.id` property is renamed to `status.details.name`. - -### v1beta3 conversion tips (from v1beta1/2) - -Some important differences between v1beta1/2 and v1beta3: - -* The resource `id` is now called `name`. -* `name`, `labels`, `annotations`, and other metadata are now nested in a map called `metadata` -* `desiredState` is now called `spec`, and `currentState` is now called `status` -* `/minions` has been moved to `/nodes`, and the resource has kind `Node` -* The namespace is required (for all namespaced resources) and has moved from a URL parameter to the path: `/api/v1beta3/namespaces/{namespace}/{resource_collection}/{resource_name}`. If you were not using a namespace before, use `default` here. -* The names of all resource collections are now lower cased - instead of `replicationControllers`, use `replicationcontrollers`. -* To watch for changes to a resource, open an HTTP or Websocket connection to the collection query and provide the `?watch=true` query parameter along with the desired `resourceVersion` parameter to watch from. -* The `labels` query parameter has been renamed to `labelSelector`. -* The `fields` query parameter has been renamed to `fieldSelector`. -* The container `entrypoint` has been renamed to `command`, and `command` has been renamed to `args`. -* Container, volume, and node resources are expressed as nested maps (e.g., `resources{cpu:1}`) rather than as individual fields, and resource values support [scaling suffixes](/docs/user-guide/compute-resources/#specifying-resource-quantities) rather than fixed scales (e.g., milli-cores). -* Restart policy is represented simply as a string (e.g., `"Always"`) rather than as a nested map (`always{}`). -* Pull policies changed from `PullAlways`, `PullNever`, and `PullIfNotPresent` to `Always`, `Never`, and `IfNotPresent`. -* The volume `source` is inlined into `volume` rather than nested. -* Host volumes have been changed from `hostDir` to `hostPath` to better reflect that they can be files or directories. \ No newline at end of file From 6b151435da45a837d31f806705b6949a34f90d64 Mon Sep 17 00:00:00 2001 From: ahawtho Date: Wed, 16 Nov 2016 21:58:17 -0500 Subject: [PATCH 48/87] Fix shell example in sharing-clusters.md 'export' command should not include dollar signs in variable declaration --- docs/user-guide/sharing-clusters.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/user-guide/sharing-clusters.md b/docs/user-guide/sharing-clusters.md index 34a53d670e..723f54fa14 100644 --- a/docs/user-guide/sharing-clusters.md +++ b/docs/user-guide/sharing-clusters.md @@ -115,12 +115,12 @@ make all four clusters available on both hosts by running # on host2, copy host1's default kubeconfig, and merge it from env $ scp host1:/path/to/home1/.kube/config /path/to/other/.kube/config -$ export $KUBECONFIG=/path/to/other/.kube/config +$ export KUBECONFIG=/path/to/other/.kube/config # on host1, copy host2's default kubeconfig and merge it from env $ scp host2:/path/to/home2/.kube/config /path/to/other/.kube/config -$ export $KUBECONFIG=/path/to/other/.kube/config +$ export KUBECONFIG=/path/to/other/.kube/config ``` Detailed examples and explanation of `kubeconfig` loading/merging rules can be found in [kubeconfig-file](/docs/user-guide/kubeconfig-file). From 8563dd273b3240ee7a0a1b2228a1812ba993fb31 Mon Sep 17 00:00:00 2001 From: Tom von Schwerdtner Date: Thu, 17 Nov 2016 14:20:29 -0500 Subject: [PATCH 49/87] Address formatting Currently the 'sample environment' code block all shows on one line at http://kubernetes.io/docs/getting-started-guides/vsphere/, hoping this will fix it. --- docs/getting-started-guides/vsphere.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/getting-started-guides/vsphere.md b/docs/getting-started-guides/vsphere.md index b3679c56e8..a1e59b0cd0 100644 --- a/docs/getting-started-guides/vsphere.md +++ b/docs/getting-started-guides/vsphere.md @@ -65,6 +65,7 @@ export GOVC_DATACENTER='ha-datacenter' # The datacenter to be used by vSphere cl ``` Sample environment + ```shell export GOVC_URL='10.161.236.217' export GOVC_USERNAME='administrator' @@ -79,6 +80,7 @@ export GOVC_DATACENTER='Datacenter' ``` Import this VMDK into your vSphere datastore: + ```shell govc import.vmdk kube.vmdk ./kube/ ``` From fdf250f86eb4e1a5b1a54da6cc9ddb6f320c1f21 Mon Sep 17 00:00:00 2001 From: Matthew Fisher Date: Thu, 17 Nov 2016 13:05:45 -0800 Subject: [PATCH 50/87] move "Privilege Escalation Prevention and Bootstrapping closer This way users don't have to read to the bottom to realize they need to add --authorization-rbac-super-user=admin to their cluster when bootstrapping for the first time. :sheep: --- docs/admin/authorization.md | 46 ++++++++++++++++++------------------- 1 file changed, 23 insertions(+), 23 deletions(-) diff --git a/docs/admin/authorization.md b/docs/admin/authorization.md index a72a855cb2..1a86359a92 100644 --- a/docs/admin/authorization.md +++ b/docs/admin/authorization.md @@ -207,6 +207,29 @@ and [enable the API version]( /docs/admin/cluster-management/#turn-on-or-off-an-api-version-for-your-cluster), with a `--runtime-config=` that includes `rbac.authorization.k8s.io/v1alpha1`. +### Privilege Escalation Prevention and Bootstrapping + +The `rbac.authorization.k8s.io` API group inherently attempts to prevent users +from escalating privileges. Simply put, __a user can't grant permissions they +don't already have even when the RBAC authorizer it disabled__. If "user-1" +does not have the ability to read secrets in "namespace-a", they cannot create +a binding that would grant that permission to themselves or any other user. + +For bootstrapping the first roles, it becomes necessary for someone to get +around these limitations. For the alpha release of RBAC, an API Server flag was +added to allow one user to step around all RBAC authorization and privilege +escalation checks. NOTE: _This is subject to change with future releases._ + +``` +--authorization-rbac-super-user=admin +``` + +Once set the specified super user, in this case "admin", can be used to create +the roles and role bindings to initialize the system. + +This flag is optional and once the initial bootstrapping is performed can be +unset. + ### Roles, RolesBindings, ClusterRoles, and ClusterRoleBindings The RBAC API Group declares four top level types which will be covered in this @@ -417,29 +440,6 @@ subjects: name: system:serviceaccounts ``` -### Privilege Escalation Prevention and Bootstrapping - -The `rbac.authorization.k8s.io` API group inherently attempts to prevent users -from escalating privileges. Simply put, __a user can't grant permissions they -don't already have even when the RBAC authorizer it disabled__. If "user-1" -does not have the ability to read secrets in "namespace-a", they cannot create -a binding that would grant that permission to themselves or any other user. - -For bootstrapping the first roles, it becomes necessary for someone to get -around these limitations. For the alpha release of RBAC, an API Server flag was -added to allow one user to step around all RBAC authorization and privilege -escalation checks. NOTE: _This is subject to change with future releases._ - -``` ---authorization-rbac-super-user=admin -``` - -Once set the specified super user, in this case "admin", can be used to create -the roles and role bindings to initialize the system. - -This flag is optional and once the initial bootstrapping is performed can be -unset. - ## Webhook Mode When specified, mode `Webhook` causes Kubernetes to query an outside REST From 053cdb3a1ee6527cf5f9baf8bc1e6863b105f8d5 Mon Sep 17 00:00:00 2001 From: Jeff Mendoza Date: Tue, 1 Nov 2016 13:29:42 -0700 Subject: [PATCH 51/87] Add stateful application tutorial. --- _data/tutorials.yml | 4 + docs/tutorials/index.md | 4 + .../stateful-application/gce-volume.yaml | 12 + .../mysql-deployment.yaml | 51 ++++ .../run-stateful-application.md | 220 ++++++++++++++++++ 5 files changed, 291 insertions(+) create mode 100644 docs/tutorials/stateful-application/gce-volume.yaml create mode 100644 docs/tutorials/stateful-application/mysql-deployment.yaml create mode 100644 docs/tutorials/stateful-application/run-stateful-application.md diff --git a/_data/tutorials.yml b/_data/tutorials.yml index ae9401c918..61555427d1 100644 --- a/_data/tutorials.yml +++ b/_data/tutorials.yml @@ -51,3 +51,7 @@ toc: path: /docs/tutorials/stateless-application/expose-external-ip-address-service/ - title: Exposing an External IP Address to Access an Application in a Cluster path: /docs/tutorials/stateless-application/expose-external-ip-address/ +- title: Stateful Applications + section: + - title: Running a Single-Instance Stateful Application + path: /docs/tutorials/stateful-application/run-stateful-application/ diff --git a/docs/tutorials/index.md b/docs/tutorials/index.md index 14530ca25e..60aab6a8fb 100644 --- a/docs/tutorials/index.md +++ b/docs/tutorials/index.md @@ -15,6 +15,10 @@ The Tutorials section of the Kubernetes documentation is a work in progress. * [Exposing an External IP Address to Access an Application in a Cluster](/docs/tutorials/stateless-application/expose-external-ip-address/) +#### Stateful Applications + +* [Running a Single-Instance Stateful Application](/docs/tutorials/stateful-application/run-stateful-application/) + ### What's next If you would like to write a tutorial, see diff --git a/docs/tutorials/stateful-application/gce-volume.yaml b/docs/tutorials/stateful-application/gce-volume.yaml new file mode 100644 index 0000000000..ddb9ecc3ce --- /dev/null +++ b/docs/tutorials/stateful-application/gce-volume.yaml @@ -0,0 +1,12 @@ +apiVersion: v1 +kind: PersistentVolume +metadata: + name: mysql-pv +spec: + capacity: + storage: 20Gi + accessModes: + - ReadWriteOnce + gcePersistentDisk: + pdName: mysql-disk + fsType: ext4 diff --git a/docs/tutorials/stateful-application/mysql-deployment.yaml b/docs/tutorials/stateful-application/mysql-deployment.yaml new file mode 100644 index 0000000000..3b2aa22f6c --- /dev/null +++ b/docs/tutorials/stateful-application/mysql-deployment.yaml @@ -0,0 +1,51 @@ +apiVersion: v1 +kind: Service +metadata: + name: mysql +spec: + ports: + - port: 3306 + selector: + app: mysql + clusterIP: None +--- +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: mysql-pv-claim +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 20Gi +--- +apiVersion: extensions/v1beta1 +kind: Deployment +metadata: + name: mysql +spec: + strategy: + type: Recreate + template: + metadata: + labels: + app: mysql + spec: + containers: + - image: mysql:5.6 + name: mysql + env: + # Use secret in real usage + - name: MYSQL_ROOT_PASSWORD + value: password + ports: + - containerPort: 3306 + name: mysql + volumeMounts: + - name: mysql-persistent-storage + mountPath: /var/lib/mysql + volumes: + - name: mysql-persistent-storage + persistentVolumeClaim: + claimName: mysql-pv-claim diff --git a/docs/tutorials/stateful-application/run-stateful-application.md b/docs/tutorials/stateful-application/run-stateful-application.md new file mode 100644 index 0000000000..443d9cdea5 --- /dev/null +++ b/docs/tutorials/stateful-application/run-stateful-application.md @@ -0,0 +1,220 @@ +--- +--- + +{% capture overview %} + +This page shows you how to run a single-instance stateful application +in Kubernetes using a PersistentVolume and a Deployment. The +application is MySQL. + +{% endcapture %} + + +{% capture objectives %} + +* Create a PersistentVolume referencing a disk in your environment. +* Create a MySQL Deployment. +* Expose MySQL to other pods in the cluster at a known DNS name. + +{% endcapture %} + + +{% capture prerequisites %} + +* {% include task-tutorial-prereqs.md %} + +* For data persistence we will create a Persistent Volume that + references a disk in your + environment. See + [here](/docs/user-guide/persistent-volumes/#types-of-persistent-volumes) for + the types of environments supported. This Tutorial will demonstrate + `GCEPersistentDisk` but any type will work. `GCEPersistentDisk` + volumes only work on Google Compute Engine. + +{% endcapture %} + + +{% capture lessoncontent %} + +### Set up a disk in your environment + +You can use any type of persistent volume for your stateful app. See +[Types of Persistent Volumes](/docs/user-guide/persistent-volumes/#types-of-persistent-volumes) +for a list of supported environment disks. For Google Compute Engine, run: + +``` +gcloud compute disks create --size=20GB mysql-disk +``` + +Next create a PersistentVolume that points to the `mysql-disk` +disk just created. Here is a configuration file for a PersistentVolume +that points to the Compute Engine disk above: + +{% include code.html language="yaml" file="gce-volume.yaml" ghlink="/docs/tutorials/stateful-application/gce-volume.yaml" %} + +Notice that the `pdName: mysql-disk` line matches the name of the disk +in the Compute Engine environment. See the +[Persistent Volumes](/docs/user-guide/persistent-volumes/) +for details on writing a PersistentVolume configuration file for other +environments. + +Create the persistent volume: + +``` +kubectl create -f http://k8s.io/docs/tutorials/stateful-application/gce-volume.yaml +``` + + +### Deploy MySQL + +You can run a stateful application by creating a Kubernetes Deployment +and connecting it to an existing PersistentVolume using a +PersistentVolumeClaim. For example, this YAML file describes a +Deployment that runs MySQL and references the PersistentVolumeClaim. The file +defines a volume mount for /var/lib/mysql, and then creates a +PersistentVolumeClaim that looks for a 20G volume. This claim is +satisfied by any volume that meets the requirements, in this case, the +volume created above. + +Note: The password is defined in the config yaml, and this is insecure. See +[Kubernetes Secrets](/docs/user-guide/secrets/) +for a secure solution. + +{% include code.html language="yaml" file="mysql-deployment.yaml" ghlink="/docs/tutorials/stateful-application/mysql-deployment.yaml" %} + +1. Deploy the contents of the YAML file: + + kubectl create -f http://k8s.io/docs/tutorials/stateful-application/mysql-deployment.yaml + +1. Display information about the Deployment: + + kubectl describe deployment mysql + + Name: mysql + Namespace: default + CreationTimestamp: Tue, 01 Nov 2016 11:18:45 -0700 + Labels: app=mysql + Selector: app=mysql + Replicas: 1 updated | 1 total | 0 available | 1 unavailable + StrategyType: Recreate + MinReadySeconds: 0 + OldReplicaSets: + NewReplicaSet: mysql-63082529 (1/1 replicas created) + Events: + FirstSeen LastSeen Count From SubobjectPath Type Reason Message + --------- -------- ----- ---- ------------- -------- ------ ------- + 33s 33s 1 {deployment-controller } Normal ScalingReplicaSet Scaled up replica set mysql-63082529 to 1 + +1. List the pods created by the Deployment: + + kubectl get pods -l app=mysql + + NAME READY STATUS RESTARTS AGE + mysql-63082529-2z3ki 1/1 Running 0 3m + +1. Inspect the Persistent Volume: + + kubectl describe pv mysql-pv + + Name: mysql-pv + Labels: + Status: Bound + Claim: default/mysql-pv-claim + Reclaim Policy: Retain + Access Modes: RWO + Capacity: 20Gi + Message: + Source: + Type: GCEPersistentDisk (a Persistent Disk resource in Google Compute Engine) + PDName: mysql-disk + FSType: ext4 + Partition: 0 + ReadOnly: false + No events. + +1. Inspect the PersistentVolumeClaim: + + kubectl describe pvc mysql-pv-claim + + Name: mysql-pv-claim + Namespace: default + Status: Bound + Volume: mysql-pv + Labels: + Capacity: 20Gi + Access Modes: RWO + No events. + +### Accessing the MySQL instance + +The preceding YAML file creates a service that +allows other Pods in the cluster to access the database. The Service option +`clusterIP: None` lets the Service DNS name resolve directly to the +Pod's IP address. This is optimal when you have only one Pod +behind a Service and you don't intend to increase the number of Pods. + +Run a MySQL client to connect to the server: + +``` +kubectl run -it --rm --image=mysql:5.6 mysql-client -- mysql -h mysql -ppassword +``` + +This command creates a new Pod in the cluster running a mysql client +and connects it to the server through the Service. If it connects, you +know your stateful MySQL database is up and running. + +``` +Waiting for pod default/mysql-client-274442439-zyp6i to be running, status is Pending, pod ready: false +If you don't see a command prompt, try pressing enter. + +mysql> +``` + +### Updating + +The image or any other part of the Deployment can be updated as usual +with the `kubectl apply` command. Here are some precautions that are +specific to stateful apps: + +* Don't scale the app. This setup is for single-instance apps + only. The underlying PersistentVolume can only be mounted to one + Pod. For clustered stateful apps, see the + [StatefulSet documentation](/docs/user-guide/petset/). +* Use `strategy:` `type: Recreate` in the Deployment configuration + YAML file. This instructs Kubernetes to _not_ use rolling + updates. Rolling updates will not work, as you cannot have more than + one Pod running at a time. The `Recreate` strategy will stop the + first pod before creating a new one with the updated configuration. + +### Deleting a deployment + +Delete the deployed objects by name: + +``` +kubectl delete deployment,svc mysql +kubectl delete pvc mysql-pv-claim +kubectl delete pv mysql-pv +``` + +Also, if you are using Compute Engine disks: + +``` +gcloud compute disks delete mysql-disk +``` + +{% endcapture %} + + +{% capture whatsnext %} + +* Learn more about [Deployment objects](/docs/user-guide/deployments/). + +* Learn more about [Deploying applications](/docs/user-guide/deploying-applications/) + +* [kubectl run documentation](/docs/user-guide/kubectl/kubectl_run/) + +* [Volumes](/docs/user-guide/volumes/) and [Persistent Volumes](/docs/user-guide/persistent-volumes/) + +{% endcapture %} + +{% include templates/tutorial.md %} From 34a31f41ba232a9ac280d8fbad18a1ad19fb4dce Mon Sep 17 00:00:00 2001 From: Ahmet Alp Balkan Date: Thu, 17 Nov 2016 15:07:13 -0800 Subject: [PATCH 52/87] List Deployment next to ReplicationController RestartPolicy section gives an example of what kind of controller should be used for what type of workload. However it only lists rc for web-server type workloads. Adding Deployment next to rc as well. Signed-off-by: Ahmet Alp Balkan --- docs/user-guide/pod-states.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/user-guide/pod-states.md b/docs/user-guide/pod-states.md index b29270e5f8..8f745e9f56 100644 --- a/docs/user-guide/pod-states.md +++ b/docs/user-guide/pod-states.md @@ -66,8 +66,8 @@ The possible values for RestartPolicy are `Always`, `OnFailure`, or `Never`. If Three types of controllers are currently available: - Use a [`Job`](/docs/user-guide/jobs/) for pods which are expected to terminate (e.g. batch computations). -- Use a [`ReplicationController`](/docs/user-guide/replication-controller/) for pods which are not expected to - terminate (e.g. web servers). +- Use a [`ReplicationController`](/docs/user-guide/replication-controller/) or [`Deployment`](/docs/user-guide/deployments/) + for pods which are not expected to terminate (e.g. web servers). - Use a [`DaemonSet`](/docs/admin/daemons/): Use for pods which need to run 1 per machine because they provide a machine-specific system service. If you are unsure whether to use ReplicationController or Daemon, then see [Daemon Set versus From 9e25fddd86e02a261215160d50d1822029a80e37 Mon Sep 17 00:00:00 2001 From: Francois Deppierraz Date: Fri, 18 Nov 2016 00:10:14 +0100 Subject: [PATCH 53/87] typo kuberntes -> kubernetes --- docs/user-guide/node-selection/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/node-selection/index.md b/docs/user-guide/node-selection/index.md index 49d30b51c9..725848b544 100644 --- a/docs/user-guide/node-selection/index.md +++ b/docs/user-guide/node-selection/index.md @@ -173,7 +173,7 @@ on node N if node N has a label with key `failure-domain.beta.kubernetes.io/zone such that there is at least one node in the cluster with key `failure-domain.beta.kubernetes.io/zone` and value V that is running a pod that has a label with key "security" and value "S1".) The pod anti-affinity rule says that the pod cannot schedule onto a node if that node is already running a pod with label -having key "security" and value "S2". (If the `topologyKey` were `failure-domain.beta.kuberntes.io/zone` then +having key "security" and value "S2". (If the `topologyKey` were `failure-domain.beta.kubernetes.io/zone` then it would mean that the pod cannot schedule onto a node if that node is in the same zone as a pod with label having key "security" and value "S2".) See the [design doc](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/podaffinity.md). for many more examples of pod affinity and anti-affinity, both the `requiredDuringSchedulingIgnoredDuringExecution` From 1b6580fe5323c9ceec47ed0e675af3dce84b5b6d Mon Sep 17 00:00:00 2001 From: Spencer Smith Date: Thu, 17 Nov 2016 20:40:30 -0500 Subject: [PATCH 54/87] Very basic kubeadm reset command formatting fix. --- docs/getting-started-guides/kubeadm.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting-started-guides/kubeadm.md b/docs/getting-started-guides/kubeadm.md index bf40b9c283..a122180b23 100644 --- a/docs/getting-started-guides/kubeadm.md +++ b/docs/getting-started-guides/kubeadm.md @@ -260,7 +260,7 @@ If there is a firewall, make sure it exposes this port to the internet before yo * To undo what `kubeadm` did, simply run: - # kubeadm reset + # kubeadm reset If you wish to start over, run `systemctl start kubelet` followed by `kubeadm init` or `kubeadm join`. From c117759cfe006eaf7cdb116f6457667a75597971 Mon Sep 17 00:00:00 2001 From: Ahmet Alp Balkan Date: Thu, 17 Nov 2016 19:20:11 -0800 Subject: [PATCH 55/87] service-accounts.md: Better paragraph flow It was not easy to follow what this paragraph is talking about without clicking the link. --- docs/user-guide/service-accounts.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/user-guide/service-accounts.md b/docs/user-guide/service-accounts.md index 3fa9252034..1df4635da9 100644 --- a/docs/user-guide/service-accounts.md +++ b/docs/user-guide/service-accounts.md @@ -31,8 +31,8 @@ you get the raw json or yaml for a pod you have created (e.g. `kubectl get pods/podname -o yaml`), you can see the `spec.serviceAccount` field has been [automatically set](/docs/user-guide/working-with-resources/#resources-are-automatically-modified). -You can access the API using a proxy or with a client library, as described in -[Accessing the Cluster](/docs/user-guide/accessing-the-cluster/#accessing-the-api-from-a-pod). +With service accounts, you can access the API inside the pod using a proxy or with a client library, +as described in [Accessing the Cluster](/docs/user-guide/accessing-the-cluster/#accessing-the-api-from-a-pod). ## Using Multiple Service Accounts. From 7f7fa5ac738b858e6e1271aa696c28f0494c3d64 Mon Sep 17 00:00:00 2001 From: Ahmet Alp Balkan Date: Thu, 17 Nov 2016 19:31:59 -0800 Subject: [PATCH 56/87] Fix broken links to some github paths --- docs/user-guide/accessing-the-cluster.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/user-guide/accessing-the-cluster.md b/docs/user-guide/accessing-the-cluster.md index 6f78ab5293..63134b4909 100644 --- a/docs/user-guide/accessing-the-cluster.md +++ b/docs/user-guide/accessing-the-cluster.md @@ -129,7 +129,7 @@ To use it, * Write an application atop of the client-go clients. Note that client-go defines its own API objects, so if needed, please import API definitions from client-go rather than from the main repository, e.g., `import "k8s.io/client-go/1.4/pkg/api/v1"` is correct. The Go client can use the same [kubeconfig file](/docs/user-guide/kubeconfig-file) -as the kubectl CLI does to locate and authenticate to the apiserver. See this [example](https://github.com/kubernetes/client-go/examples/out-of-cluster.go): +as the kubectl CLI does to locate and authenticate to the apiserver. See this [example](https://github.com/kubernetes/client-go/blob/master/examples/out-of-cluster/main.go): ```golang import ( @@ -183,7 +183,8 @@ From within a pod the recommended ways to connect to API are: in any container of the pod can access it. See this [example of using kubectl proxy in a pod](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/kubectl-container/). - use the Go client library, and create a client using the `client.NewInCluster()` factory. - This handles locating and authenticating to the apiserver. [example](https://github.com/kubernetes/client-go/examples/in-cluster.go) + This handles locating and authenticating to the apiserver. See this [example of using Go client + library in a pod](https://github.com/kubernetes/client-go/blob/master/examples/in-cluster/main.go). In each case, the credentials of the pod are used to communicate securely with the apiserver. From 9f0531ba75451a87f0350127d551906b16601a1f Mon Sep 17 00:00:00 2001 From: scotty Date: Fri, 18 Nov 2016 10:27:25 -0800 Subject: [PATCH 57/87] added "contribute to codebase" and "download k8s" buttons to the footer hide left nav area in docs if no nav is present update text link styles for better readability --- _includes/footer.html | 2 ++ _layouts/docwithnav.html | 27 +++++++++++++++++++++++++++ _sass/_base.sass | 19 +++++++++++++++++++ _sass/_reset.sass | 2 +- 4 files changed, 49 insertions(+), 1 deletion(-) diff --git a/_includes/footer.html b/_includes/footer.html index 15dacb72e0..e7fa36d26a 100644 --- a/_includes/footer.html +++ b/_includes/footer.html @@ -20,6 +20,8 @@ Events Calendar

© {{ 'now' | date: "%Y" }} Kubernetes
diff --git a/_layouts/docwithnav.html b/_layouts/docwithnav.html index 8ca0906065..c15077857a 100755 --- a/_layouts/docwithnav.html +++ b/_layouts/docwithnav.html @@ -80,6 +80,33 @@ })(window,document,'script','//www.google-analytics.com/analytics.js','ga'); ga('create', 'UA-36037335-10', 'auto'); ga('send', 'pageview'); + + // hide docs nav area if no nav is present + (function () { + window.addEventListener('DOMContentLoaded', init) + + // play nice with our neighbors + function init() { + window.removeEventListener('DOMContentLoaded', init) + hideNav() + } + + function hideNav(toc){ + if (!toc) toc = document.querySelector('#docsToc') + var container = toc.querySelector('.container') + + // container is built dynamically, so it may not be present on the first runloop + if (container) { + if (container.childElementCount === 0) { + document.getElementById('docsContent').style.width = '100%' + } + } else { + requestAnimationFrame(function () { + hideNav(toc) + }) + } + } + })(); diff --git a/_sass/_base.sass b/_sass/_base.sass index a8ac4b47c4..0feee846ff 100644 --- a/_sass/_base.sass +++ b/_sass/_base.sass @@ -389,6 +389,14 @@ footer display: block height: 0 overflow: hidden + + &.button + background-image: none + width: auto + height: auto + + &:hover + color: $blue a.twitter background-position: 0 0 @@ -874,8 +882,19 @@ dd img max-width: 100% + a + font-weight: 700 + text-decoration: underline + + a:visited + color: blueviolet + a.button border-radius: 2px + text-decoration: none + + &:visited + color: white a.issue margin-left: 20px diff --git a/_sass/_reset.sass b/_sass/_reset.sass index 9f4a43a68e..2a8bb1b6d2 100755 --- a/_sass/_reset.sass +++ b/_sass/_reset.sass @@ -15,7 +15,7 @@ ul, li ul margin: 0 padding: 0 - + a text-decoration: none From 88e32adc986da0cf4263aa2b670ac640febe0ed3 Mon Sep 17 00:00:00 2001 From: steveperry-53 Date: Thu, 17 Nov 2016 17:07:34 -0800 Subject: [PATCH 58/87] Write new task: Determining the Reason for Pod Failure. --- _data/tasks.yml | 8 ++ .../determine-reason-pod-failure.md | 110 ++++++++++++++++++ .../termination.yaml | 10 ++ 3 files changed, 128 insertions(+) create mode 100644 docs/tasks/debug-application-cluster/determine-reason-pod-failure.md create mode 100644 docs/tasks/debug-application-cluster/termination.yaml diff --git a/_data/tasks.yml b/_data/tasks.yml index 4c6cd1b709..9898fae32f 100644 --- a/_data/tasks.yml +++ b/_data/tasks.yml @@ -15,6 +15,14 @@ toc: section: - title: Using Port Forwarding to Access Applications in a Cluster path: /docs/tasks/access-application-cluster/port-forward-access-application-cluster/ + + +- title: Debugging Applications in a Cluster + section: + - title: Determining the Reason for Pod Failure + path: /docs/tasks/debug-application-cluster/determine-reason-pod-failure/ + + - title: Accessing the Kubernetes API section: - title: Using an HTTP Proxy to Access the Kubernetes API diff --git a/docs/tasks/debug-application-cluster/determine-reason-pod-failure.md b/docs/tasks/debug-application-cluster/determine-reason-pod-failure.md new file mode 100644 index 0000000000..f0f611e235 --- /dev/null +++ b/docs/tasks/debug-application-cluster/determine-reason-pod-failure.md @@ -0,0 +1,110 @@ +--- +--- + +{% capture overview %} + +This page shows how to write and read a Container +termination message. + +Termination messages provide a way for containers to write +information about fatal events to a location where it can +be easily retrieved and surfaced by tools like dashboards +and monitoring software. In most cases, information that you +put in a termination message should also be written to +the general +[Kubernetes logs](/docs/user-guide/logging/). + +{% endcapture %} + + +{% capture prerequisites %} + +{% include task-tutorial-prereqs.md %} + +{% endcapture %} + + +{% capture steps %} + +### Writing and reading a termination message + +In this exercise, you create a Pod that runs one container. +The configuration file specifies a command that runs when +the container starts. + +{% include code.html language="yaml" file="termination.yaml" ghlink="/docs/tasks/debug-pod-container/termination.yaml" %} + +1. Create a Pod based on the YAML configuration file: + + export REPO=https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/master + kubectl create -f $REPO/docs/tasks/debug-pod-container/termination.yaml + + In the YAML file, in the `cmd` and `args` fields, you can see that the + container sleeps for 10 seconds and then writes "Sleep expired" to + the `/dev/termination-log` file. After the container writes + the "Sleep expired" message, it terminates. + +1. Display information about the Pod: + + kubectl get pod termination-demo + + Repeat the preceding command until the Pod is no longer running. + +1. Display detailed information about the Pod: + + kubectl get pod --output=yaml + + The output includes the "Sleep expired" message: + + apiVersion: v1 + kind: Pod + ... + lastState: + terminated: + containerID: ... + exitCode: 0 + finishedAt: ... + message: | + Sleep expired + ... + +1. Use a Go template to filter the output so that it includes +only the termination message: + +``` +{% raw %} kubectl get pod termination-demo -o go-template="{{range .status.containerStatuses}}{{.lastState.terminated.message}}{{end}}"{% endraw %} +``` + +### Setting the termination log file + +By default Kubernetes retrieves termination messages from +`/dev/termination-log`. To change this to a different file, +specify a `terminationMessagePath` field for your Container. + +For example, suppose your Container writes termination messages to +`/tmp/my-log`, and you want Kubernetes to retrieve those messages. +Set `terminationMessagePath` as shown here: + + apiVersion: v1 + kind: Pod + metadata: + name: msg-path-demo + spec: + containers: + - name: msg-path-demo-container + image: debian + terminationMessagePath: "/tmp/my-log" + +{% endcapture %} + +{% capture whatsnext %} + +* See the `terminationMessagePath` field in + [Container](/docs/api-reference/v1/definitions#_v1_container). +* Learn about [retrieving logs](/docs/user-guide/logging/). +* Learn about [Go templates](https://golang.org/pkg/text/template/). + +{% endcapture %} + + +{% include templates/task.md %} diff --git a/docs/tasks/debug-application-cluster/termination.yaml b/docs/tasks/debug-application-cluster/termination.yaml new file mode 100644 index 0000000000..3f63748f72 --- /dev/null +++ b/docs/tasks/debug-application-cluster/termination.yaml @@ -0,0 +1,10 @@ +apiVersion: v1 +kind: Pod +metadata: + name: termination-demo +spec: + containers: + - name: termination-demo-container + image: debian + command: ["/bin/sh"] + args: ["-c", "sleep 10 && echo Sleep expired > /dev/termination-log"] From 39d1962b287b0636748c695f5199ebbbf625d20b Mon Sep 17 00:00:00 2001 From: scotty Date: Fri, 18 Nov 2016 12:09:19 -0800 Subject: [PATCH 59/87] unbold a tags in #docsContent --- _sass/_base.sass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_sass/_base.sass b/_sass/_base.sass index 0feee846ff..1eabc9ac14 100644 --- a/_sass/_base.sass +++ b/_sass/_base.sass @@ -883,7 +883,7 @@ dd max-width: 100% a - font-weight: 700 + //font-weight: 700 text-decoration: underline a:visited From 81b7f61c3b5d70a976574865014ba4eff3aaaffb Mon Sep 17 00:00:00 2001 From: scotty Date: Fri, 18 Nov 2016 12:56:07 -0800 Subject: [PATCH 60/87] disable TOC if there's only a link to the current page --- _layouts/docwithnav.html | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/_layouts/docwithnav.html b/_layouts/docwithnav.html index c15077857a..f3d885d246 100755 --- a/_layouts/docwithnav.html +++ b/_layouts/docwithnav.html @@ -81,7 +81,7 @@ ga('create', 'UA-36037335-10', 'auto'); ga('send', 'pageview'); - // hide docs nav area if no nav is present + // hide docs nav area if no nav is present, or if nav only contains a link to the current page (function () { window.addEventListener('DOMContentLoaded', init) @@ -97,7 +97,8 @@ // container is built dynamically, so it may not be present on the first runloop if (container) { - if (container.childElementCount === 0) { + if (container.childElementCount === 0 || toc.querySelectorAll('a.item').length === 1) { + toc.style.display = 'none' document.getElementById('docsContent').style.width = '100%' } } else { From 2a4501ed6306a71b320fc008ae629159c2eaa46d Mon Sep 17 00:00:00 2001 From: Jared Date: Fri, 18 Nov 2016 15:58:19 -0800 Subject: [PATCH 61/87] Removed confusing "Kubernetes" button from docs landing page. --- docs/index.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/docs/index.md b/docs/index.md index 0bdaa33b5c..c430dac710 100644 --- a/docs/index.md +++ b/docs/index.md @@ -5,9 +5,7 @@ assignees: --- -

The Kubernetes documentation can help you set up Kubernetes, learn about the system, or get your applications and workloads running on Kubernetes.

- -

Read the Kubernetes Overview

+

Kubernetes documentation can help you set up Kubernetes, learn about the system, or get your applications and workloads running on Kubernetes. To learn the basics of what Kubernetes is and how it works, read "What is Kubernetes".

Interactive Tutorial

@@ -40,4 +38,4 @@ assignees:

Tools

-

The tools page contains a list of native and third-party tools for Kubernetes.

\ No newline at end of file +

The tools page contains a list of native and third-party tools for Kubernetes.

From 0fa87b8caacb733041b4e5d89b10de91df735492 Mon Sep 17 00:00:00 2001 From: Johanan Lieberman Date: Sat, 19 Nov 2016 16:52:30 +0200 Subject: [PATCH 62/87] Mention kubectl as a requirement for running kops When deploying a cluster and kubectl is not installed, kops exits with an error. Need to mention kubectl as a requirement in the docs. --- docs/getting-started-guides/kops.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/getting-started-guides/kops.md b/docs/getting-started-guides/kops.md index 0cc28fb547..a02a829471 100644 --- a/docs/getting-started-guides/kops.md +++ b/docs/getting-started-guides/kops.md @@ -26,6 +26,12 @@ a building block. kops builds on the kubeadm work. ### (1/5) Install kops +#### Requirements + +You must have [kubectl](http://kubernetes.io/docs/getting-started-guides/kubectl/) installed in order for kops to work. + +#### Installation + Download kops from the [releases page](https://github.com/kubernetes/kops/releases) (it is also easy to build from source): On MacOS: From afcf0dd8935c4e2e50044ea00d110166ec1b7bbb Mon Sep 17 00:00:00 2001 From: Casey Davenport Date: Sat, 19 Nov 2016 14:49:53 -0500 Subject: [PATCH 63/87] Remove outdated Calico guides in favor of kubeadm --- _data/guides.yml | 4 - .../coreos/bare_metal_calico.md | 209 -------- docs/getting-started-guides/index.md | 2 - docs/getting-started-guides/ubuntu-calico.md | 484 ------------------ 4 files changed, 699 deletions(-) delete mode 100644 docs/getting-started-guides/coreos/bare_metal_calico.md delete mode 100644 docs/getting-started-guides/ubuntu-calico.md diff --git a/_data/guides.yml b/_data/guides.yml index 85c5dd4103..9991179936 100644 --- a/_data/guides.yml +++ b/_data/guides.yml @@ -233,12 +233,8 @@ toc: path: /docs/getting-started-guides/centos/centos_manual_config/ - title: CoreOS path: /docs/getting-started-guides/coreos - - title: CoreOS with Calico - path: /docs/getting-started-guides/coreos/bare_metal_calico/ - title: Ubuntu path: /docs/getting-started-guides/ubuntu/ - - title: Ubuntu Nodes with Calico - path: /docs/getting-started-guides/ubuntu-calico/ - title: Validate Node Setup path: /docs/admin/node-conformance - title: Portable Multi-Node Cluster diff --git a/docs/getting-started-guides/coreos/bare_metal_calico.md b/docs/getting-started-guides/coreos/bare_metal_calico.md deleted file mode 100644 index 7c3f7ccca0..0000000000 --- a/docs/getting-started-guides/coreos/bare_metal_calico.md +++ /dev/null @@ -1,209 +0,0 @@ ---- - ---- - -This document describes how to deploy Kubernetes with Calico networking on _bare metal_ CoreOS. For more information on Project Calico, visit [projectcalico.org](http://projectcalico.org) and the [calico-containers repository](https://github.com/projectcalico/calico-containers). - -To install Calico on an existing Kubernetes cluster, or for more information on deploying Calico with Kubernetes in a number of other environments take a look at our supported [deployment guides](https://github.com/projectcalico/calico-containers/tree/master/docs/cni/kubernetes). - -Specifically, this guide will have you do the following: - -- Deploy a Kubernetes master node on CoreOS using cloud-config. -- Deploy two Kubernetes compute nodes with Calico Networking using cloud-config. -- Configure `kubectl` to access your cluster. - -The resulting cluster will use SSL between Kubernetes components. It will run the SkyDNS service and kube-ui, and be fully conformant with the Kubernetes v1.1 conformance tests. - -## Prerequisites and Assumptions - -- At least three bare-metal machines (or VMs) to work with. This guide will configure them as follows: - - 1 Kubernetes Master - - 2 Kubernetes Nodes -- Your nodes should have IP connectivity to each other and the internet. -- This guide assumes a DHCP server on your network to assign server IPs. -- This guide uses `192.168.0.0/16` as the subnet from which pod IP addresses are assigned. If this overlaps with your host subnet, you will need to configure Calico to use a different [IP pool](https://github.com/projectcalico/calico-containers/blob/master/docs/calicoctl/pool.md#calicoctl-pool-commands). - -## Cloud-config - -This guide will use [cloud-config](https://coreos.com/docs/cluster-management/setup/cloudinit-cloud-config/) to configure each of the nodes in our Kubernetes cluster. - -We'll use two cloud-config files: -- `master-config.yaml`: cloud-config for the Kubernetes master -- `node-config.yaml`: cloud-config for each Kubernetes node - -## Download CoreOS - -Download the stable CoreOS bootable ISO from the [CoreOS website](https://coreos.com/docs/running-coreos/platforms/iso/). - -## Configure the Kubernetes Master - -1. Once you've downloaded the ISO image, burn the ISO to a CD/DVD/USB key and boot from it (if using a virtual machine you can boot directly from the ISO). Once booted, you should be automatically logged in as the `core` user at the terminal. At this point CoreOS is running from the ISO and it hasn't been installed yet. - -2. *On another machine*, download the [master cloud-config template](https://raw.githubusercontent.com/projectcalico/calico-cni/k8s-1.1-docs/samples/kubernetes/cloud-config/master-config-template.yaml) and save it as `master-config.yaml`. - -3. Replace the following variables in the `master-config.yaml` file. - - - ``: The public key you will use for SSH access to this server. See [generating ssh keys](https://help.github.com/articles/generating-ssh-keys/) - -4. Copy the edited `master-config.yaml` to your Kubernetes master machine (using a USB stick, for example). - -5. The CoreOS bootable ISO comes with a tool called `coreos-install` which will allow us to install CoreOS and configure the machine using a cloud-config file. The following command will download and install stable CoreOS using the `master-config.yaml` file we just created for configuration. Run this on the Kubernetes master. - - > **Warning:** this is a destructive operation that erases disk `sda` on your server. - - ```shell - sudo coreos-install -d /dev/sda -C stable -c master-config.yaml - ``` - -6. Once complete, restart the server and boot from `/dev/sda` (you may need to remove the ISO image). When it comes back up, you should have SSH access as the `core` user using the public key provided in the `master-config.yaml` file. - -### Configure TLS - -The master requires the CA certificate, `ca.pem`; its own certificate, `apiserver.pem` and its private key, `apiserver-key.pem`. This [CoreOS guide](https://coreos.com/kubernetes/docs/latest/openssl.html) explains how to generate these. - -1. Generate the necessary certificates for the master. This [guide for generating Kubernetes TLS Assets](https://coreos.com/kubernetes/docs/latest/openssl.html) explains how to use OpenSSL to generate the required assets. - -2. Send the three files to your master host (using `scp` for example). - -3. Move them to the `/etc/kubernetes/ssl` folder and ensure that only the root user can read the key: - - ```shell - # Move keys - sudo mkdir -p /etc/kubernetes/ssl/ - sudo mv -t /etc/kubernetes/ssl/ ca.pem apiserver.pem apiserver-key.pem - - # Set Permissions - sudo chmod 600 /etc/kubernetes/ssl/apiserver-key.pem - sudo chown root:root /etc/kubernetes/ssl/apiserver-key.pem - ``` - -4. Restart the kubelet to pick up the changes: - - ```shell - sudo systemctl restart kubelet - ``` - -## Configure the compute nodes - -The following steps will set up a single Kubernetes node for use as a compute host. Run these steps to deploy each Kubernetes node in your cluster. - -1. Boot up the node machine using the bootable ISO we downloaded earlier. You should be automatically logged in as the `core` user. - -2. Make a copy of the [node cloud-config template](https://raw.githubusercontent.com/projectcalico/calico-cni/k8s-1.1-docs/samples/kubernetes/cloud-config/node-config-template.yaml) for this machine. - -3. Replace the following placeholders in the `node-config.yaml` file to match your deployment. - - - ``: Hostname for this node (e.g. kube-node1, kube-node2) - - ``: The public key you will use for SSH access to this server. - - ``: The IPv4 address of the Kubernetes master. - -4. Replace the following placeholders with the contents of their respective files. - - - ``: Complete contents of `ca.pem` - - ``: Complete contents of `ca-key.pem` - - > **Important:** in a production deployment, embedding the secret key in cloud-config is a bad idea! In production you should use an appropriate secret manager. - - > **Important:** Make sure you indent the entire file to match the indentation of the placeholder. For example: - > - > ```shell - > - path: /etc/kubernetes/ssl/ca.pem - > owner: core - > permissions: 0644 - > content: | - > - > ``` - > - > should look like this once the certificate is in place: - > - > ```shell - > - path: /etc/kubernetes/ssl/ca.pem - > owner: core - > permissions: 0644 - > content: | - > -----BEGIN CERTIFICATE----- - > MIIC9zCCAd+gAwIBAgIJAJMnVnhVhy5pMA0GCSqGSIb3DQEBCwUAMBIxEDAOBgNV - > ...... - > QHwi1rNc8eBLNrd4BM/A1ZeDVh/Q9KxN+ZG/hHIXhmWKgN5wQx6/81FIFg== - > -----END CERTIFICATE----- - > ``` - -5. Move the modified `node-config.yaml` to your Kubernetes node machine and install and configure CoreOS on the node using the following command. - - > **Warning:** this is a destructive operation that erases disk `sda` on your server. - - ```shell - sudo coreos-install -d /dev/sda -C stable -c node-config.yaml - ``` - -6. Once complete, restart the server and boot into `/dev/sda`. When it comes back up, you should have SSH access as the `core` user using the public key provided in the `node-config.yaml` file. It will take some time for the node to be fully configured. - -## Configure Kubeconfig - -To administer your cluster from a separate host, you will need the client and admin certificates generated earlier (`ca.pem`, `admin.pem`, `admin-key.pem`). With certificates in place, run the following commands with the appropriate filepaths. - -```shell -kubectl config set-cluster calico-cluster --server=https:// --certificate-authority= -kubectl config set-credentials calico-admin --certificate-authority= --client-key= --client-certificate= -kubectl config set-context calico --cluster=calico-cluster --user=calico-admin -kubectl config use-context calico -``` - -Check your work with `kubectl get nodes`. - -## Install the DNS Addon - -Most Kubernetes deployments will require the DNS addon for service discovery. To install DNS, create the skydns service and replication controller provided. - -```shell -kubectl create -f https://raw.githubusercontent.com/projectcalico/calico-cni/k8s-1.1-docs/samples/kubernetes/master/dns/skydns.yaml -``` - -## Install the Kubernetes UI Addon (Optional) - -The Kubernetes UI can be installed using `kubectl` to run the following manifest file. - -```shell -kubectl create -f https://raw.githubusercontent.com/projectcalico/calico-cni/k8s-1.1-docs/samples/kubernetes/master/kube-ui/kube-ui.yaml -``` - -## Launch other Services With Calico-Kubernetes - -At this point, you have a fully functioning cluster running on Kubernetes with a master and two nodes networked with Calico. You can now follow any of the [standard documentation](https://github.com/kubernetes/kubernetes/tree/{{page.version}}.0/examples/) to set up other services on your cluster. - -## Connectivity to outside the cluster - -Because containers in this guide have private `192.168.0.0/16` IPs, you will need NAT to allow connectivity between containers and the internet. However, in a production data center deployment, NAT is not always necessary, since Calico can peer with the data center's border routers over BGP. - -### NAT on the nodes - -The simplest method for enabling connectivity from containers to the internet is to use outgoing NAT on your Kubernetes nodes. - -Calico can provide outgoing NAT for containers. To enable it, use the following `calicoctl` command: - -```shell -ETCD_AUTHORITY= calicoctl pool add --nat-outgoing -``` - -By default, `` will be `192.168.0.0/16`. You can find out which pools have been configured with the following command: - -```shell -ETCD_AUTHORITY= calicoctl pool show -``` - -### NAT at the border router - -In a data center environment, it is recommended to configure Calico to peer with the border routers over BGP. This means that the container IPs will be routable anywhere in the data center, and so NAT is not needed on the nodes (though it may be enabled at the data center edge to allow outbound-only internet connectivity). - -The Calico documentation contains more information on how to configure Calico to [peer with existing infrastructure](https://github.com/projectcalico/calico-containers/blob/master/docs/ExternalConnectivity.md). - -## Support Level - - -IaaS Provider | Config. Mgmt | OS | Networking | Docs | Conforms | Support Level --------------------- | ------------ | ------ | ---------- | --------------------------------------------- | ---------| ---------------------------- -Bare-metal | CoreOS | CoreOS | Calico | [docs](/docs/getting-started-guides/coreos/bare_metal_calico) | | Community ([@caseydavenport](https://github.com/caseydavenport)) - - -For support level information on all solutions, see the [Table of solutions](/docs/getting-started-guides/#table-of-solutions) chart. - diff --git a/docs/getting-started-guides/index.md b/docs/getting-started-guides/index.md index a728980e50..49497a2b1d 100644 --- a/docs/getting-started-guides/index.md +++ b/docs/getting-started-guides/index.md @@ -140,7 +140,6 @@ AWS | CoreOS | CoreOS | flannel | [docs](/docs/gettin GCE | CoreOS | CoreOS | flannel | [docs](/docs/getting-started-guides/coreos) | | Community ([@pires](https://github.com/pires)) Vagrant | CoreOS | CoreOS | flannel | [docs](/docs/getting-started-guides/coreos) | | Community ([@pires](https://github.com/pires), [@AntonioMeireles](https://github.com/AntonioMeireles)) Bare-metal (Offline) | CoreOS | CoreOS | flannel | [docs](/docs/getting-started-guides/coreos/bare_metal_offline) | | Community ([@jeffbean](https://github.com/jeffbean)) -Bare-metal | CoreOS | CoreOS | Calico | [docs](/docs/getting-started-guides/coreos/bare_metal_calico) | | Community ([@caseydavenport](https://github.com/caseydavenport)) CloudStack | Ansible | CoreOS | flannel | [docs](/docs/getting-started-guides/cloudstack) | | Community ([@runseb](https://github.com/runseb)) Vmware vSphere | Saltstack | Debian | OVS | [docs](/docs/getting-started-guides/vsphere) | | Community ([@imkin](https://github.com/imkin)) Vmware Photon | Saltstack | Debian | OVS | [docs](/docs/getting-started-guides/photon-controller) | | Community ([@alainroy](https://github.com/alainroy)) @@ -150,7 +149,6 @@ OpenStack/HPCloud | Juju | Ubuntu | flannel | [docs](/docs/gettin Joyent | Juju | Ubuntu | flannel | [docs](/docs/getting-started-guides/juju) | | [Community](https://github.com/whitmo/bundle-kubernetes) ( [@whit](https://github.com/whitmo), [@matt](https://github.com/mbruzek), [@chuck](https://github.com/chuckbutler) ) AWS | Saltstack | Debian | AWS | [docs](/docs/getting-started-guides/aws) | | Community ([@justinsb](https://github.com/justinsb)) AWS | kops | Debian | AWS | [docs](https://github.com/kubernetes/kops) | | Community ([@justinsb](https://github.com/justinsb)) -Bare-metal | custom | Ubuntu | Calico | [docs](/docs/getting-started-guides/ubuntu-calico) | | Community ([@djosborne](https://github.com/djosborne)) Bare-metal | custom | Ubuntu | flannel | [docs](/docs/getting-started-guides/ubuntu) | | Community ([@resouer](https://github.com/resouer), [@WIZARD-CXY](https://github.com/WIZARD-CXY)) libvirt/KVM | CoreOS | CoreOS | libvirt/KVM | [docs](/docs/getting-started-guides/libvirt-coreos) | | Community ([@lhuard1A](https://github.com/lhuard1A)) oVirt | | | | [docs](/docs/getting-started-guides/ovirt) | | Community ([@simon3z](https://github.com/simon3z)) diff --git a/docs/getting-started-guides/ubuntu-calico.md b/docs/getting-started-guides/ubuntu-calico.md deleted file mode 100644 index 2eaf7a7aea..0000000000 --- a/docs/getting-started-guides/ubuntu-calico.md +++ /dev/null @@ -1,484 +0,0 @@ ---- - ---- - -This document describes how to deploy Kubernetes with Calico networking from scratch on _bare metal_ Ubuntu. For more information on Project Calico, visit [projectcalico.org](http://projectcalico.org) and the [calico-containers repository](https://github.com/projectcalico/calico-containers). - -To install Calico on an existing Kubernetes cluster, or for more information on deploying Calico with Kubernetes in a number of other environments take a look at our supported [deployment guides](https://github.com/projectcalico/calico-containers/tree/master/docs/cni/kubernetes). - -This guide will set up a simple Kubernetes cluster with a single Kubernetes master and two Kubernetes nodes. We'll run Calico's etcd cluster on the master and install the Calico daemon on the master and nodes. - -## Prerequisites and Assumptions - -- This guide uses `systemd` for process management. Ubuntu 15.04 supports systemd natively as do a number of other Linux distributions. -- All machines should have Docker >= 1.7.0 installed. - - To install Docker on Ubuntu, follow [these instructions](https://docs.docker.com/installation/ubuntulinux/) -- All machines should have connectivity to each other and the internet. -- This guide assumes a DHCP server on your network to assign server IPs. -- This guide uses `192.168.0.0/16` as the subnet from which pod IP addresses are assigned. If this overlaps with your host subnet, you will need to configure Calico to use a different [IP pool](https://github.com/projectcalico/calico-containers/blob/master/docs/calicoctl/pool.md#calicoctl-pool-commands). -- This guide assumes that none of the hosts have been configured with any Kubernetes or Calico software. -- This guide will set up a secure, TLS-authenticated API server. - -## Set up the master - -### Configure TLS - -The master requires the root CA public key, `ca.pem`; the apiserver certificate, `apiserver.pem` and its private key, `apiserver-key.pem`. - -1. Create the file `openssl.cnf` with the following contents. - - ```conf - [req] - req_extensions = v3_req - distinguished_name = req_distinguished_name - [req_distinguished_name] - [ v3_req ] - basicConstraints = CA:FALSE - keyUsage = nonRepudiation, digitalSignature, keyEncipherment - subjectAltName = @alt_names - [alt_names] - DNS.1 = kubernetes - DNS.2 = kubernetes.default - IP.1 = 10.100.0.1 - IP.2 = ${MASTER_IPV4} - ``` - -> Replace ${MASTER_IPV4} with the Master's IP address on which the Kubernetes API will be accessible. - -2. Generate the necessary TLS assets. - - ```shell - # Generate the root CA. - openssl genrsa -out ca-key.pem 2048 - openssl req -x509 -new -nodes -key ca-key.pem -days 10000 -out ca.pem -subj "/CN=kube-ca" - - # Generate the API server keypair. - openssl genrsa -out apiserver-key.pem 2048 - openssl req -new -key apiserver-key.pem -out apiserver.csr -subj "/CN=kube-apiserver" -config openssl.cnf - openssl x509 -req -in apiserver.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out apiserver.pem -days 365 -extensions v3_req -extfile openssl.cnf - ``` - -3. You should now have the following three files: `ca.pem`, `apiserver.pem`, and `apiserver-key.pem`. Send the three files to your master host (using `scp` for example). -4. Move them to the `/etc/kubernetes/ssl` folder and ensure that only the root user can read the key: - - ```shell - # Move keys - sudo mkdir -p /etc/kubernetes/ssl/ - sudo mv -t /etc/kubernetes/ssl/ ca.pem apiserver.pem apiserver-key.pem - - # Set permissions - sudo chmod 600 /etc/kubernetes/ssl/apiserver-key.pem - sudo chown root:root /etc/kubernetes/ssl/apiserver-key.pem - ``` - -### Install Calico's etcd on the master - -Calico needs its own etcd cluster to store its state. In this guide we install a single-node cluster on the master server. - -> Note: In a production deployment we recommend running a distributed etcd cluster for redundancy. In this guide, we use a single etcd for simplicitly. - -1. Download the template manifest file: - - ```shell - wget https://raw.githubusercontent.com/projectcalico/calico-cni/k8s-1.1-docs/samples/kubernetes/master/calico-etcd.manifest - ``` - -2. Replace all instances of `` in the `calico-etcd.manifest` file with your master's IP address. - -3. Then, move the file to the `/etc/kubernetes/manifests` directory. This will not have any effect until we later run the kubelet, but Calico seems to tolerate the lack of its etcd in the interim. - - ```shell - sudo mv -f calico-etcd.manifest /etc/kubernetes/manifests - ``` - -### Install Calico on the master - -We need to install Calico on the master. This allows the master to route packets to the pods on other nodes. - -1. Install the `calicoctl` tool: - - ```shell - wget https://github.com/projectcalico/calico-containers/releases/download/v0.15.0/calicoctl - chmod +x calicoctl - sudo mv calicoctl /usr/bin - ``` - -2. Prefetch the calico/node container (this ensures that the Calico service starts immediately when we enable it): - - ```shell - sudo docker pull calico/node:v0.15.0 - ``` - -3. Download the `network-environment` template from the `calico-kubernetes` repository: - - ```shell - wget -O network-environment https://raw.githubusercontent.com/projectcalico/calico-cni/k8s-1.1-docs/samples/kubernetes/master/network-environment-template - ``` - -4. Edit `network-environment` to represent this node's settings: - - - Replace `` with the IP address of the master. This should be the source IP address used to reach the Kubernetes worker nodes. - -5. Move `network-environment` into `/etc`: - - ```shell - sudo mv -f network-environment /etc - ``` - -6. Install, enable, and start the `calico-node` service: - - ```shell - sudo wget -N -P /etc/systemd https://raw.githubusercontent.com/projectcalico/calico-cni/k8s-1.1-docs/samples/kubernetes/common/calico-node.service - sudo systemctl enable /etc/systemd/calico-node.service - sudo systemctl start calico-node.service - ``` - -### Install Kubernetes on the Master - -We'll use the `kubelet` to bootstrap the Kubernetes master. - -1. Download and install the `kubelet` and `kubectl` binaries: - - ```shell - sudo wget -N -P /usr/bin http://storage.googleapis.com/kubernetes-release/release/v1.1.4/bin/linux/amd64/kubectl - sudo wget -N -P /usr/bin http://storage.googleapis.com/kubernetes-release/release/v1.1.4/bin/linux/amd64/kubelet - sudo chmod +x /usr/bin/kubelet /usr/bin/kubectl - ``` - -2. Install the `kubelet` systemd unit file and start the `kubelet`: - - ```shell - # Install the unit file - sudo wget -N -P /etc/systemd https://raw.githubusercontent.com/projectcalico/calico-cni/k8s-1.1-docs/samples/kubernetes/master/kubelet.service - - # Enable the unit file so that it runs on boot - sudo systemctl enable /etc/systemd/kubelet.service - - # Start the kubelet service - sudo systemctl start kubelet.service - ``` - -3. Download and install the master manifest file, which will start the Kubernetes master services automatically: - - ```shell - sudo mkdir -p /etc/kubernetes/manifests - sudo wget -N -P /etc/kubernetes/manifests https://raw.githubusercontent.com/projectcalico/calico-cni/k8s-1.1-docs/samples/kubernetes/master/kubernetes-master.manifest - ``` - -4. Check the progress by running `docker ps`. After a while, you should see the `etcd`, `apiserver`, `controller-manager`, `scheduler`, and `kube-proxy` containers running. - - > Note: it may take some time for all the containers to start. Don't worry if `docker ps` doesn't show any containers for a while or if some containers start before others. - -## Set up the nodes - -The following steps should be run on each Kubernetes node. - -### Configure TLS - -Worker nodes require three keys: `ca.pem`, `worker.pem`, and `worker-key.pem`. We've already generated -`ca.pem` and `ca-key.pem` for use on the Master. The worker public/private keypair should be generated for each Kubernetes node. - -1. Create the file `worker-openssl.cnf` with the following contents. - - ```conf - [req] - req_extensions = v3_req - distinguished_name = req_distinguished_name - [req_distinguished_name] - [ v3_req ] - basicConstraints = CA:FALSE - keyUsage = nonRepudiation, digitalSignature, keyEncipherment - subjectAltName = @alt_names - [alt_names] - IP.1 = $ENV::WORKER_IP - ``` - -2. Generate the necessary TLS assets for this worker. This relies on the worker's IP address, and the `ca.pem` and `ca-key.pem` files generated earlier in the guide. - - ```shell - # Export this worker's IP address. - export WORKER_IP= - ``` - - ```shell - # Generate keys. - openssl genrsa -out worker-key.pem 2048 - openssl req -new -key worker-key.pem -out worker.csr -subj "/CN=worker-key" -config worker-openssl.cnf - openssl x509 -req -in worker.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out worker.pem -days 365 -extensions v3_req -extfile worker-openssl.cnf - ``` - -3. Send the three files (`ca.pem`, `worker.pem`, and `worker-key.pem`) to the host (using scp, for example). - -4. Move the files to the `/etc/kubernetes/ssl` folder with the appropriate permissions: - - ```shell - # Move keys - sudo mkdir -p /etc/kubernetes/ssl/ - sudo mv -t /etc/kubernetes/ssl/ ca.pem worker.pem worker-key.pem - - # Set permissions - sudo chmod 600 /etc/kubernetes/ssl/worker-key.pem - sudo chown root:root /etc/kubernetes/ssl/worker-key.pem - ``` - -### Configure the kubelet worker - -1. With your certs in place, create a kubeconfig for worker authentication in `/etc/kubernetes/worker-kubeconfig.yaml`; replace `` with the IP address of the master: - - ```yaml - apiVersion: v1 - kind: Config - clusters: - - name: local - cluster: - server: https://:443 - certificate-authority: /etc/kubernetes/ssl/ca.pem - users: - - name: kubelet - user: - client-certificate: /etc/kubernetes/ssl/worker.pem - client-key: /etc/kubernetes/ssl/worker-key.pem - contexts: - - context: - cluster: local - user: kubelet - name: kubelet-context - current-context: kubelet-context - ``` - -### Install Calico on the node - -On your compute nodes, it is important that you install Calico before Kubernetes. We'll install Calico using the provided `calico-node.service` systemd unit file: - -1. Install the `calicoctl` binary: - - ```shell - wget https://github.com/projectcalico/calico-containers/releases/download/v0.15.0/calicoctl - chmod +x calicoctl - sudo mv calicoctl /usr/bin - ``` - -2. Fetch the calico/node container: - - ```shell - sudo docker pull calico/node:v0.15.0 - ``` - -3. Download the `network-environment` template from the `calico-cni` repository: - - ```shell - wget -O network-environment https://raw.githubusercontent.com/projectcalico/calico-cni/k8s-1.1-docs/samples/kubernetes/node/network-environment-template - ``` - -4. Edit `network-environment` to represent this node's settings: - - - Replace `` with the IP address of the node. - - Replace `` with the IP or hostname of the master. - -5. Move `network-environment` into `/etc`: - - ```shell - sudo mv -f network-environment /etc - ``` - -6. Install the `calico-node` service: - - ```shell - sudo wget -N -P /etc/systemd https://raw.githubusercontent.com/projectcalico/calico-cni/k8s-1.1-docs/samples/kubernetes/common/calico-node.service - sudo systemctl enable /etc/systemd/calico-node.service - sudo systemctl start calico-node.service - ``` - -7. Install the Calico CNI plugins: - - ```shell - sudo mkdir -p /opt/cni/bin/ - sudo wget -N -P /opt/cni/bin/ https://github.com/projectcalico/calico-cni/releases/download/v1.0.0/calico - sudo wget -N -P /opt/cni/bin/ https://github.com/projectcalico/calico-cni/releases/download/v1.0.0/calico-ipam - sudo chmod +x /opt/cni/bin/calico /opt/cni/bin/calico-ipam - ``` - -8. Create a CNI network configuration file, which tells Kubernetes to create a network named `calico-k8s-network` and to use the calico plugins for that network. Create file `/etc/cni/net.d/10-calico.conf` with the following contents, replacing `` with the IP of the master (this file should be the same on each node): - - ```shell - # Make the directory structure. - mkdir -p /etc/cni/net.d - - # Make the network configuration file - cat >/etc/cni/net.d/10-calico.conf <:6666", - "log_level": "info", - "ipam": { - "type": "calico-ipam" - } - } - EOF - ``` - - Since this is the only network we create, it will be used by default by the kubelet. - -9. Verify that Calico started correctly: - - ```shell - calicoctl status - ``` - - should show that Felix (Calico's per-node agent) is running and the there should be a BGP status line for each other node that you've configured and the master. The "Info" column should show "Established": - - ``` - $ calicoctl status - calico-node container is running. Status: Up 15 hours - Running felix version 1.3.0rc5 - - IPv4 BGP status - +---------------+-------------------+-------+----------+-------------+ - | Peer address | Peer type | State | Since | Info | - +---------------+-------------------+-------+----------+-------------+ - | 172.18.203.41 | node-to-node mesh | up | 17:32:26 | Established | - | 172.18.203.42 | node-to-node mesh | up | 17:32:25 | Established | - +---------------+-------------------+-------+----------+-------------+ - - IPv6 BGP status - +--------------+-----------+-------+-------+------+ - | Peer address | Peer type | State | Since | Info | - +--------------+-----------+-------+-------+------+ - +--------------+-----------+-------+-------+------+ - ``` - - If the "Info" column shows "Active" or some other value then Calico is having difficulty connecting to the other host. Check the IP address of the peer is correct and check that Calico is using the correct local IP address (set in the `network-environment` file above). - -### Install Kubernetes on the Node - -1. Download and Install the kubelet binary: - - ```shell - sudo wget -N -P /usr/bin http://storage.googleapis.com/kubernetes-release/release/v1.1.4/bin/linux/amd64/kubelet - sudo chmod +x /usr/bin/kubelet - ``` - -2. Install the `kubelet` systemd unit file: - - ```shell - # Download the unit file. - sudo wget -N -P /etc/systemd https://raw.githubusercontent.com/projectcalico/calico-cni/k8s-1.1-docs/samples/kubernetes/node/kubelet.service - - # Enable and start the unit files so that they run on boot - sudo systemctl enable /etc/systemd/kubelet.service - sudo systemctl start kubelet.service - ``` - -3. Download the `kube-proxy` manifest: - - ```shell - wget https://raw.githubusercontent.com/projectcalico/calico-cni/k8s-1.1-docs/samples/kubernetes/node/kube-proxy.manifest - ``` - -4. In that file, replace `` with your master's IP. Then move it into place: - - ```shell - sudo mkdir -p /etc/kubernetes/manifests/ - sudo mv kube-proxy.manifest /etc/kubernetes/manifests/ - ``` - -## Configure kubectl remote access - -To administer your cluster from a separate host (e.g your laptop), you will need the root CA generated earlier, as well as an admin public/private keypair (`ca.pem`, `admin.pem`, `admin-key.pem`). Run the following steps on the machine which you will use to control your cluster. - -1. Download the kubectl binary. - - ```shell - sudo wget -N -P /usr/bin http://storage.googleapis.com/kubernetes-release/release/v1.1.4/bin/linux/amd64/kubectl - sudo chmod +x /usr/bin/kubectl - ``` - -2. Generate the admin public/private keypair. - -3. Export the necessary variables, substituting in correct values for your machine. - - ```shell - # Export the appropriate paths. - export CA_CERT_PATH= - export ADMIN_CERT_PATH= - export ADMIN_KEY_PATH= - - # Export the Master's IP address. - export MASTER_IPV4= - ``` - -4. Configure your host `kubectl` with the admin credentials: - - ```shell - kubectl config set-cluster calico-cluster --server=https://${MASTER_IPV4} --certificate-authority=${CA_CERT_PATH} - kubectl config set-credentials calico-admin --certificate-authority=${CA_CERT_PATH} --client-key=${ADMIN_KEY_PATH} --client-certificate=${ADMIN_CERT_PATH} - kubectl config set-context calico --cluster=calico-cluster --user=calico-admin - kubectl config use-context calico - ``` - -Check your work with `kubectl get nodes`, which should succeed and display the nodes. - -## Install the DNS Addon - -Most Kubernetes deployments will require the DNS addon for service discovery. To install DNS, create the skydns service and replication controller provided. This step makes use of the kubectl configuration made above. - -```shell -kubectl create -f https://raw.githubusercontent.com/projectcalico/calico-cni/k8s-1.1-docs/samples/kubernetes/master/dns/skydns.yaml -``` - -## Install the Kubernetes UI Addon (Optional) - -The Kubernetes UI can be installed using `kubectl` to run the following manifest file. - -```shell -kubectl create -f https://raw.githubusercontent.com/projectcalico/calico-cni/k8s-1.1-docs/samples/kubernetes/master/kube-ui/kube-ui.yaml -``` - -Note: The Kubernetes UI addon is deprecated and has been replaced with Kubernetes dashboard. You can install it by running: - -```shell -kubectl create -f https://rawgit.com/kubernetes/dashboard/master/src/deploy/kubernetes-dashboard.yaml -``` - -You can find the docs at [Kubernetes Dashboard](https://github.com/kubernetes/dashboard) - -## Launch other Services With Calico-Kubernetes - -At this point, you have a fully functioning cluster running on Kubernetes with a master and two nodes networked with Calico. You can now follow any of the [standard documentation](https://github.com/kubernetes/kubernetes/tree/{{page.version}}.0/examples/) to set up other services on your cluster. - -## Connectivity to outside the cluster - -Because containers in this guide have private `192.168.0.0/16` IPs, you will need NAT to allow connectivity between containers and the internet. However, in a production data center deployment, NAT is not always necessary, since Calico can peer with the data center's border routers over BGP. - -### NAT on the nodes - -The simplest method for enabling connectivity from containers to the internet is to use outgoing NAT on your Kubernetes nodes. - -Calico can provide outgoing NAT for containers. To enable it, use the following `calicoctl` command: - -```shell -ETCD_AUTHORITY= calicoctl pool add --nat-outgoing -``` - -By default, `` will be `192.168.0.0/16`. You can find out which pools have been configured with the following command: - -```shell -ETCD_AUTHORITY= calicoctl pool show -``` - -### NAT at the border router - -In a data center environment, it is recommended to configure Calico to peer with the border routers over BGP. This means that the container IPs will be routable anywhere in the data center, and so NAT is not needed on the nodes (though it may be enabled at the data center edge to allow outbound-only internet connectivity). - -The Calico documentation contains more information on how to configure Calico to [peer with existing infrastructure](https://github.com/projectcalico/calico-containers/blob/master/docs/ExternalConnectivity.md). - -## Support Level - - -IaaS Provider | Config. Mgmt | OS | Networking | Docs | Conforms | Support Level --------------------- | ------------ | ------ | ---------- | --------------------------------------------- | ---------| ---------------------------- -Bare-metal | custom | Ubuntu | Calico | [docs](/docs/getting-started-guides/ubuntu-calico) | | Community ([@djosborne](https://github.com/djosborne)) - -For support level information on all solutions, see the [Table of solutions](/docs/getting-started-guides/#table-of-solutions) chart. - From a8e129af8315bd612e0206dfc7ef044121500c6b Mon Sep 17 00:00:00 2001 From: Casey Davenport Date: Sat, 19 Nov 2016 17:52:21 -0500 Subject: [PATCH 64/87] Sort network plugins alphabetically. (#1583) --- docs/admin/addons.md | 10 ++++--- docs/admin/networking.md | 40 ++++++++++++-------------- docs/getting-started-guides/scratch.md | 8 +++--- 3 files changed, 29 insertions(+), 29 deletions(-) diff --git a/docs/admin/addons.md b/docs/admin/addons.md index f2bd18aaa5..192b0c86a8 100644 --- a/docs/admin/addons.md +++ b/docs/admin/addons.md @@ -7,18 +7,20 @@ Add-ons extend the functionality of Kubernetes. This page lists some of the available add-ons and links to their respective installation instructions. +Add-ons in each section are sorted alphabetically - the ordering does not imply any preferential status. + ## Networking and Network Policy -* [Weave Net](https://github.com/weaveworks/weave-kube) provides networking and network policy, will carry on working on both sides of a network partition, and does not require an external database. -* [Calico](http://docs.projectcalico.org/v1.5/getting-started/kubernetes/installation/hosted/) is a secure L3 networking and network policy provider. -* [Flannel](https://github.com/coreos/flannel/blob/master/Documentation/kube-flannel.yml) is a overlay network provider that can be used with Kubernetes. +* [Calico](http://docs.projectcalico.org/v1.6/getting-started/kubernetes/installation/hosted/) is a secure L3 networking and network policy provider. * [Canal](https://github.com/tigera/canal/tree/master/k8s-install/kubeadm) unites Flannel and Calico, providing networking and network policy. +* [Flannel](https://github.com/coreos/flannel/blob/master/Documentation/kube-flannel.yml) is a overlay network provider that can be used with Kubernetes. * [Romana](http://romana.io) is a Layer 3 networking solution for pod networks that also supports the [NetworkPolicy API](/docs/user-guide/networkpolicies/). Kubeadm add-on installation details available [here](https://github.com/romana/romana/tree/master/containerize). +* [Weave Net](https://github.com/weaveworks/weave-kube) provides networking and network policy, will carry on working on both sides of a network partition, and does not require an external database. ## Visualization & Control -* [Weave Scope](https://www.weave.works/documentation/scope-latest-installing/#k8s) is a tool for graphically visualizing your containers, pods, services etc. Use it in conjunction with a [Weave Cloud account](https://cloud.weave.works/) or host the UI yourself. * [Dashboard](https://github.com/kubernetes/dashboard#kubernetes-dashboard) is a dashboard web interface for Kubernetes. +* [Weave Scope](https://www.weave.works/documentation/scope-latest-installing/#k8s) is a tool for graphically visualizing your containers, pods, services etc. Use it in conjunction with a [Weave Cloud account](https://cloud.weave.works/) or host the UI yourself. ## Legacy Add-ons diff --git a/docs/admin/networking.md b/docs/admin/networking.md index 0cfd9d7989..406148ee9e 100644 --- a/docs/admin/networking.md +++ b/docs/admin/networking.md @@ -100,8 +100,19 @@ existence or non-existence of host ports. There are a number of ways that this network model can be implemented. This document is not an exhaustive study of the various methods, but hopefully serves as an introduction to various technologies and serves as a jumping-off point. -If some techniques become vastly preferable to others, we might detail them more -here. + +The following networking options are sorted alphabetically - the order does not +imply any preferential status. + +### Contiv + +[Contiv](https://github.com/contiv/netplugin) provides configurable networking (native l3 using BGP, overlay using vxlan, classic l2, or Cisco-SDN/ACI) for various use cases. [Contiv](http://contiv.io) is all open sourced. + +### Flannel + +[Flannel](https://github.com/coreos/flannel#flannel) is a very simple overlay +network that satisfies the Kubernetes requirements. Many +people have reported success with Flannel and Kubernetes. ### Google Compute Engine (GCE) @@ -158,29 +169,12 @@ Follow the "With Linux Bridge devices" section of [this very nice tutorial](http://blog.oddbit.com/2014/08/11/four-ways-to-connect-a-docker/) from Lars Kellogg-Stedman. -### Weave Net from Weaveworks - -[Weave Net](https://www.weave.works/products/weave-net/) is a -resilient and simple to use network for Kubernetes and its hosted applications. -Weave Net runs as a [CNI plug-in](https://www.weave.works/docs/net/latest/cni-plugin/) -or stand-alone. In either version, it doesn't require any configuration or extra code -to run, and in both cases, the network provides one IP address per pod - as is standard for Kubernetes. - - -### Flannel - -[Flannel](https://github.com/coreos/flannel#flannel) is a very simple overlay -network that satisfies the Kubernetes requirements. It installs in minutes and -should get you up and running if the above techniques are not working. Many -people have reported success with Flannel and Kubernetes. - ### OpenVSwitch [OpenVSwitch](/docs/admin/ovs-networking) is a somewhat more mature but also complicated way to build an overlay network. This is endorsed by several of the "Big Shops" for networking. - ### Project Calico [Project Calico](https://github.com/projectcalico/calico-containers/blob/master/docs/cni/kubernetes/README.md) is an open source container networking provider and network policy engine. @@ -193,9 +187,13 @@ Calico can also be run in policy enforcement mode in conjunction with other netw [Romana](http://romana.io) is an open source network and security automation solution that lets you deploy Kubernetes without an overlay network. Romana supports Kubernetes [Network Policy](/docs/user-guide/networkpolicies/) to provide isolation across network namespaces. -### Contiv +### Weave Net from Weaveworks -[Contiv](https://github.com/contiv/netplugin) provides configurable networking (native l3 using BGP, overlay using vxlan, classic l2, or Cisco-SDN/ACI) for various use cases. [Contiv](http://contiv.io) is all open sourced. +[Weave Net](https://www.weave.works/products/weave-net/) is a +resilient and simple to use network for Kubernetes and its hosted applications. +Weave Net runs as a [CNI plug-in](https://www.weave.works/docs/net/latest/cni-plugin/) +or stand-alone. In either version, it doesn't require any configuration or extra code +to run, and in both cases, the network provides one IP address per pod - as is standard for Kubernetes. ## Other reading diff --git a/docs/getting-started-guides/scratch.md b/docs/getting-started-guides/scratch.md index 84b0ffb5af..3fc23ec3dc 100644 --- a/docs/getting-started-guides/scratch.md +++ b/docs/getting-started-guides/scratch.md @@ -81,12 +81,12 @@ to implement one of the above options: - **Use a network plugin which is called by Kubernetes** - Kubernetes supports the [CNI](https://github.com/containernetworking/cni) network plugin interface. - - There are a number of solutions which provide plugins for Kubernetes: + - There are a number of solutions which provide plugins for Kubernetes (listed alphabetically): + - [Calico](http://docs.projectcalico.org/) - [Flannel](https://github.com/coreos/flannel) - - [Calico](https://github.com/projectcalico/calico-containers) - - [Weave](https://weave.works/) - - [Romana](http://romana.io/) - [Open vSwitch (OVS)](http://openvswitch.org/) + - [Romana](http://romana.io/) + - [Weave](http://weave.works/) - [More found here](/docs/admin/networking#how-to-achieve-this) - You can also write your own. - **Compile support directly into Kubernetes** From 083f07af4aa58d0cd288d856bcea72c8cbc6fa90 Mon Sep 17 00:00:00 2001 From: Ahmet Alp Balkan Date: Sat, 19 Nov 2016 18:27:18 -0800 Subject: [PATCH 65/87] Fix code block rendering issue Added an extra new line before triple-backtick for proper rendering on the website. --- docs/admin/dns.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/admin/dns.md b/docs/admin/dns.md index cc132201aa..9cc55ab7e7 100644 --- a/docs/admin/dns.md +++ b/docs/admin/dns.md @@ -160,6 +160,7 @@ kubectl get pods busybox ``` You should see: + ``` NAME READY STATUS RESTARTS AGE busybox 1/1 Running 0 From eb10ccc643d29ce6f73812e10a96d61dc0c85183 Mon Sep 17 00:00:00 2001 From: erikus Date: Sat, 19 Nov 2016 20:19:21 -0800 Subject: [PATCH 66/87] Update "kubectl docker" command to include "--" "--" is now required after "docker" and before extra args. --- docs/hellonode.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/hellonode.md b/docs/hellonode.md index b5b67a195d..fd80d950cd 100755 --- a/docs/hellonode.md +++ b/docs/hellonode.md @@ -145,7 +145,7 @@ docker stop hello_tutorial Now that the image works as intended and is all tagged with your `$PROJECT_ID`, we can push it to the [Google Container Registry](https://cloud.google.com/tools/container-registry/), a private repository for your Docker images accessible from every Google Cloud project (but also from outside Google Cloud Platform) : ```shell -gcloud docker push gcr.io/$PROJECT_ID/hello-node:v1 +gcloud docker -- push gcr.io/$PROJECT_ID/hello-node:v1 ``` If all goes well, you should be able to see the container image listed in the console: *Compute > Container Engine > Container Registry*. We now have a project-wide Docker image available which Kubernetes can access and orchestrate. From de9197661e8b43bea1956f36bd7dc68ca06e9865 Mon Sep 17 00:00:00 2001 From: Randy Fay Date: Sun, 20 Nov 2016 09:41:34 -0700 Subject: [PATCH 67/87] Fix broken path in pv configuration, minor doc improvement --- docs/user-guide/persistent-volumes/volumes/local-01.yaml | 2 +- docs/user-guide/persistent-volumes/walkthrough.md | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/user-guide/persistent-volumes/volumes/local-01.yaml b/docs/user-guide/persistent-volumes/volumes/local-01.yaml index 25acecca93..a465c65149 100644 --- a/docs/user-guide/persistent-volumes/volumes/local-01.yaml +++ b/docs/user-guide/persistent-volumes/volumes/local-01.yaml @@ -10,4 +10,4 @@ spec: accessModes: - ReadWriteOnce hostPath: - path: "/somepath/data01" + path: "/tmp/data01" diff --git a/docs/user-guide/persistent-volumes/walkthrough.md b/docs/user-guide/persistent-volumes/walkthrough.md index 30b24feac2..c82a37f7d0 100644 --- a/docs/user-guide/persistent-volumes/walkthrough.md +++ b/docs/user-guide/persistent-volumes/walkthrough.md @@ -27,7 +27,7 @@ for ease of development and testing. You'll create a local `HostPath` for this support local storage on the host at this time. There is no guarantee your pod ends up on the correct node where the `HostPath` resides. ```shell -# This will be nginx's webroot +# This will be nginx's webroot; execute this on the node where your pod will run. $ mkdir /tmp/data01 $ echo 'I love Kubernetes storage!' > /tmp/data01/index.html ``` @@ -125,4 +125,4 @@ I love Kubernetes storage! Hopefully this simple guide is enough to get you started with PersistentVolumes. If you have any questions, join the team on [Slack](/docs/troubleshooting/#slack) and ask! -Enjoy! \ No newline at end of file +Enjoy! From 27d7d4fba39274ec46e251959a2491998334182a Mon Sep 17 00:00:00 2001 From: Casey Davenport Date: Sun, 20 Nov 2016 17:14:33 -0500 Subject: [PATCH 68/87] Update Calico links to new docs --- docs/admin/networking.md | 2 +- docs/getting-started-guides/network-policy/calico.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/admin/networking.md b/docs/admin/networking.md index 406148ee9e..de2629bb6f 100644 --- a/docs/admin/networking.md +++ b/docs/admin/networking.md @@ -177,7 +177,7 @@ complicated way to build an overlay network. This is endorsed by several of the ### Project Calico -[Project Calico](https://github.com/projectcalico/calico-containers/blob/master/docs/cni/kubernetes/README.md) is an open source container networking provider and network policy engine. +[Project Calico](http://docs.projectcalico.org/) is an open source container networking provider and network policy engine. Calico provides a highly scalable networking and network policy solution for connecting Kubernetes pods based on the same IP networking principles as the internet. Calico can be deployed without encapsulation or overlays to provide high-performance, high-scale data center networking. Calico also provides fine-grained, intent based network security policy for Kubernetes pods via its distributed firewall. diff --git a/docs/getting-started-guides/network-policy/calico.md b/docs/getting-started-guides/network-policy/calico.md index 8efc32f068..be84644aa6 100644 --- a/docs/getting-started-guides/network-policy/calico.md +++ b/docs/getting-started-guides/network-policy/calico.md @@ -12,7 +12,7 @@ export KUBE_NODE_OS_DISTRIBUTION=debian curl -sS https://get.k8s.io | bash ``` -See the [Calico documentation](https://github.com/projectcalico/calico-containers/tree/master/docs/cni/kubernetes#getting-started) for more options to deploy Calico with Kubernetes. +See the [Calico documentation](http://docs.projectcalico.org/) for more options to deploy Calico with Kubernetes. Once your cluster using Calico is running, you should see a collection of pods running in the `kube-system` Namespace that support Kubernetes NetworkPolicy. From 5f8adf06160c108cfe7d85ed55dd6a0acec08b6f Mon Sep 17 00:00:00 2001 From: Luke Faraone Date: Mon, 21 Nov 2016 02:41:46 +0000 Subject: [PATCH 69/87] Use HTTPS links for all command line invocations #107 fixed a few, but not all were correctly updated. --- docs/getting-started-guides/minikube.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/getting-started-guides/minikube.md b/docs/getting-started-guides/minikube.md index 2362424440..093eb60a57 100644 --- a/docs/getting-started-guides/minikube.md +++ b/docs/getting-started-guides/minikube.md @@ -67,19 +67,19 @@ to run commands against the cluster. ```shell # linux/amd64 -curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ +curl -Lo kubectl https://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ # linux/386 -curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/386/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ +curl -Lo kubectl https://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/386/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ # linux/arm -curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/arm/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ +curl -Lo kubectl https://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/arm/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ # linux/arm64 -curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/arm64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ +curl -Lo kubectl https://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/arm64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ #linux/ppc64le -curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/ppc64le/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ +curl -Lo kubectl https://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/ppc64le/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ # OS X/amd64 -curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/darwin/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ +curl -Lo kubectl https://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/darwin/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ # OS X/386 -curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/darwin/386/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ +curl -Lo kubectl https://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/darwin/386/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ ``` The generic download path is: From 6a4646817155dede79bc34bff9789d1f73d2076c Mon Sep 17 00:00:00 2001 From: shapeliang Date: Mon, 21 Nov 2016 15:56:30 +0800 Subject: [PATCH 70/87] Modify some hyperlink location in binary_release.md --- docs/getting-started-guides/binary_release.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/getting-started-guides/binary_release.md b/docs/getting-started-guides/binary_release.md index d3b2886f87..7fdf4a2da0 100644 --- a/docs/getting-started-guides/binary_release.md +++ b/docs/getting-started-guides/binary_release.md @@ -33,7 +33,7 @@ cd kubernetes make release ``` -For more details on the release process see the [`build/` directory](http://releases.k8s.io/{{page.githubbranch}}/build/) +For more details on the release process see the [`build/`](http://releases.k8s.io/{{page.githubbranch}}/build/) directory ### Download Kubernetes and automatically set up a default cluster @@ -57,4 +57,4 @@ Possible values for `YOUR_PROVIDER` include: * `vsphere` - VMWare VSphere * `rackspace` - Rackspace -For the complete, up-to-date list of providers supported by this script, see [the `/cluster` folder in the main Kubernetes repo](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/cluster), where each folder represents a possible value for `YOUR_PROVIDER`. If you don't see your desired provider, try looking at our [getting started guides](/docs/getting-started-guides); there's a good chance we have docs for them. +For the complete, up-to-date list of providers supported by this script, see the [`/cluster`](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/cluster) folder in the main Kubernetes repo, where each folder represents a possible value for `YOUR_PROVIDER`. If you don't see your desired provider, try looking at our [getting started guides](/docs/getting-started-guides); there's a good chance we have docs for them. From 0e1935c6b9dcda52353b5f2ad2e35990812d425c Mon Sep 17 00:00:00 2001 From: "Daniel P. Berrange" Date: Mon, 21 Nov 2016 15:23:33 +0000 Subject: [PATCH 71/87] libvirt-coreos: fix typo s/libvert/libvirt/ --- docs/getting-started-guides/libvirt-coreos.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting-started-guides/libvirt-coreos.md b/docs/getting-started-guides/libvirt-coreos.md index 86b0707092..180d3f6c9c 100644 --- a/docs/getting-started-guides/libvirt-coreos.md +++ b/docs/getting-started-guides/libvirt-coreos.md @@ -121,7 +121,7 @@ setfacl -m g:kvm:--x ~ By default, the libvirt-coreos setup will create a single Kubernetes master and 3 Kubernetes nodes. Because the VM drives use Copy-on-Write and because of memory ballooning and KSM, there is a lot of resource over-allocation. -There is both an automated way and a manual, customizable way of setting up libvert Kubernetes clusters on CoreOS. +There is both an automated way and a manual, customizable way of setting up libvirt Kubernetes clusters on CoreOS. #### Automated setup From 5d6460b97953181bc001a11fc7dc5100dc90efcd Mon Sep 17 00:00:00 2001 From: Jared Date: Mon, 21 Nov 2016 13:23:58 -0800 Subject: [PATCH 72/87] Added link to DNS Admin Guide --- docs/user-guide/services/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/services/index.md b/docs/user-guide/services/index.md index 94faabcd1c..e20d072d05 100644 --- a/docs/user-guide/services/index.md +++ b/docs/user-guide/services/index.md @@ -345,7 +345,7 @@ can do a DNS SRV query for `"_http._tcp.my-service.my-ns"` to discover the port number for `"http"`. The Kubernetes DNS server is the only way to access services of type -`ExternalName`. +`ExternalName`. More information is available in the [DNS Admin Guide](http://kubernetes.io/docs/admin/dns/). ## Headless services From 3797f7015fee880e5829dbd140079bdd2942bb69 Mon Sep 17 00:00:00 2001 From: Jared Date: Mon, 21 Nov 2016 13:29:52 -0800 Subject: [PATCH 73/87] Update Weave Link --- docs/admin/addons.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/admin/addons.md b/docs/admin/addons.md index 192b0c86a8..1555f8263c 100644 --- a/docs/admin/addons.md +++ b/docs/admin/addons.md @@ -15,7 +15,7 @@ Add-ons in each section are sorted alphabetically - the ordering does not imply * [Canal](https://github.com/tigera/canal/tree/master/k8s-install/kubeadm) unites Flannel and Calico, providing networking and network policy. * [Flannel](https://github.com/coreos/flannel/blob/master/Documentation/kube-flannel.yml) is a overlay network provider that can be used with Kubernetes. * [Romana](http://romana.io) is a Layer 3 networking solution for pod networks that also supports the [NetworkPolicy API](/docs/user-guide/networkpolicies/). Kubeadm add-on installation details available [here](https://github.com/romana/romana/tree/master/containerize). -* [Weave Net](https://github.com/weaveworks/weave-kube) provides networking and network policy, will carry on working on both sides of a network partition, and does not require an external database. +* [Weave Net](https://www.weave.works/docs/net/latest/kube-addon/) provides networking and network policy, will carry on working on both sides of a network partition, and does not require an external database. ## Visualization & Control From a2dff8d4799eca424a94451e9c93503d6fce82bc Mon Sep 17 00:00:00 2001 From: Jared Date: Mon, 21 Nov 2016 14:07:09 -0800 Subject: [PATCH 74/87] Add TOC --- docs/admin/authentication.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/admin/authentication.md b/docs/admin/authentication.md index 6e13c28929..3d73f527bf 100644 --- a/docs/admin/authentication.md +++ b/docs/admin/authentication.md @@ -6,7 +6,8 @@ assignees: - deads2k --- - +* TOC +{:toc} ## Users in Kubernetes From 71f6381067688a53d6039bd4a737e650227f0f80 Mon Sep 17 00:00:00 2001 From: Jared Date: Mon, 21 Nov 2016 14:14:15 -0800 Subject: [PATCH 75/87] Update index.md --- docs/getting-started-guides/index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/getting-started-guides/index.md b/docs/getting-started-guides/index.md index b2da179cf9..b6ab61e388 100644 --- a/docs/getting-started-guides/index.md +++ b/docs/getting-started-guides/index.md @@ -43,7 +43,7 @@ clusters. [KCluster.io](https://kcluster.io) provides highly available and scalable managed Kubernetes clusters for AWS. -[Platform9.com](https://platform9.com/products/kubernetes/) offers managed Kubernetes on-premises or any public cloud, and provides 24/7 health monitoring and alerting. +[Platform9](https://platform9.com/products/kubernetes/) offers managed Kubernetes on-premises or any public cloud, and provides 24/7 health monitoring and alerting. ### Turn-key Cloud Solutions @@ -125,7 +125,7 @@ GKE | | | GCE | [docs](https://clou Stackpoint.io | | multi-support | multi-support | [docs](http://www.stackpointcloud.com) | | Commercial AppsCode.com | Saltstack | Debian | multi-support | [docs](https://appscode.com/products/cloud-deployment/) | | Commercial KCluster.io | | multi-support | multi-support | [docs](https://kcluster.io) | | Commercial -Platform9.com | | multi-support | multi-support | [docs](https://platform9.com/products/kubernetes/) | | Commercial +Platform9 | | multi-support | multi-support | [docs](https://platform9.com/products/kubernetes/) | | Commercial GCE | Saltstack | Debian | GCE | [docs](/docs/getting-started-guides/gce) | ['œ“][1] | Project Azure | CoreOS | CoreOS | Weave | [docs](/docs/getting-started-guides/coreos/azure/) | | Community ([@errordeveloper](https://github.com/errordeveloper), [@squillace](https://github.com/squillace), [@chanezon](https://github.com/chanezon), [@crossorigin](https://github.com/crossorigin)) Azure | Ignition | Ubuntu | Azure | [docs](/docs/getting-started-guides/azure) | | Community (Microsoft: [@brendandburns](https://github.com/brendandburns), [@colemickens](https://github.com/colemickens)) From b8d8c1b59612fb5aed5da60e3ada36f3dcbba7e5 Mon Sep 17 00:00:00 2001 From: Di Xu Date: Fri, 4 Nov 2016 13:05:31 +0800 Subject: [PATCH 76/87] add doc for experimental-keystone-ca-file flag --- docs/admin/authentication.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/docs/admin/authentication.md b/docs/admin/authentication.md index 6819677107..c00104c79f 100644 --- a/docs/admin/authentication.md +++ b/docs/admin/authentication.md @@ -367,6 +367,13 @@ option to the API server during startup. The plugin is implemented in `plugin/pkg/auth/authenticator/password/keystone/keystone.go` and currently uses basic auth to verify used by username and password. +If you have configured self-signed certificates for the Keystone server, +you may need to set the `--experimental-keystone-ca-file=SOMEFILE` option when +starting the Kubernetes API server. If you set the option, the Keystone +server's certificate is verified by one of the authorities in the +`experimental-keystone-ca-file`. Otherwise, the certificate is verified by +the host's root Certificate Authority. + For details on how to use keystone to manage projects and users, refer to the [Keystone documentation](http://docs.openstack.org/developer/keystone/). Please note that this plugin is still experimental, under active development, and likely From 54cd5d115b9de76dbb82dedb8f36615ecff64a81 Mon Sep 17 00:00:00 2001 From: craigbox Date: Tue, 22 Nov 2016 13:02:02 +0000 Subject: [PATCH 77/87] Update LICENSE to CC BY SA 4.0 Text copied from https://creativecommons.org/licenses/by/4.0/legalcode.txt. --- LICENSE | 532 ++++++++++++++++++++++++++++++++++++++------------------ 1 file changed, 363 insertions(+), 169 deletions(-) diff --git a/LICENSE b/LICENSE index 5ee2946660..06c608dcf4 100644 --- a/LICENSE +++ b/LICENSE @@ -1,201 +1,395 @@ -Apache License - Version 2.0, January 2004 - http://www.apache.org/licenses/ +Attribution 4.0 International - TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION +======================================================================= - 1. Definitions. +Creative Commons Corporation ("Creative Commons") is not a law firm and +does not provide legal services or legal advice. Distribution of +Creative Commons public licenses does not create a lawyer-client or +other relationship. Creative Commons makes its licenses and related +information available on an "as-is" basis. Creative Commons gives no +warranties regarding its licenses, any material licensed under their +terms and conditions, or any related information. Creative Commons +disclaims all liability for damages resulting from their use to the +fullest extent possible. - "License" shall mean the terms and conditions for use, reproduction, - and distribution as defined by Sections 1 through 9 of this document. +Using Creative Commons Public Licenses - "Licensor" shall mean the copyright owner or entity authorized by - the copyright owner that is granting the License. +Creative Commons public licenses provide a standard set of terms and +conditions that creators and other rights holders may use to share +original works of authorship and other material subject to copyright +and certain other rights specified in the public license below. The +following considerations are for informational purposes only, are not +exhaustive, and do not form part of our licenses. - "Legal Entity" shall mean the union of the acting entity and all - other entities that control, are controlled by, or are under common - control with that entity. For the purposes of this definition, - "control" means (i) the power, direct or indirect, to cause the - direction or management of such entity, whether by contract or - otherwise, or (ii) ownership of fifty percent (50%) or more of the - outstanding shares, or (iii) beneficial ownership of such entity. + Considerations for licensors: Our public licenses are + intended for use by those authorized to give the public + permission to use material in ways otherwise restricted by + copyright and certain other rights. Our licenses are + irrevocable. Licensors should read and understand the terms + and conditions of the license they choose before applying it. + Licensors should also secure all rights necessary before + applying our licenses so that the public can reuse the + material as expected. Licensors should clearly mark any + material not subject to the license. This includes other CC- + licensed material, or material used under an exception or + limitation to copyright. More considerations for licensors: + wiki.creativecommons.org/Considerations_for_licensors - "You" (or "Your") shall mean an individual or Legal Entity - exercising permissions granted by this License. + Considerations for the public: By using one of our public + licenses, a licensor grants the public permission to use the + licensed material under specified terms and conditions. If + the licensor's permission is not necessary for any reason--for + example, because of any applicable exception or limitation to + copyright--then that use is not regulated by the license. Our + licenses grant only permissions under copyright and certain + other rights that a licensor has authority to grant. Use of + the licensed material may still be restricted for other + reasons, including because others have copyright or other + rights in the material. A licensor may make special requests, + such as asking that all changes be marked or described. + Although not required by our licenses, you are encouraged to + respect those requests where reasonable. More_considerations + for the public: + wiki.creativecommons.org/Considerations_for_licensees - "Source" form shall mean the preferred form for making modifications, - including but not limited to software source code, documentation - source, and configuration files. +======================================================================= - "Object" form shall mean any form resulting from mechanical - transformation or translation of a Source form, including but - not limited to compiled object code, generated documentation, - and conversions to other media types. +Creative Commons Attribution 4.0 International Public License - "Work" shall mean the work of authorship, whether in Source or - Object form, made available under the License, as indicated by a - copyright notice that is included in or attached to the work - (an example is provided in the Appendix below). +By exercising the Licensed Rights (defined below), You accept and agree +to be bound by the terms and conditions of this Creative Commons +Attribution 4.0 International Public License ("Public License"). To the +extent this Public License may be interpreted as a contract, You are +granted the Licensed Rights in consideration of Your acceptance of +these terms and conditions, and the Licensor grants You such rights in +consideration of benefits the Licensor receives from making the +Licensed Material available under these terms and conditions. - "Derivative Works" shall mean any work, whether in Source or Object - form, that is based on (or derived from) the Work and for which the - editorial revisions, annotations, elaborations, or other modifications - represent, as a whole, an original work of authorship. For the purposes - of this License, Derivative Works shall not include works that remain - separable from, or merely link (or bind by name) to the interfaces of, - the Work and Derivative Works thereof. - "Contribution" shall mean any work of authorship, including - the original version of the Work and any modifications or additions - to that Work or Derivative Works thereof, that is intentionally - submitted to Licensor for inclusion in the Work by the copyright owner - or by an individual or Legal Entity authorized to submit on behalf of - the copyright owner. For the purposes of this definition, "submitted" - means any form of electronic, verbal, or written communication sent - to the Licensor or its representatives, including but not limited to - communication on electronic mailing lists, source code control systems, - and issue tracking systems that are managed by, or on behalf of, the - Licensor for the purpose of discussing and improving the Work, but - excluding communication that is conspicuously marked or otherwise - designated in writing by the copyright owner as "Not a Contribution." +Section 1 -- Definitions. - "Contributor" shall mean Licensor and any individual or Legal Entity - on behalf of whom a Contribution has been received by Licensor and - subsequently incorporated within the Work. + a. Adapted Material means material subject to Copyright and Similar + Rights that is derived from or based upon the Licensed Material + and in which the Licensed Material is translated, altered, + arranged, transformed, or otherwise modified in a manner requiring + permission under the Copyright and Similar Rights held by the + Licensor. For purposes of this Public License, where the Licensed + Material is a musical work, performance, or sound recording, + Adapted Material is always produced where the Licensed Material is + synched in timed relation with a moving image. - 2. Grant of Copyright License. Subject to the terms and conditions of - this License, each Contributor hereby grants to You a perpetual, - worldwide, non-exclusive, no-charge, royalty-free, irrevocable - copyright license to reproduce, prepare Derivative Works of, - publicly display, publicly perform, sublicense, and distribute the - Work and such Derivative Works in Source or Object form. + b. Adapter's License means the license You apply to Your Copyright + and Similar Rights in Your contributions to Adapted Material in + accordance with the terms and conditions of this Public License. - 3. Grant of Patent License. Subject to the terms and conditions of - this License, each Contributor hereby grants to You a perpetual, - worldwide, non-exclusive, no-charge, royalty-free, irrevocable - (except as stated in this section) patent license to make, have made, - use, offer to sell, sell, import, and otherwise transfer the Work, - where such license applies only to those patent claims licensable - by such Contributor that are necessarily infringed by their - Contribution(s) alone or by combination of their Contribution(s) - with the Work to which such Contribution(s) was submitted. If You - institute patent litigation against any entity (including a - cross-claim or counterclaim in a lawsuit) alleging that the Work - or a Contribution incorporated within the Work constitutes direct - or contributory patent infringement, then any patent licenses - granted to You under this License for that Work shall terminate - as of the date such litigation is filed. + c. Copyright and Similar Rights means copyright and/or similar rights + closely related to copyright including, without limitation, + performance, broadcast, sound recording, and Sui Generis Database + Rights, without regard to how the rights are labeled or + categorized. For purposes of this Public License, the rights + specified in Section 2(b)(1)-(2) are not Copyright and Similar + Rights. - 4. Redistribution. You may reproduce and distribute copies of the - Work or Derivative Works thereof in any medium, with or without - modifications, and in Source or Object form, provided that You - meet the following conditions: + d. Effective Technological Measures means those measures that, in the + absence of proper authority, may not be circumvented under laws + fulfilling obligations under Article 11 of the WIPO Copyright + Treaty adopted on December 20, 1996, and/or similar international + agreements. - (a) You must give any other recipients of the Work or - Derivative Works a copy of this License; and + e. Exceptions and Limitations means fair use, fair dealing, and/or + any other exception or limitation to Copyright and Similar Rights + that applies to Your use of the Licensed Material. - (b) You must cause any modified files to carry prominent notices - stating that You changed the files; and + f. Licensed Material means the artistic or literary work, database, + or other material to which the Licensor applied this Public + License. - (c) You must retain, in the Source form of any Derivative Works - that You distribute, all copyright, patent, trademark, and - attribution notices from the Source form of the Work, - excluding those notices that do not pertain to any part of - the Derivative Works; and + g. Licensed Rights means the rights granted to You subject to the + terms and conditions of this Public License, which are limited to + all Copyright and Similar Rights that apply to Your use of the + Licensed Material and that the Licensor has authority to license. - (d) If the Work includes a "NOTICE" text file as part of its - distribution, then any Derivative Works that You distribute must - include a readable copy of the attribution notices contained - within such NOTICE file, excluding those notices that do not - pertain to any part of the Derivative Works, in at least one - of the following places: within a NOTICE text file distributed - as part of the Derivative Works; within the Source form or - documentation, if provided along with the Derivative Works; or, - within a display generated by the Derivative Works, if and - wherever such third-party notices normally appear. The contents - of the NOTICE file are for informational purposes only and - do not modify the License. You may add Your own attribution - notices within Derivative Works that You distribute, alongside - or as an addendum to the NOTICE text from the Work, provided - that such additional attribution notices cannot be construed - as modifying the License. + h. Licensor means the individual(s) or entity(ies) granting rights + under this Public License. - You may add Your own copyright statement to Your modifications and - may provide additional or different license terms and conditions - for use, reproduction, or distribution of Your modifications, or - for any such Derivative Works as a whole, provided Your use, - reproduction, and distribution of the Work otherwise complies with - the conditions stated in this License. + i. Share means to provide material to the public by any means or + process that requires permission under the Licensed Rights, such + as reproduction, public display, public performance, distribution, + dissemination, communication, or importation, and to make material + available to the public including in ways that members of the + public may access the material from a place and at a time + individually chosen by them. - 5. Submission of Contributions. Unless You explicitly state otherwise, - any Contribution intentionally submitted for inclusion in the Work - by You to the Licensor shall be under the terms and conditions of - this License, without any additional terms or conditions. - Notwithstanding the above, nothing herein shall supersede or modify - the terms of any separate license agreement you may have executed - with Licensor regarding such Contributions. + j. Sui Generis Database Rights means rights other than copyright + resulting from Directive 96/9/EC of the European Parliament and of + the Council of 11 March 1996 on the legal protection of databases, + as amended and/or succeeded, as well as other essentially + equivalent rights anywhere in the world. - 6. Trademarks. This License does not grant permission to use the trade - names, trademarks, service marks, or product names of the Licensor, - except as required for reasonable and customary use in describing the - origin of the Work and reproducing the content of the NOTICE file. + k. You means the individual or entity exercising the Licensed Rights + under this Public License. Your has a corresponding meaning. - 7. Disclaimer of Warranty. Unless required by applicable law or - agreed to in writing, Licensor provides the Work (and each - Contributor provides its Contributions) on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or - implied, including, without limitation, any warranties or conditions - of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A - PARTICULAR PURPOSE. You are solely responsible for determining the - appropriateness of using or redistributing the Work and assume any - risks associated with Your exercise of permissions under this License. - 8. Limitation of Liability. In no event and under no legal theory, - whether in tort (including negligence), contract, or otherwise, - unless required by applicable law (such as deliberate and grossly - negligent acts) or agreed to in writing, shall any Contributor be - liable to You for damages, including any direct, indirect, special, - incidental, or consequential damages of any character arising as a - result of this License or out of the use or inability to use the - Work (including but not limited to damages for loss of goodwill, - work stoppage, computer failure or malfunction, or any and all - other commercial damages or losses), even if such Contributor - has been advised of the possibility of such damages. +Section 2 -- Scope. - 9. Accepting Warranty or Additional Liability. While redistributing - the Work or Derivative Works thereof, You may choose to offer, - and charge a fee for, acceptance of support, warranty, indemnity, - or other liability obligations and/or rights consistent with this - License. However, in accepting such obligations, You may act only - on Your own behalf and on Your sole responsibility, not on behalf - of any other Contributor, and only if You agree to indemnify, - defend, and hold each Contributor harmless for any liability - incurred by, or claims asserted against, such Contributor by reason - of your accepting any such warranty or additional liability. + a. License grant. - END OF TERMS AND CONDITIONS + 1. Subject to the terms and conditions of this Public License, + the Licensor hereby grants You a worldwide, royalty-free, + non-sublicensable, non-exclusive, irrevocable license to + exercise the Licensed Rights in the Licensed Material to: - APPENDIX: How to apply the Apache License to your work. + a. reproduce and Share the Licensed Material, in whole or + in part; and - To apply the Apache License to your work, attach the following - boilerplate notice, with the fields enclosed by brackets "{}" - replaced with your own identifying information. (Don't include - the brackets!) The text should be enclosed in the appropriate - comment syntax for the file format. We also recommend that a - file or class name and description of purpose be included on the - same "printed page" as the copyright notice for easier - identification within third-party archives. + b. produce, reproduce, and Share Adapted Material. - Copyright 2016 The Kubernetes Authors + 2. Exceptions and Limitations. For the avoidance of doubt, where + Exceptions and Limitations apply to Your use, this Public + License does not apply, and You do not need to comply with + its terms and conditions. - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at + 3. Term. The term of this Public License is specified in Section + 6(a). - http://www.apache.org/licenses/LICENSE-2.0 + 4. Media and formats; technical modifications allowed. The + Licensor authorizes You to exercise the Licensed Rights in + all media and formats whether now known or hereafter created, + and to make technical modifications necessary to do so. The + Licensor waives and/or agrees not to assert any right or + authority to forbid You from making technical modifications + necessary to exercise the Licensed Rights, including + technical modifications necessary to circumvent Effective + Technological Measures. For purposes of this Public License, + simply making modifications authorized by this Section 2(a) + (4) never produces Adapted Material. - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. + 5. Downstream recipients. + + a. Offer from the Licensor -- Licensed Material. Every + recipient of the Licensed Material automatically + receives an offer from the Licensor to exercise the + Licensed Rights under the terms and conditions of this + Public License. + + b. No downstream restrictions. You may not offer or impose + any additional or different terms or conditions on, or + apply any Effective Technological Measures to, the + Licensed Material if doing so restricts exercise of the + Licensed Rights by any recipient of the Licensed + Material. + + 6. No endorsement. Nothing in this Public License constitutes or + may be construed as permission to assert or imply that You + are, or that Your use of the Licensed Material is, connected + with, or sponsored, endorsed, or granted official status by, + the Licensor or others designated to receive attribution as + provided in Section 3(a)(1)(A)(i). + + b. Other rights. + + 1. Moral rights, such as the right of integrity, are not + licensed under this Public License, nor are publicity, + privacy, and/or other similar personality rights; however, to + the extent possible, the Licensor waives and/or agrees not to + assert any such rights held by the Licensor to the limited + extent necessary to allow You to exercise the Licensed + Rights, but not otherwise. + + 2. Patent and trademark rights are not licensed under this + Public License. + + 3. To the extent possible, the Licensor waives any right to + collect royalties from You for the exercise of the Licensed + Rights, whether directly or through a collecting society + under any voluntary or waivable statutory or compulsory + licensing scheme. In all other cases the Licensor expressly + reserves any right to collect such royalties. + + +Section 3 -- License Conditions. + +Your exercise of the Licensed Rights is expressly made subject to the +following conditions. + + a. Attribution. + + 1. If You Share the Licensed Material (including in modified + form), You must: + + a. retain the following if it is supplied by the Licensor + with the Licensed Material: + + i. identification of the creator(s) of the Licensed + Material and any others designated to receive + attribution, in any reasonable manner requested by + the Licensor (including by pseudonym if + designated); + + ii. a copyright notice; + + iii. a notice that refers to this Public License; + + iv. a notice that refers to the disclaimer of + warranties; + + v. a URI or hyperlink to the Licensed Material to the + extent reasonably practicable; + + b. indicate if You modified the Licensed Material and + retain an indication of any previous modifications; and + + c. indicate the Licensed Material is licensed under this + Public License, and include the text of, or the URI or + hyperlink to, this Public License. + + 2. You may satisfy the conditions in Section 3(a)(1) in any + reasonable manner based on the medium, means, and context in + which You Share the Licensed Material. For example, it may be + reasonable to satisfy the conditions by providing a URI or + hyperlink to a resource that includes the required + information. + + 3. If requested by the Licensor, You must remove any of the + information required by Section 3(a)(1)(A) to the extent + reasonably practicable. + + 4. If You Share Adapted Material You produce, the Adapter's + License You apply must not prevent recipients of the Adapted + Material from complying with this Public License. + + +Section 4 -- Sui Generis Database Rights. + +Where the Licensed Rights include Sui Generis Database Rights that +apply to Your use of the Licensed Material: + + a. for the avoidance of doubt, Section 2(a)(1) grants You the right + to extract, reuse, reproduce, and Share all or a substantial + portion of the contents of the database; + + b. if You include all or a substantial portion of the database + contents in a database in which You have Sui Generis Database + Rights, then the database in which You have Sui Generis Database + Rights (but not its individual contents) is Adapted Material; and + + c. You must comply with the conditions in Section 3(a) if You Share + all or a substantial portion of the contents of the database. + +For the avoidance of doubt, this Section 4 supplements and does not +replace Your obligations under this Public License where the Licensed +Rights include other Copyright and Similar Rights. + + +Section 5 -- Disclaimer of Warranties and Limitation of Liability. + + a. UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE + EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS + AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF + ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS, + IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION, + WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR + PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS, + ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT + KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT + ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU. + + b. TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE + TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION, + NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT, + INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES, + COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR + USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN + ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR + DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR + IN PART, THIS LIMITATION MAY NOT APPLY TO YOU. + + c. The disclaimer of warranties and limitation of liability provided + above shall be interpreted in a manner that, to the extent + possible, most closely approximates an absolute disclaimer and + waiver of all liability. + + +Section 6 -- Term and Termination. + + a. This Public License applies for the term of the Copyright and + Similar Rights licensed here. However, if You fail to comply with + this Public License, then Your rights under this Public License + terminate automatically. + + b. Where Your right to use the Licensed Material has terminated under + Section 6(a), it reinstates: + + 1. automatically as of the date the violation is cured, provided + it is cured within 30 days of Your discovery of the + violation; or + + 2. upon express reinstatement by the Licensor. + + For the avoidance of doubt, this Section 6(b) does not affect any + right the Licensor may have to seek remedies for Your violations + of this Public License. + + c. For the avoidance of doubt, the Licensor may also offer the + Licensed Material under separate terms or conditions or stop + distributing the Licensed Material at any time; however, doing so + will not terminate this Public License. + + d. Sections 1, 5, 6, 7, and 8 survive termination of this Public + License. + + +Section 7 -- Other Terms and Conditions. + + a. The Licensor shall not be bound by any additional or different + terms or conditions communicated by You unless expressly agreed. + + b. Any arrangements, understandings, or agreements regarding the + Licensed Material not stated herein are separate from and + independent of the terms and conditions of this Public License. + + +Section 8 -- Interpretation. + + a. For the avoidance of doubt, this Public License does not, and + shall not be interpreted to, reduce, limit, restrict, or impose + conditions on any use of the Licensed Material that could lawfully + be made without permission under this Public License. + + b. To the extent possible, if any provision of this Public License is + deemed unenforceable, it shall be automatically reformed to the + minimum extent necessary to make it enforceable. If the provision + cannot be reformed, it shall be severed from this Public License + without affecting the enforceability of the remaining terms and + conditions. + + c. No term or condition of this Public License will be waived and no + failure to comply consented to unless expressly agreed to by the + Licensor. + + d. Nothing in this Public License constitutes or may be interpreted + as a limitation upon, or waiver of, any privileges and immunities + that apply to the Licensor or You, including from the legal + processes of any jurisdiction or authority. + + +======================================================================= + +Creative Commons is not a party to its public +licenses. Notwithstanding, Creative Commons may elect to apply one of +its public licenses to material it publishes and in those instances +will be considered the “Licensor.” The text of the Creative Commons +public licenses is dedicated to the public domain under the CC0 Public +Domain Dedication. Except for the limited purpose of indicating that +material is shared under a Creative Commons public license or as +otherwise permitted by the Creative Commons policies published at +creativecommons.org/policies, Creative Commons does not authorize the +use of the trademark "Creative Commons" or any other trademark or logo +of Creative Commons without its prior written consent including, +without limitation, in connection with any unauthorized modifications +to any of its public licenses or any other arrangements, +understandings, or agreements concerning use of licensed material. For +the avoidance of doubt, this paragraph does not form part of the +public licenses. + +Creative Commons may be contacted at creativecommons.org. From e9591bb3ee92df5deb8ef485722adb786874986a Mon Sep 17 00:00:00 2001 From: craigbox Date: Tue, 22 Nov 2016 13:22:38 +0000 Subject: [PATCH 78/87] Add license to footer - Change "Kubernetes", which is not a thing you can assign copyright to, to "The Kubernetes Authors", in line with comments in source files. This may or may not be more appropriate to assign copyright to CNCF. - Add link to LICENSE with CC BY SA text, which will only be true when #1763 is merged. - I don't think you'll like the CSS, but without the 'light-text' tag you get a blue/purple link. --- _includes/footer.html | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_includes/footer.html b/_includes/footer.html index e7fa36d26a..5e724f710c 100644 --- a/_includes/footer.html +++ b/_includes/footer.html @@ -24,6 +24,6 @@ Contribute to the K8s codebase -
© {{ 'now' | date: "%Y" }} Kubernetes
+
© {{ 'now' | date: "%Y" }} The Kubernetes Authors | Distributed under CC BY 4.0
From aa89153cfee2bb4cee40ed002071247a6c35cc08 Mon Sep 17 00:00:00 2001 From: craigbox Date: Tue, 22 Nov 2016 13:26:49 +0000 Subject: [PATCH 79/87] Add license info to documentation docs - Add link to the CC BY SA license - Remove Google CLA, as we're now only using the LF CLA - Add "Foundation" to the "Linux CLA" --- docs/contribute/create-pull-request.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/docs/contribute/create-pull-request.md b/docs/contribute/create-pull-request.md index e79fe22c96..7f42bfb125 100644 --- a/docs/contribute/create-pull-request.md +++ b/docs/contribute/create-pull-request.md @@ -14,10 +14,9 @@ repository. This page shows how to create a pull request. 1. Create a [GitHub account](https://github.com){: target="_blank"}. 1. Sign the -[Google Contributor License Agreement](https://cla.developers.google.com/about/google-individual){: target="_blank"}. +[Linux Foundation Contributor License Agreement](https://identity.linuxfoundation.org/projects/cncf){: target="_blank"}. -1. Sign the -[Linux Contributor License Agreement](https://identity.linuxfoundation.org/projects/cncf){: target="_blank"}. +Documentation will be published under the [CC BY SA 4.0](https://github.com/kubernetes/kubernetes.github.io/blob/master/LICENSE) license. {% endcapture %} From 9544c80dee30b25ddf0b3860188f3243f9d9438c Mon Sep 17 00:00:00 2001 From: Maciej Szulik Date: Tue, 18 Oct 2016 14:39:07 +0200 Subject: [PATCH 80/87] Add ScheduledJobs to menu and remove future work section from jobs --- _data/guides.yml | 6 ++++-- docs/user-guide/scheduled-jobs.md | 31 +++++++++++++++++-------------- docs/user-guide/sj.yaml | 2 +- 3 files changed, 22 insertions(+), 17 deletions(-) diff --git a/_data/guides.yml b/_data/guides.yml index 9f732ad047..76773e874e 100644 --- a/_data/guides.yml +++ b/_data/guides.yml @@ -70,7 +70,7 @@ toc: section: - title: Example Walkthrough path: /docs/getting-started-guides/network-policy/walkthrough/ - - title: Using Calico for NetworkPolicy + - title: Using Calico for NetworkPolicy path: /docs/getting-started-guides/network-policy/calico/ - title: Using Romana for NetworkPolicy path: /docs/getting-started-guides/network-policy/romana/ @@ -85,6 +85,8 @@ toc: path: /docs/user-guide/jobs/work-queue-1/ - title: Fine Parallel Processing using a Work Queue path: /docs/user-guide/jobs/work-queue-2/ + - title: Scheduled Jobs + path: /docs/user-guide/scheduled-jobs/ - title: Service Discovery and Load Balancing section: @@ -293,7 +295,7 @@ toc: - title: Configuring kubelet Garbage Collection path: /docs/admin/garbage-collection/ - title: Configuring Out Of Resource Handling - path: /docs/admin/out-of-resource/ + path: /docs/admin/out-of-resource/ - title: Configuring Kubernetes with Salt path: /docs/admin/salt/ - title: Monitoring Node Health diff --git a/docs/user-guide/scheduled-jobs.md b/docs/user-guide/scheduled-jobs.md index b96014a6d1..0f1a16e2be 100644 --- a/docs/user-guide/scheduled-jobs.md +++ b/docs/user-guide/scheduled-jobs.md @@ -19,6 +19,9 @@ A _Scheduled Job_ manages time based [Jobs](/docs/user-guide/jobs/), namely: One ScheduledJob object is like one line of a _crontab_ (cron table) file. It runs a job periodically on a given schedule, written in [Cron](https://en.wikipedia.org/wiki/Cron) format. +NOTE: The question mark (`?`) in the schedule has the same meaning as an asterisk `*`, +that is, it stands for any of available value for a given field. + A typical use case is: * Schedule a job execution at a given point in time. @@ -48,7 +51,7 @@ scheduledjob "hello" created Alternatively, use `kubectl run` to create a scheduled job without writing full config: ```shell -$ kubectl run hello --schedule="0/1 * * * ?" --restart=OnFailure --image=busybox -- /bin/sh -c "date; echo Hello from the Kubernetes cluster" +$ kubectl run hello --schedule="*/1 * * * *" --restart=OnFailure --image=busybox -- /bin/sh -c "date; echo Hello from the Kubernetes cluster" scheduledjob "hello" created ``` @@ -57,10 +60,10 @@ After creating the scheduled job, get its status using this command: ```shell $ kubectl get scheduledjob hello NAME SCHEDULE SUSPEND ACTIVE LAST-SCHEDULE -hello 0/1 * * * ? False 0 +hello */1 * * * * False 0 ``` -As you can see above, there's no active job yet, and no job has been scheduled, either. +As you can see above, there's no active job yet, and no job has been scheduled, either. Watch for the job to be created in around one minute: @@ -75,11 +78,11 @@ Now you've seen one running job scheduled by "hello". We can stop watching it an ```shell $ kubectl get scheduledjob hello NAME SCHEDULE SUSPEND ACTIVE LAST-SCHEDULE -hello 0/1 * * * ? False 0 Mon, 29 Aug 2016 14:34:00 -0700 +hello */1 * * * * False 0 Mon, 29 Aug 2016 14:34:00 -0700 ``` You should see that "hello" successfully scheduled a job at the time specified in `LAST-SCHEDULE`. There are -currently 0 active jobs, meaning that the job that's scheduled is completed or failed. +currently 0 active jobs, meaning that the job that's scheduled is completed or failed. Now, find the pods created by the job last scheduled and view the standard output of one of the pods. Note that your job name and pod name would be different. @@ -96,7 +99,7 @@ Mon Aug 29 21:34:09 UTC 2016 Hello from the Kubernetes cluster ``` -## Deleting a Scheduled Job +## Deleting a Scheduled Job Once you don't need a scheduled job anymore, simply delete it with `kubectl`: @@ -122,7 +125,7 @@ job "hello-1202039034" deleted ``` Once the jobs are deleted, the pods created by them are deleted as well. Note that all jobs created by scheduled -job "hello" will be prefixed "hello-". You can delete them at once with `kubectl delete jobs --all`, if you want to +job "hello" will be prefixed "hello-". You can delete them at once with `kubectl delete jobs --all`, if you want to delete all jobs in the current namespace (not just the ones created by "hello".) ## Scheduled Job Limitations @@ -137,7 +140,7 @@ of the set of pods. A scheduled job does not examine pods at all. ## Writing a Scheduled Job Spec As with all other Kubernetes configs, a scheduled job needs `apiVersion`, `kind`, and `metadata` fields. For general -information about working with config files, see [deploying applications](/docs/user-guide/deploying-applications), +information about working with config files, see [deploying applications](/docs/user-guide/deploying-applications), [configuring containers](/docs/user-guide/configuring-containers), and [using kubectl to manage resources](/docs/user-guide/working-with-resources) documents. @@ -145,10 +148,10 @@ A scheduled job also needs a [`.spec` section](https://github.com/kubernetes/kub **Note:** All modifications to a scheduled job, especially its `.spec`, will be applied only to the next run. -### Schedule +### Schedule The `.spec.schedule` is a required field of the `.spec`. It takes a [Cron](https://en.wikipedia.org/wiki/Cron) format -string, e.g. `0 * * * *` or `@hourly`, as schedule time of its jobs to be created and executed. +string, e.g. `0 * * * *` or `@hourly`, as schedule time of its jobs to be created and executed. ### Job Template @@ -159,8 +162,8 @@ as a [Job](/docs/user-guide/jobs), except it is nested and does not have an `api ### Starting Deadline Seconds The `.spec.startingDeadlineSeconds` field is optional. It stands for the deadline (in seconds) for starting the job -if it misses its scheduled time for any reason. Missed jobs executions will be counted as failed ones. If not specified, -there's no deadline. +if it misses its scheduled time for any reason. Missed jobs executions will be counted as failed ones. If not specified, +there's no deadline. ### Concurrency Policy @@ -176,5 +179,5 @@ scheduled jobs, their respective jobs are always allowed to run concurrently. ### Suspend -The `.spec.suspend` field is also optional. If set to `true`, all subsequent executions will be suspended. It does not -apply to already started executions. Defaults to false. +The `.spec.suspend` field is also optional. If set to `true`, all subsequent executions will be suspended. It does not +apply to already started executions. Defaults to false. diff --git a/docs/user-guide/sj.yaml b/docs/user-guide/sj.yaml index 1b22b7f340..8f247282d1 100644 --- a/docs/user-guide/sj.yaml +++ b/docs/user-guide/sj.yaml @@ -3,7 +3,7 @@ kind: ScheduledJob metadata: name: hello spec: - schedule: 0/1 * * * ? + schedule: "*/1 * * * *" jobTemplate: spec: template: From ab09b11359713289283a7699eaa5a1fe07221eea Mon Sep 17 00:00:00 2001 From: Maciej Szulik Date: Wed, 9 Nov 2016 11:32:57 +0100 Subject: [PATCH 81/87] Rename ScheduledJobs to CronJobs --- _data/guides.yml | 6 +- _data/reference.yml | 36 ++++----- .../{scheduled-jobs.md => cron-jobs.md} | 75 ++++++++++--------- docs/user-guide/{sj.yaml => cronjob.yaml} | 2 +- docs/user-guide/jobs.md | 4 +- docs/user-guide/kubectl-cheatsheet.md | 4 +- docs/user-guide/kubectl-conventions.md | 4 +- docs/user-guide/kubectl/kubectl_run.md | 2 +- test/examples_test.go | 2 +- 9 files changed, 69 insertions(+), 66 deletions(-) rename docs/user-guide/{scheduled-jobs.md => cron-jobs.md} (64%) rename docs/user-guide/{sj.yaml => cronjob.yaml} (95%) diff --git a/_data/guides.yml b/_data/guides.yml index 76773e874e..0c1a785720 100644 --- a/_data/guides.yml +++ b/_data/guides.yml @@ -72,7 +72,7 @@ toc: path: /docs/getting-started-guides/network-policy/walkthrough/ - title: Using Calico for NetworkPolicy path: /docs/getting-started-guides/network-policy/calico/ - - title: Using Romana for NetworkPolicy + - title: Using Romana for NetworkPolicy path: /docs/getting-started-guides/network-policy/romana/ - title: Batch Jobs @@ -85,8 +85,8 @@ toc: path: /docs/user-guide/jobs/work-queue-1/ - title: Fine Parallel Processing using a Work Queue path: /docs/user-guide/jobs/work-queue-2/ - - title: Scheduled Jobs - path: /docs/user-guide/scheduled-jobs/ + - title: Cron Jobs + path: /docs/user-guide/cron-jobs/ - title: Service Discovery and Load Balancing section: diff --git a/_data/reference.yml b/_data/reference.yml index 62fdb0ca92..6f71d1677c 100644 --- a/_data/reference.yml +++ b/_data/reference.yml @@ -40,13 +40,13 @@ toc: path: /docs/api-reference/batch/v1/operations/ - title: Batch API Definitions path: /docs/api-reference/batch/v1/definitions/ - + - title: Extensions API section: - title: Extensions API Operations path: /docs/api-reference/extensions/v1beta1/operations/ - title: Extensions API Definitions - path: /docs/api-reference/extensions/v1beta1/definitions/ + path: /docs/api-reference/extensions/v1beta1/definitions/ - title: kubectl CLI section: @@ -172,7 +172,7 @@ toc: - title: kube-apiserver path: /docs/admin/kube-apiserver/ - title: kube-controller-manager - path: /docs/admin/kube-controller-manager/ + path: /docs/admin/kube-controller-manager/ - title: kube-proxy path: /docs/admin/kube-proxy/ - title: kube-scheduler @@ -183,33 +183,33 @@ toc: - title: Glossary section: - title: Annotations - path: /docs/user-guide/annotations/ + path: /docs/user-guide/annotations/ - title: Daemon Sets path: /docs/admin/daemons/ - - title: Deployments + - title: Deployments path: /docs/user-guide/deployments/ - title: Horizontal Pod Autoscaling - path: /docs/user-guide/horizontal-pod-autoscaling/ + path: /docs/user-guide/horizontal-pod-autoscaling/ - title: Images path: /docs/user-guide/images/ - title: Ingress Resources - path: /docs/user-guide/ingress/ + path: /docs/user-guide/ingress/ - title: Jobs - path: /docs/user-guide/jobs/ + path: /docs/user-guide/jobs/ - title: Labels and Selectors path: /docs/user-guide/labels/ - title: Names path: /docs/user-guide/identifiers/ - title: Namespaces path: /docs/user-guide/namespaces/ - - title: Network Policies - path: /docs/user-guide/networkpolicies/ + - title: Network Policies + path: /docs/user-guide/networkpolicies/ - title: Nodes - path: /docs/admin/node/ + path: /docs/admin/node/ - title: Persistent Volumes - path: /docs/user-guide/persistent-volumes/ + path: /docs/user-guide/persistent-volumes/ - title: Pet Sets - path: /docs/user-guide/petset/ + path: /docs/user-guide/petset/ - title: Pods path: /docs/user-guide/pods/ - title: Pod Security Policies @@ -217,11 +217,11 @@ toc: - title: Replica Sets path: /docs/user-guide/replicasets/ - title: Replication Controller - path: /docs/user-guide/replication-controller/ + path: /docs/user-guide/replication-controller/ - title: Resource Quotas path: /docs/admin/resourcequota/ - - title: Scheduled Jobs - path: /docs/user-guide/scheduled-jobs/ + - title: Cron Jobs + path: /docs/user-guide/cron-jobs/ - title: Secrets path: /docs/user-guide/secrets/ - title: Security Context @@ -244,11 +244,11 @@ toc: - title: Kubernetes Identity and Access Management path: https://github.com/kubernetes/kubernetes/blob/release-1.3/docs/design/access.md - title: Kubernetes OpenVSwitch GRE/VxLAN networking - path: /docs/admin/ovs-networking/ + path: /docs/admin/ovs-networking/ - title: Security Contexts path: https://github.com/kubernetes/kubernetes/blob/release-1.3/docs/design/security_context.md - title: Security in Kubernetes - path: https://github.com/kubernetes/kubernetes/blob/release-1.3/docs/design/security.md + path: https://github.com/kubernetes/kubernetes/blob/release-1.3/docs/design/security.md - title: Federation section: diff --git a/docs/user-guide/scheduled-jobs.md b/docs/user-guide/cron-jobs.md similarity index 64% rename from docs/user-guide/scheduled-jobs.md rename to docs/user-guide/cron-jobs.md index 0f1a16e2be..6d88997cb3 100644 --- a/docs/user-guide/scheduled-jobs.md +++ b/docs/user-guide/cron-jobs.md @@ -9,19 +9,22 @@ assignees: * TOC {:toc} -## What is a _Scheduled Job_? +## What is a _Cron Job_? -A _Scheduled Job_ manages time based [Jobs](/docs/user-guide/jobs/), namely: +A _Cron Job_ manages time based [Jobs](/docs/user-guide/jobs/), namely: * Once at a specified point in time * Repeatedly at a specified point in time -One ScheduledJob object is like one line of a _crontab_ (cron table) file. It runs a job periodically +One CronJob object is like one line of a _crontab_ (cron table) file. It runs a job periodically on a given schedule, written in [Cron](https://en.wikipedia.org/wiki/Cron) format. -NOTE: The question mark (`?`) in the schedule has the same meaning as an asterisk `*`, +**Note:**: The question mark (`?`) in the schedule has the same meaning as an asterisk `*`, that is, it stands for any of available value for a given field. +**Note:**: ScheduledJob resource was introduced in Kubernetes version 1.4, but starting +from version 1.5 its current name is CronJob. + A typical use case is: * Schedule a job execution at a given point in time. @@ -29,36 +32,36 @@ A typical use case is: ### Prerequisites -You need a working Kubernetes cluster at version >= 1.4, with batch/v2alpha1 API turned on by passing -`--runtime-config=batch/v2alpha1` while bringing up the API server (see [Turn on or off an API version -for your cluster](/docs/admin/cluster-management/#turn-on-or-off-an-api-version-for-your-cluster) for -more). You cannot use Scheduled Jobs on a hosted Kubernetes provider that has disabled alpha resources. +You need a working Kubernetes cluster at version >= 1.4 (for ScheduledJob), >= 1.5 (for CronJobs), +with batch/v2alpha1 API turned on by passing `--runtime-config=batch/v2alpha1` while bringing up +the API server (see [Turn on or off an API version for your cluster](/docs/admin/cluster-management/#turn-on-or-off-an-api-version-for-your-cluster) +for more). You cannot use Cron Jobs on a hosted Kubernetes provider that has disabled alpha resources. -## Creating a Scheduled Job +## Creating a Cron Job -Here is an example Scheduled Job. Every minute, it runs a simple job to print current time and then say +Here is an example Cron Job. Every minute, it runs a simple job to print current time and then say hello. -{% include code.html language="yaml" file="sj.yaml" ghlink="/docs/user-guide/sj.yaml" %} +{% include code.html language="yaml" file="cronjob.yaml" ghlink="/docs/user-guide/cronjob.yaml" %} -Run the example scheduled job by downloading the example file and then running this command: +Run the example cron job by downloading the example file and then running this command: ```shell -$ kubectl create -f ./sj.yaml -scheduledjob "hello" created +$ kubectl create -f ./cronjob.yaml +cronjob "hello" created ``` -Alternatively, use `kubectl run` to create a scheduled job without writing full config: +Alternatively, use `kubectl run` to create a cron job without writing full config: ```shell $ kubectl run hello --schedule="*/1 * * * *" --restart=OnFailure --image=busybox -- /bin/sh -c "date; echo Hello from the Kubernetes cluster" -scheduledjob "hello" created +cronjob "hello" created ``` -After creating the scheduled job, get its status using this command: +After creating the cron job, get its status using this command: ```shell -$ kubectl get scheduledjob hello +$ kubectl get cronjob hello NAME SCHEDULE SUSPEND ACTIVE LAST-SCHEDULE hello */1 * * * * False 0 ``` @@ -73,10 +76,10 @@ NAME DESIRED SUCCESSFUL AGE hello-4111706356 1 1 2s ``` -Now you've seen one running job scheduled by "hello". We can stop watching it and get the scheduled job again: +Now you've seen one running job scheduled by "hello". We can stop watching it and get the cron job again: ```shell -$ kubectl get scheduledjob hello +$ kubectl get cronjob hello NAME SCHEDULE SUSPEND ACTIVE LAST-SCHEDULE hello */1 * * * * False 0 Mon, 29 Aug 2016 14:34:00 -0700 ``` @@ -99,17 +102,17 @@ Mon Aug 29 21:34:09 UTC 2016 Hello from the Kubernetes cluster ``` -## Deleting a Scheduled Job +## Deleting a Cron Job -Once you don't need a scheduled job anymore, simply delete it with `kubectl`: +Once you don't need a cron job anymore, simply delete it with `kubectl`: ```shell -$ kubectl delete scheduledjob hello -scheduledjob "hello" deleted +$ kubectl delete cronjob hello +cronjob "hello" deleted ``` This stops new jobs from being created. However, running jobs won't be stopped, and no jobs or their pods will -be deleted. To clean up those jobs and pods, you need to list all jobs created by the scheduled job, and delete them all: +be deleted. To clean up those jobs and pods, you need to list all jobs created by the cron job, and delete them all: ```shell $ kubectl get jobs @@ -124,29 +127,29 @@ job "hello-1202039034" deleted ... ``` -Once the jobs are deleted, the pods created by them are deleted as well. Note that all jobs created by scheduled +Once the jobs are deleted, the pods created by them are deleted as well. Note that all jobs created by cron job "hello" will be prefixed "hello-". You can delete them at once with `kubectl delete jobs --all`, if you want to delete all jobs in the current namespace (not just the ones created by "hello".) -## Scheduled Job Limitations +## Cron Job Limitations -A scheduled job creates a job object _about_ once per execution time of its schedule. We say "about" because there +A cron job creates a job object _about_ once per execution time of its schedule. We say "about" because there are certain circumstances where two jobs might be created, or no job might be created. We attempt to make these rare, but do not completely prevent them. Therefore, jobs should be _idempotent_. The job is responsible for retrying pods, parallelism among pods it creates, and determining the success or failure -of the set of pods. A scheduled job does not examine pods at all. +of the set of pods. A cron job does not examine pods at all. -## Writing a Scheduled Job Spec +## Writing a Cron Job Spec -As with all other Kubernetes configs, a scheduled job needs `apiVersion`, `kind`, and `metadata` fields. For general +As with all other Kubernetes configs, a cron job needs `apiVersion`, `kind`, and `metadata` fields. For general information about working with config files, see [deploying applications](/docs/user-guide/deploying-applications), [configuring containers](/docs/user-guide/configuring-containers), and [using kubectl to manage resources](/docs/user-guide/working-with-resources) documents. -A scheduled job also needs a [`.spec` section](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/devel/api-conventions.md#spec-and-status). +A cron job also needs a [`.spec` section](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/devel/api-conventions.md#spec-and-status). -**Note:** All modifications to a scheduled job, especially its `.spec`, will be applied only to the next run. +**Note:** All modifications to a cron job, especially its `.spec`, will be applied only to the next run. ### Schedule @@ -168,14 +171,14 @@ there's no deadline. ### Concurrency Policy The `.spec.concurrencyPolicy` field is also optional. It specifies how to treat concurrent executions of a job -created by this scheduled job. Only one of the following concurrent policies may be specified: +created by this cron job. Only one of the following concurrent policies may be specified: * `Allow` (default): allows concurrently running jobs * `Forbid`: forbids concurrent runs, skipping next run if previous hasn't finished yet * `Replace`: cancels currently running job and replaces it with a new one -Note that concurrency policy only applies to the jobs created by the same scheduled job. If there are multiple -scheduled jobs, their respective jobs are always allowed to run concurrently. +Note that concurrency policy only applies to the jobs created by the same cron job. If there are multiple +cron jobs, their respective jobs are always allowed to run concurrently. ### Suspend diff --git a/docs/user-guide/sj.yaml b/docs/user-guide/cronjob.yaml similarity index 95% rename from docs/user-guide/sj.yaml rename to docs/user-guide/cronjob.yaml index 8f247282d1..cdb1c469a1 100644 --- a/docs/user-guide/sj.yaml +++ b/docs/user-guide/cronjob.yaml @@ -1,5 +1,5 @@ apiVersion: batch/v2alpha1 -kind: ScheduledJob +kind: CronJob metadata: name: hello spec: diff --git a/docs/user-guide/jobs.md b/docs/user-guide/jobs.md index 4e42b8fcef..89ce8c708c 100644 --- a/docs/user-guide/jobs.md +++ b/docs/user-guide/jobs.md @@ -374,6 +374,6 @@ driver, and then cleans up. An advantage of this approach is that the overall process gets the completion guarantee of a Job object, but complete control over what pods are created and how work is assigned to them. -## Scheduled Jobs +## Cron Jobs -Support for creating Jobs at specified times/dates (i.e. cron) is available in Kubernetes [1.4](https://github.com/kubernetes/kubernetes/pull/11980). More information is available in the [scheduled job documents](http://kubernetes.io/docs/user-guide/scheduled-jobs/) +Support for creating Jobs at specified times/dates (i.e. cron) is available in Kubernetes [1.4](https://github.com/kubernetes/kubernetes/pull/11980). More information is available in the [cron job documents](http://kubernetes.io/docs/user-guide/cron-jobs/) diff --git a/docs/user-guide/kubectl-cheatsheet.md b/docs/user-guide/kubectl-cheatsheet.md index 14e6850f37..b207542e85 100644 --- a/docs/user-guide/kubectl-cheatsheet.md +++ b/docs/user-guide/kubectl-cheatsheet.md @@ -141,7 +141,7 @@ $ kubectl rolling-update frontend-v1 frontend-v2 --rollback # Abort exist $ cat pod.json | kubectl replace -f - # Replace a pod based on the JSON passed into stdin # Force replace, delete and then re-create the resource. Will cause a service outage. -$ kubectl replace --force -f ./pod.json +$ kubectl replace --force -f ./pod.json # Create a service for a replicated nginx, which serves on port 80 and connects to the containers on port 8000 $ kubectl expose rc nginx --port=80 --target-port=8000 @@ -251,7 +251,7 @@ Resource type | Abbreviated alias `replicasets` |`rs` `replicationcontrollers` |`rc` `resourcequotas` |`quota` -`scheduledjob` | +`cronjob` | `secrets` | `serviceaccount` |`sa` `services` |`svc` diff --git a/docs/user-guide/kubectl-conventions.md b/docs/user-guide/kubectl-conventions.md index a22973f16f..291eb712fa 100644 --- a/docs/user-guide/kubectl-conventions.md +++ b/docs/user-guide/kubectl-conventions.md @@ -38,7 +38,7 @@ In order for `kubectl run` to satisfy infrastructure as code: * Deployment - use `deployment/v1beta1`. * Job (using `extension/v1beta1` endpoint) - use `job/v1beta1`. * Job - use `job/v1`. -* ScheduledJob - use `scheduledjob/v2alpha1`. +* CronJob - use `cronjob/v2alpha1`. Additionally, if you didn't specify a generator flag, other flags will suggest using a specific generator. Below table shows which flags force using specific generators, @@ -50,7 +50,7 @@ depending on your cluster version: | Replication Controller | `--generator=run/v1` | `--generator=run/v1` | `--generator=run/v1` | `--restart=Always` | | Deployment | `--restart=Always` | `--restart=Always` | `--restart=Always` | N/A | | Job | `--restart=OnFailure` | `--restart=OnFailure` | `--restart=OnFailure` OR `--restart=Never` | N/A | -| Scheduled Job | `--schedule=` | N/A | N/A | N/A | +| Cron Job | `--schedule=` | N/A | N/A | N/A | Note that these flags will use a default generator only when you have not specified any flag. This also means that combining `--generator` with other flags won't diff --git a/docs/user-guide/kubectl/kubectl_run.md b/docs/user-guide/kubectl/kubectl_run.md index c0a7d9cc6d..9b3dfb409e 100644 --- a/docs/user-guide/kubectl/kubectl_run.md +++ b/docs/user-guide/kubectl/kubectl_run.md @@ -50,7 +50,7 @@ kubectl run nginx --image=nginx --command -- ... # Start the perl container to compute π to 2000 places and print it out. kubectl run pi --image=perl --restart=OnFailure -- perl -Mbignum=bpi -wle 'print bpi(2000)' -# Start the scheduled job to compute π to 2000 places and print it out every 5 minutes. +# Start the cron job to compute π to 2000 places and print it out every 5 minutes. kubectl run pi --schedule="0/5 * * * ?" --image=perl --restart=OnFailure -- perl -Mbignum=bpi -wle 'print bpi(2000)' ``` diff --git a/test/examples_test.go b/test/examples_test.go index 7e5660c4f9..63fea3c5bc 100644 --- a/test/examples_test.go +++ b/test/examples_test.go @@ -242,7 +242,7 @@ func TestExampleObjectSchemas(t *testing.T) { "redis-resource-deployment": &extensions.Deployment{}, "redis-secret-deployment": &extensions.Deployment{}, "run-my-nginx": &extensions.Deployment{}, - "sj": &batch.CronJob{}, + "cronjob": &batch.CronJob{}, }, "../docs/admin": { "daemon": &extensions.DaemonSet{}, From 1bcb4cd558356dcdb44035e4a3813341f4f0fc2b Mon Sep 17 00:00:00 2001 From: Justin Peterson Date: Tue, 22 Nov 2016 19:15:40 -0800 Subject: [PATCH 82/87] Fix for Issue 1767 --- .../public/images/module_01_cluster.svg | 582 ++++++------------ 1 file changed, 177 insertions(+), 405 deletions(-) diff --git a/docs/tutorials/kubernetes-basics/public/images/module_01_cluster.svg b/docs/tutorials/kubernetes-basics/public/images/module_01_cluster.svg index 054a9f78eb..e1f92dace0 100644 --- a/docs/tutorials/kubernetes-basics/public/images/module_01_cluster.svg +++ b/docs/tutorials/kubernetes-basics/public/images/module_01_cluster.svg @@ -1,9 +1,6 @@ - - - - + + + + + + + + + + + + + + + + + + + + + + + + + Docker + Kubelt + + + Layer 1 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file From 1f685261595f13d238aaf7ba52f1d9f67fd3f508 Mon Sep 17 00:00:00 2001 From: Vincenzo Ferrari Date: Wed, 23 Nov 2016 11:12:07 +0100 Subject: [PATCH 83/87] updated dockerimage link removed dead link of dockerimage --- docs/user-guide/images.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/images.md b/docs/user-guide/images.md index e739bb0ca9..582961593f 100644 --- a/docs/user-guide/images.md +++ b/docs/user-guide/images.md @@ -5,7 +5,7 @@ assignees: --- -Each container in a pod has its own image. Currently, the only type of image supported is a [Docker Image](https://docs.docker.com/userguide/dockerimages/). +Each container in a pod has its own image. Currently, the only type of image supported is a [Docker Image](https://docs.docker.com/engine/tutorials/dockerimages/). You create your Docker image and push it to a registry before referring to it in a Kubernetes pod. From 5a54ed400cde4693c055d9bbe0fd1c8d892a8694 Mon Sep 17 00:00:00 2001 From: Bryan Boreham Date: Wed, 23 Nov 2016 14:25:09 +0000 Subject: [PATCH 84/87] Clarify that no add-ons are preferred above others --- docs/getting-started-guides/network-policy/walkthrough.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting-started-guides/network-policy/walkthrough.md b/docs/getting-started-guides/network-policy/walkthrough.md index 49da17e6ef..8384bcdb34 100644 --- a/docs/getting-started-guides/network-policy/walkthrough.md +++ b/docs/getting-started-guides/network-policy/walkthrough.md @@ -11,7 +11,7 @@ In this article, we assume a Kubernetes cluster has been created with network po * [Calico](/docs/getting-started-guides/network-policy/calico/) * [Romana](/docs/getting-started-guides/network-policy/romana/) -The reference implementation is [Calico](/docs/getting-started-guides/network-policy/calico) running on GCE. +Add-ons are sorted alphabetically - the ordering does not imply any preferential status. The following example walkthrough will work on a Kubernetes cluster using any of the listed providers. From c01aaba15624a04e858247220afac6f14edc5a7c Mon Sep 17 00:00:00 2001 From: Bryan Boreham Date: Wed, 23 Nov 2016 14:15:35 +0000 Subject: [PATCH 85/87] Add Weave to the list of NetworkPolicy implementations --- .../network-policy/walkthrough.md | 1 + docs/getting-started-guides/network-policy/weave.md | 11 +++++++++++ 2 files changed, 12 insertions(+) create mode 100644 docs/getting-started-guides/network-policy/weave.md diff --git a/docs/getting-started-guides/network-policy/walkthrough.md b/docs/getting-started-guides/network-policy/walkthrough.md index 49da17e6ef..973903ceed 100644 --- a/docs/getting-started-guides/network-policy/walkthrough.md +++ b/docs/getting-started-guides/network-policy/walkthrough.md @@ -10,6 +10,7 @@ In this article, we assume a Kubernetes cluster has been created with network po * [Calico](/docs/getting-started-guides/network-policy/calico/) * [Romana](/docs/getting-started-guides/network-policy/romana/) +* [Weave Net](/docs/getting-started-guides/network-policy/weave/) The reference implementation is [Calico](/docs/getting-started-guides/network-policy/calico) running on GCE. diff --git a/docs/getting-started-guides/network-policy/weave.md b/docs/getting-started-guides/network-policy/weave.md new file mode 100644 index 0000000000..8d5896861d --- /dev/null +++ b/docs/getting-started-guides/network-policy/weave.md @@ -0,0 +1,11 @@ +--- +assignees: +- bboreham + +--- + +The [Weave Net Addon](https://www.weave.works/docs/net/latest/kube-addon/) for Kubernetes comes with a Network Policy Controller. + +This component automatically monitors Kubernetes for any NetworkPolicy annotations on all namespaces, and configures `iptables` rules to allow or block traffic as directed by the policies. + +Once you have installed the Weave Net Addon you can follow the [NetworkPolicy gettting started guide](/docs/getting-started-guides/network-policy/walkthrough) to try out Kubernetes NetworkPolicy. From a856ab1549417a243abd3bd5c63e50c1a99eed87 Mon Sep 17 00:00:00 2001 From: steveperry-53 Date: Mon, 28 Nov 2016 11:49:17 -0800 Subject: [PATCH 86/87] Update Tutorial and Task landing pages. --- docs/tasks/index.md | 8 +++++++- docs/tutorials/index.md | 5 ++++- 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/docs/tasks/index.md b/docs/tasks/index.md index e59f8732b2..b71057e640 100644 --- a/docs/tasks/index.md +++ b/docs/tasks/index.md @@ -1,7 +1,9 @@ --- --- -The Tasks section of the Kubernetes documentation is a work in progress +This section of the Kubernetes documentation contains pages that +show how to do individual tasks. A task page shows how to do a +single thing, typically by giving a short sequence of steps. #### Configuring Pods and Containers @@ -13,6 +15,10 @@ The Tasks section of the Kubernetes documentation is a work in progress * [Using Port Forwarding to Access Applications in a Cluster](/docs/tasks/access-application-cluster/port-forward-access-application-cluster/) +#### Debugging Applications in a Cluster + +* [Determining the Reason for Pod Failure](/docs/tasks/debug-application-cluster/determine-reason-pod-failure/) + #### Accessing the Kubernetes API * [Using an HTTP Proxy to Access the Kubernetes API](/docs/tasks/access-kubernetes-api/http-proxy-access-api) diff --git a/docs/tutorials/index.md b/docs/tutorials/index.md index 60aab6a8fb..88c5b75807 100644 --- a/docs/tutorials/index.md +++ b/docs/tutorials/index.md @@ -1,7 +1,10 @@ --- --- -The Tutorials section of the Kubernetes documentation is a work in progress. +This section of the Kubernetes documentation contains tutorials. +A tutorial shows how to accomplish a goal that is larger than a single +[task](/docs/tasks/). Typically a tutorial has several sections, +each of which has a sequence of steps. #### Kubernetes Basics From a4f8d3c06a9ed5243330579dceadb4d26f9e2661 Mon Sep 17 00:00:00 2001 From: steveperry-53 Date: Wed, 23 Nov 2016 15:37:41 -0800 Subject: [PATCH 87/87] Remove old template topics. --- _data/support.yml | 2 - docs/contribute/page-templates.md | 3 + docs/templatedemos/OWNERS | 3 - docs/templatedemos/blank.md | 3 - docs/templatedemos/blankkubectl.md | 6 - docs/templatedemos/blanklanding.md | 3 - docs/templatedemos/blanktask.md | 3 - docs/templatedemos/filledout.md | 80 --------- docs/templatedemos/index.md | 249 ----------------------------- docs/templatedemos/kubectl.md | 7 - docs/templatedemos/landingpage.md | 53 ------ docs/templatedemos/partial.md | 6 - docs/templatedemos/task.md | 62 ------- 13 files changed, 3 insertions(+), 477 deletions(-) delete mode 100644 docs/templatedemos/OWNERS delete mode 100644 docs/templatedemos/blank.md delete mode 100644 docs/templatedemos/blankkubectl.md delete mode 100644 docs/templatedemos/blanklanding.md delete mode 100644 docs/templatedemos/blanktask.md delete mode 100644 docs/templatedemos/filledout.md delete mode 100644 docs/templatedemos/index.md delete mode 100644 docs/templatedemos/kubectl.md delete mode 100644 docs/templatedemos/landingpage.md delete mode 100644 docs/templatedemos/partial.md delete mode 100644 docs/templatedemos/task.md diff --git a/_data/support.yml b/_data/support.yml index 1b8e80699a..3e9ec08ee4 100644 --- a/_data/support.yml +++ b/_data/support.yml @@ -53,5 +53,3 @@ toc: path: https://github.com/kubernetes/kubernetes/milestones/ - title: Contributing to Kubernetes Documentation path: /editdocs/ - - title: New Template Instructions - path: /docs/templatedemos/ diff --git a/docs/contribute/page-templates.md b/docs/contribute/page-templates.md index 5bea261e3c..4b19cde39b 100644 --- a/docs/contribute/page-templates.md +++ b/docs/contribute/page-templates.md @@ -1,4 +1,7 @@ --- +redirect_from: + - /docs/templatedemos/ + - /docs/templatedemos.html ---