From 57587299abb61a5e87f229ff3cbc8ae67982a753 Mon Sep 17 00:00:00 2001 From: Jimmy Jones Date: Mon, 21 Mar 2016 21:45:53 +0000 Subject: [PATCH 001/348] Missing executable in docker command line --- docs/getting-started-guides/docker-multinode/master.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/getting-started-guides/docker-multinode/master.md b/docs/getting-started-guides/docker-multinode/master.md index 1926263ee6..af04aaa02d 100644 --- a/docs/getting-started-guides/docker-multinode/master.md +++ b/docs/getting-started-guides/docker-multinode/master.md @@ -114,6 +114,7 @@ sudo docker -H unix:///var/run/docker-bootstrap.sock run -d \ --privileged \ -v /dev/net:/dev/net \ quay.io/coreos/flannel:${FLANNEL_VERSION} \ + /opt/bin/flanneld \ --ip-masq=${FLANNEL_IPMASQ} \ --iface=${FLANNEL_IFACE} ``` From 84c75e2b6aaf382f692c3b1cf36ec6f4f1fcbbd2 Mon Sep 17 00:00:00 2001 From: johndmulhausen Date: Tue, 22 Mar 2016 18:37:38 -0700 Subject: [PATCH 002/348] Update _config.yml --- _config.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/_config.yml b/_config.yml index c3eaba8b58..02b865877c 100644 --- a/_config.yml +++ b/_config.yml @@ -16,8 +16,8 @@ defaults: scope: path: "" values: - version: "v1.2" - githubbranch: "release-1.2" + version: "v1.3" + githubbranch: "release-1.3" docsbranch: "master" - scope: From ebc680411046b3e364df160883a8166cb444cf2f Mon Sep 17 00:00:00 2001 From: johndmulhausen Date: Tue, 22 Mar 2016 18:38:32 -0700 Subject: [PATCH 003/348] Update _config.yml --- _config.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/_config.yml b/_config.yml index 02b865877c..5094499bfe 100644 --- a/_config.yml +++ b/_config.yml @@ -17,8 +17,8 @@ defaults: path: "" values: version: "v1.3" - githubbranch: "release-1.3" - docsbranch: "master" + githubbranch: "master" + docsbranch: "release-1.3" - scope: path: "docs" From 2448450dc74c47f623dbf45cf8bdaadf5ff93ba9 Mon Sep 17 00:00:00 2001 From: Erwin van Eyk Date: Mon, 28 Mar 2016 13:16:50 +0200 Subject: [PATCH 004/348] Corrected references to version of the default k8s --- docs/getting-started-guides/ubuntu.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/getting-started-guides/ubuntu.md b/docs/getting-started-guides/ubuntu.md index d6c778b2e9..048055daca 100644 --- a/docs/getting-started-guides/ubuntu.md +++ b/docs/getting-started-guides/ubuntu.md @@ -24,7 +24,7 @@ use a CNI plugin instead. Internet to download the necessary files, while worker nodes do not. 3. These guide is tested OK on Ubuntu 14.04 LTS 64bit server, but it can not work with Ubuntu 15 which uses systemd instead of upstart. -4. Dependencies of this guide: etcd-2.2.1, flannel-0.5.5, k8s-1.1.8, may work with higher versions. +4. Dependencies of this guide: etcd-2.2.1, flannel-0.5.5, k8s-1.2.0, may work with higher versions. 5. All the remote servers can be ssh logged in without a password by using key authentication. @@ -41,12 +41,12 @@ $ git clone https://github.com/kubernetes/kubernetes.git #### Configure and start the Kubernetes cluster The startup process will first download all the required binaries automatically. -By default etcd version is 2.2.1, flannel version is 0.5.5 and k8s version is 1.1.8. +By default etcd version is 2.2.1, flannel version is 0.5.5 and k8s version is 1.2.0. You can customize your etcd version, flannel version, k8s version by changing corresponding variables `ETCD_VERSION` , `FLANNEL_VERSION` and `KUBE_VERSION` like following. ```shell -$ export KUBE_VERSION=1.1.8 +$ export KUBE_VERSION=1.2.0 $ export FLANNEL_VERSION=0.5.0 $ export ETCD_VERSION=2.2.0 ``` From 7d3e28e7d5f249f978480429934cc375f78bd8ea Mon Sep 17 00:00:00 2001 From: Erwin van Eyk Date: Tue, 29 Mar 2016 16:20:15 +0200 Subject: [PATCH 005/348] Advice a shallow clone to reduce download --- docs/getting-started-guides/ubuntu.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting-started-guides/ubuntu.md b/docs/getting-started-guides/ubuntu.md index 048055daca..325ab34b5a 100644 --- a/docs/getting-started-guides/ubuntu.md +++ b/docs/getting-started-guides/ubuntu.md @@ -35,7 +35,7 @@ Ubuntu 15 which uses systemd instead of upstart. Clone the kubernetes github repo locally ```shell -$ git clone https://github.com/kubernetes/kubernetes.git +$ git clone --depth 1 https://github.com/kubernetes/kubernetes.git ``` #### Configure and start the Kubernetes cluster From 0981f5803dee2e4d096a39c599557a823638bcc5 Mon Sep 17 00:00:00 2001 From: Benjamin Sonntag Date: Tue, 29 Mar 2016 17:00:45 +0200 Subject: [PATCH 006/348] latest kubernetes requires golang 1.4+ Latest kubernetes requires golang 1.4+ as shown below : ```mykub:~/kubernetes hack/local-up-cluster.sh Detected go version: go version go1.3.3 linux/amd64.Kubernetes requires go version 1.4 or greater.Please install Go version 1.4 or later. ``` --- docs/getting-started-guides/locally.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/getting-started-guides/locally.md b/docs/getting-started-guides/locally.md index a49d4b1f31..d58ba80cac 100644 --- a/docs/getting-started-guides/locally.md +++ b/docs/getting-started-guides/locally.md @@ -23,7 +23,7 @@ You need an [etcd](https://github.com/coreos/etcd/releases) in your path, please #### go -You need [go](https://golang.org/doc/install) at least 1.3+ in your path, please make sure it is installed and in your ``$PATH``. +You need [go](https://golang.org/doc/install) at least 1.4+ in your path, please make sure it is installed and in your ``$PATH``. ### Starting the cluster @@ -113,4 +113,4 @@ One or more of the KUbernetes daemons might've crashed. Tail the logs of each in #### The pods fail to connect to the services by host names -The local-up-cluster.sh script doesn't start a DNS service. Similar situation can be found [here](http://issue.k8s.io/6667). You can start a manually. Related documents can be found [here](https://releases.k8s.io/{{page.githubbranch}}/cluster/addons/dns#how-do-i-configure-it) \ No newline at end of file +The local-up-cluster.sh script doesn't start a DNS service. Similar situation can be found [here](http://issue.k8s.io/6667). You can start a manually. Related documents can be found [here](https://releases.k8s.io/{{page.githubbranch}}/cluster/addons/dns#how-do-i-configure-it) From 2697a17407149f78bbd1c12f041678b3a680d1e8 Mon Sep 17 00:00:00 2001 From: Gert Leenders Date: Fri, 1 Apr 2016 08:12:59 +0200 Subject: [PATCH 007/348] Replaced replication controllers with deployments --- docs/getting-started-guides/meanstack.md | 135 ++++++++++++----------- 1 file changed, 69 insertions(+), 66 deletions(-) diff --git a/docs/getting-started-guides/meanstack.md b/docs/getting-started-guides/meanstack.md index 57a7ba39a1..072698402f 100644 --- a/docs/getting-started-guides/meanstack.md +++ b/docs/getting-started-guides/meanstack.md @@ -20,8 +20,8 @@ Before we jump in and start kube’ing it up, it’s important to understand som * Containers: These are the Docker, rtk, AppC, or whatever Container you are running. You can think of these like subatomic particles; everything is made up of them, but you rarely (if ever) interact with them directly. * Pods: Pods are the basic component of Kubernetes. They are a group of Containers that are scheduled, live, and die together. Why would you want to have a group of containers instead of just a single container? Let’s say you had a log processor, a web server, and a database. If you couldn't use Pods, you would have to bundle the log processor in the web server and database containers, and each time you updated one you would have to update the other. With Pods, you can just reuse the same log processor for both the web server and database. -* Replication Controllers: This is the management component of Kubernetes, and it’s pretty cool. You give it a set of Pods, tell it "I want three copies of this," and it creates those copies on your cluster. It will do its best to keep those copies always running, so if one crashes it will start another. -* Services: This is the other side to Replication Controllers. A service is the single point of contact for a group of Pods. For example, let’s say you have a Replication Controller that creates four copies of a web server pod. A Service will split the traffic to each of the four copies. Services are "permanent" while the pods behind them can come and go, so it’s a good idea to use Services. +* Deployments: A Deployment provides declarative updates for Pods and ReplicaSets. You only need to describe the desired state in a Deployment object, and the deployment controller will change the actual state to the desired state at a controlled rate for you. You can define Deployments to create new resources, or replace existing ones by new ones. +* Services: This is the other side to Deployments. A service is the single point of contact for a group of Pods. For example, let’s say you have a Deployment that creates four copies of a web server pod. A Service will split the traffic to each of the four copies. Services are "permanent" while the pods behind them can come and go, so it’s a good idea to use Services. ## Step 1: Creating the Container @@ -177,8 +177,8 @@ After a few minutes, you should see this in the console. Three things need to be created: 1. Persistent Disk to store the data (pods are ephemeral, so we shouldn't save data locally) -2. Replication Controller running MongoDB -3. Service mapping to that Replication Controller +2. Deployment running MongoDB +3. Service mapping to that Deployment To create the disk, run this: @@ -192,46 +192,45 @@ $ gcloud compute disks create \ Pick the same zone as your cluster and an appropriate disk size for your application. -Now, we need to create a Replication Controller that will run the database. I’m using a Replication Controller and not a Pod, because if a standalone Pod dies, it won't restart automatically. +Now, we need to create a Deployment that will run the database. I’m using a Deployment and not a Pod, because if a standalone Pod dies, it won't restart automatically. -### `db-controller.yml` +### `db-deployment.yml` ```yaml -apiVersion: v1 -kind: ReplicationController +apiVersion: extensions/v1beta1 +kind: Deployment metadata: - labels: - name: mongo - name: mongo-controller + name: mongo-deployment spec: - replicas: 1 - template: - metadata: - labels: - name: mongo - spec: - containers: - - image: mongo - name: mongo - ports: - - name: mongo - containerPort: 27017 - hostPort: 27017 - volumeMounts: - - name: mongo-persistent-storage - mountPath: /data/db - volumes: - - name: mongo-persistent-storage - gcePersistentDisk: - pdName: mongo-disk - fsType: ext4 + replicas: 1 + template: + metadata: + labels: + name: mongo + spec: + containers: + - image: mongo + name: mongo + ports: + - name: mongo + containerPort: 27017 + hostPort: 27017 + volumeMounts: + - name: mongo-persistent-storage + mountPath: /data/db + volumes: + - name: mongo-persistent-storage + gcePersistentDisk: + pdName: mongo-disk + fsType: ext4 + ``` -We call the controller `mongo-controller`, specify one replica, and open the appropriate ports. The image is `mongo`, which is the off the shelf MongoDB image. +We call the deployment `mongo-deployment`, specify one replica, and open the appropriate ports. The image is `mongo`, which is the off the shelf MongoDB image. The `volumes` section creates the volume for Kubernetes to use. There is a Google Container Engine-specific `gcePersistentDisk` section that maps the disk we made into a Kubernetes volume, and we mount the volume into the `/data/db` directory (as described in the MongoDB Docker documentation) -Now we have the Controller, let’s create the Service: +Now we have the Deployment, let’s create the Service: ### `db-service.yml` @@ -250,7 +249,7 @@ spec: name: mongo ``` -Again, pretty simple stuff. We "select" the mongo Controller to be served, open up the ports, and call the service `mongo`. +Again, pretty simple stuff. We "select" the mongo Deployment to be served, open up the ports, and call the service `mongo`. This is just like the "link" command line option we used with Docker in my previous post. Instead of connecting to `localhost`, we connect to `mongo`, and Kubernetes redirects traffic to the mongo service! @@ -261,7 +260,7 @@ $ ls Dockerfile app -db-controller.yml +db-deployment.yml db-service.yml ``` @@ -273,10 +272,10 @@ First, let’s "log in" to the cluster $ gcloud container clusters get-credentials mean-cluster ``` -Now create the controller. +Now create the Deployment. ```shell -$ kubectl create -f db-controller.yml +$ kubectl create -f db-deployment.yml ``` And the Service. @@ -299,7 +298,7 @@ Once you see the mongo pod in running status, we are good to go! $ kubectl get pods NAME READY REASON RESTARTS AGE -mongo-controller-xxxx 1/1 Running 0 3m +mongo-deployment-xxxx 1/1 Running 0 3m ``` @@ -309,36 +308,35 @@ Now the database is running, let’s start the web server. We need two things: -1. Replication Controller to spin up and down web server pods +1. Deployment to spin up and down web server pods 2. Service to expose our website to the interwebs -Let’s look at the Replication Controller configuration: +Let’s look at the Deployment configuration: -### `web-controller.yml` +### `web-deployment.yml` ```yaml -apiVersion: v1 -kind: ReplicationController +apiVersion: extensions/v1beta1 +kind: Deployment metadata: - labels: - name: web - name: web-controller + name: web-deployment spec: - replicas: 2 - template: - metadata: - labels: - name: web - spec: - containers: - - image: gcr.io//myapp - name: web - ports: - - containerPort: 3000 - name: http-server + replicas: 2 + template: + metadata: + labels: + name: web + spec: + containers: + - image: gcr.io/kubernetes-codelab-1259/myapp + name: web + ports: + - name: http-server + containerPort: 3000 + ``` -Here, we create a controller called `web-controller`, and we tell it to create two replicas. Replicas of what you ask? You may notice the `template` section looks just like a Pod configuration, and that's because it is. We are creating a Pod with our custom Node.js container and exposing port 3000. +Here, we create a deployment called `web-deployment`, and we tell it to create two replicas. Replicas of what you ask? You may notice the `template` section looks just like a Pod configuration, and that's because it is. We are creating a Pod with our custom Node.js container and exposing port 3000. Now for the Service @@ -372,16 +370,21 @@ At this point, the local directory looks like this ```shell $ ls -Dockerfile app db-pod.yml db-service.yml web-service.yml web-controller.yml +Dockerfile +app +db-deployment.yml +db-service.yml +web-service.yml +web-deployment.yml ``` ## Step 8: Running the Web Server -Create the Controller. +Create the Deployment. ```shell -$ kubectl create -f web-controller.yml +$ kubectl create -f web-deployment.yml ``` And the Service. @@ -402,9 +405,9 @@ Once you see the web pods in running status, we are good to go! $ kubectl get pods NAME READY REASON RESTARTS AGE -mongo-controller-xxxx 1/1 Running 0 4m -web-controller-xxxx 1/1 Running 0 1m -web-controller-xxxx 1/1 Running 0 1m +mongo-deployment-xxxx 1/1 Running 0 4m +web-deployment-xxxx 1/1 Running 0 1m +web-deployment-xxxx 1/1 Running 0 1m ``` From 16e0ef9ad221364041c15cf7c4a3fddfdc0b4d84 Mon Sep 17 00:00:00 2001 From: Gert Leenders Date: Fri, 1 Apr 2016 11:32:46 +0200 Subject: [PATCH 008/348] Put back in place project id placeholder --- docs/getting-started-guides/meanstack.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting-started-guides/meanstack.md b/docs/getting-started-guides/meanstack.md index 072698402f..a84e48af47 100644 --- a/docs/getting-started-guides/meanstack.md +++ b/docs/getting-started-guides/meanstack.md @@ -328,7 +328,7 @@ spec: name: web spec: containers: - - image: gcr.io/kubernetes-codelab-1259/myapp + - image: gcr.io//myapp name: web ports: - name: http-server From 46e667acd590eee1ae26d174fee5e07fc3732c98 Mon Sep 17 00:00:00 2001 From: Marcin Owsiany Date: Fri, 1 Apr 2016 14:49:28 +0200 Subject: [PATCH 009/348] Correct "kubectl get services" output. $ gcloud components list Your current Cloud SDK version is: 103.0.0 [...] $ kubectl get services NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE hello-node3 10.3.248.68 192.158.31.19 8080/TCP 14m --- docs/hellonode.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/hellonode.md b/docs/hellonode.md index 6d04c9fcf6..402a76a3fd 100755 --- a/docs/hellonode.md +++ b/docs/hellonode.md @@ -214,16 +214,16 @@ To find the ip addresses associated with the service run: ```shell $ kubectl get services hello-node -NAME CLUSTER_IP EXTERNAL_IP PORT(S) SELECTOR AGE -hello-node 10.3.246.12 8080/TCP run=hello-node 23s +NAME CLUSTER_IP EXTERNAL_IP PORT(S) AGE +hello-node 10.3.246.12 8080/TCP 23s ``` The `EXTERNAL_IP` may take several minutes to become available and visible. If the `EXTERNAL_IP` is missing, wait a few minutes and try again. ```shell $ kubectl get services hello-node -NAME CLUSTER_IP EXTERNAL_IP PORT(S) SELECTOR AGE -hello-node 10.3.246.12 23.251.159.72 8080/TCP run=hello-node 2m +NAME CLUSTER_IP EXTERNAL_IP PORT(S) AGE +hello-node 10.3.246.12 23.251.159.72 8080/TCP 2m ``` Note there are 2 IP addresses listed, both serving port 8080. `CLUSTER_IP` is only visible inside your cloud virtual network. `EXTERNAL_IP` is externally accessible. In this example, the external IP address is 23.251.159.72. From 87e156f4ef4fa8224c05250344753c8b11ffa242 Mon Sep 17 00:00:00 2001 From: Marcin Owsiany Date: Fri, 1 Apr 2016 15:35:33 +0200 Subject: [PATCH 010/348] Update clusters delete output. --- docs/hellonode.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/docs/hellonode.md b/docs/hellonode.md index 6d04c9fcf6..bd30fe4794 100755 --- a/docs/hellonode.md +++ b/docs/hellonode.md @@ -376,12 +376,13 @@ Delete your cluster: ```shell $ gcloud container clusters delete hello-world -Waiting for cluster deletion...done. -name: operation-xxxxxxxxxxxxxxxx -operationType: deleteCluster -status: done -target: /projects/kubernetes-codelab/zones/us-central1-f/clusters/hello-world -zone: us-central1-f +The following clusters will be deleted. + - [hello-world] in [us-central1-f] + +Do you want to continue (Y/n)? + +Deleting cluster hello-world...done. +Deleted [https://container.googleapis.com/v1/projects//zones/us-central1-f/clusters/hello-world]. ``` This deletes the Google Compute Engine instances that are running the cluster. From df6d4813d3e33b7c23fa128039e89ef826f4dda8 Mon Sep 17 00:00:00 2001 From: Gert Leenders Date: Sat, 2 Apr 2016 07:59:42 +0200 Subject: [PATCH 011/348] Removed sentence as suggested --- docs/getting-started-guides/meanstack.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting-started-guides/meanstack.md b/docs/getting-started-guides/meanstack.md index a84e48af47..c02c7c6440 100644 --- a/docs/getting-started-guides/meanstack.md +++ b/docs/getting-started-guides/meanstack.md @@ -21,7 +21,7 @@ Before we jump in and start kube’ing it up, it’s important to understand som * Containers: These are the Docker, rtk, AppC, or whatever Container you are running. You can think of these like subatomic particles; everything is made up of them, but you rarely (if ever) interact with them directly. * Pods: Pods are the basic component of Kubernetes. They are a group of Containers that are scheduled, live, and die together. Why would you want to have a group of containers instead of just a single container? Let’s say you had a log processor, a web server, and a database. If you couldn't use Pods, you would have to bundle the log processor in the web server and database containers, and each time you updated one you would have to update the other. With Pods, you can just reuse the same log processor for both the web server and database. * Deployments: A Deployment provides declarative updates for Pods and ReplicaSets. You only need to describe the desired state in a Deployment object, and the deployment controller will change the actual state to the desired state at a controlled rate for you. You can define Deployments to create new resources, or replace existing ones by new ones. -* Services: This is the other side to Deployments. A service is the single point of contact for a group of Pods. For example, let’s say you have a Deployment that creates four copies of a web server pod. A Service will split the traffic to each of the four copies. Services are "permanent" while the pods behind them can come and go, so it’s a good idea to use Services. +* Services: A service is the single point of contact for a group of Pods. For example, let’s say you have a Deployment that creates four copies of a web server pod. A Service will split the traffic to each of the four copies. Services are "permanent" while the pods behind them can come and go, so it’s a good idea to use Services. ## Step 1: Creating the Container From 3932b2ecdc0bad53f531b2c2f9f3a853c9c6d2a6 Mon Sep 17 00:00:00 2001 From: Mikolaj Pawlikowski Date: Tue, 5 Apr 2016 11:27:12 +0100 Subject: [PATCH 012/348] Update the kube-proxy arguments for master `kube-proxy` doesn't accept `--api-masters=`, it takes `--master=` instead --- docs/getting-started-guides/scratch.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/getting-started-guides/scratch.md b/docs/getting-started-guides/scratch.md index 00bc5971bb..456730f86d 100644 --- a/docs/getting-started-guides/scratch.md +++ b/docs/getting-started-guides/scratch.md @@ -394,10 +394,10 @@ kubelet. Arguments to consider: - If following the HTTPS security approach: - - `--api-servers=https://$MASTER_IP` + - `--master=https://$MASTER_IP` - `--kubeconfig=/var/lib/kube-proxy/kubeconfig` - Otherwise, if taking the firewall-based security approach - - `--api-servers=http://$MASTER_IP` + - `--master=http://$MASTER_IP` ### Networking From d3cf59c0d52789bb3946d42a2d670cc1b241ee57 Mon Sep 17 00:00:00 2001 From: Jordan Liggitt Date: Thu, 14 Apr 2016 11:27:50 -0400 Subject: [PATCH 013/348] Note service account token cleanup --- docs/user-guide/service-accounts.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/user-guide/service-accounts.md b/docs/user-guide/service-accounts.md index f51dbd87bb..70a3d62e5f 100644 --- a/docs/user-guide/service-accounts.md +++ b/docs/user-guide/service-accounts.md @@ -111,6 +111,8 @@ secrets/build-robot-secret Now you can confirm that the newly built secret is populated with an API token for the "build-robot" service account. +Any tokens for non-existent service accounts will be cleaned up by the token controller. + ```shell $ kubectl describe secrets/build-robot-secret Name: build-robot-secret From 71539b23da3f37f8908177efc29e362983a3531d Mon Sep 17 00:00:00 2001 From: Rachel Willmer Date: Tue, 19 Apr 2016 09:12:47 +0100 Subject: [PATCH 014/348] Add example for hostPath volume type --- docs/user-guide/volumes.md | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/docs/user-guide/volumes.md b/docs/user-guide/volumes.md index 44b9313d1e..f33289f9d8 100644 --- a/docs/user-guide/volumes.md +++ b/docs/user-guide/volumes.md @@ -117,6 +117,26 @@ Watch out when using this type of volume, because: * when Kubernetes adds resource-aware scheduling, as is planned, it will not be able to account for resources used by a `hostPath` +#### Example pod + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: test-hostpath +spec: + containers: + - image: myimage + name: test-container + volumeMounts: + - mountPath: /test-hostpath + name: test-volume + volumes: + - name: test-volume + hostPath: + path: /path/to/my/dir +``` + ### gcePersistentDisk A `gcePersistentDisk` volume mounts a Google Compute Engine (GCE) [Persistent @@ -390,4 +410,4 @@ pods. In the future, we expect that `emptyDir` and `hostPath` volumes will be able to request a certain amount of space using a [resource](/docs/user-guide/compute-resources) specification, and to select the type of media to use, for clusters that have -several media types. \ No newline at end of file +several media types. From 2ced103b4db94c402c498d603e9268b32e0c0ff2 Mon Sep 17 00:00:00 2001 From: Werner Beroux Date: Tue, 19 Apr 2016 12:14:09 +0200 Subject: [PATCH 015/348] Adds a Kubernetes overview diagram using UML elements. See also bugs #84. --- images/overview_uml.svg | 1 + 1 file changed, 1 insertion(+) create mode 100644 images/overview_uml.svg diff --git a/images/overview_uml.svg b/images/overview_uml.svg new file mode 100644 index 0000000000..4db4b553db --- /dev/null +++ b/images/overview_uml.svg @@ -0,0 +1 @@ + \ No newline at end of file From ed320c64fa40c81c6f20e4c5329e0fcc75ac5b6c Mon Sep 17 00:00:00 2001 From: Davanum Srinivas Date: Wed, 20 Apr 2016 10:26:23 -0400 Subject: [PATCH 016/348] Show how to access the deployed service add a simple "expose deployment" to show the user how to access the nginx containers --- docs/getting-started-guides/locally.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/getting-started-guides/locally.md b/docs/getting-started-guides/locally.md index cc2f9f1107..0110741307 100644 --- a/docs/getting-started-guides/locally.md +++ b/docs/getting-started-guides/locally.md @@ -62,10 +62,16 @@ cluster/kubectl.sh run my-nginx --image=nginx --replicas=2 --port=80 exit ## end wait +## create a service for nginx, which serves on port 80 +cluster/kubectl.sh expose deployment my-nginx --port=80 --name=my-nginx + ## introspect Kubernetes! cluster/kubectl.sh get pods cluster/kubectl.sh get services cluster/kubectl.sh get deployments + +## Test the nginx service with the IP/port from "get services" command +curl http://10.X.X.X:80/ ``` ### Running a user defined pod From 6ba13c62487848217cd084d7f922eceb95bdcf0b Mon Sep 17 00:00:00 2001 From: Aabed Date: Fri, 22 Apr 2016 17:44:40 +0300 Subject: [PATCH 017/348] updating version to latest --- .../kubernetes-cluster-main-nodes-template.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting-started-guides/coreos/azure/cloud_config_templates/kubernetes-cluster-main-nodes-template.yml b/docs/getting-started-guides/coreos/azure/cloud_config_templates/kubernetes-cluster-main-nodes-template.yml index b3f3331429..f201743c8a 100644 --- a/docs/getting-started-guides/coreos/azure/cloud_config_templates/kubernetes-cluster-main-nodes-template.yml +++ b/docs/getting-started-guides/coreos/azure/cloud_config_templates/kubernetes-cluster-main-nodes-template.yml @@ -184,7 +184,7 @@ coreos: Documentation=http://kubernetes.io/ Requires=network-online.target [Service] - Environment=KUBE_RELEASE_TARBALL=https://github.com/kubernetes/kubernetes/releases/download/v1.1.2/kubernetes.tar.gz + Environment=KUBE_RELEASE_TARBALL=https://github.com/kubernetes/kubernetes/releases/download/v1.2.2/kubernetes.tar.gz ExecStartPre=/bin/mkdir -p /opt/ ExecStart=/opt/bin/curl-retry.sh --silent --location $KUBE_RELEASE_TARBALL --output /tmp/kubernetes.tgz ExecStart=/bin/tar xzvf /tmp/kubernetes.tgz -C /tmp/ From 368b9cc4764db88257e0dcc93f7e4256157b5571 Mon Sep 17 00:00:00 2001 From: Aabed Date: Fri, 22 Apr 2016 17:46:06 +0300 Subject: [PATCH 018/348] changes api service parameters --- .../kubernetes-cluster-main-nodes-template.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/getting-started-guides/coreos/azure/cloud_config_templates/kubernetes-cluster-main-nodes-template.yml b/docs/getting-started-guides/coreos/azure/cloud_config_templates/kubernetes-cluster-main-nodes-template.yml index f201743c8a..f769334656 100644 --- a/docs/getting-started-guides/coreos/azure/cloud_config_templates/kubernetes-cluster-main-nodes-template.yml +++ b/docs/getting-started-guides/coreos/azure/cloud_config_templates/kubernetes-cluster-main-nodes-template.yml @@ -217,10 +217,10 @@ coreos: ExecStart=/opt/kubernetes/server/bin/kube-apiserver \ --insecure-bind-address=0.0.0.0 \ --advertise-address=$public_ipv4 \ - --port=8080 \ + --insecure-port=8080 \ $ETCD_SERVERS \ --service-cluster-ip-range=10.16.0.0/12 \ - --cloud-provider=vagrant \ + --cloud-provider= \ --logtostderr=true Restart=always RestartSec=10 From d210a5b7ed543bb443bc6e46f3791d90926092ee Mon Sep 17 00:00:00 2001 From: Aabed Date: Fri, 22 Apr 2016 17:49:03 +0300 Subject: [PATCH 019/348] adds a missing "/" yeah, it cause problems --- .../kubernetes-cluster-main-nodes-template.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting-started-guides/coreos/azure/cloud_config_templates/kubernetes-cluster-main-nodes-template.yml b/docs/getting-started-guides/coreos/azure/cloud_config_templates/kubernetes-cluster-main-nodes-template.yml index f769334656..d44b26318d 100644 --- a/docs/getting-started-guides/coreos/azure/cloud_config_templates/kubernetes-cluster-main-nodes-template.yml +++ b/docs/getting-started-guides/coreos/azure/cloud_config_templates/kubernetes-cluster-main-nodes-template.yml @@ -281,7 +281,7 @@ coreos: [Service] ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests/ ExecStart=/opt/kubernetes/server/bin/kubelet \ - --docker-endpoint=unix:/var/run/weave/weave.sock \ + --docker-endpoint=unix://var/run/weave/weave.sock \ --address=0.0.0.0 \ --port=10250 \ --hostname-override=%H \ From dbf7879dc01eed79137ee13be866a9d04eb32a23 Mon Sep 17 00:00:00 2001 From: Dan Winship Date: Tue, 19 Apr 2016 09:38:32 -0400 Subject: [PATCH 020/348] Warn that iptables proxier doesn't handle readiness failures. --- docs/user-guide/services/index.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/user-guide/services/index.md b/docs/user-guide/services/index.md index 10189ddad0..232232af27 100644 --- a/docs/user-guide/services/index.md +++ b/docs/user-guide/services/index.md @@ -193,7 +193,10 @@ default is `"None"`). As with the userspace proxy, the net result is that any traffic bound for the `Service`'s IP:Port is proxied to an appropriate backend without the clients knowing anything about Kubernetes or `Services` or `Pods`. This should be -faster and more reliable than the userspace proxy. +faster and more reliable than the userspace proxy. However, unlike the +userspace proxier, the iptables proxier cannot automatically retry another +`Pod` if the one it initially selects does not respond, so it depends on +having working [readiness probes](/docs/user-guide/production-pods/#liveness-and-readiness-probes-aka-health-checks). ![Services overview diagram for iptables proxy](/images/docs/services-iptables-overview.svg) From c003a0f4a4778e3265940df7bc4dc723b0b9d2e6 Mon Sep 17 00:00:00 2001 From: Cole Mickens Date: Wed, 6 Apr 2016 18:24:00 -0700 Subject: [PATCH 021/348] admin: update recommended admission controllers --- docs/admin/admission-controllers.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/admin/admission-controllers.md b/docs/admin/admission-controllers.md index 3af012fea7..bc4333db6d 100644 --- a/docs/admin/admission-controllers.md +++ b/docs/admin/admission-controllers.md @@ -139,5 +139,5 @@ Yes. For Kubernetes 1.0, we strongly recommend running the following set of admission control plug-ins (order matters): ```shell ---admission-control=NamespaceLifecycle,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota +--admission-control=NamespaceLifecycle,LimitRanger,SecurityContextDeny,ServiceAccount,PersistentVolumeLabel,ResoureQuota ``` From 4e9899bdf6ffd1ba05c7c635740ae596262674a9 Mon Sep 17 00:00:00 2001 From: Mohammad Khodadadi Date: Thu, 28 Apr 2016 19:19:33 +0100 Subject: [PATCH 022/348] Update fedora single node config for AWS RHEL7.2 --- .../fedora/fedora_manual_config.md | 103 +++++++++--------- 1 file changed, 52 insertions(+), 51 deletions(-) diff --git a/docs/getting-started-guides/fedora/fedora_manual_config.md b/docs/getting-started-guides/fedora/fedora_manual_config.md index 224e10fce9..c2db7ee8f1 100644 --- a/docs/getting-started-guides/fedora/fedora_manual_config.md +++ b/docs/getting-started-guides/fedora/fedora_manual_config.md @@ -1,7 +1,7 @@ ---- ---- +--- +--- -* TOC +* TOC {:toc} ## Prerequisites @@ -20,37 +20,38 @@ The Kubernetes package provides a few services: kube-apiserver, kube-scheduler, Hosts: -```conf +```conf fed-master = 192.168.121.9 fed-node = 192.168.121.65 -``` - +``` + **Prepare the hosts:** * Install Kubernetes on all hosts - fed-{master,node}. This will also pull in docker. Also install etcd on fed-master. This guide has been tested with kubernetes-0.18 and beyond. * The [--enablerepo=updates-testing](https://fedoraproject.org/wiki/QA:Updates_Testing) directive in the yum command below will ensure that the most recent Kubernetes version that is scheduled for pre-release will be installed. This should be a more recent version than the Fedora "stable" release for Kubernetes that you would get without adding the directive. * If you want the very latest Kubernetes release [you can download and yum install the RPM directly from Fedora Koji](http://koji.fedoraproject.org/koji/packageinfo?packageID=19202) instead of using the yum install command below. +* Running on AWS EC2 with RHEL 7.2, you need to enable extras repository for yum by editing /etc/yum.repos.d/redhat-rhui.repo and changing the changing the enable=0 to enable=1 for extras. -```shell +```shell yum -y install --enablerepo=updates-testing kubernetes -``` - +``` + * Install etcd and iptables -```shell +```shell yum -y install etcd iptables -``` - +``` + * Add master and node to /etc/hosts on all machines (not needed if hostnames already in DNS). Make sure that communication works between fed-master and fed-node by using a utility such as ping. -```shell +```shell echo "192.168.121.9 fed-master 192.168.121.65 fed-node" >> /etc/hosts -``` - +``` + * Edit /etc/kubernetes/config which will be the same on all hosts (master and node) to contain: -```shell +```shell # Comma separated list of nodes in the etcd cluster KUBE_MASTER="--master=http://fed-master:8080" @@ -62,20 +63,20 @@ KUBE_LOG_LEVEL="--v=0" # Should this cluster be allowed to run privileged docker containers KUBE_ALLOW_PRIV="--allow-privileged=false" -``` - +``` + * Disable the firewall on both the master and node, as docker does not play well with other firewall rule managers. Please note that iptables-services does not exist on default fedora server install. -```shell +```shell systemctl disable iptables-services firewalld systemctl stop iptables-services firewalld -``` - +``` + **Configure the Kubernetes services on the master.** * Edit /etc/kubernetes/apiserver to appear as such. The service-cluster-ip-range IP addresses must be an unused block of addresses, not used anywhere else. They do not need to be routed or assigned to anything. -```shell +```shell # The address on the local server to listen to. KUBE_API_ADDRESS="--address=0.0.0.0" @@ -87,37 +88,37 @@ KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16" # Add your own! KUBE_API_ARGS="" -``` - +``` + * Edit /etc/etcd/etcd.conf,let the etcd to listen all the ip instead of 127.0.0.1, if not, you will get the error like "connection refused". Note that Fedora 22 uses etcd 2.0, One of the changes in etcd 2.0 is that now uses port 2379 and 2380 (as opposed to etcd 0.46 which userd 4001 and 7001). -```shell +```shell ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:4001" -``` - +``` + * Create /var/run/kubernetes on master: -```shell +```shell mkdir /var/run/kubernetes chown kube:kube /var/run/kubernetes chmod 750 /var/run/kubernetes -``` - +``` + * Start the appropriate services on master: -```shell +```shell for SERVICES in etcd kube-apiserver kube-controller-manager kube-scheduler; do systemctl restart $SERVICES systemctl enable $SERVICES systemctl status $SERVICES done -``` - +``` + * Addition of nodes: * Create following node.json file on Kubernetes master node: -```json +```json { "apiVersion": "v1", "kind": "Node", @@ -129,18 +130,18 @@ done "externalID": "fed-node" } } -``` - +``` + Now create a node object internally in your Kubernetes cluster by running: -```shell +```shell $ kubectl create -f ./node.json $ kubectl get nodes NAME LABELS STATUS fed-node name=fed-node-label Unknown -``` - +``` + Please note that in the above, it only creates a representation for the node _fed-node_ internally. It does not provision the actual _fed-node_. Also, it is assumed that _fed-node_ (as specified in `name`) can be resolved and is @@ -153,7 +154,7 @@ a Kubernetes node (fed-node) below. * Edit /etc/kubernetes/kubelet to appear as such: -```shell +```shell ### # Kubernetes kubelet (node) config @@ -168,34 +169,34 @@ KUBELET_API_SERVER="--api-servers=http://fed-master:8080" # Add your own! #KUBELET_ARGS="" -``` - +``` + * Start the appropriate services on the node (fed-node). -```shell +```shell for SERVICES in kube-proxy kubelet docker; do systemctl restart $SERVICES systemctl enable $SERVICES systemctl status $SERVICES done -``` - +``` + * Check to make sure now the cluster can see the fed-node on fed-master, and its status changes to _Ready_. -```shell +```shell kubectl get nodes NAME LABELS STATUS fed-node name=fed-node-label Ready -``` - +``` + * Deletion of nodes: To delete _fed-node_ from your Kubernetes cluster, one should run the following on fed-master (Please do not do it, it is just for information): -```shell +```shell kubectl delete -f ./node.json -``` - +``` + *You should be finished!* **The cluster should be running! Launch a test pod.** From 120359b8968e929250228c65f496706f86f20629 Mon Sep 17 00:00:00 2001 From: johndmulhausen Date: Sat, 30 Apr 2016 14:17:29 -0700 Subject: [PATCH 023/348] Create .travis.yml --- .travis.yml | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) create mode 100644 .travis.yml diff --git a/.travis.yml b/.travis.yml new file mode 100644 index 0000000000..5fd825e6e3 --- /dev/null +++ b/.travis.yml @@ -0,0 +1,22 @@ +language: ruby +rvm: +- 2.1 +branches: + only: + - release-1.1 +script: +- cd $HOME +- git config --global user.email ${GIT_EMAIL} +- git config --global user.name "${GIT_NAME}" +- git clone https://${GIT_USERNAME}:${GH_TOKEN}@github.com/kubernetes-v1-3/kubernetes-v1-3.github.io.git +- cd kubernetes-v1-3.github.io +- git remote add --fetch --track release-1.3 homebase "https://${GIT_USERNAME}:${GH_TOKEN}@github.com/kubernetes/kubernetes.github.io.git" +- git merge -s recursive -X theirs homebase/release-1.3 -m "Sync from homebase" +- git push +env: + global: + - secure: Fd6wlE2mjPb1fAACxklQcJumpJWycYkaJQBfKRcjGCFlmw1XWVFGhpUC7Ni/MOyzTolqOvtb2rXnYpaujMlJP1UXqVFJ+zPbwur2lc8unQF8PcqPezl8DbPsr6HdceOjRdut/dN8zTw6+hZRDzw/mG4Rf8IVaozlYycAOnWZdAZsLXdbBpAvBp0WYHP9+8wn9xiet1L/QpSiawa/Q35Q9UM1tciPmfFBL4Fkq6Mm8/w6ECaxHyA2EX+eH5ea9EtykrzB5cMA/odJLptjnmfzsPeGS5F1MxuTqrH2Z09emZcqjfXW+fNNzioiZdD4BQe+rqOA1Ktpw645szJVJJenIMzM4P8dDiW+9cJyddGdTC9A7APgLlLVdG/tuoYcjiY1F068SWx8sUWW/xc7YenCzGj2nXIeHjUuWsjqjMorpBKe6Y1QSGfi7HsLq1DpJDR1xQZbPgM/FOaovVIPIozgAIBpS1ukdNNwadmrCw55tUKfr4s2SNv3SUmeprL37QEtIxtBpPDjL1z+qTLjc6JzRr7J57guPBdNbj6Ukg/uxW+z1CwDYb3uAIIa5e6eDxfxGQsUEj8NmkIHBLLGTnKb1WAyPXAIf9ZvZHUwQQzHvOWXGZRd32enRh00uvuwHEy3OuGEqTAOIs/31q5WAamZ1uqzW7KBTMU/nq4QA9W2yfM= + - secure: Lvj1MT/I2zaJp9mQi46us64O4JfeT9cz1m+nqKY/nuC828ULfqMYLlpw08jvrc25JhAaqHNawAwdtaxHTQqHztsmq1ixHFYA6G/1oll+YPhym29toFT9oIkXThf/L77FTJfM1gVFMyiHFbz11Ob8R15tRB1WF1+Uu1RBmtudJD7HSy0u6Uc6lPxpqtycWVRCyPdvZqF3e0KIZaTDRkRzJpcVMHa/5ZJDDcMtuJyjJYXZqS1WR3QHC1z44LlnqyB5ZM2PU4H3LyWnY8wHF2mutF0QtDDVdEuBqILBHiFuKHMxpLY92UgHm2n51RR63MxFjjEzE+iu5f9ComEm5JC0N/cc8sunIiol+d8SRC30/00Vs0tvmeAjRX9IMCExiP3mv7Tz6mqEVk6PyrVlg675hxRg0eVdaNNv92gWzCSIecZ5TuCaRG4JaWO0P8lQC4NreONt6gbnwMD350hMQZLpUUE0QoSEdafpudaD+agl4ZzTFVTOOcSOz2Sa/+RT96Msazq2YlddYXaEKZeyYzqHVkk60PiQzQcAuwCMFrAagqMe3bNI2aCFEWbc9CoR13K/wwRDwAeSzBq/UkylZ8AayJLnIpewr/iYBOQasrXLrorW969Rfr8d/nwhDN5VpRgItU0arDzngaJRoZEAtHx4zaCvZF/H/nvbORcD2gsFP9Q= + - secure: WGmgvSnT1OeHPYtsm/JzcRDnZYp+SeelgbSVNbtwOXGImkeqDo/sDIRH80HxVC3zkUu5wvRO/NkMA7N3LXN2BqENacWBFQ+j+Hf3lnDw9yETBaBpLQPO/MElKGd48GoSiukCipfaITUfwtjAQ3wx0W/z06dYC9BdUYvUM73OtsGtlrY1MEQnBfff/neDGZbzfvTdgDwg1n6d69WP+WSUJt5Aysz0ZoAbYO2rzBrKNwcL7KEhoI/ketXXQ4xEW0nnxU/qfxqhVfJbck1HpSz+HMsVGXF4tT3zdRXmU+P47KGbfYn0GmlviGrRSWCu9/elKTlX9fIqRnR1/UvdTkflwLsk4i3hWnEUflKpIT7soJrKjNQseyd06KI8qvVRXRcll92vCtsYDuOO1AlemxIqp2dIFduCf+5FWH06PbS8oHU8NBe5aVYaoBY8nK14EB3A98saJ8Un7ziB8moRaWZX9bMjfJWfxDF526NdHIusqLQrxvUthtBjf5aRULE2PeIyqYVY8/IKmBEfsy9IUENmlM5KItvgZZt4xdk8o//kceLrKabudHGVC1M1RLZo9ejkdLySz2i8nlJyJ03Hn30H9DfDjW/OMQTn/b1HU4CYSbd/yGR+6+5+tdPeKy/Z6gI6+EjHRYaaQrBLo0RXm2620cb7uDTBIsKVs2TetyhTq54= + - secure: lqboLQKFpfa6F+UkSCs9+NPv027EnSWZA4/cmAXj0hZvxc4W+qMFIlPKytcq+bbZrp7kx3cZgRd5NrgmOTqlJ9merVK1cgMsDegMv/YImHjhGVBH+3ATvqMDb7I7dLg4x/DjhKt/ogyZpoh8vsS77T9MQXaIJM8JMM3ISxYI6MMloCh0z/hbT2qv3IrPMFjpHohHdF1yorxKvSefNgPx6Y4wC+t1t+u8wS9RBhvrmOkVynlU2NgbSiMPyz3syJmnszHDTXXJlqciGT5Lc8lzpzt2OGOnZmAupEYVGok7Jx4aAxU9O8/bnoDwOcGGVrd/pGHTI5+rvT4TPy+WG1Aec55dgva4XZfrJQ88snHLvq0VrvdiI/fHJEPF18QnrBwKSni1Vd1jOzsuPIF6HL0Wwv8tklO8OX3D5wqVsXpeJd4Cj4HPPAtzxOVsV6Fz3On9vuLdV7StC/ZafQL40koaytqkNO0gX8zeUiaaVQtMCf/2MVTbX9x5m51gkGpwT0JBvqQpSMOlIM5S1fiN4X9DLBLqmsARYeZw1Jeiq7Zm39Sy1QeCgfUXS8+6t62BjWx70iftYGIkoljXtD7x/3pjKymdpwjcyUcS8KQ5W6vbMXFxmZe2phtgGTCxOIDuASPspD8zwm/ckIGArHj4qkrg94/mUpgDDIbVsaUpWXjYvSc= + - secure: bPZMNH79Lx0Wb2SCxTwyZek28w/keKxEAlfad8RDMwJrct6Bi2B0o4KjkrwFS5DyCUU7Ndk192145XnUKOWX2YVs6cQ8ge+5LvtgYhWLgX8g5Ycro+JyzOBskn+o1gQjvi8+3b42X31efcUmTEhfRdKVrrUpONEIcjG1NpLGk/mQJ6AM6hGWO2xdNfAezeWq5ISnpK0b6VUWZyTEDg3NivrTfCL2juWWCnjYm1BWSHUblXwRQ/Rl7Tcldl6cMMUVsalUQ1iG0h8YZDrxNz0cm3XTZZZJKuSHYeTCLd57RBeHD5/iMxjCmfzfq6ETNLONWLmtvA8yhWMQZ0DPFtLZzbqFIfOR6P1feBZFqP7/X5KZeFKBufN15JbcXIqHE8homLY9mS0LhyNffOs5G/P/x8ChE1DJaYiZIWCZ60umvpqibScZB3z5uFPTxLk9rJSOtT7hCWjcmg3EdJ+R4ExOiBDd62ZS5jH72WU4uysPXORRofUCL+zHycJoJxsFWQSW49GAGrohllrW45jnTgpalErxvjJFSKy8JW9w634eetz9ct2fObep7m8bfVMl8U2H3ITIoXHm+f8ooHUiNzHFLSl4wtcoAOtrCAvGtJPNfv8T2eTkznzj7Tk+XzTvFg03u+J99TmnC58Bs516Oc7E40NarmsZceOD2sN4BA3X9tE= From 4582d120dad0e067a24e3d3c92dfa94aaf24ec1a Mon Sep 17 00:00:00 2001 From: johndmulhausen Date: Sat, 30 Apr 2016 14:19:52 -0700 Subject: [PATCH 024/348] Update README.md --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index 11b9717354..30e736cc6d 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,7 @@ ## Instructions for Contributing to the Docs/Website +**Note: Changes to this branch are automatically served as a website at [http://kubernetes-v1-3.github.io/](http://kubernetes-v1-3.github.io/). This branch is for the v1.3 docs. See the "master" branch to edit the current stable release, which is served at http://kubernetes.io.** + Welcome! We are very pleased you want to contribute to the documentation and/or website for Kubernetes. You can click the "Fork" button in the upper-right area of the screen to create a copy of our site on your GitHub account called a "fork." Make any changes you want in your fork, and when you are ready to send those changes to us, go to the index page for your fork and click "New Pull Request" to let us know about it. From 9dabfa4b62931e35628e3c1552db50373a446e78 Mon Sep 17 00:00:00 2001 From: johndmulhausen Date: Sat, 30 Apr 2016 14:20:36 -0700 Subject: [PATCH 025/348] Update .travis.yml --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 5fd825e6e3..df4d6c8710 100644 --- a/.travis.yml +++ b/.travis.yml @@ -3,7 +3,7 @@ rvm: - 2.1 branches: only: - - release-1.1 + - release-1.3 script: - cd $HOME - git config --global user.email ${GIT_EMAIL} From 37fe853b1a3ad728e49cc7334ae0ce03292eabad Mon Sep 17 00:00:00 2001 From: johndmulhausen Date: Sat, 30 Apr 2016 14:25:26 -0700 Subject: [PATCH 026/348] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 30e736cc6d..03fbbc16a5 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ ## Instructions for Contributing to the Docs/Website -**Note: Changes to this branch are automatically served as a website at [http://kubernetes-v1-3.github.io/](http://kubernetes-v1-3.github.io/). This branch is for the v1.3 docs. See the "master" branch to edit the current stable release, which is served at http://kubernetes.io.** +**Note: This branch is for pre-release documentation for 1.3 of Kubernetes. Changes to this branch are automatically served as a website at [http://kubernetes-v1-3.github.io/](http://kubernetes-v1-3.github.io/). See the "master" branch to edit the current stable release, which is served at http://kubernetes.io.** Welcome! We are very pleased you want to contribute to the documentation and/or website for Kubernetes. From 9836e662f8803c62c4f291edef63ca6304a9ea40 Mon Sep 17 00:00:00 2001 From: Ihor Dvoretskyi Date: Sun, 1 May 2016 20:01:06 -0700 Subject: [PATCH 027/348] Deleted cluster server pointer Deleted cluster server pointer as `kubectl` might be run from any other location, not only from `localhost`. --- docs/getting-started-guides/docker-multinode/testing.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/getting-started-guides/docker-multinode/testing.md b/docs/getting-started-guides/docker-multinode/testing.md index 196d15f19e..c1492bf814 100644 --- a/docs/getting-started-guides/docker-multinode/testing.md +++ b/docs/getting-started-guides/docker-multinode/testing.md @@ -20,7 +20,7 @@ If the status of any node is `Unknown` or `NotReady` your cluster is broken, dou ### Run an application ```shell -kubectl -s http://localhost:8080 run nginx --image=nginx --port=80 +kubectl run nginx --image=nginx --port=80 ``` now run `docker ps` you should see nginx running. You may need to wait a few minutes for the image to get pulled. @@ -65,4 +65,4 @@ And list the pods kubectl get pods ``` -You should see pods landing on the newly added machine. \ No newline at end of file +You should see pods landing on the newly added machine. From daa2d76006cbb4572f988d1627d6b9f1f04f9e46 Mon Sep 17 00:00:00 2001 From: Alex Schultz Date: Wed, 4 May 2016 13:08:03 -0600 Subject: [PATCH 028/348] Update create-certs file location The create-certs function was moved to cluster/common.sh and is no longer located in cluster/gce/util.sh --- docs/getting-started-guides/scratch.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting-started-guides/scratch.md b/docs/getting-started-guides/scratch.md index a6efc189ae..006e5d45b7 100644 --- a/docs/getting-started-guides/scratch.md +++ b/docs/getting-started-guides/scratch.md @@ -210,7 +210,7 @@ You need to prepare several certs: Unless you plan to have a real CA generate your certs, you will need to generate a root cert and use that to sign the master, kubelet, and kubectl certs. -- see function `create-certs` in `cluster/gce/util.sh` +- see function `create-certs` in `cluster/common.sh` - see also `cluster/saltbase/salt/generate-cert/make-ca-cert.sh` and `cluster/saltbase/salt/generate-cert/make-cert.sh` From 407a05b6d2e8abd2a68f76bf95c905df926e0b68 Mon Sep 17 00:00:00 2001 From: Jan Chaloupka Date: Sat, 7 May 2016 09:49:00 +0200 Subject: [PATCH 029/348] Extend secrets docs with key to path mapping and automatic updates of mounted secrets. Secret volume plugin has been refactored. Currently, AtomicWritter is used to project all keys. Allowing to automatically refresh projected secrets. --- docs/user-guide/secrets/index.md | 70 ++++++++++++++++++++++++-------- 1 file changed, 52 insertions(+), 18 deletions(-) diff --git a/docs/user-guide/secrets/index.md b/docs/user-guide/secrets/index.md index 25eb5cfbee..7e23b14bd5 100644 --- a/docs/user-guide/secrets/index.md +++ b/docs/user-guide/secrets/index.md @@ -214,6 +214,52 @@ You can package many files into one secret, or use many secrets, whichever is co See another example of creating a secret and a pod that consumes that secret in a volume [here](/docs/user-guide/secrets/). +##### Projection of secret keys to specific paths + +We can also control the paths within the volume where Secret keys are projected. +You can use `spec.volumes[].secret.items` field to change target path of each key: + +```json +{ + "apiVersion": "v1", + "kind": "Pod", + "metadata": { + "name": "mypod", + "namespace": "myns" + }, + "spec": { + "containers": [{ + "name": "mypod", + "image": "redis", + "volumeMounts": [{ + "name": "foo", + "mountPath": "/etc/foo", + "readOnly": true + }] + }], + "volumes": [{ + "name": "foo", + "secret": { + "secretName": "mysecret", + "items": [{ + "key": "username", + "path": "my-group/my-username" + }] + } + }] + } +} +``` + +What will happen: + +* `username` secret is stored under `/etc/foo/my-group/my-username` file instead of `/etc/foo/username`. +* `password` secret is not projected + +If `spec.volumes[].secret.items` is used, only keys specified in `items` are projected. +To consume all keys from the secret, all of them must be listed in the `items` field. +All listed keys must exist in the corresponding secret. Otherwise, the volume is not created. + ##### Consuming Secret Values from Volumes Inside the container that mounts a secret volume, the secret keys appear as @@ -234,6 +280,11 @@ $ cat /etc/foo/password The program in a container is responsible for reading the secret(s) from the files. +##### Mounted Secrets are updated automatically + +When a secret being already consumed in a volume is updated, projected keys are eventually updated as well. +The update time depends on the kubelet syncing period. + #### Using Secrets as Environment Variables To use a secret in an environment variable in a pod: @@ -334,27 +385,10 @@ secret exists. Once a pod is scheduled, the kubelet will try to fetch the secret value. If the secret cannot be fetched because it does not exist or because of a temporary lack of connection to the API server, kubelet will periodically retry. It will report an event about the pod explaining the -reason it is not started yet. Once the a secret is fetched, the kubelet will +reason it is not started yet. Once the secret is fetched, the kubelet will create and mount a volume containing it. None of the pod's containers will start until all the pod's volumes are mounted. -Once the kubelet has started a pod's containers, its secret volumes will not -change, even if the secret resource is modified. To change the secret used, -the original pod must be deleted, and a new pod (perhaps with an identical -`PodSpec`) must be created. Therefore, updating a secret follows the same -workflow as deploying a new container image. The `kubectl rolling-update` -command can be used ([man page](/docs/user-guide/kubectl/kubectl_rolling-update)). - -The [`resourceVersion`](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/devel/api-conventions.md#concurrency-control-and-consistency) -of the secret is not specified when it is referenced. -Therefore, if a secret is updated at about the same time as pods are starting, -then it is not defined which version of the secret will be used for the pod. It -is not possible currently to check what resource version of a secret object was -used when a pod was created. It is planned that pods will report this -information, so that a replication controller restarts ones using an old -`resourceVersion`. In the interim, if this is a concern, it is recommended to not -update the data of existing secrets, but to create new ones with distinct names. - ## Use cases ### Use-Case: Pod with ssh keys From 536dc08a5d3d2d9721539b57b4c2c796734795c0 Mon Sep 17 00:00:00 2001 From: Andrei Burd Date: Sun, 8 May 2016 12:17:28 +0300 Subject: [PATCH 030/348] Fix: directory structure --- docs/getting-started-guides/ubuntu-calico.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting-started-guides/ubuntu-calico.md b/docs/getting-started-guides/ubuntu-calico.md index 2d454cc94a..219c3a8caa 100644 --- a/docs/getting-started-guides/ubuntu-calico.md +++ b/docs/getting-started-guides/ubuntu-calico.md @@ -304,7 +304,7 @@ On your compute nodes, it is important that you install Calico before Kubernetes mkdir -p /etc/cni/net.d # Make the network configuration file - cat >/etc/rkt/net.d/10-calico.conf </etc/cni/net.d/10-calico.conf < Date: Wed, 11 May 2016 12:39:31 -0400 Subject: [PATCH 031/348] Fix to ensure that ConfigMaps volume does create file(s) in container Document the fix proposed originally in: https://github.com/kubernetes/kubernetes/issues/23392#issuecomment-209663941 Tested with: Client Version: version.Info{Major:"1", Minor:"2", GitVersion:"v1.2.4", GitCommit:"3eed1e3be6848b877ff80a93da3785d9034d0a4f", GitTreeState:"clean"} Server Version: version.Info{Major:"1", Minor:"2", GitVersion:"v1.2.4", GitCommit:"3eed1e3be6848b877ff80a93da3785d9034d0a4f", GitTreeState:"clean"} --- docs/getting-started-guides/docker.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting-started-guides/docker.md b/docs/getting-started-guides/docker.md index e3af1df87f..37f9b22755 100644 --- a/docs/getting-started-guides/docker.md +++ b/docs/getting-started-guides/docker.md @@ -34,7 +34,7 @@ docker run -d \ --volume=/:/rootfs:ro \ --volume=/sys:/sys:ro \ --volume=/var/lib/docker/:/var/lib/docker:rw \ - --volume=/var/lib/kubelet/:/var/lib/kubelet:rw \ + --volume=/var/lib/kubelet/:/var/lib/kubelet:rw,shared \ --volume=/var/run:/var/run:rw \ --net=host \ --pid=host \ From 33fb9fe619a9f3d4703989235431bb60ce6469f3 Mon Sep 17 00:00:00 2001 From: Timo Derstappen Date: Thu, 12 May 2016 10:29:45 +0200 Subject: [PATCH 032/348] libvirt daemon error on arch the virtlogd.socket is not started with the libvirtd daemon. if you enable the libvirtd.service it is linked and started on the next reboot --- docs/getting-started-guides/libvirt-coreos.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/getting-started-guides/libvirt-coreos.md b/docs/getting-started-guides/libvirt-coreos.md index ac2ada1c1b..29ababb583 100644 --- a/docs/getting-started-guides/libvirt-coreos.md +++ b/docs/getting-started-guides/libvirt-coreos.md @@ -280,9 +280,11 @@ Start the libvirt daemon On Arch: ```shell -systemctl start libvirtd +systemctl start libvirtd virtlogd.socket ``` +The `virtlogd.socket` is not started with the libvirtd daemon. If you enable the `libvirtd.service` it is linked and started automatically on the next boot. + On Ubuntu 14.04: ```shell From dae8e5f4a910ca706cde32abc3bbfbc3da88e2ec Mon Sep 17 00:00:00 2001 From: Carson A Date: Thu, 12 May 2016 10:49:28 -0600 Subject: [PATCH 033/348] Update docker.md Clean up some duplicate commands --- docs/getting-started-guides/docker.md | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/docs/getting-started-guides/docker.md b/docs/getting-started-guides/docker.md index e3af1df87f..d8264de6de 100644 --- a/docs/getting-started-guides/docker.md +++ b/docs/getting-started-guides/docker.md @@ -142,11 +142,7 @@ ip=$(kubectl get svc nginx --template={{.spec.clusterIP}}) echo $ip {% endraw %}``` -Hit the webserver with this IP: - -```shell{% raw %} -kubectl get svc nginx --template={{.spec.clusterIP}} -{% endraw %}``` +On Linux the IP is directly accessible via a web browser. On OS X, since docker is running inside a VM, run the following command instead: From 106420a9b0ae83b0a4da1c2fb687dd960796bbd1 Mon Sep 17 00:00:00 2001 From: Carson A Date: Thu, 12 May 2016 14:02:47 -0600 Subject: [PATCH 034/348] Use xargs when removing mounts This avoids potential argument list limits when there are many mounts. --- docs/getting-started-guides/docker.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting-started-guides/docker.md b/docs/getting-started-guides/docker.md index e3af1df87f..16574a57f0 100644 --- a/docs/getting-started-guides/docker.md +++ b/docs/getting-started-guides/docker.md @@ -176,7 +176,7 @@ docker-machine ssh `docker-machine active` ``` ```shell -sudo umount `cat /proc/mounts | grep /var/lib/kubelet | awk '{print $2}'` +grep /var/lib/kubelet /proc/mounts | awk '{print $2}' | sudo xargs -n1 umount sudo rm -rf /var/lib/kubelet ``` From 0700c5ca798fce6d1f323ca70baa5ef45e82e491 Mon Sep 17 00:00:00 2001 From: Rudi C Date: Fri, 13 May 2016 12:01:14 -0400 Subject: [PATCH 035/348] Update service doc with AWS ELB SSL annotations Mostly the same as https://github.com/kubernetes/kubernetes/pull/25574 --- docs/user-guide/services/index.md | 38 +++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/docs/user-guide/services/index.md b/docs/user-guide/services/index.md index 10189ddad0..abbf458f3e 100644 --- a/docs/user-guide/services/index.md +++ b/docs/user-guide/services/index.md @@ -423,6 +423,44 @@ with the user-specified `loadBalancerIP`. If the `loadBalancerIP` field is not s an ephemeral IP will be assigned to the loadBalancer. If the `loadBalancerIP` is specified, but the cloud provider does not support the feature, the field will be ignored. +#### SSL support on AWS +For partial SSL support on clusters running on AWS, starting with 1.3 two +annotations can be added to a `LoadBalancer` service: + +``` + "metadata": { + "name": "my-service", + "annotations": { + "service.beta.kubernetes.io/aws-load-balancer-ssl-cert": "arn:aws:acm:us-east-1:123456789012:certificate/12345678-1234-1234-1234-123456789012" + } + }, +``` + +The first specifies which certificate to use. It can be either a +certificate from a third party issuer that was uploaded to IAM or one created +within AWS Certificate Manager. + +``` + "metadata": { + "name": "my-service", + "annotations": { + "service.beta.kubernetes.io/aws-load-balancer-backend-protocol=": "(https|http|ssl|tcp)" + } + }, +``` + +The second annotation specificies which protocol a pod speaks. For HTTPS and +SSL, the ELB will expect the pod to authenticate itself over the encrypted +connection. + +HTTP and HTTPS will select layer 7 proxying: the ELB will terminate +the connection with the user, parse headers and inject the `X-Forwarded-For` +header with the user's IP address (pods will only see the IP address of the +ELB at the other end of its connection) when forwarding requests. + +TCP and SSL will select layer 4 proxying: the ELB will forward traffic without +modifying the headers. + ### External IPs If there are external IPs that route to one or more cluster nodes, Kubernetes services can be exposed on those From 0924dd341c1209c6607045f5caebdacd12f93d94 Mon Sep 17 00:00:00 2001 From: Andrey Kurilin Date: Tue, 17 May 2016 14:49:42 +0300 Subject: [PATCH 036/348] Add diagram for `kubectl drain` command The link to diagram will be added into long description of `kubectl drain` command. resolves #501 --- images/docs/kubectl_drain.svg | 2 ++ 1 file changed, 2 insertions(+) create mode 100644 images/docs/kubectl_drain.svg diff --git a/images/docs/kubectl_drain.svg b/images/docs/kubectl_drain.svg new file mode 100644 index 0000000000..89c4b6682d --- /dev/null +++ b/images/docs/kubectl_drain.svg @@ -0,0 +1,2 @@ + +
$ kubectl drain NODE [flags]
[Not supported by viewer]
APIServer
[Not supported by viewer]
Kubelet
[Not supported by viewer]
cordon NODE
cordon NODE<br>
RunCordonOrUncordon
<b>RunCordonOrUncordon</b>
get all pods of NODE
get all pods of NODE<br>
GetPodsForDeletion
<b>GetPodsForDeletion</b>
delete first pod
delete first pod<br>
deletePods
<b>deletePods</b>
make one call to
API per pod
make one call to <br>API per pod <br>
delete first pod
delete first pod<br>
podList
podList
Kubernetes Cluster
Kubernetes Cluster<br>
filter mirror pods and 
 DaemonSet-managed 
pods 
[Not supported by viewer]
.
.
.
[Not supported by viewer]
.
.
.
[Not supported by viewer]
delete pod N
delete pod N<br>
delete pod N
delete pod N<br>
Mark node as
Unschedulable
Mark node as <br>Unschedulable<br> \ No newline at end of file From 67c821ed339318e9051e575fca969803ffdcd4dd Mon Sep 17 00:00:00 2001 From: Morgan Bauer Date: Tue, 17 May 2016 11:56:01 -0700 Subject: [PATCH 037/348] use `ip` command instead of obsolete ifconfig in scratch setup - replace use of ifconfig with ip - replace use of brctl with ip --- docs/getting-started-guides/scratch.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/getting-started-guides/scratch.md b/docs/getting-started-guides/scratch.md index a6efc189ae..8f081dd526 100644 --- a/docs/getting-started-guides/scratch.md +++ b/docs/getting-started-guides/scratch.md @@ -320,8 +320,8 @@ as follows before proceeding to configure Docker for Kubernetes. ```shell iptables -t nat -F -ifconfig docker0 down -brctl delbr docker0 +ip link set docker0 down +ip link delete docker0 ``` The way you configure docker will depend in whether you have chosen the routable-vip or overlay-network approaches for your network. @@ -420,7 +420,7 @@ because of how this is used later. 1. Set `--configure-cbr0=false` on kubelet and restart. 1. Create a bridge - - `brctl addbr cbr0`. + - `ip link add name cbr0 type bridge`. 1. Set appropriate MTU. NOTE: the actual value of MTU will depend on your network environment - `ip link set dev cbr0 mtu 1460` 1. Add the node's network to the bridge (docker will go on other side of bridge). From fed4af1151cf8d19f59b51f718026dc489d29360 Mon Sep 17 00:00:00 2001 From: Trevor Huey Date: Tue, 17 May 2016 14:25:04 -0600 Subject: [PATCH 038/348] Fixes "Error: unknown shorthand flag: 't' in -type=json" --- docs/user-guide/kubectl/kubectl_patch.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/kubectl/kubectl_patch.md b/docs/user-guide/kubectl/kubectl_patch.md index 08daf58185..a7653ec330 100644 --- a/docs/user-guide/kubectl/kubectl_patch.md +++ b/docs/user-guide/kubectl/kubectl_patch.md @@ -32,7 +32,7 @@ kubectl patch -f node.json -p '{"spec":{"unschedulable":true}}' kubectl patch pod valid-pod -p '{"spec":{"containers":[{"name":"kubernetes-serve-hostname","image":"new image"}]}}' # Update a container's image using a json patch with positional arrays -kubectl patch pod valid-pod -type='json' -p='[{"op": "replace", "path": "/spec/containers/0/image", "value":"new image"}]' +kubectl patch pod valid-pod --type='json' -p='[{"op": "replace", "path": "/spec/containers/0/image", "value":"new image"}]' ``` ### Options From ff6bb8a671bc8dfc2443059f22fdd293cad73cd7 Mon Sep 17 00:00:00 2001 From: Matt Doran Date: Wed, 18 May 2016 16:44:46 +1000 Subject: [PATCH 039/348] Inconsistent secret name in db credentials example The creation of the secret uses prod/test-db-password, but the remainder of the example uses prod/test-db-secret. --- docs/user-guide/secrets/index.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/user-guide/secrets/index.md b/docs/user-guide/secrets/index.md index 486521742f..cb6ece769c 100644 --- a/docs/user-guide/secrets/index.md +++ b/docs/user-guide/secrets/index.md @@ -425,10 +425,10 @@ credentials. Make the secrets: ```shell -$ kubectl create secret generic prod-db-password --from-literal=user=produser --from-literal=password=Y4nys7f11 -secret "prod-db-password" created -$ kubectl create secret generic test-db-password --from-literal=user=testuser --from-literal=password=iluvtests -secret "test-db-password" created +$ kubectl create secret generic prod-db-secret --from-literal=user=produser --from-literal=password=Y4nys7f11 +secret "prod-db-secret" created +$ kubectl create secret generic test-db-secret --from-literal=user=testuser --from-literal=password=iluvtests +secret "test-db-secret" created ``` Now make the pods: From 982e45e177791cef46a99d3562f6fb9d0ab81d15 Mon Sep 17 00:00:00 2001 From: Elson O Rodriguez Date: Thu, 19 May 2016 10:39:53 -0700 Subject: [PATCH 040/348] Fixing heading issues with Openstack provider documentaiton. --- docs/getting-started-guides/openstack-heat.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/getting-started-guides/openstack-heat.md b/docs/getting-started-guides/openstack-heat.md index 033f5ce878..9d5fe2a042 100644 --- a/docs/getting-started-guides/openstack-heat.md +++ b/docs/getting-started-guides/openstack-heat.md @@ -73,6 +73,7 @@ export IMAGE_FILE=CentOS-7-x86_64-GenericCloud-1510.qcow2 export SWIFT_SERVER_URL=http://192.168.123.100:8080 export ENABLE_PROXY=false ``` + #### Manually overriding configuration values If you do not have your environment variables set, or do not want them consumed, modify the variables in the following files under `cluster/openstack-heat`: @@ -105,6 +106,7 @@ make clean make quick-release KUBERNETES_PROVIDER=openstack-heat ./cluster/kube-up.sh ``` + ## Inspect your cluster Once kube-up is finished, your cluster should be running: From fab1e0a9f9a4f0b163afc48d7a82015e1209edd1 Mon Sep 17 00:00:00 2001 From: Tim Brockman Date: Thu, 19 May 2016 16:31:35 -0400 Subject: [PATCH 041/348] Update hellonode.md to address issue #404 Lines 106-114 briefly describes reason for the Connection refused error and resolution. --- docs/hellonode.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/docs/hellonode.md b/docs/hellonode.md index 702601687e..a784d0f8d2 100755 --- a/docs/hellonode.md +++ b/docs/hellonode.md @@ -103,6 +103,14 @@ Visit your app in the browser, or use `curl` or `wget` if you’d like : $ curl http://localhost:8080 Hello World! ``` +**If you get a `Connection refused` message, your `DOCKER_HOST` address may be the address of your Docker VM, not the localhost address.** Use the `docker-machine ip default` command to find the Docker VM address. + +```shell +$ docker-machine ip default +192.168.1.100 +$ curl 192.168.1.100:8080 +Hello World! +``` Let’s now stop the container. In this example, our app was running as Docker process `2c66d0efcbd4`, which we looked up with `docker ps`: From 7b5aaf7af7283b09ac32e59b4add7f93ce6ba0cb Mon Sep 17 00:00:00 2001 From: Elson O Rodriguez Date: Fri, 20 May 2016 11:45:12 -0700 Subject: [PATCH 042/348] Re-adding heat installation for Openstack provider. --- docs/getting-started-guides/openstack-heat.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/getting-started-guides/openstack-heat.md b/docs/getting-started-guides/openstack-heat.md index 9d5fe2a042..5cc19d85ee 100644 --- a/docs/getting-started-guides/openstack-heat.md +++ b/docs/getting-started-guides/openstack-heat.md @@ -28,6 +28,7 @@ If you already have the required versions of the OpenStack CLI tools installed a ```sh sudo pip install -U --force 'python-openstackclient==2.4.0' +sudo pip install -U --force 'python-heatclient==1.1.0' sudo pip install -U --force 'python-swiftclient==3.0.0' sudo pip install -U --force 'python-glanceclient==2.0.0' sudo pip install -U --force 'python-novaclient==3.4.0' From 6232bb49fc5b79eb69e3970bfff1432afed610ec Mon Sep 17 00:00:00 2001 From: Tim Brockman Date: Sun, 22 May 2016 22:40:59 -0400 Subject: [PATCH 043/348] Added empty line before changes --- docs/hellonode.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/hellonode.md b/docs/hellonode.md index a784d0f8d2..401de6c4c0 100755 --- a/docs/hellonode.md +++ b/docs/hellonode.md @@ -103,6 +103,7 @@ Visit your app in the browser, or use `curl` or `wget` if you’d like : $ curl http://localhost:8080 Hello World! ``` + **If you get a `Connection refused` message, your `DOCKER_HOST` address may be the address of your Docker VM, not the localhost address.** Use the `docker-machine ip default` command to find the Docker VM address. ```shell From 4380ca93f2c6c0a12363e66df2d3d599b457bdff Mon Sep 17 00:00:00 2001 From: Tolleiv Nietsch Date: Tue, 24 May 2016 23:26:21 +0200 Subject: [PATCH 044/348] Scaling the AWS cluster paragraph added --- docs/getting-started-guides/aws.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/getting-started-guides/aws.md b/docs/getting-started-guides/aws.md index 6ac90d5829..fd66b26b96 100644 --- a/docs/getting-started-guides/aws.md +++ b/docs/getting-started-guides/aws.md @@ -127,6 +127,10 @@ The "Guestbook" application is another popular example to get started with Kuber For more complete applications, please look in the [examples directory](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/) +## Scaling the cluster + +Adding and removing nodes through `kubectl` is not supported. You can still scale the amount of nodes through manual adjustments of the Auto Scaling Group which was created during the installation. In cause you increase the number of nodes, the new node will automatically join the cluster once it's ready. + ## Tearing down the cluster Make sure the environment variables you used to provision your cluster are still exported, then call the following script inside the From 3f32f6b1a01cb18e903845fbdaa55c3c68b787fd Mon Sep 17 00:00:00 2001 From: Sam Ghods Date: Thu, 26 May 2016 14:30:14 -0700 Subject: [PATCH 045/348] Update cluster-management.md Update node maintenance to include the current kubectl commands. --- docs/admin/cluster-management.md | 25 +++++++++---------------- 1 file changed, 9 insertions(+), 16 deletions(-) diff --git a/docs/admin/cluster-management.md b/docs/admin/cluster-management.md index 613ec8afa2..122a376ea3 100644 --- a/docs/admin/cluster-management.md +++ b/docs/admin/cluster-management.md @@ -101,43 +101,36 @@ that you have Google Cloud Monitoring API enabled in Google Developer Console. ## Maintenance on a Node If you need to reboot a node (such as for a kernel upgrade, libc upgrade, hardware repair, etc.), and the downtime is -brief, then when the Kubelet restarts, it will attempt to restart the pods scheduled to it. If the reboot takes longer, +brief, then when the Kubelet restarts, it will attempt to restart the pods scheduled to it. If the reboot takes longer +(5 minutes is the default timeout, controlled by `--pod-eviction-timeout` on the controller-manager), then the node controller will terminate the pods that are bound to the unavailable node. If there is a corresponding -replication controller, then a new copy of the pod will be started on a different node. So, in the case where all +replica set (or replication controller), then a new copy of the pod will be started on a different node. So, in the case where all pods are replicated, upgrades can be done without special coordination, assuming that not all nodes will go down at the same time. If you want more control over the upgrading process, you may use the following workflow: -Mark the node to be rebooted as unschedulable: +Use `kubectl drain` to gracefully terminate all pods on the node while marking the node as unschedulable: ```shell -kubectl replace nodes $NODENAME --patch='{"apiVersion": "v1", "spec": {"unschedulable": true}}' +kubectl drain $NODENAME ``` This keeps new pods from landing on the node while you are trying to get them off. -Get the pods off the machine, via any of the following strategies: - * Wait for finite-duration pods to complete. - * Delete pods with: +For pods with a replica set, the pod will be replaced by a new pod which will be scheduled to a new node. Additionally, if the pod is part of a service, then clients will automatically be redirected to the new pod. -```shell -kubectl delete pods $PODNAME -``` - -For pods with a replication controller, the pod will eventually be replaced by a new pod which will be scheduled to a new node. Additionally, if the pod is part of a service, then clients will automatically be redirected to the new pod. - -For pods with no replication controller, you need to bring up a new copy of the pod, and assuming it is not part of a service, redirect clients to it. +For pods with no replica set, you need to bring up a new copy of the pod, and assuming it is not part of a service, redirect clients to it. Perform maintenance work on the node. Make the node schedulable again: ```shell -kubectl replace nodes $NODENAME --patch='{"apiVersion": "v1", "spec": {"unschedulable": false}}' +kubectl uncordon $NODENAME ``` If you deleted the node's VM instance and created a new one, then a new schedulable node resource will -be created automatically when you create a new VM instance (if you're using a cloud provider that supports +be created automatically (if you're using a cloud provider that supports node discovery; currently this is only Google Compute Engine, not including CoreOS on Google Compute Engine using kube-register). See [Node](/docs/admin/node) for more details. ## Advanced Topics From 72e958b394a063e08f3a656a800a732c1dff1cc3 Mon Sep 17 00:00:00 2001 From: Sam Ghods Date: Thu, 26 May 2016 14:31:15 -0700 Subject: [PATCH 046/348] Update cluster-management.md --- docs/admin/cluster-management.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/admin/cluster-management.md b/docs/admin/cluster-management.md index 122a376ea3..8e4d394a00 100644 --- a/docs/admin/cluster-management.md +++ b/docs/admin/cluster-management.md @@ -102,7 +102,7 @@ that you have Google Cloud Monitoring API enabled in Google Developer Console. If you need to reboot a node (such as for a kernel upgrade, libc upgrade, hardware repair, etc.), and the downtime is brief, then when the Kubelet restarts, it will attempt to restart the pods scheduled to it. If the reboot takes longer -(5 minutes is the default timeout, controlled by `--pod-eviction-timeout` on the controller-manager), +(the default time is 5 minutes, controlled by `--pod-eviction-timeout` on the controller-manager), then the node controller will terminate the pods that are bound to the unavailable node. If there is a corresponding replica set (or replication controller), then a new copy of the pod will be started on a different node. So, in the case where all pods are replicated, upgrades can be done without special coordination, assuming that not all nodes will go down at the same time. From 081c9f969e8e7a25adcc7829c7ce3e66de5db08d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Pawe=C5=82=20Oczad=C5=82y?= Date: Sun, 29 May 2016 13:53:21 +0200 Subject: [PATCH 047/348] Typo in /admin/namespaces --- docs/admin/namespaces/index.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/admin/namespaces/index.md b/docs/admin/namespaces/index.md index 2b9f14e835..c9f51280ad 100644 --- a/docs/admin/namespaces/index.md +++ b/docs/admin/namespaces/index.md @@ -46,6 +46,7 @@ kube-system Active ``` Kubernetes starts with two initial namespaces: + * `default` The default namespace for objects with no other namespace * `kube-system` The namespace for objects created by the Kubernetes system From ca8c46d75bce447f27ff6078d766505509576473 Mon Sep 17 00:00:00 2001 From: harryz Date: Tue, 31 May 2016 14:36:27 +0800 Subject: [PATCH 048/348] Need export KUBERNETES_PROVIDER=local Fix #26453 Need to export KUBERNETES_PROVIDER=local before using cluster/kubectl.sh --- docs/getting-started-guides/locally.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/getting-started-guides/locally.md b/docs/getting-started-guides/locally.md index cc2f9f1107..84ecec34c1 100644 --- a/docs/getting-started-guides/locally.md +++ b/docs/getting-started-guides/locally.md @@ -49,6 +49,7 @@ Your cluster is running, and you want to start running containers! You can now use any of the cluster/kubectl.sh commands to interact with your local setup. ```shell +export KUBERNETES_PROVIDER=local cluster/kubectl.sh get pods cluster/kubectl.sh get services cluster/kubectl.sh get deployments From 3589ddbde77fa19442c44b2b55f994a7e62bd865 Mon Sep 17 00:00:00 2001 From: Jan Chaloupka Date: Wed, 1 Jun 2016 09:34:00 +0200 Subject: [PATCH 049/348] Convert level-5 heading into bold heading --- docs/user-guide/secrets/index.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/user-guide/secrets/index.md b/docs/user-guide/secrets/index.md index 7e23b14bd5..4fcea9e303 100644 --- a/docs/user-guide/secrets/index.md +++ b/docs/user-guide/secrets/index.md @@ -214,7 +214,7 @@ You can package many files into one secret, or use many secrets, whichever is co See another example of creating a secret and a pod that consumes that secret in a volume [here](/docs/user-guide/secrets/). -##### Projection of secret keys to specific paths +**Projection of secret keys to specific paths** We can also control the paths within the volume where Secret keys are projected. You can use `spec.volumes[].secret.items` field to change target path of each key: @@ -260,7 +260,7 @@ If `spec.volumes[].secret.items` is used, only keys specified in `items` are pro To consume all keys from the secret, all of them must be listed in the `items` field. All listed keys must exist in the corresponding secret. Otherwise, the volume is not created. -##### Consuming Secret Values from Volumes +**Consuming Secret Values from Volumes** Inside the container that mounts a secret volume, the secret keys appear as files and the secret values are base-64 decoded and stored inside these files. @@ -280,7 +280,7 @@ $ cat /etc/foo/password The program in a container is responsible for reading the secret(s) from the files. -##### Mounted Secrets are updated automatically +**Mounted Secrets are updated automatically** When a secret being already consumed in a volume is updated, projected keys are eventually updated as well. The update time depends on the kubelet syncing period. @@ -318,7 +318,7 @@ spec: restartPolicy: Never ``` -##### Consuming Secret Values from Environment Variables +**Consuming Secret Values from Environment Variables** Inside a container that consumes a secret in an environment variables, the secret keys appear as normal environment variables containing the base-64 decoded values of the secret data. @@ -336,7 +336,7 @@ $ echo $SECRET_PASSWORD An imagePullSecret is a way to pass a secret that contains a Docker (or other) image registry password to the Kubelet so it can pull a private image on behalf of your Pod. -##### Manually specifying an imagePullSecret +**Manually specifying an imagePullSecret** Use of imagePullSecrets is described in the [images documentation](/docs/user-guide/images/#specifying-imagepullsecrets-on-a-pod) From 03c388d6dae140a3f35f7212cfadbbfce9225e00 Mon Sep 17 00:00:00 2001 From: dlorenc Date: Wed, 1 Jun 2016 14:57:06 -0700 Subject: [PATCH 050/348] Add a warning about Docker For Mac. --- docs/getting-started-guides/docker.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/getting-started-guides/docker.md b/docs/getting-started-guides/docker.md index aa3bf9612a..8efa8ad4e1 100644 --- a/docs/getting-started-guides/docker.md +++ b/docs/getting-started-guides/docker.md @@ -12,6 +12,8 @@ Here's a diagram of what the final result will look like: ## Prerequisites +**Note: These steps have not been tested with the [Docker For Mac or Docker For Windows beta programs](https://blog.docker.com/2016/03/docker-for-mac-windows-beta/).** + 1. You need to have docker installed on one machine. 2. Decide what Kubernetes version to use. Set the `${K8S_VERSION}` variable to a released version of Kubernetes >= "v1.2.0". If you'd like to use the current stable version of Kubernetes, run the following: From 6fc0a6c30d484f19755c3a34b962fa27e8755060 Mon Sep 17 00:00:00 2001 From: liujunhui74 Date: Thu, 2 Jun 2016 11:48:47 +0800 Subject: [PATCH 051/348] Running behind a proxy When running behind a proxy, the setup will fail because the containers cannot obtain certificates with curl. A solution is provided. --- docs/getting-started-guides/docker.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/getting-started-guides/docker.md b/docs/getting-started-guides/docker.md index aa3bf9612a..490c2d41ff 100644 --- a/docs/getting-started-guides/docker.md +++ b/docs/getting-started-guides/docker.md @@ -56,6 +56,11 @@ docker run -d \ > Architectures other than `amd64` are experimental and sometimes unstable, but feel free to try them out! Valid values: `arm`, `arm64` and `ppc64le`. ARM is available with Kubernetes version `v1.3.0-alpha.2` and higher. ARM 64-bit and PowerPC 64 little-endian are available with `v1.3.0-alpha.3` and higher. Track progress on multi-arch support [here](https://github.com/kubernetes/kubernetes/issues/17981) +> If you are behind a proxy, you need to pass the proxy setup to curl in the containers to pull the certificates. Create a .curlrc under /root folder (because the containers are running as root) with the following line: +``` +proxy = : +``` + This actually runs the kubelet, which in turn runs a [pod](/docs/user-guide/pods/) that contains the other master components. ** **SECURITY WARNING** ** services exposed via Kubernetes using Hyperkube are available on the host node's public network interface / IP address. Because of this, this guide is not suitable for any host node/server that is directly internet accessible. Refer to [#21735](https://github.com/kubernetes/kubernetes/issues/21735) for addtional info. From e98120fa8a99510030e323e2d039fd9127827507 Mon Sep 17 00:00:00 2001 From: Tolleiv Nietsch Date: Thu, 2 Jun 2016 12:07:11 +0200 Subject: [PATCH 052/348] Scaling AWS paragraph adjustment --- docs/getting-started-guides/aws.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting-started-guides/aws.md b/docs/getting-started-guides/aws.md index fd66b26b96..b8e56d584e 100644 --- a/docs/getting-started-guides/aws.md +++ b/docs/getting-started-guides/aws.md @@ -129,7 +129,7 @@ For more complete applications, please look in the [examples directory](https:// ## Scaling the cluster -Adding and removing nodes through `kubectl` is not supported. You can still scale the amount of nodes through manual adjustments of the Auto Scaling Group which was created during the installation. In cause you increase the number of nodes, the new node will automatically join the cluster once it's ready. +Adding and removing nodes through `kubectl` is not supported. You can still scale the amount of nodes manually through adjustments of the 'Desired' and 'Max' properties within the [Auto Scaling Group](http://docs.aws.amazon.com/autoscaling/latest/userguide/as-manual-scaling.html), which was created during the installation. ## Tearing down the cluster From 774832039cc4f05255bf239a3afc435076aa6ec5 Mon Sep 17 00:00:00 2001 From: Adam Eivy Date: Thu, 2 Jun 2016 15:37:21 -0700 Subject: [PATCH 053/348] "rather then" -> "rather than" --- docs/user-guide/replication-controller/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/replication-controller/index.md b/docs/user-guide/replication-controller/index.md index 9dbf4f3a93..44546a4ab0 100644 --- a/docs/user-guide/replication-controller/index.md +++ b/docs/user-guide/replication-controller/index.md @@ -15,7 +15,7 @@ replication controller are automatically replaced if they fail, get deleted, or For example, your pods get re-created on a node after disruptive maintenance such as a kernel upgrade. For this reason, we recommend that you use a replication controller even if your application requires only a single pod. You can think of a replication controller as something similar to a process supervisor, -but rather then individual processes on a single node, the replication controller supervises multiple pods +but rather than individual processes on a single node, the replication controller supervises multiple pods across multiple nodes. Replication Controller is often abbreviated to "rc" or "rcs" in discussion, and as a shortcut in From 918fd9139302484a7c76f554e61a73a440be31cb Mon Sep 17 00:00:00 2001 From: Brian Grant Date: Fri, 3 Jun 2016 18:34:29 +0000 Subject: [PATCH 054/348] Improve homepage --- index.html | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/index.html b/index.html index e494580285..101e351654 100644 --- a/index.html +++ b/index.html @@ -1,5 +1,5 @@ --- -title: Accelerate Your Delivery +title: Production-Grade Container Orchestration --- @@ -11,8 +11,8 @@ title: Accelerate Your Delivery
-

Accelerate Your Delivery

-
Manage an infrastructure cluster as a single system to simplify container operations.
+

Production-Grade Container Orchestration

+
Automated container deployment, scaling, and management
Try Our Hello World
@@ -30,9 +30,9 @@ title: Accelerate Your Delivery
-

Kubernetes is an open-source system for automating deployment, operations, and scaling of containerized applications.

+

Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications.

It groups containers that make up an application into logical units for easy management and discovery. Kubernetes builds - upon a decade and a half of experience of running production workloads at Google, + upon 15 years of experience of running production workloads at Google, combined with best-of-breed ideas and practices from the community.

@@ -75,12 +75,12 @@ title: Accelerate Your Delivery
-

Automatic binpacking

+

Automatic binpacking

Automatically places containers based on their resource requirements and other constraints, while not sacrificing availability. Mix critical and best-effort workloads in order to drive up utilization and save even more resources.

-

Self-healing

+

Self-healing

Restarts containers that fail, replaces and reschedules containers when nodes die, kills containers that don’t respond to your user-defined health check, and doesn’t advertise them to clients until they are ready to serve.

@@ -88,11 +88,11 @@ title: Accelerate Your Delivery
-

Horizontal scaling

+

Horizontal scaling

Scale your application up and down with a simple command, with a UI, or automatically based on CPU usage.

-

Service discovery and load balancing

+

Service discovery and load balancing

No need to modify your application to use an unfamiliar service discovery mechanism. Kubernetes gives containers their own IP addresses and a single DNS name for a set of containers, and can load-balance across them.

@@ -100,13 +100,13 @@ title: Accelerate Your Delivery
-

Automated rollouts and rollbacks

+

Automated rollouts and rollbacks

Kubernetes progressively rolls out changes to your application or its configuration, while monitoring application health to ensure it doesn’t kill all your instances at the same time. If something goes wrong, Kubernetes will rollback the change for you. Take advantage of a growing ecosystem of deployment solutions.

-

Secret and configuration management

+

Secret and configuration management

Deploy and update secrets and application configuration without rebuilding your image and without exposing secrets in your stack configuration.

@@ -114,13 +114,13 @@ title: Accelerate Your Delivery
-

Storage orchestration

+

Storage orchestration

Automatically mount the storage system of your choice, whether from local storage, a public cloud provider such as GCP or AWS, or a network storage system such as NFS, iSCSI, Gluster, Ceph, Cinder, or Flocker.

-

Batch execution

+

Batch execution

In addition to services, Kubernetes can manage your batch and CI workloads, replacing containers that fail, if desired.

From 819aae8c35ffc51903a7c3b0ae7f573b1a44731b Mon Sep 17 00:00:00 2001 From: Mike Spreitzer Date: Mon, 6 Jun 2016 14:05:17 -0400 Subject: [PATCH 055/348] Fix doc issues 617, 616, and 614 For 617: changed the master setup instructions to install the kubelet after Calico. Also noted the additional file, to fix 616 and replaced `rkt` with `cni` to fix 614. --- docs/getting-started-guides/ubuntu-calico.md | 78 ++++++++++---------- 1 file changed, 39 insertions(+), 39 deletions(-) diff --git a/docs/getting-started-guides/ubuntu-calico.md b/docs/getting-started-guides/ubuntu-calico.md index d1adfa2c73..be3d26fac4 100644 --- a/docs/getting-started-guides/ubuntu-calico.md +++ b/docs/getting-started-guides/ubuntu-calico.md @@ -70,42 +70,6 @@ The master requires the root CA public key, `ca.pem`; the apiserver certificate, sudo chown root:root /etc/kubernetes/ssl/apiserver-key.pem ``` -### Install Kubernetes on the Master - -We'll use the `kubelet` to bootstrap the Kubernetes master. - -1. Download and install the `kubelet` and `kubectl` binaries: - - ```shell - sudo wget -N -P /usr/bin http://storage.googleapis.com/kubernetes-release/release/v1.1.4/bin/linux/amd64/kubectl - sudo wget -N -P /usr/bin http://storage.googleapis.com/kubernetes-release/release/v1.1.4/bin/linux/amd64/kubelet - sudo chmod +x /usr/bin/kubelet /usr/bin/kubectl - ``` - -2. Install the `kubelet` systemd unit file and start the `kubelet`: - - ```shell - # Install the unit file - sudo wget -N -P /etc/systemd https://raw.githubusercontent.com/projectcalico/calico-cni/k8s-1.1-docs/samples/kubernetes/master/kubelet.service - - # Enable the unit file so that it runs on boot - sudo systemctl enable /etc/systemd/kubelet.service - - # Start the kubelet service - sudo systemctl start kubelet.service - ``` - -3. Download and install the master manifest file, which will start the Kubernetes master services automatically: - - ```shell - sudo mkdir -p /etc/kubernetes/manifests - sudo wget -N -P /etc/kubernetes/manifests https://raw.githubusercontent.com/projectcalico/calico-cni/k8s-1.1-docs/samples/kubernetes/master/kubernetes-master.manifest - ``` - -4. Check the progress by running `docker ps`. After a while, you should see the `etcd`, `apiserver`, `controller-manager`, `scheduler`, and `kube-proxy` containers running. - - > Note: it may take some time for all the containers to start. Don't worry if `docker ps` doesn't show any containers for a while or if some containers start before others. - ### Install Calico's etcd on the master Calico needs its own etcd cluster to store its state. In this guide we install a single-node cluster on the master server. @@ -120,7 +84,7 @@ Calico needs its own etcd cluster to store its state. In this guide we install 2. Replace all instances of `` in the `calico-etcd.manifest` file with your master's IP address. -3. Then, move the file to the `/etc/kubernetes/manifests` directory: +3. Then, move the file to the `/etc/kubernetes/manifests` directory. This will not have any effect until we later run the kubelet, but Calico seems to tolerate the lack of its etcd in the interim. ```shell sudo mv -f calico-etcd.manifest /etc/kubernetes/manifests @@ -168,6 +132,42 @@ We need to install Calico on the master. This allows the master to route packet sudo systemctl start calico-node.service ``` +### Install Kubernetes on the Master + +We'll use the `kubelet` to bootstrap the Kubernetes master. + +1. Download and install the `kubelet` and `kubectl` binaries: + + ```shell + sudo wget -N -P /usr/bin http://storage.googleapis.com/kubernetes-release/release/v1.1.4/bin/linux/amd64/kubectl + sudo wget -N -P /usr/bin http://storage.googleapis.com/kubernetes-release/release/v1.1.4/bin/linux/amd64/kubelet + sudo chmod +x /usr/bin/kubelet /usr/bin/kubectl + ``` + +2. Install the `kubelet` systemd unit file and start the `kubelet`: + + ```shell + # Install the unit file + sudo wget -N -P /etc/systemd https://raw.githubusercontent.com/projectcalico/calico-cni/k8s-1.1-docs/samples/kubernetes/master/kubelet.service + + # Enable the unit file so that it runs on boot + sudo systemctl enable /etc/systemd/kubelet.service + + # Start the kubelet service + sudo systemctl start kubelet.service + ``` + +3. Download and install the master manifest file, which will start the Kubernetes master services automatically: + + ```shell + sudo mkdir -p /etc/kubernetes/manifests + sudo wget -N -P /etc/kubernetes/manifests https://raw.githubusercontent.com/projectcalico/calico-cni/k8s-1.1-docs/samples/kubernetes/master/kubernetes-master.manifest + ``` + +4. Check the progress by running `docker ps`. After a while, you should see the `etcd`, `apiserver`, `controller-manager`, `scheduler`, and `kube-proxy` containers running. + + > Note: it may take some time for all the containers to start. Don't worry if `docker ps` doesn't show any containers for a while or if some containers start before others. + ## Set up the nodes The following steps should be run on each Kubernetes node. @@ -192,7 +192,7 @@ Worker nodes require three keys: `ca.pem`, `worker.pem`, and `worker-key.pem`. IP.1 = $ENV::WORKER_IP ``` -2. Generate the necessary TLS assets for this worker. This relies on the worker's IP address, and the `ca.pem` file generated earlier in the guide. +2. Generate the necessary TLS assets for this worker. This relies on the worker's IP address, and the `ca.pem` and `ca-key.pem` files generated earlier in the guide. ```shell # Export this worker's IP address. @@ -304,7 +304,7 @@ On your compute nodes, it is important that you install Calico before Kubernetes mkdir -p /etc/cni/net.d # Make the network configuration file - cat >/etc/rkt/net.d/10-calico.conf </etc/cni/net.d/10-calico.conf < Date: Wed, 4 May 2016 19:48:38 -0400 Subject: [PATCH 056/348] Fix errors in how compute resources are handled --- docs/user-guide/compute-resources.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/docs/user-guide/compute-resources.md b/docs/user-guide/compute-resources.md index 0d90e117e2..bd8fd278f2 100644 --- a/docs/user-guide/compute-resources.md +++ b/docs/user-guide/compute-resources.md @@ -91,10 +91,16 @@ runner (Docker or rkt). When using Docker: -- The `spec.container[].resources.limits.cpu` is multiplied by 1024, converted to an integer, and - used as the value of the [`--cpu-shares`]( +- The `spec.container[].resources.requests.cpu` is converted to its core value (potentially fractional), + and multipled by 1024, and used as the value of the [`--cpu-shares`]( https://docs.docker.com/reference/run/#runtime-constraints-on-resources) flag to the `docker run` command. +- The `spec.container[].resources.limits.cpu` is converted to its millicore value, + multipled by 100000, and then divided by 1000, and used as the value of the [`--cpu-quota`]( + https://docs.docker.com/reference/run/#runtime-constraints-on-resources) flag to the `docker run` + command. The [`--cpu-period`] flag is set to 100000 which represents the default 100ms period + for measuring quota usage. The kubelet enforces cpu limits if it was started with the + [`--cpu-cfs-quota`] flag set to true. As of version 1.2, this flag will now default to true. - The `spec.container[].resources.limits.memory` is converted to an integer, and used as the value of the [`--memory`](https://docs.docker.com/reference/run/#runtime-constraints-on-resources) flag to the `docker run` command. From 6cf91c527c08dcc61693bf8ecdfb54c7832a76fb Mon Sep 17 00:00:00 2001 From: Janet Kuo Date: Fri, 20 May 2016 09:18:40 -0700 Subject: [PATCH 057/348] Document kubectl rollout status and kubectl set image for deployments --- docs/user-guide/deployments.md | 117 ++++++++++++++++++++------------- 1 file changed, 71 insertions(+), 46 deletions(-) diff --git a/docs/user-guide/deployments.md b/docs/user-guide/deployments.md index 1cd688e8c8..a76dbff5e7 100644 --- a/docs/user-guide/deployments.md +++ b/docs/user-guide/deployments.md @@ -76,47 +76,39 @@ The created Replica Set will ensure that there are three nginx Pods at all times ## The Status of a Deployment -After creating or updating a Deployment, you would want to confirm whether it succeeded or not. The best way to do this is through checking its status. - -To verify if the above Deployment succeeded or not, first compare the `.metadata.generation` and `.status.observedGeneration` of the Deployment: +After creating or updating a Deployment, you would want to confirm whether it succeeded or not. The simplest way to do this is through `kubectl rollout status`. ```shell -$ kubectl get deployment/nginx-deployment -o yaml | grep [Gg]eneration - generation: 2 - observedGeneration: 2 +$ kubectl rollout status deployment/nginx-deployment +deployment nginx-deployment successfully rolled out ``` -When `observedGeneration` >= `generation`, the Deployment controller has observed current Deployment; if not, wait for a few more seconds. +This verifies the Deployment's `.status.observedGeneration` >= `.metadata.generation`, and its up-to-date replicas +(`.status.updatedReplicas`) matches the desired replicas (`.spec.replicas`) to determine if the rollout succeeded. +If the rollout is still in progress, it watches for Deployment status changes and prints related messages. -Once the above condition is met, check the Deployment's up-to-date replicas (`.status.updatedReplicas`) and see if it matches the desired replicas (`.spec.replicas`): - -```shell -$ kubectl get deployment/nginx-deployment -NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE -nginx-deployment 3 3 3 3 9m -``` +Note that it's impossible to know whether a Deployment will ever succeed, so if the above command doesn't return success, +you'll need to timeout and give up at some point. Additionally, if you set `.spec.minReadySeconds`, you would also want to check if the available replicas (`.status.availableReplicas`) matches the desired replicas too. -**Note:** It's impossible to know whether a Deployment will ever succeed, so one has to timeout and give up at some point. +```shell +$ kubectl get deployments +NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE +nginx-deployment 3 3 3 3 20s +``` ## Updating a Deployment -Suppose that we now want to update the nginx Pods to start using the `nginx:1.9.1` image -instead of the `nginx:1.7.9` image. - **Note:** a Deployment's rollout is triggered if and only if the Deployment's pod template (i.e. `.spec.template`) is changed, e.g. updating labels or container images of the template. Other updates, such as scaling the Deployment, will not trigger a rollout. -First, we update our Deployment configuration as follows: - -{% include code.html language="yaml" file="new-nginx-deployment.yaml" ghlink="/docs/user-guide/new-nginx-deployment.yaml" %} - -We can then `apply` the new Deployment: +Suppose that we now want to update the nginx Pods to start using the `nginx:1.9.1` image +instead of the `nginx:1.7.9` image. ```shell -$ kubectl apply -f docs/user-guide/new-nginx-deployment.yaml -deployment "nginx-deployment" configured +$ kubectl set image deployment/nginx-deployment nginx=nginx:1.9.1 +deployment "nginx-deployment" image updated ``` Alternatively, we can `edit` the Deployment and change `.spec.template.spec.containers[0].image` from `nginx:1.7.9` to `nginx:1.9.1`: @@ -126,17 +118,15 @@ $ kubectl edit deployment/nginx-deployment deployment "nginx-deployment" edited ``` -Running a `get` immediately will give: +To see its rollout status, simply run: ```shell -$ kubectl get deployments -NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE -nginx-deployment 3 3 0 3 20s +$ kubectl rollout status deployment/nginx-deployment +Waiting for rollout to finish: 2 out of 3 new replicas have been updated... +deployment nginx-deployment successfully rolled out ``` -The 0 number of up-to-date replicas indicates that the Deployment hasn't updated the replicas to the latest configuration. The current replicas indicates the total replicas (3 with old configuration and 0 with new configuration) this Deployment manages, and the available replicas indicates the number of current replicas that are available. - -The Deployment will update all the Pods in a few seconds. +After the rollout succeeds, you may want to `get` the Deployment: ```shell $ kubectl get deployments @@ -144,6 +134,10 @@ NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE nginx-deployment 3 3 3 3 36s ``` +The number of up-to-date replicas indicates that the Deployment has updated the replicas to the latest configuration. +The current replicas indicates the total replicas this Deployment manages, and the available replicas indicates the +number of current replicas that are available. + We can run `kubectl get rs` to see that the Deployment updated the Pods by creating a new Replica Set and scaling it up to 3 replicas, as well as scaling down the old Replica Set to 0 replicas. ```shell @@ -163,7 +157,7 @@ nginx-deployment-1564180365-nacti 1/1 Running 0 14s nginx-deployment-1564180365-z9gth 1/1 Running 0 14s ``` -Next time we want to update these Pods, we only need to update and re-apply the Deployment again. +Next time we want to update these Pods, we only need to update the Deployment's pod template again. Deployment can ensure that only a certain number of Pods may be down while they are being updated. By default, it ensures that at least 1 less than the desired number of Pods are @@ -237,14 +231,21 @@ auto-scaling. This implies that when you rollback to an earlier revision, only t Suppose that we made a typo while updating the Deployment, by putting the image name as `nginx:1.91` instead of `nginx:1.9.1`: -{% include code.html language="yaml" file="bad-nginx-deployment.yaml" ghlink="/docs/user-guide/bad-nginx-deployment.yaml" %} - ```shell -$ kubectl apply -f docs/user-guide/bad-nginx-deployment.yaml -deployment "nginx-deployment" configured +$ kubectl set image deployment/nginx-deployment nginx=nginx:1.91 +deployment "nginx-deployment" image updated ``` -You will see that both the number of old replicas (nginx-deployment-1564180365 and nginx-deployment-2035384211) and new replicas (nginx-deployment-3066724191) are 2. +The rollout will be stuck. + +``` +$ kubectl rollout status deployments nginx-deployment +Waiting for rollout to finish: 2 out of 3 new replicas have been updated... +``` + +Press Ctrl-C to stop the above rollout status watch. + +You will also see that both the number of old replicas (nginx-deployment-1564180365 and nginx-deployment-2035384211) and new replicas (nginx-deployment-3066724191) are 2. ```shell $ kubectl get rs @@ -305,8 +306,8 @@ $ kubectl rollout history deployment/nginx-deployment deployments "nginx-deployment": REVISION CHANGE-CAUSE 1 kubectl create -f docs/user-guide/nginx-deployment.yaml --record -2 kubectl apply -f docs/user-guide/new-nginx-deployment.yaml -3 kubectl apply -f docs/user-guide/bad-nginx-deployment.yaml +2 kubectl set image deployment/nginx-deployment nginx=nginx:1.9.1 +3 kubectl set image deployment/nginx-deployment nginx=nginx:1.91 ``` Because we recorded the command while creating this Deployment using `--record`, we can easily see the changes we made in each revision. @@ -316,10 +317,18 @@ To further see the details of each revision, run: ```shell $ kubectl rollout history deployment/nginx-deployment --revision=2 deployments "nginx-deployment" revision 2 -Labels: app=nginx,pod-template-hash=1564180365 -Annotations: kubernetes.io/change-cause=kubectl apply -f docs/user-guide/new-nginx-deployment.yaml -Image(s): nginx:1.9.1 -No volumes. + Labels: app=nginx + pod-template-hash=1159050644 + Annotations: kubernetes.io/change-cause=kubectl set image deployment/nginx-deployment nginx=nginx:1.9.1 + Containers: + nginx: + Image: nginx:1.9.1 + Port: 80/TCP + QoS Tier: + cpu: BestEffort + memory: BestEffort + Environment Variables: + No volumes. ``` ### Rolling Back to a Previous Revision @@ -346,6 +355,7 @@ The Deployment is now rolled back to a previous stable revision. As you can see, $ kubectl get deployment NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE nginx-deployment 3 3 3 3 30m + $ kubectl describe deployment Name: nginx-deployment Namespace: default @@ -381,8 +391,8 @@ You can also pause a Deployment mid-way and then resume it. A use case is to sup Update the Deployment again and then pause the Deployment with `kubectl rollout pause`: ```shell -$ kubectl apply -f docs/user-guide/new-nginx-deployment; kubectl rollout pause deployment/nginx-deployment -deployment "nginx-deployment" configured +$ kubectl set image deployment/nginx-deployment nginx=nginx:1.9.1; kubectl rollout pause deployment/nginx-deployment +deployment "nginx-deployment" image updated deployment "nginx-deployment" paused ``` @@ -398,6 +408,13 @@ nginx-deployment-2035384211 2 2 1h nginx-deployment-3066724191 0 0 1h ``` +In a separate terminal, watch for rollout status changes and you'll see the rollout won't continue: + +```shell +$ kubectl rollout status deployment/nginx-deployment +Waiting for rollout to finish: 2 out of 3 new replicas have been updated... +``` + To resume the Deployment, simply do `kubectl rollout resume`: ```shell @@ -407,6 +424,14 @@ deployment "nginx-deployment" resumed Then the Deployment will continue and finish the rollout: +```shell +$ kubectl rollout status deployment/nginx-deployment +Waiting for rollout to finish: 2 out of 3 new replicas have been updated... +Waiting for deployment spec update to be observed... +Waiting for rollout to finish: 2 out of 3 new replicas have been updated... +deployment nginx-deployment successfully rolled out +``` + ```shell $ kubectl get rs NAME DESIRED CURRENT AGE From 940fba455af6ae9db097cdaf65bf2d5e964bd016 Mon Sep 17 00:00:00 2001 From: Cole Mickens Date: Mon, 14 Mar 2016 12:43:35 -0700 Subject: [PATCH 058/348] azure: document new bring-up --- docs/getting-started-guides/azure.md | 155 +++++++++++++++++++++++++++ docs/getting-started-guides/index.md | 4 +- 2 files changed, 158 insertions(+), 1 deletion(-) create mode 100644 docs/getting-started-guides/azure.md diff --git a/docs/getting-started-guides/azure.md b/docs/getting-started-guides/azure.md new file mode 100644 index 0000000000..e41b544c35 --- /dev/null +++ b/docs/getting-started-guides/azure.md @@ -0,0 +1,155 @@ +--- +--- + +* TOC +{:toc} + + +## Prerequisites + +1. An Azure subscription. If you don't already have one, you may create one on [azure.microsoft.com](https://azure.microsoft.com). +2. An account with Owner access to the subscription. +3. Both `docker` and `jq` need to be installed and available on `$PATH`. + + +## Cluster operations + +### Cluster bring-up + +```shell +export KUBERNETES_PROVIDER=azure; curl -sS https://get.k8s.io | bash +``` + +NOTE: This script calls [cluster/kube-up.sh](http://releases.k8s.io/{{page.githubbranch}}/cluster/kube-up.sh) +which in turn calls [cluster/azure/util.sh](http://releases.k8s.io/{{page.githubbranch}}/cluster/azure/util.sh) +using [cluster/azure/config-default.sh](http://releases.k8s.io/{{page.githubbranch}}/cluster/azure/config-default.sh). + +You must set `AZURE_SUBSCRIPTION_ID` or you will receive errors. It may be set in `cluster/azure/config-default.sh` or set as an environment variable: + +```shell +export AZURE_SUBSCRIPTION_ID="" +``` + +These values can be overriden by setting them in `cluster/azure/config-default.sh` or as environment variables. They are shown here with their default values: + +```shell +export AZURE_DEPLOY_ID="" # autogenerated if blank +export AZURE_LOCATION="westus" +export AZURE_RESOURCE_GROUP="" # generated from AZURE_DEPLOY_ID if unset +export AZURE_MASTER_SIZE="Standard_A1" +export AZURE_NODE_SIZE="Standard_A1" +export AZURE_USERNAME="kube" +export NUM_NODES=3 +export AZURE_AUTH_METHOD="device" +``` + + +By default, this will deploy a cluster with 4 `Standard_A1`-sized VMs: one master node, three worker nodes. This process takes about 5 to 10 minutes. Once the cluster is up, connection information to the cluster will be displayed. Additionally, your `kubectl` configuration will be updated to know about this cluster and this new cluster will be set as the active context. + +The Azure deployment process produces an output directory `cluster/azure/_deployments/${AZURE_DEPLOY_ID}`. In this directory you will find the PKI and SSH assets created for the cluster, as well as a script named `util.sh`. Here are some examples of its usage: + +``` +$ cd cluster/azure/_deployments/kube-20160316-001122/ + +# This uses the client cert with curl to make an http call to the apiserver. +$ ./util.sh curl api/v1/nodes + +# This uses the client cert with kubectl to target this deployment's apiserver. +$ ./util.sh kubectl get nodes + +# This alters the current kubectl configuration to point at this cluster. +$ ./util.sh configure-kubectl + +# This will deploy the kube-system namespace, the SkyDNS addon, and the kube-dashboard addon. +$ ./util.sh deploy-addons + +# This uses the ssh private key to copy the private key itself to the master node. +$ ./util.sh copykey + +# This uses the ssh private key to open an ssh connection to the master. +$ ./util.sh ssh +``` + +### Cluster deployment examples + +#### Using `kubectl proxy` + +`kubectl proxy` is currently used to access to access deployed services. + +``` +kubectl proxy --port=8001 +``` + +Deployed services are available at: `http://localhost:8001/api/v1/proxy/namespaces//services/`. + + +#### Addon: SkyDNS + +You can deploy the [SkyDNS addon](https://github.com/kubernetes/kubernetes/tree/release-1.2/cluster/addons/dns): + +``` +kubectl create -f https://raw.githubusercontent.com/colemickens/azkube/v0.0.4/templates/coreos/addons/skydns.yaml +``` + + +#### Addon: Kube-Dashboard + +This will deploy the [`kube-dashboard`](https://github.com/kubernetes/dashboard) addon: + +``` +kubectl create -f https://raw.githubusercontent.com/colemickens/azkube/v0.0.4/templates/coreos/addons/kube-dashboard.yaml +``` + +The dashboard is then available at: `http://localhost:8001/api/v1/proxy/namespaces/kube-system/services/dashboard-canary`. + +#### Example: Guestbook + +This will deploy the [`guestbook example`](https://github.com/kubernetes/kubernetes/blob/release-1.2/examples/guestbook/README.md) (the all-in-one variant): + +``` +kubectl create -f https://raw.githubusercontent.com/kubernetes/kubernetes/release-1.2/examples/guestbook/all-in-one/guestbook-all-in-one.yaml +``` + +The guestbook is then available at: `http://localhost:8001/api/v1/proxy/namespaces/default/services/frontend`. + + +### Cluster scaling + +The `azkube` tool used internally during `kube-up` can also be used to scale your cluster. +Here's an example of scaling a default deployment of 3 nodes to 10 nodes: + +``` +export AZURE_DEPLOY_ID="kube-20160316-001122" +$ docker run -it -v "$HOME/.azkube:/.azkube" -v "/tmp:/tmp" \ + colemickens/azkube:v0.0.4 /opt/azkube/azkube scale \ + --deployment-name="${AZURE_DEPLOY_ID}" \ + --node-size="Standard_A1" \ + --node-count=10 +``` + +### Cluster tear-down + +You can tear-down a cluster using `kube-down.sh`: + +```shell +export AZURE_DEPLOY_ID="kube-20160316-001122" +$ ./cluster/kube-down.sh +``` + + +### Notes + +1. The user account used for these operations must have Owner access to the subscription. +2. You can find your subscription ID in the [Azure Portal](https://portal.microsoft.com). (All Resources → Subscriptions) +3. The `AZURE_AUTH_METHOD` environment variable controls what authentication mechanism is used when bringing up the cluster. By default it is set to `device`. This allows the user to login via the a web browser. This interactive step can be automated by creating a Service Principal, setting `AZURE_AUTH_METHOD=client_secret` and setting `AZURE_CLIENT_ID` + `AZURE_CLIENT_SECRET` as appropriate for your Service Principal. +4. The `--node-size` used in the `scale` command must be the same size deployed initially or it will not have the desired effect. +5. Cluster tear-down requires manual intervention, due to the fact that it deletes the entire resource group and someone else may have deployed other resources since the initial deployment. For this reason you must confirm the list of resources that are to be deleted. If you wish to skip it, you may set `AZURE_DOWN_SKIP_CONFIRM` to `true`. This will delete everything in the resource group that was deployed to. +6. If you are deploying from a checkout of `kubernetes`, then you will need to take an additional step to ensure that a `hyperkube` image is available. You can set `AZURE_DOCKER_REGISTRY` and `AZURE_DOCKER_REPO` and the deployment will ensure that a hyperkube container is built and available in the specified Docker registry. That `hyperkube` image will then be used throughout the cluster for running the Kubernetes services. Alternatively, you may set `AZURE_HYPERKUBE_SPEC` to point to a custom `hyperkube` image. + + + +## Further reading + +* Please see the [azkube](https://github.com/colemickens/azkube) repository for more information about the deployment tool that manages the deployment. +* Please see the [Kubernetes docs](/docs/) for more details on administering +and using a Kubernetes cluster. diff --git a/docs/getting-started-guides/index.md b/docs/getting-started-guides/index.md index 81216ac79b..6fdff08367 100644 --- a/docs/getting-started-guides/index.md +++ b/docs/getting-started-guides/index.md @@ -48,7 +48,8 @@ few commands, and have active community support. - [GCE](/docs/getting-started-guides/gce) - [AWS](/docs/getting-started-guides/aws) -- [Azure](/docs/getting-started-guides/coreos/azure/) +- [Azure](/docs/getting-started-guides/coreos/azure/) (Weave-based, contributed by WeaveWorks employees) +- [Azure](/docs/getting-started-guides/azure/) (Flannel-based, contributed by Microsoft employee) - [CenturyLink Cloud](/docs/getting-started-guides/clc) ### Custom Solutions @@ -116,6 +117,7 @@ IaaS Provider | Config. Mgmt | OS | Networking | Docs GKE | | | GCE | [docs](https://cloud.google.com/container-engine) | ['œ“][3] | Commercial GCE | Saltstack | Debian | GCE | [docs](/docs/getting-started-guides/gce) | ['œ“][1] | Project Azure | CoreOS | CoreOS | Weave | [docs](/docs/getting-started-guides/coreos/azure/) | | Community ([@errordeveloper](https://github.com/errordeveloper), [@squillace](https://github.com/squillace), [@chanezon](https://github.com/chanezon), [@crossorigin](https://github.com/crossorigin)) +Azure | CoreOS | CoreOS | flannel | [docs](/docs/getting-started-guides/azure) | | Community ([@colemickens](https://github.com/colemickens)) Docker Single Node | custom | N/A | local | [docs](/docs/getting-started-guides/docker) | | Project ([@brendandburns](https://github.com/brendandburns)) Docker Multi Node | custom | N/A | flannel | [docs](/docs/getting-started-guides/docker-multinode) | | Project ([@brendandburns](https://github.com/brendandburns)) Bare-metal | Ansible | Fedora | flannel | [docs](/docs/getting-started-guides/fedora/fedora_ansible_config) | | Project From 5ba83f14285041770725fd94f22dd3ed5cefef4b Mon Sep 17 00:00:00 2001 From: Cole Mickens Date: Mon, 4 Apr 2016 14:39:47 -0700 Subject: [PATCH 059/348] azure: repond to feedback --- docs/getting-started-guides/azure.md | 37 +++++++++++++++++----------- 1 file changed, 23 insertions(+), 14 deletions(-) diff --git a/docs/getting-started-guides/azure.md b/docs/getting-started-guides/azure.md index e41b544c35..4179b20c97 100644 --- a/docs/getting-started-guides/azure.md +++ b/docs/getting-started-guides/azure.md @@ -48,7 +48,7 @@ By default, this will deploy a cluster with 4 `Standard_A1`-sized VMs: one maste The Azure deployment process produces an output directory `cluster/azure/_deployments/${AZURE_DEPLOY_ID}`. In this directory you will find the PKI and SSH assets created for the cluster, as well as a script named `util.sh`. Here are some examples of its usage: -``` +```shell $ cd cluster/azure/_deployments/kube-20160316-001122/ # This uses the client cert with curl to make an http call to the apiserver. @@ -72,11 +72,23 @@ $ ./util.sh ssh ### Cluster deployment examples +#### Deploy the `kube-system` namespace + +The cluster addons are created in the `kube-system` namespace. + +For versions of Kubernetes before 1.2.2, this must be done manually. Starting with 1.2.2, the +namespace is created automatically as part of the Azure bring-up. For 1.2.0 and 1.2.1, you may +execute this to create the `kube-system` namespace: + +```shell +kubectl create -f https://raw.githubusercontent.com/colemickens/azkube/v0.0.5/templates/coreos/addons/kube-system.yaml +``` + #### Using `kubectl proxy` `kubectl proxy` is currently used to access to access deployed services. -``` +```shell kubectl proxy --port=8001 ``` @@ -85,10 +97,10 @@ Deployed services are available at: `http://localhost:8001/api/v1/proxy/namespac #### Addon: SkyDNS -You can deploy the [SkyDNS addon](https://github.com/kubernetes/kubernetes/tree/release-1.2/cluster/addons/dns): +You can deploy the [SkyDNS addon](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/cluster/addons/dns): -``` -kubectl create -f https://raw.githubusercontent.com/colemickens/azkube/v0.0.4/templates/coreos/addons/skydns.yaml +```shell +kubectl create -f https://raw.githubusercontent.com/colemickens/azkube/v0.0.5/templates/coreos/addons/skydns.yaml ``` @@ -96,17 +108,17 @@ kubectl create -f https://raw.githubusercontent.com/colemickens/azkube/v0.0.4/te This will deploy the [`kube-dashboard`](https://github.com/kubernetes/dashboard) addon: -``` -kubectl create -f https://raw.githubusercontent.com/colemickens/azkube/v0.0.4/templates/coreos/addons/kube-dashboard.yaml +```shell +kubectl create -f https://raw.githubusercontent.com/colemickens/azkube/v0.0.5/templates/coreos/addons/kube-dashboard.yaml ``` The dashboard is then available at: `http://localhost:8001/api/v1/proxy/namespaces/kube-system/services/dashboard-canary`. #### Example: Guestbook -This will deploy the [`guestbook example`](https://github.com/kubernetes/kubernetes/blob/release-1.2/examples/guestbook/README.md) (the all-in-one variant): +This will deploy the [`guestbook example`](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/examples/guestbook/README.md) (the all-in-one variant): -``` +```shell kubectl create -f https://raw.githubusercontent.com/kubernetes/kubernetes/release-1.2/examples/guestbook/all-in-one/guestbook-all-in-one.yaml ``` @@ -118,10 +130,10 @@ The guestbook is then available at: `http://localhost:8001/api/v1/proxy/namespac The `azkube` tool used internally during `kube-up` can also be used to scale your cluster. Here's an example of scaling a default deployment of 3 nodes to 10 nodes: -``` +```shell export AZURE_DEPLOY_ID="kube-20160316-001122" $ docker run -it -v "$HOME/.azkube:/.azkube" -v "/tmp:/tmp" \ - colemickens/azkube:v0.0.4 /opt/azkube/azkube scale \ + colemickens/azkube:v0.0.5 /opt/azkube/azkube scale \ --deployment-name="${AZURE_DEPLOY_ID}" \ --node-size="Standard_A1" \ --node-count=10 @@ -147,9 +159,6 @@ $ ./cluster/kube-down.sh 6. If you are deploying from a checkout of `kubernetes`, then you will need to take an additional step to ensure that a `hyperkube` image is available. You can set `AZURE_DOCKER_REGISTRY` and `AZURE_DOCKER_REPO` and the deployment will ensure that a hyperkube container is built and available in the specified Docker registry. That `hyperkube` image will then be used throughout the cluster for running the Kubernetes services. Alternatively, you may set `AZURE_HYPERKUBE_SPEC` to point to a custom `hyperkube` image. - ## Further reading * Please see the [azkube](https://github.com/colemickens/azkube) repository for more information about the deployment tool that manages the deployment. -* Please see the [Kubernetes docs](/docs/) for more details on administering -and using a Kubernetes cluster. From 060de443e158cfcb71323e0bb7329fdd0553f200 Mon Sep 17 00:00:00 2001 From: Cole Mickens Date: Mon, 4 Apr 2016 15:15:29 -0700 Subject: [PATCH 060/348] azure: add tenant id back for pre-1.2.2 --- docs/getting-started-guides/azure.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/getting-started-guides/azure.md b/docs/getting-started-guides/azure.md index 4179b20c97..17ccb9eeae 100644 --- a/docs/getting-started-guides/azure.md +++ b/docs/getting-started-guides/azure.md @@ -24,10 +24,12 @@ NOTE: This script calls [cluster/kube-up.sh](http://releases.k8s.io/{{page.githu which in turn calls [cluster/azure/util.sh](http://releases.k8s.io/{{page.githubbranch}}/cluster/azure/util.sh) using [cluster/azure/config-default.sh](http://releases.k8s.io/{{page.githubbranch}}/cluster/azure/config-default.sh). -You must set `AZURE_SUBSCRIPTION_ID` or you will receive errors. It may be set in `cluster/azure/config-default.sh` or set as an environment variable: +You must set `AZURE_SUBSCRIPTION_ID` or you will receive errors. Prior to Kubernetes 1.3.0, you must also set `AZURE_TENANT_ID`. +These may be set in `cluster/azure/config-default.sh` or set as environment variables: ```shell export AZURE_SUBSCRIPTION_ID="" +export AZURE_TENANT_ID="" # only needed for Kubernetes < v1.3.0. ``` These values can be overriden by setting them in `cluster/azure/config-default.sh` or as environment variables. They are shown here with their default values: @@ -76,8 +78,8 @@ $ ./util.sh ssh The cluster addons are created in the `kube-system` namespace. -For versions of Kubernetes before 1.2.2, this must be done manually. Starting with 1.2.2, the -namespace is created automatically as part of the Azure bring-up. For 1.2.0 and 1.2.1, you may +For versions of Kubernetes before 1.3.0, this must be done manually. Starting with 1.3.0, the +namespace is created automatically as part of the Azure bring-up. For versions prior to 1.3.0, you may execute this to create the `kube-system` namespace: ```shell From 3f216dd33ad650429bad747eafeabee39e91a835 Mon Sep 17 00:00:00 2001 From: Cole Mickens Date: Mon, 6 Jun 2016 15:43:40 -0700 Subject: [PATCH 061/348] azure: add note about manual teardown before 1.3 --- docs/getting-started-guides/azure.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/getting-started-guides/azure.md b/docs/getting-started-guides/azure.md index 17ccb9eeae..78e74e7231 100644 --- a/docs/getting-started-guides/azure.md +++ b/docs/getting-started-guides/azure.md @@ -150,6 +150,7 @@ export AZURE_DEPLOY_ID="kube-20160316-001122" $ ./cluster/kube-down.sh ``` +Prior to Kubernetes 1.3, the cluster must be deleted manually with the Azure CLI or via the Azure Portal. ### Notes From 8546c704199ac2c6153326941aeed516bdf63a9b Mon Sep 17 00:00:00 2001 From: Cole Mickens Date: Mon, 6 Jun 2016 15:48:43 -0700 Subject: [PATCH 062/348] Add new Azure bringup to sidebar --- _data/guides.yml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/_data/guides.yml b/_data/guides.yml index f5654a6efd..a005817839 100644 --- a/_data/guides.yml +++ b/_data/guides.yml @@ -142,8 +142,10 @@ toc: path: /docs/getting-started-guides/gce/ - title: Running Kubernetes on AWS EC2 path: /docs/getting-started-guides/aws/ - - title: Running Kubernetes on Azure + - title: Running Kubernetes on Azure (Weave-based) path: /docs/getting-started-guides/coreos/azure/ + - title: Running Kubernetes on Azure (Flannel-based) + path: /docs/getting-started-guides/azure/ - title: Running Kubernetes on CenturyLink Cloud path: /docs/getting-started-guides/clc/ - title: Portable Multi-Node Clusters From dede608aef8dcdb82e035a99aa8049f664fb8a60 Mon Sep 17 00:00:00 2001 From: Ben Hosmer Date: Tue, 7 Jun 2016 12:53:15 -0400 Subject: [PATCH 063/348] Document how to create more than one node in the cluster. --- docs/getting-started-guides/vagrant.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/getting-started-guides/vagrant.md b/docs/getting-started-guides/vagrant.md index 99127f5233..dde252be21 100644 --- a/docs/getting-started-guides/vagrant.md +++ b/docs/getting-started-guides/vagrant.md @@ -38,6 +38,12 @@ The `KUBERNETES_PROVIDER` environment variable tells all of the various cluster By default, the Vagrant setup will create a single master VM (called kubernetes-master) and one node (called kubernetes-node-1). Each VM will take 1 GB, so make sure you have at least 2GB to 4GB of free memory (plus appropriate free disk space). +If you'd like more than one node, set the `NUM_NODES` environment variable to the number you want: + +```sh +export NUM_NODES=3 +``` + Vagrant will provision each machine in the cluster with all the necessary components to run Kubernetes. The initial setup can take a few minutes to complete on each machine. If you installed more than one Vagrant provider, Kubernetes will usually pick the appropriate one. However, you can override which one Kubernetes will use by setting the [`VAGRANT_DEFAULT_PROVIDER`](https://docs.vagrantup.com/v2/providers/default.html) environment variable: From b31f05cf421ab80f1ffc9bdecfea1f68cd73f1a0 Mon Sep 17 00:00:00 2001 From: Jan Chaloupka Date: Sat, 7 May 2016 09:49:00 +0200 Subject: [PATCH 064/348] Extend secrets docs with key to path mapping and automatic updates of mounted secrets. Secret volume plugin has been refactored. Currently, AtomicWritter is used to project all keys. Allowing to automatically refresh projected secrets. Convert level-5 heading into bold heading --- docs/user-guide/secrets/index.md | 74 +++++++++++++++++++++++--------- 1 file changed, 54 insertions(+), 20 deletions(-) diff --git a/docs/user-guide/secrets/index.md b/docs/user-guide/secrets/index.md index 64bf51a1cb..3ab0ca8e38 100644 --- a/docs/user-guide/secrets/index.md +++ b/docs/user-guide/secrets/index.md @@ -214,7 +214,53 @@ You can package many files into one secret, or use many secrets, whichever is co See another example of creating a secret and a pod that consumes that secret in a volume [here](/docs/user-guide/secrets/). -##### Consuming Secret Values from Volumes +**Projection of secret keys to specific paths** + +We can also control the paths within the volume where Secret keys are projected. +You can use `spec.volumes[].secret.items` field to change target path of each key: + +```json +{ + "apiVersion": "v1", + "kind": "Pod", + "metadata": { + "name": "mypod", + "namespace": "myns" + }, + "spec": { + "containers": [{ + "name": "mypod", + "image": "redis", + "volumeMounts": [{ + "name": "foo", + "mountPath": "/etc/foo", + "readOnly": true + }] + }], + "volumes": [{ + "name": "foo", + "secret": { + "secretName": "mysecret", + "items": [{ + "key": "username", + "path": "my-group/my-username" + }] + } + }] + } +} +``` + +What will happen: + +* `username` secret is stored under `/etc/foo/my-group/my-username` file instead of `/etc/foo/username`. +* `password` secret is not projected + +If `spec.volumes[].secret.items` is used, only keys specified in `items` are projected. +To consume all keys from the secret, all of them must be listed in the `items` field. +All listed keys must exist in the corresponding secret. Otherwise, the volume is not created. + +**Consuming Secret Values from Volumes** Inside the container that mounts a secret volume, the secret keys appear as files and the secret values are base-64 decoded and stored inside these files. @@ -234,6 +280,11 @@ $ cat /etc/foo/password The program in a container is responsible for reading the secret(s) from the files. +**Mounted Secrets are updated automatically** + +When a secret being already consumed in a volume is updated, projected keys are eventually updated as well. +The update time depends on the kubelet syncing period. + #### Using Secrets as Environment Variables To use a secret in an environment variable in a pod: @@ -267,7 +318,7 @@ spec: restartPolicy: Never ``` -##### Consuming Secret Values from Environment Variables +**Consuming Secret Values from Environment Variables** Inside a container that consumes a secret in an environment variables, the secret keys appear as normal environment variables containing the base-64 decoded values of the secret data. @@ -285,7 +336,7 @@ $ echo $SECRET_PASSWORD An imagePullSecret is a way to pass a secret that contains a Docker (or other) image registry password to the Kubelet so it can pull a private image on behalf of your Pod. -##### Manually specifying an imagePullSecret +**Manually specifying an imagePullSecret** Use of imagePullSecrets is described in the [images documentation](/docs/user-guide/images/#specifying-imagepullsecrets-on-a-pod) @@ -338,23 +389,6 @@ reason it is not started yet. Once the secret is fetched, the kubelet will create and mount a volume containing it. None of the pod's containers will start until all the pod's volumes are mounted. -Once the kubelet has started a pod's containers, its secret volumes will not -change, even if the secret resource is modified. To change the secret used, -the original pod must be deleted, and a new pod (perhaps with an identical -`PodSpec`) must be created. Therefore, updating a secret follows the same -workflow as deploying a new container image. The `kubectl rolling-update` -command can be used ([man page](/docs/user-guide/kubectl/kubectl_rolling-update)). - -The [`resourceVersion`](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/devel/api-conventions.md#concurrency-control-and-consistency) -of the secret is not specified when it is referenced. -Therefore, if a secret is updated at about the same time as pods are starting, -then it is not defined which version of the secret will be used for the pod. It -is not possible currently to check what resource version of a secret object was -used when a pod was created. It is planned that pods will report this -information, so that a replication controller restarts ones using an old -`resourceVersion`. In the interim, if this is a concern, it is recommended to not -update the data of existing secrets, but to create new ones with distinct names. - ## Use cases ### Use-Case: Pod with ssh keys From 07d137ee1c54b974a8cdbaa22480b71eecccf2db Mon Sep 17 00:00:00 2001 From: Joshua Perry Date: Tue, 7 Jun 2016 15:33:26 -0600 Subject: [PATCH 065/348] How selectors interact with headless services There was no information on how the system handles headless services differently. --- docs/user-guide/services/index.md | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/docs/user-guide/services/index.md b/docs/user-guide/services/index.md index 334e53df8d..2084733810 100644 --- a/docs/user-guide/services/index.md +++ b/docs/user-guide/services/index.md @@ -317,18 +317,27 @@ Sometimes you don't need or want load-balancing and a single service IP. In this case, you can create "headless" services by specifying `"None"` for the cluster IP (`spec.clusterIP`). -For such `Services`, a cluster IP is not allocated. DNS is configured to return -multiple A records (addresses) for the `Service` name, which point directly to -the `Pods` backing the `Service`. Additionally, the kube proxy does not handle -these services and there is no load balancing or proxying done by the platform -for them. The endpoints controller will still create `Endpoints` records in -the API. - This option allows developers to reduce coupling to the Kubernetes system, if they desire, but leaves them freedom to do discovery in their own way. Applications can still use a self-registration pattern and adapters for other discovery systems could easily be built upon this API. +For such `Services` a cluster IP is not allocated, the kube proxy does not handle +these services, and there is no load balancing or proxying done by the platform +for them. How DNS is automatically configured depends on if the service has +selectors or not. + +### With selectors +For headless services that define selectors, the endpoints controller will still +create `Endpoints` records in the API, and DNS is configured to return multiple A +records (addresses) for the `Service` name which point directly to the `Pods` +backing the `Service`. + +### Without selectors +For headless services that do not define selectors, the endpoints controller does +not create `Endpoints` records. However, the DNS system looks for and configures +A records for any `Endpoints` that share a name with the service. + ## Publishing services - service types For some parts of your application (e.g. frontends) you may want to expose a From 5b256c4a41bd3d08f0191b880175419ee97e75b7 Mon Sep 17 00:00:00 2001 From: Eyal Levin Date: Wed, 8 Jun 2016 16:47:17 +0300 Subject: [PATCH 066/348] Fix ENVVAR syntax link --- docs/user-guide/configuring-containers.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/configuring-containers.md b/docs/user-guide/configuring-containers.md index afe942fdef..99eb9221bc 100644 --- a/docs/user-guide/configuring-containers.md +++ b/docs/user-guide/configuring-containers.md @@ -83,7 +83,7 @@ spec: # specification of the pod’s contents args: ["/bin/echo \"${MESSAGE}\""] ``` -However, a shell isn't necessary just to expand environment variables. Kubernetes will do it for you if you use [`$(ENVVAR)` syntax](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/expansion): +However, a shell isn't necessary just to expand environment variables. Kubernetes will do it for you if you use [`$(ENVVAR)` syntax](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/expansion.md): ```yaml command: ["/bin/echo"] From 64655f70021384178f54c59a57e4254704933703 Mon Sep 17 00:00:00 2001 From: Eyal Levin Date: Wed, 8 Jun 2016 17:03:42 +0300 Subject: [PATCH 067/348] Add link for more info on configuration files. --- docs/hellonode.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/hellonode.md b/docs/hellonode.md index 702601687e..2f1e0b8feb 100755 --- a/docs/hellonode.md +++ b/docs/hellonode.md @@ -290,7 +290,7 @@ Building and pushing this updated image should be much quicker as we take full a We’re now ready for kubernetes to smoothly update our deployment to the new version of the application. In order to change the image label for our running container, we will need to edit the existing *hello-node deployment* and change the image from `gcr.io/PROJECT_ID/hello-node:v1` to `gcr.io/PROJECT_ID/hello-node:v2`. To do this, we will use the `kubectl edit` command. -This will open up a text editor displaying the full deployment yaml configuration. It isn't necessary to understand the full yaml config +This will open up a text editor displaying the full deployment yaml [configuration](/docs/user-guide/configuring-containers/). It isn't necessary to understand the full yaml config right now, instead just understand that by updating the `spec.template.spec.containers.image` field in the config we are telling the deployment to update the pods to use the new image. From 2274e3616ea128fb3960f909c41a54309fc08a01 Mon Sep 17 00:00:00 2001 From: Alex Robinson Date: Wed, 8 Jun 2016 09:08:54 -0700 Subject: [PATCH 068/348] Add table of contents and fix formatting of network plugins page --- docs/admin/network-plugins.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/docs/admin/network-plugins.md b/docs/admin/network-plugins.md index 4034c8a467..99f58dd246 100644 --- a/docs/admin/network-plugins.md +++ b/docs/admin/network-plugins.md @@ -1,6 +1,9 @@ --- --- +* TOC +{:toc} + __Disclaimer__: Network plugins are in alpha. Its contents will change rapidly. Network plugins in Kubernetes come in a few flavors: @@ -32,10 +35,11 @@ The CNI plugin is selected by passing Kubelet the `--network-plugin=cni` command ### kubenet -The Linux-only kubenet plugin provides functionality similar to the `--configure-cbr0` kubelet command-line option. It creates a Linux bridge named `cbr0` and creates a veth pair for each pod with the host end of each pair connected to `cbr0`. The pod end of the pair is assigned an IP address allocated from a range assigned to the node through either configuration or by the controller-manager. `cbr0` is assigned an MTU matching the smallest MTU of an enabled normal interface on the host. The kubenet plugin is currently mutually exclusive with, and will eventually replace, the --configure-cbr0 option. It is also currently incompatible with the flannel experimental overlay. +The Linux-only kubenet plugin provides functionality similar to the `--configure-cbr0` kubelet command-line option. It creates a Linux bridge named `cbr0` and creates a veth pair for each pod with the host end of each pair connected to `cbr0`. The pod end of the pair is assigned an IP address allocated from a range assigned to the node either through configuration or by the controller-manager. `cbr0` is assigned an MTU matching the smallest MTU of an enabled normal interface on the host. The kubenet plugin is currently mutually exclusive with, and will eventually replace, the --configure-cbr0 option. It is also currently incompatible with the flannel experimental overlay. The plugin requires a few things: + * The standard CNI `bridge` and `host-local` plugins to be placed in `/opt/cni/bin`. * Kubelet must be run with the `--network-plugin=kubenet` argument to enable the plugin * Kubelet must also be run with the `--reconcile-cidr` argument to ensure the IP subnet assigned to the node by configuration or the controller-manager is propagated to the plugin -* The node must be assigned an IP subnet through either the `--pod-cidr` kubelet command-line option or the `--allocate-node-cidrs=true --cluster-cidr=` controller-manager command-line options. \ No newline at end of file +* The node must be assigned an IP subnet through either the `--pod-cidr` kubelet command-line option or the `--allocate-node-cidrs=true --cluster-cidr=` controller-manager command-line options. From cfaf885568d1239da98c8026455e48bb31ca4ced Mon Sep 17 00:00:00 2001 From: Avesh Agarwal Date: Wed, 1 Jun 2016 16:34:37 -0400 Subject: [PATCH 069/348] Fix the path to dapi-volume.yaml and update the example to reflect the current state --- docs/user-guide/downward-api/volume/index.md | 28 ++++++++++---------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/docs/user-guide/downward-api/volume/index.md b/docs/user-guide/downward-api/volume/index.md index 446d8eb563..3cfdc2c2d9 100644 --- a/docs/user-guide/downward-api/volume/index.md +++ b/docs/user-guide/downward-api/volume/index.md @@ -17,7 +17,7 @@ This example assumes you have a Kubernetes cluster installed and running, and th ### Step One: Create the pod -Use the `docs/user-guide/downward-api/dapi-volume.yaml` file to create a Pod with a  downward API volume which stores pod labels and pod annotations to `/etc/labels` and  `/etc/annotations` respectively. +Use the [dapi-volume.yaml](/docs/user-guide/downward-api/volume/dapi-volume.yaml) file to create a Pod with a downward API volume which stores pod labels and pod annotations to `/etc/labels` and `/etc/annotations` respectively. ```shell $ kubectl create -f docs/user-guide/downward-api/volume/dapi-volume.yaml @@ -46,21 +46,21 @@ In pod's `/etc` directory one may find the file created by the plugin (system fi $ kubectl exec kubernetes-downwardapi-volume-example -i -t -- sh / # ls -laR /etc /etc: -total 32 -drwxrwxrwt 3 0 0 180 Aug 24 13:03 . -drwxr-xr-x 1 0 0 4096 Aug 24 13:05 .. -drwx------ 2 0 0 80 Aug 24 13:03 ..2015_08_24_13_03_44259413923 -lrwxrwxrwx 1 0 0 30 Aug 24 13:03 ..downwardapi -> ..2015_08_24_13_03_44259413923 -lrwxrwxrwx 1 0 0 25 Aug 24 13:03 annotations -> ..downwardapi/annotations -lrwxrwxrwx 1 0 0 20 Aug 24 13:03 labels -> ..downwardapi/labels +total 4 +drwxrwxrwt 3 0 0 120 Jun 1 19:55 . +drwxr-xr-x 17 0 0 4096 Jun 1 19:55 .. +drwxr-xr-x 2 0 0 80 Jun 1 19:55 ..6986_01_06_15_55_10.473583074 +lrwxrwxrwx 1 0 0 31 Jun 1 19:55 ..data -> ..6986_01_06_15_55_10.473583074 +lrwxrwxrwx 1 0 0 18 Jun 1 19:55 annotations -> ..data/annotations +lrwxrwxrwx 1 0 0 13 Jun 1 19:55 labels -> ..data/labels -/etc/..2015_08_24_13_03_44259413923: +/etc/..6986_01_06_15_55_10.473583074: total 8 -drwx------ 2 0 0 80 Aug 24 13:03 . -drwxrwxrwt 3 0 0 180 Aug 24 13:03 .. --rw-r--r-- 1 0 0 115 Aug 24 13:03 annotations --rw-r--r-- 1 0 0 53 Aug 24 13:03 labels +drwxr-xr-x 2 0 0 80 Jun 1 19:55 . +drwxrwxrwt 3 0 0 120 Jun 1 19:55 .. +-rw-r--r-- 1 0 0 129 Jun 1 19:55 annotations +-rw-r--r-- 1 0 0 59 Jun 1 19:55 labels / # ``` -The file `labels` is stored in a temporary directory (`..2015_08_24_13_03_44259413923` in the example above) which is symlinked to by `..downwardapi`. Symlinks for annotations and labels in `/etc` point to files containing the actual metadata through the `..downwardapi` indirection.  This structure allows for dynamic atomic refresh of the metadata: updates are written to a new temporary directory, and the `..downwardapi` symlink is updated atomically using `rename(2)`. \ No newline at end of file +The file `labels` is stored in a temporary directory (`..6986_01_06_15_55_10.473583074` in the example above) which is symlinked to by `..data`. Symlinks for annotations and labels in `/etc` point to files containing the actual metadata through the `..data` indirection.  This structure allows for dynamic atomic refresh of the metadata: updates are written to a new temporary directory, and the `..data` symlink is updated atomically using `rename(2)`. From 90bac918eb06955412ef72fbcbb89dc7d94ec508 Mon Sep 17 00:00:00 2001 From: lucianomores Date: Wed, 8 Jun 2016 16:43:38 -0300 Subject: [PATCH 070/348] Bump azure-cli to 0.10.1 due to incompatibility issues with node v6.0.0+ --- docs/getting-started-guides/coreos/azure/package.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting-started-guides/coreos/azure/package.json b/docs/getting-started-guides/coreos/azure/package.json index 3f7e05f737..2ab720ea45 100644 --- a/docs/getting-started-guides/coreos/azure/package.json +++ b/docs/getting-started-guides/coreos/azure/package.json @@ -9,7 +9,7 @@ "author": "Ilya Dmitrichenko ", "license": "Apache 2.0", "dependencies": { - "azure-cli": "^0.9.9", + "azure-cli": "^0.10.1", "colors": "^1.0.3", "js-yaml": "^3.2.5", "openssl-wrapper": "^0.2.1", From ccac6c8ae2890ea34bbedcbdc1fcbe8b5da65682 Mon Sep 17 00:00:00 2001 From: Euan Kemp Date: Wed, 8 Jun 2016 13:53:49 -0700 Subject: [PATCH 071/348] Update github tree links to use a valid version The `page.version` variable is set to a value like "1.x", while github releases always have a version with 3 parts (like "1.x.y"). Using `page.version` by itself in a kubernetes tree link does not work and each of the changed links below 404s right now. In the future, it might make sense to instead have a version variable that includes the patch number as well. --- docs/admin/network-plugins.md | 6 +++--- docs/getting-started-guides/coreos/bare_metal_calico.md | 2 +- docs/getting-started-guides/ubuntu-calico.md | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/admin/network-plugins.md b/docs/admin/network-plugins.md index 4034c8a467..297e90afba 100644 --- a/docs/admin/network-plugins.md +++ b/docs/admin/network-plugins.md @@ -18,13 +18,13 @@ The kubelet has a single default network plugin, and a default network common to ## Network Plugin Requirements -Besides providing the [`NetworkPlugin` interface](https://github.com/kubernetes/kubernetes/tree/{{page.version}}/pkg/kubelet/network/plugins.go) to configure and clean up pod networking, the plugin may also need specific support for kube-proxy. The iptables proxy obviously depends on iptables, and the plugin may need to ensure that container traffic is made available to iptables. For example, if the plugin connects containers to a Linux bridge, the plugin must set the `net/bridge/bridge-nf-call-iptables` sysctl to `1` to ensure that the iptables proxy functions correctly. If the plugin does not use a Linux bridge (but instead something like Open vSwitch or some other mechanism) it should ensure container traffic is appropriately routed for the proxy. +Besides providing the [`NetworkPlugin` interface](https://github.com/kubernetes/kubernetes/tree/{{page.version}}.0/pkg/kubelet/network/plugins.go) to configure and clean up pod networking, the plugin may also need specific support for kube-proxy. The iptables proxy obviously depends on iptables, and the plugin may need to ensure that container traffic is made available to iptables. For example, if the plugin connects containers to a Linux bridge, the plugin must set the `net/bridge/bridge-nf-call-iptables` sysctl to `1` to ensure that the iptables proxy functions correctly. If the plugin does not use a Linux bridge (but instead something like Open vSwitch or some other mechanism) it should ensure container traffic is appropriately routed for the proxy. By default if no kubelet network plugin is specified, the `noop` plugin is used, which sets `net/bridge/bridge-nf-call-iptables=1` to ensure simple configurations (like docker with a bridge) work correctly with the iptables proxy. ### Exec -Place plugins in `network-plugin-dir/plugin-name/plugin-name`, i.e if you have a bridge plugin and `network-plugin-dir` is `/usr/lib/kubernetes`, you'd place the bridge plugin executable at `/usr/lib/kubernetes/bridge/bridge`. See [this comment](https://github.com/kubernetes/kubernetes/tree/{{page.version}}/pkg/kubelet/network/exec/exec.go) for more details. +Place plugins in `network-plugin-dir/plugin-name/plugin-name`, i.e if you have a bridge plugin and `network-plugin-dir` is `/usr/lib/kubernetes`, you'd place the bridge plugin executable at `/usr/lib/kubernetes/bridge/bridge`. See [this comment](https://github.com/kubernetes/kubernetes/tree/{{page.version}}.0/pkg/kubelet/network/exec/exec.go) for more details. ### CNI @@ -38,4 +38,4 @@ The plugin requires a few things: * The standard CNI `bridge` and `host-local` plugins to be placed in `/opt/cni/bin`. * Kubelet must be run with the `--network-plugin=kubenet` argument to enable the plugin * Kubelet must also be run with the `--reconcile-cidr` argument to ensure the IP subnet assigned to the node by configuration or the controller-manager is propagated to the plugin -* The node must be assigned an IP subnet through either the `--pod-cidr` kubelet command-line option or the `--allocate-node-cidrs=true --cluster-cidr=` controller-manager command-line options. \ No newline at end of file +* The node must be assigned an IP subnet through either the `--pod-cidr` kubelet command-line option or the `--allocate-node-cidrs=true --cluster-cidr=` controller-manager command-line options. diff --git a/docs/getting-started-guides/coreos/bare_metal_calico.md b/docs/getting-started-guides/coreos/bare_metal_calico.md index b3f4890398..06ca2697a7 100644 --- a/docs/getting-started-guides/coreos/bare_metal_calico.md +++ b/docs/getting-started-guides/coreos/bare_metal_calico.md @@ -168,7 +168,7 @@ kubectl create -f https://raw.githubusercontent.com/projectcalico/calico-cni/k8s ## Launch other Services With Calico-Kubernetes -At this point, you have a fully functioning cluster running on Kubernetes with a master and two nodes networked with Calico. You can now follow any of the [standard documentation](https://github.com/kubernetes/kubernetes/tree/{{page.version}}/examples/) to set up other services on your cluster. +At this point, you have a fully functioning cluster running on Kubernetes with a master and two nodes networked with Calico. You can now follow any of the [standard documentation](https://github.com/kubernetes/kubernetes/tree/{{page.version}}.0/examples/) to set up other services on your cluster. ## Connectivity to outside the cluster diff --git a/docs/getting-started-guides/ubuntu-calico.md b/docs/getting-started-guides/ubuntu-calico.md index d1adfa2c73..fb12cae859 100644 --- a/docs/getting-started-guides/ubuntu-calico.md +++ b/docs/getting-started-guides/ubuntu-calico.md @@ -436,7 +436,7 @@ kubectl create -f https://raw.githubusercontent.com/projectcalico/calico-cni/k8s ## Launch other Services With Calico-Kubernetes -At this point, you have a fully functioning cluster running on Kubernetes with a master and two nodes networked with Calico. You can now follow any of the [standard documentation](https://github.com/kubernetes/kubernetes/tree/{{page.version}}/examples/) to set up other services on your cluster. +At this point, you have a fully functioning cluster running on Kubernetes with a master and two nodes networked with Calico. You can now follow any of the [standard documentation](https://github.com/kubernetes/kubernetes/tree/{{page.version}}.0/examples/) to set up other services on your cluster. ## Connectivity to outside the cluster From 05ce7e27dd2a006852edcbf739c0fe6160582047 Mon Sep 17 00:00:00 2001 From: Ricky Pai Date: Wed, 8 Jun 2016 17:04:10 -0700 Subject: [PATCH 072/348] Fix self-referencing link in Horizontal Pod Autoscaling doc The link was self-referencing. Linking the walkthrough that has actual example. --- docs/user-guide/horizontal-pod-autoscaling/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/horizontal-pod-autoscaling/index.md b/docs/user-guide/horizontal-pod-autoscaling/index.md index 8c98d9beb7..dabcb28daa 100644 --- a/docs/user-guide/horizontal-pod-autoscaling/index.md +++ b/docs/user-guide/horizontal-pod-autoscaling/index.md @@ -80,4 +80,4 @@ the horizontal pod autoscaler will not be bound to the new replication controlle * Design documentation: [Horizontal Pod Autoscaling](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/horizontal-pod-autoscaler.md). * Manual of autoscale command in kubectl: [kubectl autoscale](/docs/user-guide/kubectl/kubectl_autoscale). -* Usage example of [Horizontal Pod Autoscaler](/docs/user-guide/horizontal-pod-autoscaling/). +* Usage example of [Horizontal Pod Autoscaler](/docs/user-guide/horizontal-pod-autoscaling/walkthrough/). From 6dbfc3456d87dd6e8ad1a6887317e959d37b6439 Mon Sep 17 00:00:00 2001 From: Andrew Smith Date: Thu, 9 Jun 2016 17:55:25 +0100 Subject: [PATCH 073/348] GC gc's unused images, not unreferenced Tags reference images but from what I see the kubelet will delete a tagged image that isn't in *use* by a container. When I first read this I took this to mean that it wouldn't GC tagged images. --- docs/admin/garbage-collection.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/admin/garbage-collection.md b/docs/admin/garbage-collection.md index 2ccd4bd2e9..81aeb772c0 100644 --- a/docs/admin/garbage-collection.md +++ b/docs/admin/garbage-collection.md @@ -4,7 +4,7 @@ * TOC {:toc} -Garbage collection is a helpful function of kubelet that will clean up unreferenced images and unused containers. kubelet will perform garbage collection for containers every minute and garbage collection for images every five minutes. +Garbage collection is a helpful function of kubelet that will clean up unused images and unused containers. kubelet will perform garbage collection for containers every minute and garbage collection for images every five minutes. External garbage collection tools are not recommended as these tools can potentially break the behavior of kubelet by removing containers expected to exist. From f066890632ddae67b0c496871d17704e3defb981 Mon Sep 17 00:00:00 2001 From: Joshua Perry Date: Thu, 9 Jun 2016 14:04:30 -0600 Subject: [PATCH 074/348] Delint --- docs/user-guide/services/index.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/user-guide/services/index.md b/docs/user-guide/services/index.md index 2084733810..587a740fea 100644 --- a/docs/user-guide/services/index.md +++ b/docs/user-guide/services/index.md @@ -328,12 +328,14 @@ for them. How DNS is automatically configured depends on if the service has selectors or not. ### With selectors + For headless services that define selectors, the endpoints controller will still create `Endpoints` records in the API, and DNS is configured to return multiple A records (addresses) for the `Service` name which point directly to the `Pods` backing the `Service`. ### Without selectors + For headless services that do not define selectors, the endpoints controller does not create `Endpoints` records. However, the DNS system looks for and configures A records for any `Endpoints` that share a name with the service. From 015ee234502865c6370df42c44ea4acebcbd2676 Mon Sep 17 00:00:00 2001 From: Joshua Perry Date: Thu, 9 Jun 2016 14:10:20 -0600 Subject: [PATCH 075/348] Wording update --- docs/user-guide/services/index.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/docs/user-guide/services/index.md b/docs/user-guide/services/index.md index 587a740fea..b34ce10d7c 100644 --- a/docs/user-guide/services/index.md +++ b/docs/user-guide/services/index.md @@ -329,10 +329,9 @@ selectors or not. ### With selectors -For headless services that define selectors, the endpoints controller will still -create `Endpoints` records in the API, and DNS is configured to return multiple A -records (addresses) for the `Service` name which point directly to the `Pods` -backing the `Service`. +For headless services that define selectors, the endpoints controller creates +`Endpoints` records in the API, and modifies the DNS configuration to return A +records (addresses) which point directly to the `Pods` backing the `Service`. ### Without selectors From 72e24e2c379ae1e83bcfde39b4b91acc6a7f26d9 Mon Sep 17 00:00:00 2001 From: Jeff Lowdermilk Date: Thu, 9 Jun 2016 15:08:48 -0700 Subject: [PATCH 076/348] Fix typo in multiple-zones.md MULTIZONE=1 should be MULTIZONE=true --- docs/admin/multiple-zones.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/admin/multiple-zones.md b/docs/admin/multiple-zones.md index e420bda304..f3378069d6 100644 --- a/docs/admin/multiple-zones.md +++ b/docs/admin/multiple-zones.md @@ -73,7 +73,7 @@ plane should follow the [high availability](/docs/admin/high-availability) instr We're now going to walk through setting up and using a multi-zone cluster on both GCE & AWS. To do so, you bring up a full cluster -(specifying `MULTIZONE=1`), and then you add nodes in additional zones +(specifying `MULTIZONE=true`), and then you add nodes in additional zones by running `kube-up` again (specifying `KUBE_USE_EXISTING_MASTER=true`). ### Bringing up your cluster @@ -83,17 +83,17 @@ Create the cluster as normal, but pass MULTIZONE to tell the cluster to manage m GCE: ```shell -curl -sS https://get.k8s.io | MULTIZONE=1 KUBERNETES_PROVIDER=gce KUBE_GCE_ZONE=us-central1-a NUM_NODES=3 bash +curl -sS https://get.k8s.io | MULTIZONE=true KUBERNETES_PROVIDER=gce KUBE_GCE_ZONE=us-central1-a NUM_NODES=3 bash ``` AWS: ```shell -curl -sS https://get.k8s.io | MULTIZONE=1 KUBERNETES_PROVIDER=aws KUBE_AWS_ZONE=us-west-2a NUM_NODES=3 bash +curl -sS https://get.k8s.io | MULTIZONE=true KUBERNETES_PROVIDER=aws KUBE_AWS_ZONE=us-west-2a NUM_NODES=3 bash ``` This step brings up a cluster as normal, still running in a single zone -(but `MULTIZONE=1` has enabled multi-zone capabilities). +(but `MULTIZONE=true` has enabled multi-zone capabilities). ### Nodes are labeled @@ -124,14 +124,14 @@ created instead. GCE: ```shell -KUBE_USE_EXISTING_MASTER=true MULTIZONE=1 KUBERNETES_PROVIDER=gce KUBE_GCE_ZONE=us-central1-b NUM_NODES=3 kubernetes/cluster/kube-up.sh +KUBE_USE_EXISTING_MASTER=true MULTIZONE=true KUBERNETES_PROVIDER=gce KUBE_GCE_ZONE=us-central1-b NUM_NODES=3 kubernetes/cluster/kube-up.sh ``` On AWS we also need to specify the network CIDR for the additional subnet, along with the master internal IP address: ```shell -KUBE_USE_EXISTING_MASTER=true MULTIZONE=1 KUBERNETES_PROVIDER=aws KUBE_AWS_ZONE=us-west-2b NUM_NODES=3 KUBE_SUBNET_CIDR=172.20.1.0/24 MASTER_INTERNAL_IP=172.20.0.9 kubernetes/cluster/kube-up.sh +KUBE_USE_EXISTING_MASTER=true MULTIZONE=true KUBERNETES_PROVIDER=aws KUBE_AWS_ZONE=us-west-2b NUM_NODES=3 KUBE_SUBNET_CIDR=172.20.1.0/24 MASTER_INTERNAL_IP=172.20.0.9 kubernetes/cluster/kube-up.sh ``` @@ -235,13 +235,13 @@ across zones. First, let's launch more nodes in a third zone: GCE: ```shell -KUBE_USE_EXISTING_MASTER=true MULTIZONE=1 KUBERNETES_PROVIDER=gce KUBE_GCE_ZONE=us-central1-f NUM_NODES=3 kubernetes/cluster/kube-up.sh +KUBE_USE_EXISTING_MASTER=true MULTIZONE=true KUBERNETES_PROVIDER=gce KUBE_GCE_ZONE=us-central1-f NUM_NODES=3 kubernetes/cluster/kube-up.sh ``` AWS: ```shell -KUBE_USE_EXISTING_MASTER=true MULTIZONE=1 KUBERNETES_PROVIDER=aws KUBE_AWS_ZONE=us-west-2c NUM_NODES=3 KUBE_SUBNET_CIDR=172.20.2.0/24 MASTER_INTERNAL_IP=172.20.0.9 kubernetes/cluster/kube-up.sh +KUBE_USE_EXISTING_MASTER=true MULTIZONE=true KUBERNETES_PROVIDER=aws KUBE_AWS_ZONE=us-west-2c NUM_NODES=3 KUBE_SUBNET_CIDR=172.20.2.0/24 MASTER_INTERNAL_IP=172.20.0.9 kubernetes/cluster/kube-up.sh ``` Verify that you now have nodes in 3 zones: From 43705be9fbb2b997d7f759f3ceee36f630c58787 Mon Sep 17 00:00:00 2001 From: Justin Santa Barbara Date: Sun, 12 Jun 2016 14:56:04 -0400 Subject: [PATCH 077/348] HPA: cleanup some nits, based on a readthrough of the docs --- .../horizontal-pod-autoscaling/index.md | 41 ++++++++++--------- .../horizontal-pod-autoscaling/walkthrough.md | 39 +++++++++--------- 2 files changed, 40 insertions(+), 40 deletions(-) diff --git a/docs/user-guide/horizontal-pod-autoscaling/index.md b/docs/user-guide/horizontal-pod-autoscaling/index.md index dabcb28daa..c13be2d63d 100644 --- a/docs/user-guide/horizontal-pod-autoscaling/index.md +++ b/docs/user-guide/horizontal-pod-autoscaling/index.md @@ -1,20 +1,21 @@ --- --- -This document describes the current state of Horizontal Pod Autoscaler in Kubernetes. +This document describes the current state of Horizontal Pod Autoscaling in Kubernetes. -## What is Horizontal Pod Autoscaler? -Horizontal pod autoscaling allows to automatically scale the number of pods +## What is Horizontal Pod Autoscaling? + +With Horizontal Pod Autoscaling, Kubernetes automatically scales the number of pods in a replication controller, deployment or replica set based on observed CPU utilization. -The autoscaler is implemented as a Kubernetes API resource and a controller. -The resource describes behavior of the controller. +The Horizontal Pod Autoscaler is implemented as a Kubernetes API resource and a controller. +The resource determines the behavior of the controller. The controller periodically adjusts the number of replicas in a replication controller or deployment to match the observed average CPU utilization to the target specified by user. -## How does Horizontal Pod Autoscaler work? +## How does the Horizontal Pod Autoscaler work? ![Horizontal Pod Autoscaler diagram](/images/docs/horizontal-pod-autoscaler.svg) @@ -29,34 +30,34 @@ Please note that if some of the pod's containers do not have CPU request set, CPU utilization for the pod will not be defined and the autoscaler will not take any action. Further details of the autoscaling algorithm are given [here](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/horizontal-pod-autoscaler.md#autoscaling-algorithm). -Autoscaler uses heapster to collect CPU utilization. +The autoscaler uses heapster to collect CPU utilization. Therefore, it is required to deploy heapster monitoring in your cluster for autoscaling to work. -Autoscaler accesses corresponding replication controller, deployment or replica set by scale sub-resource. +The autoscaler accesses corresponding replication controller, deployment or replica set by scale sub-resource. Scale is an interface which allows to dynamically set the number of replicas and to learn the current state of them. More details on scale sub-resource can be found [here](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/horizontal-pod-autoscaler.md#scale-subresource). ## API Object -Horizontal pod autoscaler is a top-level resource in the Kubernetes REST API. +Horizontal Pod Autoscaler is a top-level resource in the Kubernetes REST API. In Kubernetes 1.2 HPA was graduated from beta to stable (more details about [api versioning](/docs/api/#api-versioning)) with compatibility between versions. -The stable version is available in `autoscaling/v1` api group whereas the beta vesion is available in `extensions/v1beta1` api group as before. -The transition plan is to depracate beta version of HPA in Kubernetes 1.3 and get it rid off completely in Kubernetes 1.4. +The stable version is available in the `autoscaling/v1` api group whereas the beta vesion is available in the `extensions/v1beta1` api group as before. +The transition plan is to deprecate beta version of HPA in Kubernetes 1.3, and get it rid off completely in Kubernetes 1.4. -**Warning!** Please have in mind that all Kubernetes components still use HPA in version `extensions/v1beta1` in Kubernetes 1.2. +**Warning!** Please have in mind that all Kubernetes components still use HPA in `extensions/v1beta1` in Kubernetes 1.2. More details about the API object can be found at [HorizontalPodAutoscaler Object](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/horizontal-pod-autoscaler.md#horizontalpodautoscaler-object). -## Support for horizontal pod autoscaler in kubectl +## Support for Horizontal Pod Autoscaler in kubectl -Horizontal pod autoscaler, like every API resource, is supported in a standard way by `kubectl`. +Horizontal Pod Autoscaler, like every API resource, is supported in a standard way by `kubectl`. We can create a new autoscaler using `kubectl create` command. We can list autoscalers by `kubectl get hpa` and get detailed description by `kubectl describe hpa`. Finally, we can delete an autoscaler using `kubectl delete hpa`. -In addition, there is a special `kubectl autoscale` command that allows for easy creation of horizontal pod autoscaler. +In addition, there is a special `kubectl autoscale` command for easy creation of a Horizontal Pod Autoscaler. For instance, executing `kubectl autoscale rc foo --min=2 --max=5 --cpu-percent=80` will create an autoscaler for replication controller *foo*, with target CPU utilization set to `80%` and the number of replicas between 2 and 5. @@ -67,17 +68,17 @@ The detailed documentation of `kubectl autoscale` can be found [here](/docs/user Currently in Kubernetes, it is possible to perform a rolling update by managing replication controllers directly, or by using the deployment object, which manages the underlying replication controllers for you. -Horizontal pod autoscaler only supports the latter approach: the horizontal pod autoscaler is bound to the deployment object, +Horizontal Pod Autoscaler only supports the latter approach: the Horizontal Pod Autoscaler is bound to the deployment object, it sets the size for the deployment object, and the deployment is responsible for setting sizes of underlying replication controllers. -Horizontal pod autoscaler does not work with rolling update using direct manipulation of replication controllers, -i.e. you cannot bind a horizontal pod autoscaler to a replication controller and do rolling update (e.g. using `kubectl rolling-update`). +Horizontal Pod Autoscaler does not work with rolling update using direct manipulation of replication controllers, +i.e. you cannot bind a Horizontal Pod Autoscaler to a replication controller and do rolling update (e.g. using `kubectl rolling-update`). The reason this doesn't work is that when rolling update creates a new replication controller, -the horizontal pod autoscaler will not be bound to the new replication controller. +the Horizontal Pod Autoscaler will not be bound to the new replication controller. ## Further reading * Design documentation: [Horizontal Pod Autoscaling](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/horizontal-pod-autoscaler.md). -* Manual of autoscale command in kubectl: [kubectl autoscale](/docs/user-guide/kubectl/kubectl_autoscale). +* kubectl autoscale command: [kubectl autoscale](/docs/user-guide/kubectl/kubectl_autoscale). * Usage example of [Horizontal Pod Autoscaler](/docs/user-guide/horizontal-pod-autoscaling/walkthrough/). diff --git a/docs/user-guide/horizontal-pod-autoscaling/walkthrough.md b/docs/user-guide/horizontal-pod-autoscaling/walkthrough.md index 7089c53aec..ac67ca9a3c 100644 --- a/docs/user-guide/horizontal-pod-autoscaling/walkthrough.md +++ b/docs/user-guide/horizontal-pod-autoscaling/walkthrough.md @@ -1,25 +1,25 @@ --- --- -Horizontal pod autoscaling allows to automatically scale the number of pods +Horizontal Pod Autoscaling automatically scales the number of pods in a replication controller, deployment or replica set based on observed CPU utilization. In the future also other metrics will be supported. -In this document we explain how this feature works by walking you through an example of enabling horizontal pod autoscaling for the php-apache server. +In this document we explain how this feature works by walking you through an example of enabling Horizontal Pod Autoscaling for the php-apache server. ## Prerequisites -This example requires a running Kubernetes cluster and kubectl in the version at least 1.2. +This example requires a running Kubernetes cluster and kubectl, version 1.2 or later. [Heapster](https://github.com/kubernetes/heapster) monitoring needs to be deployed in the cluster -as horizontal pod autoscaler uses it to collect metrics +as Horizontal Pod Autoscaler uses it to collect metrics (if you followed [getting started on GCE guide](/docs/getting-started-guides/gce), heapster monitoring will be turned-on by default). ## Step One: Run & expose php-apache server -To demonstrate horizontal pod autoscaler we will use a custom docker image based on php-apache server. +To demonstrate Horizontal Pod Autoscaler we will use a custom docker image based on the php-apache image. The image can be found [here](/docs/user-guide/horizontal-pod-autoscaling/image). -It defines [index.php](/docs/user-guide/horizontal-pod-autoscaling/image/index.php) page which performs some CPU intensive computations. +It defines an [index.php](/docs/user-guide/horizontal-pod-autoscaling/image/index.php) page which performs some CPU intensive computations. First, we will start a deployment running the image and expose it as a service: @@ -29,13 +29,13 @@ service "php-apache" created deployment "php-apache" created ``` -## Step Two: Create horizontal pod autoscaler +## Step Two: Create Horizontal Pod Autoscaler Now that the server is running, we will create the autoscaler using [kubectl autoscale](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/user-guide/kubectl/kubectl_autoscale.md). -The following command will create a horizontal pod autoscaler that maintains between 1 and 10 replicas of the Pods +The following command will create a Horizontal Pod Autoscaler that maintains between 1 and 10 replicas of the Pods controlled by the php-apache deployment we created in the first step of these instructions. -Roughly speaking, the horizontal autoscaler will increase and decrease the number of replicas +Roughly speaking, HPA will increase and decrease the number of replicas (via the deployment) to maintain an average CPU utilization across all Pods of 50% (since each pod requests 200 milli-cores by [kubectl run](#kubectl-run), this means average CPU usage of 100 milli-cores). See [here](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/horizontal-pod-autoscaler.md#autoscaling-algorithm) for more details on the algorithm. @@ -59,8 +59,8 @@ Please note that the current CPU consumption is 0% as we are not sending any req ## Step Three: Increase load -Now, we will see how the autoscaler reacts on the increased load on the server. -We will start a container with `busybox` image and an infinite loop of queries to our server inside (please run it in a different terminal): +Now, we will see how the autoscaler reacts to increased load. +We will start a container, and send an infinite loop of queries to the php-apache service (please run it in a different terminal): ```shell $ kubectl run -i --tty load-generator --image=busybox /bin/sh @@ -70,7 +70,7 @@ Hit enter for command prompt $ while true; do wget -q -O- http://php-apache.default.svc.cluster.local; done ``` -We may examine, how CPU load was increased by executing (it usually takes 1 minute): +Within a minute or so, we should see the higher CPU load by executing: ```shell $ kubectl get hpa @@ -79,7 +79,7 @@ php-apache Deployment/php-apache/scale 50% 305% 1 10 ``` -In the case presented here, it bumped CPU consumption to 305% of the request. +Here, CPU consumption has increased to 305% of the request. As a result, the deployment was resized to 7 replicas: ```shell @@ -88,7 +88,7 @@ NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE php-apache 7 7 7 7 19m ``` -**Warning!** Sometimes it may take few steps to stabilize the number of replicas. +**Note** Sometimes it may take a few minutes to stabilize the number of replicas. Since the amount of load is not controlled in any way it may happen that the final number of replicas will differ from this example. @@ -96,11 +96,10 @@ differ from this example. We will finish our example by stopping the user load. -In the terminal where we created container with `busybox` image we will terminate -infinite ``while`` loop by sending `SIGINT` signal, -which can be done using ` + C` combination. +In the terminal where we created the container with `busybox` image, terminate +the load generation by typing ` + C`. -Then we will verify the result state: +Then we will verify the result state (after a minute or so): ```shell $ kubectl get hpa @@ -112,9 +111,9 @@ NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE php-apache 1 1 1 1 27m ``` -As we see, in the presented case CPU utilization dropped to 0, and the number of replicas dropped to 1. +Here CPU utilization dropped to 0, and so HPA autoscaled the number of replicas back down to 1. -**Warning!** Sometimes dropping number of replicas may take few steps. +**Note** autoscaling the replicas may take a few minutes. ## Appendix: Other possible scenarios From f8b3c2be01d6395b4572b4e2b7950cb7362e28a1 Mon Sep 17 00:00:00 2001 From: Eyal Levin Date: Mon, 13 Jun 2016 14:04:30 +0300 Subject: [PATCH 078/348] Remove redundant 'the' --- docs/user-guide/identifiers.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/user-guide/identifiers.md b/docs/user-guide/identifiers.md index b5fd025e29..5c540acedc 100644 --- a/docs/user-guide/identifiers.md +++ b/docs/user-guide/identifiers.md @@ -7,8 +7,8 @@ For non-unique user-provided attributes, Kubernetes provides [labels](/docs/user ## Names -Names are generally client-provided. Only one object of a given kind can have a given name at a time (i.e., they are spatially unique). But if you delete an object, you can make a new object with the same name. Names are the used to refer to an object in a resource URL, such as `/api/v1/pods/some-name`. By convention, the names of Kubernetes resources should be up to maximum length of 253 characters and consist of lower case alphanumeric characters, `-`, and `.`, but certain resources have more specific restrictions. See the [identifiers design doc](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/identifiers.md) for the precise syntax rules for names. +Names are generally client-provided. Only one object of a given kind can have a given name at a time (i.e., they are spatially unique). But if you delete an object, you can make a new object with the same name. Names are used to refer to an object in a resource URL, such as `/api/v1/pods/some-name`. By convention, the names of Kubernetes resources should be up to maximum length of 253 characters and consist of lower case alphanumeric characters, `-`, and `.`, but certain resources have more specific restrictions. See the [identifiers design doc](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/identifiers.md) for the precise syntax rules for names. ## UIDs -UID are generated by Kubernetes. Every object created over the whole lifetime of a Kubernetes cluster has a distinct UID (i.e., they are spatially and temporally unique). \ No newline at end of file +UID are generated by Kubernetes. Every object created over the whole lifetime of a Kubernetes cluster has a distinct UID (i.e., they are spatially and temporally unique). From fe21dfa5b307c9e3fd0424ae5c57eaeedf5cf71d Mon Sep 17 00:00:00 2001 From: Lee Calcote Date: Mon, 13 Jun 2016 08:33:31 -0500 Subject: [PATCH 079/348] Updated CNI spec reference CNI spec has moved from appc organization in github to containernetworking organization. --- docs/admin/network-plugins.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/admin/network-plugins.md b/docs/admin/network-plugins.md index 2b1adca8ec..f5e2d27192 100644 --- a/docs/admin/network-plugins.md +++ b/docs/admin/network-plugins.md @@ -31,7 +31,7 @@ Place plugins in `network-plugin-dir/plugin-name/plugin-name`, i.e if you have a ### CNI -The CNI plugin is selected by passing Kubelet the `--network-plugin=cni` command-line option. Kubelet reads the first CNI configuration file from `--network-plugin-dir` and uses the CNI configuration from that file to set up each pod's network. The CNI configuration file must match the [CNI specification](https://github.com/appc/cni/blob/master/SPEC.md), and any required CNI plugins referenced by the configuration must be present in `/opt/cni/bin`. +The CNI plugin is selected by passing Kubelet the `--network-plugin=cni` command-line option. Kubelet reads the first CNI configuration file from `--network-plugin-dir` and uses the CNI configuration from that file to set up each pod's network. The CNI configuration file must match the [CNI specification](https://github.com/containernetworking/cni/blob/master/SPEC.md), and any required CNI plugins referenced by the configuration must be present in `/opt/cni/bin`. ### kubenet From 22974547955024126af108423b9d650ef881072a Mon Sep 17 00:00:00 2001 From: Anhad Jai Singh Date: Wed, 15 Jun 2016 16:06:45 +0530 Subject: [PATCH 080/348] Fix broken link Linked to a non-existent page, fixed by linking to the correct relevant page. --- docs/user-guide/walkthrough/k8s201.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/walkthrough/k8s201.md b/docs/user-guide/walkthrough/k8s201.md index 04cb3c5c37..a2d8619a87 100644 --- a/docs/user-guide/walkthrough/k8s201.md +++ b/docs/user-guide/walkthrough/k8s201.md @@ -48,7 +48,7 @@ They are a core concept used by two additional Kubernetes building blocks: Deplo Now that you know how to make awesome, multi-container, labeled Pods and you want to use them to build an application, you might be tempted to just start building a whole bunch of individual Pods, but if you do that, a whole host of operational concerns pop up. For example: how will you scale the number of Pods up or down? How will you roll out a new release? -The answer to those questions and more is to use a [_Deployment_](/docs/user-guide/deployment/) to manage maintaining and updating your running _Pods_. +The answer to those questions and more is to use a [_Deployment_](/docs/user-guide/deployments/#what-is-a-deployment) to manage maintaining and updating your running _Pods_. A Deployment object defines a Pod creation template (a "cookie-cutter" if you will) and desired replica count. The Deployment uses a label selector to identify the Pods it manages, and will create or delete Pods as needed to meet the replica count. Deployments are also used to manage safely rolling out changes to your running Pods. From d7f1884266792c27afd6e97e56ab00a2c685c6e8 Mon Sep 17 00:00:00 2001 From: David Xia Date: Wed, 15 Jun 2016 16:03:29 -0400 Subject: [PATCH 081/348] Fix typo in connecting-applications.md --- docs/user-guide/connecting-applications.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/connecting-applications.md b/docs/user-guide/connecting-applications.md index 7cbe76ccbd..9854a05a16 100644 --- a/docs/user-guide/connecting-applications.md +++ b/docs/user-guide/connecting-applications.md @@ -105,7 +105,7 @@ KUBERNETES_SERVICE_PORT=443 KUBERNETES_SERVICE_PORT_HTTPS=443 ``` -Note there's no mention of your Service. This is because you created the replicas before the Service. Another disadvantage of doing this is that the scheduler might put both pods on the same machine, which will take your entire Service down if it dies. We can do this the right way by killing the 2 pods and waiting for the Deployment to recreate them. This time around the Service exists *before* the replicas. This will given you scheduler level Service spreading of your pods (provided all your nodes have equal capacity), as well as the right environment variables: +Note there's no mention of your Service. This is because you created the replicas before the Service. Another disadvantage of doing this is that the scheduler might put both pods on the same machine, which will take your entire Service down if it dies. We can do this the right way by killing the 2 pods and waiting for the Deployment to recreate them. This time around the Service exists *before* the replicas. This will give you scheduler-level Service spreading of your pods (provided all your nodes have equal capacity), as well as the right environment variables: ```shell $ kubectl scale deployment my-nginx --replicas=0; kubectl scale deployment my-nginx --replicas=2; From b73a097e4bd0d6f360fdb87bef1a6a03aec33a15 Mon Sep 17 00:00:00 2001 From: Gerti Poppel Date: Fri, 3 Jun 2016 12:56:04 +0200 Subject: [PATCH 082/348] Update dashboard documentaion. --- docs/user-guide/ui-access.md | 32 ---- docs/user-guide/ui.md | 200 +++++++++++++++------ images/docs/ui-dashboard-cards-menu.png | Bin 40191 -> 0 bytes images/docs/ui-dashboard-deploy-file.png | Bin 26652 -> 25598 bytes images/docs/ui-dashboard-deploy-more.png | Bin 77575 -> 79309 bytes images/docs/ui-dashboard-deploy-simple.png | Bin 57589 -> 51962 bytes images/docs/ui-dashboard-detailsview.png | Bin 0 -> 46116 bytes images/docs/ui-dashboard-namespace.png | Bin 0 -> 7282 bytes images/docs/ui-dashboard-rcs-detail.png | Bin 67354 -> 0 bytes images/docs/ui-dashboard-rcs.png | Bin 68587 -> 0 bytes images/docs/ui-dashboard-workloadview.png | Bin 0 -> 55575 bytes 11 files changed, 141 insertions(+), 91 deletions(-) delete mode 100644 docs/user-guide/ui-access.md delete mode 100755 images/docs/ui-dashboard-cards-menu.png mode change 100755 => 100644 images/docs/ui-dashboard-deploy-file.png mode change 100755 => 100644 images/docs/ui-dashboard-deploy-more.png mode change 100755 => 100644 images/docs/ui-dashboard-deploy-simple.png create mode 100644 images/docs/ui-dashboard-detailsview.png create mode 100644 images/docs/ui-dashboard-namespace.png delete mode 100755 images/docs/ui-dashboard-rcs-detail.png delete mode 100755 images/docs/ui-dashboard-rcs.png create mode 100644 images/docs/ui-dashboard-workloadview.png diff --git a/docs/user-guide/ui-access.md b/docs/user-guide/ui-access.md deleted file mode 100644 index 04faeffd7a..0000000000 --- a/docs/user-guide/ui-access.md +++ /dev/null @@ -1,32 +0,0 @@ ---- ---- - -By default, the Kubernetes Dashboard is deployed as a cluster addon. For 1.2 clusters, it is enabled by default. - -If you want to manually install it, visit - -`https:///ui`, which redirects to -`https:///api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard`. - -If you find that you're not able to access the Dashboard, it may be because the -`kubernetes-dashboard` service has not been started on your cluster. In that case, -you can start it manually as follows: - -```shell -kubectl create -f cluster/addons/dashboard/dashboard-controller.yaml --namespace=kube-system -kubectl create -f cluster/addons/dashboard/dashboard-service.yaml --namespace=kube-system -``` - -Normally, this should be taken care of automatically by the -[`kube-addons.sh`](http://releases.k8s.io/{{page.githubbranch}}/cluster/saltbase/salt/kube-addons/kube-addons.sh) -script that runs on the master. Release notes and development versions of the Dashboard can be -found at https://github.com/kubernetes/dashboard/releases. - -## Walkthrough - -For information on how to use the Dashboard, take the [Dashboard tour](/docs/user-guide/ui/). - -## More Information - -For more information, see the -[Kubernetes Dashboard repository](https://github.com/kubernetes/dashboard). diff --git a/docs/user-guide/ui.md b/docs/user-guide/ui.md index f8a79057d2..56b0b68eb3 100644 --- a/docs/user-guide/ui.md +++ b/docs/user-guide/ui.md @@ -2,48 +2,82 @@ --- -Kubernetes has a web-based user interface that allows you to deploy containerized -applications to a Kubernetes cluster, troubleshoot them, and manage the cluster itself. +Dashboard (the web-based user interface of Kubernetes) allows you to deploy containerized applications to a Kubernetes cluster, troubleshoot them, and manage the cluster and its resources itself. You can use it for getting an overview of applications running on the cluster, as well as for creating or modifying individual Kubernetes resources and workloads, such as Daemon sets, Pet sets, Replica sets, Jobs, Replication controllers and corresponding Services, or Pods. -By default, the Kubernetes Dashboard is deployed as a cluster addon. It is enabled by default in Kubernetes 1.2 clusters. Click [here](/docs/user-guide/ui-access/) to learn more about the Dashboard access. - -## Using the Dashboard - -The Dashboard can be used to get an overview of applications running on the cluster, and to provide information on any errors that have occurred. You can also inspect your replication controllers and corresponding services, change the number of replicated Pods, and deploy new applications using a deploy wizard. +Dashboard also provides information on the state of Pods, Replication controllers, etc. and on any errors that might have occurred. You can inspect and manage the Kubernetes resources, as well as your deployed containerized applications. You can also change the number of replicated Pods, delete Pods, and deploy new applications using a deploy wizard. -When accessing the Dashboard on an empty cluster for the first time, the Welcome page is displayed. This page contains a link to this document as well as a button to deploy your first application. In addition, you can view which system applications are running by default in the `kube-system` [namespace](/docs/admin/namespaces/) of your cluster, for example monitoring applications such as Heapster. +By default, Dashboard is installed as a cluster addon. It is enabled by default as of Kubernetes 1.2 clusters. + +* TOC +{:toc} + +## Dashboard access + +Navigate in your Browser to the following URL: +``` +https:///ui +``` +This redirects to the following URL: +``` +https:///api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard +``` +The Dashboard UI lives in the `kube-system` [namespace](/docs/admin/namespaces/), but shows all resources from all namespaces in your environment. + +If you find that you are not able to access Dashboard, you can install and open the latest stable release by running the following command: + +``` +kubectl create -f https://rawgit.com/kubernetes/dashboard/master/src/deploy/kubernetes-dashboard.yaml +``` + +Then, navigate to + +``` +https:///ui +``` + +In case you have to provide a password, use the following command to find it out: + +``` +kubectl config view +``` + +## Welcome page + +When accessing Dashboard on an empty cluster for the first time, the Welcome page is displayed. This page contains a link to this document as well as a button to deploy your first application. In addition, you can view which system applications are running by **default** in the `kube-system` [namespace](/docs/admin/namespaces/) of your cluster, for example monitoring applications such as Heapster. ![Kubernetes Dashboard welcome page](/images/docs/ui-dashboard-zerostate.png) -### Deploying applications +## Deploying containerized applications -The Dashboard lets you create and deploy a containerized application as a Replication Controller with a simple wizard: +Dashboard lets you create and deploy a containerized application as a Replication Controller and corresponding Service with a simple wizard. You can either manually specify application details, or upload a YAML or JSON file containing the required information. + +To access the deploy wizard from the Welcome page, click the respective button. To access the wizard at a later point in time, click the **DEPLOY APP** or **UPLOAD YAML** link in the upper right corner of any page listing workloads. + +![Deploy wizard](/images/docs/ui-dashboard-deploy-simple.png) + +### Specifying application details -![Kubernetes Dashboard deploy form](/images/docs/ui-dashboard-deploy-simple.png) - -#### Specifying application details - -The wizard expects that you provide the following information: +The deploy wizard expects that you provide the following information: - **App name** (mandatory): Name for your application. A [label](/docs/user-guide/labels/) with the name will be added to the Replication Controller and Service, if any, that will be deployed. - The application name must be unique within the selected Kubernetes [namespace](/docs/admin/namespaces/). It must start with a lowercase character, and contain only lowercase letters, numbers and dashes (-). It is limited to 24 characters. + The application name must be unique within the selected Kubernetes [namespace](/docs/admin/namespaces/). It must start and end with a lowercase character, and contain only lowercase letters, numbers and dashes (-). It is limited to 24 characters. Leading and trailing spaces are ignored. -- **Container image** (mandatory): The URL of a public Docker [container image](/docs/user-guide/images/) on any registry, or a private image (commonly hosted on the Google Container Registry or Docker Hub). +- **Container image** (mandatory): The URL of a public Docker [container image](/docs/user-guide/images/) on any registry, or a private image (commonly hosted on the Google Container Registry or Docker Hub). The container image specification must end with a colon. - **Number of pods** (mandatory): The target number of Pods you want your application to be deployed in. The value must be a positive integer. A [Replication Controller](/docs/user-guide/replication-controller/) will be created to maintain the desired number of Pods across your cluster. -- **Ports** (optional): If your container listens on a port, you can provide a port and target port. The wizard will create a corresponding Kubernetes [Service](http://kubernetes.io/v1.1/docs/user-guide/services.html) which will route to your deployed Pods. Supported protocols are TCP and UDP. In case you specify ports, the internal DNS name for this Service will be the value you specified as application name above. - - Be aware that if you specify ports, you need to provide both port and target port. - -- For some parts of your application (e.g. frontends), you can expose the Service onto an external, maybe public IP address by selecting the **Expose service externally** option. You may need to open up one or more ports to do so. Find more details [here](/docs/user-guide/services-firewalls/). +- **Service** (optional): For some parts of your application (e.g. frontends) you may want to expose a [Service](http://kubernetes.io/docs/user-guide/services/) onto an external, maybe public IP address outside of your cluster (external Service). For external Services, you may need to open up one or more ports to do so. Find more details [here](/docs/user-guide/services-firewalls/). + + Other Services that are only visible from inside the cluster are called internal Services. + + Irrespective of the Service type, if you choose to create a Service and your container listens on a port (incoming), you need to specify two ports. The Service will be created mapping the port (incoming) to the target port seen by the container. This Service will route to your deployed Pods. Supported protocols are TCP and UDP. The internal DNS name for this Service will be the value you specified as application name above. If needed, you can expand the **Advanced options** section where you can specify more settings: -![Kubernetes Dashboard deploy form advanced options](/images/docs/ui-dashboard-deploy-more.png) +![Deploy wizard advanced options](/images/docs/ui-dashboard-deploy-more.png) - **Description**: The text you enter here will be added as an [annotation](/docs/user-guide/annotations/) to the Replication Controller and displayed in the application's details. @@ -58,63 +92,111 @@ environment=pod track=stable ``` -- **Kubernetes namespace**: Kubernetes supports multiple virtual clusters backed by the same physical cluster. These virtual clusters are called [namespaces](/docs/admin/namespaces/). They let you partition resources into logically named groups. +- **Namespace**: Kubernetes supports multiple virtual clusters backed by the same physical cluster. These virtual clusters are called [namespaces](/docs/admin/namespaces/). They let you partition resources into logically named groups. - The Dashboard offers all available namespaces in a dropdown list and allows you to create a new namespace. The namespace name may contain alphanumeric characters and dashes (-). + Dashboard offers all available namespaces in a dropdown list, and allows you to create a new namespace. The namespace name may contain a maximum of 63 alphanumeric characters and dashes (-). -- **Image pull secrets**: In case the Docker container image is private, it may require [pull secret](/docs/user-guide/secrets/) credentials. + In case the creation of the namespace is successful, it is selected by default. If the creation fails, the first namespace is selected. - The Dashboard offers all available secrets in a dropdown list, and allows you to create a new secret. The secret name must follow the DNS domain name syntax, e.g. `new.image-pull.secret`. The content of a secret must be base24-encoded and specified in a [`.dockercfg`](/docs/user-guide/images/#specifying-imagepullsecrets-on-a-pod) file. +- **Image Pull Secret**: In case the specified Docker container image is private, it may require [pull secret](/docs/user-guide/secrets/) credentials. -- **CPU requirement** and **Memory requirement**: You can specify the minimum [resource limits](/docs/admin/limitrange/) for the container. By default, Pods run with unbounded CPU and memory limits. + Dashboard offers all available secrets in a dropdown list, and allows you to create a new secret. The secret name must follow the DNS domain name syntax, e.g. `new.image-pull.secret`. The content of a secret must be base24-encoded and specified in a [`.dockercfg`](/docs/user-guide/images/#specifying-imagepullsecrets-on-a-pod) file. The secret name may consist of a maximum of 253 characters. -- **Run command** and **Run command arguments**: By default, your containers run the selected Docker image's default [entrypoint command](/docs/user-guide/containers/#containers-and-commands). You can use the command options and arguments to override the default. + In case the creation of the image pull secret is successful, it is selected by default. If the creation fails, no secret is applied. + +- **CPU requirement (cores)** and **Memory requirement (MiB)**: You can specify the minimum [resource limits](/docs/admin/limitrange/) for the container. By default, Pods run with unbounded CPU and memory limits. + +- **Run command** and **Run command arguments**: By default, your containers run the specified Docker image's default [entrypoint command](/docs/user-guide/containers/#containers-and-commands). You can use the command options and arguments to override the default. - **Run as privileged**: This setting determines whether processes in [privileged containers](/docs/user-guide/pods/#privileged-mode-for-pod-containers) are equivalent to processes running as root on the host. Privileged containers can make use of capabilities like manipulating the network stack and accessing devices. -- **Environment variables**: Kubernetes exposes Services through [environment variables](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/design/expansion.md). You can compose environment variable or pass arguments to your commands using the values of environnment variables. They can be used in applications to find a Service. Environment variables are also useful for decreasing coupling and the use of workarounds. Values can reference other variables using the `$(VAR_NAME)` syntax. +- **Environment variables**: Kubernetes exposes Services through [environment variables](http://kubernetes.io/docs/user-guide/environment-guide/). You can compose environment variable or pass arguments to your commands using the values of environment variables. They can be used in applications to find a Service. Values can reference other variables using the `$(VAR_NAME)` syntax. -#### Uploading a YAML or JSON file +### Uploading a YAML or JSON file -Kubernetes supports declarative configuration. In this style, all configuration is stored in YAML or JSON configuration files using the Kubernetes' [API](http://kubernetes.io/v1.1/docs/api.html) resource schemas as the configuration schemas. +Kubernetes supports declarative configuration. In this style, all configuration is stored in YAML or JSON configuration files using the Kubernetes' [API](http://kubernetes.io/docs/api/) resource schemas as the configuration schemas. As an alternative to specifying application details in the deploy wizard, you can define your Replication Controllers and Services in YAML or JSON files, and upload the files to your Pods: -![Kubernetes Dashboard deploy from file upload](/images/docs/ui-dashboard-deploy-file.png) - -### Applications view +![Deploy wizard file upload](/images/docs/ui-dashboard-deploy-file.png) -As soon as applications are running on your cluster, the initial view of the Dashboard defaults to showing an overview of them, for example: - -![Kubernetes Dashboard applications view](/images/docs/ui-dashboard-rcs.png) - -Individual applications are shown as cards - where an application is defined as a Replication Controller and its corresponding Services. Each card shows the current number of running and desired replicas, along with errors reported by Kubernetes, if any. +## Managing resources -You can view application details (**View details**), make quick changes to the number of replicas (**Edit pod count**) or delete the application directly (**Delete**) from the menu in each card's corner: - -![Kubernetes Dashboard deploy form file upload](/images/docs/ui-dashboard-cards-menu.png) - -#### View details - -Selecting this option from the card menu will take you to the following page where you can view more information about the Pods that make up your application: - -![Kubernetes Dashboard application detail](/images/docs/ui-dashboard-rcs-detail.png) - -The **EVENTS** tab can be useful for debugging flapping applications. - -Clicking the plus sign in the right corner of the screen leads you back to the page for deploying a new application. +### List view -#### Edit pod count +As soon as applications are running on your cluster, Dashboard's initial view defaults to showing all resources available in all namespaces in a list view, for example: -If you choose to change the number of Pods, the respective Replication Controller will be updated to reflect the newly specified number. +![Workloads view](/images/docs/ui-dashboard-workloadview.png) -#### Delete +For every resource, the list view shows the following information: -Deleting a Replication Controller also deletes the Pods managed by it. It is currently not supported to leave the Pods running. +* Name of the resource +* All labels assigned to the resource +* Number of pods assigned to the resource +* Age, i.e. amount of time passed since the resource has been created +* Docker container image -You have the option to also delete Services related to the Replication Controller if the label selector targets only the Replication Controller to be deleted. +To filter the resources and only show those of a specific namespace, select it from the dropdown list in the right corner of the title bar: -## More Information +![Namespace selector](/images/docs/ui-dashboard-namespace.png) + +### Details view + +When clicking a resource, the details view is opened, for example: + +![Details view](/images/docs/ui-dashboard-detailsview.png) + +The **OVERVIEW** tab shows the actual resource details as well as the Pods the resource is running in. + +The **EVENTS** tab can be useful for debugging applications. + +To go back to the workloads overview, click the Kubernetes logo. + +### Workload categories + +Workloads are categorized as follows: + +* [Daemon Sets](http://kubernetes.io/docs/admin/daemons/) which ensure that all or some of the nodes in your cluster run a copy of a Pod. +* [Deployments](http://kubernetes.io/docs/user-guide/deployments/) which provide declarative updates for Pods and Replica Sets (the next-generation [Replication Controller](http://kubernetes.io/docs/user-guide/replication-controller/)) + The Details page for a Deployment lists resource details, as well as new and old Replica Sets. The resource details also include information on the [RollingUpdate](http://kubernetes.io/docs/user-guide/rolling-updates/) strategy, if any. +* [Pet Sets](http://kubernetes.io/docs/user-guide/load-balancer/) (nominal Services, also known as load-balanced Services) for legacy application support. +* [Replica Sets](http://kubernetes.io/docs/user-guide/replicasets/) for using label selectors. +* [Jobs](http://kubernetes.io/docs/user-guide/jobs/) for creating one or more Pods, ensuring that a specified number of them successfully terminate, and tracking the completions. +* [Replication Controllers](http://kubernetes.io/docs/user-guide/replication-controller/) +* [Pods](http://kubernetes.io/docs/user-guide/pods/) + +You can display the resources of a specific category in two ways: + +* Click the category name, e.g. **Deployments** +* Edit the Dashboard URL and add the name of a desired category. For example, to display the list of Replication Controllers, specify the following URL: + + ``` +http://:9090/#/replicationcontroller +``` + +### Actions + +Every list view offers an action menu to the right of the listed resources. The related details view provides the same actions as buttons in the upper right corner of the page. + +* **Edit** + + Opens a text editor so that you can instantly view or update the JSON or YAML file of the respective resource. + +* **Delete** + + After confirmation, deletes the respective resource. + + When deleting a Replication Controller, the Pods managed by it are also deleted. You have the option to also delete Services related to the Replication Controller. + +* **View details** + + For Replication Controllers only. Takes you to the details page where you can view more information about the Pods that make up your application. + +* **Scale** + + For Replication Controllers only. Changes the number of Pods your application runs in. The respective Replication Controller will be updated to reflect the newly specified number. Be aware that setting a high number of Pods may result in a decrease of performance of the cluster or Dashboard itself. + +## More information For more information, see the [Kubernetes Dashboard repository](https://github.com/kubernetes/dashboard). diff --git a/images/docs/ui-dashboard-cards-menu.png b/images/docs/ui-dashboard-cards-menu.png deleted file mode 100755 index eedabf87c4250e1d77dd8e11a0c1a3cfdca6955c..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 40191 zcmb5Wby!sI+BQr{NH-4Msnj4L-QA6VgfN64-6hftAuT0CgOqePNHcVIcbCLFy7#lc z@8^ErKi*>&2geNSj&bk&j}6=tMm5{e6M|hIUHPCyqu(% zrn}K$^M`o4NssM5UZ0u8PX4=cvf+a?i=Lm886V!fY-dWbDjpwqWyQ>JwX>N|y>Rft z!%PgfYMuzaj1lLCARE+e(9XO?18W3UPmq}i&}U}|P;H3U@i0$F&#EiXRV^rsiarkz z5W_S_G<-36+3>Wz6#6qX^5@{s>z{*uX~ODigbwRV%gv`<=h55s*G6YksoS^FNV5Ka zU*2eM#b42hBK&=gM#4$aB9{F9u_%HP+S{{fzX!OdSU9wpzb=3F0EMOVipqPL|C&z} zp#_NvP?#e3Upx4U_YJhMLIsca*Yyo#PW0-(mJr=WBC*;ae)jj>RPfS(H`C*3&lLV^ z={8x~R;ItV3Wvrq_EVPj|8@RBLx%s_D9O8=(@tY|p{MwY=H>_O59e8NtWTbgT5JEy zt?UShxZs!w5$`LSjkQ8c_Q&bX+LPn$GW!(~X7XN}kL5LgOMKK4x)UEk`PflkEd1#7 zYx&+-iCYeKzH0@lPt#7C|3Fz6cmEuV6<3o#&ad_>C_ZXAUJ|F6Km4MCPF z4apM^%)5LZEaX-}{bG_CH}%1m@jC=ZzV~Ak_Eh*k2fXV?b7lr)on9bGvAK-vzuHnE z)kVz@3U-#4DJ>V+?2H{^{HLH$NdxR8X$fdI*2dY(o`t=7JYoFdZ$00ZoftKfM(aAj zYGT%P_qUeXCh&Hofx-`C0EEa7F5iTyzk#|^DlTSgtqo+(iSuf>XFmS@p#NH@BN7z@ z85cYn5K3uUo$B|HX^iIh_;8D^n)82KMi;oZpqY>Qnag(c^Nq?+kyU4=s)fjmN*?c?+2 z?euR2>tKOWkPLYi<0bpyxr?(JC1!o$9T}5)sA71Jjc}`@o!77J=*P9M(O^gKiy7Re zzXbAb0nM2KP!UbjwG<-kH_Qsif~zKwEEqLnK!J86LqkU)KKh;a(uNEK%+6NPx{TCg zvQXJ-#eePy2LT~Htmn4%%SXtuK!Ly|ftYv+8ZO6Y2Ip{T^I}sQ8S-#%>4%j@KU#vq z=i(XrQoIdP>#UUg6;WKqe5Z36(dmnop>h%P`NXWQx6>lwt#*PJ9%wr$P5{d9OI{X+ zh4UquN$lQdD(Mw1I>Kk~4tIGc0V1sX`Dga`M~s85d#S=FphGJ;sNqvb=H6J1^5XT~ zwr^eDn1JX-oZCtU(o* z_vRL*e1xyB#paUCvhPYH2tHLhY<_XItsZ0L^vHg7XuTt$_jbYj5EFu>8jb|rP4H&Qm$#H z`(>16fZO>lrO{1CATb4tg3^-b9;PAZBke!^pDLjAk41#hT#cQu2)!hzxB)xqX;-Gbv-4SD;<>~)2z&jQK-s3fI{%S}tq8=v8Vi!F=vP1Z8QCG12V zE_I}8d>%gya%78rv6%Gpb2im^xZs}RwuDyN!xOhvRE_w8fx^V0%z@QUsBhK(OlJh*h3J3vAuny zi0pkfKM^uw0IZxp)SnjaSrMow7NldnTY9?H@pD_ccYDyGC%=xDcfC~3xv*O~4M3q5 z<8@)N-5&!;Wf%4ARZi}g7(5+iwmi0_&d-+_;t1Ur@+woK-p++{$6W#P^3^%w{+Wg^ z6u%)tLl5z(V=kltm``aC+Jk)eM^VW|Oy*q_#*H^OVZ0)@dcDrQ88UM02pQ8~{<*hEo&YRK+8UM51Q+Q+ z1%%&(0-;l{Z!pcXoKi6#7~`{5t_Z=NJ+wgUX>%oLU5)P@Pkhy5+xGtRB6Qfw^Zv)zxhy>~a)m+)izq~F^bY+q9q5krC)6H*#e!4Zf#rCP%hJMq_1O%Q;St3alULQPfz4!<=-`mmu;zO@gMtt+@?r3q>Ig1Pmg zAAwtCSqIzf?t8|amQ0M(V^ciUZec?7Jq|Ys?@{ZxUzH0JU_O2-Iitw(rSN`h%@9wf z=jGkVV~0^MA#33hYlGLJ(-hLrQn@qnf9ALVD4wdiD&1O!$lYnl^wMq6^-y|p*xGX3 z(}s@rl1JEMNaktkreQ0%)%W1~s67S3m=Jl;d`2EBZJUYevP}TXimLuGpRSG~F#TM@ zrPC~)wCwj#J;Kl{ys>rA3YUdXw3;yr>w5}`%(j*F)#_B5?)cd_ z)|VX3F7>>20n_4q$=OT;d+$=nY#LU0&g~a6{Upkq#5*#{TqV56Qv z1CrIOWRzGbJx@A_C;W~YRr0i3PF9MwX-c(oejLh7V|Y;i)8*PgXrL1J4|~Utw#WI@ zs2{>eHc--{i$9iVM>hj=5TU4gV^n~2E|ufaP#*gWc9(Ewtc|=qDTg#t+a0Zs2SHBd zYBCa2Hk4AdiFnP27z7hr>(_-YO7t`n&Td!*$oA&)ai3nMny1yB zjHU#?54$PT`i5z0scBe~O5f)bg?mzy zfyC3l6zQl;YLH##MYZI1+wspwUuZg6qVF!MG`~8>t1u4IHg?N+qAUx&3hckTtt38* zQQ}F11<2E1OLb3!4LhZWB~F{@!}M|$v};4r0QRQBoUOSYbjzidS$X)wx4{0#@F97x zk>2vojiRsVb)`B6lRSO}-<{da4{eAk@Amj@qQRBZ-g}qqg4M2AG}>ARyOb@r)JL`L zg#Q14r|%X}nd)!Gem(9KvNs*Cm)(~=DMW{>8Amy)0{S)8dZX~Q-{62fFDHoFrC$&Vq*`*Bg6nc!^~{&rc9xKpyj+A@9X%<81aT9EfJ&&{2}| zVRNxA!auDK4hJI2fsf;&O=3|HZP(Iaya zP`y`4+#PV`b+^f5UbU9lOo~Di^U^eu@~&ya?COWz9>=eQYnp$moho1nEt$EHSNOKg z3kyS&bFaf8>3Z?T{Nv(G1xRv=iAX63&^6d;aVm`}(nUyGvoP8Utt9h#5(^cN5|6*a zN~ixXu&j$SYO~aI+yik!2a74lETFq#)Cnl}LU0$bA8z>v`Ma(nTjsC}^G8GORFW4$ zt-v?=wI&k^MSBU3O;1#k4%GQvQQbc+=>MwG?!-lZs6Q>9`#l_)bdjX%6S#HxI9%71 zz>qm)6b8g0P36RTM7G`* ziiHsQ4D|ZE_cd!YBx@u9?__0r=&{0rV9VabXB@Ds=%j6L z?X$lv7Kz%Q!)zmOo8snjnN;As7qUeZ>&(~+9zi0taj~G8)UY;9~?^*pM>*g4qHJ|>eS}H#H zlp*)=bGI(L7X_#XRUao%s?RWAUnv2RExWk?GAh}3)@bhv$9{%n8a-N8D7*YwNR@SB zXr7$_Jc#vIjHC*n#*JB+@Vo9f(lq(@yu7hP4R&iI8J5&~407*;uZ&6OU-R1{96#<1 z=sIr2wO%qV?h1@(9qZPwjU^1c$DLSu^0{#1`aw19xfd&{m%pok4J(R0D}dV zv9p%KPwc`AjXiOAQ3MrDJ6ng`KFpZa;)|*h;XTw~WM3ufzq}fr5x&`Ougqs>FwCtv zpPj3$6@ohsP~$xY6h#8nCH{g#F>M^EJ80n6D;Yn(wSKn{RUkyWAk|HuR%)FC8Kt%C zFB^&R5A8|Kc$y?tAue;*L6xGl4r=J@A%)?X%1R~xGvF^7$i5>%qUtFh8!TJ8(exuQ zYu)}cy*_#fjcPoP&w9dO@Shu*sAJZ*V&!fV zs#!9Bsj0?05d(=0kNu|s_#;V+iqe4Pfx`9c{iV+9gWZ;rbT#}&eI$lq(i8p0K;hes zI7a2Po2%N^2NLYRg-#U^fO|9S)>7Hjbie%c%g^t$xpgc7)MQT2r4cdp^rAM=d;n~_ za66#CF?RG^rbLx@0_PGb@YwgdFxfbC9~tNm>4Yq|GHaPB zHG1T={A)?!_$g^|x1p!_u0iFu7_)*47&$~4g`e)vaUzrM*N9u)FDG@3SN|*8&~eBj zB*>fp!brwXXJ8YmWn-?9VTu^@Rl{62@vk$CLlE!Wh)FH*QDW z#KwY7TZ8DeOAC!-`(|hb*nb2=0G0q?HevoxuxX=0#qb@*J?210LguW}WkACS@fRvN zz2I$y%JfZpsTD*EWyL}q_yPmm{^J}w-0=DF0n?j;Cx&UgIe4HQn{AGrb5t`2It~H^ zPkq{&AACM{7YZn4hlU52xkwX8mtrO5rq-RGrBzKl_f&(C?f+w`qonXJ39EcZ_fU)5 z&XZpwH0)n;j7mUiO}dNa-T{$LHfYcp?qHhIvW2Y1ti9ItkMRo0&Xciyj~P>N^MK8RqS7cNGQnc^{imyj$jbNM=oVP z-EWKB_NioepY?4=d6yG6Pbb-AKK*Q!Qc|Uyc=pCb9uhC=PS*nbus%D5@Pu|0d5cwe5&Gir@LVftK^`ai82BTEF+C4CXHOdp7=t!?Qx9kpQ(9W8U97c{ zC{dHH^tew!_%1)G^(KJtf<@D&{%tVp=9^SCzj#WA99*8jCajM5KS@;-9ig1=C0jcA z=bWKcCYpc>+IJB>uw@ex2)(B%8yU(w0;&9C{ZIYX#GrhrO>9meq?1)d+82*#2tF!) zPaFF>&BYq4)0el~OAiVnH}f(TL&$gOd_ba_3kDNc$T_QO{P01d_`67YJZ8(W1Q9%o zsp;OCxsY|C(W_kcio7CA)*lvQuR-c4}EFLQ9?XU&MAY7Vj6ab z^8cH*D8hUqctP{kXYNl%+b>~&3R%n_;((*6$AI=Z0M-(Nz0625B&SS1BUQx(Qc+_G zV%@w@$KitQ^1aJDX3Ymg-|Wp?a-9Q#s-OH|+5?BBuQ0n;h3yMPDsl5mTLId?lo@r$ zyzwBT*f!k<1&m#v^q_e?VUBCa2%@@s!35NT^9^)>E=?qkKuz~A0g-f|#AyK-0{oSg zkNeA|(9B)|z(t&c6zk2fjME`c^-V~s26_0P1lQi~Y31>7gnaJtqxqX#m0E0>8j;h0 z!`3FZ#(N9Syt4!3fo>1)DNHJKs~yN-$~{~|^rRt@3D zS#P`@#sL4TDjle9i{CH&H&U#3kH0Dpu7Z4D7isBr*P{*^`z{PFm!F&n$4UOnWE{R^ zfNFBl5#ZCw!ll{gO$DCUCnp=fH5w)^HY~Sy3sW0>&mDot?CF&XsUY`#Oh*!6z!}ZB zq67Gy^eSyaU~H~n`tskkwu;>H;pFwrSM z-LuLc#O&qwe$_mksnk3zFJI!u`CWkn%qVIvCISGH0pnYk((?b7UQRTX{nCp7YihH4 zzFBoY;Yj;GOQjoX8OrY!Ms8*0BU`^n)Js#5n=5+BsGxve$hY;d5=uWF(e2s$;&x(C z(YZx+aEN{rFD*uYw59mDnLw0gdxKwf9Vjzhar%Wcq6Rxw>{D@gQ@u z+JqVN5I;Cw$XC$o3^hJ!t6DDI_#j#e6Qlww%a<=Sfp(Q?-_k@e_q=q888&G z;1i@64i1Nhh37Xq9)KXb5JosI(h0X4buwQ7a7Ebaaz; zF_2r^STogcFECi`^TL~fz`>k}8DJ3J6G!>0+W2vE`RR_-z`z=quDrd6M4AYw_+YzeUZ*#>R;tu}7Fx_qDRB*L`s$Q&%dv z?lez%X>gmD0J_hg_dAs=eymVZHFc$q8?xHe(!*<{leXcs1bm54a4dR4e#XZxZSlGk zGr1Zk$^e*1v@vqZ eK#bwtgwM?9Z2j-&*Me2SvUp?|2MwE_<=pkuBnMKb z5lWDZ=#EW|Ru08kkr>{qO>$O8=DFBvR)Z!EgOrT(rg@jJy~?WdK{+5>dVtE({Sr)= zBKLPTB#b9Nf5?5v{!!m;E5J;-xpmgxc19Jjip#;$y_uWqUFatf4LJpulGPo4Q1L8E z{Lue~6FcGg<~>*q$F1jUUV%`}PTE6kQ;|2L7qhquLxN-QmrEChi`z=;O>G~y5RJS~ z2~#EgEROsJWd{6=L||A#KAzo7&kGP0^_l|@)t_KK%|@0;tur2dU`lo(xCyT31FAua6O5s$^<-fUJ(^pCcc|KF(Pl?E zonPOpq5neHGtt&F+~@XJi+NX&a0`u9n}k4Tm#j!SZOyn5))@eqH-ks4*xsAB!pzOv zt&S`GiiddS}^(cWIhk?T!ZWB;^660qxs~NWtQ;2-+!| zozHtOm8}tMWj~;Nt4<%^-OQPrMg(^1_JkgV1%n^K!wLEOzfp+O1+ zRbumqDI3>mSNM9baQH&={T@vve`}L*wowD<4LyLCLSe_bBJ$JSH;eq;MuVJ3KAT51 zy)Dq32>?>zUIMgRZo#U(OFF>Fg3n=sj*s9mfdadT`$K~Kq|j5szT4r0bJntW zQ3=TVF}Cseel8`kc4+yB*Yu?^{;C{wh9Djf%=omyTdMgL7jkN=>}OS+3JrfwLPyRS zl!a-q1S&$KO0o8+S1rB56c=R-fmEPFLE(aKmV}d*AFIn!qCuLhKlD=>U*GUf(kHBL z3RxJ0N?+D~Af)=}eUT43m`CjT)RQ9m9t}OBkRRMKmv^4`tSVoE0Bqf3_7Pg(gx3^u z2h0shwbAI=g5YxFRWfroa`V5*G_2vp{_#b4(lDpa0s^fE+XjNZ6Z~-MOMbN^Xe*`n zk`1I0b{px09^tXwUq}8zE4*ho(_gLy!U!#poGzWcAE>!-f3z8moEBkfb+3;ZsU@;s zMD4GEaQY2s=@>o9?n6d4&Yx*nhWF%sqX+_dpG7UVob`)%-z)@&_ArjU8aJP33I;IF z3V$yDAkAz!Vs$Wm5dJ&nm`y_}R>#$`A!@=#LpkQ*+51dIDz{2B;5^Vp=|#V+8EWQ7 zZL2z@9_!1Az3EiDn$z~5S0=5yQ8VuA*?vpU3$Gg!@W9aewBG*Ev$L?PY8>!h&!i~@ zK0w1$E8SgRX^)emepDbFZ{tuCsGTGU?y=xjxHYZNWFaJuGX)r$$PRAkzD?$y&XQ%* zeNhdDROG7Vg2;}}Nv;|%834v{ZEL0RcwvdCfuR>W6-noAc~?gbc)d&e$C?1{r_*BB zTngy)rlp%4e9{65|vUN(Rt z+1Yq+thdgHO(?g&FL&V-Y}I@2!A;ok8b8o`PBV20at-po_zC#TVmXlJGj5WD>*TcZ zf@gjsI~R1JXPE(UFg-vZ9TN(WH+sz}PQrHo3uq)*`LN2+uWCQI^Xu4Yc~=p*H~ZN$ zqiXwv6EGxYR;})u*QKfDsqMUEKR*jo)O%cMpar{kpmoCpo84R757rdAjVRB#?UbW! zIf6>SEK+9IDpPbuv)LeL))wyTw_cle-1L)Ky~7JWa5P7(Dov3)>q67VV})6&;T9IG z7qszTwZUIb=J?)MoTN0@-P<_c;$tr%kY}ris^Uv6zQns2oEDBsiMlSR%i|skC75L` zllPI@ViSI%I{I;~%;N*q0kUj*MhjfV8o166C1X6Bkz#$k^{ab2TE5L?eOigFovsFX zxAP2m;$=Rou@Xf6DGUzfd1d~q)}5Ue{9uUMTV16%TZt2uVP{rqHKz^y>Tx=?0z88N z_KVBbt`jyR>%(LepdjVks%dp8I8kuU~DlGZmsa7{UBpqL>1((1W zz%m^SK^w(}_| z`phbHdCQnljy{m@nj)s^2Sru$ILxQ~3`~IAJRh0XuxnI(i~q5YQ0M;q=^@E?7n`E6 zG!^2YkkXo>=|j#jD}uVrAj4tpp@6r;3Io~0a0uR)H0&=xyC2Yx`vcJNtR!U!$qnq1 zi(6&Jr>qMPC9_@iXntHYTiIU7I5kIdYscw$Id^eOZO#-L(vZ`PzJ-cE;I_n`71Iw9A`JRByK4LaD*?&|1?_lM>mzRtOY;n1Yl+r-zug78JkA8y=gTKV_S^Ht2}ox`S) z!kE=;y3NWkW=A^~o;vgsUr?r+3qc9+Bq|tHr9T{un_p`B>?w-ZEVN~i_UrQKg;q&G zQO)uXh+psgWE&jO*Jv(zrZK>x$MfaOOWyD4VSDIZuBMvIfZmH(3e{>aVz4}G_?Q<0 zt{C?M#B{eKnhys+m-Pv{l;>dz=E{l)G*bo8m(#l-g%<;OL5}Aai`lnfp*PX*`eMubB z04tNcul{OA>da5*^};2Y0=m%9mb)#KuNIKkdHIi5t{&B-&`_JbM)imUy^lUr?Ce8~ z)!LB^;}QI`ic@}>(wzJ$dP1qVjEokQK@1wFZwSFf<~H-T%(YaRgJkGhNr~mfH6H7_ zj>A?iZ#atxz#1RO1;NLGAn(ILA(YalD{kY;X*wn){NBtGQy7Djf{_^CXP?Tc6c^es!V;h_6i$I=e!;cE_Eh# zRJy-@_pH}6k1yKPNZ4p3MgMLpr-NH=+2T3?@8mIQAPi>NThMu$)EE)G+1+<5{;U&! zl_K+9K-s1VMh#(PZWR`AITj8Yc5@9ZIA)g+VS0+&V=ZV#3cmh=#1QYV#!Og_h$GYj z@=|#pCx_&W{)>qtWSTNqGx-FMBm-1+?6XQd&n{B?5Miv!>L~i9O%ztLDwm(^=UNef z5q*wy-3xw~$e&tFlvaey`wBCPYsB5QE#@IMT0ZihgS3idk_ex zro^WNSbJgT3#??2htAL&8%ZN%)57Shh#y~9Y?uKu*Q#g3oiJvhZ&w}zxj7W#Fw;Ez z3&u)geEKtj5A}vE4hb1nbvt|tGG#=|!O%Ru=97yHg_K!AFcOT0rW>V) zF(C1f3Wb)YNF-?)e@F%Kd!F`bbzsy{)<$}W5^ThH0ttpElq5pBNlIHsXW4VV#JU<# zRd41(F&8wI1qSVw_c9#IAgoKnp6^f}sNB@PTjr8;3#dh5nz+pUk0>P`7JfMT(Er}_ zadA^tl>;Goe}8{_B$JZq>j^0#|i2?k(P)Y}-u>_7AkIS6%SgFi0%}M-! z4}Z#cmXS^UXC6{-JV{!zx&U!B%hd^Rwuhp}G$-sY8LrfT{PC5GX^b&%aGx2$n6>(U z%!b;>RIdac2d1Bk>qVg}2rhf{ZWyP47kY_B#vUAn1Q9x&OZ`XJ= zH>AaVxz8@#>mY9t<%spy+prpJvv+3{bR5C|o}YvyEza$r`KXj}***JWQ^IRJ+QrOK zpnj_lrR7GnFb@5@lT}O4N#S#a1ZDuE?#<;`mAYIcu9TTBh31r5%CJEM~! zEt5)|Kx4CeXK=$F0l1l|7_H^1Fh2w47MJ6Fzp%qZMH%x`r=3=8dQ^aSWeImPN;=2Q zyLl2`ZpU-g*9_-xZu;1lofqxMbd24`ea<-Lo6`5~_?+JzPae;?7~TySXK3Ufx2z7_ zNVA7{JWhIHFJ6*(moc`wzq@`rI&#HmKi~K(I2FsV)@!=amGX|}xu8dH*5a|}bmPZk z6dZ0gzx~$>w%3!5WvLnotUkAGM-q;AlgABrqvTsk6pi~a6Wa|EAy-Fd(Ll(d1RxdG z#fv2U&F5)AckDlQC6m?dB9p`!axg5b>z6=Rewz7S29U;UwVC-Y8}b&8n|yNyhy}KL zAWCmEyiqjEQK|cIJULB1)A*~6{HHFcin?6YxW($Y)KKSG=;Ka9@%5hUIz0E7uCFX9 z*$m}ClNBxTclny3?>dgxS@SJgzUv2OyNIUKu9V4fuahWbvpvgg_M$NfubMuCT6zv+ zpR4)&j}{p-+&@7$C9zqGxnyBXVp&04|NwFbG??YRx>5j2nS z;*Dq>7`{uT`;w#0YQ8I@jY}@F@BM{6rj1@=sw_FAt9zZr?KUPT1hmd)Hhs7*m5~2B z_pHSP1!Dkv;bZz)#YRsfwvaQCR)gI%ybG_x8dl$OMw8j`Q7VPH*453A`I`L zjnJOOxmu5GTfYzNLe3~_;V3xzpT$2X(Y6PWI%}%5?6SVU?IB(6U_X@s33-bwHxu_B zEjGsgJlKdqXf0s8OMcezrYes?*cDMcs%WSA=8D}^O&p=WP=n?ghX$L5u_U#o--dkY zWFuYn-FH-^s(c@Oa3ZJ2S=4?NrxlQ|w09vR-xg3nh1_eR)%1G9fcNdo<;-~g%5C7+ zSZO4rvU#7Cyf-cPqjL28n<5S{3Mps&45;1LiWcH}EENB^h8V3kPI7oLafXP(*2uuA zRACzRTmu+2*@=JG?owre~ z>xWCAXza>%B*TOqoJYk4Yjd$lnekDmPMQ9BiaBxjl?Lw~1I0isXoZ#jG+6|-qQJ+icT z9j?XRHXDR)+YGr1p|Ioh`emKobT0G1upE@vLo=0@eD8lNBR#xl%Ly=4^^ALLhllT_ zRCZ?jshmpn8Sy(#fNdy~V0t49+K*Cxuq9<`qp}~zn+W$D8((Jy=+1|ZIAcL)(UJOK zAFfvGK3+b;HA(4*QVGxgjlN$;Uy7=Tz%Z)H7h4Nttq3LgT8gS8#h1_d_Mv?a1D1jr z^YJlfPV3hh0T0C7k{?Ssg$U7Q$m%?ar!wkS^Y%HCDkh|Xrr?l9yP3xQy<)dx;dmmH zf+PAA%~k(HZSMF3O3CHJ7P0ukOps*M%#vZ9xFiixWK6?1E@Ohz9a;dj!EwhNq3$FC z`f44yP@_)${R&m4Orp@zf4tFQqrqUekRotgALxmJ^+lF5W$p!$vsJk?DTKIVw)t8> z7o!V*5murerFOCy$O3qDE}s)w#do_dyv9#xLfn!qTg}Y__7n7{(gheRb@k|*&6N%< zGOkrrgJ6037{xXRMwF5@)rti8B~rZ)s_CG63p)<Thh!syf97`(x~Ls4}gxk(C^M^gngROQ!jE#+z8s4tNfnn&D2XFQ3HPhS;r< z_x1Odf%oKOA9=Z&;&ey(0^sp3KgL{g`3^=Gb_D1 zWytyZZ-?_VPG`mKKU~7U-_U?+!uv7htrr?)RLVOrb0;Yd5~p4V`;@_vPcc_KHuHf# zA~;8^nu!reYyZ`gej`2K(TRUMyw`;uPNPPfV9JwW-U> zg+QQ_^=oz;@uRGk`27^leiz4!%MDo4%gtp+^MkAgxZkjOTh?h|*jN{+O%zwbzsXEM zymlJ-Luir0$LktwTIu)-ALw3cKctQtT!B7fP)4l5`o0U2CfO3Eh(3V)N)XzJ>1*{gisto$3so8N#VP z-{)50L=W|s(QsExKpZ(@?^ew zhen#+Na8!)1qSnbo<+@$)OL~W1Zif~e5elWnOe<mkr6Nnoo30`!w%j*ECXkmmMKtH2 zLZsxO(n}AAi@UeCq0+}UxTJLJC=xHNEX08?L^j`twD*}O-7n4?u?mIndl7(1UUFlG zT)P+$gD2?)XYStH6F5C=@!3N8{23xmQE!&d>+N%vD&4MrQr=%>82_ri_B&XuG%zp? zR3b%)Ag!hWL-lmgMFJ}ghk05?7i+n~2L|$vcd7-1k!8MrKSQzQtx9PRppKM|E9WXC zW2^05vGp=0YyNZ{0qj^~ppSZf>?UX6Qf4gRf}F228dxv<#{=7qhw~urh+SKxU%I=D zp_I=DLa}6De}X@zizQsQC*VdSxNd3Gbtucai)aIBG2xRk>&!$T zO|yWgRYPrafkZOOcbCL&Ww%#=hoc1LoozyL zNP~(GavQJ6gmCKD1QC9zY&1#+p#)u3a6(d;H}Jy4!C6q2l>H>MXKI~s+Tz~dG{?@t zvmfrL-s<7yH*5&!9G#@O&IGO%P`(@DJ9`+dM}(2{8w68ji?QGBnGBN- zVMgf=BwxWbBljM6L(!RCN_2olih#?WLbj2fFA*eHOVg5-#8Vr9vi_Ge_v>%1~i6)=)2Yi)u0v8$dc>zzxp?3Vx~voc9q=Nwok(^sp|znd_|Z|I&)l=f$o&;X3`jEhJ;Rs&gcCTe=P6y020;1q)X%$g`XoyG@%}$h zxyh%xz2|onxisQ42L_yQvi5x5iJH-LX!%!h%Pa78(*kfa!jlPOZ8u~!m9i6ov#4E} zpVt)l>m$yT^+S77b{4**85xNW&c1cF8KVXJLOAo6=c0l>ehrfjk=XIh;>Geo<6<#e z6echVF3k4H=OM3B?;hcYfxm4WBwz_t|tj1@ROj=QH*rHuC+6 z%zN-}mI>;m!04;8o<+(OqEiGN-m)q$-_?t!+q3KzTJ0#w@s2~?`%G1b1ukjufWH z!+{|i+%w6^DQkga;Ve^*s2P;+Ec=_<(*&-KV<8xMQm5V4ZBp+v^pWyqosC~UM=KXH zHTr%r;gMd=Mp^F6(0t?1CH$3?EFW|MAZxW(PnQG)lBVVD7JABkb5wS`=iH#~8?@E` zEU8J=WkA`U_kp8H=QvIJI>hf1br2?py;jJYR8Dl{rWn>+MHQvR5Q|GQy z8mxhs*CaobU4D29mIQqql+EV1co@@IWX_`>l)#CGi~f$^{E*qrza#XEd?hY}CEzf* z-t^N1Yod}#-h#$r_N`EL3aHEJFr{qb zCPzEbV%-AZ+nMPdNxF1{JFxQ7l5?o3zTKglAAlJJ+o@aNx{?YZZ2`GyG9kB6HIzMF zJaGKYo>)ArqR9d|cP7d8Ozrh9@q={>#L9|PjZqobYjKp>GGzOIAZ)1%e0lRJtWEb- zuThiuBJ4jnS4m#c(Q;cqBfr6azyTNiWcrcMVM2;ek&+=03iC{^&CMu^DVeuq=7T|( zhO$=D>^V8T*cM@CJajLFudEfK8i|q;U)JWtHJmIA|Azap|Bp}C+iKhlZVITl&_*C) z3=jHv5fWhSYsAj#uN6C3P|4Oq*y-Lb&bK8KWP#kvjNXi#5BmZK9=?y!^iPmePy>## zdSZJ!gl$Tk&pP3SAZLZHwUGyf5tNBnN>K%)^!;WU`YO;oVpoc-N{OI4zp|~lqGJp8 z(E@t0fcZz`37p+ed*G~0t7g@puY((m0I3(WhN`8VdVBDRK_3A|Wa*e{$Fndd!~CAA zSG(2yLZ&cpUB(U`R>{A^cZPih#ND8kb72{UWcE+~7uoU|{OSpGBS>W~A^vf%PI19t z;=K%wq&F?RMC#nQm$88mD8-yG%qpqL#@F15J7E0IUcK}}-J`h93;N5ErZJx{4%QS$ z>Ap=pN@bwzm%ZO}PZocME z6oZ$)PYCw3-H0duA?3<<@G&z(T+bN~*zw4s6=fUPR5=xE1?Hqv{x(aD1wU9AJ_Vx~ z_{eGwWNc8>RSP-*ktP_(#;htfn(E62Kr9`N4ZkgD{3U=CQ@J|E5aWOXQwDaT>peF1 zOjlRh^AoMa$}RDeae&milY~!_}SWqVx|Y)1-w& zPsL*rK3pILH4R!#qO)2oPJmOxv9FHIl>};4 zMyJT81QAv1Thjck{+BLf=&LuOJq$|8ktP$KnLYo9vA2$@>WjZc4HOVmKtLMlM&i&Z z-AH#HfkR4zlp-Am>5%l$-Cfes97;e!x}-xy(7TR&?|b8o-y3(_|Bl1W-fOM>*(>Ip zpZR%af7&uO*l1_Wp?ZMoo9|8Etp8T(gX8WpiZnRafCT3sm0)5jLHtESWxl5M65A#a zF*@tnYuaPaHFy_dSy6hlL+p=)4DfeC_>&8AI%FN?puCb@l*eY_g(s2@$g>xZTY%v5kThy zaff%jFi6S)wk7{g?HZygGgnSPTgxBS${Q%C0eN(r>oqRkKhbJ$^y$-Lkcxw23~&-A zX{cv&G!KnGJvieFKzsl&;cTk(_}@KOPq8uU;(_c~@#exLpEtyqkx}9^Qmb2Xha>C& zY9PazxB(6LMI_&|fvmRwL0It%o_}iN24qunNWv~kdH~vLQdrFazzF#pI6%dGvs(Fy z3Sg=_g&MwXs6pHnqrSo^Z<=u7#%2p#Gk{zi)z;PM-$C^>AhQ$@dLxe1ih!m2;$V^O zbAyKn_@S~>vL60}Lt<#c(0Zc)kUHZw>&UExBua5o? zRV<_$4vrP~#JoDq7xP^XdYbX}&o7Y1v~jvUv2?xf5nymXL!jed@&N}k2QP;Nhf1eV zs2(@oA)+R2vSE)hi={%baAxg4+O4 zY#@e< zpa-xB;4S&X(noof`2l=|dGOsKRTeqJCSc6xcXR3L*V+)pgONn&z%4u3=nE?=>yZX*^M{PL^gnT67z}?&G-=S1aAT9ZQNoN^jBV)o ztCP#$VQSj#pJR-sU(jU7j;_oHDYJpQebid?cQz`rW zxOJ7IiN!qheCX#@cA3I$@Gxtwzs8aRVlbln$1PM&E$Ti6N@BLe-2oX+tWY2%8 zr;adwEe%&?j6mSRokdMzmx$JLgG#PSo6_{YFpB3~41!_#P-}{+ zD<+-dUkv_y>5VLSAF7zxmzNP^iHM2wm*u^Zd;|3fR2&RJ6WExrro+Ax<8xdKEWLYo zZsK;ROes1k_9Ib?Utu!jMQbnapknj0Kgd!-;gB-O8NClWiR)dt>nmIFLTyEYQ2wTB zfEeS$7eZy^6uHd^8Vh#1!q;MW^77fuv~!{q6S3~GCkZrJQPFz1`#EvDBnmi7jPuzS z%~QsMCiscsPEkAyGREq(1b(^f!W^`s4?cls{YX85Z;M|1m`kHXg>yj z>4s6vEq*ig-afw_0f7Bv?3-2Xl`EeCrxuub^NX4T_`p}d&ypdq|KBXRh@5!&*T#&B zID(El*BZw9jY#HZ!X^^>;70AAh?=m)|34Ep^lh*dT~heyQ-egUG}XMRqM~adg%nR} zgME`X>G7`s3_d%}=D<1v5ay>oK_*V@iTVx$_lF~&ySlUdo}Myi|4QyQ$<(N6ACwcH zs@RPH(`Lr_?s}b&LPT}LEm?C1d5=`Jp}du{r5cv(^0!GUN(7hsYk`D>VAzx(`q`E7 z^0cyNUsPdVRh+Afde-d(2{PGcHVV_iMs@oRO(>{%_yLDyPq=Z>gs)c87=i>aNf zz)q~#b=0Bp#6`H1?Kwrf>Q*hS=oF*Ju{)Q7qQE{2nH??mX34+(Lwi#fESn^J`xU9y z>>K6#7SG;K=C~krjWX;v4(jSEj=%cgsBLH*?BmuxfhXi@QkSN2W^<4#Zs_kQ77`cX zrRuEUWfCC@Eev1k;|N5W5C^#mr;uJwdr_S&zmIIE%6tMkVhAxWaQ#J#O-%Hf&+AUl zFHPq36jdp*jcWdg~e*x=gwhV@ME}@Uu zqUH$}(=Q=&LoWk){NREl!X5?+KkC*OS* z)0t~?D{D+Duf<47lBj4$CrP7H;Ka@ia}DZK*IHD%mUNuQ$~scqxrT+%Qs*T1KsOLE z+9(v;FX`3C?s8jX)h|=;7`m`ix#vcjvh_$;4)Gcadj&zzz}t*0&u;&HEA)?6X^IT{ z$#fniBh;8x71*ZpxZ>AjWs=U>^#70wY=)uS2E z5%Ux36Gqtk`#kh*CfMmV~l(IRvCRUjQFQN;)=sdu$^ zXywIc#d#lSSBD$}^~UaOIOL^CXO~cE=+S&QhodF9dgYgu`faCRMnBCrFt+O?m19k>H@Elh1isq67-h`v{cz`A)ydl8MD89VkYH|1@GXTDAT%qpXW@gyE2DaP63G z_x1XEkS14(5*?IVjjcQ@Y*^0JH!szvv%_{NB|?APt@^E2TCD>`->`IG$87Y>*A?f= zKl`j!ah9!DBrD=YV(Kp(1{f*=i&8Wo^reqYL_^7xXM%lG6L9p6AeU3s#`^kh3Dv)a zw+qc(lZq?0;h8&wuyQ7gySiC;otbs<)3=vAsdgwwmlx**vLVf8+m#_g;pqtqZ)m|Btu${d?;4 z6w`nt^xe0qVtg!SsnFLSFV43o=`Jn~NYq5{I`?mHg!LS8@<;uIYHcoE5R>`5RHF8< z>NtzVm`r<-Hi?cTnu=461Vk};X4uw;H^WhN2wX}LE4)&x$;a>S4%$5%lYrE@@1z+c z`f_n!BYjSzT_6G7yz9+WkFnbc`g$XI_%|RQ{R%e=k6Ie`o(8H7xE<4qAD>f`w6YhJ!)lg-Eq5|9QyFIQPNaz{H1E&yZkyM>s|fPC z^gNZidHXp(j>mTpm1fIfi3nUOO7E-inHUdQkuMt~_NT~P6(p?9Egr_&cotcf%0`m3 zBT@4!i*ulgQG%;zNL-Ie?TGg%VKTD+9zOhRQo;VqgTl}cN~Hi!Ut{apI3296iJnwb zGP$FIwR^}>QU`eocaVB`M}Gb9q;z(DhRU~xNDKtmt&;td5o=a(czl|FqQ92(w$AtaY`XkCK%%&yAIzt2_#q zIs4X|-+Q)FXN+@E_xge8ufy_H3I}wV<69rsU8#eZIqM_Qc!f+6^H^4hW)qVd1=R%E zj_vn=Df5-i&ifQ=yo|Z~WyMEZ`-}K9u9T5n`%UMqyRQ%qME_0+XM*=z6EbYp0}Am; z&qP1#P$WQi@ZeM7T+8lWHmztmPj*fxy`q&zPMd>j$sVW=<5b6AG7<6X>S*b8ckQxP5kTw0vU89 zqxH9ks+FU#n(7Y~Me%;--Sw2qsCrIKJ_9Kc8+8TayiA1w9S(2ln(e9}nH`+Y@wXKGS-JB%{PZ<8*X0I|M+BEj^ zMq#7ft`~DcZfu4mA*)pV1sg+YHfCyxRM8e^LoIYiwL?=HXPcy>8mqoe2G22k@Wv@f zRl;;r?AXzJy=_CvX{zH&0u}zAtAe zSz8^7bbcME-JPDz6Wqm0tM%{5#e14mBFyQ|E zwXf&8N@!D85*9e3&7QWl)X(%9W816-T?+ChujTzTcT^HNGSq!66u)juzoA0&sk-el zD%)Cj^>eEb+M?;s`!zDlbv>3hbqcC*)7+0}+C`j6y-t=T1J0P28`TlV0~2mvSyttm z+I${4my+#TV?;`Wm|pD2*F31AXp52AX%nbw((>rjnM12E} z?ATOhRg3+2rdKZ6!N|^*lpf^j8}a>V;Yhqqtc>Es^)w*_t1w67_MEPds&P;x(T{vSaeKR`X}28&ghZ7_9V@E50)%1#$&{tfvU zk;4qJ(V?p)W3KCkR6~^g83*XpEzaJPcVNJ5JQT6tpN@3h6`ruVME5CL0 z&OQruryv#OB@H3*0|0z}+Pbf8&Ugb5 zNNzt|392_NDWXNfsjkKl)tyOMa zcwxXqBJ{3Dqc82B{?2LWcghrSWG52-2mU)98g&Roi)W&E0^+S4d|4zvlSsdN4Z{sa zoXxsPSf%`Pq%`qCu8sGp|40F=71u9{2ex`@st=2T>?{jcOgOU*8ROyN2^;bvxQv?{ z6DKEF8!=~le>mWr+3`lYtmyW~uRc_Ql?#fTiRDC>9P8*i0TN|Sewo}Ev6JVobJc$y z#ExK(@GgG(BZJR+xP_nEa_A$c206ZcjFPw`KYzZPrt*zqq|+QZ+O?P8#Y9N+bVmB9 z?w-n^1b5#|{(3^LlE#~{(Ihw7vELrfD3pn_0rulCVY=yg2n$JlJv^`c*w}Wwol|I= z?jFwNClYA7wqfes_^mU7rI}*^Ubb@U(U9;npI}C)e@~+ z_%^w6GgpiQ?J%%Csjr?nPae@$F;gFRVK+ETcSr(T%7Fu;w3F z%94=o^5Q~S0ft}<8Bk=f)gzlpz8}!l!I93Jov-bk5Z7{xkd9E7&AfqtnNF7|?BUka zNVWb{LRqniy}=nwz-j8KvpP)9`U4F61b^y$1c>AvgL0T12mjQ4tE`S~H5;4yM*%nu{P-i{wmy4YDGi~(VQdiuMuD81GwO`d;C^K#OW+GvS zOs>6#h#)xS-1c6{6%PnWW!Dc;#8t|QJ-A^Eg!vJ7r}EK$J?rk^xb&mkB>@}#>*U1n zjju(Qq)M&ZT$imuR{FCPXSH935I^5gX=4rEvxqt^mQ{c$sh;tf?>{a6`u$7us82fc z3ANVCpD$m}6rB~8vXhA1mz~Pl7Jj0d53<_M_$dw7!I+!xDyZ@>Nb!)6g&3?jd8OZ% zz5d}x1;B9~h*xCtSI0ut(>cjtH%@6NPQWTYe0b`GY`28lzYh_)U<=MzBO@Fozuwh7 zX7~{7llPelPaHp^rM)Jk+}G!1v+5TY4|5Qmlnt$R0Q!p&+sHfpzm1+USqzXi?=MGC zK2LBGrP7urFSl;7?(t4)u`OmkW;u}zzy51EHqJW1M?p{&BP@_`1`|lVkm~h?&Ak=XvtXvPam_WYx13GLbeent0;C&>3ggtM~~`9 zh^_6&{$3Bs5f6Lba2?Y>uzg6?{i;@32++G=O};G$JLPGI9C;IS)_9nfqdOg>w+Z{a z?uFsPe3rKu7X7FOJc^on{QZwZWr*T7WQJJ#`o8lo`C6v1AB;f)qvW939iL!i2Qj3H za~cr$$tQod3mpPh9TkOD%1@X)uIyTIfF_ZKq*bP5pGPlvaGAQXl~`#>fW#rHVlD09fFIXol{OwYv-?waw)e zm=l?=X-KB>XO29!y3W>k%9ZmGl}_RA8pr}?x%G0(S9nWAKGM)ZCoCl6IwPfusO_Y5 zgaM?e84@mYLK8xg?(m{Gk3aG+5f)4podI~s^{#V1RuwS_lda zacx!~8Y~A-l_#pjQ>I(nUzJh(+<$dxz*cCeT@KTH)SA+z@0ydp(}x_8*4laTRyNUO znr@u5;JROV5z zN+w6f0rmI%v~EULv-6sCoY!yVV@ue$*Iy9wYszn(m1i$LMGR08G}Fo(pUmFJ%4HcJ z2o(^$;v*tU$Y?59R1r*2G7d@J`9xPF@?>6YyZ|g7izPm(=h^SK3-^eX*NO4rQ>08|x=7C`pAN8Skp& z-Aj1T6@Hsbj4hjCy`B)h<67&d{OG`fAm_B#cvASSR*rGdXIV&5kmIW!@`SA&LkwBA zne3-yF-aHBCqa%S^O6P-1asB=i|4=J23mg)X<*+cI7H&w@I2u0vttq zymVX@?FF-2l$fS~%OdH7MuN8;3E87y9I+X*(vN-LCv%h3jR!WM)(?hKUaAS(VzSQQ z@Dw*Hdlj{sYd>;V5IN2efP2}KzopU6{E37)PWt%i{ZRfvEc))Fn*T~{V{+Orge@J( zy1}$h9M9j@1!*MFG?nm*7f?}5-Uf+KHi-$>rTGqyX^*S|Cce>&OxZfNxR8j*3_ZW1 z!CKa3kK)PaeAi*kL_+?2_y1W`prMx~sYRY96ASU3wsKhlw%*=B?c(SM*Gx&dkplm$6(b)+K8(bPjN^QbQ>+S#IQ>hI z5eWVtf{=G?Bsw}eRCp+TN2To@7GaG4oH>CcfFffk%IdTerFsN}FLbj~ zg?20!%tSsw8-fvuCVq>?9}z=irtqI&=aq0p3Xo_l+@#ku%MP)~n@Rzv%=wuaaiwkG z0*Vi(*$Ek2ApGAxnIKhtmqNSX|EQWUfEc%U9u4y?u_g^ZSv2K+PV`?_!+T9XdrlLf z60}}Inl2kqc^80-sr;hEn(LALZIOL_0}qgPB#LYW@X|;h=8tKq%Dve!a^`KO4%Aw@ zay`&W7;8)-+p%Y~IE_DwcBJkWu6Lq!~M_;R9ae;bhe2HX^$Oq+5Mo|B|jSUW}{(-gdTY|w51-Oi|?L&F}N@; zyK(t5Ud6nmNvyJdjI^Ga@E*HrNR{p+Vlg#| z)e6=z3ym15GP~CD@8_fR;|ikMfNRps$nY&+^|(`BzJ?c*bBTBRHT%a-N3{UD{JJUj z9O%)#82I5b!aDa(__i)}#Dro>;IG@+@nH;H0~4#9DAkcQAENS8Gkk^Y(kS@7+7PX| z3lYO&On^x)9+r8gX=3c0`q8u7Pj9!I1$1!bi~4FOT0#4zXDG@${MY5T>Yoj%3m0Nf zzM|BByme1=$?x}Gwbf{%LN4(4L|^)IUjMgOuuWioRng>f9GNZKKE3~0fWJDRh|TxB z{twby-Q&6MNvdMzb-Ib)(%Z}3*X#@FGj7AOIgz=|Vy?qw-9{JRlGL03^uNB&Z|^)H z6g$o^LO=Mk!S{D3ajA(u=x^U^(Ie5b)PA8KZ8GiIKRNHC>{EYefyKz++wIFys6E3! zM!n29SVHMb1ima4K{{Kgr&2e!;QbnV)>RK_0_*O-Od)J>Fa3O-+%4%zA_o1-8}x|} zpZVXXWG{64G#+sL>k|iQtr*9bF%3!UM{m_B$%`6kMRmSxMx+q?oFr8eM09VQep%%- zM2+QB)L7O>oVba#RwYMMz59UukS4z=t%mi{d-#q4;bT0GWjaO2*tyIzlwHWc+@}v4fyyt;1{;}BAp>y-y&F@2x ziq6IM+s<|T4%!Q`+Ws6ZUBJcuT(17z3piN#ok=+^Eqq$Ivc4#QyZPAMYOY9PPIMVQF{XOaI{NIO-&z`yN+7yBSJ-}se{eukjy_W z;xCtl+5X`!JEA|cRIQOPbdQe_y%Nk4?QCg}B|L4WzIRbcEHILsg}g0?MGmKCvUN@* zTNlfNBieH$%toA!L=ld_1tSn&BWkE@pjy%T0@L%MiW`&G2Zg6c{lHS3x~JpHmE=@= z^=$^ZU3Z>Oa_{*LD3$cXO3eEl7~J|@T1%w`zx7!qUAky)|1E@(JdHlmkwUIn-jIAT zwIr6u?s*#5{wuX!L|@ST)z5?ZH|kH1W~}^Ne#V3-=7#*e-e0;Jg6`-vc9+vT=f-*b zjdIgN=OZ1|K(20ioQn3|lDX3xV0WPO7tVgiuZun+(27T&BfC>U)gHpn zwM(6s|4I`pV#9Ek$+S4=HhsmE4bRZ4fY*V2zX)~cDzrV$PUk|8KwjF(3`uzj**YMKfpFg&u$Ms^Rnkqg!d`p>c6w=QQ z588kKq)d!^dy;wWcmDf0^~q1a^IcUB$H7X0seRj1AumcO%5K*`ob&s{0Rj9p^DY9b z%laS0;f}ynnO!N-Pwdop>XZdJw=ut7LGa)xsvjFGXc-g3h}_4ApEZ!XJSkUhXcmZ{ zFCE9iXJ~w{cU4F-O>(~}+KA-!$Sre$GdNup0RcCp|d+Vd>v+rNX= zJ0iWT0cZs9J``y5PwLtWP$~toDgPi{j{N;IcU078 z7p=Wo@*um6j>3P!!CratTXs)D;geerM>wocjKPY1+y0R> zO~?b3`FqU`e&|dKa%xXi-rdjl2xbKm&1d}n?|1?B!mpko5AKwEbEg{|>ofrfwM?9< zP-@mcKE(O3H91rn6#qXnH!4L`5fqeEH5ygS1P!218zApjb^ui@A}{_%A(^`xFih{? zh6m35EGmRDz>X#JIXG>Cv|E4yj!~k((HPK!F(DwLTqLeS19$%pngCF@{~xI*?U)Ww zQ2F-sqcoV8Cl+a7j}nHmSHFaUjh8?y_;n+&lmyo3v^W4eO9h)9GNw^lAVyM0P0;y> zBWVioXbs6{4QFK09hkT0Y5&vXJ{*~GBW6*$0|hkpFgQBy8e6V|(Fp#}8ArzI z5ZI$^qrn{MTSI0=@i3m=48Gw)?;I+WP`Tm?NTSX*Xo8&S4~((zkw!_CowOX&H_fZu zx6sZ2SST=K0XorMGuQhEFaAg4==C5Opc(+GRsvSn6gR4wpLKEHJ@lvZ7qPZw2OeTY zP(3|744?shM3$ZweNI^{eFrN2tRZe<8f#}gP|>Tg;is($D0Yzf=QN)IIOZRAptleF zD~TY;aBKHA=^d11`ZOajub7I6O`!9S#=~GF*h<08s2b&h{pkb(bz8-DjtBp?xG$;M|%1dy7xrQWstLkxTKC`*2Cf6akUuc zu|hWW?DpYZhHVX-<8y6a6d>!No!b-c1rv#U^(?lb8zQVP>Pj^V6GX7oH%1Da_?xOu z#uG%UcQ*9U34=Dw|K%;9FP}PApm$ zqfuM^9+g0IFQksuWZpw>&$OlPwIv;1yQhNmdF1gN?))xSTDwzdg zCv-EnEvC2h36>1La)aIa_s;t^Flq3F;VEj&$(Ph{7*_JUV0D$4!;cSTQkIe4TEFht z_wo}x3I4q?O=ny*xs56LXU+aMw;qkx_P|KWTnEv`Gq1)Jc+yA?-0)2%R@w`=@P|(icFYvy4G-!Du3P#S13MfhwGfYLmG!QJFMT%tHE>x(HHh-$h zV&)>fw!+OExpDDk_tcQB2$G@Jgn{W=VjkX`ZJ{Ig(pAk)S%t+|nDNr)MAv8inN(h- z(}@zB!D1(7(r8IIqPUE*t;8LFU0iox+@I0v1m?4TIG(5h*;fkN=Q=LNom41#^&?VF z#OW9A2iQCmb8?mXrSfg}Y&pX%bsD@@j8^nLl5~xSOoow{;WDA4LLJmKsQ2qqf_anuGCmHMHCov6VqJs`;?_^1G1X9{Uz@C~H95CeIOmt^GY5pr2V~oY4-smw zba9XvR5sx^PI>WV_??Ad5-7m|A8p>hi=+nAK8CwSH+Mt+4(F6MNxTqoAPQZ%nh z@d^Nnx+7xcWfK6KT&a7Cge`j9MI2dbOxYd*ibNKd$6gwkqYcSbn}XL84M;6)>%Iz9 zwobq^4t`|lHlU#y#x>|tHODnXz_(?^RqJbATcxtSl0mLt5$5$~laBexlq$p_iB9Hc zg@URtzna-9;dCNyFe z8=vRlxyGnBXH#HF`>0k)YFUp6!1>H~=vM;u`UiO?G~E1NyIpUP!@(G+X^U)5D5=8v zp@)F%ll5)f@Hnk}H# zXeIDTZp}Nden;pR^fHKO;cN1$#SIIbKH%f`nb0WSt?#plrBxBCDu`p=`(pM48Co(Z z4O|6%GlCq2F%xa5(mYC}sB5p2nB*-fZOk&REA`O^*SbrB92IP3sW&9}`&^xwESY;J z*PUn(8<4iD0%y_Pd-kG{L^RVQjYEe0y^GFw1)nTS1+t_goJ?BsxFvS6mVa=ZujVBU ztm8@BQCMr*Oc-LIC(THFt^GWDz7bYqjFGcEBnU6UeiST}#IsFk{4PM` zl`w|)il$l;f@;|~UMSY`fLc2mBu;!>U1!kb_hhfinT`8$tZ&P{KxKdhx$-%z8yX216h)_u@+d%s6R z?%6XCyMOd75>c^B0oIMa>`Yxn5Hf8z7@06l;)I!7lF$3*LhjyHZO!eos3m>6Af)8Z zM8fysxglgz#9hB#G_+UQ+xs!O#Y0!HL+k90YXUE3=4rr?1b8thOa4_NH^O32w_=RN z|1hk#tX11s4m^ts^s05v+*lR-c0fJ1v`-ghvqAx8lc=G=4Y<7NdzAU0Y495+)Gh-= zFxj93G#BNCf}&kt`2Akvv)FZ&A-qt-8b8KB8NKYYa-i%FiA=z~SHF3b2!PqC53LCw zE@S4F*?8BiFQeFPZ=+Ss#XBrOC~{S6(>>^+g#!iLfOI1DZH(>0ZV+#Iuu^QiJ%?a1 z<9T>w*?o{low9~a^bCxoKrDSGrvHrfZD>osZ} z+|^rX;G@embHO*_uUVo+7A4_W(iFu~f#;Zvu)=b%+ESA-A2>2W5)-S&4 zAE0n?83}&x;{dmpP-Qi4;na62Eh#Ryt>n&6;;pmRQ;)br7!}q&B@aLsp=f;7GgmiR zqz+w9?OS0vh%6m+Ya`z*&3{PMROFu3VaC3O#kJAdcwy8PG~85}zbuEGsXSW02fncX zIO}K>JG>Pr?txGvbESMN6g^>WVjI@GS9+p#4xlw)m{_K08UJ{Ho||UFEmz zpqZCN@_dbAA+TEo4tA32-?9^Y+^!F!=2~Ck-j=4pKaNTub+Li#xtVVEZ$+vHALq}t z=EdT3R=@2VSWhYxE9`M|NMKXq{g7|-yJ~-n@_alJHuZ4$xsdxD^(>=t)$@0a(pAyXrA3vU|F1CXytM^{%jNYE*VV`8iWeX zMp=%wg+6b^qO!;%P*OzY2M`{ET|XKKv$(McySpIF#P5JYW-JdFC~P$QCU`&e#E%*o zH|zC$bRQwoBd~an-(oKN(2Nc=S!;*BJ|R|2W)V&Ampi0#VYl= zW+c&!6Ud*TtGi1a(4YYMPT;Jbcc=MR$^^J+gbvu8wc^X~nzm-Yu}wAho1kwkk!o z;Ks;h^Is!#{6A)?joB5Q8{`v837Z~tcj+PT@>^Nq{}v2?FkYYNsqZ>9FR8vA7+mmN zMTj3N%v#^pfNsEJPNQJg3$vHnE`3~=9!VQNR}-aERU6ufyFPt%F<_>lY`;iazL3^6 z5~r_Pwzot@9wOPNLQeOkj5aTl)o4C@wA{A+EZfp=d%AFzKw!ANV0hJ~Xrs!16Bt%4 zk#QD^8ZHPX7zhR3UnwXtV^A`B=$kf5s#Yzq7JR zjIGfwEFq6#6hJ2^qLBbJSxO3XhGj3WNUnXC#@F?%N*P)&+>gZeEo4pt_@OwNYpQ}1 zB)-Y~(yJr!f#0zM2jdq%KlWZx2+{faVMv_OK>{MPp;=al7wt`(Rez>5&EJq!Ewi9z z0y`#F(?)ljSFIgf3$Ueg-zgoPE}R?~X{dZ0mM4^jsr3tsheLD4Y*)(Mq8)9RHjOS; zSL!3VwPB;)Y0g@eF@;mkssc~E&Al{>)o^kJHIuq3F{2NBMNwm&GJ&n94b6;(7@t^Z zWMzf4zPpYpNsd&=aZMdOa>FksM>2o?hCJ6hFCAXu=SjHr0){^X@9ZhSgJ<|5t?Sw6 zxE2wXa#ucOgz^LE7}>5)CZ{XbK3*Ks*{aL28Xx1)ppZs(5>g=*ok=ZK;>TRrQlzqr zY8sLxt1s@~at@fvB-2eR()|lw=-HFzE07%5TDy+Q2aS^kYMiRexKfZB36sylLWYX0 z?-2)&Y1Rffx+h_C%jg*O<%Rx6;MF!83XJXNXl&MsBAl!w&@by$BrggDtfM$ShAX5z znHoCPysJ_do99dq-kitmVq-jwUKS3Vxu8Kq;K@tq=hie6aJl4433L?uQE7BBc! zLqC_o_?z~2XdiPF@0^;lm40?emZ zm-$)?XiO zcP13FfU#?2XUYrGzz^;_(i)RPGP#a*Dtf=dffS*u7N`%rjuKKmlCg>@+*!?vjtR6L z(TsS02 zxrAxC_B86R^70k@flUfAO)U-l{`9ZOUn!zWA0uQE-rKU_xd>Og&p9$RL2)*#)C&R+ zG()Ed&ZK=FA1iIVr~av4Zk#=?aW>BSICZ5uS ze9{x4J2{y+-ITgx+a+O1qouW;rC?cp+2CkXMxRnfk-NgGFqyaSkU~>br3-A-xivXo z*O{pqCX|)LrK}&iyG|H1PiPzsOL%zo0|m@^#AEwS34RR3959H{0td5*T&IRoa=7VJYE61x^&9ip}C`ga8@^XV7bl55}VT>?Bm<^H^Ql3%Oy@ zxb-kaLT7msuTO8iaW!*Wkdp1fF;sJrlkcZ2g5B9>?oxIBBJGWC1;^zJ?H4u~|??I#5I}UVc%YishCI6{#xt`qI8`J=g$3 zlI+Qppq6T^TXbKMI>o>?nFg)q>^^CeLA~PB%D-slwq7CF$l{A!6Sc?L+BVA1ngzhS z?-sKtq^geonDIEwvB29eJ3rg^_1=``mmIj+)#P~}JhTx617d^FI8anuMS%j_quGSL{-r6AQ{BMPLiYPs_0m4i5P~ zw<>tQrb&ky#>UWRlhgTzZ9X#3x-v>LUs2K_s zC1I_+xI+U@D*Q&TAJu^Zgx}$OARF`nIJ4x1ii51w?kJ|$;B11U0ycRXr*ozn;NSsD z`^h{)!N^QNN=c;hG_97}{|Ke3lrDeBn*hLKL<|(zD45Qn&M--~BXhCh`~bvO`PLk; z2R4kRIXC|ujsi?3?j00qC!85%>t@KZ?Q#Ero)na*<>*wUUqI(cgd+Vx0m}b<*BF^7 z6f$m2Ggb(*o9BIb>aQ=OUf%k~(M<9n`<)0?MMLhNrb8Q0ywR!mqhmA7>ZPV3T~MsznLnS}=L-=TK~rZ* zt4TZivg|zQT|&=G$xAW1PI}IMl=v%)E(|`hr+|C!%)&9Ig!g!inwb!@JY~J-rKT>s z$f!Y$F7^BJ6Ou2t{QVa+vTYiTY_Aui4yLc}gagXMqoNh8B#{tjY=D_6;rcI1uTDoq z)QT|GuDo@-=BPuDKU3*ZwVzK}SbZIi<()Doh}f^K-L75mJBc+KAr_`~a@vwuL%$Yi;TNoPz~-L>ATs)_P^wJ@W^ zo!x0ofw|7(c<)PVj3rhr`J$1w?KTgEV!QnCvo^^(ZCNDchz9%6>Bno++$nP-TRj?6 z+Eh}L=sc4H_ukVod&@PAkwZL=YraU5_C*?OzN|8Um_LbWfPUzECbIi>)Ii z#nAxRie7$xr>%1hN+U}1W+;Qe`hROqii4_vD1^8h?#s=hW2i3_>mu!pZTIkl{k(K+ z*gJtAb0x1y-?kF>$6;jkXiGcUzb02Lfjm$ouhIOCWBH0ZtW=}1RzrWaKjRor?ae)d znN+Is($%AcF0X1&f(Mc?hozD^OdO#NHkty)_Srb+FL+S$Z({}sWceL!IyL_i?dqAT z=S|M{2EvGirV@PzmZuT0U1HcO_A}225LPxlmvG$GEnrfB4>uTn1`AWhx3c+7!=ffd zvfF3LTSQR!L*Dacm(&*r**#rpsZ|2*s(dE>!dmkP|TPLGQ$}3@aP++m<^Sfe3LC08FEhw_R;Vt@m#LTJs zOP12z#iV=G+96~M=n%(lIueWQo-Ui0@^?KZ$MTOe;s2|y>yBzN+xDUk0_u>2s(=Vc zk=|Pb8H9j{NH0>Q8JdE05D;W2fzTs{Hj>a10U;Qq1V%wZkz(i|Ac}NQs)cgrp3gh? zy;wKTszkN%~50X$W&jiH`BH|j2`%6wfOlMFF(O1b5hi)Yezb@vlHvs zl14H#o#G2Es8TgDw_ndhJn!!slGv?1@6C_8CZ|6;c3@5v#^&R@df-ZIScb>RyJIga zdfo;boYnHxYT?}A0j}_GX>~?me*a8;U`mrN0M<2!<4p)`kG+bKshBlcJT6)(D<(|- zsmG_$c7=(SLrmy6B4B&QcHO%47`zdaR59KV<>}$9>wfBzr;OX|(ia=nKM*EC-KkYW z3$5z}sjnr}hDps#(^~2ZO0`9-F_XEca+r1HrnIn z$K5^**R>q~37{$%1(_K9kVWlke~}yvJJrNvN36x6Ctt_}Cwq)INN)R^>oIJH=MjkHeDPro_fdV0mgEJ`Uq(w}rE-GIhEA%bi`!gS zwMr3WxFwzPG5cUq2~ZX-A|mxonI2D7wAZ@7!_IBx;vMlGhlcMpoUOsaRLYIr?>`Cz zymF2#%y&(za?6_m7fDkXM9|WfrKHj8@)uY$57uUtP=G#|R@b zX)+(z>7pY|1+jL8a+3tJvDfQ0ZL`-K_+XqL(aqd!zEBUfwcE@=j%rvU6pce`L!d&-76{L{pVgX#Q?lE*WSjAvZpG6Gp~xt$mc{TIz4<9*S5zhbJNo%L^uFGIfpRrwofLrMj|ug zl3=4AhFZV#RJ*6D@8#pqybk=t&zeMPAZWu>Q&Bn3D(`!9 zp-HHCrm=J$SSjaWy@dzom4^b(Gdk*)Z9%ys<qJ zL!u7uK*=CQ=<1t{xUU5$!ySpN$l?L&rMg6^*>_&g>7(IptHYI~SpECL(#a3%ALa)= za{cV?`v?b%#{h5Vr>nK;+#rM^$JluF*T1eac+V31G2mD_l?*QA89LjlU?tx=|27+~ zhnlMD=-V#Sj(2Vh=VTJZ$$8sa%wF(h`HdyL{#N6!^ShqAuK5`XA9wERD9^0nKeCk8 zvPakG#edR?Act)m$qC)H7_@+I7ue|+l~g}Sq5};)BB7}LLvRY}#xM1-+#w4IY7v95 z8B%4uLSV@HZ`&GE$I4~yr8O@I55l5GG`Y;x!J7`_pg3w_xzZsAZ-R1T1COEb$E<31?U~*$R0-k+}!&^*wNyG5Gly=tpIeh-e86L$|xyleT#*1rK8FqqP)xS z^;A#0F2*Er4h@vBGoR8lEu^dNgA<6 zxT_y_*Bopj$lNnGIZ#$5X33%|y2F$DUr$}lMeDVC45y|w=@Z>qnhiP@YW3E;C{ss8 zr9Ox&?FB_h#$^Dn)@6Z`I^ogVm>6kh3+o{jcqHfdJoD4t6%qU^CC(+~J)Ex+42cHH z5knS!SQ%0|%OPuf-yEJgl)&!PNHo!X)JOz2oFWvJ8%v=hj;@^!OfD2c-;E#-aPZiQ zHQgvd^VeQnT!)wnkPp;SSNIh$VhlLg$xs~Lj<84}e(Ixy8Lm0Hejt7F2TeMSz$0cB zt)VwIiG|yli6I!*aE7Cz<`duxmNjkvrU+-%HKq#0M9C z6-?|CkaJgo8xvCWPP1xxdF9woi(9p=NT-$P<)U{;FMqUkJj^;{k4)mmlW>$pgA3;G z1l5Eep>truib`%zgEc0{x!>pI-wAP4CfkJ@8ccCi38YE}m*;Se-g)J%wtH+#no%>) zywC zHL>PLiNDy7Rb;SpCz^>;U9!$;RD6OES=m~0+)iRg0w=xAX!XU!IJ3yT~KNMhkEbO4pjh$b7bb)orA82sxA=TlR z5s&pi@tE2ujR_$3U62-5dBs)$wCOhlZyb6mLiEp&XHNRrCqUiq(2i14=s5xH)F2oj zFWu=@A#@t+x$qJ;mFpjbu9eB-wUzAkk6M661?Y(7bz}jS4d80Hp_>!9AV<*4v4H4n z2GDq`W-;>V%^Kv7{))*$hmhD@$AxJ?%gWBwI-CjjT6bY?-srC0+`nyIV?SGz)5L~k za}q8*I2lx)?sM5N5tdC;Fcpb>{J`lU0$6>ti4SNrtkN&(OyXVxIe zpR|jw?!`;)iQgWc}7ie9Ke8LCxp%$+FhVQ+V;;hChpf_93dO{Q{d1lB~_axg^A9PzZ ze92b1HA{(r^furgP0M|<#!Z{Hme;%iztjB-s3@Qe(QUAmQr^6U}ELvGPzxv`Ln2UC_r0pc?fuflhD0I|{n;#Er28G8*&Y(K0DD zLQo*ai;x)D!+8is{X1M?dN5?L1D*neT3Y>I6x`wR>28a85VjI9@IY)b<(5l@V5O~r zC6yM%Ltz3BzGB^xK3hJl?^y>ry8!&@p}pfz?%Ekp7AFc$K#sdQpuD^X!~hIGfX$jA=Rk)gSE5Qz+QMGJD= z32`38zSLm;e{20Ok^})wUq8#%0Za-s^Z#NXfs6fG`)hS*`4;Gvh^4e6r&xMfQoA|v gCJR*Ec#tNQnpthzKYtErWnG2-4EsN=r%&%^0XiNq2Wk$1s2h(hMmv3?Lmd zbjNVV`aSP@pXXic-gVc#^9O6bXP>jrIs1EJe?EH;AsT85q{MfK0RRB0lA@dz0Dubt z0Dx}@uVG6nEz3Ls0LxA#IT;=AX%y_--5Bg!Fm7J0>ss30Js` z+Q>2VmbJEB9mnfi_kp+M=s$fv*gBS=nV(mNWZNnGrWdOlb}XB;5M;u?9s8YrH^(d! zJR!XQ?k4un(iB{{<@7u0pQm>>_fP~tmfwdc7lHd7O&6DQO=eMR8LR&#Gs6EuwJVGFcPL97 z)NRO*<^tqnyKkDF@!t8{$K9JiaghIcWoJbaq1l)E5W>H6=$>GMa9XG5)-nf%i1el7 z*$6W!PH6n?cFGp0bE;il>R4~O3*G6wjK_`w&-3)-{ID)B%U{K)1c2G;OGNn@I^S>A z8Bc=}1$&OS6Ls-)ftM!wuN99}090kYX!(Vj=mN^-i^Vx}r}N82d5n%^g+51Ic$9U{Lp2t6hWnvMfc!ezi$DUN^+sN#+M>~ zwjh_jB=eb?k^=J!=LX8ATm6=^tyBR6+ZR1$cZN$C%nL#5;utF%`~bAJM1FSIfdr<` z>~@`%N|#bgWa(c^q-z1k48?0nY++kzbLYGHJm5?DYo0P-vVfVA8`A>5fj`3-b-s?G zeb&xvm2~LXh*B0P8|A$6HDetrIXLBd z!x_SG*yQCqwuvz`l5g)WAZEcid55=yd}cVY)oAO|d%xx3Z%Zt3^i~A#N-O*b<3rlH{pff_#hVDy1Hbq-alFHYgO zIMKnLf)~x$2f}-Q7q8;eEgHt2VJRqb#Ni4$*VEn8Iu+a3hj3e;p1qamjP!XBv_o?` z0>9b}YT&rq-!7Ac;oQF5_YZF#H(o<-uB>P2ip4d(1dcgpHN|J@^O$&CAsv$lXHUXomVg_GgVQ&RA78_wDDUc&$+uE?cx93U_@CaUl0t6RrWChBu> zmM*&62-IqhpPThJdcH=(SU5%fa^APvuA{Gp`=U^=lcrkpGI}-2|7ezHz4Z%^8Ohw~ zMtU=H;(K7kY_xh^Nx{0$1A({8`~F#c?2INn8GTJt3Rhxr=$(O_&Ap?7V{0TLerwTR z%}lbNkLCALc5}ts5ym{^B4hk?F_ZOtH_ zG#Z4R%0^hmL;z*$npZQ?i{|p|Sr)#tCY$JWZrP}ycAUg0efVrWtMg@a6pQ+s>08nY z-?>lQYLEm8p9U@IQufYpb=M=w^y<1Om}|0cWxF5{ULJbB%3wApd~hk|HsQ;1g#iCp*1x(;zC4j~ ztuHctawyTd;uk4dP;nU-qn$Xi#^nZi3Tf$sDN_j%hSJrG-w_=*At*oyx zT0Gv|7VqSnV13`RX+d9}@$&;K^km+LI+MfJWvM!ay{U5tPiV4p-hKp%6jI)^TQfC#6yHpL=t+a->K2R>WBz<3l;d;0TGm(Ze{K+isW=GR?%NeUefsqG z#yn!1uH8GYK3?jhd)nxF*dn3hcrIfIxKdyaG732FNjP5K&v5BGh1V}#odl(gr2E zNv5+xy#q3cWs{C^w~Zsj9jqVvnM;ba-tRgCT|_heneX(aJI|FgW@k)(oj;DH0;llG40b7v z=_gDZq|eqQ*O9qM*ew-nRT`w0zVgy#sol(Ek1EeGHvzwQp%;zzd4W@?`SpB#B*0kcvUG)EQXnLd$5fh*2T+lnf9Z5d z(&}dyryj1``Nefxb6{U|v%rgNd_qU60#q0$gY<$GFUA<$oQ>kPT~SFpSW+H%$HQqd z$iksNYm7_-$nOt&p?NcDeJYXoGTL2)L}0yn&Xhu@D(og}0BvV=X`Py?T) zR6O;DqI7Mm5-|^2OaI)JiwazB$MuXoE|;1Cz%eC*nmZ>)=RST~Ge?67Rq>M{{MEaE3#QrF6MpjV09zRi-+6Cwk zoN{zqwCwgq?!&7EDI1l=%MGIWfeTz8Teoks`8R|&GiJVQyIf^K$b zt864+t;*_lfG~SSOSQ9lotD2wI^N1$^3_Hz-&@-)ML6SJl`Up0Ndjhl7i*$}U_ZCr zzVoBHR!@+NO731ZG!d#t;Q4)cHVRR8Uv16_8DP73P>7z=kCggUe!nKX zH|(<8OUsMnAM!j;%gte2E)!^+v+j?JPPE57c&%nG_fNh#xGj}N@w>b}LKC>5-DD!< zj*Cg`HqT%EkeYGp+M-ffT}(S(_o(Eqn>u$pyFC0<8Sp{XFRh`*)QoTzwb&a>>J!d- z@6TpnkmqvSEh07k;?a1m>_yc`!9HUv;bl7zFnqF+?DZL2XnSzb%&z)N2WIxC|m)c{up3>&OJm`%_*47~1&!_Q+ zQt)fyI5XM()A4+(Rx>6$jftOhj!0pbNY%~5Y)f*IV}`9$?##b_`5yOjxRUBdf>7HM|%Drv*o)@oe~Xcnj0()7dl*Ez=pRpJwf5_kE4AX!b7$&`&(3M zNa{t*-wv!8@ii#1uI%V5Khc;13GLN`!IHM7%j9z9)JP7_f{u>Q| zmpU^ZM{uOPEpFbP!e4Fbum7K7)&GLNH>K~q{;U0ez_kBWulwt*whYWcfg9$+=kzC8 z&;zNn;mE)JzZ1_n{mvJ-OLDr`$G05H*D^V30`ragtG1g`6`Fn&o+9khJ3XCTT3)`T z1WHnU<~-9N^Y8VMX~7$f4r^UznmkpGn5G=-Vh;}wwR9f-!`&#;i@{}ZWm+?uuHI!X zzUx^n9`*_>1N~c)Qvu0DnXA?VGiW;XCz#^A3d3WKzw7C~Sl52BFKDtDFK|(*JA*v{ z!w;FLFrVW;Sp9P{y%XP_>PfnIvKC@dLeXH0vz8!* zWspl5dR5a>Dxw(y;6amr;NiHw3Gk03;0b;ICdHW?{!omROaEMN@2A{R`|{s1K$~RN zjeOn}EM`-gr=*nT5##0KLzE2T{+703b~iBR9m*oWq9VW_qE-u9N|1`G9 zFfcH9pbH!yI5GiTE?$j2gjEKa2cG6#plqcUe^D~n4~jA)2-ApuF>4_0StGv)I93I~ zwlWfUO&9)CVZ&nM#fH^66Qt>0)|TU|SDIQyxBk<`$+~*bOK75KQqxom;XFJDVWa_o6^6>1i-rq z#QXhC8bJQ12(qdGfC-;pk;t>8; z00A&Tet!XA;qHGg`Xjpcw+aM+^QYi{sQKTE<^zc>)bg7|b8~V;(ubY3U+ik}v}`AHG3@nl5(yolA0)ozgemJ0o6X5~J+jv9Ac zx%`$bxQ4TGyDtn^bHB&=7&c0=?%G0<10MnZUM0YbWc-I?tfr>PU)}}086p3q{9=x0 zhxS%mux5xyJigd1At(A9$^{i^G3T>`_8|!LU0=t>Zq;{b`yW+zwIsA~E-pDRy-u3Z`q|YV}OBRRxsg9^lQSoUD0R z*TRilId$vApP{S8(P@BW%BWx+<&hu2m&dBgT1B{-IZ*1hFx7Vfaivbi%G9lZW^cXYZd#FZ0{s$JTAZK}NrUg9#?MLjt z%#U-@fv4f}WO49WVWUr3KnQD5QKH-v=e(hr%=gemkSY38k+W^H@gsXPM!|}0Zg6l| z$4m1$9RM=mJ#NeHCkaq&sehIr{i}2`b;SWT<;7wqP#U5!Tq{R&P9%As5Uk?gq@^l;kX_?EkzKyBh~KlAa7o!Bvtz;VQNcLGcx5a?1p%ub+cK4q zjaGezQU;9fes-4__%NpIg_yWEgqW8d+XDAPh3@Y+Dn=wbm~2IqW^89!Ma*sevKl-O z5bvD(mO@>>1>D505g)DC0Un)Vl_jYaE?7=kDC~HlxxTe$##lrFlgBBf>e(CcG&r83 z;+|_J%j@8z3*quH6{J}2=^b1g_h2HQ3l9R<%Q0n*V3^8c*#))+f)v2}&NYqrN_kpU z^K(|b_hB+~n1NHTxar3%!@0WpA8|~#_^*9CN*s(nc=usutALhSP&4Gn3)*GVF=`TH zH}=u`JNxwRmx?lIp4f;kMZ@4$Z=C}tWX0fHIo^k+rm+6_;Oc^nyERb9M>_G(BvzyS z?e(onj{N&5H{K1r@8fS3PYgDG@DVSj~S7 zibH%w&JaCj%As03^7yWD{@u*wJNV^-&X|wWgV`0;4EZm}MY1C{PEP0bA_jf#CN9oI z>-!+L#BQCBR`yR~b@JwgP&gZ&QYh4sQkGD!M-lX;W0o$^V|vJ3(D-(-uzgsg4Ki)Z zpeJ{GS9Lv-RK}cfYVTCu$|EaE8a)>Nn4Rc&e`qRZ@~|?tq&IB@o#8d69C!6`C<5S{ zk)oJ=pb)N8;VzC;DhUD9ozU-_rgY0KvkM3#7e6{OPyO1V(tr$WV64`0N4b5;#)>FQ ztt{>+=vHxZ-7Hk(5ZQm$FVlpio2ZH$hx({1xn%A>Zk*9^(ya*0gKuKi=FD+{s$t6F zk}S0nRDG+hi<6Dm?gZ!=;_BW{`QT8mmAO+oD~-fa2PJ5#e|%FJEc#*CP~m+F(R}Rn z&PC88Q@^ov=>#bsXu z`A)G=WZpV=p^b5t5dN2P)Ob|;9iZc(*otXNQ&7jwrJ3Rj2zF8cco{&xnJ#eqecjj6 zcX`CdRlzCRWgecoky0fS0eo8?QjU)(UljEfRFqh+^E2OL1&50W51T?0UyW9=B=jlq z#&8l@h+kUaJzOPz8myr!R8@^EuX6Q%a;w}@{4m48mg2Swx5Tl@_io!vdA1Z{3yGec z^6>4(84vv!cQe<>T5kS0=~)oS?+U*igM)2$@cKCtiCwJvUsb(2X7*#*ng3WpC?bTW?WTm)Nugxg1d z602V(zXQ9YDp02pNLoc}anPhiRiu3M=g;2i`fqz=jmV`dNY*rrO{G=F%j9iVbO{4Y zRg5XxhK2An*fT>;*`)13&wI@XGlkcc_G3#1BiSMLw4PGuS>EUmtuV{o7KPfFHIzKy z(#c-1A;NT=eS8b-NB}%w)7(H=PitZa2U%4G1jMv)$6EK3w|$Y|R|X>w6lY3`Ykb|4 z?PLISjyWxWv!>Tx>;vZ@(4CiM=LMF)95sC@cgwkP%7oz}dnFc%XjRpQ&9mBj5HtPu zwC^NFv`Ph|(Sw4Nb(go=VmoGtG;pHKLL$|AYDA9ZK$=GNt`+!SYE0xGjLZvIk5I;R zFz5hOEN_03^~@{%J!VoHUndnnpQThvepB|$?-4ti0e@X=*d=ZXH2ElE$|vU6C;W-T z8QiZ26%>Mr98yDPvq%c-;qoy>vENfXRjL+ta|JLxGiz+4lo3Ou!;-mlsaZxk`GxAxeuT$S-$8TpuSfK155n z@slV0j3MhcQkP6)>b4QmilFSz*J9c{gyPF*JaY2rn+1B1|OU(Z4Z*j`VO096mKy=3z(iXVqz^)b@D;kVD*s^J3G zgnfqgX~^Y+283pRRGtJkMm($IhOq9sR4w@V_|LX){aTbm=crAVhZoTV*b#b5;yp}= zzV9*sv#7CQ>&15o(|NpNY%$Xk94F_ZDEh6%b3|Etj37@~Q-owWR&2awwqdK`x?B;$ zta;CSgpy*t)a+XOwmwF1q{)hD^aVn$*RrE`lO*4|NmxfT!UE&@O?$01~o2G)AH6JX+U_{dv7WN+IRl-M0Z<-@%UupF`yWkPSNNa`! zxwJ%j`I(BjA}>c}xf;>wbanvn(E{p|)1l~lPo2^-OAbC6$>)^#o{=XE2WSbK=8Z57XyH z55n=eczVq?(tq54%`ivm)yXuzA_6x5IT$e_>a5MfgO?E>G9n4%b;}xg_h?+WN)p$j z;A@3ZhATm-PEE~@kHS5cpBh6*{Q@qTw2_UQ99)7+de2iW$-qfoHwgM-*574ojrwU! zZ0}P)BYT&&NqOT@!|+@<;A~>VJ%S^C4-Ktu{^I@ME+-$(wi@x`HbbK@PT{N|D{?kh zAnpFKfMLsbjNs-pb(@ZhlG)p+v&A#ti;OIxh)$oz`kVe5OfdeCPTxGSnW?iUJL}$M zlY%!pvozuw>d)y*j0jtp=ea!?iQ+1J)ZYZp|F(c3uvE1G58A{k6g8ct933-eSJ7zC zJ-2l233-T`PKZH~-*ioG{^CmhV~uI`2NW@KO~EPPD@pcjs`Kk!hus?4Nk0tp?9aYY zCLI|)5ag6Fm6_X>x8{uF@OK|AM&xwdnsa z-+%i4P2gr*fd;@6Ds}ReY)Spoi`;&#<+9^8+8<3UL1fSmd*+s+}6t8m`Fhi5C@a#(#AFIKlDX>c3t%( zr&bmTA%MD``j$hUUo?hsAWV;-^%CtO6`b%=hRwH~0(D(keMB^~%T9@DVaE2a%0J+!Q%Wy))0Bp8< zPNoK;bFuqG(F8E1FcdA69sDR^NpB6*ZPi_Ms}~~rB;!6{(Dihq$(dyMUnOroo0G z8_^QfZQnNcdh$g+{N_2$%%rekJPE%m5%RVw!cXF{@czpVl5g?k^9oy*>L-a+#-N0{ zRR3BYu+Y|d;EM4z!ubhaNE0dje!a1Ws)ZQkH|`GEUqF}1X(~G=JwmyVnPr#Vnn2G; ztElQ;o-Z!N+B5nCkWcwZizo(g~{z{MfBi$sQX%5<@z1@Do)mL~Fp{VKBo@DcCU4u$X54Xx@6 z1^snZM6@WzsH7x*+Aro(_t+roJ~X@$ist?%-wDq1Ty|0~yr@A3BgIu+2^h2LnG4b&+^6Gyw zgKE?SKSvUdIk<>M(tonC7Qg(W=UcbkcOoXd)cNz7AsDd~LpCw6f7TM#e1?%Y-mOYC zkqXkj;S*Q%%>mSM)*V&4#^dAXI)L>$n!>!#M~_XqdrB0L%0Hk!NeD+4NSv0(>K>1e z4CB6829}|2jd^tx;PEUXZ7!Z?yLO$_M`D9vB4u+VsL_DBO};DNY4`oq;My~vuM&(` zLiH`2-IevAWhb*eQ-V=1#;|gc?a?}4xYIW$m53DgfMsNg*VFx4m8J>;(D87S@lTVu ziT5s|tK(k^k2O|IfHTF;S0wezy2ni|LttaIiY7O^BR`a#e1zYsO~kg`7%~WHxmWpW z!X!i2=P=FF2W=Yjlk^m&rDhh?CDDpI4WpTFq>wI~w`X;!zu5U86}!H^Uf02P+qlr1 zn|3qGfT=);IcxCCfyW2d+t|guzdg&5foI4;i@u>$pht#lN^z*$QaV<6@JxqNrlZ&AF=IkM~1i)!IsDVFZDbYUHxh-RLrU!B+2 ziIv*K=ZenFCQloUL{j1l{VraYGprMx;Xh>WY0wuWRc495Rc6ix73*82+{5GI(T15) zd|@P0)PJN&QD;NdcAyja4!KX_@ifm&b66yUVLcqL4~>$_IEX!XkLzq*ufTSvXQf?k z-IM<7$Br4|FL56ekDn=iKQ~F-=juCq(;*97SK95n1~9rM?V!oFcnOL(iGMPzh+Lp$ zw)z&`FGSO}upx5pxf)NbRTH)o-bJ4^Ix%ov@k&ukZ{P~|(^M#Ec8-rdfm7deW)mC3_e)7Q9OQ1KWlh216+XD=D} z^l+R?+FiCnim^U;`L+KMrr2K~Il&EuQ0PD%jS>AsdsK+nhVkdmiU(~m|D%ED?wXBVYbkstb$#{*uBy|w$BZryk1 zY!L&I1LRm1*r=-Ia;Q5_q(b;^(Rcu(*oNQMh|{^ayk6R;8LH9zX)k>t#-1O=b;qP4 zZl^Lz*qopit_dN=Gr1oQZ=I581t%fyz zy}a{Wa+LQNy4knOdO}@jiBPftAn;-uUn2oLwCfLjYD;igEp=WUZ=R7NBz#IQKbx;r z8^=N=&CCNM(kIl?WR@^p@YYbv*Cq7LmxD1Se4@R-8BG+xoA7x{TDqxVr2K=2iZp&j zgujj+fQE${6|30JNuv0GcN}Az=i7hO)cI-{pB4?GxNZ7CMblPJT;23GF&+OW#JR(f zN!(@6yD6JYTcetX$pRp{^#>#tHrai5^>rxDEZOeES}e=U0lXb=Mrdh0kD`*I6>SSV zpqBJ>#1=MB+Kqy1%Y>Li7?^FN){{3aGt?C(2~&`3(Vs++_B19U^^twIv+~--$E@p@ zpB%Q7Vrj$!PmXgN)|fG3R|EOca!AUNn;MU`*@Wh7pmY22T|nC)jvY^gzJyKBg?H=q zbmOw(_EY?!i}P6WmKc@1iCYQkO8)a?$hRIzb*RM=~#Cds9{xYhTO{Jp+P`>|qqfR!_b zV0%2OcEwbB+e8TG$i}Sv_N!#S3N0(taf=W1U^XAeP9>bYq=d|g!^j*d1X__OaBJYa z^c^fmHD6%z$?q1BEc^Ztx%%G4=Gg?HUdClksz~MQO;g#dGcldwB^mCLH#0vQT90mu zgYiuWGsa=FIpEEFnjY3%%w6S@jt|_$)N!=c0h{70a}yCbYf|}XcssBAX&e1_k#9`+ zajrI?;1bTS0K)lS88^EHDiSDaQ>G{N!#6k|t5Fqcw&lZj z+M_%p%D30uia!pdmw>pKEimr;VBoVpjY3(bQN2#gVd5rHrqxJvWv|mCrw)*KlszxR z3vn$E?E96T#$KshO$si*n)A$twWQcz{y=$k1CbC=#@Y0CqtO@bJSd3LJJo_weClAm za|4QqRLpcxcDpDCLa6Z{N{Mk_QpJ3IOw~MH85jtxv3gZHmq1YILiul*O^9grY5%c)m*YJHb*|(3c!R^J(W=z)QL@wP@`_3^v@`-5#gz z(&yk(c1N=>(e7Bj72s>i>XLvz6jzo^qNuVRltoy3M7}o{bQUfqHAc`JZ(bnf(eq|ueJVp6?Xq$~~tkj)% zy+h#!J_q7>-^9~kb(X^aav@b#!}6ht)nciOI`|IIiU6)FDXtzhV5|2}pgu)*lCH+He zH*Ivj&q7!ASO(w$;rJTXgLCM^N=`^l995fT3vuSw@yU1>x%8-4h^J2_Tv$!JD{%z> z*3GuJ!}vPTS6lH%O%UGOBI|KQpU%C@mVBc1je&PbeW==qh`?vxH&3s+%(a0vENRLj zmOWk+(&%eqc2Wk zT~~{zaN=s77Qh`uzaDGxXnkw+Wf_j0od12NxM^|wrIMUS)DWit1y52sNYsU3{ZlzgHMK6x~mX`q$iRc7Ca+xjVH~9@4 zt0D{d4pX(L{IKw~AwGy!VnU1!VzHX*XulKHoGK=O@V}nfJM`TBZO{}|OGyNv2r%2@ z++*+8pQH|uSnt`G6pZ7vT}b2-%}14`N?g;3^y7U6bz6ubHq`#=Tk{sMQMu;e(%U`0 zxAdq>s_JDtxbUWQ8vcQnnb~NFie%F^A$%p@uT=g(sD8OxPS4ESQg>DlIO96Fn8@>? ziOe>GhbS`wcu|PC-NtNlRe@L}_Dm6(yWN&8!LZAk#$sD-2RA6T1ei-gP4(*ze4D+o z886k59Z5HBoF=fvW0s0i51dvtip0=$eyT0}p)NHX5Yh67nme8(^4Kd~TqCB<|6;cb z8rZ6aBOo9!x==zdvmSOk$?cg8mwMC)9;`tXJZrDOP>P=v z(V5z#>~Y<~R@!$aNd>eLAVZHk`GiKp7`Ql?%N7?idXgTQw5khmk6)supdiuL1%%f` z@J>o8f$kLF#hsjQ&zU`GVwxuuVlS*iDZy!ejJ<4T{-yy;u#eDYm zXm8Xz{0>lSuzV21?|0oo>C?bI@(52LeW-w?*b;~fYro(F48;RB7P&-8>{bGR{S%3*8Rt4zeE4w@{a@F0y+$)G0fXyFZN~Xl(7yX zua%Qw&9dCwF=k-I*I(3fm7UHr{@&zlG-bfq9fgm{%JC8$eLOy$XtQFZK-VbQhDu&V z%39W1(_N`;a{`SL?fPDU*oYNJ8GrN$?y5L%oUO#TyCi1a3&{@|$2qAYFJ3e=rY}!3 zJm@yw+SM7r@nug-m_S~_P6nNP5H%l6(@MCEx41K>C3I~OcW2PtnQES+4t+FeL+QTr zMt4E;WT(SUu=7-(T@5$HWqEn}h}r!juJJ7Z2gmy%o<@S}P~KgyP`#^>bq~62gU>~F zxsP&e6z*|LY`wb|0l1b26+`tO7)p@X@v`nJ7LLIlQmDkTbb^?s=r_0rqzzrDhvaeW zu4}xp@vD@l$~Vz^K@Rrn)2Qz-iBj>?y7f^Jtei5CY--ymvOd;|noz;k`pb2|tq(Vk@ZlOfMk z&d$SJH&n9(`PUD_5dgs#n=CDbB9P3-C&(gW#UrGLk@ZrG#PEn4q2qK-H zQCdmQB!X~^G2+M}!d2Nb8N zDkk)r6?03Eu852IMaO|+J}E!E8E^4LK}(Mwh$2*Kp56@O#uvL04Y09lM@bC{u##+e z=*dUFimNX4F39*4+z*!kMfHBPPFH_+kmybrhv=e~c2|4Nuq$RE@GI`aGPFs#ip|Yf zXFvTfyC(P!O-V}Bl-M$#?k{ZLIGY;h3BmX{!|@&pk6UAqJQAk&i9Ea%ZLur~&7GMw zK2=4R#}@iYqHoAUPD@mi-gPj^tTtk~7PZrN90EMUt_9j@8yivS7R*@d3r_Q7sTD+1 zg>{s446r}ok4xYvcw?(#oSRYAi9Fv~kF`uq zILD+>M2NwcDPR<94-x$$a1p9fd!-b1_aaSZ?nC?`YC}aQ)bNK)7CYJtO-0b2 zWldQBP8RrR1VJ|j?|d^A5w(@YyM7?~Cld`01Kja}1`Odn6xWPrE_*;LQWbzr&aBuj zLLC*Diioi;9cA_B3Fy9|ifyRIB`SGfbFuB90jvQ2kI!PMm8Z8%3$QuT!y-&HPOp( z>=gh2wC(dky5{<(LYw{We&o%hdJ!F+EeP1sghfDB2Eb22J zE|5||#}TO7#k$nn6@l{r%XULq?>^dlUS}C;?7##u3fNdLWw%uuuN(^>es<~^VX-da zJCg9KB(d&#lWuJtp6h*Q70airgA;++s5nMjrAJP^yCqPKY*?PEZ7Z2Lb3eJ}daxMk zll@CFyoUuDp%EK(LK%$8kY8hk0DNQGba;Wojuf{82I1#YOyHE$& z8zxi%DDLlgOHZMs17niF1`c(!l2Vu4oVDcTWXxu7hwygM}+9pKp9mC5UtkmN(t~) z0oe(vK|eLUiAnM*_VUEj;QB<*2@z3oBOwY27V~0-knDcObLh35CLJqtJj3E|dcTpR zJzO+NroEUS_X~Ipvuu*Rtz&A?b&DM0OfL~ZC))Zhf)&&&kLj{;hLl1}6wZooC&kfi zWr=kB(U!D4{`h4>>4N@q9xiqgZP;M+?G~&bgE!=q6TwFH<3|@j)RG^{PCc`kg7tKt zj9f(gzF_f&o*y`7*>Dhc8WLjC(|k$3)RjA}a&p>e8aKu%6$bU^N7r zEL2>bailnzc3H!GE9(B$K>?>O@;TT`v%qNqLu&x@1c(K|*+GHT<>b;-3*L5Qcp9n1 zHtX)>*qs8T{x6|!e^tN+8)vAnzeCkS zDq9E|o%&JiefaBh;54Y?6t3ky3}D}{TUx)K(bWG|{;G_!Wi7E4>2dxqLJ1VNz$x@^ zX?9Xc=iSb9xUfH%6GhmHQMUY1eh)%m^1nqn|Hr`K{}3zuucAMy{`Vl^|L0NDe;xk! zywv|Sr1}4LnDd(j4Uc#)v4J+DT%Pm*ZbRj;J_~W)Z?<~53(+(yR=Gh2Hv8~NAGAPl zDD#MvIG2y4qdCdx(wBqCex=uQA-}xHwrUqQpQH@hnAaG|_07!~%-0w<=U1zUA&JJR z9zJDI@?JJgfSzQwJ*rE&y91I{acUY@&LxF^?;PEIHK{MJG1ZBBFNnRSU)x{ z3Ak@!GFW;`rGj47>R6jJH6Y#C6VJM9Q}>~kyVm=~!TsHzG0{6x<|ffGD==K!c&>K^ z4dNWHMkD$>%WYp-%U?Tt&srH9_eS@Y!!L9&_;Hj)JqnO)NgqlUdR$p3S@Ax3O=?a; zAfgv8iPMlWXKI+F(%wE`}1FMIjBRc zV6bO*hG1f7g&7>(ilX$(p1w3i*+t*%ZsXw z$3^@QuHwYjL!IVwONlWBth|T4fd0r%4I}He@Yk^6+**|APp>m^AU+;-vf5@~Ngw2tvp$@kqOXo6oaL|qKKMLK8CCW6Y zs;XfMe(`qy<%!VT>DIV##YS}`U~9#Vx7VAbwd7Yk9~cwnwMZF}g+|?jn{t$L|=HiBve~qu~QRS{x!=MD79dyenLA;gpB%5=;U{!D#oR z05>7yCWhcn+I^o_gVsAs9JG7^Uj9{@tJN++Y}eF!`g^eM>q#Pua>jA?(ki?GAvh5v ze!tJNh1gR{|3?*z3gT{*wtIT_#<82$d!(|P|58~0YTw)xDs^g3`n0jJ^t^oBedVId zy2zuQ1}y|j1S8}TvQowYTb5ggp3}+|KT~79s!J?mMqYWt9gHIfd|E4PtVb>{4wJUG zRC8vpcj_h`9!Wf8idkHdUR?sv^4RdqG^e%o z416X#c>|~6aNJbnNKgLP{~{2%1!HCOUxsbH?oS)&?fc*|35T6Hn@J*E(+#gYp0)UR zn3_crEGUw6Xopx@6av!52+e9qh}q58UANR9lim1SEKdxC zpcfW)4b{s7++gRs1KgP_-Si6sBzCE9JpmlDRm{x{!8czUaDl{r@a^_IHEu_xbhnTz zano3>M#@^#4B1TdIjh4&>WlFWE8-Br!_;jJsYXWeNniK8z~uloWv6KkK#Kp3JL$-a zM5Nl$99>w$I{%a@SQjurdNSW%#aw>sqkuxvU6zvjNU&4A&6X6NS8Cliy#25$u|7re zd4!9!jb5l=m64Cx^wY=AA z3>lt-L5B-U%yqkWfVYawfv8QoSb=@R{n=*jbWV~GX47wO&OQESrMiuFR;+<98}peA z&l@?rn_G^Sp1Mao9c>n^MNS7Xb+_I^ckR8)P2>o_^~fP0*2H}SO0!pYI`o_i#@lay1}+i|L~ znN%#XJeH`Z8YUMG@17}Mru>k!H_3kdUgfeYsCx9o^QmEZOC!JVYZ}`GFD$fOOd5!X zBB}Jl6oqcy+I2P>81QO_D_VN`usAq3yq6#!rn{6e4<4dy(=`Sy#;8QNCZINuhzjEuD>4z@p-82{EvY)nN z2kHc+aR)G5P88PRJl4bDzTuMggqtIp)bPJV0nVf-2kQ<599R}(KaFUeLNm9^1lTH9 z+?N@Qt1ck>3gIG9JeSbjqDI{UmD1CeWs-Pp{vVgm;*VFuioy{=(o>q)lw8Xe0s+pI z<<`2}_gf!S@1L477`p=xrduxv2>KLlTDPH#=YcnL0;T#)rBmv`&*+SeJY6otDlwHy ztRg6#ghTg0n>KsnsXV}JV{DwP%ZlSq_jfz#&4J#}hUaj;u39Q(VDyU-07rX}^`oyz8eU!In?8NNn|{*)tilwX+Luj8Y+5;7~HtVd0` zNU;shy(cAx@~YhHYRo#ELp=&Km09UJqa)Vd=Z8#_qQb(+=r{y~Gp3Qn6+u->!W_)X zmS_4Gp}H^f+#b$*)o&(X0t-}%TFU2XxZT5^s-F=LF7ee}pB7F+6VJQp3Jy1Fa(hbo za|0dyVR!TZ2`{-IgO!F={@-f`T%YL0GnfNX0?d;A*Q|VwV#7$kqstR5ZT(GudHqv? zt!BbbbD00DnC}d0D*N_6L9l?zI1V5p;0!7n5LCK=#2FDq1r$P&rXZkn2raZ^#&J{- z5RoEX5s)r5^x!Cj8bpOq6NV6w8j=u5=RJutbMODT&->wiy5~c(le5oWd#}A$IeY!q zZXb_%p}?9fwXL#9lPG+3B1)>(aa{^tB4 zpN89a{PL?d+V1;bE`R&{X-_hwC2D1wWq`>lgwGphrM~9x*i&{5QqC}rD0lbKK;!4% zyPjBY$H9kn@;;78-ePQ2wsqkRa!VP8)nnBW+`g)Q(9;N>5Bhjbe2v4_X=(BFnC#7A zZ{cy^u3IXow|`4+ky~#{lL&OY_ylN=1O0F#teE}rAC^fuk0w_OR*F|5Zi&Bbgg)-E z_76X{5IdL=|$&QgQ0QDYS;eW-r$kS8lC;HVukUH{>b49@SV$r z@`|(xnt_C1&yPDlnG^k^4B4hH4h6Pk!7n*CO%xNqjHSE=6Hsy_r&+CVIOI?YI|qt> zU}_M4Y@uv5!~I~`X48A7xAyixNc-mBZ5Gle~*w@att zI9dk|VtPByl3rd!LEi|RhVMyjDNEbdu3o;IZq1d3a9l8AU#rGj{?xu_2kb924^xwY zRn+CI!-06VjKc_Zy)v<)2}mm|qL^r?CuR*=w@FUl^6>cN3h^Bw*28Vv1PM^|wZu;2 zEri#0k6XdW`CWfx+5&_XjXZ+)*5FCPVCgKy!=HSe?IrTj?n>k|im-%yPUwZz)31cw z4?OnU>7U*G%{jR5Yo3QUqbTX2{m-L5d;UK2x`|ec(;i#Z4e+=3^ue`gBXjC!Z3>rr zU~v)-XY`C{pX^l>mA~3ddIvmJEhNdn`X;sUIS1c3GDj~1@mEsLSZVDYXd1$l#lmlR zAye+}ZXxdpS3!@jSmCzs!FS(y`Sq&)OY);=cR)chA8$KOO(^(W4chF{`O8y#30Edy zINd#0drehZ)8J2Ka*o=R1@;NR_g-A#7~xG-(1Uu)Y4o*oI}*lwR!0n$Eo!d!kGmh; z;c4pxS=KtEOHHI)IoQPPVk6!AJeN;9jcfGPdd*I{A$L#bTO25$I_A`?x>b9`nn6O& zX@)s|Nv)m}vsc>I*vvT4Q92mw4hgvv1V#=g=EET8QUY-obyM~S7|qURBN9hS`v>~F zoruPRpOo#@z0|V|Pd`TZHk-^hQGVAV29NP8mz3p?&4)(ca!JD{JXeh_@mHXpO=u(# z&irOFlK`2Vs)qArdKCw~Q`-$U^uctJn(Uwl@5AiYGY4~GoGxR=ecSWg+rhz&Psn>Y zD1G+rTj*%|1LTr&yusFOFSqQ=B+Yuau*5~w`i{9_>YGof64kov zSImMbe7GH;+|9Ae`m`l01G?xy{vOB|n$ON@GN>oWzh>_GGRbo?((?4Tn5?v*-F|U3 zEkL7!@;duaSj)A=t1+uj;)b<|I*~aswK_`!UyEz~aUd7jLMiRa(^tw}oKi=X;lc4e7**C2`p@|4g;Q56_O~gv zfcDI^&~4gg;X4zL|<@^XF7}&C8c{Yv)5xZeZ)T|bDFG)>Qw?h zbz}539TDaEF?Jw>8&hu^GIuR}ZaT*kl5?mIH`KM`;_NI7)~TMKFg}rWk2(8Duca7S z`L4g)P%(bWV?263ElyzaX!l6oT9<5xf{wGXE2mqY6|UO$qZu(+*`)ax4o||UMkXgj zecDRYGrVg{j13rgK|?5rAl2V>%-aEw4M@~!<{!0HNWlCg1^Pz6E-oh8eZZKveN-7=974*!^qMy{)-sBN}M0fsH-k*3m z`cAqYYc9jQ{j*h zuVlpoBqWV&(97g)&K%J6>omK<#4T;z=d6ZQFic0^p2F3S58+P&vyPa$)5E-pV&&_MSrcBI8#gOj)}Q` zjveUia(3Bhlm7NzxE+4P#5pBE$vM1X|3IHkO!Vhxw0AlNcU-sw1wO&NKf|WnGok|E zE)h#btsAS&mu-pdcY1b!r%IJZUBy-Y)c;SXqa_cgbb#5dv09lamNn8IKIGh1ei54u z!o^vOzxw+5_gu=`_hD0g?3r2cN_ldVUO-QOsX;7bvJpv@dcQfFsEQ6K?-5}gWAF6M zJmK@4RqjiB@PqM>$eGbkV{D>!-!BWE4fe?Lt&p8%>JesB3*?2dCva`&Z7K|foIMF< za6gMa=Ss^K@M`n;PcDjq3#LO?e63KLs_+gGo2xL;Ep{IDY&U zbpL;@=518bFD{Ny$k)sO={ES&B_cv0oFbzVO&$wouGJ;?*qNMMRY#At1b^rzO{8Ne zDYqFUZIN6kSwX(6#LznBWRe`xJn|;z<~%#eP-tIYpUwoU$1?UODJiK?jERX!fC)8- zxd?9xB6qdB9G^nHv^vf&*VBf&a~c~8iIc1$G9#Bj-ZZK5J7Z|*s1n>fpo!(xtO}T% zdr_XXW1se<5)wJ1#d)#4m|enAE1Lli`@|yn<*MkkLx!2ju2eqvxyT#QUj@v|ba*)8RE$>(u# zb3+BatEW(W0_r4+p@(vhFHoME15*Z6xJc=#RWR9|5z}VXjItc9Q-m+qIWtfDeOBIL z;R@XeS&Dv+F*z>VDkc{gF5Jl`gjp=>z4G|P8fLKGhj7rAZnGQyw2k277>NTB0ieiC zr+g2?h(#=%XRi!E(PO{02D6H?iFf}xU<8?LRK*p{kqV(Nl}J!$7z6+TM`~z}NoYkd zQpQL|QRdIZ(hTz_aTAs(jy2c5OybIeH#e7U;?`+XlKI4l*EdcW-aO6lhxrp*?_qvb zsuf@qAzu*APEK{0!k-+~Wmj z6kst3Mek7!TQ_ROtusMr8?h~_u(TwX1a2gdw1vB%lLVCCp4O2l2O7oQR8Xiifs|H01pnet8a5*TLQgA^nQB{6{2g)Dp)lMhcRF(y!iJ5*CF$QZ)Wxgfs#w?3%sPgoNaNh68c| z6y%pKuQ1VlR^X1gSOm-qGHjk|o)~A&JYXWDAhW;9u*m!w>L)o%06uE8>&DP$@>B%~ zvSPJlE+%y2lu^D?*~F)J3(Mg=v9FiQ&GRJFZ&1Yl-VuiZ6F-~2j{L)-;^zgjfV_U}Mu{verb5;KX2s{Mhv32t zGl0|RR<{|@fC2Hf)u|Sl`TR3wR7SJ6Hx~(p77)1~7IfhnO#3!My@6kys@jZ=aZ#*1`rbCdJc*F{iFECH!1wE3RL}Fuo8cv2N^L{s%mTho^RD};h)ej|-fEtq>XEn-;iTMmuq_WlqmZt`>FbA`ssn&NM=IaA>5A8|M)e-IiSDF!m2Puec`p$Yh+zc zU0dB$9p64~``=a8BKk(YX6VYu3mY4|#JFpCFX|q+_j#QU20US4R4VtZ{E-!5&hWpH zp0#^FbWkWBKkr1A3WK5|&j$Sd|3Y^DZy-Sb0~N&ETA3OQc>bUrN5gi?7|QDXc>BEl zQ}S;J*@u&}F8ES4CmpZza^jaX+8+O!(MbQjQU3_;J6X*3l(?bZ&Mw#e-&Y)Oh_ik7 z%I%(1!KEEs)PzT2wrkL1zyD=92XOG~-^;Ykhj z#tMo7*qMf19lTC9m~U9kKy_Vkmf}{uckiV2ksH&;-^)Ke1CJZTH~dGZ)Rw9!wrg!)Ydw z3)**GX8cD3bM@9@fh&-Vf1i#58t8|FJk=J>91O@5aUYzIoY1?0oJtI?qnHMbDghDNH?U#L2XVxgZxE5NHSC$Epg;Z)(CRFjh-{Q=Fm`TE1 z-xgSBlDIs!FDow+nj7hD?K?IsV|>3}!d35JTML6>NM>DP;C!x+)s_fo14Cn^5O)x_ zzg&vKo}9XW-En~YjLqB!8+Zt_^+es6sczcn*rOUnvxjoKIymzU zU_My$1)y?WKLMC*osR&WDr5UC7$18bl!U{;geov^I`&TCboO<9mOJm>3qZZVpFWw? zIriracz7ype1Nl~te!PNeCW!>XobgjbMKbqsTR3ywM&rPqXnCzZy@^PwLYuM&Xm8# zTE?e;9leoM?QATK-RiDZWL8b=iwYb0#n&->w}vd5>1TNAyaTe6@2yt}AVC2PfAco#cLOZ+O5uP~HifpNq^`No-%_4Gw$PKh&Jc zmQPecHu&3F?%oXhf{nd$=)jwkJ3FR=);pYzDD|X=;-xy^Mi%mDv+eWeZ2#^W_h3hJ zPhWk6Cy7n~1chG5=!Kb8ZKt-#im7AWwK0QwGk!}_821`M@?W`+Mo!!YItlNxgVD4KliqU$AW`=Vc?~ z?(QVKbcc+x*N_i@=^VNZ>K`=VTjc}nKqXenp`PI7%7%uFb*Ub?S0tuCmz~{aFKALi z3?_7L#Ox1Zng5W4-I~4u(cid`11ZF| z8WxbMq#%~ODQ6IavR*un`(TkNF{ieDs;vMk+416q_qtrQ-lmRJ6~u`i z!c1iQlAON`mCla#XpS(LV|FQE%5?!%&B-UfX!3vHDlA3%m+$Q7Cxk;3In$npR-5Ei zHT8(+>mId~(L#a-TtBQCn&{w92@W`G4!a#MSnD06*Azc7xpOWj3<_*LSD z=ca~YS}_sfczjX#eECM`ha#hb@H!J{YVRZ`sdM!jf*_tKfC7v#={D(Cw7*)=8e96( zM2R{EC`p@UsHR_&-=snsS77VwCt2Ad4d(w`;aw6=t2p*?a#sw;&9gztM^7o+!O;)U z&}N?xU0vF=`0%n-`W9K8WD|PUWvOXIUqyf4pq%C=?`ZR0Ja*G{zNsnwfncd=qfz>n z-)p^Q!v>j`cEmg?>sEc^rt4tklLowm=2U$GqJz|@mo<;X@m+(H}}7kEaV?;D=d6d#D{@vpaKR>U_PjqMJ5q6 zlA^M3HHdVy&{~T0^)->`f_Jq=%QSi&$j;7Ik446EI;7nyAFOalYC_92Y7V5~wl-EK zey;4CY7Q-F;QOa3D_`{=AzPV)Htf}o9UaQ`7&Aa2khi!dLR+j0 zX*_jtd1#Jh;_&x_pc5i;N zm?veWyVu{x-{70~Vz;-$C};E#T6of)u$72@NXE$#u&(zvN2u~7U}k1Urc?Sl@6X)_ z4gOk$FnPo@-Xp~$i(drEplVoepI%!Fq)mgYHB!opre7X%^8nJQK(jw2BS0A{bejaD zkc^dvRY(MLahSC+5kM}hYKJNbjW~|XV>7;UN)&Uu&8si~5EO-Xv-h2wLr$FIE^7^fbvoSXAl@FQM~b%Ac3^Km)f_0wk8tBy^tWmNwzNebk}lg}>nH>RPQwj3B>r zLGhVL6h76}%TsubR634c8raHJ9V13ARYXwRHsj`cH+r}GvI&Pqn<{qX7XOvm?PvgoQ`W)si&iK!Nedft=dAn z2J106auDlVaYoSVvsKemGmz$k3EkB}AGk$l79qz^FEO}OtR{Z@9R2wnLEtyKBb=Iw zUVN`;%xY9V=F1fmf-#V#a%xFS6V|VM%tt%D>M}T9B>S|8j+SHO0N-Wt++IA?mG02K&}UA=RD#kV)<==P^Nr#XW^V4L9}iqY8nXAl x{k1W%eBE$GPYU+!ajyF{AXCKG`X0fakJzB*4{b5OgKG!4Y-(|#^5{XfA1>6rik literal 26652 zcmeFZWmuG3^f!!%fs~4Xw1PAYO4krl(ntynCEYLs49!6lq-$X4luqek2ocF4q#Fc= z?rwO;bB=nR=l^_vuJ?N14_tH2J?q|kt-aUUtA3jxRb^R%dldJuu&@Z^UP-B8Vcm4b z!n*O-U2Mz~Yx|lm%*PFgn(Rxgl3uD+%oiLB2_*?Etj`g67sj_R-*FvZ=|Hfs2wQ&s z-Dr2nGsVKP_$((Sq2Xq**>LUf$SQRiaV>w1o^4Z0k=c#Cqr`HJ7w)Nj#MkEbu1ihV$J`g< zSlIXf!oqoib>ptM(E`p@7<#XmsO`k>-$y@x{nCc}1pbltADW*J#K-SGftyJpoCb4o z?*1B}Y#Kw=!SFw+rmuU`ul_qnqyqZkpGnhgaUxOW8B?{Z>e#x0ep~(c^5F!Ed!Ti^W_2UXXvKhZzD|bWcaql&*-b+hcy@Zdwd7 zFSVVAKKxB2KKzh57gUlN%WOV8b3+6Fw|KB9ad&k2tKv`%MWwExRWsX%(h~pjidk`6 z62Gvn#papi=Bch^iJdZeQV(i;SmFIgkV1Rx61?AXbx9jtf?3F%iJ4=|E9$h`0@USK zbXs{^g5_k~5p{9kKYT3W7O4Zw;s&~q5%7>8t(lYFXQ8F!9nQh|(jiAw6s00}OYZ(2 z_DPu&_BsrOjEEyC0}YZ*-;nOg4xfU!*>1HaU6bmKxwSCg_$`Te`X0^*3;doQ>-RUI zpe{0!2#t~6rgN2rR;pQAy-0)>4UYEvKUy;;;CI-OO-W3I%Eaetr#>v?d2RfPn%uU$ z`}&*=A|zm&aDBC%wqzQ$?k3a5KT`$!seu2i46BL}5PWLr^Mc6+pK^}L!ou{uAER}E zS$eDq->St`E9LQ5(R+J6*QXJEQ>*uUtySFSbBczx##iR0wtb6H*%46KMS6FQr@BI- zvPZCc8bem$#U_g6>Ay4N@nNq=DVX`%#0#^-@5x+UKts4XdMmg+zGTI`+H{&zY#6Cj zXo!?r109!yYdlit+=FPg%`+9*3 z1Nf}Y!z3Q5&#bxz&|;J0!j8j;2~W|K_#uh?H) znN-U`*3|O(bLVO1K%ObN48<3ZPY4IR9~E~;DSTk{ZvihE)`3k+esVI68-+2V!hm0f zi+sSW1DNzN{~8nJK5O5hCn38WeowrMx&@s78q5*4`ZDQwBc$Vy)2zf}efIhiEpx%8 z2WUSwlq>AMqn$HzWwfy=s#E|X-%_}}*aVf^ZmiqM$Gz_X0W>9x`Uu-`ob<$e`x;35 z!6b0uyvfT6IZ@ z6q*wm7TN@~P)pYBuen9q-Sua1;Efqc-hJ zTS;gt>qi>OYZ@Y-o?zIERnQB|J|EmZh0mP#S8!Lbv8m&=B}Irdm835Xy>-L!)cq@QpntFyP)gB_?zahJ*S|umQ>y{|0g1fetie$DeWSlyul?dpGAP)Aq+a81BLt;isrv1%fKD zYMEmlMD`X7NtuR`n#ug!t0b;(yjmqSU(d}Fg3SsdlkdE;cVux{cW3O>m3t1=3{sQ{ zUy?|aNE0bb_5 z-pdf~!~JGB@f8l1MTxoKrxfVu0NpPaQxGR^1!nOM^5Tn#d0F!{8w{N)wYStuYWt=S zR&kqq3hyC)hP7ClV<3ajSUc#S_OKlopz3M1R?*(qS$#qZnnZolyF^!Bb;L^$EZ&F1--%~#IlI0Wqu}DET<)0 zgnOT{Dm&SuGd^nl1(wPlW*rw>-Mui?~hHed2 zx-}E;o#yhf!}Q(1&MCbdM?OXY&e15pGk}}DuV4`)(HO25_R53e+3|ZgL)U9X;55do zuZNk2Y=3IPvc&s+@QeX|zFD;vQ(-Q3nob}tb5)BAvqvS!tZLDN_0Vq^@ah)xIO=4X%aDSla5Lv&f(j21z~=I`(`ETL*~QVLcI{F3B>%*3Mcm;0 z!K=$z6>H^BaW$bZ1p;>z{j^@9SH2be%n9|T?$KVya?rXyZlSVi0YeSrZzjiba7Ks` z&PsOY33A^%(}ePI9Aj31&$XHJbeI=G1+2!g)%NNz4az^!*UtdZ^_s&R+%mf!f7kO9 zTJ7enV8kRYZ8|PoFi5m{Uh1J%9nOE~&mWFOf(phlyeOt6FgnF z4ptU9kr_w}v|C?V^7lEagKZY0@Bqfhs#1SKyBNA3+Md|3Zz=F=<%FZf1sB1x*qN_? zXB1f2t;FI0^5|c|5N3OrDQ}?5ZJ9&(J;MKlq{6XS;2_Y;Kf4DD!vgE@!GsUDi6(CS zmNuB_FAyF4Kckz`{{@O8Fx!>si~9Z4DEFZ_AWGraaq>X^ zp2LTcD&gVb6bP@ggVk@}z6ntNTAH8C@rjGeTcP*J|Cfhd5??wNA^*!nFn^5bgu$@? zY_bt1eVe-bTS6@Cf8=Jk^`Cs@5d(ZNeoBt~KUBhgU)Hc20zgYx@o)YAg#USR*n+xm zUcJEUGIR71cZeqVtu5l|0k}B2k3YU`d@F;>7H&?V#TwFGw@0}Ahe5wYx*sCXzrVBy z>U|8k;CcywbaQRM?3<6C{YNnIp8JlsS4@S8Dryk2Z-vQ4|ke3Fw&qabgW#9cY;rLA6KFGCjoo zA4JTro%cC!HaS}@Afiy>0Fq6wlbyNC^J9tsB(hk3&;7O=&y_Qo_1U!mB+IZcQMa8R zy(z-!e-s*%h{u2F==g|dAWwlO1Ms({Ay&?Z!g@2EGadq4T6$B~+7$Te1 z;0OrDs_d!%koXP%RLiIT*{J`6S{6Np+lPW~&f6{fYi49Tv7R zwmd3|S)R;48;~L=&(f2ufR3KL^@k$e4ZFi^w@BbWr3oVfOnIT^?jHmDx9}k+@gIIV z7>@s_{jbu<{{@Oz7-|)wznVJ)K1le8fSq;=E(t#H`ggGehQUZ7%2#-umscuK! zm9`oE^2)j;JFeodRlhiW7Z;`Whl}i$;yNUCTA4Nxmiz|{A(jINesM6qaH?MdC=)kE#%kM%3+n=^#~AOL^P3184a%3K2xhz-Wt9N`yz8zi9Y?J$E75xh6%Wi=avRMarMTx1@+4CEu zc6BJt4!g{zGzNnT-4KeYwg(Fvo^>)K zjne|yzjjm7sBm#$=&5^ez;G80ZGjesta9*X;@LyQu~B``N7u`_a?hhKk4131w|sCM z>f7hrhtiN)IS6FSn#~u!AyCP4_n)DRa5elvB}LEby2;$P_}$*77iRYd@UE{>CQg2R zE$VQ;ih-2YlG9DG5|o$_t~B%4MM>Mzk?LoIq5}6PHcKYueHtZ=W_drnI!rgl=8X0} zEh*Db>T-JZl!my9GjHdFdb=_uiczG_NU3CWpa@c^{sdq7EspR4w44kSl(d=zX=R>kZ5AsG~A zzuzhAPb&x)Oh4V}bgSunq-Jl_q;KQwL>;sOzcRuQ^!txTG+qoW4-df$BKP~Qb;f0C-o&PH^u>U*QapEwF0 z2N%XxG7Mu!Y_|8A zm5Hcb+k-zBY)Q-vl8W~}5LGe7Gep^}Qg|=357;F-PK-H^&G*KkR^NEqD2$A3qzv0? z4%r69^iw(03nA}IN?w39p>Wsv^Ftcd8^6#tX@XlIx^PblLlgF<_v-?Psg& zh>$AuK#z)}JquFA)`Mr6lzEBpV6mKKpDe<5)<&*bOspDpO+7^XkJjIpOP5E3&GK4> zQt_4r`#Q)OpbJQ3{R5NZ;ne;oqXNqcSLhxp#KG;4EnQ}PpBXl-AvaX`Y0GD%FDu;$hdBwZ(Wz;@tOrS6Xw7Kr?A~awmb0i~;|-Awb*6nx$2)v(hwCl!)%OIEr#=O8e|Pf_F{ouAs^*Zv?(4en}*8at_3t_Xxg( z&BPM>)+xVK&{^~vul$*v!_h|F@-*|tzg1pa73^UKxP6hE#~0kxQURLr2W)sV+Ta6i znqD!-b>$yVSzvWn4g_8X7dNU)jhdc^*@%ENa!%S>)9aj3`tI`p$`~2 z(@7ZM4_f*G-(q#H+bMnd=Srho)u#%aP;N+GX&-z*H@Z!J>cSbFn31y!Kqt@ZXm z+wkZy z1bw@Tbbn1^Py>?b`PwZvVc4FG5|6+;*6#MXh~034dpnEk=Yw!ncC(@+$tSN$Tgx<313{kO;+H7QAZ4J6B?nj~99=~~>(SfTRq zf^U9Q3||V9*!Z&{y(?=Tk53+!RW0BCD)cqkL8odwAOWZW7K#aboK*?7t;7>767J|n z?XGs%?(#+lj%*<576+0X?(b63Prirgk*AlOzG0>maVjzED={~e0SXe8daV`{YpMc> zU7Ihk`xcyBSDSRYsaH6qAL$ZUNxM|?dzHOa6BKRxmCxkSVCy}BABN31nyC8^KED-B zXu-K;LW8?XBes_9%xJN!WQG)ZR@T^rRjcErPhWN;7})%7EA^$|DFEdicVYn_pNIZu6d&WOuLcpIl5wbXXLj1-%0d?CM6uYqmd7fG;Wp{S@o4Gjd|tB% zw!<6!K`2p8P0c!sd`aOw89-LrBN8uNAksPC5Ocz1NDuYUohP`Qua}Q_l%`%2=HdRc zdzgM*wK(9C(=AKUGf{U!#T@ALeDE2^x3MiTzP1j{ku2OxjPrM}CP-@iJnN6~#Wg1dsHNI1^HjBb*>i^JccO@OG}=|m8V&MP zB_8}TV)9u^Dx$;yBdOXjmB2U6-W_`}SasRh2|&!0pKCZzX=UDSP*U>{~sN%okKn+np({b+#{{>0<1n z_xn7etvpS|94=ZEasQ1Dt)HmMA=oWE@z85=n-7%%Ax@}r{pfeqBz~D&;+zt}l{g2A4e>q8^*w9a$RRsBZ1%r% zaOaOWU#b}~na}Bd+Vw@Q*rgRaDH6Ipb8&GoH8nMjgf^Y66*e|D`d%&j&W!Nni~N2s z&j~v=4s~>NG<&hc{~;YE;36Id#qIg&zVAyLv5pM7&nsJ7v?N@3ygioPR~Z@Y*iC4- zlj>7d+ZZzn9a%3-3*Wq)UsOBUSRCq?>?GQ5GB6A(Os+bxJWIK$t_?>SJiqlay&H#> z6)twQ8_{%qbv~?@?zn_$9L|!BcdcfaaD*Y(x~mI6AC7%@j#?&xys^$R)Am;AK93L~ z5gQp$HPqIg6;bUCd88gSWs=j!J3LGqUb;NABN${7b{M8lZ$7FD(bwNseSNDe&C6KzA20F zO8}W+y|2+lg{;H9Dugpa(QxIl=XUUiT2V~f$=zkQr)#h9PL0J8EyW~s)hz1Lm}pWL zp>az}E!J>NgsXbHjqThfp}KVu88lROs^d`}e`MIi{<;yRK^{DAkHh*5-u0tj1T{Zr zd9|2$3f!&McOhthD3OsLkNd?=*{FXak+zk+kvs`(nt|##9 zci&A`+qa2u10f=sA#u3i@6S|cx0+k3WF}b5f;_%BU09k(0a`eS)-TYGi=ihS9AE1{ zIrKAaXM2%&wk; z9mLGC`d-IDuMc9+Xp#u&)~F*US8G%x9Qu7+Uon) zvSsmTePET^?n|u#-sOy7atg~-* zXZ$|boU19Yl?F5&=pOiNtE)F$81F8^)8qgSasYphdEFRoMpr0%ZCCen>eMvfzLDUv zZu2TaNH|zA@&WGjGUxM>reVUJIl{~@JgL$7n zFFV_%zXuwrA^boW)amAB>S5OU#@rjCXA?vHywpL&3-AC26Br#Sa?2f`o`B;*ZX?$z z5|Q=&E{2GQ`bRi5Ro@l@*VifPE*|zIxi2(!zVWOem#oni6r8uVybVj5xG3bWJa9kP zF0{KXJ`b|V;`p4AeOZ>7*IZWH5Z9m~x&Qg3($1@lH?=ZqwBsz^sAze~7@xvV_CDcGV^l-Tb|-aV|%4;^6{2=deIB~MHs?0WHbnqv>K|NWjFE~TmW05?vJ_R zfET%3wr#bSgE9j5R}Z=yLdqL5@*R_~5y?%1v%@ZDOBzBzF(GENdPcky+PJ*bCZ>Kx zAl1g!O3BxSn4-_zZhF(tKbe0JR3gGU74WM*@ti7!O$3ICE{dW+oVoLty26cH$Vv(Xi)@mbxxrkv0Az}8V+?etT$0cLaLD!*UCcz zz5-xDWIFRwzn;+H@S*yGL#GRdF>j|zyNw1;H9qAy8kp@NpyNHGgUUAWxl`L4{W3FI zfY@^jpibN^s8KJlVQ4R`i2)3jmW1*^3zdQNOkeO-MM@;b}U zjqU-hWlW|ZZx!;GN{b~Z>Wy|xz{GiI%mvPy=ttnD z(@jId)o}#<9A0+HH2KwF#tAZz{%M)gU37=pnt?BKWJ2BGZ2n&+%#ACk?Bv%os~>oK zIL+(KJ$$UY@QkgMEuf?0fmZbJl!i8u9`ioFIcYl`2Xuolq@t$fc;X;WJ2$~*VijJM zDJkUX(EAusseKRdLy1Z$!f-39Q3n4ca1ktcX)3a*W*myfY_#{d3}7;q`qFYEMO5qu zKk`ZT`tkdLJhbS7b+eYyo2gU=f)pfg)p8SW)@ZI1ufnP#R1!mJDErkh<%X5d(E|^U`;=2{d179Wd|9`_kH~7z zhP4@VpJ(7CeCv$M(OQU_b2Atzb2As2Ax%jtjg}7$byMdx%cMX;cUv|z9uvbMTN-)Q z32k~+cI6Y6)!k)oue!S_8DZ;$s6>cPx@Z*GO8MYxO5OMGh?uKS>P>unSBd$-b{}%^ z9u>&R03bY5w^@p>DFXQ3S##x4=q5rpKffM-@ZmkbRd~xp4T-FHIu{NYt#)M&?6+HO z_g8-4-xH${ltewzYYSxjnn2U!{<*0JsgQ&nky40SbnFnaJN+=k4fL3}=iuR``nlr^ zX%$idss0q9C?;alhL+Sb4OSaTMvX$7e6Fr!7q+S78zGh(#Wzzy9;>{-Mo?D^*Ep-I zqE3HvcXc92UM>xkaF+oY-@hCoc?!5i2XCX;UtenL61y-3-8|U#5!#OxW8XrJug~@E zFAhYIQP>e=Pp|_vAZ;Q^LNkRQ)9*f0`s@xq9cD7|Y&yvCBjteikLsp@dZ$x;jyr~K zO^(T3NBDJliPv@=F{lb6V4S28`EL*h@){q@rbAsv_1L=mc|TAvNF|0?*}kygriCI;h(C)IBR-5}81?!s4|6t`4Pts+-}D#sRom9XO`d10{BYS7ag z3QGLTymu)aCASh(q;gPp@{u5P%gWwvoAP=Z{SiM?%usuDKBxp_$S$v}K#VUbm; z7cYs!^su911)*!g^dM6bSDfC3_fH?7g_B7Kmna%&dqGbRAAh)}YpYSY|308=foqHrxF+ptXTKP9lTvvAOLFl~KjYo)}b441MQ!zT)ekV$a*n zwTjeusO&0KTjL30sqDbrA}3&@@>@J*K2%nwBb#LC72?1;DgWc=(r)%XDg-RT>H=6X zUKp@guIhcS8!9%1yYD~ZGN$9D5<6Tz~9sD&fiwrP^o`}$Rqa}DMNIR)1{6RoHREI=7m zLYh}Tmgb}-?9-#RFX`bBbp^6++n2s?rb;0B8Dlj>B2|NIgBC(O@+E@sMDf$TulZJ# z(s+Cerk+i2xXws=Nr`R`{)FAhZ$%{V2E?_IiJg1Rz!n_H5gOGe#R&GQqUS6IR5>OHe=xd7$$On7e*ujEet2?GWetf5lzJ47>V^bubs<vA#DAXzWc&n=Lgg5Jcx9p=)iRH$pN&oVy|RRAbaUr_D$Ze5|4?sA(AE|Hp5YmbWYFASYmxxVr61{ zv0jdF$sj5<<#3}hSO!BU(fw>!Lh#ov+#XP?F0+vBs}6=#i@peaI=F1~xfljecD**F zp46GjGd|C!FO$)H$v1bIqLS)6H8;=#iQ_LHpYNg)<7q6-eQ&#aVN=Y`X}D4)cYKHO zX#GPPhrufVXLqBHF5xA6j;P2>G+6kePwkdVQ6$J(|7kHm!^<I}7>?jB&^ zok#de3~#%n?mhHSgbfun<^Ra;jDFqSM>s&4Jv~TT5dtPHP=SHuH|mENW69**yEGVL zzSE<uMZ&-;=kt&PJ<0#zOv=-Xn^sJ0tvEk#jtj?F}KMbNF zCAv}I@F>`kv~gG~rp3D(Slm%fSDCCXr38l4-DBp5eo+6=9U9=_D?yh;t-)Iuyf=Fb z1F-VJ)}_rqcSZ+Gt#ih3KgIG}W5PJFM|cFeMTjT;HM2gG<5mp2jBeJtdLC41hcSu*q^ZODzvimTgTP4 znXl`DpZTaQOxv@X^;3d%cvuO4VJ3{uALdnpf??5~JlAS8`7lQg!B0w@2Zc*IC8rvx znc|qreB$AYbqZa{h)b3Ntq!x-@dE$$M6vufm5lJvYF}Wp^bs&6>uhH`kBq4uUT(K) zek*ID-w++^zc8gAGlIXegMMQK(n&EGhN`jkO7hRH*NKlcEvqcE_YkOt+X-;j+jj2PSzKQ4RgZzPJJ?z6u@BX3Iw&3~gP z%H%LtER?VnKL6iisf-sac=2QLdLQ*6iz(;2MNIK3u(vZSO=W#l?3uyHTcYp(Vb}kz ztsm%W@W+K%a)KblZ0>lrAvx9e((CFl?3}X?q+=@v9E4i=-1(f%@<*bjyBfNH>oqZz z%O-6yJmWa0D~D_Mf+Zop+rI&Oekm_8>T*hp%xNUoH`zE}f-3yHD)M$GymC9=Mj-Ak zb)vwCYE2KIn$Peg&Xe%H&EqRo>a=XeA!5ao?z~}|Y@n7tVk$WE(z%2~GYH8Khg+X;}sBvk} zPGwsfk4f2`PijUzq=WY>RWD2z2ND$D_8k+R@9o47Y1g~;6x6IiM^jT=WwJj9Rb;Mc z)qYJ6C)o71{`j@3L^c!6*`(51b!nE-72nJ)VFyLC19Lf~k zc_L_{8nrJLQ|IW7`tlK>?Dm&{hOh~IE@l(IFP^3GMSLo`19u6zc3y=gf_=zTav%&G z@l4x~CJ`bC?#A!nuXH8&*24WtKeWo~iRZ!0-CXt8Z1Cu1KT=#*WYK3M^uVi6bZ`TS zD6k;4I!lhE;+<-ik4a)o7+eWA?B2hp>=)LbMNr*7wL(IKhnL_%W+Z5h0~$KQG;ju| zj6p4ZaZbt6#2|B+Syhn@B~EsmNko>CugN)*LP<3%s%vU68LxZvd+^)8aYw6 z3=4)A>6pYAROEpF&_nG-Ve_fFq`rv=;?Y7(+mEmwGG2BX2Ds~O3C1)wz+k0*_b(LZ zm$-*{Fp#Tq7n!~S8z&qruQG`#Iq_BCv?TDfCD)gwE4hwgd&jGGgp{WMGrF@%_?(+=}4N5KtnNnDHndf5!!t4M#sJ$h4q(deD(sM-W#>vu5| z*a!`rVf0aDVi2aXk=9yQ$(o(pl6m;$CpN9HZX+ZUj(gn)XxK{kHCS5!QW;$gQ{t$bT0cNOwh-oIQyg4 z)Ksf^j7ioo>7Ba7D%OaayWvp^4B~0(_DXa<-HpbwcKZ)KZ0CDlbwAtUp{dA1wb8{+ zaXmUFbvz)3D!x?lMhV6cr&I=sy5j=vFZZhY-&b~AXONR*( zgOEm2G1RY^N0sEKT3HvSQqmZNz4F|kTjM;4srcyeV*S4RPW0yz(rv*PhXg(PFT_&X zYIR%Y_&687u}hK4`ruBIE{*Ph1#w4ABj%U~OFh|BLNOqKg*M=i5p z8RNP+Tk=RM*!^`1q|oK&TSgSWPBwAP2bci0U{`)?RRBmIF(kjLoAye~BLknNrvkK} zFVb2r2?$|mb@~(bB>#i}XnF(rK{WXmJC@nbmn$E@Eu+0ZSS(st@O}rJ;^`MS)iMCo zETR0bU>Flk=V4=lP7-Hq%|AC|g+E|{5$Yb%uD`C-V)?ORdfr+ROscnji&CbJxd1Gj z&1Chbnfd<@^*4Iz|66Ot#Hu<-N3Bj{_B>WbZ$)R z&|#p`H*zs^AlYyvmSS>lZDm9Ls?bg?pFK@wN zr%tihzQwDIso!r2IB{B57G~K=I@mf}i^eL4rEab973-*Q!U{V}l27M|^$h~w+r;}u zrT4Da`>ZVGlOPmT2--`4>~aq|rZ_N{{DzuuV!Fl^dM!UTvZp(7AE_OQec3_!kg z^Js0j&{?RVo|wN%p%~Jm;mLN_XK7+LyrhDkwl8-uKi|{8A!T}*c1b`v-91|t5JqmE z^kYIP#`5r0z6cr5;dd2^(S zF4uTVMdU{tR)sVi~Jr2gb-Xyms#}lb|W3r*?7yWzjcVX#mHR-e=mTggko< zDoo~#i#vV!on>T?gHXX;J(kRaZ_d`iksD-nV-XD!>(O zpkcAX@+~&#D9p<&Ad8s%VP&yBo}}#22Zd6LP0vnF50Ce|Obzko>OJ~;1U0Yg5}y1~ zDLJIW>*Z(&beX^tP-*oLikMCA)EM4+tl>DSk#&B+%PAu2>+pR<5!&qT^2TyU?0D(L zbicRHP9?!-Xv#C4Z_V!>d~=a?H(!j~tf##yUJ;lN27mXNR1H4Y5(=FfeB z&PNd*+er`3l7ia`Ut5P*#a!~zY77PDeUWZR`l2Z#t~~a1>)@oS0wN^sqjG0X zwzEd`@#OT;MmPO&(|F;Wf;#4w>|ifcGzB4QVI+i%GcGR9wO*(0c0smvsjhy0s$5tqd^x#+ zPbfCq%^bRn>uxA?DBwDEOPy<0=Jj^YBlSv)qLoP{rsJg2_LYw8HcgTz4Y{2ALy`m+ zGNEtS4DxM6X(A3?t+Dw2imHRuomXxshL0~a`VKDZq{8m-_R zQ6L0$P8eFSF;|@VMjV#tOceptkg5^aejm_k5pXUn9AQ{*B^h#MJ^`S|HRd&D;jiRC zY-}_gyS>t~Jp{*gn051ty#!==JZ&nGak`~fwj@jPig#f=8dVaWDzW~U8yecE7pMMi z#*RvX+D+?_+tooR^E`8t6+-*w!_lHz?pQ~5VaBQOmve5jr9?)ou9|YR+C5iy4KtPL zM{G73GRnf|r3s$4An(N<Xla+7oM7dNZb|Kd&x+;W{${F>N8i8!kspY z0wdyuJWMzK?r&pY56NTGd=RL++161-4Uv-mW8a4IdgpqRjg&q(I$Q=I7Jstg<6(8$ z({<84L)12NPIqW}dg#vneG8z^-aw} zHUR-qOv^!^BB^@FuO%)gEvpgCr+xSi~8?8pRP*wb(*S<)z~DZK85y(^cN0u&1*?=aa4 zg+=O@00}8mAuK?0Y9R-6eb=J{BEksDk2>3v0Xi~_iW_LKMo#ugn*(Q%6Q9Dlqp{pS->H3tCuk&?3cvW)Sz6lJ0Qzs*o-h zfwrzJuY%~GAVhW!*qC4yc-z$R0^C(W#P1_ftk*}m9{0zjuphF2*v-kwv6cr68&uW3 z1Fs&YU#v{%5F8}ga72MWa^mOnj??;DLiokkT1If(28DcUF z$Tbu+agu_gMpgepr>QT3NQH9u;E;n^NC>=22eO{&rd~XEd&+4TQB>SEs-lUS;hFw zav!1D(xZvz@B`V3yTa3>4wk6*LUc1dSGvk7Kp)Gtar(spFBR9++NH0am>%=iP|>yx z!lbYD^Z~P9u*-1&gv~>I(x7gM9XM2yL6)fP(45Krd5|~M5U_C`0Qyyrpb{t4jDhZ<7 z{v$N+du`_v^D!Ow(^2ze>7{A>HF~l>z9f#(BnYNQ+pRcMmYylhe4TjJ(-Eo4znwZSRm$_q}g5OQZsw|tEF9W ze0$BVz6SF>Xjk6j&T$1n?i4$AC$Yep-ez$Jj3_RJRVKWiDAu@0(6Ba8I{f-ol+;0d z)cRde!6JurWgST7==IET ztt0a_USaqCJC2CKftr&Jr#9Uh48)mS)`~3=(-;e(tY#`Lyu56}bi=84D!aj6z5+k@B_ z4}9U-disixeDQM|RS{6Zhu^sU_HA_F`AJHiBf%N|^z|4-pLZi_$Y{y|=O~c+dIEkk z)pP9a{2CEKH3JChJ-%;`(K${PcBoR7^A=4#ux~o1;VBlpT>>x!uY?h#^*F2FfcAEqLBuR#m!GYlcJGy8}J!2D2Z8zMS3h~fA%GXhbt~c z$=&2qeBaRP$vFA&~n$pLcI- zy(9K!8AUt-h0YqB#>r^>3JyRYq*Wf%lp2WG3%!xCEm?fJP(L4%?Qc@eU(-`QFg zuhM!o;-1ac?zevf^zBA~oKTwv48PUgl%x^8_G?Ob8YaN|jsZ+goD4v%X!B?*ACp}i zYjY~~a-`y1i-`0KWS^e9uf-bZBSzMb%+Pv zHK@Qa{ND2kw6fVrG|!!=3MF3isYIlt&U+b&yL$gR5|^5k#ZGz~6w9M7LAi+4aniv+ zYruBzRPwg4+A&n#>|*9JF&A??0DMFg#zjYh^8x=cc?60P0l9ul0KD1vK7=xmSO8Qm z>H^1a^{*G8)Bw&Yo$N{k263CE#N2CL1`)}lK$<;t-6tced7Ml0J8ifQF z&8sAI;aJqeQ6|E4xGCeaYC;Km-F#)KE!xbIISH{^AytKrvCJ+G#jXrVM38{2eDbm5 zlVL_DNXk@_AHlgi_ok${T(>Kd_J%HRdV5S+ZEOnOexLnq>Q{`$WE^5s@N)A|cm4-B zDqhTkZ5uz+K2k%s0?bcj9f~!gnAPn+7Uk}36Y`k#uoLL%72shQJc6kCv z;G#VheziQxL0dhlcpmYqz)KJr{Y>Ar|CZq0T@P!ohXkLG`xPendDYMUKeDA1lV{Z?2z{|s9FkQ@CWhXu zyH|vE*;5NSf0IBhmBj~$3J*{&ATpsW?r(E1nS_Y12IMtQrA!8Fn#Pgyq4)Ca?M5ld z(gXEO0QXxY%2j+6Jtoz))X_R}BHWaZ5jC{mu}7ni9`M^dr@H1TW34{4>9X8@Y-o)G zet>d_f+qQ@*w< z3=F;!+PMK#*Zgs-yQ2heb91&#Kexr94)2Op{QmCa$B*X~G=IoNN=ZpAjr81!<<<0b z7_H5hSAD7DY+PZwPPUON+E-vq&XcMRDg~CDAVRlprIc%^si7!N^x5q1toNBBYxro- z_$Y!hnT*kkwt#U3^Z0@)roM!Vk3}rib&+{D4|24y=z=GJw5Hco@+WAk!(zL^=@;>ePt2$ zuZtIjhm6Tlyq;3p-?@oXik9e&Q2C15gFkY^t@R9uFkY9UiFJy(L~p8;G+RU?qGs8S zCC|*@zc~Z%@_a`QPQQ6hO>0y)GUlZhgqn*k{E7bwpA|mqagkB4)xGW2WVNunfrbyR zP1y|URKr?GCR}x#zhQ>DGBxOYp~7p9EbczAoQBQC;jTdK5ofV{Q(cMuokGG+;Tu@3 z$GY+^-SY^^%H+IB0s5f8ytiF+iTubMjBZu7o|rUxE*enfmdE|bzvU#VsHn^@nYX=? zK7KIdj9|!f-s9)GyWCo{+&ayJa6a*%SE)YPMRk|8rQlWxK} zwY^hKUY5?8;#BSGg;VfsUl;jCTH}*YS*{HAh?%`>y+{58r#0fi#%% z1#x+NLkXYo;?MEd0=ErfYsq15iYv6k-B)K5wtcNDR44X4;^r=4WCCUf$@b__OG4?zGlNFz)dm8GPJ3S%D#q*mMXjMKQL$c4v+U^I=l^X?@SG#Cw&QD!4LPl8}~O&Q1W& z_m&RLs<~^;=_ZPlF28^POr}e7dTim@EYG2g8?)T^dKqhdtatoD_Wg206XpN~03|=R zliI>sz3O)wjp{`gLQhtHo%;6atT?LsxH=7;M+7T9Ka$Y~xq=~=0w*05G3+e22{e-g;lD?GyZNuY+$F;WW z>94+=vGkWV0Nq>EE3*9Vo}5Y-U}O4rU_^#!%LWOmo17`R1SX)l z4A0JE^pj!twE^{% zMbnj}a>k`#C;D4*$AY z%AML(CY3o@VD9FnN%&_Nh7sdI>*c5x)oFfvKsP>KR#3yOH#Gw*{o-g4YpOu9bLYom z=MII#)GK@2w~czRU|p3SlNz%d$!|q|OOKwR=@&qyT?XVSKF;qP#!t9QWG2pj=Fwgh zm~)-X=*yoXJzGo66#{%${vv0pyEm=%>% zlUGxuvuHX9N+zMKz>hL|`ib33endJnP7zyrW2)2aBM0kZw_3i=XLRJc) z{|Wa2-O7&1=S|C0uW&fe5vPXe&0m6+$y#71fDa-fIpCZ*1ME@|i?#eqtx`TO4dJ;L zV+&T=?@#$KvH-G`;{7hwKiHTi8Nk%oZ#Yr@eT5aAV2ra*V*l# zg}(m9`67t`-b;bL!v_Ox34=>|Ug*1wn4^c4PNoJ%Dr-(N{(VIO^e^1mA#|t;@qdH# ze}nYDzmkd~;oH++sZXjy82s8ELsHgx!d` zkg)V+h_ExUOhh#XqzGLdPY=$kpe4tEAL+xcAKb0VFv2BJxyH|vibuPaThG8;>FG?4Q3XCW*4eM$1Ykmswhh!D@ITWPJn1rG z6B6<-lsF%NVb@PLosbiL6jczz&XUv0`HGmuIQWJpwGUsoN@LdFOshM}`87zos>nb) zgWKx=(`*!GYH!a=z<}rRcv}iFC;Ru4F&by|j}*I;*+`67k;=L;adDn2lS*px2l?CP z_{3vcY8VTp*UFV0q8E>jua@(`8Fc@wWQShHy_!+O5&`c2I2S zSztd(HSe9AzZjqQEv1m$S{{7B1FVHxf5v64HDd*4|NzUS7PtUiJB zARYZ53*MJXDwk@fa%FQfLd91}5I%0i$BH_cwy%3Tl{r?whb>B!m8MP51T~?f7toQt z^O(XUB?VB+bu+t9pCh>WYP1;ZtpvksrsnGt^K=IiVBD`}Kg2mQ#nke+Rj>bCXl|bJ z!PwbeA$*flGOmN~{n!`b!K&O3dAqO=Be9_@lPl?7j(m?jEfvazFItFVkch9$GkY{F zIm)(Bi0JKR6SNbX!KL#frLt_rEa)ZAVk}ZR<6(F|iMu5%$VmJeT;kBl(NXue!ME%C zcSUI5!lDSGsJ$FFuT7s@o2)TTaM5QLa|yDTiV)RmLOPkMBx=psEe+-^w90%&qcUYf zQgClSlCBpB8+&YGB4-7465I4LVe0!IFcco9z&8w>INusOGp;7J{ZeVM)u->##*3Ik5Ef$EO#p^_;ywr(Q|`~2fti!xBUUYUkL zYWoq*;n$|rWl9hGw!2%-%0ZIi zcU6LYPy1khVnQ_3V?+>XS<}Xkk%%!$UvbnPX83ewEN$-)VjCk2QpNS1pp!vn4!FUr z27lk7x+0Xt$DR*~`Pn=bH*YG58-0wBOt|U9J{*rV5TVs+RQ`Pu`6$zX*_oY&snbc!2t>1Al9=g?EKj}^-+Qvj zKCAOB%9HhbK%%j!i9w~kwpZ^h4wFkLSnV!NBO7AtvD7L~y%)xc=7-`&elllFbFeEYqY*5MlEy8vvP-LVRGfpDS|;qx$nOa3E*`<2O|x7a0yEEy3)_C0Pd zQK+kspX{{sOcL8drno?sxWz=5zSI*s_->nvh@a@ zo#B^8@B)fT{P0J#dYD)gW&=(1k+3}-bGwy3U3LmlFdP>u!BQB7j(oftHd2Djx zt29cvhZjX$Jh)1VKIFs1gXAL~^I_+3>KNIH2e2+}BQFMRyT+j@a7P3OJt|+a_)vp|S#$V6 z^c0o1*nXqzaM9ZF{B{Qpxgorf#Hg8gnG`ubNew#A-4#?`I5PcB9dL6)L-F*&UNiSY zuz2S9Io@CVgf3R#cYEi4G!C(dwXzt(V0qVfajiMLyu1;wvM#>8a^mQQ>_=u746A*% zs=#sU)~%;2o!2ho@X!RY2VV^LL#|0`gIK7{(4IddYTB-1D=?$1}ePWxuL>s>ht9;73QJ{x*WVk>g;8bP z2|3*rmXtpid0Gu%nLySD+8p}w@hUb#v*CB0Fjq%Xj+rATR+LRzZUPZ0dq#WoYXro+ zh7vK3v3BA+dP3%+&mhJUulBV*+@VDO&j!Meuu>HLpYWr948##kBlSXm#W$g0b&T-r zfR-m+B;N;zY$7~5-|C%2JI%DM&-d{dXchd`-euO@CuKuY%g=sD)0(^C`v50X6#-fCL80$;jjg=9cUJk=7=LyDDdhilj&&6v`gVxMso%7rk|BY;px;!Ni3ZM?&%5n5fDrxG_UDjVRW*W8+mJ}ixlSUH z5aS)cVf?@-#H9LSbualQIkRT0#;Ely_mo!e&Sbzj~mjptPCb&C|6B69ro#0O6?rsf)4(@K3 z?0xn==i$F&+=n~v(;cID=(Wn$tXXr`_kBf>!UrilYzk}u0D$*ST3iVLK!ZK}6=GpL zJX!ZWo(BND0Pn=#s5s5-G~L%wb~|sx-v_kp@C(r=YJPHX7>u8_0c)DZ&YHZotKi=F z0-%)ue1$rHz`MblyrH_8%u5gO*%J^$Q9L+s?2C|JeIv5db5oa27-n zG69s&3=H3#nap3u5|g3IIdlGN!evu%C@d-m0bGeice|iW18TPX1O$5?y^cL?-n{iV zJZConvipO7M*V9<_W-!4DhK9>tnWOspai4l#f0I786oy^FiGs;#PVt?oyBplsH^E? z)bV3-lmx``x#{WH{lvk^;r*rcF15)0B+dKNa*yK{tBT`)%;kRHMyE#ESvFEcm#Ke0 zM2zPVfGW{AAVP9SZAyIFms#q6OpM|Q)??X$$vyjNuJ=rg&`SoqK;wBZPBht^0=ma1 zbsiu2+xCtVEeF5RFPBAJ5h+aEeb=5h*EsmEKt`PkfQ15QZyOBGA%a)jL4Y0&O}K>d zM?A*lCTxf|_&T;lp;0EQq~8m-x%yZUNW6Eu4{_Zbw5OknO+QZ-Y+_CooUv z`i+};8rekaosQdlA&sU4SHF?!EzPI~u^_etom!0eb{4RB3`d>(6sTjyGXqw&Z%zD)|s;hRS z!BT&~CCa%r6N4noph-^U2~HQE8&`0v`_V=YR4MkF;fQ??e!qtuz1?S#xZ93+FvRd_ zX)nzbTd%&brz&Lei#59J3aka~8IHcqugB;H3Y&@htkJt$r}IkASHU7nqHazeDeuw! z7isjSJ@5OGB|WV6LN_TT`b*AtEutP$9{23+E-I-_8QWLhQbXIrBd^B$Yfh@#_?r)^ z`nhHsvr7^3^afX@g6c07LW79S@7+5mIdI)i=hZJ>%KmYV^Z5I2PglLWVbR(=*w*cd z>5`hwh{UDDugn(PH^AGuOMgqx)edbJ$^k|=#A9Ie$~0Ba&ERg>^Vrh8(etGD*oB(3 z8qQfxV?;Nt0?qK_9rl!`f@t6tyDCpEzbmORqvz1WdfA=$aM9p#S@F6cYb zwr01;r3n{x)4PjZPMM~e@8(o-Stq5gUIRPN-{RkY((|;qPFJ?xKASgRq6~+*d$Ns6 z-jA2E*Nn=0JX6%GqB`rW3Ki`(%@h2N0^q$#di~Tk`Ap}0&ZG??aNTyVpo@7TJb`nk zf4?|?HMhe36K%`eI8*Q9($o{#UOnvXAamaK;I4<^Zywzirm+URVG*k|7ctb@bPhUX z%5gphqQvNX-t{%NBk1qP8z{;S>=5*XRIUwBEDaee%rc)30*E`&Tr0^WOymcv^Qvl} zqC`Un4$W#QC1lvJ@7Gt>VmtV^Yd?@R_9?1xnfnd@!d1Pxrh}KEl)&q|%OCrL6#)PU zh`Y|(6sl|WsD0i(Ps$4C~0qrek_bI<3N7mkS> zJ}cdfRG%R!UmSg;0{BUNSLt{f$YkNGsJu+G_)S#d8fD`9!D0Ws4b$?>fxTe*O_MFc zp|=8sgmVmsz}Hewy&Dcc^SrDJku8fJ(n}wjIrmqG^SyN*&FTFLm=nF55v*?n^UY!X zYK8XKf}%}JdMM5HJHn;qR*=ir6^S@G+EvK<=~sBL?LABmakty#HePqUk*?vEukZCK zCoXGb66>k+@VER^Z3cFe&~^jYF~fzOUoUK&EIZdXHO4Lc7Tx z9rgvUXYlr_YrKGE=jah+-))=$Zvt-&bKXY1Da=KbL(N$~r-AIM)D#T?wQ z<@@zgIN!Lu$y2A!y8%*2-rc0AGrgn8dckqWFaERN{+7w~I6xG4vS-0`*}d+DCT~1B z>N#C0HZE%>s~(f|1yP0@{FQgl*+HmSD6l~0;M0DW1RSHPi+fK1z5xLTD%sm2bdJWH z)sXumJnP3mdxN`+m+nH3cAUuY4xDxe#V~hTbRB+Kx45ceUSdtH-5wH9&eOGaJA2%o zn76&P>NBs9w$F+{PKeps?5A^?@?p-Tq+Vkx-F4ZkP5Rg4j-(YfX7pa@t1MA?_Q%9UPcxj|!LR#*Y}16x!_uqK zUl_GCb{Z;tm_WFC(!3pPQMNo$d8(e-w6?1dA**iHD_g2RZ3*MbY^)_=Af3aEFTY3P z=UzE^+1!MNZWa_F1a|9&s(O9k_UC8^t4U<#m|~JmDWb0Lk1=!J(NzYyq>0P`MZ1|p zOI*XHCaNU#J2=Z0_fu?6FH10M93R1YFz$VMZ;0sk-hw+C(9?@Z7fXF0UNm~2pWxGk z=^I845UjpJ+3AdLzaVpll9@iQRBCcXf$6E}4Rp&|S89X(mX{i{1;0}VT((ax3@~lg z6Mr4i&8|810Y@7p`7Jx3HliaD1COr1_fw ziZvqFtc0SeDP}IzggK2B&)a%E{Gu;pz0;BS06?)eYO+1VuTvvDUB{;htsU~<7_z@x zOvC};{f6`D%gIglCD_(Z@vF=u9E(OHb*9mf5E2`DXkh4esheTKc7%r^Z(K_$^di@j zsCh)hn#};$p|{Te9b(_BSrzi$=EiAR2po!hg{pCm24t_T;h8aDFiCQrmj!n$2n*@9 z4?AU`erq2Ok06evg0EO1po_;QZFD5bL9m`ryNMt^LQhR&u(Yma1SJ;aeK`mIJyG^w z)+YiU-g_MFOFktv)YJV6e!cIcQ{_ zoA~YZC_huezN5Ifs73L=ub~SFd4}d+pI$WLY1v$(;DH(G~$swxwnCx8-SewI3$QdIJjEu2d!b z)UtG4%M2EWJo8V%;1vVuu}8XXx>s=3)_Qe;DPx}okQ($bR=@KEDsvV{lkF#JG3VORmm6ilaa>g;nBWNq>pdPJfK#3e8g(w|uLt+rB z2Q6-Dn6aQ2afN#iAVV&XGUthKtQW-O5enRS_i~oo@P==;F<$d-onXNp1s$S?GJSE0 z%&agrQFjqE8BAL*WS-J3w)-o4XFb^LcJ&Kki>^UKVRVTt5J2I_AK+T2#TfAP`C8&j zAp@tLgr#I#;F~Hy6;Bo-KpYM2==(nNz#JRv&X3b@aXV}f4$k}lskWvN^{ctPQ?8oN zfa@qCk?gjtg;~4}=*m6{i%_{EW%SC#Za&WDp6iryD&!bLoI zu%45bG+Ai4kztT2>|O8r3@b9>Xg>%*Uu`Eil&Q~0gCzhmP&2^f`PF_fxs~p$8}0u7 zrl#MEh+d3C-KHTPZa-(6{G}2bSTo0-f*Ot9L`EHK(y0cmo#f}hCAF-XhGL~E%J~@< zze6D#rP-T9|D47GmJ{@7dDwsN@X^AM)txvKS^An!*d>4Hj9`>@^aud=E%8ob0bAbmPuZJ1Ail~Zpw%F`s0 zX2wTz`57~&-?hkv!%(Z~r9zuw`c zTJ9{32}E!_HVIrNKwAvY*;=s%=p+%%j;?bkzeb0}DfvBYdUeUxIH||29c4-b>~PQ< zfZA8-Pd5qvjp%n%-jELGL+CVF*Xo++*>tSEP+}6%<1|lmod4a`d8A(W@4lH(B21(3 zG`BVNU1p~bFW(ZHgxRJwtSHolHeqiT$Fijr*u{FHb8Mmb)NxVqmRGw%*d-h?zNTCDk z*w^;%hg-p(k#T-uQNmsX_9y@&LgyQq@EO=7DS*r033Jun9=O0jh8lVI>@OUC)#eQ+ z%R3N25d(^$D)>#S0BBe1{2Kfr185xwVk~z)mrQ@kI#2={Sf4f!i1q}>P?69CWzAoE^_F9rfZ1rZ(M#Bigv9e*3?6Oh*Of*F zO0Lkr+|}r16#hISE=>(XqFt}!)ns>#g^gSf;|gZ5UnHXAW)QaPXzy&}Hh|LkO<`bJ z?k$@E30%TvNp;+J%5vDg?ICrU)zvO~@N31NhR6@S!Li}rieNOEXQdqS z7BGy=a4LCDqWyNEab8_BZLzGKeGfBKx{-+acE8s=d)oxxm@E{96QgbY8Ta8f3$&7+ zH1A!@+_QdK5BbPzy;3!qzloZCVUW?UuVVOWKRUK42~3Qi5ZwSf+0B<(L8v8VlBhoR z9U=(f?IC>LoUFR>dOh1-79QkFsf4t@?85Wk_^fG*rpL?qP%t*5m6=roo4Kay7uolw7{RnZuG6>=m)8)J&RCr=aXE-nkZ$Lfdput~Iv?}TdLbv2F zh1Ni^7}`-;$l72>tL~GgnX3Hvr!?$rXBQnf>h{8bs9vrdSaivF%C=4-3s}cd)Jt%V z!Roci5=RZOX487#C7}6&m(q!<(_zTFO0bOjF)c6?m9t5PKD1Bl)}_39Zs>AdfI z0|xL%NQqD*Mr-FWy79+BN;l@Jg3n8WPmR$&oEv8={Y_@MxOUwLnv9LDctuKFxPGM* zvcNF|bO6-8>B3j(XZB(Y7^-HxEoi~sHLBBOe(k|xs zu8`>>E`z7tt3v-!j1T;x;{%VDd$%x{Odm)rbsC%@Z_(s6sK%U*`W7X#<>H#uneK1W zY}4IQh~_BuP`sLy(pynakz$Mmj4O2LmIlik&*d&fI`-kY3L=~ z{cFngDw%XKZbpV7+0j4VBXkMh%7y2(-dm;?{f19HHf587X$F7u3Z}(OXi!c|f z$)-4OkUL*(=2jgF-V9oMC?^c&@KE4gUQM^KO`5Cr{i?9Au=o${_h4$rR{g87Se_8? z-queBJY`w~mh!@<6FzNr%Xeor(+|}2p%Y7K%U%7=0ZcS%TF=#Tqs<`A{cK0`vJKyZ zmc8YsqVZxi#HHKXZH@Bl;i|+x#{GN+B;&v$LHpr7*lK1_6*sTvemaTnyV>fztp!K; z!s7)Tp+K~S!`?{yR6_CZ zGLig*E5Lx0gM0=z6hkefj&;!1piwrKt=O`i?w>kO5#ONF|Ed2}0{jA`^iOl)Z_tZM z(d$Y-NN@(jUnLKd9EcK7{uuU86-9hn(5p6%$ASM_=Mk{7m4grQ?LUnewotqwY|@bb z;}}nXe|Kh&d>WelDM5(}-PJ{-#xewP(qW8KYWu`Qr>3JQd_ulARXL$L%Ph)dWjX)r z<(MNbV|A{pOW5pdY8Gc*DcpwVdy7s;ynZ7My!c0rAO4<=3vI12192v;|J#^;u9JOC zaFR?vU-pFdn#G>j+-@6-!YPcHv$#HV-w7JW!i_laRxi-sSRPm2tPq|#aTi&Z2%^#Zfw?Vj4B@34wZz=}-;Fxzya_9XE{aoKF6r(m+ z7Jv0Sfk~{{1rNvg9rR*_eIf8DExVD{)V?%B*|ws-k>x?dAMtQuvu;5fODx#mUpPl; zv>gDA3mxMDQ79EmcW@qxFA+3M$v^sruY(nYz0qcvjyZJ*j$WXV&B z6Pd~qWn^!4wyq7`S(KAxG%Zd`6%|CM;am+^N^1kYh~K%Fn}#3TvYxBXKVaY%`?^>TaH{A^}x&=0sF_tXGr~B%zXz>D`x3BLC=LBVK#bnTmvk1WQEG zA0|AUQ?pC)uUwdSwl#0RstWtOvz@~D?JID+bK1d4czzmuoQ4Pc2r1H1K0VB{`d~aI za;(k2268^{YF8ils(W&4KA9CcyT^a~3Jtj3`?TdwgjsU8Y71T zrizMctIl5Ta}wg(*H0~;ZgY*y=co0t4FCXdloipX-xF^>x)ZO*1`cRpz4dxoIY!iR zc~_h5!z@b?IMLXwB8P5Fq2q>*@sTr=#T&?n6aG_j%gIos84Fe*(kc1WE`0#oqu<6W zvCo|45$sOPT3g9$D<7@Fx1OyAC{kmV`{KoGRRS%lEi+}8myd!EIWvY&dL6N0{+gFL zw+TMq9^fIp`_C`g%*v=QIlh z_#A(K0!Iv$+AMj5h)*daE3^2Zr@S&Mdq7Ocl%X*y44D(nyNZ2b}9?c^P4Sf9PWswrj^i5eO$n? z>+fES{B+je=)&?bmjv))UkE_Zswq*w1#hK%mzh^$&HS7@NknR?De{i5rw3GGem;5o z2?yL^^Nn9+`F$pB%l)Q+;4bvHQ`(i&yDEA6yz6^&u-#-+nnAHhocnA3QOEIXkN@sH zeXtymIV?>gLO0YSW$>4>w?`#|oj6MbrMSPx}?o5bM}vxjZn2C+RK!PTbV{G8`vl$Fw=K6 zGo8j%+lt4vF`TUhlJRZF-_#IeR4U%b(LCoYvdf*G8J=g?<&n=Bcdq6po9^T)L{z<< zN11W?QMO~QJ>4(Od08T-D>`aY&=S-BC3WvL;?#|obIx+b807~esO&_ zZ?0``>xc#4QBOBQ2hb}lAC+w0MP7Dr*T3^6-*$b8@=L3sLj%(NwMna?%aC0;3F;il zGG~Z=m+7SDrCNKs2N^{K9HvZggGjuDmip%fuhX1lHNUMC} zt)(Z(+F4G_xmzuVLCna+U=LF*ml2LHNSjIe*l>mlRGpVZsTch$2_Yv<=G7mevCmRt zVp`#h#uL7I(q<9o&d3h(dT^>swvaIw!taSUT3tHy7#h1AYYU z3tCTKbBuu0UQ_vda8k`R$uiz&u$&1n_cx>xG3VUf>jhb0R-`QUftOKn_u0JGp=_u7 zJEtzufN^1O)T*@(-&mVU^G8a^eu~~W-ih&rI&wvVyUx3*7|9JaW4K|>a+1bPyAIqi ztcs~&5Fcs9b6>E*JO&7xDENd`5g}qbA0)6E?4|j<%(O{T;8|xo}d7X z23~E}uXB3I{5t5cx?eMNpa9FnXx=T8Pr&1?$kgOM!QmkU$NAC>Cuqa{^*Y>eCIT=2 z1U~6;+-y7s4I~Z@3+UiT)3n(UzO-&fIrmUN+kYYqZbyZavJ4P0cn>wVG8jz1%p z-HzXrhe>|1rt~A?TAwP`J{pr{?>*g~CZ~NkWjq1a1i+oh@)t*rJ2_NrCh}!%zi}c7 z=J|IydfHK6WOumn5VL4W@<uvb1@XfK>$@(mS>H;ihGO-(IkGe*@b*7Lk3;gJ@#%N5KANVm%dGZ`p&s%25d^>A=Xc^;PKgZj z0baO=H@X@>J|f*OZ* ztGH%q?AxDH<)5{HJccO@s_Cn=$=1a;`B3}ZnJboJm}!Np^i8$P#QdxY(Jf5>CfVy=x@_|BT2Jzr9ARK}#~V3j(v>NKt1JIzyHtwycQzcPy>73kSD z37n(?YC4=UN#3F1RqoMJ${ljCGgtMca#$fJEDUecq2cJ<^Zxvb?67>vX$|kz(Jg$J zb|*v<-EY;Y2Q>rt4{*7=Y4^vu(gEN1v$qs&Z(TRDp@pCR!A1KKXDCmr=Pm1O)&iE! zHwF{y-Jv==Ar2w_Eo(xhjYoYfkl$&^S_O=L!%>k*FOXAMRP@WN|wD7x77 zSxrl^Cx#bFAHX`6T|b8A?f`alyYH#*8NihZN=!_gGbMFY428LWQ+(9nzVEs`{;PFgUgR>%iU>fr0_+ZkMK@n<6}o7f(3Ru$ z1+0mgRU%V8^>W7ZMc73Ww*oMH%| zMYa*Ni{k86;nYKK02QL%Zh_lmc0PoWPR%YJP{!z5dPWb0$mxV~<#DFY;kW4*pjD)4 zinHWvrtpy4QMI18T@OM}$MPXRR~nf84@i0fWDI@=Qu2cnL$PQdeS6abST*Gl-%%^f z_9lS5jZ_q`26nfW8|6XH*!d#}4gz+K!E#J^FhGA6AQP-l4O*>bP{&Oo;EJ{pOdniL zFWj{K7hydCa<32&5Tj8HMIhqHloEJaaW%!gC_<0cz&#{ymoJALPnye(yyn?Jt3rUc zCIrV1QbbVd+9hn$_|L5V23XHPC68eM<^ecdVU%nKpm#x~2{!o!a6D}1X21`yL>)6= z5h6k(+jw>6B8+Ewfjdu!QwIb++rC(G-QQ$+F@z2d$|yc?B?KM{4n-UrwKyRt zS%M*dp_;#tAG8(&WHj??$u_XhD3D=@xODyTZ&WhWld?W~0Lu{YSt6`rWxr>{aM@{y zZUxu&rp&!jz${640UKD3%DC~Ut6M_5qkVa;$i(Fh1+K7!a~VFaLMV68e^DbAj2~5_ z6C#14PF4}`(TT=ob0H`qCCScjN8}aQ0td9LCy4@#ZZvPMa6mXn8SKn(_x zP5wwYV20#66BkVC@u6Adxn>F(#oE_b`fZUy^#6d|2{e3|V`2qKR{9^D<_am)d*u1L zC$`yGn$rWtE!^iFgeCJCT$t&iFSc?Y=98`>_vP@{s;f(oGJ8PF7rTf?6CpC$%-_<} z^PO~OwVrH@w;7HPPC@u*5%L;zmK5exwFln5!R}MbXKUJ|77zJ@FuiwJ*c$I2PI77; zKes9cPHjk`P{3`1o7P}p>_#BrsO#ONGpD!RDP3PJ5!9G!4L0%DFD3kM8Flz9U?{b;p zOY(BQw@;!%pFB6*7)kAcuJ0899c4E-pi*2(RyDp(V^w(N4ST}qFjt$!4jSoM1{qri zgc~`2_v&r|Hv`zNprP;TVPM%`!!4qp?4LjQ{a@(6-qEipFx?044-h7MN6a1)@vO^W zEV96ItH`L0>7DV)kk5F+N11FzqHggiMfAI>=Aozcx>BJGM4B9+WQl``W$aIyeb`sn zojDju(cMG-aIpmsVFA-OMPHdwXAiQca+FEt6MnzzJ_Rf3$M*`KJX@OUx5wTipI(w#?vqR|Ds;qg^_jOnfJsGO@{BYx`?EL`K z@m&+utktKTJgZ+Z;-PW6H1aMlAw^wIQ+jLljFQf2-0p2zg-GlueIbP%3_la#{?0DV9iTkw0BIuX7Xmlex2uCO&+^ zW!8(N);(WPZDQIY-yzdj{B}G)?=J0*7kdkNB_{OlB>FEbn6sr|K6Is6(ULvCKvR{i zB=_@jhcUSRY&UpI$_Ar3f1WOm$8?%}S8K~zcI4*|SC$*0K*~2)_Vj&sg_Z@2#P|ei z=fkS!s-G^8_p(hEl}_cVskUZ8E;od){&qozEvwO=s!=}UKceXpL>1EVP}|R>g@^;| zee1D3GU<%5bW~|S|BE!SZR4a(fxIVAL7&4KIdTRC9Uj7gMyCT73!3M%#R_t7T2v2W zJCbcKF%lkKQd~#nGG$5tm&&&V1u;d7UY-C5aoOBpdTjyAxD`Kc!fhD~6{gtuk9?ZI zT-1f{k$A8>g~{2#t>5z5`pU>2A4p9m>Rtwq{Yf$EbpdLWk`tAkAo<75t@NgXJyzH1 z<#x`mTWhB(Bi8Rx}BRIGZ>jFZI81!JCH_#smAFDP}=9};_?T^z1y5zcera$CSu`8+{C z<4)7UFO~6V^tKq}K^8Or83va?vkxOyK-0Pv0;^c6a1s}N21rmwtJ0Kiaa}{~ttNpk z($(h2)y059Y7`fN;~2jj-38;ynB6s(X_GX;5e0oT*M?i$U+&SmchTo#OyCZNhpm@1 zBz~UwPO7Rx(mc;n2s{4GZ>UHjyPvA6`v150* zEA<^8?5Ra@er{Ma2G`r!f}o0*0lTd2bm65w+e%cp7`8({q8TxR5S1DDS+-`8n|z?N^k`hIo5tRaM|Db zW3o%SS^sK9SnZYjU#ob3#vFlh6;?QTl!e{JJL-Ro@(gLNAq1+2*4nV`F+n6SETy~nU-T$y7;5VsVH6&q+PJ;|PNeoLcZ zfviH4L#+)Vl!wR7^s8fJUKJWxk>67DN*hl1fz#sMWcdlS+o-ONqsR^gc*>!V{+&ri zpE{vFLu7PBsX;h0-`uovVfvd7pT2nm1JN%B8{zun?Q#fMM_AO38LVSrTUh3n%w1WP z89pH+fli-Ok-q%{4`!1Jp?b*VNR?^lI5b$lJKroz$w?xyu%D_vev~sFcwtaDX2&)2 z!+ZQl*1Y4WEls%0@s;V4d$i(RV{YThakxdxR545#cI`f_<`Px)y7C{ zuNlkA)o1NDDT@ikzxge#>1}H(T`j7+?VkN$a1TyzVx_eGFn<0c zWMZcNhsJ(dzuFI2EaKS!fYxxO0+AOfz2VRa0dA%tM?@H?8Fs6K4$XI3AmBHFj z+xMy)(&2TmUi9$ZMX5O%(U0}}Yan2YM zh0@mb^}HO;sfPQ9((OP)$Bw4$lH;YH<9*wWxe}J|Mj3irTT)vD0+cve^CNs+x2}}` z^IfNa{vRpI-`>*o=V0&LxZ(qe*Q4aL=f>lDDU66MKI7%CUV2}^gX7e6t0^m&Vxe&s zh1b|XH1cUyPG-ogRbg=eT& z3#E~xqk^p`?MHO6!CImvZfI?YfHgYo!|N@Gx6@k5Ts5u3jX2W1@eapefhAaxV!!}; z-kn9w0v7U=45tRuN>K|+{RL0=n87^E_W1?0PvkH0a_Jz})Iz`q=?m8cz{1FjF$+IL zRll;Al1pY(X!}$}*-qir%hj15)v0z*fqrT$>>Qf3_DB9=t-|NRuU2Vz()x<3Jj@dfvJZ(GQG{LLQY zP6^d0V^Rv~PxG)E=91;#jTadzWBkaxrQF%U$pfBM%AYaHKYL3#wYu|u$y75wbIUB| zbav7nC|Qs?#I5_uo{V~$=AGaY>2yYw8>`Pd?)oLeW%kp?GHP_~t<&!s{s%7V58w$c z#9%Bs!BWyvIeO^DwYQI`xomcv89YzviX#3a<67lKn{-7{PUf8Do!5(Fts-UQi48_r zUf86tUVecKvdyp}42^tp;#Pll71U6B_rvP_&c#)10@uOOp85j!hES!ZMGsg*WaDS< zmCA%d*U{xG6F7(&nfFVLN1EuaS;%|7v(|wx14$rS%l-PkFEa>0Kf;S?xu`O6W z(_(*U(~9G_cn7L^i6HP6^cl>_SZqy|R!8@v6VErONJs55(#5!1!0y>;r+|W^t9^bO zaTi5Qmom{hh^x~V6e=njgd&QGOk|&F^b~I!jR+PLaH+Xey3E@&{rK8YAop^#;TkfT z14xwwM@{YKxlfoj?|^{BGduxI2I?cC7EevP^%hh5E!C8MmEh@61xDCu7k+t)Ifb3{ zMN#w!W{*BFIx^ij{B^}ZzT{YmWH36#1M0IK6qzZ!=qTG1%A&2dq436c5md09N1IuA zm8JOPK%hkZz3#P=D~rcUf9)@pPtC~J(OaU}N6+Kp^Niy@snR@elzb5M3Yrl_M4OpI z1x3oInAS5R2ZZ|P(bGoxL(bisg#ra4^N;(5H`P>@Ur&3%!M0|BnZrM9!>RNv+cVn6Z7GwLSaqCx(`V)sx2W6MAD6I8fI}5N%x&24XNZWVxn0yU z?hd{HQOCfj1(+}oFVokK%x>uH&O(#GE(-gOD?K*O7Vu#|_#mEgI1MnubEQl`GGyWp z0lbS2dIgIPkiUQf7_qxtmMpZXKf;&4tK&R{SblU;E5ZFBaql z1r)UDa{Tp!tL3I84p6lWR{x@pGSnR5CQf7NEoK{zwT;KD+O}gSKsgqiZvT#GXmueLk_JE>H)Dn|iF~;WSUCwslUd z18}>U+>K{T#m>ncckW^&D_qg_9%KL(kp}40kCmeU_3liW5e5+@wiHn#m^dNupv988 z={+|5xd!n>zZS=9aT0xQ|cY6obWaW5wRV)ix$mFYM`(LVa?&lk@o(Hc;2HZ_vJgiPq$; zE(HEdM?V-!=DlEpZV6jY=ccR}by2Z)Jm`=_1x|X<4@(!?OMakIggPG3ecsNMV&%6{ zxb@gkbZ$KLi|am|eSACn3}p3t7bs#U$v0%x6AuG!S*#N!?pQpWDkhfN5iDX#Z3>e; z8a=PL%g#PdWDPx{j4KWyd$M_S7|zscOv`kd-_@{v)INP#RT@-JS-3(l-HTN@>6}{Z z=PO)4#HfI6N5jHC!8#H`#T!%UrrnNc+#S7-C534P6dj24>Z$U&hkfI(;vz|A+Vc2j z%Nn!Qcm=L!Uu%x$MR2e+3|UYr1p$O>iUb?)ZfE)WwVYcv+6|{TbH~DmQ*GTD+YL`< zHD_nQr1mApUpMl68Z%AT8wX&~)vLgtHgAE{Ugjbf+xI73gm_Gu~Ci$;^7O`I=C*2 zy(ZNKnWB5ry*+yCugFw8nWnO|aAAvd&sBbID>0KG;R@$0wq~0Hh`uwq*!TFo%?}MLa*p) zMEg@D|HK9wP*>ZJ2br#lKssE&{xfh)QI|($BAf(8(OWb35f~%|)gUOGir5&bA~x%= z-8~AY8RBO#I;7Zny(z7Uo}mH#q-_h7;w2`NvaJN@!K)`uGoFpsB|AhRlJdApIgtv! z-?0g!~K}JugTt$%3kFF1L-4m3c2Slk}xj`-|tluBFF=klHvEe-%#Ba(h(w}Hs zOFGT(vKulPN(YZ-yRzX;x+fndV*OaWlb;Tcnj2Xr1g_Qjx$LDh>t2l&IdUOEF{9Hb zMyv%N3RsIelHd~3VH>{-Dz>g$rOxW;;;7?YS-G|vA8wO4b2xHZ@Z%q&;Cx||&b%`3 z46KClBWDn&=P+K*u&=dc_*WK944>X5i|%_&jui9-IpCVq$)jZ0r;b5xO(w9BxTYd2 zv=4O3_#8MtZFj}SS;zAyV+2iqKPV;Xfdm9l$4Jd1*B9#QIJN9JZJj>w@$sD;nj9Ai z*=&_v?F1%?bPxTtRHPu8+FH1Ugh-WcUA#uG-Qmn)fv|S*PqI=H^5T`x>U~Uc38gFi zGzl@bZq~G6PMvb&Fb+1hP}jlmBk+mK#u@DYPbxlsmP&M)a~i{f(HFo6_o4paMl z-gXIrk(|-KqF-0RlQQa4yiMYs*3I=sOW5UDJFi{9b;BX*d*3J7ST4Y>eJOVeN4*Ib{b!m{yIvH zzfNlt+F5nL`)y@`g1W{FvkCk2Pk}PM;>yHZCUVaSZ1orKi!Tx`>z_2$TdSxoT4$VN zZ|H^)+Ybqa-(eL-Qy4MSvc@pYYZcr1v~- zWo-DbxL`Qt+mF%UOjvlbf+!59-=asz*6AqS z4C}Lbm6GS$iCfi}uhg&famuUS_bwWnms6R?lQQ^J(L|=;qNs;R_DAu81Vg2t0QZId zb0rY`00$lr>>nKakFN4x0f*tC?!%V;yZl$2`b(<&^Kc?rnRai&)l+umz3F~U=3fK+ zsa{nIh;rLJS?%AZ*>Q;f31XZgWUrL(*?pa*!qaJ|bv>tcm-fmb-yO3A#_rzz&B(QW zcu*iJIR19b0b8coM5L6;u^#m>q1&@%E84!a>JR?fnKz~=r4Kih?SA>vxeR6s2k_YB&bLf^tc_Pu{vrD1E6^;i=mgQ;2HtF2Y z!ZGUIv_02{t5vnndAkWAvE3@}1<%*6S0`~CpM!F)ME|f0e_t7F!a8DQeEMaK(v|R0 ziku5?Ikv>-z3&}9itptrUkSQ~Aw=Nm)3ID)1;T`{u{s>3#@NV>RIXgfb}qT7!#n4> zU5>KUWia3mmMC3X^1CkLg`)9f({2?zxmxlx0bazXWzPgKbPxWdq2qL1=r5t)^yDOcKW+Qc={ zjs14-yVVWy%)4g({a43+!TGMa6kmg;}nd$=>GG)-%su8oeWi04H*+H6ho)3A;j-0c8A!tGs85U{ z%~yJ65nKU^^yiwlx=)EU9t5NB;ejB4k2FmF*79>N*~ie<+#q z$#j%>Tt|UKpeodh2jp%F6LsC|J>Jj1Ug>P@Bjx)?(O{iIvf!+|bBI7gEU=7pT}B>V z_tQ(zS8_R5CDk_le551_IcjvU-6YKWLBc8Lz$qDvhft9bKL}Fa(ig5~m=Pm11uJBl zE#(*9?8YV`8Dd{9L4`~rQy;6CSjkO7UnVou{$ z_CWgc#Q`j9CQD)-(nP~=yQhWNR?kO5w@m(MW-6PnEN!r#Fw|KlhvA~B*f$mSR!B@U zWELGlU~{dLr&PS(lCAu|JqD|I7c7Fdigs11{0&v?w9}HP=q;A>&GqHqX!U9k0ny~U zs6d5T+x~R5F-)$;zqdj!@;+~iWL>iNfno&>X%g&I=l?JEzB;I__FI>_L1;@`ER-6> z-4dKq+@ZLY0;PCx4YZ|@3Qln=#oe7iAuSXQ?v^6KLeL-~a5vxg{eI`p+?hM~%$#%o zIt(+J$xe3Oz4v<8yVkRw^}P2U<-WMt_hNK%MP+v{WiGx}x2}Unyroh3~!vd)r5X&?yp+ zu7rs_#a7p&fTg8M9i~704&SNy@+#N$FEU9}#n**!#kE_5N8^nZEb*V;aa{mWH1MMz z**mZ6z?LaHcfJ`)hRRmwvJ~00LANvC*P3QLs;0F%KY{vX|LoO4Hi(N|=58M+WzwL5 z0(&8I=j%4C`bAIw=1j6`rooGDlp-dyd1O8zLa4InAI0yhcdxd4Pp)4C=}19=5?AeI znc>&E5I3(pwSH$o<663l;1C_1$%o2@%0uu8uGMsRWv`++YmpRdA%Wk&NGjGK+>xa&YZ6mo*eIxwHSvyCFY#c<=e)yo7*aWxKsT>zu}L^B4`w!RLwI zz3OX`g$${y01eA(lMgF?t##Z?b@FT!Hm-;pbGJihA&gRMZcU;COPYgz&+uR zkoz=GCY>b;-AKM(D@4{+;?7AJf_&9qhMrT*xO1EfZ|Yi>cD{Y)iM?c%1B4NZH@pej zzYju7amV3J4r!kz-h%a1N0aUN8^#^&=!yhIh3*URtlk8ijXm z$z0h*dMu>3f9UW9pfMCLA0j5k;1-2MUibIjmn%Pfpbw15>N)3N1r|VCdW{7)nRMef z^7BN87V}3iPkEEPNprbh+S0zmEF5VU-n|@2X1kasn>-tB{_8gL#bs&E2LTS3SyT)K;aR>1R92A$gLEfRxlk#@(I@ zX9Fa+-l9w`Rt%6z-Pg!r{~Pc8^-DZna8pclxa2H>FAp{eu!r4u5x&wZ<`4tz=0@{v zW?@@pe~fz6)D^9Et<-(>4;YzI-e2gfLw>>PH*+)Hw_TXlTWJE%wWu1{yW68&GDi$t z^=(ExHFfDN??0vlQ9S^tqHD9LuGx1zZ*teZf5|gnsheD!wu4|xnRJi}QLMjQ_`$k~ zzMNMPzSj}VM(V#iP;3w2ZWd)!MEX@cSzE;_zR6z9L*+^Zjbe@Sp9ru>-7R^v%kB@} zhxA(eSwf6)^H;(?(rPE)w-v$Ttr^8Ru7szJ?8_cUET&gCi$lchIwJ$+e@%^%o9`U7 z9Fk-;U&yJ@gA@VIxEkA-W<-YG_mF!~gWe5yK7PdHJDKr6W9 zTUXw>WQwxANC)05zTJv9A{H&7n|Td7cFMn8mi9V&z(;(vp?jrDT>{p;L`8>DWsPpjjoUhB?y&n6DF{hA2$3e-DOZ$Gi2Z;D~uAM5t~UxG2Vj)6Yc>V zd-pL`aQ4!dAZjmN^^}LGz2DtnAHDo^jG;#t#Ev;}1cVYTlKs7ci_EwKft2K~|8)cBsJ?o1zCIQXz zW0V){yE(Pb&F2O}J^E$d0X@5Pe4m2E@=>xZ_jS9U^Q2tF<);!6Z-@2X)jCc>qNA=L zAaVR@=-^*AY_H9)z-R)(JSZ-lYO*I=2qpq8pJGL&WsT_*LaEkyjPXB5*^W38+droN z4s3a{!w`-6F0~w&QL3J$`=f^w?cefnuG(r%);ib}Vp$znX9o~$ClRQkquWbh556Vv z(v;o}I2X9nei&iiPzQ&HbixoYnzYeR_dadO``jI0gd+MIpEWR9n0R!*HuoA5lnUp7H!)nJ}f1XhF>pzInrkjUdtn_4X?kkk~PYw zxP%H?7NoC((fn0J9z81}>&W03ewfGaCDjzH4BgXXXLwbmb;&7zI96}o+CvSgJiE&=M; z!|%b%B9?(VhJp_nYP}PfSJdUuUBPhb@5} zS(aaAfGrdDr@PLK$KdQwKL}h zS$;GO;|#sk5%GP4Tgd*4NuhPKT@0+29&!hHwI}b5bK>Ucpv#}-7Fhgbx`oU3mg@~- z8YJGhD)Re=MGE`{h*4ph`{=%=lotei-RIzd)q8gg{D9ib%Gop(;{2pB4s<^Tp$0b5 zFWQ}&XI|8=G-TDiEYY*5#Jbv|GuK55=JpCVDe|{A)h|LguErH|r=r(JGgG~;3Cn8k z_Xvj^ctY&!0SUgE^9uLxmjAo^k@E9Zg!1;{I`9ZT7JU6TumSPYz})--14mpyFh=oX z$DjbT``7;?u*4in>0i`afCBKpAt3+i&tUm%$O*JC^7^hEq-Y%dTK$_s41C|v%v2pf z6>z@WMLX|Q>N!`!NDhv|F|iIxdD!U-?n*Jk3g;66l4kjC6_f1 zW09Fwn+qMz|fALh$HWXW#2p%)>Oq#s(CltX)zz2zN z#tPC5{}iy}^NmAFMq-vX`~onx5aWG-PCCnNl4Ke^1C`tjcl^DmJWAEf&#nBgzBH4J zze2`aE4)x{IvGTk~3nt*hnr zZ}7_%?`^4!n_lYQL)ms0EAp%;2->#4*NJL-aGqyKcJG~1#?vrblP*KEsddoP=|l$8 zn)j?~;xk5#9eZy*x{sT{1B|Yd)6MJF7xB_`?OudJe5GIMITT z(yAuf@_QXGsbMs~E6k8Aj7L9>w&xioxfu_)hYvjl_ZRjhJWWX^aKi7&?L4RH2)hF! zg_*?8Z=-Dt+AM-HA1!gT%~^NQDAJy%I$Sf{>_hzWC%K`@ul`d3kSq8t?Or-0t0rCy zTkDE81BLI+Gp7A+KiM$!x-T^qE_OI%ox$gK^qq#g-5tq<_CMh~{DI}_MCgo5!Y1^H zx%Ex8tc>0ZoJbNv0wpvu3k z;z2;}d%H_Ft`{Evh;zR2c4P?lO*%v4t4~0_gsYXtm%tsVHVWlSe==@(4$Q?6n zzca}HSrIC>lNyx?&Zwu!st#$CI#x6SUuxZBf?X$mPG99~aSgK3+q)~nR^O80lVAT2 zT%l<)I3jRSsLCu|Kkl7lJ3z#40VAwBg%&&&7@v&N?CFqqL)4qwO+k&Cm2&C+kMru4a3S@GdrvQP11)TfqC(^`wp#cC&*!>?E(Z?A$21?1Jlb zKfJMbctucvD-MGSQf4J^`-Wz0pz(Ox8?N`k2 z?O`p!sOm)jOqa*zqg9Z`yXK@V4`tXJC3m^|R5rFe30KBiGV|VMa42jnq;Pj1#8}4K zyEeR{zU@jMn9f$iL{a5zni3l}?~YxZrh=SxP)fdt7zC&upZEOmItCG#Xq7)DbwZ>ei+c9d@{`_(RwtKXTwJpG+$Or9jSI$Cq_ny5m4 z>bnrz*9U#);J*yl+3hIVdu*{W4%xj1I@rR{rP2+GtV!BlOsQ5})cFY)&lF6xH_Xc< zS^IjEDf#QkrW3(S^1lkM=Q?)$>Y%t})h{nx+}SR{duv|4<}pN!JI%*Em3Alr^}G$Y zO?hV*J`tnmX!!%f%=J6kSq?woyNwBN=3HFCW$hUI)3%SPfakNt^4{WQ%#ad+GRNT7 zS+Ni>B_7JhT#jc9t8%sCbcv&!g3IH8yCRe8K})UB6&?OAxI(V<=3kX zb>u(4);G2E(`&)h-e(NR3^!A*94?-NStYT~anfp5^R0>e9Mu>us_0E_14CX!?B~6? zy7WH7fjezyK|4*mFH>qWF#Lmz1Q2i8NycaY;JN~mjQk*Dc7Fus#olep^_5uNxV@-& z@vRNy#oR$%NWI@HHuO>}hSPh|j(Q<5t>I4Wp4Ic4pU9PQ8l*dO z70S}`7L3GxUhID85DMgQ=Wd!Wl|F$7;OBTs)QFUb+zeWZB$bM`UO?98Bi*f^E9rF} z%5t}uR40iEPJHD>o*gsus;N%ddl+R?LeSIi4&%J^3AzHeagsuP|Zbwl_R4SE_!B)~@zy zHr!_9ECDOP@SGP-xvTs`;Xe2k^lNMpbqcKyW~xMO+&CU6GM+uyqPbG+ojZjgH8c|O zGhEl?`gyb((y^W9lbShPJ@0B%Aun*Y@?WyO9*fWR7E%b7F4uWocWX=Kp2_5<5T`C_ zz75igyDOW&7Gf!BXewma`*6v7HIba&r`KE=J!wP@GFf@)Z)U+#jPGz8i;B{`*?_zR zZo_@7`>dH=uWcxfg{6nllDJBGj?>Ru4qx`^|NbM|94JBBIQN>rz2xv>oGd>2s!Mxd zI!DE;lVp@$)AiiRAy<%QvFX}0tUr!jZy_V?DrR##hJO2*9UJ4p!E=8+sP*9zSmOBV z60_;(mmcHemZgDY6cn8A+H_Rxr@=7kE!gMYaBx9SVn?t$!T20P_`D9cx~T$}A<&}q zHw)9h|1C+F0GP2WotSu{RJ8IlA>|1ov#Sw%%v@UPeIO zI}tK4E1ftB*NbrRaNfj%+Mh$8a04&zJXy<~`wo^i)s?&9I@`GtqnQyBqa7J_ccew7 z7~qL!Vt%rh>1JxOE4A3ZK2I94N>S))f!^88yO3E@Cb`kAcUu)`N?a<{3K)tizgu$A zFs@$vuG4y>zK8@5z!qT6?LahdB&E`Okq42d_B0@ zGzuX}AB$rK@*36FbgrG6nMg``87s-nanDRYE{l&WbDI^#jsEl1#>$6$|E}S=t1AR% zJSOft5^o0I3l>ni{8eKJS;*dW{&;8mTw!hf!Y984J3DgKtbfP3GiE{@pLABbPRcO) zbh1W?(Tk&!H~(so`>Pj<<#WB=!k1>W+da$Fw`T>xe>t(gDue%}gjW>t7RYI~iL<2X zC|_)+n}55=39g*>c%-J8(g8Yjc>K)Yc*#{s$IY@>HHGWhRc z9w<&?kXI38O+Z^Eg@;~=>2Soy?Vqr#u!~Zr4IZkeVSGT};N@2T8pr8@b}wAj=$PGq z7X`BQ*nbhm_E&P!mTa@` z$dXRir%x;BhvBDN&5Ju1NPL6j+9Ev>0$D2O73s3wuBWtC2Kmz~X}1dW(LPgb+mL!A z8`JUhR(~d)oCedQkKbNof7hIAyH4{+gNo_i0}iV3kH6){9$oDk^@WcTh^{l%o>?)) znFYRZ=VBbzLas&6Ot-;0TnL-BBMZphe?Z;X^8_&?V&qgo?3aKy$N&d=^X&D^D9>IW zfK5yPen$#&@$8M8pmS$0e;+0L&mVg0w`MdO6&`Rr@3J?nr_2^Z=exeUGUx(WK^HNjHn5`LJ1JWrs@VA_bnKS@nQ#afp>(?2GaVF z%wFDSn)F*<;!%4TLo^k$+HZNuBTeu5P~qTk{CEW*1F&yO#B=*^efa|kBkecAi3{aj zT+#g2y<^Ri2Q8EnJT!aV{OI2fHPf&0lHS{y#(2%4)!llianr|JO;iSgMAt-v6FxvOV0y#N&td~i*Fs|uw=I3siwJcadEis zyiQGr*~rVYK5Io5=;5`!9Em;Fao`b-k*asOwezW0ct}S9FjT-rUj5r#0V@IiZUT}>XRlI# zy96HZ|CbK{PVnsS?*Y91dpr;R{`LR1Ll|9A_tmdrr^h?x<>k*D&Ms97(iraxnuYJ1 zKwX!}tFE1m2?J{gmKk+lL`WW|<1pHEe^LR5N=ts3fCpw1q!kECtU>UW&vb2-kobvj5 zTeE`6t1@h7*L4##i-qNys|KbHM0f%vHKG%o55Jsp9{A{8k7a4VSCSb5vFdBhLAj)A=NeZV4JL=#nmpUwbKyZ{B2>IU=9WCf?<2g8D(yy zz`fpGQceQ7i^ws5a!u~(NNHaC9SVfV6G3n}-9V}gF7j`~d;oR~gykZAMB2H@ZIuv7 zt9`w&kGUrdDXugWE_7V(3i$*H+neNwk8-GVM;YpZ{`o%a3h!z%3h;h^yXc!BfQ&mN z`_gS}d3$?%Q-T5}a;kA5B^bh>m5)xkw;(XL5}^rCmADt&ZBr0q8!?1k*&O7tzx|hg zzX{^roQ3}7@c+*h;qQ<9hwH*FetVT0{ zuKJioc#-epzt*%LvNu6-v9U`te~vCzWkKcGAsdO|0?1AHYlo2IxbMJUcdU zu(>7fOZbi4pRqe7{o?l51h^tumq`S9$@KOqlpX{((vjzQFe{M3K$-IWCl;_QnvKi5i{ z{a4rASXQwT`%t12(s4~gi$NkE8G%`o88vndHFoJ>+75N{3q1d?Gwm>eCANP*Cd`IQ z9OVM*yI@J)ujHX{=69NERjib`6*1gJaJB9asGj z%0wx-FTYvd0isKo{tAWg)B~ic)A}LZMkRuWZd+GZ_ow?bX4Z4YJ4XxvcZJlc5|5Cg1%HhKAER zY-)6pQ9SQz~CU2p7Jv$q4PQA%t|B;ZlxG`os#w^iF-NR<>MSdfF=HQ0xVUE zQ*J>7FgO2;mp#D7^Q$<_b+%KCpv#*8;H8ng2OC)fZ>cXxNM+0zsJsmsXO>E~Wu zN8mh+Jl4k^awB&KHk$&7H39R57l1=Qe%%96efp3&XhW3fhj|?zI3EJyrR}iP8Bb2Y_!LZQf2I~;1)Wfhh6{!05Z~wYlsAuw#z_Z zk3X%J<{(ELN%(Bik)5tJpD2ehh`vTmWrR!E;(b^1a@OsDpwfx*Icn}DPw5rXgk6au zkO!T?5gtb4_9K^n#F_aU8HBf8Kw9)O!b*Cwz4Ut~8c!YbiO+e(lJ&pJGqyeFVV3k;n zaJvcL*edT2!QU5IdkLfAyLINE#_L>jNJPM6y~D)}rZ$!Y zkk$zB(Hn^3wFykB$j=wPdv3pTgJq^;XQ&_+IT^6D-xj>-H|v zy(wKnIGOveaEJlRju zs!tQ)*6`g614CcoMS*-8seZw#W|ART1|B-Y=x9CRYx`z6wWU;8i#o0UM_$P=;J51| zfGK(AE6_4@|I8Y%NCfIEFCXGMJMsbbMg0Gk2*Q17eh#Z$2uqzk2rp8$| zv&H}j4a(gJ>U>*v#cMFEjCt|va=w%t>KkYXSW8A;@>6@I* z^|Mqo8H?e0%FE2H8V9T0qgZ=4q~)wQ+>7UuD>1p*&Z;14U)o?|Q$>`1kRT?femN!9 zEmzF?E{{l282Jc=!jXZhk{^HCaM3ChTyWzI2?6k!w>u)>r>_({p3eWdV<>&Wl@DA$ zP`vPjHhg?`MAb4j^RC-<3R{~`%r~@tKUudZd#=m2bZXW$KIg>Dsa&L4F}yaC8R4>S zv1MtRZ+@e1O*tc;5NO+dG1RZe3GBnb`UI?qEsx?m4Q=mk{-R17BE(YljLj=WVuXg^ z7&NIu&RC!vK;t-})sZ0QUK$?cGgdonTH2WT-okxA)&?*T`VsB$F1sZ7evL&D}R?Y((VIuQQ^3M@yn8c>XTKd zy&dT)0oVXA?Q{>FH(HW}4k!(EGj&tUDV|TN11)3O zVt*@grQBh@pNudc-FQC=IfEpqZ6G1mS>t)}!LOcO=k!#CxEXmnt6a#Q*|C=?h~lGY z_&)oxA~ZtZuoL&r>uJdFPME{nMb)d$s`JGrN{MoI^hpEE)!Q!|MiE?Rc)z z@rQ=kjNLhI+xy0pWvQYi6!a!zN}y6nT4wlSVxvu_H7-JSP@jWIJV?V>{4 znS{kI55869RP>$uWL-nMUt^P~O);43;Uc)=o$B2XWmB3>l5=UcrBhWj7`R*%tn4bb z)t!4zsLEU%Tf0O66WEuKt!;HSxrzHO*By+C>A+QDbcRcg^7+}3>kFivvUJ#+UfFbv zcZ7hsscMQA6L>>6z7^k5G3h=cHs!&Ss6jox?fiF* z;--ra3)@;r#*?m)l16KU%+9!=w4x-{iFMvaoHkO@FI>NzMb%e~(|~XE?N}uY@<#Ab+k-%2u)*eSkMRLGc*b55{(uGzfI zGm6UhF1PM1juOI(jkPCWhlciTCraqWwS_M~1zZ0aM5U9!33D}>qslebYbqS~DG*fd ze$k`nh;@AJnroM=G}JnmDgA&CMf8mGRn%c9dx4eJuZ;jR%;r6+Pyl|y*7{-awI7EWVGQV zLV^lL3}h8Ihz#CqoKKW2taQ3yXvoOlZlDtXV3a`Lt1M(J>)Q~k^08wgy_tRu+WkEE z#(tLVEShp8(9}u5zTYkZuJ&8Y(=Z_o5J1H0kBp?rhBJAEV;b8YJXtjSa|xfoAsAS1 z9#(}PE!Z>;t;Q=oE_gWpp~@YF@e;R}Z*4SVHq}HRu5aIZ1q%erQV_1ij{%LT)hT77!S;KR zh?ow0NVDIP@#EH18S#A#loN8|O*h#dJl+n)4jJ=TuFH)^18PSkoMM6T7m zGA!+v`^#-tD>W63u<`@JFaU4-@e2$hwJD7IC(TBA08m#=moikeaF92YM^w!2H3aPB zN5u3w9q$(8_1~<;aN3773>4pc679AgQLVzuSLpUAciH35O5!XFAH}wd1gB#AtS|!s zxWMX#3<}aRZ_;JmS3UkAO)HF*ol)w{>o-R%J#41Uvv$JmfEs0K=Cw-3uzJy->H=U9>N#{iHqR;Be+yEcs{23L$b`r)8p!s|V{ z@~_I17%w$8=EaxC-_1O~BcJ%AqoSRhc)*EdH(9&3u2UeduLm}>5ijjnC@h$$=^r9P> zST+&Akw$E?>(NHyrno3rsC~u>39CI^`;iEvu`jr3ZjGP( z{7-h$x`5zOh~%H6o)!xPoa4rGVuoU5Z$pD*CiPjBm9T@B$>yjYkb_c8miR}3c%fS$ zl4;Oqm6T6|Yf8e)ZQGX}sI=URBMD1|xU!Wys8rNTwWw=SD?U#b#)4!pGNTt zM(wfV<~l7mJF7$=O_wG~BcO@}L0TkeM!~BBx9)9J;XL;fp$~K>C=1 z;ms|T{-CQ(46Y^0WJDMMv$ln5tw~b-R>_{ zA(dE>WvKBkojk-Z0LVHH8=8p?L8-*$jPOn~(y&>_wC5~xWna|)xc0dDbpLb#sHdQs ziE#f9*T_osA>dwaI{yMZ<$OqjoQd_zKl(Mmqo2(!!@y{IErutTIh5yzI$ftH6xf$@KH#aFC?>gHg;NCEXYcwBiWtXic@GG? zbD3koz0G0Wzy@c1rDv)3Wy@xw+uW;6gIg6*S)h#@V=nm{$*8 zms~h8p6U0~rzZ<+Uox3*G*wPaaTO*;z##2;+Bpk=qIAsZVcKaKu~`QL1CW@rBr9oz zlI&#*Y+1NEHY($x8Q&WvRvft=Bgfq+i_nN5RQg4B+RFt;<2ujmrp=uUIMe8gbrZXT zrFr$4_~IyZp}TtY;G$#wSo=rDq*CCi(>zMP^7`3p zv4hW_w_aGv4ku8oLED*5UU7?r=wO*ulg2F&S#8YJOe@M6Uulag^9$~PsJAL_oPH=! z%6<~3E_?4s0y*MZgJB+j-J?7cawcK6!Dk&@L*~A$*%u#*EMPjQc@$yUZ(127jaFbo(8+5`1n!86$NJr`0S4JC|*<2x44{uWOIqn3k*gy#_wY zW4z~lOcHKzO1g#^C`crQj32h1ubM{)5>WS9%qbZ7y$j#ap-lspK`63W7pLZ-BpFDi}TOf?DXtZIZF*~o?5S- zcAvSnw-ku5@o=|Vwh}1ibbrM1$|!_lv%|F>(gR5IqLZPIQ`7*J(BMX((!V_yNEsF+ zBOVWw0vSi)edgZ^3JT7&!N)c9iga=Q`HPEI4HVPz434Yy{to&E29dzlD^?u(!7_>& z7>$}{i+0c?ect3bvNewcm-%Q&@MW?DAd7-Lvg_2)ATJi-YS+|;;vl5g; zI6u0!e6?Jr+zWv)`!*U z(&E@tiEr5bsFl7cx1sAP>R~fGje4-%#`R5o+Yu z5|KaT3+yAX#AZb)%LbbZqZK#%pq@vf@rOr2<^FQBC%JGw30}Wcm=^}vN&FZk`IR}% z&*i|=trr6zn(Q@GE?*dK=hql>G3yyS{~ChjQcy6kg`WHyAa0N}7$MHuW;Mg&bo8POR&Wx4RaP)=C&>90BY! z4+Ct4l%>Tgy1UDGf?-@Q9q8g}=$6PPk^r1%4amF4GC9>VR4`?iv5YM+g9I@%&T`MghAE^%R@t zBZDUZmG264r_(9HX|mnTG($*aw@&1RqVH|%Ak-PqA!NMTL`F7a#;lfHs{N7;>jr>u zWQxzFU)Q$2EBE7RLGA%3 z3+g97Hz>*GNI9pHJ2e0zH?X$HLJnN*5-fN2ww+VTiQfkhc4#i!l|n{jVVCiAi*Zr? zbXVPh`_%yzl}|*#u6%$cDrF#5)@1wzBE`1W51df zx<7=uGAAx#cu{@%d@;&pUN8Lj@KTu(AvHFGyNNRn@2B@iamNgPVj|B3hjp?|OgDL# z)L6BnVnCbAt$@+Cd@f2bAb0N7*GW!ZFX{oLT=Mp&QVQ6}=YF_t8S$x*P_->)I87c>lHrn#*=@s)+Uv5OnNpe_eV!ny_V{#xgvnU6YwGc7V^ z!@2I)#T-#HaY1r~US?R=J1a>)8_3Yhn)&tu7yIGpY09Lx_e3SGC-%eMyq|Pfyt~t= z?efiU=yJ223X6E9k0A&LZ~xHtczeOu9|qUD@Y0263L)A^Opb_4kg9L+?4*o- zs<6RB4Cop>sa|W?9Xl#_+OE~pTR2IP&#V;Fya#NdUR;NDsB61u1Z4APcyR5EE5u@3IrRrr&B6vL!xTE{>NP6`x)i|Z zJN%)^H~S+0q$|xzf-k;y%8NJ|g}a2W@OkFgkiouX7T@d}xYfQ3Cmi5i1+F4S#hhKn zy1eY2(2dxJjD6z~2PEgpMuK}Qh8Al3Y{gx4Hd1RehG>E3xk&w29mn`Nrtd>z175@) z@iJ_>^?vidv0_doGmf)vVyY5>ufK%CbQVxaJD5TmAJc(M>iDlB*qgmZ#XS9*-5d1A zaK#h|1fda0&)CnUXql{l9G4*k2k$w4w@GDgjW;f?NLrS{hJY3%+&FY&!aqq5#J@j8!>Z*GFp z1lsG$MVyU zbH~Ww6SLC5unXNkh;LmyWr4tev!bp~^U+RuuYYx?fnuVaMgG{We9suMH?q;$cD8*r%%S{Hb~mjRNn}L1Zl=^9XZ(S3C#pT8nAsaY6TYW znK7GsGp-Pi4ZvmT#j19+MVAkW$7R*s#q>pQXleaPSmGh)I!Z{Jc7>eKdM43+Vvc(a zXk(`}c$oFJA1R%AyH`UchZvp-g(^VKE8&&&Kp5d^8L}i;NcdeA7YrQ#`f|hI69gWzagS7*rP@A2x+=q+R z>`}^XIHCED>O;q@dj^ry&)Sx_g7KD6VoR^$A8y`Qp~yGu3}QK8Sk6L-F%zDT2>|06Oe-jmdd$9=&~JM zKd$i_HB>UbHl^Ig>uF{tC*x#yY9^`mqz^&Ljq&m#u06HH%BN3qxv0 zs`w4PF|C|(D3P`8NEJm3EGc*Jyd-25OSgP=(CaGHi)-}1rh#{20oQZ7iY+C7oxHF$ zl`3#OyZd#p%&Av>agF1+r~P=GO2+h{v$vL--MG_J@M_)i80M$z+F-ml)`}0j>aTMw z*E|}X&nnD3J2B?pOO9}r5O(bP@YuD9jll>lvbbBy+~u%l&GI-VZ0!ojRqc<#bok2v zhZGMI8kLOe#zTjWY+7Mu(5af;F>9ZG7kudGjz?uG+IxCxZNSoL+A>8DJJ_LUBCp_O z;##aKMZQGA1;~ZF=7}S$R#e#&%dow8Xh_rSH}mQQmILV-e_e}6?!dq4x7DhU5D+^~ zar`2AGnV;azjPQZ@WG|%lj%K4T(v}8&0HXZpOLlqHQpE2)bx5@JH{!;7* z6m1%PfKg;>?t|YkzS7@9$TLR1X%0klY&d4&kYa-uRoKsGl3ml+1c6u)VH*E?<^c_&EL6BW;}!`WmnFA7;UtVeJHl# z*?_8Zbj)E(@G9*Qe3;7|V+j~7xes%;s;A+k>#|bKbQ!+P3zA%MN+{bht>L5ufWi;L z+8_+LYp%>3jnRMkF&K5JOMLlVmuy(enrCLwyuzMwd(IwMH_+Qiq_AAnN;8dB9MS*F z{~ic&<0~AMLI=81<<^I)D}zBwBtrFyp;)Vw(yRjYw!lbQ z0U!Xx!HI-J#!ORnfQaxb4-Hg@~ zD-`;QJAXI(mV;^B*k&@yo5&~@TYT}0tXo#10jw9(iYdhi6X>%s8-#9OY7!g9=Pi4nWy}g9%6R|^m$COIr98YB>L#l7sT;cM)*H_{Z>Wyd^vJY5uWldt*CPqo z;lVbl1j|&I<-L@q8vHiuou`&wRYppt1!4Uh-$QFeO3+PuB@dN<3vWlOm0uQcKg+=nAuhqxaem#f@MI^h|0)G1ffkfUeYe<|8>tY(t@6#HS%uXk<9 zH17BZ^&q!MBWv1WsUxL|QPpIG7Mo#BH6Y%;(Nzdr#>Y_-CP6boiH6%t2H4b%cLP z$)YxDSSfmZJD2Mgh&vgUEB=};)=eRskh!Ajd#}ox{m4_n7b!5cY*)G8ztdRO9t)8D z?RKF)H++A$j;>flmn4xRB#Gnb^&QlGAqqdbh1UPY-dhH>*@bVMHENX5x0KRCkhW-X zN^vQr#ogWAT@s`%lDEa(p%g3bPM{^lCAb6)5G(`>gyQ?q_rE(kJG(Qx-*#t~PfYSa z@|<&?bIyI;zxz5S{O)-{Lrno#3CtO;D4*;? z?B;{cI%gK4}7OLyReJDA~XP?JPLs(4cG-TF3okW-!A9i(0UTvlcVOH zR{J1@r=LBHJ=(@r%=LUIt9U(Zk>77Oaf#T;jC8eU%8{s>uBSeLHDv&aiB?$anteIE z&Jw`9B#)%~1h%d}2r6pL+V-9*E|?*%FPvFu_AVDSf{z0QC5fg251=nS_3?E%&mjPL z<%hbKt`2b=;rwyX^u~Aelp#!v}%*z-uku!=VqazHH04c2gNd#051q z&oP4b!lmJ31c+)p5LUm-O%fbeKDwJLp#amiTO9lOBvee4MzquRmmGoQ-By0t!(24O^s4pld-f)GLkGlN$#MNY6T?dGY zyT#upmCx0b%O}x_m!P%-089%;40|guc5f^Uqa!J=)!tFqsP7Z9(10awDoSRB(vyUo zG_4Qia)Cp4vb(+!f#++I-1M%tP8l9`Xq6*QlzHK4sb$G8#hYb`wS+$(j?`~jJiUhC z`f;F7*!5uomVF$coSePdcwOvJ5?fSt!E@Oti6Dyxx(LFqt8=@*@KM?IcqU`K6WbK}b`U>CT!3a4ssfze1l>mU7O6 zpXHy8mJcbP7)DTDyJip!ux-u-yny2!Ugc)&1+^oW&C}7hWT71A`MTPuv2lS~*X*wE z2X83H+I{`>l26jPJT@!WvA`pDJ_;E;Xjmh09l-@(b;T#H2j7|oT)J%=iZ)B6R2w+a zy+y+ds$M)vhC5W z#mwegEX(R{W`4lVVtds_ZvEP4zDp>!{gNUTczWY(Sg;*k0~rHi`L=E62N2w60VPun z&L@6z$Lb*cUEh=OafiaZrPv}>fPMVCcU9c|m%%Moo~h$Y7ky^GEW?)8Fl}KJ1AUUL z3|nHXUK%U?%6Je=c1lub^rtqJ^VZEe0gEo4a!<_~+KG<)vMgfsA{*%1&mbVSuz z9lKBAi^-|m2P%ndGQViroS0b2N4=P>$q=*!#*|*hO5&?73vP(D}>W5ngga0*X6g}bg^>5pqI;fOe0R~OK z>yypVn+cQ7GThWol1Ghe;z+E_bF|DAMYWu@qmj-1auJkU8=nx&-W+Tpr}%7Y`BmD9 z+2<%x?M#n8*)I%9?~7rRy>cR|RhN9i&m+wA(!Uwy?Z1Hr8Y7L_emL*d*c9wfVI8QI zZvTng(zv=2DwmP09VJ}rZKYI5T?sZn%dRP@x zKl7RAnwoO4xiFNqr6W!i$F&Ij!pLETe%JbQ#JzO9V2Q^q`zNZ z&&{0dYay!}*6p!%T~N=<#>0z>80^X>;z+&&;UXR_HBniP zO>qS2F;IOkvL{TJ-|S&%`$xnknTHEw`TA?9B$?k*%Ga7Z`H!2(WxMiW#YW@l`1h)m zR?;=YS<1l@3%7nSj7}+O=S>#NN*!^o-v|$Fk$7o-VYrJ(xW}8fV&k)Cfkdae*idd3 z<|VS+REE64>xit8YSZv4h#q2FC;FV&ts@6gsxIAoU6tm%W$TTB^=mE{$AEn{RZ$dr?cM5RlEhqQ{kv*sabldB*`E~M8TSUF zLnlRnvFMbzRF-SbLv@}npM?t~>+WoI-@*$zA7_O?7_HjC_;T7`=Juh?Y$_8}b5R4K zl2n_ZK6d5u?n+Iv+um@;8)n9Y(yw6XA$8+YS*FPJn{s_a?kridhi#27SWN|5&K-|Y z0m@oh$f=_#(|2f$vA^=>PDY}6n~$NFBiN{Rt#Sv-KPmZ_>?iNdCmc^A4wdMqED=Y> zut^r#cHSv!GNJpp;nR>AaBY&|==xY^?sQuO0W8(q>1fqS7{?~xZjCz%>Pj~!1m)1G zSb_c_kz6(ztH6!3s*zGIFP~<2Q3Qwkb`{O=Mxbn#n~X>44C7iwtt0OlS=t^3xpS|l zeY7Q=y>)%`DNXAj?~B`i)>NOQq?E}`ju%OE|Q(1P;&qN^srA@_9s9rB`n~?UjF8A1P|>j2t_=@2sAt0 z`NV_Lw|?hXin`yPEjW=^=0kSqyJ>A<1VCeF%B@H%?DFi%5{W_u?cqg)zZD|_tu@vR zoR$Cyzi8ZV`;nr(BeL;02%FMKV-=^X+si=jcz*<&=I>Eel~3vXql5zJ(GY|=6zX;? zx`X0chjoy-%}_hbH`(7<*e?RFhm;Gxyu&LHV5=KR1Xw+s0_7zVRG@{}<|7zVoW*#? z_TAb%aJCVUIt0dTyZSuuXJXVNLnbilc8qO2{~c-;#up?SxO=+2vad`j-$pKz9;N*Q zIvpQ#v6)`yGygM>f*-YwD^GT|Cpx{RA(Rl2lqs>G`8x++fg|s5w`HfrCDspeL)rE7 zJ0g9RI%CFAWe@k77}q<4980Tdhd;jF(J^cu*qY)KZKjhK09Ugfs!gOL8m5i(?rXL& zNx@hdC(`4R=?mBh%YW%FML~hpC-rMiP5{6aUp-O(2L)hFkv-)vmyMr>hlt=}ivFT# zF0Bfcz%GI6KJ3s`z~HnonH*%Dw7k_M%5_L5wk4QKYcp{i-hU4^KZ_R^2Rr=`8!Y7n zsBY$IjrL1}^~OkLQkcHl4j`aXp0s0swm_`=STth*;<(2_{iL*op2u?|#r|VlVfiyc zV`Y_*pEw^Gh!p2L1G-M#MkE?gTwHbfC3DsXsqYqYFFt|p zv@g!nIimKa&U&vM(x+Y4WwPP5Kyb%x|59^}g}gV`Q%yazCjw&kJ%c{TNnhxuGr-+I zOU_fLS8L|Oi*N($-pwPV(nnrJv0v_(I`y_HSgxs`ARORaZ#(!YLFcBB#zbMfsE3Rt zPo5}yPNR;w<EMOp0{0?MG3TKm zA1_Pzr}HTdBwAHC{NQJ*^k8>(!JBJ@yo>pi;e}vH2#}%t_x3j(2i~JXTT1pyE{75- zP>|-k!8-ELzW~$=(Weze#Je?{7f(HCeY14%4qL6oV&llDAjIkt$pJ2N?2*@2AWJx9FQ%ebt9FTezLF?kP0^&<+u88-MK@(y|k=bdZBl@(_iweoh;7O!h? zUvmG+Sn~%2dKN-)Yto~Cu=Jq1_%bgH_bZdT&3*4`F#pA`eucLGmvdC{*7xlQZB5v& z^24Rw6s&+q5kx0Q+R-e6Ct;Rf7Nn7Xs|b2rfPGF9NV~aTHjhfT)FT7n5Rwbb$wVoF z^5aJaT|C>>o(tYjZ+aVMdlze-@~!Kqqc%N7$&$4;b4dtX?BWGyb(==kKcJ4CU(d=O z2vmM9m@$g&cS-1Ou|075RWZ`o#fCt~751iEo_bQDi;|S=bU^M1v`|-(`*iR0`u(q^ zsyq5^VtRY7ZVB(3)hu^~PIb(ug!!Iq6`}8!M&=STh^yG10~Fd_K)U(yLJ05eX9kJlM`*lfNOQ{R@`335Su>iRDw_Vc*> zeg~Cy%0Mt|RgIWA{NJH96~Z^hDF5q96k| zu9lwa#?=aN7dB;a@07KOu_}i4ZI^E}C{LSHj`2@6!RHn7_~!qU%1R&}yGllXbYHCj zpLcv6BW=1Ug~~;GN(b&hcX&VC`p<^r+D^g$upH7}eub<_a{)RW%>_);=S=h&TfLZ* zBSUHWi_OBtY3r4ts6ZC}1@|ejqpwo{h_U_bGFwyIYt~c8>1^|#MwEzGpAm^iX3Oqx z)y7qj|AoFA-WrERDm}F&P}U8SZqmSJrQx8sLA+>Y_ES|5vE@ z^OOq2*?jIOyK`h=>8UAKSnm+seEJ`maS81Y?#%EPG|6ms+chLRnCCN9&9bL~w%_W5 z&L^B~GJ}m`ggEXbDxXSJmHfgdRW*C-X+iqj>hC4gpHHP#kxH*Lb!m0pT-^WaHxP`# zZbJdTn*V8nwHIKcfd6=k8R*#UR{!FQ1=c%(EZPf)k!K5=wuRzniT47xnSV^bwfVeq zF}i$Nrz8hiGi~&JdK=mO$JoQjo^1mV@ zngkT#XYyZmGG4E__}$RS$v{Fc5y~o zBb@z+G*Is5e^sb|X@vh5*nj-rv>+M0V^q!<;_)v1y_TJS?{bTH)?6(8lAx_`JW)QD zUoKWwXeqaEWUd!7!Ya>`8IynIC<+h*gN-Bo9SoU;eI_NYE%ZGAKTNIHrT#t0<16Fc zRlTM9h1+V#-yJ=Zc{8-#r>bU7tk$2cRVPHLgPJVT<24{NKN@bc7p|x;{Wg2_y8eAW z^P!Ooosqno>*WQSJtpy8DK;r*GUhW`(yWdgJ~x9uV&6}Hz8obhFk9r_MuIi>I6`T% zOhT#3rR(N98Pstyfs2`khq?MdVchb61#S*E;YGQM57|I*U2iFM$-LYYU0E_V1{wqO z^+fp<9#zGM%M;EW1{$xb+=zM23^!_oc&0eArTQ4jUwan5!nLwy&cs=6)U#GP3t{#=T_;?>HvdP8(MYX-LClQaH%CM6sX*6*EH_VMa zUltlF$cPw{IcEA$J3cMQL(#r5N|o1j&mwFkODVJpfu$^=PnV}vUqV#k4oaj(zcI+` z;>A$H7MXFRP}r8Fe5~ScN2IDqN?h+${kknn0#9s(sQkSi&4R$uoxh+%(0oBSTkY*% z9C-r^f<1Y0OTQ5~XbhbSpu60g7v>yCKjl7`YNKOe+S=2LUH#d%KmgkbaH9+2!G{oA z(ycUdvH+k8PFQ}{N@*$bp5(kwO1;80<(2!fB!uZFmCQ?K>GFgGpYJP4 zr!73s#vG5e&&gOqf?uY*8|1YJa@Tf{X*PnR(ef;Wm@!`iJ><;3Etoy`@DPYrmiLO8dQDUvWd{NyRfnvRzY>K9QP+0H}y%E zdSclyu>sT`JZd0pN^4mKsZy@$b@|z?tY-3dSHL(}{ZC4iywLiS!=@8om(a;9wQBLA z*ZH7=D%%-o=9hq{2kuEv4aL*_`uq-R($&=11{+pCU_T2|gg}frf_J#C7DxxCpiqn2 zI|btjdW@(BUPmD^ucdpVeoQINs_VPwE7OmSmoAbtSE9K;<<|@l^nM$du=~fWMjN4AjiL|BVSa5(ICIraHbolHGofVnd6-C z;XBpAt4DkE7OD=p>F2vzy;I41S+|IBL>20HQ=oPxE;-~`XI)RJRzOmB{_l%kru%U>Kqc=s`N~uRwv{fQT58}{6x8f5g zNJyf#LlN0Cy7H#g=^ zokk9#U{@EtC@NI`^8Mc6YGr`rzMe8=5Q$R(r{_$Zaa`FG*>b1$T0#E93ubWVXa9QI zf_~=ojU)hup8~R4DN$_E#$04{4G;Ln0mFbb{~xn=>ixW- zHP1($zJHe#8r(@2i#U60^Sa|Zy26;+Sz+T8P%0)vllY^JoDXpIlib<;x0(qRNo+j7 zUrk-4cB)il^{Ae@t`yP~%I@isDBXn5n}} zpe)E3;_uCt9vX4k1L3;QeX%!XlxFIg%3b2|((;wWvxz%7`$JscF?BJxb!#R*BcO&| z{#f+fezX3(s-qkcVA#`}>L7qg_o{jDVr|Tx;>@?K%(yLQ@?&d&X-x{C9~roX^;*B2 ziP?hybA}z9?|RMnQPvLVWEA#WDHyRU#M2v7MxB~9wr^NXN*s(x$#TA2Na+3%naTq5 z@c+*~q@rUsab@|JORFiB8tfWlC+s}Mg`7|82?-3p=CZLn@ZCb4(IrJH9&{ew0TA6M z2QD^v1<6D=Tj831Sxhb%$fi#CfFcV`A`6WbBRE4JJy?F_b$g`ycURwwsx~~rbjqhp zzD+&W;;9G4UD)u?(PDZS$?h?_qSZv;0cB?#$yP7mp<3b}f$~A3Yq;t@0ZGyp0Lw^u z>x7NXL;?k78u?|sxW;X#7eGnU=-G;yaNSW~eAn%|+w zSVh&_G91k11hPa5^lqyY#tAyLxK^pQ#i>}Ah7XwPY@CdTu=*-E73l_#PlzvcKVkTg zjmo^}=zK3QhL!Ov$vlOZUJT)@NnnSSSjQK21f94U&kowFYTmWVHw%M(HkAq8{sDCT zQ}cJr^=Sw%28m&&NYex2ZHanzYU&qOYXVkV4?nb>is_A+L=aVe4a7{^hIT(!;!G6x=DOfo-$;ZAuJHQJ}<}mjx z+xr*pR#;XEPOwA?z^G1Zj0)tc;=Rt$31MSxJttyoln+T@7pIf?2i;{9p!y$vd{r5Z zz^*i)o>ihCyCI{9p!U1r-=db3VR`d|YGcRQkUgOerK$+?})xXm1x-5K9o{{A95HuGcy*Yer z6c#Ee96z`vT_J$@hCx!4khTlz<}?N=l5LqGI_)Lh^3jPTq1>qXuJ3J&k>aH6SWXr5 zcI&=dqD<94(-FHqfUXo!vD@oK3j1}zD97#|;L$PkJ>v+-NDetSq~UJkOFgEn?R&Bi zlt5_}P=z_@%_@h8q~JT9U`B-~epL_MM46T_g1v5+J!m3f$&+$lwt^u{?&ifMG%4OL z^GH;4zXC<_CH}}9yT3WC{0pi43Pna_+(|!LYY%bXu!yR|G|gM~wb@i7>;{k{bWbkz3c@P)3<}mrLV*XzQ$6!b9$hjLzDnd? z@5zy0T7(+ajTLhL&*&fxsO*xVL?m!iH|yClp?(%y!J_i;zW5#8;sn0D$7Q$Hb6>Ps zPV#@^A}?u8iuGfuiXeHwO3f?Hnt4+E=TM3`COI^agSw19IZnaHFOXQ}^$s#@6I=x5CUF5a#gf$Mn)M>PAbRhbw=;C>EvG1b7RIt$YPe5z4sX0?E`Y6f44hFCWpJ>AQIox^f(a zlc>j4@|T}fwO&NH6Nbo8my*zf4kT{&>kDH)s0FFWd5kFGOz;wNMiq0uh*93E?1x>5 zkoqrIQM;r8{SI`kOVaBNy*EkKMM<^Yp}4OYw>MbfHcA{hds?}^_X;lus$NSFYf)~k z8ZF6W)-l%_QJ_xMxLh?|B<|F>=3)We_BZujjLPt^S6Y&L5=CzqFUb_pSX2v(QDnMl z|CmS1CUQwbk0B|G8XH*vcyAwmLLgdGh$o ze@U=F-kSdG#9P3%4JAPELH z#{~s(UbNNsC2`Kw+D|q-y=MH=A9A>=K zvFOoYAL7vA*Ok5_$Tm|F=QL8MR%cVX_ewAwWx-n|yBlk}@M^jRcjDAdE(&W#yx+~| z`>~dOKr(CG6HwjwMV)H<37bA{W~|S2&@O1>3Bx=cOz9e}>%QhQ!!+FK*O?=LscTZ| z9BeuZo$%eAFCcu2Yoi4-y8ZdxMcB;Kl@Ei0tkpcng*g4zf3xewvMsp(nf*P;xWn1% zJ4n1B3=3NW+!-j|F~#l}iJo|$wM8VPM~ex?{j&F3#YAm=YF}gBub_j$l1lf-vqewb zEUf1R)-K@v5jTK2Vb)NAkqnA0t#94+OBWBEhrnq*->tYIngo=ox{r<|J>3-i#%W8Z z`b%rLDld|q{myG@9rmR9jB{HQ{o0KNdRPXD+mGyG_TnVsMDIsQNXzqzA;%S$KqlJ0 z?ESb5Qtb?Q{1FucY^O`~;}Lty5=~n%VUQ{dWsW4tDDhu6a+)Dyibj+>v%s9o;}EQb zO>GLizYRI(N9{z+5d3mqz0b0d6F)sNH1g7IeLE&8lNR-oQ_4$qhf|QH_2;EFFb^3~ zBykw--`E9wPP+lK-mEDTuH6NlI|shS-E=AyNDsRaMjpNLm zuexxQDlEoLnQ8kAPW6twl>TVn45b3XpzzKVU{v>W)P#5*aCHrAW1A-xi*!FK-zJ}P zb~#-rpx`t*Kw=b;iiTLgd;yFD%y`TwRgocU2 zVo2_s@KxIMp&8Y;$(cCB&p5c zku7?5cs%>ty}Z%!0pL8nvU~FWgYEeeMcXd?{i8=j4jayhwsV{_RPZj5dc}SxmGhss z(w~OwTMtX~KFZ7_>~P*7(&*hOqQiOJ6KcCW-zlFfvuAH?HQX;h+l0)Q9r_CIU73&- zm$x>J5ylxcVW|9*;F(LY^YQY4TGd{9pSCj$r?MyR(0~-7dT5z`k#i3@2^fo`=I7@X zgbs-!eyir(`+1g*kkeY?;~SWu^0rGnat9R*UVc>q*V6TCSlN`yZLg;zNZh7aEYnwT zU9l9Nr6ukj(oa6Tq^c=O>{}_V47?%u03j1bf{iG+00&J(;s}hzn4uoWa{iiidbjmD z_)4AVpU(i>M8jJktQ1wV9M=AlidBN(alw_ z@G14y9(3V!p3No>T=cg3GBVcW!>2h}jLPvO|JME7$B*yj4dcAd>$k5!&d7eWDrKxD zJYE~Od`TPeB5UVo0;{%PW;{RU!!OO`;v92Nqr=V+XDs>0^FLkUgqabz8A{myB$Oy@L-1|di0W}opp3MyM%2gsX$L<40IkDR=28#j_rW8;7;v^A2O@veU9_!X zjmj6v?=aUiEI!*z8EUK*HQ>jv`$rzssyC}-w*AOl6jlU{bBr^N{Fpxs`3da4yjCbn z?ZqMGr-qYqlkXh^MQBbB4ljVJW#JKmC%d+xa4ml=aqU0n#r7p0qepF#hU2#>c2YbY zr7vHJpBNb1tzQiLwx1W*(GF3lQ2IWMBt9zK_8kYlub>C*!jBU{j*MN)aNyN1y#?0a z%Cx(kDaVh!+fEHH!9oMI%26dlf(cOvc+E8s947JJd3ASOIKpPAMh;RtGQA@gUMj2~ z!Mmo-n&F6IhV6ZSEu`drqOvi5g;N(CCu0?$d{u`|8d^ZEE?Lh)87Yf4!(V&j`t zqk#^cwJvB9RNu1O=#M+L$Q$A$!3|SzPU~_SZ#PhgRo#izRLJt%BZCf=D?qR-DjVs( z%VbdcXB;Qrj7vSl!Vi9c8GQq|jo@SDLMLZaM#kLlHQfFTO}=dQS*QQgjV;P>>;wkI zM{wpNB9_fJ3OV1{&)GfoKn(ZqyF6ZJj$%rX#29{meqGaqJXGb?u4+@S{N2v@J9#-( z*t17~z1^B{Kw7Zk=LFD?cv7O&`?Ck$4>^fk<!FlEEna?)}h+bR@4)ebM%W0bz7j0A|kgoinkq_)s2*glhBY|Xt4PIj?gk0BcWH)M~tI4dt{<@179?cpfJC7By?v7%OsTEVpltwHz`d>m{zvE z+Qu?Tx&8CG#Zv}EK2*r_ZYTIN9g%lIj3iODX&{mN{i@GHKki7l<$lyyQ(4T%Gv8IG z&d>sp{|R}^(O*0Ug~w;nb#I*6rrj9g`mMerdF~t+`MJ*}eTCy7pZQOmLHU`~EKwe1 zz?1zVx59pj2~EBX^-npQFAcp>+Mj&y5-6~GT`V@FhlN-&RLp+q{*Q}VLmcP9e6YFp z;kTbvKnc{yA&93`hy-ZRq|bB?cH$d+#n|W5s!iG6#z)z|m1pQ5K^B7@h00Nm(UOMC zF6;>uxbi^6LNTNqc&rp1rIFDdf(MozwiqtB(00U+EXR=;pOEEt3ZZw%EKf%xHlVTo%wS4B6KYLCnby)<#b&c-GA>9?RsfQHRhS=WfV6kQ4$U z7|%2gZD#8x*oy}ln!C+Uzsbo21#_%Q92Pd+J=Tma`^IY^O8`9hfhP2Zb*O8g|w>afh1M@ zC3V(9@w{ivV)4Tb^dekLI9`Z8pypn7RK%BbxA2iaNxSFON_-ZFG^vgdeAA0%h z35D82$`M6l{7@M2jQ{i_z_)Ox$RB2!x$!}kuQtmO!!);j)AqSRJc5o-$Y_ZSG8q|% zP;|8BcD5B9OA@!eF8T#L|L=*Qtq?F#Wp;%j7vl_06#-9IzhqjPa8h`Ek#H?3uE0L`2(8C{)u}X(i4IZ5zKWz0B$7MrDh&rb7rny7W)7S%iGKP! zvDv#HKV$BCaoa`|wTJC`HJJQ<(>dqM-X8%!f*NnxHq=v16Oa{~i?S=b^c%Ur!b~hVJ5o%2+k<^XROmkQcu=<6rh&v` zpkBn#+wP_5j2VrG5mCCF_*teNgrT=$BxgjmaRi>QLGy4*2{itM!PJzf|AWNTSb)+C z@kezomm$#S2lQpJ2)cMkX_K^eMxgqPY9_+2^+bdnCZy!3-L!dX zUgs%q?!Xgac9-YT@Oh7S{^dnXJ^M={+JnyocJD-Qa|b7>@*Vnrc;UX6jCQ zz0)UlR6%eDfjOK;7bJd+92KG#iWUM}fTuRrL!&G@azEDtFtVX+6_C5M=#(wqjiOG6MP2VP1tq=_&SAeXN#9*cz?BuHxFQJz$m?e<)tFr&)V7{& zQ$|^H976AxU6B4uSPuNV<-YI)Riq=2sDEZuH`>D@!>aFg9F;z2{Y44}WzTJm8C>~& z^z`qS(FT-_P1y>*j0dHL1A6(hm7`r^GkvCGton5o&^gfnX^e{*&TQfUS{2hlG;K&X78)n{2ML={PX zr!9~CdcVhPWfQT}*l9BBcu6WUcg4l*eC{j8pVV@`HzJb)DLa|LbfD8@!U}r&l4?t{ zH2T15wXb0t-lPyEl zx=T)W-aU^R8y=fea9XAySj@Mn)!PAY?to18MD@ut@b4V5^Y&|_$L<9H$mXT*pq%jB z`}h$Fr}Rlc=S+Crqg=>cY>yM(-lku3pUfVq5h!Nu6`6w1uV z6WS{5t&!VJScA$7kGi0Qst@=9d&iV0p2WL@D3zYEl{4RcPkRmGy*h*G)YD;ZjGo;P zRc}p`C}$tK>Aa@Uxj6xlhqPDqpzE8iiZh;$5NEB#*m2=y-R9H-O8iOT#O#~tA@}6& zh@`c5bDJ2ykyh5aH8ZpogYV8@rEv4w2RcpXsbDzoL95W8BL0l zYR#En&FDYjFJN*0_JU}YkB@A{OxXD^JN;(eKB@L3&be5W#(qwXjQC!tn0_sx+FcM_ zD`&(<{ztLPuozRVwW0LrmpamP^z)q$=cyP2y0wmOA~)yuYj(y}!jSvk*T^_YSybd<=WtcLQj^wzAllacuFcAg<*%&v z1CMu((Y4#)h*o0y0y3(tC;q->)9T;WKiBSOk}z&}qcanDo(mXl33_~TK*&nlBEM8g z{j4pGC6T~<2X{)i&)@_y9+aqd_xHY`-ag^kX+#1lnZ;^j{pB}a$ z2usoU_RnX80JDy;f%9zLcj8A!op_p%aG_Gz56`}uCaZxsiUk;Ky zWK>S-TgUbhC9unWCU{P|jGqm65Zqh9Wvuu;w`4J7ONPU%$@W@3{d~afmfFF2it`qs z&9JQm%tKfVnC~8dE_u@lv`3ZI3nw_hE8Jsqb5L?!l z-MaWMhHJ$!a9*ZG<&Cs-7FFyL#QG`nj)$30CY$j>1ZGx6yf`$Wtc~9(8tl{?71$Z? zI1wD@ZysaF0Bd~d{(4e0EGe^3mEX6F`iM<`*u7!j_itip*4!<5+c9~{gmZgu_tQ@e zNqO>rBTDrL89t4@dn6kz9yO!xwoYGA$D9RWh^?oGZAwg)>T&PfAfo^uqmB9HL2=o6 z9q{Z3y$^kwO?$q2(nvB0cTFXDt6z7PlAirtuBmH;X(Oc{KuR{P*?Mt}sRxL?Io#he<)n=#K!CuXF`!^lexwg5yx5#T%$8ssnj z+kxd@MF_|~4Fb~cHRVZiE66GtZ16{#LT27h`eBsAO4bFzB#r|2h$%@*bjJ(^F+w_t zh}B_Ea;GM+)8EmhogaY{WQS1^ zMP#-!7RrY*=;3=X=Rea!|Z8)YS?(&QEDbc}*|F^9jdic4@v>Ak4 zF^wdKb-Q%uOPDt6fbIm5;;~srdAFc`{Cmk!E7pU?Yq>nedoH#*8@bjIYEE^j);UJW zcZ34=!rL7}y6j2?^jDUCfL~>n9OG>xwO4u@co%GGeD$;?NhqGQ|4Y!vYo11y>{6_` zGEFD4Xq0Hpm#qyI2-|cPdxnn79gP+zOjsA&tjZk%wyaK%@?utAEC(VL_o~FD#Y#9D z&4u$cCAL>hxQt}oGHDkW3ED(4Ps*?9vJPqy*h+M-d-x6(2}`qhmA4RT;f$nG>&9Y`K>dt?vHPO4KX>WrfSye*Ibp9yKW*Z1=S+7L7!gs zhlI{s6^n-bNYOuGllmL7e$>y?oM7n^kDa5O@0q(EQ~Jk}i0Vz)#nG-^26qzW;_vX% zdVSf@kTP1O2mD96;s3z^KQ|L;MB*&8pgiB(z+=7H&ZURHZ|s?j!w2c@a{ zhvo+^`UaV%i2doJ!}7KLlGB(IeM0}&l7$kTUzqHM>cErJq08G2!+OTMY6X8kYxaqo zfKw8CTCuJ;{8n3ljZjkM3p2r}=Xcocy`*Fxeg)4%fJ|-0Tn!<>=EoJwptU{wopwDv zvSid@dNJ~k=ujbUXN6rGSAI6oG1`NEcOLl5O<>O3viouAH0?BfQ=|wu2GPM6`rl0+ zC4)D+n!SmYZl&SRJ4<7MIEnA4-|DA5r(ac=-J5IO=+Y#_e^5kMwLQ=D&J26la!#gZ z710b_GGNC;h(;&yacLt)`6Z0_@RI0%!X*1if5FIZv>BD;-}Z0de`9wQEKKmEgcA!C z)$yM)V82|(VEy)APIl!fE!h%3v(>r=+c9l%t=k}L$Z4GEjgcUd&K(t zn@rYbsLii2Ek_pD#?i16SsbTpQhBmsVnxQAs{wW*Qdo<#6Vshwr!`<*_W;PPx&JcL zhQClkvrUXIj>@yvbn8f5mIy-!_Tc^ZT;$L-=&5f0dWUy*yUSOv8(u(w=jSmD2Ra zmrrGeVNw%T;H+IvBjkRu{>w$ES~)RK&ReJZjtv$O8fm3qRa&@d7Z3}`?n}p?EJmwb4xvQN=F8ouyw-CgO>J#ke^HTi#zgx22fdn ze1xHPrpv`vNxd7+ztSE`_C2s{x>*wGvOrkf&eZ>LnE=GL$+g1@h$dO7+h9h33qXtnZaoDW$t zel(n`|7BEhJ0jHCx-1;3IVrV_#AD{B&jp+)=FX>6=7Lm_)LF{EdAiXyn*ZzII7;>? zO?AJq6hdER+HBTnk&LlH5|JM_c}jCAxLnuiBXVlg8-eDUOcT9Nm>BwAC|ssVxGw*T zh|8RbblGT*$d18YS*f9%7;?lTacs+%9u@}mh+%O9|Dv7@*@uypl zf#?l2)d%(eg}V2QYHEGkeU~oLWg)BuA}9e=K$K7vX`!kpy-05Xq}NaqkPa>oA|N1L zsUp1vq=X(+L^_dP0z`xmTId8w2%L#){l|IVan6S`#y)%RvGXY-BV{Ie<}>f|yRJK3 zYF5bMzF9xvYLkt3e-MX+`Mc-h44n9#ZjKBwve}HHq+7tQdD=QoM`3Es{me6WIV@hJkhp<4JNnQg)aov$tSU|ScVAr(f5@XiSipOHi)soY*ekEeZ-)M5DOPGbg58^= zOrslJ-kRM(rLWz|9&>G<#BOTBE#dbFDz@;CMx-JzSGrrlGa2(_KUcXi3DHPFjYhi= zm^*VwfuUcqW0raSt^x zTz60dwEOo0yKd2xc~(ZJy@9-IRyI1cX$o1tHZw)RP9FA7dU3bQq539}0?SRLh$e0s z>N9CC+-3e(U9hj^CzZt$aO%^!3)8%vDv`PsBhOF8MVJZfYwW!zH(oEao(T%nR-3Tr z5_2=Vmv{kG7{s>7$Bak_5{J2oAx@D7;PSW4uXQIUe0;NTSW06NQenZ(dY;O>xUn2| zo0Y{#_mqEv+Y?bqufLa45J1Wqak^=kG!Tq8o!(sP31EhLg*d2duJBJI25iCE?t_%S zXPf(B9*fIm!|z;goHWh54g&9H&3mROAzZz4_^tIaUz;777KAx*|G0w3I(*xE$!b<@ zagF;nX(ab5nixw&iYo~2^U|ahYX=->+K{$*YYwetEkBqu&q_qHmLx$h+Rt0EFflx| zJ=Hr5oOO4)2E5;zsb9mZB`tidBC8E^9mTO8&5|_1wCVLzN@XiSee3EVA)8CCb=kT& zD`gERi$`Ff?jgXTrT3p?P7RWEEfo^-)e;2dXoTZhW6Zbh0H@&$LIn_k`gUxew9YL# zfGX0#Ycu8FQGRD6D<`dAtb~r3T>p5(IE$fkW!_^5E8X|rZWr;HVF9|=O}$~jVb{gb zsr+^KGJ>7SsmHirSYFf>*$}MT!fDns8`Y@MFJqB%cm`@J*>i{IWVPj2hm8?M^vey$f0^_y6!F63iN=F2rf)H6ui7eUNQ3xqoJWX*5)aoNU?Z8E2T=V>lf*-yXM}pLaeTND<8D{fs zgQtson%-a+IvRd>C=4F*_YL&$p4uG8IrXmZ1mXT(5H>LPE1@ry^>3wZ&v%p*qdrJ`z7MI2Dvs6ZUtD(!u5#i~B z@Y#JYppZP})~x?{GnFHa>8*R&{%?$f1BfXjjcyAzv=DUnI+4Amnh4o0dA* z*96uJ$LThQ8k@TH!&?yB(_Yu(e|8X zEr&1V;nC>GH~Di+;6TrRH(e`i%)eb>N9w8?ZmqMGOxwl(qlqD88fV-*F4TQlbO! z_N55nvTk<9=j9m}A!kO*63psMmpZ$Yj9GOdOwS!Ycu#I|#h3?R66f#f6&aVyt_$h3 z^cpIf!{qcK?+g`Y+*Y-K7qsK%pLHfpo-~8!dXRT+dip zs>?!#d2>}qd$cR~FAH8{Ac%L>D4NxwH2I1^ZrZPC46h(S-V_s_eoe$##_t;*^n$DVr&DSIPShtUO^ zqmyxQ(=kW>)auolBDRIoi)_{x_H+kIYJUa}WVSzV{{D@kHet3EP_dT;wrG0ZCv4tm zdt#55DtJR@SpIJxLeQ9c{8Rawbw{zPZ4!APb=Ew4PZhz19P*nbisbwO38PjX-6ZlDMU z{XX{FAU;cJl>aCuUXdP18mVp$5F zcv`e0JcvJ?^WN9B>B zx$3oA&4UXE=YZ@F=-|C+YqIXo;Z+-C@n1gO(UtR6|6 zBD4X2f-0HKpPk_sA^IyCL1inguYf?Z z@6K7cP$JLw(%%X1iC`bpz@?s`z`zYSP+d@MrFx^JHfc!q0b|baQ_Zd^0N%5UA71j^ zy%zDr_2pMR#ru4o(kS^O44!$)krS%A$Y7SCstkLL2lz9?8z^b|D<;h z;5J1*cLQ(bBsLe{6g$)n$<_z)J!8I|f2U#au3lv3_q)2Bg#J9M;$JOUk~yv*23nWp zE-tQD!v9izKbVwphcQ20K3jfFy#oc8zV*ubPwbnD>zNNJVwc4KaOm_getk+3v$?dE@oa-u*H)$h`sa-iIrn`eF6f4utGhg#Z;+z;W+H9dL5OR97p=y2r`y{yn`1 z!1g($j?wcjX`B|AREzyiuKff|kYijhJt1?O13kHd3wg#AxSJuj^^&$1-U&LP?<3=c zXg{F=-`5pt_|sF#&z@Lh`B4=ER#&pe+qA!u0_RtcO$4?}@|(&eZAw=7ddh8U^80ItSjXh_(DHvTRZ)x1#YPF`?Tkfn%}Ul7eP2z-g3vOmNPRi&R$2oFGYimkYg zXnU$!3Fax8&yH%38vKnZVMn8=#SeI4d~BSM))2h zS!7(4aVA$e`?ho4HR4j+1{^4PENvCdL5>w{)2)|2Gn{M^zx}o*J=#6X8n9csdAFNK zWOv_5Y$3c;O)H5DA-eZIz^Q1VjX$Za8-v(ifPpwHKumP3Ww7%NLaKbVa-vu2{06;# z5Aji!DP-HD8f;N-YZ*-830UYezR9G7dL9b}*51kb{nWyPG9sT7k=mqDKW|MvP)%d$ zKw*#kZmI#Q_k1AwL{-BvfC@s)lLU~R4IZ%Yy!nkUBua1f+Pca5hi-|J;S~RaIw-w8@#bC$hJO9$e?)76L^R2-X)r(sYEf^-i%LW8SFwBo zR=^4yy8Yt3U@uhmCTeXwRkSNmsa2}aE`Zp+P3ew+qSEG!sk2sUu_sl#U+NfiN*i%D zg{cJ`J@al%{$2TH>L+BjF|oyXxW>45PP-#pVShxM40&g3H)wVXwMjFwJd`pywcF-f zJ2(&SK{#UhH(a#V*0c>Zf_B=~A;Bp(;}w0hglv&J#C{YY`uTNcdWd-R^T-n&l!JJVJqR2>>c83i3t>O!^y19?{JC5>q&=xJr z3aRgpRvi?oEf0UP9o5&(`1i=uR_}+*Yy^wG+Ye#_fesx2MMQOtKB_yQX_~S=^d-R8 zcX7MaCP{?M@ieFin~cR2617_#XX`iCrJ=(INrtzPMI1qBm!hgA!u4m;*;^$$NrmdE z!Uyo~rf$yf_nbv;R5-IG=x36QTB*sYhfl}WfdC_3HoS4vb7`d!-P>L2pPb%TfJ^VL zKorU5tPRWm=d!N?sxWU_2tOG&xskxf8#T$lvDUQif9FENhalzL_)I_sLvoS=3uh{x zF3bV~W!53`^OlYC!5N7l5&0+a0RDybq+a^OTlU)cEfI+{P)?8CeF6 zaF@bM3QuN-9mz@?de_y}NwR1YAKJwy6_+i+3G_9ephjse#11gD3-?$kL(k6uTeB_z z)#u_h5J(-2H@?G^18Y4_tUSJ+?yF$s=WsY6L;{@7xwK#Z?kTI=A#>9SS>Bj($1 zJ?x_+HXZGP{=d5UR2f@d=D>2m%+O@$yt#M9p(_MMm6=!maI4{xU+Sd@XdadMf!~Nk ziK}UTpFpyyc$$t za=&!?avtxc_{&t7+=~Q8)foS&(gQ&3D7W zIVc@~8ZHtoQP%a&3mQgV^M|taH#JibBaCJ+C-LGm_!q4b$6|Iz~&Y)z= zA=|+JW?ss88_wBUQupiR1J2Yo5;(#l=S6?=m8PN29E$edZ8dago{(@b&*t|xYGmYn z;!`>267BOj7L(62z!73YEw(_|NGs*yGgLDoTJocoa}sOH0a66Ue&i>ID1uYXmLiNA z>%0J@wW2ImuOW%aysf7BmwNL$%I9yebJ4StAU5%tFqsdAGC;gVORDB{v!W-g8(yL-=5sEa9x!bS zOD3C)iClUza^zfUC9a>JFE&=L3q+~E0@@ph+kU1nC*ADu8!1+vY>{g0TI??$9ji&5 zrpmaQJ!UQv<8x3`Tq?j7;&ul&tYvXb78IPUN7BzSK)=~*N96ZhSyonbY4g^x7zFmv z#^!=L1}80&NRs!aw5tHpmwQ>e;^{g6ash>tV+7Q@sID7RTt&)vFEx#LkK#P?*s#Br z`w2(deC~TKW1HJ<1zP6UlYxuTteld+ak2lOOX2|b_Mce%)at{pAkk6Xk*7iYK@E4N!_=*YqxeXYii~9)MaOK=s!N|330UCf%vkdLpnbV#v%3 zU{p^!uZ6>M@Yu#*3oz_JJ!DB_0#eb8=;_~!I~q`^vi_*j{`;rt%dDp2Ck*OtcWT)T z*Q7Z&zm~LNKP%kb>%O*VmHi1KYCH6#lAhU2ED4HUI|a?SWIDR~8sliGyuyGkjF~wGMhR8fM>~Z+UiiSH0}`2V zm>yEyK-aqU{FuInpQv^KV&@kew@}bhri3@}YZFCdmDaa*C(PAtY4o7EK${CWj5|>duXWlN2deMS0M-RId((J)HBciwiK``_f_x}L@t?m=w!dag zqpvSMrHfod$lEOZ2F~Z=#Z0Gg)SuIqBAyD`a*Rk@q(E&HMiRLSs3riFq zh}Pp8A$Ot@?n}WhT2x}GK@jg2!o2#aQHHPQz@o2=)rGA8TR_y#poX+z_m%Q|QO&*b zESyQLPMxkW)TqT}j`g7%po53yI|JO)$ivZ8&o}y?^31DFFwo4V%9SnYssdWmRLs

awe*xzcFq{DRzYc*KFdB&TVU zg$CLaXcgzo#EnKTORU6BFv3NQFbln}3M77kt7P<`23i%-qQ(koe4bDW0nCKbLMnnz zz}VPANZ7(>i;)WxerBLP#{+4RDYk|a?sTNZmV=VMbFEq$qVK%6-YlN9@Ab@kZ*Nte zkz7*vs2~5f)w0@drQhw0PY6m(BFOgOh(NZfJhTjr~ypY?*?L`kZE@MOwrOLwlt*AwyZ5iR6+o#J!6M#zMp+&!MiKHd(Q zIWE6&h!3H;<6HOHj=$_l9W^NIk;?F7v@B{_b?I=1Knoyi(~A}Cj!#}RfjJ4mkG3q2 zdV=wmhtVMeSU}Ay!YML9qXQGa@k81Q(J>Mb(g!e5Oc*#?QlJ+eWrQr?4zC96ju<&~ zteBQgO;Ug{aTC3h|iY6g$_|Zxk0tjkHtP% z1WubjEq#7(jl2&`zD{AW)ZeV0|ZnV#w2zq*H?{fskt7OMe z&P5mQ6Zjg8Zjr!(rmK;4>-|fUDUiD3N4KTBkWX=8^=RdI|I$5JK%em`Xfu8ODJ#!M z>QD$(o5oc;H^1R?7)zK{pdD`GT6m96c3z{bxr*c6R<~ndAM46O9A*3?yc7GXOekbVw&$bUSa?K_~yyK6yLJ$(vC^ zAlcGdV=jqc`J??gO1?-?*Luawldg5g%aY@*dwWM&Y@dLlen33^nmHotKMa{vWXpKLH!Ch~1xYT1JA-L#-N7{=n&_~lD>QOcnf+?AK{5}=3LL3h{h zc|IC*+ckfYaHg`FE1cK3j52+EN^4;X^C;ILH?Ph?G;3chaAWTDjJ;=2t zWo>z!5h8pTf6($<4S%-}O(b(X(7)z^UP6L)bwy@GLJoi}P6~R+5i;lAiwW9JCuG3Q z=Gj{J!=LDQ&1bY&;Ew4=N8Oemk2koF!eTmXMDj{H+HaIU1Pv9D$RS6#V`!pcmoWeY zH*%$!6^SK>M3xu>xf`8EZl%}ZkB*K6TUrk!j|+~;DyrGB2Y^F@IQ~im%1v5po5LIC z&jC2U{#aXwPJ!RH-_(KLWF86@U&{3O{0Pdyq--vY`{W#bk+8_>AFO1KFHbrAvl{l% zxo_g`_s+BBjoe>~XN`!Nw#G}vLozww=XGxs*@h#eCj?QR+K%Y?$Nn`A?SXwihX4@G zv$P(jbgYSuSQc8a5%yb&OE<<@=FWaZ)H?}(7}P#joOAAbs+Em7`fY&3l3BH3RjACZ z)SiPIp7_q$<}8?bl2Hvq_8u!@QG24sRJF|xX;`$1<}xhFdOuy#q~(b{tQ-?ZEaD&7 zLz$xg-Y%g3{M0H}F{Ia>lS+k+_$LP8y9#aLihdXuo9D7YgN+GbpctG3*3QgRP5 z22tcvgfkGC-IWO;t=Ak=2uJCy-zKjt+$rD#wt?t*Y>%DW=7V@$}VWKdPdrT}GAmwVSh+M5cL|1dMqgajARx;H~w93MwSx51#GdC*HOXymNNnYGKRN%hw%DY@B?jN3pXQp10-<2>|hMfGG5*J(_SwS+ME;(e+<=Mg|gYCT-{6 zQ8QhCgogy2b8iGh2psWkcr(yQ6Tj-k?Sqn$l5Bn-ImGQ7)ZCDrIMWu3?5_9)HZ>4X zD=ZsY+Q*5N>9@GH(tM8HMBF-nb_A1LjR7uaA9sA9K%41OpmZFKyWU=whU$xCKh}z| z{flU1&0#2|sXkLM_oZRj4QQY}067el*RuAG{X7u0LR&Uu0(1MeCj>T6U(p(*$}sdC zK&4bYj9ra!%1>B00HDDr@jM*{VSB@S5vf7lY;KRO4$oQ0WlanRCmaMFd<&LM-T1mR zeH+?a|Jz-Y-nR2AZaUFVc8t{9bfq^pdV1qWo=NsXH5OD160ERTWv@cHH*9-o?y7MLzxq~Xqq-(v?UoGob!I*L$BE7Maj%d z%34uPqqj>O8k-iR`M2+jbN3jP`bS(Fy;^}DN~SeXj8%ad*}?Y`hcnvr4tI&y2YAKH6s`%I!p zfauB^-ZqDyaOLL;z-S{AubR6B%~+N2uF_d;R#h;PAMg~zAcemYo)7y5l1ih(bQ zEdl4~Y#@Bhx1h{l+v~UA!~<0yKh#eQGICfmL6pvWj6QGL3^Z8)&tMU^k9tB5M5<>L z`-*9bA&X20R!P^G4}6aL2>|fI#l=NMI_9wV)zP?t>$vY{T{+dFV8S{rsG7l<*LPA< z?u53s_S`cN$WjDRB zxwE z0Nm#r3$IE=R2v9!?0%ViAUnm@G>hFRB|V2XKW7*pp>dj*>-%Mh=raQlh^!ey{lfCy zCPNS%R^R(bWF}(E6~7>18P*wcHP7Zt>J#N`KIMugTbtnHg+u3jNJuPC*;|-q*6+b1 zQ-FbPjO52HD+I@~| z)+)biCRc9?;W80iT4K^-q!(jl#13Jm{9_Qq#j1=p;=dGab(=xH8FZe2S+7qPrWj z<@y3hKx4=Omx z%OT_nx_H1+erkQiso#F|C$|k`CDYwU_}aDr#i-#yyeCkmdzt>wmhSIFF)K`*i6!3O zywLdzL+r{4cG(P}R~!>bD$72X#GA!0aRO!4dLH7_o{e?Ggd6aSif*TXs@<&8%50_G zuC3$N4lhZjXT(bSJmYS+?6K zD)d}ervI_l_MrP&f&W@2`sX?StqAr)CWYhUwV&PGbAUa;uQH+59TT1vF*Av`Q(ur% z-w;w)Yswn>5S#xuBH#p+WB!5VeNO#TQkm%`JU5f=fJ0FKxS`GbM3Lj$SYfrub#7x) z%k5ysit+Z@L%IxG5#--Grqq|bR+r?!(c?-_%+uNy`r{`BHY;mMf~k=qwk^gkcmap> z8ugSegECzYr_0aj@tO$*&Zn+v2^1q;dyyiR{h$ zLA{p9!{tfp-3IHE`{RMMi_e;fff@X8{1B$oKRNGc=LBc>_lL^*9^a>JA-N}mEUBN? zLMSW&OY}+t>q;QfmPujrZvy4VwhirbxNlE`q6gSnr2GhktfiM;6^ zXdNfdx_-QIF?(WyRhL5D7B2r%xcSko^&D0jGe*c9PsFvauZOJZxxngERzv#wk2drI zc9v`jy@izSyAu*%yrSi`$vGU;G$w8}Kc3c0%+=lRT=M|}o?&H5FB6kd%E3=OPr#3* zxYds&+`-r!fF_(139*i|`|0ny$dO8Zf#?3W4jlT{t1P3>tc8Cz()GJXg05ql^@eJl z>qOwRf1>fTYvHe1_moPBclD|ko>}Xi*BGu&d)%ek|263D8rk@US&he{W5wbSId-al z09y;J{*i>4k1=v6n-!^3y#|?iO4m3iBXL7l`GFFBsVN?wp>UG+1 zii{s8`V6CuCCqArFGstMsiyTq5%^p52#9)@>6`&#@nuc3Ls(>K7a)$!uwC=b#b>NP z)LinJYB1o`9eHpQR|`%{^{IkgbeHX`QuvZcVgZ&s01=%uo5G$HQ;4R^Rq} zssq2brNYMGJn{SaiLsf?$Qtz0uKu6>G%kuOi(8U71>aJ@kK@ZfFG@3$6IOflJRn{bq1{T4apU zc2?!m(Fr&`wAfih;_d1(R39va4~*X$@X80@St;#CHaZYJJ@TA-4 zNFAS<{KA>$M;K@qa@{W}<;26&I8adn364=2A$*d_) z((F$>Y>|{9eVxoNOTH2yo*I!HTULw^7_iwY53~nB@VGgf+{jW=@$_A2^Zr6-&!|jJ z7=+I$$bzA z(L>nMsC@eiit^xw+DsuX(G{YE)KLdOl7FlZZSjDOX?lA3mP&F12}tvrQM4y{+`H=bi=>;UO-@`KtEZp(r&aT8O^i9Usx~4rHZK6AtjM9oJd$>)%*1PJ zgk896bUCYSnge?lpXahl% zgSk@IH$5Cx3v-^(gdh%z(vl^ljSOxje|snOn@+fx^0_B>gy3~*AfWODCPNRm8nfC4 zzH)i4TQ+zE;kGhihop2OiY(j;K<(t7fV+u{eGzd}Wd?uz{`+WUy@@=G3FtDvw? zH(wRO(*#B;M&QGSv4h=p&Pm3RBNk8=2QZ{-TL$-zEoR0$ldW}r`;^sE>;f>(++m)S zOUlU2x*P*e4(Rc66cj2~ypxeFdDSOx{@N)QKonY+)OKJ~mZ*sAwHKBxC=Y&bf-R}( z06Z*fce&IR^P2{%-x*FDiouOu55CS-GNjbPbFKMvv%Jn7Mb1B6n@V6GSA}L+-3M%f6o9KsL~4%)4Zd;SQiSD2gy_8IwkC?V{7!<>v)Z)merL8uSP*2Nxi4{*W z>{2&`^j*2B1MC6J_MZ|A6xj?Roa%+wDlRRLkEoqfZTNdfz@*9Yw)Xb971fqE7Q5AK zjFxhLgZsH{fVi@wti5B(htHDCTZPF4qf6^7LNWS?0q19FureBQWCzk$}y>DAAGO1f(d~r;~@Q^?)o>Tu@pdA1?JGuSO#Drwg`{fz7%>}Ph z-CWg$=Two0QW>2o-kF#3R;6^8wNv0p58GJ_hf}-GGbvoH&C9q?Yh-YW$=N-wt%iOQ>DgT9^(QwCEFcK%s1;*v7gPllxHLwBI5N``5%SpdMsF4cP%6+XWn)yb!Kt%QqnDY zm6VnVmGI2A1x}NldDr&fY|G#Ki6}u-V$QI}4~s?lv-ErHY849dIm4Q^#yx7c4n#@7 zhrNe{YZyAWb#+m3bvGCIx@@CFhcr*qU3*nLZ3qwl`80kF5Ts|2^;%g|?B!CsE@+xg z?<=DTB3#O4k;JWxBcz9u>dO3O1)br-M)&?>D#G&Tfs~9uxXYkmECe%cXihM> zS7u*0v9WKe`aM~wCoZ31vCnVJ?o~;Bd}!Wv=6!gqf>6gDHTAu1 zn&I`I&)m!xzJqAdXU2;1#9E{}IaS+aaifpAHe~pCY@klGA!JqCwHPVwg-PZ6S7M9R z18mDDl59d77r|dJ+j*1sKcnNDEyP_yzd1{--sO}%C&wb|k`Cg1dq%X#PxlH-2Ui)PN_VF9%+2 zRsCmFJMRpx`tuPlyIScF!B-?*sMy11qm{1lDgIo;&-NJq&&-i#onZ75!EBUo(dhN3 zc}0^U5%z8+r3*PQyvP%c>}5yGdwb?XAUtCnH)WzsUz%r#tz%H!x+Xh1KTsGDX&sx; z8h)rN<37W1@lO1$Mzy-|y=u54Itckp!9^nQ&&i{FMU}9}&G(BjCPVJp*HI7}Iqm3v z^ubh@U2n^TIQ>z2tJ`E(a*5MWUp5~Eees{&xwm^3up#z(n!>ff#_&+$`{<FgtIW`|=C^K}1~ebcBCNjWZ@*0N8536-{UJPALB^f^PHE}s ziCk%VZd@E+nZUwOXTx(=%i(~ynTWD(JO0+V7NxHHK;UZVW zHtodHKw+<|JLp}~P+1c*Mwe@qJ|wJ{u}_d>=WP9*nWNP|$2@yFW5*IUBW}FoAbuad zV@6h!vL4SU%$##vE~i5bB=H?7y+W{F@Iql5IM+r9)Hf|wCN|w#R~|i4D8?TXtZmwt z)!;Vkx1e{Et1f!f`+&|Blp`vF!c(%nvoGp2iRtp3HC4(&OiBW(+9Bl}DAQPm@}e&% zUZgb4$r3iAfqi?JyJZrc5)kTq3>(Yf*JgF`cn;Yu3;uDmokK zND*qdwZYL{5AT<&efs3PyBaiG{ByYt*Hsx?R|s^tOrzX!5|l*c) z&H_66OoFJPBMjX9^iuC1T2zk$GN%t^`^zhRSlxlV3{vYN5J#>mxX81WeJ6`#=^0kf z0knHPtZ)7x^HD-js^;E422r0w5MEc<4th#+_f%Edw9V$bKgRG6v4sTzvq3IpW(>Pz z)vKmhrT1FFZz0EH!X_#-LAz$Y%26%P8h*%Cf+O(e6#~1>E}@-8f$2J8yRk;EYv7HU z;K>{Q7IG$^2eR5GdvluJLWgs5Cn5yR#_zE++Og-$c)hZk?&6noH^1ppBhU9}Xc7DSFu-9#E|i`rg9NNw6lf%(Yd z&Z)0g9qpmLU)tMfql73z4cN^rm2W-$we5HACwqqWlY1;q{rJ~-)p?JVqv|!}jqbeK z)R;KgkKnYQtFk{M#B{fvC1>Wg5$tg==VT6~BiI?w zkAy3BL*^ZHJib6t;r8sZ;mDV=#c{VdVtr0SU;hql?b$_&ebm0F;l?9-=h>0pep7)ig&r*h^RO6Ba&6znZk2y@g)I%mt#I|7Dkm0P z%p5;J3NXzTS)!1Xnq|txa1F6_L$;So;&};zq1y(;yQ_ON|2RzyZObOT@k~oqeBz_} z%6LkPnqfxZTR-j*xu$wm91{s6?%i+cbn$$oky7luBk)U8kT=r89MGiB3yV)DM}X|5 zhv*@?tm;!~Uyuo6(`G796^jQXMMVInt4;;@7VcngFCh#R=W$V@`H^^p!_D~A5BM} zIDBZgUA+)ySKM1>C~YEe@p*n*ktR7RX7fVCLZB(1h)KCvle2d>uv$MjBZVlr(1K0E zw_ES`K8!HQ-7!!vXeq3!aqgd;#2{kk4Kc>$Uc}vLUX;XEivNukQz!T5eE4LA-~ST6 z%*-_e6#1kr@h^vxCEE<{7q-e3K|;PkUfMSgmtTH7rqE07hGHS6DhK{Dh=i zs$aVh{K#Ij)x1dCNMN(&t&qx{?6lT~0s?wXL49px|NiG1Gem_uRq&sH1xPAhC|~^# zqxt{AFVj~FY!2lw)FP|c94JA3mXlV8f4_F&S9StDj6(RXlw?STdO9C@&ZdDCp2ee@ z9Oj1)Ac!4fiv=5~tjB!!t{Gub2LtCh`OF_pZFxqQ>%N;E1ccAUYxA_1&&GWrFaY|y z)?lbCa;apPgVE`0T-)3^ku9p-L=)n6M;y@(dm|Nh4Lai#n5#U2i2;uJMMdPZ>wO9Q z=sC6Gy_dmm!iT<;IrdH5vB!Sz`Tirvy~NQcD0nfe`GilD?t1U>oBhWTyBW4UawU^pn)IeH>Q$v`8YVLY%rNEHT-P2@T=|BGUt@Cd)>~F`c^I?#MLS} zEeZE3aFJJr#_z4m1x#SIK|aLS6YGcl4A7wE*Q;EUVQTcP*o<0}?jgj{;NM`SY%k@0 z6Nq9!6WeNVp;mTki+|ozOxaLXb-;kyz#YArSiMwga(!8K71D!;=x0P6JQ%XNwKEk9 zi)6Cy9%zJ3HSvg(wE zv3gSW`Y`*mDj4I$tk&W*lkJ&9n4gW4KkcVD2_)(^i4^Gd?%W@D^R9n;V+1ooou9NW z6GaW@x}wsWz*-@Z)YVo`z`CuS+u8=^u3B;heLg3F*uUL58FZ`1F;2+k| z47Voh9;nEh0y%T)nNJ&7A6E!t$n%f1Ar)+tWhyi+p>*`5kaw5 z!ab=!4WP}Qw~a)yvM?;aeewy7-06vxB4TBAR3rY$R6UpHSCfpLNNS`Vd<6zd6qo+FFK(%^Sl1uwtW)hCnPLtEFhYz+Zi9#R&-5m!GMgK$vg z0Zkq47AzldVg-muxPH?l^7I1^{j*Myu#TvxWRWII{qQhkq;31VHeYsPOi%Hc4j=$G zL9vzVw^xLH)4KWC>ehCdlw?Bayeo_ef?7fd=UY^88#=Eb?Ty56zCrxrlJ})Vb$u0o z22RB+$}WGdl6dQqYbM!?Rn`fZN+_K~lPV^wegY;VodZQ^Oc`LNQWY$(MN}G_FOzui zz4Ge7kjy!RZyp0@PkFA{CG}G;IyhK!f{dd4#?B;KyzTmfI(XZj&*pbrMiSnJ<-@gm zf(=8j!}MN@La;l|44!bqFUf(m91OVv{8u+GK+~r?@r!;p=d_+6w2ULniqsV@lmvvo zt|z7?HEXbokxGhJdmcxWSDm-WrEiHZeZ!}WdSAtQCHklNC#OAUpSCq-5$P(jedJjQ z|EZdvz!c^V&tCBI{b@U%P(^yV+nbVim7xJ^|3Tx0`Q4wj+2wtpq3DIQefW_$GLCK#RAnf?|UI4wIBSI$k`CK+-)SU(bD6aKx_^{ z=fGzoE?3WrPxx%>nx!zO@AQ3~Dz?@9u*K74VG@xo&4!nU78@GXG(gu8!;%>=)$WGg z&?1wL0g75(i^3Z>U>1)1yE$SWS({j6L!ItT<)miO_ZxX{@n_j{(8Rm?&Bu&ELnhze zr>FKNyq*sQr(rfkuioKh1rp^2X~XKQ;d63#nxOBdZRfx595m0V+@ouLv%OtgHWRR> zpTq+#D=SOlGNZk0-BHi6DovW=*FxXS-27^9Wj<7xxQwUtsZEQ$q2M(z0=4rqz3Iq5 zQV8yiE+@r{#=Yt%PM1O#{nIdFTO6^^j8w-~YOPc@&O)2_X85@Rq{u2oN@m~nUS{a@XkcTiL7y2jbMH;NQ>BT|f5ka9o_ zN)1&FMtV`2ks<<-B1Hm94K|1%9Rmae3>`s0K#*qCY^0=6141ZLlF$N350G#do_+qi zbLQNcJ9B65znSFwver9mCF^;g_xEHsD|XIfy+d8<*`s!-rRs!pE6d(&uK5}V$`0lz z*Vp;gfTze#nN?N0{2Qu^;aj#Y!7C0>mZ?5w-c?la$#e0vuAXe?#d94~{ zx5s19ucDw(a0gp?W|G`7z90_~5TgIMp%SZxBDFi;$gDY1m_zfb4Eki4y0R1l{LZ++ z&7BfQywne#|Lq4~B2@T|n(8y<_qkpF(3m)O`_iQfjw6+qaR&~$_{6nDk#OhkSj7|P{ zE(%!e7wj>$potkM@wr+IoYpwTtKh#U(~#MUfBlg12pUqmGIdk>!SLG=@HCdm7^X>i zl`_I%M-=wc5d3b1k@E7&w*j2(l2+ksjsYrny)(VGd;M^1*YPY3u^bG6I?!bh6`buQ z#8Q<|Gu0vq)(y*{=Q`5U2-e=^6UqGBJ#DXF>bV3Xxo9Ugn}d@gP7nzIdnO_16hN;y8`A&i4wNj#7gKq|e zH3=2Z^rEM)jHGu1(`B$i6#M1jDcOpVl!V0-;yZd$Hn4T8&$`^!2>Q&u>&vrU^N)73 zr`F^8ngcoya2{~j&9p8jKn1fr%#6cL)}SvjolJa@HSA1b_g+6)ND?l)E6=xncJAE z?f+6^L-pC!_sP&4=C;01GQxZVi_RsgE^j3au76ZtrDL7Kqe&qI$|t8jy$9Sj6({^16<5b!@Bzl%_^)q?nFggr5fTQ(gRI8N)f9s{QHA zzy@V#e(B56_tHGt4F7b=s~&jHM*hWY#PBL@%ibM?^cOR{oUGY(S<`FZWcE=k+c{2V zvoHK*ugE#o7lHSqq&q+|$n4z7S6i0f{C(s~$5L#nWw}EQ9am+fvqbo=sY(ca^sw^? z0L2YHE9Yax^+{iRSW^*FMotYf6|d+Zi(LskBrh0(DtDc%zx%}wHhDVG(BuRIT5l(Z zgbBsT#ld==u`r%PGHtcY_xT*OD#lY$T@gK3?)=?n&~w1xRzTBxyxd_R1=CubA4{Nn zOCnZ!kiz?Cij~ST%G_P^2g;rg_QxX1WRf<;Z!N>C2U=@xwMV0G$G>|)ay{qG@Y#xd zM!8NZ+?tqp4V7ANDHKpUXd;J}w{!8B5fy2P57$*dKUp;zwCKwdKu|I}yyeiaH`nED zA8h~Lqo@fhPFzP%dotkCYf(n*Haeyl>9vxM5XFlyuUS^q52epN@9EYm02nV~y2PX@{l7{X={f z8?umg_^=CD7e1b){5G6XN+bTG-!-upo$a`!r#)5IF5K_Z0?Wz-k-)(f2y|*2j(pb+ zqH^}mQ>MmCjk&P7Sxp6SK~2;5*?_|3_m)CMf2NS=8Ntupe3J=&j<%K!+{^Mlni`VE zMK;GeRlU82NQTD`4FZ@0Ywus$TMeMS-A<$Xr>Ew<0+lyLsVOZQyi2c!nw_kWTRQ+~ zY*n!z{68XSaEQr)ZtKY8}k#-D@wKm9mJ#?lxyOHe% zD#PK%Orx#lhFAWu-vhs7T_q#1gwRTfL$h|$p+iN?ZC1W0s-((Fh*fDVwUT{eJ%W{h zz2PZYAhb(O`eP0~>m!9;#Q(&dgh94Higa{ywwJejVas_=V!@&P+5t2^dS&H1P{>V4 z*zN~Aq5Z)NgY_X}9DNihO)G-5@zrEvhrP4(VSA3gL~#iAS%k9bG-sF56C-If5@hFi z1hWfK4)A6JRLO&uxpQ<2Ce>l=CE7AObw#w|e3{J_hHl~8PIxxsSy0qTY(DtA4^v!w4Tl*d!wyvn~742Y$PQY~(J9`DZ~ z3&cMMeVhubFQxmnqhLU{&GJuDow~Ge?!5Jx4?M={5R&n|Qqs7U)jq9&@-xI5*F4Y= z>6Jnx%nh-SadLTtn+upsjaJA6w#MlU*`v?~Aoxh(s>@3F#xg zgI0`GoiviTU@we#YU3u)Y2D|VIfHPxUe`;;+_vTJNetCK-nGL~4Kbwp5n7?PAQMb*lb zI46yMq>FJ{z%mUVTF$8h)_nC}`MBm!K``i?eb$)U{(4+ZskH|<3n?aeW%X^(N%V@y z_AfQEhlI)E$3!zaX9UU}q4H~=P9o~zLewW6YCLuPc(K80>ncm0rtRJ9lZJSV7 zK??P@s(3HOP;9lkAZP}Cq;razmO!Th2)#u?y`vUSs)H}#P~IXhBz5NJ)=up%HGZ8em0q# z*>hc#ILFAphRbGR?G)s@XEVDyun!~q`j>-kh%5_cCs}cF@linrG8rRL>gLd1Rco)o zWPpS+f65pAEt3kA)Z)FSa;sWA3rfpz?}h?I+Wl0&ci))d+fr?pEp1PYwK7Fx%Fnxa zL0$tk-)+-whxe!B%A1Ddj7J4+eHHtMdRyG)mnP_XxI1o&s`e^PR`subFF1UZ9C%XE z>77+|P=iue!~A$IC<-~}{~iP@t-BDj84y1m3o=rf8Z6OYZZr`}6F9XFZ=d~YJ>YOA zp}KSi(KK_Y8FzbbX<<;raR6wuaS&Thjq}$^GeT~>O*IK_@UKeZz$gm?2~n(vF=iTi zH$`3G@9zTt8(tC=BbFk@zqcnOESyt_oIRA&*?csbmSrxzQW_-Eo7A+_gC6h5-@q$E z1S)8AU;v~1I7Gqz-ReS&3oH=T-8OQwJElYr#a>9xDR?mjT%xSIn$?Gq+J6tm2ZixP zJp6K7`LC`EplYS(kYh4+%d8!;j7aeqhj)GXkhT0x8g}_Ra6#Z|5xbe zgVQIZ*k7towe=DuLw8L%uXz79cxdmh+@aiJ8 z+58tl0TOn0%7VxtI&sf#ERN-{xA%B93H(d>xf?y?Om$H%Dj?w8?$!WaZJ}c0^u{mr z8Q?u^bHUt?jZ)g0le^}V-9n%~LrLxBY{H+!t})e+lmi{D}ngS+qpWEFNCSOIqZI&?cIvJQ5lhaYi`?(=U$lUshl`d!XCc8I>>#D4_<;d9vzAu10)mG8WipSu^ z!pk>oLmClOJ?^H9xBzdDyGk-xrN@7B1HbFQqoAEV^UO5A39~#0; zYH!2S=)>29M92ttKkb~8eUELbWkT#RT4Zj^;AIRn+&8x0RvCRU;{khQ5p^%e>o{!x z18=KvcgSAC@r>J2m&KVruEZE}hV&*EnrqFd!OuaN%-PDBls4SxO}NkS;#y$X?%qB; z{+k9JIHNb^qs7o4H9K}fYn#BK!vtdkDdW%DaJ(e=BhKv|gU;)}2Z>GV6s#a7>#IFh z9lN$KZ#V0zAU~Q2eQ=tSFCtD^pk3#dW&-dD)d_pcJ_hL~`xj3y-4J}CdN7`Dp?vmd zr-Qpi>EeLEgdtstLMlG;_Ql)#RT>Ah?=OrtdnpfXy=Vb*tMnK5I|LY_ug>8!AhGza zbf-j^Xj%7$M{tHn0$phoNLI}mEOXUn9^cy}k#HR@B=XL<+0lH;;+ca(4eSK8zygi_ z)~Xi`^88UIeoV*F-X;Vk`k8Q9+G4vB^;Yg!3GZmmoT^u65Js;!6n$>P6#?ckRU;VySpb4EVwghfZ&6> zev{`t_mlh7_v=)hpQmb1&EC6KE$LpXyRW`_N2n@;aj_||pFDYjD=#Ol{^SXY4dV9& z6B+Twpx)5{@rUH94wigUG4f&u@qlV2p(OF-Np>zGz`xtlb4GVDj7q@ecHaE$fSxn*Nj-2211s*y zKbN>J@)XomejK9ouV&(KIk?=vo}VQ@sW;bEzsr#L*USHVjX0U$R~((L9aqiGwY9F- zFt~tN+fC<-c{*=G8l-=&pbtP!S2*VM)xZFyY7BK&+cBkBOLTzM=*ssj&A6umI&&uf zG({pz`3gBJ=?%EffOp;3v+5^UY%hXRt@{W{;;~(R6<|{SOGvMR zSnT@ZO1M-d=x7K(yW0O-)Oc(`#WsC~M5K#Ny`uQ_IxKiTaPMk0%;y_H8L#Zg{^`Ce zk|-q@5;`^cEvi3^yqqHHIZ-n=FfUI>f}7OmMKY`RD}@L`@Kp{8}f4VtnDx$tXa|N7^fcW%enp*#Qoz7rr*cy^T} zq2l`qVOK<~&t#>)>`*<6?|k_9*||nX`XS`erGn#;Vr$Z&y+$Je!1RxH1Orho@&Ezr z_#9Hi-e@aP^LVIPrN{(J+awh$jP+xr428>|TI5gX9jbTN0TK`l6v&G0pAOXFhR}_vZmFuH# zyr26D<2&(*Tdc^U2mC7FakH1IRUF;~4s84K&oMU9M`fIaf;(RJ{^*Lffyagos47aO z;M3wF*89p{m{6UP(Qa2})s>p|&GER{6^GBs^o0#rFhFhd@m$LJL%``Q!|c}QfW2Tv zq6U#Rl=3hSx3=|w`-gmlsBo80OJ@(;&Qq_uDzDZUAJ-2dyXiByc3VRi<#Xwr-(cKo zIya>&?N`&y1|A3dcQIB5Exc3btzykCUwtS6VmGyW0k_xehvMkTkmEszS)nM>iYlztzHiUf zaSt-BTR%?+QEOH{#xCX~j0F1gtKVV2pND&54050NWKjFL;4&00*HFt9zifxBD{p3$ zHeEa(*Og0iK%6Qc@6ea8@9`^^J8DB_9cSOmo7g;V&W=2Oo|PH5Tw5f3Xb_Dsm& zn1Ia@LQKB8)6~2<7Ha=rFBGJH9TbW&8`e5TL=YTqE8zRErNIuuVTL;qr^h440kUXq>D|LT^G#&yO<@$g5K+Nw~Nuf{n_+*lul|QQTC++ zizr)>JsNT*n!8nbH?dj1VD>15)%k!iF&uDa+!&8q{wVTT622L5xp$)a^&8F4Z%hx? zk5|AI664eDiJLm$6z!Ms_qwfG>u5MReUvB6?$$jOURyonKmFJUrdU6$FFQR?{;!jJ zf=p!F_pLCv6-$!EWj^sx8>z$Kp?-@Ac6kEhG6qL0Jz1K@jQ0UmuyIK@Z1VL~HxSpGU>WFYhO3IyfQVO2`XC#MsjMINqpM z-p11^ZBvlei^sd)!^ZN4L!T>Yr_o>8Tim@7mhq(hYf_p}8KQ~IALVoW_>$4EgMU=wktlf_i zlJC|eE#_A16_^PQ8y+r?2Pl2KB%5@4Mbg^>Pk0Vt0|H~ZSEn8``y1BpeQ#YO+Wu^( zSW&nL=uk4PS}(it?hF&0F{dx53udz z!4oWVUyi(zS=FslyC}qcjn@7Z5u26Kf+aCi2&hc) z7<=N2P0dy2$25A8Ii4FOX=Pd-YP;CsMycZ3cKG>6TcZJdo1d+UcMdQ9MaXwrqhmS9 zm%Gcy)7SJps{_zy9S0$vM@2Ngp>-)7Ba1!PoNH^T(%TfiGjjEtZvXN{h{LH0l01@7 zLlccxbJD?8L&K<6{xI&XwlyeDVJlWYQakf2_Qo_WDBth?s8`c>#&P1<_Lyz2C` z(;mhm%^_+t$Czmm+Nibf4u+ZD84M7g9aALX8~Ks1<39EJ@!|2J%%k5p%(kby@fU&Y ziw#QZxA&BfAMbZiVb0Pv)?1YY967_DuSUh7FcpP>E3u63TFP zECV3bO%b*J!$QE~hep3c*AEPTL&>gCnA(wZQm zS;0M-yi0bCm!GVLxfuWJ*%7fC31VvjOJ;2tR9>E%u8INLiefvS$=;_>&BlwAp;~J& z2su2S)ZGmI^&$v&!U*JShDiIVhE}nOdgk%Hxoh^~i3^;gMR#<(bERI%%_0%q*Kbed zqeAD^>+^j6_i)8^O=0cJgT5z4quPc`N;BXk1dat%abJk!f;-nm*?o`nx7CW&ttX2; zQqVsAH94U+Vb z^pgxZ$QS$fNs=>xpcWJW#(S#qaj4m;ugQ+)ad{N~sxYYdSxNy!I^+zc8{^cnmjVz^ z-Q5?c@=jV0^RL*4pk0aeK#CiweB@tky%n6M5BX6#N77XJf~nNzPI4lx$Cd+s7g0ru zJUI)s#?_?ZF6(}jYTy%~TKvg;qW^d9DfoZl_iNsa`oA0S3D!_50^cn4{Ow2_*%V6^ z6vn`3r8c_z-zY?G^$B4%Bo9LWuWg8_7zpEHD@I|yo2UQRFg!@MU|aCt7c@ats6~M8 zPieJ9R_=etLB#Q@Aa&5cH%uslIGikb3W@l^p739C_%p+L;NKT<2m~ey_uQYW_E`4X z>f7z#I_-SC?aWdxJZ8lI_qbZXqt?#b)_MDjk79SfsPD(A+s|hVz_C!C)|-!}iMorYv!CNoq+3$(J{E>zkXF zwexqorIpGVLhfrl5u>8F`yu$EP2WQPZeTf+bIe2qh9U`TNoJ{SsUm4>8H7{KT!z8E zw_1&SDW2Wyi=}?NT95Vc{I^ERsE~ADd`16?^%dtU{#U}U&x-*%PIOg$e?|8!7y&LZ z<%B={YbOIhss3o-O;Vti3fSC^-EKk>lrPPzx1;QRP50J{@01I;@PrPnbTwr9Rc~%_+Uq6% zlxYiIBiqa86jbh4JcSDfNv|9jc}#J}1F@PF%n`x7Lhf(}Lj6Nofm)-|KooF`BS0KJ z_lhxk3iGvX{mUONQa`618Q#&OYtCq1wG3xNzF?1v5Kj~*Db$aIBNqrVw8_Lc)Ld?0 z)d3wvM@4MtYw~*!{6C!4tA_$=IINX+GJ=iK%+RqwUx*~39R&_k$ehz>zkz;^Tl+d=Loc)vX*%C zoUdYiIe}>ss@7_rJYVsgkN{|~661X~2ViO`?ImXr`i>U)ViX^4&-Ej*a8Db_^M~7?=?`Hewt5Fq-_u5Z!<#yjq zl@;We4es-qM~?~-j{er{%m+bP1t=L}`sFrRGeqZe&lSsxk3v?{rGGsfaP6(_1`ub**;d@4fK)$iBMMr83{&E3bt&2r@tgiOBF%- z&d>34!gxwJmRyiMTUv$pbgp%UoUn0%%AQ?B_zYB6VSOBxKI~n?%p^?Ng0Chlo?yp< zKMD(^S(Q83JAJ}vq;QvA);jJ+P9j|vZUhhiKFw-V|FgKrA@P{PXOUsD>|Km?$&@%& zhCUU$nvm3MGz{;p=$5i9A>V34NddY+wO6vQ-c93F0)Q>ckmoZf^b6frprm! z#`C~sPYrv*du6cJ+sc6LK`%-@^V!!`6=dqe=g@Y$B#xQ;6tG$K)tvf|icP}^^aZ5D z$gg~ZkSz^ysd}oWkuy^!1anL{FSK}THGVdWQ7(E1x1gt5i|;%I)@lVKfBJ;HvbdwU z@fn8^h~Ac?pXXN4cR)5*lF`LN@D{esg7kZ%+7?5Fb@a1r))Il3-^^|kJ8W;qniohS zU7H#8bx&zhJzgYcdedr+}R3NfUbRRJ~(znya1MuzS zy$dJd{4$wGqlS6%Xk8QjoA`6u5>)WBOeOuS13UHy)e=FW;Q_(pdBBn|RlZcxI=38i zzfU_GNv*9_I^LQTFSR9 zT&iAizOOE^Y9Q1ex}7KsZYd}T8{{Kgvfm;>>nKKHeUU2|XZ^Wk)Cw?FVge_73rk`7 z#ctc5M^OP-N?kRXV3-S&;d4r|^l)M@ZUgSm=1iTVQa24M8qN3S7d?TKc?O_C{j(u2 zxKY=8{Q^3`h;954WcOzqzeWbu=fb~yc(bQic5F)>aF&`27}g)mip)JPm@Xlnzyeal zFu1;}u8Uw8W|HxX%(?uyUVC5zRs+i&V9h{nf4B`>YHPHOn_Zn*#9z{DIE^uM(`eaf zsUN@1)lll=uAlMdE?|>Qum&4UN5-9YZ!4zpQ~rL-O2rdR_9Bj8Cr5qT1}xGF8znmXp@X_b`$jpXqmwQuOxMM|Vj2Ra+>)93e|NPY-|H|^MYAJI4dfY^>64oe30&N0oG^9?7MLOad&!PR? z8r)}9j@tk$1b8N?fCll4b@9YT(0ymVU6MMs29D!bf$%7UtJga?jPnqdJ2)c=;abY5 zf>dDylIZ+Bv=X=esp~MnJHAu)=$ac3eRhX^?)1SeoYz~%ajoTHc`a8bIKglzBlnNK zIe;N$iX8zZ6Y*GDUi@J9ZuqjKTrW3TwJ2-pwgm)?HMhTK{z$LksN-4a-lZ?QaM07g znPx5m0mtUN_#ot|D$I&zE0o6CP0sd8Wh+(6vUOTJm(Ab*YQe98qd=b@slxx53V02Bkho&z#nSQE>yaj1W{ZCt9mZbdYa!q@$fZfca(bG@f6 z*0M_NU8TM-wL$849&Q0LJGYAdinvh!swFN@%p!n?a;C7Jc3-u5pK_L{gS_W(Vyu2S zGN&XWVbYq5&Lm~Tggv#{a%hg6IuWQ~VD$6JL z$Oda7r%<}L0fRAqWyz{iCd#!ZewM~i168+K=ymBQew$$CXNc&NtfL>{G5F2PrdZX3 zGYw@8#+mh?#4O`=Q5cO}t}(uFd8cWuug?H1de$wiCKDD@4ccV`^u)77QNL~CB%EjA z`Be{m74MiE<=B=R`FiqXEcADM_VWe%>=0wf zK5_Qa!-|^@YG6s~pf}SobDR!+YO8gWi22H4G4W`mVpaY(xx(SV^6UMuUl1-hF6zl( zRYR$rS3pBFhfeRO;v%J337z6_sfaIiA3AP+aIq~{A3k8e@bl_=9;`B3M_WgFNG6Cn z1qEdVJiM+0NE@^+i8`;LKD7DWUbvB_!Qt?|*-HGErzgLEQv037KM)JK*xA{&)YtO< z30cV$Xc8fs`uHS*Nmwrw7hX5Cb2Ho$%lD`OlZW!m9W(j372Gr3c}=ZOB+=)wek}XK z9fw~n$ZaeN?-@;Hj~V!c@M2aBia!XH& zo9>5fIu%9*Aw2ve-W)!6e^HiW_emMXLFMZBXHE9Sb=K0p1SbV#jOy^R+i`rgYgz)SiZ}|A-1?qxS@qgK- zVqH5Q+7Vha#gL6;O(?yEfHIZ&tp9;Rx=0AJ>Lq9YzL_B6CgjV71}O8jtL&Z_rUV(l z)Vj@e3onuSjC%R*dNcs>qwtt+$R0UD7!7(+c@5Ze)>h(XUhD85BqNIHj;z<-Hiwa&tC40>Nb%bX@7n-qv zQ6z#1YAZ%75=BH84YK!)IH_X=o&-ezHRQGlN30^vr%Jp7?BCcSwh8fQ{~~;l3FlN= zb1#Td(jZ3NMCJH!zpz-c69OdYdvYn&?_bMMNDTW0H5LGUwqRXmT9uDBV8adrHvJp} z8F*9RX{yqHVR{OC(ET{{NLnguhgtRv|CLQ!m_;anpg;o79TpbCL3y}p(J$zkY--)} zGy^+CZ2W8-rUt72DxucY@K(~X<8%ZB`v$)$`}H6z2=x73^c4Gf<*~7C-~0$DS@|C} zt11O`vm8MCU;>&VA3nmAstBT_qxhvQ>juMF)lZ|vrSDu*=ROo3#<-b%)*nk!nT8QO?0Q*$&D%fA+9ueZQJfEjU8A*TY@9;} zTR}qg>cyK&CQci_F$_RvkG9>7xGSE-r(;lT*w|7}yF;D+-VgWr1LcOk0n=D}ds5N9 z?pE663VD6FLUozI&*YdM574tQXhb3m%CyfV3-Vqd(zU3JM-aWKqc?5&(@>+>(=yr` z-~ZTZnI|ABbH(j9&Es6`wT9g!KdQgf{xsHgnX7|wx_o6A{5Aufdtb4fsPw6u8Zm{r zFI8$zALC_~-1jBQFGv=x;DSip}~Xgud!UP@!8BS$t78J zP*|AgI?-os&`1hp)SsLuB8xKAVxMf_$`>)+|6M&l*?1mHFCBOb2_7k*Prqh-I;Y=M z`zjr~^u8n^zro%i8W-~J)U+IFu|57BlNq}LNW3h*&5N*Vh$c9Az$8mlz-7Y83=;2{}H zh35xL6Ro<83DJ~ujUXtzIZ65WE%#glgoVI)KgE)rXgb5W+T?Us0bC03P}!7554`oA z5qazo#uGBqon+@xmvJzg!q9Bnjr9#yBW^Y4{e4ghH^C}v*Xe!MHkT(%Ak`UT(NVw_ zVjDP-nix#r%1?3$10;+~>&1k6kRPP*mKSIhVCc*FYH30qQkzB6o}%6Jj@+)DNG(b2Rp z!Y+-3aPR#wR_~B?kLl#urcgfW{gi|xQW*Z+ojN%#N)~&Dt{cO%$3D5sk9{oHRJ-eXjZ}d0t<|4p>m~GPCj<&8{7whs<7s zPP}Eh?5hC>9ZOESU4z1y2di$CVHk8+8#Xbgj%}-b{!L^^C^5;lWWGYu_1p$r$3mhc zDIFduDZ0Jv_j%6<@5dqoPseblHpS3|M^Nz9J>xYMI}&!tS>d;jfX|D+@9_ z!Z2b%%Q2?tSdW$jyKf&)j^`wPWrpdNOKhjdAp3_G6^j)CLJEdi1G`5s=6rQwX_d9DDg8$*!(0-LL;pYpt`d_LQTR4ldN$8sRkO{{gd)6!RG}h( z*4xaQn|TGzyq>CdjB>|Thug#@4c{QYLF7=;ucEql`!EdGcW$>C3P@(EpkC9}QWkLa zG+ZvDgMD?#mbHXfT16bS-T~ZWPN-}#?Zf~ueeFic7$E)-q2L^Zu`zfBhvI3vC#|p7 zbK#%rfpmjyFW=l$dnTm|;C6H%siWH&oLtJ=eon|lt3PQ8SFN2qm?P0Be0Q%u%x{2~ zDr4qR*See@d7!B%xuvDgZOQFsWE^nRWcL7rmNDc;m+648$rdE{$)h994czykyH=_z zb$Z2ziGGLsU+RFFC&T|q9O4NEk^2GhYeA8}3arK?U+BFVkj{b-v+$~eY?FTVX0gbi ziXcRRfO;2outy?t0k;*~TD6h< zn*=IRS_}<;(*MfmB}~mhU_lo1c*W`$8)2ivB4hmh3imt?x7bz51! zBM`Ex`l%ees0wH1Aj5F|XpjSWTC)>E{IWdyJbP(e?DLlr6N0xtyRcqiSZ`BvN9HS^cK%uJM@>=U=YT=N-6)Sx6%dD`&x;ffHNNu@-Wo$?h9#b8%e!4NfF9x`iSF(RU)Pvwf>@ohWzz6IqP$c$V0ORm8 zc+r=P_tK@D;5z>zh-Q7Oi+wmEjc|lD2Gwqyo)}d=Uqs&n%}ut{*EXqQjxKbRVigY2~s+n;_>Hre_>9 ze2t1QRpFrnGT|s$8*nWg2Y~#_jB24VZ7p-?mB(U|^Az>us#ENpe`LWRX6KQ#A?kG* zO2P1;fBU{--`~6w1F~60R8P%wY0mF4y5N@Jky`Czs7=I$ROA!06GEBi8>CGpc6@4D zP>ZaVU|*?dUF}a2^hY>$l^c)fKR|SSQXi?-dc0XU=DEA;fFxePC~L9WC1&6&)z=gu z`_PPerrpdm1J32K$zTgo;iC=+iWU*Hm-D3E{l% zF)DQ8=rG1wd4u}5_|3dHx}in$CB)c-XspLj%t8(F*F_@{PuyhM?etFx@ur~9dweQb z4)@-(-PgBjh*mvi{lXijNjVhHNC$pb1@{DFsmnxBek#wRq&Q^$mE%pKS;MJ4YkDtT zC7m<3pSGm^eBeGl&$|wKM>h7uQ_1^Vt#v3Qu}j@xi`J8XZkh@EV{|L(JB7C5JME&1 z-_uio)UJ`z|9v1pzpa09*tBkuNf$u=SX5v zjy70_qMhrbi5gb7GIFy<>r3^It3Rr??dYB7hijq8Ulc_6dpGnhErSxizLj!1Z)4s# zI+Yn^t<8*$!l3NrYV(?rsI|@;ZTY+x{nnQOr5!xhv}f_2NdVMXQF&Zf__;2tpF~_& z1cRmDN<`4%3?GFl?US)gErL!FH0WT<_zz8BqJw&qYGjp-UPz&i$JOOeo{c3F7O}nu4yAA4i8s2EvZBqfrqT8dteQYI0&U)_RqX9Q_q9E9db4p*46-3Y&O--{v|9pu@Dr3e9 z(Z5Q$=nCO1xgT{#nHZOAGgMfyO z7c!MNZWRXd!=JXB=CX{5p~9=MaW}2LZ`06UHW)VGQbU$xO3`Cc zUu+-cWh;17t0lfI7{}_cBC_{W^ZC*Pj%&TS=YvSql5DfC%}XxOz;BI*Pp>(D0GQ#P z$1&i8mc6fshBy+1XIF+Ee)kK;p@Ip8e;CLvVJzJY7rIZ(d%tDbUt@G1So7}A3{JR1Xd;);>Ruw0>JqOnFh6toHLLlnYr0kGlK8|B!1Z@@$5hS~(NiuZeUV-dIk` zS~QL78oJxB6P9+2{uR80YN*qsSC!5+o7txCaE$rNZP|_427B-pJyr@y6G} zwftK@D}hX=5S*G-_n@U2LFFh#yJi;VN+R})u6_$5?>bIDUuP6QrqY5Lu&R{jjp z#u?X=P-|fWDphmGtR*bw-()rtg~Vmg-+PG_Wm{rs!pfgd^`L+j8T>@;$^n4OcAQ2Y z+$C7FMMaK3f9TdqdSy!LjhS3`$LnaD0?hSr(zJ*4e8cLk*0+@b%nI-UNr{!l>gcVl zb&1D8nBOfTi98yu^Pla2i6G0=LSf!N>sn$Y7mY9%e$#6Y*!F#oOp1}2)T;Cj{mv5| z&1Pd+8$PYc8!yMjW~wmtix8NmjmQOlfwHz1aP%oQw-pw(LHaTwn&`=>8)^R`(l~AE zvEw+^o6k-S$hA z*E15UMhb$d?o&vo37Gu4URp&K*6{GBuG0^YsbT z$Lk0Ol0nirRCKb+xcPdceK|sMG>DK7!N{k-ds$+WPfV(brctA3lapg)J>sjr8vBP5xDJ4JCYbmq}G5Pcg{7I&m=gNgV z8A%n?VTqvyD=)|$chl3So1{0pS_ibl#x)^ehYvmPS|YR?jRLI`jO|RX7f@aqs*%3* zm2lNn!Cg4mZq`df!*ToyrOaIjESc3%vyELP`Q8e5KVx~z)^Fzj`5+6z8{2*|{Lv(? zZLgwI#AO4V9;`AC=)AZRm|E}@yAee# z))Wn zS~`)^ZK@hZZi<%+uGzJ8Oyg~yF6^W#U3UxxUpIs-q^#j{KMmBe0Tc*LEk_P3x0)~P z5<4E7+`Rb4$d)t6yXjvR__~|32!H}s8k%NU2o&tKV)riFYUZQuxXGB8@}1e0tftr{ z+|MI+ud#zAuxz=>#rl@FL9@`7Utt(ip$;NQl**Iycj1Vf_rt*BC(asHP>aQr*G(Jv z@jbNHtsb$vne%TsqLhTK@i6Sbexx9QvRi{8gIiGy?4i4#Zk-5Ib`vI8WBdy6vORk2 zM=RCixQqd4`TX3e<0N;DawROfZSPp%=wPH8XxzD1fsuJ_bPVPQK*3hefK5V;eJQg< zgM*>%$2dkJXOW-2{->72BnvZo3fd4$R+G_B~v|MST8P&u1dp!IV&Q zL}olml!cY1ckU0ks}+xWe5Ot0WU^t6u(kYy&6>iNWNQ^M%wB7wwdhIKX6;0a&MMOS ztu4v&-;^*3B;@qXF?sZ}Z(kA>81`CPahT+^jNF#i1P4xf4;TescX$7B=> z^!nWJF`lj*)c84%#q7De%h=C~gQjtkeNA{_~PGJKFt41}E&o9p;F5K^@_=B5x zDc#>-;~eFtRZ}WhM>c)&u{3jqP6aI5t;R6ttu@j^7#Z^Sjh&!7wkDI-`i5}Pg6M5c(5p+qYJ$d|!y}Ld; zAml^ePTs2CbBvnN158|21t}bNl;2l6500g@uer_ZazBar;e*w*#dSuaYe1|{42$Kg zLj>2GAuu(s+>d-sRv!Be`a-T4o9f4dH^SGSHWOlgxax)XHegnJ(q)CDOnDK1T?Z^} zF*HN^0y}&PVz0P~-cJu2C@|v80Y1R9mk73G6PYQhK^J z$oh(pbP@$5;=!yPxAMSg5q_44aD{G34rgxBGbp3m$2VoiRZo~k_*9n>bwzRySMI;J z-AX6z>)t%a*A3MGjXq|T0^kr}?du|?AzSOC8kyRY2 zLy%MipLQQR*1sM)-1yC`)Ml)Bh#ke~KtCvueIXG|mim)&zDNCT4F>fn->w{y#oeWG z7|6QmvvF7oq_{j|Vb+#@E*h>X^881*ryzi?7}$6-81D%VWs&4c!=&U*baJt81G0I( zPXMCgxX&nh4J505U{cH(eine)y;1d+e4Ij-guvahRZX@(tz;bP!FIb2K%qn!@qFAG zR`;(jkAtB)D>7`7DBpZoSCwY$Q9r@#Jgwo|Js{R_NSnF%$`Y&AnAQK_#d;cvEHg>X zZvlj_F!k$&T>_FmC4>t|dn#DxSsR1i*^XI>_)cjKF@LgrQf&;2U>0x@Y6brQ6=;P; z;my6gIJwdKDZ=E#m-X?JG_OM9*X^@i{pBonAXm6I%~ S(?yt^(C*$^JhDq!eoh! z4B=`xMVh;jzo+g#OEjV=7(R5oC9htTt=myUJKtS-i5bF1%!-(z^+U@3b{mUn<%t3P zd%RNPr`*v(*FWXi5_DW@1!wzCPUetd0FHC0Sao}QdwqRmouz)jiQS!*;+yn_*lHd*Q9 zb|)axWd(2k(1Rso$^+586Mif|YRndFzBrOJC>SVP>=T>jv4fCeB!1JlY8CoWVBLCo zuGb;b?B`!bYGwIyA}yMw`8f#fa%FMImmPykQHC@M5@KT#XM^)Z&mY+1H=S?x2E2SpJMxU#%wAz2=L=srl; zZu@s{HiH2i0}t{uYQdf)ATMWW^+l!@uT%c`rLXwL8ZE!ELR)M`r%GVp)W$Q9HuLSh^8tq!Cp7H@6DKa0VVz}T_5NL(>Gi7a-)ZeL#@z94k$j8qq zQynxS4iY_cskWf*TQ$?#ogWr(7LIBK*JSHeVVdx&p6p`o2T{b zZL;sEV9rK9m#vrP^aRem3v_@pH(*^f{e{t~-L<*p($-FQ5U%0pp z0fPUJn~}+P?bg9R>Wp9#H@uozxHv+j%2NUn*c^4t6YCFxfAI`Cv;Ebg(lwskB%^)U zG%Mr!#RW?{-3HRObGKXx;N699O8r~9cVeSHqaTWu-UeTQGpjaslKD-6mhb1c9d01U zWI@Xb8-+`QhKVn#WVS5I^@f(776X_l9gJlD*e(gwmUDG&hnK+&Ra^939ja2}&6tOTp zOIaJVRJ8>7%CuMFqZJ2BXy5n7;J0EKBuOxZM$`ctQomijZtW>5N5_Y8Il;G6#Z1-y zl*Gy)s#A$a|A;q;3$;qgna<2&Fa~Oq3a8)~gZ_q;ez*8+X%)NqtgA2=kUwEHoY*C+uyPfikp_3x^rfKXz$H1WB~HsRGD-9J0$}_&5Yo?2-Z<*k*;NdHFAVr=+~0X8`uS$SgQE0RN&qRYb{7>r1IYX8b7eZI(5k5S zHB$`&O35NV>x0rHnsPkUJT+#8dk))d=?q)&jO0nkJMYKeMr!_p@4I9OMe4(c>v-&1 z;%ls`xUB{%2I%U}8eldoqFN#hNqcw){mPSb!Sj2yA<+AAIgYQ#vf6V>Ox?lI@(qp4oRYrPdLy)Q~Y z^9ec9EwHCdSXxRaB9As*c{NnTOXb{(Y@aj2gPk-DxnLFeB=XbE#O5FnCEzi6O9ZcK zx;LlcgSJxHjQ-+*o~q$QhV%#{@o0s>7r$xq9d!*pRk8vX@7g_UI%9|IZ;?zR5VoCz z-gR-~nIP1YwlhytrBPR));^EFcb)lB*Dum2Lzb@ot{tywF%+NMkW&2f4mh{IpV;JE zTC(@KEN>3>KC|zPX+@LRL(2Ctow#_sG9Z&=2P_ng`JmIOUAch`|=20 z03suPQFeC|Z=PxQ%1q$2Qe@q)&w!?Wz2fL(qn)U0a6Rd1CNe0=ZK}Vl@eKi<9?DY@ zZCN&(tAo!T<&L?}Vz=uD?K~1cFnwa1z0v=eclcLVHm23G>TuT=}5m!eyzASf++S3f&>l|De%GtI-1+ZCcK0QrMmaPe+$rtGzMwU~s^aa6{PqNgYK$#K9I9|gEkbSp-o6rj zTA|nyl5GBDm}EaGUal<=&me=MpQV4~S);RTZAzfzxT~s%(LF!mS@m13-i8S7=(4ub zO zypaT=aYn4+*7X(&8_&yxgmGFzatrGoJGwdd3?yfZamGd1zGb1mk-MJGW$ai?<}apE z^?6jF)c^jae*Fiv->IF4$jJ#Q&pttvkrA>2t9Ot2{3H@q136g5+e)(fv6) zhK@ZEOOZaL(jnRgb@fQKbbJv}yOmxKDQxmFDC>^6L6e(}f+*VI*DWNQ`qCd|x}T$^ zEtRB$a+mFqvJqcB?Ty;RiFR|u{%B3+PZV~ALoMsAT7r(slV&L`-!Cj`Ul~rCBYb%N zLc@%^-c8z;qf;3n<*N9_Pnmv@?rXpJAoV1;Bi{u0!97C+sn{W4^40(l^_faW00Vk!QVoA{nWUYm~|yy=OPKEZ8c zNrC3Qm9iTzmRZO2sTc2y`mlPQt=$UFnJt*iHR1jAPkX*wq-!|TPay5hTdfeGp-pc+ zy`p@f9s9T?HNHVUO;9tpc|qM_hep8aWm1^U!ki&nP!_tfoU=*~qD1ozW-6)mdzvD0 zf0YppP+DKd{gZv!{N)VqY760q^Vavi_s343U*)75``+$(h&%wj>T7}81sF!6BHbgk z+cH2C64N@!kK7t5|Ahxymzx+Xs_V3&d<62&E3%3!1t`X@z(?}TM}3FuH62Mh@0(gS zgPhuY_n(9%bHI+O+Pn9m-We`u!6SnoB61V}m9sHXbaIPo$ zu@`iM>!gs6_;$;)L8c7nC2p`kJCKF**;%)%L)bhZWbecSJQWQnqrs9NblaMQE`BJ-%uKY_T&~6pyj3TdGJ3mgDY}}eH^=0gt9tbwx zVSLkKBH&yHIg_CS!JLN@FSrN*2cR{=6thw0{0R)v=a*+)p~PT*|LuXc@WR;*0eMb@ zk{CvnK>n3ygX@ zD1OD2riFO_WolB}{>99ft7%xsO9%0?D@@Q-ZNrv;B5 zT937w8+p1F8*+W75OD3Z_4irQIsQhG2m44wK^yU-t9D_{%OxI3kZnd&*zvYJI4gX@ zVJtr6{Z`4WGQCY4gt(RBuBX9WfTppHW4T&@sjXNQ+WfkScl#>2=Cx2fp45qPUjBn; zpBY_pMcPB%%h@`Bg{)Rm=qz+_!kQA8aiq|>uys9!1>Sxb?5}+8J0p>Wt|C_p zABs|DrzZL=`$zEjBZ{D9_r?^DgNHB&jYO&p|7)Z}tn(R;p6axi-B}0AB2N4EB8VIeD-~e2ddRdjR3FQ7sYsu#qeX@9_E`(ph zV7vKkgURF$9-6yxue?g8XD0Kuaf(-g-x^D&%#7U6NH>Neq9%*N-E^cixvhARk`eXHSPjM%c;k1YB) zcu22YQAuRf^yi|!;GEg2>^;xEc#W1rlH@DXdsmF#B-n%(i?`qCbLTE?RznlB4R1-I zk*A3T7{O`38hS!R{dA z#T*#|QP&~j(sQ5!UCH`d_2fA|)lz5G#7NaHo zu1^Q>%bT0~<%EaedZ0EEGxmn%D$Cq`Fh>!BKd|%C&GqiL;EiV~cGJl^#F_1Jd6>dG zrh|=QNE6yPE{zCvrP*(hg3WD`c0gA+R+v~Dn6FjFj<@tKh04=uOAo{symOf#@&bC6 zUzXHJ-h&DSQeVn#VzZe_)#nn%s#Us$##9?0MGOuXqiPFp)Wm&26j!3n1JG>R<3by! zIM-P6=s!+F2g9vnp&PIkub&e6%>zUIh0}$*XK|tioD?BAhL_Db?^lF&xr-`TtihVw z$~bW~)J)_sD-nxTGg+a^gjNx+nGl|__1fO|^AQ&Zu1~xZY!C(wijqm2oscA=QDN|N z@6D#CP`b-W>DUq5!CDc4qUA(5$}|^NZo%0q`@XN3V3KzXyxj?ZYqAS#-X}0f*raR3 z1;0t_c%7H?qb;`a=hF!&f@kh&37AD$nXx;wSw{-*`6lXn?9ldjBGCB|XbP@T`kKyW z7J=Dc8RgV*d?FPd7$?tw@yYT)mcIOE635PiSOZu%N2r7YVchzF1fY^*WEl_rxn(c`H(9iynH{_ zsAwj{9qV_v5N34qn|L70If15NS}VDh4VBZm$D(N6Y`Xfo{UfI-)R1?0GXJxw%2bE_ ztQ9+pE`IwR8Xc{&V$S2zO|j_J)j^+evL{%V-T3u)`_y--)cbvc$d>Gsib{9ehtBzj z=dQJxD)h5=vg7T^%^^&dv-@)zhRM@xj6@$&t*1l;lF{P2crj@gz$N@Qic%{q?@SpUS-t@lIl9VyD%Y(V0>C zz}OD%?rxCo7LeXZgVG`0Y`R2Bx?|I&)TT=iIBR>~ z|NqbDJkJ^9yf`_=Jq9lrx`DOknrqH$Uf1{gxym+nw9iww{yT@ zyUP9{Bb7M&mqv9TgOVH0-;!Bl_Fu(V#G$ecuvjiKiyHhl>jW^5y#bN{ZoQv`0M`fN z$o&k-GOM{*gX4<*qL^f~OWFt-D5Dd;-k?V$1laA7J;^`kTu6ZN(CY^nD;B_CAM^I# zyl^k%nzHaZ75u=Rt;(EaGlzi|CV1~_G@Gi;#~}@dJb#iaVGP~Ks1;dm=5sa&i^l=Z z!V78O563?6`oryz4-K3z2&1X<983-N!)p36K>z{ z+4vlIQHjEwxWD6-=ea*diYP?bOE=PkY6XDXqkkSKWPa*RyQ9@SryMEETNeX(#bQ!a zCAL|-G@xtyWsiMK)%If!>g#L3t}F?W#&Wjw%ZoxWKDx7Li4lb6T+8B$4oQ?Tma`Ni z8dKG--B)_rl|DBYuL`T zqG45#okm0bvhQb~&sE$QB=p|2m`6e0ex%GRhZ-7j0TJLz>&5xjdzi9(sKr@-RD=+`1`=d?T83c<$^zJmkt0?SNUZ}$L(rPj87)a zcOPn7uU_M~rOWni`E;sRD6An<$oo{FqTO0Jz;k0fWn}4os+TZJ_pqbxc$GQB(y&vV z;!@&ed%*XA(Ru+~Dwy|OlFMaI$Q#D5zBfEgN125>N_SVyiS@NExAK?sq-w;Y_EI96 zVj8q3!U8}I!KbMY^NY{{6+hK?Y6TW@8fW49%eXb(5}4Do{+3t zn~q>qbK%Q$@m5`Z)hE<&DJZQ$!>13fAYQPNFjX4+3Ms67F{rKf0hcGDkqRzA>J-YWq zy}0HRqOiEb=xt}lY^C}U6VHZ)K!(ZK7OH+IPMUUJW^5+Dsa^h-CRcEDcgVASW*ltp z`FYkhaQ0x^M#>nZK(uS6(NK++imIN${g{QEQ==fiBHyDZD!zck)!#=56l0OEUxF=Z z0m|3`M4aI;f<+r*lM>@?H$T>(gR@_ULJH^O7!_ip)O7>9GgzK4n8*}*%ytG+kq6mz z9jn}1UFsX%4&$&K8XOEA4E-*eVi-h3`5e9&&Us$l{MgC>FJxtP92UDch>LykPgTOKQ`h%;|z(yjZ>x_h%^HM&80PBrAO?g%UFBTL)R1-?}oHrg& zTdxu-NBMrK;0fO*qnGZkN~!XQ<+ZKSi!dSV!5rJ)Wpb*P~ACBpePcSxFTzfG8@-iPgg@jSUW^J-jTgM9zy(vN?op zMPdS(%yf#7X5LKcv3;W|sgT{`lH}H|Q$bzE3UcqG_)bd^8)4Ax^m7Wg{vgV*rQOA+ z#mz;x^y}x@Ou?zh%<-;wr}GJT5Pos6@YOAa?9ft0YmJv-7u$aKf#-p}{fk!Dt?|P8 z2{?Fp;$=WPfLI_^ntOjbvrtmk&csxC5!vQ{{rh%Qzvv3p#)#q952Rhsey?6pv??i53hBI=quuf384?^W>Ec%1G#Br6%K*Kh&)@CWU-cb_bGHDmNK;J`JQp>Hh(Or%o0UxFW3^U9KT zrT2Bf%iPBIgniXFO7xR@!b^S^J>@sJ4$Ev=kVyztoaN-sBqU4^ZY9hrdsa^d_R$7= zD7MZJobMzQHZ1#(uT;H(UlxN%e=>iBIXU%4vI4sZ{}>9g=G`*^rFlHlrg&x$P+*!I z6WFaP|LEUh>IpA0Qww;0c~(a&fKpm2Qsl%3!%rJmF()MA^UIx6?$szbzsLpMu8(k# z0#`r~LOw`))ycg35SnNDNaJB01)NMRw8q4OvLz2Vj&$3;A8B;WyOjlw zAr(&rTLDyqc`rMj%!_It_^ysYjrj-bzDFhxW_>vVsU)l7nzNcE!65jHXUW_0dFh64 z>W4hx7`C4VKE!EthsOn9sNsi1nAF<3>rTHH$BWDu+gsA8LZirb{zwNHrLq{7@f}Q* z=|0v3F^*~(l*O3tLrijGo=HyZ*Y8P$KuW@=sNqHDf;6-R^5S%8K-uBbZYg);BXyn1 z>P%0mvUf{y@;%`}n2W&Z{e$Ba zL6Q%^{@eAKm_imPUaVqD@0N|!0wzakS^U~0hUM{u-^?f2O{_APulR$6ZOwSwTro+!a4I!Hj|l# z+I}yMCdf-l_=7Z~Y>Dd&VhWh$CNX^5MMeFh5a&d{Nun9(|1wdd6R$2l`; z7Jqd2u5R8sh+zz}PSSB!Cp+0bn_{^%37Lq_`}tJ;<@t>Pf0nS{Ugye^olz_Ro3MQN zC&=q`M3y~K*jQURlDOQ-@R#T}MywbZz+}YYkhh~ld}bdPse7x3Q<3J_;M0R0vQe;- z;}oh2_yCo$l0L|f7AU?rY-@8MKFNZNq$;B7G#X}pAp%N=IgKi713!)qEaCqK6Z~7I zXv6`OXI}njtOg{F|7{X_3KV)Sg9Y~fp>G8BAQX?O8@&i!$G;=ePJcicxrn=4mhg{$ zk_4fjG=Wg){-vM15d}g!k)m3~e_~H9dF0a>2;aNKtNq$h|C59)-=nYSy1Kf7?r<0s zLD}ezAoN7yX{qIxU>Enw92V9#zS)j(keO;I5@8VTd%kbB5{qa<-0MHJNQl72@t(Ph2aLDy= z58cq;gLQ%jxS|mLHxo*Hn3eX-e66LiN63@6pUI9Hge5K>Q_k}_EI;=BU;OS8P!PtC zZSU13I5-3{#d*B>Jpo{gNM!OjTRH0duWx~0x`M4vB@6HB$BD9~(qGEd z|I<&s>K^9A(eyvej!0m3XgW>JTmCb;snNh=H(UIh;!{Qg%r$jNw~MrYX2)}2hVnn* z`&WJKUjtZ*i2I49GW`A;{qKN1MzJ?W^REL86L3_~>rt`HLKqBNU5tVC#elhxFzz5bKRB;=jsu8(!)+*HtcR{RQ1D zqX}xV>5GYpQ4+aTGi~?%bplw8eSVIB25bB;YO~q^x9U^>E8~`>Ua8p%eP)se~R3{S`N_(70W>4Ba!*9 zT@px^NCP%h{6OZU0bnsdMi#*RguDE+Va0!eTVdGW&ktlF#(K9n9ypFKh9^KEDeL#e z)MCrB<=&kIK%Bx}hICLkPnLp=s}Pp-HT5({V;{CKUWs3@iSPdd5i0hN-S@|;_=nZf zX&bBNda%t(osVbF)88ZP^t-V}@gaj0DUI%ZY#qPfg4>;}T1*QkbKDm<(pR_sSylm1 zaVwM-hGhDTj#@nP9bxO-v<-12UZ(j~_FB>bVL-T52$aAQVDbKtDU#6yySM23M&?pe zDrE`B00;$u6ZJPl?+N}KI=r#14Lc0F@Kjo|G2b{kWE#)CTlhF^awJP6*7Zrh!luTC zPV2^SZ7G(D=D#m;Cw(}pp1x=1RU*n|0_TN0-lG! zCgc}5!t~d+A&LliwmYlm|GB{`TtvWwBvATSz=MjAvSdH(v-m6E5k>?&=-Xw$3HZOZ zArgd-)$}YWQ2SDEj9!Aj|%f&6P*iT&r)q=`EQ_cAj*F}_MZXye}9GmaQV4a zQRJ9jwJ)5aFR_D)htNsH{_L{smdGv>5SIhu(tk9DKv4n}Sk$$Z&C~u`z|(;RJjF$C zp8208_dg%|&w%_tJ41qgS@#h#D7M}lv^3+AvKjnd4aT(?f13PP$VkTyMn0W{ICR{) z(-By6lWz|Jp<}>JLx7Kao5^2zRUi}gCyZ@NRV;K^Zr!oz8y;35qc)C3fO8-RegI_% z2%p!0oP)>N_Bg=T+Q^;yFJ(9h16+oeN)Qn4zH(zXkjZJ|=ufqRQ*5LeEH|5@HsQH8tF`(GlZ z(@Y^hid74#pWfdD+#h-#*Jj;5CKc1ZAdcdE#;vlCAf-YV>@lRIWqzj(TQoE@q_|nl zv;Fld3jl`+snqNG`uh0qn?JJxD*R^pAfOw7Wopd%d#0hi07^>-GH5;SrGPPBA*iZX zM9p}+coxz{ohw%{D^)CexOc0#bzhxLO&`LJXZ_)`j+}Op_ZtDfV}aAeCGpK($K76) z#(IcGDe99*g0rp4){{qP+f!*Zum8CUEe%BGX;q6U$%tnPhCkEIHoaye1#i7u!wtZo zr7@tEC0IWxt!$6%F=2nR9{8oz_ZKJ3?_^WPN=l0(2LvE+V#X5h;T^YpSKT*}x8*C> z#V-lA}y?^4Y9hQ9iS678%LLgcWcl@Tt1l*!QS$VnWcTup|udfGW zMIePq-_v2f93o?u5VKa`kRUkgDD+Jbvy^UgHQ=`Lp7_p|D0C7n>J~_`ISN)ee zZZR!*J06gWLiVC?B=IkJawqVp0SJ0DtzPdZ+Wi3bn>Ez8x<-Z}lp&Z$Gv&HR3Wm+D zG98s|LyfYZAIIGx-*Z-UWwCS2?74Y1;4@;fV!$ypd7@JoWf$;oB12zye=kDa+RzD+ z+ZA!bMA@ssVZHeyss7jNsLH@D0_Q*f(t-*EJHJl`Ynij?{|T}mU|Jy$W`m5S#7B6f z68{ObrHy7#l10HIpYR9&hp3;bg|_dncwE{@7{$sX9oKT^Ut7qZq~;(ZAb%nm=LaNI z0eX;VwyumM;nqly6&8gPJ4Rj9u~mu9NHd29P_ZM{@pK*d`Z*-yEzPGgSIZb9$d)R2 z1fRDQcqm#j$QoP@;<@vmTHLkDp;~n6pMNx6U zUn>yf?UR(h0hvY(UlXb)fAy`OM@;Cc7Y%%l0>F>|0Vi9hOTqcEBU%0&5I+#&)vD8F zLZo8S>-B%tnKra1LnLxo^N!_Z*k)^B&z+m{8Aq%HTb7GR-g2};x+e<07*ra69i-nM z9n`th%CIreCy01nv$rdxtrX+>oR}{+&WZ&(Q#Bnp&uG5>))K{G#@?q$H^0@ILS~6D z#;jtZdP{w*$NjL)y|3N&8K2tI#K?-4l5aL7CMtbh)A~x18xmU{Wv|o-v|9o|7)pJ| z4Bl)%wuST?riXf>&7Nn3=hRAaI$XY1Q6a`6Hkay7ljM#g(dal;d6vV!YHA&_jBKIJ$?=5N{NzCOL#voml)OVYtTAOeV zDpdOo!tW%5doPS$C_G0kQBnLdj!*ydeJ@ARM5oLuHw@o5(&mfNXpeYBq`ND-EN^Xo z&3KoyEf<={g2uvL2VhMbU*MqVsS|2*dBSmYeLM0FEM$ia69(pRa|q$WhtMpC@w(ZG zO2yG~^XVhgv;CHAbX@(o5u4U(d7R5idHqFl?C7Xw*^Q$WpK8}?KL6|IQS`N`iVuYm z<&2dzMm^lUPBly)(jtdwdC6fv?!oGmh8Z`xE&n1|L#2m};@u~hle2s4pO>Hu4{s+Z zv{YNx1WIteJr1!68k|0Ad&m1oIaNYCs+vGNcgK0KZb;)*DBC@MU9h>E%_|nsy^r1& zLg-&n3#`+IX*QFKirxWwK&6&*F~MBHO05f;xcrIF8Wi-RbvfR(nLN?aWo#OyiQ!fp zCcIQD=IJ_Pglv}pw*_UnW@qQnofwj(2 zzP^@^Rl;W99oFNlUN>e}Vlh}EVgM$8P(X5^$b(f({ZjMAiwbTR(!(88Jq@ev*+*BA zk4|AAA2`5c@_}(w8QZkdL_h_4i`p_FlD7x;0d8s^|?0g66K+e!5ymaYSzJz>YCJQj8 zv!8lo@Xqq_qu$E) zxL8an{_vf9I8e@}kA3C0U>IU$e9n*!H3fBvy?}n$83#x-OZC{uRH5ryiQ;JG40SlJ zEYx@;1G_)U#$>OJ))MY1o}e6jH?eSUrlFhi-c9|PD5l~s`3Ec{-RC)dqs#=D-3QMr zw2%nt7iwq%`cDdHWXkdfLT%l^17G*3DlQ(i@+yO2Sm3D~?Ljgc5Y$c;D`)CiK*V>( z5O>$;;5N?C$|Z5q58KZ!fpUG{GweyRL0+3;$W2YC?{u*$TtbhSK~`o{m0LUHIv#4s zG5y94JmDdafh}QCv^4%?E8A%bzE|pc#t#%(21{m@V%^o)mJIQbM(i{{py8A@zy2=4 zGc)#17;ZH%ig_L^3m9@*Uv@l1eqNQicc=FRKemyqL*l{fdqF4M!}6RKYL5@o*5GQ) z1GNJYRX)5hO~cFP?6CgCEUg1ds-dLgr7jJj$y%^LorfE#r)Bm;^a-Kr{g8^=rz^T2)CDDNjIJAUXLAT#ld5CRT1R(>R+s+TzN1lEm*g*_Dno4P(yHRq*j!ojo^)+ zTRICt<5ap1rOW*pDNYo{yD_4 z+ae3Ww~PAJ2FAQ*1Yo$!*4nX4Ma_Vnm36kBb;7=N{@n|#hw_iH`;K>N!Jm{f2M!q9 ztHf0-NlFW-SdaI3N|)^FhCAIBcI$Ou+`)`|$04sSM5pm)wFp~(;C@dUU$0J$*-0d} zw{{&8ufOdHISTnuM`-_k1LQL!4wmm!TOKaMxqmz0;{fAmG;SG(FdDN1R`EI=*p*$& zHU+G1AI()D;Fn%cyQfs;Qgva=Isc=tLBKMy@v<&&WnzwNt%(p&5>S$9l$X7Mfrd5m zFqK6K`IW&XjN#(Ote87yNbm}ki|HQ47_AhCdxu*;6D&6sfkW<34DSZTX*kLlzF6NC-k)Vr`RrG(+`(6H zRu+J|=bppea7XcIhv)G+)ONPZaF7czXN^cuT45@cOeSB(VEE6U6@Z2nYuC|yVkRM4 z1o$Y=zrM{~EwS|qdCHW4O_bw}#qoZiuv`pEhbKq%Dma@tsVtACnxF^JOxBm6MsyRT zeZZ3q!6@OE6nE>HeyAfr@VPK41@oI$&ifvzWX&RuI01s&X;O*lDALEx8u|3{0e6o4 zZ4-lZu;D z%-bw*FT`*H?tEgvWX@fX(6jmdJdT)buw1dC+Qs}mMLibWDy$c7Pc7F`xK&$PaB_qK;rOqpfMZ}koqp#*iaT{?(+8XiR<5`6 zVlR5bG!ap8rvc^_G>@e|m&z@WlB7FbHk6wH;NpH#d~6F`L)k<@bb?q~EodrUz4)|j z!@Z`E#b?~?dBI6z9F2`-pf{%=R#PKwmzqK4z|)w5j7{6`nC=8MSoO1m!$}esUlPW5 zs84=qqZ>s1##3WIDihz@IOw^QsgKI2fBaIG)>cD%nk+vtx8T6V&2>vaipR9@8%?EJ zIpubuqPv7HR(yuh&j8PL5Pi$*Dc?uiZE%M-P_a>(u&UDayGYJ%2`U39fMQ$uBmVmO%9+q1=Aba|u zS$O)n9gQTbk9;vou-e zkAcLWZ9W{U>1?ry_hes{Z?}~7{H$Hxf)Zv3JHW%Q@4k!tVhA`qyluNbdV2Q_=nGN= zHEmhCi@iS<5kBdsz2AO%n@4KkiNq0w$N;Qd>jvy0&gHQl*~rG;zB>geN7sk6vlsIAP!vm5z4!j zc0a$>PPVzwL#;U+mJjYB59{NrO+nu-?2ZI&`o7P@_mbdmWvMtz)(PO?kL?482)0`0sH?1 zulx!F?vREHO|mbH%9}A|aBW!R_cxK>3jsF{wgZT+DtWQOj~3|GNrAh&|G9EK)6i=p zm41X6hz|fqA|b>cI04R)B425EZN%{Tj|P;zWgMijlEUOWyP9OkWCXO$c?&3&>EJmOOLQXdK+n`vv;tKdN zy82n_X}~YGd-LBU`XJ=}7l`VWd`739-we>S<=14uwbI5ys}lHby5m|C=oxc&uMc9E zr)g_e0(W^nu-!^f5(b(wPItU#yD_}q3y2|{*x|0Ja#INwwr!PZ*_bNC!lDB@a|W*N zbewwJjbvS`20ZFI9=&Z6DO@7|ckdaXixE&VjXgEI!U{lo!De+3L&28~uNx&Nz!Sq0 zm3FkDy};K1dL@SUoxN72d4-01AbO2OM5h?Y{78pzYqjgp@6gBZ7}7jv#_0v1t|NAi z#1Q&x>cISFAQyHxdN(f;_4D?nsK*>CBAqg4i0m>A+1|Y`#_8`v=vpdop&efUWC&8Y zhoJY__Vu+u`fS15VkIz*9ex`3Q$QR9gOL+4)Yv;Bx6TFJR^FcjRaOd88VqM;u<+>+ zSI6(cj{P4dhiq0e0O`0Y7Uih?M6TmZ?iwHa-*)%*Og?Cs^AK!G`-&hF^ z#Z-as5Ci-SZ|(DkSWE{oy}K6R4Ru{HiugTkc&lz^d4Cgde`x6cYpXERTec&X(njxn z;;!Kh)bOU^{$gc4!?~%fJpP)8)9zz-8|iCA3C8FZ#u#cfI`75Fb2N=kkjpgs6QGxBfqHwWb|2jzWYbLLQ3`seKn0w<-z`9>+v~*G!)sWG;w~6IExy|ATIm+>lzB+veA=YXS=uhe>q*vDL+iFt1xUkzLul7 z>@YD{X>wVI&a|-^2soXOoX;dN;MfcLQAVug0xD+bWB5h+p3bJwwD(=Y=M)DW2SBE& zv>7>(GJ*v6B$Lp8<#s-@HJK6i439QrpkRR!{{0Pwl0Rgkj4wxgLtnpVGK??tN|~y$ zG*UgL$qRk`D{-_ zj?7UoiV9HW)5~zdr8-eKW4+`!1a%z3SDDpxmCay_PQO@ama7TQ6naFF-(dfFW%o2% zDLg0zG0!lAURn3i=k&b$lX5!1M1g@s{LN4AG%Iawb6vDTN%^RJ&tdgf;ZWG;8o*n) zkyAdq;2sJ)z+nEea?pq^a$daTk$}K=m0(MTERf+17~&z`;5>liGh~q zg_peyCN#>$m4Z=?y?O4f zd3%pmnq1GnfBURNQb`;0!+USn!vGGG;l1gNB?clRN*18rk8UeC2T&e^wrHtg$B3a&i3ML~K2P(J^oE&KnaBCnGQQ|sK7yMCtTLTiYKjqmf89gRhIjT@J5 z@xSu9-wYk8N7aH=wkaN$s)tK(oj>FjQ#GG0r^}0QwzTZ?1$ey{wafc26XE8DbsLJfcke7F>PD8NfNP!@zj|zU05DZc zp(L?iY`iOCGcQCFx_3<~N~4B#n2MAR+Ez>ZX{nYd9CbDXVx5aXKSk5r{wx)W04o9mwKPHHQu9I?&qmnXidkrBjP|b=r zFH4w?4NA+fV(DwGX(xWxx=TdZ=rW~@>&tJJ#zekkA|I_-_(JtU{}Mx?CJ8!Syg9`q z$@w|hbh_RcLYMcc_z^Nn;kTD4t2GN1$yd4VwilZ~QCi0T)a6b>92uq$ci1uzW7kyJ) zo%C#$q-B|dUBpA|}Y0Z~`aU%|nte6pzy{0o%N2_V@p8GKR)peOg@5Z@@EUvLQG9agy3@ci_3PJAP`>KIHnX`IESPi0g3o zF7_|8&-t6U)sF|(6ZIN2>57W~!5>yfek$iFLXw(7>jlGP$d>DK_&Cb*9M4!15WU0R zwFvvhQN3%--wUOb>D_7Qb>N*HWM|r;M(TUXa4i$iw@||%?CQ8)^oce|E8p0h2E0#e3CR!pTaEE@2ijmN#@~6$mu#heJ(=K} zOuRbGQjADUs%1Q%1K7|LB#qJc*RKCWfEC&oR=1sVU= z9i+w)$Eh*m#um=& zQvxejKcZY{EeVDG&biJPdjcJ|XJ4*PspRm+p>`q-LSHAq^=*XUX3o*~jF4*G^aI72 zG!pLO-|kX`E{{%uBqq`1&(|ZjjxBkmZ_a<8-G0PKe<(`|o^icQk;MB3nD8PBq^X@n zdrS3e6tZfkAr>tK@b4Oh$x^-b4BXeNel8;m2N*nP3cQv$DMEco%$jxFl*@kgDAEKy~4-H1Hmzx-*?l|=vAZ} zo|+Ga6X7pqJ9Be>aJDLCtRO?ar;o%xHV_bh8w#;jvFHj^`zS4P{=AE*n&xx4Gr0!8 z8G7_MBwq3TkwC=#n~d|f2AqnC#c0nwm~_e>f7=WP2Kl)SVB(kXCi_>i(HZSld0AoU zsszRMVjbjWx5m!dEW-psYkKMKtzZe_WM*E=5%A=uUspi2eI7{=Fe`>w~W8B)(zId`(9+|RvcXyi4SvHqzrtCxPPLIbUNt}f|LGmmx-7&io^)#MFF|1MU^{|HKUlqt*_JD?sS<6o|bYOQ5x^0 z2xlcADJIcYnZDeu%9S6zA}Emh%SdF?W*JtkQ=Td4<*c|erN|@l>${R{!P_Ri4=;ZM zQUO2`O$P&fhMH?Si4SZLDhohy;(z*Mc;}*E>79d}oqU6QvLxW{qQ(HGwJb5ozS^!9 zO!3HdA$EFP@wt0#mD>*o$r>HE_g@(Fb=w#$1*oe2)jQ1BYo^8#?Z-erXKRJ3YTW0@ zR(*SN>!eynL7VQ6YccGS9kUakEroQz7X_1S-ja(gSN5J(k9@W-m}0-IMxHQ@%A>?(8iLul+sEl#R+u6$hK zib!$35RR#k5YYRM!4mxIoA-<%5{z*R)eKqbuwF3;l z1Z#QCcDBN;MSt#ydhkeR;Z)B!jP#0-VRZ-)=GaD7U95p9s|?qDZ=VbIyz&G1e>os< z#z+R(!x0Ab3X7!@io75Aec-;-tmm_d(Sp$FJHiX(Qj~E<&|7LPM=ziQJd(Kr4IVls z>rC8a$(p}Ov%HnY>Lq>EN~g0GRIa{eaN0cq(#%0LtGf8OYJ*=U$ea=dMwW|-_-DoR zvd$N6;FI<)r76VaL_vO&5M_OKl&%-QcM7xCdw%?QZji^rUpGiL@yJ0;dM)Jq+>LDW z=y^U>nIAv;A}1VJ_q)Lsco^i~+=c(C}Y zl`IFOTTY@FY$37W_?&5C>os{-YNX_PV4*ISXyWh5tUDrw_n{}G9apF!wE1S$S|I}B zH%>T-d!zmrT91VS%yXrHR1uO+=%uC&Nlh&s&|3SNbYqnxZ}*~bj(!T_Stj>Z z#J`3)&Jf6u&OeTY$$V@$$-KGb}*tq~?Q*WG1G6GX?fHZwwy$dc8&Xba0s^fK_M_S0c&#%EmW$V!^6)^)m$ec7-D|kJR;DQ?G%v&yc4z5T<#kVcN@BHN1R}MnnwvU80lKS+ zIqZDsT1EHy?2oN~1?1Y+LrK&>>N&2iFFX|^bnrga%)YN%!k3h?Vq?X2dS&P1x5E!S z00RBclVHwoznUwSU?3;VsT_^Ssu#SU72weyxVt|$Be664ANP>*V{?uVdk1ZXa!SitY=gh(D#_?U;>@vRLxh{lZaKAn*4`ISjT^j?1X%pxGvD zdwA0_4N(r!!7&kiPZ>fLTKOcPYd7Me1^Q;y=CaQAPTbJdHLnzOZYwQ$$+T^1Bek4M z$L@yQXz7$NSQNW}O%9M?6oj!~S!b~=Pq|BZW>ar0 z`X+u2@UL?Z*?rGNUV@df>MPy+71%}?dIzy26+(1q z@(CiqAH4wv7hN`vrMNX-M%4!~&fM3n!LwEj%VCq+QmQ$_jFgAjGRvM)4M@_JqND(5 zv(FZ88HP>#hF}gR1mR|52vUEZ-NA8HYM1o$!3~fcWoJc**shjv_dq(hIX{HGC=B;t zfng7K@x+vYc6$PDr6N@2#a$4~&w7gYYU0wC+}^4RWeYG=b4iP7Z{P(M)zoEfgJ?P$ZdX zA2v{&ROv|U$L8?fPNKUE{9{ublB1CSs@835C;mVt~@=Eos%%1eC5_uM@yxr?NF4F5R7y9YviQ`%TLK#XbTgUl^u`DXAj6dFsxSR&$%~IIpC8u4%tl z-cuE(T1jeyzS&v}YS1tGDeraR*^#QMS=4T1xaOu#OVx?f;wUOoCHSddJ%ec}ZJSr0 zJpV&}T&zR_JyMZj@;f^$V2D%(p4#yfcD*z4XB zYwDH++l1@R*Pb0Liz8OMcZc{q1z5Hh2AjpX%gsw%K_=F24*k4o7YdOolZj>@Yaf1N zN0Bbl`E3uj3Qz%4)!`fa0Mj%%CXl5Ufc!tx0HBWE@ja4iDnREmcY3U`aR?X=Xa*|= zatmf?UQL!+V$;$pzA9Ch6}QVOoV|GHW5V+3y+G}d1^(U&hD$!@1l=I33Ks;`xlmO` zDrU%ga50FF6Km28wRG1LP8LBIh0Wa&Y)c(flT>f*MiJU*N5dG^h|!~mJuqZPG&S=6 z!l%SbqZF^!XE)lTFg>J7om84n?Pus)x@wI>Ee!;GgsG!lJ3%s8Z3*ft_(}N@YZa;E zP^rf2VYfZ2PXbU~g zsEmyp<~&?5CtJy{GE(8;aN)6`i{cDxJ%gQ~y_qUWyi-S#VrmLOSlfA}$M z9G;lI%fZllMs5Q%Ay~+t-OI#~g^EDW#G^JdUFF3S_S{hz#O;c6p?RfPl#u2QNbOQc zbbLF?CGrxUessUGb=>^Ur{r+jeg$2&fv2)`jbU}VBP6#kb(Eg0WP|k0*|()M59dM| ztK;y^jiVMlhtyB0;Ndo=O|o)4rBN2ehFC=?u><^N%GUI0Wk-FqT z>k>av&R`kQB@G!mULczpuzql%2GGa5YlZKZ$E%r#rbOJ_!f<{&>ix)a#RbgAI~sy~ z00J<3gUpNP=)$T5Dc%s9Gh=JHdn&4MmXqW66<$kqvxDF1tJV5PhJ!(O>v`^iW8c2@ zA_@jbNGm7BLlLi7Zr;@8d3Hz3mopD0*o%Z%+ps2tKXL9H%6ZcQjAt3F-BeuQ!vsc^ zfh5;~Kp|_O&OT#XzW!Fr1Vo6#((;y(&A?VDYX<%S%v%z}Nkc_gph3}gCMPxh*fPmo zYN-$e-!ias7iwbYrRx{qY9tjaLA0G?r|4Dnate8~_#({pFr2xiz-Jq4>>#zyC*1vP zz#zI9?%+P)`l)cwnPOoQVuHqMk#Mw4@$o6U)}u<`89ralQW3T0`78U-^MQQMRz!v( zu;FO6m^dF4Hbj%9=8PE&+A}sKV&^u1^ZQb#w_~0)HYc!~v^|x72pfO$ViEPODfdg! zB5i6nR9;YjZ>qLhtpIza>2&XFd+Y~!X}0BYDCK5l`zvcH!Qwb{L?}e*h|G0eb5VX~ za!6+Gk?2@Lw~l`5Bqb1;_Y&!6Rb7R?^N^!KOI|AH?TMwCl9|xi4_#)tyuAjuyOT#D z&yFD@5(aOwOF>D&?CKwa$2Q_gkHcr4KeDZwm~+Zd)V&q^vS8Dhao7Nc4alhfyxz*_ zc<)nfTCY3Mh`wj=&3y%p=}V?_A|BG!$~4dU5PM6W5;(GhlTpr~DTL4o+dD_L8I zD&Km*QEt*@E0wjy(|<5$4Xyh=YnO~a%Tyy&H0$4zhq>7}VCj#QuJhHsuTz_-40J3rN{ z0Xyv@!0eJ{<+>LBBv}m@d?u}!1`!ckP$*qCaz7pAILRZO%g41~rensOd&o5Y1=i*p zK17dg*3PcfskM(Ec@!(6;u#5UKQt^%Lw&XB=jv>D=oir-{%{OFo|&n#kiH#9A!3Xd z_0Nhm#dhhk$B)0jda%8Dp@dC^AmeCN*f`MXt%-~!quyGzJCE+ynT*u*p_QQFw~2T4 z1O4|O7sG(QB@hq!QiGB}36MO09cgYWF?m0udeNi$NpciFW+J-QT0rL}NE?;ShvO0!B*7iyphB_|4uIeLcNEicGf#Po(}nVVC2h=Zb26-te~Biq@BW zi5W*Y2Jxna!$4YQB1{bD`@D|Yr*pvxX&Qkw2?BwPvZeF^<(VCVAf;%^F~~p=Qv@`= zFylq!kFoO~ZfVD^yjzm%q0^9DN#|W}+BjCToHy{WhY?%&a2)RnI`5L&;FypDi7I!s zAKV!Ql;?m7=|YU2oB3?KaU>zB;@QK+?|N)jvDH4*VGD{-g&&XNXB#p4-lOH!o(nPCbz*6ba9u*5HF35*K0qK_Ov|L+$QWWVkhQL7qMd(>%~ zxw4Vkcq!WAG@FiYZ5$kJ&WGEmnyBx-9z(n}V#)SaIUbRdr_o&%1cBwYu&{ALEUgM^ zufIMh_NC%00O^sl7YNbPH_a>N3RA_eic>`yK_7Ce3u}y-g8Pu3Xk+=ep66QO!blmQ z8=@5W^+0TEG>`FEUpS>a@7ay%%<3k1E|v~=KH;I$CuhOK6PYaZ+%>!Ibp+UAvD$PH zi1ZdyEWe zN+kCv4rla+Bp|_m+j*8bc(Yoc%&N-^lt~A9bN6u(3(>#6h9H1Fbbsz|8lcwyZ6E6t zrFFbL3%$XGu5@^f!Fnzb*U)Y=pmcmx=J_1>_pb~9E$gT+oE5NQYuWuT1P;#X z=f^){W)0_?4_~x4{hBM)Y}hJa05EkIWLFBeI9;QGF)xXdeNF!@>t~5`S(6133tGe4RED99G}% zGSTdR-n++Uzjz<%<@a+WVpiv}-E*gV8a%73naN*we0Ho0V76Q~*B9Wrv(SE@lJnmT zd~7Rq%Rt>dv!A$tEd#J9u}3yL@9t&{7J_l#P)iCV$u+3J62f>he^VA{JBUT}wlqf; z;yMT(=a}2}6{?xOTMlaUy#x4g{=T;-!@5aNUzfNmjxCJ*0$k~<0Dr81=a2rRmL4x3 zkJ{72YOUOG1~%v2!icO7}#iol}Tw|SleS6v^^ zdh>hU#AFAkjbI^KE{dW>4kI3~7P2VpcxHhulDX+piD6QYGa;fJ9iWW`coM!dSMm7< z-8kQ%iYr_L`0>>|k?t(w%CO-*%a_92-(jMdmoL(`ry-WHU3EM53qKmJ1vjw3cy|22 z6jDRrV^C?a5d9j1a!`=rkbmL5W)T#yvaTpVgWzqi>)WjZR{*pCOUGj~d-g42_8uX` zlMWt>AF{5A`u-4&J^*h2&8a^d(>wtN#6vIw{W4uHGen@Z*01%nHNRYIhm@Df7Y@5n z-xFqNlE-Ua3?1NYrq5!D;UV-{+S&A`Ply#2Sg(@aN1y$-t>QC@2#Vv>cYNZ z3F$`Z0coTgWB_T9?(XgqW{^^(YXkw6?(XhxB&CN2kxpp@--Gva$NT5^=liXPbry>` zoH?^+pMB2W*LD3uhP8EBFM)lmG#v=~BCU647MU%j&CorHf;AU8KlaY_WKXj9Wfvxq z*7XHKDF2;;xX=M-W7dyMwM#S5p~0}&VvhY&PD_#!Yf)=UkNTzr&8Y!x{nW!Ahk~-+ z5}PR9wihK&rYyWBM!iBj_HrG&!UpgaHd&meS&P%ci|Wtt6@JVlxW7XSqhmQ_id>J( zDbgT0pPqtQSa%J}v@I|ZAH-7XgboMU7A@|3Kwk`LH>G!^;U_q%<${v!`lf*jyX z58~slW9uy0in6sQ9dUawdMwtOv@h+=-YFze8P;x{AY~UDv2JbD@gsA`$7fKgcJ~wJzJ0|n%l1r#xf36ps7@3AblXe32V3y z`)NCfG{#-MTe-A~RuuN+T-5}8pxQ**%@ z%8I1MFR?>b`$9jbW$2HHGD{;45t?;)cYi%neQb?Qi#b;L4MsU>3l+f+Ctg^qvA7E0 z0SIbK*^U@&C)Xw29zCT)azB{mVMn{nieXuDRw4{C+}O41D_KS>UKLVaZPz$xb%JYzP33yk&*R9r%v4{7wNO1nGB#YIcQcGEj%c`ea zC|zzI9Y}=tlGAkJ$#jav^T3zop6S~P!-*T(BJDkZ5zQiMP>~KncAaX zo?P0`h?8Fyf1Qz77oc2^=q;vg<v{5mdEHC5vB-RA_ z%~rH~Zhousn|xoa!W#;r7G(cI#B_;?urPxJ&~0~LBTQy1+_gkApv za}3WSJ-c_qJ#Keo-cK27Ii1ZAcNYd6pqejgMW0x6;(>%M|%3td(3}nKj&< zZZ~YddKt5v;0KWnTdx%o%E`}^3P4^8ME&Uy3qLW1VHyA=----mp}h*C-YoBF?XFjE zx+|v6?+F_DWWbQ2*7H|HXX?^9hCtdS`dQgm1uRef>c?;nt%De+p+QGaEz_*akD%(L z_k}g3PU&@4ukHc_rfGJII-LDlu?)5V5B44l0u-9_Min7@E87Jv>BYtfxpN82u(3nI zIZ*o2#0l&B42NBha;hRwucke&tfB%jg>&Pf<>Xp~xU<*W77GAy@F^i$AZko#zvI?M$Z(ATfHcBgbrpa;@p_`%Ew}Lb3 zKD>(8(QC_y_E|yI;ER?4`@Ff2#v^u`Ty-XJF5&f}w5_>vP{O;=k~kmv6`tkpY|y#U zhUM@Vt%JA={$Zar&h*;Xk|;o*ojJt7Pu}5RCI8iD@zK2l4!JZ0j}V9BP|US!XUI!I zfj{X?-SxiGnTJqtraceec=}%CZF*?xbf5Gquim4Cc*;HoygcWpKlAqL1<>$l1s>4kJSnmQy0ww?xT;7Vy9TC+#zcCxWn z06R=*B92~^$)^-j9gr1b-w&Gp0VF}Kldl#}(yJjJJo>^XFp<|wBnM0Ab@74IF34A>O94CT&M|oLErl3a#0yoxy*ULo@8fe?;2!d7f7d&7w#|i$JzI1_#D6U4+l)u#skc%fu|i&f>=5vs9%rURvwB9425Vqg`qp< zNNmLpTWk#l&A{ZhrOdsK@w$EuD}zXJ0ModPel&4X8V(<$(+*FP9D!*Wh5T`({2mMa z+=eeo-+fSfz)hliV)!LpmiY6}A9r|Zs|G}F@DfeoWsz2^m%+S8t* z&8Ki1Ye`}}GZyOP_u28E9Z99dRuGv7v{aTl44&zN;cD>&+>wcJrY(9G2j6y&{mtWI zZ@8L&RQQT9U3S=luCd}-Ml9(SJwQwmT2@=4ZfM6dO~R|c=R7FTI6prka7n9m(8g^- z#r-k+tahW5g19aM7*=NJz`)guPVq1WJP08SHPeNbTI+7SvEnms9M2bayy~sD@W|0` z)!uvcE91(fF(Jcj+bTsJ1p2`=;q^4kQh@hzR4PXD^Tsv(f%Tcj48h6KeQmv*qJ+wt z(S6EHP=aeDj$D6$S3l`Z2|ckNTW*}M#ys!$MgUh(4w{f29GTwPSNr`F)cR&7?fQAx z`L)C-sp$%OCaUtLo@}-nZQ5|ojiLbdwC*<+4W~Wbk)yoIL!oOF6NJ&`GqVveBgi^xfFt+{d;0*N(WxgLW83pgf0RS@xKH^ zo~ZB}#-oj==vC_-ixP+pvr}G##SFC82f*Equr6DX0TnkYME*a4%R{q(yW~GY!+(b_ zx5_|pxf1#g`>*ovUnxZ(`fcEp><#&ku1OmqAVzjcZHn~Qss4P=2Vw>Luxe%IKfw@P z#nFQ*oFCfUqlaShIC2HGtlQEugdqk!Ek5hwamo*~r!?e%uE-DGNJ~B7Z3;@j#}|fv zP6fn@bmINTO#Sm~_oi}ImP~h9|5X7YV5?)vj zIgDCVEn(+o&kgGzf!I{GV8k=r_NZpGT0xXGWG`9+*_Tzp5*CccJ{m@b_3XE*M-2-{ z=8P*}0z^vB4`;#+rzeuD65SLrJTKM~MBZkH*Q79k!Kf4fLqv-Qw9<5080~}OBJw-D zPW}%Zf4%X+2B6_m4(xYR*D|{+e5JWJ_*wr_xc1!pE09VUVQhbUU)k+G+Ffq=Vz1o! zw!8Ep1#&N)(&9Gu`rvHPVbDuK%Z=Up*Qc`eeMVwR4Xs7u#$)R}<+o3rTFhF1-yNXh zuBD9h8#mX9EFoCh0UbpBKwf#*qw&z;&yW$Sfx4dtJ^uStdv+qFNNA&xyp%;Fb4~g@ zYkroD_7jR4TFB18Uq%2?a_YC4tq|Ep_O_XrxMIDnCs>ax5Wxmm4 zGhDKC>!mDLy{Lbw=UD9Ma|5umuv$x--_OxerJvW`?Q>{p^1l=$i2|RfL$i25BOfE|D-H5? zaQ#Akd40L)?P6bCx+`W6o<>|gkg5Q*2$i31)nGOQN-zNFD$mIN?VH(hlXTCgtZ#SF zbw7FD)~PTl&TLAjMQhVKkQ_ormu|cpmuvMMvM%<~q|bH8)+d@-GJr_O!eNMVXZk?| z;LzU4aypf#1u(LQgJp{aq+^SSA-p+7%o_Dl;3gXs*eihE0%+Jv8rW@1YuzQsM)!-> zK%Dh$3_jggHTd6j-Gu_gxZtJC*DfYnM-h-k-;10y6oIzamK)(?Kn_K;=kQ8Ghn$w< zZ5y_omjo$q>NjT{OjNN!A$(9?(|9@AJ0W8WUb> zEk&)8XE{z!;sr+Rxs#Zn04D2sch@_}BG+G_(5Np6j5XSyyKA#aXvy_Xw0&^C@Gwro zYJqr`8W$0JM~}`2LMk^Yh9Ps zMN#VASEmS@ffalmn7YI8=RTM_6!qR-O#bY<;d?jSdyDJT?k}SlPQ4EURy$Tu-&`B0 zueJe~U)~7$1Z}bkRr>p2@Kn^+1GZTl`A5+M{!Et2&jaupjwI%gATb-Ut=|+7(iX2l zio}5Z+#2hCr&c_DEK0=Oa>P%r@D)Hi@qww>tji-io-5a*|*+L$i>_t#rIMINKQ65C@=&R)$Auq%^hNNIWXcqAZw zmD$GC-*vfc^?+XF=jV5w-KZz@S8N*cceBaX1Ionu-^@y!MW}RCCM%D1K{>r0KG>VO z%96NnV8M4^fSlWZs4Kp`o`hlagf=f$S}bSYIi5#`PCKSv)?SuPi7;3n)dpZ?v{>!| z6O*(ASq0)X0W0&w1bW2}Rqkg{d3pS6)Lbw`FT-`tDlf;j5S+ppyvP$05$>I0Ld z-|Yq_fi<&*cIKU(>RDPk1gF^Tby(A*Q&gA000PJub)3(JG96&r?GeMc4!7;irk zK8kv9HE)x?B_}+n|sg zGN-i-vhNHib|sU)2UN;nlp;k7tCadS%?Oq{Wrh4Lp=Q^+CM{EsM?=#vJhAa1udPUqws8aqiD1h; zfE_JECF@uchq6@#cDQANk-GwzJck4iO@=Z@Vp6eJ|O&H^*Uk zFt31SN+==2uk|sRfo?|$iv9^*?^5T;8s*v;_2*M4rzPivkd_k|b^h5}aw20yCL65zT3y zmY3C@Xu_MjGl#7sS#v)BTij@9O2=Us(UR!-+O=$g&W->B5taNusMb*mJ0ki7shEUfXGY~QBl+t9Va_HNV4>Nir^pS-J6Mi<;3S5+Ws{6gh z1;o~a&S{=G@1+s>(iTS()d!s}QX99stm|6WL`^WF%PxjkH)n!Ghsst^_k$6t5QY@7 zDu})3ZDr5~GV7LY9sj~R65f7O;DU?Uh(Wgb8?22y2>oz9QaKhy0?W+uGC!1M1YC8O zWU_#!G$~cVD zt{mxN)#qdytVFfe(#16$T>&q$-=>&C> zpxuYt+#>-LXBGfp{cyDZU(H$sh!ktJ}K5H)ytB{ zRb1JDGF0B>bU3gkdPURYeWex!xh2%Oo8_WngM?6&CT%==k}Cqlt#(fWa{}WhF!}Ez z{p~E6d9jmn6awE*^ikKFb-&BNWFjTbx-&*1*k+UM!|Gd4g)Spta8CTtWnwD;AOxkL zlekCjE45SWma5V#;wZYDFxBV=?GHG3YqgYl;;UX%=}kO%wYEO|_r+|Bhijmuj-3Ft zkTVZiQo`pskRHw1mpTl4vlYc>V{}h2AER?O(SE5Zn_oB}Gi_=Y>gzh1gk9YQl2dt# z+cy*FsVr@NiC%uuG5gR*6%@!n@7#bYc7Ul0{+{!VgV79`F zADS=@!k}(;BN(*N7y7$M4izl+O9H?M!s_fpdWHRFH#$)=`u3wg*Eu?TTH5F4uR}AUuJ3^lZ6sVdf}r{97%$kcvSZ|nO+Qcw|Pk& zrJ$!!!o?21iRDt)W`nVTUYILxvFA_)d$VO*tXbV1+Ux9G`jdMMp@Q#!~zBxZsV!2e#1+UAZ+Nh*SWPl5WFa=gyVOO@)-sMLR_2eQFf6XN|O!VJtaOx zi$3F5yllk{yAbYD4*xBo^G1>xp$9B$oPygM z@`z)W`$++4Eh2aa(PrH@CujR9F{SZUdj*5|*qR1V0Xx7};2!Nk@KHBy$u2`oLQ2T8 zj1MOmpW+0N2U^Al1(GDUX~kHyiF*Sw(W@DEUtMu_gRojUlCaUknx1JN>!X@CWf3vr zB2G9%fyrXVxito?KzLM8ODhz49fG8R_O=O*FU)lVpTIH~D;iLd_SzFoocbjEMZDR zc1sUcw8>??vDSpR=~Q}8TEKl`8usqdpK}ZC{(GS(Bzd+>MF5NB;rb61UyP$Hvigk`_uuW^kCI}xtAMj)IrDbsw{K8C z+3_FF8g+R8fsR5RM*IYJn6zMiMaOD;+1nwd2FiU#ULw|bRUP;w+~&CB+G}O6x34H$ z1ro?m>)Qh^z~2>L0JXZTYou;qZRVJW(PsoZTc{SYBZQ86oTR-|@yv=6r7Ewb=lG>1 zNOgX(skgUiKa02R{s_Ciy{Xgo>GD*0r8C!J4(wmXW<5tI|3IvhRrSMk8YPU3=bk7I7kPcro zz^;BYU#HvNZ3A>!PfG0&`W(OSi}+K5)6?U8vxCqz+0f9_IZT8CsvbE-t4lHf6qqaW zvr)VM>a0y71OquI3sN*^)^5!*vF#lB%e@qW(3p8YS z%sD=!HU8c>5bR|~QU7@fWF{(rEJgRn^*0Ys(7!jp7Ypg17r-B8K>Qj)-~035M9Mqh zT`y4o`??E2xL_$nY&~3_zc*E|JI+5ZfIor&ZE})NPsaW$^9&T9>)J{G$uu;xsPv=0uO?Qq!L60`RA@ z1M3@@Q|Uab4Ivlpr_xYHYrdacvKu{i1#HqQ(0fTc#Uucq33dHdNfJ4B5V2Ex!HiF7NI^U~x#P%xHy8kN704IT1_T-qHAbCMYi0 zXl~x^D7x;WcR$@CZ+x{d;OG7+ZB=w%tCsw9jXbP8#oDh2w|lTD`UXggbD;?*JUrMT zN@8}hv$c#0SQLIs-ZNoBd0Z!K40pRDy8tkmONVUGk%7p~cV_8EUx>k_+gC4;E7L$} zfq%r-kS*6bJ3hR)-n_83iCe%U13Rv6Q+%3TVsZv%O-o#@lqFLqYU;Y9t$ysdM^E-O z2VW`DyjtJ&khBT*-bqvgqpkq7cj!%*Kr+aCv&KYEp0+Rqpc?FLV^N!xL<`9*=zi8| zP1yf16=VJWQQd$ywip5#N7CZMS+baOaBt#kz+v>#@tFP!*we;FF<_FRmF;N$EE<+$ zHPSr4O@`kBq>r-89d}&}S{Whip_AHb8)}P5enBgVcrz?ph|7$J>nG>8yy0@l&)Gh) zY>V@#OZ%i!IcC}{KNcu|>%713lo?UVVHpKm`<#)#ceJ!6E{Zs-B#FOX2Wb}K0@W} zcz5k6y{ohI;OO)8K$D0LABtkpDrwmP%-TVz9w89^yW2G;vuO$QF}YIS;vwUQYYRXo zqBG@Yhd9vWiT$W0_h18l^Z@9)?REzFWGWKErNoNn;7v$7xx`L*k2NNQL9QG-Z^EwPMI7j>(9sH47)CS?sU9cr%mu6~Pi~n+~9M zfmVZ&jD$Vg+_pGOpGd*zR8)iB8m|+r1f) z{J(6wRUu3AK37XTwV+R87f>?$%tg~mC9OZ$C~AnCt4xpi^t+;s4GH2_QA{}9jAhG$ z7L#67{;y|6TjpOLs~6TxIaq!6VQfRW=BP*hk-GvWUxa7)pZgyd7$YkdslnBpvrU^V zREI-6~+y`hB_vl|#`0Yjl-3xFR}l6GcKJ(!F|M{PD^c;{y^84ll*k<{RT| z$l_D>v_H;P&2+1g&>%C3w>x`$;+6_xJ>=ghpF|$WK>TLxNh^qDbJ@?jirD<&tL6|m zXhElVl!#}|xMl+uCacx-BtJJ>V6ZPJURB{*wK-cmZMi$>%uO-g)~K{)nKL2D9Xnm5 zo1REb;A!VU%$SXnCmaTCGN_zX26FYE+Uoc;CB9o)LH70Jxx;9*v`O;|zsb@DuJY-n zj3}EtD}=U$IP`Z~<4(hz%c3lv;y9WeodR7wimfAn>{s-*QZQr=lB)?gv=*h0(#5}g z=HQq8l66_Fm)iUJ;P>M%2e1pQ z+4^KGLcf{jU;JAS5KlTzDnvk0WjM_IK+R+U??Vh*s{PQmLd|qU5Bm1&8d?g6))ewh zDh+;Spv--4#TIC(_QJJmxi*{NL%4dekuo4G42}QpS_(GLREr!31XkOTMNs@j8YkIz zkj++T;0BKYsktR@F3h7;ijHhKAkW=T7m^28cDa2^qY6|8g!4BInQ=DEgcER&^_uND zN+4&9VBAQk$9q*IgKa|TyAjMZa-7Nx6%L`H;kSfmPq3PPT)Cx^1gIO_;kZ5CGDXf@ zu{lt5C@-n3wzIs)9o71ZUZ{#VYDdiw^|r$o=FWE?w!yi+Q!AS8Y3hJT^u zyi1?pn_BaL2ujnkiSdV2j%Bx$r3pvAB-iC@Lz^-^%NB^H^11Nh*tc&J5Lug6%}A%; z*;cJB2y)emQ0AwQb4VP8sxU^$w(%p?Ym5MqM`1q=^J&I(N& zJh-sFS$#NXQQdGTP6ydHFdmTb>YRHmtYwQdXOk3#3g=+3*39c6G99-dtW3fj$*oKr z0&g~FVF7W>ULlvKB}jfV0iFli;9Zl%G!Be(NW30L$Q-o*L+u88d)XVIhZQ&l8lo|P z;MjFQ58~c&M9?ENRYUh`N$<%jle&+RAY-_%dXxB*H76;t@UQprWWoURTx36`h_wrHCF! zxoaWiSyviug8&ghaEL9t2U6hA4QLKJm#tS{!1yHX_{_cD2F0c4g^YhVbpohDa zjDzeBYfgjqVe1ySY4Av#&ExtpKZmRH6y{_^Oj%A*Y1q9|7# zba)SLe%rX_a82h7%K6u=7VI?5i;>OrwKSWvg8t`cFG-jS%BbO$vdc%+l)ti6D{}|{ zaGv$utzs`2XYtEMO|}j5;qJo66IgRsZ`_~k!Dr-OC$Nd2WIfhCoQBM$FWfGT%wzB{ znP^>_XbM9grJ&?7<*}%~ZxAwI);-8e=!;FInB`V`&YRE|*%TWG2z|DWp&Zeipit3_ zNG+cu$GtEw#9%CE#1t2i(o2~AS{)K!u<2Q5C!DI(h|VMqC@f6w621CVkZe=>voMx3 zd2dq>HKKyQ2B!+z>oxcFtJ3CAIZpL(jlNaT7mlMRgA*VEv%#GLP<&x?wc4S$2)7%eKp$6cSrX;Y(4I6jL1CI4A3SYGFs z$rGW;mliLdKhtohi3oHaeqg3Ng`T$pwq1WOKl5b8_7(i{KJGjf6@RtBQa8Jdl;0y!^mh1LG zl)y^rABF)HR$Mur`KV3$arWHrjX9@hv{IN-JlScF+FPF7NO}Aweq6gzge7wO$Dzg_ zaop~7_`5jALfd=p+Y7!Pn6;|}Ex((ILGO@Gs93&r^B_{zW9`uQ`S}PYPo`ijF9_%C z{tb`;IRBX*$NqTb+P@hBPACvZzm^@vco;!Q~G_KMnTOOZqE)Mm$# z?a4w=enedGpCr*bK1r>u=Zw{C>$0&!6$0u-S0`ooS65S){;cZoXcvb>tL>ECZsS^7 z{>tZN3qHSBW0m^i(+ON^S$eCM1N_o)YS-}iwN7=Y^ugX&!@6L7rP@u~)};fVk2AKe zfRZ_AsI@B(vdAKCtn!dM!BjTQ>6EOK@B3&KDeX8OIr6CTWJR6Ao_ssr{}JB-EI<-& z)XzV0A?o9m;d={dM5D}qMItyy2J>YMmZRTh0z-PSmUucD1cBci4>(l z&Tux3?KHrL3D3$OEKvd`kQYUuR<}PwjB}g3P}~(=xDTAWS$xJA+`aQRAG|zVA?2`K zg}J6THa7d{Rc}1`(t9qZidN6-41?*hLlWK_In(G&tK4WCSSw&gJs^F1LX!z9pJ(E0 zk&k{kFTqy=8;qGA;rcAtKrI|9<%|6CauJXET5&RsZ=)>RzD+QOI9F>#w1(Foq88O$ zmUS<8K3}?|++K$DSE z5N>Pp03vOqHgz+TA*a9V6vAy;brLQsxTt-*hR%X>Ul8_K05r&j5e z%PiiI_`GKpY-8}wH5gfE){C`z)xB&7vPeI8=)!`I0iv_iZnx* zbsg#=D#Q~U-=lg33OWAe^$^(+oWYI1igWO^esS-av)d}BqwaroAT|owzWxpkv>~F=Od&E z9OGt-)u_`&8}Wn)+SvhSaKo998lq!$TBMn^{AIFqV#60S%9aaI078eVFhx3XbE5-97N5@^;zsIc1qM}6mx!KCM?3(Pfz}8Hx zyK}zpQuh6=1Qc#}`RV@N=eyIS+8tTIGY&@>fzTn}_{Zk<)X-jZ*=hVm3G|AEZGFNa zzguXvta>P!S>ARZ9Sop!`kXu58$@-}Sw@~qe0DkF6Xchp!epI_Fq1NWu>fPx1{>wvObkYA8jm`xuB$yg^Fs5W3!#g zkyn0@Mv#jw$_B_YA2l!CCBJ#O1p6C|THf`*__S-_S&kZS;rz~r>7Ta!tE8JaH!<>7 z%4XUUs@+i~9eeJ)#^<4n{%y$pB-BWexn6FF=N<6J1Q< z44sH%t7>^lN6p!O0`nV!{$p#vsRGL7p@OHQ{o->(KNc;@tcBE@Qj}VYX-!NwmEO$9 zv5^j+k-v?O;TaxtP6>9($rh@jK>=d|$Q$r_>-g4XFYB2d{UxBYel13CYZAs8SZ;M@ zh_})bF>j>*2O=UaYTV>eIJpBTALVH>CtAOKxBH%hua_{_qF5+2(WV7z|P`$7?XbKdLKoP8nV6Nl=JpanyY84~UcML2;~#mMl$ zmL_(2$&dF#Twc38YDrVBmoJ{Qw`)!Tp*?bXnz7j0EvOAP&p~&D-tBKj3riZ9cfK0bTl#71Qgr7(K$z#_0v`l%|2 zmY^e=4vk`Pm2`R%CV5EaiPY;(FDulAf`dIr)k-b__TF;l&|#>kIflCpW1S89Zt=(C zJRTM*hdjCTLtOT>lWVrz@+lOBw;$eYfQaEX*augUCchAkT3>I$tn?lP(=RUg?}Z`z zbATA#nJFMY^s6fv(<>lOKtSLDYt|( zh#T-H)T2XFkm3&sVo9J!`4$fK_~`GaPusGjMkGWV`Hts-X+EyhcnEkYDWhd`k9sl- zYL(Y%Hyb;kh;rqcfV;?<-taIJiubbjM9G{E zI$^EyV%LqHIYT82RjW)<_PyNCkNl6aqbx7O&Q)mmEcBH&rX<84Q(fScG_RY@=r0Vi zAKUVPGM<~|Z(C*mQRFL7xotwy!ue1mvP?jes9t>iq=wcpLmr%zYTX;%hxrZ0e5%q< zVs3lN1*RyI<#4?z3=<0{KyFJi%qa~}jrA%N>2PkbI7Y^)0yE!&G-1Ql{O`StY^s*X z5*K8YUs5xA$wb@LMWqO%Kaz2itt>{fK);L^0+jF)v#D#cN1Nn}eVMBpKUHji0_A@W zKf_I56M!F+7SM*@I(M(?ZB4)=w0^w(<6QDZFYZ*MM)m_`Fqe8g1*H=pROj^|{MBWe*^{oC=ob)3 zPEI~O)J6pD8seqdz=!zflE|HCdwuEexjYLrvliY%NrCjHV2}jV}7Q?L2$L4g47>L!%ykV5FO-?Kjsk% zudVQzhx3+A6tW-xZane3k0k*}`7Jg!^v!Q(ve0MMnTz>94dqqFztEPEWvO8i!qxir zDfWHVr~7fIL%u?TuX_|rUP8%LKT9TP!TpQLglwe65&q>eDr4}N!z*Ot5itN=UXa>L z5uJMwG_!mz7rJu<+pb;qB0lgq0PcJh8D|=U0SRyz6ZRBLeuUA=G1`lnHUbX2e@#xw zdFR34-bb0DIQYgH=VRHbr`9XMW#PS50>ijs2j~-(=G17a=={{0pOoU&>@T@!^vzI5 znZ|4%FY$2)(uEHbs$_B}pPz}jx1Pp@PQZ3ivE6q&*#NM-AoR@ z3$EkRe+9waz9kO+k#7dHdSyHE%YNvvIq1thsB%?|i7Sgg>Dt~#0pnDIyMzidB@!D& zZSQb2L5@B*e3n$a6o!KpKgDZy1Trsb(gMYTs8!q1@zJ|8{jt-zOQO6Go}$qgtpuS2 zD^HhCDD!^vqw;r3`M{H@dRz9sY+6U$8?m4b8P#uw9i#29%BtW1P_D&2-KMCi8!-R_ zfFRw|XI*Cw6cy8r661T-0HI;cZ>I92py^hiYml$+UTM7tTDH!@fe|>%wFUzbU+qIo z1#Jy&QV>re2p7(3^B|D~Y%&1A{yP@Bis{=LS~zOr6 z6)_hrk^v`7j!p5h3^bvc>KeG40m?+n)I+MJJ!3Jaaw8*XonU5C?gO*;ICaSU!xadk zdoae>Dkpk#^Nt>u0FUF@=qgUgX*ar8CZXyv<1%z?%EcSkF!HtQwuGx#6LpAs`;K-H zbdm753%hD6&DrpA_zr_xHqoV>c)^mN%!hjR0r6A}o_*$ej%D^B3f#2+kgrh_I}8Ed zE=p(Je=Frt53b!r_$vRcgjWC&v^(?v^uV8f_;39@;1${;Z@OsB^)I`5)PO(oGk|K3 zeLp>k6eIvyZuJpxafCYpyYqmLQG77Q{5R}Cn}I<`{oPM00o1#2?a8V_>X4Qee1yEb zy~=Y%!|V0WN}cke5nSi?-{1QDIGkU4+itY!4DXc!!Uv<$CJSvQqjn-jEoLt*B5(dhWA;ymxox^0GkLOK?IYb9wRJbuX=ZL4|U@ zVM|U{x!FE%!{9P2qT68Xt+@$NQ@wEPpK2kHHTX?Z9SX$Lx38YbfsJ1xJ#9Eojhi_y z7BGHmx8c9tl?rAwc38Sy&n473GITL|vp(K8*?$tlJpsFbReP&ETnDePP9Nh{M9Uk;sl;1kf0U3)JqZ+pHZt#Sq;@jxk4WjKDa?9YeG z77+N#v~Fb(0e%A6oF2I)7>{%}eCc<8zs0WUd}!#hBIu)q$+tH$(3dSOJf&T;5x4@~ zj~qI@s+)=YTqX=>+&6^^w*a~ePj%n??1RD_g)hSWrOM(jIOR25L{GwhmKaWyFEC-l zPnLl$={ENe#qBY^qvT76OR2M@r(|OGk&8uCcQ0L$OP7f=xc(=4B$KFEGt-RFWoe%=U&?q8QcOJ-@A zZPhzZJ}_$^72NDCEzK z>2c7*wBDeNi_^0|MbR{r;}|Lu_fw@1jy@0I1ot2(z_=`ZG<>xX4}1t&kbU+c?VsdsTb9OHPMlI3uspGLdkfU@b~Or2)JS^ ztP$&|5qlSxO1ev0V$H3TLX~YfEcCKi;aN%xw2^h^cgwG>LwLTcX3<4_S8eY!m&CRi z-FzWA)_vI$$}^Pxwe8>eQp^Q?9OUygF8AME9ZZB7VA$z)mCHZ(9NoA9qDTMo=^s+s z|Lo-)Lf(dD3peeP|Gl($gnstv(Nbl-upZgJyP~)enl_j|u;2dY7H*{%gYL8aUw3u3 z;=99!;OyH8S*oR0A1Ee^>BN6N-2~Ckv){Z@Fq2lV8t+F`KRAzDIu zax(c^rRnVd8CH_Vvs`~I0#qoMN-oBY*!BnJZ{qZ>P~X8Y}GXP|mv&;}sh4E&Ts&Br#se#Q(& zyy!N>Hv;G+cYqW$I|s+cJHSZYq{Zt5h?x78HA6pbOB^>Dm+whb5JG}e1uvIdeQsFe zA%O3E4T7wYI2aj-gCSWX2Zv$S7VXX#!^2UNT?(JsGe??|ZC~8LUX~j*hmcjGhD5u$ z--LCrZL_hltryIwK{q2WL>uEqOqwm*t676EU(V1Q` z{E1ZS;8lg9qgSR%K){YhMh@U+Hi?6Q@U6Vk?`QH#&*r1+0cq-NV1RKGKgQa3#@?Ol49{f4uU3Xb^4W8X%iurc~6zmD+F zh+7B{YX>x-3p6NeqwRWkzQfQyP^*SotiD!dx%BGzwQA-?OkRUq6_!D3iIbM25`a!U zQ!s+194<2VL1bL~jto_hD6DVQV?U@SGuV|i1+BT)!70d9!-i2xyE(g#t0g2c6S?JG z+WOM%R`FuEksBJ&;hzKA>!rQ{G6BWFbXouFLpGtTxps~}Ah z_A{ZyFvJN7&K`e?@NHU1*ORr_Cd@-YCKxhB%KI)^QASU&@4&q5|5TZ_sRR6*Y1dz? zQpi!ihbdQCFhZsFm7q`gH-^RcL@I=wr1hgbq_8_*48$#?FHDM}ic39FItr4a`I2u! zYmTbWjj8)gQS&G$uuJ3Ip@uNpXq{D&_Q=M`JY4?OSYlfxac!s>_HOun7Z2xHF6_vGuT4ThznA_G~j&O`Q?Rh5=t?0fCOb^+61@XU^L92KqU%*{zHQe?edK|w0Hv9O5MY_pN zc8(g+OO>nDTVG{XqR})0kHHa-gDo!^D$A?I(ID=r%C-GOb9ffCW<226W2VYggV8tt z@hB`Ut8<&xJAycjyRK$jj~OPBQp+ufVjhaJ?eDn*6U3|qi>VT`R%k5ZkD$C^g^3fN z!JT+sAn-^>5G-2Y}RW3HdhQWkOnyIlQ%PeJhuM&M<=Zq6l?iQW$(YoaWVljxOl8h z>%Y73gn?bF(WC}{U|fIg|KF4SZ=V5cr~e%q|1aN8cdq>p9_N6k?(g!wgT}42?&pCt zU4M8^e?dy#Aq(Mehl0Bv&w?M33tF{m9ee%#lzK?NoD%^5q*-lPwsu;LKiBTu`0Ifc z5-pY;j}}~=Aj7dRU0~UrX<3+Wby4NFkWo@fGUN2l$*WloN-;Or0xZS=FWL5~g9MzB#7B#?D&uToU-BTL4&=}<@8f`O3# zBmHYDD;7Enmc>frv;id@@Sbh8tcaO_WpD|1{5{*!Km~u%WMjDd!g&?ZHMJyP)(CnI zI8D%P{dduUPurG{|KXPZC}$K-f;%O+L;0qA zzx&-rAY{D6?mOlIzRp8H;FJ?nmgWkY-y3ZXa9Oc>!T=&*tH|NdBz|I4^` zUH(J%d}5$MCZaqopz%cRS@va+rk8JY3y}mdSJ5dumx!fti#d5PPOZ7N> zTcmkpI|_8C$ZCO63GmWW8WW+3;yNm?4jLWSAl~R%;EQ>q`XB_q4f~Inbxw)LIm=heu zlTpb#uk)0;#7D4rOSmGX%{UV^8T~W1KA%wws)*7@ghE1xwJ{ZdgifimGgJ}-Meyi# zfW*P)yx{(wF%8%Y4ek@~juI4l6DCUZRzVxbIOZc3jl=@s*7tVSs>r#x>u~uFiQSKr zI9Sh8q|NEfR|ZKmC_II5@>KN8q{_KV2UNIC$P}_?=wD+>m9Mxtgt-ikZmjS75xV^( zZqueCA6?*3e*d^+je(`IW5M$wQf|&kimdDywsSGCGjL+p5I3i zo!bd!FsGixhZ`-U7z>s>r4J;#b)*L9X;JOnz;}Uld+$TpG*QxCS%#IO*rZZdgRb#6R=&? zH(j9?Zcx;h>C~U0^NpVMUr<1QlXt1%BB#@-*FCA5@Fnk;P1lFvH`C#}$XY^12~||i z;`ze$z=GsGuge_oO98JH0nw{pJU`vndZYJl=J!M#SG&)=Z_gHLpes|}H&Y8PW0X{?OhMJQB&dHMCs}D zHs7c|X=k@=O?RRC;iIP{L@C=HYVSlx@;F5Pozdf~$4>7%T?C0D^`5UjdXV)_{X|@t zNHIirQf{U@F2x0Xh_vWJq2Dj^r@Wk^uwbO7sPJqRBAVynC+TC0$X9H4!i@$aIvfQd zzwxWS+&fyFg=C@doxZ1n6Bv)Rg%&t1o8_g_+RZvMRz?F^KZSmvKL!(Ag?`PS^)#z6 z<&!GI%u>x_+u{llH>{h^CFU8%F=ominTNuU3%Gn?z1oIhC^K<(jnMWo;|H|gSkx7w zg3C>~1Glf4*{OM-6Z*$U8I!(Aluy@449plRB_Xq@r9nnkTdp2ww`TD(QU7DRXXqN$0SNQ~>k&szrO@S%qKoKe`%({lW5LN# zu@HTo5Oyw?oO&=qDvv3PK#X1>B2i}zS1OK^g!B#*16VWasXoQ@coY+-` zScpGu=;Jd~wLLYCgbe}7dCMc}w5Px@J0ZR4z6zL}_`#1Zgb{u}M?qHywZ?|joH^VUp(qahWW-8PC zrINCe+rPCsIs+0(;iPd9!Cq$JO9Mf4<5*?otRRbd(igPEvIWhx5{D{ zT1<~?dlEp5A`ZZ1F|QJ6f-78<8^fue*$O}*`UMxc5ueJ^36}yz=Spl5;LE2mR;ixJ z(}9I=jBE#WaX+g{@b#X{C5$4G)Ld&Y?$wmL8d_1F$+;y=hdzWv+A8>02y&;C@~4{On$H3_SG&ItrFH14cVP`bFu{=i_QhTWV-j{n?*!YO2^^ z482LTQ$ruRKb$Tx9WKb+8t~_yTMj18G`Lrb!d(Fx0RcKXI>Pk1T{HTfHNRE(=|JR= z`u7rblKGZC+X|I_Ba=@$&9IQAtw67nshVS?0|~~QfZZO4eCnf@A4`6=G!Y3QG&n+^ zo5pqXXY?ecR7j6I(uKV>>(B1su^My(ZOn%0cYri1d**+lnT?gV6d%E(;&Qv`Go+1+ zGDV?~kq<-uGL~a(tCVw?H;BS#YiN7dclF>)i`T5R4o~DTRF^|{(afMT3)F;MtvdT> zaF|bz$_-;P*wQv6qDa|-3%#3k`Ec>n9;O`4Dnb4>w0cm7p7hm=k7ecQ415>c*&Gq? z?VsXL1wUVauTjXU<cy3EQjOT(q)zGv7KPP{(m*o@>j<`UT0tu<{5=#BDzm-lto?Xl9W zdeawe`ekpii%2mnlpjr<611NF)U=%9Rctb{Of~OaKKZW4a?o>5ROreaiYr3q^=q#B zB7uHS`nXFi?-Q+AzOq-qL(0Ik7EUcT?`ZZ+Vn_lgJ1@o4u;Jqo()LJvZ`ZxBux)v= zqZICe0t+bvy-dTr&F)yfl`!p4oub@0;?J7q)k6t1_b01wCf{f5MQGlCd-7ckIdahMmK8dFH#|OHIfsz$_`bd#N@d-x zjZ9uFGoNo6@k6{yJA8fD2@;yh_dQMww?1RkF}Ht0nv1r?NbhYR)vVX$7^SB7E5`r? zkbe{&TlK-C2)M4O6<2xi^+29;z(9NXq+rTzY6-zQ`rbRBEaNL^1+mDxP7SYoA%5YQ zW!i6;$qRLQCp*WMKlycnl{opW)j<48#u-^n@z8Fcn2bms!!#Q6{_O$vEzmHMO@05E zJTPs2c@PeSLugI`ha{MK7bDV{J^=oC}p`^6ta9{(CuiWmppD!qLJh0r- zN$$}Q%PZVEyA~Im?KnX0Bga`mZz&{V;-mLA1uQv4+Cn)=o=?MVn>--VAN@U)CwyC7 zq#2jDUbV)scPc{2vA(n?z46tZ>Xm7vcf_XkU{aOj62lf|T~J}X&ZAUWjPz^$@yeQ! z&4jMhlZ^+z)+Z}LrzFPl08dqJDioa0Vu-?j;=+;|;d9t&7coK_tV9!}nltL^#r*am zoIN2iFY!e7?#^kK$tY7vUV-0Wlz*ILUSR6>U3w^AQ=;$tXfc)#fmLt9`c)fZr`U$HAN@dkdElf)AWA;n&AwN@cohf9{wH z4x&p@UL!I4BEb7RU@)qh3cH5hB&?fTlJ{V`%rq{gb5Icq`T0OZ3TL^Xq%mK~9uE1g zOGi)0B~_U7#&G>lTDK`Y_-C*bg`+9UNx2w{Do8!UCC3Tz#{eZ9hGmOj{9a85H5rK# z#ty@9G&rA3R_ANsl#X7-AOUw@w ze{ov+&(3*sb@0W~Tx~VpjKP<7bZ%xVodnMA$xd>OnK&?wC06ZS4z=uJesgYZonn&4 znyFqvJpF=h^Qy9jC!#EU>AK5^;)z~SxxhrWjIvYE&WRNACptIg&JLUG8EU*}B7T7ihn3n8?u)byh^cBmRRrU~48Nz%O?_nT@0slTdH z?I}W`ZHc-6Ncj zAh-!~iw5_m)@paItIZt(9?gS%&6H~NAm}vp%?Nc14Ohv{#YXB%5s+<}1b=wn z@Z(|Pp3n~9%p#W6L7>H1KW@{0U6Szxcoh*43hF(*sA=bZW$S;3dTS7*h7F?&kTx6v{5;9|j&k3KN-zap4~5+KtJ% z!KX>(^QMO*OWA;bQ>oSiBWkE}!tfn7wEiD(Hm z_e#A;-6{IIxD1Sqs{GRRwOIxaU_RS9%L#$6IOqF1Q}cz!3E`-}M}g8ZZ9j)nS{mc23%<8N+A!|37+M~LF3`({LdE;!&j?P<86JN1Pl|_gazhqLd}Kor&%Wg%^5^7 z=TxI*##A{J_F3LgS^gS(hI%M`t}CKIrpEvt!(*4`Ug@BF+SZmh(N=lzYds^)$_}K| zG89^%h71RhyuPGebotFiMG$7m$HqI2->2nSQjLRlnUa@Whl7$(u4cbG>^TZ}s5$g= zMUGm34fbMT01&-l*wll2 zLe484S`Pca2A|~z$(@-*1m`ciZGJwXEYB)9*)U2_d&KTHSg>ToZzRlpHGZ(;jx#9w zkt$lW?zE^QCtQYDiK;|l;c06?pe-dg09(^i;PL-y7}Sm#==Ftwkmtnz8>pY4?n7sZJqKQ~EFi@q=#GrpH7-y%~}p2D>317;+V7zrr$bN+)( zz1$8^!Nq%KZbRuiC5AFH8|s~H>C^XlvBEEWV_gk~Q0LLauf&pGG~*1A2@?#c^WRD!Fz7NzAM##gBFz^pkojk>M@g@1Y3c19pOUfCGKOjcm zetYIUTi#5ov23`%u3{cS5e()p@3MqL_Vf+CYG~plYVX{6vGZ2)jk<&JK~LL$D}=2Y zE&Q?OFykEQ`!RHKmqp-Ojz0$(wdeDCeJHJ;0ZzMD~_WYy7A|0 zOkv{BmuPvPJr36*-{#bNMnTMM5CZ1ko?uORP^3#|tdMliwB65vK~J4$Da>uaZGoIi zX(hUKMmKv;nYtr^O+h%_^*xw!lKtYt7IiYW=EqF-4RH^M?8|q zgZ&d8&3&Z~(8A9WgsY&ZqaA#l-kAXIk>{ll?QnJ7TKtSeOv7#~YJDC{LL^gG4J~+f z%}Bg1SysXqVo>L3I)p8dBl3BhX=sXJQRx%9o6Yynp!DNg^Fw4A&62PlT2j<3NZt~z z9$>MAh>_66nLs1rQc`5P=#ASjDdK`O(9Dq=Uk=QkR8&jz#;d8V3=GVjM-lRN;9I;Q zOAS&^4i4HbN_Ym2>dsa|mZ_YwwX7hVjgVoRn+OsTBI^M*y~sJC$JX{<4B$#hru{E? z%gqrk((8p^h8`Y*@aNhJ%XYk3KcDZ9;o6HHshST()QZG$i~fuO*D)xab*vzs&7uxQ zs*>C%1~ZkEG}=W)(C3T&Y$^V`!uld|aAdFd9qDpl=8(Z*0|TD=oV@pSQ6q-VOM~!n z6{$5t0`a+A@L-MjC!qGKh!U4_YjMnss$52#$V!zozL_=YeeeGuabToC9VYpb*7oCX zak&PIH9w3S)JV18Ks|l1f%!>o3HY+Qy-<^tbw7CPEJ?m8M8vKgRB)1mW8^)TSX2XZ zf(|Wx5h#{5G0}o{In`%AmZmD$diR|Weq6WG_VM|~>CYeSuG{GskLbQWO$R@8oSuFU z46n_24)rUtwsb-yopsL`7)W>`%D8kyLaVqNpb+y{>Dt9=jhTL!VMMA716HndAJS;M zRUmP-Kov;cJgiT%m9B1@0qvp2EeER6(B`>!?*GV~z z<3GmO94p?zMn+Ng5DhJI4F=fu zxWdGn@5=ebanUAa%Lbe1dd=*EPEu8@h`pL}$FtvaIV20kqPlf3(`v`AxrwZ5CCwi+ z=)gL(2;A(n?^VB>sR3eDXZ8Xym~_%Rnw98|V+0xfZdiY`thc_~>=83PY)lJqeK8U5 zz(62MM}C4drD?4|s}KT|N{o1GJ9%JMcez}~!6g@N5}-yZl-I_tFx|4DGkbua96bbd z#Id~z%@84)?<^@Cf2^a!g;--98TY%{3B%OhmRe51D5ZP^$-pukJ7h6dfVbez6E z1bRr}d2J9;JbtcA5&{?OsD62_(W|arQh{KIk_^t&Ol#L>N{x(C^|RBfV?b}Fq|Cc2 z4`2ZH+1t=6%*%IcTWatU;~__ScSo;kUDq3A9)-J^@k#y#VkaD&QC3}obW`LQn_tth zM8P2P$wpqcooZIeC+Ju^hGK2(O32Yzm!a)bKteR>V4nAfqU2P@=ye>Ols|v1*V+qY z4$`4OcF_JX3I(_XOn2VTzo@yE%uG;2(EoaG&T3~$_X|*SzTW#IavF+a{LAt06Xy87 z`_wkP(Lk;Ep4H<5fV3sm0Hzcbkj`nyziEdX@W@ZQXR`xZ+J8tB>1g0s>X4Y5{~>kf z0kPC+3X2@jp6ef;R|SwaQBFjdH~sg6GHGW3&)Z5^G+Q&DTtHV%*%g1c$Z*P zPVWkyxC6@6i$>Y#U~mp}*&m2VR+|Q1aE0_ZLqIPuG$CA|50j zpg9ccx*ck5ZN0v}Mk0}b_Dn8o6wr?iCWEIc@QvU670k2u+d*M4Yr*_EQ30zpRVVH7 zG8Jhv+ctv(nCpqx(^znE98VS*ejSHXm$bPsfh_E{Dr1fl+p}ynq8T2nwk<;&NC^0C5UG`w7~IT7_^13HdGV-D~2^`(~xC+DZUp9?SvIu zaXx`&%n2`O;&S>KN;2KtvMbE^Errw7Or`1YvBL7MEHb$xf_r;Rms~UhFm8G)=_Gn8 zlLYxK+Pgx`lbB12bl3XbJ_zH-c&f%c&SHKp6Rrp8EYG@>eR|X_HHiGep z0)p0_>(Fw`*g8Je$PW`Y$_#wzY+4$ubgJfayjFS~Bqkhk1V?J=9ZVt0--`_4_k>`a zF^a5p((Lr6R^i`&Zx0X7WKZXs(QDf1-qVN9hB&YG=HGHNOSAhNqA*I5E6h$Ln66Hu zs&O@+WjeI7j+4potyPXM&)keKcpXi6&pe`p|Xv#^ydv+&g`dC$(iTf!xA)ArFZ+1L?tealhP z5W|YD%V-ZY--*9Yk3^K@i|FQDcCmxIPWFd&E86SN5nFDwICI4Kd3|KM_mc-oVmjp~ z2PF%i5$MjDV=6!PXl1u-)2s%W+titM;rJ8rRB*?)-`TR8yFhp)87Q>J-D@PC%~uGC zOwT3mAMVvw%7nTx^K>L^A#g71Gg aQyAbnAslGszapHH%1#Jt?$pQAkxHM92Z0e z#q{WULhF_hUQ<2!#PMsPR3cpRPt3$Qv4$P$KJ_&x#ec1JznlKTS0-DT4;sI+tuhs{ zSwINa_xyP+hEHmv4?P$6#g!z9VNVr=LOeL}4A4X_(a?5FiLH#v=1F)Hn=uFcxR&6$ zv(E`SK5nUQ5bM|xvI76TScyOP**ksd$QRNg|DJL1iZ0(%?(5YOXsY!Hx+5 zUn`YU^;unJg@+{CK(Wxi!|yy$^dfdEMPs~$kWJFOOM2XULSbeU<5k)Td&6v-_n@ww ziR=PiJnu5=gF=-O&7wk)90sK$eKwhXfR9k<6?g+6GjXNjMFEq#WGDcyQ;NUeuZbFp z?DmaxPN=HuDsiYk%X`q$=%Agw$QWnw7EEq8FjrTJGRz!?R1WV)ju;zONNED$|1cg#+L z$;~}<90;hl0V=1va8w2CJ{;cw*D8K!FrZ$KQt3=Ta=GxLQ%~1)G5W>JAib^cntZ!7 zYFF|(=)C#WRO&)>sXzGA04slJ{@^gzl$R*65~K2*KO*&EV3nw#f{EUK2^R{AomMXM zXIk8^$y`Ws`lw*X1Ag4X69H-GZgp!ZevaO($!tG^J(0#rB^I-8QtJ&HT)=?p@`cCA zwX_%gG-nBq`t_hzg5BdiwR&cFIBz2;YNTNgB$` zFE3>k^Ea#Pt{Z5__v#rX-l+x zkAXzFpnMSIC!^PDIdZh|YY#zJHOO1}BwZ0(I~=@R$Xe6+HTa!mSoB_8pK zpCNfHcWX?qW=|35_eRePw7xe!)*h>IKI?dHz+_Ew{MqfyeAp)C`2uv};P6T^5W|g2 z5-$VY3Odmjec^zmT7Ssl`Ql^(Rn`_g%x^egEn$HB<3!-rSZBp|jN!+FTklTVyFSSehB z&8t90_LV6=UK$?TLLq!cHExgeRuHUDjjM={&;X#EGH6dU;|;I|D&*G1dNVPS#T0)E zin4O12J91(H`X_Sr-LA^lI|>)?|K(2_MpJrUXdNun zbh%*|&2nViO;fRPe7%AYTEKS=As~uODtfPY_yE=PaxY{tbm-<%LRpB}X4xYq*+xFTy!A}FPflupG$i|1GD`w zA>nLakzFXjv$d14CXPQ}P_d-=;oxjAG4Ij!7DmgsBM2j~(!WT%4QG+fe*V4yISlO~ zj5aNzzE<%K=GBUhrU|B2WvO7UiW+j!>VxefGS0a(i<`)Jy%)+{i;4~L%QcEZw74DU z{jGF~8>w5qu_vcuaoIl=6G+VQu6fO&9)UN{y0Uf~q+IK(dp*z#<7dnJZV0zgm+K&F zN?mxP>EF(7ASSZy&d7@zY207^_If`5_q!{DKeJW@L z?f&r3>ZaEiyG?tf$%R)76Y4Z=(E$q|26b@~%VOo~r*y9ReM`0wx!*NQNhbVTu^L8a z7O)(qBr-(N$+eT_{j8CrSd=|C3%ZmPa+YPY;`3X)mIf~^POOeFCzDNeLW4PbG3H>| zwS&4Z)$9gSdkNGU7D?pHPww+syex{|NSB$TGbvoQEs=t(mCA~*#aZD5;Gln5{C3Mw^!wp~)r)c&cF>lv#rh;_eqVnj-ST~ldiUoLEYcYKcJvsKXU)0EV z`$1)$eZPMy7BWm0H7~cAF`P;L0;{gc{LU@N(me0ooaS<#z4Uw6xk;{J3&H(_%rjk@ zAg?BQa|_NWxYKX2vX)Qztn{}asmjjhz2la$ff#gm;{SA}{@SdJ_pWh8`N=(s|Ehyu zWBAYj0?>BOvj5SwauNY0hy>RYoBRJFwf8^YL2===wg0<*^Uv%K^*=0txkK0~PZMrQ##eY|I0Lwuxmk??JRC7x+RAT%SB4Z&h_fP=w zr?}a;8Aka*vB7J(eJh0GRn>0NM7=cp7nT``_@4tvDBKadL$0WeO9tnwR4KT;5k66d zcQip=Rx3{Wx9;$#D4a453&4?87Z@bVMMNKGcV9$?STJn+~V>Uq>;G*|n`^JCXN*O%*TorGX#NrMj1hSb82f?R;-|q^J#g^Jd73IebsdO*efNxOtqb z_xIoH3TI_%=T(XA+B54SAaol{$MiXRzy-@$UxHFeSDs)^Dx&Vex+W-oFoDRu9LT8Ez;E0TgF z*L>bI%ps>%Eo2S&Q^CS#%h3UeET(|bDg=J)<)ezmOm`QFb-%$g6g}Wx_lb7-z?C3$ zS#w%J0!JRSS9I>=7{fn4PuC0wanAQ7^W(gZIqmx(dk$RIO@>n{-*l15s6o(`k!Z5} z#%e|H?^B>rPhZ}YyOLqVEFvAteIyR9*gbe}kNxtQxnX=O(Ivw3dG z{WIG&F61Rt-lr#ddB{E&tt!ba8jt<8$eg3x$t`k8G7&)sI|~A+dv_%ot5}la|#g^YY`KN+QL`R zojQC@`EVc;>l5s5jvSt%8@;(gTbEx|%!Ypl+XWgfAXPdRjd$R%7Rt3kV-?gOx(s? zhx9!JSoe)?vMWmsOo>6Zexcjoeq2W_8&w->_(5wVFv;`3eN7cdsn}cbKAOBcGH#** zw#L#XbI&^J0Mn#_>@BgczF zH{q<5B#L13UFAq&O~L%|(C~utS&HFNk&UuCT47Dnhzx0SeX;Oo4?_H8;2C5aDr1C` z@&#&Y%o&Pmxb##!1c%6Xdf+T3kw}}W?@_$4Cw)G2yW?SJNru*lKN}2wiEo84zeE!>`+McFQKY*2g}98v^BET zNz)!4z?y6O2{^xi$_nv3r{LD2t4K*fSitm$VPx^pla$}dT0n~&h1R;0LNU&{mvdr$ zAS85aI~ibd=3E_|r^+bo^(Rz%9*ym3Ho;s-z-N>t$@Pr&B%(2{or(nZwH?mt&E13{7F9Gu>YnUXO=nJ)O%%|D>mU41{R8d5T zefVW4vQ?a&e!^QbtB($3${XMVM%3exdrG-^7Bfh6F0QlWC=@cmGL&6OZtYs=`l}1D zrW;L(pl*)tUn9u!XWM#@2z8$s{G24fCWVM8@0;hS_=wLA#MER)B=oB7dym}37I3D{Ycz1JDgQ}Ck%&5)B^Xym9xPrL z#FESPg-5Lph@ND-+FmTJojr2~6{j@p6ssVi8)Y3869ID7XwS7TGsyu3_WGWsFs)dN z7e&H#*8s)*%+5@u1C`Z&FoPO#uU7fQdK!-$#mLwGU}k=s+}DTu>;=21D=B|oR>cAk zwj*K-Dj{ie^e2^khs3*ntvKjhPlu`+(=&ALM>gEG7lvg5o4c1t=vr@2gG`p>i|itV ztR-%4XdB7FM=$5Dtptoa^)W+ojdL}bn0tgTHR%L8Bb4t`+P3BA0EjKJ7bEie+ z_~P@D6$6XAtYaw^Gr7@B+yu!_X)~BOts+Oh^0^juY4_{CeI`FnTpLxBMvU(Vma^d( zV*zeAUhdwElSE%tgE5R?(LX$HCrm4bJo_weLF`U(}d1wgmlXgWAM_iZniUGEUoLY%WJ7R;brgqooKrdb4PYK$?iGa!#FnLfRYWt7F!j4mj>v<5`j?Km5bX4uS7>UvOjK+k z0@)S5uXMNav_6z!3Ey7_&!Wxifylwf9w(Eq`-HEst{W2xi>o+&)v*9V5ze^PQ6!dt zg#^5aIn_!TNjKzm`huSNg>zkJyJf!e3@yR5TBl09EFm_4b%1s-S&F|wtZV75luzSW z`+28&X@rZ$hzduGlr!SUnA#QeelaWt?1^g=7xSu<&jF?DnRU6;@^+TK>{D?DE`)YH z(H&R6l*xDflWR+jU&p@`k5THEIVAA{s1P6Q0KqZX5dwKX1l%~UsIyR(Q|luH3NypmCAYGxgw(g!M`MFx(>3~YqgXzn5#j*gdH zSz2}xc(O`u7EQj?CWMmzv~A#|w>#zy9yCX3Ir3QeVd;v~Y8tD^U&J&`KX zC%qpUR_9m>eZj@->xf=6=#`Qpt1E0ApSBr}$Y&e{DgoHIF8f#ysEpyt+`8qW0%E=t z9ywEUwwZpT+HHD;sM9pklY5sCQb9duIiw`;EER|hLyrbV0Z9^BpfMluT{yCA09zf) z4lXJD1g^NJrDm@`umi8dYD91HwMaW<>reb%YJ!QQsu@RfEuQQ)Egw$lqI21+_`Xk$ zfEc|<02QZVOp0GXMjSFmmQg4R)+dI}&93IcDrCQxFp5T4SzDya zFgUuf9TJGfeA=z<=sNfUzAs(BPb%EmZ zRw4P#_==e0mTbEp`o#)xBI7S2odhwB7+%6j?CQItY!`9AFPQs4jM>Rp^d^BFyaWu- z+O6v{hv?WUtZ>DPfWA;d(s}NQa<3NHCt;9e##bsTTxM{j%8Lz$aZY4NkDge?Z)n$p zel??0%1fCids(xU3{3e`4R8oCT==#*e^LOgT_omoemTcF;aI8%NS{hY%y$12*qkyL zDel?wU^)?b*?L>)>sh&cao@qVYaLGM%O6u!0_adzm$SI)%chp;ZQ>{+DQG zoj`5=^lz8jGKFf@t_u$ZUCcgkWIIO4Y7}jb+A+q# z&(QIi8IT4!&FJ;7x6a+Iq?+dA18!MgI}%WF=T?v4epz(lntXqe=I@p;ZTfoQe!}PL z;7eMKomjPUO)R-c2Db5?F#?&V*bggoqe3OibQaWrWz-@3qY%#{&=>v~!nMa)j!hmtwgE4QMaYyUrBk|vs>BvK#wZ;oXHQMW<8yWjiE-u${;5Zmg785yN9^SN1hWpya&$=uQe5gm91tggv45_B5}f8X~v={d#Lv z0eBPvG%>qRj7tdng~m>yIaw8-!p21ymHeV^pOeIrpDqk}Nx@s5ty5BgBXyFm>J5*& zdhYd6zo_9yFvjE5hc$O!RRN+#+nnAlKG1g89Z!@>$(tWq{n^j;#>RCa%@xh03al0y z-la19iJi+v)_Gx7!qT(93ZvhICz)^S6dvMr+)e!(r$Qmrw<|R(3LfmC7uI4r6V?uD zszgPiKP~14S*EH@IzXm-VY;}ld4p2#*y9ey9DdOG9A4d(QzXSn@Y4CELhNbaM(yNJ znT&UR3AV{Sx&Tg#g=gXv#51-_g;@fYb*A~HV&ZKzP@|Pylr)kJ-j3#*9C}bylPQox zmrqwfKw)FFhtfB)FWT1HRf{JcSX=##4z!mZsXkhqaa=>$GbJm7-+kTUUq`&Y~14?v$#W`Ccs|EMd-AEqT&Y)zRUUkl0ehNfb~@ItFV;h>pPfYPa+Cgi4d@PkTu<$NeE>)=a&mIYZUNWXW*EbK#}1oT zv0$z-v6`BiAb~9Y>)$qqv-dfXq$V4JKN~$R%6}hSp6rmi=@X=Sp3e9_!Y%EBc3oV! z>Z@Fqc64;udtc36@?peadaXnO8gQYhrP;8>f5Gx#z&WN&os|~!wAyxH0SZA5a%Fm6 zo$kiRlXYhySUp!L7Mi`7R{%xGOob(|Jo*lnwzhWj-D*2j>)ri(GI#Wk)&~;fy4oWs zCr9Y-%w5e+HCiMv)%o9c?bO5fmidbBt_r_9=0ZnCMxG}uIYabihGiA*O+EP>>*BNX z7$PnL&GG~qfg_X=RJ^Bw{|VY3fxuZEfEO-s?5@9xNrocPnD>Sqhu`yZmK0UYEFuJd zjbuVFS(b?Ws1C@hz0WgJ+Wjb_`pkc|KRoe!YJ<;20)CA_s^3mtMZSDX>pkr|6-d5Z z-g5FujO+N^sw^A4T;KJG+9R#vIaG+3=weHnr6^&;kzW)L(7B9F;pU1JM| zcDXg+Tu8_e%*FFq!U!0U_yX<&MEFdSWGB}B`fc(>Y8&T}*8p$Y2oAJ>U=0XttUDC(D&-ve>0gv<(I79RP zvnB7ZQeB{>`ztS$phbPvgy}K@~s%^Os5&=CzuzRD+k@F(nE2#Must zfrxR!gETl&ZQcg& zNi0%+&rcL>(2mCk)1{GI-;3&%9~yEO`^B(WfWYmF`V{B0X-AX}5rR#m!O1n8$i3w| z`u1NH8&a{&ABq;*{5KqcNB$lp;OUU)KZASPA5${Bw+Dj%?uZ6}$4ma%>!1Dk*!|_EqJ}{JAoj6)*<6K~(;S|jse19K z@vlx344f_)o{n8D@FseE3iY bxAzuvOMhAg?R4G&e%?wcN*0U1`}}_Zb!a!~ diff --git a/images/docs/ui-dashboard-deploy-simple.png b/images/docs/ui-dashboard-deploy-simple.png old mode 100755 new mode 100644 index f0df6f93af3ff431d9d0b47cbe23ab2f4399facc..c85356765b863c2cd525e535268a13afe34a976b GIT binary patch literal 51962 zcmdSAWmKDK*f7|cX=xj*5}*_t3X}$Sm*Q3|SRqI$P+UsTKw#Puq{ZE7aCa#ZXbJA_ zQe1*VaLA@I^Zwa$_S-$@{jvFxlic=Pc3=JkX{ad>10Dlz+_*ukq$mr%apNzK3g>RN2A|)+`8RC|izy^=RL!@f?$}HgPCH|E$dv^Z zCL9X5>W$-Kh5|zq@8ppsRU!!@2Ey+yg>#enr&}aOzX&LlCXCRz#l!TFoc(?}BOqIj zn2R_5k2hQlJw$I-EBe=JPyVF%+A0`{K0Ah`999S1blQccc`1o=O3XLz&F(fb`rVVh zLy!O3>GxiESCahT*S|aTCzj#Tzkf>s?*3STU-mcR-{t>J(ks&cp1hYC{`;&uV8x$j z-l4w<`u73vf&U&Y1t3kE_qa}GoMZpwynje_&;^NGp_5aCYW?1{Y#b{c^UYnN!Ny|G z{t)L|=Qg-n0Uh5y377c|QSI%lQcVKirL&3y6I@dN=7Y2DKb`-5=1C;bjp0MbzhKRn zfju5k+A#-+;55QIRT+L^5H9)>kQOiDYyM+qcf#yRz;Dc)6_k)46PX=Q9j{*3+fgJ6 z2UGJDM*mBs8z60ZZ3&?-W6=KZqI5>v1^}uRjhgaD-aPFgzcCG;1|jIYVZFyBv~R4| zO*1ukh1|Ln7gWy*T9_Odf75D44WwOpvcvT{N8G0_ut)Oz%CJ8zO;9>ly6b<=VMNRg z49e$eEf78sze?>&K0e@2e}BsC3m^2U-iJ0F_wv3&E0Xz#|0ZmQePpvF|AUlaV{=4$`~G`4iav zZRK=2;lukn=SLp2=fc~<+6KL%+V8ZvCCJIFDGmM?bme{6sV2LbAzK zx6z|xmf-sOr5bI55Ni{qBs1<}yniN==lXD;_dq>O_yf?!<(39njqc%f1FcMn!oWN< zcX9nbAKM6n)w-3d3L!%#ON`<|Q76MgOS{@Mqq?u0&wf4OwYYcJ!MxGVSp}&?YK-Ld zH#uRg28W!SBGlCxJ18U6#qKCP00-{_Y`~cib?ToFCZI(%NDbs6DIu0qF+===6gU_4 z{<{BWFZa;_v)k!r$b~Tgd;9ipVLqt`UQY@3cVVfX;8w0!4;-(TlegcBfj}cE_uHBv z4NaL`2XU=Wk%Uvpb5u_@kti<~SO_>*M%lGQrbO@U_?3G-JFB*0%3MnY^Am(fH26$i z(#gLt^g5_wpwAI^c2?J)FAHMP@c~_qop5xJ`W)28%y#QJbcOlo#vC@^%iBdQgI)-0N-%hlG}& z;uU@Y0W@8wFFnKY#5QGF5JgJcyBFYQ^uW1K3m=R-bl+9qJDR^XMNY+BBsjjM6c4Sr z6C&6%0#THoA6FAgVdwr>V+g=~g@0|lkl$X|(gBn_>c6T_VLeb_rC5%sk*MK08O&3T z+S@qMp=EA_MDsHkcg=)x@W*T)76@AGW4n(%aN7^74uQc$4sh>LAKb-iz=pqYs?V$k ze7o-ucfBuswyM(S%7*fsG{(rNzuC4mU_SdKxP=;ois26AI>{U+yiN-wg02@<*TCEk zd8~atW@T1PFVE(DYkN4wehSX(LrKY)DBKd2<+6!4rtq>tQphae6@k*i*QVYs$?L0O zx5n=!2}Hiz!Pb9SVqkYEjGMIW(`-2xd`}=%<5lbhhc~crSg`K_om%&Y#(vyK(YSp* zDiN)tN9S;M%W$Ag!jbFtv%%esCf8>haC$_Y!e@bFNM|DRySmaB1!wO=`1(YTbeZf0 zOEeG%w;hcdv%sg2c?ri>xIs9a;%rrRSyT;jdP8&SB+aiSuzsUyjypL5vUAQGl}KMO=wd{_%tSdz;)> zX|ppZmT?wo$6wmAY`c@f$E7orZMeG0&D<(LE(4f<&&Ts1x#9Y-+&i+M<2vDE!{LI? znR|9hamj|4%T2>Un(bI|Un3vuxs1rXk_`bj5gX1bcO9R%bx-Kh@+aIjmWJ0>^k&>P zaC^Ju#}%I>i`@yy2HFJ5OaRy`7>B)yZBEEF#EWg05VTckjS>_reS4ZiVtaH`qb7Ba zQ+{dwW3%LC{ojl>v^T(B9*=v-#lypg4!BRKt#V_0L8Z_(7Y3Cld+ zNj6wdEk?)hmjSb~{N(n`kl(&rucph*W|)q{B>af8gkNbPqdzqUO5xa%B&uhZq$4hy zwG(OW2s*yivL6`a;jQq$Wm7=LOD~;w)xRR1pBNH<(ug=$yLP0vzLWYse5y3K{_n`OBW- zALh8HlopDn@FXaSgczqdj-RGxC?i*07W`?zHUW^vE6?dvZPXZAaiK+}HUH+ZGalIy z+HY>_sH3?Hh_Bn+SlacdY?i^Axy^}4)6Jg-J&>e~%>)IK(tZ)VA9^_*5 za#^kV7kv*M)hDg$!E##{j$@2FXL{7Wr$cljC9d2Tf=r_JUGu`+=Q(PS>yuT0LS*w& zyjCvS0B?}Jr^SxFcgsChvPws@u~OjmG|_d1rjqh;8^t`@`3pNyP=ij=gx8{Lwg(yz zLgIM@n+nwexH`;1=!C$hf_dd+xg{@BDrT2L_8Iqb!4un_?=v+hqW1lp3E;#(Mjn0bCltAf?5I*$+%&}gxj&;6fDYG!^AnWVg=(cOl4Qz`CQ<*XIC!Rj_k=<70( zr^bw7L_AI@$UPu?ENMI4I%Sr-e*UNi0;r-Xs%LT8^sXiIVxTqn!Gn^6>}lCk!GaSL zKi%Z6^zn;T1wE=!WY&3V+qllXs@s}jG$W#(tO8{xEh_c-u34($0ijyQr1Yb=HP2F- zo>40Uuz%8P#dz3c4L?m%h-r4{f5?askYlIf7LBZ>ry^vZ_2G-?lb1s@$X${+Dcc~RRS zFl8>^5~rT9VB5Ve&;{*{LGTO@xa+F-uuS?;hPUMsc-GRytA_d+0(X{!B~_Y%TcK{5 zxm=c&N93D4lW!MRFCMG@B$N+y^GfwyJqB#P7jpg78h<`k8;(miWevxjUJ;JYX_tr^ z_iy|5f7V0XWz5tf(JD{(5j-nx$?t+c-eNT=HT$kt`d3C4x!~bm3_vO(V25 zq>=^mk7}PN_Bgpv3JQ@@LG8zb}oQ{m`Oo@SHBh6$;5xXTKIqbPxqhaAO6O{F2{bsyh;lsC5qz?DfW1(LHneEX1} z0gO{Y(>^^P*!(Mp3WecPNrC1{-ot%&cWxK(uADx!^6_naRian(c|fUY)EIpDA^S=0 zc=}>aK`0Dj>p|(wW9voWN{TL6rJj<*QTS?=qu=}wB`D$st8=RvTn8T+34v?my%ziX z1dAl{KegnRUiH~YT`%mW#^O3F^|8Elusq+4b3Bg_HhsX7Y151;xAEheowujWwxMxW zYk?=mG`jg(ZATMz_d>6m1hM&1z|}sJ8i?`wei)CAGUmCv-T-Fweqar3<(i>HVoKE! z$^0($5$w-(6S21GSZRylCcLPSlSvKMhA^3}w4=_S{qn|_$K;vREtK^ruF$(C;XmBP zvop@*2zMWlc92${R_(Ao)AglfoL$(I)HTegUAAB$zKZBX47rP(c8OGf6ki`)Gd6~$5O16(_@-RkqdZ_G5RJy|Py z`nZw_aRSF>9~Y%RfbBohJ^3$4+zULba!jp)p1U|UyFJM zeTpT|vL=hs5GOyp^E>Wb7EnR*y46g$(S9!e+eYt>5vGAlH77<@M?y#1x5zZ?m-mLJ zSd9{Z-#6yVvIE*l1TW$*bH^DbI7HI8tmq%(bjetdGtYmKb*J%C4>vx&OukmFgp^;S z@Ocy9>XvFUVXG}rRin8QMfddAx;wRK7=T}-T|L0ft)dLhsoAMLslBn>cKL+@IZtpC%X`u}r`zO1XfaC}_daJHHhsmQn-A~+XL zhdXT-C~3MD`3+sTmzar=kWjd~#F^~z@$vFfm}K#4Mwo;MSmJW;-=LhnhD!k}&`Z&m zEX|1d!Lw}i}2-UjtK~}@-32IL9H#_J9!q)Vp-g~^{zHBbK*DQbEPO$P{DO6nZFeYOR zRJf1koB_}u$qq>7>{awjU>>#Ie)yZ~CoaxUn;2ov9zbp}0nia4Jw7Fa|0ENIy!Lnv ze#Gc!lW$Y}Ys`r!_e0(%8eO6=RRC%sJGfWEF+5pb zJVhJ*@pDp$e&pLoYlE=A54V^mXoREt>I#0ghbFCM{dFvJivk_o+~`Hh7GCh&bn+8? z4mz33D=PkH|Img@RgK2%mTcyLWSWNRNRrCwemh-vzwb44=SNcM#*MFc_vgHJ^(hWl z$@4R?JSBS*nFCZJ=R5Vg&U(L|qISoX%az-;eEKP(E-0P+mAkthI1l(A{HK%sK5y_a zyqLv9vB2rGPY4MwDq{cdcq9#U^Yx}VGY4X>D+BgAWg_<>^}OdA&3_B_htsYr>n4Jk z5t}xOzljJA1pfK+>3<{s8}fJwIwDz(lTyEf5(k{%<>u;SKCWioC#n9|QYUGv_T;*F zkp$lo>Hg`*a1zkXpGQ0gL`eU_B=g??%8+9Sklb8&gc%MG*@*i#5E#sv(<7>u>iop6 zdr&g5&OJ5tg><$&nrCFUWJxoqC@afb&_vHvRi2;+>zB}5K;HuD__-n0Z$@@OnDo|Gr8);SUwoTk4d{q^VSxS1~jC> zY~4XMS5ay-Cn3~jp~Fi%RUsWTZ)NSgT^9yuBD zhaOS8dgVK~x{_&ERJ0VOk;6J!4_o*8XDu9>dG=J215_ENtPFP72pdb$Ohqf3>XQ4z zQO6&+l@9AEejEmBUkDk81F*JG&??KOS#0*NMQTgsxgCq?8zerD0l zJ<&uP@Vot8@2@7Q!td1F`L!Dk&J7ufA6BFZys4-nKlYk|N9;Rs2z8{Bx&9Dw)bh~M zhd$v4AbqOc%j;^e)!Hh;d>|=Fkc(|$-`~uf!hGVm($Au3hRvozrZ2fM!hCJgd$J`u z7!i=!GzEPaIhbeejux+|P@BbuDPpv;n0US)F^1-lzAdV5o)Na1D$rK!Y<~Ggz|fV8 zEn+1x{5;_4n1u(@;o3?V^^_Y|i!~1ERn8yq4yi7_-EJ#xARe7_ver4%WU5g8x|5%h zRJ32$#A*`c``nX9OXl0SVTiDnLsq^SnLl$BR_JQz*IpmmxMwik zY7W?l&J30{?+U2ZP4lDt8~l(ew}FPneR2oUs^l7{=FgnnW3 zZFBineL@1Fl+}Yp#q@4~fBqVPQ}B^7tjjaikaF0wwyM<+79+iD$U{(UQNM_>l^~0V0ab@DugpOFn`-wpc5Y$VEyBcXdtW zGEjXebAc{ZNs(9G7h4hN34pV&JqPg(y`t)vLhFqWYlq9WVAu_EkH1?`7$T{Z#_sBy zvQwX4J#8@cmZ!zsWUGJ5aK#QgP9$0Akj1?WTf)H_#uEL6(Ii*mvKx5|`SD{)cHIjZ zAWEngM|(L^f?| zX^qaUO*N||*yt04M3Z};fcPb5>VYtsAau}XMZ#m~CR46n{dihIBGBKQWp?lOz1>H~TE!$oTwUFcSafJezmShYq^{#zh zcxp)%i>ax;?d5Y*`cvboNNEO@ZY}Ph-ls`@M%!mczS%`UH~rMqRAzMmq^Am_4Cs!e zrn*zOw23kz$vx|M?rN3e^&y2;Cxz6&p;5iB^!rX0m&T31WHm|7pgJ+l|Cl8tu`(2M zahTz28aHkznzDk3>K|yoB&^cE^K`o}E1#cZTELiTLqHxy55))nn4%XqKgGL;PQ4eJ zd%}4IX3Z9*E-4uv7W`8OeJMT1bop|Z+3|{eQqYXhnJE>P#m_7 zFmfNJZ)?=HomJX>aC9u@2^M-*cC?UL8ve-DKOm83#Ia_vcbA%kT7E-3?yuqy-4*dt z5-&rm>|__O&s1ru>Ktg*@-a-m4g|#4L^w8+7ROCq{euJ3kwfc?^?s<0e0u?kYn!ShFMBRw z3_Jm&O&hn>67($*IU1mhji;qscUaGb6u)&}h!co+lw#GwBABFqtOz!Zr{p*|{yE^6G>b4(m-HqbB#!Mo z;g}`)WH;?mB+yb%Es?n8cAASj4}PUnb7zk@$5xNGyUMsNJLZaEmYA-H-QYn3%sj(I zbksA@R$L8q)FlQiwT9`igz5pjyi!Rx5efS4Uvg@zwLeMT6heRcP|b7KLw<&K`d!EU zfH0VE1J^ikXjCoff%8sK$;lO$q~3Pc0HiE#A>#)f6-{N(W7FR7j`=@NWC!^g>a8Vp zsY?DRGIQjtZbVL26ZV=@G@L=Ch^Jg!_haNsa=T_ba!Obyp z1c4IG-j2=~PUwZw#I$ceCs{i^LL84tMDvu!_W`?X^p&&THlH)!=m087pEly#yHa(Q zic!f#z8#ZzBWPA(2Jnr!TW+sSe#oDj5M&iz%}vDuKk&rn&O4Y-&OVUcWmo&*B>cb* z=VbrIz{={gK$x}Fi?R2Qt0pZdDRb?ow!Pgt?C)Ua9IJF~W}i;a`F~$n$T!3y>gF4K zI;O>ZI@k;J{NKTX^6J~1?4u5fb5zL0xS@h3bhP>QhP$a1(+d?7{%(gVvz#7eX;f)FYz|YwbgW@Mq#l`(Uh}aYZv?y((&fG z^t;2G3Kk74N)}3d{l}TkFE$2qTSrv+pw)!X4cU-Er`F{)%m}q!}jL6G0=dGwBp6PC9Ll{Y_qg0SJIFoul4)Q2fwN5LKQngiuw(<9Re7%L8 zU({dS_E%u6j&=5}o>D=cRB+$hc>R5AOJf$0g}OEz^$#mnvSEM9fc~oyMjPiA_T-1? z&QyRscq&bp_Q#f#=nGhyQL_D#dNjIpKVzfJJ=z#)$2mPx8HGQ>e|qrG0lUN?HpXOx z>A6A_pWh!K+26Y3LJlxoR!T&AtVc2M^3@;2|GNB{d)E=o+U5gBI9liAhvT*0T8S3&t&P)V{wwuj%V5 zQRwd)<^*n?_@vf$J^V!=#RU`#V&0guv&NXEOU2PJGR?P`?bI&-c+4oAp^Xxh)PP9=Sv2T)a@+V#bu9lg&DPw9=cytDtEayeoNhP!* z`$IAR)TVpy<14pQE27)V_!>!nN+hSxZoZ+l_lALfst*sV$<+;SbcR+WTXIVgtBSMv z^K*p<;45`sVJ0mGnIXrnAv!#iMu(5^;hcGwSZp1CK>v$QCtPXEP+weUX|3LZ`EI*X zNMapv_Z0oK0b(Wmh**}fb(iLb)o15Yma?F`33s}Bs$C?rZ-HDG*Afex3?gedll_2m z;iPlVx>VBUgakkhk;0s$zAMpU-&c_x`e)}5z2!xG^^QzGE!W5kUVh-RO3=iG_x=0U z!v{apHlyt1e$;&ieFf9S{YCmq1(hDfNin=a?(-{U@kXjKF_P~H663g)Qoj0zRdnhn zsG8gEJfwuHj9h@^-VE0BU>Y3_U?-AQLp#FKfk8D@N`#ux`=Vs z5bnu|knUdTUny&c9$e-Ksy_#?9(;_5SD4GCE&)pT=*~6Rb@^`Bc^~*_HeN1+JdZ{* z3K33%Sm2zC^Y%yCG@kLRKGzekn)BM)r*!6B<@QiHiF(6*I2;=YCiw@L8V}q|=b4A) zUH8AwI#4}3|0H+T-sD@yaV`$;4eR6fc5J%fCd%vh=ZIKUDSM}B_3m9NO`?(%Z%HE? zS&Q~@XdsZ_C&U~GO$J0Q0jO|Uz`|N&^@z4@%^Y4cp%3W3h%`)&&h>dJz`!~!w@(6@ znhrnn7@_49seRV@*BWnD5#`~WXM$jRLi|kl%!c0!c}sIRTy3%Xgooo(lBc!vSVNJDW#mf^#`x2ta0dZx+g_&ARd z;l7!3hwiyy)->`Jm9TMG9IaQ*Ys6yV{et#X=f;y)jGTuKVOlA`RZjgg*cNdOyHl;* z=C_98cGy9s*i~QEn*>)N z3dUd9Ea8h6t0#SrCtZy%hFp{+_PX4*^BefQ3}~Ib4rq&f`h@3Y@O4Oqo;BJ;epiNZ zi;EQll$Zd#tu|X%l@q$PrURHq%`q+(bwQ7bp-oRYbh>@^uPU;AJ|1*deLgs?W#P>> zZZtRc-4gEg^jtx!9AMY_zPLIwliVeE4)|*@RbqSp@bETauk<>FjN~WDMz;CMb)RrG zn779HxdPMV-*+dqm4|Mr-@9OYQo<{R}N>c1;w<~!<#ERm?vCwhC2}xMr*a^>@ zSf&TGE&(|tNG!Y>f7g^T^I_$Qw#l;n=JqNi&C!)JFk16BIkOUx?ZnKh{dBYR!E<<=!FH3l9&!sW37hU% z-uArZniLH0(5B*oUrTItuPqsC#91$;7IQ(l7M}@)yLo5F)}>k!&RFmmX#B;z$>-ci ze9G1HqBU9UbNctVX2P`Y;rk3S^Q`sa(q#I2LGg{|h{dgoTMYC$D_NJlPo=CkL6;w) z$^FH7pD986<~!vK5n(8p9EY)%l!!fI<%4ZXmPW}&cbMf0CLzhbMB#vmK|E*WEckw& z?ds#~ii7VJEJ~q3@x5hX&y`_}^NO^7`YNpHXtOFGD8XmFym>%edurVou~b8iyk6GP z>dWctg!Kikt+4VqVpgD9>VhkOL3eJapuj_ch2t zBlgTLF`A?_Pp6dcpiToK8?P3~f-Wsc#l0tQO>5{yJTg&J%HEd@cPKG6@lXcqJ{T_l zj8?hb-kLC{8{QIl2Jfa&a9Hr68D(tNc}86hFA0Ekk~XK+LO>$NZ+97znoN{HC|j znBB4+V<$EnPn0W*Hv=hZD$XohJs*xTOOqo5LH!zN*MK-e2~&Vd`i#Zdw36wYu7)>V zJhnR~Z6Fns`B$#jrWL?V?_Q8n=1`l}Nb{iZtPg^qG$|2T};Rn1JSn(=~60gy4A1U~9Ec2RRfnF#K@75wxEK9DtetCG#^ zy=X8Q<;jqaf!%Lay-378YAv#c&apFuvmm(jr%FalhTZa5Olw1wf;e`RvhX3e^HGBE zE?JWcr7-LwkrfdEJvidcSzgdL*^R%?8B=hUe^cb(WE~RI2VSR>*rZjkI9ROrWZ*?4 z-ZU5#W#_F;O34}soYo7;6-Ua5O&f+WAcW60@z!{Ye{HJY_zQh6TEVfS+FD&K6P2Wj zRtVXG_jk-e<3bu9bAR#>FXl+wl-H@faHQ9T%+tl zS`i**cckslsFnIul5#GFy zG0~>PiJL62dVP8@jNuSeG_^2^O&90*I?R`|=kL8Y=IObWkQ|Xy*a`CdzMlxxrEp0& zQd-DS3o}`I&0KA{eGi=Ld^Smy^B0Bs84SG$+V%oCR-$Mftv2-1dM8awOkl;&O3wVl z#+0TAi!uk8_3%iMd(?U>(PI71r1jrqS245OcE9eQlm?i1jd@==Z-<=(L^U6;2y=iM zPJN^rkF<@A-(q$O)tmnaZx%HIgbbueT7wGp!!u*sQ%nP|G9sTkE>Nlv@D#iqf46l89(m1|w! z`NPf;jWnXr3qmj_qB?*#*SI0I{j)G)yKCO<)M{!2$?N)%mL_hpN*kjCfsA}`-%|V0 zzA-+nKR*~3Gr3?eJCSOH!mUIHlf*;U!@s|i8jK?(L?#~n`Bj_~p{BkdZ=}AkZY|FNnqtN}`j>NT9as*6i)&QL*a%LNl)fer#EKwal@Zpx@u6s^2GpCXch z&T=J7y<|MIdtB@EtJd1ewmK)}h6t0%G3-zKuxU!qtR+D9tja>GyOGp}Q;QoEV83ga zgo)_ZTltX8syWVbuu3RTl2-e78kQ09hd9EAyWqG-$J|J67tx<$tmM8jz*)6{cWaL0 zVSvB7pw0g~lgLlvhe}Edi{KFbGN~wmuW;PcxMr%L#%*@e>ju#== zLdKL#70HM6A*5Bn!ilW5P!@!PdQ3#di1Qb$Q=jutthI-jCe5E1cySgmyIoXy!HNo0 zn1p%6h>+3h(-^3OfxIXTI1wpb^QLM@DCfiSdsNU&AE%y(bF`vqlJf}*A~Lk&6)uo5 zaupYl>=pNzhRZcB((nExvX0};jW2Oo14&xBhuEX(zRR!HO3PFMJm+gbatml%;L=d_ zL`@8|9M#C-HZ^x0DQg5`qaZc+V&ZrxqP~}zIY14!?E^suy*GLvCQl!%0?z}&>dL%n4((YRhg?-3&_`2a!*bHBa> z!Iu$nPhAxA`amw`m>3E|hki{IF#isG-%F2r(ux7$yio2v*)bmXuqW$fmcJ&QH;ocB^iw58Nc!~kJX0A2w~`7}%B4D`WAkvtmm|u$)(LK zqw6$9OouhGghO*gPReGa0TqNuf8h@yIx~O&;IKS^FbEXHp1FApt9)N;Xhn}G3J7y! zaUVx36p$5(`&~T%5O;jOXmfU!B_L2>6dTAMr?{~r4jS2|Ti?vsu|dujOBhT=g?PFn z)az~5LSwkhOyzIp%<3rRSXjfA79S3$tGV%j*Q43FcTM0f{gfnAj0iPP+k*JV3CZ)} zsK_seK;{r?!7()6=elK?yP7feA&0QogVkKZ(;(Gju%lDYCDmjp7C645%J7m@&Cp49 zyLs*4657Z0!ifEFkb<+@@{eieWcXULk>=qyC5ETlnHrT8o6@dWb@pz(d8^pyHaLKe zPx|19su2#rTe_Gcy~9x*Svv1F$U@}@poE1v7BN;3)_A-~S@S~QvP#`C-Xi;%iJP7j z{B-j00DITbqW|x^P(8ZQQOPtutgz?lr~)KL;Uw4FV#-1wpGwb+c z=Sjr{t)CnX0bYwib=s(Ofur`!?^`sQ#=kHjiskEu(Xl}NZMfD#qVqARCo`&?OTaw9 z_uOmX+@GO58})9(uSL4iG+bUkR!4(CKDkxC9A5$??R?%5L+OsBtal2^%L%?OZ8X8i zmt-B86bsH}jUvHLE<_Y14^SG|^gZNWQ_5i!7Tqii++YVUag_21)!BJ{;{oMRv$VJp z_Et9Z$=8B|sG`fNg3rA5d4dA(GvcIq)%w#o)+0xZ1?%s8Iu6egU#dt8V;p)dx*!tM z6avgQ$J*+3>S@_55%^+#MC;th`*G9jYwmittks1q^)$*r%JaPkJsV@r{^ z-><|bYB74?F@LmK=Nnzbe&a^rX9CiQTkFIOi7Zo7W>e9H-ut!ScGIRCiVHNAo>yB1 zS=sg=pylCtzx{iGl3iEPkNJ z2+c+ zx+wC;jqlePY%72&{Oc{i>hFT7k`xnW6M@I~<{r>DJTwO-#g4ou$)b*(%wW?B`qe3b zS0G71mb$&3me+$H(~8;hA!=4f^Z2g2K6? z^1m^(K*NfF%`{zOnqnP&FQyNwO5nZnxT5)s`ui^s@GDOP;>-UK5>J@Ma2P{?m9f@7x>}LUaElh_VyL*xXJFR1erXX^VbB} zBTR8YuyLRG>~txt-lysKiO*ru{MMP+#=g`BRbu1|^a-z)%#h%{2yMaJoh|0Mj*8hl zHJjwjOf%!m3(NOOqY`*lwxR}{s}~T3Y<}mNoP4aVCBRrF^EpYNi)m-+QNlPri3=tX ziM!A*LRUha6CQ6yy;Iz*zWO}jCH|;j7Go#fhL#yBS>dB33hA`V-?sroCO(cC_~$V7 z`dg_iIoNeP`7sq&3BtU(ED>N^O(Jh#>?{?DPXkZE9`pAbC_njd^}x8JPUq3)HRaDv z5PS|YQksTmN|n@<0pGBDNEFem8_gs<<=B$DPzN^S84jF=+H=u!0(ptfWx@ED6= zy8j9>see-8o)3(T;<1q}krZPJ(w67hh-M3;y32dQ9eX?)N7iV{@LowN3mh3wQ}tRL zsKT#^tR34cA8eq_94I=`WJuU>8M{A#(AV$2^GixjN`RZ(E+h-4gTnqOHVLS&5;E3R z*B|@RHnq{*Lbwp+M~FrFtrkq0GDt=Qdg4B-M_+=4US+oV7H%pP-Z&;bp05R4b<;(z zNMG@D64~zNK-yeh5a`54VaFgZ-6Sw>>!X>aB_{F@5XD)g4R10_d@EOY{eKGaS2zeU zHH?2#C}2fF@;sujsEDQnm@S_*ulSsCkp#;dft@J$s5L0hIlj{rFSF~(hPC`gMFM2RV(snG}U zmT!3x1SnWC>8w)x{X>NDFKXrDFZJbDbb3ML^;sJ3j6J+rf|6F<;0$CrmW<^R6QR z_4N2B(+AjsIv;hu)Dq?~@hTA&S4GR;9M0b1i3~<;alI%RXQ82THZ~bHMS%1l(Nr$p zV?%%_>G;6h<$r@1oF+%g*&&d#uD-|m8*vd;mBT6%LR*8zLmE}a<%EBT>sIHmvjs&1 zzL)l#7jlukmF4F5@&IjV2OKvh*xpfR3U0AT>YCRCsODw+UleMDwv92y*!%bn9e*yq z)sx^--1^W$0ibGRF>an|(v-IO4570zoZ9TXZCoAO{GL%hSdk>GJCVd)wMfZ4w>Mes z2jcj|g)V4B{Eq8%5ITKYXPrCL1b{AIwS~uxQ-jJG2r-#lOcC$eTJ@8W@mF)1C_958 z%K2}FVn?5NZMeT5uDGEUx+9fxFtE+SXktN<+eNrw-sQJ{m)5*MfCScAWlXs+py~h;xWowkjx+gArmDGiF(pD=-wR;5wV?Ju8Z+c1SX!d9d+=|J~DUwH}(6+rO zyPduM#D9S1!pljvYf;ih=4~7-yz4|2Y|J6KM`15D_cu!|@m(q;7|dZQc>N(}`Mqp& zD5HWQ_Yn+`VnHNzEq|&)zDVx|*i75$UZ7{8Y1GkA5P$QTopTS>p>C3GvFN zh4=^p?=zbuZ)Q5x*6I~Wz2;%jlusrH-|8nRJUYvADujSWboee@t!$+m`r3E=Kr{nX zYxzp%f4_Vl{8ig4%MeuQ3P!CKZW|pg@9%*pYDq+m&1zp|vaOTe=tEUQZ}Wy2*{P9! z9v8I=7|`uDFUJnupcmfMb!-S;FZ1GdOt>* z7lXkVH8p)F`K!p}`7n*hBj>fLTEgDbtO0=uuSXIFkkN8B+Ypq)PO1+><3a}7bym_o zlZz(eo&VeuR!~5!wB>1zLv47UIpfmPltkCAb$z4>K$X2HkH}xaVxq>X7nxORM&lOS z{8V%JmYclI$q2{M&{6vh$BtalumK~uOXXYaVAVgP3RHbMkcgxBOs=J3zb^cbD?Vipia*3gPp_DJEuN!!V)zzg3}W{+OlGBjcx2c2^{WeBy_rQ=87LxazInGq z_eP8Va0t~V-!>W%f-1)!^Dz$IZ`!1;7Xk5&#y@(i2QDfaqhv%}W!&vF$9UDhClA`t z4xsS@Y}5YJC(XCuXa7s$WR_P7B0GnOa2Y`UBF zOy)R=jmkf_M@-0BhT^+Di)08Y1BA^sWhv(Gc0SLop|AXb`9HupvZQg`UVx<7NHq}U zprFZ=6h0AHll9?#9I~|K!6bp^0VX z{(6WQ%}p4YeBem;AhHZ`zI0O==~2)_8wcFHJygy`o*(NzFoVJOtljw0$MKyuTH50E z>-3x=5WcNcJV-vHCfonR>VMswHEMtuN4zUxgneW~y1@SM-x_In^E3nEBZnA16RLCR zVxc+fES-TH+ufjUmLaB;`&s7w(QSB`r7_rQseP$^!0FKKyQUO8`jb`TjK+%=6jLm7Fo6 znM5%t&!Dz(CcWA#tj-HH6_+;B{KYgU?7L;T%CWW^1@LvU)8N$4-j8kT4?7?bpb)Ek$=!0aJD|4Rufyn43fsQg;gH_&UayJ8gU_;4~cfvZW~CaBHQ z+=A7;#j+K5=pokc2}hl!6)nNv98P0+EV~>q3AS;$57>=-p+bY-PWyY7uH7ki_pTpmXxNCN&am^u@`Y3_D*ePJQYHm!>hQA zhpY1i^=jzaSzF?EM<#@Qe?drTj9fyfO+SLxtDV{GM=ev3*a6PY0maODLKIDzm@(PG zDh`;2NR^JvlDMh{UQ2UMYYC$ZZ_Qf+7~!D4z`7dc1zB&oF$(6C@z>BuE4vllV^!1$ z%<#2}S2jyBCIMm;kr3}Q&qt)!fU$lCwN@8cZ4gO_Ze4EVbN|veaX?-`QpK_#AF)>s zErPHPKS;7Ftbx(9I*h7lcIgdYd|S3nC|#ODL-JE zJ?)3=Xt0cP0)41BxoI)j=%!{*F+_yKgXV+|YCr6>4{bS2@LcU3$P6upa z&tX|5r3NQT0s^apE9s_udlHKyuGbpC(J7;pHN{7Bb}D5MiD(Kn2UR=gH^lWZv>d z2|bkLrz9!yDo-V~q&llpGss)$)CuqSs&SQwdXkr*xTIOyVC}5)g;`Bhv)hS6_bcC= z7u{p`86W((#$Pf9nqt+vE3XQCG$VPV$_=*nwLl~C-_q??y1=0A?sIb)GtSccz?e>A z1qo$QmF#ys4AtybRP3vVnBw^Owy8+#mLANfZDU(GBbn?&&x}uyNYi%lcl1SQ9F zVNp?W<<=uR#iw%VjS;gmNoK{C9H0v(RBC2VyniBee$z|qIPUcWh|A-Sl@ERq>Ho|< z)(V4;!dzhLl@Znfmc{KMTTH5`1ib?Y4_!#tqH!(rk1D>MjIUVRTA>nnk$gCCX&0DGS<~BfkZW`i)`xZ=K2`vLkajBJGgQYZLlAIbyR)9 zFk1P?RYaQ_j3Tyg4U3cnncJV{p71W!UvF{^xVaARKxr}u#`NHx)Mv?Yvj_K6i-uya zJXOfyY5cBS!$$#0tHKBo6HoZ~w!Muz^)#=*8mpzIfa=+sYUQG=fny#j&3z-ovsWqt z?jN^(IV)qDSI=w23$h)ypc60EP7)Ju);2IeAI_b<_c3z&qG#VfyfEh~;)bWn#`VvP z%!&U)8wQtVN&{d>9U*>(8(Mu&UoG9$r%VoypSj(ckYyEy259!$CCEdP z#_J-RY5NNv@S-ekn&YcYA;d3B;&Sk*_HCn2y`64-m$=OPsKwBpc2(5Miw`V2S4>TU zAUhEMtia4)0i$?76mFcY3ZH0|5IFgZ2}1GsV?rIrrpDqaxGGrWlCO-9A!^mT>}3xL zGe#BAT`o$Y(In+Otdl|;fPecHN7X%^&T&~jz=(id4NhIwiPYNa32sY_D*<}>TNkk? zEqS%~Yr(EKyF85+=EQE%Nj}or#+-qTw*^-VdE0LjGtmVB-c%r>FLMq)bX76<_ImAp z*3mje*qU8r6*=^v(64#}wB=GVm6{U;bQ5yJzaGE+n^HHjzg0boOXy9GR%R0PoJkLAJ- zqiS^+?z-h>9apznyq(>c1tdm!9DrX&yynnLf$02S)V*g^Q(4?MI-`yXDh?1p0YT}Z zOO-w%y%U;rk=}c6Dg>kk2qe@XkWi)ft{`1%sG&&-y-4W&9-VpKcij*7+k4l#mk(J< zRu<>%z0Wy&|Jr{`PA1mdM9Io1$$fdtpD>w>@TtywUD`8qJtk3`hw)U;a_2Jqvl%t; z_xQ=iOR9S#9$I-_Gg^~JSHd6FiW*$}EiNQG$!5 z?$2%f6uVQe$0&{8R94AG7);btuNmRhWt%`kQ63R10mW%Gt`%J??frA)neN;;e{U9; zr^7`W39X<>tx3BqN^3QC16Hl;q{{+>+WEtf4_38=&@J-Upm^`wJms~lKq z?bsNR+BkG;`^S(lL&ai!*2L@G07-_5Ik$*}PP@?EWU;{onz5K5ubkl4VYV7 zwEB|H=PWtWknu40LGfk3&eSBAV&0hieuUz5`$o?{!vm!0F5zak6|%U0kYymBcyk)^ z0I`^@lg{_f3+YbiGMiN`{diLAuMe7cI=)!)^Z~|^>L`yOKplTgq^TQRwj2E<&7K=zbyAbmWQ`7>yuN+edy+HsiscFw)#HQFz@ zriB`JV(S@MzuxnNWY<+t&3W_R8qSCA#Co&-<`3D=iL7_lgJk{Uq^%E!^XcSQS#Ua9 zwE;yi;P!rkF1u{rblF0<6N}9N10v=$Q}?XL0y5{f$yFb4%J+Z-m^IDIIo{$Zp3oRR z-$_rnFbjjF-cGgkQS~blFZn$Em&lfBQS;ay<*I#Y>CUJwz2l*QFl2JQX)pPEY0c|E zh#vRnk{#<3@1S;Lv+8D$XiR54&UEyUXNw#T*#z0p^%lhF7+0%P%>awPjKBD*0z@Z@ zXWD-bE4en-5Z*Q>33hDH?jKsF;n1 z{%rVHU4Qe23+m{&q`iC^IOq=WjF^odObd7OHYn3eW zCN7NX-DHE#u$fBS$jV*pnxLOigNNcT@OMrpF}|7^Y;s&55^@uuM}^Mnwer+@#>Wwt zXWvu!$Civ*gJuzmQv3Uv*G7>Y48^Q}{PBW%`{2mU>OetFXQTSCxR*Qa7^y;=@@F$@c`@V^2jsV7NL z2vLY`Z|^Uw4qmRLIgbab_2sV)Wfh-QM`D7CGf$^mFAdKnB?e%@7fW`#pG;um@S_g^ zSp&>gfVKdcydhE2nDu3at&@`pwB3#DVLqV)Bx9L1+5(Mq2P8PCy01#;WPEM|Gy|QPMTiySWS<1EDI@0i;WuS!-@oy1A6AXo9b~=XkyL~Uj~_o0KIQueGj|h5n`+t0PYP*i zC!8pgnjBb#EL%Oa!K3!--Q!=MPg}U^CG;4t_YUvqgPQ7Ta%T9yUu zAo#5cYY#Nfy9lIvq{n0x`qsRZ4X-4tbjMH*lFjFy9;P|aGZ&;8>lofnDEThWYRRZ4 zk_|#w*o(4b4ZG~MYShd*M#d1a3JZZ9Nf#7T1Bn>{eyZFibsOiY68`{}2*%$_OSK%x zO#E0%9X86E`byc*SH8r9|I|l?azf97CAT?1&Pms+uvsf!Nm4c_ z?n`F!+~>_PcBj(b#jt6^*K^3_MYN0~&tmjvheo$N1|d>phX~hnm4hYi^XWa6tgY^W z2GZKzxh9#V%|#I-Uny`DTx}{F9+fb|qsQ#%Y;Lls*^H|DUc4(5Aa{J zj3Qb!(IpvI^UGuM^h$HDHA02LO%B!cYYld?6r z?geZ`Jy^nl*hFEb*m;m9oqun@eACe{JMUxfXU#^MEhkCo0jKrpzS~tziLK9^CB2ne zUKl2pN-i~BXGMHy^36K+u%72zjZBB<0fwVviJyrgfES|gyTt%k`{Rg~;AR%*OvLx+ zl{@}}N(aWrS4JFtSano_JW7o5VcGlaHw2(Vvb1DZ>O*#D?w|A- z%Dq7a40t1m6^nR%ci&2ML`f$VUxTL*`{X z7c{J3gl5as1`LC*7bDY~bMNhmKu*5ESC^C2OZ$@USw_gI^&m}}DcPXx3H!0TS9T`; z83$mIEKHR*A%o}cm723f8S+V(g=TGtZd0WeEPFGR3z#o|`?!3N?Xcw|p<$>iWY_ha z%rcgL>2N^s{?w>R*W^Wsh@tF}-XdQ;sf+ z|2#ASK%`TBG1G5fXIP0YNwmsiBG1BN!1wdMkOvk#X>ndr18*ZZ1(w>*{v1YaG_s}T zZt)vp{J@+}IR9OGC4(70JZdf>o5M%bAJ$vZ_z2z;lTOhZLXk9VU_5O<3E1E5ovKpC*0n zDOAj|cEWm&^>5=6NxNI@wUy7@HQi|$mHg6AWE?+YsQlAE`3A5vmCH_XJ?XYi1PTixa)06OF5Mmw>a$#o4BeY-kLnwD8EtCbyOpI zSVpG(5TH@iGw53HP%j?*oCV@1qC7AOIDY~{@Oi1kTnMOT2m@-FS0YxeCy&9mfa#l( z-D9VjpmL8g9a|80=mz`J95F*it9{nX3)Va@EHxevn66UMJ%!dnh>wqtf$|!0uU3y9 zX9>;3B>T8`X+O;d%(O=!AXZT{stCZ_+H$6%2d@MGh$HW!O~~f2crtqS&}at3md5Xz zqF&OA7UR?uJp$|iwZTtix8qZzdlpToiaHLSLg&OA`i<&qQPRPILW*YR6<%B5(M^Db z+A3(-Pe@O{y9OThrytc?8=$FvrJy=Ru1r=smlL_g)4|l@_xFoZxAxKY3#Zc)G_Prb zeDq8+8SnA#zZs!Q!bsl4!wD@P3$TYjF=NLUR2v82J9HkT_6=N!^e~DoV}}Ya!@ueK z`m_lj4}BycA4w7`Jd^n4=IFm&&gibm7$oKkM($U%7VHl$Rs;>bBrZ-LTVdG9=8r1> zAp$*Jj+QZxs#q@}F#zxMo*jgK)Af{0aZj%sP_25RBr;`UirmhwGTN)G6InWNa$L51 zMM|>*8rT$^`b+!!;~L~=^mfwhvt;b1*G?+&^wmnC!`AKm@T8^W&pwcGHR59=(}t2+ zz0cEW4%fUp-;>(5Q-~s*Z*^;Usx35Z3HD>`O?5d?!@g7Czg}lG2W2d+aqH!{-z;)} zDK>g6Gnx^lAAeCZz~hN8HB(y{Pj|o1l`1^%TEgD4L;KQL#Y#d?*hs)BRG{3{4WP0i zy4yDdcd7@NGS2g&2EL#Axu`z&_ICf%Fu5vI>trN!pb;wqUFieNoPe#6pmddMbow)0 zd|7FsVuurwy*Qyu+&m_4F8A>*SthEs zC{XPV2|X80)^H$k&Q96aI*~zL#{t8h?xChe0@~Qpy{lKQ&+%!bAs_$3W`}*Ew8T=7 z^a?AD?oCf(F0suy;%qsG6=4gWGOy7<|mC8P4&aLtmzh9cMB#=yOF zaR)tWiI?Ef34W?&B5zsjoittgVx+XhE~|4oTz+Oup5AXI{k!#zun#~OLQ8^apYi>dfBh?` zsl|>QWq<#?Q`gR$QT)WJCbBkSF<{B-5Q`p<9Q zws_;Cjo=?SnA`g*`8e7#U-y^4;n&&M(Xf4nTJkor=>ElD$tP}et88GulL<7ui)I!Q zV2e-Wcj9jR8}a9^JNNoz8EL2FW9eTS;_qzckDaH2LTp)<>f3x|G<1l9LTnPV2zA0< z)!09ZoUGrk{ph2+-`HdwdHN|yXC;4*0TR!@FySv-mrHHj{f7!K)&r9Od-XHD8?ue0 z%(STqRvjD)Z#@*+BvaCfp-c#FWh>%eO#bh=`klWtP=CdYXwWK65{`P%H`MJ`jpQ|N zA&C(SQTx@B|JoBZw2-qgD%Zooc?7TMzseDab0Av zas~Rz?Zt9Xd=+!B)SnAnEuW9g#={2Qs2GDENf)f!)q<*l(+x!n0qv&{*L1elzb?<;&7%GAvC}JrIwmQ6oB>eELba&x=Yr_u!57aCyI!)e0;zs|Y7R0= z_0&<`27y&pxPJHn#&6T2L4uIY`B%{F5G^4VC{ro?RQ- zdMX>;IXcQkeXA~EXKj$A@yM8Ls7{JWssY(L5-Z7qY=wEXj*lKL`+>|Ae&3+)8a0Xg zufG1pxi^j8+}rQ0HEmDMxlsu@DWhrgeNVhbpVyxSdc0F7cz>k&hv@f6 ze=+8OeznT7InKAVWlznWGNw?xCog0xt35mrl%&ph}6J z!L1`XH@4;Dc#3f<^VIq5fI*9IN;9gxjV1GW{!|9U;})1xIg_l8bJ6W@u*%9SZhPH_ zIf}Ek)KW1lY6Tf88@i!x45jk1@ncvITi&8%Z5Dnz8qq24+Gp2#_CMz{KF>v{JI;Pm zLwS*D%#Y?5I0)M-=c&e~tYiuMFL!K0l7UFsXGd+rlbde+kot>AH(B!v!Kg6E>?d`W z>shC1+gpguY3fsfb)g;`bRhmd2jHuZzJs3~ty!RNgx9jCl{hKvOzk>{}(t5Jw;?)F5MVzNs^^P-3?Y~_X0 z=2N_rlGN)1t6-#Csc5o#NYj)0!P%D9gC(^?&(^^EpR3_Z5Nu-k;DjJ-A!eQn^Zap4 zeP+F+nd{>{!Q$G{lhT34jdPph-E_iXVr;okHBeO|?A1*#SPT5pu(n{eLoD#24A$wW zxiEg6P;JyQ*S9@rThp~*TuDp&`$C@)gs_o6K+5eWvl>cz_ILQ}F+%y)LEyFWX6C(g zIq`ZW)(`rkR^hp3Mk4EmFp$%L@FSek%%}{K)r$+IAOc_Oh9ob?OAB*#4L}~L7hv|8 z{C{YDlcrGuE+=tBrY6@$2g_F+_ueUK?fmOqX~2D;AO?EseYCji1Pq8KH^dr zoJeGw#_|$}2&Cj}W>%iDL-VG_6>TnCk>gLh7h0U+shs1bQB|VtA{|7*R+HM~I&jvZ zEITSXo(E}ig+({00qbmYmE> z38SZ=+2Kj*&k0oKwkmvJ9oYDhuV@p{6IS~UDTdJfmc#65URt*~mR9wYhpI1j#m|~4 z!JKCfxt%*scy98M*O%}3v+FIWqOta7w7L_{_2G7%|Hz6>g=i1?1mp###4c2e_nJcv z=d&dV67ZlB?%hL$g_3!ZBFN+qf%6g5nTJb=5=4G&C*KMKo$|_C&aoK&7D|8%V)QhQ zG2$*l8Sj#+F8d>@y97GT#;y@*%(=OqO*Z;H@fjY7HyJ<2W zEU_vtu!ISvXX+#&Z}$3v=z3m9eP;tN6It%vm`G!dO>K9qd4(>UIf|I?J#(xCDT&!C zyY(&#(8jl_5?5Q*(CSI@F%)UyrjvZc>jm?Rs3Td+UY5OHL^kJIFjL1HU!_;EMmexU z#g2mc@=#5s0>hXb{+?eE8swElpS@cZyE7>9Fv~j=iV7f`oc1Dl zKTz3JO37KfZ!2rb%aw#sv zUS`S?XDAx^u_rAkHZ_9Aj|M5hDo9t{=w_LZ_S{hp10)|8Sk@Bo0`{%e3z#VX?B8!! zv#{knc4aKxB!Sd5&nQ!A+t#lyQY%k;^?8~K->#{KBX>P#xS@8Z7$>b(>-hNVHLWxt zFR^o#;r<%%d|TUdkoLy6eJ<_vUbn3{ZUUjiykqAAkB!fOdXTvztsd(m<(M?4>Dt)2 zI@ge(y(rV6YWoQ%eO9Fmv=gw3Yxx36R>U$HhA=0vK`=){-C6u|??g9-BDP6G-n_{r zpRh1&nF{US7!go89zAI%In34UNQdL)w_Wd#tuCG>8_0!x-a?NZhFD2c3sMcYh3tS6 zn>`9;>`Y-8#Ko`>*y1I&qUR0mfQ=E3wT2t|05z(nP19kQY1~o@MrU=DdP+nlB;T0A zvMIkSIs$d_Qww-cYpvPv*{V%*)7D1b6pfW9W;F^@Gd(&e|90|fiq+EM&R#c4dgpqa zX_mFRD$ix&O1;N`)oPZT&jPD9lf|k)<1v+Q)fbxH zs+$EBM`L%9K59!&$j(VEpHHC-*i%bd+|WCzgK-| zdV!MsYaH#V5uWbbSU~$o1mZ$6Pfs+(!#5$f*PK)B+QScvf#20rr&qm*6?J+0NOxDn z0R%5kI^A}tdjYQYI9n3}CpJV+>w39;k}Y;pp2fO=bb$)Gh|Ql&l3d}lDWJZIx2^L1 z7o!taQPXbBkjafWb0HF+Gn>{I!VYuOHiXA%TrcqqI5Dwkgl+HLy9W%~g5$QEGl?<#}$E|-AvhR~W z4Gd{=#9SiOc@IUc8k}FVYl5LVc*HKzl>O`Bq4@NCeaXPlzD_&|$1S_gFLq+KHOi{y zt{_bS1Vdz@zW@%gB08tTIB^fVzKszHizzBU)4+w)TqH zxfB!`8c`SVw@GV|anID%mE`5cs#M;TgKm>LLl7ZdJYb$@c^j@SnYHerWIc$&4BN=G zUf_AwH^z#TZsF&z4^6Kz)ao0IfbP9|4oj5M4hR@jBvQIf13(1jrw0B&j2; zBVR-?MJ_AioSXK-3PZY4;;eYYV7L7*#kM?MYgzFY-{3wFQPF&+$>73JQg3s4Hqokk zst}Y=DkF%wfLJNMLTmJbKQtbMIQ=~|?c8ddc-}8OukFa;8{K7om@31;Yz$6Y&b+so z>(*yKA~eF25G$jPk&p##_Wr7U3n+30`aHglX$JbCoT*;jHpv}sGmYqWHj=#2=P_qk zv4_&UpK@h<_wi+z>AV?&Ti-J-tMH4Hfhs(e0r4qr7pF#9fV3&cFeD$h>3#jxVs)vL zuMh3!wPHykZ4{?uM%?Pzqhb>)jfCRcj8>NS8GKIt`_{D;v%2$_Su@Ga$P3`DveNq> z(*o28@Ci3?qJS>6Oop(mVW`~{uFwg zOgRufT65Sz*6Zq`NA%u+dUB|HLuLJB;142OTp#jbA79&5xiW6Wt&+3T`?Fx{kNCFG z53b>9rQ@mMlSO4>(E%b1k_`R~iVS)c4i!pGx)_bPv+fGji0?hCGO0p} zS$FkfNA+;esH8|W*FnAt)FVQt_B6qt4dUC}>M=%mH|54eOJA*eJ?7VOgFUPN_APLV zK;56aU{E!e3LD4W4jgh8xA2Et%=CtmtA{cb{&JOS?aM4K4Xl#tuR zvX~^b>|U!xyxGe&*kjVo4*tx+0sOCDe-NLy!0pWVhCzzK@08GeQ3I-wS4YG}e*f6i zgA(-(PBJQ1%DS63s*2F<lUGuk(InH89z8mU5O}jVp{>E~-iUAlqN;x#n-?YV-+%!} z01EGp_HW|7NZcxN6{7EEe$;pzFX(Gb#}I$|lA9qKFA>6^gvaD3l%P}Q?HC5-=}zqv z5aNW^zkHjGZuc-Ed3|AtS?RG)E0X$NQ{XEcIck@UTbMg?-H=iorFdogeM4Ob_W#{m z+33G%=_0}+-$eeQM1OhGB_$^SFfdNctUKC$$t*QBb(*Kf-!}RSHFKwV1B7QievxxP zrK)Cy){8}|R*gP+eCr6n68y0v362K1fQOzm?+v-2q%qqEHpd59@YxNx_xX3;4|F!ogV z$%ZN{D=Fb!=jF3gvFN->cST`Qfkvn!=%9b(H8SXV!BmDuGHr3G^^*{fiD}6zpxSC# zM4HErKRp&nt#_AS2H=x^nxN7W5iq51e{23G1Tk@cWmzYTi+_))c7}mc1M62GU-P67 z1J0n=RknWL&dUPhF}$gDXj@88);~KguJOYG(oZU1%5}A3juB$%pw8vWQGxG5G1jDm z5)7%ZX$t@~JrP`fq~0^^rNWAWuZGuF1tx-UIiIiVO|2N+^h1joO-vQn;8fpgfBFO0 z=hKoM>>X=!>Y^^XjK^LFhn z4~tK{d~>d*%+s0oO_bQtVXAnd$~0MN27({Sy;yN)jmpM45ihdX0!Iuf%dAtpy?~H{ znIE~eSYt$`lebNJc=iX$lvLGOCphb|E!|ETyL7y=naE{aSo-y1eQJO<{C>A!HD}xM z$kWDB0Uj}Y~Ugv0`~ zy3v92h-)B0HUBAZ5>#-z%oCaoSOPF1M|1B40-)aKku43Qwm43Dt{xS z!~IEP3R|@mi80Cb$yIwhT1%uNj5bHWq~gew{k&$+zk3g^v{2{4couq5qn5g9X}cR_ zT;s?E@8N_@_I7?9n7_%flrQ+ndEXnhiI;eyky*;&52AZXOzyWicJJWJ(&~Ckmt=Oh z>1n`T%Maqf%Tw5iZ_suRY`sAESTRGboF2=sj>-!+ISEzUvxhbLhj2Ib`-=O=pGRF6 z%>}>Ts-$=y>D%oas501eKs3v<>u)p?vmT>2Qnlmnpau(ahDB=emULRI+7kTf1Gxr5 zw{JHew)r+(pWZw88+mz!e0z0&;xHrX;3s@lCxzbU8G3K5P}G$1yng@SY4A}z)$-os z=AhKg-a}uF>UP$HPMv=V(?2HJq2l_EB;7Ew$km$_G^e2{tk3u&+~oyeSj8aaTW++u z@IGJ%ub44NmCyKU1nQ-+V7oEw?^~}ZS#n;%HL~5w?e>_aQe^BHUQ}6rc#->5jqUA% zo!uBD`Fn%7Z*~4Al)Yi;m{YI-g@@ar0t_TzeeDq8MWWi31gc#IXdiqP7myv1AfeY| zqT)UvKVit8DNURM_N)uibw-SthQ89&uQJ8OiJy#fiWCA*SpzA3#riuHR zuW^aB36i3`Xw!Z$J|YmHR+_kR5%MLuT8-Q|T6OHf>qltRLRGut8%N-7rF}$s z`;E36xcr2+$()*JzAlZd&->42=&hW&B9l)XM<#Fw?;Nul1s18`-FYSY)apg?Z7Tmv zV>M!^Owanav^1EZG+nz3U4s>%mH0NOk7EmWJATtmw}rb8;uE*~e#{jFXU*)#^U)Hl z9Sc3%48Y41XAJI{!!%zX;O>=-XsKuB$MMic0+f+vwh6hnz9ryo^uK-G zfg>F?B6H$cbG4s8{ysu0f3F~Zg4uvHR+}zF*2b-!Ff1U|jORi#?dePWG}|<3l0EW3(kZ z&wjLy#XzFKCP8h5!#v!S=5b9`hySJFy4}tIr7b6c=E>1Nj`1+0ii0kOH-hah zrLs)YvbAriJh5=}JY#fbwFHzQ?o!;((A`)G6=Q_cD+L@ZGpGJ<0%uFSmUCPBPFwyD1ZrRz&CNvKzOJlXtPuOCz*oo;-af{Ai(Hg;**(PmPD+Jv-#Y zNM^Mn*fa=cxw{@GIawOl)}wV@)g{ToT33%!|C_6o<+Lef#r+w)?T$$^LUH~9 zb?XIb)Q0qp5>tEh4Eiq_zzca10Z#Fr$kvV! zrDaU(CWHpgcy^BnqGoWTbliwZF4f!mg$4%D(VyS07Mr@@26+MnvrN#704>A#`ggbW z^(R=zmToF8Fl0kA)n6D`D#Q*8Sp3WJKVz_a5s8Xt zY|EQ!toC(|$M!LxiW!L{u4*Gci{0h`q3k@=b+TIdA1FUQV_ba^Xs4N~3}iB+N=(nA zY^I`a0jWD)kPUJ!W3oxuxn!T!0XL~Ri<@d9lLqw^7x2g2BO&T{4jR})x?QAM4_Ta! z(V(*y*d02bAMfwbg$Jd}fD;2<5xNx56&dXo0!-51J{k-g9VK|^VqDZDw0O3k7!VI* zlsmJZFhcQ{4vLD8Vws>XYgnC3WF&>~m2|%;6>Tx}`LXD7e;4O%YLO)uQr|_<*qmPd zb6~MR(W?0@2*@SM$)d|zO6@VchITd{NK@d{>s5RkRrdw#_qk)b9OI?Ogm2>fLX~RJ zm5oeMMsyQT?d^R@SfJ~WWtIo4dw()QF?;@@dzB4&ygzE}mzC`{OZuu6&Q}x`fO#uv ziA_^kA;v3{wt2%(;v>j}aeE537IDmebOp$#=6n8&86A>stf;1DNInDf0#~biQve1+ z6@3I<9WWy%kjFLmj1`G_G9RpG zy)I8ewJ;%SA}aEb)vh!lZAzJ!7mH8ETUzMqZQmn*z8?-O&Gws>E24R0xVsl9-9+CW zqNGWg(#|&CAd@eG8t8lHg$hdIr9}cgMpr5bGR=2CjpE5weMSbiF(hX{i1z9*ET_u; z#ADM2=&1Xl+M}8{%?v$qWebe*(oY{W@7_q0T)GcgzfCv!t{fgEEu#GmSC{AFZt4t| zCX&YFdDnEgS9|bA)Wtf!!YAz01f2!ugls-hVu>#S0mg=TLC z=I7ld>+MgEckbx*^~f9@uKm8UmD!z+Q;Sna7) z3&ZEbC5mJ6U@Lj(-iVLeNmzMrWE&xnn5!148tL6FzcLn-N$Y+o(1Xu6Vxc>Fnlaf+ z6yqyaSkt>h;e8*%<|yWb544D%u86#;O9+h3z+mNt^JhC-=D48*lnahgSjsqYdd#Dt zZ(E3lTW!loBCyIT9iImis(=d{i;TIoujg!7OmaT-R9dycCQLYp+tKW3)wvsfR>(T= zXXPqNP*_i^Ny?zM(=3`^w0|nW?BNX-zc?un4fNn1q!?~MHq@w_|y+H20wIo zBtONcYp>xHe7a{TnA#ZER&ALCv+QF(CI@Pa1n07PwRnP!<$QCm*B7-%klZ7TZry#2 z6V5rH-p%LDCv5>s>p7NSZOfp2Ek8CxGXHD$%mnwNGe1^q$cgBLyK^w!H3{?6_P;Lw ze}Kb?GIBO*>=o} z{oqgfpWHk-tiIg2t!ul!47%YemKrFDt|{h0EXLjxHDQjnnMuT8NFJ;bQ3Kl5Z*Aij zME#%B1>QdWcUt-N?K{wK>Gapz|Md9ZZ~s63;{U(p3Q%G~y-FR%_Oo;nO*HboU0Qh?d8BO?!Gp3tT{;kVbcBfzmZ-Fx|He;1lDb zdIYP2ybX2jSbk{4S1qoDl;ec^!Z^8L78M_(b0}WMAKx#qTwnK(X7S8hr~pEh7k-Gm z+2ZI>C2f_Zh${h;R6#9emU8`@PgpWtEZom6@DmOcF&|Ir!*=2rx8n}gjd5uh7v<~8 zSUa)iX3lsr+40Y2l+Pk<*?0Tr8QmP#mi_kJ=AZmS?3tG_9@B?bmg3=U`o(64hmQ{G()omaJ6^Ad-Z1WQZ-v0vbT<)jaZ?>p+*Q!J=Hhns1q^tCKvOA@`=UPS&w=a^{U&H z>Mf#EOg@5zTBvV(_FO48gj+mHb)DE)c6aKh{k=Wwu`8S+@#@BvFRi5Vli_KH@OsAO zO6qt`$Yfc;u6rZPG&YIxqYHKd)5D)J}Q^~kv;@B z{Tr1rVCRcG%S(zoh2k;5y7RqEthT>`bmf+4!AGV@Pz94ndwp-PqyzgfuYR<&AWPl- z3R-cc>EJiE7DteR2o|oxRJUotuV!C=f&P=ow1?eRQ@yPZ5nar%V(wagQ>?q~(VK7E zIR*503ejc5KX#K8zgRx79ral+{NoR$YDJls+PFC;X`T4CemqfF&Dl#kG8QF2!x{Pk zCIe@4Zfnuo%YmrKpR1=6#^ga!*Ir#Ss1b@ly4BmBm=F$}igQf;lZ4_4edGUmO8p=a zU5JK$Yaa#TnklU&;hvyro36KjU-wDon9%-dvd(JDgcZ`uRrBR(~tTZX)s|$j0q#jgpmuj%G z&M;;~vi92&!Gf&)R0a^{caodOi$LiFxBk!b6)L)2mSnf9UO%HAPKM4YksH83B{B1b z6hzYq7@`k;DG`g9t`IO`bJjAiSEW)G+AJknm+#+!Q&|PlL2YTqYxGr9UH^9ns)=T) zysofxP0h7b#>^ZlvK7k_`;xsjkz?yyNbPG&&co{{K2!IK;Vf1KByxKe6*AI|dW*;6 ze=aGe6t)BEEXs;{ue-xmXIyBzXPOc$t|U2)b}___L43-t^plYK6HJwx@cHLGbboEk zkG~lpx6YXi_v3`*Gbvc)l{6`zsls!0ANLn4nc|$d`o+&Y-aW4t$@tjEg-@ZECbm2+ zFQ+Ylu_`NP3M-ij7zEKrWg04E$!LX-I;C2;G~YMYvYgc?W*vjD5oVpIEH-TY713@~ z;F$p2PoQ*h+sKTrLC4AKW)@3*_Pk;4ZDpg6lZfY?(jq&e7OwubVXE5;x%eDB-#Gg5 zMABdXEoOEeiNAM!P!aAPna$>Zi_>4OgYR=BK8HVLYelnD;H?&C?KGc=opu7KCnTR_wsf~hzzm6vSy8~X3Z>ymn*oa>Y6L#otMfh)w9@yK_}pIK zgT-y)&1woPbBEa;-^(4#sXyol$!}o6se%?Bw zh|fgLOw9qo@A!zx7b;kq%49auK7=#jRuE+esGoII$KRc0YT1CLS+she$h-q3Jr#wi z_}O50tuCd&&V*SxP!s%W9~ZyPoPY^8;hBAi;C7Y>kQ^Bxlg+T$ywqhKn;w~~%PXr1 zu1UX9h*mgSJ;b(Xl0KTE(l=1^8I^$UMc>9Z=hE~wvadM`a*w^_QIs+1VrmvCF_KK( z)$$Ch6o9>vZjPm0aT|JG4bCt^(Nc5de6;Oc$}V3(fEZ8@)~6jZO@?Ymvp7u>ckF3< zgQ`^hiM*72Uc0e3~oM_FnaNz}Ln|V2vIjq-0py0cIlN`tn|G`S>nf*zJbi11+xyU;&Uo=o)Pi$i-Qrv(o z@1K>t9a}^i-BrBmfZ#MHxBP{rTP;@pyrU;^<6!t=w9Q7z*pLUcTjT3yEHn6)`8L5o zE1QHP57t|H)lrbCllNhV0a|#U?l66e>7{4(1^~$(W;|4<#D7VW3z3C!RIxw}M@qy3 zt4e}5oFBx?hBaUU~32lI@1-?i;=SIu%rf&H=~OWGQIxO zdXjIfpsT?QzWGQw59UN|uK2~an0+kc2*_x?pZzhMkm26hwp_BSW?`vv&$zqi}@f5bAscANSAOW@_VCHM;!u-z-c z=EXV$JE~ZFo+R8WiNN(YS~xU37kN?hT)XA5=>I}PKvu_MGrq`nly-nEdeIR10R6Kh zzO9=c5==|G|Ds88wdMGEV1X-?`%?~Z0zfGDj~}-b&0a7lSK5H)7I80$cgesiHotAC z>psB@??AfcR9nL*4(XDTGkJZ;lBBDcsDC~3@5luvP@2<2J=--s<20i-HWt;aXmR$v zf?}G{6if0trcW&en(ow8nf5itl%-ql8APW`YVomC)S7XeXbo@N^3PS8nA_|9biR9U zE9le+gM!^IXUkof!Kv8ZSI(vPUh)A#%gER&wH&+w`~8phXHy2LsKgGsTS^01Buxehm(2l`))kToKSH)4^m`$OE@Qa%rn_uVU;#cb?W zGWJoOh1D)US|U54DH><79yO6WoO`6e7AGW3c^VTU8w9MTh3(yKGfhy6WFu&EII0@f z5m;T*By^OIv0tWql62Hs%<1pm($my+=th}98&k)_C-U&j!Y!>{HAG5}c`I$RC9W+w zSC!86^3zH5{5aK8PKDFOV^$hg0&im?$>zrFD$g(mm=Fi9UM&5mjSWWY*GKi1h zitAvhlox8ppJ48{l5x4{PBnLLJwERgkZzO*Z!f1e$F~SPTz}uSa{y1ebp)Jw>jeHMK!n5gt|;b`39BDdCz0hch(1cgjV$*OFr9f-n{2~ z&bufe)jQ|^zRJ5Yb{e*x!a9h3#Sc5FjEe=GuWn>R5<$M|>=;LO)Wn9uggw=!b zgR$n>h$FVf5-#{-IF8+LLTlD^x% zDgVRMsxY>E-b~SUDe5Yz{CLf!K^IW+SiZnBkC56s__A9 zak;hHA~{4+oX6p0CEYnCQ%~)6((5h7xr0eFv^*=5y2tR1#>Dj*;rwIP_QnQU$Ho{J z_t|kq=+~i@fy4QAP%+CTVle8hp#kQtH8 z7@E71^yDIKfvtfN!+qyHZjyRZd3mpY(p{N+Kb`sHz8jR=Ydz>p3CZsbI+SCH%ksOF zRXpDiJQG5$k19XkE~fm@cdfBB5_w{rS8H;WtR~U*b`p44{Erb*BM*_O?%%fvI0Te4 zwQh_&^-oZ*Wu&e=(Ql6{M8H^D6g1NUC__?I){ILJsFkVIN39hvE3iHrY^3@!b#GID zp7&o|8e+FQt$aYONR^d0Bl2xmP^!HaGvzNak(+3|tUcLYw>7D?Vi`eNDSJHa9i622!3^KjsJjU){ZRSXAr( zF05O(fdUFcNQZQ)bSly*gLFwVq;xaZ&>`I*IdpdnTa=EG9BJuhh#6pjfwOS${XOq{ zy}xs=_d3@({~Z5e*0r8hu4g@;y6^kre@^O~8`5DF)1g~og>HYblq6+tW=QdrjDuTl zk+;Ha7O%$Bg6Of85P<&u479eFafp) z3z2$L<$v-gcio?V7gCm07M8z&^-hiS32sy8y$b!<^Imn2I@~G%_fkox4HWc}T(E@( zX5q4)c$7h_NN|Xech^+Ptf^Jt-!~#3cWS<`dAGOKfFPk8Q)ESLr&PRwDF%WBQ0ZE6 zeu-GpOCFEc)g9(7k0ts?YG>%Q(S)|IHiI80ze3iHA1Kbc7MYf6M)uOzkN1{VOVm$3 z(#{%6?I8g_x?wa|kf{_HfCx_NIBiC^`ny}q>g0Xy_EIUK^m|$mr9vRB${cuHCNXkn zp78Y74stzaZIv5BzyHzH@BAA0O}S*1LNXI=DC8XTW-4AZ2%UQ!#zM z7eE*|2hMwI5O@g`A(&_(dHj-JtR;aFu?ac)Dqzan0>q0l=LPB3F9V!X-Y{(Rf~U{t z`(rxtkABr>;NJU|Jq7DUm0AT9Y$nJGWZl+(t$K9LHLd1z?`Drk@~A`~IA*1l96p_) zd;C5gKFg+noX(S)x17@j>xIk%S4`VPJcg}=?AZa1@GJls1p{ySsdsKNv*efn~8YXZxzY))W_GznbqdY@le3+K-q{-Qsnwcem# zTrKJAx*j3?B}(k4Rew?bWTuQwGJ?*>xDH;KMPFO}EmP(t(fC3`cg=OdItS@TD8KJ3 z&kQWE1_)GXN=8^tV#6j;r1nkz0(~Us5s1}<)8yr31bYQm?&21xv?rQd+p!Wfl#`r; z`_!B$TgX~0IU*TO1dj>JKs`}%j^YAfDuQDA)A=|Y-aR^fGtqXyV^NAKSpO{V4BMqy z|9Ya22(y5oRt{)+R!l@*KcZ-@aY&TOhL7LETY5{E_J%DCXMR-f;Eu$lU zUZT08lC++NVg*rGTMtNaW=IdcXQ6OpaXx?MsJmE&NKOd_K-G3KTn~b@DxKZDSab=U4Odu<@ssUZTv!HspdkN#%%Z z_4bOyp@C|Wuu}H6*|q{^wB&cO;%`sMD-zWAm0r@0ob5$@&v5hu4hN5(C)5dm{~H>I z+sI8_DYzZE{gx}-n-v~i0JpaT5Sm`GGQ3$^mTd3(Hge-S!DBtS2rU3u$;G2Js7`Mc zI_4hd(9p{i$4750rTx0bpy*dAiCU389ByV;pfW4eulx`j|I*B$v#2~ME=$#5oQ~wW zQEi4(Ly|guWPzk>#5P8Aw!kwm8eyrPo2Z@s147P*(5cgI9(W40H#$%wu2a&mS@CqN zJ3-IpRhI9nH01p{+OovY@zdG~_QRtm#&{?XC==<~#U{-p7}c5wOH1?50nS6DQ%fzK zo@)MENBtFx$!egP-)K0WB^y*q2p(+O)K@KQ_sHrve_LRPO&U;L!Nsc=#mWCJN7(QU zlTL;fm>P%&xa$+J%s+DuR3^59N6%f(z?aj@^a!qs&pk{72Q3GU_c?Cn z^h*uta(!C5@S5*E02utI<^Z_1|C@jPUqYF?iY@+s(EI?dSL1BW7Uz=(P?o;#GfB(45A-n zdha=zl>uqi{n}phQE~sryCQ~VGx7@|+~vr^LJiet&ho=A`h7~l$w&6zwtsx;PkH=s zu-oyeV4QFF&vy;}rLVcVastqT91-wkn(AnE_dNb9h+g;U4THr7CDimX8_<`D7uJp& zbY2H~B3#Du%z+?#nYgHc96KY! z(*@ggB2kGAhh$@_KrK^Q8r-}c9lg((J=KPrM3vj^zFh7gFH{x!lT01BGR7 zh$uC&_KEbaVpS7O){m*P9CFl;87RGaghrljcm{X9&^h^neEKG4%fRyOX{?Q*+%1b# zu@Y=gjx>*S+Fy=NaRSC^%_Na_c`I(g;XROLY|oQgCS}3o&fvrj+r`g`NpaKlEv)L? zFLKHICvz_1`1LUv3$gZTGbkVZ*bz%#Sw|IQ-(-%N<_o17wZzNjL+>-4Siy=TpPdIQ+>WEyjkxOS)<8|{g@#SHy*aEYa zeed$+i^kGr;G0zIZ->)sS<})f@Ej~AR58W$MrB0)nXx^T++pcQ4O$d}B(h?k3JH!( z`%JlQJ`!s(X`Mmb{w(w8#?JYvfbvq6=l#;!Vrd2xcO*o*+~FDSWL3c8;5M(Rc(@Bo z30bU91!enFy3{3Gl}V8dgqD$?diZn&0EG5~O_xu4)?j=}Jy@6fkL84p1|~v2nEymV z#_GqsUQI>Em1>Y2ht8ojrXSyCURuD%ZpaQ?PV)jz$2pr5Gc-jyTLf|A1Z|L+N*K ze`C*?bO`!;u5iKj)4O;w9riy7zp5~laDj}@u&_joe91eOvhjgAa*)Od6Db z$52KJp5L+|E0T#FzuDd*)O6qW&Auf(iNHbtvcKHVDtM^{?vWX#8ck=LvAYmh?v0Ki zi0XMB&eES!YM*Uj&>Oc@PElO=dA?@su|uLphqo(gNQ%Ob7CYzev*at;M~ZJ2QW|yq zJbu_X#GTK~mJ#2zVcFJ;b=G*vk_B)Nm#j_e4u_dw_|9byNTSH{kx&uaGszgz-Fsyz zM|GRxgV`~s45$ic9@fIDZc^*acf{5pwkPe@6vI6~lRWJT4j3Tygd;RcZgN4d^WJI> zt(?jUxL6wXe;qz4 zJxwwV-vDIrdn%$oulzL1Gae=9nouquG8a>ZzDhatMkUbb*6PnT9yB7|h4I5de>%LW z%3Joc1lXK)8$HB8cVL$(GiqG;0iNOV*J#KX~bfwzP~()JYJ=!O@4)CSf*&t(77$24pnZQgOZ#8Mr6zK_r`jiO`Mc+)DGij2dA8L(zY6|8b8Z z+k3l%0MSllHvrV|f!&&~>S(2I9fqLlE@N8HH|j8fx| zKdK(|lPZvB4F3>M23b2kWq%rNYw0%ambL1GRIxBSI~Y#g*V>=^L0!0>oEVASml!H; zDw@-_kj!z>TH7bFwutR6Oe&2`C_jHwMQO=R&rGG>F(sr&HYeejvX}O|*t5yHGJcpp z9~|FN{Nyvdt%xU4!Z$Xuz4J@%#@jq5^^y%#!J}Z%r|_q@@FUm;CRp8)+e+$uqrW@W zPfpI5+LB`DJH8)pW<_CnRmpj16vzWe6t&4=ME4D0$>YX0TQMD*=*YU$2_sUVs#5?7 zeSRpCP|q3QT#TeUrt9V_2L0yd7&c7jqXW=>$^>$ER@y7tFT?`&zTQ^83B~oBjvxb% zp84Wdk~Z+i^SVs7`s7fb#{zWG8`77D(&K&rO(4t5SXYt{jcG6xhh(7GOseBfRZRzzkuh8(yX6=et|Dn?7m)1(O_I5!Z1lq@OjDz_wnE zseT#N3317=6xDy(0s|MOmuc`A-c+=?zRANwZ@k%3YT{U~D^fc#-N5J(#X03ZeB7d( ztynv(hkohyV6U5dM9)18G2^1K(6z61mi?|EfSik07iyx5&U1B6G(YT)!XCH zT+8g=DTJefTI*jBY<1cA5+aEiVI@29p{7ndF$(y^GIdD4t_lL0g4f zkX@m!TFJTODVV+E=TA{^W0Be>tJj9>1S?@p&lpcMkYpJ++zaBST@n+hzA)GlPk@qh zydKWF{-?I%)CfiYX>~7VfLJF{Z9>JYdArKj2Oee_zf}Wd+9R_P_Gg^yYV!T%Iw6+A z@#{|I$3Zv5E%md06Erm=Y7(l)d*v$+hA+z4! z^41qp{vH3X0d&9u91;0O&;F zu}L)Y%wYE6?=&%zF^ZvdVb@lHErpnl$Frr`IWQ*J{7IMax^DYvg0fa0FY-#Tv6~a- z`YL?;{gakuxeeEm>}_$59S)-m_7co;)7bqn6F$L|91Q;h-J~{=60nGpmF7`fWe-5j zd}fh%8(zvoKX?oQ0^z5jCwKtF2#`AaE`%pdL0Z1oNQk%%)o~$cd=sMMlyNWB!4o8_G~5R2@?bG&OAnqV!}ye{!yxuK$FUnxkmPO zi>fqbuL-hCh@Mdn}jYWhPoHajzGu2zdWTIfwJ8jDnsR59;7W2!8BpykN+ zwdkL4L*)L;=C>$JU^}2x2+;%u1iYU5*#$EY{Z+E4J{jw6QCb&Z4{I ztP@|B6ertx(`4v}5;h&)aSJSFc+e3n-n8Tz4RB2f7}ijZMSl0AI9U%q%Fq$?TejspVyS>Y3bwC*z^9I=(r1VvpPe_?XNN%}PY7T!xPTzM8T|zDG6b!|}#s&_gRx z_s+zXt{JrTsByYf@ex5e+P;KV+TJ;1v~ZMrN*4QaLv}_DaMo(u!xrOL9O`J7>HHkU z#p_&~GCdqCS!+m&HhE8<=SUdfOhSSqL8*?L+!<2M{gir;8|3!gM$(!M#e+fxkAT$2 zmK@b=ebY&djC#mUo?lklq@=n~5|4Pm>$rW3*c~AiHV%3nppz06uD6i(Lfzi-VXpwV zaFSQBMhk`=2XLH5HsUIbW$99zjcFRo3AT-89Fs{b!ye$|tjFn4X}gJojhfFL%uFQg zXXmn;f%q*`uJ@#7asJDqq1(M+4UX>3jU19cTd$bIh8JA@N{8)-0{Jsd@A}-}8d*BQ z6TsWM_KQKNwb>sN-f8R>-JCDeFYD95l5}l`ZXzjq?qI9bo#98@H766|tDyicaqpV? z&&S^~9;83;&(7md(L{MxvuDc6@YH?y+Af^_kJ`@FIEL>aHp=E-_gGt13wrJUX z-1m*8$u-X7U!3Ky#=Ex)#8&^zOo4m38T~ObZqagrWprS-iY&6I$WY*p zddLrt`y;c=eRhDXgB5ss`m^$H#hXi=TguJ~bho$9EGHNPAnAgN4F&>x#FJAC@wYy1 z+-=G4a#1vsIn>OOuQ>q{+}BSTjsSuY(V&yA;E{#z;(qjcN{#Ry3-LdyMB+e?W=>X5 zC4t=Rl0&!PrUr5iTR&FF;Kr@0Fx#22BGDm7mVy-F-1ScbUU5Iz_(Xgo_)(wzYr800s`hJ1lO)J2Y)03ma(tH$9N|PR?;$fZZ^;hKK>Yug+dl{s@a!VboSJ5%K82WVX0`K|e z5Xr7Ma0V8{hXy4O^{A z3Cx4yNlTF{rfIONkg_@?P2JO`nl&@+0H{fd1|`m&Pq{-rqgdACdOhBz-DgnH87=9$LL1fz6$oGNk_+u&}T*jZI;?59-D>2aN+g(8fy2gtI!`z!p!^8 zqNI6x_A>4$l&C@174byJmhKJ?6|@S+``{^vQ{A^62Ks(J(i(nT zrSZ`}-JZ1x?~ik1ZQ7xWMmlW&G_DI{h~Kg(BR*-qD){a=DbCh({06tGwToL0?AtWY zTDuT7ef#1r8SQ<(F{x)fQy73dD^RFw_Zd@OP!l^IlJh<3J#?S9Slf6{Awy+4DD4%A zz>jphUs@bYDPZ>cbvHivF8p`i+mEI5`6TnHv|xaKm-YFm?@zJ(IcYfCDay7YhZ`w@ ze<*oy0LT1Vf@Bi6`NO)Lq4LBlR+Z&HQt+K3SFF8s&(o=oIHvd%uy`7Hb){m1EP%M%p#56N?fRHPOh z9r64(R{LEX``_{VXZ;%lV_W1Jt5X6ocYP&t&EnYF$^@wLSRsUIr-9$&P!3N`D0pI7 zELVDGia`f?x>p5U65g(N4w>8q9S=r6cYa@29hVi##Nym;d9W~&-%D{2zAYtZ6!Laz z;jT^nV?h-y=SYqaQmm*7YqqE9O3Y2>M3T(HlD+X}kPsEC2kZ4tiR{dJY)(eJK+SV( z-ki(xIH7w9II+cs*-%?r|49(UeM6p8Ur8jJJiH)RCIIMs8ZM;FluD=% zsx3H;d7el{ZGhUYbqoc=UXsg{xBtxSc@S+=mV%{Ksi}2F z#vv?vty9LQAz99)hC4rv5@a6wHJO-~=V$4zLx0M&1l)tl9zQ(sRcEK{6?uEl$BEiia^R#n9Kt%hye)2lKU;kZFo&{x zX1Xzp8=sEKt8Z3sXOQ$-=%4>e5DnPCsbId2r*Q}9u0JHNVVi3)0o1aQCV9diwu(+d zL1Dkzs79~Id8>o>F!?}jZOaY@lH%~GJaK7GLIz>J+%ft%eKA6qfXq!t=Aun>JA7CD zxV<%?ee=#nMUp7+m3=5vPt^xxNd7mY>ImEsMR~pRO^H!rHSwtB8#C2 zK~A|#9iVNeDlb_{fn~hLms~liuinosGC8_NM=YOZq>g-IYbxMgFjUjpv=HdY{N8!f zpIfuC_8Jh>f@eqkEnQPTBR{bF5mUqvK1{6!xjqsxuCc3G4yznbgnHP~-vY(w7=%{? zGE<4AU)(v1Nd(#w)K|5~3iP?RUJuX6>nrzr^q%J3!CK?Xrp#MF<-u4F_ax}l!=#szhJG2h{n6?g8R^a4e;ig8AloZVxSi)|@ zskSk{g;LJ|IrX)aOZRoAYN7NlVwf@`ZAI>m+Lu((jGGw2OR&9Xy%pq#B6$o(%MURW zOJ*==bNo?Ed1^{fpUY)7uPS_J=jmdO3?sISpP%T4Gp?jlkSm`$v2C-Oa%q zNmJ)a3^q=NfZn?fba#LW=DbYey+I)Pn#EF?KxL#38foHxBq4b|gw6M-i}$Za(fPo~ zJ}R5@75d8(v||7YpslC|n;!4sH)po#eEvomjmigs__hGw{>95o7-)_JP-tL@eyUq; zlntbh9|0oq@8bRm-<|C?xE{uMgH?YKal>9_IwT%W$r0%Pn#eC<9pug_U@f-?H!8X* zmEM}pfBU3J&9+%=AK(~5w78f!Jvwv@F9c2Z3g@ePm&G>DKd<-!_T4~zlCk^g*qXck z9Nh79$2c@8|5ew=2|e7y?3l=`8}bB6_f+HRjtFc&ACJGteDi3a&2NxI+6ye!@%BVWZ&8al=%VZGKbeBZI` zf}|~#RzKd3m$A|1@b_Yb8MDC7Hrq5Lo4~szy6X$vvw|Gw{E<546Kh}5J1CWf{1VJq z&DE-1kkLS~J^JT1N2?(CM7;e_9P^!22=5s?=ak9_TCAU45yTHaW0}VW?f1ido{WZOheMmm}{| z)0f$@x_!7!2k0SL3qeGpDL&{>a2QPy>nF-QBKF0`jx#Hnk-xV)vybVRFk^zbJU^7k z<5Yn*nNkFup05H8R$vc ze-L#tRU8H);$nt*fD02-4e^U(hB&8TBEab6DW#_H8k^duScF}%_Q(-Q#}mgFq3Kop zQhNt$DGV#sp5poZ;UMw~&44`q5E0}R-*}oT?=dV%3T~6R(jIP`Gv*kzk_*T$Lyx_n zNrE$VkGgc2HowRd~J9J+ic!WKjtg+cR#dcnc5o-mcUN-l8 z*|Of~02!xWCqUc>IpXp76-JTYuM}K;y9CfO@`x=MiCc>$G=G)Zg9fy6GG%I>Y0H3+ z!4gG6h2nF#P2K`yBWVG7W$ZLopq67wy&~WB_}Gn)6gAblb!MUVb7%WihREzzGMa;? zH=3{v^DD*Q)Z7iEqqrLe;>BKHWAxM3C$P~}1wA?QW1IDj+Z#%`Ti{0j(?aW}WvcLx)?*}J z%_S=NMZ!s0w@w4oKm}*A)15_MnrBS__UTIf{a&Nc$n@kXQ}G~I97uK0ApK1WZ@%#| zGnPx;eD-z2M|lE6TGxTDgN@u3`!{g|Yq_o?V5`q<@AvClw@xjJY_CUO7oSFPz~j}) z4|w}yGk}36M**++VDrgeZ{>Xa4rKWr!oaV@hE5+>n$se z_^0P{Xgpm2(Ar&-=>+J6^C=LhPnpVHxN@9-ls>t^@*#=WIOi(USpYqSL%|hoCyU|A zB>B7~Ea%^(&tKX#OeZH?zxEfcxmd9 zkL0UgNfb%;OeKh%#=7#=bfK6r`8&g2AOf&{PF#6s-W2P7q_t_%e3zqD8!SQdnQcm#~v7n2y_Dk^$F%P?qXV85L=!?re;>%Zx*$yLvF}AG$o4<_Maqw`GalL*8s$=qa`LHu`koM)f=L~*#mQ&6% zmk~3FQo5=ZaB7NJbVc}|*i0tqnxqilnDlBbk_ zj-;2CGWOlb$02Xi`@g+dSvpyYMv?|60V%Y)z~XCC3V=G6yx6aIH#mXi^y7B|Cc!{q zCvU;Q)EbGM`q{BVF8l!-{FpYAl72zc|C+DHtHCsFvDYrbgB_=G5S@9mceuT_d_pCX zC>n+NPG_T{D$T8CKm!o-*A3#=#_`q)nv`qAU81mi6ubi!50MXp3;x){g&(%V)idl~ zUi(Wgx`R|*gptRvzI3y`H;!%X0rz(~%dSZ3Bw}XCtYXLWI6L@(Nw zqQnofEIjyi2sFfZYR`Y(C1uT1OQXpk+B^PoLdkUPO@sm?kZa1hB2;T$`DyHcEAjja z+Z`fvwaq{v2kd{xKEs^DFXi=v(jwx~F0fg9B^3MbKF}57Jm=N__n`k%JAli{{txb& z;fibg;Od4#0Ij3{`7qh+tA~96yL$NNps=g)n(SA{|JOT$Ew}39chjfiqgq`ZPrI5t zHcs}pMrPz36VR8_g7%~OUJix618&Yz$JBom_v&8D3V)YNDwszPOc+b1fw!XT|c zay+)V#jpMNWbw%VvO#((wta5G?M&LZk=0l-Kx<_`&f0M(dUiznEWp!IZKKi!yYcDn z)-8X&ISUD>>de{!BocyOb;S8O`&Y8e2ck&_wWC9isT-sZz|)^imp*46&-|!$44!7J zH|UtN3}XoS}~VjRP~j@fB$#VI9&=#JK~YDuc%)hizOFtKC`qTS1sy>IvOUzN2Z$ zXyLW2gp;$*V{$|j9&K`tcgmQ3jg``9f&_@|tnMI5rEg0&l`-O%{(-x8(&gzynE<xlD+u}Z{jNHOz%3$%U0Ra3-3kW zOTOE{$9FFAU3t_6eHc@cse6B+!}^l|*waE{3RVjbfow0t`R<(EA_j3=i!LZ#omrXTISy z;rhO5*gDr(6#30JQZFWa%?H7keC8Z%;kQ@=ojG?uINC9k)hZpq>dKn=$Q>@M86G-@ z>v|G3UPeCi)71$l8|FWQ&81cB$fM&Zf7e?L7&X!^0@XoM$>2 zOdtynPA1-6>XO%W4z`kqN4PpApD zG$YY4jcS=2!R1Qz>5E+c@7c+y!Jh)ZnwW~st!lV+e3nF+8pGL3u>hOEi5;+ACS3uG zKjy1Lel@l=wYAu>07{MFZv1CiT8&Pl_3q>Je#G;Up>8U|LgH{>+0_(bdiY1Ds2QG( zXx=}+D?9jOXhMewoQICOu|=546S{99Az4o)4$c5Jwy zHt>eRC6-X*6#Y1yF*PT}*rQ!h7<5-0{&qO*{n?WoAtFK^(>gcxd6-k!z4kFNcx7U? ztl3Q)qt^v`Fz$;NOuUO)-@Pu|=VuUPzkaHBN<}k7tvPU)-B-_;_ZzY~m$)+jG;);W zx_Fu)0<+R(*_&MpaW0BBE}a95%GobNJ8&w#x_=@ws-Fb+5d<70 zopsiyqWKot7Pp<9Jznc@hM(YO&sY20WNp6(n6n(1@>L*p8Fru|cBSXo_RWpACb_M& zi&87!Y+t6oq%c)@(5(!q;(;GYDm+wk34u6#GnX=Iz;EWd=vkHa$^GauE4G=h%U<== zpRE+^)+v1}Z!7n+Q$a%ZOG1UzDMZd}PMz4P&rD<=nRa*-S~8fZOKdOaoE}ZpK&DAa zpZl;R$>mI|qHeAGdwt|Lg9d#7toj@jC3T;F__=cBNVuo`>4})CDMi$)&m>WHhL}3q+byD!XC{Z3s{i{hCud~zH^7BtFqfFDMrYjKaN`ek z39oYt*>o?;>ADHIsDFI8-`>!F?allT-JJi;I~HsxVY9Qdkm-6ypL(a+)aK^qkwhND z0S3}(wtB6qLG!c8+bx$;z$AO}2)2LE$ltL;KID z+}Yc-w6sMvHCh-#3v<6tPwJiD$5F$8PtDNK@N{>@{?pu@)ld8O2S?#MB2xhmT$nLI z(p9taEkH%}jjFsEjVlEgPnxhZxp!{z&%i2L1=l+kKJ<55W01B-z=KC@ut$(31_j-6 z5e^I8p_q2b)70C(;UawSXz=%yXkmkcgE)`xvpsC}rViaP@@7@Uz=n;$8Q?p1Ie@xg zqN%B=BWzjzOuO<=ssMu)wJgK762P!4FO@ra``R`Bf?WQgP7+vdbjQyq=Ex3n`jD1k zxdp{z;NpP8&Hmfl4v}I0FaH8AHj#_jC2Fy!rghRNj!vAt{-mw%%2RL5w z2sfpGaYtH1u))5PZlLP^$uY_^e{Fu35`#rv4tCXUFO{GF1l$=lVd>VUgSK{FM~y*A zK*;T00~FK6qD6gZpI~a?72Pp0G2#A=P&n@fu>^m>3YV#p0)2pcsx^b?aGHg0pe7EuZ#a`zQDXVHvL-*P>q7@G!@$E|g%L9KPo*SBP0Rn6;C^_Ji zhjd7C6SUM9xE~o)k0d}HRDtC^G^{*cefQosCIaS(MX632H|&%$Xh?r7aVV|&iLpDn z3xl!W;gE9OWor=H79c?G_^8f!Eu_(Iq@D@oOH0pF7kWugrME*nK6yY4k;iOENU?;q zwy|zhk0d02@54P_MN>ehbrk3kn?>ilp6!0R0bi?Fy4~d8&Q;#X6g-7$zrc=-j67u^ zuqaSV6S4*EZhJ5hL`xj>2sqER_JM}4LV^b{zqcUIBiT3m*y}c^6|xpNF0qoFy>3zz zv9|-@VDzu zH{Y+p=KRwlhd)rAX{Nkox4F;pq5s)-jRuvllhL@{B9PNf8K%$8|N0YkIHE z6wo-%BW%^0kKU`{>h~otXEi}x_-}yMt~*Ot61h;D=EsaF>vMb;I#T(|j2o z6Z-$&+=vp}N<}!t*_q<^E3YDlHAs)do2~)Tj3AQ3Xu7eClc3)`VXsPe>L6?>Dre76 zw{-BheIt7ixR@DyA-$!c;?|Jfk5r^6clTl@3GA8mKu;C<+P&TNK$pTb#$cM*O9_|7 z?-&@LKVz9mTQettbIVS>fZOW8`5HXax0sV4_&}^~h$CGppcx49fHl32y|ukdd0~81 zc;%&{Mc_$^hqmU)uZ{_fX@fu>{cNQat*1-CxEmokZ2q(}3NY=szThHHtiZSMezBv* zl4x?ScR)Z+gOPF(u}wdtSOWUc$yr(TbtC%98!ay$Ln1NlnH%k#s8CIF@Bw%m!-xnz$>5rBYM=)y~)fv;y#FiLE*5CddJbkzI@ zf;<^XuR*m!Vp_ z-$h_$^r~RuaJyHNlEOW-XBv8m9|bUQ<)HzeeJKHHZykbg+GWM6-OUlnW7PP?xPp>_9cH&f32;pjj*E! z>NUxc9lN``=l48XC-55npApa%1sP{bQbyWoo-5<>GZU~FRqbhMX|OqdR+;7#`qkfN z05XNvW+R>EsMmF9_-)syU`Joq{Zs?S z-Bk8-U`~#lq1&_O6y#h?0I*y#Tlhsx@LcknX+P4DpyQwpb zox5D(G)3ObsO$Arit9W;!K^I$%ZX}EWqCR870MZcJ6QSW5N`eI#|>BaHpv^>;Y!(+ zuO3cZY1-s!<%`Fnc7T4}?~ET)aT}DL{5oA+VhW8a?DoCj_hm|GvoX$Rag({VtO8+|$Tx$mz zm(KD^f0BtWNALovhC~cq0A^UiZEdJ*06^&%BceEVZT>x($Eh$FhpgHYf?k~0zWMrL z#%S1N_|3}U-C(!%bvQX}<&eMv5F)&{WZCT~|LyC4JH&(%#m z8Xawq>)Qd%M|XwgJ~IU}w=*XD*;r=DthZyDdj8gCxR=|xua;&y^g4{1-62umPLjit z^`;_x0PuL^-;dRG-z+SOvze2M%YZADi+n}ueDB+JdIRb70%{LX*Y|r0G30CHU1?#$ z6uN(f3R7XJ4krwqSK=6eJNeHUq~yfOjLEW;D9f~Onz`_rUA?cje!E(s9g82Hxt!Z^ z>&xc{Vulrb#`#$jpDhtm|9ks5Swpf6vO%&PGD01Hp@p|Z)cCR7N}3?eB9~Z;)~|&V z$s%2X0PSQ^=$|8=eAH;l3PXfVgsp}hao%271OBJ@xrd%`6R|~xx(fb&QC*bJPC3L% zM2u=1D+d%m>}f@IT$t&pwkrL5j;y)(G<$&;l!upGbY@css6IAmnV2#Dk?l7~DJB!lFvfCPy{&Pg%? zk|p|$``LTz^M336J3r2kcddICYwk7OU0u~(ebrUfM5?PQJiwvEK|w)z@Jtb?iGqUW zje>$2hJ}iJllrxy8~NuRR8v6~rDB+B2l)lvRt7ACf>Irid-ME0@;kPxq5%{I1;6Xp z=U%VNdn*)_m8fSx8EtRVZ@005kEeZ#2o5{OoLYRxJoQXmG>dsS>+GoObPVebqFbFT zHyeuI9r%*!sB$(r+q1l&<_ps5mPccT;IOz8M%Gav@JZ-}xMwu1W|F{jwcw}^>|sQp z*9=UuuixikMZRpm5D^GY|FjazBu)53_4LKZ)xmUVg#g`?g)K?jyYVxJ^pk5D%$I`Y<1*RDby3cDtFiWoq&Ra?OQE{6kQ{(;!$h z%fpkilZMC6yGQdHyeyUiT2MoqU~)Sy+*gRJ#sg%?#W8{es}$q#tqJy!kwLqsyA@m}=ex0eQizs&?OKy@GnOE@1qRG7){Wjta`B3gY$W#_^>TzR`9 zitkCe_48`Ydc@J4Y6NkT(k3|T`cK=SAysoQ{PK*}I9x__|NBuk-nUU*c`K{AD3Nj& zJZqOHdyjTsBn{-;HE>?Jhk&Yk)3>qZbo{28R}fFv43bV-erkI(Z!OvioK$&3Cn>orR06GPZ0)P)AmjCCOA3sCj ziUWW8&f(A7l2{rIxo<%;&*O&?H)8_R?H6-kyfq%KCmlZutM>#dj7LTSSp_5$Tat~M zguTR0sb04fYVAe`oX?*~wY~K7@!9AiC~&uHoboblej(_G`9SJ+ce^4O-! zLiN{n@8<-$Ul)GtJ+0(c{2Xv|u$Cjq;xYBJo;LIDs-LUGii=w<-|Z90^wg7<$8KK7 z@PGGq2JGW5h$PxrmJAo9MUWtu|6Nw?=u?>8=GBhOjB83ZeLm*YSw8K})U?47C;7f1 zp%~~SCqA0$XYbB^6LV&GQ)BJ%m%cSU-%5##T`=2Mzj2Wwumu62_Nx>>t64|n+ZI#6 zqq~VmZ9Wc7*BkdUuS*GFbD}wI0zdA!zg-@1;(Dd%eV=otR6XL}{~2)hCO zE1W}NK(H2X1H1n%es11-)||F%SO;2uhb`Fq>h9yvLKdmY`A=Cd47V6)&SmxNdfl7# zrhg~Jq)r+%8O=i06!PWaTwaQ@3W}R5t5?tZse|oRVRraIMjxKCF&E|8W-mwBw{H&iV--ZS|j}OHi!}NZBht@QnHmF5zGGt;809QRfWAr$4Z|IUw zl?0MqKez;y8)?yr`XJ zkB^ikjOO#=PlSb-0agd>vgMNGzgY7cIR;c`~e**<-ef#v1G=gc-Z$wW#@ z(|H4q^1nyX`;?Cypc>a=PqEsahu|3|GLbQ~4d3UCH{(ajc-OKF9uKv0);M9JlFcdB z^K~aO&?B>a4bgi#oG@W8&pUkiv7@zXTV@)A#=K6_D4X7K{x||1ZX=iifT-iN4e)Vy z%KvJ)0^WwzC|j10c!incVf|XyFZ#RpKc4Fzu|9Q>PL$26<~|LgR)F7$xs&F7epa2` zVR+bXus;a8KTz+0Xw7Uw+v~*o5J$HgSV{=Cl5h7_1)hY`qp>vn+)l>qtOyj1(|-mW zXNMIipG2^1-sXbH$1jYp?pVh>$M3e5M_v{>w7aaZY!W|bqSk`xdIf&x`a=T2zaJ8uWs&)%&L~PNgI2hNPh%Niak35G7MRhX0 z@H?eCar|By_`~LIRw}e+I7@D;>S-VI<`K$;7LStTD`-VS4M7R%&|k(m3F39aa^9Wa z`@zjGxi4hVNUIfRKGSjJ^%n0-;2&%IDQfA%q0WN2)wnA8?RRaWh0yFbrd@qYu4%VM zZQCqK4_Pg6h)hc<1o-y<|LulKb?!oIT7@dkJE>I3?I%z4+#w^WY+#lL~3k0_AHG(#~#ert5lg;XvG3 zc;Kr3kDY<#w!+{#%=RbeF|Nh_-;Yv1M<({H_c@NM3dG@H7 zC&RJDLIN%;UKZ0@L#&gT;#Y5dg`w{?*s8Q2^-#LpXs>U zp&*}6-t#)0lTOnQ?w5EYWPIP@x#Uq<+r5{TM{d6tN0ky{qGnhbU9isBetN>jZsZKXNhPDRA1O0j z5GURoS%hQlvSct&20iEd11NqB`(-3}h40c}FbD6>wHmAO?ogsf&{iKb6TFH7h@Iub+eDxv&{v6%V}^f@Le+YR;2Wz{i8lZgI<-=gW`2Aw?uPhfnB zxZF(s*|h+bv~}Zd>-$5j+DroZEk@{{TNrhg!L z%KuY<;`Fid;h%j@r2iixng6>zK0#2?&`3fzRTvV4{jW~rC(ikw{?^B@AS(?8otPzE zg@r!l2_h2)Yj$1j{PE+*_0&1d|L=!f#QfL=A^Oe0ljDtnnwlDR=ii$Y8IVFvt_30V zo(@D5N*bhEcXc0U_}Y2>*NnJ9(tf$+p>xD_d}Pfw?1)qS!f~E6*&~64KSeSOdnF{z zJyriA#dQb{H-MyH?Igk0Pft$^)*c58{3)rq7Rh%HiCV?^Tp?BJMXpWQ28e`pFX))` zzeZ(Rp<8hu>b&P3h^XyKpW}aCR;8!O{QK-dnveA!8v_s#=iqBx(+b&LJ&u0FL(KWd zdPTt&#+D{J<87vquG9vc0iao_B7%Z~e0(xx|7EciCKONz+wM#7yx{)M7Kjm64`E;s z6%p~dyYaF3A43LREcp?@u}q9(Aq-?V%5BJX($|Mo{byAF=2hay?d|vY_Og_mQII)- zAoD^|v6I`2<-qIxhREdqvRNkjlFt@(1blcRMAUZ-;2$0**;CrG$kQ|L1&P=`$hw z($9kI*2TKw6N@KoUS920lKfBQwI206_LY1?2FA*VvJb&UU=P7d@_(wX0O=5X!vADJ z{-!hx*#5ACG%0dU6MT@0-k~!F*w?RL58JN~W@~IC-~JC>@Wx^yRa=`nHo;O{K9>CM z8DhO$?vLfFE`p>&Hp;x2{;P8&siE_6E$>gcl1XwQy2DWlf0{*G<`vxFJ{OJB|HB=c zzXL+=x|g^Qoc4y>uH!PvVU_S>9V&aKIoGCg%&-#SqU>;0rZ^hNe4P&Cy(qL#iSOhk zIMvS$PgpOU=08{;A|gL1RNTH2HF02QB&qfC7}i$*dYR~k<}|B?_(Yo2EBE)7%_YTh ztc4uRmDWr6dBWb-S?l@1F+R-p!9}2Judw3z2PMp(UvAd>vvO$Umbi@PYZm(Jh+b?H z>Ga#Hl{S02$!1mPghJ-bVmv25*Xn^{S^xGp(p^L|<_LGYfgANA^dUk^QOyGMeF|u&Nb~6O0q1D$c`B`+LjG;P8|O|6o>7E&>R(wq{_X-Ca^UiA?V8Q3}=e5)?9y;_?&8ar?Guw(5xEWnAM{lYPBaKj>$erNe12Evg z(l2S%u!f1n3!_1pIbwzZ+BEdC&L?uc_zme@M`AJqwKJXFGu(c0Lq><_mZw&uh4=cx z#Z0hk1hR-HYD%>Z4C5~JBHPD94{%u2Xb@+MezLn?+f$~*JDl`JSGOSUEubaZo9)P( zdrsl;4<6{Gy(@O825mvsBE!in({@Q+c( z2_fPLK-&x3CHfev7NOWIzxla2mBsGw&CGr_@;T%y+5Y~)_jt?O%5mhIk6JSNt@_$E>2+5&<~zF7Y#IXz{QJy}?*T_0)b zRDzc0?BwL)3f% zdSc1J!TbELQ@kVZt-M=b;}{*Y2Df^)7h$8W3ST!%%5BMolOno=z9vViZdm&{Vjsss zqF9m}b!XdafKav30_SW8pR<>WUfQOkDsfJ|EM;=x?mrt0NCsGH5eMRqBY=#Cc1RKl zIIAmHf~3$#-**CvtL=tq2yc4&b02flr9OXKFJb7|E9C4SBH0Y}0UM07-~Y>ndzzY9 zxGj@O{5=-|h2*o|m!^^Ka&=+`{n<1K0A@ngRl%?Zvy;Pgkb0I>la2yfbvlNz`@+k$ zZiM-Fh^c^1LUr|_9>rQ6d)lbGj0DCIQIhFfh!4aKZd zHh5snl{)Q*FXO+<0~Tju@^|%ZIl+CN-dYzO=W;``!SbD@oW#Wh?2Ite8h5{<3L|E( zTy|JW6)g63RB6Vd$;Io(p<;hK0&^%a#27L0wf}^*?wdigr6!Hent6cub>Lt|yL~`a z3uqh%CK~>ky2pIB*c!I^xo#w>Z^V5GHHnLLLrK|#+KzQooSby0P4`uVBVHFsB@nlH*lNDW7 z^3JaNI3UYjzn^-W2R8M<^qIOJ^e*&y`G1aLLzpyLRH4JR@kkfjhjdx}b<3r>a#=&? z#Sf@<_i-oFoUQ!7?PaP!kHZghi?=pEg;4s*zI`_a0%<*{f28_^y^)!-^g9eV?mAfT zLc7iB3~nH^tiukoIq0nK8t&H*BimxgDCwXv^a4{8cW;2k+nq8K)cq|DD9AL`nZ3-3 z5Vf9eIGR7S~3_cG1h9-VJM;AIHN2s z?JOk&bFOVgecv@0`;S;S2!Ji?K920wVuq?68^4DcbqU(!eO&Vt5-_yegxhmv?zHy& z#!0iES|11y%s1pp{UYW1H?XKWLZ<`+SWi(mTfUV^C^z7oGK~E44pO0WDi!nA6(I-x z+JNb2%Uh9*J8#RL$VphNWwgKg?}wL(!@=W9oWs&+(AOjmJ0<6p#+N~uR3 zU>Q5{nv)WKgB3G{l$L6nrm>Ke{gx=;I83|Nsq!g!`pa0TJxqCH4|H2zcU=9WB+J1Z zzm0M6;(G$4!uL~_MIB^Rai{kF!*Wxh-L9Qw@E}2pf%CP@N zwPjf!F`^64p7UJQLJ!*+dyUm32A>k(O3+l(=rO`Ke8u5{?vhDKm`b}08=s=j;Ty!K zn$e5isn1}F=obdIeTm%hKG{FuoaQi_9q&R(cZ~a+IZ2VdxWn6^*R2kR-eSY#5-++d zA_3n&BDzD=4pS;zL*bG3+2Un6N;TX(A|&28`IoJkPJZXB{pp*OTe>xRtgzSZz-1jK zm~V5(;bvck29G6UZ&EFgfnai@WJcu0jJ?D_2U4B zAXHRjpz`qVeY>KJ#Qmo|u$F2tf}&IQ^#|2f9%_q9H**PSKx1#Y7ZxY+WmZC%+?-T47|661|M# zDEa%{|NiWM74myE)Cvd1J{nrZZD_yjj8$mf{JEu+PXh-(wXR5gq)qyqTq z#e2Kyoqlxm>4FTI`A^0%63-EDb{VPX{@^qZU4gi@cb;&F3@RSX*L#?C-n?_hqh4%q z*h}FynpdsZ0&x^aT%=u+_%%36h}XijvbI#H7Hd4zT(v!!=j$Eqly$XNMuWuL+q08ukSrJYN7W;Cim;R6r00ilp5+@EtsvfFfPD@z^?iEXLcX1BjD ztMEk_XEyp?-0ieXbWra@gaRLh=AU zi-Sn(xRLd2@RJo|DFTnYZDTjr5jA9LrvwrLAqubrv>I^gW9s~7Va-?GFv&Xh? zv$7&7{AYl!kEl){ZV-2y+b8Pv7`MFkk3CT=b9isx-&m}y?p&L8lm{3WIX&N52;t-UNyfvyUN+rO^6#^laUsUBJM`+x)%%e2R$z>o;gj2>PZyiL3Ue z4e069#*XP{0P6Bwh_h?z@o~8l3Y-G#vlc#LumHoCfn$MUH!9wG!JGKIRSOFk;{1GOQG6$#3CuTbrsMKrtAWu@7KrKR`A zVuS}2nN-zrG0fZ~^*m9yOe`CyO~p&*bi|1@%{^6J6|X1Sd8J#t`Jf{*0^dDh7tv5l*9n1SiPvz>C(JCoEVbo%I~t!uEnMj4hMR46`h2-+2kTpjqWNm~ zy`AA%Y7Szb9H0E8rmsg$egvwA+W;~>X96jQ`t7?u>tfSY+plBgtn@G!8NmL~mLasr zw!v#*G`Z>SVB!3HFqC!|5E=VM3dNLI(u3`wd^*Is7&Xh|TYP+-eb zX#i0kFmp`)Vx-qNrPza{!j5+I_FbsUSt^ABB}f{&7nIM~DJ%d0h<1En=xM4ezx`+` z_`cm^WR--(^Aic{+&m#>@*<|_$*P*)MT&`&3>1B7tTHlJRL12xEDYae@dGC$nXQA1 z{Bbt28YS_Cr$@5oyXG+d?be3H=Bo<-tj#;kx}jwU5?1_g8q5cCUhBN@((}=#=9K1% ziu!kQCnZPc^;>CCeOFTouWJj7#E`U2zQJd1vH3Z*%}?n%3A*1zh-*2&ns1j!X0FnR z($=kPu|Ia(_qSmk&T+pnUkOJNUP3?TW4Y;q;OMPzyO=X{G?V^*$0w;2d%>5?ChU9l z5E})HPv#fqqNxLMy?OCyKjzb^!)njtWJF{8z27rtek^8yQ6oZ<8a}+P5ff#&{hDt4 zM1mX_y}385OW=53+L6wBJ!V4sk-H$?NV5GYh8R5fxK~(!3U>w0ZpoZVlj4hSn*{Mh z_X~RgGk#7-iFFp+XU8a!C+*mM=+YjeeZVrNL)S_2sGsNIcX2Z?3=!}{370EATtWNK?9#!&unz4DVSGZb? zDh_Y@;Cd1iED#Uos{AqO#GKW3vGOgEq(Lvm?Wk|?yqDgI!QP9y`FP>;eDrjT^!2I4 z2#+0NfoM3}s1bT=q3eFBQ@x8jGU6QgLrL7j*7`Xg3wN`RzxCz9(ZqRgWtNe?sY+1K zY4D*enNIrtNzqtC!d3Bt*LnX*)926vZ`<7pB&NME9PyKr7IK81< zX*PTN{o8>Y^b8!dVHQ86WR3DVeLH_*jWR+-QZzT~ zx!ye*a>7&S$z5Q=*|W`uS9~ZAcO%*&xdtYx;)7N2vK+%J?7%uLUNrLTB<~_uO!0cX zqH;t|aW7%ShzU|k(aiXKRjNgum2`f1htqO(USxWD9tmc>LB}Id*XKmGxisO(+s(=7 zsVRhS6O;S_$-8&&`rkp~nPENZ*TdW`RDv#I^d8#hHkjto6TNzX0tuse(NOd!=#8S0 z0v*M=iaKHH$*S6)?Lz~>RB-HQ^bNF5&{K=81o|L?901rPv1(n@I#{$r-f^no@g5iz zgzXI+Cyy3E&FoR+dL^CDcmRoHxgo;058z9sWoi>h1s4sCvbagheai07jIg0&Amhjo zHR_C;G45dd;bg-*_FTl-d7(_p21Ts=gRZM09e)}TLR;?xk2wo$Q}g!u`!Y4nS6YBn zw^U6LiCFoN!_nj#We)gkz(#^X(cwWYYS6T^^r1zeag^>7>ynKNC~SDrKz(e_rJzAd z+~rGw_qrn>`9ixA@`Qqb8Wuo!;(fDQ2oyLQEscgous_+@te{8VqY~{9SP-c}3(}dk z%H4pDdb=%jfm}xRF<1Vg8XK+<%oqpR;qxDXRPSdfHi$}NCLo% zpQK`)cxa=kp<-biA0$Mx*VtLCcoVIQ9rS<@T$UP{9OJ%`4PN46f&EC9#BiW_?mu)K zo1QQ$%`q0Po+C)1h&|9wzxOTO(vQi9B&cmzVz~O8e!Czo=4Oen$!2Y)y$!RQJ?n+! z^r%n3pegBo)}Dn*OdPB@t4cHGaIX>WM;Y6_SF1o;>ePOu1ij!0P2!evKX?PU3=Q!< zdtN~`#Bg>*3Kl`&1+U5Wlk}D`>68j_JjtkxvcIp16E(@i1EXQq`=q?l*{KfFwLYJ$ z@OPT`EP?E^$$l3663iB8bc99xp5R_Qq;n%lV3+$T>2x#1r4S;!Hu2pn09SckAeHi4 zG)Pv)5laz6DSET2OIqjhGXU^(seJuuY80cqT?V5kX7#*lgpKnDg`R!tjKfnFy|~RR zXr*b}PsGGxxXpniz<*js3oukH@?06PYIT|Z#D0N7qw7Xj36NKmsyZUd@}&>!%rdTt zAH)E&coXU|{&j~yBD<6BPD0H})j)15<^3W>K#4k_To}$)W(gbajJ>XvFcW|i0dZu^ z<)gR>hqU6=4g;p}#Z|V12eqc6O2_IA)cUE!l92;3mS|@ zY6yVXy}T|)euj1(3b?`|;p-48dM~X(L`|f2(<1wFIg+5FSbPGM0KN09_p=CKGhbs0 zt7+K4tkQWkAz)_IvruQ&R5w%M)XuO5u~E_W+yrT%W@UqEx6<*}jc9-7ooJBqF3x(c zC~iHA#PVx+t-F16VIf!z(!w;LzNb_jU}9iLvoJ|6TIDpYVa4K@xOmcgz8ULv(mtsV zC1!i}kq2(q%d7Kp{3(2gOw<29Vodr`So`GO2cn@8kXA46BGJ%2J)qX>G?@$URL zY3b)sWZBfubtVoK06`C{5fr^b@f}~j$nDdaWShT0XcsmWLAQY!S|~#TvH41tD%*&c z{`v6`4udmNa83PXR7B)PjjRM5`)L=se}I;rN0d$V)x&;7_pE?iq~)hBh66Ezj2%=@ zYo&-nGcX6yP%`2?OJ9LwwjG)pQS`&cl?_opDS{}4U&AmHKH4y@WW0+6KHE-KYL8(u z7V!K=H59`ncaAr}6FcE5{$%!T1SaIiuq z>z$o~$)u|fP*Y6fwm+ynAZJIKfs=qH(?^k?NRMoU@md{t;_G`P(l1$y^jM-W@`}Y5 z-#;kz4oWNrCAhCc7*>y`puJlo^PxzjMCeUvgbLL04Ro)^292vS?ZXHSqaK!bg@0fJVx zp%##*9ga4c%)e|I#M9t>AV`bIl$jav;&0UgL{Ijr0k%jKf_ux@HL>_Q($RBa`>?K5DP%6NNnKVl!ctj+C<)n++EaNx)dUkU^_^-`In=Eg zw)PY)T$b+P?`qo}G|y*UBH5M3*tm)+>~oPd)3(WqA0X^d1{B~=8J|hBP4OrwMis$Q zla`%x{FQn>9Cp11rfI%oHKv96fB%u(a=}60_S%NH2?ur9r1*qTk0%K9R2jF&>rR3tJ z;fUz+-TUm12g60Rj4*-`@$X3^WhYv_Jf##z0{nZ%{!9#3KVHFv124Yn(e90o4@xMr z6dP=%1qHBqm^DTjY=ur68W{RMfwLUaX8NP>lo@P&2A(uI3`8r(1>*L|5eUEyf~C-h z^w%*WSh6hxy7`K!I4OLLNu-damYsN~32p;BInRy< z5PwpZXz4wb@edPG+d>kk^QUFB$Rn(9VsKF9y&Z?27;9Ba5~L(laNo`3X{2VK$v$hC zeE8ceOsmdj{GnnyfEa=K7sO!q8{^N$r&?-3N5$n@dGwWH1UbMr6KRU(WQh_KitwTj z?X0h1{9PE;(t>Yzd|PTH^2lzvbZMm!A?xYmaP@4Lo}mwcVpd%J%Fm*bir4js z-JZCFJlhUGD^WO@^pjUhYiY1hO-Rqs*97QzPq1be+k1R4F|#x>y4FC**1e2po2%&B z_vL0#G1&z`#gr$eUht?;3FtjU;=#s65g)rw(>TcdQrDEG5+_NPlV-4x;Di1SNhHCK zFKu*z5(*XS(9}=I4bYZiepc664NOu4w+=j>O`3l_Y$yNrtZLG5U2%xZK21k??|Z;H zvtq6tw)s$jdcz@y0(jm^Cmr3UHo1Jkhy;hxLjok>JOlW}Ysg@Y@LhBZX3Y=|7?)bZ z8x2e62e(K0Vj}k@2BMxu*?fNQP4^G2^Kp~8DglmdpGH1`$O_W3(D5wu2?_KLnw9Ds z;JOCnM&(ZH1FUa{xLfdrX`X6pl(WG|pRdXMTeXmpfnEM%E%CP`12-$wE0(1&3(F>$bZ{1jYc7hAj)U<90mRn2L*c~2uV%izw!Bp<3!F1 z0z%#r(f#I8pZ=R&+LcDqqnLfI=)YUmG9VdD&U4XumEWZj9(2-(GBhjGKT4Gh6OaRt z`eiw>{9Bb1BubbInYzc%^Y4WWY1DGXSMb@Wwqc<^$SGMbg3hJH_89TPb#QRz+Qe=7zQb4jrsTOo1r z@jq8ErGE50Z9npbfe+d5-btyud*7V9<%*^7T?f`&l`Vflo*%Wyq$|tRvuH zp-HPq)x*;>;OF(S3&XG@&L0}03W|L=#HbM0dUHJR`AwJP3+Vw5tjB-oYe?3-3wVNS z_M`Lqu)~a!u`|=`X6+HDjD)Y%)-R%kDjXTu)L9!{t4AiXzHSlU(spCcv3%=fRA0+n zDsHgbV0{=#(Q7Brdv2j;8!&x?{mu2MgEgq4-WnO$uQuh1l7nfIZ#Rt0tUcPoaldED zko-_tQ=G3TZ@fz37Q3K1pmNi73fSHbYMWozGxDmhqaFC^yv=)lu^qD z81ucwtaqN>QGd=?=@d>46NEa(W5Id>$&gM*66f*E9c|2MGXNqEIiGrJFdkSX20u%f z!I?_CzU8%06!^eSDus%YQpug>mC+#%j9VKX>6w|<;Mb4_;o!%Ndk`Df$+Hz6=aH@L ztAyaXZeO|gw?I|Z|6#K*ebSSkKE8xmZ(%seB7id*3sw91Waa53gnexsnzqjHxR$y3 z3pzK5Z^l(<#HNri%mXfID4S;F<-CZ3NV@vwF$3Q|(go5~$6|QNW>Hs9`zSDi5epxX z{G{d(1Kf^o)GlSa*?|p~+!6V2y^$lc_(>juU5)(`S<|&hNV4+Sse+4TawDkAb=P34p5G;XHsH~6($#c zmez_t-DzeWPG-~qQO&Gs_8z%+vF*RnKpFo}kpQv29_nK{nEpqs{?z0K)RWPlx642M zo#0K@BMD5eLwmZPQEW|O$KfL~x4T~*W}hdL-%^XRFEuJ``ITTTWHUx0($gcY2z%4e z+TJN1cHbqE4^oSIO5@c_1H$PX=g+Ah!_v|zMCpY#GI&ITaa|U@+_WpcQUN_DR8_`( zK77FfD<1iMb05z0JQ}7Ck>(^DvH+6>PcW{&6R?1SgcRa7w>2ngmp$mb-TKmBlvq~! zsWmJ707=%Ra3Deya~3dXB~gPqZc{QO0WU>K!LkCdLedc0_c4(!qR;Fj}lDMiV-O`pb>kBV2o zqQSrtM}Bn#%cGt!|Na&hv~rcqZbv#&_l4%(Y&y%4E|d?X)cHD7z62~=5#r`iekAL(-t-YY%!OPir^}6eZhNF@@%}% zzg8%nB&$3PkcdA2H7{@Jf@PR0e;nT?4%5Hj!2YTQ#|p<>>t*@U`6k3rLs8ItdCKSEut{TU zf*=1IGoTjxJ|C0tdV>!JfHN@qNb&EC-+Q88_E*8^tO?|BAz(rK{Pk$8cck>*3x!ja z*!f)%MtlA@67Cf#8C(MH&O;fVkK;tO?B3WiEvNxL)11f%Qm{NBLm0C^1Phi>QBh{` zHX!Kx>63Grg^U112FX4$m`r?H_Byc;U-_SWFP*@oJS?zt#fII8R^F(KriW(nX`D;; zQ%wR!njVQFk^Ne^g4q5#aQ0;ZV>42Yj+TEt&Kv3dU-6?wIYXJgwof82309}hrgzrf z49f|$3ih?%{ITv8-=iraL^p7kBrR2d&t3agHuf8t3WjE4fIJERj_rtsAS$!nwDQ6^&G=2?EqY8DOIS4lcn~ck->f#Kx985FcgV zgQ(_KU+2>fZp596!u;mdus*FU$8kGO!hPSHHb{N zRo>`+P5AfJ+53xLJnMxHavHC`e_I!;LMITpB6)HNF!HLqPco%VbyRN>K0h?e@u4+A zGSA%SorIEm6&XT9LN{)RT(p_MGd~4{LlZiv-wfe#sdYLG^6Yb1J(j;@oSTccV5lWc z;M$gqhwRFDxr9BJwAv#CYpc!K#7g-A#|6$0jQ7SIjkbb1SyIV2XeC~FSF+VutQ*Tf zPP-H8C3uyr+nn~5K9)O4+99!fHs!hx{~M$BNvfZdku1LUr!&vnR+6-3Dz}G!?_0}X z`xZV#O-*h6NW^z5%L_H=qv1UtH^8(Z)yThB5|Lo&eZt+dn#2Zod;{FKN6 zbpU5Gi8tYI@KGj!1Rr3)eUjiemDoky6K-6VArE-IGamB~NIrP4N&Ij~>DiI+-|HON zslFQOV=fp3XA|QS`|}k2Wa`JifaT?^)E;(_^`DjEWm3QLl4mX?%Kx4Szd$1j5?*0! zqw5C!292;^(2v3|+3BCcN+f{8Kq5rPSE}Zuf50#bWPNZ(wcaetZ)yIwyZ_bQ9}6Yd z7qv%GuhENN?>6#4!0qc|-b`9e+JBPMH@L&#GmKyfQj>Zs5mG%dG4cmM4E|&)GteRj zdj;03BYnDuYUSyEk(BG-l&!+sVk^Oq1^N}~L^ID8D2lz=MkP#Tl%b@w2HAd2l==tF zd(g_17A3hm$-wwY98^#Zy;ycKasfjmP-hb=;^MZ=K}yW-bg%>u!t4cawHfZX$IP%D zAk$2N6RSXqN}L_>1`tX9s^F=zbDw)9Eohsu%}AI6GfUe?j>+`TiVM{GrfWoi0%9U> z$W`@ni^rfFqSbf~3m>S@$*{aWEAS_q%y3M%Z|OlBgbkZv;}X6(G;v;`Xk#$*d@;VA z>Yq5yqL_m?ai)0?UD?RqUqR2uP1ZLQF&J{I=1l1oRouv|gW=Jq&jTS4`L9~Bgor`w zq8_kiCU9mVW-5hb=$kncA;Q;`i9BUMC9Y_CKtRu)1;*(qks%9!76+jB^m2SPZ6{Ig zU27uFOPBJ@kQvHjBh&oDJ8kaIt>-=OYbLy33qfzptex|)4&noulriOcK0 zAOVJjVHZDI0t!RZr7u$9IG19FNv{L6{dsdAKZ-i;lugt+l)Fj+GO_b zCVm3FS3P&B^U6rYbFfvyHQx8V_E=Gzl_1^M9$`w~?1Cc_sh66LmFvSI4B-AYA&m;K9;+pD6; zBq%b`pTCU>yDs|y7uHp1bYXH@=eKOD$MLogCMc5De`e{K-z%;tN&20r(wHN-Ehz0J zj+{BKa8A-1bSJ0+WqpumW&Ot}=&dOro!|9`xk^4-W&XPK1)b~@d7L~(4W>&J>Ie*3 zb&c-l^1iY`X>3NTgsm^}?1K~4v|s)xHk83lG7*C-9YokX!^$?-SIFfQNe3;y{jLQX zyZbf*K1UB;w7~1pW>^E)K8)c@PUq+?Hl!q!%S55*D^NG95#|gUGXumPEfJRR-WS)x zeyP?M=)`-gypge4xTgN#{zG;`zT^CGzx71hXDiyyst(3=kCOf%OGYM{SM9TT zbL0RXK!Mz~i~!sIyicZQ$(`hfxW$#iaejh<4T?aaF#_VNx4Xg}7nF*dmJM;L54RJ3 z2>-BQz25Ll%+QEEQn<3NyiNHcka)-`bG?7~GN<%OSrve1(0rr{UGdCBPd_3aa`i&X ztJk~;=XKxyaaziyW2VxWJ=CKQqqtLeIAmmJoxv>hBY2{snW8irMa6X-M^Pt$#X zmCz#uNZD%!73@+Etr)g=E#@ep^^$So%olraC)sU)&-@Rfa(=A)Qd%y`DC*#A47d`& zqj|+d#PfGGv;q75(gB1~vCSo8+7El9ASKdzfFJ^&gc!%35F2EzU|KF5{W1RDx*lWV za2iK8PeneYgwik_>sU^BRV7q99)swq&NAoKnuc9Nv$3qzR9tIHG-NKZT3r?U4u5Ch z;rY8v`%u@TTX_ZB`w0}#l!zh`ioQyXI|b1LmGFR_RGUV9zjTVhx;UQc$la`5Mn6%9 zX#r?|^1E50h@A=f<(hACNpL0$a%%Sg_7A?F0>euNy?#a?+CbCSE-eI{UdO zy7Kjgi6em$qJn?w87lWFHwI&ufa9X*_1TMEGB7iH05ppFbsDwWDX>dsR=qZ^MZ`8s zS(^a+?HgTa?K~Ba&MIHWGthU4%8zI48N)H%vWu3*pCF2Alk7amM68l%6p)SjU3wET zp+bg@ftm+7eC4-oY@|@-$A!CGv-SpSWB3f!0OTgkouw@ypy<^GiT2v!sE_PC+z08Fgx@`;}4RU6#yR&TqYBlJ|CQm8h)TCe{B0+e!;lMgb4K8z@oLQ@)no!5-V3>uMFeS2 zEsS;X0k6JA#E2;BYXc^KILE8r>(T@T5P{!oJqMCRMg;K_f%WNox_WiiE3J@8I_*%B zmRlf+PDN)Ow7xLPh@b!&zx(d5O^a@G%&d73mzyCFYRL>m*DChM; zz8Pl|lV{aCGFs|;jF_;)p8JWDGQx4;B;&7h{G?;un?wu>(8+!mHGiGGK@2|SU?I9x z031mS-gL6=z_b$k=#Km=lOrH=Xc(et+p!R`h{7*yFH|kR;CZg9p|vkHwUZ^Wms#GKaagO&$p;W^2_{sB;sDAAh`HoakrDG{9AhXTFCZ$E=b$Oym5 z&=e)47?e=kujn#s)3d8?FtYF!$u(tuK>%Nqn<~wbYfdU{L6n#{L%0MOGicP@?|&zH z&j`b6!f%8ElEr~*ia!$Jm{(WUqHii8yIg|M?T3upSlR~4q9P11rC7dht}r7ER7qB% z&pOZLrfr|Ep5j-tAK3^Fq1n9*RR(qftgoUZN-+&$61ugg`U>UO-hO_Mny(AgDm+JT z)}C9Lkmf=L3SVkh%^jsToi@f8Z~YGTCg>JX2hFT$63J zF3HUqMcO{i1Ql2wyJ@)J6i&Sre)a8?P(Y^6H9;FBYF_B2-(Oe~Kny9bt+hCeU6L6oc;Vdiu%iYq-CtO?;J0qezf8m)2a?&gYep40*5l#Efx~y#dm)i^3;xUnR zFH~j|Mia6b=BUR{Ci$K&Wb;x-czPB0ugdsMH9Z2E{y08~YPq_CE5%Q|Jl8k-R0eI# z5J?pxjM~ATyMq@?@|V9<)u0nY*%qcr9JAhS8jR<_y{DBs7%6BCBnhwsqYbOYdObBA zoEF&+F3!w?v9mrTZ|Bw zh&LCn^<}xccqKY{EfrbdHA%YW3jC-65dd0?gDp&{c&4|vmuS{%E%!+#jwTj4;DGeqVo2SL#Q5#WyBP#3BJtH#LnA`P%p-6Mc5tZ}Ijs z{SQ(>Cw)|RhuZrl!xjBm%7g9;w?tu1n5DmQXR$-c>?UQqn?|aqn4m`#Gu+A+Pt@hIF5fJH@K+ zsfZRj(%9HoZb_zRR!ge~F8g1J^&AesDj$rJZ9)gol$ACg0A`r4G@RlO#!5|1#Sv=g zjpyjO*qyXs{0N?#0k&Jf#snxS(?C-epF4yo^kACBAZyqvORZZ^WWt~kwC`<_4mE#zJn*iYr#Wt!dG-0<@+h{*>V?P0UB5^TqS z6_pfTt*QgG#&D`Tv|eqp)f0nV?jK0#KZw<*5I_R2TZuut@C`Fd#YUFK-}5t}@+(?D zoHMz1wi^-zkqk$8aIm{OZxN3;<%LMHe+n7#;@b$k8lVKYh~XXL;(?mR)p)`PKLu6@ zvbe~)xTry zr~L;v5`UsTOZCt*?D6};@28c-dnk;-Nz5~R2nHXd#Q8m4Uv%E)>po>p5Y~GoI$Z3* zI%otJPXBCO6b4?HUM(Q zr>t8GeL5u|k!OJ%)Y5qMMGAjrm`)0)kel23`3q8(GS-QlEtW`}riDd29cRRu)1q5M zSmI}1^D~b#WkKXAn@#cgI>3NEzO4wb1y7qfb-l~I=}lGPZsiBJNHE8JvXmR6}|pU@Iirz;NMB z|C9`$`!7D>UbO&sLZLx820=$P_l>mc6~;#T({5yip}RC-Ziz_307F{wBUL(FPU^1Fp|wxgu{C=+^Wm=eAnrY>Pw!^iRZ!>qW?`5S zB`d}L;i1Mjnmg8>tmrNaoh3~qiu`P#9%$ST+pDt=&i~F-*U4$o2h`jIe?H`OQKv4< zu-al-$@sZkHZknJ=twp(Bkt2Fg^mKN3VF9?jpQ5QXTlpkf;^N|)wUv5=2Bu9L-`Fc zXO!OL&5xCtR0QhRK^9{4Lioqsv|sbZH`@rJz&jJd+)WQZE{z|oPlP_qM0*xF318oi z89x-C3W6S{_UY5lYTgOEfbUls+15G&?jM@~Qc~LLO1k%+dL~atzQyAh&key7Vc(i+ zzrFoAJ-HOhKdS|fOW6{|9DFOa9$>#mY`Ldy|$5GcWa$ybqAF)F* zXlQC*g|NrlFpU8_z~g>9nOy(-?A=r&-C*!Z&WD$^UjF;nA$`jpX&Yx5sh>}b>J$x| z@1~x3`+Pg^cD40vK#fP8eIpyvwR4PZ(0m5CG<4H#8Pbdr-hJlt|RTJK&JaimZdR|MA7|eE5kY@;xDB4Fd2s_lUhjQB5ml%qNA^1ycp0iz5*Xl>^ zn82C9f-~*RlE-nC;~NkU#zTjG`KN+6a1Cl4V9BP{0!NM;3)AQ`eUD>ux@y0VboqKy zL<|}MhHm)lk=w@Ks^f&Mq6z+cXG?8Zyjs3V~3Xs3dIphZrM<=N$%~van;=-`A==8$8b)UX`p#nD!+OgVt+k zM4m$`2vA~juCC2k?-(d3TfWlE#O>zmqoVLmcA>WJ)53RrNg7H2&{Jeh7zHLR9cDr| zZ_6tGs2~6vQ|`z4K_*w)eLsz(JUG-S3Hr^V27;oP0T1o(%OzIGKHU!jDUkc8KrO>T zK5eu}T=BRq=AK^brW7bcKiq1p=^Tjn zhvDdOLO0}SUXu$Fl?Gx$mNL@_f+a-`!Daa4CvZH5!l@)tvhnfZrnxGl(TWON3LY-G z)l@Tq?NBCZ+GcC-A{JTh(1T$&5~F&g-sQyn2Id;-m}}9^n4Pvr4qdyl;Ju9@?-LGBf&tUZ~VBLZ`{P|w3S`)XAp$! zNIz4N&%J;ZC@9EbyrirM>XWgxSK9aZa4`uJz8O2ZmfcOrkGKfK)3tm%(bMxANg7nm z{yIy!!1G8|M8{{Z{UChje57X_35z-cFX(jC^>jkW1#}F%OXlM`Mw>hXy+?NL z(gnR+))%RAxo=F)Xz~Fw*gUOQlNc79kE5>l>6}qJYB$c#PS5YqSG^QDOTRPV)o}CHA^+6^VF9 z$*S*%CHTS8SZrb0k}MKC$qWc$9ko_C$_uaVKJXBCBu8av_E@^yq}$0AwYyTk@6QZd z6$to4t;F4B%q;6l0AubYw6UU=P4q#$AzzZ+y3_X zTb&G(-$6kmyq-MJ2!c13<35c(LQfV{R8Jcm;nUgw=fgt{P_j^T2mwQzDRG1yyklQa z$KF5J(R(Y_93(R)j)bs&d9u>OFYF&od zlO+Q3x$3-h-*2iOCoA$W`}2pOalj_4gdUh^*8MvBzd+(tr`>6Uw>H-1yYCav4L~t@ zy}k~+vnilZ=YrA#ZpPenNyM>a=+ZOeA?By4GXMZe&A{b|;M6JdT%JEk{2)ggfvG(D26nt)NQ*%sJYc*Pj7Br8@Fr zQXmIqr)v&CDX7T(@~a>x1vc@;7)eENH{Vd(TB=@MFyZPc&?*4#dPc*ho&zw_NEE$M zGC(0I&gul3S(>-9au`t^yjECrDPHwx)UOXL@B%o7@!o$qhPO|TZYOXCFb{k_qiXZ# zQTxX0`i`g7pvDJGKx^Jk2r#JsesUXu!5Nl!Eg3vTcmMg6kUb@$Y8Mj!yMpMZ44&5D z+?qY>KUvvq41n!AmUvVD^Gz`Db{(HTaQXra^?!ciKn55+-)#H8fawN8;0Nrk_zV^Q zS)}f?z#=8HeH;GIHw}Q*Qr~!=QTdOxR$wbIx_Jy9_WwL*e^+=Vur`+revi-pqqi1c z0Y-Hg3i+1@W#ByEs%BsLXl3_L{+H|c#2b_xNBald7PkX_;EPT`ed9la|Nq*w#fZP; zqeKh7*KCm~iE6?e@S-8aM5OiC# zILr?1Qi<@Q!pMi8lOw&d6>(`eu+P~brIi#mhAAV|Gu+_6GzRFMxZ>4A1j;C*FX2MRl0`7hiZ?*K(x(Pv$0`*3WnA;BMy~eH*B~Mqp z3VUvBvGMVc4H9TteZG;IKp7<~_(^1QfLVHm_|QOklewdKBw{C$lSed#wYr&rQpctd za@m}itm94Qgj`nECd9Wy23o@dJc$Yh7`eAkn_c=rx2H@HjMX<4l)$DjgEs|D$0902 zWMMftK8--Y=0um;+*~49gx<@0xM3)2J!yVo5Vm)nCqHs&qlh3{)H>(VR_EVprCPj0 z8!UFwaH(mV)UWF_w<=&S!$j}LWoAoXQ;-Hd@Agw-;3ig2m;{6>Mz*=KbM3UV1iG-C zBe^l#%-L!8t0$@Kdt^_>;$@yu<*U$7uoFExd~`R2e18k(Z{^8zQnmpn<)MQ;rWqclW|RqRwpm8h+O#s8qFQX+}}Rwlep1jbzhrAq{YogZ)LKq1NEy> z@xF?LGFQ)b-39Jx7HtJaoNQhevuJli6LL3BR0B}xy{#v6`*yGnP0=Ct;jzpa?(lU` zMCS?Bq>qzD1AI!VjMPMIzvND3rK9C#-sugy^^IOr*;Viz955)Q&@?)!GS}H>i77n=u61Tr|;G-eFyV;(|+8G zH(?jX@f1-RO-7zhW3jQ{{64`}*JFtEBa05>m;#>mxxiMuR}2u2gp=;%gi)1&-p>(& zl5YmQ*hN2^y%n%v4#cuVaT~FI2bg)MkGE)z14oY$m z6qk$x;3|ya5JgsQ3;*V-d2&rG5gf{BV~Li*pWZe~=HQk5Eso-szw}8YtPuJKq`IRb z7>r6h=~(69#AIW)Q@2xvPLNri)>#;ObF|X0{h26vveDb=Bkt_nmn}Xx$ ziC$tW7h!MA2^Hp!J?lg(sjt%7TM{UTgONY@=c7T z7H`lRDzTeZ3HrGWZ?Hhy<-R&F9Vu@AtoEG%!dG+J>Efs?97jzom2>^VCX{f$o4d9#btbX$uHx;hpB?#n@E!8oBcHc z;7xb=eS{tY4|nQ88{#vhE3tg$<@wRU4Vz;3fi0fh`jg58>CwP=N$0s)b*I1U(iwke z_-&FpMA(kNiY`l#1s&_86_xtb2<_$2n(rc& zd(ehbhUXqgpZZZD-+6|XyI-@e``~R$)+bgnB?q@%TL!bB-RpMwSGXlkJF$35%^WC@ z5Z0VvKOlE`29^uC=yVpBb&&XIYt8aS$1J~skjdvP{H5lzLI8Dz6<1MjHF?g|dEnu< zK8$h#<OfHC&NwSZU^4{AcUwK(6 zw2Yy^Bsh!g>7h23m4uN+Z9XqNOS-_!Hm;FHk0q0_j6Tpb+3G_!6Sz!|X7dOJ@RX}+v@0kDACqyFEg?VW(Iv_svrD^_7B9woH(|C&Z z{CI&4XjA*%7H-i%ncnDdDlhL*M`@V-X_cxU`#xd<%1V{J7(0C@dJ3;A!V1$e@ixl1 zh-sq^aAJaxeaa&G8;N89p6Hq{R@;*bbXg#jsb|-*Mx(We@v&>Ad3y3L+^!&up*OjP zFIJfWG>{u|UIZAV+upXI1p(?idRmBWU0Y<>_O4Qk3#dHqZzin^F$A#M0ilQZ*q0+V z+xG$%6JNk1rWK@mH{adRXBzYm?>q~I}pkKVvMR3~HVS{v@!Y>Scq=3D?5V3jw_|vf+{I=C?zp`o7 zXETOx)&FM2ZlS936P&_O%S=LM$a0J8F5oBH?tVDWtZ<$A=^!uBjy>$JfzQVvaULks z0QS+G3{(S!UQ$uCSN8sa4y#Jfh%pOYQCL|~Rumy6E=6K6UV%{{lo$(0uDy$%56hOR z9QXd*ofSt6OQ`QrBy5&1?LIWHO^i~Pse-J#KauYG$#9znu4x7W8)coQNKTUdwcu%khZpXjMdC7EiE;s_ToMT>R)Lg zC*5oa+rQk^iYzY85aq%iol}+yeP{%WQZbpGfD%LUFay_>5u#~S%Ik}WtBMQp0#lc! z*3SZ}Mxo-BrHd2cMII(f-rd$mS~)y0B_Ia4biA17d1)F{3s!#!?NZ842^RsSW3T+xtoJhc`*fa9ufXbn zHkPD@CJD1Md;2$i{5?UN^Bx8cai}BOEzPLXAKGj`VSeTcW zS6a%XjPPFJSGa?Vi-z6C_Gs$hzyJdS10oXA+J}eZkdPO*^l!UvH=@7;DJ;6qV=-iW zPwu}IRCCu6u1UjmsDpOFyw+A$&TN?Cwf7Ru_qVtF*QNmsv;kiUiHOu+vIS>N=rP$7 z)9*o|V`Dr2JU-mNHkxIl&fr0tD7e@fy2G25~L zB%|n?m9%Br4C@5hJbc9{o{MU=iM0079jKh2F93C;>5iZ(9!Foe)KodOd-zeYysB{) zL%_sxTC1zaQRJW%(6%uu(&4Hk1(6i$wL-TmbusPZ}8|3f^4*JTHo7@TTpg?`g-48tI7CFmfqW zv;A5J#_M)&p+&3^viVu{Bnxi}pZha_r$D_=eIy+{Kp3;I#h4X{S zX8v1jB_UZl9cI7$o4+UIpiH9QcBW4d zum0a$8tAn&&IpmqB;#v(up&9F?^VF;( zCAUg17tN_Uw*xLe`A*y4r{lxx;cBX-sgK`fA{jtvb=zn!T0O1~w12bklW5^GAi?@_ z>sM`rax_6_q{Uf!m_k7XS(sYRHgf151pVO&QTAOCFTfFNOCd@+ad!J|{e8 zOCN`zN^t$b5A*$Sh6xFKT%>$>XHmMKswnXNJVKtnl&#s<65L0$ZeZWl$%S*m*NOP2 zK2b&w_zXt$NB>AjYm4#u%5dx+{^MO*{_};tY8e?BH&iQ5Sa2l`d}N+2GC9LDGbM7y z9*Qfa{Ap-}xtT9NT5ALbQONr$5nv2>MrdaSMlWIn10LHaV3(ERjU7Xq|Eg2Kq1e}e zKOxWK0{ui@5!vH@u;P|@hx1VTM&_!3e6g5U#hydXcZ`Dmhg-+T1U8I})d;-WeWF{| z>+h-J>OP2AA3_utt)8z|7+x*ET`j3$@MwG=J%KcZ#zF9ZH@GvXzC#S( z)fsZ9gtOV_G?&W}mg zqWEt)NhW=5*9#j2a{pClW_BU`!d2|F*dzbbuNPokJst&AT|f+hJnU0JvV;s{NL9%W z>#Y$*!ZXv$PrNtrD@BSbbr4#%ODU!CqP8E-Akex%_^gn*y>RYft0#4_PBt09P64-#CTh9}S05lDZr>R##qr`WZ=+~`*fmx=SDBKgN^Iyj%nm7Eg<`&}L~RqsBu@5!I-pn21p$13M(W`^1!kw}j1%iZLp^ycl#? zoIY4Er;^167}8ARBr!S~rsB}bME$TN+@bUrSg;aH7V5g^VP3+C9(LJcqZ*~7J(r_; zux}6$0Lvhr`H^4b{fr%KZu^!~)7y1tZ8_-EM*r4YG(n_lVfFLwJK0$W95%gJvRavk zrTMZfTCYT3KZUH_HEcg3@n>3Mb3&Y!9}_>23P&!CSg#2N)?C&2?gp)>cX^$)&7QEY zllN@)p4B+*YVJ3JAz3t77)|9SJF&+xh+0b8(jedD@l zrXR-!bkKC6i?QXCpc&8OoR-sD0CI8JpP$L97rHLbQe`d4@|+On?D$`kF18NAwhn($ z?Ch0T3(Rq8#x0U>cdRK({?6A46QdeE-AiFAWW}*2orVgEB0qm^Kt&z9!w+V#CPaT$ zYu&92+XJet26OUwdz|74`IraDXOm%;Z{B`QGXK;Z~fGd=CH6w z2s1a4t+6KVZaR8bhc>cBiQ4aaB!N5;vzd8Bw=5a(I*QFr>fX}Hha(gTEw;@<-Vrv- zTOvH%1VVTg1jbmNki!`G^v1j73lrp=d_cUwBs;&@UY;s7>G}oqIW8B0Xh{HSx8&e> z9W(4mEzNZwr|(ZYZI9O?^W{%Nlj5I9N_<^RcEN9#>hooGTcb8w~Kc4hlJEJ1`=$09&7srU% z%Dd?JX(1yfG}v z*?(|Hx_-Is8M2UIkG}`#Wue{=A<;w)hywTgXP#eHIV?tO0xBNA*`?6#?6_oH16)S=mp)En1F(*TY zfhDIv1*TP;@m{8p>flonwYE;08^XW_t}-WKcn;Hx14i{%)-w5PFPn5hIMYXfh2t+p z?jM79&rGfCO|dyemdKkL!Z8WtS)F%(>EA(_cl1Hf^hu6qCd#wl zK+@$jtq956Vx4i$Mqwljk~3p=EcVlH4Dc+uu`QG3N&=jibv8cNF`>Cc6tO9Uy=ue} zlqg|o8K!Kmi96Jp!q%>HVJ`;n)a^H*QVRuzvB4Uutyjo9e{0hb&Ci%1CzhWQe(|a+Z7`EPh z?biCSajqbku@*6{qG;o)06JDQ3SjB@z3#2aM&!I<4=OC54JVVX$MFquf71L=!-qgo zOfLrT<;o2WkLO-ueYcFP*G$zWy({tj=|g%f1o1Z zc3L(|yAnTI8&;o?0_=#77};AU5U}xw%#|ZUg4bG`S1Uzp!_V5=?l=1SrDr1K$|00S z)#4zuM7%(@5{qG8Mw_}W@Ca>-F^;ltSX&|PYDgBzV@GlK7u2s}oRRNiB@t*@ zI^n7K0^>b`a+UyutAmLUiv~23Jm15UmC!@Q+qiafShPW| zAYh_YQS;8<+ZoNFqX8@hD72)Fq8WjARQRG>qVTFn;r^z`QSU6gC^#@m^;So+N9U`i zak&tFps~#;+Y+`3OiKXE`1#OW2?sPn@%CER8#a4KAjsZx8tcB)VFJ`E+rF|unAhoW zPvd&Hqb5YpBfxAcs11|Cw^fY4If(-Nd~P)PcHfl<5}oP=z>>pvx|fP85lksy?OQS6 z3A5&363UO1l>vEV43NV${LgqzbVAio<-VD?NmE1Bd3lW+%2Badqnek-z51o-guL5w7K>7_NT zvMeHTRO-nKYe4CxpFoOpkabF_yK|uUk;Y^bGg4Mq25uai;RJ{W&9~N9(9$uQ#e6R9 z!3P|^MyVNvrZIp5y)(^mvWU|msGP+{TYA5KCp9pipDL!E z=)aUm%?v3G<9qPLng~Mo*Gve)ix8ooAR|$~0Z>CruB=Km*6PkPPzYM8nPEASlm+GF z;XO%5+sVlHJ|U5#I0W9?p1)NXwejd`k=)@ji?s81U`e=Cj37)|^h`d543O^^x+f^5BW#&4a;ltdpWW+vo?-d#?XX1)>JVcD8+a)4-`_o)}?d7uf z|LeIFZ?h8xwC3d!{Ii8u+M~bg5nEI?(1=8(oxdB#;l84TRXMm;eo);VZgn;0h4^16 zxq<~wzVbC!g1>Qf{XQy7+LuNMq1@XyDdiqjJ6ytpsQ_hw?|+W9M+kJQD@Rke=FxLv zy{X5!0yo&v^iZT|n_Eh&>8HFcq^j`IEcZwmnKW`A7OtfM0L(~^vzPxxsP&xSi9a;q zq8>?wUv&;xy`GfAz^8MqsH2eU=!Zgvu-wnJz{PF*v=>OdueU*vaiNIsr+1YWU zYCj)gxwyDUxfujJuixLDZ;!;W3%k63e>RrR?bLBK&y0Gvjup@xS?~34YaaY3e)Xi^ z4!85Tkmt$ATfVNlam7?N?byft;+zkEe&38zcD1(~9i`RQavJ%&xYXHGhae%q2Wn^b zYyPu1ato1xO2&s1*Bl{_qs5`2a8?K!8k*QWEbgZTTdMC_KhfF7-ENNT&-236x1^+` zJ!ro?0O7y`HH1+$Te$K|VIlodpyx)@V%6IYpNpMws$V~lZc`{YLng9Ce6I(^9yvYU z0;H0G7_w$JUENfNCVqO#xJ_L4u3vRuRJ9P1Xb+x5DhG;BLZS$%jmCwy4;w^X;Z%cz zgX*02V!!uQ4G)UUEdEtYz=a2Dl7D_p{t@Qn4Af(2P?ZI(hbPD(Tn#i|Z|^HptseUQ!}WJ3yPNHQccA-0_f%{Vrv+ge0%etH>edL- zfV)u>89H3{&c~~2NmOdcPq|;n55f8p*m)6hmr{P}RM@B?na4+`r?PaozPV`zX6_Bs zqKL(QSLG)CaV68bo#zD2uSZ~TFrZc2X}Wdx-LUEh`1-!MCa1N~$Bh?NA=pg3Qi9&P z|FA=zp?Dz!l^7^5N4t~%p&_z30H!=E_#(nzIRA!1s4tX?vo(eY&?lp#z-E2I;NGKG zTl^%oifB-@N4ABI<;w!jFOiFM(b&G&X@2L!jQ-Mi^e^L4szYYYE86SpU4hcPEKP6( zfP2j1(`0=C3Sj7^y>DWzn({QipanHSDQq03 z;NA5>zBNBhsvxQ>V!=cP&Z?MN7i9Xk7QK4B*OOyorW_PF%CCLh4(C}#28ULEguK!( z_B}vbztv*4q@(?^qq?4#dqszeI%)sR73ovwg}m0FzY04b8hr{# zcK}A5eRk(N#eY4=dBA2(RwkrJ^52&Qa3Gixcz^8dD}D)nI$b2rP|P5Fo|nS>B)E~4 zZ;lA{S_!((6C6(wi%9dFq(+&sC$I{qS6)B}{JU;_X;?FYX0psp*wU^O1QJ<+oy!=tpY{Mk+)2zdH~to zl#a=+S-X9Ex;G+F{&z%cpu@}G`L61>-O6|%fzRcq4!11%_-y#rCs39c2+p1_~Y+| z^N`|pOMiuK{MTUR1p-$P%=DE}AKW%K=OT-GOUnOJYUWCcXw$5fVpJ_zZsjHxrii0n z^yusr-_K+zj~Cvn-<$*vN1O4|%^DVRS8p-+1~Eh{oa94fK&(%fR&~N6Ii?DNStC!! zYP+^+PqSBjr-{m_yL~Zo)0{w>Gp6ONDfC?fz~oaLtvGG*l_RYhPhU~~^|pI~Fg+~S z?Br4>W>1!N7^Ek~O;tOLE+g2(%$)w@C7|uafv@<=LQ{qG@B|g-mOgf<{!nG^D{ZIX^l|Dlf-@e) zYt7$zrhH0kMd~UbA&Hd^W9m886?x54KcLL4JtdSqu%i*w!3-itzCll;{kq+XC9>rU zkYreh4)-OH1VSfZwiNy+d&q;@LmWc$RA{9qF^j2Olv+1|8- zMRbRPWU>(P(w71M7>>ksZ@HR9J=)Rdab?@qii5PbO=?B`L-B(NSo3jvYwE7(h*t*! zEYTL^@Cj+$_z)#7(rXfUt(=IRM;NS^53$3#6%1kp5ZfE7crqnUrCepQ;S2(VY( z_GV;3^|UM9rcHj+he_M~7UUZXti1&-S{iK7t*<1wqh#7vdZNWhnut~0%Xq<|<{(33 z@wvUH-^!`ob&AEBscm6DW^Ca2A|;OLuBc;oo_OwN?>gK*co zlMAtOK4LZWm`gdQlHQE`{YdZz8cwHt^pG)k;R%~(fut-jii>4;6+~-axMj#PlJR6S}3L*d^Z&!!}4OCkT z*YYV~#1EHW`<{W|C)DF#Cchazet7WEiLuS1b;ALz*rR-@c?UbOQV`6p&`@ImpN znRQD-pq}yD=iC<0gU;8T-c6HSBTGz|C z&%CAxt}Ft@femnEw*x+CGh+kdWFUEgp1UI=Mzm}|*TDf!{P1~6Luo2Sm3p8HdX|&l zOuf~o=ek$N6xJApsts2L6m$wERS&-=rdQq}a*m>B*{RkqEB?lJO}Bb1QooT3OTozG z>jn%6c)9t``J#lNHdZ!Ur&J4o?*$n^qrtj(xCGbcxjq<_F<5;m5nLo$?Z+JInDeuf z`u@iOxva($x%yI-Hy?7erj#Sp{j+1r>UWeZY`H>Y&<6m1Kp1d$$(k1Si?e@(5L4Z0 zf%gpsjs|Vv0C5qP0N7niv&T#D&YWRQA%! z;-&2-uJZSvN7>ATu!^z`28^_Iv+csg+Zv-SGrp+1lN8D4j!B zgo6Rer>&>CUqbF=Z)&D9ia4#q=p#4FLUoL^)XZDduV+AU&*{LkdLtt}J>9I^y;F9% z1^O9$zovq$g*j-EVEPg}@)~om4|mP#KFhIrIzBzWWu9DF_~C48I4S40k1>`~_06FwUoXRA7V-)Y3REuVhu{1zzG2)gEJVwC)1{TtIx48PY+4N(^m(G(8?;*w={%g z-7m}TwUj=`f)&tbC}4f6mQtE?gPbu;NCCa}FALDM)D3|0NdPr4m12(lQkt*AdOAcn z#z5vn{_Aok>|>fyOU!^8BBh3DY_}E11rsC8WxMWD>km*TJ^C6G2YwuI`ZX{pdR@Er(6NRVB7uPzHusmX{ta&^*7W5(FK=yX z0-NhQUaIi-Du^TUMrtp3){&{RjKlQiz3No0wo24JhQCX_N#g2nYXZkIxV-kIi+Y_( za?-7lv>8hJEtamRP6ohpq^4p^Tu^b~jiV(ie}r-{T=)%J9VT{faY0MZVS>(tiMJdR zQ6N6LU!`DGg2G{t%W#$^x2DnN%Hv-}xibqhgxU4ka47R$anR&-=1rtDo-xz~#3_kT zTFopwQ;m7ohT$0kbob!VlFzJ#OV>+QjJv!s6I7#L(e?VA-A|Q72B#5xKv0n%QB9th zn7fS4a*5EBwn;Bv^gMCNFg|}@$i+bxB<&s~LTl42r;6!@G+waaf-~+H_s~>p4oj|9 z^)Mtq1bB#>35NF?hZp?5T0DxCLJP5<|8-Yl=wAhRVdSxPr9h`4?1M zbIjUocgpa+{V^Bt%Wo@%0I^|oOi*>m&q0J;M>PaseU6j#uAIGnl_wx0wi?DnNC8cp z(BEA;`*nvR($$^`9--$b)iu^;=ogxYRkSx4q1Kz^I;x2|k!G9x)r>b25GfC5H0rjc zzH<2xD&G%uo4L8(w$3^X4(jQ2T`3(XXz~<}Ng>(sk^X4f;Heg(H|mj-U4JZq_P}vS z?oHOs;Yh9-ndvSs$;3HQSsZFv5wWE=q3vCDC?{0A@SFann49Bdkxb!cv$nQhLo^sZ zWo5I|p~48Hd+xyGQZ0c#E`J!PrsQwt>R9pt>tt1&1OIqH$=`0?d;K`r$!TZHM=6%i zy80oi=ug&|Ae`a2;dCqK1mKw z%m2S&F5(mAJ$J%X`(I&hIx;8{%!{Y_R5M z7{!^RT=VOu9JV3^z8K>dz7y-Dk>8h#zfBe_EsvWX%jXgU8qw3g&EvtO@H}|R-gSz( zcBafptmnjgw%8gViUu8eNP6u%7j)HMM#o?L_V1RCDfo7Dd_^Z|14n?m2#fW9wAs1c zD6>i+?w|y5DN3tON-}BJ7-(s>%BoOl&bt-}AQUFG6}ENHNO0%;mW7h4fYlnJkDj9PREx4bbgWq4dotIqT9F^^OZMo9nOL4+F zIS*T=mTEr;3mMLpTZ@YXlV$kieyNgBC8C@YWqUYlcUplL2%y_kXKZ8kGHUoeYvV2D zCDrr-VJmU<%a8@Ml6FJq$7FTIhA!U#MI4CeQN8xa=$p?69$TSIoyMzUrg#)Y8qPOW zf)lHL%uZsF#_v}BJgy@-3DNeD#&_E=IkqEq(%+~$Y0b1)Nk>Llos4n2wqdEPH3sZ- zZt{(LS>F#>#7!0y-p`ZGd; zJCjDPUF;hF4-(<7SA~@ouiuV|UY4}>^|}T&;cG9@pHx)5Hu3R)?38qgSW$f!N;JP; zIcR?Hv}TCL+W&UT#ixC#F^#jQ@E-*8SnY$p79ip6aV|Du6Iw;GL+eA-keNv9v3cYE z%?tf;PAHMrWocLn&j6ssWxmgx%Py%Tp&vc4iqLDBhjbQd zP%W5INE@W4?IX*)`Kv755Re8#x5M4IaN`BHbfQZG*=FBpjS`3XNaIZpJq3A^QC3zj zaglHGpA99prd^=Y_;x#IRs{i(k~G>R`nXX#fQI0p*ox`9mMDsO#bKpY!Y zO4`RP4{WB|QeGZ)Q<9sfr6J7CM}YN$=Ini**isztK%s=RbUp1Ncl5;jjjKvO8`xN^ z8(h(?+Y>IU7Nfq<&`AZ{scY6f7RYESZvFS~)!7Pb`bzA=mp?C`SLdH70xbtly$6~* z>x6Iq0~R91X%RNlx@`8qKkhJR>h1Qh)!TkP2MUsrUNf~Hy-on%xkpg%M zfZxJH?TV(wr^kL+MuZndy4oyp^nTj0p^aznV{(6~L;f)VO>1u%-xUMVh+{4?>0me22?}+jvZ~iJGaw)&_TTuD_@-N>ZU67OX_u^B$mcB23 z>q|=%>lsnN8rM*CHK0g!6m{kxXf9!O-pRYDQL?9j0G;Xz$4)1lcgJGGzr)i*z3-A^ z-@I*mfBLwj=d+~$?W$?#ig@gkRJk+W?HMIwV3zbOKP4Z z!MxZ%&=~2;b{CBt)X!vvxR}-Dl$lax%nlS1P7sc?9O$btXQU*#XpP?#u4XG12ey~% zskYqDR+R>91JO8bsL)i@-6KVyvu+N0R-1HvQLQ^8{&##&b zeQ)y8+qRpwrDn{6ceaJ0-cy9in{QzALbx*Hs)`1pp2GBH2QEwBizz*Ik4*2;rRy02 zqHQ(yLL;n{4SFS>iW#OeIIox?*FQ>xWb0jE4KxtPtC^3H(jd#}H~`N1z|0Qe^8FSI z9xye%=MwBJrG@Cwaj`>W1PjfhiIJ!znZ+MR>QqC2&=T7(RJ%jU3@u#uOXV|7h+z$% zT0N>P*fQ#>3b^6vwhYb>J|Pz^WrW*Iy2a>bh(&a_hSnZ`E$3EnSW9?JX_D3qQ5i?k z^@wFSDB0*h?p0xU*Zg+dL68Qb?dJGcxy}MMY+(p8sMh>;>NGUKpPpL&B7TG6;&m=ahfzM;P>gp9U_~5pq9$C0C4;q^nBA zOv$BOl|cAwN-G&qCAl)4qr!)R5m=@aSN=#`@Kg-r!@Ee;+PnSqH(LnT=oSXC1FpXYF0Mo^p7lOD zAa+O$QYAB4 z!`58NZ>Udrfx&&b7AbKdVd68ggL(~qG@lVM=)Q@!A;{eyaYhawk0 zf3^3PVNrE&)VLxN(v5UCQbVUOgmejrgn+<+NOwv%h|i8_2uOD~^FR3b z`1t?!etSQ>KQ1mWuVMCyefHV=-uGJfS~1_t#ZPoLvW^O9lfSW|@l%Qf&?%$006rhW zdp)o!0DvPLGHky3v9j(~Mb!P8!i`WcdyWqS2y=2bShRp>BU2FdJe)xveLE7uno zGUet$J#rp9yf30{*nY8gs_5Epv&U;5kcUi^kdb$6D3@Uns3hNAtPF9pp+_;X|}si?=!D2ROtB@+GL?;C}9J%=%j(OR0?50 z3aE+{@#>BLg+%G2f@&ds_nfskEg6d@2-pEio6_AEK>EEwzrNsw3aW(A%d3rvJr?nQlPhG$oH?8u#D06IfH2fB-dU6H_bb zuK@Lbhx>oB?=qDU0Wj&TgYtGCAb`UZIGRCxd|KIsEr2SQhK2?&OKR6Y_$WvWs0v;_ z!+9fzu%+^2EMJ3BIhB~49H=&NX+28Y(`$5AGBj~>tKT=i!}I{1FBX;qf6D7uuCA^G zjEDHYMxXEMohV9__iNU>v^w^{Ywr9ZUq}l8@Bs2*NC>F_UgfeoC+Kmy9qn_yJesE> zB?0j0TMi;^L;$P`;Kdv2=zJOEF&}KY+$>a3P!Mo={Tk8SC?zeeD|&4xV^SO-YfD!) z7%al>%7cd;fnA8wWR4X!s;0yjLM4c)x%rxk&*dn9A6^33R$^izF(swXZv7f(O;jr; zu|Y;ghRLUD}&I5A^|cfpJFX-LU4)bUVbkZ!hnw3eRg)%dOGvQ)i+H= zysvi0_)zFx3N0`V9CxqLr3ySOZO&(_*>sB-Al*=|p9U56Z;$;`%NdmOs1NUl@=*L}Nb&VM?%kVsW z*0jF0CG{Sc1N&;MH+f(AlPeA8qnBOLCyxY9!c{oCXgG)q2oOfDz( z%Qc}TL@=nw8FKS~Cr2MIkJ^F_K97yrtNTYiI@Rx1-hK zm$hY-qzpGyVWEauWX-Y};Ym^`Z?eQ~TQ(aK(LQk8>*4_{Kvr4bKl>H2T~ zlrR>Uhh%s~>8=2>I&Hgl33{3{0%vwWtV<<>Oi(+gDKduhEF!N`MyX988OanKu>mQU z`-KU;@Ekz|-O4FPZLu`S!MG;t-5h_N$z?YSu5Ih1Vnbv%`C36x1;FG1Afw*C@7AO= z$zMrYBVK>~@a$+c6B6$sb$Iq`41YqbqP5WIG0j72@F$1$(YUbIYl<6#O|~-awt6R? z7wVbWi!@UZ6zPB>%S$BcTElvlEN9bgbj>(%fy&@I<6jt%+T3_}F7_@^K>8_C4^Yq9 z#VE~c+nRW45J4C$15lvZw742Da0UoydBu|Qe_9_lhdX?1ZhN)&NhkwaKf^CH^ou8^ zq%0A-T1NgnW(dh4;0b_zXs8uRd@`3v!4qQ_@+;R77{-ah@)U;1NOssgrW=epa|!Gx z*0vv^bwTkxkv_4Z74>I9Nls~xqbX;SSE~p+I$4p_KPdNO!txV0yWa-Mcq|=PJ}l?r z0#r_CP8m#xM_Zd0Ykn!}zW&5&H5e9)9Z4&|$P4IwCBj}bvY<@T=V@xN0CL%Kih&n& z!+DugFQ4#E;i4^-GL3 zyvC1FG#8ERea+t>|72R&YE#mg?MY`_A?XRTkmK&C;YAHGtv-2H-Ok8Y*QQ35$^6XK zDAHUXW0d8wXGg4&n~8QEu%{Oa@>nV3CvW^podhRDV zp8%9NE~dWNMrLoa?pzr%C`omLCUB^32C$9cS1oy_<05^m@5K0>91)z7$qo3FaHJ0}C>PxY?FCsm$`$=snv`?iwu#fqRD z23(k;>k%sV1BQDR2C6B6U(h>aaef~5M;v@t%|z?__@L)Art9O6khJZOL#Ee4L8Ihh z3^+;-KfbG}$Z@zMcCA$`+nrYhR3;adYv?1mThh-N7J7=&;IHq^ z>oU)YptV*nPZ?^y`ZCL%eT2g$zMy1Np_O5396M&?!yx}xJB2pSm3Zty7A#=fv^{+G z97&rO?cj}E%gJ<%^#H+Q%Tua&v%2JsS^v+p-zfx`6Xa!t6x&>6KsQ6}E?g(hJR|F8 zQIastJt}bgiW2gRQ5z*M)^AtOn{A-I_!4V-HdC}wnhS62Elwu1T$&f=fs)D$7x|>w zUOd<~XUC5uuBM}tg?sZ`%aBZC3mI|kli;HBXZQn7NPSCppG%u?p+!_n(F?zMzWk!cs^4bkbfxjM@HfeoQ3J@<6OGEQuW7)j zT`Kf+Z!D`mXrUKN>Pj!|4E@+X|M3podDx~4kTnOmmV*lx_Ipdv_@!cUf(CCczd32Q zefxldD9^-4*zNMLvSQ}+koRnk^MgSC2E^~UQRZbL@^Ib+j}tnRt5n}BSODf zYO4s~iHEbog zx69nAnAF;!%liO(Vd$!6-t|E8w@=0WI39U7n3^_xTi@-QtBYkCXp_IYgcS~Y*J=O& zmlKU#A(GCXqhKB1$5E!_=Uz1XuYSEI<_PIo4wa4<*LK(A{4&Seb4yTWKYp@kX+K{H zrWxTMOOQqJLlY!K z1sM=eek|sLWZE4eE_EC3I%X+AHI!4jK^0`mRDHf}yM`Q(XwYx&YMr2Mz#^ zz`S!)yqQqPEkYP*gqi$BpXt|2PFDR$6XD4T9DY?Cr5O34BD>m?wGoYvOoe*jQI(J) zl>F1WCFsVip|j1ZdT6=PJhqu@&f+Jl#1l8DuY*yc?1`m(+sMyYTJF7QgyOjk+Cu092;X ztf$3^c%#6z>G44ubaN#to`5u?U)H4N2Ge{&Ji&r>81PK9YXmA)!x-u6urp~+rdyJr!4ZHHhl_jZNvZ#k9O zdM&YWOmLIJv&0T`ddtv1TL~%>e3(-gefeFDb@uCZ$}&fCNXb9;u+GM*4j_B`PGP*s z@}?M@C^O95w|#~3M&pk|KI~hX8H9PU+{hTvsNbWHROeooxx9K!LU?%QCs!Qn>A&r2 zD8Td#X$6dzjHEoX8{JOsnvVqP!xw0TW0U5RFatKJ=P9BdD6 zO(yE+$E+&|)b%<)cf6bER4AxlsL=771$?e9|zyI2BDlh^Tt5JPkC%>}Ff>xh;aMPw+XOw?O!sk-G&d`VP8vLsfQl1)^}lCr4`gf?K#sD|xFc2w91j_>ON? zdksW|pXaJ>)$`FLC)O03LtcKIHf_~sT`O|PSUtx|%cgI(q~Vw$-}9xC8_A&MjE}W@ zb|&F02*VfWGk>?Xw!f9%H@})XyZebZ@+D?9!{;;1&E|Wb8H;Ca`@(Y5yGzZ|6r$G` z-M2e{@^C6*beBb%hPC&+TaS~@wm1^bY3nl;h}dI`q>&!IF{t|-X%bdf#hPJ}ytnB2 z)+^=5=a8_eBrZkN=#6Q;f;0kIP)D=Z#fvpF;_szshY4LSWt>t(6YXDJ@rgWBh--n$_H@;q)WE%o1@-a?rTzxBV^()aSJo2PSTz9E;e zvkVQ+beM2!o7a&6`A!26Wu-OanU|jpb(yXwfh|oRQDN@Pmdm|~r)B0h+6QNaep1ZI z1QJWscX9Ha-VngAPr*{n3V5qX+S&W40+TUr4CRS*(yG<)--{i=!n<2OX3e~ zCLN&v+K}D{d@f76-@g0&>ho-K@|ZjRH*S)+FPEVVBT3m@1VP}RnuKrd8O->VkC>Aq zM~RA1@7QiuEWq~bWaCM84c0fk4Ec)K6Phw_D04l`47jf>%&q4GC#=EHdT+KDxPO!5 z`~d=T?SsEQ0SL$dTe0QfE9nDEDbYLfC98Ic=J!5;<+?JaF;({##WyMG@x066TX%PN zFII<}%acOw(h}{YTI+eq>d8V-{2zjyYBJzDx(aOG5qax#Yx48+wGF+TjEu6{#f1M6 z7hn}h3@#SFFPE}gS390-XtH)`unvf3Zvfg56NQS9tv993H3m6vPwl+{)%*1p2kT1= zS3sV-iBfhfsAJpuZkK=3cCLF4P%%xpEyuyu|8zQ?YQUGT;SAg0NsPs3onIa0&@&tW ziSQ7YmXnE>fS~l{y=_6CTanX<$Vd!KOu%U|{Il2dr>ftPs;S=X?$`YQYU}SdjEgQGz`g~!e=E;TRB4vqu6eO?CNftiyv6{ zqbp6=<=6)2ij0C{qRWzBS5L1~i5uX}rU@aY$kEvz6*Xc{B3KoFyS`o%hpSb4{dhn* zRc;dO9a*AhV1P7*OzF9=0zX!{quXBlB2!2Zy<*%v3U56RZ@rmqjqJER2e$dklyFx% z+{J`uU6)akzHXEF=5V8utg7TxOPwK5>9dVH5H|Dv9d5g1f(~QwOt-KL@Py1Jrlbga zoEA}t1nNlOk8FVao=jVv0%?=;oskjs2d}I)SX{w)3uMDe>#~lNvAauM;hFv-!IU@a zvZ|tvWZnS-nLcZ4YX|HIF&n+T&$a{NB$@0Q_#{0KIME*jO1WFl)h&V}gzqnfkRp+K zhks`D4sS$&$&*t}o?$sl`tJuKLPOtd*g0rEa4MUxiu1t4raGH)-kCYst{mE&yod0v z6@JwUU(b$4ATsD+@Y$;MRwxt)9EKUUN_Pp_A6_xGEZfti+B!85J@Y zR)VjZqfKq5Hu?0suJgpTXB3D0;Bo0+mvy7=#(`LgDk^zX0QS0y@YFrA#a9{UY~b@p z%PzVgYU~;s2@or#xlMgkv5CJZG?a>l4R1bRchpXnJ^8#n)4}qfFu~CK@si(z;Ylee zDQu-}aNe7gHf6F18+_Num+g3iReOl!CZk#f(E)mv&;7FQL)b+uPf)Pfxwdby_PFSaY@j@%6*P6c&Dv^B)my?RB=wsI>{Vo zdhvJ*QjgTzL(+7o4@|BsDr#R{bzZl7%13Vz6YB{S5sr7zL_Xa!ahw=2?HYPMl*oKr z=Lp9;`1vg)1IuJOzVn?o_la(;6I-7?f#G>(CPk&ObyVJ|=5SQM4!59Iw`c705C#gm zkHljt`7_uswqTTKOe{jT67z@Tsz2Y;FSq&gL@OprnHO^8Js+BLg#|ECJqe`36RUM} z#kGHTp!61LigWYP&KES#G5tE*wq52hGyl0N3K+PXx*01iP|eCSd^korB$p81s_aou z20NYrIy660RRygBX=5yF*cfWDfLiC4SsYl4`~>6qc-KrOQ*z5$1TcXw?gPvmt_51^ zWKmI3RUgMKm)5KKXwRPjoS@xIoqj{PTU=Y~LZ0VS++chnq9!h?wk7>Um_C{&deb~8 zVSemEU*pfm>$^(Z>J;7VDS&8OPP&uc5r)beUxK>v&r z`kY^Gj5H|N%b4k1)@)&P~_m`hI&F0JQLU~JJna*-;V!CoZ` zGB7>TZ;?bwfDo58%96qR2Of%Z&iG-gGnWlo<{n_Y>_y_ippnIt2QnRK(s)m0J)f0T z(p5vCqvd!H=G-O7Qp7TgAT;ieUbMx8IGc<{25cKM%4b8gMz$zm*L?uV9AK4?oPk4D z_ilIhZhJGXQSJH9_7;F2!)F)tQBN^}yO4hRd!=r<$>=Dc|7lMZKFa&A1fU!*bKBEV1UCF|Sr2SJlyM zsW;l{9lQ+PIjnoj_4(R=B3rciBDi374K-%_(e^lH3!ze_#R!DhL6rhhGBFXgDLW~9 z)z4W~r%Q#4_a1i6UnCp>gBW`b#wJv+?2sX~1^P-;rxgDwhG|AP| zW8C4%Vl~{kSsSreVkEMO;{I2CO&lQT_0g8pX8_f?TKoZXopX=4eAn=7C7#oUaIWrB zX~k8Az|Aka6%hOW#c#MP2>*VkgRWv?&~W0owjFgdCFroAf#y)LHJHSh>mnlNve&jj zp7zz4r4oR(z8usyt|X56hrSF~76E7%SVMF0F<^3(FFzA=GC3bFJlj+NTkCNiAv&0+ zC(RaP)-iq%&2t`X7V?{RdOoc#E6~5ROr2ouf zbXcboiG~<|6{4YZ{xRynbJX2{vme61z)XvSuz*M9#;NQ|XSoysS&}7tcN?f8!Nm0y z0|pkS`56jAbrkx9=TQY&dU)#bRo#wp9aU5Scf}+MC#rMc-kYZoGU-Ko*T@BTutu)&!0XDHt0lu0=A7?t}f)_-q%Fli$s~p;#Za8SLQya=0f4MIedQ- zJzL6*wCA75+t#wVc!vp4PUR96#qfopp}j?YQp~uT9qU=-7>;3P*h}%9`9N`d*Yakk zH}?aW#Ah4yjI*!!el}SGV(yV#pM!)r6CQ7&A#1+vWXUAzu%Hd5r9E$&td_(2ir{OV zpR91lC-I^c;NuAzjXe&NZ~y{ zYz|FwE>ONp!i)P-t2&~yTMg`t3t+bk7;FX1QYAZ-CEw7&*mHqiz11+TN4WtNw6?6r zE3f*|{Vz3vob&JoADPZ$E9|pGU-_)7tmq)_fmhLCg?%|k>G{e>V360NGn2ShXi43i z;PgSAn~iDEV&lAj0kYjycIdfv_Ga!W)cT-oT3Z=h0G)?!1N7nG31pKMk zLkmP-U2RZAa>w)|N~MhBch6!M7AVAFdQB*%sa}Vk)^>{%qbiCE7@>ME006)G=HSg5 zVLXur|E{>e&5y?OK{lwH_SJR%@mo8207W=_ygV|JhL*QhL#dcy_oV6pOe6qmt*4{Y zA{JARoI4fsxWyScN7UB#(zjN1<}h&0yjI+kKE0u`qKw$TUB=pA5?XKg+zeH!t-fwM zIPrUuhYLlVr_kFdESs@~aT$E2`h?0C?08eJ=!6253u}Nj^y9oa9`I7EQ_JG%-=jt5>Mw17O_j!|j|Kym%SMq+SD2!e zK#%Bk-yW?(fRg#@5fYQg`hpDbS2~?UEI4p<2Jr@F!Q@rc5?|%5>6jTh#oF#XPkVtl zJg!m7@ruW^?&z~_LQhCHL!#FCW0nzmV48kQMW0xd1`~{oSVNK&+8T-|ZBad!8vR{a zupxaFK3-%lsAiKdX-}O^$y%Mm%$}swtk$nJPhC(c#b65OjF%(|wJS=i_UG(MOOK{y z=WAF$Kf-x9c2eQgD|)F12EkUczAwi96#rR~RAleYsEE)0VMY7dg7Q$M9Y)$3!{S?R z+fqbjgE|{NFI^zhL<;3#-TF|lyu(58b?nDIgyA|=;lr~kr!%1~KpCWXMe_9>H=Lv1 z%MFkRlxHp1h3?VwT$fs-dyenF$-Rnx9}W5b*xS-f7SW!0cBwP2k+Maosj4Ee#mVC7 zOa0yYs0|Oi&lMr7yXHkbIbo3W5qr0tugw%__^>;*et9JQhLhJ<$6lpdi{#8Z6tTI^ z-{B3s-qaPs!6Q`3vy=k{Bh^}6Ulr?P@%B8N(lk)#9Q)u^ZBW$OeR2jV)kiG%dcT-O zmm6atq&lbUljH%LwhQHw1DtiDyxOe2OqX>M^fZ&!bj~Pi`@uOs{LLI7l%Mk(RwY~B zc5KYHqmyS!b;87GSB`!6W?JoSNX++85mMe1*FIL@j~lAvy*5;7(9)YC2+>Z9Rz1tl6FeHEKpFrJqY=CrPZXMoB>fWk|U z@e^5KCPYm89w3C&lU9Xg0&BA^GQ>zDHPOGgkYVY%XQbCE4=628ziBYDhE?HtJ8&w^ zc_iWV(;k_jCEzPr#Bi{p1+0|}E}&n$_D}6!Z95}NfXvj!D}Zp8cJ?OH08VRBMF=F> zu(s1eKc%S)U~1N?h(7V3yzmK8qM>pg2$3;!p-is78b+5Xk;VXWN~Qr}(peCBd<}P& zC(!zU2NXg-6XUCFm&I0MpTGjVgjj<=%rgBwCaLfd)>=1 zrX0JWZxck*kz*P5;kOI$QM`g}3^x?R>m{O?kP9N810uuob|Urs*js|2nn}AG&ozRV zg2=fckOJb|oQGzfl@%emtVP;D`Ca=wZi14M@%2>L)6Ls*+7Gzk(FX;@KlB121xLly zrU2iaCAl)ID5aegR`qdhuKHugxUDOtOw9Nj^Dq1h2s{D>UrQU2IAykuje$smlB-4< z&r3}pqlE?Lh|Ic!q7SH80@Wa|SVPgNS75g-s~Wup;L$KhSaBO*V^A(R#6}LohaH=m zIu&-dJ3#}jhsWqHCA&D^W>w#joj$z8eR>;&pVQym1Yub?Lmbk63=^3ZbU4$98cwiO zy*}8OSD8=6D;>40+lA%>3yoJv(YXwu@$G?S;6139Id`G$++m;r;z##lHJmbdLGHgT z_obMxMGj`Ql9G2Zv&)fe?9emNL5A}96S)LMdR+EKsvzy!63>h%cdlgk#LtxxXudtD z!^@;DgqldLBaDTSJC{LzU8Eu4Q(LhbXgt$gV#Ueb9`w9_On8#Su2^DN4{5Ac?BYrS zs!2^JPBMM$itr*12enj)PgV`r`nTGmh?+LO27U1<6%XM>98lz~9NJmM^psn0c(OFN z4nJ@&75z19ydtjTk!eHZC#j&ZB@GIgCSIt_+#AG@<0N~w-g2|AY`H6t#Iovx*BP-} z+P#NRixmjTh;CeVegrC!^v_bN^nA@0+Oblkj2pBw^A%1dMUFlbRf|VSPd%TnKaLN; zi9?vBUl0P;Zi<+IE@7I`-&whQ&0ew`+T=vH337BAO6@2`fJfh}zx$ZJM={DZulQ;e zlHTfm*7gl{L{%4CvPa*Q-=9p0Kpt>&H_!*;2__E>TH9oZ zJG_0tKp|)kI+9vrtYB-rYxLDH_k*&NCrKrS@>EvVrqhyuE^UcpB|5-8nD&h?2sy@Nje6h7+QL93IJ6j!^ z6s1){VSOsh(Z`)0KbX~li3qSpco4H|-A^=fz(M>zlo>lL2`|-a(Zf7V*qvUqh!J9P z=vrzrQBMF^6}^h{dVH;-^SEMj`+9Ekdh>zhVbv3ilg^)naQS#5C1L|ZMfRVTgF=%8 zTC_h5Oto8kB_E#Xc$K$%y;!kPCVjBW{svR@!-M0ovuo~(Tj+*lz{^R!^Luz(<@WCa zn!knQ=2{FaPVzUs1daAyt3S}nkp}5MSZT~4@Ncrc3`@stS%$Heqds2#ik^;45ijZ@ z?tM~@$vfbgCarO~I(G04;6X+73u$kpmt;9yd+w}$RQdFJ;^KD9tDES_+x?!4N3_XN z;@Y0i?oxCE5)x2(q1O|MDWZGexdOXFme7aaxo*B9Htbe^hyxzLAiP04?I9l-z@+GK zLS_A>BL9Z|9g_iFfvfvE|$)L~Qh&{&G~uw%la>N?-&UoZIqvB32Y5ps$@ zEa-wx?$Z`>J6#k(4KoUlSd?S|9+jhhAJQEi^xdaSj8mG63SZUNWw%X&VwNRb))Czv z199u5x#>O6*@+je6Z^)qX%hA53%~H|wvJ_QG|9ZZlL_dE%lS_xK;j$T6ep~cWPXk> z;7h~Y5vjU^fo7>y>*?F04|*Q6F8~LHJADCBRI5bDL}g49;onKW8-Po{HI%A-moWZa z2oQhAiV3wgqy4K4{1)J@oR|-?{&iLbAX!-h=CAx+{&|;yQ}qE-k(WGc1b?-p2d?)e zG-X=u-{S)uG;yFuZ<3C}erN6O$T7IU3D{u+R=+G%@2(dRJIDYiWw-rq(*L!DpaLg| zUpM3X@E=udQw1QB`>Zz6{C8Vbh`58$28cTt#QfJ%*AF-WeM3^@ zUpIdVlub-z2$ufkNXHKFpQmx>`(^&R`Tup>|GMqhAO*(h|M6)1f)uh^IV9$>|D&z7 z)YW6Z4cSg!;TY-fQK|%f^#F{wq{IH@d8+yjy{pK7gZllKOWdh@Ffdc)3}4YwtR+X= z@_(<{JH!b<`ozJ@t;XE~@OxacfuGv=w*ykJ-mB)`tA{J!ij@O)0I9gUcifkdVHgmj z4+UrP4(w*{cdadDvn zKmb!0TdlWS3%94WC(8Ia(Vh1W0AEcJ?EtFX>B_ac_|FG^=0iy>b@NhhrPdor9$zH* zJa*XO%8U^8xvKTK3RUq$n6$L9fj72ZHFi__=uuwWbx<9R!Sy2L?V`hwXfP4yS~rEq zc1h!|$#d_K8)DGY5T;HjHv5pf#G?e;Z%j`x4oP1dq#|5UQ3~j zwxK(-wQ%493riv@KXgY^%Dz9@U+j2UYkkurE5^;u4UKNQHM|)&>>6ynNNH_zD$i2| z!EYAeH|ZX`usa^!vMUzo_RoYfT!fhX0#s_y{K$=jnD|}c)9v9vY$f#44uONqJB5$e zZvY+K(kvG&k^Os`7FJg1%DE4Z9Wt)SoWD0XpX$GZ-S^=&;b@CGZ2#S%EBfg%cSKeXgHakpyF`C@G%BKyu3Nl*8rS? zHiVGjb1dtF7zg$|29%ZsaAG8{Au36-5k~;=w9L7#)N_Tk0lSk8-VHyeBzmIvj!^2E zQ_m>j0N~g&ofHe;0jgg{oS05u67#h~L$WM;zXDpeb!^adcX7#-kaKqyKwPfTrBv+4 zot8`SKEj+8zHBtbO)LBiZk)F_e35axkzpwv)H&;GpigpV!}QO4$-poy0(~Li7-cPG zYd2f-mN4C z{V6gsa#{kDelTvUG}Wok?GW68Xa#@3@FLEzjkBSI&VcH0GoKE81Ym9BYcnjUHB5-l z5X#KKQBLuGr4X9F@DOglMk@ z^n|4G53u@5MirSa}hieG#bZ>#21j$~q%^v~J)arH75i?&md_EQ)QxF970 zdb>SztC#{>-J-gWPZN%>*i_>H+BxttgmqF0%fExP8*>Si^=*J>;G$W!Xbq*Us__sBcm3Svl)dYwJL~MWDGoWh5ReWC^Ith{H zB)+aTyyB$`MM&mTGp7giZg#H@qJ7AA{65`|ir?UoL{)}`GAwmct+tH|+3?i-sx}1{ z8r_c)9|6now6$JX(LNhRjWa?bdcel!zpetmmT9XyaWAE@7bz75A;!k46yF%_S^&KY z@jJZ=fUyQFpAtZk8o3Yrn4 zf3J%`Wj7eroOIA#C74ZoIMU2Nb8MbbPv41X=^64`)- z6rXN%nb@BpNH^8{%%;hiAyvo61+qvgVZMn*@yyh9cO$1C_Sp4&m-Y z`TUU{!cS~~ygG75+K&6d*TahK>qSzesK*x@xCRR^45r;6s}AK6Vz;c#leYWngC0^p ztxG-6Db;;W1HTO#{up6GhVK{pY$j(kIwQAwT`?+5;dzO@p$rvzHFRdc!>U*GnLOBR zWHJZQDqSnHybi=tvvzc$)oIw8Uuq5SkXA#!Swl623KJ#gHUzrpa z^-mUe7IElM)zi|ty>R{dWXFX<*(>)A#dDJM_k?b@H@CEv0=K0Za`2Ue)#G}qi#_pv zsK2Vpw#ZG?^uva96)}%_B>RPB2lzs3qInk4Y2XO_XwQ83I=GcN;Rw1m^k=9?KE3O< zEyhatvc@Q!aHxG&?5NnS4$TUEN+HC|SO?t(4r3KHClrH|t9>r@GIV6*<=~q*T7K9> z(ac165}bkew*9o1aWvyxzM+i2nr&&J@gp?Py(0&{6z;&$+U&ZWL)v<#!d$0)noCUT zxL=BpE4SF~b3JigDk@M}2Ik+a7h;$mstrgN?%%aZ|>xPh~)x8Nv-;OgS=k&sT-rg*V@(P7COpc5dLlm8Z|FA z;neMpTLc>XM-h{nS=$>CRmE*tUzDrGhhN>Cd?{c748|h5@JCN_mfe^L-QEkb9+f$& z5nss^;n;y+g7Qk`SUJP=U5DK4K`vf05yP~}Q~q3AwDT8CY^M_2*qPO6U{~UVcW+Ii zd1Uc~)pAG46+#%eRqaF$vA2z>yQe<##_#m7O7;k0jVEQh2|BD*F2s)^TubF{kFz;| zNKc2*SpN~~8{S0Z`v_^baU9Gzq714zA+`?1z)|{1wZw+VQN8n24{4z_2TpCS23V@K zb@4+(O>E*L_3DX$`lnLt*9Xj#JthEDZ(G6hZ;?D3fC{c-2ogutvr%ddL#- z&p1LEP|S6KG@g+hmE;@|j#B?iduw_yUT6UzV)MC)90ta~F_4Z&Fb1ZJv6YT<*ux%0 z+gzcl%f_TXyK5xcvLrCx`YUP_<7w!Nc5XYmLwv6MX{4xoQ`s!N72v4V3^4{jNCrab z%{VfBSg5ER4S0g;r@D%dj2_1LmsqN_x{Wzb_8F6o87k3)bh(wX1&U8HuQ8JY#}rfF zH};%MUzpEh zAl(QRPA5~s3d4nl9=P=1!i?0c&Ix@VtLZkToq!cdCMCB(&N|#OYcUJ?Lzk;)A7;Z^ zAZ4`#g@0lVDnz~{2=ptboG4|J6&qd*XAgZ)I@d@sqv4jtGClYxkosAl2yy+Rgv${g z`!F6;yuwWZ|3G(?+iC3q>5y&GN4iM<*9VQ^LdFHvMMn?S3{nwP+Dow?-^}~S8!gp4 zOp*Jlu$K}LnsST;6hf-Up`lGOO+oau-J1Mll4fXRGUHr>C2%DJKEUsXlZ0AEhXZK| z%adJPzT^#eTnomylodfbvPIyWJHs;Ht$?HRx()&1Om$Ubkqp@kT z2wMR^jpy;$5?$#;3m{u!SK~5@4}FpU&aRn&b%q#htOON}P1raply;b{hinWnMh?th zO&Vt)i+2$V72sE~^rBGmFl0z5*>I=PT1DTU zZ*i`R>=@G?z8I{Ig1+@UsjOHC8tQ;U7nRy?tV^(&ukAHr>JG-^$+3Oc9L2!ELv0XV ziY>Q*cz;N-6v;EXT>kgkI$VLsmjkJ>vMFEuv)a;?g#yV~el_#*g1Hq=Bde^5FF87FC+5@)PcV$2euuuWBH7y?bxT^Iz;@Pc5Zau-M^?ZgXpZ_UzJnpPq2Jyzy{o5D@SfNl_Q&5rOPeNUl5&@7X zBF2$H+?uI=!)~0^kObk-)nB~V5cye}O@qIZ1$^RGyU+4xP(8;1&I8_NNmH(kg+QAj zPAkxSoa$JJR^aD?lyu}jo;+j05>g>%d?{&37IAhCJEM0;t%q2G+3Ay?Qx^<125SGQ zF#jBO$MVdr*K75@8Oc##CxC&0dw#U`U^+xs9#kl)Ij%J1|0hSfDV0eL#+!C!XG;Ek z@??ls)Mv3hLW4i*MnL5f{y^jlCDaL>dg(S<=H%z|6C!`)K{IiHR|W8QC1EV?>mprj z?iBj7@pAq37ks7L7r&D^0Lp<1)T~N>gueJ#ONjQzlz6HWdeI+E-GyAKtzOuEyp<=H@}j)V*9@0=ll|&APLYPU#7sP`6$&>Eo@2NManY-t1wC^A}22+Wc7c zk0}h!)>dh@VB;v=_E6UfJtG(ZJ`?zHEF2!~&3H7g-3e*E^=HGpt>n;ascgdCZ7fMU zyxAaoU1i@|6)i|=sDk{R#m0%0(foQL7lk^^-{A6(NwL#SH4kiH{&qEkA_k=m-C5jHDko!Yc z8CyOCDJ?D3_`AVGE^oH5JAv@^XsDZKJ)%3x+*2B_`$sMORtlzigY{H(`vQI|{qrPy zr!wS|ch1>Nk4ggeGyOo*rC1>_s1<&*Cq-lEePR0YemPhqo%55=_4l)u-~~(!46mQG zlotSvc^7}+E*B1z%l~+QCGrD*05|6;=nf)yM87q_i7o>o-7g+wxkE64Ux^!3AjN;X zbyo6VAs9}*(hJB3x0=X;-nO8ar+eNoEM>MpFRr5dmwLKA`kNHV{GU2+U(A-Kw0+x6 zAbQ_uQMksUSBVLoo}b5v1JmT%#M8-)xNJ}P8hQbO&9~DTH`9Otp2xZf{v*+&ejb6d zE+WIK;6a7hRlroB%NT$gTR;9m2q6HdOPkz1hchtL#Z4^Q-3mYH`c?xG^KxJGK_2zF z$f?H?kFD4#vt7UDza10@7SH<}vEvjT8Gq=m*4M|vMnZO}$Otzr7 zc(m}nV6T%xL#HqX16n!2Z_X_>5nU4s3R$JRfl*@Uqw!UzV6rm#@c^uoEdv&=aB9@t`_RPVVp8m_GGDj%C#!T<(=5qiHjyGssM})u zLjf9ucxQD6ZVoJRp#aKg3en9}#;D2GBjMI%oPndYA(NSNj++q<3mBUki34csI(KE| zH`?;55_@V68miZe+^&6_|Hd0~O=#ewx1ze4<$&(FtbMtVK9o{9xYH{E@U?4lyvow0 zkI&#w0$*`u4>=CdUg1gYEQG9S@bL#iB*|jTQD2}V`6uokP5RtS64UscjBA%ux+Rr$ zXcz6=p$%4t0DRy+0VvQ`r+BZh8p6a168KrPNWw1HWyNRqrpknHX_D_SguS`?y1I4Y z5JM~T=TBN*e77lL{GF|;!uK6uPi`nIE2E!)RpliSm1LvJ9`_5Kh2&O#3uf$>Sk3Zj z4Xa=4=>iwwYxsz$^?r?pNdDmDMaI1VP>l6EV(INZ791N5+h`3CbHDEr0j8;0G%6z% z3q#!i^2A^#u+UrBF*`<}t3BxC09#3Acq~0wGs{{X^m3B}hx8vu`QcbZzH&&3`&0vt z^4)~f9Ay?LKbaW(Jn%E~LaG)1@2j1PN^Qdg?*75l_;r#;m zGce&-nl9X03H)v|77tX}K%c>ic03|-elKwn-*7V_>cBd}esyZz@q2m7tN~JfKzp;d z3npx7zvltn->t2tYsE>G?=V%b%@u{o#_!b?2=@MI{jKF@`j-JaunK)< z|Goaqo)Ihj3*^Pq4`V~!ZZ4gQ2EES26?xlvJT_H>#V@y|tB8tAYt#4X>EbfOP`j6y7)hp@B(6qjF_R^Z=FMPMlgXG)#PxjXYHI{T2B#?Mso$hpXN!LwYUta@dJ{kj!Sx!JS z6m}QpxbWdT9}JWI1Mh=SfKw?Uo>Hmz-1>VPMG*If4>bJ#x)$-^82=+2-X7AR@&A_k zR3K)_)z@FwWVMQOQCW;B{*uS?&=5^A{htwbM~%=xh@?;EearGMruyrqJ~IMYT~Ui8 z-9KRx-LT{x6Jl)Iisknh0w&q9j{s32y~>UK-&8&zxd~yG-%0Rb1%X|du z?64{`Dz~$8uT~IQL}Gm(9TAZ#>QR#(w_5$D|AGIIWZVPeK~eMrK$7It?`P<$pBuEs zUwlEn{Z{I^-U?@Ifj+B;f9d z5LpoQdjjdZFlof6=ynH) zo5Dg?x3`N32OQQ0vtlvwMLB*|qW(MH%MbzgQ|d5XP*&86^qp7+v0k?wAR@G*sTJ{G ze~|bBb~%y^;Xtu)*eW0aa|a~@_6)HjD3M0w|G&-%{xUp_4aR9ev8bX=&H>{g73kx! zQV;a^>j6tU0K0i?!56;QZ_(jDR6B-?2fn(G`p6fF6JM1&I!NNa;Zm5ea>ya4mdM1D z9^M|`#)3FOhgEOSxk2^|1v;wxsB#hrxbNQ~TKnD$3t@tW%b5g0;!V@jvme9g1YHLi zYiD=t>ze{o4)}k6@vhArrti3A_3X}YS1Mwdl-a_y3bZ%Xg+wIjLM&wvp_;= z%60?Zt?<*h$vsD*w1!ohJ(7Pfl8(yvieA-Od!qXxZ+F`(PLjxB$)DKLMMwjbSF7f% z$*r>><}mfKPYWNV89$<8ek1zHyV!7O{nu^orhX}(kK49W`AkJRe{>Glld z#vv8O6u?c4s-2_padX|cT!Q~`;j_;PxarCke>JB_lqljr3VB2m0HF@-x$*W9DQIrs z;W&Xzh*Kw%QT0g;y$D$m(CKVIVS7G+O8kMyr=52nMO4!_TxMNS=Ink$T_4<_-^ zS&a)b;~VCwPov#h-(0lKta`@uv7#a7cYguT#+3y@?v)ibI~Y@~Gh#?Y%*uOZ%&l49 zAxZ~9f8`Z6E3_ow#&0grbY!{)K$AuNrn*81e+$oaR!p}le`D4vhlXZ|9Xxh z)V@=%YQ8y)Tz@h(SP)XN;K?x03XqQc=V9Fqj4z|OOzcgslDeT6sP+q%F*onBo9_WX N&!m;XC6cfF{y(dU3Vr|p diff --git a/images/docs/ui-dashboard-detailsview.png b/images/docs/ui-dashboard-detailsview.png new file mode 100644 index 0000000000000000000000000000000000000000..9182c1eeb7eea56ffffb98d2773084dc8ac0c2ea GIT binary patch literal 46116 zcmd43cQjnl-#1D+A%%z`MD!rYgdqrmMDL7VVniFgM;X$H5=8I4j~2ZXK|~OB2%|?A zL>Ya|yCwNO@4MFhx_Sxs`@AuPo0#%h|$Vq8QiHL~E4&63rvBO;=gL~>6gG+&JUN`B{c15T)(BFr6(-V#W7c`4w=_1ljnuC<5i zd|<>%<(d176-(X2OT(Fb!nyg?^Bfl%#zdwY}gR!Pjl)CU{Hs<=dmC;@+oj z7H;NlZlm#DO^ZnfzS|2uf^V-$USb6QtdAcz2hsQ5`uo!*Mt8ewQvY5{kd2c3JM1-t z;=g`La{4y#-aG#P`HcU12V(%^zdv4U`|$4smk_dlPIQU!9Q58q1w*3c@J5r(6M5;- z16Suds|WW(uloHv^jked{!%qVXN6=9_r{bQmxj%ER&(s)wI~11&5?_M+nEstFN)Y;9`qK6PI`SRjg{UM%?iz+ct!qw-`R_1ht9*8(=^AeSxZ&?1FUZ2}v65g;H zE&@N{|EL(*iwMRa>UyZ(<(U89hurlqrm_0?L+((rGbJUmFMo{sJJrZC|0bl?5Al#gWsb$8WkA5mq%2yXxi7yn&-hX=H5p4}!d}vmzGaHt6#Gqz zPtzyC5#7g&3KmfV90lrYTd5nqKW7hGYR=Uc`V24$W6-WG{v3*(yzoov2;4_| zvN>V7zCd1PY;_l1tc3nm1U@&iV!q>l#u#+g_hn>so&Q1~h+AjX-_AK>gHk1Fw?*^4( zYU^#Pd)K1AKCPqoi+qG%vu8qIm+d{ckks!q{)xZ1Gyj%>^!C$;-eas8OKctF%88Ki z+eLi*dXI(yv>{@#F7b*1G}E{Cxz9wBiQoFtL80V8?jq}l)&f9HDzja!SrMR*r-_Ed77vgTLX9{zRvC%18r7Ir@&T^L%}&;8}Q7n49&igv7~1_mol=6Y2k z`S&|Y0ruClHf#ojmS?Yz80-4wnQ?qknmiLpMrgzj+iy>~w9zlMcI6Bl+f_5>QdQJh zJ(YWW@e0OmSe)g=$wACYu-1z!k#69|_cjH~cxOTJ@mGe-A%hYl~ z)u-*2>VuN-Yh!k`6MeI8N zIILMerMcJTsRZ-uI-OY`J1bV4zkTTS>P){bHM{VT-1!r@PDRgAEHu zHy>B`h-vtG(o{RA5O>ET3@f2jbkQgoS>KZX?Z=B}Nw!w*bK`I?F?~Ad(6Qcn3!iq+ zp2y z8EuCHB(C+WK=4DE8X1IvA_$2aPSEBLRu{@9b6OBz~ z>Ax$HX=RVS^XP>n;%kdVQd@FRm;J;-hIuXn6+b&UtN%*DUAe(K!N}L1I|ea^%9X{? zY&QS(V~>$dMX#oHb^#7-sEwySjFsHo)Qco(Z2QpOa-)Q$(}DGUG3laA<_C0q{sKt~ zkDMl}TGthL_Pvl>6CwwmrgP^-&g2fKUw?$}k;6TuXv}3bjD@U7ja)1yM(ph8Rvz@& zyph?vhkTYa8)Q;(`k>LilqPJl#D*Ic91O+wH#ZNtXuX-1EyL5h{>3@9lX-0NLebglbmU zB579{g4=O{4MlDD$3xaw-N!6RQV2e8(-$sFmyoz#eL|8QlPzIWOkPM)^T>7j-tw)9 zi(T3!`P6Hs6H{$H%$wH-N3za0xJ|5gF}6G4LLbMgZ)+0nJm9nnh7rzPxt+=T<*nI> z9jjv|0@URF%R_qJkA(tn~Q9 zpML*=qH|J1yJv`yJ$7^*y=>X)<4F9b*YZ<*d*XmCgD4yVgY@WUQZ$^rO>es|CWO5y zTcPNxe}qok*T{TEkDNR>D15FVs&WHbZdyotBov+Y;iu$!hKOv7gE+vc zz167jRJ6g^8JQNP6}ZySrM$Y5{OOh7`MK6uacWSN9h5{}6G`FgT5DhL+lSQ+lT~zO1^We2 zBb9QufBJ@6jAewfXl#vmE}ge`pq=>Aqr?Cus6Q^1U)?ywS8kh<6nwdW)O zP2^?uYmz5l5*BiySYlB9GEpY^d6@w`3%gY58BL-2G(xOmoE-nb)Fid(N02&VtvYE& z`(ocTW=-*yZHdfS={lK=;I;~WX_mi(etWI$BSc9aPGa-uewA$C_El+Smp5`S8TfH^ zg@a%dFVbTpq|-h}Z$L@*A9n#vM0&_ugZaJu*3d}F-FWBB_x^Q;eqNKlKaU*31LS2{ zbG$$NhZ|gOSGD3he!ccH*Na*@LI_4Kx^c@x6HkumRdw8Bb779DdxiY`lnHItU-BQ$ zV|Ny+dEvWa=nr01=HRh#Dear0I!bEHDe{k5Gi+b4r_20fL~}QT(jR!C^jf%SXC;Hu z69VeKJ8&YZ)uTH;SMzTdS#U)O4uosNOIdEuZI3(S9YdQO&1l3bT6`)D2NlK*MCTr6e91v z!3&O+#35=ba)DN*U;id4>wRE)S;ga}p+Zee>#X_kA-|VC>4n69H|N6xPXC~{%9-It z_WZ&e(USl9wATNF?)~FE*ZyB(kN^2NCy{`3V!)RHR~DgdyY}Z55%k>O@BjJvn$+K) zACvt(^ndmMdyYkla;5gr#UA%Ajr|*4G`wU1d~07=UWCQkeE*n=t74A-l|=FE=a-O$ z;`;Rm!PLCw7N_I?6cTKr6qK+IbFyLQX#)+w5N)jUegYYpFM>*D*^FBWmsd55r4EuvY_ks^X!Vb`%w^|hdlko^%Z?bgThyn z$t#M%z#iVTQxuHe+THdQ59TwC&5GO5&Q1uWq+An_FDxrlaK~4Qg+Nmkc@FkMATenT zD#=Jz2f_*d@9^egn+F>u}}4SaDooGvk30(UTa+)z1j9# zc-L7=YtpBw)s3I%F%1hNYsXGv_;Y>w{PvOhOv*FQpHjzla4SK3_rhSXx-UO&s^8i( zCh_vw!HU0PpFzU27BRD&5JB#T$Pi7V_&p2HPhQbT$Ka8J<-;OlPRyk5U zvRfK|uJRaH=U}Y4cgX5%5toy?NJwMNUU)GT*xzhWcGM-{MNenHU?a=LV%;Bg*>9PfGbqRlqZ1k+#fCG>C?4&uDfy97b!(`InjP+K z*z*`)8LxW(o{Z1t$9;`*bEW!*L)>bj)nJZ?q}x&-WBol}{#0W!i{2F5>U^j9Zws3p z41yaC+o1=0Q@)pNk9x%kHnjYf&y0=h-E8ZxO4{xnA8hxO_ljAsHyznU#>MFr?iGz3 z?_%w9bj!`%nmFO;I7TV|xbE(5Nl6js1;y?4`i%q`2Gr1c=2h|IzP2x4ocu2msVa_z zDShlvNlq-9^cbzcue=t$DyP^<2s!cAD$5@bA}|qxY-w`3vYs5h*vS9ne6^Bp5UORU zAH?O)H=LvE;n;tm;UN_*Nm7#lyU;Zw-oeKlG*i&Zd+ry5`;`J`EIN6+U1d>j!#naB z+O2kXbiZA7v3wnu+9O}qdrG$%>_rW#^i1i6V$FItGUW0$3?D-2errS&@hI!WAPmzA zN`B8wjht9E8*R87q(*ub;!uwm`@QHA-9tSKPJE*hAm936N@pm&Gm0sQ*j2niEs7SI{wq(zj+?-R0IT=JI zdloW?p=2f3;&3M3P!U4VtLI;MV_Q}+RdSb7*JwW5TudZ zctyhji(mq;&dTflu#ZO6S5Ojoae;{2h9p)vvj)SkUFYP6wLyWy=I3^V)yMtro)O6- zhL}Fzn~u|pC$YMcy9*=KXPWZXaS3d-!>g=OhsBmgwyhr;2+t@suTXBq7T^sHphG7k zzUpi2sN$(PRMTM?&NF4Yc6V;3>gj7VgMMI5!qf^5xl!W5jEmPa77Yx1{-lgjK11?E zyz|=wr9=T(*rTlg8Zqx0st19oF}Ml$DJqJ4a%!kJY)ZnwxjWnFzbD+Zc0MU7oaSY$iT?;m5q(^WdW<))wc5Zs_W85#L0; zdv!1F*J$|%h}iE>XLrl@e};*7|0p%>!S7Akw_|w2iHKDBDM;ue>Fe$mnx2Ga#P`X}*0*O@Y1 zFCQurf3@lLKv9~!YjIk{%@9Z(TeMH$pg=XvZF|3d{MY;hewL-PYJQ3(O0u=P4D1FP z1JgL9J}zY^g?@#ofG|x_%%y#{T{T+B7o&Ofq?bMSoqZ^KzS>h5x|hQ~7PCQVJ;JNI z%Rp3-Tz3X?&tJT#aW_>Cn}cG>-=RcvZWpv_SOz_f2&2bKGU~kw0-01&_&kA? zRNlc8f32#sii>I z8QPZlkiAFvV0bI4yKt-07q1Sv_YQee_lB^&2BliH?+y1&k*LBvT>SS+*WzG^Q8NUM zi%&_^nTFUIABkTS43&}d?qCj_={X^&&1Oc_$u=D;k+T)a;?6y0c->WUo*j=SJ5*UP zeRHAcVD%GKiSDxspa6##r<`Mk_U%5Q&HoyGh-Yx&N8H^UpsG>vy=(B)@FUbWK9Mvb zN?1ZoXevy@n#(a~2uB%uuAcE;j}wt&30 z`!!+diF==3o!u=h_#p2ML+1ydQ|v7$qqeHI-$B{`_}F5dL2FfyDbzYsfMz3l?7@U{ zDqeoX(Vr5vg80~+*jY@ey(SdO^IQJ+@r13*6%CP%xT2V+E!5BPOHVWwrRrYXOW(Kj z&TtR?`MsAZq#(@|Js-|vdD&H4%{_fKAYGjNC_2@SI6bS6bqq==iGvzw{`T;vkaF!E zcmx#bYpu7gNu?9CIVoyU^zy$IRzz;gk=gGO!YHmw!FpY6jTfmeMK_b=J4C0;(sMH> zcUym?5-rh(KH>7GxNd00K?>h?sZ{LYV-;zkR|sbIXGfHsDSoiuA67$EAwIwyl-~5! zQ%wDxu3>U;6=I3S-8z7QX^%N$#Uu+~vhewE)~E!6R;NXK6mt3;nyk)HE@!kk^ylmZ zOtF#}XW2Rqx?I!bGlf}2^Vf@I_dETbmcSBJFy*HxGJYml_^uk{GIv{q<+~oz`{|eHLg#_#SGujpV2yAy|6M zF5Nt@q;Z$lBAiII*r5YTUFkNksiPKqi@;BBI>Dbjba-(nfE)4lMJ1WKnW~`}f7Qzc z?UC3F7V{qQ*v3){7oO}BRnK3fXZw@*6%1g8YUM*)Jbq7M4CFVZP;7RDyVRHUe)v&H z!R_-QlQj@FYE3djYEEp&$F75yy)CZkoPirJz>r1>;_}&H>0^ScZ?S}Ijq^&z!{U2} zrI_aF8ipbV_U#oL^dhSR08h{7GMUzS??Tny8RD{~vIl8@%p z6;fvqhQ&wacCYmAT+<`VoK*#ldo9$?m zHycHXe+q8nNGLFEVc%}1GKkhQ>@&Vc3msb0Te(Foe~E<%DX?^*GrpGRcNZT)I5A=+ zD8Bf6exUh%yZCjPed#s_SqWXQsr7$l+Hcz0H!CNyIs9pH8rp)+ zuY;Uc-qDQLHr|CEqwM!{O8aesg@-OyuMq?pv05gsW;}EhpM$8l^n}zX2Urzieh_N~ zA-3i^cVVF5Uw+YK+7UX!>d)J>pOLRsr1zNc*h_8pN9ir3JncRIHy5u~gZrd+W;dpP z{_GluaIu-60})OanZ!;ZtfF47Mz8OhuDN)>m@53cnTup$@Psg70BSslIU~Q??`PPP zx#ptgtlpWKkY5+6pDcdBJB>65bQolsXqr)ukYx9d*qgpQ&55iDe}L05Wy`~*+};p& zq=GhFfoeKi;u%zf#gnDncSAm|PK^*x{j5l$I0%)YXidZuGKZvDG1QtO!>CPzO2zHN z`Nyww8R+(sMB?31SNzT?i}R}-#88JolzH;}ZMP=aBnk1F{ZUx~6X$zGV5%VmlvUi= zG*%*yB(FK}Neyo09bA0wq{V`tB4=!(z-i%-csu}t=FDMO+-v6cX7(C+G!&TTb6jaO zar^UBu5yoaWpc6#9REjl-N!s z#clu`_{(t{gDT?W*F-kKx6wtnwK2F2biB+g7*&WnV%2=plIM}@#WU_ky@#q73*IGo zyHx=i5<>O0rl#iQ+R0WI@7dwZ>-xB3G`!$p@VS>x?|7ag+ja~ks&fYp-s_nx>o7ce z+Zc-Ls-B;^%suduYNh;c>7bul<<7wVwG%b0s{!Vz4^Co8=p|2$NWVh|ZaR2=eYWTV z6PY_j3FWxM;OmPV4I0^ffhKwL%a1Tq8*A|@+Z(1!;ucTIg4z8~jvyZ zEi~#e|E(JqGUEf$Fq8y~y5~gM8JfYclwsawx<~Az#m2AGIubTiQq$_>moEOKIP>Je zi^eh!*F(4Jf<9kKoUx|H7)IpsRPn55TOT1+|8qV*%{+bm<~7gMlUJG*lk!bic-}-s zoASl81k5|o**GJ{s#j{(zpc#}mEa2tjMm+#4h>kd65M|r&jn=AueedI3!&T&@9m%l z8hu=#pXJO!S0~m)I~x1A1`9t1U^&OI-fg!sV&o;^U82C~BDTtQHDm$Q&qK5AXlqG1|uDHF$sDdR&wE!5pQ?ei*iiO)25Uo&!+0?yjm|5|kmE zVGB&}i)Qu3sKO;+qndE(XioAgnO>B2p%o{Q9)?!x8ka~C&PgK{OWr^AdgC==Cn>l+ z^dN<^rgy?W?N+?P4VSJ9FjSslpWCEsUChpY_|Vd~?rpp&bFnt&;Y5|>nGCku5qS=R zHO)p)7wv@#8=$yv%s<-H5brKvEWRc`T&dy;R~06i|b-y^ktC= zg7$W|efCYTylVJ`q-(a54YfLmyDVlAY%q>!%jKKb>Uy8uWeE!t-HmJP^ahS9Tw>yb zQFW|pvoOaQ!I3u;xtK=MUZcyhI@VodBNveKss}G{D;P;co>?9tb24%_csHzo z^9&KuxFWxvn0f$&$v^vv8pz)~K42Qd8a|=*1|ghjt+ZiPSw_U^Ux{8qz|n|+8&+CE5T47(@QXl>x&tyhM1oJr!aol@o%kk}@ovlH{_yTR9=Pd|Nedi?3vsnB)$_2=Wk_y5KNK7|C# zZ2s&Z|BtvUBB=RKH(lQeVyyo&ETaEe2{pJi+*x-ab2D%O1UPc+Hgx#r7N1AD35_UY zy;jiqKQ}<53 zl9mGu+N$mE2C?Q0zG7UXV&A`c9po1}7*8XHPv!Y_YCk@HI16guf zq?MCESil~=%cD(P%vxI<%;EQRi-%L{YG692FJtpr55$iW zV{wt*nbvog1JhIUHFC!e-t<>->s5U9qPum=x>|xt>CdQZdfHV#B zqj{Z+Ukh4DC7=pdQf?E}ROg+`s+d(+O6Cp{BI2p;`bTQzjICtMl`Reh=g2Q^vlTz) zhodwphm&5DW}_pO&$WUs>lz5MM?GKtVtuTHDE5F#lW5ZFA77wd za#Nqo;(LB5)MZwU-?(a+^y&%Xc$`u1epag#@C z&o}DY!q-fuc|N}6N9`8?G3?!a4{gRf7mfZjzF7Q%N*-p5nWCtZML>{-8AnmURaZ07 zZ=zXW#X2kfY}m_VD>Uv_uIkb0VoE=Re2?erNNhs1lX96y2*avPt9@>pR#6E%v3B?l{ zPlruZ4bqIi`R@c9x5ZGA&6eRDa`wjChv=}!8hFw-NBdN{Vd<-jxjt`wSo}f`JI0X4 z?>1jibz2@mNJUGQY5QG3s7%6K%ckpNSMk_^9-+02(?1*nLq^2|dRlaNa6~CR*wGi5 zo{$>~)a;#&DRCrJ)6Ahe4r)YrlK#qe=P+5seXE)JbSTqq)KcU>GW2CD4j)Wv&E%BG zl~;$oaNns^lcTSze4EupzCdx$AHAEeo7c6x|Len>UfJWJ8#t$hN~#(x;%--!`3;WK zspH?W#1$_#w%8s!p_3H4dCC!a@+k=~4kJ@i)_XEfP!XlKMLpxPva&$a)z{uPEVbRL3Qo(z*UG~x7FWn02=lKXxTMXTznlPr?;-HC*ii^vsn zMGY+ycLu5En*7n6=ZbsRb|{O(X7)FKN(H6&fx`!*c4pgycipJZUc6F4)zzOVlZZOn z-4y>bLurNogWdPP5J0sw*=opr|M8Q*p5|ay9=CtUall5o1h}=EfKN$Xf z`QL~V{%f57d#-%gR8nl#>sEOZfi6ArXuF%w{ObosHdfYbO~J!=3_kb6#7}C3y!St% zjsauAc7QQEm?nw1e|K3M!)TTmO+4D^lgjbfUNFa?;lCh6L~S1}p}z*QgAW!G?N8mA z9w3I>o?0sE8XB2Kp;^jK)ZC}tH-r}U04!DMQT=e~w7)Gnr! zcX+bDJKEc*4uf&)O1nJ(lG)8xpq6^|_JFD+Civp_H1N`%o%+QuZ!K0& z`ILA322jYWmz9kz;Ua@~U&GFT!s!JgAdQfPxw$yKN@V=EXfAB?Q)y}c<*DZ8e4Wya ze9iob<)QpYIa%4?S9ng6#C<25q_x?MU(B5Mx{uZ^&p80bPJ`>xvn&tXp(+Bxz+f+S zvTwTGMyc|{)op!z)S$-UmC7vu{?458_KuNWv7>v_2Ve=0J#2ASbM2=;>HzIDY(&^@ zb-&SWLRdtpDDb#sd9*wi)da+q9Ka3zhDCi(yhG=Sh~_28+QI#>NDT`P-o9O+`|YDi zYvB+)Pqn4O7CXHA)S%w2$YU;C#%5askDC8R1t19F2U~uTUS-+EOYW+=P2L`wi2X0O z?BhT>p{2rh^1~1K_!^f%b1Ci6Qrh@YnmJlw6^SzH z&=VcWl(~8eg9bmENO1+Ml+;-C-X5%Xw--SHpKf8g<&9b(V}}#bOQ$>j!uOXBPEnE6 zyDOv3&4(*okufpmeBS#GM@d&8Of?+Kpb`HfmejudR8jGLvN62FvQ1U{W+Kdb0e+8~ zTkkoWa?;maP)1bG)#Y(552iT@53j^=3OT<80Uu;WD0Xyoba#`t`53XNTV)fEqC<07 zsUg0W(h;T61(?*ktu+MAygq-K6a;-o>s+m6p*Iv_?m@Zrt12#Df!~1n?yXN`Qb@rp z`dwgF%Rq?NZT1mOz=ZEHN>Fj!}sM1S#C}$`sN|)hG*Lc zQz2j{Q%f0+4%%|LWUKg!K`r>{kq+uCjqt#qvi41m~1o$X9CS6eZ-O| z37C#pk9ybDFQ*rFjTlC5*zm&f@wQqB$(?@C(T#MVLq+JJ^G_Df*F_Hj;ApQyCKPj) zD9RC;{akie#Ty+Bf224BuU$TgN>OOpnPoX9tcI3?Rs|i?l_nKs{L;1e7e` zB~RPRi^CtLVetTkjx8h!o3F-+n0!7b62=)o!9Ovj`k9^IGw)W9%g$22+Xo@hVMiUy z-w)jl-1nr0?q+%_?+0%ntP^(nk%EhW*3gP$-R{z+dOR08d*vdoh+8eZv z1|4SHy`s6qOJfyQrePTLLmxbDgcvS;x_r%K|9=r$`P+p4AF#fU-VR%Ha?T@$?xoCN zQoJ<_b^6I@`2*;QiBu=sKiW?@4WbZ(~Ov? z4P;J+iw&^`P2LBg6s@P-h)%?+0&7uaDO*^vf{7mY0{W8^=aQY9No6Hkt_npq6;+0wPRy zv(;<8w&S#UnYRU8XKcUE%G&O-JkYKQYOH3l!6n{uy5E{PhMb<+y~>X9Nek);sv+3{dy~CQJ%f|@DC~a z%|4B&2H;6*Ffj1z!R*hUjX%p+>LlJ=Al`rj*!~009F95gtFi^cxpyZ~+Hp|2k4r}w zgZ+9l!G}w!1#x#dynDF=1^B32LL@!@*$QqjSEX8;7-w1{Ki(bB*8^&3A%E@sDH#__ zftva!=XikUa7{_S!cy(4ZPX(x(L<;>>aap|zsCHoaKMrb{%bh%?qf!1val;UImdSu zQviCRkZDOo)5+oL0w~6NF}Ki&yF%Z;e!aiF*jv5om|^fIT+-dGSYHHsz~vXoyTL5V z5$ujbPAoZfdIk8O#qW{&u6^Ox3TV1c!zn&TYl0h%vU!;9r>jF=C$qu4`~GK%9+$Q| z4CW~9*3eK>TZH}eR0g{*aC9=Ee43S|H3ECjWG}u0R zRc!y$ud&K5@BNu~TGw2?2TMy;QpMC@+R-0}^VC|ku1W>YIQNQLg!R|GCZNNNmy^^)(GX05IWUZ)r@&UNw2Xb<9=0FT^qX8%>$_qMl^*KhKhteo5?# zG_!wtC^kF?*mRv`Qq*zUg*qmF>f6h+WrT+FP@u%~`Xhkcy8vo_RW!62vO@Y(WTU~f zz&~L`oLh&?*yq5E%kD~NypL@19l0+%B}e*HjTx$xR4~5i=PNInYBl^G3`sl(FnU*% z1jc~za;karDzaM^MTWl^3Q|M5G;Y|Uysdn$%=9*gsV3C!r~HDHpxyi|{wN62*m}y} zlb6DYz8h3;;PWBFGfe^7x_I^wjIT(q>fsPxV!I$aY++<-g<|$M*p;!VFhS2Bh7Llv zJq@msP|^f2^Ojr6>h)ID(SYo@-XtW;Zmx;GV6{bW52GK0~GrJ)R<*$h3o0XR!hq9$fQM9CN;NM;^V1+m8bv zahNJ63!4#k8AOsHZ0zh5#Au<7t~Dg$@b5 zn@siHQ3tuu^}A3_5l5rpCOzcd?D#A3HhVUwLap4yf*hx?S8v?;zON`l68Vsgw6xomC5z^?Q!L^S#sx96YR3P)?dEyR;7U}XrPvV-#ShLuedri)khw68h6Rb zC-@(l$y&Ph>DD@#y@ZGXa#&?m-0YimD@5s{nQh6TIr0uIU|kUa^aiEX2`}jsnkHIf zm`>9WiQdRIb$VylZZ{=4jG$^Ja+Rs%`(*<&HA68MyR_|h&-f$qR~Pes)&$?WdM5%B z1Q?`8X0Wy9P9leYX=&;1qmGbCH2yW@%;Q=rshZkz3@?L(?*KLbQxW67(K}k9hKG1g zVxntOf8xR49@d?IhfYJj$}+D;f0Q1~r8w2PPm{!HNojU0wB9O8#;~Be3x1&B{&yg= zjr+SBQ}(w3L(5wb1%1W?w0gz0vC1BRTZ+AsPXl8BNpOiUfDR1UuQD0n$=0VNeEVpl z*{j#E)~Q`g&}mL~*N!A`1~9nib>JqS;P;Fp0g_=tyjUL>;`7*gF80@VxJv`oWL@2w z?+P&C2sFu*4(SGtqTP#RALw|MZDPK9HYvm3Z1s2zWD+!mh4go~A23V5zr^9s!NHNE z5X+Mz=|`Fa3>WjIzH~N0!4YL^Y*mrf|2^_aOis3q)y-FD<>CiIT6u3^k$!ao@Vb7i zy*N2o>;e7Z(LsOJsZvl}57JEL!-cw&y9{F9KPr%zZ^a5u_0Kyg?DcGH3MYy?7<|Wv z(D1H%1~q+&Y_*=dqZU1;VWQ?@`_pGZ?9wt7C1TUjWhjnzr74H)Q5c?Z8+udX>Gl@Ea!F*agdAIodJ&Mk9f4w593 zKdFsgv=r);GCd}%yt>nLn(x#oMLY%hIhqspKR4#iY1AMB4a9^*a9^hAgkADJKz2dw z>f3m#?}-(-(C4{A;o*=!_(KrjSlHD^zh6eD#q(Jt2stmr9j({*>VF4AUi(SIXa2(~ z!F~7FsMZ5k6h#H_7Ii%$?&Rd=dLe$~Yp?^He0vax*IZH&_Zjme-pgg3=KErK8E<$t z*-g|{sqeK@8|x5HR@-*=wiE8~@Mi$Zf1_;GHu7lKHv&$0k9A4o z%{xz4jM-VNiV;5=ekC7hN##`Xc&-*F#9bOn8X6j{XY7h`&#Ipe znj#Rt_@o2mxT(cSEXCe8ffD60hub{qMGc8dk*86RBwazzAcqu-l@@NsmQ( zeb`XLiB$_?|5V6gi}1-0ClWqTgQ*efngQTjluOQ%>C&3FJvly@6gX&CwGFcb`?q*xIIveFE=h$PkMGWsZ`TbDD~(MS*9bEML;g|10pXcnZmqU z@hL_DE+hq)h+6@*zEj=Y zn|=~Aw<3y8P4gAG0q0@foR2jg^YKe>Q+m?d&0^BnX-zd-C#s z#`A=XRexn;k-8P+4t16RMhT(6FFojgmBGjc8r7YT&JD^+FhU!ro}4;CMa6QLFg9={ z0ao9LC+uEGC-jK-@n(mSt5UUfA3uDH#T@-S}KPbx?xTDQ(6KY9vm8$YO|!?`zv>p%OmYJtvo#~-md+fEEZ zrAdbu(GhuGoB3louMhxEQRDzZgU!jH1-qj2fkUl?TOFyM&2z1lPo9`o<^Z37?fYd z7X^_Qt|WkD`^-n|AG?Goz$Bhsli}}k{D0|JFGVE(Kj>f|ZNd{1t&0t65$fvdit6eS zwJ$cAYCHRVMovZ+(|XVTnONxCL}@Lp7&&ES>F;)#OWS>mpdW+w5BaPVF8SowF1gd6 zeiSg{>I4XL0+?v(BUixI1x3Af@0G4C0Z|wzf!}S9X7Xy8qCaDson*_wf1p7`RFnBf zTUcnh$k3444m`a#yN!7SUAfFAo3A5l3@GK(gNOoV=uYAq zolPFMihyU!Q6id~_iT*)4%?G3_YG_XJqiP@(SNqph#ZIkwoUl)R^UMjs)743;ba zJ`Xz$G8|05xe9cT48gU+o&D2!6uZK^UUY-r_GtmkB5S`6tL;~aZ)PIb2eR7 zw%A|ZfNqs-fF({401fy)@R7+m0^L6^WdZ^#2&&7yL4*!1261`?FP#|uBU)m(^%DR= zPxziJl6cxqzf-JDpjw39 z^?!GTzN|}?$_dopQ$?K*v}YbC4}c3m3#kPkrv$o44*(0@e7%oMtM$PmW^g-*c&HTl zvWakVq;=^}hqFonu6ViCAluQ4PcMnJ91{4g5bY{GLLh+W7bIkTJ2dcgU*(2^dxQE$;17N%P58+wVmCrp*Ok z0HV7=4>2diJV60MJIN1U1PufifT;1>v#UQ=%ak6KPnku#gvWpq1#Dn_4o6JeslB_G zBX2ZPY-n5Aw+U{-7-!ZcJNd&-_S86981etbDynG}wGnI&nn_TWKBZ0S2(<<$1WAYYV*c4eoMK4XX}m8+haj zcN;l#RKyOlX!tEZ2r3GD`F z@XPnwPMvBuUuz!}kDZWS&RYZQ2D)!ubONBUDhjSmai8NC;aF_3p$OL!nq}3#x-mS5 z@lVR+CFG{mAFs0Ld7f^L@-XWYg z=B2tNO3!~XcDNFN7%l_PqDT&s$+KQJ7%1~CKvf56*x+Exo`Ku{)9=df%Xe>NofD4w)Q|80Va>dulCD5BfuJ0En7cEn1q1X;Xb(HTt&bFh9FLa(2IqxA_HXXUIk4J%jk9R^ZNyQz(?0 zns`&0jMYld|3X0cL)izbBPGUjTgQmIR?a9gU!d4x?^ObO#%&CUfoof=#-8Hnhe%Z# z#+ktisOgsj^de*T2m3Y3W^WWD^3VoqR8p{+A&Z>7q7I6mz`)8V@`*hON@vk} zZAb^z98_KtX^fq=y^KoSb8t>dCx-@Q>>CWs;+I+B7k zA`Jo(N=WAbg2V!(1*D}_U??f+E`ebvr5gpMyF);wTcrd9&-LK??cYA1bKcK>_kPde zv;J8s&&)h?KlgQA-?}++oewGW#2Uer&R9<_Dcv#jjC0&Z;~RZILZS8Wxx(=n_?w35 zr!yBOXf((ytcwzIm;VxH=7NWfq3J9UH7`PQC|Znov&h6vwUM~g+)pJ!E{|Ma>S9rjy^I{k{+;l9 z$RCLle#y?54_fZXey&6pqA4fGF}^YWd!N!=6iM>>KN0RdDF0ml>=VTw6Z87tT%GOh z+-+@b0b|xzJQ?>NBUcA3;-(XeWTVZ-b^qi4b$Nz@0kzH)=&p}mdN9N)_N0guxz}0Z zWU}5oMqjkhJbqUy(jO4Y1!;?J@)%*g9tUWR1&Fd8UqYNvUYC%swJn+P0WhH4zxR5) z+dnz4l>F()xNl+niD>u{I(Wcpq|iF%rZ&lS%trk|p~dK%bXTbJYxcnWu9dE?KIQ?O zK^;bTb!edw)8v8f8<-7rkj?|5@DMa>kZW4{ki_P3Ut$C0KafzN8y*<-UyV2b++yAIksaXpIl3!)})UL936 zGKvciew{hHJ^vWhZ{g3{l@L@$Ef(g`F ztW8u1z~8&kawYLvydrA?%T(glmLoF#E8l5SjSBei@#EN>*#2T>If~cWtwq%X^0g*pa_CGJ z9!X@hqXr4kBn`rZEkSGW4C@IcAlv9;5I-))(!h4C_@n0E3@^x7(VWOk!GwW2&lk<6 zRX)KD?owOrhZQz_z&G#^iEM>ifXp^|x-JJ{cow#7c_$|7++8+TOM1qg2&wOk(lat% z5&UNpV$LyQ26oNLZ{}?ab;(~$T)^UIf(Tp`1Gp^j1#;du-d3&&h2E3XupHUwG2}_J zfCrb*y`F1;PLNSl^a%4or7>f^2zV4e_npKaqe-&&;9HxKjVhNI zQmsgWR%|M=a&L3t;S!;K>gyGR%-+2D;LW>FP>SZ^yV-suw-Ww9?&lUY!)U?|Fw6;v z->uGlHvtw!93avY*(DLI7@Zyvkv%ri+>jsG5``6E8q1W6wBTLmIch)GF)>Xo^rVKt zW>rutxTMJdl$?(4^c|@oEX z2SY#T3Y34$zn1S4RX^R+HV9N(odwAPV3=bK8t>KBbp#Z@Vtd|ed;#=5G21h@1)K)E z+JMYG3(8ISj`KGE=FYmlBX?aGV3}FoKZyq*W=Nm?`?%2iaV3YOle`V69jLpSDM!CHmDy5sQqE)&FIQlXg2AaE zi!Img60jP=nV49Cxf&`vH*SrP0TYy9Z#h9%fHYX;@%j74tiN}?f15FIL@DIr=k4n( z+X8Rl*FF^IE+W^_T+z%1eJ>) zRE&rzHSlv3;*I>of9wDRaEg%g2G?p|roXjKE@PbKt0gGnr&YQXtcLR~4(jSaCq&Bz zsVwiAQay|kahBIe1ei#1Te6hFY?$R9?3x;>yhu!~FR4VJS!K`{X!`R#q;M$Qi-rAz zD$5i}UJ$Q4^gxe(8coRHR9IzMYQ)Sn()tF9?N1Rh7x(RWOde|gpS}^#M3=CGTq-ai zp;S(XK7)J>9WAI)0a%D&PC^mR09NLK0b8n42AUR3xOHoJOs>(bg6V3JirYo!sTZZW zbLP{Tlv8-&-_;eo%VmW@5$C#s&}<*oCs?0v2gj5Hw|3ASJIZwu1t&%9%UvsUsylh)Gsw92JJJ@VN4|cwaw!hJ@{&_Liyp%AwuM{ykE>xb?z+GU$|B^>Jw zW?I4oU=QO(#>8A`VPSX)-5qc8CrShsKORw|4T8quCPJl^8#bL$2Zk2`osq)p02osu zY?>1mz{h4ow|wdmL!_=|r0U&0-siwEa zEvx*uk>G^s`+!R@VPL~4o+w(=EwhXku#INb=mZp@u7#ua+Y2pxfPbqINt=StnG37c zXCY}Jbz{6N51D?uP7>Yu8wTh{sgd-SVuHLI0I~mcKJF~U$1XPsOg(LCA3%m^nhJ}7 zW@PZMg*LSctRiWGx;3Gs9<3Cn@sX5-QA9I~6Yz&_8d6f@ zoF%@59q@N)pBS8eC1SU;s;x{2d`t?+a+4Xad3+6IdTheY#@3-Zj}XWv9H0z2N{o-c z_FY8s2F44fj4!g5sz#N7h}i0Yl%>w%A2sDCVk2N3r4L2}2cRIV%^V)=PP9P#h1Ba3 z?i|zTgzOQS@U03`p$60u)WjVG2%2tB$}r77+~3|j1sF@VGc})?(_dPzjDCW8B_5xY zQIY{;V{p!T1bUUAKOYVcHaK*oQrl!Wtwbku2DKy0d?2VS^_Fjc@v&EfkinEh=qlfu zf|62s(Mb8?+efPX*pd5R2XocSuUUb`I_L`k6|2*HmV<95yfTe`0$eh>@K4%KrSXSv z;5OgcfK@cwo}S>n{Ps++ohRXt*anR9JODP?iPK1IYoPg5UNPXS8+?E^DiI+z-$!1; zaip$zGs~iBnqaEdUOF#rgxj{z10f49EPoDqOeCXXFf1 zOh~Xq#0@3_E1>Pke*$J6;p&aT$1g z_eI+h{jDATZ6+fX4$FM#)M0{ywZ>MO6V zWA2fOD{9fZ!Mp?3PF4%(mj9Awn5=WJ$ki%Jp?a?2GigpgX#U|FZ1-h1aF%$~w6vaU zWh41%{Gfm+wVkH^hbt+j#bo(;Z>caPk#~aJ4f$Ee;~Grk!DV~?KHNRlBq5g1f?+?) zGsm4h2&GgAKLQE+C{Y_HmU_lrHB#}|YF_`d*v@%|_O;KtU0>YetSn#Wu{69R5nOrhI?RBe!a9>Uqete zS3%R>WfH4rDhdw7O~gt9btGPRy|PKyZ7B;uiRYZ@fc!n4(BSdRLH~K{?eZvAA(!tT zuO`A+XUH#8O^Km6>5XIK8>E&)(^T33<5Yzr(a^3wu``T?!r9-U2`^3c@%;iyrvkjX zmBJc@2Cu`_ovn(U+-YIQU*ijanHoxmQCklfm9b}i6e``wE39tkHSc}*62;XRq?H_2 z-e5Z|YWZhze8}NMC*)*!k*FSn$~>koWfc`K{91!q(bA@RDf`VF2^va&)e`v5;y51z z&No^Ot-<^&`CD9Ie7N9=8Gq7AVK3EB{Qv?vg?&N80OtENaY zQwFo;L)9fRr2MiJr^4s!yL0q&CcOttRh$R{*VfT-{xSx-;!@JjGN1QU$FZT>i@Q1; zw6wIVmAp1;(;PtPA1U=od&d0S&W^MEeH5jBym+2fl3?^NpFjs;`}v|1a!VC<>m)Vk z2R4F9F(i+4vh9KrqUIqWr_7j=%F7TdlO}p~cb`CtJCkYxA=Ebjz^4idwe(0I9{72m zN#UO-4?~U6Ii$*W44Lfv*w}gBzdSVr0%ucovBPp0(2e}k)r9!2g)~#xjF+|@*#P8C z?3domxLm>*5r}E#W^K=H{k!+A#pBmkrLL5PA@sV&*_lw*kgY&BKu8nI?h7Z2pi zcmuHW<|CWkq2#_9DMw>k3tfq3Bd`qF`0$T!%2#C+f#P}w_A~H4jrFmAM?Q;0ITBaE zDi{mh+x!jk(Da>X5(Z+7+{z&TI&@D1g?cq_OzuYZ)c$B^b3W4#Xn|k7ys~mdYVv9%ioW>8iH@AMu+I-6O}LDXhtl1QLNNif)-& zj~2H{#D0DA39+}#?gyyE&kJ-K>NSOom8v-0sq}974W&U*zh+XoMbYg+^&J83WYX+pz*OBP^>fml=>kp<0;AGgb+s{y3*%9 z;!Au76fGWL>5|s+8+D)czpAicNnVrqx{cL|S)*0o1J5_+Q28~8ZS?>jYzcX9WHRH# zyz3DNtZF{NYE_V#V3}&+&*pF`_J3ZK#Q&?h_Dqh0kd&14!aw!L5Ot+N`kyL(ed-2A z_s`?_UspePA@OUlJ(2S~`TQejqb#5x*o_>;u zdwHkt7buP<01g}Lv`o`pl|`*jRP?0Wx3MV%LrC7?I)w3Dw&A;ku@}`U&~cjD&K4q& z5c~wBXwBcd22SpU)y|FDvdgmCWe!ZG6g2lV1G5LG)kLbyS2)!aR!$D zzA1@ai2@ZJ-g|Rb50b}gVV)f4m$Qyd$*~dMUqk8yl6WM1yBVa|H!P93lI>OCRL_;g zFK_j7OSNuyCX2W`2l{@2&8rjS-)18NJCmN7ZfwMPX4T8CfQNQLV&AAIq3^S|Tf`sc|Ui-fgqMK#zxTfYAv$=j1 zgNwA!pZ#Q&3?m=-DVpJ1rWCw2Ro3=i=*_tgRV>f zEcU1zdMQzINw@3*?ZPDf{RU3Br@s%chFN=-UepR?FMAHd zdby};$%XEmNwrCJs)23I$v%4~@yxxD2wy{98ZI5Z5j(j{xReh1Z?%moUf5H`;mYZA z_gXTo5v6fTO)(exw;t0A1C;c5gP0L^pN%VjIyYry)Y0Y{XTDmaN=PyJqe)5V4Wye^ z-Yw3LbbTQ|BSJ`7eS+7tFgnrG>WiJb%Ip#Pknyl{kQZnDL4scFI(+J;+|>b?#Bs z%X39OWCcoBVy{rqgI`@iWXRHZn*yuT*gHg8Cb7TT?BwbPE+}f(iMxVZpynX7ZV67( zz!z@;gO_!wQtR8DP^wMQ;2Zc7L)KI^5`B?Syw8-sx8KpYI=NH3;;Js|6Iuh!GylCT zeuIcBRCy8+El|Wx6Ew*jfS~XD=!^Y`RT{y6ZSD(C+9GvI_T z0ztFr{_E3s7xIO2gy>H!Nc9j1==X9E-dLOFAcQT;@X!MP$>c5sOv%cg_BG5R!4#^h z*uEk^QN)S90U>7RR{ri746iY{d%XRB|B#bOd4wLY|s!jQ|NU{Da)8e@A+Pe7&-`Bcpv z=Y~$=L=&X!EKD|dyh!68ohWCul2P)I^-HJow6Aw+F!$Ck)hqh^lAiF;y2d+Sx3+Bi zV;L#VnDwk}PSalbL$BXGY_Bu^^U8awE}n@m@g*ZW=<8nUh4DgC4=+QM#qYbugih=X zj@xyTk5rt`T_wN9cDvIhe}nw}&Un zfi=4^s}L)R!g;Y7Ef)F-B;*97I}FRS5zf3V2pd+yuBp?j7%$aEOm13N(BU_I{naef z$e9p^DkzkH=4wV1!bs?3o7)X%pZpyV93VXFfc4q;@J;_kqLmXZhg(?>;s2}gjU2uN zA2w+4w&9{0zT*H3WxdR31It)=&c{$-RW zvpS^`qS3kTI)xiUF}W?eYpJ4a(F2MZY5T9s`|y>v@8yX&OuR%h5o69t*m^q5{6yP+ zGAk&WuDa$(HW#RY!coi5vJ|MTWv8sHYg5Xia5ECtuZ0xQTG1p6zkJLa75c7P8(e%l zJTS`Lz!|lX<}fIdRsJc23(c!c)i1X@SbK_u(}phUfp7ZmQE*GSsE%;@LY4y!ha&TD z=3aIeIO0%CmPFAJ)@>_VM^S1ij?xWYlw+>Cc42OCW$aoWXH=`Yh}xm5Tf|} z{MWb@^04gf!R~5iSGz_93s=jBe3`k;etUlDh5nL(>!&w7Z!Bmpq&uv^pgMZ+*@zgz zr8uU>_tO?|bVUiz?st4z%>W#cE(V#Q;;z&!VL-+x+>0n4^SG-W+OG46ip^#f*; zW_NiKR%*e1;cojT18!qWfmOi@R;DbGvaL=)i)DOD`|XKiZ0VBADSVx0KQc-;#LxU` z!U1kEJog7qf^?LOY>)GgI0vq0UdkAzd4vGBS>F>a4Ttl`589w9d=HS|7OQAJ36b@ z1S{lZC0Bk`$?AQ;|3Pa`Ao@7w1^tYZvdZF7;HmU!x1x2uC60CuHswT@eacMdXUU87 zQ<9p9>i`hqhgqT8bEoj4W1;Jf2isuW`4f0gQhtHH1yNCELv5R%ZHpx1Al!Yu!R5X2 zZI>kYt`u2fmzC*A-!w(24HkX%r}@4%kL3}IFYRSoYEznI0o#;45TE6ha-xB&a}#je zGT#IpZ32?P!w$W4!ZFr$LEzKhM;pJ7dgz&$&f3gC?CpHbMmoN<4sYe!l6FuZJT{9z7bJ#`(ys zoxwv}4-(J$BIYwTO;k#|h{;jkcN(tXMn&1gPO*jqGxJ)*AGxz?RNvlqdJH@jPpFy+ z&Yb`5+-ACtX&|q1D|pq}>dMqrlC0J_sTh`E6UlwAgInop-29FHE|)mIEWg&eXlS^0J)H(Ag`9^8nnr4J5_XK0+pK3HI zkfsvI>ACECYVLfpdgnx4*w}CDlXV;;yubKAe$swgTSR8#F!IG=LNMHZm|DK&JJag- z)+;syVzEl3N^xdF*>U~^QY*VW+!r>j(4vNwF#E1}7&r}S@ZT$jasS{DhUiW(4O=6Q z1c!xW7oEtawc&x2`v6GtB1D^03IIze5_Lq?0=$US=c8Fit4Wa54%coEKQsj~Px=N} zR$Mm`vd6@J3+7-l9p#oRnb0IZkWG&%0Pkvl8tXk82r7MB**$0rj6hsogE5gQ<~7^Fr-D}L@!R1pbAC@55qIB1(!e;Vb$Dk}H$a4@J zc%pRx5J^(TLATv?>y_Mm*dd&2H$tg|A1`2*ea3xQ)U$nk)7L$vDjlcuHXGkMEGSLP zV{hTu2R1(X;*%aTp)UIdGMu@$@*-#9dEx_oETQle!rb~(yS1)o!pcr>%N0G`QlE(U zfh2#J>`!E+=dm9Ez;d!EY?$KC(SscQA6sS$ z=O8|G>Dt-W%PzI{M)aav+KrXqivP1n-M*if$a>2nhg!x8JGBdi-qPy{rnD%UsRLkr zg5|Rr;KSuUjlq-)+9g5V^z8lXpJ81MLd1gtKz9v z!}z??WA8_JZ$I}C7rtnjtCB8rd;Ud+K<`N(1YchpwLc|A$ndVXxOnxj;P70atkkUT zpbizerF`q zCm))LY3*FLpXGIx4_eP>-xc^sPhc{s<;*d3Vne>YB%{ObVLE42uY-~0T47Re@zt8w zajt%iD<78rVr!&-LpgIbm79i0A()wvL7Xa>;PjY7ayMTm$#d(LUwkP`yOVS59piDrD(Vtq3ur$r@B1rBK>Z#gFj$kZe?;lUJ${ zaAQ8($YxX9;Em`LE4i35W_v45?6WUt1(%BnYA%yGr_P~kp%l@kXK@+Xj6ow&pkCO( zaD~ii(?(w8lblolJs~KnF7HDKt9)X{>xLm?G5gt8fToDZ#?A}nd=1sI2(XDy;t&+K z?+mJO{(h5C2E;8cPm*w## zq;69|fX!Rt7IT@~ZXazP+~50DmCl?^2+?T6#b)~UE!}>; zL0>XFm2FkGN}(`Z!y1gv2B>X(AnKLT}z>^pKs__xoF2K`(?8rJU8v?IR%+jGJ$55hM{`je$_H| zDb{2bGCp&WgU_Sk)|tK}VHd{fCme@l8dX$f1@J?E>o(uomv6l{cL`IBK*l?dJ91Us zqTC(<gT9@9sV=N9h%o<;=sSpTQ1Z>0kTm za5ANI?aL~!Djgb|I7(}&E*|YKh5db>!g$duZ$F*a%;e5bCew&__#~L+CvH%BANykz zSUO0u?zZH~#LI7QotKajjV`Cgck;q2YF*|#msO9OVLz8OWBE%tzeA3WkdI%r=cRcg zr&^G!iFky`5Z5&$vg^)KT%Vg!F=e-3YNM3KL>Ux5^Ck2#(_geA%C&B0e95kPxgBGJ zBHw(E%T%3NGcD{O2&ou&(DcQ3vE`|= zxK=M&-*ll%gY~!RyU{dydaSve=s_K8sY?Lk1!PjBE@p(0paA}$^NOIu1pSl>MlBzy zx}$uPxSV{kq#weB9uE@FhvvTSkYgnrlAWirdMw-p(GR(Y5zR1XZQq@#O6Ek*zPac3 z<@w8#Qj+>J@e8IOg&#h-rNKihNQ+~Rvfxg5(8l(umd^2(h~tkR1Dd*Pw^4eipzJ1s z#LvL&e!n_pNY7S2doS|LG;dV#wh~*eRjE@?R;|gjdja3M zwd@vivc|T~whFO?7Eu`$l}|a#+t-WbzB!~AIX?|Ojl^_*^`)_Uk?)_b;MQ;aK3tGW zwwSu+TDLg9$K=n_R{V-UV$!?1d_1PGra2<7NoEbZ{T^f`L*z~hvXrd$*wTIiGEByB zxQ>*se&m><+)V;22uWN%855T`Ez~y1epxOkLsq;BUpE6d&HTTFeXz|xE$FUpxS!{H zAIw{Dcqohd|fG0i`c>rfL) zqwq&5qAVZ34Al_X8INJS`z=WCH>%0JC9h%b`qI8dQm9%`Z6jxYS+JR*`NX?xJoh%> z)3O~-rOR; z5S=XMCzZ^=Y89Q!$Uv*}fG<3}R|MOdsK%>NwDn^amfdokkpG}FR#fQjyvq37aePHy zQbiO&;O(s1JR6p#_gQz(X1%hEA3t42L{cgwJz}*Ezfz$#O{V7{gJLDe2^G!^<#(>n z^|mL4XU_G`$|i8Koh$oZt$BAX)VE+<-O24^@y#oS%!5j-(HeiLEab1G$$G*!tg_zt zWWRgq`HPKhmZGa{1SXmMAwdV{X`qkMlof>cs)4JDIfv;BNybVS*er z=#)UhgtcK7(T^u?zs$UXKPAUbSn{J-VKTzWn8Xiueq-+LM=BZTnpl1MH!oqL4N%px zJa~A90i36vEx*)%ILDY8FX&(jWsy!mY{Y|iB{P7d$_0U2R`b}4H=o`WMziytkynpt z$$OSZkp08X7(&GG@Nhf{H@oqhK#K}{$6~K9&xV3`%deak@5WPK8fp$FYPsevl&pUf zk|W05OyI;jq3PPym4XNfR3=qp1|i~*qcwk;uiedu$w4XMkZC8eld}1j_%?~ZRUzDk5=4CYAI+rt51?CE4kpZu-Jgp$hMYX_^~(Ue_xyR2Z4i2{)+*xeW(Fsg$0lRkbsefy#PUhJ`=2eNyhCY(=01gn+Vi5fp|4Ub8X!kcC zVoA2QFP4>)Yy3AV*K7GYE9=_7MPFV8(r#|m|99B#-|8$t?zk}jzNTn%xPUXfQ~3F? z=T2DlkNoU^B2;5zxj@0s3!`S%;GYY!26IA$S=GxiGRXJjnU`wCxoH(p4L$^q zvKTMTg+NWte{R!tYTx6`4!~OqE?ndO5Yz}m{c#iqkk04+!7o)N{k~vTGlOnE1QJw_ zF#;CW@v8+$7&6m;h3Ja)Hyen)R^M%j!C}&AB%Z>i^1U?0?%IUbqe;M|;#;WQJ9S;> zqMa!K>H@(Iuv5H6_4~NV7`iCr__Hz4NiqDRak=K3?yCo)6p2*u@mRH77X=fvX>Z!y zY&d>sK0DEd8`1sjfq+$_0f`J#lp3mrK)z&fvs6@pCTg`>-;nvl(;*G5co0^_5#NxG zL$b}B3hQ>p#ozwXx!}z)6RNAL7nr*Pnzp(C%5fxfZ1deoqBxKn_w-up54+jcVURby@dB=Gn0x5XRE2~T92Np^9(C*ZR*1XeHz9Fn(8)g#e)9R5HW=?+ zAt7$97!-gL)2w{x)cax7dXI!bl*SdS=NwP=vRM9B)rVS;%1L=GC^Kjbs!mXX2&=89 zhsZNEHumbywLjnJtv8&6+*tu~Zv?k0>PJu#VkgfKzUjCu@(IFy4r@Svh)7^sku-=S z{k{*N>KsHg0R;c39+~eOjsl-w0wbkW&=p2{`naT|q&L&7YS?L@|H@mP5oj@XZK`e} zJ8XFaFhnvoLaOo9W?Cd}$5}7L5Y76>@`V=_$@!9Z?`13qz9y#OZj`qcL)_hHD9atE z<*2sy_TtD+52(b*yOG)Wd5;UyJ8M$mHPovAe2!>YREnTO^C$Zbb|eCZAkjlQH&aX3 z&d;;Ux2evHj)#Y5OV8Q4yb2U{O}<}1S!=UiIdieb`o5c+8*|V<-(0i|DhNq42{ie% z!ip$8TFzMB$#6N-588o;Z?PkKNZM#9g$IHh)dR6R4Vg~^cR=xD`t#eDVZi>FEil`tsDCI#!j+XY~pss!v{&9f9XCkAB!MD>(9ft8$cWT=hpsL zJOl0wC}p@Ie2~nIu&Szx@x~3GM(ix)8{RD3B|^5HXRK|A&Bp@gU3`JC>2bUa7v`@(qOe@nDluL!E0XHi27=oGq7~mUeyh z06@cpV>OQkVe|~32?#)Mg-AG%AaOXZgiNR2>(I>s>N+so-r$gte!YXqE3@lf;SL$JVK7NF{~kh z4Yv&7x!$&}ePn7nKE3?=I>ZKgFX;u&L zx}}|Gba9qLpTv}4lnNiD!X}cPO%$-@hXl%D?f$w9rSM`BOjg0cT`EB@!^L%n_$2AT z%Oa^WWKtbSYG|rN&@4-k00LjaZtCxA2;KU<5e1XNX*u*teW(TuKN}F5F{!C-S5NiS zG8Vmn3+6;Gn1W8Cy3mA-fzX&e2W~6I>3WnjDrU$5Et-k!~ zATW=h6v-M!MR20`o~rtG2v+v3=YpSQEZ}Xss_JE_XHqw}#k=FY*vu4EQB+|`Wq|d^ z!+IR!muTSxljDJQwlSOh!t%XhP(D2z(J&wj`6it0_8F(tVWIVKiSp`raeE7RQt~;! z6+=9y%HT&f!XcKgbw!|6NuO`$$BJ4dr&ZJBQX>xe;cJu1N7ZB?4A`xrR9xfGS4~Ko z{u2FkfKQbv9xy4i5(z4Zx0#jD49--{R!*hfNlx-MvA|vke`-gp8(pLJ6TQwYa@I-V z-hqbu0Vt{cxrupA?N&$MIt!Dy|A?ry4xjK}ekS0F?y=u}S#YsYWj7;*Yf7fF-*h{< zeH;KoE{(wkAis9>)$Ui+6nX70F+D7IW3))bU3D8?DiI>uy5qncoCf^p&i+M+A91ce zB{C5eIMCqzyVnieo@KkG2E4eF_Znh*X10AaTpWw?M6v6ZBBymk&1+~;aiBR&ctZ{-(M7-%3dq^^&wtC!pQekzn- z>nyE=XDTM^Q*|}gKOG%#TwDC%PmZvLeSBIRRc?xhKW|9&wGY#9y<{ZZu8_=CW22WD z>}9jWHvu)oR_nDN(swf^)k(JOUuu-II@oSJl3jUDopS5r{qB&M19u^fg)Q+xk&Wg> zbad#6G7TY`OSTt6j=a40?+pZ}vtD; zosTGAvX#$BcCb5&?rtk%fX9@_aFOyxY{onp#_f%(6oyfDemM*hf@|T{P-53&Fu0?h@3fA|lzR`uTLh7>;xEt@v zirj2iSQob~J_#Ki$eGOc2m4_LWF868e1!7hOXpTy;ge#$O!-9g&9a2b{Q+PH3w{DD zS44C3#B!)uKmVb%Y^lSo?uJ(l3)|=M3?DEys$g@}jqY8D+@;N<4u=MHH04@+l6pm_ zw*>Y*y1T^XU|El}8ejLuy26Fix~SP+Yy_GE{hABY*Uk@$u&A?q`pXt0&N2+_!tfP7 z*k;Z%Lla-PU+ONGAx)%uD6JH;E6Q#m!W_uy7+Rk6$g?Oa0oTiNeW6kC3wLb$oMa~4 zYLXQ)46^t|YBEAv00#TljSkR2?W@bqP)Wu7%6$(-Lz!Wl1lD?XhzkjklzDBBFbB!i zsp{;7)bhM@X$Gk5*ROr?AH-+)+UTE^DnajzH|#8}y;P&f=f_o_lSt0dB9xuwMK9`1*>)1;H_*$@%5nV+}X z^#+>NL&ewD`xCFs7j%=n8E9`B5M9EU+#>W-V{lX#G{nl~x98n?s+B&(qe*UvrBKTy ze)2_S=Bs2{g-btQrT$$N-LZYBK#w+jU?=3uZklUau@#wpVM^oEe_n#t?A~r1TanCe z6ot=ByLR1xJ*V!q=u+w^NmP4;UEL?SI(4GlW-GL5k^|k{n@oN2Nlx3%%lopFHQ$-N z3}qf_?uGNLyM)e)%l`e-FWv31<nI5LF<0u;Nwf_Kfq6m}A%Bh-A-|*dKbJ9;<4N z*eNPnOcpTtl>Sz4%JGiVxymh-g(P7vtkTJ{Sg#W~*i-|QrN|l&bMU=;YK?tmV{$xb zrWA?#XrYsNk(`b_e0vGi0?Fb#06FCyeNTy4&jUznM!GJZ@|F?Sp3tql>U${pjm75a z1J$a70e2rdjNJLqaz*clBx^~*n?>kp%_4op%;%i7oFHnBwxMJq9O}*eu0a$`r-Jja zkNZUE_4b?uA6~m(Oi32?15Z<3r7=3%oHLq46e8wPp=tU|yX^$A6@~Wfg^x9C?1OZD zUh}eultvw8_!m?60_J9@cCx@4M?2V*PdBxqKIxaf4q8UtLC$`WEbf6(f>`q0VioP0 z*88$6cYB-Er)%D?+mM_YA{7)JuMe##J})7Ht|O-NPdYb3ja%z)(p$*7%rkkv+b*3w z&x3hCtyDPc1x`ViVjEmKSBWt;omk1VjHcyDmu*@d&z+G&^KCrCr(UgB`$TdU!cznb zlX!=fFT+QSX^T}gdp<7V^m8%And-FHAfwyB_b3}|xn^kPxQxhj(q|P(nQf`LLSuFE z*Yeu%;8s{?;V<8I)E|%OzQ0zyoG#>`_i$NHD{>kgYnMGGRCsW|hbcbs15|xIf&#iq z5{K#>y&6i`H^!>$+*u{|IOTc<`C&`ACmOo`U50AH@+6uAq|SZ%W!J`Y#7daV6OYR2 z#|5u08050&-YOZPBVj~IX({oEtZsc9y70#tDU#GoN@Ob;PT3FtSlg7!y#Z0$`sEo|!5qt^32bQBly?K2> z{1$jo`Y3{|=1S@g^W7++le4whS~O8@-51!D?-D7m4b(kUpt^)}upcfhR8X}ViC58% zaQJd&$6Yn*Zp6uvftBUbBduhY(DT;XbCNUlXs^Px&20Gt1FiHa_vqwt0nZIz!_xiL zqJzHW6OG?~Mcj^*(Q>Vhr_o!`v};r`&$uX3g66lM*C>o=7CRUS)~mtpzvh1@#+F-b zhF&kEa&n@VK`oEvo{LXh(u}wnn_bv_O3R!6dhIo_11Pa+mUUGXSQT)=ZTuV`v)Da}Ub3p|y&-cfa_053y2I0#;mO3AG4re7Jx384@}Vl;1JbL$(C`%Ws?@Qv^Ab+pG|g zfeorF#{<6Ma93m&4v+yr=^<^-L33E487RdRLHl+Iqv4yz{Sb~PsRAEX(`s?YEgtaM z-&>?U04pMr3fI{sF#Y&<=zP?vH3_G-hA_6m%Eomj0fCBEe#GCH0%y@319u2e_jvTu z-ZknqYL3E-C-Knq$aylN3KscyKp3WI-q3Kk^YG*OrdY`CD9>p5~Z4o%!WIFo9ol)^>i zo&0ErL2oqBXOS+-dmfmd;J^b58$XgFnE=L>n`yk!8kBvvPVZV5v1KVtb~CwhzyfsuN?2~`0IXW z#uN2^#?79Mqif{0aUQHy=(8R-d8T=(!%f$Je#N@h>J`V=&*vw)*iXrBKF+!9 z;4+YzJ@NWwZ}H24)7g6gjub2dZer|$)(xqpze=lezEzNDAp$Y!UV1`-xd9ct+*`m=mh}61^Ixx!8Wb z=$0RmZdn{H6%bIk?bf#;N*xCxFUKbdOTueKt%hL++zP=FEgcG|c8l-*poZ84?tG$w z%<0FD62J}bIuo31=xzuyLG}YToH6mH3ch(*pB|+9lZg2=^93;O#Z~Zq75wb%yyJ2a zEHnhx-jE+NY^YXrfM_i%vZm}>zzo)9T`_5jpoB z3f_ld-72?aX?&-Jp9Im-8Lbu;7P{z>9V+3Gz)eV;ylRJ=EC$PqBpIIlDxA<@xZ(yX zvVu$BpwA+zE-L*6cEC4tX-X7=2cV0X{cafq6&ORVV@x62oyQ<}X0x#_m zGtUfSn#m;XZM{d?fZn=Yu)?qRp!#l%Q{f64R`d~XHjp%e0s+leU_ zH|);~m`y~r1nr zqK}WOaLdNvSMO2uP&F-6nRQa7(v&rhBnSJhjs0?9Ne`N0S85GqCe+|aAX2W#5Zq{p zcR$+v!Q-(m%NBEJ{!rOPhTqD0cC?3Q#6ZRbylCsjcg{0FV zh$xuKYL@_h(91?kO8H0u%Ts6OsN8xB1RqE|dMi9g$g}x4s?C z%NoA8wR&Ru#!%;jL^7dRrxXX(4~3JXGhjFlm0wuNhy{ffRWMz|{r>JN+S!{<#Q#@q zXC4l9-|umCw~-P?_H7!B5m^QiEf_luMkq^@vW2qm3PUAi&#sx!$d)bHYlM(x8oQ9l zzGTV1obR}w=f0owoO7;cIp_S#^>ew#_|5P8{=D9=&sC?_=x@{V3+KM}eCRBexjrKr zw9FFX7>3UU`51Lc4YmyhTE^t@$aR+%7(I)fhvyk1a6)XD?~Qm??vgipw!(GlRt=an zbx3Bi@zFn>012{tRdh3G#5V+d7mKD4k>%8hhq*Pa3;YJ?RHx6l7N#nvVf24p2CYZ< z?9eu#DCdSb*0Q#A?qqk~M12p2h9YKw!{$1cXcL7y6B`a75n>TPn8*dDHwHUoHY^E$ z_W&JD?n6-x*;K<9)#HX8wyhe86*izwY4eB(OL{3u$yv`i;NLK+66Y%MN2`CWmiwh+ zfL=c*Dk2IyxkLDAVYbJ@rZ@Yp#B!>0`-ACn(>Ek%X|x7pZuOXQL4tSY-Paoqe(`-b z;BAltD@Cp?iIKe`;Fa;u@uNlcmWVFEL^b>9ueZ~MpXig(gf9{`EMmLaHP{e{DZ!_U z=`Fpe30!VXjChQnBwQ3ASc0X`W>dZgkJOAQa0XyvB_GQkf9A0{o%`N0=Eb$K(%he_ z^goOdsHT}0-~_{>>Oo}n0aF@&Y;Wb7h156idT{WjzRm%<9iW+QOHpGC$y6RalJ2)^ z!nyOb-WH>npx{+RWyzJMYSA*Fz()qMD_TfM5=?!dOEF8mKv33+IUb%=u!L5z7~TQP zrm+nAgLT)%_`QJ0hAuk{hXrgaRGy1zG$~h$TVkS^;mIzb(n55Ye%t{9oN)lBp4ig8 zRq~FO;kyRgv|NUDP?$tztPsX*?19a;J9t(g!$Nf<{{dk1*zZNwL55Ig;ZpKpepqBE za|nPT6aZZ`^Fb)!<)UXHU0s@ir1n}9dqCra#Z|IWxSnfcncD+L44`!^0C%~{E_#b- zleRjD{rZGR65sHw@yc~@h6i^&Yv9&Ij&I7pux2%H+?#Rxob}p{Dfu`*eejW=%@S^U zg5h(^@~l6E>4ERZ4oC??#KQupBLT2^^705{tF5-SHWEBq9p4IXDEDgA|F8i<&6Od@ z<$D`8W~jig@7h3D7}t6`yRbHtkYj3!L1$-h{3e8y%4+zfs;PnNRY@ikBn=wbF8RY6 zq~&5NS0~QUOS6YH=&LEMjRTOHTc%GNe5euHv^nm9IqVAsa>q&VmzK*#V?FssFEd8< zcv@Lj&f0${FZqZY!x0JStR|aV0tNS%Lp#b>I+|Lp-q0%EjG1#k=Gy<2dFQ&lhjPpf$kHJ#lzZw zanic+r3MV%|7z)Yk_34HX&07s=+Q1Kb)ivu%jS;4q8L56G1~_x&r8uV#64pUnGUcA z`>bp}&r=;|Eff(|WJHh1w(Fio6?n35bRVBgTh6jloD2AU%nCV!40B4nNOk9>d|CZ!^aXYhT$*)DK9Z+{=yZFkh5v|O z9`ghq7A2`pDENd#0Aae_R@m8aLsm@z`d14nJn(h%^d;{?0$Rq7LVQ=?OCGw!26Gnq zE%aTRX)=q^kFivO-_)1PZj;|0t09l*{IalA1L}qs-_co*derj%j-vq(4FG(CjfxX4 zc0ilj&>ZJ%W+*KL~RAY9n{(Gb8Il!~K(0s$inD?xTble`*l7(tpH zDO_wxPM5yqcKs0hZ&;h<*C1c$&(rsqu>Ta1BdL#5DX2pCr#9#bmxh{1^7H6bWPkDX znE%>T4-TSs8XD$Ccf1lJMwkn9P}A&y^a zH2LYdlkShFPAw$tt*l>7;M?{ndyp)fRfX;5Hdk}_ZU4IS49Mjzz}>Whse7~Uz<)+> z^yhy+OTRuS{qBDOob1aoX!!hb|JOCO-+#Nm!aJuz3L`Bd?~Mz#NvD6_{n*KCFx`;kk~oOUqI}Ucb=trB;WU(9&5^?ddS$X~D96WW^XXDcGi5yWV&iMq z4?OB}cO7@vk4baUKqbLa(Ph8{xPTCdZ(h!4H5aT|(az3Q*Z|L%I>3jaL)F#-v!94L z+q}lGB6tve9MCV?K=0MJ{sKp+s)H+KP^!b<_7_UrGDKja`%N9@Ut z6-`vBA5a65$0jBo#q)xSUl@;Kzrw~?4Aqc=oe?vy>ZPs{0M?QG!o71AH7UoZ`GJn^ zbDH7+o!p?TzpEk`8#{7H+u_7>p@-u7sI1g}$BYW9cmYO6X|0#)xBw&SJB~FQR$Cs( zS1=pi*IK)NrcU)N?w0^mfO=c#Fd(xG6i3N`!fB(50j=snCfn~1j#AHf4xniGn(p7t z66(_oG=i}j5zBqL!dAU0F@gmhwZrawJ#-&Ist}(j?hBS*>X@6Sxm^R|4|~Vj9d{}J z?e*jC4%C%}U|g^b0!TI7qsqf?@rcr}T-`(Ydu zVPvURFSjc*ld{12K2F$u{{I#r2zM^70;5)c1oylJ4N((Vmb$>|#}dr7H2A~s|6bK> zwxj4TRkOUv^2#Yaii5tYjY-4QL5i6a<6Xyr;qlm%hZBxlk zLb>Q6C-L?%mg!nv{|mN7F1!63TEm$MUYy1MOwLfGNMT|b1~zAlo*zIuj`<020vW0q zj3qrWaU!2;l2IB+7sm#=C3MdC*PW2wTgRbi{R19XB<5Hw@G zWng<}_b;l|ivJs+LU>&*Y&U?G^X39OtsCtoAdztXYv6f28JwB=YG3IjTTv}bB4cjn zc;%Gne9rzbF--(kvYw%HeSH)Crg0`pN=h~x07kG+J)iDJ4C`8a5g~V}BE2gFQQzzY zK5UPuo-njnxgkG9LnFr3T(b#+80yB2W`#O2D=e``-?M&ruVgR%G2#B#c$bK}@r+h} zy-9l#FSDlGdnU7;EbD&t9#U)Hcz1h5LhfGZ=_lC?o4ituSI2K0aNn9xQwp3f=AX*y zd*kXNS^0ieF4~&MHWk8j>k4CCe>zv;z;nLn%iC4={qS>|9S+)`3%BCLXB4H+l%%be zJ8!Yaq~5r>CUE7Q?+m+76_dlgU)m>Vm1)a1_E?N_JW?b)eEmD=EIh=R-dZoWXI5O? zc8fMiL1V&NO`qR6oH`1cBiG(l%;nS|dD9&0U56e>ZNlh(Rr+a6;D)nFqq#TWp#t&k zV;oXB_mj(zazMVtni|wL7jRlbQ2I>mXV4{JkW+Bh7Sc(=K(TOZAyLs*fK}9)g5D#) zo)DJ!BUIn^pGY9$_Y;T|WfTW+j2quhv|!uROVkQ$!3N(C#Nl859zZ06mSf@{5kzJO z(G*gx2(US-VoC?ZRdbS=Q2CKGKmskW4>A`~5ln!V)Q7f#gyO633Rw-JPY*G&Vp;-j znaA%;=UzP(jCbc{T0<=GVa@>ns^-lCFr`01UzkO$RUFFta_t`Q@+C{=^LPT#_&vA* z<`o!oVNHDr91`iEwh`b z=tJ?A+usewM+0m@hH!{zsfXwD$*0II3NZGr)<(%LZq0Kocgi;5e^m(+<$@o@d9Ivm zyhg6uN)`&Ht$TCS_gQDhy5fHK3XbS?oLz+WC=+yhZax@UJ(-nO8ig_AE%W0p$PapI zCl3ll;}&`i0yo-vC-JN(y=v0^)y=aa@ujV_TDoy2hgTvbV!Hs>2efuj?aL}a+2(Na z$)z%F5*qLRNY?oWYl!1)#N2wqg&z4f0=SotgD&_n(U_nh@v~_FN$a;s|2>tcHX-A% z0AgT?Wtu_1C847(_Dn&mvNNUw6ac~0TfG6+eD>F}kh^xWiZDq9&_yLiXxuL_Qpgj> zs2GzNn^oBmhM2xnKxO!$eU$W8|9j56ld<2YgXNDw!wpQ{+zg;Dq$Sl*JnKOF^U%E7 zC<}0g-)mGv#+3 zXf>46k~nN?wxST9m*b?EJ0^d*x(GWS0li z1If%F@1B@mLriC-AL?xBy?Y3v1)Hw9Q-4*@nlT0ohe&LXwktUeK%2~q0bC|_3*M+N ztZ9BU^E{R)UZGsw3m#AyXu*=4y`%#@B1d*eKfzFh-Zijhdr2$#VyO68ZjP1>sv;g! z2D%fp$K_I^8ni!b&?;x53QURx(QJ0nd=5ry3Apn8jN(&aEr>Df|K*edWqBCv02z9- z08QdQEMdeg4WJeS@KCnEb{}!br=CT zFLn*v>!KMlby4oj&-LvY;p;?+(E0VZ^X=gXdX~PY5Q)xtTGu--uz%@5p&2jf$k0xkpAJNoi?qsy_c#KFeh9NX?( zc9Qs!EZtPQ=;-K!!D`_5h%u=GG@wkyaaL8fX8c+fjdzRMjApFJuj};ek}!Kgt?(T~ zh}kHE88xVAn=-RbFsg9{ER1L5=pO-nsYfgbdWPFvaOe_8yS@GMRMGR##iAoq zP8mX)k?cCTHVP^GjVEXP&8T&!s60^ELvqJEUXM_4Y&@)aRFolX&jIHep*^Oa$VP*C@AEAiv5%`c3o50~h)&aBlnbJOLrt7Ia}mw{+V zy6tMz(H_pc*ixYX888u(?zlFZxGyCAZq*+9TP`P%$EB_f`Lsu<#i*l`@XhDon-i=d z)5p+>8jmeL8O4396bFjAH!O}7?Jxz*ML0$?y{=yj!(0{D{R`mIro<9Ilr$Ot~VPNKqk@V>SH^^j7k7T$ProFP?gSK9stWNW=88^g0JlXNZ*oy$if5fK^ zXo^q|#&##N4OJ$k69+Rs65d#)CuLr|`J{X^FJ|_(@Za~3R3ZeO7gei}e^)=ko%^$X z)Lh#op-GPj$K2Asr4uuHex%z?KSZZB7TtAQ_M`}XBaiH})BXjPJd=#hS9~RqipVPw zppjzP?T)k!vtW~FoUsycSRQraeBu^(!}5&uuAWr&^m!pMkxJoDK~@5W(|Bl*Qt?<* zVdQ0l6(uR*_~+sf&2WA19Q~h29R-rOyz?pP`s(e_&Y6q)-r2P-0{)PgA;mlRBesIeh93Y-xYrw&~%P3FC!O+*H>gAQ$%1$o>N$3_kS_j z=-_N^I%V5E?aUft;$@OG|EWUa2}%A^rv$sTB+=^*=oK>Ml=RDS7PYLbp9-0P!n}V3 zf0;){Xl5HSJ*HU<`Jy9jFa`ce%Gm%#Au8N7gAmkBF&=5m<_o^S(~33|#^FD;woCK2 zZoFm_?*Ryr zuJhaf+I;+QoiPNA?yNxj-ik~>S%K5A6*YDOTWkXV#P9>U0OTCm2j+bmAS?zeNoe~3 z6E5-BM$nt{Z#ckHfuNjRAJ~if$7NV28)<+=)BlZ%^V%6j#lf3@l!F%U9#j6~hj_Av qBqk=B9{cMT#G!w-h>WZH_l_=FHtuFVye2?>T{Td;D)~w`1OEf37#G3- literal 0 HcmV?d00001 diff --git a/images/docs/ui-dashboard-namespace.png b/images/docs/ui-dashboard-namespace.png new file mode 100644 index 0000000000000000000000000000000000000000..4271c92b7383bce8456eb817314aa983c6b50b22 GIT binary patch literal 7282 zcmb7Jdpy(M|6ls(Lb|zSLMXRV%0>v~GNC2iEJ7h|=9;_Qx)5U|iChaIHk!M+l?){@ zm$}R(qwryq`(?I$-&UW`r{ABy^M{9XUa!}AUEZ(rdY$uJ_s=hDl5L3fEp;JsW0ch%vfqum!sJ(9LJ=hapb|TY^Sje$rTD1O~_7o&Vx+ z@51i*_e@qmt!=nQ`oznfFMrmsSi$W(Cx&cqNBkbAp=TbbZ`t>2#33I2v)%|c_&w|d zWPXqR=)GLN-2Cqm5x*H46-Q@3&kT;dc&uFRIji3|cA2&DM0fk5|5@BQ?gptdmjrdp zw}SSrpE)n&90C2MV5GBKo|Uoa$@uzxmoVty`WbiGe`)c;XbgSk<&D(WJfZp-+t+U< zEh0R78v;yQoqV_V?BCUYx0=UiDAh@)l55ReUo3P*nA%@4I|gheLy+6>|CXZ%zm7A{ z{`6(mYkT}Y_H5^p^4IxwNndgDHm*!;6hhTg_ii4jL%rCMGv1iIdLGbV$z;s7+NYKj z$02{kp%Dx^M7iLA6En`19yr8}0%02{0xEbA4HHC#W7I51VASzP65Kr_p`f!k2$ow`r7Q9vZ_mse=-TG^D60-D>NSFQz63R472QL=9M>! z3rX7QZ2yWcc;nL&CMv>z*f-@lMHCUWq3M6Fc`jF>AL2mvk`Kv#J85;S5*nDYQr2NF z8C(#2_Iaf39-~qDd(6h05`WVL@UBb?UDf3SUhSEh75Ll{)Oe!cFK@yTTS0=NXfLQd z+};F2LebT?h}zk}QN`^(yqViyQ)|uD^`V zQVr03e#bq!-sv?5;i|rE`F$#YbckUhRPWyD3hg*`Yut3CR1OF)qe^)+)YZefG{J4Q^(3~uK-H6t`ed@ua{WC07!rD=5w@=@OSqiGaDdxG(Gy0p;M-iupXfl~`=(xMo^N+G+!<~3z^X}Kp+0WqSEUXiXGnMSs$4>Q-C zWLr)xdKd=;nv`rb?CLISR~+kAMLx=I227;&;?~$76DWe1=$qt>#K3+2?ZIm%`0D-f z6H>!igTj#$Z>8>Or+ru+lW6B`5aiyn4r8DgmE_KB#Y^l76YT!aiVRF)R@cC=ID6Qu z`CzWh{@2_}83_ey;DL$F`g=b6=kX8P&k$LAvrPT&ccs+;W*K!LEwQvH^=D1grb~n- zalB{$O#aSDC_bpUVCg1JN|l0;EfCjtb$Xtn_I%&LdGWkn&1A&F{1Z6&bo`g#{Uk8< zl^9>piP~nJjq8u@b08Qaf+?DPlj97nf3;a0;}H6|2d9UJsE(q?@>5SWFBGgy&rB61 zuQpxCMU-?~hrRV}DRZdlpxLH0#kyKN7ISubUOubsn}-;~yJUqz_7QS&6vy=*e8pHOcTYZrIE1Og?G)`=4(;Q`)8TgdR;-HpjH))31X>^o6 zRSzAQ7;Q$A?cmr>z1DxT>4N%Y{sDT1&GC$Rg5#N(c!a8eJvK*vD*xi#sDZg*Vi#pc zvVrVzNHT0+vpAk+WD(LSq^K>&4h|E?R+7sFu|LQxzVm|n(5`6~(}jWAL6e`2*>5jd zKDIbR$n>~MfJy6<4WdBP?X#}Y7xqQf=XbhCNy&?&VZ;mr0xW75`nDZ!#_RNgW|Yf8 z4)YDl)^ZXtI{nN@Y0VJ3*>xJhhbVi|5zsMVwBVxz?js$_cPEn&9C z(zUlRkFYVon6$^F1UNgI)V8f&2}>(^)#+G-F1=+j9)ii0nsRSnLp!EeuO^zTETxS$ zQ+V~3zYT0#NnH@VHZUJup2VuFIB`q#L}9V2O~E_cCuZYO;>^3|$Px9TeD8MFO8wRT z20ClvQIRvMJfqHPx*TsG#~--DFVT9$G=GqAv#e&I{2(J@I5#WU{%!%4ff}WA?7Y*2 zDJ%69%nQkCdRBmMM){o2pojcIU77F7fK;7paR|RumQAa~=!zj#G(dML&`iQpW^rJt z?r*=v^zojU8N`@9+{mw@Jx8s-f8k?@o5=NnF>$bRN008i(dmaiOmeXk^J~q$l@V;1 zgudj2`)t?WxKJvmzsRBwcLF@L5{*!8v%{VjLes9Djg0<+1gp4wvJxj*xu0h59~!1L zuz4}wJAWE5*VOE=H%pEdQ=%$BMcq6^BHuHc=FBqXa*lf*TeK{{daot zc?8?S!l))}>x=8u+6*;>c~Avi5y-pm-`m>Ah^z_ScjX`MHhfga~g49A!(5*x0uelg-USI8o1p*$`rOLw2`bzcC}Er zH!~2vOjUJnht=aj3M%#zHYx>NQvFnMaAvZvKf;2(w@@ zjf&P|(_XVvDb-f^LpAs}Bc%hMMs+7BUh-$YjxOU@^kPGTtfH7&ujzj?&2TTnyIu)N zghxS>MZUxB=S2jTY0(w$-|~|SQNQvDdgZKtT4NOJUy|k&`TDT z57wuM4I80cV)3iQ9ysUEPp{Lqvnd+0+PfAf7?hTJw4>B(vG?(Wni{ozqyP)FiZs^B z(NKk5tL7?m%N|qDl$g|EcHDusQ5Uqh=BHkyP+DJ+#(`+DB|=uEDi}?xz{iQ#uN}I_ zzC17}s7f?!e&gacd!OygxruV4yJE#ynRJQ#sq$FPzS4z{E6k6PHq)G7MB*Yd=TbJa zc1&dH!&C99`8&1}b(bty@aW5SSP1WG;^RQk&k0yRKo20oD;2G;W zt9>o?-`qFe)Cj&U^R)b^dv+`cs<3$tF=%{y5Z#Gc&*f8mjx5DIfmI@(N962AWj|po zJd6}YYp!dBrq~HeorWwq_OL_CmPmqd@yo}4cDrP(V80|)l#%HA)DDQO;cgBjZxvB; zEUWE!NQtH+_&F>F$rl3)^eZyULN6|TefIV7ENMYJ#adxrDiM~XkaUJz%tlwk zVscE{MkuE;fg9>=@RBSv?2}Y*xYqR$+}!e52re_uwacsQXyUbMa}`Xavi7B@s!N#2Iu@o~fjT4RPMkh= zua!7>RcWferUEt`?O@QB#xJ8^+H`;Y*DHnQ6$4F{1kxtVgQPkd&mcE}U$3hAn$EQF zyMYr#m_3V9A>) zG|Cg@>z!+m$XIq=p%jamkfaS|NHRJz+;0Y`hEdnnfUlljphJCumrYYT5t~~-qm)3uO>B0zx`*%K1 zEn25MAU8ByCc>iTo&IWr7dQ=#)$&FNOV^OF!pQU;QUy7_D2W4j+pHiKZPuGn=H%fJ zB%y2Kkloc>nl~%#2Bg`!fft1VZ*+YZy-(nIvt1?;i$p=}qIqBEV(zn_Oh|~ z%s`BZ9-t5*0W}ZX$Um7mP{-i%Q>T#9S{oaJ6R_bcL%B!2r|#qEa+;epG%uTuBHXPl zS|6SalBQhfLdd$kw3?`&Fa=IRW(_G~`xTG#RJ*ia z(8poQuISyG#=Siu5D8;`vXl0H-h+ zdQkZ3#`J65ixc^UZ>wnCOY`3<40D1phdb@_EK z&RiE`8W24!$RR@YCD4I5GIJiW8eD6;K8(t49SvvqvJ73TSkn2(Yg=`XS?*s>ICrYP+uWQPIPa43jGuJ&ZoaAh3gbEu+ne zpmW3{DYz}6Zqk&NYp6Jl$LjSr(^M$PfOz$XjtXTq@J!waR^sCIgEW1~-)A3qGuXAM zII)uNscWNFm(b6y?K=+~`mohdBW?|}6Ci90MX2(Wg-7Qo2tctYh!R_YM{(QSXbcxH z;o>fbN3Ho~6PcF9g?9$+;FiGK{p>M?=>O=^4gQL4u1_8*pH7Xoyo6BTnfvgEbaZD~ zP@LhFOvq=s8zZJ@&&0_X11S+9?o)8{Ew~rmgp-LX459kukU`BnaRWbZp}!4 z|0-coQ7W5t;So>hj{g4s>z0;aPa7L+YZrM41aiPGC@83#NF?T!m6dT7o!(99?ykI* zMTUYvGV1ERyveJcUUmYzBOr72m$E-ga*>1>o3y712?_lVwr)QS7-J_OC+bAo+IHzm zHrGiH2PdVg#>SC%?%eSR=L0o@FC6dod(gx&HT(#D67S!_RfM~-6`&l{DSoe;>QAxaWQ|r) zTQ(`bnYAd$^>`7j*7o*^T;9#~PynH71gCyMWZ8;A z7Fp)dVM}OOSeVbZ_gjK3xx|P~B=Ezh80fH29u9Zx|E@+uzkmOJoHG9Xdw*RJ^S2F2 zFoNUkaIFt5Ql1zx7W?>?3UZIm>Z`Q-j4Gem+Lai8&QXiP(x~Ia2sahLvt}*ILG6HU z5;&-1>!!x{1ab6>V>j$3GD7NHhKN0B+oOGP{&OTz`i~U1AL6?KYsih@S(8f`DA8f% zwf>W1s;coBBAcS2_NOUd(zU6Ks)=+Ds_y2wd;~bRNg30mOpeS|i~tw$;Pok=o@*BU z8*2;pcG%^)jfM7V)J4&o?hmw=y03`r_-imIphj}K%3}=X%LI~+F0Z>5ndUQq^7(Wm zv9I_}Yk2gDMBCshW!t*Ivp9{3R8PC|B0+N?hKGQ|1&nV=F;LaJUrUb`)pYvI^xYN{ z6I&jW;+&S-ymYj>*37|t=l0@_{oA0ueX8we1O+tqXfHk$ztKe|-+q0m#l*zKxhYz# z+;7fqWr^rLJ=1-~%)2WafT3&@h*LQ!gUd_FTmJLRUXAiRA)!j8#B<{7q)WB$Y)n%! z2E(T{l^u_J_3BVSh?7%(n*WcNWL@3)7TJ^h{OVAsKxIeEQm?IBQ9V1Lli>KSc`NWC zwnwV6&yR(HS(ysT%4M0EGOeFJdEAlQ^azyWh%+`b^Szx@TdN(A64C~SqdR*ss_Dtc z-%!0C!~u_Zq)?l3#UL#Qj#BDU%uTfetT{pfD$`RS^MG*{Cx&sMmiK>>&u|_Z8ro^! zmIF^2Y)RhO2viRkb~7_G1Mc$ihWKmo&z|kao=t`s;n|3^o)^sq{Wxrah#Kz>)Px$rs-&@Xsn;1>8Lc)K*}; zKPir0bX%jkR=2^ei8Rd7nX+D+s7R=bi_3DOpjoA+}wh6_FjMaW~CdB$2U=8=m(bJQsoATtF(k+*%Gq z`)u~P%uUtYLJw5_P2>caa?kNOyEZ=(_~K0HCH;4s^@1zo&esUO$Or|%sehMp3RAyG z%W$)qD<|D~2zdJbyNjNH%zh_e_N%IFqQ`UF|Ct>I+RuKMe^@sg${&K8tRH|g@}J`!4|R#b3P9(iAUm@2Zd| zTS0yVSjNsquw5I{UO+yjB~nLsj{#S>DVf1=MWavz8qH42{aYx#zofaixY(J)W_=A} zE{}Tr$iG%_?8p%krO+BJuIBu%yZb7&(j)85n>QNjZo>!1fF|n>t_IqzshL?tnwGC6 zkg5sKpCjOK0YW(NWzSt)-KjRVsNL^d(Ux@bqvM8L+rZk2I*^|cJHqr_m9AP@SuKwI z?d|PtUK>&{HkU>FQtmz+DQ>1p>L4#ckepVLZoy8`#yjE zJSe2Vnh^xRxhw!*3Q7b_&$n(NC+Of1ZgA%=~O&)NjFG$cO6u^rKCZ+I}VMK($d}C-E|j!zTcTU z_n$j=<~MiGI0w(!d#%0KdgFPX_dO@@lZ@yK6hagP1cVpjVo-Smgr`~v2#@QYKL+m@ zU{@l77i1eTHG2dERIG=8j}Q`)@WGo%4&u_nNDGgjqT!*@y@K{4AiPEphkj6WncA6k z)>Krv6TCY==|>aJN1=O87a)P9k3f`5E21CF*fn0wQcy6+erd^p(U;I?H4;5anW`FT z!Pc>1rfBhqJ~!|)@@raE0#v2$x4&jKn+2J|C|-XXd$ zut$9k?gRm0LD=zx;tgKbzyBaGbFGm7`|CZ^!$Tk-{C|CQIXp-`K@+-#mIzDgy9EVq znBZ1$XPuo;@pr%aH=uTaCiWIh0=pu~63xg~VWNUrfC;pQEj#C;m#TG2rywHi-)TQX z7f#|e4t%iWPGv<43LKTmBPF?_^j1oQdvGw(hamq>szekOj1*YYbZCYeU~K;PJShX3 z29i+3)iwUjs!(e3Op1SJx(L|INqu*b4+)WmnxC7WXO35!oInq(s4R?AHb7nWIbFj~>SR!jT+&GmZK~O8yFt$5s($ z(x-_>p3r!8_T<}GvO~1<@WrBRLPMg6hxotQ4>IisTlvP7$QFx}7B@C~?@#cgTm0J@ zExE>hz-=Ov;(A>LcMQij+xcY(ee3N1xAecT&foslH}Gj24uqR{5?m{$uhqEpxEw0m z(|d8`j4-I{tZlaGr#?gdH^tvIJIHiN`B{aV_&!uX(RB@hA7FtoB2c zr=B^TVp1~Mx-+U9-FFSgo}wU+4+r`)BmWz4N|_1e5DPLm=}t75Tx+rLvM7FgXjJlb ztPKaF6{mTw5ZR(;gP~?!p54%q{F#2#k674MuwT)e1Z;}w%9>=&md!0H)pz6VA6kmW z-|jC-XIMIgTWuPQ#w(F%HC|wFUj2FA1^!;n$ehsoH(`n(V?Xkpwg-9scwkGrMhAox zmoa(gVtC0(JsIqa;&!3=nab#P+}B-CU3-Z>^TG1TR^pq+r}eQl^pr^%>v36$boq|T z*BGgddY_HqMDy}dM(8oI=8V&_3xTr#rohN+KNa@$@XcXnA6gcw>aR2Lbk(g@BU^kq z*sa4>{R!JHrxQ`*V}0Knc6zVDmk~lOZx}e4)pfL)vCCw1SSxeket7J7Z#vFm(}@2K zsLqUc3CBySEwnxodZq?_uL5&FQ1B#zv!QKs)Ywb(N&HffC3^J}%lO^HLKI{brp3af zJ>o4I1HrseHN9_bCGCF(G?}cqe!q)7jnXqngQXA0o`gGB&AxJ-XST8ZZaa5X&ajk)l2Th?uu$jz95+&{0>w*PKt zgsy~!hRgfLh0{8*IyLw9QYI_jAIo|`e*8Aa5z%mM@HUf`1wP->5&?5XIVuzhwq0*O5EU6IO>?L4`$2B#V_!DA|!N%~Dvg_yxyUY<&>8v2)a zKdXhzP1hnB;tPW?q6`W`r^CnHxR3R&#jeg@Ha8tP!2*%%|M~c9JpT9NXWzcr*=L+T zF1xi=*)Fv392}C4fwGZnnwj&8I_+*ue1bxanTUWN)cP#%Q<+@+oM^GL|4>`It#2to z1F=)S?ds}`xgSVnf3qHk8(L6Q)pY4ctg~=*CMK+>Zuvr=D&4t*>!Rl9 zzWH96GSx6Z>`#vMdc73E>uINk4rLtuYSkQu5>mlTp}m@L$w{lLZyrzYJv$8@GNb#1 zJ(w61x5S`PD$#Sa>{x^6U85Rz%}&AEj$(aAn8OaxN|TWC@(w+n_K}mwn*5qW-N-`o z4o<<60!_l>$dg;-?ye_>_cvNPPeC*$=^HBJx2?b)>cE=X z?70F_UzSGu!=--B_38ixj^`6-4imxH&agn+26YcVjv==h1;o*A87rbW_Wds!lGwGj z`CZHOCA zgBBgWMX$v-_BG&0Qfi)BX`k2kiu~cW-ZQy<7fIcafC2|E$a&2pCItl`Vc;3JzbA}- za#p8JUW|)^B6DYc@WawD-su^;7=x2j`1WQoSVS0~=50Gej#twjm#OO;{ZfJM5n>!o z@}bQlXd8YJ^kjK4VPMxy&Bb8{vtLI8#!z3&FUw{ujz)dORC`swVg|OwfO9bV>ye>? z5+)fEhW`VIL|92jGP!`NfJKrG*4oZ`*xwvx0+b$U3qh2e`(@MAfJ&b2FXt$Mv~4OA zqmm2r+Hc7VzKHn|kG3c~WYjegdR zNkCZ-`RQB8^6?T~lH3JeZZN)1Hy0KmuXE@TgErRlt>ygHrhAJim&%mBww{i}B(AEW zJ0R1N`K~Sz-UkeIm&Po7{q6PLA_uEbkmkRD!kmrEc4!9fx~MS5l zFTeP2cE>3v>KxdUQ-5%NBu+X5M8|v3YuLILO?XzE77Pzb^Li8DT5>R~!)DC1UBwf8 ztJ$wPCRm(PuU@E3e&rU(s4W{Mao>$05rWJWF$-m7F`NBk|{CGdVzSGVD0a&E8Lmy66-dey^K zw{0n7j_ghcR|

6_v1{MvI;RwPVjE9vt&@=nsIT$Z_6#m zd`^1xJ&Wlf04dy^-?O31_lph_Pj=WIk4Mv_8a8OU94=y#rB-~lyzHAk^&SkeOt^nl zfP9Jta$}nXIkv>e?H63HYm$Q@b_UfXLG=@~Noo#L9+B#bdG4x1Dw;)-c#L z6dh}cCgCNOy~1JBR7Uc-*XkOtf>u)zov0<+Hr>hb5~s0v2=&!)tl>7S$mzRnhiI4f z7E^zXS=?RwF4p?#|89NT<~#V$c+yjcB}>@&s&cLCS^jjjM5J4t8}eAcV}v9Za83FC zP24a!Jk%mrj+ zof3}!!zBoOxPaOm@sZ6gxQN`Jo*UH0UOHD!bsFtG5qr}Q_#X4{}A z^B+f>b#e;=a>vwsAIk)k&}QdMw0|9Upe0Wa*KtNU#p=7wCU2hbvHp)Nr1)??L%Coi zTs)z=ow}U@$Pg>TP%wuL#)~qYus2NL=zKXqn7=VB+}4!!-R@?euVrJ2aT#C%X`l4K z!ScA2C0VaL9#7=m`Uew0SU_HaJNN*CR9}t9K$y|sGCXl$`?0#Wg16CW>=WWO>&SkJ zKnCD&VE1{8sslfb)ebQ`E`on|fEJuFU zjPaRA=a_{z!imnq0VPQPo=N@DeOl*CvhwzRdnz`C`2|8c*nJYNEAt*n?8vJ;3uujl zfse!SV8X0v)rHl+OMW`qkQ%PUG%IQW^1~&fwy#^VqoA2c0@wA81irF}zY3@e_;@k+ zKN@OGfd?5ZR9Yo2XG&h8#j__d7gA~|t>Pr3J#9xdVz(|{tS72j?^2a-V`WP;<*pptTT|Nr)%{n=e#?=%sd~#FhUF1^T0`-ofI4Bi%D!E7xu@bx|1nDGZUgNPO^qxqCCc^74 z)b6_J>=lMa^zJvFld@FnJY4q0_#Gb0rMUTc7A_W5ws3E-GJkx0?&I|3$ArDy&gO1B zuA3Y4XRNcOF`fm3?GTu%7meU>&;?l6Zv2#`mo1%r- z>A3~1Q*+T$5|gW^D&BgzS3%p>BV$qbW5gy`9o7fYx`%GbYSeoTLn~^TRJBzt_KRRfQd7=y5|+-PVR^Bb?DUP-zhf{$o(9QkjJ*}4 zS3|9Lg-MHIQ1xWTvn^JDl3tSG9GE&MSdHbEO9yLt%uMO^7%;{18vJ=VEop4Lj{b^g zXBy7T$vOTb*lNDX^ZIP}%NK+|=+Fl34k=GE6F0Z($;RNR(O|vvK6oZ9IT@M$zC=zW zH2l}tD1m=||6aU0-QFHAQOQ$eZak59bUclJXKXc7Q}+4yW0G+HK42%>6PgC3K}Mk% zCaW&;(Ld6u4P6w!)cB$%SJQJ-Nfzmf?cfrpyd+v5KKZ7yMomRyyoO_rk4fKnOQDCA zzMnUd*DEa{)7PuOc?BCRUM(1YW46bz-`low(xB(pCb6GlyDW-Wu#r-q9TcU~kSb+W z=9P?BlutzO;X#~5WthjFeoz}MT8jl?5srV8_r?)*d2RL`IT=&BaOm^+n@Y;*Dfjab znE=#cqGndh(I9B({e`VneBh=l;Zf@4Z#L2$*}uxJwFgERB?TE5=gNKeVdE){de0AW zU`-X>dRhB;Bf<&$kp<0D8MB8PyBD-?t`{$C#P^wm$glK_M>|^CyS(0(HMTdeZw1r! z?uV*`Z8KA)KjNnX=)ryZa7ws@r?PL9l+EN*BN+(^iAu31OT3haKx%-Uy?r1pEp2X~ zoT{m*>Fw32y}f+}=|qLG>{)nFP*5VL_1XUXyfTHu#$U3E$(yxA>$?` zqwaL%4r8g5lxxHx<@L^#>S^h@9nK!Yi?tKw4RrWiH8`_z0~@`)-LEp_NcrenZiS)I zIu>-p>UTAI0mzusTdve1Jw6^hacSmUWB0*I2Q`)Zii{8fX=QS{0@e!a(*sff-~EiuON2TqFrZT1QftxQe;|oF zsz?)LC=&(soTfia*r8r1X#oLUlKlpMc+=Y9xPh~nM*TrHIlLeAwcM`8q66A(_d;{h zivrC=2n6isa_i`RZh+N+8{ILp+;|@fK!wGLLp zI!gRru~IE#VWwyI8gU^lleO`|SY4;eF^`@D{38nUIy{xPZ%piam+VYjF&TWK_IS_W z_XOA_tatbDs&$^ho6*8EKi`kT@I*qFG+uz2v1wE+WyUjL^(T z+mhJ0fZU4C+3Rnav`)+x8i>m1IGQ3p#W@&)|>E3NtbbEC=1=i_Ad> zcvY)P#kskqw?*=KT*NY;K7F#Yv$L`)Zftz@mS zXLRSXzLS>Lv%aRX4G(jokk6S^igOi3q`>}N@!q-;C_nSG?5SKO_xoSXYU76MgG&?2 zFeBuJ>M?#mT4 zG$f%`V3~@GEskp?W0~TmsINsFLc>}Jxomz4r2!8PZ`U*JpW$`bcxyh!fQJ{0MI+;J z*p8>;b?YcDj?5tjY|lHz!83vp8xlcnza7+DB<*zv8^Cp=ynM6So{1J-UO$jS`;Mw2 zpe*@N6v|0o5;%K1E2dRgZ1AVC{5Lk;C?R3H>MfRT3Z7?|K>?@(1^2&gBAFOnM&N# zIoFRYU4s4NBNKL{fgeEoQu#5O99FP06Q0e@^QOj*#Xo?p>OAu&kQX&uOg-TFWwexz zD3Ty=Ww>RhLj;*{&DveuFvSGU)=E1b!z*fp7)w?^`u!?~C0W;x%!Jvdlw&C1Hc zW71r(tNr@*D+aM|0F+!#Q*)-;avFAjdv3(^K1U`QmqGQXa>4M>P=5;lbsy(EgL>tk zCM!@ZTM+&uY>5z+nAHAVqwuy!Sjtx2Zfh0`X1yQqS*b$iwEaBwTXRh$nf#nHj}+BM zO69=qMrP%}z<|L@hI2lftW!7gh}=8ub~=`b>_`%>cOtDXAsI4o4bf?Lv@;@?u?x2Y zV{`?Z08J%69oZ)6AF_1x-3zEg(O(zmWe!zIy1owrEg~P~#ItI3oFS0pIuqI=Rt(gL zO6SGCK$V7>k2f90aOa%ODI+HGbP;P}6ciL5j|&@MCcsd_!^1V1ycTE+OG``VVb{CA zzCIxz^wz*IGB6O!8Yg(#twH8k|hmjJg5D*$Z)r#$4ahg-}_c=Mln0A@@ zHFVCt)3-e+^m-;ZGLjG{)FdHgmVE@vOJj-}DY?)UeMwJh{USv54f)V^d`HEkHx_<9 zpA|AlmS3&<7P%L47tr^Vyy~587tVGlx$MH^V&fXMT%nb_E=6-${f%FZ^cnz}v}7^g+|e`mMyhuwT^ zWNb{}KpfiD)rHpD0HT6LfZi~kI~=5^U#Ky`dx0YH8sm|7&4hG|1CCzPU;*7De)R&yA4kK5?1sq)xeg{6E@EY}t;Py0<)3zgva^LTKib4-;{ zRdj>5u#_8?TZ3wh9l;{Wp5c4jcAKk6GP+M}Y)~O6s91;_I{@{d(_eB$&uWO^739U{ zu#IDc;Q1o=hN2;GL|ojNm-_Qj6lTrRfG{(gS)me9C{HfTMpF+RPr zh$!-CFa$k?6o6xPd#xM33-@71AU|S~XU_LJ;!bEWPr=g9uAhtoo!}S~o8E|PH0v?O z9R{kFn=^zurEu0JSDkLHFm7+cs6MkomEn4KVji#UtZHF*cXwgoZ?Bxi((>}8F)2GE zCFM#e=iJNS_Po42;MH3bWy^(X6_58-k#drZ_bvz#rw z3KF6goP6NUbKl35F;Li|O7Y6?g_{>VWzhEgjcA!{%g8~x<7QVu>^kDLd+^CtgH)AY z^lt^MgmKevbZ#CNb9F1dp*=^9-nWgyiiQ2%^hVK(ixcJx1om5kB+?Dj6;TU`wf!&r z2Lhr70_yiS-SIH)x(UK8y=IO3lP25h=jP^mVwtViXz%aNn|F11czL~U4m(c;-Z||| zl<9Yl5lsNg*_*A`=LN#RN_q}nXwZ;+f|lSA7d$r}d8u+Gy3papM?LsLuFphGc7BQ? zq+*RuAyQ%ZXlR%KfaF#zy=C&futgHi{gOHHuC92eeNoYiN|Q@P$THT5M%24wuZftd z|Lp}}t-#;k$PRKJDXbL`?|P#aziEa^KYP=fp}$O?F#mag-)&5DKYC@QvY`&y#Xmee z6xB(zp67wHxy;$1J^Q%bKR9L&*Vxfn7hcYWA((``fBl{eeENR-ud=CnbE;MKn8`=h z_Vk>057#_wWpmSWgoWRqbRJ}nA`y+O#q|I9`2wQDjxR_Q@=RUSiVRNpR#zkOR6wBs?N`Ef4)ZeSp<3hl=k6X zOMhB$*;rK_+|9^PZM!Qjt501GN)#>#p&8Lnja~EZ}_?$vM7y1zj zCHD9y7bE<=;sThqtN%kz>ap|g;DemH%q=2i{WGCl*G1sr9w`2rDQYnUui)7f@(agUM z2-+QpF{QdpsP9+q9(&PKAf5+$d03^s-9*Z0Bl(8Jm@>Bg#*+Rvq5iR2JSL=^*kEvoS<3Q^hnc7sEyZj*Fug%(OQr{aL~60&J+D(aQZY^~aq zQvL*0cv2^pxuzxgzG)dsgAS3ag|!w>y_j4xBM%QtC(t)M)V64q9G;VR9U_Rh{tQcQ?;iL8V~cYl9zUf$u+QBY{8C9I5ZTe(LD=S z-oBa@s~r>&@T*^5R!(lvIMws)YP;loV{kcMJA#PQYG=Orz9WjIkB+cP+ z=U09cdSBuMc|w@sh~e*0jJrL(}7NPAogSsAMYA7Z6@?Iq=#mg4)bnd2}W#NWqvhUEAHf{Hjkw*w|>A%kT zE0+1rirbQd3x)dWkq;Dwxv6w+q#&-@OWpH3nV^@h%GcF%i`()6DZ~BT?AcKMG4E-! zoJRWnsS!g@$+P}$f=&h@mERlhi_d;6E|hKMsrRE-tCH}c4zJgr$W^meBL0Cb<;_72 zl#I6(=yPRrYy5r0e(f%cDcYX*P(@K)W3b6(HI9NC9z zzzQ2O6+CD6$8fpg7dWs!kjNPxAVsa9tULzfll8vD_07%8Z+JaF7&W@P zyF0>O(Z}}K+u1eUo=wB&+zRwN!{X!Pd7bw(%MHX8keG^`Cd&=?8;-;3>#sAIkz3!= z$fh*dtx7mLR%tQ;@i8h{u2_sh1 z(rR?xpOa4HkWLl2O{2NM=e%1ZO;B&13D}sbG6%^`bbqc&LN;1%ylQp<~GYbkuvpUdfENb$bjm z4y>b69$|kCq%Zr}<~3fElOqkrxj-OtuF_kkT4}K^{q*U)RRp;8c^b^rWy1mUl)a@q@;|O=m^>_K|8}ST6UGO{3V`AW z3ej?A8tH^eyVY(G*MNeRhQqOs1l#Z9X5){sqae>ytOFt}fJMR3qQTEgrbX0z^itHr zK!615%tt66NqE4B*=e3aj?l-CI-aLWPGah;zmS8qK@swfj zDc)d)oZ{1-Zfh*dV3n<8HQuUtehOM7PIZ`hG#pI7?4nl zhK7ct%gM(t+U?bXtf9!WT-Zi`O}zq_)E z%Kgpp0GMg6OyB<#EgCYi>yEiD8zHk+9h>Ws!SqWNmGO-Ylc>#5@O=ezT156b+b(BA<&%V zpPi#&RS4wx6cDlhrhRPn$FQzZ1e+w_yhjW{eD;i*lCpF%ue{t2xH@3lv$M0)*L$X) z)YK-y_5<^H_3D+rzCKa4mPb$lvaFbu)gDlVB_$=bG&TJ&NJ(D4{Dm$EySZ3w^#@#3 zOzaO>wAPlEE8t-^It9)MWa*aE)ye$sKLWI2m#b+X{V@l@^SA-dGMpoeHF$BjynpwL zzh1CUODj-nO;X8O?@O1as#X4VN%zsO!&9XnH@vCL!)l2vq6O7Adkozn(q0qtxm4ux z2i$!+4O@1okS?meI;Pz51k}JJEmvpeb}n&&xQwd}nS#=C3u=F^h+ChC2tK_KUK9pK zTCJ$d$RB^iZO%nrSO4m=E1OYPZZOqHw2L~7YJ7IuuuJ~WOYs_%uT;%G0WL6SJip}`Us0$p6$bYqlC_ZYdRWGc z0=jU$({7{~C0Z*WquKfSTXMw`bl525JbpUOdXGG2e>=l$l z+5)ja1oYUQs#-7S^A*xHFjxV50Jgl9L(B$*UT5c2KG!23WGn+#khXh3k%~n%bnrIMn)ncB2BJS~C)Zgzjc$%0?69z~ zfQS3>{`Es}yFIIN!)oxjWF-oknVIo>LG+MMQ(;%87YBs0gC>>DHrjG>y1~gH{ubxw=jY|s+AMx6GFt-WA&@cXVz#<) zzKxmz=(6>2KQ9Q|L{>o*E`6~j5`u4gb+QSJ#&P5Chm66C>uX+sm$|tQUHHDf|5Wl5 z8^w3G|2xPXx4;)Tt>+A48vx=3wYvgd;;GP~A3dFBRYrz}Whw^lXFIi^f|4vl6CoBr z%gAU6^ue7u768+{Znt%I(o+E1zdfHfcJ2ak`ySMufzwu{QoGB1gu1%A;xTJ8Ff(%! zhI%7m7->jtI@E!x7T;NRh(*4~hS6sbHvkC*1qg6NQA@KX_U*u}r!Nv!ZLjwmh5fVL ztO3fZ&<*zond;2hAm+!96hSZ8?{A1;ouO>N_D2g;I@{Yp@&ek`{VKB;Agtu+U!Ne6 zaM^qbC@(L6`uLH?=E}}a5?HT@2nZ3;w39LxNj<=vj@x65baX)h&(QF{r>3SxMzVt8 z0Bm&HpG$tVx45WVzcyr}Q$HW}bJVQ)=BVe}x5rGe8VU*-igc50j%&RH)1e{*HcYH< zmC1|$RzE{oT#{?4tLLKhz<3mqmp8b27wO#UvJ+qUH)cMGLMQxdxoL`HxVqh$)sANK zxA?LfnG0JSPXS?%^Ww;B#*|yyHUd65mX5%WeY^^Im;{yQyvCe6w6AsXzLw4?iZ#4P z)ao|03uzJPE3A2G6A&P!?Ctqyu#Bwp&i>}NemjwX{+P9~fC*)qtTn6!k%l3guWkhw}zh2xBfE`zJ-`MZu3kAusb58lsCw?+%HvqdrnC-(j> z9AEB&(mWj0q|ORiR{@D70gZli}p_^z`gZyUD{foP^))WCKD#ptdi9dTrj}>+1_00r(#{;&d@T z=x4M&#Of9X57@?O&k^zFJHd= zTciO8pZcv&Vg{}N0tn2OfswIFmBFoX?yMfxES(Dw88D}Uf&zeEWS~F~mCrkwskI07 zWs!`s{QT}}=Ly0(Y;0`RBIRkEr@`&=C3CGNY(HXSr;kV=$i6KAF{NDC6;I zTl9}?pMtMqfaC?cyELv+p9MFd-RSlvrrV9wM6W&3e#7MC?+9*p-yD5OL%G=~oyICg zJv)l85C=9y4s;wz^Ak0=U_Zd|v(pj1pAcw2y>gt-9XRjZz_bqn(A2OYeCLN4%9zLvdw zkGo~8eA|fp$YynJdc7^fN+`7NRSF?!wADUxX#f#&f4;f-a$Uq?zFa8nQ=VeEaq5m* zuqUcfaJ%013s~6lr`ZSqroaJ07%_Sd2#5erXoCc8-F&Oej2ADJ4T6TRzrSqU+Yb!5 z)}VJpI485hctE*e6rg?p+`*s~Mtx>=pLN`Ks{sD25KOH;IP;NM0CE`$1_o!m)I&rD z;zwUDV54WAP8GY`AP?h`kihP4E`hHB*DJAK>&+Wy187-8@b+XFV77Y4?T+O`0YHb0 z>pTEq0?Y}ZGmClv!$e$MxYol&nA(t^qT(``xYXG^{kdV z2TwjD;c@sgFi^s+6#^>Yce~bJyk20+<8=UI0KXk6HE+sRxgkTBwoIGZadmSOfkJoY z8taV*QmWgl=RXyJM>J!Pm)bcH;lv;%O$ESVmVdkEyjw^^RoZ1ImU7FIa%X_v-F&-s z*Zi!`aT^yCvmEF=(>S!(T>=H?crnQZv!6b+ythr|I3-r~eO(8qgnhU8fS2E2tnJk^ z)GOzAf-4F?c{(MIo%Q-7Q_xoV3bi%IOsaycQI z^r+5|zQzNph4yRv0G7gm;%So;c-#v_=qn;zF8Bbgsg3{+hqC=D@#_b8xfqr!NXHgQUafsH406qkXhZMQevZUC)J1J z!EYs2*Va;a9J{Bdd6}5301yEjwt}6?cy#;S``0g38GZme;W&)%4A>3tA0D0@R_uS^sFIOQl46)U3)wKnFMpzu%gzzR5g}Lnq02+#e@Ot?tHUO$+*9t zpG+bLBOBWZd>&Q+KNPfSCtZ37e?=)ZHC%qW7H}EhLz@mf)2G*f=o5sBI6FJz;o*7z z{#7NQ4#yaZkj;h4sV;CeBw0|=!nVQv6u$m|xx2RWZZ`EP#C|Iwxy~Y9NkrIxes}Y-A7aB5hK-GI%=FLi>A; zG?dgs*_HJ>^9UiBp-ZmvCti4vxl{D_zdAgOJ2EtDB6sbItsRrEQ2QV?WE-)tGpe}PmI z#rKJ0D)eKU=^X?1qaj2sw4UNh8~cdo^I_|+W~?q&N^L(jyoXRG&X?J_`Z)>J3QPr$ zO!`%*Zyo3N7QgT1jm#fFA%S6Sx!>0=4;Bx^$Hm0~$$~aU0(5l%&W`%@@!2;MD6|dm zY|!rUQ;`mXgzrML_@U&NoSfWze|HVCFE1BY?&wxXNC-%Em=TI}hWONif~ovk%g2}g z=&!aw%mERFS-auS8449{aIGfOw0N?7o`bWqthBTt1=P)r$NiZ;YrNCm%p1%=z^!Z+ zTM$XYRZ7Byv*a(Yu0Zau-Ft4#22%Z)lbZw-27|rOP~dX{lH7wN18zVx1RsKfa|+Tu zh-bOLOaU~QhFhl!dNqkrO8l#6r=_JC8X6LY`t8F1HvLRWP7V$Z23p)Qz<>z}-EXB@ zWw5tGhm3mGK_lfeZ!$Kv=tj^q1pvYN!ov5MnA0Ii=F{Ej^M&8u62PSuryh#{u3ZO2 zC&Qi}fT#U2C|_AIsMWj!I@W|N2PGvXH}~Y39ndJxHs$DIoVLey#XXu^{a##yKDWro z$Zy`>RqDHd5!V3_T@q!*%v=&39eun$Kp)c`9gRKRvpdyp#V>>{p zF1(;{Q(slKt`7|kz6Q@bvD-H}$xTaZ0x0WF0%(BbfBW|P3l6YE;PwbPt&)C!=CWOS z%IgN?8jM#wC&vSV7zC_sKtxN&j^B!T!qev!n z16+e~SPcE$pZczLb0Kf4@TPMTlJz-sqD%JFo#Fl4S!mlKDmJr=5-x4&n(_81)RO|y zS&(S~19`#XX13;|Wgwb~_Gy5(%cM~Mm1Pxg`3r6+$t&MrorrML1~cDfEGi<;LNiec zC6O6{lDsCTqPv??ThT`mant99ll6zrL+YqTzr($h%5`gHX3KYdmm$=t-wP-0YStEd zx?Pa&;?7Gc6~}eRc7Hzk#0J3OIX3 z-_{rqxJ`#LfqdM2Hf0WcOu*LE5HO~Apq(> z^Qzr&fG@fK_qzaYM&l>Q{CU-Yv;j;t54YBlkO&e=+W-rtsj1o69S`V&kFPISUSUd1 zVbBcCW9fB!Iu0bJT7aqrQs;{_IIAgw+=%XsbY*$w?9ncJLPX|J}CEy`8P?=1>-DeG++u31HK}(bTKVaHeOyTB+qN4cHeKGcz+M zm}p-@t|$E7C}iIN@CNn)R3SWR((7Acqzzc zRc=59v*_FwO8b7xtw0x}kRxMx>))I1LGs6fV|wpp4d@jGTKVyyC@H(W?6_GK5r@TR zjq&GCA6HEFo^c+4t>r7hdrcCImpN1F+VFZpV{k(72ZP#!GU4TQZ$#n&f^*jx_GI>Z zC_urUt1&7T@H5i_Hqf+iAhbg6!nApM1t!%3p&HOXrr7JND?dL!0f+f++Ekh?&=9t| zx(bx*zuTu4&XD13Y4f`c<$_Z{qd;Q*uUrS}!4KuSO8T?)R#bVb3BD9}P<)#N+bSgV zO_7ct=wmVTD!7d501_n%B37w;p6#erpL26^!oPq2?xqs((Axmo9MnHbOZNdA0OiLT zO{@3sfBlJ|*gmaAL$DFV4PFJ(8K6Z&n+`zP7ZVe6+2J>)x&m4P-b&0 z@BcpO`6C3ohhI~uWU`FOR=V26eZD4zeQL5)&BY zGse=2hJL>K?PXTig$s zuyL^D<>%8~#{%$p#_TJUcH1*Eqj?<*hU8}q7`@Ml8%%yxFO!ED6lhfG#>UG1%LUM$qMPv9b?;O*@>hqD5`ES5=&@STxVqLzAad&{cb^wCq$ z*rWZl=yZA$i?@jevTBT5Q4&9+)su};Y_4~HS6o;3vifrNWVI(J{PTQ56JJ(Yhx2KI z4@*1|uFM`OP%E%Uymne%I8V3YWEHx(GhB;o1Z?(11#PXd*i|J294VtAqiDs6M_|IrUS#&avuZpRC%{g1)zc*= zSSTDt?J;C{rj_aZURmwQKF2`Wx`$CaL0md4rz%ZBne|jXbiDO#Z;LM~_<8`4f@(pT z$B*=FVjsOqG00gPqXpv)uE(GUVPa7>c!v*kkz@?p5{y>}-Kw{$q!X>ESJ_DqI znhX5he6z|70uW&I0wBVGa15+V;A$fSNHVVgb%#aOrS3Of12=<#Z_cOzo!!uo3c{ZB z`XM-o6`a<8QY$1QkCBH$qnmZFYH-vjk1)dJ&4z}s?gfg)Ja(qJaIObWLIW3(6>>yY zsH~;uFV+c`?swzy{Toic*p4 zqQ9~oSX!t%ulT{(eT)%~f|rxEo>c;K7j8Sm__keS*4+Z}k{OF~c8TU(&C0gcWUE$n zycvFen9MoTl0m+4Z@hoEoS46N|F!w65R*ceurD-OTHuJ{Ys9nLAJEL~D81H;2p)!b z6OHz}m&)p2BU#@dW`bf*z%5Iepr1KvT{H{f-^thgiZ>CTa!ixie#R-TV>&QB`oBnf z>!_-_=xtO5ix5FlN(o6p=|=DnlF|)|ba#gZ(nw27OG`Hxbcb~3A*34-?mYPZzB|4< z#y7^j`?hwxg|pdJg`7&5h^}?( z7HoN0x~$fNk{XllZSeGF=p|A=r^mh~?`H$aQ&I(KJS zOq)%&K$4pFJOuDnx7|z%QwP)}{+1w;JL)q&M7lsss)>lad2&x3RMY?hCNOXtm1%<1 zl|HbNDW7b`aTor+H)dWKOwJ$e2bO^I#yj%M`)=>Pe@sjyCm>K!SFZ%c1)G%X*YGeq z7GySZNdm4YHJRV(y|3?qqKdPMiVA?i9YEtK`3dNwe4UyDIP|^LZy4KsXitRAO-yb~ zSXf$OPvjLA;!L0{$7tN|sw>}RD&B~5zHd_e9yHBsR%7c2-?~1p+y+cIW7KPLTLeiC zQ+{R%{_zU;yR+D%CRVS$4%c^@KWBN6@y4*@vBpz+9le|}Qdd`>$&`73IZ?Ymq-D!= zhW~d!Z=qauDU#3cD9nrj*SNjMnnqEe#MxNvSyOi%-_5WL$<2l^+Ui7ts@T|3g2B<3 zh>-T;DZYn1N(y1Yl0_?B>jbm}{L>TZ_li|iHT?aE690B)?$q&17qgdOO4;HkKkFmo%^9;qTt_2S;98Ojzp+_waaG}! zm)AFTHN43rZYD-X#sfkozgB&RNBy57@xfYzzH_vP@2K|c=D@nCi#_sSLUQTHpp$V| zPld~~NY#>x)O3SCQ?eMT&HXtAPD|$PI20o_B~)Sol(?1Q>);Kd$7jS3m4Gp3%)onMrZ@MH)RNELsVU-JI>XrC!{^bNl+;|n-exosjey$ELn zhO3(`{77cFqvom(f-1j!{%Xnt5fOxHT7y}VhY{}qPsYIoWr!}dsEGX9)R3)?b=wvd7t4Ve1As!5Uh1BVl!j%-OGxK^E?tZ$pf)68T+6EbQIhRv$Vq z6Q@@@p{N3Fs7N}9LZq~SN2EKPla6KD={wVswk{Q3K($+#3srP$6K{>)q^sGMi@%@Y zU(QF0RpEYLM%!@1KIfYG8oDAV+~qen=2{A?STn|M>3)d!BQvx;j7e0wley)a>))hb zG`@aDLNqrwx6~Q?bW&Uj-PGDz*!{rb#mnh?!Jy4tjUwZ+P@ggRz;4`)9voTe5~Dc? zIT%!EOhI9%3+#~R5Eedpi*EwsHzx;|TS@FWrbp9n0KB2H1#2IM-`!pQ=AVzTq!BdF zrj*I(wFW(t_$q#iA+W}^rhPujFE}3FPRxrE$JtKB#K5Ck1k&+G!zXDY^wZDpXF6^N zZR!7!C%zo&=MgHJOuF+P3)hwGjEJXVT)OejS!q%%rBZ&OgxqJ(r3CrUPu?lm+i+$V z6o7f9+?#{9qZr6kWv)=0WF&#l9 z>>ij;pJntz0cibBJ5x;2PLJ^UX_EVfZIw3KR^}M+*W{CA0xrFWIX`DzcG~E&WMX$F z?0a-dgq?xM-Jfd;+^k+J0LLsp9bHTE7U)%XDJfYa+EP9|cYlwT3=R&~KU1wfq|suI z=HIAH0~OQV@;VNM$JY1jsA&uEWcP2Q26)fUBLES&p|G@$1XxQ|v9}>wmnS z=qNUH*4&dHVAx|~K1pa_NW3YEC&tujQZz1^v_bl}nKM2wPg`=~kJV<;=(C5kR5jmc z?#(@moa>IMj`*^WURbf(wb3dW?Zux^d6r9mv@a3YW;$uweOi6*H^msuwQvTB3*`DW5df`tYu8g9c5|{!M}qWHfb@( zcWwK!Ha5y&3RR=J^~LC(vm-E?g`hrjPr1xPo(CMD$W=lX?IZ9#P1|)mOp^%p+NeUJ zj2RFdz*2mj!sW$yxh2e{!b70;y7@Qvcw!Q#AzzeF;OXn_?e!G(kuW^3HLP4nQvt)u36$s$TcM4D{YKAqF^bfx^8OadNqKoWU6PAFQ1n2S zQ+n?8f!(z^loC~>>M}!@>{0Bn(vPw~wSK%0rFvK%ra(lZ+_>cyv5MJPZi)^0`IcRA zoPuYU7du!RJofq|l3nKfYik9-^^ln82`r!l0)Za=m>3Hn6Byf2pjjENPITztgM>}g)7K{<=YM_+!HoWdkgpM>*bUAHAH*SaYRUt%EztNGHBnOG4Ay~ z#o4!0`D8MyPiCKNTUE1x9_z9kRybZcR8rq{e#ck42Wz!O89@l zOvi5YwD^SJu)8mph8g)t!$P--eN)+CqCKSe*gmfh&yM&Z7nOpjopl3WY-=gUGf#!T z;k=<=BAJcz0TP5qGt)KaD|~EHOvaBVemK{;A51~Db5xAYgm;^lOpFQ7vakAWq`kf` zmzENVac@gkkwgbcR#knUbm~zjaWo4XQJ8wJQWQ%zj>nzaj{E&^sBXpFBYG_QIP|XINgl%3!A-=i!rL+fS$N=4l`Zk z55e>WK1P3&AMbBAH8p|u1kP15NGc(p@2!0z0@e@3Y}BpC#Ke48f1<6Zg0f_QH4qB? zTINCLkLR*n&7jMXZ$LZFd=)O00frXTgk5YG8`ksogvIc!wy*$j$~V)8tE;OJmqeyp zJHhJ-pNWo+K4YeAxzsl@;sQw)#9zloKU~n!i&`kgAwdR^U4Jwe*dz2Oueex|9v|Yj z71)#@Wv4ulajjG*I)d8=oqq2ZcbRz3+@|w z>A^%A`{MC2=mAH+CmdFMataEJig%=f_rtb%{FdlvV6~XX7e*riS7v6B@ktVp#@8blUx_)?CE~o0+wF zRCP_u_mU)W3p=`k4E+66Vt&(}V_xy$lFbUNvcU_>S~SaJ!nzeWxU|9RsOwfHYS~2L zr7IUUrD$zIBc3Bt34}pcJ%7}JS!9R(v_J7eL&~y5>Rf!r?WWTM?Wdw9>A>u z2F2UExBD~3U->gEpi;mW(I9NnK1`9TSqXBE59T`-7my@ZE?#eFv*eR0lwW$oECP5( zotlu4P*{Jm3UWAADvUOu+fCR6H)=LR{QVWOl~LQf(9sN9m}Ds^%ur+sQfIqCF(g=E z3+5vO^>qtcI!r?l0^8b30aQ=``TDZYH zrlZXJ{0ugWcYctc0d}xHTaov!j(&=Za&3UHcwS2l!0`}!XAiq)%UgmH2m*i;w;e3Z z&DVg{0>Yc!?AMo(5%u#k6i|pYPI?^Q1rWn!Ipsq%i_X~8ln^KY;FkU4#Rpkpa$KAn z3;|pp7+;2lhj(2Bg2hW9gnkyD)yn}+0uL4$#)h9qsuiyXW2O&wuA?({y>1FzilL z;%fBw;(rnW8fFUBhXMh4qeeaKOMJSH^YK=;vPrQ$BpVCto-+uEHXi zvgV<0*%q&Pu4?sEX(r=qdt`Vb!(`5#tlyRxBOPBVxU+i>t>5vLh$gz;WNnp>i|aOR zK)bw7r;OZC`NR5Lr|)kmGP}G&oZ2?D#o>J6(j~V;(6nj}qoC6r6fcb$C+igHi+OpS z!Da~VE$C;Ws(R4L-h=W)LRXa1(!K1gr0P#-YI*9muw4O-88WY0sQb61b`1g)r+~oe z`Pt^V-!hOwV3|?P<>cqDDJyfFs&$nYIyUIFi8244Uoo{xIS3hF>|~8IsEGqa6pi}Z zl;=k+KvjD|oup9-Fa5}!K55oE1CG0x7#LPyDFw&aboar*0mQLi-1Wa^ZJ_GO1wbF_ zXL#*1`%Ly0Q&Bc*V99G{eMzHP_0FJB{B$3n4a`x9SZxYv;u-1b+6RY6unq{og*(%a`kI@Y0U^Eu+uqsz7$Dfe?s?|Q`VQUc9wks(un&07`P02^?@Zv` ztwJVAOG;J(2hh>pPDM@quG#alHm;CuLO1;%D}STLx(Ju^uXZGc({(2w6wc|ej<-sN zy1e<%3_AU4GHQo$lKA!(VgBIZ5}fWIRnT>x^3E?=hbC+6@0?pvJf^w7T8){s*cDne z7_hD2Ry46(OqQ^?khml*@WbTSbKvJo3|jfocUtSjTyiK=~|6P51ck zrgx!gt|8kfe8+(rMiCaKb(mwy&<|b88Bx5U2c>;h<+-)?QB$ur2OC zZ1;fe7TPLi9J)UDY8Rxe2og!any6EViiv>);RKwKhi-LLbeL5*ij}SsML)>qfkKg4VVSM4OcwUoHvG@vt!@9IlJuwO(`rbECfI+Yq15*oGulDWDrQ3upUwQJ0K45p^6~D3Eth> zdV~}{=Y{MDw8SY$13b531@=O757x6=K(NWm$%O<4Ql}!5+~rc9*Fx3>>kfo99V4U6 zbJfizD2)V=YW9e@8v2jv`~{S+s;oQ#IVZq((A>w1UP*Z0`i-Uj`Tdvi-#1VlFlLON zTbuQ+jzh$-VK?!8_xVjAOa?vH7Pp?nvi9@%7Gt0|-^8CyGYU_Fasi|c(yVW3X9;u@gKEZ1=;>9F^(DNE_(o-L%KE9tAkuOBy>FE?zvmK~9NF)CD z+!u2M)8V*wdlqM{yk@v1IH*4GeriAl_XqUOalWr7f$6cRMkYcAR&EVF(x*+bQK>5gT&c zJ>L%XmPzCrJPg*(SEb~!xyQwI09m@RvT|gf$?rgCZvDm^p;_iL!7cMup9JO`*2$PFzb#cH@$}c+PwSzTt`PoRh2OEi3d1J z%sexl5)8rdnSOL`Tdo5Qh!Ly1U;kue6cbqytn0J*b57ZOg7GZ z_D$BT?X}NMep~hp4C9=AwW{H#tEa}t?du%u_Vya$dt#4!>09XhzT0Nyd`RFqMejT# z6PoK{!kcDp%1I#UpsNSOli%)*bHD=Z$~TbiIgfToG84Mj>Bi^3%GiLN-^*(=+54<& z)Z*6Z5~?Z#n3yA?Dl=2lm9*}FH=962fqfO|E*D7Z|MGj@Njn~3{`p9Ri3o>M=oAKF z+dI>^$Eh>feW5iB;wEHc#}rXF6Y^g|umpkp=+E@|1}S2AY%DG&#u?H5_O;2TigP&Vm60Vi0sXHo67x|^_Qtg05HfoR@@Wi&RvGNfg#Q|qz~ zC7o|WEX~Y7;~s>+cXMNUn42?&w(Qb%g7fBlIV$tAm-wsqPe~h=s~z+8N<(Ta>2W88 z7#9Yj3%pgw#{B&>$G`O2%;7tWr;+J?6tFAUA@a@^cCC+#|D845FxM?h?0TMvTz@UX za^vHSSL3|il~K!y&>qul@3x)gde?mIA zK2Y{kASyOt{{{{~gzO#-jRDMwdhNeE9jt$fQH624ph`5hv;ariHvJWvIUEBI0&KWP zRL`f!^Pzw+K2Um@RE&d-kRLRNsTQltSm;R+#`UL}ljqYbTmhlgXOyZ|u+@;aj4>T1 zpU!+doYm8lKQqmgzrU0=(!e=yizAL#ReW;`rHXaX;Q;5t!tex27PXj97LJ=>J0p1!* zj*6mU=khh+*-{!JVqurfW|`@Xb5{syPSBM0u3Gyv9PlSj=^}wfQBLqVGTkqyCb+685QN&BY2n& zZE85;Kt{3y66;gwt!wo;6>>r!?K{t>6^zG^#znavgq#8(DD9ze|2vb7Os=&3vHsM5 zx%=_Y=pIsqm#1E~+ul71w6m@E-(APWC7_uenUb?To@QY@bLxC}Uz(#X*gWcV=ZZYq zEm3Z!OIV%O<1xxpYx{Cwa|Yw)hYu+C9||&oY~c-h24rO5g#&@)&st$a5|^c>uI?1* z>qn5Qb8~VYQ^rk01`hiU&|QC-kN|dUpjZHgi6#|=Yxww1Anf1;HoZpzjg#*EE42r(6=fv}$EIknA-dz@ z)rSp)rRAG&Fw#nr%2Ezo6xMqaPgE6m9CZ@k5bW!78`o8Sz|;GvCo`RNGxl4}i~`L5 z9`~gY*JLR{X>T3DcJ7)<|9Of|=y$1xYOaF<_fRzlfqt*c+6h|S)#-ZAHTt9Y2H@)I zuPVA}8#Y4&KeDqROOKTh!B4v^K@9{gO?NXop{B8s`?R-PSQO_Sj zkx`IIk+pDgaLB}SVXvQ-#PitdQ+9D(Q+p3+fFk&kjVuYG<)iiE$fB_1c6T*Cg7=0YmaGC*@X&`Vfyfg)>%dA&k z=eEE`(p3B#4M*U)mKN!qJM-f!dI@I{@YfaSh5kpLGAdN7rR$eoAi`RWA9JYS5C z{CmNxq!xPrsjnnW_+m#)A=qhG98R39rfNrxN?^1spl0yRb&8p77o?UU?bjTrzGT1; zKyll{4QFMnL_orU^!V8u6V+6v>I+eKb!}drqWw(rtqDh04W9?xab%ZH)PV>DEX2;v z?)h{HAb$ZANkTa!po;>{vViV}cB$ur;PaviYvtz7Rc5SPFCTy8cMI}Cv%wheBL`ya zt{rrScdFN3*F$l`>6#c8Nc9j85exBI?!dGI1O5iJA5xZ9zSo;MYiw-%Hb^(^iX<N7b68vY9w+2Czd-i3!bx7b^yu^cOYg8JDY%+gZV`-153#(uCbe_9Irs#~eiAWo z+%#i&x{%k!WZ$Nemp`>SNBl-REqF;||61VBpW9*>vZ68)JrU>X^^z$=&8ei<7oD zW2t8AmZs@VrmD}Y95P5lZh*pOqyC3>S{JMfT4ChjprrXk;wwd;1zY|1GnMp_qzzs3h%6o~#bfK3wVeyeON@;?&bqwg`#T zz890~7e_iWy#{h(D#4`}pJ6f6-$hM;lNSH^q}!!eE@3@^6vPBIXv>t;8bT+5mtNcx z^+sLIUi_UF!FPSAXJB2rWE1@KzsE)`qzb+4x)DPg=iiz@zqS;n>D=Sf7I&v_eelf> z4HBk!1ry=Jx2Qv0nqll6-esYeo^s7He{;WEAr@b)FuFweaC}L|Ux{sG)%Oao)x`&) z0LWbRd-jDGSnxis}v3Q0` z(dw<{K@n|=`G_IqSY|FJYA$WUBr`#4736mfCZ$(D`B|D*na1-`eniysB6etRD7_@l zR-%fX3gq2WeWocU>u7srx{{Bfuu7}4Jy1*kMd_gX!C<8M7d8f!O&Th5U5oIfKGol& z6%V;dq zW?9Y#L{(xnski{aKW6P|*~~0I&ux^S_*|ULYwcgS_pvevRY|?SC){i6l=Y94UJP5Q z;`8r#eg|JFFUl>3il1AAg-|Mrp(>WKn2)DhS$$D)fsm%PNwi2H<(}H?7e-F|GmHW1 zMMYBsserc@`?sL`o+Xv4RQvuuSAAjU~yPG|)oX0VcjVG8fCASKn z(5?iLidwcgJyftydPl<$-i@Z=n*M&zh>oEr`{PT$TI4ex@@M|7RzmVBf)4go>#Cu{ zrBp=y{g6df#4m{m>kt`g2ERvI)V)t`#(=9ZRj%co0a zvq;{?3lpfR{xn>r7WP~_aq7>?0z=zNe=*ylp&j4w&%O2xD$fRTI~b_xnB+ETK_)rL z#An!G_%~WzpNEE2QK5kY%qL>h@aB_q9@`0a;{1yXzcl>+_epa9=O@pdv?ahUtdU{l z{Efv~M^`naE%?Dj&-K^l&<`Is?5ByzQg7qMY1m*RseDTYiGNFqU;O{0uReP^gI9dS zO&%}H=E#&iI^uM;&!6gD-m<~cOYf!ml7SUg!d2Kwt5#I*mK*pA-7c6b&UlcN`4&zK zY7>2|xHR$Vn94_!l-k(ekHP1l%&s-q>UCW@zeU;^p?~BUC<3GFt2~5=ZWmZ-9 z3o{279`Sn$rln!}dR+;V29JJe>D1@;=VT;iV7sLaHa))LFPf&}jDHPvey?-Ik`IWU zrE=m~*VT_d6+0N~XN-Rmk=HTQ#MGv!WV*hrJV&M)BN>q=9f(7nTmPkXIm|p_puZHa zA|nmyUosp$!~>gbB8zO-*j|RC9MxRhz6zlnUg5w>ICX4uPJ4BTk{!$SG|9e2qb#wK z;IkjTY-vx^pA8wVElE!_=Y$A7tf(r@n|g>!?Y6`6%5sXkpCPR54P*TyxA}B5Yu}eB z9~Tl_4CTi_mTOGIh(PUqx3#J^N#fYg(xG7(8omTfDp$GSZg zH$$BsznwbjO83f)A5VNym;#KoXf#1P=iRVjGbcP-7VefiZB5J_0a0Tv`HH_>85IoD zBMtp<89Do|gbuFK=r{JYnKGDVe>7CjR%Dp@%@yL8U=$&JJ3n|&CAF`WB(y_TE~1UL z-y~8$njm2C?$sW{=GG_w231Cqa=7RJyv?Ux`+55@)&Q?T@clU+D!Xx=gpROe70Txy zsqq-o7MF5T1wLp4id(d!9se11<9$%e?2=^bqlCfdX2rY$^#eP!WVlEt7viulyHCV( zWW(@mRX=>NS6MW09Hh3frD=M8(UlRlFX?+hzv%M}efh%D?Qx8`BS|K5j!N+#OKaPk z%5{mUHM{zw$CaW#Q+xDMT`#DuQ+3pmD|b-YFfXDp%e*IEXJV+Gk@`+zo-{7o0vFLz zyc;j2M59_`76Jq9Epg!Z#RQtk>Bm{CX45Yd-zX!#D|q%oABoPGaU`*TsoF>v`tH>T z3l=VKmt&?8Uch455)%#DX$mK#<^-1isbZ8V9e8}w|9?eJj_3)A>h2de|-KeoS!@Vw`_iqbOLU%EF2iFLAR^==tr={6W{mi zx5HRhS(Lb%1Q#A7^;L?rDFf%4NvD#Oev^4(-$q8CZ-!s;ES;(+1KFqT^8RejTs*Zh-dc zZ+u?y`peq3zPHE&P;uz>neAMq@gvQDxTe7NzkZlD`89DZeV4yQVa_kZr#?@=_z%xL zp$tx&U&S-T)fft7vXG5x!|OVtM;++WOisI)+6{B@O6I=CS(b~p{_nWC`ZR&p*q52A ztXpbfGw7JE-ZmB>->eWjvR6$PtzSIN{fkI|ED4R zivZWtX&Ho^XCAXanmu}QokIvSRaUIu?=p>0M5_Ut#!x9NQQ|a5QYRuUVC!+GI6BxM&HRzyUCTF6X7)eC-G>j_?h64*z~eGqd&q^!?*pro_sB zpz33w+iXFXWV^FzyR>3D5o1p4L&@h-8Fr3K?Aur3r(V$64KI>>T9A!3%T{L&Z5w3J z_KmZD7dCEsxJ6@@p-dlbZZFN1k$Z#?Rgp1%WEXF4=}{&ww@_9kiAT;rgpslOdv4AG zTN=^GhCpCFMKs0+GET-#Ob}P8GpiUIOGIS%FY#mRMZ{)D2WIS`>tH3L->lw~tWwUA zd8QE7weaoX1@f|K!dxkd6Z8!lT)^&-Qz|A>7>kkflO$uHs_S~XP&Cro$D^i7_SZYxGu`wHTL1ChRyE^xf61iiOg}B# zO|rTNn&jm9eL1l+yZl{Tg)Ibh><;S~B{k0why~WpEfRg7yz_9^_q-_kC@My{(JE(G zp428(FJlI$u8~O4^r;X@U{?Q#crN)TvYz<|2K;o@?&l%1oI-ZxjP9;51&EuJWRLGbX>4o`aJ= z(W#_w_1>Gt7{d|C;2#6aFQ`0={D_NHm-1S(OZXCCW@QC}E$~9d)4WEt3y;dh#sj11B&TsqXMPUC%5Etm@cNB2+ zUh@7^GNjY5HEhEuc&Lg&-@?U7KKiyd%fshOfT0m>=p$OB82@>OSH_6@?!Sk>UQ77s z#fQW6GP`g7fxiDlSDD+>w4Xm8U#XHx^T+?=J8XNvS2v}6zFx zn_?gMZ$fHQ{N>!df}HhhdF97bQ(xNt;5i-eE4#ECC&f~Fta!tDg*nbr9Brnin!KZA z+AVLos}<{lRkBv#Gh%YDqEcdiW?gKqm(;Xb<`Rt3`kG?seHO~^=BM{Ix1y+&oU*~FT9j~UVv9AOEH6Y#-WTHjHx)ci z?1cDy*KdD(uO9Z=e*AQ|zlxj-M~_C|kDxecyw9_q&$%SK`?rCQVuogXcmV;=YL}dc z*~NEld=^qEitM5$%=?Bl6Rj%7b+fpu(c2~O<5H{n*Al*KYr}=xkd~i@<9|P<>2o0p znOM~fE>YFrMq9b~g_&ZEmCZYBik5NZs(NaXoAECE8h6^9eut_p?~Cz@18Ts#Jikg` z@N9^9&o|Wr=dp)^y2p>GQjXuw!Ulv%r!&`#IyMU%%z3Q1P`~&3ve|QxBsEhCMHRlc zx#=|wn=FN{_=#VNc>xt;{}D8FA@0}G2u^ZrL(9mw<>_sM{&>zFCMSsJUYrpYqE*au zriLRNgo#e?@4QqUhnq_Oj;?)Df&(=LI1(M zVfU9bV|)1G98W~UdStn=?)uhXp^pFJ#ah}KRmi-Av08)uSIxm>YjNV(tzBkohot8m z8&jlbN0)E}#V*ct*oXx$XFy4Rf!TiQ%0WMGF-f)t0H5L!sjQj<+|i)grI6vcU87e0 zeJiTs%=RBjO#k4IhKq%2euST29xraO1miyM1>_k`_$4(RmcXDuxIrr6oP_2NScv=& zo7Aqpp*_a>jekdM6%(dZt5ZtS(NW4HI8bcW(lZNR&N0v7ZnE(m)X3~U?Ne88k!Q%x zW{_JNMk0Szv#vfd*X=h$6V9m2%%qYv7#jCFaeo3Oukd=pa>9V#Ho6jKm;G_2ERw2% zMhtlA!8XmCym<#9li{5cC=hTLp3QPkt8QNZR;lW$&dbp%hA_o!0X9MY#VnzK_x~Pa z_gZ`J&f`v6f>%cZRNgmcw?7rc)J-&zEtm7xd6<6cJ!jihDxgUDS0-Dv+}^mwBF*`~ zt|*2-{}Cc#c)V)V{;)^BIiSfSW^m}39~tql=W+ydX0C@^6CFvV0CBu%GEYIbGeC$V}C;y)6&m{auFJ5L1RjB9Nt#? z^|E9P_w-pL%KkeUwNerAKMgNyRV*PIIoGB}DeT*rzL&3$EqC+#K1c8U@V{Yfx`jBm zpItshOk`sd98s@JRF;+n11xC&Q0Lr?t=L&|t@ON@B?W}T>fT}FnsE_Irlqcabcz`Z zgz2p$6KI#vaDnSYLGc8hjVfON=l_bOFZBU2`rYK}#kVV)@Bdf&{9h&Xf1kPZ|NhTD z!@0bU4n~eD)`G9&$}ZbJZI&#Z?YZ{nN9HAK_Qg``N6Qy6^s}}7p!JW+|M3F!sk9dK zv@u#P_5P#4{>vjSsl|Rg6C^k0Cc)ZQB|>F5`GfR6GA)=^_LkX$b6W{D)28{Hzck1w zd|4GuO3fJD&jQx=t_5~A;Q|L@@`_LD#QAz+xA&(3i(b-vmJnP?{JZ&@okxdpc`P`f z6_jy_Zww(LHF&9V+S~A&jI$(3sQ!mV33^h{Z$i~{!J@R@k7nn}ecG#Vv1%_}tMyuE zwToxUU7g%<0cF6lzSZhWp1o83f~mu!ocE=r3nnR(&yBv?qi$>m7K=a1y&G!4%o(+` zE2skR7C4-g-MjZAV4CcDHVQQYK`DxMHBmhP9WJ#0rf zNd9+dQpR=dCRJ5A^gZ`uBx#^(sSPh|+z2UKLt!Z6l8W z7SlG)$eosSzna9JbsOAVyc2oW@GZexL=sn4k54WZ04QYaa-XC+@wjl8GRSamD!1K~;+i>n{p*JH9`@>IV$BRR#eK{kOXyLVBsYlRmemZ^4&#mHMWheI`1{Xu!mtdPWD`Mml zaxKs{ZcV0(2ET*)6<&V?fkA$SZDW9X^ZeYWUGg3b6mfG?;PTJgMN~q_jk{lwm((q&j{al(2L@YD>7j-d0L=yOKOJM1JI~-XBCS$;ANKI4VvX3C)Q=O85mC|_Z zYu8z2?$ZECAvSd)RX4ifRI_g6sjHj+*8_(Dw>&^`6{k^zHCgeMl+;d zl=4iZgBKo`o+hCvuN8jhibY`QY7@dbIUD@q22fgLAtHQeWi!=;~4!r)T0X z3FuNALCP3`*RzbxDYg&Vp^s^)6xZ_C{|D27`&iM^u2e)%NqIAVkH5*8Q`9iM(d-MO zy{RLE9nfF{L#t~6JA#cj7IF+^X#(SKqVD4i2IrTuG6JB8(Yl~+sSmZ^^a{(79b$N3usKNUbT!e1X){4+_~;KiR7_@ zVj=LAOQ7i<{QJa_?TOiYMfj=uf#gtYZL+&tJLSZv^ZWL#T?||sE3SJ)6~Zbce;t$# z|H|@5x8y`hWw7zt&U+1ivnJ!|8ABuQ&75Hd6swWotI_(=~_IGKnCTN3~aFYu$1PGBejKP|UFM7~^(hg7LjPEm?#b*2-Y= zMHO55h9OY|B57Drr7!oE-DE?*FnQKaqT6^hkKymYhm+kKy#2PGIh87f$9hiWBjKy0 z#DwuDb0R&4onjvs!Gpa8TuD!8%5{4CRN` ztCcx3(R{Ur1#AWO%=d;%EvpKPkeOB!)7w^>JBWGh3oO+v^P9@2Bi%Bn~o{~6A*;*~*ojOJZ! zNF=z+;gD-8y?QcyjER|(BSknl^>s}$baSplCpkzo+}8PRnQQE{WWQwaw2} z1mblyialc1`n6UBSc8)ZI8R#5v6Dh1E6+zNH59I-{$zc?GAiu$(p6S=P*bA=GkcBy z=~URYsoL%@Ar^Rt z=4OLKF(_GYM0IH?eSN8buu1+=>C?nHXr9)Ld6ZIo74v6y_V@EdK94?8l4L(ttxCJ^ z%koZ+-HVS*XMfzd{Iv7P)jT?cAD;U1!JppYV6F*Q1ZR|^u6S)EJ<6O=JcyDtP`%b} zc3rSO%vO4sIp||>j2N@-?n33O!t?w#=NCuqWjGz8SLd^swM!iCa+Yb=-^6e33Ap>W zZ4H}(%8v8*x!pt_{bk$h74>!=Lt|wnhAAoYX{rT5sc3}`jg`d6aDRWu>^wroq8Tw0p*=X-ACzEkUb7+iU zE&bcWMY@Nu|E3bMCXu^+H&3IOa;wm(eLkAL#jtDF(GfU|aCOdKz47THF#pXukLlQ} ztv3^NrOVgm@Y(EnMv{AL;%vnC?U%j@y289^D8#owDw1qxJ=D1!7l}_!M8Rn}lm{L0 z+Wa(igA&hoVi1T5VbwBwTYApAkJYDTzL0)ek0=NxuyP1QI(gy=xCElli@GLJxA5GZ z{`vGQi7+NccuJW1a%jumk!tk;rop^R5+M%K%yc^>iK}2Y@#d0c!6#>4QtQ*aFMZEy z;wtGiews^o^rdCY`VHGNKRb3)%YOEd0)v)nM=#mpd+OF-Qcn9r2{GBgfH*3NRMCVuc;z6?6?et#KTu{ZyBt@6=-0&PB%0a~?H} z!%8kpAsmky6&p?@RchZrrt9=%v90$nSi-nvzsH6ixrM#7jePCb+`+~1CA^$MT^aBD zqvDLgOjp@Wijng%CX!w5cGaQCPA&D9pIV@Ig2!cN8`plfo*z4Lc(Jc5(XzvGS&%Ac zmHs`Sgsf_|OHz;E>NIN$DM`4^-@W$u%L?OI?>c32ewF=-(A8jGtVA{aAU=;5=g@$< zf~Ib3?(L4*`JVL|H`&(tRENFgDK3_+&ioRzJPRi7TZ=%XM4&I2ZqHpuJ+S)HiAr)% z8az+oIjifdp|P%bOhTXfnw)vtd)@Q24`%rJLQ~Frrv5gThNauidy9rUZc zH+5$3+iRugHq}hd6hXl{vO0H@vG9Jjr>gr7olV=({E;$s(DJ4UH>esL0|CGwYeSGs zTQfL4GwVV3D|Mf@lUoH%#(QlAb)E@_1!C$~J6ih*!h;vfj$XdBtu&rs^0JA%b)+C3 z7NXx$8R<(8$(2aM~-h43Bp5()lh?d zsfC$U>KX4$F&3_ZxMNG%UWRH1hkRs8hpdjfMdIGIt6q*?pOHQvbrQPpo>58nFh{>R zR!PEP;foo57G322BRMWfyyNFjxx(SK-$kX&10j2cZDx)IJEu7VV{8ltG0mAPrKcl& zE7&{Vo8CmXteJ|5ks$CBD2S7$ckf;)NTqiiH6zj0#&$zq&-=ONuT+pUH0x?@<7ivC zKdU(=dw=xm&N-$r*oC92nIpo6BM9kI9doZMxUvp7k)AD7_|=f9CrwVPSzs zvbY@&Rd$~_ z>g^xM{^ZkY&tIc?#%kteBlyk_eKXMiCRYWOS~CV4rbeW-k=6AL#`9XuF>XDf`4R)!hVrG zRaw^b=U+vVz71@ZiNEzrglkVF?&l!I&ElrOY9Y&js+Uuv9JM6> zORoRCe3y2pF8=+}!aw@&QY(rtg4f6YX@|#pf-lOR7+hDdHH+~Rc(lSaGF`| zgq)f^}J?f*sFTSvw9b=!h5l3x-MJUE2H-Q5x>+=4ptLuP9gZ0hoP?AaoUzp{3DERVJgTNA5z+LT zLtUXD6nED7HA`pfw1L2dj$kMt8cYpgLn2@Tu>}JvuqsUrv42@rkoiT!(+u~ei$c$; z)Xd+qtK|9%jzc+8*ug!c;|Vl1BgEp;l}3zd0?j2-#9I3vdJv*W)72N2n+MBlN`H;_ zF2l(;8gFmk+BBW-L_R<_?(-M8g?m9DS&W2DE-4*-I8zOf4%n#kWt~`%o*m)g6 zbzJ)RNrYu}*MICMcB@@n<uL5EIX1DyT0qgi00`7g7e< zoJb}5;U)1Y=zmAXz|DgpF`?_Iz~Wr>=dXu2f8}}r4ox%VFy6V7(fUckRfpa z3bX*N-zVrz1*~^?bFX++?5qx<-3e?u zZd|*oh&bIL7HKqdqzzdBB#rK>9R&j(Wi39b}*OT>gGw zC&-MPo|bLAka13~L}8r0?t(A6^&M0K(%9cRHbsSVrT?qVfLt;P#X^{5@n{!8KV{S<722 zpGRekWnWrJCwV`ZkZd5O(&No;*+1X2);6j_tWQRA4-Oa1NdNu+AK!*Fo*XKHM}B|! zMawpz#;qf#>(=zb=W(X+na2<}HzrO8BA3YND}$#YZPk%I{%B%|AY@s(WUKGrLwwDt zjfq>FSc&46_6Q>w10=Oo>mP5OxF4a7>Y-JvmSXlpmdcX^!&yr%5ipS&q-T_!RNH+C zo|0%*UOkH4{${0L0(Zru7)8sEQ%z{_Hyg2BlfQ+s-e(hsInq(T5yrsAwc;QMDzg0Z zbzSn0H!%J57B={?kxx^XN*Q%U?Tth)D2r^jPR0+%0oLW;e3=oJ3{mFLjjLLb-UXL7$gG=;diZpxPJda)t^u-wS~;e zG3;J;4WBhgE6z(-1sT?5h4^CAo(L4|oqW~O|ACmv&b}TWPM{jctTyy^|QK>c%nwQk7#$)Dqy-`5VKDX zNApWwQ4I24t2Qh^Oe_fg@+`Yv4C&c+8+0|7%-T$G5uS%i+Rt)oaXU^)5T9heNajfC zGLjnagoSSfW)AHrO+rdMK7Joh1nZtg!RzL4=@I{w>a~ybNR?95e3`mleH>cmB6N(5 z0;UvA=w;5vj{+K+GOTnT*$qQ(i`JOPNJ+4bvvN=6$42EDja{pBQEe=P_Z7xYBG2XS zUNC0~hgqlSd5qj7w%Esl-y?P~383TFih1R2SISaWFt4vFG2bKf@vBaozPkcPxX{)l%>|9=lcB8PuLNEKf;XN_{EO&VbxJp&xBFzDD2g^QynV46V!?d z&E;bIQhmVj1%8{QG-qq0Q#{msdzBIy`{sFRId~(Z(K6#n6CZi;g+|)B z16r%PLxruNF0a5dR)RFcb*5I8o421SKrwYTlV#PEdu>p8)SUt-hn;gs3sEq#j;2Is>PNS^yz46|9j5O6 z6-SBkh#jCT?Aa2b*eR*NT2r38N$_?xdcZ%d{44dCf$ft1G(3VV^Vry_JH2VojEvNk z6FaH>+JBVOBxF|I))4hHDfRY1p`d7rKE#+_aWshEW%tkguq6ejAm=_&iYm>-t6i-t zyS@L4)ex9WPkfx^{P6*2{|xcWtK{V%#xifxFn>_%n4GKkDUHLcfT~x))`XA=@FL49 z6%)C8*RxJIuy2MpPXWEC z$yWMxZeIS-!QPLN&0{_Y17lw$xwmGyRvJv8d84vg_Qm9W_{D#sNrpTQ_GVvU>xe0C zlq1+0YYC19&@_U^vGR2Ui`x!o=QkGYH`}TzeXsn;7wDvw6N?hoeFo9hct=;OT3CS5w5$t4Vt#rI} z{Z}PS5!yw4WqXt)>ZtUYlJ-4>OTjeRS$vSiBwl3XtW;mF4C6DQ4>Df;qJhbR>b=GM z`p=Vnw;*3SQyU76QWFv^m^MN&t(KrMGOX2UeABNnwCO^~gd9tAyoZmUXzvGch!zgv~oqIct;W!j4~ znrjBRWElp+c2jj{;R!9O_&kF&#^o`Wm>gVA;o9`~;4h~g?N3s)qZ>9$7x3LKQcyyV z#g~nwkaY{wqq&GDlX9+Aro~-n?yCrB1N*34u^Pa_P*|)M2Jw}23wg(QGD76`*R1xVg~vzI$|)DhXlwEP2*e-HIEn5+8m5M!rxZ7gDSDqNRFsr8KN?c{St41~oUbIGC&ycMIs7n5o<$%S2cfq(T7} zYLsR40|U8MnzOj0TQfKu;es|ZOQ(Sp82!Ot_R%pV#l;j>=lDg5Gq>J4Iqz{Nj=DGJ z)X{|b-bue84?X=hDW-O|pHhr#Nqu$_x%7={F~ztnUdXl7Q>PLUQN12vOa=zy0asMc zno0qVgnuZc#yO9;H#I=Rf_Rp;WkQXL=TS6rPE~c}iE?}~GB8f&qLto{ht|IP>7mqN zXq49c_n7ajH_aUo3ZQOYTUlAT6M7Ja1eKnajNz?2oo389O=x}8feT(%p81Uvd0!>< zEcz_8&)uBV823fpbUhS^0Pim4*zfe6A=hgT?7D)n?%F6uGI8>GUvPZ6Lx&k%rOp3H zR7J=iIFvH-aH%~*HA#@Ni&|A*g%rhp$D`o}hV@*~Y80;oTWV+@6fL=T z?`t;^{C_h{KCQgx0ESdZ<2!bFhTdcwgntjQ^rOf%C2E!dJ)daFCz~zRZQcw1U=Z(> zKjuS!W^Ytqj{h6OG@bgtW|&I3#2@^T`3vCm{mE~$nX7x>egftPu)yA9;eS(h{u6&R z_k(;+zY+kU03rtHu7#?Hb9rk%*K1Z?!beWrb1U=b>=*#m_2ARuS6sOFr5d&v_+0x0 zkb7^u>r~X>M!FwyVCS^w^_SV2=UEX0)`w6?<^_NLWfl#B;QW|4V*8f{d+-hTD1{4cq0pmnB9oa<&DV> zH!NZu=M5;Fu~!b?tLBNT@m}1)M(vM3!XO>Rbjg?~zE!I+u2UuKP^emdabCGVu^OzO zeVSMgJ7$^#GuYbJ)@N`2yd6Mi9#r}E?b(LIhESAtHUK*MSACR_tOC&RGl~1?^i3nM zHm87smanGbp4LH_NDxdo@qA5&#<5g|zT6~8DpXI7AwPaLk=W1l6fX>`0Hz+p5L2^khNX~v$abYQ3zmP!N z7VU$xvYPtEpRSS`Ng|nVK^^gdQ}X%89R$mc6a%LMIs>-$&T+j^C^kNEe;-P5^4g5kRy6tgh=1H@B>Ka5W@f~Vh;Se!UDW!syv$%Or zYH4wB2@I(o2$ejFAg!db1opug3uAlV>$&g*xN$k3UjtR^;iA=g?&^d_8grHG>uJVN zu3R&#Wqe^_bnU2W#Jd--6D|5Y1GOz*WQ`M_9?bqR$38$!SpPWu6f-z9;fuh?(GI84 zLb6j*?Tda_B#6GryI4e&^haPgSqM0^RIVv&OKIk0E4?4?Si;qIb2)&AfaeHY#1T$FJ>1yv>?~aZdrpilfYP&yuhn!Y{7Va@!Q+xByAx42$aU4X|_#*3VWP4rd~cJ z`nqfFx}v2^ysy!H;zkR?g-^SIM@(E_yeDLf|5~Z_Bb$y`NjpEjLakmSCPyxT_yoFS zIkA31d=$-AU+5RYC2yj^9?zJa9+21NOJMWQ1HtTB(XZ9h?zM`er*D&g5`jmi9hR-d zpXN7==tS3Zor5GPcp!R`C9cy$CN$VO4u{13ouL`o!O=P2|3pOeK?er_XBvR0Y&=0C4QNM^4>M@_jA?@$j_3aH{thhpav zXJ-@VriR5k+{My$&U!v0&U&Q>ZgQ8`L&e39hl@pYu zy4v`S7D42gmIH1vu^Lc4MSFETv+@Otdne-XOyVxtYuoTNZ5j=OdKnu z5_XbNLVC0kP<+btPm6jaqv^=cwTWZY!1&*a&0wZ%$wbT4qYyq@3s1(2N}8{=@Q@k5 z&p?l0qe|>Jix6kNw36>CT||Y=6B@>DVR@3ny$(2aVIIRWM3H|K6@I!mH<-BXCK#lbqv*v;EWjXIxvb*K4s8&wb1 zQ~rIEWMe>;Wk`LQn2RIGVqT3or_bETMmGmF&{%B7RnGrRY9#!$(_s_-GN;@8dpc9b z*(3!1S@{R{bK8Hw+HP3z#kScadi4RHa&B2;>~)RB9Bw?1vIM2R6C_;Evn|-2dxdsE7S9ZoD3Lc1q@>T#m0*0p!PfxJXA4P0^V_M z=-=>|m+imN@-JBEj6##_n0+Dxs@J4@F2Zk>6f_V0nA)WY-}AW zwd(C3>~<*m-;S5VA1lZyQT@Nb>0mpMjDi_=LN)h9X+MNkH~TGUcW?k&vH-VGw6|~| zFRtZK$I$&o$wR2)tHU7g`29 zL{iAzpJ*>T3`E>;W$iu3jAoJxjOz;d^g5$*6zw+cjx70Pl!Tj`Lu9WR z2&KucU(~u#f|(>!8xGg173GwF~Xy0qwm{Up@ck=JWHy zJxx%(9RnkF%AOH2X=#~sQu0(CKJZYcPf4qz%9zxj@1s8vhdWz{b4)I7-+m>HN6-$5 z{p@pC$HvhCfHt>v$i+N23@+%C{9D4P#9Lvwo6}>npdC`N0vWX85xsPM7cEGFlFJYx z5jOI;t&A*1FK)ljSfNl+cgmGIdd>VfNI$=zxv&2K_~wP$nHC7qq}+?IJWg6(I`g5gnVI zZWFYcBtN{J8}nyVNTU)j@wh@-n_5!1vA9LmN&rF!-#UGpSzaga=sl@jnHF7!7*vT< z{gTg_A-a}JkQJk+Hf!pb=$+z>Fqn~X<)8~g8WG18$(d>bZNw_2=n%yZ`fhA1;np!T zrndM}8lj?sjY7)o;g@41)b!bKucO`nTdh*#szX55#k#rWV4FeczJb4FOx_SDDyMbn zYaPpChDoNPb=Tv3%%RTq@PlBDr}?zocp<(8D!?tc*k+AJP(>sa;ORi<+*0$ND!O@5{!_SJ=S|q#N*g98id$LQ?P24wSu6)6v5J;-AT@{c215NK#$Xy z7)L;{3C+{vCpW*l5Z}Kc00<;4B^516@ejwxJz&HSe(tAvfd1Qg1b5Gf|Cordri?xV z4^LAjYsbC+q-pa1k(=-ncVRSouEoxZ5a-abyJCJ(Ogu@sWL#Y5E_vWx5u1X(>USW# z-b$~`)wTKQEn5*=eoi4vC_rWp39}o9CJ1IHT)Y8$E63A$POoU}?Ssgbao1!j9Rs25 zQBwvf$9o`FHkZ$OrSaK*_sJ6TaG}0K(9kLdg0hO+>Nx37z)%S_)9{AfY!Rd4z7v-^ z6xuD!$+Hm~KMLd`s9RdJ2rXa)L&1aHYdSm?IVYyYdU8UMZ^fDMlJSX-%g(EgkGcEe zGbgEEH%o$Vk}xD_Dy~euuls?N1=cUua`%(Aw>|(-Rpfjqj_Fg~_~M1nY9`GDQ+2G9 zPH_vwdDJKg22!Igv=b|ql*9{%S9-v+q*Pfyx^SYF){JtQ6TdGGMyS_K;H14tfG`a~AL zY72hwpb%RH=8GUvdUijc#FPai_%lj_jVUGe>!9oOPd{0tty%7;0vLv5b1g>;F2u2b z=grpDJ<98mbY%@iiD^NsB`%#S=H_@Kf4u%;k+WmXk2yYCvL0 zVXytsTUKjR`COrw|4OWBoV!m}B2&c?^s`|vPdQ~;gn=LEHv<{*duBwxI31DnASGC#>aTJ0U8gf}1Jv`ZBgP-Tr%+2${@z^F`Eun!({uOxOrDH> zfMe4C&Bynz%pQ9R55*T*_vU#pS8J4p8(=+W!Q@&(R9KY1e`OGQQ$QCU=dcR7?qb)Y zhvRb2h5JVwk!1D7U8dRW3brTh)=qN`em1+vC+OoK>O#l{2+REvw5gkRQ~o17lY#q8 z*ub}h0v5OC7FmhVk)6HxEj#5_N7Utz9GcfMLBD=~empRu1tj#{4yG|=PY4n zus|1tbJm$L<2P8p{A>((=Cfx$Aq9kxo9WrKF5yDfhA8~I+r4EDQ8;2AUq=L;{S&=L z;?Q@DQfSx6j$hTZa}d1}Xpq>Ss2P8Y5iHk@`|Vdup_fa-#?)nDT=#doaJUW;ERK@? zj~kCyQVkL|N~fxJ1J8SuOz1$)dT0K#7YA8U^wo50Sp_vU;;~bBy2g~A;=fGpCR{hi zDSw{r)Y=j{?V)Vc?-4;iRrSJp{1Kf)$Jm+fmF!eVEj6Q(ZLXMsf*tVbV(lTP4Lnb$ zq?4F`XQ#OCd+^P-aX37gw#Q+J-LM|7Vdmph2RJlagxzL{-=>DLr^t;??Ss>hKzS9I z&LFgvea>N(&<4H_ajS2`CZJtpsyv7KDv{_HS9>$Ll~FZqSmquIsB_^&1XLD{kKv=W zN4Nk2Uu9T>Ss&HUm(b@6RW*~mv0I_1~EJZKQ=xVo+CD%T+N_}DDvNZOA zb?SP*;rv-&?V$|`$gt9-=E#zlTq>PsKm8N9AVN4W)@ zV$wIfrZwCa-qit;Uc+{5&M1z^ZRU(f1Z-Luk6`i8#fK@!mZ0KvJ+|og3x%ZKDX$Hx z%f3V-In8t~KMPZSzsb64dI1wIf#>GFL={sFFFPiS&+GFmewiB`DP%Nz38{&Zi;L#3 zep}3HD9h{1E+Q?Tx`a(=YJa>{85=P_n~l}9^S$vZ(%ZRd`D*PJyT(Q6W8Y&JA?Rzd zJMhvO0fza!3w7`M__le*mh@7Q5RcQ%V@Xw*QK)o~LUVay!7Dad(mCmsjgU*j^|Eiq z>GFQ0B>|Rz&YobIm~{R2NttKvs-lrInw)#5vH8now<&!gQW$6|oRIv+d;hdDsosls z<|J{jYln8mrn9ryMl@4X`<-q)srT$6DeEj=xQl~NH_n7S^U2JY9^ZN%HQMZ77ptMO z`RBj>Y&1vKL)M!`Q80VYSAF5PA&i|bCj^qXGmu>ug%gwRxc=5uZQnHZj?R7ckyGq~ zy4>|@hB#aTm@*glhUwt3#d9l>#uiKq+|^AIj^MPw2WhQy;+ZX184YGkO(8y)6kmv~ zJk?D)HaaQu)q6uM%ptBU;FfJ*^}So*W%LB)vX*yo(>ssvbGWPwm=)L9^1afM*iWtw z@4Se;$yB}=SP74{kmmLwB!CXoYiXB!4k5=-?A|wiuV&Glm>~vEoNCY?KdKkap#sGq zxMq6QDF-XWNf>DsapSnJM^@>4jSq3h3d{$QCBL`U9%-|U`DhZuEBXwugC(7hBMx}I zZ%dTe!77NEor}v;za8ysOSkexzl!M8rX4y)(YSa-*%vrM^y0#xN~IVFt;$ym%9l_v zaMa#7^S;pwqNO(`+&3AsQ!Sb?DnA#=p#)(*=8E-*qwBh0?#oWv%8s`08CiR)#L3w& zj|e-<5_3XcZdRfysukY^+q5s{dm+*@*;ZN3jKW9flt@pw#RW#mfR%i3R(=3#X;WTN zOUBB0g|)jMTkd0Z0kieIh=X=FLkd&O$ME*{Vzyr@sWoXGwD*U)&{5S_!JPsXlAe1I~>Z#Do?!*z@x1oehLIw zmS!G$&bWTInxboe>W)malsw#3Ekq5RE>CT31bpB=&tq3D?BQywnv0nHUspxM+|U!1 z80~K1#+#t>8FojPJn&wS7)YE6*R$WGnpeL_lue2v`gA|$S8=@M@YRm*GL|9~`iPtx z35pwWqi>35TsjeZGW(Oade^CwlnODBz*axM=S_EnGN3)sG;(`$xTq| zR&zDji#mOql}ft3jU<+-Ucz?VeI0)>s}SlLX=U5VYFR&;XM1s~3{N%hSV2FWaua$W z_4N@&QwMe7?Vqz}uf(-iB1JG1r2;-`ZygesR ze#FSoo?CU}uP9KlD$$BAL@?Tk_~$T$`ljDaKqw`9QlDakj&}Z}E5Fy!x`|mCMl*k4 z*RP+?2QhnHRhOz*Xo4{}hGNmNOrERNgMLn2{i)iLpHV4YF~)A7hEoPNdyrhklJZ;vlDM*%JSJy6J3R5uZ{a0^&elx}(G7MCpo%6Ea-JRogSy*PfjQA#msSxoS#3u{?d}Ffn@AO_omgv~e!h=DrfQeRjK) z%1cTA;E3DlQDoDubFse&J-dytz#k{eV>(cC8^*v~^qXjoW;zLb4la2QH8la{ zaGNp-6xka_xbuK@wZ|1%vst^fUXPkWbiYz+-QZ6D^Y$M$|uuw-7s zkk0P6zD0*AWRoxU+kN|y{i4Ep_N!RJvK8d7)j{qZ$Um*%X@w^4(|R=#vS!E_)?3ZI zxNajSR!T}ueh4^Yebc6*7z!pEpU)mAwGTBuPFyw{Zk@N=Nn!nWF2Falj$MaWTqjR4 zE8l@)l|Bx@F+;-9^vxE58|youak-wyT{v<^tK=F6B(-4FbS;vwrr3yZwDk_Fn_Tb_ zKAX9TnlrNEaO)4yv=AUB7oQo`?ctFPXNran&C(g*8ot+XaPdi;m#LoGA0ESDsR&Qo zL06nG+DyZ&R$|uP?fyJxHlrw_`fU&uwL0AWDicTDY>Hud+x_W%kVM~H4}4IV*(B{a z&7wnR=iKu$zD&$wxb>#YbNNf%RV#||ChAew(_%i@9tlh7ypV2x-@LxfK5og7;FtjDc~B^paJdh$BJ2*4Q1h*UC=eK$BE_O7J9mY|fgzjFSu3qDgk;j@4k(o0(tv4+ZKu90yQ4!mFN zO5V9KA|&Zh6;)AD)g>0adix$0Jf=0m`kG5h{0)eA)q!TkBo3AM%*}`}eUC%-RcpWj zm%f^EgOJA@3-Tn>b2)YYQCFIsb3x_)vK?8?c29TM?Wfs)>TDFO%PgP2ip_)^}f%xU?9iD39Dn)Z=`szpv1|J%j+x^CX| zuJyC&wmbM|3pzm4TVi_cqw*a*_-#EaaFK`Uw4pl5pZoA@EgBLNVI-uM(24|w z>2_o`5kG0O@O-x0^yYFbT@n3tfRx(7psjZ!RXM{I+g)zL&+#PO=nco2%yQ6>;6lQt zU18Jyfbr-B-*NOT*;e^E8|)xgBIV7i0i~7?-Ue?(b1dtZYgraF>Z+(oQJ+1DSzd-A zPx5t;pflXfl|letx5|pln5;D}GxG zrP^vJxz^C?ipeL>Mr=Cv4ec>XStZTZdNJ>y1Cd)(Uh#Mwq3);{7%0P$b9NXzPu6pD zhlHe^)vn>Q>W0nO{@4bc&ZxNS6Ph8j+>o|1hBpNo@B76Lq11ZE{){gMy?|3wdEB`> zv;JDyJSVK-_X_!q6mR0x%H$oSG-Yw^i61!BnN;CUmMT7gx~OpKn{}Q z#(}kFj_P`m{7T^wlC*2s<&bBtX) z)TNrKZ6jgJ$Jr}Pk&PSq9jnJl6fh#~R68ZCV5W*pU}!bXSG$Kz0ym0s+1k@?`g2&? zIXP3dq%d_)5>9dM3x6}&!0Hb^HP_3BWpw2;wX6$sxV{Q01$~JlHooUxTj|3yrql9n zA{Eu2>gh6gzlXXl65Cc6&e@U{s(aQiR2?}J*z(|APaGzmWMUGqCR>w%>eTdM%pWkL ze+Egl+U5#i(ux|o4|a&LQh=vzRyl{_tR#?k=jXv5MuaKxOi0^GEJCPR{X2k7PV$nP zxiK*Uh!ox!$DcsXHNnnx@JkoDQhu-{2&8EG1Kp|8m}OxTW;Mn-P0WEjJn3U>Yt zd9>CeP(wv_;JjV`72vu3WkuXB+7Yjj6erNCMguZ%FbhfiZf9ai$@|Cv^!NR?=MZ7) z8!ZdV`7}kEffiaORU%uQ^@-l+>y>gEoIU6Q`8r#r57borayX<}sJXc*N*!*@S4on_fm#!b;UQLZmk|9MmBP944v2DQ@uERS5VdAr>Qa*Q^WDi8HWPvqyyaGq zC@=L4e}T_XCB`XTxgiri{pxEAiWXc@5~dW!o&AX_8YNY;22lkpx12q$=y#$!tH;am z?WAm83tx!lTTx;k{HmNeYiXgDq-x!1nVs29Ay-(QR zAoA2^az>Dak7~rcxpnG6QIUHd!PJT&{`>z8Jkt6+Hu|dmk?Ghp&oXd39r((wgjW%V zM`%oRIm3SXIEn99sIet}|MBa30g{^H?MsP=A-TT0v8t+WfxeF9V>&74S3qmXC?n7g z^;`->mma8I6x@6s#h@&`R95#jCZ_1rb)V!awt4{n4j1K=*?eX!P{xX`muA*B4;^a} zG~^OIhzcVcGTIDZkqE0V{Ir6sHw8cq;CYSbGteqxY}33CtADo+8J!?}skdlNAyF#! zx6GDw8#*@B0}H$m$?GHxRE$|(dl}Dxl{Srg61g94vvM%>58$E!bQAkCHmM{D!zsGN_J=n>^=Eb-7U^DUwiJs?59; zt{z6e-ZxRMyIKD_F)@a3W2>j@*1%WaY-JXGlNnXluX95oLeW=nhZJJ+ZRX6JVx9MG zdwae9NXgYPi}W^LWS*GY!FI!s6$O?&T#RN^9YM`gC+!K!A94pk;!`)3v(pnAwxQ?{ z@!P@7HHjh-+bJvwKuVnXG;)7M-03{_AE`JG?ymZ>ntLbn#vjXWwvl%Oa`uc_&oNHm z_ViEF(8WV=WdEB1Sg%5|f`5L~B?JIbP1l3o+bYf}0|XtJ9r;MV@wxyr%bT0)))LWj z=}X@o6YuqGimmE?rIv@0#t}uKf2Le6jx3(sIc|T$vuhrT-D*Z(hpNXDX_7u=Z!5m@ zsF?r9^jDacew8pcYy4!~aZTYUgYXpMIGM5WR8cBAf8UT$c}k@q{`rnXxKm7{&@qJj zR`O+oe&MNrdQ5Ngm(L)?#_??m(|uj|8BxTBNAy}*mCI{By+d$$YHuF!^37lr-W#l} za+!SN4mt^IYJ#sHbjgj2mdtjRL}d6( z8C%1d1q81WtsDb}KHrbn-uf^rzNP$tR2=t_GZi&6wFJJ8;AOXandt^ru6Wj4uq1jL zi0^L>@=fLacKFigDVu5#!Dc6PTOcn6rtrMEJ=vkIwxORRykWL*blI7K!eUayksQNW zZ2ON&YRyS3hQN=I;7&&Ch4pX-*^hl|Kg;Y`Bb45T@O`CYLg#1VtM+^2_(iXK1dQ5tJBzxCRfY2QRwlWTo!f1Vu2a||y@W$svtFb1WI*4| zrMG1lEDnkF6|C6i7^@IJ@VV>5sCY+xPY}*|PkC}IicK#8okbX2xLJS#I#~1%tEvFK zWCFg~&BJpqi3{p-EwlNqUniGdO^fRVnb|@ab~K4w$LC(6xjWXpdW67F@N7=`i!FzY z!SM`zZ;3wn)U96=Q@^!O(~M8kvPG3Co!U+if}W9Ra;mb^_qoUq6iOswL%$O3kY#PO zo9AV@DTQE@D9Gg*vSCESIYxa=%WNfY-hB&7US41Opr$`H+08s=li0O;Qef~*5k9eX z^qXM}DjD2keb_>fWbRzw*b2FN*c`;&H!@_NHjliDpxMSp6>Xp;M{VGR=lF&(|)nPj*!i3x|+7Fu+Mc20iRdFlV*F95o zi~=G{^_s5!bR}Ma>6654w!LoyHZy#^wk@R-5a0a1C)i*S#@5lRGoiUdtvYZ( zG2~3P$$1uRny%=au&1nwt{9r3>&AIG>pD{2?%PPjS3>SP0Q%r3*2d55EuE;gMfN`q zCedm+kSYEZsJZY*vD~jW(W2;M z)iRFMYDsFL(Cm3WzNee{LBf?*k1qc)D|B4L1#TDexNg+tH zdC@W|YlbZzg-A-$va6gf;@HMWq(Rrr@+8AS=Q5^AO#I#?%2N2IFM=vs*wu~!)FU-o zZX;iLF4(e80SfZ9W?Rwp4SG%@4p!RRLd6R@-LHB&N2D{X(Ic}s;UlD*?wy*3@p65wsOC2%SzoACR}pGS+Y2H!}rfPaaYw`pb__7}7#xZ=jKK3&Mcnf9&BbP+;btE6l> zQewgtzc2eK@vosWyiuFB_(GlkU%(-cQ?IL@>(**MEMgtL~-WLp1#Yb=WbE z_K^f;omIg=;qw?zm-FFob0(!KearQKUWDuRZ1V(%_Y^LCOEbj4@Q`q<1Ah-vD_pir zI|VD%*_z49>-JHK2o`BFC&CEM5ijfd-iuRYz+Fuk=2>Cq9UWe2SJ5he95sIS-`{=D zAL+I_K&J9F1)|sfx`c}jyL~$moQ!6Cgm0wvnf|_TbIiy z_e_R_$Nb&yvC)9kcsC6$Ara-j7L5m&b(R6!g@XLA?HY;QopnQRpeoEO^q(sXEH)vo zMzs1mtXiqOx;K&xTPEO@;n6RY&-_hQD|oP$p#+ZR=Y})p4*7kg!N(qxE8b~Lz8c%H zC|)6dtx@WpvnZDSp?#!eW!K}oq<$|0SBIoPv@|N63bS$5&NobMjy{}V>dvPS$-cq< zcOXlpS+JD$57XX zk>TCHvwt=({UqPf{{Z@o)G3?Zdosq&Cf=_^)nKTZiiN@YCp{<%kK8b<43P*<#fmQ0 zo@UF7MZ9{o|KMH|>w&-+FqjwOS4fevrGp~dKb{Vm=FgsXiBCV;7pKyla4~C?qY&>j zB&j9|`yXB1WKQPYbd&grqNk1*q+ty$W6n%Zu}>BL;rP-g#puUwK-?Dem*R_rFo6tb zKXi*(9nO5Eh*B)!a8|a9D@R#;?-)%KBh|NRRc|qsSIxm6-=#qDcQVXxaWS5U5MXjQ zez;cA*ZHc=P^0q@Z86Vem(Q@k0ntYP(rIMmWqblP=Dy+2zjwKzyxf_t^P_w4b8%(9 z#qVBkc`JOk$!1&PZv&tC=RrDulhOXI8~i`NN4{ZBJ0g~{Ic>rcXe|K<1(k5_eSb=XFx$zluSxbFxUdK;hIDR{?t?na^ znXgFmisy>Nc|N6r)IYI)5hc2E!I5Ud7ppQYug_(ZCXP2JTiw)ZhC@0C@1sA<#>txQ zcCK9_RS+eQbC?s0@({^CJE!3#m9LSzQJ zfG0m&W1il{syj@FAwi@MUWgomBrh=aYQT=%4IuC2@Pc6vZLz(JzbUCZbk!!HI!<`6pL+0mN)i$n;wE<9U{Ay&i9@^FcGM8b-#1pPv z{kEJZr7hxV>$24Eo*d2imOSG&L9WQ<$cYGExBdDUxlzl{1xzXp3s!BuOlBq$P#24= zmYL`-ziL?7I4^2b8zXAeoIRY`e>F0HT7_bdz_LXx1iLQ!xzC7T0}r*NsdrL`wD=zd znzyTnf9z+4DB`NC<+#v4Iz0N6_&!PypQRLma919pMb##sqN8yD!>@I{l8I$6N?HKJC0xu?oZ)tc-~HY&TGdS^v|Ut5{{zt{eX%lC#?GFhs&BfET1&iTzDf= z{k!``cYqa;m)0hk_c-N2D5p=0uD+BhEp^lQ#3wjD`Y5b?QWV@vo%3HEWJ;C5P&w*n z3VR3$UC8Ysj7jNi>^5B8Q#9yJbiM+f8!UglleA<=@9{-Ah`MyNcJnc7Gt4qdY*&iT zn$M~x5grt)kSqzVRx8p0aW9%e8~HZUo|UbFt4sGcLp5U)mRSp4owv4g?mWL}bK=M* z03ie7oNUfC4r&qqQwKO?XQlJSb@9Fo*KF;`bt>~Pq7@pfUqV);#Z^Qly>#cwwAqo@ z|FLYHmTXbtdE62II>DOej?mj~<6^%&RgTt?wLua)#NT8CS`FhSg2VlKcKofqYFu92 zE2JnVwLW>>c7-46@qa)^lenYv#tza}T*aw#Q1lw$oJI(O$&P%V$@ej$dXSd9^##*X zh_Cb!vauFCj70=yh4Se?6A$T}w?F!r9Lq`2M~H6S_$nz`RN|LuCEI#Vn?iK*;a`*K z$O?Y$Vxm`KhLd=3lspo(syUXY*ZIuYh|zs!82O!9-Awr)?a{%ZC-_}wLWqRqjtj%2 z7|X;oG{T(*^a7iFlRmkR8`yZ&sZ_m;NLovB->Af{9j+H-Pp%cB^G52%b< z7jXge+W@Jikz|p`9doP< z$4~HBpv(mT~Rb2>!XMG7Qi?(*AI#8=Eid z1uvzphB;$^;?kFo_rRxD&pMy+uU`D;+WJ>m9iYYE(_k2t7Ewgx;TpWwwz^IcsT(J+ z@O3rG8|E&Y(6RWB!v1s*Hd&uWC`rMToYME|O#jx?I{eD_v&?_iJyj#el9(RSM#G&f zm2={!tKDYqoKga>%w!>wbxy&=v-&_EdH5y>u+9hWrD0l^wk7*(VC^NH+PGz3(@%EIEXZW99tFakd2>`146h|2TgNG1&q+3`nv#CHgH7d2ku#YT= z9^dFDYti_cOWzUmTPZR*x~4lqRW&gKYIxc_Lg58aLuO{49n>>jWtGhGju!5iSjsI9 zaNYuEpgcRJB5<&vi?hMmICH|>uY%9Z5?9X0W!bOjX=G!w;p?)xTdmnGT~P)(cZkwO z@b_gR=G4F-67CfUeWe;KfQEZ9>ejDZsA!j3E?VrxOjPF+))AJwAGMWmR59XEo%}(# z;t$2BTd%kG#RdCw;wTO{n1}#(lL7WRZ2S*m%3#1rp;QgVe`-0*bnheGL8kY26T4*j z=gos{fTs@|GT^2o|0+{v%TB2@9`W=k@s;Cos=T@;$p3mebtY%9SSvx0uGezroi(@Q zi`Mx$`9MbwW*A|5B&K8{0Xz7QH~rt%2%UeaTgWbZ=K!Ri zZN-4Chke8>|HQW}QB&TNS=r$H&@rLgF0|!ulzAl>_g+7vI$p9lIBkCS*4+9Gj;CXm zhF7b6N++GtyN~Uy@{yk(xsV64Tcyf;qAR0Z4GeBR`=J^ae=E99i`YzHjSYcsMV1?k z^Daorjt^ZeY|uR@liD=M3@LAvw+S+BUve<`aqV)V&8;5J>`TPQs5q6z4#dJ@-C2@y z%Xqk81A(c27P0QR8eEh@-bg9sk9jWKarsN7HY+8k{rwR&eFdwg=-(FRJX|T5Tl64D ze|O4G(bE1GHe=RbI8;!b4|vOOdiOAzMw6%6#JN8c;MV3q-r}3H7}}Yrjw%!n=9GYT zEIqBvjN_rlpL;)^w-C+MS87xJg-tuT&|vsVdj5vD64f8U3N>egii-UM@}&Q7mfIUqwRX9@E8W1{+}NHV$y$x zpUwvc_F=mIp8f>lr3JfAwLEPndEltp6`z-g$hxcTLV;AN(TIt|&N;Qd=4JqacCxnK zu^z3K=f#SzY9pz?zG5 zkC3T7n&ZnOfghf3HsGD?uQ|S?#KEO}C4Xi15)nGP2elVxfKvV|WZ5mct|Oq8y^V|W zjf^X&ug+*jsdMQV*dq@LaOp*9Ag;9uPh%Z>fW1^@Ny5z{PgF&H%AGg%wSA6teO@UM zKI3xQQd!wz8OnA1%VZY<>d6hHFA(Em5ViqWhSAwhJ!z4jW#zUdBM1x?XB1r0dLL zFFZ9L#;;ecoT*Bz@9KR#xLB!=AKrJ+n|*QAOH-P<;wm|H9JtfpF*7qilz4QSl3!XK zkS}sPGcv@=riX$%=kiwUt#3B!Q|9YDCnQ%)4HXPaPb4Le0G#)_19E z;TGo!D*i|WLg)fJ#^!x?ZudqBOyC1QuASm@Mj$fwI@h9RHv_BvF=939lkxDA651}H z47`VBP1wmEGb(6`q z=6sdc=cqjz>a;iy2#g@gpAnxOrPwBV%yz0s;!^KFIa;GgPaN47O$2rgsYrVIdr5kq zV0}bRI`z*ta=SvRZBNf@J^dRm$IH$h$>N~R>HDkrzi!rM!{=S`UbLgF(g~VZ&tnoD z6$$B@aKrN2c2^y8UCeab>Yh$i)>!4Av>es1 z33}VA-4oYI6nyRXRF^X~&P*;(!6SJ6FGSfGAn^|0s8-bW_hdUCTG^O$sD&Tzh>aU3sSoF$))`&7>mAQ7W$Ptvlz_R8`Xe>*DLPIv-ic1%&^WOtvh3BDM%6Q`}8 zn}_;2$sNxgRzg$5x{?FtJ0?e<0Ja@>e8l35(D=)Y8A_Z5R03c zW%?fbJ7ry7hM*_jTP=P1)jnG(+GW~<(Y%u$2W+_udfr{qbLKwhA`4yBuCuc36<#}i zY&DRrO)O4OqW3=8@ysVY;cDlV!Ss#9lLnGvni@~ zUGzcRD-L*fY-`K;NRaub=bhRMtceuooI*SGwhDWb?iX>|#Lj-9iC&4qv55E+zxeKI z{+%wSY{mOReJiKCcsEIEA)kj^ncmzr-gZk&EW((o!p$Lkd_CG^xwIlqlTf5T$ z7NqBP)Y6yagO;y=ufN?;-@vV{E#Td*_BQdGtavGC7f#t-Ww+cmcJb`|%kH(a;9)kE zAmB>no_@JxolISv7v+k1t+=+?dDOTZ9kIl*`S}6)KidP*33%gw6ek@QDmKOrs4upt zV`OlCMt;J0So5htGSo@OEf0+f?9w(g`q=+mE-KDoI2v_iDyhJY&uT zC~>b#M!s)uOwkHmulsNlSkC%GyV5+YpPc9O>eaZ}8%`T$bCztZsWxrxp5;j_a?)y6o?zqZPlyyZpn_}iHdB>4A7E7Q+voUZGkY#vZzly z5Rrrafu$B0qX(fzl?&eTYIR)3JU`W+azv+LAC`5*#?s)Xy=CG|hl^6BDD!HKFWt{9 z0Ad=@NN}pBJ;vM~>vTk0@H2YNlV6AJDriD6`1d2g>@&d~PD&43*R*tt%KM1k7#RTW z7Hlx!2snMZ?EX)eG8{(>%U_$jcRRnDTJ3%O44e9DSdgU&AlgEpl5VEQAY0?!+xE-@ z^lKH@f9YrXK0urdG0L;!Q&Ui^es?bzp8m*@C2Zeu2^D1EBH@(fzqog8df)4tu!7a0k1C!?Z{Hybj{`WYY%%_X zzml}j&;9zpY2yPZ`8_`E5>sAMquXVHQ62~GJ0!fXm z*Yel418g+L85HOPt>WS#Q>@H68{L#4B{u!cCWTB<7(Nh)z=fP`Mn0iT>p|2Fuh5aw zM4PvlQFkj@H}`kb-i&Au?!%Daa^J27K>^-pVD`tM&+GjfUQ~3?j3KUzWVeWWe7eU< z4=7xgKFnJU$X(J2Vp!K*Wh}p?R`yam;j8OKB>wF|4r}H0sREmr@g-2q)AD$mhME2+ z1{`t3$bk0(S7h+?*1^u1H^`C0lETlz5iEVLVsyZ|CT*-VcW7G$Z&NHo4!?y#oxCEG zS7zj+eQnzMo6r=^mZ-}pCRo{&QKiTgM!EU2*igC*S+2>HU(8cw@g;!!Ds;)$N@kXY zD!;KO$5@M(*^KpXAMv-+;CcL4C!ZWO1;z->xv~;3aA1c=gr$*~%osK{IT9q&;nkx` z*RjxV8CfV6R@S#DhLf9A+16S-XxEpf|AZ?8>sq4$uLL?%jMk-?l{?rJ)cFYJsX`-j z%@}%S&9Mss#AeC*!M@MHV49cs>4x_96&@1;84U#uPyZ#$VKN`w-Wj^W%{;lkl=yZwdgwl3gN%jkU`Ul#%;lU9a60L9? z@}uOWv?n^^tQw%}&0D>Hc6D0UZY<__Pt^c8do$;;SPi2GiTYkju~L$4#y&!JB8Goi z0Nq|89IJI^y*j%38({MncAh+m)zyzoutN3B0u6P$&KXw}?=5_yD3bIVd&xhrSX29j zYI${HOr)Z;w-U86HC$2V!<8~}h;tR)pcg-INE~~H_aEud5S^2F-&g(LC=72%=(lrh zZ{wyYlil+&opM4;mWQEOc`#T{7PphE%@}~odBMyHWRv*W)wJmOt4IB95hcZ}-|(*# z2l(NMLVR{LkVaOLs0ZlQ8%Y!)3c2%dB&U@P;BYwlO-rh=6)QRB>}sRm1N-TMmk)|8 z!sv+~v>k3yxtW)%xr!yd)}$YioS2LifOK$uI{#>OqB;LB>2y_0#CA^uidnC-h?$=F?Xeb-$--t7q< zAOV3am-Cwn#rB>;&Uw;N*YwAl@7gxWLnJI^ncm0nsn%X0f|zpE1akgSK5L0t?a_;* zgA=djUim`eJN$jWcRTE-wu~zTMwSoF)y7;116IE7TIR3x6?e3){$6Izo$Ec8UY4C` zs&1xVUwhNOd*Uf}q~KW8Z&NY(3E**7k~(c`0#2fj_1wK%7FLo55XE!hdj%YNWZQcX z^}zOE!?M#j)_ALto^u5uKhv@}u&G)F{el2Lmudl0N_PMFxM%`yS;e!w^XRKJSX(<` z!UY&NKU^VVu&fK7gL@#0CEA+EA{6p`QUMfAe!NT*8ZW253QI|;IS*2R#=k5mp+-66 zoLlg)I56d{GfBJfF?Wv6KQ-diPo?56rke7dZi!*EJ?X21VP?%&AlTl^ZWjE7mg>e$ z(zBLioxkt<_ID1cl|?2OxqRF&RQ=w(PSAF`qM0*sl{+UdLj-++=g#23Q(8R~A}wy) z_|LIf`_5lhFb2&rOqm)SyVITbnR2dncO(~YBO%CaK5(r??t7SQc`@2pVru-{r`BZ+ z>rgK?^KKm*{Py5tgMyn1RcpV3-F}AV6N;T=B`aAmp+Y?;bRZ!#GH^>aCQveRIdsxP z=vJSP?2VVd;rB_~DaOs0hjy#~C>Y<~tCy>bp(39ko$N^M`OYUCQ<+nNERPCG>=8uP zZ#%Hk40Pe;3lEES`krF5%M~-rkXa>jb`@l?wH)3TpoiHSj!mp2_to)PZQM(aO}!an zzqbs`ulO?2Xdx7=z23EUD8_fGFqe5&zo7mTm@c^kX|)_4VOHyerDcSRB;T##0O?MD zXKEjccBHjl>R8g$4Y0`s?vwPv3#OyAh&Gl5);*=8(>kODlb{O29P6#zQ|JP%gKvL% z7EqeJC9767wsQqo$vas}PQ3zN^%q(@yd2tzs6&iZwu`&;f&Is-x7P8$g!jAIRkkLT zT9ipYzI94_>7Z>kV!_ohHUV1?%o+-ZvKt`;LfK1~?7{x>U3rE5R=xajUOobc{vx8dmRvnXoj4^E=`dqwx<&Ir zW(8w)4ilAhB1D}|tkfgHq!yMFj<=ARwz`TDO#9JJs{E>+_#S+fScGVS#^xx-w; z?*+4}t8bTcwK_W%?Udx^Oh1b3%j*WEt5wP#jm_sYA&Y%S(tI2skjiKI9EqfaE0|I2 zJ7exg4*p_=+S_-U&9Ew2{-9RN-y=F%BhVj2RyIC#3}GY1QeVUZv>ncAm5}!V(pN-4?cIL~Ivp=VqqR7?GRr%;KSGastymKL z#Ea+0vYwU%wnjF5CCZ*SN?c#SbN^dwTdH^%3|^MjO`-!sa=a2uf);=A8?$O!tH*Vw zW}fdbL7`nz>k=Ok!|siCw*|R|xR6!&oN0btgewxRGI2=+;hAJ}xcLf`;Ek4nY$%i* zG!U7-Vx#U^ldpXzWs=Tl{(lJH^6_wwG6Ug5Us}tgSN-=SdJHk^J6{-c6AQk* zqRTM)ev0})tM!!fp*YyEy*x zQBzFPu{9mTqSQO*M86RjJ8h0Grr2%II-Qq&x~ijp z#x=&}=jW{QSJOo%MfplTuZa_hWV5MoX zZee-*x=>bo0W*Q=AW8|nm*$!>z!B+ z4&K#xkWRlH3tQImo>}bqQQFDt6`oau3P>B+UgC7T<6P)Vy8A{bvWL7vt(DHKOk2N> z=DzV;t4EH}eRDh|B=HJ`#YFP@XAv;tUQKykXBL5QJ$paNQn>ODT?I&Vv4ohmjKY_( zbB#AoTWKT0^)-6AawEtRGF&)u9)x_>fP>LLP^Kvy0``zGr~4~NMP+)nLvFXn1tp`h)5rgLDBgPf$K|w80x@Mz1(_^QKA4{ewQDpoMp6!Y|cl$dO73l<$tTU~{ z7MJpDo=Vm$Sf!Sb2O>JQ#~Y!gX>q10%Dq~+t(~R)KI#Xp3N)or37xzzL2z(Dig1En zuiRL_Oy5!k?KgKs4CMnQZXsREn6xOxZXVrmghzBj(L2i=vu8r_I=$rXVoaS1bSrtk z>^q_^XYMQ3(I~=7j{MQ0o82d#J=U`OIX3F~So}hEA|dD3~2sx;pGGsbkLvf}W*Vfz?rLhMsd{_f9%aMCt@fNL8lLgOH3wcYc1s zp|ktzZ^uiVE(!>GEM%p1M)uXs0qJ@oNt`>$3s!k<>kb-E4S%7IUKcnBlLx0+C2BzF zWc!ea=bS*MiSQF47wdP)jH!8glU3%4{k)FPzjY_Ha!K9Q@6x@y-LY8*w0fKK1s}XN z72k!SM6;c3PWCt|f3Iy$?lYUh5ALpyktCvDM1<@NpK6BJQ$$mt(VCIOMbX>Uu^lH2y&*9QT76hIe1owTA=cFfc%TEc%ux*w~t z)eCsfN63u)A!rVHB==U=NL@g3`0d5SIK9A&wh`ad3TYrmF;cemlIgzSCziKLY#iwF z6iSoRlP|Yy{b1CbT;wv9%0#<9F*ULto{jz@nG}K>Pl`y(ww{w&XC71oZpw{A%-B_m z;2$b?>><+7_-GdAsOh1=&u9gz&I7O0?*aO2zBqQ9)2|6Hy}tb>*Qrw6Z}&IvqJp*H zu!`sDqStlDHxETwKT2jqQIZY{hNGAaPOF|@wl3K8A{6%qmE<)g?t!he)jmC30hj)lo|IX5F4?Bm(GrNk}V7Us0kc72Jsw&2! z2Ywtp$#=`QuFma!h6XYZ$q=?OH-aX3%OC!bs9GJ;wSKMkcZBR=xZub`NK<(H^^s4| z2=>>HH_yC{#OaRB*Z3}hvSkt@+jRE0~Ev~0{U$*)rfZmoVKdWh6Yg%!sz_`g*RM<9SOnuv)>B9OC( z0Fm+fz)RxWYLjwJxnXg6-#lKnSK%vTb<+hd#TKKt%MyVCx-{H_YafB&+;G60y{B(> zW`b(h)4gYX>AUHKIY11CW_alt$S%lzxfw2%PEy3hJ@hcW$;8q)EznR3HnbdzNyvLo z{-APg0PT-iGG_TgiU(G?LMB#&g>+p%z{Nn(s;s;(M@kG!SS0IVp|LU;6N$WE{d0(D z(0?P^tKRfivVW%VBPFM#xF-W`3@&W?yP%-pwMbupcrXFTmXM^)j|pG#2mVc}%Bu3x z6&{ehejDLKbpU?~^7S22}Q5TG%EMcv*K!SeoZ|u8VUT_fL9-L{1_cEhSKI*iBdT` zuRvp8)U%u?Ifv(_Af=q$Vk>mIv_FD3N0B?f;T4~&?Lpj4#wgAg zgM1Z)_pG!mSeDVE2?bAGU(Dop4#buwhp7#=P;cmoX*}XiXChJ=DO9)#>};|p2qz4a z)$041>4?>4x(1PIr*(?q6SoJ?WZ%5?vB|S62P4cYDqR=Lv>}qCF$$Feu=9xaP6+I8@$UYk7 zVTan?%Q3Ns z5#@$rn`0+=k~Lim4XW6)CM~Yi#!|LO<*2FawagpB z0#()_IR#^0MiZk3ah^cg1Qp%3A#GL+?c;k6uV6zvK>E~Jsa{(&i+veD!GBO{e<4rN zAba&cJr+P+DORc^X+ud*kZmt(9r}S+}~&GlvpeDmfqy<$Zi@)HMB6ulakv#q|E}5XwuhYWIu~A zXs7Ua)>=v9&b@XAC2j#Tz={x&t}0v@={d{kwnbypv>Ty(k3&~o;C!!fTfbs3d;}{_ zmkacb)dVls_3_t4-e935y}3`BF%+pc+=g+f1QB51+q%ia^|3*3AesE~kqG28U%lA; z(E#KFCHn%I_#LRUoOe#tmxC9R~^5jAv0XHZ4?i zk)51Alxr)B(Cn{RJBy5#B+OUGZC1r2i&bc}a?}*y;2WW@oS3v@BubVkMTTBqB30qd z&>S;J&IJbX5~NuIflCNxkEK%84);36%_(pW+N?V{#m1GGR+@-=VfGX}oZ^iCqNcdI zmQ%1p4{OPoKn(`jLe$-v4b~3Sz`r(UHw2A6DyR}N+%hzEG1H!-Sm;bC|xcAyu&3VX8uVWr-|T9P7*36mrJfl3B?beGoEae zyaEz$;Ehanr5OD}S&>AvE-e-i^9g<^81F!wcz4dXT1CgVF7@4f08Q4Hm(Ri%=D4~{ zx0d}0v?Gq{jrYx+{|6}bpRBnX|G%ES*Uo)ZB5Ui3c6uT&uU%SP0h%ZJ?&JRe&wuXP diff --git a/images/docs/ui-dashboard-rcs.png b/images/docs/ui-dashboard-rcs.png deleted file mode 100755 index 5825cfbb470124b8d661f0fe45dda31644bd5004..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 68587 zcmagFbyOAKA2vF43J6GdhlF&4lz_B!w{&+mBGTQ`4U&hJ5D<{=I2^jWCGSzczxTas z-9PSGu$VKtfUM+OHP6LEu#YRzPPXV1CibVd z_!72KJ?@p0vk4^&acsoHLH*TI8dY+K4DWG2M&aJPHfTfPFSnT;lCG>XB=!OsOgkj8 zDXyDNx^+^ViplV_1u6^;jDv;)VNb}+tN> zg#dpf?5|&(_&Hu z1i>%=9`s;+`FB&e(cc?F+2N`G9=Oi2?>Nl%g&hth$nS-~16rba8DlS8T@f6k{$7w* zL>L@}wlFqS$BSoi`9eB#42&2!Kt7rzBBQrYbk|ZuUhr?K);0o{`}xg@GthHr+rF<^eQ?)@l z2EgmxlPJA*AiTyavM)L#66k~Q@7hYN_BD~&Zl>e#d#w+)kA9{OtbowX;)txO+iTQ- zYpHKfq0jYf!OaeTwKOi2zS7I#%xA@R((-YH$-*@1Inw*Y^3VRPk7Rr8SAvQJ)5%YQlG2@8MgCg_A=-Udz(5Oqbe?@J-x~{gx`qDq(JX~z@ZL?2@j&vL*R{nEFG4jwz_(=fK|vfk%&k66G@ zBK>T186E3Au+0AsLN@1n>~1bQ>k?Zv|1HJsoI*6B_nQF2(UM`bSI_dv6T2kaG0kyq zq&w+F89hgoH(b9Vg8k~O*dIM-#8@@Ji#CW96fh66I6a0;;K$r7@1pED5&);pQKC74 zy#4l(Q8%fUA|Cf!c%lGQQmC7;k(|ihec6hs&a(n5f=Fu9wk9h{Rj--o?9u)NR-PAH zH}SElerv+RV84`&c7>_FXXDtZKg}@V1&jbWTo=@tEpyi<0%{D>JKso69<}2wJCdV$ zVEzr)49pL9QmW3xBI9& z86!Rpc6NVKlTnkaXVc#!6zzSl2@BQyy14s0O(T0vK2y7GoBNi&%JIQ~mLfMvcu2?1 zBYj$c_jjLxmie?-B-g{w4d*If{U0z5AKe{qxp^YAuQiu<$V_SmL&Y3UgBx#h1Grt8 zotpR$%Re6uA#pluW9_~$uspGOU~r_CWOM4fZ`8KY>xk(1QIBc9SqB?nxU}KPa5Igm zmD_Q#Ws{ZFp=#2L8fa9y;ahXvyH;koU^wi%7WvwWk`w9c*!%2<(wjy|Si~;&t?sM^OBD2JXtwdpi7?3$E2?w6UT52KJcd-zPa!Bo5!(&TZO*{3}2<;ss1>z z;QFAuKoYJd19Xjv?svGF%iN=%oJDX(p>ZmbZ4QXKoTmFw%#!);Z@?TabwagSC@#)K zQ^)`1Oqgo7p~FeoCPv~^ZvFIP*+gTrM^HF|G`>s;wC%%uK>47J?*=m&7FPK#SBqB5 zslJ?6<_2R>YtYI!{zk^)(i{IO%0TObjL=<2a!<0b`ZEy$%24ntx8R?-dO@Qrt5@?y zp{up(H8p{vZYucgds1PQCVNBuEB0G8J(sqcC@uZu=G8&pHWIrJ;^-dmREmCKu{#_; z>hDfRJXWj!a%PMtCK0eAb-seP4=YwpxE#3IoQ34#og(@_-Xm8Id8)-}YbH`!KGdDL zJh1*v0TQQtQa82LH=&8wkfI3j1M)AEEZxxH1Ws>z)6ZD3;>JmX6}te(y&>giBd*L| zXwYy!?GZ!$dd{au%4La~D&q0R=fulZ>ayPOdO8aeQ^95tW`A|8Q(y)Th^yrp)@m~e zag}Phn!OhW;ta0QZ{mOLhJiXOfAa~!10)(FPs`=e%t^C990{$h+kbmkj<|6Hz&HNx zRATRp8W_N=W~cKx@z;w?PK4RcQ7iyD-I!T53P#8E5X%NP5csA2SMHoMdD+k3O}!t5 z=RR&>^{G;Mu-~q{^Gl5&_NHf)?O*)0bF2jC@VzcC)}9A1X|wYg_n>TG-hlx2@z(B- zV~p3mnGxxR=1)Dj35NHv<|%ys->&Tp+c$oo4fq)Fi7c-Lk@2!Tz41=SZyZdt-@)&W zcJXu|VV3y)AS|fSeRW-Por8{{;Ie@wvb^R(<-z8prSMR%93vRK_3!1 zj}+dD1iAIY<>2i6&LhQ0&w`Q|4+C_cDpz`h9+(=A=sxxCNJ!kO|CPg;qbOyyFm^DF zf;APH-?c&b>MU^P?L>gklcU9i&MY*0&nhAAy&bpYPah2N97qnld}}f?ba{{{Y;4b@ z{Ty`eGjw7a2=X`)4APy-6=82hS-!f*`oyV$HpwXxW42j82LS99{_N*iSU+Xq0@Z)- z5wfdvk_a#DDq{sAT%P_6c6b95*uhbMOsM#Pv%m~Pq5qBQ1%<3(%;BKiWPsi3Ccbvt z@kWkkq~CR7;8kK**qFp@ti&z%f8lA?!n0t?(XZ9P@wf$M=EbYS@p+3|ZEo&Da7lj2 z49!zaRvHka>AS(zeUx03cWm~il=t5EU6q=NXIJ;omh5->P!G(}$wwb@EeZk@s25og z0=9|*#DK=jt%`oX7|sjWoy|fRF-6J}-(?>{6kWcFO(Pmc*~JsYhC{C1e6emxOgHev;LihuTxupMm1PANwl|-=eE+X zK7%kbu^(}l_zOBrmk~T*-vnU_@L||f@A=Z|DhVxLfa98ByASDy=)<5vXo6!W5V$%q z%)ax`YD{@Na(F3Nm<|cBR-gFHR-tGTi~OgefX`NmqfhU_kl*XJ?&F5HGbIZW9sCj< zVNVb^G5i}{KSJg^P7PPEOMqz9eeO-NR~=elJdb9OryQl{gDHnbjl3wd@TXb&>~91? zs}`{^<()JrhS~;u!vSJ+sWS2cm~R#-uw=36^{a$7c!eS7V*v^81HSHm1oL7Gi2edo zb5hakdRJ)@1Q>dr9GJvh6P8bkO(%R5(J1s)FjI6Qm;30QQ#g&aJvSNcw8Zc_r*I$&IZUbj==s{;{tTz3ge)I=10l0kEH1}FcURcz>(7gv zrG63Y$28^s3C~B>VjtNyH$MtF8xr}xMD%XzK6U6&&Kl!?zzOdq_I%8SnKPqx@>{;o zaoviEW>#9kZ(c>kbV?&8|MfHMXP6CIoy;#zEjiC*2Y9~(h z!p0-dRbmp)wu+`u2nV1k9mOE;iJkZ zoP1z+UT@8vVrx&_m_IaK!*SlO2zDlRn3g!VyBrwlGQ1NL@)l+GptwE2bP!HuA6*=8 z4C)X)vbv)o_(|XSJ^rFJaP6+}`*w-&2)LeUKRWh%xxnl_nx>}$<=JEOkN#KjXtOmr z-n2YSjtrM8f{&yfam*BC+Jej9gk@zB;hrDrGb5JZ=3F?)UVZd`ev9Y(miuvdBqO%> z_VKTRe1AYJxnSeRJ(f!#_l_5$dJk<2^sCJ?aC1*<>(KUbZTqBINrDD9TDo|K>6fSF z$(5{VgN3=bi`{W!&Z$ey%3(>uScfIc!H$nhJ^vSI#&L=1&Oo(?HtlELm|*sHZ*%X9 zC@*Nk1}(`&TsFNr8qj|&%-I)vW4Q2idv4IT*k~4uRzEt+lcW6Isx~lSvrAG>SnfKI zJVR~PE>JVL{u~-GY{Xu&LX*|0qsO@yafeutDvHVYJ(K0<$Lz66O6}yQpCDv_z3KY-`lk1&8QSfmeKdrmO$zFk-fzJb>>4+LJODX+r`&;XG@NoaLZkf6J$- zkjntiPu@pCUaY_QC+$48BRrL{|M!Zstn<7~a!7iZf7MCsI`ORgQ;PpAs*!w7NC{Ce z%qn!(_~NFvyl{b3KZw|n_C(6Nf-NC2y~*kE^b>-OM$y#sX0=R&me02{&Bq-QwIPb< zkuL!n)eQor{V4`Q;eYc8o;mEX3-{yt*-XgX$!5MLg1yM~z>V&l-ND$ZVn#^GG(8_{ zMB=PoX6A@2^PIP;8-~BBgFVD(Gi*cn(zEOGL;O7=RcbZ5{t1dzZVP>;%!dwt&b@{Q zpUF&{3;WB^9KS}SHrT&_0!tG-Ntg_S;P}O2fEwOiwnq`VzMk`mA=gd4rZ|yHFo{rf ziV;A7Dh18P-r~O)z2c^vMFzVtR4MZw?A^iN{oN|qoD1lEe_yN?F-!cJ^GV3Ff{gAmv?fl1FDQ3NB?=y$xqdHwTZ$Zd|F%^dkg6mL$S#V~I0ST|^DTf~hjE1L8HyvU zgf~{E>fVXmQ+hkX2(dc{);7J>&Gd zWh)Xa^vP2_*z{4d9y|ld@K}-!AM8(|Ce-yS@D_yjbKB6JIhU>jallOMe*;r}>CQXR zd))1VSmBFXk=>n};DdHuj(G_g5f$ENF8CVS??E?@m1V0jUcu52l!G_~5H-E#aP1G> z(9^zz4&f4iT{!eNKGm8fU&GkbRM>}dzw2D-UP_rW%dG~&v6}@=9|EkI<97N?qhH0o z3-ASJ?5B5j(kk~Wt!y1 zPHT8l#AVEOsKjqO2ii8-#N0db{q0@y`tSn_CM==K4|NalcFjw+yD52B5ULDFDu8x^wu*}m*+GQA){I`A+!OQP>Y@LMhCHdvU}`Lsqt1t+S>o>omQuqg zS8ur<480&0?2`mvRHDEC(jL7Fy}Ax>{KaH7+uwtJKtIhj^h0(@E3MX;7-gzUTgBGTawI~lc*~x8&Pm7Tzu;6J;kMByN$PHdd1Af^sCY5b!4tcs z#=dXGz1!{2wNO?6^1`*(G| z8fm{;x_srrKbOGfPZ%AJgM{AyAN*HL{?`o9%j-j)!VDJ=zlsvQfEwa9u=_uxaQs&= zH0`WK*c<5F(z2X;h|_!lQ~p)kxGgoiun_zgAA%*=B;je|Fb2tYtq7+7A#-B{6P z5qRBvEem=6GP`I&l>cD5k=L;A_5wb8x8eC4M|rocBNQ7Nuao^LF<9c9Xg!e~wjWLU zFy{j}is-#0dY-1z_?KV}pWT~1M&+Vd{=u4-{k7Pw&a-R}!TqTG*um*;wD^ulRDb=# z3MHa9NQ?pt+3Xo#3?jcXd9P?RBUoTSXiuEu*LK3Nz`RBeIX0?au;mdH7A6JKAh-Rz z>to9Q@JGwflxg{;K|wIV6nOqt>CBpbM>dATzkbmmoAAN!?Cih++kdT2$pyypy}v%{ zlECc!n2PUhH7Gg^{R|D|OE_{7nX`UAjrr$UNisCp(b5um%0xFgQ1x;h3%eR32UAQ= zP6C0Ec3ppkwrnLpm)6nI@%FwuI(a}KAj0>watPRAT+5|{|c=>4wR!djzZ8T@fU5#Yl%-uk$(4!Z0jOu z`5{cBUCiD;_S{RLcxvot*MF{OK`SCh6xC-^s|6MeXc(89@3?sP8jqQpI%Lp{&K>!G z9%>O|k77})B~Mp>Qu|#`Il3DQvKEHDdn#TI6NKFglR4)}@- z_Chxx*RubR?L2-w2lUaOKM7v{KE5Jnz!xv8eiq}`ItPFGg5H`e;G~C4NMDS1?-t4omX>AqvUPJEuK%TDLciN_n;@85)- zc7Wa6+Z)dx79*n9%F1eC0qS7Hv7DjfRM@Ct5B>o(*s?kZrqK~tIz;&s3{om!Fc=OF z4(h3%7sEB*Z+*9S0-Bk_2yYj8pD*i832(N|EiAC%yDjR|<+pE%`0P|{XT>Pu5)(sq z;6lgqd(AlN>*`>G#JGt1h%hM|)ZP%F^JI=}oB@F&79}-$TX&Eb9$ROGl!_?_0le&* zTiEb#-g2Irpmq}y5_*PGhT>5NQxp>i*Y-;wv-Fc>WoMH{OLKB?@R48?h@CD}^e-Vm zniI{N^-rV647fQ^;o;woQ}S66pXQ@<$IH+Vps&}4Vd=C7cdz0K-kSW8s-2RLg@pw> zl098Mn1T-q70-1(A%Q1y*54h=!?c3@PcT~xtPJl3B6=y2sr*HksW7@PwP9jeLdVy+ ztK`sagrEl_-ROuRWUSG>YmR&*8pj>Eh)Y@bkCVgHw# zTw?v>oP}Vfp5M!UzY&d1Ivvf?xsP$G&S!6AI$6X_>Z}`pDq|fb?HD1R;K%}PCegQ{_1mnNw zF0DAb{x40%+5h^x@X$`j*rBoEZ(H+&bai|-`3~bWon#N2DQe-OCD;SrZ(5aq0wFx- z;)h4i|DND@P$88R71e6J7J#4PhdO$=c%BhE!CgI48}D)eHHpPabASEg$Hk-0*K_2P z4mX?*Hy;fjae&L3dv6A2!K`FI-O~{njToi#?D74^OZmDqd4Y4feu(FGj>*~oBINf_ z_Xcr@MBe!LxH8lg!o;D5c(`)7LxZ2NytJSZ7v=K*wduOS$|;;p`hIe6Z_jlMi3kkm zxqpWLf8?iW{?IbQgIxVVOimoiN(%T3R&40v;k(>N-0VtiO%=uN;8) z800}n5vEk}$d3*J@Ud1X#hk39UpK%^>kA$h|99Xim9&(c9L16q&xhzAtr9neOJ$AX#vcd%>*^xAL(sANxqd-OXrwn$B=0&@9Bnk{jno%_U|NJ~dbUfF|^UC2tIDabn zwjPmvFv{;rCKN{14B=)*Ih6dDxCnGdVi22n;u(QC zjVx2uGe5b28f)+2XMQBwZ@X&FD6$DOj4Gus+{wuVky&)-v4!3YZCt=? zT_~f~dy++UZ*Iz}qrR+dZaP4C)qXe3=yyK#3>N`BsUQ$~a^-tHi!aDa*MkRe29r@y z$Uy^>8(a*V-hx<#YOJ(EJW*0L-}qnPbtFYBA+mrxH|hu!H1PXUebX1Z1~&eB)CE{V);DYx&ky$PXX z9m%i>=W0VIDdX}vBzh^EAV&-|UvebSn4?5}J)}GODQEQs<<*1Zz}tGMnxuqYm$b$3 z^92Q+ty7&u>=rqPY!$f6#+$XNJVJTI6Jt~M3`3#F;3t(h`UGa0sK>dtL)D3iW)Crl zZ90+(*mfqVhOO=&()>(2cZkf}@y27H(;3bEl7ok_E51l_6s>R++obDDB@{02x@O>r z6cOu#l&KvO3f1y>iVyn_Cf^UZ@fE)w(t6yjYXNSlIGpAZI8b0#8(L3q-wmS~mdBxb z0`P`3sfoi)RStVMC{0#dsF2R6SfI_oG*9g7WYMA0 zn`NnJ`glf}oB{hLq8W~o6jq9|S;i660$Cg740TvwsCZ@(kkrY}QbKOfnKb3yA0ZRZ zS9_1}poK4;tIqO4+gD6(6BVYp(xP6`*d_h-VyJ&FPxN9xHLdE)fK4^gvv8uEGBGO6 z1fd~wPTdzw=~G7B@*p;Fz<1GJ(%+DiUS*VPbm^@a3oK%$}c7eBrJcL_Q zyUX(0YN%*$<;*zZ$IKhGU&yr#SkCn6D$byj^-CI6DU>sQ5eZPLx_5{hJ1RQo0O{Su1-<0vAKoY7RZK|B7Qq9^ zY66Yg)x6fxsP)+BIt56#a@iU33yyiSOF?RxBTRSH))hL%b-N8bEpNqvPt)FEQpS@u zP-`p~&1o@US-GN$0G~EdUm{m}*GI6aFeKN;9o1W_;gM5vYMFw(OVORvR|?-Sv7pH6 z6~4(#GV=|iTT&*gpV;4!b=|&2Q8!nWY1HUgpjKDhdCCJGZtyvmW4j@}sen!GwzwrW zS(IFLJ&d6-SNTsX#Y|;xW%nw1V-A~3xr*>CIltt>ZZR7;tg?~fH56Z-XrVzPK{8cS z9t^e_^Q2){#l*p~$vD7>mmsSvG4r)RYs~Dt8%NkgC1=@)Y<wJnQ&Dm9!vjJi6u{ z9dV!-B|ZS;;sSg}isUq0i1)rCz}JK=)6j8M(B(Zm+Vo<43Rb+lIN*8cN^8rj;|r)( z39LxVeNtnZ*!+lM*Zw;Zi%}9-b?UEhQ!Ua&?JAohLf7fQxkc@u+afuVPDDdaci)uk zODuoo`_-22qwNz>b`}y(Re$Lek+4{~n_RbKB*gj~Y#h;u3|y61SqjknI`TWaoEoY# z5i#nmTmjgM*h)2@As816VBO(rle`Reuq^he!$oKENOgdQqW6%Tm6#@oR?sCXk)OK1lOh0z(qM#{bEgxz4oNnwhU zLjcH19Gc6|Q~-EP>3z$_6-#zYRaecIKVQ)Vj^MV94OrE|#Gw+B1A%PGD8S)eJXu$g zehK%)`yTi&DTG7jZ+5wes#e;w22k|wSh@=wY^af81M_XfVyC2F;wJap!wI?H1ycZl z`Pc$^@kH5^`Rw~wfY5Il`0Ir){qcgMUf4ydMxc8DIaj-+{tHQ}B5I&>i?0}_wzy1# zK)RIq1fg+{|0}N48QEN4*XmHtHq7t~rT}STrW8zcfsR2FC3&PDSt|-0xPg?FY2*5w z)~?7fWd>is!TNNKliM&k%cZGf{$F^>7+(gCnBORiVdq31Fn5j^5w-MXv8gj6XQBZz z3x&}Wru$?ojt>YFfmmgmzL^W9aRRSUTjZ(YKiyJAa^V>(gvUqu1gn*r`F6@H?z?If z;9ODxZG&Yg^KA(@UbH%E-&E$sna)b1zYZeqCx`jp0*a8vP`K~ zpw%rQMfR-%AdnZcF;}LHCR%_VIjKB8rkORW+lUCITO?ie?K8$lPC~oEVIDoudv(;# zzFnAjIa$RfIT2Jq53j)KLAU;{H7!ZC>IJ-Nku){sZ=qo4ef7mXNO=>iCm|zQ+#C2- z(Ok1dc^(ZtWQALWMt&H;7q3IzG-*Vj52yWBN@OjwxzjoQGCZCaUMNg_vj9MTIt-tq zq!EacEu11&pz9cc2Popo8wHX>kydL7U!d^9zb|4UB){cAD4WtzaL0<=MX!wOV?#oY zpS4}Lag&bfvqkEZESr;Mkt>oCTQ41l5{C>7Jbevb+mG5|6jus#-~e6b!sruHmlbn%wI1~P~JLs;ntb;-^Te6cl-@~Ge5ptVvzqycSrsai$IaBR>at*mm#E?}%WTeCpicj?^tIy5 zEenPECovoD1;!qes~{N`(#DsbRLLt`9Ld99O9J10W`{2@4`)-C(Wk43(pf2xc)jBk z>;(&h*=3c!p_0#-sbQLjT&6)!%}^n2s^Epa8YKhUC340-BN8QT3YVi-lN>lEv`~8N zi-NLRJK_{8ojc6mx$()tBJ(IfIC(u9WdNm^{zo3=xKJ)04QjMv$<*tWfj(I-opc$f zasgYN#))KQ$jglY%#1_9LzhK;!1rZtnzdiD48_Z?(`GZyZ7$-=@HMG(c&c={j~s)@ zr|HpUCx+X@xKyXw3KofdNco}et2B*`U?W!;rE;vb?${J!af%OCt^!p$lsf&G@_MtF z!&r=FLSHL&LWJ|=xaL3N2OSACE+An?Yf#SZ(}-!$EGE4urYg!pN|H+CQ*9e7)+}GA zzMp)V7PsHP5vE8D8!KC-#6a;D5L=hX{>@PY56y{b*os1|^*UvZzo_tWe93_Mk+WkD zkv2t}bZumVd)#Hw5cW1acbRXK;+B(a%m83m%`1cQ?Lh5LO9S^4JdgxVP+lM$K$0EG zC-#c3J8Ub)H=c!-qAe9Vv%e1wm8|1UaDd6 zz7|!3a8y$JZ%G*2-WGA)i{-1BB1&6&nXn!-;+MH`L=BlJ%h;+$2A=X)HEHvpExB1I zVnPd4HxiJp$edu!VqFE!XvYtcF;!lCC0!tHmi&{JX3a&1g$FZ^=}t{ryOD?)0Xthm z4meHLNtU%|x4v{9KK*Lr%ZQ>;B3pDRE0eB6y#Z%fJ*90%RG~rlX4s61Y|B}v(*1MX zs0Ejpc)rH@E9p#WChqsLrwewyyor;3aV0l`Lml7lrAmL7IMrZw=iPr$ygUkkhjB-i zz8Ep(eBlwH%(~LQx<*pU`?E-{{%ujB4i^=yr)m-WG*5x}c2eZ+YOSsvE0fhLI{b86 z)dXHW((_Gf2_Z#Q!{{2mLY*qfa&BgGvLaPFAa=y|8()d$JDUd8sKxT+q1PNJgGi}h zlg1D1gX|(gW#X_r#DO%jh}T+|h!s=jpanf#pp1)Xa~>a-MIvd~giBi=*N{qcDAmsN zlq0~FL#dpXSWOki9HjFL4_~R8+gj>uJBd)r2=nyS;yeMyWiHO8356ul-I|?9=7JfQ zc9u|?s>7QKDXarcn9ZN#=7n-4t3ktGV_um5(%fxOh4Hk-i{hLm`;`ja6GZFvxai1B zCoXK`406*|yydCzHYj{UJfoU$I$x=g`Ho^P*;4`0O|LSoa03 zmTBH^9%yY?G9l;s6t{~m2jlsL=?=t@^UG**NKzkoe}{vzQ%+m3io} zNz;55G!cH;{t=-nuVzTwr%Ax#T?B6{ovl&9c`p5`E<2A)(8F44c5146sQ#?Hs>VJ z2dG+*aZVphdZ}xIkIC0;a4s=N!|0#LA8_c5Tq90tQ19M#_d#oxII;rFzdgW@b23sB zFA)5FrFHtyI-Dw1YMfB+utlvv*^da{d@p?9p7HjG@%Hlj>aYk9_)AagV(0YdhVnUI zzhQ%!(R+4r4pLy z7!d^%498bqP)rjFlWM=li|Uh8QW|&>j5X^yT!`3f7jsJTnbMOI6&@Cfvo)8T1X^+t zp@xa)VM(cwU_^{qZ1B8AwrQs|3|ZrO=ZT?TnBwX2>FLS8GV^$d*UQN0>-X>Eya?i2 z8!Sge=pgi~D0~aGIhD{r3J`V@brC=eiUccQvp7B)Ey_yA=Jfc;V zuB2q^R(G_F)27zD}o@okdTlyM}qGb$Sj3KvCdpaZ1_=q>s5jWil#E@PA{5P z$2jWCPf4$~1TV7_4Mz`G_&wgK(NJj9K1%JT5Tt+71j-d`UGOVChVNe`oHau0 zbk0-{V^bp)(bDF!+7={b02_SM8dEOVOg*+))|CI{{6djf(bW}FMvY~RtjLIqTGZNE z#k{0aQ?o<~HOtn$ZnzS#!sF-GqV*atRg=7+W}}_n-5#uO2=b}Wdd*rP;y&SI@_fog zZ=yM{(i9FhH>veJ7!6;mm{WX=scVgjoD2G++w8lw^pjEt;-|e~uz0G`z|-A+9qE7o zOcz+yaB5TLO=O$Z&McV405}C5Lv&WyuXPeXhh^#FN7 zm8L{ojW~L;paWi4;6d@!b^q+4rkNVmH-n#dF&}gOcz3sU<*@&Sr6iJ7bKy9g=eg1y zUVIm0+Ou|dsln#}qay@%zkJD1OGO6|Lk6W*yLgO4OmFt?vUT6hAQdpD)f)PiSLx_= z20S01bJRB5f}TwF@w-(tHCEyt-mwnlL2BJ@A{7D0L$)Qa;OcjeRc)YS`y%@5Hr@W2 z_KkyTmGwp&xBASC(!~>!oRL{lH%aeEUN~ zqkE4zKN>AH1y2mKL5<)v-4bLC?r`&97WVO&ikY`K#jGi6&dM>$_*g1w`Z0MuON)iI zqI!iA??mKr>F)eoixX7PX&cdr(*c`&RFjDUjIN+fKbH)vqTx4$L z>6q2BniHM8B8c^QJLKci7-_(y&4bg()=^CZDA%u}dDyWq8W!;QprF5#((iJ#6Jzf_ zO)hYR6qx^cF_$UKw3-%9-|nl>%F2_yoxhF52YR46bDv0nx8rH8#ihm;)Ajd&#~mTB zY#|tC&wXNX2mQT~;;h8FEO@<;GP%#ME67meLi;TIg$gm7A#j_AHq?8VUs}FTS3g)lp%fbC&(ls}wvS z@OxukUd82Y%k_0YMQHI|@kBiP3_b*N2{Tt{$^TmHDLvy(i-_RaB=5Wgi7D~n=avU( zS`CCs9IH`e?}!dxjVNcxKj(ja+jLCrHn2Qc(WdgUxeQ9Y7(S~%+A$~#JQmxxhJamv zVwHM>Iv$rNR6(7ceS+W@g4$a=*7YT}0D_arv(n_mqh@2yQG4rmKc|`y! za|uEf+l*o1-zqRF&!tLTdn~JN1q1WDZwamY;2O>mG~Mauni>Z%%(u66LAZTo`|&jF z*`T|}6#JnK`|8&$z^eVC2H+1O|60H0dV_bxN(mGCt-NmBsUMn1x{DR0Ob-OF;o98o zhL9K-SVu2Be#5krRvoYJ+}I7#e+r%)GC96AR_<4TrC9d(+PqP;pD~4SZw>ll06#Oa zoT6FAG=lUK(piDa+T;c+_H?}{XIZ{7wOU;Y=Dxlu-%K2@Akl^40ja60NCKM8RdGjd z9&lLVS7#fIna#lR$Wqv4Q)XX`81jfuH2$dvf%QIEiKnwpm>d95nXExA#Qs`TtP7@Q z%r+pGjr7a%%;5a;f{zfA55|SJ>-A{wqb^_q+oXP*t!!bos&>fW+AXTFcWOOPC|4oP zOoU|8-63RTgIb(@U|{stSk;|weq%TcwKC6=%uVdGYE*>keYKgJo)O7>vFhOi#-;Rq zx*gPHIxo=;QP66MUOHr)Jqa3OqQw9DRqYBo1*d-5=xeb`=BoK03SNr*kV7nyn#CUo z`K5Bd#XM{xJmg+I!3zpkSL_L?9ak_={gHrKGs|KBb0cQ%jy)txY3%t+f#Y1Yf*x>O z>=$ukx1s1RivJc%vTJE(E8<~NQ^Xgf!S~2H4PP(!hW|e3Gb$7o64O#4;e+fWI)V($ zVn@3MXU+FZ?k9YBWvy^H99@`gmo!zxp!vgG>VY_PXI$`eOXXb7i`w2q1viqiY8#h( zmDQebl&)lBG=HlspybBi{1mh8iEF57I?LXWnaP{1R2k8mxXQ^l8 z4@LFE=iOV07?#7KuWR77R(u=twG?s7Ix>=S-Q(7}xP5T^#4S%o*`%YKz zPA5$xpuSITL@E!>z9SX2?L_Tj^?&4j$E$GOx3i5iZEnrPMrT%$q7m3r2Dw*syl76E z%dI??D@9il*(+-H(lbGw@%L9ZHC_E=tTY(A7G!sCjoDv=xNTmq^jWv7i1nUvl}@l; z-doY>9*)$F_AGKXdf}Ta^TU~-uvMZRD0hhY6M~oh%hdf{Xei}f$Rl|yL78tZ8RX4g z%k$h**GGvpHK$z%_c0`8Y!17lOM2Qqr@trW#1pgg+F9CiEt}DSKBQ4Hu!y}zV(x0_ zt3H3fh_>kfvL+IOtM`(1Zcg&guA+=XCpR9TC*@@lpS-l5AykReEMsf@@g9C*?eEAB*%-h$T7WHrdJe?fB0VsFX zIWj$48*0$wZ8-AIgmQJ^J?Vn}lS>*j+VgXi?@O3n2*9aFKX(2`2#5IxvRLnBe(^+Nb;(rip(ajWm_+W*e z$I8Q{x3cWnP&?@$#ARV7Yk$ULoN55sMgYEVNcN&20fG0D5}cY>LlbaeI|naZV;Oi- zk#E6INs00qy={-UfO68@#N;6|eDSyJ(}h@M@C;FGwx*V{nvbuIBB#UEIv($s2cxd{nR zQAF+X(|LaDuvaY(I5z10s#zu(PLqTxQ1e4#FXpPpu8nm@Yv~Vy?02r&Ru{0IY)uUY zqy~yM;L%3~4Aj8k5z_(8&*@ot3BRnF6497H*AQj4@!An5+9p;FPb=NPzx?{0#nfHd z*x;s*2MIxKXRkP>>olC@*nT$mxDa4?7xgqRWqlVx*n^b?o5-IneXOv*XyO6Jp{1Sa z7n=|`byqO|gp+q?X}6NTk6o*!2ZrtI`wgsGid-f*ovXm2xsz((HyktKq#im1wU-GI zO*m~T>#m73#_2X3tZr9RzbJ4JT(uYVb=CFg#WAUEZhty&T^9#V_7EjJ!dlA%+n^5) z?W`=F;l!S{Kd)Cr47u~PBr0%(scX#Gm&6F^tJV!THh#rwE%}yR45G#ozfj%S(j+Nm z;nVl{@fLa{(mbg0fetS!*K*fC;W$yZP@vKzcb*Y1YS>_(^>Hr^uSU(#m$iXADY8%L zpzL$M=iG`Dn<~b}d$*B63HgcY-nOrMP0mIw#INq&Qo^iSy5p+ReWC9M`!_qi*VIqf z($ko0Y&knEz1IjWUsOHGqyMo^PSko4Zffm6v$hyKn}J>1Tv#mOdD?|%(*Lp4tUhCC zv*)vk{Yr}6hK>i+L)4UR!jhMoS$u}BV}+MnimcV2kbLBR(7Jv61lx*(R=X!wfSqxj z^r?^5t!kNt(LVQw%Wk^~l)v%%lnODVJCoO8CE)pSDuamxyp1 zr2Y*TlMT_d@AE4tyXICwVtBbE*FPgObJ(OdMK1bXqk1a0o#5~7(wqiaL%llg`F#H2 zU%gfp_B%*Qoo=2uL6Ac)QCJseXFPq=)9H$~HhV^oh*d{Ev?f%mii%moOIYRX5kw@9 z;qBm}7JMl80m|-6@rwp=*?ik-8;Mkifb_!qM#P96V!WwIapn}v9Kk>g86-ROzp02@ z9~xn~Qi`fUoIjA8iQdmIYJP;`5k5@9!cgu|d%tcSH-Wtiw8N0H8+4EOga!`_-PiL? zu8<0s6QE;6gg^{J*SFgQ;b%QPYX9LD0`>dH)w*}2)ld#m;vfmpAKU^81RRukJ+2g& zlyIiY^JGSMP+;{CIiyb;HIKZ^k-#rAcbcQTPJz;S$h9`0kOr~>f`Rp4F4q6&U0}<| zZu)o%=-ZUo7!i9;ZFn0BI+q=9{~#zPYsh8)y*pe?gN2Naj{bV({9VC>G6ODZ7{lkD zc)XY57pQ+YUgOvk5XkpZoYOQeA$In02qd=c6=f(CZ~0m&7vFQ(`y&2}K1=+b{(H9; zm(BxWI{vb6%AU(#Td@X8nCJ_eH0~5)M2H9KEof^5kjl+L}sB`bMz!z3hZE+F`u!! za2k~6&$vE;M$mel^`lh4WGp%cMtHCBa^D}3KG1Q@XFJjHL!XA9;X`yz&51&!#ecEF z63y<@yNz$Jf7yXNu>y zK93cDkdaA9V&Ebolr_ZpU8xfm3SDdY^pbR=(udAB-)gJjFuypmG!}y~#lnd=ko*-2bCb*>;>@wA&Bw5Kqnd(Ro*PX3* z>xf$Uw6^I&a2;`ysz6$reyvQrKv>`hw!cIV#)-x5>VzMQU;XgHWQ@-y-ltl*T0?0J zg=*%`FO+E2_qb4@6b9^~ZvJ-MXw{EOSFC{p`SDTE#WQhRm(h6L7Y|7M`dbS~ZwTZ0{M2EQJk1(69Yjo!6bWLu0bl%)%a;COR&GJOa z-6CPKYD7hn#Y&4@N4461*vSbrf%K|2l8W_i@~Db+%b{TU51A$^5dLQGSXsM^5|`nCoX z9oo9Tx@rS`hWgNcu_6GR=FIDB_z6%>-zlA}=m=*ycd)d9G(alj3;ThS`b%iQIjGd* zm4FUO((xMn|F{6tT(eNSLK!dp<{ZtE){G9e>!*Q|Hnj^!(vkWFZ|;FEQ`NH9u-P+v z4SO;bI#LmuZHjhZap~HoiWBcl))?GSVSuj;SQUlsccVL z^>Z`aGenMVa}9bK%`v%=g)JO6>>=Pa_DdJdUtRpP`WCEOK!E~4gKSe9I%CP;=m$$& z8W^E?Jwbqgbb76?C z9wWRX@65EDy_L|M|M9NHyT`n>EV3 z(!O;@=oxX3rgpF4NfKZt--1!lAR3W^? z4}}vlw}4Qj4z+Uz3mH(-Cnx(S%HSC_F~&Xyiv#$uI`)DUhrGzNpIf_D4&4d#b z^Jisl;e};Ci(k-U=O{RQ`yjRu?R}AE7w_6aRx!bw4RNx;0$(!ntqT>Q?s*pGNkVSl zC~3LRW=tV2mtpW_YVS<~-g7Zw+q9Td_IY+8EWKYK{U`v_zN7otmnLh%6u}qH)GI4vpbB7K(j2^OX@vn!L-c}|dTTyJHCF9gJH8PU z&}~6sj%)^`i@L>ZrmXk^Rvo|pzy#jw+Js?%ISBfV+C3JZqW9f2->`oNoW5OizVQgu zW}*hi{P-`J&q6qHB>2a4@|05&H0rnTG`C#8T1Dc}6L46hyKAhHU;QAULz9M&lYK*m z;=@Y*Yx?!BAPO#BoP#1Yn|QCh*vWMpV@OZ%Ya}xN)7Nq_hNE1}QibZMs;q z^m9jK=cjubC7B>1!;&z+lM#Zx7ifjcMM&5vEmx)t4#j;S}1Tp!GpsOca5@8x1jH+&eFp?+3OsVD3mVKhxj@3qo?)|i^Ee}NTsJrBXfPg_}>BITXL-*y1)eejHz}A+dW9pOD8^?^(Ii}G# zjc|xK6Yv%LjkJdBA)<9~!Z7i#)tpfFuF~pDP=F-f$fg!kf(?jf|9Q;bk+hi9J1SKTDO25H+=1m!1?HM~`(v zug4lUzWF+ax=~Vh8nkb}u)mzsWlq`0H`kRqpwM{0$EoYd92IfPs~-Njkz^_oNt}fi z0{Z6JL-cB&0I=de{lXmonMmurqLG8=OY7$0cO`(wYc>YCBALL^ZzHHdU2>lS6%_I| zgSae~y6HpUfv;z$r%bVp1MSVewnyh~*m~A+9aou`?lVhVJgyNz5+BZ*#SCIxb$+*c zH{VR%e5CO|yKEcVN-=luYS>sW!s_0R%dvHE5S_oq|F{x38q$0IkGm*&&qv^^TF-6O zF5g^%C-`iwdhfaNCw(cvsF>pLRr@HS*!+c;AY&>hcWK6{K!`#Fd>Ymg33(AO97>xU^!MN!gEdZfC2NG!Zybdi?`*9KJ6SDH2FfjwWEk@wfh&m9~HY1 zbEH_EKqU-1aD5;H;phv0?-QZ_jxS9g5SwQiD}_pTNYQ=W2NkJrSfeI*bx1CMDp@B- zRH-H;vZk^npV*xr!NrY!=4uAMm1+&ofVb(|#_au4*>B=p&}%ZCo{a;T@$H`TGWAvF zPHn0QbH?T_K`EtP6=Ybbhs&wj(@e``hXWKeuiBSNk|X5O$CQ6OZ}llMWd8a^3e?BW zOTAidqRyh9+H?Jh)|OT1*ZtwGRcUA68bcu$J5F*+w-v|*JOhq2+x@O*n}X4nbQ_)h z6N6yaoX+}%Dq|QbG2D1|_1XdTZDh&)NIMl2wdBG5rk&q$DqTyOJvVoDdNQ;Q!~6Vi z&r}EC$_{U)3_!D<(M)^)({$MSD+Y>MWabnS*?48*CpYwXr*`o%IH05ZvwP_b)+C}& z#N@j-r!atAyo`+SKl)a#lKdq!(lt{$_9l#9=x}5@ zj73xQkfmHre(_hXmfEQOkeqElk*gKHRUgq~VNs`(;>tp#kl2lz{heNcXjH#zgmLv% z&*exF7~J~95xtNP6E4lAU5BXS^Cx|s7iUU4obCF&b&*a?9WkXBiC1}@ z0wgpKFlee7C+7BDc=SDOLUczO){7zIJ*Si89^B!uC?;HK^R@)v$?{M$3S8u_>)WrRkv*U+U z|I>E=`+>*1`*p3~t&eNu{*Nktx%#>tm#zN4S0Cru9&X4Vw~y~R?v6yS=~o|XANP(| zXY-8i#vhkOAMqd0{r}XK$8L4rl0RDb9}|DjfqUHZzp;4SC4b!azq)!Dx_XekzwPwC zompp*!GNZp^>63;KSa8pjIedy9n_Amcy4D~cxjdtF&0bXqyJKVMa@Gv8|zSZuy0nD zJB_+?TNdM1&qnENzmjwSZL4#!FO#kcqPdn^p+Fl}o#GR!DxRIFp!V`Gk`#Hk-fUd; zyM;8acqwzOIQX4q8a@7*LwACiCw6E9=T^F0m9bzC6l&W&Dhe`7G;;X0$XE;NL!CPV zzVMs!N#_k~Y197FU~ZKTI2c%&sCh{Nac0u{h;mIMz5KNZH%kG#yhHX`bk;_r|7~OE z&8UOOpu+7IUuOggfq48jjGu`Y1XD@a_4yof+H_Ddyx3)Ju$GQ8R_XANx{#@=x;Z-u zuxDI}Y`v!UPJ>UAG#1VY{i&j=mSDNZR3ru7`!FJc^g;2c;3DaM0jiC=uAKsw#< zSXsx|zMC<+pTRfsIRQ4PRy;Oip5E?D045Yh#hs1x9G)5}P>7`CB*vlbW-HVGe$M}% z&;N|?=^+gTdG7j&9quk29xpQ;I&K;tW>>ozkl@ANJY7VgG7J}sE`tH@8M@ZZ4}!|4 zKWX+r3h)q8;AbT8$lrZ9U4M6g`=|*1IZ51*natKL+~{rt-@_df-c#9UUvITrTnxMb zGg|a5cshCv|IEz$B?d=^_^OCTGi)zDBoY*8GHeeecn%rte3eOwHBaAMexnU@=3GJf zzkkl~ZTC+(9$U)mb1#=PIpMU(^=$9&Rd)8a%v}PhF*@DGCi;pj{M$vGElCS7Kmhq- zr1O|`QOSp2IJ-prWk`|l;Kkzyq$i^7Gea9FN*;(xWTtgziq_iHgGRy*{U+tF>Om)F!6V`e#*BKQmKZOK?@ zG!`yCpjg$8t~@tks+^6!y8ie{b2(OOe5ExHt@uzFD4gG^{~G12rf}Ew(EaXD1>W(p z_A?t+U(s-9W;Qil60N@tJ4hWAFZh_{TzJ^@s?(aF&J~Eywz9LP+rD3-DUTijJh={~ zzHi*U_fRfTPSO@^hP1M@a1D$`*`F<{hpe#PT*kPQ)VNEIkFgaa!8jcEP&;h)>#E@b zQgy}hX9tw6^Cz?F&nOjnXV#NrF#>Sj4@&J=k!D04BUW(mwDZ%?5h3D%9RzlyKJHLk z0uG=J8EJ!%eM4tvnEC+whC4fde+}lYZbD}684S~2#;W>k@KEu~qc=;znl0<_U%m)L zP_A^G#7f>lElpl5D!amYF2Cl0VTtuh!Q&38bAX;NR4P7~#J!dIFz>Rjv5BcmZZsfg zMigE3ukqGcHXT@;Mmdl)AA+>bU6SS`pMFwRLNi$EpR?q#Z{#eWj)!wR0~gAPB0$o~ zQy9-cEk+oEQ|TWtNQrAFYPa%SW54Q*>d&PPw;pJCjX;NC7yK#lK3eFLBL zR)X>M$&eQQuRHzsuE<5uVmt}5>DDycVDbYjm>Tlikl7f3${I7&F9vYTO!=ypdaMDN zLlPsPg8T$pa#67_nL06tEoo)mz> zHmXx?Lsh(|{dDmlFJvJg?)08YONOi^!q@ zfa>5&y}eW_a@BXsXE1c3{haRqa9y0+Y@ZscH1v=qrH43JH|oPOXEKTP zqy;MvU8VKrn`;1IOwL!Ekp8@BrG86knl_f>l)LTZW;?fY!{EsdiQXR6x`QT}nJl$o zWh_jZP;s!VEay-?BgIdy-YQaW#j9saY%mZI%Q}=w>d#6;I@P|p-i!mS?jdM(_nSvO z+)s4p{7`*0yrcU1$q*~m5)W)Of^}w6Uf9Hut%M0Jx4V$J@xwE!TR;w7M!UHc*VFyf z3Tmv%Hr!$kW@kv>FKEwmW8s1pXb_ueRlGiN9IztYV%UQXgE~@V5cP*T8(#c)frra} z>f?#CjuDJkWZjnCSg3#!-@~btsq#&#i0Cy8u)(Vo6_u`UtKgTj#X=+DS(=gXsqW%l zM(6TVeRlpL_pkoxmx^w~&y*;O6{QQTmG7qGSYgh286h6JB@uNk`Kijo9y-VAp_9C5 zm1}Da@p5mOG;pt%sF6<7823WcHC-^jsQVfjOZA$3oO3Iy-g=E=~x!Lk|(v9yZF10PtRW{zS!SOfAlLE=)8;dmc)~b09}g_ znTlI@)~WdZ7D@k+Pc~BlAXcR6se?fLF`}&U*$5-H(@QCNlrESxWTCp?Wx4kr_2|J9 z&c75X2hg-?+m~GV|K(7quFI{~*06p>18*dx%QKlB8wJdCdF_w|>^Iq2LQ;~%23zez zv))r3yw+^^m8OCx^=i*BK}}?p2NsjT3-hllG8BrVBB7%1jCS5gE(DTLH@1aeP=>h9 zV*$NpBy_pJ;L*eAt^Eidm)PkDyZy-#!cxa}3tdBrG#u=W#K{pmWq9UX8cdDRta?%t z{ENT73`+a^t8mvq6JmFh?Si#YfsDco)M`%=R(G&04}@1SX~ipriVSR-2{2-%8Qd|; z2}sz46;vfX7|8@jI`dy?}i}0S>SLRISwzc+l;cBM(5V_BDnCwu@?PBx?V4FU-3DxpI5ie2e4dHwaUg zWB`S-T`P{;g4k6r4)4pttM zR_(*G!2Rt?c5roTwM(Zs!J{)6P{eZkoZdbYSej?y!7Y+?N4 zIfe8#CgZ9Z5}Y~I4EZ=&&WNy#^EG2W1Mxktl*(1o@oF@A=z;jnS20m4!N}r1I4#LI z6gQq3jqyl{F=YZM_|Sc;HL;P%=22Q5vS{Sz=$@r_crU;@`oSP9V#dHX*@wDh+s0!W zHV~^cqgfT|EBA0@-ADC^^2v@Z@>i|$Q>2}mx>>hH)z#?Ff0%&`-WljO6IK-FQ^FNhN= zLU7Z2pWMX$IUmPVTT+wcsdN4x{t@x%ldWPT_rE(?bm+XF1%dl1m!VtGviy`&p!;+2 z9?6`2m#fLH4p=uOiz1?U+#>if1{G#eA-2Wdhd-|BGsu$aU#V%%ipJ4Bv+vtgbyWi@gQ6Ii)~l6fz^jg z7ks$Ffrjl@YPf>JhBox)5Ty&~9dG354*r?4vr%sDuR;IpYUwxdqEeHy?ma@!vpM=- zC?G9PyY~|al9DHrpl7!DIqlwkEj^Cj{bNOD=kH-MW=we4f!qFla4x5c53wet*C3}3 z?CX-Xod@iC-WYx_*ZYZGtzE_glv1pEYnoIo*YMghy~!C9YhiRv-3@NGYI8>33!N_+05>nK<&6&co7RPH>MVOq`) z!&HaqXJ{1suB~L;u|nF!g_?g`^9AyPrFf<{^C@upA9IQlHGrso4qYj!fxo6^h^}g2T$IH_+i$_M= z&dL?pOS2K+92#}JJc(z1_6ZtW8zVKDybU&M+{Ep8ZjScW5u(#K^8a-zN5j1T)2F~t zptYPi>1Pw~9XKS7K9?-_72rSapCokVEaP%0&JH(hb1!gdeQI%|b*=8IAId9iq(bxpuFeU8 z8kM5uw#*FvFS9d2SBnh34s&QghgScH@tNO@i|XOJ@8KVoR*<>DAZhkeIWs>WM%=!) z9xP;$$tyte!-E z!%7hXzqT!q3|{`NOxfA}n#}DiW!YN=nUY5K82hWWcIs4B;Na%^Ns%0NYH|2xu!NlM z_D~o+!0`zSaDtT>svi<4$?Ex;ezptmOhjcQ3+KJY0te9XAMi*a6Oo;&&*E^kQ`$Oz zKVRDHA1;Ok{8v3y&HFg^Qk^my(*Opjor}Kft%8oCSXJw(s-hTM>`O~0WI!Un@9i%v z-5)qqg=xs`?MxD}lq}Ch(Wh}4jTC3M zkVEUE2=#ul{IdJDiI!z|MM^UV1^YNSwe4Fr8uQ^DUkz&g)r>FwhVj^2C>eION%^n|XhYo!^CVib8>0=Yk^0%c^t;E+`9y2gbgRgLcNn`4Z52%;^!h4%Lo&I&az+8-ePD-oL|%KXkkz4@;i$?{u%Su}GeG0tx$75$ z!FelzVH_22N&J^WF~Xsv;F|UwIGaBg%H*VZ=klTkOua)59V=gQc7~E0!*RsYjCtd; z?2@*e%67?KEl+MTGGN}n(l}?ewgui108r({tx;*7sB>icmCq;^?r z;{XATGjzRCx;q}Sxw(UF<68UINawpn-qD2z{VhRF8O1a2i50T6$T`wTVF9OMfRLhO zry-ux6c~OMGeX1(g*JYbg}h}A&z*~hp$Kv9`s@YW<@rq{S9*wPknr^n2yKLL+g=VW z#IZ_Vo_U0muW39}5Jd!#YBf{w%Ck{n8x5j(*2)ys_-@cJ%ms_eegW}~9N{KO&oJIb zv7g((hcDJ0Y-A3~Tz9czQruP>SuP_1nmkMMehR?+Mf;|!0g$M%{-=v}(3r4spQEd- z?A_5-6jGFMws`y#kI>RsoV+}_+rVmMa=A!}uHcmE9T>LLky8jMPj*EPz>c`RX-nhhZ?27q~%ZKIo>h8th$w0(7S^8;2v^>MaZ z2HNGzf_rW+Lu9943*%S0E`rsUo`wDVyVkEJw(0qfTZ*Gd^%3OCNdue8>z4X{=1hN! zs$A?c1LUh^xB{uJ*_B@Z46P(KS0Nl_v;cI=B5$g>+?K{fi+-r`2q|WBm}_HgUTY9} z4(Gym%IF%N7INrJHZ=!7HK@cDOLWZIWW~89c{f@{EYEfuSt;SUYdiLq^3@`4yI3$O z52;^qO4`4*0K{FM!dSw(;=tr9gn9jINh>?Zc?-0SaO0#$_`m$g<_eKbXvev(6Kh>+ z&u0Pv$IcMuFius8&W%xv(C)l7uyg75F6yN`o1r56`ieS216pp5a?~Wj8chA!_j#NPTBE} zx+CSj2+1bZlI8gtr&!i^oQ<-piq@<%ujo_<+Bwa{yp1eAXJgx*N_|1#+boLVUP#^M zk2V6)-$Nq)xVROFdKHkhwz^Wnu)Ryh-;+Oc0!urXQ#3!YE+}b7*{{it5rdTY4f9Ih z6AxTFF54?%i+AJUlf7@fK$qimg7W?{x>r8&HN z`ZPY!c7?09t36qM&PVxv1Tqpd!8EBFy6)_pumB*c{ z`|O);l^7!Ek;y!?(}S0I5qHkupgGQ9SPDzVv15pOj#$Ul{3DfUUyNt+?`oIFq>opOHriQygA^E>RY3-S)NpEQWjPx*Os(SzvKHfIov9KU$R7d}5E&i!5lTnR{5@g7 z)S}Q*>y9Es2-}LN=1nmx5TGG1c7Bp^kpg*3yPc)w<)$zvH`Z*~Vv-l_EL&qc7Dchd z{mx>z4BgYNcj9B}{ZB)0hFCqRqPWcXxi=-x%b+0tohd`o-U4#rO+F63|8U<&kx@T% zW>|LjIO=mMo8Xu0Ce2lTy1%Cqwo3%8_T42}*?dosrtFI~Y?3PMJ;*k_*soh!FPLTA zdR8(l$J}4d&t?}iFtH|=@jGnQwVL($-F+yHlyeK(-jh!c<7d{%;DPC_#v3TCt7IPO zY1(Sib@i|uZ*{tFQT<*q5s3{j?I)0yd=w;mEy|r=!)ga%5cF?Ddixw^GT&4fR0~Z& z>8mK;9aX`9iIMY8+nks)vK8Ob;%(rpY{#gHzbMQ@bI<{ImIq{Vcn?2@p`X=SJgo zx$@kM{qU`{KHodzIcfaDtfeN4zVV5oKf{uk2vZjI9>@Jm+FW^4*B3s!jVFw@niMFkXwOd&rbARY!zIvc&qsAq0yiKQL##;CJFECET-nGh%R$F8Z zF=$_XY2+#>10V$*wUKCThrYTco@wv?VQutcthL&uKMO5nG}DScAOConA$vlttq5}a zT4!NYg-1zY?#9|_WgG>8EBLITpMv$~qw~V68F9gdRzLLj@4Qa@s%c-R&$9$F;s$*&OH$SbYMER_`Sh+%v0 z$Rnc!r-INB25PsYZBqst5I(DJm2+>0;FBAI4MJeFbR=O2+`_d@gwd>iy5<{-HjzIs z@Bf2lB|qx_MYG6cq7|PNdyvH@Tp}xjMCZO$zM+NT7ygvfFw~@INI*>pBrhO|vSDsC zz?a6D2IC|r%w1&UZfR#|Vo*ph9+|suoVDfhF6>0YVB|sUP@GyI^w-%KO4MV0wkVXQ zYt~D}n+n#30~D-#zkabd95RJ^PyYCHpBIup`P{j-bGzm49YWbvQ`=OWsj}W`#t$^f z(M$WNrV5yE<2jmUFo1dzY_1WQ|+KvhZ*TU_nmCV;&B_ywWdD5wnw$FCMf&K7dTR^H-Uba(V@_%=zz3H z!eu5q^4?4hSLH(vm>@B8%urKAjM=~LBKC56`VtvwbV)Q=^FZ2KA~OBQqG-!D!Q|w@ zjBAmjKneEhf$Bftv%auo@ek9FGICr+$XZPYlBC>z66;L#m}9!Zm=UEdn5Zd9I&P}0 z+i_#MOc-huO)$>6FA9xNr()In9Yt*xQksGZ(rE38m<93OS}}X0mE_cH`l1^NI#8Vi z-U5V$7@7EzuTUTRAx4@%=Z-!Ch53giqpf);+5KqNRohX1;!`m^Z-PGs1wRWo$nG?1 z#_S*LBT4rOG?202GnDGf$&yo9gtCU7y=@MaB78P1r_8uIfjFy(f3aTtXN#HYFLE3N zU%}b|>l3e=etWJu*@AG>RR#irM7i-%U)gBU=S@K#N3@7H^7gHj2NiphzkWvQk-aW1 zZZ=62L2$;N59c4BN}+wCUor3CMRD@+jCbacFdD6pRn9q&y=w86*@<|M*1_b(%ioga z1w^+crGSDd_1(Kv$@O_KIn?B8B!6m9<=3HAZ%7RoCt(2kgb%N}M-nRTcQ{Q&1L^_n6);t2r>u- znBVC!bJ!V*IImNxG2c**+fgIU6odAlzm(=f|x6RYo+=w*H>>MfOjg4;rgJbe8jayU{w>81sbH zj6)$rXP*2B@e+j{8*O{4Gm8mX9jq3uEPwJa!>q(C-LI&Jh!$~c8(86htxecszAnlWolt$~uJC1? zHz&i`@*m17v%1_Pv=G)?y9z_8{2X>}gaa3gZNt6R^RlvL1F!GSac#v@%L?1oxFa8y z*E8RNFbiHr6+}T8;kBymVC?WbXcN~zl<3nF5WC}{pKP->Ym{6IE->?q8^mcpOZDVH zApj!Xyj*eFY52}l^VKonZ-(1$1}n9=FQ$3)>EEfM6k2wij z&RglE&&&EV&iSmoYG-ftJVvM{tS!7H9xm79+UGsKuSl4gnZe>JZ=Pe#Bt96 z&CS02F1vU|-UKn-x{RovA!DKGhX5R^Uo&KK2M{u+R?zix62y4^#{7p_-D-={Z-3M$|Lfa{LGoa1ux`1)Ac(dX*1xY8;C`X@M5WrK&WhC1e18qHed7RDNF zN6Wum)2yCGZHMS@T6*C5sh)iolroghGU!~)`F?ghBpMRHzc_ptKL~B-aa^Yn1|b&C z=uU%`Wz}q!GUY`6JLtEIAgRU{Gwxr%Ud8x1f)dwf6#1p7{>#-~WqFD=CQ7ftva%5> z`?*S<8zc!0w18pV$qz3t9NXke{~Ru8^m`&nq~iQf8WOv=y7V?aB{a2P`~~GnF%uB} zrJRan%+>lGXeORSA*a?G6-Uloc7C;-eQ~wtlYsNJhJ5z4nWatHNQV>5Q;o>*CZ8CH_94S0|apIyREyO z`RzyP>|dPzZZ=lbU`7IdyY1{R+vMJ3MJ1@cv(9i8FCVJX1lG+ki~B4F?F|#YMCu{n z{Y$`5#o(voY^E#z@%NuJf!T3X^%h0|NvlDtwU|ChSdyarCJ zGrfxR$YLwz1B;rf0F#n~wHG6e2`OIjG!J98rissd_F#qMb z%g3g##4`(QId9W^ttpM9=T{l7;Fq~qRgNPYZHtL57pD9WiM(@RO#&htd|6QY;S$cw zU#2gCnjsO4!<84kv=SDdVs5lf8#5xXJe?>|q!?v_@1ddK)dDFA(kQl3c>5u*0@KfN z$f}u*WB*95ES~K#N-XPzR^FDN1ea8!*YQ>?8pIVyC6=${%}eJ!ZhRdyqb3x6c2bG; zc5dTNrq}aB8>Fh=wzY%j?ZtA0OSdPNS8~sa5UK023i```NtX;Y=OhM>0FVW9rXXnH;M* z;0yUJbJBF|9zy1F5sY8;Izd9oU;t#^lkjR8XkRTPaChQ?yijlXRH3dw!F966%mS>nZR;;yG8 zK&P$0-G1Y`;6(Xz=A4WsiXAHtk{yzkMNh4k~Em_6ba#^QtWW zMKVzdN>Tg|Y;y#u{@i)%6LWl%<)p(mNe>IkQJ>8g8N;6A;_? z`zc6Y&lcwAe&1iT8rjf1J;9&O*sGB`0PVWC2nnp%g}6YVS*ZqKA;Zw_N9lOP5jfV& ztowKPc^7t+O2&KU#jZDs)H#!r=*=7#b6I>~s@mv+TOd>_Y7zEua^b4_CsnRi@e(+gtWOzejWWnf zHZL+0va?bJ?Ix@yUZ&Qy95hyUYiq!J9+tGGj%CJ4+M@+A^Ea{3s3ys4Og{%@P+|@R zoJuh9OG0CM-|HBNHE=;fx};a`{7puRk{ct9E2!8potn)$P%ih>)&Cw0%J3TfIQ~)X zben?09GGrX3o7lO`rp;34|C|~Z#=Ua(yS;g5!`SHKTeRmA5PLzsuh$T(<=_C$jnd+ zI4E&U<>6DwF&DIF?DTMkilw*7W`hz1^;eq>_CO9zGVk zqrHTR=@-Y{fnLvrhIp&Dr46P5p?|EgEX)Fr>M86ANF1)YNQ8CcoP9Nt-K-O`A(8A? zsNo;|rGK^Vx!F{FnvQEPC~Dr$mOUTa{72v__dd}1KYE>r;Rc&$#>u=&Fn}Xj3XQ8p zYiP68ipi4Ga0RcMd)Kd>JX=8md~Qb3khDUprH#7^&%&x1Jsw&MP*ivjZ_Cj=R&$sO0*awKx_nQiqu&F5`- zFI7j15~W3?%S#1OkaJ9jt11sP;F;F5-L;og+UJ*@aGPzQVx59uO3p;m{_GX>eB9;IjNCF}H-J5G@{i39rTTaR`}+ z>68T7cSfpJ=j2@_LzVoCi&rx$7dj?t-J1JL`T{?%G>8xX1Rx+kaa}Gf5)5V<-mbfE zo7@APHwmN`d85VB4ty4e81rh{{dY0);GKq3qh2ge?-54!&TIVV#bs*uZ6ehi`+4Zn{JbQq(Xj^3#^oqZ;2L}KaMk;e({+03>?R9;L?HG zIdZ^3n^;D$ix9^d;YP{FfUo*L78bPEv#PX(Y&NS6g#yCH&u=*Obal4cQ|5$ev%kw%cV2Tg-7j z&r}gueGhN>C2kR$0_0#dbo>mh41~XB=~$6&{v!EMh380oTcR|OjYn(UdcCIN7lvF= zU7g-`r2vK^n|~z1fot;dwfGWP!Xg%_oBZ`fHiz4Ja4o$jVR22$*XIU&-*Y%Krdaj@ zHS9(k^0xNx_tVwQDR^QLmyT%9 z#$L((4ehr6^a6K&y4wVl-T-y1`OA4u_`+547Y*v>Hr>a${}8m{RD2MZ&U%dhC7@_$ z64-82zr1S*ef`8Q*UJp;g(xZh=M=iNxBj~~~H?5h9cqW^VIoS)}q-?;zz zdh04!&mbZ!D0q95+wr&(Ci(|@a1fRIcoTN)_d7^sY|QT==&JMn;mYrH2!Ag2aTnab zE!!z1c)L+>-2Qk3UT}kNVUc^c)8oE+yCS>o?RVaeE^>DQ?zFJzyo*BbykBwmzZ(KY z_J5|1{eK_2civ8bi&L=qpUQ$aSSI&(a=Yx46}{_m7rpKpA0HbV6Siu}0DB2XM~9jVKkryXC8n`v-JCKR36_A#@RsV{h{8Lw~<3 zbpPuKd^#0-Y*!Mb-ZLH&ZC~WoN5^Uuu zm8)u>M-~64C)joDWb=F2Chxe)Cikm}+iqRGS(i1^Upc;u&C2#Y?Ll9;FNg!*?{cwm zHOI*B{Qj!bw}~9l?`9Ky)z`n%IM?qyF^&~|QFhh)G3dTd>lzwH-gdPd_u}#F?6~vs z;Yzey7T5C=NNpgF(1D`a`s<6%o0+Od$m34#{oLcw@m;3T{hw>{7b8)9Y&MmO%nTTY zHu5reVlGHaR+ZBofK-xROIf3|!jJ=c|c`_Ui|T=8*!G`6vr3CXyo^o989 zVpBD!cn2^crISLp?L=SMWjFz*dQdn+)oX{WN<4(b$V9ta-*IgfTV(YNhAL^p-i=%rjh`_Tv#ZkC|UG^-FQ_}v(?DQ5}ll>d8nUONmJrJvUuRJD32mmw5z0aHO-Q5 zM+?VLXgWk6&=hN5wJz0UEv?^xBsOBdQfrH2rc)@mnS}&#=|O(;))}j(54Tu4^Ua0# zCsVqd!s)YmjSYIu80tYdMErhAq``<4lv{ZD}s;d39jzSnw&n>W92hKEuNX zLY^Y5SWu&z7GPM_|LHxfm!qk8q5^<{#S+Vu`XrsC>{Hx16t@rvt>eD)d7jE&{er6` zY1eb`MU^%1yJR8I+vLLrr=(d_YX)5nKxR}@a4Q)oT=M!KP3Hx} zduNmgd6cA22lnIa5(Fkn{_d3K61&NME$ZQkcFPKB7GFX)#)_vV$J3`~VjWcvp!S}5 z@f+2W69O3Y2GvP#br)QlRE>J)-3FK2QgAl3wf40-?j2{B)#ToNem1(FHz&|1McZS6 z3{p~UctUM;8F7LBH)sBy&YKSRv#ET1539L%M&PF4G$@l;^nSjxsrlsbhOe`^`DFE` zQRR+3arMuBCh^qc?c?&&b-B^~=xEI2`tjq+XbgFCoW1>Bxlg`A=fjKzc#pB;tZSaf ztH+6m(1*Ck3;)*UljD2Dr&AwqJNY?IANxda2GF}JJan*T;_TlDeJ>b!0|QhsS>Ecz zxMG?^5JXt#q&~RlK~HO3Hp&$3_%tMYHo@A0=On->FM^{XZcw;xaEFnCUN%I)a&L}ZkQe~|)RM7aiH{kUqFt|_o zvoaL27NcmcZKJ0uYF)2>Hl1a?f68Ot>B;mAPR<@R1P6!WJ8CJSo7R-){wu8)QM$xt zTV55rrOT!)-`Sfckv=ro1sNiG4p+U^tZ2OFC3?Nd8=c3g^vPrx-f65$DQcX?$XA>q z35tWYgtvxaF%{xP!nQmBDl?7NcBx;ONJx*~v)`2cGA84tQVhH`ZpX)d&v8grYTysSk_|JDLTJxup!E1eL3Ib5UMy}TH1Q+JCQ=qh77)G4#? zn(0M4J;iKsFt&wHH7}Aqr2hVSSKE1P|5L`gOv+Do_msP#iXI?a3b%CG? zA1#$Dd}2X&*LBE~OUf-=?DGse??g%#YCVqIlAg)vk5ogD)V_2gz<~?MKUAvbPKa`Y zb))vJV>cNv2Z=gsSkFnp|3y$xuK*8-Lh71ns#UlTD|DJU0#9dCR@`hvnu#H|#t!;m zI?vZLO0B^%%+QNPVB3|v=_MeRp^RXFk{9|NHC)6Hdn^V`W>%74cz3)fg-ul|X;w0= z^)eQX8Q-KEEu*>EH7mq~44f7k#|LJ>Ez)^1GS00M`c}6C8(xKn-5P`^Qea zsl=7}uAnER*6uxpO%mE-SmtOgO`ySbYQ;m+tv#n8tP~lhGjrc@B7@$l+EaAw?Kx$Z zIc=%ewPBvDTW&V-thbak)s2`LET*ttbSqD_@G(Zul-)pn5Ph8KY-v8>@D2o32>&Uv zd*Cw6AQmXKE8F4P^Zv_!ljg2PPn$LgdaS=VEjFb;PAAF68+CeJKH`@f*zxdvX{McE zAY-F9n1aou#d}xioyu1*JMN|;qth_p6HPy-Q-9)bo2zb4mY}Iagxzf z(hx~seCi=8@4Bhq&O9XjZyA`f#*b!4a5~@=MYmYL13ded1Vd*U2I9>58weYoT?3l* z;!Jg;o=dxH4OhOU3}15LkCBC_#4tci`BUcT@lpJNmL>ghYAnlg&gVfL;eaP8BI-7FGlqWMT-q~2#|FJ|k zC`ixTvgGH51o@g|+x^~K;4dez_$28RACX%AI4AcR|B6YSnWU8cc~H^N^L5Xh?k(;7QOOdU>*JR)w@&rO*87#i)iC4m za)rqPSrrZ6{ZiR+Y|D%nQs~pqf~K(8;QQ1BU@5Gb%KtEi2fka#H?`#W_5H(1(^vCM z07F^R;@H$)kp($_aP~IRa~aio3LP0`Jk1Kmxm5y*E%JhlSmDf6;{y;db+^D4jcO67 zKk-UF3Cp|`y+PYy$z%*EG9>pMk}y2M#BZuiKhhX2Tcc`+$9A0`OK*CjLc!@Se8?w1 z+$%2@PwBCCA_D=ROS3GvVu!&wC>km}2m#D?ZRSWJb&skO#cE#BKY_1lw?jtqhv*cF zlsLTykSLoS(u_0UepI|miOi0G+*XiYqvVDao2NNhqBinj(@-X{@(hc+V5@Ym7(FfH z2d+LqouTaDsle^nmbJ|}pkNk+xPms-ztkQv@H>?JWmAdCdSSv@H?1U|L&8!SlJvWs zYA7PB3I#`RQSpnhjV~CsA@o32#ZUuTA;pJAE^Hclr{-Mfx4G@mxaad*?SMl-t8*r= zo$lS*wMQcD>`!u`=IgG~6S}MNdk4{ln(MA(CGO^BJ4rpOlOU5$htvIU?zubKcWdrs z4mCHujjP-OZ)7cM3&z2%R^n7v@N6IY!}{i`&fS{I!xbco{O+ohj_>!q*6!oCEpUbj z%h>`RX(CJiiAz3 zX4(@Ww!PAwpD1vR0@yU4NXAVi4WOQeoN7ZCXy5-XY^cjew(c2t{J~2h*nUK-^U?Yw zNGVJ*DXI!z^g2w`^3<%bVA?dp*vq1aa^dsPkB}kj#L>s!-m3<8YsZ;~*u|L#wUGJ+ zsMu(zt8i#NQRYX5sXC2`Ph*1V~010X9{R1ObHB|G#_=?wlfhMRJbJ#-**pC#i+?(IKI9L+wux& zz#bini#APDVq1A-^F?~6DdhdE=7Mgtxg%`gps`@z(_)7^WO<$>eNs4S7pGPu@?|Ir z|A(!&42xrF!ax@b8rDfV`*BjSUn1>iDa*76s zp+KpC6~6$EJ^i=kJ4ImS_-5{ff5JUY?=8bJ#qOwd8EG*)l0YlTdsC-FpWmB6>7Ma( zO6^Qz>a*HGBVwN>8Yt0)awNZA~KY^l-i3Acs^E-VY2_`E{FP z4<|_$?CO`@n&&BIX|3>tPY{As^CGXaQ?%;^X+ty5^lxorYqA37|(PveM4)qSpVW6i9j9@HsMO)@)O+QJR7HdMv?l0kyi1`2JvN@PNn7IlTfzDA30>@N6 z#r!dn`WyvBB_m`iiwY#FJ>S_2`r|`hQgVp&0fazT@fmw$&d0!dH=WuE1=I~e8DS8M zr-fsZBZlfaqvW-Kr5|!Z0io+-{$w7_mf1{CG>e5IPp5YAN@qpDi*k=89X&OVwBZ5d zGH4wqMP6uZj8APxeKBc=TeWQVi)Ij2txbDZc&ZcwLRCLHaN~F8Q6sgtiR{EfW6$~$N*8T7fki4E? zQ+Ec|8_SRT&Q5{2KiEN`G+Zq76}3i$a5SDwPxneaMjti5zc2j7N{s2qM#M;g(mmKj zF_IA$+;z+~Ej{}u{3`{%Xb!-VMp7@s-HOQ!&X)dABjr6x$!adPz!VnK?_0tBxK8*g3MN5D?|D#iqZ9c1bJ$I!%%r9) zFto9;@rNnUT%taOeEZro+80Y^DxE8M>(8aCzV(}owuQ3dTc1+b$EZFN=DE*S@w5QC z=8DM)PUMFUb(sC^fU}4=o)e#r5pD>`CJ(Z8YOr^m9)5loQfksen+fiZir^9?rUfV_ zreM5z^$thqvG2Zx9Ehf2f)?-P2ZZ{N5WAhX%wfr4t2$7v>mQ1Vp~!RcalsC0uS{*V z@$nry6HNzd0PZUVb`f6GdreU>D?7TSF{@kiv45`?+W!oxU zXWtpW?Fe_uajQzmkn*XkgP4j^#jMA4-6YN6;@yOo)J7Ib3isN&%@Qyq_Q1d&4a@}P zx!Cu^FnbiqckUf~;lqY%fA^P`VIr#kR~YsK>OblbC@%XuU)epB^iAj-3u!Tw@Dbk( zR|W1Y{L!mCuZ@*X&3aRxE{f!AfKhW>hvovGwpxHm8H)bzcFVgS67O0Xhg031{zv9c z84$>FW522cBE)8O(zQ7*vx&o&`Q%Z8_&oti%vvty>C4x3+88!e7t&oX@!9l-4yG^v z>*tDs&#{HCovE4e-5`;8I#5lJx{?YcX1ve3C!U%gFDulopx(ks`CyfrAL&q$oxk$; zn~eYW+2uN)K9IA&x8Za5`3RCUP-^tR(hLqTcYW^m=cto4(?V zk&Etl{(i3WU-;~w-r*^H0huD}sr9=QR=&xCE)mW77*8DSCk7a3iPLIu3kvjGV9LF# z+aH?WO=n)xc7n2zgvO#$IhO=e;PRDyhV}op%b4h3d4ciN1WOc~Qy+?cFC@`k2(kTk zHxZ3l<~qD@B`SxrknfIG&nshPp;O&Xva|OJB%i_Jzs%H*q?7{^^FERAdOjLGC3}()?`a!kpPv5p_n4C2X z$ps&)r#$($#c;)8K6#UfxKEtMs%+;pT?9s|r}cKu2&uaz7{9D6tK$osq6zY44aE$- z2tXB8R&yFFELAx&c2d}$N*s?hq^fF>aG)%!wY(PQzoRu(CG_vCYAP75aPN#pBaaBJ z)2(2eGV`r8eL9dGK3!&UdmmrodazDns4Tx$5>3Ku<+c;*Fs#{ubT9|go{SON2~I# z|N016sNl?aKO(ieDBBA%9X3?uZo5)_o^&+ zxhV{lj~yC#mv)aBMv3Z@U6+Fg^|6{6Ub?2MKM^s1ks(yk{Dms5NhO&eS5ln)*@^1dM`N=T^-M-uya`$vY>D;$>y^M4T+_ zevC0lE-r^)jWuwBW+u*hU36Q0t|Ut3N&o>7ElL*`(9U&ZadJ?>|124FV%`@}qBzA* z^z~qiTU)e@>h-^E@fm!W)Az?kx#DKLI;#P>L#5s`*_0}$zgnbi1T)cpd~D}ytK@n% z+3$6%?w=L^2%_n#IpWr-H=UY`%5R1aOcD3nhya+9SeQi8Y@ZEg?kJM%YXTMMn_^pk zG+AU3Xa=KQ$?&ds=6+0nYPU9$EUA~b3Jb<4SM*()5M`_bbIi-NH}M`N5>kBkH^^3d8}uzlFMAJ3i&HMw8$#!!u9j;jTI>F&~ZGY*uZGB6<-a2r#eH zn%s9e=t0}nfRm?t2zwpYQ2nyNLEk&eVyXc)j+R^ zkaxb@b%0dPrac}&2+NRH{F^1*#8M#$S1R7;Ba7aEf;Ki+t&N8lqV}h#;HpPwqbM%* zbTiYwsg60D*6f8RMx&`s=c-|9|5Q5h-ePDd(8{H)NF_EnIrXo~>=t^IVglE*jT0LW z)j<*kGBF!{5)H0W?rG9gFqgb!1ez;VTWOLma8^BrYOWWlz$#T5OjB)BmBCX~-j5WX zCd?UbV(*`*{rHKHNaA?IQc2v!S5XW<*rA}>eY&^>vlcue7?4&I;Ni8Mz_L97S3Xl$y&o=m!q}SP?*F@9jOQvxVEOp_ zs=_ywu3@H+EG{6`EM~XIO;~=um)66>oJ;HJyEuEMf&6FPjFpY=tNt_D+X(?DpI}rY zYKTbd*U5@AG6nO*k9*CKK7V@guxsk0YtJO_`?;?%On}5rsH*R}X5qAZ+I}#fpzzFF zj7w5v@P5Y5e%Il}t{ZH7aAml^+O|Jz=5a5uG%25YYCT;{jHu)P%$Cs}v)G*UK&ZcH zZ9AW-2u83d(@xOHd)xO5^H&1f0jE#!=&Is3li}&8UhJ>~wOnUPhljLs0nr<6R%_mH zB+yUy9|J~to4iDz2KmHXkQ*`S$2-D{V|vQ%Rhh z!lqIbmG_ji2{WDZ>`qJ~=bF|e<94QYm#y(utM==3NWzMp_AO+09iGdMZ9Fb~)!_vE zBr+HeN!nBP9ve}%|EzM-M|0Q(WC&lU)wFh(@eU1nZ@-_oU)KO=SsFComQjWK+4x)0 zKi~Z^fANl2`l^P<>un%yLb<0S^Wo!DyWJil9iHD|NPNk5X$ge)W&X`c4AyyidsS&f z=s|f7BPhCeipqB>mcO)KR>f#>WkjIn1+=*8=AQ*f!Kb;Ds>Snm+Wr@GbA{OH$hyo1CM)1G&@36k%X2mrH!5hGD7GkG7=cV@~XZb>%jsrJ*B|} zJ6-B-w6%{?>OG;G8&jhwjjg|~e$vk#NNco4&wU^q`X}BmHr*7_aMiWk{5?fY>j@nW zur|tLC0vNz_tU89C0|WDv)*)}OGGUwW5ea4RP*UsDE)B3tVdm9>gJH*S?bd>h3kGN z=N8Txy+urg8sI_7uPJ#;jM96bwrI zzIr60T9u{^bdhCc{#d-f#SA)-;p1kmAxYFhm?}zPF?k+F=-_GIlq|60{J}F+D*X8= z2Sh$DZpd63D*+gP-zy*or6{JS$nS(d`6B*QU$$a-P?=)@bm+Ic-jW{>=j;1RXaiv^9l1!v+@T?ZB zYV2h5%W+5I(P~iX+NY}X|Ll1 z(k0=S@%dDV-sTQ^Oe+2&vq#dk)Mg} zF9fyPk4WVJb1{yAeQRu3mNrjxR*vfVl3rC5QlwKeJK`cxkZkP+5jlJ@SsI&8Vx6p# zWD!?qVVf9*GE7^U?f#L}*k-wa%9FI~Im6S~QGtonafb3sTo*J#DcZE)A#E}zM?XI8 zLPv>fy9+4Mk&(ww4pn-#C4ApN#A%!fzsD+$?)4T*obda9gd&e4rdv0Q)H@>~XLpMz${#N2els-4m^rVY3mWfn z!qitv>aQ!Iu68<3`PNq!Q{M|DCTDtQ@#(Dn)($oh{kEs*XrmV{7W`O1qjH1?=X)J6 zwV0tFaG~*e6pB82LK&?&2W`_?`*=g{>Zi=drF-frf>Y;!WdCKj;kSaHG!&>8X@0sNDU`ttXJ(P@iiD{w=LN@u z81tjtQIWt+r6N?xsdx_u-fTt~<2tgSnc0C%I+sKCN1B<FqcnGCT)C%H7FWxE-0_ZKzA$9W3Z@v;f}}SiW_}CLJ7ra z=)v?7cM^B`1O@tH*(}gd)By>`csKdzGYkB5w*S&XoE4hr1bM`B)tmiy4+Tji%~D=UXlpG40p`l{#%q-~W6liTxGD)N-Hc z)qp?xxsV4o$9pELeEe8e32ZsKm#4*Yv=2l`j38h0w&y>%sq3#;p|gdrX`qpXL@%yWMXl4F`{#USxCx)4HOI2i?RR3I!xkU z0v%Xu=pybSX*ZC8oR61=e2qmNqX#j^W~og}3u^=v@`E@O!L%SCO%F3$9WGewTdLfN zl;cLl`G*BL!uHGxZFHMjTU_e0;Yya|(Yf8wEro>s6^Q{5L@UCGqn+wL(rT{6j^*sAVHx<>T3IWU& ze8&QYFg7I6Z}qP!Lo3sc1oWj6B3HY{rK))&S01-XB;G@AKF;jL5*)`VQ+n1*D)kFK z`FvWn zm%(&G)}qg!GCA#0aKvuwgxUSNGFV~8U7b6A5(n+{k<>8;m>t^-)|D;|EV6N9??ov0GSjothrZ*4|RAj z`6)s&3{w4&TX?ulJ9ammMwmV0EaXKQNb(VxqBIp{H>Pks=i^ji`4m?G>fV)^Gt-%V$Ixcupnjt#d?TJLHKz6nXjCm_Wm55#h1TLSF`eY zEA_&4Ti90e#eU$W#x{fz$?Ea9U#g@XNF@QE)-P@mwe9eu;L$b7eoUr+ z@^gyH@wy?i-95kFVu_unMrqrqqAl~p3u&_Z8BL&aX0wr^`(}Zu?sUG<9j|)XlV15Y zXUucfdjnbR(KpA-Ow8n>&Fejxnlq$@Vc(K6`Af~JFQ}59&9oOMm7bX4*A`zQrY@Mw zj7_XI{`|iBv9t(Rp>^!3Q4U_CSu_023oEod#6y0%n>`vDk zBf)B^&@$N0!tt>lF3uDQipmYKi zYnda6(*AbbuCZu}><6uXk^BuGiNnXd?Y5>plGCD@c5GT6cLgDn?iU1=U|@{Ha{Bd|aapSP0@M7E^T z)6*Z9PyKHaG9O{falLOec?{>ml()?PSX>x3y7clFxn#1QhJD)Cx`AR4_% zFrOrdR6+ZBRS@4WWW1XJt#xr@(1X>*y4Yhn>5LsHtB~s1#C#?+CYRd0#urI*fjrYr zy%nE3!-J*9*?%zRz>M1o7*6S%($+qE0rpI)%ik^PXf$!cUn^PbM*2q@=y5z9hVwO5I^WGR(_x!ZK4_?;~!e*;Uh?k;V!&yTL6GU6n zN84I1hg1cW%9pBoXi(cl>}-HF{lz?%dFCESRY1w{UY6H}Xh8Z~Xp**|QStT7Ulpm+ zSs%Hm;CIk{-v~jVkMFzduu(ypvfEKyI-N~ET+Y7ZwkpPmAgaB5qg6O;;?RG+X6n1qJLpt2BHm=d3;@4khaGX5MVREOS?E{Q>qVKOW+$&`K>=wl)VwWh zqjypIRY!@`@*CeD$VlfYRXU>lX{tlcOG~q}oIxuciVafzf?Gmg6&6|xGp5}_cElB*ek#2Uqv}BT z+Le&@z~LGMMJYq6V;aK&CzWJ|?Ku+lv~V?c;)a+;oSb0BbA^Q@MZ6PCiw+_evr5|T z_@a*Ah@9l0vwrV|m{SX_THez9UTUUQj+SOiJ@6@)1*aDQ5D04-Ps zKOEbu@;mgnkcH|})a3Ns?_!;SCk7+P=0#NxHTcx5g>n|2Inc04T$c#di1}w}=>Ubg z$_*FdqjUvrXkiSZLKi#3LA)-GETLKb3I?6yHz{7hCM=z#QdlD1|3-x+s6wc(xNZP0 zJn=Iq+=Be=2DByaostLs`#HJi>^L zU)vCwF{0Pz5kgO_l@vedR~r_F(pKP3o7KD$5IIfU1IOgd)bRCt_UAF@4PwyOo?1Uf zM(8^G|CmV|w|Eg358quP(lSM&B(f#T>xT(p`x=(?8m8P9+#k3R4mc2b&hf!o5%d_c zbiWMbREsL-c(9S;BG@#1s0@FMC{|o3Y}ad|->eT4bkRR?%-&v4W9$6a8_7IS!}=y4 zifSqL=Kmp&Ke$^|nJ?Xb+$VEax2VcDg@_Tnj7;Ojs`P7{{PuPXaU508#|V58BsUHBWO18V_7@PjonLTob0N=r*UNB+?@ zPSQoh{qNg)T^~+|8a@jFlxgb}_n066PQnN#s7;A;z*%TH zQi{16K8FH$6G{KIVbPBIKijav{@2Egr3};H9>@lLK1dNd-krM`2!zGd)H$CmYSWPV z1OpDt^I`Gi!>_ zSxO0n{|2qanv;_=AtAxD5M9D%ZflErWfE8#fDM-pelrhnzw6!7hQI#CcwGbkHvE5I z{wZTIDj2$ygM<_DcciRI5j)!k$vmvfCJVs%<%u! zND?pje~s#ZMz5hOXc+&RZ{C{StZ5LoruTB~3mq9=IH7Tlm zO^(->maxGd#cHwU8e_ngQh@4TWN;&G@*k9GVnlBVMPNC97wN|)gCzZ#OU<-W`05E@2@`L{#2Y_kiMJA9$qy-?FQrFbjN)43K-Q*+4W)p zLms&*xU|;1z`I_u+%2Xl3coZrz4Xr$<~a}g&`n1oqwejk^EPNkkS6blb}B7Kg}*kR z(Q9q(eeItcDiu^JUE<{kllJ2Lu;ZogUfozApyMjb*}An;xC0)BwBUxu(w-xa;%B>k z!BCSj1O;MXtC?0v095+vyqE-IEy*CzeSF8{2n!-d<64Mjz@Sz$j$%gfQ<1$OAHQ3C zQ&?OCRP_A%y+aJl=Z#P?e7icWhC>IZ{zENxV2T~dj$5I%A2<1gO@T!XM&XBP%2)z@ zq@Mf;H9~GQ7L8Vx7It4-A+GJww8!^{RMS6zS8T>pnOX^5!20_`HqEezc%88!tk!l4 zWmH%H%yti|aF_~9)EMBqai+8m@2QOGjFmF&_tpflF*3qyVZ+uDTJYW=g3!~XhtG1T zEAWxVoBF;MF%)V3Ti%|2Yh0r27AATpMCPBg+;|RBB3X*TCh$cC?4U*meg^ajRN=fN z9Ee!*h~+>dIp&;vkkaEgxVgRKIz~+Aew0xCN}`z5QRRdBU*>4d8O4LJoJtC7qrG0$ z9|)jd;7oLNcZm;0b|%)bzZ%-GQoy=C!9*}$6!r#cOHv!Z9`(c~Gv;UghIWj{+A8^N zO$Z+Y_;iiLO?jPvE%G!K_s2STYyWnsrokERnDg|_BL^0ZrJfcr310GPQP;Ug_mGI-!HjuDMg2MSNG0SSuexd4d^nXb^=Hz;mAlYEhV%U0bk_ANN&*Ei z0>~swSGlgwGHdr(u=1PGywlY3N@iN$-v@2Kw`#2J&%p*+9xW`Sqm2JX?AXs9r?OZ% zLML`uLjcC73%w;)WcvadM2^^EswBc{x%r~yjzi}EzDClZf@JI_V;&x~M%`WI77x{L zGM3?jmrA{VVHZ9$t%gkpr$5W0wWb+9K0lkzPuEE#-XU0b2yai^0P6UfYL+6mV+h2` zfU3qJxG#tXc>jsE7Vva0@|imJ?~y5bqyAvj`NS~#z-S!!d$qK9T{ z_4zZJfCA#PQqaxFoPvzI=(fA7hhkBWTnJ^reR-nM@GzGmC69ADqLOgY$0@Q}s%yke zjw&9r!e2h|KG|=XSwBTf)x;btcJXIzPCzRp*h zic5$`DgHS98w$|pEnZP_EiEi4<)zDAUsKbdY%Pi%0&-1Ye2*;xli5W$;{wDTTbX_Exn?tOzR@S zL$r`4GOuuEG2(kL*)Q@Uck~tA#o)Q84rN%vraCiz3>>?;wPfVq_FHLR5oGs#z+`46 zzxjhiE%KL}FSOP}%V?dcqjKrliCbK~~zfDCFStpkXIbo;kO3lsMpHtoAYlJVxSM2XnkEl4{ zxIupuZH-2?7{6!m4C)4eSqe#i`;+tMkh)0YuPloogwE2Wh5>{20|#3k=j0%xgDIfd zXHC0;0+0>U<`L-lAWiscNmFo{Hv>ZWDMRtCRYZ7`br>AoP*uMqVPmTCXlo;no0_ag zS9G$Pd}0IhHHsGGbV3g{n2}SAN;D;{qZ?aH)q(XKCdI~!Ki*Eo5TVW}R6W|U%6_rV z?C&;gOI?kTq!51+R_FaeFb^p&*o>y3W>r{>hOkeQxPyCb6rpQxV#H4s`&i= zYIDqdZ9)Hj@dtXa*$`hbr$n(|dB*|;=oLm?kDFCOLWf|berIbLR7(a&zQ`4HPZ+Edi z5H#Z4vUjk&2Bu&Q>{*v?c5bh-q@db+O@!2W^Y>aKUCnF%LCU@OKE~K?pCS^Xa568C z#Sb5W+_<6WmdpYHF87gPEw1ZXU(}v@n|gqE%c&?M3QuA6<_P>f6XCfPgo6!QD{SF) z2pNv|g7@S^;yNOiw%o5rl1;3-KSD=(>5nb^2Otl_EVZk|UzUMHsnq=?`=$el849GnhG;7LO<7iRfzF&o2qq6?8uq~!GZrV&C1V_|>SyqDe@Dkz zQQ25#QgD779u7HuYQGb7ET**O-5ed=;SN%2wkrg~z$?T%cYSn!RZS))5$dKLr9__*u$G^M&PB=R@kcNUUxG(j@K zX#u0Y9y7?zPE}pvJ6h|1QBdB0j)Akjxpew&PuRq`W{ZK4t7zntMr7tKS!|1u92;XM3G$CZD!-J{l?}0d-2Z&)bgg}VC8YM{z0{U$ zytGj|Ot~baM;2DnXqtT5R32BWe~M4RVUx#ILCLF60gC!lF*FtF$#iy2=SE}7`L@!pwxki_=;rUagf;a2V$ICL;K|>a8-|{w_8y^g z#f6aP#aVN&RSh6exQ;DDUQLpgI23#}8NaWGP}aF2WeV}(z$sV&L;-3TCQ~719|%32 z#CSOk3tBH(EfyrbtWv|y}7vw6y;%2hkMn50L6@% zF>!EfYb%xUUF+YMOF6tjn_@5>9UW65Qd`nL-IC(A6L=&2DZh^Hz8ez9vgP*87}Qo# zKHsH;U!Q=FqT=Qd;{WuA3L;+nlKqRDJ%?5&$HOPVm3g?Er(%=(X17{xy+fSX*A7uS z4?Cx)r!fNvNay|58aN%Z>g(%0R&7G4YCn8n6MTB`_VMw#xoTHA#31o|2`exV5ILKi zYHe+8eptP8y?vTHSZZ0_%O+{{y~tYzE-SV>I$L6cKxO>!?r6RO^&b-TPfnMYR|9~4 zc}9)4y3X;en_Wdp<9xEZckgemuCGO&x;p;E-!}vs-SnSpFAQFVT%H^T>`uW^(d=zM zufX)h-S8rD2sJ}I2$!3v@M7n9L>;UD&K(q7Tm}VoVgycfJ{O<`DBpGY*{^mun&jSv z5tGihHn+cot*pT8N~`Ovzm7YgPn{-j`)3pw_`aLHx$e^t@wFLJ>mzv+coBvLqfy^_ zb)LvXMjQ-*u@`1hFH81laoQj3v&ba5(&xWeSg9}9ZM=lo%`HzQGk~Jg1>ZT^?H0zB z`}&&qotx76QC&Uwp_68ZXUn#t-{M^FZvr z-fP6m0oR*M$+5@GY{Y1vM1@5hy$*rSbnnLZZf6ch&vL5ojBTTy*AwGbX9VBC-?Lp4 z7bnj(@*#n!cJ&4M>u!z`nKV|G1)cAE9%}m(8WML!8=SW{IsK?c&pRhZ9>yMz7f47H z56FChikV_l63{Gh_Fh+$i3X$x`hH>mmbei@n`lxR+E1aol9f4mLr7h#j-l%>l<``~ zo{t6U6?)(aiD1*v+@VrU?(W)(J@l^vhll(!68MKEsHVEp{qPGJg0;o9u*!_yY*Of7 zyLTZv;cTcXpuLqgzr!w+zvt7>Pr`a2Q1@$tbxZp_8nUxg*LS@SWSv0wHs51{$;UyEpI|)26PU& zhA+7Gfa#be5#){0BZCQyMGd^@-$S4g27;saj-S%-jw-b%p!yU19_l@o_5UMP*mRL} zzxp&#TK_mpL-7Vpmy`kKM<1iGJeZ$ILH4Pg(Zt|K1nSK-4{1P=E|jMK430Kya$mUq zk?^XgArV|)NmgK)FAMqR z?L^cxC`oq@AMt_lGXYVEFi^)gQ`D`Ej&``G@ zIoLm0of8uqpYdVrNEj(MIgU)^-MGvC@UwyL!YV>Za z0#IP=k7l&it0+mah}F|mGS7nDWHyi0qHo|6=Fdb)iSmi!)WA$=%WP*8}jor=OJ|1sT+>VbZKox2MhlYQ=@^AM2o_6=K zkb7Yr{GUs6mnA>AUtOOFIM?pRm@1i)z*F537KA5dCoZXg}1>G-ePeiX$S1Lhdi}4b#?CU?i)5o z&wKw&o8uufw%NB4z!~p}PCW3*%jN7?^Yv(uUth$j7KeJ(mQS^I*rG4wH2Uua0+a6| zA@{{-(DKdMg%Gs*Bs50&+S2sIL?vz8my(dvs@Kg94MA9}Sc@Kvn(1R_;9%zmMy86B zIF3fWK0RswYn#4!A=g%(uTJ&}rPcAIu6(Uukc}(pgLYY^?un@5P4LOy_(MqdC6wCi z*|{Hlx{XIe!bL$yNao6jL*h`q7+^Y%_1zU*^*XBTbKD5_`K8Jzd_8Y+criyDI-8_c zqT_#j+0i(djaH_6cRL&#j|Foo3eDl^vRYiAzdU<%626`68!)dC!e&4w#OwLQL#1`| zoa?Y(pSt|}ao}#^_`&ByJ*bbh;naY{N33=xxBaTOv()3hc5`lb^y%Kd%{0j&2w;_r zgS|b^$MZf!3IGoL3_n%BwzILJ3Kcrt4+Trt=0on6kU9~_d|-S zRzf_Q$f|QI{AZWQPi)60%YM%I;qdNcd?zT=ip|f_QWk#48F<%kebw)54OfwCpGTyI zmSKLdW%^oqP`Z1d{-3b`c4;2C*9NgDq@vDc`u!8x_Yl&WlkCA40peS{TKf z*PVZIAbUb*mLEjoZ&5da64SM3_%anCTIjnjGPf6~uWIPs*zVyG@DjAL0%zihdxVtg zAR<)Z16)CFZG|8Gvx|CU?+RpRRe3ml1rOe4Jc7x8@XRislHG5{7!X6r z4OSr&YInWS&AX7wJ3Z=i@X!k=csc736b!g?=56)e=)Wt=CDY#zK`w9QbM#&+GPf~M zn_1rQu-1OQG<}gBU-2=qz#Gp4b4s9sTb=XFC z^fSTi*0p;4CdyJ~a1e04-pp8r4Ii6g6Cv=q10L(3e7nS8O}GKPMq;Yb6o+hg(wO0wz>Qy^sIQWn?6;SZb@xt?Y~X zm!1yi8C;HM2X#MvXc85eno}F<#TydTU)HvPp^HWNo=c{?e@N)R>-f^KrqVti5LK>M z0PjCzq}wzJp9%I|?~f!&9!2g-KTz zZPMk7ojlH^X2Vbs4n955wV{zfX8x%*My$0(M$RsO#L8(nprKF#W@!{e=?yqthnCAI$Y6JO$m z8nC+JEfiZ@>+9Zo1G2(LTA{6>*kjJ9TvAZrp;DLh&N#Gdc6Uto{!dGuGe{=R>H28m zSP!Hpczr!Rn+hKo%i;%YI%0DQ#|ypp2P34^I`s~x`nn|snMWRCqLAL}ZCazFc>q@c zSW8bh;?m9}QX3(smR{v;ymdG~R?!M(bR;V)O!B*{Y zu>}qhVY&WcdBYq#QSh_>54B(|j z8$Yj9)?dHfzArBmsWdoULJ{-U7rE|zrgIQ>Glw4^?C+-skluaghXP!M(}j`_(D;8y zuzp1iGBWTFmO+7BPmd29yw3?c_UI(H?cjGayc1lk{DFx&I@;I1#!gFeuQP^ci$3N8 za45O(7}?)RN9uX3N^@N23K!;`Lw>S^O3!OvY|>T~J#*pOFJ&?L=ReIG5I&mNYekK9Z&dudM-ahCkTj||YWYs_pZ=^j;UTSRWR9tOV^+AE#@Kg@m zqb=)vbOuMQE0xDtPQ!at^{i*aO_U`aNCWSinfdL}i3ha4f-1`qcF#Y84#q{x3B*d~=BX6*L;wFQ;sll7nkKQfYI zs0M$h29w>wj5H|2!)Hn1lXFFB{goyw*BYA_A`5T;j5KSD`)&UKSk3{zy~TL?hTYkI zZR;w;BA~IJ2+-A*cX3H_gp_IZOli+``Tb&J zV=rwo%8Nc0LGb(^s||GT4cJ60Q9!UA$KRdH9-ZDMDYAB73~X<=`<@#-8_7|&YS~=> zRctl0+xg^yrcw5t4jgc`G}fIc$M$sFh&TFtSPaS6(H3ZZG)T4xazB15W|#evyy}fS z`f_n;dpBfi_dN#6@VIrRcFLl@WoFc_&7Ghv{<&K;ucA)-d2a!4H2vX)Cl2msb61)( zyVu6d>a%;j)QH#JtkP=1SZ0L5BSh!aK!nI4tc(OZPm91 zRln4gY~2wd6kXs9SFy9n^F~mz{*4CNEO@MxU3XL__EFJ%MuDf!oyvcbj=~r~H~at&Zy+)!Ys4SG*1_UJIRz z2R1W;bYr4UNJ(SDe%DV`Ev|5&S#^Ve?JGOHmJx$dJG-W3i;$PWIYm+T=Uujd%?yJU zSF5*y=|ZwA0f&<~flLkt4Yz#@M9rdYSG`-E_q9V7nIe*F(^$KTZII%FV{TBx#kI|L zSnFRK!{-8hZOXY>+Pkaan=w7VTwjPV{F*x3k9PkhDBZ@}E;Rr9#UdC@eG&fF(1q5U z>w`L`8d9J8{c>Gq106SzP4eIzYvxm$%Md7ewWZEQddPD73M|N05uZE+>GW{W82Bl3ZV=a#mhcc0y>A}H44}-@0)`6dY`@Sb95VGIW*QDoE;D zd^M);Y4Y4kK>Dyg1k9=b`QZ&nK@{?MlYPHLr_lx(Jh&K;$MC<~P|VczI_PuAcpMlK z6?zLATJ^u1DJXBdSWZk#JZ-sbF3%LKc8iB3yZEG?>{NHMiJo6X?fTzDC9{@xE^OUF zH2lwjrtPI=;B+$uasyFhheuDd?dE`&>xEr|$7==<-&40ioA1d}rf@AlN&U<==dQx3 z8T{DO^w!bMm_hOLyGEA= z`$I%#X4~c81SY0chB>g;Rq}2!)JpSN+O`GHJN9o5ZMQ=fWl5`gT3#2Y0WLd#k~8_; z671T1pZ_Gc-#xo1mX?$}-gXsU-dQL>b)=k%w)(F3DL%R4?3A75jEPt%`FP!#?2cvX z3fw%Aw5Q%^?HvCY0xFN{x>?yR5p;7}_(T3CjM--fh5JJ8E{RSbpH!F>MIL%-5LS}3+^-y!GZ)5EV#P{cL)S$;}U4x zHMj%~5a1PipY!gy-@Et6>p$H^HzjLTtvTkHW6pWE1x{cc(c02CttJD@-AaLa7)Vb3 zcz`w)tF-Un|JABAE_%5Dd$dv_y8eTFeXYAzsnx!EUeV%I-hLLmPgd3Lozl`feJFr-HmvAEIS(fGZkj(*i5{xEI z9MtgzYWa;}U`{lKZYSdoDqz2t=IlXcZ!ao& zye|D~>yz%|Nwg`p>x-orN1LJR_sx(A4X6@9IGBw3Y+qNg!JTF9`>!jOME{K556|C* z=vmF2%aYv1+iP6Z*{^F8+{K$rMlaHa+DKCN){J`#2eSK$Wm^&@hUbvNDdP_|514@k z`ZaQwwji#Q+7@v>74g_Illi$-7>~E~C*s9Y=lWsJ z8<>q~l4$}Tl&lU3`}hhEQcxI?M!jC7;LNdST2My0A-I9q8FAJe&4Nk5{aH31S%&zm zGi5nE?p4o67H4%s@%45UewnzdjUyhCANd%s95W2?k;6evHm6oK7KIMpqyh^1QWyfas|8(p_ zWd11!T^E7EdF)O9_xW!)+f#aGnGMlCZi`wn8dl}QzU)B-lLl(xUKvC40kP0t?Bh)0^@NC2`^VDvt4nh=FvT*S72-d%bu zI*AiaoY;M{Vu|4B7(<(ivVk`G1OJrQ;Ua@mJp%D!UX4UIoEM#;L1R8b*@DO9zFiBq zGRro=l*zZ1*`L}A#s)ye1fQH&79QVKrXrr{UF{?dKB6_wuUaPLScSYH&0p200M!L` zrJxql2v)@kPZ;Nz^7Ko_-*Y;_2vB5%xRdziatOhBO+_LC`M&q*SMBnC?W}7D?I|#? zmBup!S3 zJN!X48Uo)$PmNhWrPtaAd`8s_ekgo#ybP!qYZpQ(EJbPN?ChyUV+N{ASo*mWm*C@} z5S2>ErVR<>*Kw{sq~g|S`5cL)Mg=;SQ}eZ0AweIB0*knUSSFNfKQw-`ez^WT9XX+K z_q^K02c}rPd`=*+2IH9uD&S>0@$oPAi0dGnreCpcM+*&)H`$K@%?FP+U^Mr3uQ=QP z_N(i0ZWv|v%UxVa&tK(tC93J=nyKC@$v()?s(Icx6=CkVNzdnzkhed|BbyKEIC%7o zH~qx|kFV+n@xj8T*_Gh+r8F*n!e=}+Dn+4~=~?y*GYi57i(o;&vIc5F3cWgCqf%fR z5!bNeW(eWll!blP zGOK@(JLVhNXSfh;1ZT?!C$Yyr z=W3!ih>WG;D9g|}JdnLcUIB+jaA#t;>V8`5Rk>=w!{y=5%VeXQ39Xl)dAaR+NyO+- zRrX5wa;k3p!Ng*_%BU5in)m*G$TREya?uU|frbCC!zzYhXj#?cMOFKgZJUZQYI!Pk zvvPN7?6*_Att56k1DXS^@piD;JMh-~UQSKq?&m^ZOL^N(zLUs%=vikXI`|ve;j{fJ zqi=uQTpagw#3=sJ8RpVDJ_LS%?TiOlTy}HcJpAk!^Lu3N+@8&Pzuv~^e*?y%nsgss zE(?v7T}usGU4Ak?-Y!1g_rQ*il8u@#=3YG>HFex4yX|EM=u|oytt}K1WxWsnvn+Oh z)%18Z%*ftx0W(&^J8398`5tg}6L5dj@vwO_JYB^u2=Kd$`n~pk=e=1ttS0RG^B%nn z3z+fu{1s!BEud6{dz;wQaq|beqsDDG8|xos=nvecHUG}y4nP0B%%&BD=Ckw&yPfs` z!nHPZCnGNs%Ach1>6{qXD`?D zUyqTUWc*oI8_Rx1WfhQ;^;p%k;)4yC*xdM@jhG=Ojf?tO?hS2&S@gwYqEHX-^A^_Z zyQ|sNj_Uo+3L~$_5w4viIDoz5VaT93`*CpI!iDVLr-`GV5Ck@PY@cj z4?_#v3Q}i9mP>!xTumAK-yNJhp5NJ(5wWXrohI@bdj}Vc2jDY?$e}D_V{b5sqvQr3 z&()Znrz@#ti(z;C-f3Ft@Z}X%0!;R{2IE2S7tqLk>gc}B;=OUB+i@=pMy)1evfr;C zcC_F29_kJkCw1LJF&P~C?2;etZl)@>X0-xtj^e92pzBe?#o3}w7$C&_el<2N)_y)U zbaCBrbKJCY)BkvfZMEYkxbzoT#0$&dNm|-zAl!2bqMRzj5}k2hC(dWrHRn6){{>S2 zI%=YCb@>(g;?G3`1iAXVugjviq04b<-ARXz(RcW@mG^BezY~i)nmCk=_59wuJ6NF^ zwZFOlIR%UF>Mnjf4tVtV`Qp)KZ$$0E!U%rl5(XN8CyA(FH>Iz$Y17KT0oIZVr|MzZEMn#I%%DnG*Dx(UXq?q=bB{6U zWUg2WC3n%o8iZTEH=M_YAwLXVZZGx>d>)pwcyIcm$X@teOyqtKJOrmBM)(Yc_Zh95 zI(d#wJoeGvWre>CNFH@vb|PR4U;L2rgibc!?K~}>ppLeM?C&yE6wj2Dl#uiGwv)gm z{+*2tswX=be&GP4&!O#qwqJ?;0sDPz8lm+^Q@sDZ9@YW=Y12^#WtrZq0ZtNyf)e~^ zY|-CqjM=_t;NrZ}4BhOE!V~p4oCA+|x)zMh&c;R|7Cp-SFU|3lC8sEH6DX}d!Wysp zM|u0d32woV@ZSMb(B!PV$f!JXscw#$@V9KCc_|o72CB`TB)dN*20In?-X(TFkGwK< z=gkdAUc%aqfQ7!o3tfae;*;AD?ebt!+J>szx=o$u2VyxCN9NN+(hdThsksh<7UZVj zh6Lj)yM0v>6UiS?b0{eXz~`aDWsR)LjxXO!c0>PCGo7S*F_2~P>^zux)x!qP4cSk1cwC1250f0@5Alhhb z)$lb&c>pGNEnG$!+j^`SHI8^{+J<(77BX4#i?bqFjpD|=b9Oj!N?0u~tZ<3Zv6ofZ zreh<^Ovx|dn6YSJEa)6?FZN)(5XKK{8N*BmmU7}S3lQPnL6dGdGwUf21X1~S-Xhv? zIqHQhKWk#?#t}XVJW!B8#ZmK0aLLvJHc+3<2m(3&aUrmN;SE`2lZIMW6k$b_?a6#Z zpmTqvy7L+ZhXG-nIVu3Ud({EgXNt&9Y%1BwAL*!R_H^Z58WkUl6oolYnupZQM?^Hzf>K3mt4>Jgqcdp5^llPnf1s7&hEL2k^F5=a62 z^ujh>H$@)Q3D_-JS-BguxS=$-W*{wU!mLPT%hLbDA^~`dh+vK1q~ZwGL-d|oOfU(F zbRbF&HlE;HWs)5T57)8|vsl38qukPvyg^9D08-X+?j}sva~H*GW0!|VNz$g;CjHPL z`b>Z`ixL_ckKzGutxN_p^4dpM8~GXLWs9YGZ}eJLo2f4mGAHKG*v=%dRXCDlb_m6q zhm{!0VjC4bQ~9Es-(#=CIhPIUCpw=(p4s%G2BHWI$|9_>c{wL1&&f`dWQp=R@%)gmibC2SJ!8Erfq0xz zt@;llM3fB?SkOHW=NOI{{ia*+)!G#a5D&tTOUX9~@eBh1D~dn_=a~}IdMt8aJ1DL4 z1qFXu7?hJc-bKPq$fl;wY;_%T?#GeUnm?Vt{zw5x>>pL1{#cIbp={{;z%^47h131D zu20a8V$+ya;fuG}n4j~y(-WZgy4f+6B|+jF!?N(GPr%KI45B!S&M&--t0k!~4SKCM z3{{fEDvs(B9;M^Orvm-HU;Ru{s8WeLkTT}{F+pVUdw!FavAR6UK;>4_uP@|$?ax?; zOUF2)k!{@4N#0z4ymirP%DVVUlJFNegpY_LnzDf$vb<*>rwN(zLp0xwuBg>!w7xWR zM=dnBB9si8 z)3~KiGsiiYvGA2X2cP)F^U5MF&YV&fK^iiwwRfuw-uT+Md=3+HIU4|uLOMiBTV4<= zzIBJyQ547R@I6JpGGA0nzDbLY+`9}DP{cB4zoHg zf%_0#;76I)bYI6Q;vX)$O+d&>|8*P_h_^{Sbx^X@(Bb$v+W8bqydNr&BSE5kEyagN zX|hSy-7SUuyLY(ZlYB!yX9Xu5ApP7uttfVYVqHRqC~cbegPZsII8oelSw2AA5KDv|&5aOO=K4$}%G!GgS9*M#TTiO#y`Wl% z>6ePMA)#SEL{!@UkD2#gE>Z8j=Lw?4#}@mi zbP^m%IR?Zs?IC_F0qjy-Ksj43&Xv8}cZSunMfiD!t61ffxh8zIA0bgZ6jwE|W*l5? zhia%XmkG=guZ1z^zVBbg+iM0&ER;Ixw`qu`GxdBGL`B?rE33y>*}*dKnR^Y~hm9Aj*2FneiUccG+%v=GtZ$f8ps8w>q-Rp+pwh&T5W;;>Z>wc} z?t=*Y=XwrXf9`mwqRzaW?8~Hxl(*YDMeCoxKbX0dslmH+fe6Z66p1|b%LIud%DeDI z!Sf*w!WP^>nxHe;E#mP{LV=g=eXU#JqS`Q|q?rxJL4vdrO^t(?K_-Ww{nOC+hF=5h1N)Ahxxjsm6{0@f>acVn5?m4CIrGN z5Y~jDnnub*8O??yaH5T|Vl{an#zKxbz;SFO4xXSeZ>{AjMd9BOwvGHMMd|sz>h%y) z^W~DAoYK65C!~48M`0dAAJUal?1jwp&+y<D z*Am%N&j)})bf9_?L}(PKau$93vD4m3eR( z4(0I`Y6dI9-!dTzfq&<3{m&>;*o^B8AR?4-D0sm z%SAGlEoJ@4p(flZDdMKAJJWYwuGyzSDLd)mAg13fO)M7#A4qmrwAbK16>m!J0uPK` zs4UwkrQk@D#WKXfzggxz;?VCle;$^xF(YHlgb*`@okE(B_)wn$8c;v?aD$-C%CqHL z!*a@Ju3kRaN9qKZ8!PHo&xG$+tc1!(c!$$*(FxEm$H?!>;hqqvGfbxfJuYv6rl@rV z2qasy%+b(!?kx|2MaOTwy;f!Lj+M#cajXrnH|G1;?;YFkYfl;)%=Uw5xHC<31g@Mgj2S+$n%Sl7!9Nag@NFyK-V#M*gmf z2uCi2YuZF=dTF0Z4F(kAz^-m7P%P=cL{X~fO+Lk;zi|U4SBE4X?E*vHy?R@4Z>DFC zOvo?)FkuP9n`v<;;pTUZOD2TI7I+!K&B=t%7St@Z~w&;*d2TfQ<{NqP7;S; zx(E~dp&~MiErDt7g^1dOp1GOn+fpI99y+t$3&`|^@a0$7(P5^Fv95b^L06JVo!Po+ z(qt{SDW&(D^Sf_w<1xMJp0w8xp)-z4;(5&ybX%Ki$p*h=($t@*JbrpS2x>dU!vg5Y z&N^gC=14padQJKa878@uzqKk_c>Lqg&{TD0umN!&-@b|Ph6hG9`Fl*7FA2aeUxmIwFZ%B0ix~qA;niyolBfC%Ym|LV)sjB>28^1i9|3BCYWR2gk>Vh1lM|>8p1xtW8IORc0`V^ENm<;aB4ob zn`Ur8zr>_nwrELQf#6Jtxa63N$2ukT0V%g+k;dsOBR_2)KVag2$mgEg6oRTcaG0tv zHv?G_loTQ9;}=C*wtS`&Y|&L@mbQbgd6^#}Ku8JlovU>yxd})L`%Vv9WKIiNdHX)k z74Tkz&t825VS9fTg(NEf0j~woT_zttt?A8PlUbXKrzy zK4S%)YET%y=%uYEdZ=x+IyMFi?5!%kbW(*?6qC~I_)&a@S^!!+Ylf)sT{4H**q_1J z3YJ5+3g9i&5KMW@Yr8tcQR9SN_rV3Ga=%hrmBr3Sa5h_2seDP5b&}c+^Ke;?rI9?Q zCcy1{_=$k?p}&#BJE3>-^5{0|gJr1C`?e6+@^QvQaN`&o6Fln_?PzJsanW5BpEifE zu>|NGp9*!ebt$81>V_(v4V-t+Ii<5Wk>@VO^cp>2eZ?qrEd5TKg%hcYyfEsWEdrhk zu8f%>0Rpu>jI98Ly0YwS0w9n$)B>o4KU=^OQjzvQF=1WdurVbYt!@>LHu8Gq-;sJB z__mE=d1)e+POuILOTHNxE}9nJ0K1Z|$iAY+HO)yEJo$h+xVsS|P^^wgxUV7cZCqQ+ zdcf0v9Ss>FN z}OG?rfj6ZF@tcgQ8_E;s<#E$MERI&@-{=UPHT zK_^%?lrAE$;3FH&Tg>`+nM-<3Gg%Ax6Lnk9w&a`w7i|wpBP{DjNBf+7TQ`8`CJf}a zOq5Qy=GyxyUxwf;BxjVq%LXwrcSLVFM3>KoiGPPiJ{{^)j&Dk?z<$mqCk*NJwrXl; zP)=vVRpk+hVwioI#*U7(vlLS|mHaiWQZf@U$dx_6oXo1KR*K{D&_3*L_SBQ4{;3OMyo84c|V?06Gyt-+bcl9UA%AEHzQZ^RQ9R@#M} z1RBRTB;kNvnlF26Cgnl%SV1*Qze%)XV@z2F+e*g;F^epwqg%C|TKzqXC|;421@_mm zu=%E9c)cJ2HK=1*?W>x`lN2UlMI??^f8t>(gm)Qt2gVi$tKww+N{J}#e_ebL7;E3o zqcDEWlI+_$k5W-H%GhuHbfMn42b(J62^urjth6D=Jb#D%B(OudeSFGD4e1J_^eZ6v z^E9a!V?tJXlO@zd=XHun&v|VVW*p2T_!q;CN=fGE5Eglr-dF7=*y80CyXMS^s@TRg z>X!|@s56hVSBa;~)VXNj#Fw~82<-Y_L5j`_f(oYfujdcFm{|&Ed8qMCJApnurxSquiM*ivH17*t&wG zN5h0eIad3@Q3z#n0D(%FeaK5t_gQ>wM|ICzU6Ecrki#L`etq<^E$c%f(H$a?VN0M^ z+esTT$doenk#h<`skRe=ie?agv61w@$5}a~_=dRSBXM4Z(=Ru&x7b8w(+2^#&=E-4 zPj01M5Eg>WC(UGDxG?8qns_Um>-aH=0rYs_<3o}@u`sg$9PkQb;D;>$^Q^-V zp7B1|PR+UAAC6{pH&FJ_9r!x$L*}x4>DZFVSL(T<^qtD2MFZ>b2gwH#(4k&A8sSRh zjFuV;whj4qZ&x&2EaH<`Th|wMUpe1>PiJt_qPgt0t|4kS!{hh*Ao-ZSibg3PW0>Xw~)hXbXFjN84yi6QHe^+RT!#0#g^Hfze=#fCjMlsK^F`A-nt~ z@+&NoKoXyq@hSK@G@~JocWu5}bB%{@bm)TMK$7Hm(LZ0yj(4$8{+&%?-vR5DcD96$ z@@>eyc}vHRJJSK@X*^8?=x6sG#G23gYr0MGZ|+8$97HarG^O2WtR0yO%)E9YTxBXA zBNZBL{@fq`4Bn1az5OV=yFw(_eL4^EXX%iYLuklo;|Qb<}I{z)mR954If1lhmVEax+qqnyZS zz+i!%0oio4a5qLI2jYppaMMSt;A1=|qpBo_y7C>1D))Hxy8Ipo4YAq2>!4*g_A8xIjTySK;y701vkj1eq z^|rJIMKH6aqRJ+{Yr@Dj7dIE@w=axkyogoOC^H=GM)}5JSb&*U)H}iZ&Y~E}61mal zQsm;-4YsdUFj)rL}==a>8|Y#wTt0#^(r3CBvJ&kKWOD2CQXa%4nzb5Z+h zcgcObz$y8S6SF|UWr!zxc$+(FQ|Lt5gJRZO`2x4Rk>k^Z>a3DVGv)xq0pELL0IYty3)i4W>679!KTKRafL7EAdI@P&c~N8|~tEbw@4Z0hJYY1$(m z#+j!|)K_gd((xVf0&V(FRLG>l39MYsX}E0X%GE)?`2`x<8WJutZhsK`db`+>Y`*`> z%qJ=!>*T$TlP^zXJ_6;eRJ^^?m!hgfc`bzxr~J4@WFuRr)f1Pz2P?JtHMFR9g<<;p ztcFhb#0Otoaewj-ZLme?x^H&xoxWnArx;Qca4uzEY->#9mm_q~tl-lbXr@@I${m9n}{(Bx;3A~x0fd(fL$~fcshfD78e%< zjr6|xu@Abkn2RmA|6MlGiFw&Pai=WJ)K_|l8Ay`0$9*pS-xygFiC%~X7GdYtT5Vwr z5fjc+Z9Qgm$HRS7xpP|_aRXJ+-Z_H5mLxER;~Kx-4z?CTU(el~G~93rij?fRo7<|t z=fCclPn)C0-jvh>6p4|Xu^6O^H`z#1IAey(G5Uq=FaULl{`OhNN!vBvvCoS>wJK${ z);KS~nC55FoeJ@+xtG_1~u`WRvAvCtFT$y=Dlo8@EE6no&;U1tRGyvcEhf`Y7@ zY{R7qg}t5u~^%NiQa9mm$4-iys{_%lgb z_Q1`wP6RgRMoO(--=N(sjj zryY6EYW^Nt)^M!2!mw9#VCi7&!5aPhRt_xMEE{wedc|+Ll7+<(!fNlfrY9znSK6pp z9kZVVJFWNs>B-b{YaJi;V&@`$!%}!bTAx8(E@-&CxUR!G7Q$R1HRk;WR<78zieYDe zUcclMDuOGMIVp&j5yqo%Ac+Gk&gMt*y5M0(NbXPV2$y1{XOY8lN@=xzXGAogfA%?Z zWGz)RE{=1Jwbg^XMgU$hb5%vq5h)-<+KaslFCWt*Lm%^A3QRc1f=LrjA1?bCCEyyI z8j)pBeC5o1+iZ8s$$Z6W@|(dMbxt>esFSIqFo`c2@~&s4(A%G zb&;S~Q^Bd;edDYy4c|x@W+=b(aa|m}jn;Bc5ieZ52v287Ub9gz@BW- zGL6nlyK!QsdE$~RU|K0n2=^JFa<}TuuRYAR~0obZ_eWu-+*mL zplas$+Ha&PQS8ncTDEaxSL=WPe}3*Wz()}lrhm>x;sraGnNjNyFrO)LNS98CCm3w< zr;45NYdH;nZ6#nTbZ<>_62`_x;M&n3-O#>z;^OW2KsCRyzucq5)PoUuoIpE?CtR!> z5w5=LbN1yp?5Jn&=auVGCfAIB^H`J=%m#T3F47%Ol|3*oGR|fdi`pd09q9ZMh#B85 z%pnCiw0~)Iva(?+zEC@ZRbKk?2q`FtOC?EFXzZg$wX#p$t1qRE{w-$0$i&kH3O#XN zYS)Uzrg$kdWz**OL*Hp1C@bqEn$ARCH%WOvB(EOd7witab|6n4g6D@oJ|q9Z%6c(T z?bx)UuCfj5A}}-Xpb9d`aO%3Wd5ZxP)SbF^a-qZBE?Qi&t~l^}eab4<$T z0cpLwa(xN~dY+v-)`=^L#D#@=qtr0q5ob>Jk0?A)d zhX#|;uxVEqX&l+S6d;P3ir@no{r32`(cJFJhF;7xi`$J$^NMwdy@<{_|CTJDAgM2^ql(x-bOWlW0x(C|B^MW zE{mz+hN{bDoaclTusHHu;*#8%Q985dC0Rf|Zbw{6Z~K2)WQuu>kT0|~7xu!xd0h9` z?d1U&yy*f3PSNktsolie?v7hyL~nPW1XtlzEI9E|9vu^ zpSYyNaF};ZZyGK9pTiksh-mY1L{oc-QPT_(g?339lxVLMsNPWu@4aTl{@WXF=YS

3Va(s&}u{`jecjO1H4-{S`e20J0I> zZH>`BoRK{;Dr4tFE8cOX5(XxmiVRf^Qd*uvz$Q7lIw%n;%PY)?&WCoIKS!p7`7Co##d$Aw_UJ8h#70+$A_W#YP z{_^>DSfO5Fd*Hzl?ekDDd1X`rlBr|)9^5k;xPMV+Po0cOHb}vmL7ICk^u3wZ%n*D+ zXcR1E6Lrz=gV~b=M;)p|Mra9g&5KP7HB_VTV!WV9&oDao8jlse5Xz8#$vqy~k{R>X z0HO4V9!P)x2MHaq^8TSa>Y3PJ1S?hP5hgI#evA=wP7^`T-(&;!*kT>7`%>CMK@mu? zuMcBl`^sZ7j>-alS-Y6I6gHT1Ugu*z@RLhjWze8^=v?%|V z!gIJ|p?t663(6)~e6c`zA`mhdHGv?MEDrM7xpF`_Bk`MqpH^!~=-0lh`9Ol}ek4M* zz8>LkN;t`H7raq`#Zo>X-u!BppSaqG1C&{yi3^LJj^s3jMo|1sDk?+fl||qI9Li_P z)9ze8po`Fu)cCxVkC}!+m~;O6J?m7>EET*0D$aPaIXT9XW-i`76b(x;M7vJX!T2(+SX2y+1*FDe^s|O9nUug%0 zf?eG}1yTDTO#;rV6`vQVp0GR7`F6*s?1G>m7-S-#D-<71RjA2;+<68URKY?Q1Wn>i zNoR1sCp?^1OBJKimd^Uu`^k4MvuaB+SxCg)rk_ZzuX&)kC%M=@5qK$y)v00LR1D*ul1C(THXlk#*}1=a!?4$oTM>BYW** z5f<&gX~Ib(VYlU7#|rh#X>c_(b~l;1`r>DS0-_U_#P{h*i^V*VSwpTKI4*8#tX z0)9w@a%hpiETdla{a#HI-h#(0nmGUyzOEuVHf1IXosls+i<9H6dmd(Dii8LVSz{ephvUW^J*`VO2_9lg%0d!J}Si2G)y{Jm6CJ@-k&Q|w#M3a%5=d!gpt4}HkH6j6Se z9$CAJhD!Qhld5(VmtW8U;xc|)>q(|34vkawZtoy`m8qg(rc}WipRuA#QR*L|oyY`) zE=W>d()?Gi|aezk0prC4Om5Z1AG3e4yr2f56qoUfDhH5YX& zK?gLr?(x^eWhQ>F$1dWZY2$9!0oGxhR{}f2^68mhr?0%}1a_9P>%T1UZb$q&MiKGsHnp0AR$8nI?mGai8W3Gzl@DXu8zcI85Q8-jKD&77-}J4Cgli zv_Hm04*`M4&r#oH!{gwFW;tHtH9;WhIjr!i07}eRuU1{32^H+qAPRQ;m#F(()Glyd z-aLm@q#w=g`M*_u(K-)GSdsq@v+i2E7Ab9}{d8|!%=D0>t(7R_{UFH9@QF7D9lt>l zyKuH}{uIfw+QnmFlRU2F)brb?QWCudn`MG>rMeZk6kZiUBMkk9ST>lg-5B&gQX}9l zX-!+emfeu$0VxHC9+y=is7`EB~oNSzil)8tL*BPV5EE8MPBJUr&$ONl$9xv zr~p^1;u&mKvEXZgxL3x=#!=rnDm9pM@c{|4x(y$Ai~FLrD+m_#bKT8@T`2lrp1m=` za(&-ZT17bj&(cip4o#99jKCpzoL7|+(4Jzi5^MfW-fzreshTypZmz|q zwiU@W+F?}%3neokdZWdfmyZ9%eQhL`777E60v*C>fw_(W0OQd&Pc;X$i&B3axoKaH zSsfcuqAa)ngSx0W8rcFFuOHy|HHsnFf~ruaN&x4r!k`_^g2cXI$^UYoPG;e!^uo&5b*CnbyIPbYs(j_6Ni*s8f1Gu=C zn`_WYD-)537i(YBY(4o^e{B_+SeATtm=#TB$Bb7L02JqaaNhY3d@QB)UsIVh$72}f zA5=D9kW@l>V@3f0*znfFgFMz|y3d!5TBa(}`~?~2<~(3Tv!7jJ;KS8hDv#L$wN&i9 zc8^SBX+G(tZ{P0%8IEU~=ix8@a_bKYFB??#EM+CYCUWRVPxxmQL4k|-ykrNH6red* z#;Dn%sa*DH;=*AOPE=y=M%&=T2gJ=a=+$L{Tw^_^R(&ALLEGg!^PHoNSTsfcdh&@D zZAh^{KMeYg^-NYr{Ze^*G^=O#RNe(!j zh?HBY*rl0`FaC7@ccLLvE^0&_GZ7>!*8=CEbuyZ@?%5= z{r_~DD?K@B^&NPpphxQec(?jg-kZ@8Gvo){S+Ust9(KU*9`GA1MJ#_DaKKQGUV!3v z1AYBlu&nn;IYpHCsG)*jRtUMjwr>75c(bCc6)4|d-K0YP1j7L8|-zRe~Fa5lmOt(M#BQ5P= zVih)eW`1Syrh&VmYdZd@^Ee~PT%i1 zzCJ!a{{9cg%R+P}0DmRa7R~m!-+V(%Cintsf&w? z`Pl&=;AMju$^)`|Cjh=sy40rL5$wEmZyd$%6413=5$3$B)UE-!&sg(BlB9GpAH&iV z*RgNP5sleS!orQHJHoEB3)n?NcwGWJ8U3ZZe|$ark7xa_T81Kh3Hsmw035<_P6x#r%#?U`#ikI&Mcez<$=Of) zX$vLnnBfeDJP5D#7TFDVdIz@@=3gO8c`RZ*FIbqFy|%J7Okw@gv8Yybup>TWxkxtD z@VTu0wN^&x!Ski-zUUhUq?j4i-Z+(Kl<9fDdLs;bU)|CSIv9SdPb}F3XCjd3;AR%_ zoE87BnJ;Y&f6yiSbLxPr>QV7poSML^yO(K|4#Nnm zMJsiFDenHxt%%CQ-Cn)neWnZcne>setrcHVe1z~K>fF08GbumOPkY3rGlioQOm18R zUW;~C1PEqU4%e19q;n7h!1yw{0mVC#WqE0()hYkSMQD+GxpLlMl(h1|uG;m0bk@f) zHx|kSMSSK$l+Z{%wH^1?muvKWkgO2T1~IpHjt8;^0r&MIja)Er!0Tq_n1?0Pe7JCw zIBQmZ+@*$(7F8bkqkwJVN%~U}KU2b0dJC3Gfmbbg8l1gdlo~#eKDIj4e_ne}^6{Gg z=Cz&y+kO$O1)3Jl^f{t*%tD;|Tctz4A&h@4{UQ;wb*8L4mlqRX(n3-5i7*c^2P{CCw!v znWfg0ZFMbn%JKAOv$LE-M_*R+?~hmvSEr)iOs3j36-N+%+m=OBg|~}L$Fmeh`ijSA zRAKK&l>b1qW-LZopx8e;%2_Q4W@6M`ch8_&#-eoy9#_+u$?HfwOCA#WUraONbgEoF98ecfSp-=KGlka2^x}>^usK{8nE_bwS(oL zh5f~e2Nb+5aQ-Aa7t+w^LOfo$60L6QZCcXgap(jMG#xd#pgJwDD}QgUbQl^sME96Y zd?s;=$UsPgXmsbjn~bSpGm4Bflj8)EiMqB1&I0gGV$pG<^Hk_s?%!RAL}{ z7|hjO?Fa~fLZR;N?&K7GU`T~3?{XhUQ^bXFhWhv6Ax1v}5RGMaxUmsbv)O!x!qo); zY6G?nE#JTQ@&lc*;Ja|r0SOKmC#&O1tfH&_aag0Ae~TcphyWlbd=CT?;RTi2t|4a zg@iytk;ENdw#E_*=8Lb3~l$6PYqBtSWp$$ z@z8_^oL*`8<``Wd_Et+%OVdoy<<*^ZQ(a}2%F8O3sUI|h{2nq3^H)7ZEVvqftbX8g zF|IoHl7I4B>>Tr&(5I?Rw#iOa)z0Icrhc{+wdBW>6P=+7*{K`0LC0*YnmEL~j};Eg zLF81raQpRj%Ac?Od*vsF$2~R&dzf${-_*H|b_D9S=P_92MUV>=<6y671{2y9$ z87HFJy?iVe2E1}oj~m#@ud0dlN1QDW|EqP{(Ej!gk1|tsVQ8qFq$_52-+s|gjq8u3 zRa>Y4yptlq@E9-L&L45&SM{GV6=(^VDNgNA_mdxp4NQ(Dz__N+Tx%ZTzY<)|KtaKf zg~+_hWEBO8u?Uqv!|h5=1PglCeJ@pES`_x|d@h5GQ%kG#a zdZw{c{NeeNfU)aeA~m>3=j|_h7KXZ;0Hfm!+NDUEPBJ5e>Ou61MsB`&yP9B{xosS3 zr`zqMVJ1kN)1;iU_4lmVVK!Amu*>1!jRH4!5Y9M79U4;Y}(Jqk6RAV|s(zlF;Mv;XCr9Jqe)*uoPogT&}&Ij7GTi6lc>pn(&^C95#+2Ks60%quqm7V^NP}HXY_B=F-8C zDzN4w|Ni}x$Y=SFC%;f{2W97-XrzMX`g;1{TXy)(oEFW2uE{OzwDEK!NzYo_jI3;r z!>R`5YGs`Gkk3BaM>h?~a- zjz^*zaZ1LA)(o&GhOUbXc|%(jgx5!@l zb|O+SGxim3zC03iJl)=lc0c*?CX*L-$QWt9>m>i~3;lg^b+t?}z-%WI63fyG2)xVM z3`NSb$7Gc!zfXf*O0~3y_IAac&%SbsCAjP#G<_jm4}#ZIX2k$kwsFvM&|G9D6H6DC_wcRQn=Ci%hu*yFvWw2~5pB-mG&5fO4MENF`^j*c}&NpUOGRk-s z&#hfSS2EuiBHQ+x`W_x6dcQ$oyJV0Y&_^294!1M<(5?ID8yV|?lJ06HmC**ALt6oZ zGpGVF&z<$C?Sej69i&$SOiKK>9;qH6+d*8R)qyIq!ck1xx;{5XJfS1nYA1xy?YWc! zGFuCLwxBoAztwWtMhmUwrguC&e=Di|U=JKMFJ(a(=`-CCJ;{SG%4-jhq>gbp=a;#w zGGn;Q>p;glHhs6Y+8DFo^P!;Tr+s$yi6Fea03PLLm>_FGze6Q6dMK7`Fef|&2&*-2 z2McC}-j!~SA*RGB*L$gkxeCbx->@20C-p4Opk3~RBDZ*$tU4Y`+1bY&uIqA|-{dqjBqpeXHA&sW`4@&*ylG00K)cnj#g%x?^L%=u2FSB8y z>6L-mowDSF&J&zf4$jsZgt!5Bs=xx9X!}m`uwPeLe7`T)!HKw)_#W12h*vrRBPlSKRG(= z;1sTL+)QTGbob+d$4a^=e@3`%c6uyNldOznbFSmMTUzPAc0N`@Yu}y?BRoHV?*7(r z+gZwv?Xid6C8GZtb$)ctaOhq-glKc*|25@gZP{M=bb70DI7Yi);4R6d_k{4It@-5B zvS$G@=`duE`gjKnB@4M5%9wD#0zz8nMr@X_1$zp&6vt)htv-z;4Qm_>9K#rCZB3_@m@5p1aum+?ocWHO8FiU?OHQaHaeeh#tzoW+m#- z%Ya@i5?LAVlYdiy`BwBuCcVro#Rv@xiUZ9x+Zo*=%;*52Gr^9ks?LHS(x8Kqv&)`u z|4nzRI2IBYj7__H@-pMx_kR!Xt~5zA*3=_`_JdFzx{!ji=rRyK`wfo*T2W855T!9V zNLz#@x}QgtY+6=@gJj2%Ktn01+VrO5&P-mjhv=hP8?#hUS?&NrAzALk(>VVrL)6on z{$(FRF+TLjPs~CS5ITJe>s|TOM#bVNBbZt9u@>1{^HFVnNdS4bsunZ2&Ri7cYlRU} z6c+C6z3+Hf_IS(^upxt~lyh}0@Kfp!hcCSjb&gR?or9x_A|k5 zhW&G?ae=~Xr-JCQMl+L^$^V|kv7_KUKx6Hh@A{0_+M>#x zsROD>8S9ouZI2U8#a<; zo#KSigXL?nPTt@2)a4{N7k)TQF0hP*7?y#m4{lg{AG+XzJ97@=Iztt&b!$`UVdK(XU6Hr4;Bhbemf08k3x=d z(8Y_V524XbsZP`L%)E?u!PN0L$R^dDZ;?&R@8gk#>VI7Z>D{;pDRHPL^fEX_E?@Xh9hXt3DON!1WScGWdy;AzW_t%jRI zPdD>oqW}UNv74~J&uSH@AEz zSe?_}tD0;YGiT;f`|z>KNvG8Jnaon8J9w1g1c#5KKH!@0&8+7A^_{3~LzlXy!dx#U zH(tk#y2In>eEZ}mm16$D0=7GG=~Z0fCuVakN9+Y;=TVp1{VIo)q0EXMq#VNH8`0hC<->hz1N_5$e7dRzfF;G&iqpAujv{HQaD; zX1@)a#32jbUT%z5VfKnomqh&#Q3n^O1(7Xs+ny=C%1`(2$Eu(SXzS6{E0{2pnfqDZmxd- z`6D-0&>T847<}C)VaU8s$K-|}!d#;^gJ}16R`hT%E4lYUbjZ0y^KQryv_Dq=T-V>W zu9U&ULqdy}sATyg0PJ8yULr&MU-%XW!XLc=1~*+qmMN8q*Dr{5a?0FdfAhDv5`=eH z8Ag5$E> z90ayumSz-xm%9qUcrVsCiCofX3eq3qDeF{qq_*$jehi%YknFe`|Ics16}+i|a-8IH zMK6w6R4o$3-6{lp!b;nh?D*up5!qVZ0Boqi1MDCaZwLN6e_MlCo7l!94%{Pf+N={y9IqAiPiQa8w0xC&7V(aV>hoF#JgQJp;?;D z$WX{zl9A$dC)UOoW!FEF@eNd+!y?%T9|-s$Emvz9;dh8kn#KnG7lydQl#sR0GO7HE zHVjW6H^q2CtS>M5=;^r(vE`D6Dpn=?165;QD|SUrYM>G%%HG5Wdsu3Qtz|RD-SB&= zVFL+db@XCAzuqe-pG9$=jZB~F&B@;q8S`6M30gUx8u#r!;zsC#H46m>n`!iQZ~c ziC5~*u{b40s5QG);rHHITudLo?K0^Aj6kc4E->gK(KmHW(*2M0)MfI-7#Lt_OZZEg zg~&b!T7sytzIy)FJ1KCpgk|`%F9q`I z$b>nmlqqB;y%|`;VESA(&ygl(8KH&w^MT^L_T+%$y`l26HDlVfQ@cj(jqBBZifZjs zyB8NtRr{XoLEScC5tGj0|EPGi zMW`$CStl*Y0}5(31nkZ1zb{j)jgf|n)c1d2$8xc1_G_cw^N_zPib0$e2 zS_r;?Q!JI7{yBW%bofwa{_(DOt@W%1QPVK`= z++_*(;(}6q-wN(pH=~!E#fFVeK!1wV?#6c8ExSh%jt>&WEr+CbAf6aQA4Vph-cUkI zGLzuFq!=GP>sf$gZc(Tc^6etuKUHqfk`T9IG$LfSMEE8P?wR+u0J4qjQhjk}pineZ z(ie!%Ao7{uBte=~6UJ|Y@R^q*m#>;@`nY$H_Svc~-&}*7+tJj}Oy>YE~ z?MX(}*_v(X7J!^+T*ZUXls&!a9QwQ&dZ!>?Mq5t~JeA}jXBQV(fc{d>^%`b(Z40G{A zV_A|3{p%k)Er;ZnO;9q|-sdk*M>kwcdk|KS^a(!3@Jmt$9ZvDQ7o3=N`|zVLroD`i zF}_cT-(JZWi0ekvAx**{Muem0>Q}T#xjC8xpoli~+C1o#; z!Ak)H$z3%*SMlUfXB(-5D(66z05@0)U9Xuz%KQwg2;Y`T`RWDeNU7m;(v`*zfxBt? zOS?R*w1uxVw<7_H3Br(!WHi zp7n-R2N)F%AVw3b=9d-CN<(pPX~z$=x6?x7kEa(s55r^39$8fl6sTgpUUklzD|#1v znS$X)2ef^l8P&z8>JEHnWuKYy(YtBG@mXNC*(S0jzDDj#-sne-as6XN#y4#fcSFZc zm*7;}RYI63)Cg6A%xEqr6zBNNGLE+x0N!BnXvW1E7RC&z{#FGZk{b^5LnjX0957L) z>DW0C6B1*50)~lJW&mBRB612-jWl(`P`((h{DQ`or%#^AKx@)#@F6jD=Q?=dU<-f# zzVwUr%!G^d)4dv^cn00(=}K0yv(Oq+$T!du3LgBn4YqP z?{&fz)%JEoWPbD}?Ty)rG0FtxxlHYNG#>bR&=F0Gh|(PO)=QWNQ@zL^12N32w`FJs zx)a6&rg)%I&8D7PZ8<#%!!rFz@@N$ITZ!BrQ{xC6zm-{rDE9rs)+(5hcIf`p%)H5$ zt!pcH0E9mH*k(%~{L(va2{@TMDbIpM-nAHEYMnhp>A8}Nr?PAZR><2OeG|YII%85y4FP2SJ>_C z!BqCe$~NCiyJD*SDELtXFg{)@kBf0HwNDF;@mRT2H43)q<6cw^MZHX{JK&1BPdS@D zo)dvf@;f#g$3lb|jD=t}6S&IPbbC zx1k`bT6g;EWbG=t^71a6Z_9Jzb4(nn#0==N^R9h1 zu`9m7YdOEjt^6O^3^E(*u#<#EEzU*`5Zxmje!C`RnM>4pdh1C6!R0W9d6ZUkUvaAKZYofXRKge25$nO$4oZl9d? zSZNmtnzjQ6)kC%@c#NI13C}>Z<_UU%;8*U~V0u-cG5KlUJd>-BNynfuJPu6Id^bC; z*aab(L*@BR2B_%{yXmnWmD0}=+Rm}{F%!+y0sPiD3md>%NKkZAn1JIYE69&crhdoK zbG;&9X~CXJ&tQP7i~$i>t!m+OC9KK&V}=U$l$R)t^T&+B$*7UJGIA)%{|jqgpYiBO zp%JejwCqVgQ*>QLsN=;tv(Z7?KylJ5p(I_(`aQq;I5TT&|JfuC#8>3s`mi39ENoFI zpo2}!jk?KK3><)3Mu7U0CG-S;cDuYpx)bf=P*dZdc~&-p-J^tx$16!Qfh!@8C3vk~ zfP@-#O7MQ&UVt4RdZFEp8f0hxDd%2-eg1@S)sPp!eL!IALXrlBHk9#680n~j+)b~- zhg1QHh2Ug~L)(}hc|?G@830L{x)lKEXxzVjC+COOq&U#YDj5{8pc!7JS7n0LV|9ID zw>8{^9h_jiSd%uYOmbx10TqAo|K|7jN1}S7~_hzs1T??n2yGve3GUNj5FGc!HykLWOQpZ?S8q0X3!O2s71ps6fPeA(xF(r%6$mo!R_ zs62Ti_GJZR#%NdI4x&!$x?sl#qR8>_B|ZR{o$p6X9tj?Co^{&wre7fZH7#S=LrVAe z0FQ1|n9|<^jO+hKP|(+c`)LE{^!jY$L_bUB-#}*7dx;W+(Jxa-DHUp&s=Y|sL{sOB1@0jc_!lL?rjN<;b_)niI|0_f&sR{_)|8f2g z3Z?{jAO0Ee#BD8Y?NZsl*p2x1e~;>Z!@iGWv8cH@>%W7|&6l23Tv`HO)kW+axSNsPx!?)&KsozPgMUp{7YgyRoJ z48v5$OB>(l@<~9+n{Z>7Uh#_gReSC#b>8g0oIv1?)qJv;Fk?79>lgeq^YLQ!Oml9` z%18sq128-hl_3p}*liSQqsZaLuekg5C1;ab@gvpKG_V!*w;}$c*t*tlw6nh*Z z?j}hLwBCj2uw^AG<4?}72k|-9$x4CvP0C?i0OwG!oIw zx^Pj>vme^e4bKw$wX;+h4+N(2jwLE9M5ybdwqQ8V8?4Gk?o7a4UVGsbsm+4yes~X} ztJ>b%E%gz$-|fU*z8q@4cAGh1mhen5VyQJLl`(%en+;C^CD2*1>F@ozpZ`Hco8Z}NbHXCZud4M*EFo~#fIw7NniB%%P!3@1VvtDJ+E}g|%sYG#jR;;ppViZWQGAgNm_i1Iay$KhDP#=LPHZKfxMFd`L}8x}mz`Rh+B4vc2V@93FEexo0Tie!B$@k z4k^KlX2z2F0_rV#)}KfM!T{}_QU9`BMi<`8#Gmm9vn378;M{uBB#=B^hbAu$$mW8G4w%>0Pk%}eWU7K=)9}*>j@=;fgvx6JxOOn zF>3j)aUcwf9uF15lSJQ_ITuc~ImETv8tc2~%? zWyTr&s5;H}u(4V;F`?R?Pcxs<;u5kOksLX6Y0$Ik(cU2cJKgBTTj@zisW}D`D0y$i z;st2xv(NYSp+OVe&{EBp%>!A?VB?y8Tpj;2`4y(%oe>$u;GRLL>Y*CVYG9uZyk`-^ z&1~cY{<$T56lP4Ck)qdDpn@O<@66e<)@*?70b=<5> zOF=s3pA*a3wQl4`1yhJ-oE;PjvQP#@KP@K+!FIADWwDi6&V}Kiv|cn4$A)p*NMWf{ z@f_HpCLC%}GN33IrgGuAjK;h5lu^`;Fd7MG@ecycmE{7k%fuY1YKlYIucx?1W;XKN zUnnh;o+O_}v&mPo!`f>m@jEc1bf>IBszNF z-!iair8;HW)uT@a9?16@6I#NH*EEl-TbTRN&67i3JGpkO`Q|2>~w~c8PLa=2CHl;3n7wH7C z=4k)i+-bD``5wz|}T4sZwuH@nUb_N}C)on7S$~#7XC(`5;2JKU!np6}2Z}{WXi6 zy}aAv+Di%DF&4(%k+&J^nTe6D5YNJIdz-DwpxV_Zmq?+*A?p}V82)a|BYhnUl+S|R z0YcXJHmo1)V>txF@3Tv}!;+$(@h=tT17Y}-_{i~6u+MAz$UZYdG9}6{ZnHHD=f>b) zF>aw>{GODLLNS7}`}nqrp%QB(;ZTp2Eec%EqG0sn@~@n?--cfAk~Y2Op+a%`Nt-q5 zqqu!y#Vpxj$IrWKG1UXe81tqPdC;$r#vk`0fQ?Tv^O$`W9)20|8Srpu_+-~B;%#eS zRubfRC^#T!xuSkhy4#e(SL~T zzs&bvZ|Ogk|HnFi*zxIm|0fTus#OX5hpm@vBX;qg{kQjffZu)xWrb<}dX#@XtpDci z*=it7quRa%m(q_BB*FT4$ma4RI&kwEa=&D|Ugn|?Fa9r18ZP~k^6sc5DOKPa0 z`@0XZO*Gfk2iedVk@R>|wcrMVdj$D@1os-F7_AYT{s#w@mmyosn1|xPR6|3hQzT|= z?qWDQ&x)z3t7bV^kU>*3#`$_(CTNK(;m)!4Kc6d=j5=ab_f&Gm<`Wce#l7Cj;!00+ zxN95q0iuOAPDZ&5S-0zw$*RBy5vS^kK}?~#dOLeJE042SvTtpEyJ0V$W z1wg>Xv^}*Kv0qu9cn7zJpE+I#49K6VWii^4S#ORKHhM>hk`;Pd=0%0sM4};gv*Nr* z4*JR|oOk>~Hp%;hS6uLSOgX8&lf=Tgmc+YHy$)N2gzhgY^9rSAKsi}P~h3V>TOCU zs|dt(=S2-9R*x46yNT@ai#swlE|5b38zx;=`@+vVHQUZ&gS-bST02X%{EZ`&^`3n+a`*#TC>C#hB3dER+& zQP%v{=0S@&Q~*k?3%5#WdIz{;JXmdpda#_5tSRg#2yZj)j@>zYmU)I9maYnjCQZCG zGwRLNz=|ZqI@~Fo+dj<~vS?A~M7&kb2XnG+FJH~SALia`+uV&fmXxNCd%w9ydpG@s zQ34n>|In>lyLRA6RgP0Lt#oO~qk1yYM#}Hqpv%R&M39hU9lqs-(1DgnyKSX=ZghKg z{1Z(QXnngq(xumlSQaz?c9YRLv)pB-(3*}vE=Zvq2I^$duJAciQuz8+%)n{qsiA?> zsM-u8!cW0q`BgrVjT5S_#e52ZSoC`RvYv~=qmUHvr45~0e%K~c5-KI&3^r10Vx8zP z!*aefe>yB3i%&wuT0uvQl5CUiD{Jt<%(`#3_4S);f1r&jxj->neLau{bFAKcAv}r$ zUoy73-F|ro%5sINkJn3v-)#2goYQ)P7B+4G=MxKFwD&n)qdr3pZRTryd|ez@n{ijA z-46L^?&MS~st*WT>T~(J!Eemq^^{H@4bf?J%)6meNnt(kfzRI$zTtP3+=pv%7!Osl z*vfT5xalh}db2H0^zr#6HAK_a0Z2~+^etD(l3Kv~;zA#T?=6Ghg84@4ZWzy}Xl10I zpAaHCNWia5z0!ID--brrJdAZrjiOhC`5w-PDKLa_BXu?}nr3erXM^QcUl=ZLb>X(A zC}D;9vGK>okwL2ZgZDlV9jwu#hoIy(Oa-O&9Vm6^ZL=UAo_3LemW+KBhfAvR)Pct%AG}(qcEe|I?ijHI>&||A zusV2s=R)9aS0`{@Uk>b5de@HsBYiS>(F^fYhUPBV!l&0I=3!C#lVZmYZxN9lI>*gj zsV@zFOtJuji!v$H+7<#O^L-1-iq*>=cBr_kAIwGXaYfoxa>}W}8)LiZ#pNQ51h;F` z9aa&KsegxNU2;2_Ig8%J-&$Dud1A+Kj&j)ja&X)7Gu337;?;%#-VYRg(M)}eZKoL{ z!Z?gW)sP2{?0h$APW^k-JlFotAMyFi8k!A6&8#a)Cy{15e01}_MUMX*Q2#lz{=-B3 zU*ge}XwmV{FVugsAZ|Yn4D0z5D|QLO#u@y|tp9apar_A!cX9r&L&|?o-V0O~u#S!g zsp;urb2quUxw}l%B-4{tRvZ_7zLlT>!G4DGQ3+`Tb3D=f(iqS z2!GMW7uvVJYSX&23zwR{vN|Q~DrCRbSC^opYLhg77+c%N{0}XqZe1(rX&bd#;7>3S z!8YF2A-S116u-E0W}k04Kd$L>Sgz~(B`$5#e*-x^5S@6~6ow(BB}$r}7~KHzD2}DF zIW=2>a{CR7jp1Zt%{STY{N<@syB>U>leH>F>@7+Nxa3dl8Dal160PAGuzF z_C7hiU>+xaoKy2)8f&qLbMHu>d+bhtuwlnuO}-T1fK{x`<@$bPmr*`5-;He>p=7U+ zWj}Fq!V@d%tt_LXhbETeece^`(So#lYR)EhMITG-tfr4L)%7=H_d%{hG#cf$OvHvp zj0k7tLzVap?};lLd=c;WgW8O^xK^%Cqia--I$Y1&rLGCV7$d^l&Ell%c>3U+j zLN#&1--x|=s3-HO9?&>2;89@J4Zi3Bp^c>Qev*Ybz{l1g6~Wcw=Hc_8mtz0bFV;wC zII|lWyv@ln-sB#q48-DbP$_<2-fF=^O2Z<8eJm{;XHOY&2*uPomq zEaQhHZkB}J@U3;j{!&$L2U3Mg7RpDolE5?i?$+kxRO%N9?X7r#xJ=ihzk- zl@;T#XTx?Z0}}_-_Zp)z8pMRa{Z|-=sl)8wou+v0>ElT^@mQVcu83xt)Z_=yz2(*e zIl^9jIDR`9KS|YVOtr2s&P|4<3A!wPOj3u6SS0GmZ(M9oTgpz9g4UM76FgTl zWH#JRQf8{g#X!7oMs{_OuB&noH`1!@N+nC92;ZBL_GaCun4fEAj9B?oW9oq6y};b+ zy!%W{Ou3F5dX16}+JU}iyOL}0Mev@cW_4Ow$nNgcXzQuXT5(O>rp{#fv8#ha@~c^L zeBGGLch|Tj#NHE!?%nXm->39yt-S&TGv!6^K3)TF+;^Oq7nBF@F%2`N$zh52JSGjZ z%PX?`&Ib>{MaTV491>*O`{&#kYcQ1u4)rYa?mEu06IK)bUscT4_q;VCE43oMedp_U zEm#e_5>^!h9;S`f^pkv6-O+k7Ap(Q8r{Q7{G#y$eP z)^POgeaDNuaT{-+_NB{)zG=8T`eQl&u&P;z*RTFv`|1Sfq6NTe=*N~uGaA%(E!0Ec z-D{MQXIX+meS_d|uc};br9As5A%bPIJnc}z(0xaD@qu#dY@TF^knts=TFy^9)jUL{ zRgQQG)w7oFJRnV^hCrOz-oW#6Az0(aG+~uc_;phq=pjCQxHZ}miNTIRfnk1Ho0Ob5 zp+KEB_%W|$L*nbTHc_B(KKQH?Z$^OD$gxZXtz_q027-5^vo*HiUDDkp`UTcUdUeCVBwB| znCEJu`G8kz<^U11Tfv)@3<5kP`Rz6~QV$FBMD5HmSqQM8UrZlL*WMjSBasq696e&i zh|Kb2Y6~tjXG=aPvWS?TLlb+G!N9P&&F8i5Ywe*ufYzD8 zV=b@t@7BsmJ}0eM*C@bkI_%4Q!ZZD-cOfv<-yZ2Y9KMH_TemO6YiKd}h^PCoefCY}0qBT@DR|ytysa z)Ln@*Di!7r!#@l3=WkU-cS8^jO}z@nO-jSqs@LTJsEf;u&>87LT9aWWc|A=Knva70 zf*yFTOyK!oanU6pHvt+* zIp1!Wr&9N1M}6HhUdBLIlM9O7M(wm^xa_2$;D;QJ5vd)^oYju_LVunXEB3<`so zC7Sq!C*2CY89w}60uBa771^&WoYb=OtuWo`f)vaM_ol~h>`HZgnyUetYY4C=efpF| z?}#rkboG9G2RNXQ?b9YjKk4mR^oy^;wayp)GkCM?6fnLIb)JR5biEuxqP$ITjs95i zdo?_i8=j=*yL&ZhRe5-jLwKN~B+e=0$rh z$5b65i^7*~)KCG}y0lAm@6eFl0joCRE#5gxo|?zgC60Wf`hoh3QP5`cC?!7 z&wC)UKlmlJp99@>T6ojECBx+}fo%{Xvc3 zdK;!|yYyj2cKSo}ULy{8Cl_5Q)>3ZB8E`ay!M58slF*p1GFEa+XddVBaonyO5wC;1 z=%g-78u8`tTGZGz5e#ig2hrGW#1Cs_}Klrs%tW0kAKbP>}~7rV?6iAt44gV7{{(f z`er)yJG|b3TvbsIxPmn#lN*)G{H(el6#A1NCInPX;)fmDn*)ZPj{`0}^q{QJ|b- zH>vK1wGNAqPZeQ}?@FkzaZIKt)+0btSa$(Zg~L>A3-G|21{U*fPx?Vr=TT)No^f z?zQd}aCYKbp|14%v!T^B1B-1xOQz-@AWoJl4U|1!D!?iLMCoUiX*D1P=J*dI&$po# zJ$x{2&zkYX3j1JGUgMAVLAd3ui3NgGTI#i{)(QRjcG;*1`#tRO_aOzC`CaCksJu5^ z@)N?dBxZa(cotWgxANXn@=)(u_ebon8pI53Ya|*W825-bH=c*Rn8Scm`WX}I0@7w``Os% z;cP7QT`<7lg)tDM0jQSg&R?K)a2H^Ge`iI4Eq%cu#!+DS(d7c`Z0E>sh$_dLxqm|V z@{@&Aj?MxbTeo<3q23gBeF}%ppubU`RWAh9(tvAf7Wq8&^4Jt_>TuqHX@z@-s$*y* zm(gv8DyLt3?70!;;=upPMQN?Ty#R#sP{`AH7N!uTXTT0;04Lppa~WgrYg8>PdVWuI z6H1HY{87Rp2ib8hp}hUpi^brRepC$_ubJU6_^{iMiuCkd2c!{ZaN-Sk(Q|^5VlZ&v z(U^2>sVD1`jSBoh0}Na0swi-Wq1zIS&r;Hk9zkkG<$3s{>Wargmk$elN|Zet;Dg4G zGBF+;d9OX%#U)Y@MSu+jw=3_4)9bL64`KjL;uM%|0Yh|DpyTy2W(Nivf!zzuH? z@8vaDH{Pvst_KIEH~UAL6jv^zfDLk900f)xlZ4h@*EKE z1-|O!)aj*p+S)mA7V6>?iL!y#7zMW-u?lT9xrl+wO3R5U36I~;n)a8q6mVJ%Ib|r< zd;7QI?J1juKctQbl?Voqe+Z7L1oKlE6m9~}l@4r4t{dlE-$KK_;&pNxo8aOWJ{2mj zPqqD|mrl2u^*T>^P&)h7tKf-ad*@-kQDjJ_t+U2cEyZb(Ps3;H>CN+}fLikgRQr7S z%_7;1=}MH~2F(wf!Ywmc-U0W`#G9oQLV4n0enCMTN{}6P=yUq{)kfz8!>^p{F>k-e zP)+xBOQJtYkIsjRiXfHn)M!~4yxI(-7j?|~;+qtI0xH`d*w0W+F`d$0US1R%6idNV zr#=QC_&X?AYO3So1FGq@^5z(@&wl$MTZs8*Ij4^-JX?+t+l_pZkF17xL}~LEF8oX~ zwX{s^?(Syghg#e|-1&_YORmXJ7)alYe;BPoYt!6%-9f9KOA`mug6sS3I@i8z-O-;1*IHradTe*ErfUM@#yEPKRNUlCapjG1^3K0P)y=)JpUj5 z{J;43U-n{!e-Ht3kBf@n2>qlQ%bQw}srmrMZv zg8x{QL_Df@@026;v0BLXB2(i8cblQPXG;i`&Dq(13`~Y%KaCo(JAVGfUvw=}!gv33 z{QriU#xi*O;8#589>9b*rbq=DWKb_nzj$aCe~IhkImG!F_fMO!1s_w{ZY5l-*Mxwh zNKx;&IyQVGg}YNJLwcM*bMY{Efm#(<+wX2lQDm>w=MBbO(d~C!2YnC5_75yJ^Pv{! zC%#GXp6fnX&@+AXCSFrxVp`}|M!k!-db1Qq5tcq8Ft?V!($Lnpvds0kpR34UEhtNJ zj5g;uUZ^`$+3O2OMX7w5FHvKe zPFO<6gKS$XvURZOZ3!SWl($;~Ra=+k`eBsa_ z*5d`!n-^h7j3rO7n}3c3Q$M`e`PUQ~W` zL}`E_%_@!`Pkr}K(A4ChPcr4GUe7*7!AoBA_2ZM{dJ&5W8Ve5jJ2<32!yvJc%frU> z;=*cM*;@2z~uTVGUS}$>r21 zlUBS7t)}qB+(PYTnM#89Ep-3}_km&>V>b@PpC-_{xC>As7Ati1&6+u#ygKR)$`^Cz zgIwe1a&9^PaCcw5i&2%5@vSZ1d6M^#)s5OcZ`?r!>T=TrK8eyWacHtk{yEACi*{Xr zG4q%&IGH1OoFeSaur$!u7zQoGi%<26b2Bpk)EgoxV|nognn zqJ!i8sw07D7Rh9hvzs3?b?soK(t81xySM1j=EM7tFhK#BMw|Ws3o-Wz@}S|$O_dUZ zb%rJFZa)$;#@2m`Rs`sHro-pzZjG7&Zpdko7CLicRXOF?Zop*TBzRw7y6tvEAeJcB zoHM~oR%ULesQj(Hua962)2}2BdGom@#)Niisw_jSx)zDV6l0u4gkH&hYp{CtsHN%< zW5I+RUu(GOVbrBcdQKDY*9MQ?j^?j7Bf@^Vywn%^=#T^O+4H}#vH{IXf``hQ<@44r9&irdW zNuIgpnwe{^@xJ37Vy_Ol<t_SZ=`<&}Z<173ZN}RC2uP+BkOaBuTot#AFmnc=;%04|hdPorl~2_(`0<4eB8mX2;#=x5*X*;*ihd(VSW5raO3>TvN}A zqki~YN$t@-CD0XJl1u)#3&Gct@-j^F2FCEfC7 zg&s7qL63XQ9zyOG%1v9B4DX1}Y*~wwBT7R3@^*rf2|-2~+uqH%9VSIUeI=b^aUm-c zPNKD8YzUV@@5**d3q;TctGPaUVG;v3T4!4A9m`S6Zt-p_9Wv{(xn-aWco3}+Pf}_1 zh9`kMM$}0`kGMiE`b3ns#P1*MC5FXb$qB>ogH$(u!HcJ&&0MDsLfUizOrI|DG@j;; zzJ0A{1ox8xnD(}MsR|_+1||84f+SnY0AUuBZ?Wh#18t3kJnlhdTVI-J4v8?p{M%*n zUI{=;*EppvhMx<%LgH#8cB1@U!<3`C`5h$6t)i18@z~e&Ozl47lVL>XoskLMg>`QA z?ZXYu6G+WavLFYev51oq`f~zja*{f#e{Ff$#w}uUi#p};=5-v_l>M3PR}^~p_J>3! zvgx1N_ostqR2G#XgDbxB8NYD-$YA;6oLteYA2{XuE0#BO*rArA<3Y;~&{?;E{U_Z7G$Vb_AmEBa2Ea^!UiHhZ74P{Z+# z!{x{t?A_awC*OHOy_3+|G+jtgH!`dpK?n1X1!ChnDTFYTN(z%!S+O8yj90RCh)7&6 zZo{@Fj(-VB zz@Ov*+-rTs(`%&sut3gsHuGnakH0DB^H%`yyXcoFKCEcbti%9#-3Ke%I$YgzPMNew z6LfL#OT_ZgcXqm6JXS@hHsiEcD4X&+P+DSi(64Hf0kW4Rn}-`uoo3+=9Y3xE!_EB{ zw85R@PCg>bqV1QmrMsQotw0!bfkH_eRf5OqkgC7<>b;Wxqx6CD-SbZhsqclVYj8F$ z%w>0}9g0eX8jLZeL)79Ny^)(%!i-)`za}IRn$1C*KDf6`y4+r`@tSG$z=qjrFCV#! zqAFi427Qc6T@2NzWga!d>X^jX=RNWWAVE}pM@L6ViSQ$F=(dxnzJ1jN@rhoR*sUpT zPlvd0ug-aV{5kx$&4SaG&&didFe;3eTC!KBdg})+{1L$L;dZ)b7nT<2Gs23_*Bg9k z3MxHq@o!vopGIn&s$=(v3fAzh%LR%yx0sNVpY$EHjp@u={kC44zx?OWXW>gY-K~V` zh6d*N_;}P@i<^&3)@$q_*oY`s*aIKj5J`gKSlMN}`qw@3wB`ty$qy9p-QZdr2t#B} zOAuNkzY=#Hy(n*2q4L9Nlk4t&ZAw-a=)MlV-vV}sTNCOpG{S=DM^SsSY3cxlJ=N;C zUQ2q!70I`Jq8_cMnbmjL{tNrqTky97Z-2{|dHsFN@9Ze;c3gv-rk036w5ZtM;O9Sg zbN)-A^LrAv-dgpoJQcfq@mHX*WSvgoU+U^_-8ullygz?*>mP6De>*_`|3YiO9{&c6 zjg5jUN~Z7LeY9UPuhvV+%;Yy#xoY|irm!&SDuN-z^)@iXG(f0$K zP(3+6>Ll@+H#7cQBsxDCc2(1APS5ERZ=AB&blzJ#3afJ>Tfy{uWiN>S&+6?ntuVGG(~`GobMQ8`~HeWXJw32Csl+-@ZB@q z4N-9D_+XfS>u-q2ABGk4Bw8mrZIQs=8Hy?R=iwr)8nLDuo=A@?@%QLAPm8No=xx6m z=%+^aeZxxtU3uTxmf~6Ru-OhB9tk^q@U^Iyk&fU9(@7mEZ<}%Ri$55+#`6cbOqM)A zZ6;?2U3PF-zpNAxRnJfW#0Fn3m`=N{!0ek0N_~xYMb2KR*4) z)#X5VXQhZu8|5a5^UyOp%J&0nFRW)yS2^=Bmwj7_{|Sszn2%j&al{H|BARO2w>lRs zKp%MLPTYbUCM9Vh@QKUU*R5-)Pkc=&(cYb_wxlXLA+Pwox1P0?N{N&$E0uRO)$z(5 z)bh)h{4eM#Dh`D8W2N*vtG`u0U$Ab` zmBp;ob@I80Q`<`iArS_q%e51a42r}Eu62vuom@&Giz{X7MGe=n&c=FPmy?#p%=v5Z zrjMQc-o~#`{k_6Qz>WE;-+f(WK$cNbt5U|u=sXL@lP4oTf6{{5eY8Jkwk^_Sa9o@| z8L2rfG^B9jtT*w+LNa^ydY$M~JlS?Lk&Mav5wp+9W9&eu^6{AyzN)@{EQhB>#+#a` zM#F4Yul8!WR3TxD0FA&`J`W-n^Ddh6s;B3&T0vZhZ!sb_F^>)F>RWY$pq}y?+2|>z z2NWj=PZrk&t5$fF`LWRm;*rMZ&rB4m-7dBvX~-`XCigFVU4AZH9~eWlKi{Mr zkqk;#b2~|BV`;vE>Mq^y?E2uFNJcmtomKgJ*#`stj=8J%o}uj7fryZT1Ab)`pEoJGMO>X{ynGnHgcUg1PbO8SBUce z!EdUiTgL$P`Hju>I8-ChoT52FwvvEoj-I)x(sZqlUO!^WTjrWx(ql>?pu|!tK`lK% zK~{9%t}7wh%T_I8DgEfI9LiqC&s=|Of#gnL^QC2OC?)9ttj4)ZAW)H)_216#CqfbGStwotdmzIkDyH)kXT5$&CTXoWUSG&YcMiCJx1GxvC$Z8I+gEOK;*xC zj^LA9_Y9N|`h=YJ%*a=DUrbI&8uYIqU@>qR8S$) \ No newline at end of file diff --git a/_includes/templates/concept-overview.md b/_includes/templates/concept-overview.md index 07a0d14ce0..0cb0831a81 100644 --- a/_includes/templates/concept-overview.md +++ b/_includes/templates/concept-overview.md @@ -10,20 +10,8 @@ {% else %} -### ERROR: You must define a "what_is" block -{: style="color:red" } +{% include templates/_errorthrower.md missing_block='what_is' heading='What is a (Concept)?' purpose='explains what this concept is and its purpose.' %} -This template requires that you explain what this concept is. This explanation will -be displayed under the heading, **What is a {{ concept }}?** - -To get rid of this message and take advantage of this template, define the `what_is` -variable and populate it with content. - -```liquid -{% raw %}{% capture what_is %}{% endraw %} -A {{ concept }} does x and y and z...(etc, etc, text goes on) -{% raw %}{% endcapture %}{% endraw %} -``` {% endif %} @@ -35,45 +23,21 @@ A {{ concept }} does x and y and z...(etc, etc, text goes on) {% else %} -### ERROR: You must define a "when_to_use" block -{: style="color:red" } +{% include templates/_errorthrower.md missing_block='when_to_use' heading='When to use (Concept)' purpose='explains when to use this object.' %} -This template requires that you explain when to use this object. This explanation will -be displayed under the heading, **When to use {{ concept }}s** - -To get rid of this message and take advantage of this template, define the `when_to_use` -variable and populate it with content. - -```liquid -{% raw %}{% capture when_to_use %}{% endraw %} -You should use {{ concept }} when... -{% raw %}{% endcapture %}{% endraw %} -``` {% endif %} {% if when_not_to_use %} -### When not to use {{ concept }}s (alternatives) +### When not to use {{ concept }}s {{ when_not_to_use }} {% else %} -### ERROR: You must define a "when_not_to_use" block -{: style="color:red" } +{% include templates/_errorthrower.md missing_block='when_not_to_use' heading='When not to use (Concept)' purpose='explains when not to use this object.' %} -This template requires that you explain when not to use this object. This explanation will -be displayed under the heading, **When not to use {{ concept }}s (alternatives)** - -To get rid of this message and take advantage of this template, define the `when_not_to_use` -block and populate it with content. - -```liquid -{% raw %}{% capture when_not_to_use %}{% endraw %} -You should not use {{ concept }} if... -{% raw %}{% endcapture %}{% endraw %} -``` {% endif %} @@ -85,69 +49,23 @@ You should not use {{ concept }} if... {% else %} -### ERROR: You must define a "status" block -{: style="color:red" } +{% include templates/_errorthrower.md missing_block='status' heading='Retrieving status for a (Concept)' purpose='explains how to retrieve a status description for this object.' %} -This template requires that you explain the current status of support for this object. -This explanation will be displayed under the heading, **{{ concept }} status**. - -To get rid of this message and take advantage of this template, define the `status` -block and populate it with content. - -```liquid -{% raw %}{% capture status %}{% endraw %} -The current status of {{ concept }}s is... -{% raw %}{% endcapture %}{% endraw %} -``` {% endif %} -{% if required_fields %} +{% if usage %} -### {{ concept }} spec +#### Usage -#### Required Fields - -{{ required_fields }} +{{ usage }} {% else %} -### ERROR: You must define a "required_fields" block -{: style="color:red" } - -This template requires that you provide a Markdown list of required fields for this -object. This list will be displayed under the heading **Required Fields**. - -To get rid of this message and take advantage of this template, define the `required_fields` -block and populate it with content. - -```liquid -{% raw %}{% capture required_fields %} -* `kind`: Always `Pod`. -* `apiVersion`: Currently `v1`. -* `metadata`: An object containing: - * `name`: Required if `generateName` is not specified. The name of this pod. - It must be an - [RFC1035](https://www.ietf.org/rfc/rfc1035.txt) compatible value and be - unique within the namespace. -{% endcapture %}{% endraw %} -``` - -**Note**: You can also define a `common_fields` block that will go under a heading -directly underneath **Required Fields** called **Common Fields**, but it is -not required. -{% endif %} - - -{% if common_fields %} - -#### Common Fields - -{{ common_fields }} +{% include templates/_errorthrower.md missing_block='usage' heading='Usage' purpose='shows the most basic, common use case for this object, in the form of a code sample, command, etc, using tabs to show multiple approaches' %} {% endif %} - {% else %} diff --git a/_includes/templates/kubectl.md b/_includes/templates/kubectl.md new file mode 100644 index 0000000000..1b50a7b95d --- /dev/null +++ b/_includes/templates/kubectl.md @@ -0,0 +1,37 @@ +{% if command %} + +# {% if site.data.kubectl[command].name != "kubectl" %}kubectl {% endif %}{{ site.data.kubectl[command].name }} + +{{ site.data.kubectl[command].synopsis }} + +## Description + +{{ site.data.kubectl[command].description }} + +{% if site.data.kubectl[command].options %} +## Options + +| Option | Shorthand | Default Value | Usage | +|--------------------|---------------|-------|{% for option in site.data.kubectl[command].options %} +| `{{option.name | strip}}` | {% if option.shorthand %}`{{ option.shorthand | strip }}`{% endif %} | {% if option.default_value %}`{{option.default_value| strip}}`{% endif %} | {% if option.usage %}{{option.usage| strip | replace:'|',', '}}{% endif %} |{% endfor %} +{% endif %} + +{% if site.data.kubectl[command].inherited_options %} +## Inherited Options + +| Option | Shorthand | Default Value | Usage | +|--------------------|---------------|-------|{% for option in site.data.kubectl[command].inherited_options %} +| `{{option.name | strip}}` | {% if option.shorthand %}`{{ option.shorthand | strip }}`{% endif %} | {% if option.default_value %}`{{option.default_value| strip}}`{% endif %} | {% if option.usage %}{{option.usage| strip | replace:'|',', '}}{% endif %} |{% endfor %} +{% endif %} + +## See also + +{% for seealso in site.data.kubectl[command].see_also %} +- [`{{ seealso }}`](/docs/kubectl/{% if seealso != "kubectl" %}kubectl_{{seealso}}{% endif %}) +{% endfor %} + +{% else %} + +{% include templates/_errorthrower.md missing_block='command' heading='kubectl (command)' purpose='names the kubectl command, so that the appropriate YAML file (from _data/kubectl) can be transformed into a page.' %} + +{% endif %} \ No newline at end of file diff --git a/_includes/templates/task.md b/_includes/templates/task.md new file mode 100644 index 0000000000..abb7ea6b10 --- /dev/null +++ b/_includes/templates/task.md @@ -0,0 +1,36 @@ +{% if purpose %} + +### Purpose + +{{ purpose }} + +{% else %} + +{% include templates/_errorthrower.md missing_block='purpose' heading='Purpose' purpose='states, in one sentence, what the purpose of this document is, so that the user will know what they are able to achieve if they follow the provided steps.' %} + +{% endif %} + +{% if recommended_background %} + +### Recommended background + +{{ recommended_background }} + +{% else %} + +{% include templates/_errorthrower.md missing_block='recommended_background' heading='Recommended background' purpose='lists assumptions of baseline knowledge that you expect the user to have before reading ahead.' %} + +{% endif %} + + +{% if step_by_step %} + +### Step by step + +{{ step_by_step }} + +{% else %} + +{% include templates/_errorthrower.md missing_block='step_by_step' heading='Step by step' purpose='lists a series of linear, numbered steps that accomplish the described task.' %} + +{% endif %} \ No newline at end of file diff --git a/docs/templatedemos/blankkubectl.md b/docs/templatedemos/blankkubectl.md new file mode 100644 index 0000000000..7403b84fb6 --- /dev/null +++ b/docs/templatedemos/blankkubectl.md @@ -0,0 +1,3 @@ +--- +--- +{% include templates/kubectl.md %} \ No newline at end of file diff --git a/docs/templatedemos/blanktask.md b/docs/templatedemos/blanktask.md new file mode 100644 index 0000000000..026c1afc41 --- /dev/null +++ b/docs/templatedemos/blanktask.md @@ -0,0 +1,3 @@ +--- +--- +{% include templates/task.md %} \ No newline at end of file diff --git a/docs/templatedemos/filledout.md b/docs/templatedemos/filledout.md index 7eef09134e..ba56528a20 100644 --- a/docs/templatedemos/filledout.md +++ b/docs/templatedemos/filledout.md @@ -1,32 +1,80 @@ --- --- - -{% assign concept="Replication Controller" %} +{% assign concept="Pod" %} {% capture what_is %} -A Replication Controller does x and y and z...(etc, etc, text goes on) +A pod is the vehicle for running containers in Kubernetes. A pod consists of: + +- One or more containers +- An IP address that is unique within the cluster +- Optionally: Environment variables, storage volumes, and enterprise features (such as health checking) + +Resources are shared amongst containers in the pod. Containers within a pod also share an IP address and port space, and can find each other via localhost, or interprocess communications (such as semaphores). + +![Pod diagram](/images/docs/pod-overview.svg){: style="max-width: 25%" } +{% comment %}https://drive.google.com/open?id=1pQe4-s76fqyrzB8f3xoJo4MPLNVoBlsE1tT9MyLNINg{% endcomment %} + {% endcapture %} {% capture when_to_use %} -You should use Replication Controller when... +Pods are used any time you need a container to be run. However, they are rarely created by a user, and are instead automatically created by controllers such as jobs, replication controllers, deployments, daemon set. The following table describes the strategy each controller uses to create pods. + + +| Controller | Usage Strategy | +|------------|----------------| +| Deployment | For running pods as a continuous and healthy application | +| Replication Controller | Used for the same purpose as Deployments (superseded Replication Controllers) | +| Jobs | For running pods "to completion" (which are then shut down) | +| Daemon Set | Mainly for performing operations on any nodes that match given parameters | + {% endcapture %} {% capture when_not_to_use %} -You should not use Replication Controller if... +Do not use pods directly. Pods should always be managed by a controller. {% endcapture %} {% capture status %} -The current status of Replication Controllers is... +To retrieve the status of a pod, run the following command: + +```shell +kubectl get pod +``` + +| Return Value | Description | +|--------------|-------------| +| `READY` | Describes the number of containers that are ready to recieve traffic. | +| `STATUS` | A value from the `PodPhase` enum describing the current status of the pod. Can be `Running`, `Pending`, `Succeeded`, `Failed`, and `Unknown`. | + +TODO: Link to refpage for `kubectl get pod` + + +To get a full description of a pod, including past events, run the following command: + +```shell +kubectl describe pod +``` + +TODO: Link to refpage for `kubectl describe pod` + +#### Possible status results + +| Value | Description | +|------------|----------------| +| Deployment | For running pods as a continuous and healthy application | +| Replication Controller | Used for the same purpose as Deployments (superseded Replication Controllers) | +| Jobs | For running pods "to completion" (which are then shut down) | +| Daemon Set | Mainly for performing operations on any nodes that match given parameters | + {% endcapture %} -{% capture required_fields %} -* `kind`: Always `Pod`. -* `apiVersion`: Currently `v1`. -* `metadata`: An object containing: - * `name`: Required if `generateName` is not specified. The name of this pod. - It must be an - [RFC1035](https://www.ietf.org/rfc/rfc1035.txt) compatible value and be - unique within the namespace. +{% capture usage %} +Pods are defined when configuring the controller of your choice. In controller specifications, +the parts that define the contents of the pod are inside the `template:` section. + +```yaml +YAML EXAMPLE HERE +``` + {% endcapture %} {% include templates/concept-overview.md %} \ No newline at end of file diff --git a/docs/templatedemos/index.md b/docs/templatedemos/index.md index d199e6d074..30bee2839c 100644 --- a/docs/templatedemos/index.md +++ b/docs/templatedemos/index.md @@ -1,19 +1,134 @@ --- --- - + + # Template Demos -This page demonstrates new doc templates being worked on. -Click the headings to see the source of the template in GitHub. +

+

Concept Overviews

+
{% capture instructions %} -## [Concept Overviews](https://github.com/kubernetes/kubernetes.github.io/blob/master/_includes/templates/concept-overview.md) +A concept overview covers the most essential, important information about core Kubernetes concepts and features. -- [Blank page that is trying to use template](blank/) -- [Partially filled out page](partial/) -- [Completely filled out page](filledout/) +### Demos -## [Landing Pages](https://github.com/kubernetes/kubernetes.github.io/blob/master/_includes/templates/landing-page.md) +- [Blank page that is trying to use template](blank/) ([Source](https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/master/docs/templatedemos/blank.md)) +- [Partially filled out page](partial/) ([Source](https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/master/docs/templatedemos/partial.md)) +- [Completely filled out page](filledout/) ([Source](https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/master/docs/templatedemos/filledout.md)) -- [Blank](blanklanding/) -- [Filled Out](landingpage/) \ No newline at end of file +### Usage + +To use this template, create a new file with these contents: + +```liquid{% raw %} +--- +--- +{% capture concept %}{% endcapture %} +{% capture what_is %}{% endcapture %} +{% capture when_to_use %}{% endcapture %} +{% capture when_not_to_use %}{% endcapture %} +{% capture status %}{% endcapture %} +{% capture usage %}{% endcapture %} +{% include templates/concept-overview.md %} +{% endraw %}``` + +### Adding page to navigation + +Once your page is saved, somewhere in the `/docs/` directory, add a reference to the `concepts.yml` file under `/_data/` so that it will appear in the left-hand navigation of the site. This is also where you add a title to the page. + +{% endcapture %} +{{ instructions | markdownify }} + +
+ + +

Task

+
{% capture instructions %} + +A task page offers step-by-step instructions for completing a task with Kubernetes. **A task page should be narrowly focused on task completion and not delve into concepts or reference information.** + +### Demos + +- [Blank](blanktask/) ([Source](https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/master/docs/templatedemos/blanktask.md)) +- [Filled Out](task/) ([Source](https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/master/docs/templatedemos/task.md)) + +### Usage + +```liquid{% raw %} +--- +--- +{% capture purpose %}{% endcapture %} +{% capture recommended_background %}{% endcapture %} +{% capture step_by_step %}{% endcapture %} +{% include templates/task.md %} +{% endraw %}``` + +### Adding page to navigation + +Once your page is saved, somewhere in the `/docs/` directory, add a reference to the `tasks.yml` file under `/_data/` so that it will appear in the left-hand navigation of the site. This is also where you add a title to the page. + +{% endcapture %} +{{ instructions | markdownify }} + +
+ +

Landing Pages

+
{% capture instructions %} + +Landing pages are a set of clickable "cards" arranged in a grid. Each card has a heading and description, and optioninall, a thumbnail image. They are meant to be index pages that quickly forward users on to deeper content. + +### Demos + +- [Blank](blanklanding/) ([Source](https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/master/docs/templatedemos/blanklanding.md)) +- [Filled Out](landingpage/) ([Source](https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/master/docs/templatedemos/landingpage.md)) + +### Usage + +To use this template, create a new file with these contents. Essentially, you declare the cards you want by inserting the following YAML structure in the front-matter YAML section at the top of the page, and the body of the page just has the include statement. + +```yaml +--- +cards: +- progression: no #"yes" = display cards as linearly progressing +- card: + title: Mean Stack + image: /images/docs/meanstack/image_0.png + description: Lorem ipsum dolor it verberum. +# repeat -card: items as necessary +--- +{% raw %}{% include templates/landing-page.md %}{% endraw %} +``` + +### Adding page to navigation + +Once your page is saved, somewhere in the `/docs/` directory, add a reference to the appropriate .yml file under `/_data/` so that it will appear in the left-hand navigation of the site. This is also where you add a title to the page. + +{% endcapture %} +{{ instructions | markdownify }} + +
+ + +

kubectl yaml

+
{% capture instructions %} +You probably shouldn't be using this, but we also have templates which consume YAML files that are generated by the Kubernetes authors. These are turned into pages which display the reference information for the various CLI tools. + +### Demos + +- [Blank](blankkubectl/) ([Source](https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/master/docs/templatedemos/blankkubectl.md)) +- [Filled Out](kubectl/) ([Source](https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/master/docs/templatedemos/kubectl.md)) + +### Adding page to navigation + +Once your page is saved, somewhere in the `/docs/` directory, add a reference to the `concepts.yml` file under `/_data/` so that it will appear in the left-hand navigation of the site. This is also where you add a title to the page. + +{% endcapture %} +{{ instructions | markdownify }} + +
+
\ No newline at end of file diff --git a/docs/templatedemos/kubectl.md b/docs/templatedemos/kubectl.md new file mode 100644 index 0000000000..1c106b166d --- /dev/null +++ b/docs/templatedemos/kubectl.md @@ -0,0 +1,4 @@ +--- +--- +{% capture command %}kubectl_annotate{% endcapture %} +{% include templates/kubectl.md %} \ No newline at end of file diff --git a/docs/templatedemos/task.md b/docs/templatedemos/task.md new file mode 100644 index 0000000000..3803492f2b --- /dev/null +++ b/docs/templatedemos/task.md @@ -0,0 +1,62 @@ +--- +--- +# Doing a thing with a thing + +{% capture purpose %} +This document teaches you how to do a thing. +{% endcapture %} + +{% capture recommended_background %} +In order to do a thing, you must be familiar with the following: + +- [Thing 1](/foo/) +- [Thing 2](/bar/) + +{% endcapture %} + +{% capture step_by_step %} +Here's how to do a thing with a thing. + +#### 1. Prepare the thing + +Lorem ipsum dolor it verberum. + +#### 2. Run the thing command + +Lorem ipsum dolor it verberum. + +#### 3. Create the thing.yaml file + +Lorem ipsum dolor it verberum. + +```yaml +# Creates three nginx replicas +apiVersion: extensions/v1beta1 +kind: Deployment +metadata: + name: nginx-deployment +spec: + replicas: 3 + template: + metadata: + labels: + app: nginx + spec: + containers: + - name: nginx + image: nginx:1.7.9 + ports: + - containerPort: 80 +``` + +#### 4. ??? + +Lorem ipsum dolor it verberum. + +#### 5. Profit! + +Lorem ipsum dolor it verberum. + +{% endcapture %} + +{% include templates/task.md %} \ No newline at end of file diff --git a/images/docs/pod-overview.svg b/images/docs/pod-overview.svg new file mode 100644 index 0000000000..3243c73d55 --- /dev/null +++ b/images/docs/pod-overview.svg @@ -0,0 +1,4 @@ + + + + From 630cc830140c33315b99bc631739f8db9cd2a3a2 Mon Sep 17 00:00:00 2001 From: Naveen Date: Fri, 17 Jun 2016 20:03:39 -0400 Subject: [PATCH 088/348] Updated index.md fixed sample which was failing. Updated index.md fixed sample which was failing. --- docs/user-guide/configmap/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/configmap/index.md b/docs/user-guide/configmap/index.md index 17cc9c154b..ccd3b74ebb 100644 --- a/docs/user-guide/configmap/index.md +++ b/docs/user-guide/configmap/index.md @@ -362,7 +362,7 @@ spec: containers: - name: test-container image: gcr.io/google_containers/busybox - command: [ "/bin/sh", "cat", "/etc/config/special.how" ] + command: ["cat", "/etc/config/special.how" ] volumeMounts: - name: config-volume mountPath: /etc/config From 0efeece39d92ba117afc4455a48a4343ef322f34 Mon Sep 17 00:00:00 2001 From: Matt Baldwin Date: Fri, 17 Jun 2016 22:37:21 -0700 Subject: [PATCH 089/348] Update index.md --- docs/getting-started-guides/index.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/getting-started-guides/index.md b/docs/getting-started-guides/index.md index 81216ac79b..81adb9e39c 100644 --- a/docs/getting-started-guides/index.md +++ b/docs/getting-started-guides/index.md @@ -41,6 +41,8 @@ The local-machine solutions are: [Google Container Engine](https://cloud.google.com/container-engine) offers managed Kubernetes clusters. +[Stackpoint.io](https://stackpoint.io) provides Kubernetes infrastructure automation and management for multiple public clouds. + ### Turn-key Cloud Solutions These solutions allow you to create Kubernetes clusters on a range of Cloud IaaS providers with only a @@ -114,6 +116,7 @@ Here are all the solutions mentioned above in table form. IaaS Provider | Config. Mgmt | OS | Networking | Docs | Conforms | Support Level -------------------- | ------------ | ------ | ---------- | --------------------------------------------- | ---------| ---------------------------- GKE | | | GCE | [docs](https://cloud.google.com/container-engine) | ['œ“][3] | Commercial +Stackpoint.io | | multi-support | multi-support | [docs](http://www.stackpointcloud.com) | | Commercial GCE | Saltstack | Debian | GCE | [docs](/docs/getting-started-guides/gce) | ['œ“][1] | Project Azure | CoreOS | CoreOS | Weave | [docs](/docs/getting-started-guides/coreos/azure/) | | Community ([@errordeveloper](https://github.com/errordeveloper), [@squillace](https://github.com/squillace), [@chanezon](https://github.com/chanezon), [@crossorigin](https://github.com/crossorigin)) Docker Single Node | custom | N/A | local | [docs](/docs/getting-started-guides/docker) | | Project ([@brendandburns](https://github.com/brendandburns)) From bdebd9895f5a93da02cc0266f7902986459c5dad Mon Sep 17 00:00:00 2001 From: Matt Fischer Date: Sat, 18 Jun 2016 10:29:10 -0600 Subject: [PATCH 090/348] nginx service/deployment needs to be cleanedup If you plan on recreating the nginx service & deployment you need to remove both or kubernetes will refuse to rebuild them. Otherwise you get this error: root@compute3:~# kubectl run nginx --image=nginx --port=80 Error from server: deployments.extensions "nginx" already exists Additionally this commit fixes the numbering in the deleted section. The numbered list restarts at 1 due to the indented code, so a simple work around is to escape the period in the numbered list. --- docs/getting-started-guides/docker.md | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/docs/getting-started-guides/docker.md b/docs/getting-started-guides/docker.md index 8efa8ad4e1..34445cdd64 100644 --- a/docs/getting-started-guides/docker.md +++ b/docs/getting-started-guides/docker.md @@ -162,14 +162,22 @@ See [here](/docs/getting-started-guides/docker-multinode/deployDNS/) for instruc ### Turning down your cluster -1. Delete all the containers including the kubelet: +1\. Delete the nginx service and deployment: + +If you plan on re-creating your nginx deployment and service you will need to clean it up. + +```shell +kubectl delete service,deployments nginx +``` + +2\. Delete all the containers including the kubelet: Many of these containers run under the management of the `kubelet` binary, which attempts to keep containers running, even if they fail. So, in order to turn down the cluster, you need to first kill the kubelet container, and then any other containers. You may use `docker rm -f $(docker ps -aq)`, note this removes _all_ containers running under Docker, so use with caution. -2. Cleanup the filesystem: +3\. Cleanup the filesystem: On OS X, first ssh into the docker VM: From 2bc2409b72d81d4fc94544eb2f383601a2eb9760 Mon Sep 17 00:00:00 2001 From: William Martin Stewart Date: Sun, 19 Jun 2016 21:28:41 +0200 Subject: [PATCH 091/348] Stop Downward API docs from linking to themselves --- docs/user-guide/downward-api/index.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/user-guide/downward-api/index.md b/docs/user-guide/downward-api/index.md index bbc0bf58f0..3227414cad 100644 --- a/docs/user-guide/downward-api/index.md +++ b/docs/user-guide/downward-api/index.md @@ -91,4 +91,3 @@ This is an example of a pod that consumes its labels and annotations via the dow Some more thorough examples: * [environment variables](/docs/user-guide/environment-guide/) - * [downward API](/docs/user-guide/downward-api/) From b99eaf0466919e66687deac3b728b6b7a4f1d12d Mon Sep 17 00:00:00 2001 From: Marcin Wielgus Date: Fri, 17 Jun 2016 17:26:27 +0200 Subject: [PATCH 092/348] Cluster autoscaler doc - part 1 (cherry picked from commit bf73c18603dbb28824199a7dfbe3cc796c0601b0) --- docs/admin/cluster-management.md | 58 ++++++++++++++++++++------------ 1 file changed, 36 insertions(+), 22 deletions(-) diff --git a/docs/admin/cluster-management.md b/docs/admin/cluster-management.md index 613ec8afa2..16d4a6dce2 100644 --- a/docs/admin/cluster-management.md +++ b/docs/admin/cluster-management.md @@ -66,37 +66,51 @@ Instance Group will take care of putting appropriate image on new machines and s In other environments you may need to configure the machine yourself and tell the Kubelet on which machine API server is running. -### Horizontal auto-scaling of nodes (GCE) +### Cluster autoscaling -If you are using GCE, you can configure your cluster so that the number of nodes will be automatically scaled based on: +If you are using GCE or GKE, you can configure your cluster so that it is automatically rescaled based on +pod needs. - * CPU and memory utilization. - * Amount of of CPU and memory requested by the pods (called also reservation). +As described in [Compute Resource](/docs/user-guide/compute-resources/), users can reserve how much CPU and memory is allocated to pods. +This information is used by the Kubernetes scheduler to find a place to run the pod. If there is +no node that has enough free capacity (or doesn't match other pod requirements) then the pod has +to wait until some pods are terminated or a new node is added. -Before setting up the cluster by `kube-up.sh`, you can set `KUBE_ENABLE_NODE_AUTOSCALER` environment variable to `true` and export it. -The script will create an autoscaler for the instance group managing your nodes. +Cluster autoscaler looks for the pods that cannot be scheduled and checks if adding a new node, similar +to the other in the cluster, would help. If yes, then it resizes the cluster to accomodate the waiting pods. -The autoscaler will try to maintain the average CPU/memory utilization and reservation of nodes within the cluster close to the target value. -The target value can be configured by `KUBE_TARGET_NODE_UTILIZATION` environment variable (default: 0.7) for ``kube-up.sh`` when creating the cluster. -Node utilization is the total node's CPU/memory usage (OS + k8s + user load) divided by the node's capacity. -Node reservation is the total CPU/memory requested by pods that are running on the node divided by the node's capacity. -If the desired numbers of nodes in the cluster resulting from CPU/memory utilization/reservation are different, -the autoscaler will choose the bigger number. The number of nodes in the cluster set by the autoscaler will be limited from `KUBE_AUTOSCALER_MIN_NODES` (default: 1) -to `KUBE_AUTOSCALER_MAX_NODES` (default: the initial number of nodes in the cluster). +Cluster autoscaler also scales down the cluster if it notices that some node is not needed anymore for +an extended period of time (10min but it may change in the future). -The autoscaler is implemented as a Compute Engine Autoscaler. -The initial values of the autoscaler parameters set by `kube-up.sh` and some more advanced options can be tweaked on -`Compute > Compute Engine > Instance groups > your group > Edit group`[Google Cloud Console page](https://console.developers.google.com) -or using gcloud CLI: +Cluster autoscaler is configured per instance group (GCE) or node pool (GKE). + +If you are using GCE then you can either enable it while creating a cluster with kube-up.sh script. +To configure cluser autoscaler you have to set 3 environment variables: + +* `KUBE_ENABLE_CLUSTER_AUTOSCALER` - it enables cluster autoscaler if set to true. +* `KUBE_AUTOSCALING_MIN_NODES` - minimum number of nodes in the cluster. +* `KUBE_AUTOSCALING_MAX_NODES` - maximum number of nodes in the cluster. + +Example: ```shell -gcloud alpha compute autoscaler --zone $ZONE +KUBE_ENABLE_CLUSTER_AUTOSCALER=true KUBE_AUTOSCALING_MIN_NODES=3 KUBE_AUTOSCALING_MAX_NODES=10 NUM_NODES=5 ./cluster/kube-up.sh ``` -Note that autoscaling will work properly only if node metrics are accessible in Google Cloud Monitoring. -To make the metrics accessible, you need to create your cluster with `KUBE_ENABLE_CLUSTER_MONITORING` -equal to `google` or `googleinfluxdb` (`googleinfluxdb` is the default value). Please also make sure -that you have Google Cloud Monitoring API enabled in Google Developer Console. +On GKE you configure cluster autoscaler either on cluster creation or update or when creating a particular node pool +(which you want to be autoscaled) by passing flags `--enable-autoscaling` `--min-nodes` and `--max-nodes` +to the corresponding `gcloud` commands. + +Examples: +```shell +gcloud container clusters create mytestcluster --zone=us-central1-b --enable-autoscaling=true --min-nodes=3 --max-nodes=10 --num-nodes=5 +``` + +```shell +gcloud container clusters update mytestcluster --enable-autoscaling=true --min-nodes=1 --max-nodes=15 +``` + +**Cluster autoscaler expects that nodes have not been manually modified (e.g. by adding labels via kubectl) as those properties would not be propagated to the new nodes within the same instance group.** ## Maintenance on a Node From 72a3fb3c3719f15b056d5949ac08f6e4699faecf Mon Sep 17 00:00:00 2001 From: Eric Tune Date: Mon, 20 Jun 2016 07:59:19 -0700 Subject: [PATCH 093/348] Moved jobs examples to docs repo These are not considered "examples", per https://github.com/kubernetes/kubernetes/pull/17345#issuecomment-197153744 --- _data/guides.yml | 11 + docs/user-guide/jobs.md | 24 +- docs/user-guide/jobs/expansions/index.md | 192 ++++++++++++ docs/user-guide/jobs/expansions/job.yaml.txt | 18 ++ docs/user-guide/jobs/work-queue-1/Dockerfile | 10 + docs/user-guide/jobs/work-queue-1/index.md | 284 ++++++++++++++++++ docs/user-guide/jobs/work-queue-1/job.yaml | 15 + docs/user-guide/jobs/work-queue-1/worker.py | 7 + docs/user-guide/jobs/work-queue-2/Dockerfile | 6 + docs/user-guide/jobs/work-queue-2/index.md | 210 +++++++++++++ docs/user-guide/jobs/work-queue-2/job.yaml | 14 + .../jobs/work-queue-2/redis-pod.yaml | 15 + .../jobs/work-queue-2/redis-service.yaml | 10 + docs/user-guide/jobs/work-queue-2/rediswq.py | 130 ++++++++ docs/user-guide/jobs/work-queue-2/worker.py | 23 ++ 15 files changed, 957 insertions(+), 12 deletions(-) create mode 100644 docs/user-guide/jobs/expansions/index.md create mode 100644 docs/user-guide/jobs/expansions/job.yaml.txt create mode 100644 docs/user-guide/jobs/work-queue-1/Dockerfile create mode 100644 docs/user-guide/jobs/work-queue-1/index.md create mode 100644 docs/user-guide/jobs/work-queue-1/job.yaml create mode 100755 docs/user-guide/jobs/work-queue-1/worker.py create mode 100644 docs/user-guide/jobs/work-queue-2/Dockerfile create mode 100644 docs/user-guide/jobs/work-queue-2/index.md create mode 100644 docs/user-guide/jobs/work-queue-2/job.yaml create mode 100644 docs/user-guide/jobs/work-queue-2/redis-pod.yaml create mode 100644 docs/user-guide/jobs/work-queue-2/redis-service.yaml create mode 100644 docs/user-guide/jobs/work-queue-2/rediswq.py create mode 100755 docs/user-guide/jobs/work-queue-2/worker.py diff --git a/_data/guides.yml b/_data/guides.yml index f5654a6efd..8f4d76580f 100644 --- a/_data/guides.yml +++ b/_data/guides.yml @@ -59,6 +59,17 @@ toc: - title: Using kubectl to Manage Resources path: /docs/user-guide/working-with-resources/ +- title: Batch Jobs + section: + - title: Jobs + path: /docs/user-guide/jobs/ + - title: Parallel Processing using Expansions + path: /docs/user-guide/jobs/expansions/ + - title: Coarse Parallel Processing using a Work Queue + path: /docs/user-guide/jobs/work-queue-1/ + - title: Fine Parallel Processing using a Work Queue + path: /docs/user-guide/jobs/work-queue-2/ + - title: Service Discovery and Load Balancing section: - title: Connecting Applications with Services diff --git a/docs/user-guide/jobs.md b/docs/user-guide/jobs.md index 9acde65d0a..0d60220873 100644 --- a/docs/user-guide/jobs.md +++ b/docs/user-guide/jobs.md @@ -249,12 +249,12 @@ The tradeoffs are: The tradeoffs are summarized here, with columns 2 to 4 corresponding to the above tradeoffs. The pattern names are also links to examples and more detailed description. -| Pattern | Single Job object | Fewer pods than work items? | Use app unmodified? | Works in Kube 1.1? | -| -------------------------------------------------------------------------- |:-----------------:|:---------------------------:|:-------------------:|:-------------------:| -| [Job Template Expansion](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/job/expansions/README.md) | | | ✓ | ✓ | -| [Queue with Pod Per Work Item](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/job/work-queue-1/README.md) | ✓ | | sometimes | ✓ | -| [Queue with Variable Pod Count](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/job/work-queue-2/README.md) | ✓ | ✓ | | ✓ | -| Single Job with Static Work Assignment | ✓ | | ✓ | | +| Pattern | Single Job object | Fewer pods than work items? | Use app unmodified? | Works in Kube 1.1? | +| -------------------------------------------------------------------- |:-----------------:|:---------------------------:|:-------------------:|:-------------------:| +| [Job Template Expansion](/docs/user-guide/job/expansions) | | | ✓ | ✓ | +| [Queue with Pod Per Work Item](/docs/user-guide/job/work-queue-1/) | ✓ | | sometimes | ✓ | +| [Queue with Variable Pod Count](/docs/user-guide/job/work-queue-2/) | ✓ | ✓ | | ✓ | +| Single Job with Static Work Assignment | ✓ | | ✓ | | When you specify completions with `.spec.completions`, each Pod created by the Job controller has an identical [`spec`](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/devel/api-conventions.md#spec-and-status). This means that @@ -265,12 +265,12 @@ are different ways to arrange for pods to work on different things. This table shows the required settings for `.spec.parallelism` and `.spec.completions` for each of the patterns. Here, `W` is the number of work items. -| Pattern | `.spec.completions` | `.spec.parallelism` | -| -------------------------------------------------------------------------- |:-------------------:|:--------------------:| -| [Job Template Expansion](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/job/expansions/README.md) | 1 | should be 1 | -| [Queue with Pod Per Work Item](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/job/work-queue-1/README.md) | W | any | -| [Queue with Variable Pod Count](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/job/work-queue-2/README.md) | 1 | any | -| Single Job with Static Work Assignment | W | any | +| Pattern | `.spec.completions` | `.spec.parallelism` | +| -------------------------------------------------------------------- |:-------------------:|:--------------------:| +| [Job Template Expansion](/docs/user-guide/job/expansions/) | 1 | should be 1 | +| [Queue with Pod Per Work Item](/docs/user-guide/job/work-queue-1/) | W | any | +| [Queue with Variable Pod Count](/docs/user-guide/job/work-queue-2/) | 1 | any | +| Single Job with Static Work Assignment | W | any | ## Advanced Usage diff --git a/docs/user-guide/jobs/expansions/index.md b/docs/user-guide/jobs/expansions/index.md new file mode 100644 index 0000000000..892dff0df3 --- /dev/null +++ b/docs/user-guide/jobs/expansions/index.md @@ -0,0 +1,192 @@ +--- --- + +* TOC +{:toc} + +# Example: Multiple Job Objects from Template Expansion + +In this example, we will run multiple Kubernetes Jobs created from +a common template. You may want to be familiar with the basic, +non-parallel, use of [Jobs](/docs/user-guide/jobs) first. + +## Basic Template Expansion + +First, download the following template of a job to a file called `job.yaml.txt` + +{% include code.html language="yaml" file="job.yaml.txt" ghlink="/docs/user-guide/job/expansions/job.yaml.txt" %} + +Unlike a *pod template*, our *job template* is not a Kubernetes API type. It is just +a yaml representation of a Job object that has some placeholders that need to be filled +in before it can be used. The `$ITEM` syntax is not meaningful to Kubernetes. + +In this example, the only processing the container does is to `echo` a string and sleep for a bit. +In a real use case, the processing would be some substantial computation, such as rendering a frame +of a movie, or processing a range of rows in a database. The "$ITEM" parameter would specify for +example, the frame number or the row range. + +This Job and its Pod template have a label: `jobgroup=jobexample`. There is nothing special +to the system about this label. This label +makes it convenient to operate on all the jobs in this group at once. +We also put the same label on the pod template so that we can check on all Pods of these Jobs +with a single command. +After the job is created, the system will add more labels that distinguish one Job's pods +from another Job's pods. +Note that the label key `jobgroup` is not special to Kubernetes. you can pick your own label scheme. + +Next, expand the template into multiple files, one for each item to be processed. + +```shell +# Expand files into a temporary directory +mkdir ./jobs +for i in apple banana cherry +do + cat job.yaml.txt | sed "s/\$ITEM/$i/" > ./jobs/job-$i.yaml +done +``` + +Check if it worked: + +```shell +$ ls jobs/ +job-apple.yaml +job-banana.yaml +job-cherry.yaml +``` + +Here, we used `sed` to replace the string `$ITEM` with the the loop variable. +You could use any type of template language (jinja2, erb) or write a program +to generate the Job objects. + +Next, create all the jobs with one kubectl command: + +```shell +$ kubectl create -f ./jobs +job "process-item-apple" created +job "process-item-banana" created +job "process-item-cherry" created +``` + +Now, check on the jobs: + +```shell +$ kubectl get jobs -l app=jobexample +JOB CONTAINER(S) IMAGE(S) SELECTOR SUCCESSFUL +process-item-apple c busybox app in (jobexample),item in (apple) 1 +process-item-banana c busybox app in (jobexample),item in (banana) 1 +process-item-cherry c busybox app in (jobexample),item in (cherry) 1 +``` + +Here we use the `-l` option to select all jobs that are part of this +group of jobs. (There might be other unrelated jobs in the system that we +do not care to see.) + +We can check on the pods as well using the same label selector: + +```shell +$ kubectl get pods -l app=jobexample +NAME READY STATUS RESTARTS AGE +process-item-apple-kixwv 0/1 Completed 0 4m +process-item-banana-wrsf7 0/1 Completed 0 4m +process-item-cherry-dnfu9 0/1 Completed 0 4m +``` + +There is not a single command to check on the output of all jobs at once, +but looping over all the pods is pretty easy: + +```shell +$ for p in $(kubectl get pods -l app=jobexample -o name) +do + kubectl logs $p +done +Processing item apple +Processing item banana +Processing item cherry +``` + +## Multiple Template Parameters + +In the first example, each instance of the template had one parameter, and that parameter was also +used as a label. However label keys are limited in [what characters they can +contain](docs/user-guide/labels/#syntax-and-character-set). + +This slightly more complex example uses a the jinja2 template language to generate our objects. +We will use a one-line python script to convert the template to a file. + +First, copy and paste the following template of a Job object, into a file called `job.yaml.jinja2`: + + +```liquid{% raw %} +{%- set params = [{ "name": "apple", "url": "http://www.orangepippin.com/apples", }, + { "name": "banana", "url": "https://en.wikipedia.org/wiki/Banana", }, + { "name": "raspberry", "url": "https://www.raspberrypi.org/" }] +%} +{%- for p in params %} +{%- set name = p["name"] %} +{%- set url = p["url"] %} +apiVersion: batch/v1 +kind: Job +metadata: + name: jobexample-{{ {{ name }} }} + labels: + jobgroup: jobexample +spec: + template: + name: jobexample + labels: + jobgroup: jobexample + spec: + containers: + - name: c + image: busybox + command: ["sh", "-c", "echo Processing URL {{ url }} && sleep 5"] + restartPolicy: Never +--- +{%- endfor %} +{% endraw %} +``` + +The above template defines parameters for each job object using a list of +python dicts (lines 1-4). Then a for loop emits one job yaml object +for each set of parameters (remaining lines). +We take advantage of the fact that multiple yaml documents can be concatenated +with the `---` separator (second to last line). +.) We can pipe the output directly to kubectl to +create the objects. + +You will need the jinja2 package if you do not already have it: `pip install --user jinja2`. +Now, use this one-line python program to expand the template: + +```shell +alias render_template='python -c "from jinja2 import Template; import sys; print(Template(sys.stdin.read()).render());"' +``` + + + +The output can be saved to a file, like this: + +```shell +cat job.yaml.jinja2 | render_template > jobs.yaml +``` + +or sent directly to kubectl, like this: + +```shell +cat job.yaml.jinja2 | render_template | kubectl create -f - +``` + +## Alternatives + +If you have a large number of job objects, you may find that: +- even using labels, managing so many Job objects is cumbersome. +- You exceed resource quota when creating all the Jobs at once, + and do not want to wait to create them incrementally. +- You need a way to easily scale the number of pods running + concurrently. One reason would be to avoid using too many + compute resources. Another would be to limit the number of + concurrent requests to a shared resource, such as a database, + used by all the pods in the job. +- very large numbers of jobs created at once overload the + kubernetes apiserver, controller, or scheduler. + +In this case, you can consider one of the +other [job patterns](/docs/user-guide/jobs/#job-patterns). diff --git a/docs/user-guide/jobs/expansions/job.yaml.txt b/docs/user-guide/jobs/expansions/job.yaml.txt new file mode 100644 index 0000000000..790025b38b --- /dev/null +++ b/docs/user-guide/jobs/expansions/job.yaml.txt @@ -0,0 +1,18 @@ +apiVersion: batch/v1 +kind: Job +metadata: + name: process-item-$ITEM + labels: + jobgroup: jobexample +spec: + template: + metadata: + name: jobexample + labels: + jobgroup: jobexample + spec: + containers: + - name: c + image: busybox + command: ["sh", "-c", "echo Processing item $ITEM && sleep 5"] + restartPolicy: Never diff --git a/docs/user-guide/jobs/work-queue-1/Dockerfile b/docs/user-guide/jobs/work-queue-1/Dockerfile new file mode 100644 index 0000000000..cbd36bb620 --- /dev/null +++ b/docs/user-guide/jobs/work-queue-1/Dockerfile @@ -0,0 +1,10 @@ +# Specify BROKER_URL and QUEUE when running +FROM ubuntu:14.04 + +RUN apt-get update && \ + apt-get install -y curl ca-certificates amqp-tools python \ + --no-install-recommends \ + && rm -rf /var/lib/apt/lists/* +COPY ./worker.py /worker.py + +CMD /usr/bin/amqp-consume --url=$BROKER_URL -q $QUEUE -c 1 /worker.py diff --git a/docs/user-guide/jobs/work-queue-1/index.md b/docs/user-guide/jobs/work-queue-1/index.md new file mode 100644 index 0000000000..7f2ba0b6ac --- /dev/null +++ b/docs/user-guide/jobs/work-queue-1/index.md @@ -0,0 +1,284 @@ +--- +--- + +* TOC +{:toc} + +# Example: Job with Work Queue with Pod Per Work Item + +In this example, we will run a Kubernetes Job with multiple parallel +worker processes. You may want to be familiar with the basic, +non-parallel, use of [Job](/docs/user-guide/jobs) first. + +In this example, as each pod is created, it picks up one unit of work +from a task queue, completes it, deletes it from the queue, and exits. + + +Here is an overview of the steps in this example: + +1. **Start a message queue service.** In this example, we use RabbitMQ, but you could use another + one. In practice you would set up a message queue service once and reuse it for many jobs. +1. **Create a queue, and fill it with messages.** Each message represents one task to be done. In + this example, a message is just an integer that we will do a lengthy computation on. +1. **Start a Job that works on tasks from the queue**. The Job starts several pods. Each pod takes + one task from the message queue, processes it, and repeats until the end of the queue is reached. + +## Starting a message queue service + +This example uses RabbitMQ, but it should be easy to adapt to another AMQP-type message service. + +In practice you could set up a message queue service once in a +cluster and reuse it for many jobs, as well as for long-running services. + +Start RabbitMQ as follows: + +```shell +$ kubectl create -f examples/celery-rabbitmq/rabbitmq-service.yaml +service "rabbitmq-service" created +$ kubectl create -f examples/celery-rabbitmq/rabbitmq-controller.yaml +replicationController "rabbitmq-controller" created +``` + +We will only use the rabbitmq part from the celery-rabbitmq example. + +## Testing the message queue service + +Now, we can experiment with accessing the message queue. We will +create a temporary interactive pod, install some tools on it, +and experiment with queues. + +First create a temporary interactive Pod. + +```shell +# Create a temporary interactive container +$ kubectl run -i --tty temp --image ubuntu:14.04 +Waiting for pod default/temp-loe07 to be running, status is Pending, pod ready: false +... [ previous line repeats several times .. hit return when it stops ] ... +``` + +Note that your pod name and command prompt will be different. + +Next install the `amqp-tools` so we can work with message queues. + +```shell +# Install some tools +root@temp-loe07:/# apt-get update +.... [ lots of output ] .... +root@temp-loe07:/# apt-get install -y curl ca-certificates amqp-tools python dnsutils +.... [ lots of output ] .... +``` + +Later, we will make a docker image that includes these packages. + +Next, we will check that we can discover the rabbitmq service: + +``` +# Note the rabitmq-service has a DNS name, provided by Kubernetes: + +root@temp-loe07:/# nslookup rabbitmq-service +Server: 10.0.0.10 +Address: 10.0.0.10#53 + +Name: rabbitmq-service.default.svc.cluster.local +Address: 10.0.147.152 + +# Your address will vary. +``` + +If Kube-DNS is not setup correctly, the previous step may not work for you. +You can also find the service IP in an env var: + +``` +# env | grep RABBIT | grep HOST +RABBITMQ_SERVICE_SERVICE_HOST=10.0.147.152 +# Your address will vary. +``` + +Next we will verify we can create a queue, and publish and consume messages. + +```shell +# In the next line, rabbitmq-service is the hostname where the rabbitmq-service +# can be reached. 5672 is the standard port for rabbitmq. + +root@temp-loe07:/# BROKER_URL=amqp://guest:guest@rabbitmq-service:5672 +# If you could not resolve "rabbitmq-service" in the previous step, +# then use this command instead: +# root@temp-loe07:/# BROKER_URL=amqp://guest:guest@$RABBITMQ_SERVICE_SERVICE_HOST:5672 + +# Now create a queue: + +root@temp-loe07:/# /usr/bin/amqp-declare-queue --url=$BROKER_URL -q foo -d +foo + +# Publish one message to it: + +root@temp-loe07:/# /usr/bin/amqp-publish --url=$BROKER_URL -r foo -p -b Hello + +# And get it back. + +root@temp-loe07:/# /usr/bin/amqp-consume --url=$BROKER_URL -q foo -c 1 cat && echo +Hello +root@temp-loe07:/# +``` + +In the last command, the `amqp-consume` tool takes one message (`-c 1`) +from the queue, and passes that message to the standard input of an +an arbitrary command. In this case, the program `cat` is just printing +out what it gets on the standard input, and the echo is just to add a carriage +return so the example is readable. + +## Filling the Queue with tasks + +Now lets fill the queue with some "tasks". In our example, our tasks are just strings to be +printed. + +In a practice, the content of the messages might be: + +- names of files to that need to be processed +- extra flags to the program +- ranges of keys in a database table +- configuration parameters to a simulation +- frame numbers of a scene to be rendered + +In practice, if there is large data that is needed in a read-only mode by all pods +of the Job, you will typically put that in a shared file system like NFS and mount +that readonly on all the pods, or the program in the pod will natively read data from +a cluster file system like HDFS. + +For our example, we will create the queue and fill it using the amqp command line tools. +In practice, you might write a program to fill the queue using an amqp client library. + +```shell +$ /usr/bin/amqp-declare-queue --url=$BROKER_URL -q job1 -d +job1 +$ for f in apple banana cherry date fig grape lemon melon +do + /usr/bin/amqp-publish --url=$BROKER_URL -r job1 -p -b $f +done +``` + +So, we filled the queue with 8 messages. + +## Create an Image + +Now we are ready to create an image that we will run as a job. + +We will use the `amqp-consume` utility to read the message +from the queue and run our actual program. Here is a very simple +example program: + +{% include code.html language="python" file="worker.py" ghlink="/docs/user-guide/job/work-queue-1/worker.py" %} + +Now, build an an image. If you are working in the source +tree, then change directory to `examples/job/work-queue-1`. +Otherwise, make a temporary directory, change to it, +download the [Dockerfile](Dockerfile?raw=true), +and [worker.py](worker.py?raw=true). In either case, +build the image with this command: ` + +```shell +$ docker build -t job-wq-1 . +``` + +For the [Docker Hub](https://hub.docker.com/), tag your app image with +your username and push to the Hub with the below commands. Replace +`` with your Hub username. + +```shell +docker tag job-wq-1 /job-wq-1 +docker push /job-wq-1 +``` + +If you are using [Google Container +Registry](https://cloud.google.com/tools/container-registry/), tag +your app image with your project ID, and push to GCR. Replace +`` with your project ID. + +```shell +docker tag job-wq-1 gcr.io//job-wq-1 +gcloud docker push gcr.io//job-wq-1 +``` + +## Defining a Job + +Here is a job definition. You'll need to make a copy of the Job and edit the +image to match the name you used, and call it `./job.yaml`. + + +{% include code.html language="yaml" file="job.yaml" ghlink="/docs/user-guide/job/work-queue-1/job.yaml" %} + +In this example, each pod works on one item from the queue and then exits. +So, the completion count of the Job corresponds to the number of work items +done. So we set, `.spec.completions: 8` for the example, since we put 8 items in the queue. + +## Running the Job + +So, now run the Job: + +```shell +kubectl create -f ./job.yaml +``` + +Now wait a bit, then check on the job. + +```shell +$ kubectl describe jobs/job-wq-1 +Name: job-wq-1 +Namespace: default +Image(s): gcr.io/causal-jigsaw-637/job-wq-1 +Selector: app in (job-wq-1) +Parallelism: 4 +Completions: 8 +Labels: app=job-wq-1 +Pods Statuses: 0 Running / 8 Succeeded / 0 Failed +No volumes. +Events: + FirstSeen LastSeen Count From SubobjectPath Reason Message + ───────── ──────── ───── ──── ───────────── ────── ─────── + 27s 27s 1 {job } SuccessfulCreate Created pod: job-wq-1-hcobb + 27s 27s 1 {job } SuccessfulCreate Created pod: job-wq-1-weytj + 27s 27s 1 {job } SuccessfulCreate Created pod: job-wq-1-qaam5 + 27s 27s 1 {job } SuccessfulCreate Created pod: job-wq-1-b67sr + 26s 26s 1 {job } SuccessfulCreate Created pod: job-wq-1-xe5hj + 15s 15s 1 {job } SuccessfulCreate Created pod: job-wq-1-w2zqe + 14s 14s 1 {job } SuccessfulCreate Created pod: job-wq-1-d6ppa + 14s 14s 1 {job } SuccessfulCreate Created pod: job-wq-1-p17e0 +``` + +All our pods succeeded. Yay. + + +## Alternatives + +This approach has the advantage that you +do not need to modify your "worker" program to be aware that there is a work queue. + +It does require that you run a message queue service. +If running a queue service is inconvenient, you may +want to consider one of the other [job patterns](/docs/user-guide/jobs/#job-patterns). + +This approach creates a pod for every work item. If your work items only take a few seconds, +though, creating a Pod for every work item may add a lot of overhead. Consider another +[example](/docs/user-guide/job/work-queue-2), that executes multiple work items per Pod. + +In this example, we used use the `amqp-consume` utility to read the message +from the queue and run our actual program. This has the advantage that you +do not need to modify your program to be aware of the queue. +A [different example](/docs/user-guide/job/work-queue-2), shows how to +communicate with the work queue using a client library. + +## Caveats + +If the number of completions is set to less than the number of items in the queue, then +not all items will be processed. + +If the number of completions is set to more than the number of items in the queue, +then the Job will not appear to be completed, even though all items in the queue +have been processed. It will start additional pods which will block waiting +for a mesage. + +There is an unlikely race with this pattern. If the container is killed in between the time +that the message is acknowledged by the amqp-consume command and the time that the container +exits with success, or if the node crashes before the kubelet is able to post the success of the pod +back to the api-server, then the Job will not appear to be complete, even though all items +in the queue have been processed. diff --git a/docs/user-guide/jobs/work-queue-1/job.yaml b/docs/user-guide/jobs/work-queue-1/job.yaml new file mode 100644 index 0000000000..d2696ed022 --- /dev/null +++ b/docs/user-guide/jobs/work-queue-1/job.yaml @@ -0,0 +1,15 @@ +apiVersion: batch/v1 +kind: Job +metadata: + name: job-wq-1 +spec: + completions: 8 + parallelism: 2 + template: + metadata: + name: job-wq-1 + spec: + containers: + - name: c + image: gcr.io//job-wq-1 + restartPolicy: OnFailure diff --git a/docs/user-guide/jobs/work-queue-1/worker.py b/docs/user-guide/jobs/work-queue-1/worker.py new file mode 100755 index 0000000000..a20884515d --- /dev/null +++ b/docs/user-guide/jobs/work-queue-1/worker.py @@ -0,0 +1,7 @@ +#!/usr/bin/env python + +# Just prints standard out and sleeps for 10 seconds. +import sys +import time +print("Processing " + sys.stdin.lines()) +time.sleep(10) diff --git a/docs/user-guide/jobs/work-queue-2/Dockerfile b/docs/user-guide/jobs/work-queue-2/Dockerfile new file mode 100644 index 0000000000..2de23b3c98 --- /dev/null +++ b/docs/user-guide/jobs/work-queue-2/Dockerfile @@ -0,0 +1,6 @@ +FROM python +RUN pip install redis +COPY ./worker.py /worker.py +COPY ./rediswq.py /rediswq.py + +CMD python worker.py diff --git a/docs/user-guide/jobs/work-queue-2/index.md b/docs/user-guide/jobs/work-queue-2/index.md new file mode 100644 index 0000000000..434859093b --- /dev/null +++ b/docs/user-guide/jobs/work-queue-2/index.md @@ -0,0 +1,210 @@ +--- +--- + +* TOC +{:toc} + +# Example: Job with Work Queue with Pod Per Work Item + +In this example, we will run a Kubernetes Job with multiple parallel +worker processes. You may want to be familiar with the basic, +non-parallel, use of [Job](/docs/user-guide/jobs) first. + +In this example, as each pod is created, it picks up one unit of work +from a task queue, completes it, deletes it from the queue, and exits. + + +Here is an overview of the steps in this example: + +1. **Start a storage service to hold the work queue.** In this example, we use Redis to store + our work items. In the previous example, we used RabbitMQ. In this example, we use Redis and + a custom work-queue client library because AMQP does not provide a good way for clients to + detect when a finite-length work queue is empty. In practice you would set up a store such + as Redis once and reuse it for the work queues of many jobs, and other things. +1. **Create a queue, and fill it with messages.** Each message represents one task to be done. In + this example, a message is just an integer that we will do a lengthy computation on. +1. **Start a Job that works on tasks from the queue**. The Job starts several pods. Each pod takes + one task from the message queue, processes it, and repeats until the end of the queue is reached. + + +## Starting Redis + +For this example, for simplicitly, we will start a single instance of Redis. +See the [Redis Example](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/redis/README.md) for an example +of deploying Redis scaleably and redundantly. + +Start a temporary Pod running Redis and a service so we can find it. + +```shell +$ kubectl create -f examples/job/work-queue-2/redis-pod.yaml +pod "redis-master" created +$ kubectl create -f examples/job/work-queue-2/redis-service.yaml +service "redis" created +``` + +## Filling the Queue with tasks + +Now lets fill the queue with some "tasks". In our example, our tasks are just strings to be +printed. + +Start a temporary interactive pod for running the Redis CLI + +```shell +$ kubectl run -i --tty temp --image redis --command "/bin/sh" +Waiting for pod default/redis2-c7h78 to be running, status is Pending, pod ready: false +Hit enter for command prompt +``` + +Now hit enter, start the redis CLI, and create a list with some work items in it. + +``` +# redis-cli -h redis +redis:6379> rpush job2 "apple" +(integer) 1 +redis:6379> rpush job2 "banana" +(integer) 2 +redis:6379> rpush job2 "cherry" +(integer) 3 +redis:6379> rpush job2 "date" +(integer) 4 +redis:6379> rpush job2 "fig" +(integer) 5 +redis:6379> rpush job2 "grape" +(integer) 6 +redis:6379> rpush job2 "lemon" +(integer) 7 +redis:6379> rpush job2 "melon" +(integer) 8 +redis:6379> rpush job2 "orange" +(integer) 9 +redis:6379> lrange job2 0 -1 +1) "apple" +2) "banana" +3) "cherry" +4) "date" +5) "fig" +6) "grape" +7) "lemon" +8) "melon" +9) "orange" +``` + +So, the list with key `job2` will be our work queue. + +Note: if you do not have Kube DNS setup correctly, you may need to change +the first step of the above block to `redis-cli -h $REDIS_SERVICE_HOST`. + + +## Create an Image + +Now we are ready to create an image that we will run. + +We will use a python worker program with a redis client to read +the messages from the message queue. + +A simple Redis work queue client library is provided, +called rediswq.py ([Download](rediswq.py?raw=true)). + +The "worker" program in each Pod of the Job uses the work queue +client library to get work. Here it is: + +{% include code.html language="python" file="worker.py" ghlink="/docs/user-guide/job/work-queue-2/worker.py" %} + +If you are working from the source tree, +change directory to the `examples/job/work-queue-2` directory. +Otherwise, download [`worker.py`](worker.py?raw=true), [`rediswq.py`](rediswq.py?raw=true), and [`Dockerfile`](Dockerfile?raw=true) +using above links. Then build the image: + +```shell +docker build -t job-wq-2 . +``` + +### Push the image + +For the [Docker Hub](https://hub.docker.com/), tag your app image with +your username and push to the Hub with the below commands. Replace +`` with your Hub username. + +```shell +docker tag job-wq-2 /job-wq-2 +docker push /job-wq-2 +``` + +You need to push to a public repository or [configure your cluster to be able to access +your private repository](/docs/user-guide/images). + +If you are using [Google Container +Registry](https://cloud.google.com/tools/container-registry/), tag +your app image with your project ID, and push to GCR. Replace +`` with your project ID. + +```shell +docker tag job-wq-2 gcr.io//job-wq-2 +gcloud docker push gcr.io//job-wq-2 +``` + +## Defining a Job + +Here is the job definition: + +{% include code.html language="yaml" file="job.yaml" ghlink="/docs/user-guide/job/work-queue-2/job.yaml" %} + +Be sure to edit the job template to +change `gcr.io/myproject` to your own path. + +In this example, each pod works on several items from the queue and then exits when there are no more items. +Since the workers themselves detect when the workqueue is empty, and the Job controller does not +know about the workqueue, it relies on the workers to signal when they are done working. +The workers signal that the queue is empty by exiting with success. So, as soon as any worker +exits with success, the controller knows the work is done, and the Pods will exit soon. +So, we set the completion count of the Job to 1. The job controller will wait for the other pods to complete +too. + + +## Running the Job + +So, now run the Job: + +```shell +kubectl create -f ./job.yaml +``` + +Now wait a bit, then check on the job. + +```shell +$ kubectl describe jobs/job-wq-2 +Name: job-wq-2 +Namespace: default +Image(s): gcr.io/exampleproject/job-wq-2 +Selector: app in (job-wq-2) +Parallelism: 2 +Completions: Unset +Start Time: Mon, 11 Jan 2016 17:07:59 -0800 +Labels: app=job-wq-2 +Pods Statuses: 1 Running / 0 Succeeded / 0 Failed +No volumes. +Events: + FirstSeen LastSeen Count From SubobjectPath Type Reason Message + --------- -------- ----- ---- ------------- -------- ------ ------- + 33s 33s 1 {job-controller } Normal SuccessfulCreate Created pod: job-wq-2-lglf8 + + +$ kubectl logs pods/job-wq-2-7r7b2 +Worker with sessionID: bbd72d0a-9e5c-4dd6-abf6-416cc267991f +Initial queue state: empty=False +Working on banana +Working on date +Working on lemon +``` + +As you can see, one of our pods worked on several work units. + +## Alternatives + +If running a queue service or modifying your containers to use a work queue is inconvenient, you may +want to consider one of the other [job patterns](/docs/user-guide/jobs/#job-patterns). + +If you have a continuous stream of background processing work to run, then +consider running your background workers with a `replicationController` instead, +and consider running a background processing library such as +https://github.com/resque/resque. diff --git a/docs/user-guide/jobs/work-queue-2/job.yaml b/docs/user-guide/jobs/work-queue-2/job.yaml new file mode 100644 index 0000000000..ee7a06c732 --- /dev/null +++ b/docs/user-guide/jobs/work-queue-2/job.yaml @@ -0,0 +1,14 @@ +apiVersion: batch/v1 +kind: Job +metadata: + name: job-wq-2 +spec: + parallelism: 2 + template: + metadata: + name: job-wq-2 + spec: + containers: + - name: c + image: gcr.io/myproject/job-wq-2 + restartPolicy: OnFailure diff --git a/docs/user-guide/jobs/work-queue-2/redis-pod.yaml b/docs/user-guide/jobs/work-queue-2/redis-pod.yaml new file mode 100644 index 0000000000..ae0c43a793 --- /dev/null +++ b/docs/user-guide/jobs/work-queue-2/redis-pod.yaml @@ -0,0 +1,15 @@ +apiVersion: v1 +kind: Pod +metadata: + name: redis-master + labels: + app: redis +spec: + containers: + - name: master + image: redis + env: + - name: MASTER + value: "true" + ports: + - containerPort: 6379 diff --git a/docs/user-guide/jobs/work-queue-2/redis-service.yaml b/docs/user-guide/jobs/work-queue-2/redis-service.yaml new file mode 100644 index 0000000000..85f2ca2271 --- /dev/null +++ b/docs/user-guide/jobs/work-queue-2/redis-service.yaml @@ -0,0 +1,10 @@ +apiVersion: v1 +kind: Service +metadata: + name: redis +spec: + ports: + - port: 6379 + targetPort: 6379 + selector: + app: redis diff --git a/docs/user-guide/jobs/work-queue-2/rediswq.py b/docs/user-guide/jobs/work-queue-2/rediswq.py new file mode 100644 index 0000000000..ebefa64311 --- /dev/null +++ b/docs/user-guide/jobs/work-queue-2/rediswq.py @@ -0,0 +1,130 @@ +#!/usr/bin/env python + +# Based on http://peter-hoffmann.com/2012/python-simple-queue-redis-queue.html +# and the suggestion in the redis documentation for RPOPLPUSH, at +# http://redis.io/commands/rpoplpush, which suggests how to implement a work-queue. + + +import redis +import uuid +import hashlib + +class RedisWQ(object): + """Simple Finite Work Queue with Redis Backend + + This work queue is finite: as long as no more work is added + after workers start, the workers can detect when the queue + is completely empty. + + The items in the work queue are assumed to have unique values. + + This object is not intended to be used by multiple threads + concurrently. + """ + def __init__(self, name, **redis_kwargs): + """The default connection parameters are: host='localhost', port=6379, db=0 + + The work queue is identified by "name". The library may create other + keys with "name" as a prefix. + """ + self._db = redis.StrictRedis(**redis_kwargs) + # The session ID will uniquely identify this "worker". + self._session = str(uuid.uuid4()) + # Work queue is implemented as two queues: main, and processing. + # Work is initially in main, and moved to processing when a client picks it up. + self._main_q_key = name + self._processing_q_key = name + ":processing" + self._lease_key_prefix = name + ":leased_by_session:" + + def sessionID(self): + """Return the ID for this session.""" + return self._session + + def _main_qsize(self): + """Return the size of the main queue.""" + return self._db.llen(self._main_q_key) + + def _processing_qsize(self): + """Return the size of the main queue.""" + return self._db.llen(self._processing_q_key) + + def empty(self): + """Return True if the queue is empty, including work being done, False otherwise. + + False does not necessarily mean that there is work available to work on right now, + """ + return self._main_qsize() == 0 and self._processing_qsize() == 0 + +# TODO: implement this +# def check_expired_leases(self): +# """Return to the work queueReturn True if the queue is empty, False otherwise.""" +# # Processing list should not be _too_ long since it is approximately as long +# # as the number of active and recently active workers. +# processing = self._db.lrange(self._processing_q_key, 0, -1) +# for item in processing: +# # If the lease key is not present for an item (it expired or was +# # never created because the client crashed before creating it) +# # then move the item back to the main queue so others can work on it. +# if not self._lease_exists(item): +# TODO: transactionally move the key from processing queue to +# to main queue, while detecting if a new lease is created +# or if either queue is modified. + + def _itemkey(self, item): + """Returns a string that uniquely identifies an item (bytes).""" + return hashlib.sha224(item).hexdigest() + + def _lease_exists(self, item): + """True if a lease on 'item' exists.""" + return self._db.exists(self._lease_key_prefix + self._itemkey(item)) + + def lease(self, lease_secs=60, block=True, timeout=None): + """Begin working on an item the work queue. + + Lease the item for lease_secs. After that time, other + workers may consider this client to have crashed or stalled + and pick up the item instead. + + If optional args block is true and timeout is None (the default), block + if necessary until an item is available.""" + if block: + item = self._db.brpoplpush(self._main_q_key, self._processing_q_key, timeout=timeout) + else: + item = self._db.rpoplpush(self._main_q_key, self._processing_q_key) + if item: + # Record that we (this session id) are working on a key. Expire that + # note after the lease timeout. + # Note: if we crash at this line of the program, then GC will see no lease + # for this item an later return it to the main queue. + itemkey = self._itemkey(item) + self._db.setex(self._lease_key_prefix + itemkey, lease_secs, self._session) + return item + + def complete(self, value): + """Complete working on the item with 'value'. + + If the lease expired, the item may not have completed, and some + other worker may have picked it up. There is no indication + of what happened. + """ + self._db.lrem(self._processing_q_key, 0, value) + # If we crash here, then the GC code will try to move the value, but it will + # not be here, which is fine. So this does not need to be a transaction. + itemkey = self._itemkey(value) + self._db.delete(self._lease_key_prefix + itemkey, self._session) + +# TODO: add functions to clean up all keys associated with "name" when +# processing is complete. + +# TODO: add a function to add an item to the queue. Atomically +# check if the queue is empty and if so fail to add the item +# since other workers might think work is done and be in the process +# of exiting. + +# TODO(etune): move to my own github for hosting, e.g. github.com/erictune/rediswq-py and +# make it so it can be pip installed by anyone (see +# http://stackoverflow.com/questions/8247605/configuring-so-that-pip-install-can-work-from-github) + +# TODO(etune): finish code to GC expired leases, and call periodically +# e.g. each time lease times out. + diff --git a/docs/user-guide/jobs/work-queue-2/worker.py b/docs/user-guide/jobs/work-queue-2/worker.py new file mode 100755 index 0000000000..49e5dae798 --- /dev/null +++ b/docs/user-guide/jobs/work-queue-2/worker.py @@ -0,0 +1,23 @@ +#!/usr/bin/env python + +import time +import rediswq + +host="redis" +# Uncomment next two lines if you do not have Kube-DNS working. +# import os +# host = os.getenv("REDIS_SERVICE_HOST") + +q = rediswq.RedisWQ(name="job2", host="redis") +print("Worker with sessionID: " + q.sessionID()) +print("Initial queue state: empty=" + str(q.empty())) +while not q.empty(): + item = q.lease(lease_secs=10, block=True, timeout=2) + if item is not None: + itemstr = item.decode("utf=8") + print("Working on " + itemstr) + time.sleep(10) # Put your actual work here instead of sleep. + q.complete(item) + else: + print("Waiting for work") +print("Queue empty, exiting") From 6f26a1dcd599ddfdbbbe208a4be669543b1f3e47 Mon Sep 17 00:00:00 2001 From: Rodrigo Campos Date: Sun, 12 Jun 2016 15:11:20 -0300 Subject: [PATCH 094/348] Document all node conditions in k8s 1.2 --- docs/admin/node.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/docs/admin/node.md b/docs/admin/node.md index 13332012b8..afae4454d6 100644 --- a/docs/admin/node.md +++ b/docs/admin/node.md @@ -47,15 +47,15 @@ must have appropriate conditions, see below. ### Node Condition -Node Condition describes the conditions of `Running` nodes. Currently the only -node condition is Ready. The Status of this condition can be True, False, or -Unknown. True means the Kubelet is healthy and ready to accept pods. -False means the Kubelet is not healthy and is not accepting pods. Unknown -means the Node Controller, which manages node lifecycle and is responsible for -setting the Status of the condition, has not heard from the -node recently (currently 40 seconds). -Node condition is represented as a json object. For example, -the following conditions mean the node is in sane state: +The `conditions` field describes the status of all `Running` nodes. + +| Node Condition | Description | +|----------------|-------------| +| `OutOfDisk` | `True` if insufficient free space on the node for adding new pods, otherwise `False` | +| `Ready` | `True` if the node is healthy ready to accept pods, `False` if the node is not healthy and is not accepting pods, and `Unknown` if the Node Controller has not heard from the node in the last 40 seconds | + +Node condition is represented as a JSON object. For example, the following response describes a healthy node: +conditions mean the node is in sane state: ```json "conditions": [ From 7fd579d2acb234fac385c4ef4658b676cde66e3a Mon Sep 17 00:00:00 2001 From: Dan Lorenc Date: Mon, 20 Jun 2016 10:12:26 -0700 Subject: [PATCH 095/348] Add minikube page to Getting Started section. --- _data/guides.yml | 2 + docs/getting-started-guides/docker.md | 2 + docs/getting-started-guides/locally.md | 2 + docs/getting-started-guides/minikube.md | 116 ++++++++++++++++++++++++ docs/getting-started-guides/vagrant.md | 2 + 5 files changed, 124 insertions(+) create mode 100644 docs/getting-started-guides/minikube.md diff --git a/_data/guides.yml b/_data/guides.yml index f5654a6efd..dec1026a5a 100644 --- a/_data/guides.yml +++ b/_data/guides.yml @@ -128,6 +128,8 @@ toc: path: /docs/getting-started-guides/ - title: Running Kubernetes on Your Local Machine section: + - title: Running Kubernetes Locally via Minikube + path: /docs/getting-started-guides/minikube/ - title: Running Kubernetes Locally via Docker path: /docs/getting-started-guides/docker/ - title: Running Kubernetes Locally with No VM diff --git a/docs/getting-started-guides/docker.md b/docs/getting-started-guides/docker.md index 8efa8ad4e1..80bce6a5a7 100644 --- a/docs/getting-started-guides/docker.md +++ b/docs/getting-started-guides/docker.md @@ -1,6 +1,8 @@ --- --- +**Stop. This guide has been superseded by [Minikube](../minikube/) which is the recommended method of running Kubernetes on your local machine.** + The following instructions show you how to set up a simple, single node Kubernetes cluster using Docker. Here's a diagram of what the final result will look like: diff --git a/docs/getting-started-guides/locally.md b/docs/getting-started-guides/locally.md index 84ecec34c1..cd08a8b141 100644 --- a/docs/getting-started-guides/locally.md +++ b/docs/getting-started-guides/locally.md @@ -1,6 +1,8 @@ --- --- +**Stop. This guide has been superseded by [Minikube](../minikube/) which is the recommended method of running Kubernetes on your local machine.** + * TOC {:toc} diff --git a/docs/getting-started-guides/minikube.md b/docs/getting-started-guides/minikube.md new file mode 100644 index 0000000000..e4633dfca9 --- /dev/null +++ b/docs/getting-started-guides/minikube.md @@ -0,0 +1,116 @@ +--- +--- + +* TOC +{:toc} + +Minikube starts a single node kubernetes cluster locally for purposes of development and testing. +Minikube packages and configures a Linux VM, Docker and all Kubernetes components, optimized for local development. +Minikube supports Kubernetes features such as: + +* DNS +* NodePorts +* ConfigMaps and Secrets +* Dashboards + +Minikube does not yet support Cloud Provider specific features such as: + +* LoadBalancers +* PersistentVolumes +* Ingress + +### Requirements + +Minikube requires that VT-x/AMD-v virtualization is enabled in BIOS on all platforms. + +To check that this is enabled on Linux, run: + +```shell +cat /proc/cpuinfo | grep 'vmx\|svm' +``` + +This command should output something if the setting is enabled. + +To check that this is enabled on OSX (most newer Macs have this enabled by default), run: + +```shell +sysctl -a | grep machdep.cpu.features | grep VMX + +``` + +This command should output something if the setting is enabled. + +#### Linux + +Minikube requires the latest [Virtualbox](https://www.virtualbox.org/wiki/Downloads) to be installed on your system. + +#### OSX + +Minikube requires one of the following: + +* The latest [Virtualbox](https://www.virtualbox.org/wiki/Downloads). +* The latest version of [VMWare Fusion](https://www.vmware.com/products/fusion). + +### Installation + +See the [latest Minikube release](https://github.com/kubernetes/minikube/releases) for installation instructions. + +### Starting the cluster + +To start a cluster, run the command: + +```shell +minikube start +Starting local Kubernetes cluster... +Kubernetes is available at https://192.168.99.100:443. +``` + +This will build and start a lightweight local cluster, consisting of a master, etcd, Docker and a single node. + +Minikube will also create a "minikube" context, and set it to default in kubectl. +To switch back to this context later, run this command: `kubectl config use-context minikube`. + +Type `minikube stop` to shut the cluster down. + +Minikube also includes the [Kubernetes dashboard](http://kubernetes.io/docs/user-guide/ui/). Run this command to see the included kube-system pods: + +```shell +$ kubectl get pods --all-namespaces +NAMESPACE NAME READY STATUS RESTARTS AGE +kube-system kube-addon-manager-127.0.0.1 1/1 Running 0 35s +kube-system kubernetes-dashboard-9brhv 1/1 Running 0 20s +``` + +Run this command to open the Kubernetes dashboard: + +```shell +minikube dashboard +``` + +### Test it out + +List the nodes in your cluster by running: + +```shell +kubectl get nodes +``` + +Minikube contains a built-in Docker daemon that for running containers. +If you use another Docker daemon for building your containers, you will have to publish them to a registry before minikube can pull them. +You can use minikube's built in Docker daemon to avoid this extra step of pushing your images. +Use the built-in Docker daemon with: + +```shell +eval $(minikube docker-env) +``` + +This command sets up the Docker environment variables so a Docker client can communicate with the minikube Docker daemon: + +```shell +docker ps +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +42c643fea98b gcr.io/google_containers/kubernetes-dashboard-amd64:v1.0.1 "/dashboard --port=90" 3 minutes ago Up 3 minutes k8s_kubernetes-dashboard.1d0d880_kubernetes-dashboard-9brhv_kube-system_5062dd0b-370b-11e6-84b6-5eab1f51187f_134cba4c +475db7659edf gcr.io/google_containers/pause-amd64:3.0 "/pause" 3 minutes ago Up 3 minutes k8s_POD.2225036b_kubernetes-dashboard-9brhv_kube-system_5062dd0b-370b-11e6-84b6-5eab1f51187f_e76d8136 +e9096501addf gcr.io/google-containers/kube-addon-manager-amd64:v2 "/opt/kube-addons.sh" 3 minutes ago Up 3 minutes k8s_kube-addon-manager.a1c58ca2_kube-addon-manager-127.0.0.1_kube-system_48abed82af93bb0b941173334110923f_82655b7d +64748893cf7c gcr.io/google_containers/pause-amd64:3.0 "/pause" 4 minutes ago Up 4 minutes k8s_POD.d8dbe16c_kube-addon-manager-127.0.0.1_kube-system_48abed82af93bb0b941173334110923f_c67701c3 +``` \ No newline at end of file diff --git a/docs/getting-started-guides/vagrant.md b/docs/getting-started-guides/vagrant.md index 99127f5233..12f02b593f 100644 --- a/docs/getting-started-guides/vagrant.md +++ b/docs/getting-started-guides/vagrant.md @@ -1,6 +1,8 @@ --- --- +**Stop. This guide has been superseded by [Minikube](../minikube/) which is the recommended method of running Kubernetes on your local machine.** + Running Kubernetes with Vagrant (and VirtualBox) is an easy way to run/test/develop on your local machine (Linux, Mac OS X). * TOC From 55cc4d09f994af0ae48dbcb9da4854db485414c7 Mon Sep 17 00:00:00 2001 From: Cole Mickens Date: Mon, 20 Jun 2016 15:32:13 -0700 Subject: [PATCH 096/348] document workaround for lack of TTY when piping into bash --- docs/getting-started-guides/azure.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/docs/getting-started-guides/azure.md b/docs/getting-started-guides/azure.md index 78e74e7231..47050db1c9 100644 --- a/docs/getting-started-guides/azure.md +++ b/docs/getting-started-guides/azure.md @@ -20,6 +20,13 @@ export KUBERNETES_PROVIDER=azure; curl -sS https://get.k8s.io | bash ``` +Note: if you receive an error "the input device is not a TTY", then you need to start the deployment manually. + +```shell +cd ~/kubernetes +./cluster/kube-up.sh +``` + NOTE: This script calls [cluster/kube-up.sh](http://releases.k8s.io/{{page.githubbranch}}/cluster/kube-up.sh) which in turn calls [cluster/azure/util.sh](http://releases.k8s.io/{{page.githubbranch}}/cluster/azure/util.sh) using [cluster/azure/config-default.sh](http://releases.k8s.io/{{page.githubbranch}}/cluster/azure/config-default.sh). From ad3009c822f0c33e39ff4b7158cdfff8b8fcc561 Mon Sep 17 00:00:00 2001 From: Mike Metral Date: Thu, 16 Jun 2016 14:49:10 -0700 Subject: [PATCH 097/348] add docs for recursive flag usage --- docs/user-guide/managing-deployments.md | 43 +++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/docs/user-guide/managing-deployments.md b/docs/user-guide/managing-deployments.md index 611d56ed86..ee85850c0d 100644 --- a/docs/user-guide/managing-deployments.md +++ b/docs/user-guide/managing-deployments.md @@ -82,6 +82,49 @@ my-nginx-svc 10.0.0.208 80/TCP 0s With the above commands, we first create resources under docs/user-guide/nginx/ and print the resources created with `-o name` output format (print each resource as resource/name). Then we `grep` only the "service", and then print it with `kubectl get`. +If you happen to organize your resources across several subdirectories within a particular directory, you can recursively perform the operations on the subdirectories also, by specifying `--recursive` or `-R` alongside the `--filename,-f` flag. + +For instance, assume there is a directory `project/k8s/development` that holds all of the manifests needed for the development environment, organized by resource type: + +``` +project/k8s/development +├── configmap +│   └── my-configmap.yaml +├── deployment +│   └── my-deployment.yaml +└── pvc + └── my-pvc.yaml +``` + +By default, performing a bulk operation on `project/k8s/development` will stop at the first level of the directory, not processing any subdirectories. If we tried to create the resources in this directory using the following command, we'd encounter an error: + +```shell +$ kubectl create -f project/k8s/development +error: you must provide one or more resources by argument or filename (.json|.yaml|.yml|stdin) +``` + +Instead, specify the `--recursive` or `-R` flag with the `--filename,-f` flag as such: + +```shell +$ kubectl create -f project/k8s/development --recursive +configmap "my-config" created +deployment "my-deployment" created +persistentvolumeclaim "my-pvc" created +``` + +The `--recursive` flag works with any operation that accepts the `--filename,-f` flag such as: `kubectl {create,get,delete,describe,rollout} etc.` + +The `--recursive` flag also works when multiple `-f` arguments are provided: + +```shell +$ kubectl create -f project/k8s/namespaces -f project/k8s/development --recursive +namespace "development" created +namespace "staging" created +configmap "my-config" created +deployment "my-deployment" created +persistentvolumeclaim "my-pvc" created +``` + If you're interested in learning more about `kubectl`, go ahead and read [kubectl Overview](/docs/user-guide/kubectl-overview). ## Using labels effectively From 90651e731fee725cf3f075f50e17abb2902bc170 Mon Sep 17 00:00:00 2001 From: Nicholas Anderson Date: Tue, 21 Jun 2016 22:43:17 -0700 Subject: [PATCH 098/348] cleanup whitespace and correct misspelling (anoter -> another) --- docs/getting-started-guides/meanstack.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/getting-started-guides/meanstack.md b/docs/getting-started-guides/meanstack.md index d6620ed1a4..11a1ab4363 100644 --- a/docs/getting-started-guides/meanstack.md +++ b/docs/getting-started-guides/meanstack.md @@ -3,7 +3,7 @@ **By: Sandeep Dinesh** - _July 29, 2015_ -![image](/images/docs/meanstack/image_0.png) +![image](/images/docs/meanstack/image_0.png) In [a recent post](http://blog.sandeepdinesh.com/2015/07/running-mean-web-application-in-docker.html), I talked about running a MEAN stack with [Docker Containers.](http://docker.com/) @@ -14,7 +14,7 @@ Thankfully, there is a system we can use to manage our containers in a cluster e * TOC {:toc} -## The Basics of Using Kubernetes +## The Basics of Using Kubernetes Before we jump in and start kube’ing it up, it’s important to understand some of the fundamentals of Kubernetes. @@ -34,7 +34,7 @@ Instead, you have to build a custom container that has the code already inside i To do this, you need to use more Docker. Make sure you have the latest version installed for the rest of this tutorial. -Getting the code: +Getting the code: Before starting, let’s get some code to run. You can follow along on your personal machine or a Linux VM in the cloud. I recommend using Linux or a Linux VM; running Docker on Mac and Windows is outside the scope of this tutorial. @@ -72,7 +72,7 @@ Then, it creates a folder to store the code, `cd`s into that directory, copies t Finally, it specifies the command Docker should run when the container starts, which is to start the app. -## Step 2: Building our Container +## Step 2: Building our Container Right now, the directory should look like this: @@ -410,11 +410,11 @@ web-controller-xxxx 1/1 Running 0 1m ## Step 9: Accessing the App -At this point, everything is up and running. The architecture looks something like this: +At this point, everything is up and running. The architecture looks something like this: -![image](/images/docs/meanstack/MEANstack_architecture.svg){: style="max-width:25%"} +![image](/images/docs/meanstack/MEANstack_architecture.svg){: style="max-width:25%"} -By default, port 80 should be open on the load balancer. In order to find the IP address of our app, run this command: +By default, port 80 should be open on the load balancer. In order to find the IP address of our app, run this command: ```shell $ gcloud compute forwarding-rules list @@ -436,7 +436,7 @@ And the Database works! By using Container Engine and Kubernetes, we have a very robust, container based MEAN stack running in production. -[In anoter post](https://medium.com/google-cloud/mongodb-replica-sets-with-kubernetes-d96606bd9474#.e93x7kuq5), I cover how to setup a MongoDB replica set. This is very important for running in production. +[In another post](https://medium.com/google-cloud/mongodb-replica-sets-with-kubernetes-d96606bd9474#.e93x7kuq5), I cover how to setup a MongoDB replica set. This is very important for running in production. Hopefully I can do some more posts about advanced Kubernetes topics such as changing the cluster size and number of Node.js web server replicas, using different environments (dev, staging, prod) on the same cluster, and doing rolling updates. From 8539c55fba02b1cb5777a33e7529415fb1b6faf2 Mon Sep 17 00:00:00 2001 From: purificant Date: Wed, 22 Jun 2016 11:29:52 +0100 Subject: [PATCH 099/348] fix typo, change base24 to base64 --- docs/user-guide/ui.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/ui.md b/docs/user-guide/ui.md index 56b0b68eb3..fb20ce6952 100644 --- a/docs/user-guide/ui.md +++ b/docs/user-guide/ui.md @@ -100,7 +100,7 @@ track=stable - **Image Pull Secret**: In case the specified Docker container image is private, it may require [pull secret](/docs/user-guide/secrets/) credentials. - Dashboard offers all available secrets in a dropdown list, and allows you to create a new secret. The secret name must follow the DNS domain name syntax, e.g. `new.image-pull.secret`. The content of a secret must be base24-encoded and specified in a [`.dockercfg`](/docs/user-guide/images/#specifying-imagepullsecrets-on-a-pod) file. The secret name may consist of a maximum of 253 characters. + Dashboard offers all available secrets in a dropdown list, and allows you to create a new secret. The secret name must follow the DNS domain name syntax, e.g. `new.image-pull.secret`. The content of a secret must be base64-encoded and specified in a [`.dockercfg`](/docs/user-guide/images/#specifying-imagepullsecrets-on-a-pod) file. The secret name may consist of a maximum of 253 characters. In case the creation of the image pull secret is successful, it is selected by default. If the creation fails, no secret is applied. From 2ab411ffd09fc9344dae295303ead6f0e5ebe5b8 Mon Sep 17 00:00:00 2001 From: John Mulhausen Date: Wed, 22 Jun 2016 05:48:00 -0700 Subject: [PATCH 100/348] New Docker-based authoring instructions --- README.md | 58 +++++++++++++++++++++++++++++-------------------------- 1 file changed, 31 insertions(+), 27 deletions(-) diff --git a/README.md b/README.md index a1eb9c58d4..93741172dc 100644 --- a/README.md +++ b/README.md @@ -4,44 +4,47 @@ Welcome! We are very pleased you want to contribute to the documentation and/or You can click the "Fork" button in the upper-right area of the screen to create a copy of our site on your GitHub account called a "fork." Make any changes you want in your fork, and when you are ready to send those changes to us, go to the index page for your fork and click "New Pull Request" to let us know about it. +## Staging the site on GitHub Pages + If you want to see your changes staged without having to install anything locally, remove the CNAME file in this directory and change the name of the fork to be: YOUR_GITHUB_USERNAME.github.io -Then, visit: [http://YOUR_GITHUB_USERNAME.github.io](http://YOUR_GITHUB_USERNAME.github.io) +Then make any changes. -You should see a special-to-you version of the site. +When you visit [http://YOUR_GITHUB_USERNAME.github.io](http://YOUR_GITHUB_USERNAME.github.io) you should see a special-to-you version of the site that contains the changes you just made. -## Editing/staging the site locally +## Staging the site locally (using pre-baked Docker image) -If you have files to upload, or just want to work offline, run the below commands to setup -your environment for running GitHub pages locally. Then, any edits you make will be viewable +Don't like installing stuff? Download and run a local staging server with a single `docker run` command. + + git clone https://github.com/kubernetes/kubernetes.github.io.git k8sdocs + docker run --rm -v ./k8sdocs:/k8sdocs -p 4000:4000 johndmulhausen/k8sdocs serve + +Then visit [http://localhost:4000](http://localhost:4000) to see our site. Any changes you make in your local clone will be automatically staged. + +If you'd like to see the Dockerfile for this image, see [this gist by @johndmulhausen](https://gist.github.com/johndmulhausen/f8f0ab8d82d2c755af3a4709729e1859). + +## Staging the site locally (from scratch setup) + +The below commands to setup your environment for running GitHub pages locally. Then, any edits you make will be viewable on a lightweight webserver that runs on your local machine. -First install rvm +This will typically be the fastest way (by far) to iterate on docs changes and see them staged, once you get this set up, but it does involve several install steps that take awhile to complete. - curl -sSL https://get.rvm.io | bash -s stable +Install Ruby 2.2 or higher: -Then load it into your environment + apt-get install software-properties-common + apt-add-repository ppa:brightbox/ruby-ng + apt-get install ruby2.2 + apt-get install ruby2.2-dev - source ${HOME}/.rvm/scripts/rvm (or whatever is prompted by the installer) - -Then install Ruby 2.2 or higher - - rvm install ruby-2.2.4 - rvm use ruby-2.2.4 --default - -Verify that this new version is running (optional) - - which ruby - ruby -v - -Install the GitHub Pages package, which includes Jekyll +Install the GitHub Pages package, which includes Jekyll: gem install github-pages -Clone our site +Clone our site: git clone https://github.com/kubernetes/kubernetes.github.io.git @@ -51,15 +54,16 @@ Make any changes you want. Then, to see your changes locally: jekyll serve Your copy of the site will then be viewable at: [http://localhost:4000](http://localhost:4000) -(or wherever Ruby tells you). +(or wherever Jekyll tells you). + +The above instructions work on Mac and Linux. +[These instructions](https://martinbuberl.com/blog/setup-jekyll-on-windows-and-host-it-on-github-pages/) are for Windows users. + +## GitHub help If you're a bit rusty with git/GitHub, you might wanna read [this](http://readwrite.com/2013/10/02/github-for-beginners-part-2) for a refresher. -The above instructions work on Mac and Linux. -[These instructions ](https://martinbuberl.com/blog/setup-jekyll-on-windows-and-host-it-on-github-pages/) -might help for Windows users. - ## Common Tasks ### Edit Page Titles or Change the Left Navigation From a78baafe371ea1ef53dcc3683b1dfd39bc65c0cf Mon Sep 17 00:00:00 2001 From: John Mulhausen Date: Wed, 22 Jun 2016 06:08:19 -0700 Subject: [PATCH 101/348] PWD correction --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 93741172dc..e43118bb87 100644 --- a/README.md +++ b/README.md @@ -20,7 +20,7 @@ When you visit [http://YOUR_GITHUB_USERNAME.github.io](http://YOUR_GITHUB_USERNA Don't like installing stuff? Download and run a local staging server with a single `docker run` command. git clone https://github.com/kubernetes/kubernetes.github.io.git k8sdocs - docker run --rm -v ./k8sdocs:/k8sdocs -p 4000:4000 johndmulhausen/k8sdocs serve + docker run --rm -v "$PWD"/k8sdocs:/k8sdocs -p 4000:4000 johndmulhausen/k8sdocs serve Then visit [http://localhost:4000](http://localhost:4000) to see our site. Any changes you make in your local clone will be automatically staged. From fbddf6e208d177468737ef41f741b28898afba23 Mon Sep 17 00:00:00 2001 From: Gerti Poppel Date: Wed, 22 Jun 2016 15:16:30 +0200 Subject: [PATCH 102/348] Corrected a link to a dashboard image and the spelling of K8S --- docs/hellonode.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/hellonode.md b/docs/hellonode.md index 702601687e..307ade32e7 100755 --- a/docs/hellonode.md +++ b/docs/hellonode.md @@ -139,11 +139,11 @@ It’s now time to deploy your own containerized application to the Kubernetes c $ gcloud container clusters get-credentials hello-world ``` -**The rest of this document requires both the kubernetes client and server version to be 1.2. Run `kubectl version` to see your current versions.** For 1.1 see [this document](https://github.com/kubernetes/kubernetes.github.io/blob/release-1.1/docs/hellonode.md). +**The rest of this document requires both the Kubernetes client and server version to be 1.2. Run `kubectl version` to see your current versions.** For 1.1 see [this document](https://github.com/kubernetes/kubernetes.github.io/blob/release-1.1/docs/hellonode.md). ## Create your pod -A kubernetes **[pod](/docs/user-guide/pods/)** is a group of containers, tied together for the purposes of administration and networking. It can contain a single container or multiple. +A Kubernetes **[pod](/docs/user-guide/pods/)** is a group of containers, tied together for the purposes of administration and networking. It can contain a single container or multiple. Create a pod with the `kubectl run` command: @@ -200,7 +200,7 @@ At this point you should have our container running under the control of Kuberne ## Allow external traffic -By default, the pod is only accessible by its internal IP within the Kubernetes cluster. In order to make the `hello-node` container accessible from outside the kubernetes virtual network, you have to expose the pod as a kubernetes **[service](/docs/user-guide/services/)**. +By default, the pod is only accessible by its internal IP within the Kubernetes cluster. In order to make the `hello-node` container accessible from outside the Kubernetes virtual network, you have to expose the pod as a Kubernetes **[service](/docs/user-guide/services/)**. From our development machine we can expose the pod to the public internet using the `kubectl expose` command combined with the `--type="LoadBalancer"` flag. The flag is needed for the creation of an externally accessible ip: @@ -287,7 +287,7 @@ gcloud docker push gcr.io/PROJECT_ID/hello-node:v2 Building and pushing this updated image should be much quicker as we take full advantage of the Docker cache. -We’re now ready for kubernetes to smoothly update our deployment to the new version of the application. In order to change +We’re now ready for Kubernetes to smoothly update our deployment to the new version of the application. In order to change the image label for our running container, we will need to edit the existing *hello-node deployment* and change the image from `gcr.io/PROJECT_ID/hello-node:v1` to `gcr.io/PROJECT_ID/hello-node:v2`. To do this, we will use the `kubectl edit` command. This will open up a text editor displaying the full deployment yaml configuration. It isn't necessary to understand the full yaml config @@ -373,7 +373,7 @@ This user interface allows you to get started quickly and enables some of the fu Enjoy the Kubernetes graphical dashboard and use it for deploying containerized applications, as well as for monitoring and managing your clusters! -![image](/images/docs/ui-dashboard-cards-menu.png) +![image](/images/docs/ui-dashboard-workloadview.png) Learn more about the web interface by taking the [Dashboard tour](/docs/user-guide/ui/). From 5bfeb0697d57b4e18cd19b928c0163eea4317ef0 Mon Sep 17 00:00:00 2001 From: John Mulhausen Date: Wed, 22 Jun 2016 06:36:57 -0700 Subject: [PATCH 103/348] Using official github-pages container --- README.md | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index e43118bb87..aa5f73050c 100644 --- a/README.md +++ b/README.md @@ -15,23 +15,26 @@ Then make any changes. When you visit [http://YOUR_GITHUB_USERNAME.github.io](http://YOUR_GITHUB_USERNAME.github.io) you should see a special-to-you version of the site that contains the changes you just made. -## Staging the site locally (using pre-baked Docker image) +## Staging the site locally (using Docker image) Don't like installing stuff? Download and run a local staging server with a single `docker run` command. - git clone https://github.com/kubernetes/kubernetes.github.io.git k8sdocs - docker run --rm -v "$PWD"/k8sdocs:/k8sdocs -p 4000:4000 johndmulhausen/k8sdocs serve + git clone https://github.com/kubernetes/kubernetes.github.io.git + cd kubernetes.github.io + docker run -v "$PWD":/usr/src/app -p "4000:4000" starefossen/github-pages -Then visit [http://localhost:4000](http://localhost:4000) to see our site. Any changes you make in your local clone will be automatically staged. +Then visit [http://localhost:4000](http://localhost:4000) (or wherever Jekyll tells you) to see our site. -If you'd like to see the Dockerfile for this image, see [this gist by @johndmulhausen](https://gist.github.com/johndmulhausen/f8f0ab8d82d2c755af3a4709729e1859). +Any changes you make on your local machine will be automatically staged at this URL. + +For details on this Docker image, see [starefossen/github-pages on Docker Hub](https://hub.docker.com/r/starefossen/github-pages/). ## Staging the site locally (from scratch setup) The below commands to setup your environment for running GitHub pages locally. Then, any edits you make will be viewable on a lightweight webserver that runs on your local machine. -This will typically be the fastest way (by far) to iterate on docs changes and see them staged, once you get this set up, but it does involve several install steps that take awhile to complete. +This will typically be the fastest way (by far) to iterate on docs changes and see them staged, once you get this set up, but it does involve several install steps that take awhile to complete, and makes system-wide modifications. Install Ruby 2.2 or higher: From 17ffba1ef1984c1e8e584371090b9a8b4aeb67f0 Mon Sep 17 00:00:00 2001 From: John Mulhausen Date: Wed, 22 Jun 2016 08:11:35 -0700 Subject: [PATCH 104/348] Figured out official image was dog slow, so once again going proprietary --- README.md | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index aa5f73050c..045c21a7b6 100644 --- a/README.md +++ b/README.md @@ -11,23 +11,21 @@ change the name of the fork to be: YOUR_GITHUB_USERNAME.github.io -Then make any changes. +Then make your changes. When you visit [http://YOUR_GITHUB_USERNAME.github.io](http://YOUR_GITHUB_USERNAME.github.io) you should see a special-to-you version of the site that contains the changes you just made. -## Staging the site locally (using Docker image) +## Staging the site locally (using Docker) Don't like installing stuff? Download and run a local staging server with a single `docker run` command. git clone https://github.com/kubernetes/kubernetes.github.io.git cd kubernetes.github.io - docker run -v "$PWD":/usr/src/app -p "4000:4000" starefossen/github-pages + docker run -ti --rm -v "$PWD":/k8sdocs -p 4000:4000 johndmulhausen/k8sdocs -Then visit [http://localhost:4000](http://localhost:4000) (or wherever Jekyll tells you) to see our site. +Then visit [http://localhost:4000](http://localhost:4000) to see our site. Any changes you make on your local machine will be automatically staged. -Any changes you make on your local machine will be automatically staged at this URL. - -For details on this Docker image, see [starefossen/github-pages on Docker Hub](https://hub.docker.com/r/starefossen/github-pages/). +If you're interested you can view [the Dockerfile for this image](https://gist.github.com/johndmulhausen/f8f0ab8d82d2c755af3a4709729e1859). ## Staging the site locally (from scratch setup) @@ -112,7 +110,7 @@ In English, this would read: "Create a set of tabs with the alias `servicesample and have tabs visually labeled "JSON" and "YAML" that use `json` and `yaml` Rouge syntax highlighting, which display the contents of `service-sample.{extension}` on the page, and link to the file in GitHub at (full path)." -Example file: [Pods: Multi-Container](/docs/user-guide/pods/multi-container/). +Example file: [Pods: Multi-Container](http://kubernetes.io/docs/user-guide/pods/multi-container/). ## Use a global variable From 81f85e7486ffd6cad8ef3dc080378f95d22b3dcc Mon Sep 17 00:00:00 2001 From: johndmulhausen Date: Wed, 22 Jun 2016 13:34:37 -0700 Subject: [PATCH 105/348] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 045c21a7b6..d2ad5c1682 100644 --- a/README.md +++ b/README.md @@ -34,7 +34,7 @@ on a lightweight webserver that runs on your local machine. This will typically be the fastest way (by far) to iterate on docs changes and see them staged, once you get this set up, but it does involve several install steps that take awhile to complete, and makes system-wide modifications. -Install Ruby 2.2 or higher: +Install Ruby 2.2 or higher. If you're on a Mac, follow [these instructions](https://gorails.com/setup/osx/). If you're on Linux, run these commands: apt-get install software-properties-common apt-add-repository ppa:brightbox/ruby-ng From a643ed6e0d8d2b13893aca3e99094985e809f14d Mon Sep 17 00:00:00 2001 From: Phillip Wittrock Date: Wed, 22 Jun 2016 22:12:58 -0700 Subject: [PATCH 106/348] Add more detail to Template docs. --- docs/templatedemos/index.md | 135 ++++++++++++++++++++++++++++++------ 1 file changed, 115 insertions(+), 20 deletions(-) diff --git a/docs/templatedemos/index.md b/docs/templatedemos/index.md index 30bee2839c..a202795fd9 100644 --- a/docs/templatedemos/index.md +++ b/docs/templatedemos/index.md @@ -5,41 +5,136 @@ $(function() { $( "#accordion" ).accordion(); }); - -# Template Demos +# Before you Begin: Get the docs code checked out locally + +Check out the kubernetes/kubernetes.github.io repo and the docsv2 branch. + +## Step 1: Fork and Clone the repo +- Fork [kubernetes/kubernetes.github.io](https://github.com/kubernetes/kubernetes.github.io) +- [Setup your GitHub authentication using ssh](https://help.github.com/articles/generating-an-ssh-key/) +- Clone the repo under go/src/k8s.io + +```sh +cd ~/go/src/k8s.io +git clone git@github.com:/kubernetes.github.io +cd kubernetes.github.io +git remote add upstream https://github.com/kubernetes/kubernetes.github.io.git +``` + +## Step 2: Move to the docsv2 branch + +Docs v2 development is being performed in the docsv2 branch. This is the branch +you want to be working from. + +From ~/go/src/k8s.io/kubernetes.github.io + +```shell +git checkout -b docsv2 # switch to the docsv2 branch +git fetch upstream # get the latest upstream changes +git reset --hard upstream/docsv2 # sync the branch to upstream +``` + +## Step 3: Make sure you can serve rendered docs locally from your dev box + +- [Follow the editdocs instructions](http://kubernetes.io/editdocs/) + +# Writing Docs Using Templates

Concept Overviews

{% capture instructions %} -A concept overview covers the most essential, important information about core Kubernetes concepts and features. +A concept overview covers the most essential, important information about core +Kubernetes concepts and features. Examples of Concepts include `Pod`, +`Deployment`, `Service`, etc. -### Demos +### Example -- [Blank page that is trying to use template](blank/) ([Source](https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/master/docs/templatedemos/blank.md)) -- [Partially filled out page](partial/) ([Source](https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/master/docs/templatedemos/partial.md)) -- [Completely filled out page](filledout/) ([Source](https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/master/docs/templatedemos/filledout.md)) +- [Source Link](https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/master/docs/templatedemos/filledout.md) +- [Rendered Link](filledout/) ### Usage -To use this template, create a new file with these contents: +To create a new concept overview page, create a new file under +`docs/your-concept-name/index.md`. + +The file should start with the set of tags to apply to the concept and end +with an `include templates/concept-overview.md` block. + +The file needs to have the following `capture` sections: + +- concept +- when_to_use +- when_not_to_use +- status +- usage ```liquid{% raw %} --- +add +tags +here +e.g. +pod --- -{% capture concept %}{% endcapture %} -{% capture what_is %}{% endcapture %} -{% capture when_to_use %}{% endcapture %} -{% capture when_not_to_use %}{% endcapture %} -{% capture status %}{% endcapture %} -{% capture usage %}{% endcapture %} +{% capture concept %} +Name of the Concept goes here (e.g. Pod) +{% endcapture %} + +{% capture what_is %} +1 sentence description of the Concept that describes the function it performs. +If needed: +- add +- clarifying +- bullet points + +![Also add a diagram](/images/docs/file.svg){: style="max-width: 25%" } + +{% comment %} +Include a comment indicating where an editable image exists: +https://drive.google.com/open?id=1pQe4-s76fqyrzB8f3xoJo4MPLNVoBlsE1tT9MyLNINg +{% endcomment %} +{% endcapture %} + +{% capture when_to_use %} +Clarify precisely when and how to use this Concept. Disambiguate when to use +this instead of similar concepts. e.g. Secret vs ConfigMap, +Deployment vs Job vs other controllers, Service vs Ingress. + +| Tables | Can Be | +|------------|----------------| +| Helpful | When | +| Comparing | Functionality | +| Across | Concepts | +{% endcapture %} + +{% capture when_not_to_use %} +Call out common anti-patterns to be avoided. +{% endcapture %} + +{% capture status %} +Describe how to get the current status of an instance in a cluster using kubectl + +Use a table to clarify the response. + +Consider status from kubectl describe as well. +{% endcapture %} + +{% capture usage %} +If the concept interacts with or is embedded in other Concepts, clarify that here. +For Services this should call out the meaning of the label selectors, for Volumes +this should call out that they are defined within a Pod. + +Include the example yaml here +{% endcapture %} + {% include templates/concept-overview.md %} -{% endraw %}``` +{% endraw %}``` ### Adding page to navigation -Once your page is saved, somewhere in the `/docs/` directory, add a reference to the `concepts.yml` file under `/_data/` so that it will appear in the left-hand navigation of the site. This is also where you add a title to the page. +Once your page is saved, somewhere in the `/docs/` directory, add a reference to the `concepts.yml` file under `/_data/` so that it will appear in the left-hand navigation of the site. This is also where you add a title to the page. {% endcapture %} {{ instructions | markdownify }} @@ -70,7 +165,7 @@ A task page offers step-by-step instructions for completing a task with Kubernet ### Adding page to navigation -Once your page is saved, somewhere in the `/docs/` directory, add a reference to the `tasks.yml` file under `/_data/` so that it will appear in the left-hand navigation of the site. This is also where you add a title to the page. +Once your page is saved, somewhere in the `/docs/` directory, add a reference to the `tasks.yml` file under `/_data/` so that it will appear in the left-hand navigation of the site. This is also where you add a title to the page. {% endcapture %} {{ instructions | markdownify }} @@ -106,7 +201,7 @@ cards: ### Adding page to navigation -Once your page is saved, somewhere in the `/docs/` directory, add a reference to the appropriate .yml file under `/_data/` so that it will appear in the left-hand navigation of the site. This is also where you add a title to the page. +Once your page is saved, somewhere in the `/docs/` directory, add a reference to the appropriate .yml file under `/_data/` so that it will appear in the left-hand navigation of the site. This is also where you add a title to the page. {% endcapture %} {{ instructions | markdownify }} @@ -125,10 +220,10 @@ You probably shouldn't be using this, but we also have templates which consume Y ### Adding page to navigation -Once your page is saved, somewhere in the `/docs/` directory, add a reference to the `concepts.yml` file under `/_data/` so that it will appear in the left-hand navigation of the site. This is also where you add a title to the page. +Once your page is saved, somewhere in the `/docs/` directory, add a reference to the `concepts.yml` file under `/_data/` so that it will appear in the left-hand navigation of the site. This is also where you add a title to the page. {% endcapture %} {{ instructions | markdownify }}
-
\ No newline at end of file +
From b76d25b7c17b23052a9abfa1f017edc2e3ef9fde Mon Sep 17 00:00:00 2001 From: Dan Bode Date: Wed, 22 Jun 2016 13:57:46 -0700 Subject: [PATCH 107/348] Minor changes to volume.md This patch adds additional content to the description of the hostPath volume type based on things that I did not find to be clear from the documentation. 1. included an example pod definition in the hostPath description to demonstrate that `path` is the name of the key to use to specify the local directory location. 2. documented that any directories created based on hostPath definitions will only be writable by root and offers some ways of working around this. --- docs/user-guide/volumes.md | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/docs/user-guide/volumes.md b/docs/user-guide/volumes.md index 433deae10f..9f561acf17 100644 --- a/docs/user-guide/volumes.md +++ b/docs/user-guide/volumes.md @@ -117,6 +117,30 @@ Watch out when using this type of volume, because: behave differently on different nodes due to different files on the nodes * when Kubernetes adds resource-aware scheduling, as is planned, it will not be able to account for resources used by a `hostPath` +* the directories created on the underlying hosts are only writable by root, you either need + to run your process as root in a priveleged container or modify the file permissions on + the host to be able to write to a `hostPath` volume + +#### Example pod + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: test-pd +spec: + containers: + - image: gcr.io/google_containers/test-webserver + name: test-container + volumeMounts: + - mountPath: /test-pd + name: test-volume + volumes: + - name: test-volume + hostPath: + # directory location on host + path: /data +``` ### gcePersistentDisk From 57abef3e780b52a87420b1843e2c30b4cf6c143f Mon Sep 17 00:00:00 2001 From: derekwaynecarr Date: Thu, 23 Jun 2016 10:48:05 -0400 Subject: [PATCH 108/348] Update 1.3 documentation for quota --- docs/admin/resourcequota/index.md | 189 ++++++++++++++++++++---------- 1 file changed, 126 insertions(+), 63 deletions(-) diff --git a/docs/admin/resourcequota/index.md b/docs/admin/resourcequota/index.md index 40559f058e..59be97e38e 100644 --- a/docs/admin/resourcequota/index.md +++ b/docs/admin/resourcequota/index.md @@ -4,29 +4,26 @@ When several users or teams share a cluster with a fixed number of nodes, there is a concern that one team could use more than its fair share of resources. -Resource quotas are a tool for administrators to address this concern. Resource quotas -work like this: +Resource quotas are a tool for administrators to address this concern. + +A resource quota, defined by a `ResourceQuota` object, provides constraints that limit +aggregate resource consumption per namespace. It can limit the quantity of objects that can +be created in a namespace by type, as well as the total amount of compute resources that may +be consumed by resources in that project. + +Resource quotas work like this: - Different teams work in different namespaces. Currently this is voluntary, but support for making this mandatory via ACLs is planned. -- The administrator creates a Resource Quota for each namespace. -- Users put compute resource requests on their pods. The sum of all resource requests across - all pods in the same namespace must not exceed any hard resource limit in any Resource Quota - document for the namespace. Note that we used to verify Resource Quota by taking the sum of - resource limits of the pods, but this was altered to use resource requests. Backwards compatibility - for those pods previously created is preserved because pods that only specify a resource limit have - their resource requests defaulted to match their defined limits. The user is only charged for the - resources they request in the Resource Quota versus their limits because the request is the minimum - amount of resource guaranteed by the cluster during scheduling. For more information on over commit, - see [compute-resources](/docs/user-guide/compute-resources). -- If creating a pod would cause the namespace to exceed any of the limits specified in the - the Resource Quota for that namespace, then the request will fail with HTTP status - code `403 FORBIDDEN`. -- If quota is enabled in a namespace and the user does not specify *requests* on the pod for each - of the resources for which quota is enabled, then the POST of the pod will fail with HTTP - status code `403 FORBIDDEN`. Hint: Use the LimitRange admission controller to force default - values of *limits* (then resource *requests* would be equal to *limits* by default, see - [admission controller](/docs/admin/admission-controllers)) before the quota is checked to avoid this problem. +- The administrator creates one or more Resource Quota objects for each namespace. +- Users create resources (pods, services, etc.) in the namespace, and the quota system + tracks usage to ensure it does not exceed hard resource limits defined in a Resource Quota. +- If creating or updating a resource violates a quota constraint, the request will fail with HTTP + status code `403 FORBIDDEN` with a message explaining the constraint that would have been violated. +- If quota is enabled in a namespace for compute resources like `cpu` and `memory`, users must specify + requests or limits for those values; otherwise, the quota system may reject pod creation. Hint: Use + the LimitRange admission controller to force defaults for pods that make no compute resource requirements. + See the [walkthrough](/docs/admin/resourcequota/walkthrough.md) for an example to avoid this problem. Examples of policies that could be created using namespaces and quotas are: @@ -38,7 +35,7 @@ Examples of policies that could be created using namespaces and quotas are: In the case where the total capacity of the cluster is less than the sum of the quotas of the namespaces, there may be contention for resources. This is handled on a first-come-first-served basis. -Neither contention nor changes to quota will affect already-running pods. +Neither contention nor changes to quota will affect already created resources. ## Enabling Resource Quota @@ -57,11 +54,12 @@ in a namespace can be limited. The following compute resource types are support | ResourceName | Description | | ------------ | ----------- | -| cpu | Total cpu requests of containers | -| memory | Total memory requests of containers - -For example, `cpu` quota sums up the `resources.requests.cpu` fields of every -container of every pod in the namespace, and enforces a maximum on that sum. +| cpu | Across all pods in a non-terminal state, the sum of CPU requests cannot exceed this value. | +| limits.cpu | Across all pods in a non-terminal state, the sum of CPU limits cannot exceed this value. | +| limits.memory | Across all pods in a non-terminal state, the sum of memory limits cannot exceed this value. | +| memory | Across all pods in a non-terminal state, the sum of memory requests cannot exceed this value. | +| requests.cpu | Across all pods in a non-terminal state, the sum of CPU requests cannot exceed this value. | +| requests.memory | Across all pods in a non-terminal state, the sum of memory requests cannot exceed this value. | ## Object Count Quota @@ -70,12 +68,15 @@ are supported: | ResourceName | Description | | ------------ | ----------- | -| pods | Total number of pods | -| services | Total number of services | -| replicationcontrollers | Total number of replication controllers | -| resourcequotas | Total number of [resource quotas](/docs/admin/admission-controllers/#resourcequota) | -| secrets | Total number of secrets | -| persistentvolumeclaims | Total number of [persistent volume claims](/docs/user-guide/persistent-volumes/#persistentvolumeclaims) | +| configmaps | The total number of config maps that can exist in the namespace. | +| persistentvolumeclaims | The total number of [persistent volume claims](/docs/user-guide/persistent-volumes/#persistentvolumeclaims) that can exist in the namespace. | +| pods | The total number of pods in a non-terminal state that can exist in the namespace. A pod is in a terminal state if `status.phase in (Failed, Succeeded)` is true. | +| replicationcontrollers | The total number of replication controllers that can exist in the namespace. | +| resourcequotas | The total number of [resource quotas](/docs/admin/admission-controllers/#resourcequota) that can exist in the namespace. | +| services | The total number of services that can exist in the namespace. | +| services.loadbalancers | The total number of services of type load balancer that can exist in the namespace. | +| services.nodeports | The total number of services of type node port that can exist in the namespace. | +| secrets | The total number of secrets that can exist in the namespace. | For example, `pods` quota counts and enforces a maximum on the number of `pods` created in a single namespace. @@ -84,45 +85,107 @@ You might want to set a pods quota on a namespace to avoid the case where a user creates many small pods and exhausts the cluster's supply of Pod IPs. +## Quota Scopes + +Each quota can have an associated set of scopes. A quota will only measure usage for a resource if it matches +the intersection of enumerated scopes. + +When a scope is added to the quota, it limits the number of resources it supports to those that pertain to the scope. +Resources specified on the quota outside of the allowed set results in a validation error. + +| Scope | Description | +| ----- | ----------- | +| Terminating | Match pods where `spec.activeDeadlineSeconds >= 0` | +| NotTerminating | Match pods where `spec.activeDeadlineSeconds is nil` | +| BestEffort | Match pods that have best effort quality of service. | +| NotBestEffort | Match pods that do not have best effort quality of service. | + +The `BestEffort` scope restricts a quota to tracking the following resources: +* pods + +The `Terminating`, `NotTerminating`, and `NotBestEffort` scopes restrict a quota to tracking the following resources: +* cpu +* limits.cpu +* limits.memory +* memory +* pods +* requests.cpu +* requests.memory + +## Requests vs Limits + +When allocating compute resources, each container may specify a request and a limit value for either CPU or memory. +The quota can be configured to quota either value. + +If the quota has a value specified for `requests.cpu` or `requests.memory`, then it requires that every incoming +container makes an explicit request for those resources. If the quota has a value specified for `limits.cpu` or `limits.memory`, +then it requires that every incoming container specifies an explict limit for those resources. + ## Viewing and Setting Quotas Kubectl supports creating, updating, and viewing quotas: ```shell -$ kubectl namespace myspace -$ cat < quota.json -{ - "apiVersion": "v1", - "kind": "ResourceQuota", - "metadata": { - "name": "quota" - }, - "spec": { - "hard": { - "memory": "1Gi", - "cpu": "20", - "pods": "10", - "services": "5", - "replicationcontrollers":"20", - "resourcequotas":"1" - } - } -} +$ kubectl create namespace myspace + +$ cat < compute-resources.yaml +apiVersion: v1 +kind: ResourceQuota +metadata: + name: compute-resources +spec: + hard: + pods: "4" + requests.cpu: "1" + requests.memory: 1Gi + limits.cpu: "2" + limits.memory: 2Gi EOF -$ kubectl create -f ./quota.json -$ kubectl get quota -NAME -quota -$ kubectl describe quota quota -Name: quota +$ kubectl create -f ./compute-resources.yaml --namespace=myspace + +$ cat < object-counts.yaml +apiVersion: v1 +kind: ResourceQuota +metadata: + name: object-counts +spec: + hard: + configmaps: "10" + persistentvolumeclaims: "4" + replicationcontrollers: "20" + secrets: "10" + services: "10" + services.loadbalancers: "2" +EOF +$ kubectl create -f ./object-counts.yaml --namespace=myspace + +$ kubectl get quota --namespace=myspace +NAME AGE +compute-resources 30s +object-counts 32s + +$ kubectl describe quota compute-resources --namespace=myspace +Name: compute-resources +Namespace: myspace +Resource Used Hard +-------- ---- ---- +limits.cpu 0 2 +limits.memory 0 2Gi +pods 0 4 +requests.cpu 0 1 +requests.memory 0 1Gi + +$ kubectl describe quota object-counts --namespace=myspace +Name: object-counts +Namespace: myspace Resource Used Hard -------- ---- ---- -cpu 0m 20 -memory 0 1Gi -pods 5 10 -replicationcontrollers 5 20 -resourcequotas 1 1 -services 3 5 +configmaps 0 10 +persistentvolumeclaims 0 4 +replicationcontrollers 0 20 +secrets 1 10 +services 0 10 +services.loadbalancers 0 2 ``` ## Quota and Cluster Capacity From 212341002aca8667b6101caf07fc7e055d65299b Mon Sep 17 00:00:00 2001 From: Chao Xu Date: Fri, 17 Jun 2016 16:23:55 -0700 Subject: [PATCH 109/348] add a doc for garbage collector --- _data/guides.yml | 4 +++- docs/user-guide/garbage-collector.md | 26 ++++++++++++++++++++++++++ 2 files changed, 29 insertions(+), 1 deletion(-) create mode 100644 docs/user-guide/garbage-collector.md diff --git a/_data/guides.yml b/_data/guides.yml index f5654a6efd..6b99f48574 100644 --- a/_data/guides.yml +++ b/_data/guides.yml @@ -58,6 +58,8 @@ toc: path: /docs/user-guide/config-best-practices/ - title: Using kubectl to Manage Resources path: /docs/user-guide/working-with-resources/ + - title: Garbage collection + path: /docs/user-guide/garbage-collector/ - title: Service Discovery and Load Balancing section: @@ -261,7 +263,7 @@ toc: path: /docs/admin/network-plugins/ - title: Static Pods path: /docs/admin/static-pods/ - - title: Configuring Garbage Collection + - title: Configuring kubelet Garbage Collection path: /docs/admin/garbage-collection/ - title: Configuring Kubernetes with Salt path: /docs/admin/salt/ diff --git a/docs/user-guide/garbage-collector.md b/docs/user-guide/garbage-collector.md new file mode 100644 index 0000000000..e9d646e30c --- /dev/null +++ b/docs/user-guide/garbage-collector.md @@ -0,0 +1,26 @@ +--- +--- + +* TOC +{:toc} + +## WARNING: Garbage Collector is an alpha feature and is disabled by default. Use it at your own risk! + +### What is garbage collector for + +The garbage collector (GC) cascadingly deletes dependent API objects when the owner is deleted. One use case is if two objects have functional dependency, you can specify the dependency in their configuration file when creating them, and if one of them is deleted, the GC will delete the other one automatically. The other use case is if there is logical dependency among API objects, e.g., the pods created by a replicaset depending on the replicaset, Kubernetes will automatically set the dependency (this will be implemented in release 1.4) and the GC will delete the pods when the replicaset is deleted. + +### How does the garbage collector work + +In release 1.3, there is a new `ownerReferences` field in the `metadata` of every Kubernetes API objects. The GC monitors the cluster and checks the `metadata.ownerReferences` field of each object. If none of the owners present in `metadata.ownerReferences` exists in the cluster, the GC will request the API server to delete the object. + +Currently a user needs to manually set the `metadata.ownerReferences`. In release 1.4, controllers will automatically set the field for the objects it controls. For example, when the replicaset controller creates or adopts pods, it will automatically add the replicaset to the `metadata.ownerReferences` fields of the pods. + +### How to request the garbage collector to not delete dependents + +When deleting an object, you can prevent the GC from deleting that object's dependents by specifying `deleteOptions.orphanDependents=true` in the deletion request. It prevents garbage collection by removing the object from its dependents' metadata.ownerReferences field. + +### How to enable the garbage collector + +The garbage collector is an alpha feature so it is disabled by default. To enable it, you need to start the kube-apiserver and kube-controller-manager with flag `--enable-garbage-collector`. + From c2e4bd9147cfdb374a088426fcfab849758eda9c Mon Sep 17 00:00:00 2001 From: Eric Chiang Date: Tue, 7 Jun 2016 16:56:17 -0700 Subject: [PATCH 110/348] docs/admin: document RBAC authorizer --- docs/admin/authorization.md | 159 ++++++++++++++++++++++++++++++++++++ 1 file changed, 159 insertions(+) diff --git a/docs/admin/authorization.md b/docs/admin/authorization.md index 3b521b5df1..81eabc658a 100644 --- a/docs/admin/authorization.md +++ b/docs/admin/authorization.md @@ -19,9 +19,20 @@ need authorization. - `--authorization-mode=ABAC`allows for user-configured authorization policy. ABAC stands for Attribute-Based Access Control. + - `--authorization-mode=RBAC` is an experimental implementation which allows +for authorization to be driven by the Kubernetes API. +RBAC stands for Roles-Based Access Control. - `--authorization-mode=Webhook` allows for authorization to be driven by a remote service using REST. +If multiple modes are provided the set is unioned, and only a single authorizer is required to admit the action. This means the flag: + +``` +--authorization-mode=AlwaysDeny,AlwaysAllow +``` + +will always allow. + ## ABAC Mode ### Request Attributes @@ -152,6 +163,154 @@ file: The apiserver will need to be restarted to pickup the new policy lines. +## RBAC Mode + +When specified "RBAC" (Role-Based Access Control) uses the +"rbac.authorization.k8s.io" API group to drive authorization decisions, +allowing admins to dynamically configure permission policies through the +Kubernetes API. + +As of 1.3 RBAC mode is in alpha and considered experimental. + +### Roles, RolesBindings, ClusterRoles, and ClusterRoleBindings + +The RBAC API Group declares four top level types which will be covered in this +section. Users can interact with these resources as they would with any other +API resource. Through `kubectl`, direct calls to the API, etc. For instance, +`kubectl create -f (resource).yml` can be used with any of these examples, +though readers who wish to follow along should review the following section on +bootstrapping first. + +In the RBAC API Group, roles hold a logical grouping of permissions. These +permissions map very closely to ABAC policies, but only contain information +about requests being made. Permission are purely additive, rules may only omit +permissions they do not wish to grant. + +Here's an example of a role which grants read access to pods within the +"default" namespace. + +```yaml +kind: Role +apiVersion: rbac.authorization.k8s.io/v1alpha1 +metadata: + namespace: default + name: pod-reader +rules: + - apiGroups: [""] # The API group "" indicates the default API Group. + resources: ["pods"] + verbs: ["get", "watch", "list"] + nonResourceURLs: [] +``` + +`ClusterRoles` hold the same information as a `Role` but can apply to any +namespace as well as non-namespaced resources (such as `Nodes`, +`PersistentVolume`, etc.). The following `ClusterRole` can grant permissions to +read secrets in any namespace. + +```yaml +kind: ClusterRole +apiVersion: rbac.authorization.k8s.io/v1alpha1 +metadata: + # "namespace" omitted since ClusterRoles are not namespaced. + name: secret-reader +rules: + - apiGroups: [""] + resources: ["secrets"] + verbs: ["get", "watch", "list"] + nonResourceURLs: [] +``` + +`RoleBindings` perform the task of granting the permission to a user or set of +users. They hold a list of subjects which they apply to, and a reference to the +`Role` being assigned. + +The following `RoleBinding` assigns the "pod-reader" role to the user "jane" +within the "default" namespace, and allows jane to read pods. + +```yaml +# This role binding allows "jane" to read pods in the namespace "default" +kind: RoleBinding +apiVersion: rbac.authorization.k8s.io/v1alpha1 +metadata: + name: read-pods + namespace: default +subjects: + - kind: User # May be "User", "Group" or "ServiceAccount" + name: jane +roleRef: + kind: Role + namespace: default + name: pod-reader + apiVersion: rbac.authorization.k8s.io/v1alpha1 +``` + +`RoleBindings` may also refer to a `ClusterRole`. However, a `RoleBinding` that +refers to a `ClusterRole` only applies in the `RoleBinding`'s namespace, not at +the cluster level. This allows admins to define a set of common roles for the +entire cluster, then reuse them in multiple namespaces. + +For instance, even though the following `RoleBinding` refers to a `ClusterRole`, +"dave" (the subject) will only be able read secrets in the "development" +namespace, the namespace of the `RoleBinding`. + +```yaml +# This role binding allows "dave" to read secrets in the namespace "development" +kind: RoleBinding +apiVersion: rbac.authorization.k8s.io/v1alpha1 +metadata: + name: read-secrets + namespace: development # This binding only applies in the "development" namespace +subjects: + - kind: User # May be "User", "Group" or "ServiceAccount" + name: dave +roleRef: + kind: ClusterRole + name: secret-reader + apiVersion: rbac.authorization.k8s.io/v1alpha1 +``` + +Finally a `ClusterRoleBinding` may be used to grant permissions in all +namespaces. The following `ClusterRoleBinding` allows any user in the group +"manager" to read secrets in any namepsace. + +```yaml +# This cluster role binding allows anyone in the "manager" group to read secrets in any namespace. +kind: ClusterRoleBinding +apiVersion: rbac.authorization.k8s.io/v1alpha1 +metadata: + name: read-secrets +subjects: + - kind: Group # May be "User", "Group" or "ServiceAccount" + name: manager +roleRef: + kind: ClusterRole + name: secret-reader + apiVersion: rbac.authorization.k8s.io/v1alpha1 +``` + +### Privilege Escalation Prevention and Bootstrapping + +The `rbac.authorization.k8s.io` API group inherently attempts to prevent users +from escalating privileges. Simply put, __a user can't grant permissions they +don't already have even when the RBAC authorizer it disabled__. If "user-1" +does not have the ability to read secrets in "namespace-a", they cannot create +a binding that would grant that permission to themselves or any other user. + +For bootstrapping the first roles, it becomes necessary for someone to get +around these limitations. For the alpha release of RBAC, an API Server flag was +added to allow one user to step around all RBAC authorization and privilege +escalation checks. NOTE: _This is subject to change with future releases._ + +``` +--authorization-rbac-super-user=admin +``` + +Once set the specified super user, in this case "admin", can be used to create +the roles and role bindings to initialize the system. + +This flag is optional and once the initial bootstrapping is performed can be +unset. + ## Webhook Mode When specified, mode `Webhook` causes Kubernetes to query an outside REST From d45e0ce4df68870e363ea72e4dc582681caa897b Mon Sep 17 00:00:00 2001 From: Avesh Agarwal Date: Wed, 1 Jun 2016 16:15:50 -0400 Subject: [PATCH 111/348] Document examples of downward API exposing container resources via environment variables and volumes. --- docs/user-guide/downward-api/README.md | 27 ++++++++-- .../dapi-container-resources.yaml | 34 ++++++++++++ docs/user-guide/downward-api/index.md | 50 ++++++++++++++--- .../volume/dapi-volume-resources.yaml | 40 ++++++++++++++ docs/user-guide/downward-api/volume/index.md | 54 ++++++++++++++++++- 5 files changed, 194 insertions(+), 11 deletions(-) create mode 100644 docs/user-guide/downward-api/dapi-container-resources.yaml create mode 100644 docs/user-guide/downward-api/volume/dapi-volume-resources.yaml diff --git a/docs/user-guide/downward-api/README.md b/docs/user-guide/downward-api/README.md index 93c5e6f5dc..00a27bdcb0 100644 --- a/docs/user-guide/downward-api/README.md +++ b/docs/user-guide/downward-api/README.md @@ -1,5 +1,5 @@ -Following this example, you will create a pod with a container that consumes the pod's name and -namespace using the [downward API](http://kubernetes.io/docs/user-guide/downward-api/). +Following these examples, you will create a pod with a container that consumes the pod's name, +namespace, and resource values using the [downward API](http://kubernetes.io/docs/user-guide/downward-api/). ## Step Zero: Prerequisites @@ -29,4 +29,25 @@ $ kubectl logs dapi-test-pod | grep POD_ 2015-04-30T20:22:18.568024817Z MY_POD_NAME=dapi-test-pod 2015-04-30T20:22:18.568087688Z MY_POD_NAMESPACE=default 2015-04-30T20:22:18.568092435Z MY_POD_IP=10.0.1.6 -``` \ No newline at end of file +``` + +## Example of environment variables with container resources + +Use the [`dapi-container-resources.yaml`](dapi-container-resources.yaml) file to create a Pod +with a container that consumes the downward API exposing the container's resources. + +```shell +$ kubectl create -f docs/user-guide/downward-api/dapi-container-resources.yaml +``` + +### Examine the logs + +Grep through the pod logs to see that the pod was injected with the correct values: + +```shell +$ kubectl logs dapi-test-pod | grep MY_ +MY_MEM_LIMIT=67108864 +MY_CPU_LIMIT=1 +MY_MEM_REQUEST=33554432 +MY_CPU_REQUEST=1 +``` diff --git a/docs/user-guide/downward-api/dapi-container-resources.yaml b/docs/user-guide/downward-api/dapi-container-resources.yaml new file mode 100644 index 0000000000..2a3abb0145 --- /dev/null +++ b/docs/user-guide/downward-api/dapi-container-resources.yaml @@ -0,0 +1,34 @@ +apiVersion: v1 +kind: Pod +metadata: + name: dapi-test-pod +spec: + containers: + - name: test-container + image: gcr.io/google_containers/busybox:1.24 + command: [ "/bin/sh", "-c", "env" ] + resources: + requests: + memory: "32Mi" + cpu: "125m" + limits: + memory: "64Mi" + cpu: "250m" + env: + - name: MY_CPU_REQUEST + valueFrom: + resourceFieldRef: + resource: requests.cpu + - name: MY_CPU_LIMIT + valueFrom: + resourceFieldRef: + resource: limits.cpu + - name: MY_MEM_REQUEST + valueFrom: + resourceFieldRef: + resource: requests.memory + - name: MY_MEM_LIMIT + valueFrom: + resourceFieldRef: + resource: limits.memory + restartPolicy: Never diff --git a/docs/user-guide/downward-api/index.md b/docs/user-guide/downward-api/index.md index bbc0bf58f0..eb9819ef14 100644 --- a/docs/user-guide/downward-api/index.md +++ b/docs/user-guide/downward-api/index.md @@ -21,6 +21,10 @@ The following information is available to a `Pod` through the downward API: * The pod's name * The pod's namespace * The pod's IP +* A container's cpu limit +* A container's cpu request +* A container's memory limit +* A container's memory request More information will be exposed through this same API over time. @@ -31,13 +35,15 @@ Containers consume information from the downward API using environment variables or using a volume plugin. -### Environment variables +## Environment variables Most environment variables in the Kubernetes API use the `value` field to carry simple values. However, the alternate `valueFrom` field allows you to specify -a `fieldRef` to select fields from the pod's definition. The `fieldRef` field -is a structure that has an `apiVersion` field and a `fieldPath` field. The -`fieldPath` field is an expression designating a field of the pod. The +a `fieldRef` to select fields from the pod's definition, and a `resourceFieldRef` +to select fields from one of its container's definition. + +The `fieldRef` field is a structure that has an `apiVersion` field and a `fieldPath` +field. The `fieldPath` field is an expression designating a field of the pod. The `apiVersion` field is the version of the API schema that the `fieldPath` is written in terms of. If the `apiVersion` field is not specified it is defaulted to the API version of the enclosing object. @@ -46,20 +52,39 @@ The `fieldRef` is evaluated and the resulting value is used as the value for the environment variable. This allows users to publish their pod's name in any environment variable they want. +The `resourceFieldRef` is a structure that has a `containerName` field, a `resource` +field, and a `divisor` field. The `containerName` is the name of a container, +whose resource (cpu or memory) information is to be exposed. The `containerName` is +optional for environment variables and defaults to the current container. The +`resource` field is an expression designating a resource in a container, and the `divisor` +field specifies an output format of the resource being exposed. If the `divisor` +is not specified, it defaults to "1" for cpu and memory. The table shows possible +values for cpu and memory resources for `resource` and `divisor` settings: -## Example + +| Setting | Cpu | Memory | +| ------------- |-------------| -----| +| resource | limits.cpu, requests.cpu| limits.memory, requests.memory| +| divisor | 1(cores), 1m(millicores) | 1(bytes), 1k(kilobytes), 1M(megabytes), 1G(gigabytes), 1T(terabytes), 1P(petabytes), 1E(exabytes), 1Ki(kibibyte), 1Mi(mebibyte), 1Gi(gibibyte), 1Ti(tebibyte), 1Pi(pebibyte), 1Ei(exbibyte)| + + +### Example This is an example of a pod that consumes its name and namespace via the downward API: {% include code.html language="yaml" file="dapi-pod.yaml" ghlink="/docs/user-guide/downward-api/dapi-pod.yaml" %} +This is an example of a pod that consumes its container's resources via the downward API: -### Downward API volume +{% include code.html language="yaml" file="dapi-container-resources.yaml" ghlink="/docs/user-guide/downward-api/dapi-container-resources.yaml" %} + +## Downward API volume Using a similar syntax it's possible to expose pod information to containers using plain text files. Downward API are dumped to a mounted volume. This is achieved using a `downwardAPI` volume type and the different items represent the files to be created. `fieldPath` references the field to be exposed. +For exposing a container's resources limits and requests, `containerName` must be specified with `resourceFieldRef`. Downward API volume permits to store more complex data like [`metadata.labels`](/docs/user-guide/labels) and [`metadata.annotations`](/docs/user-guide/annotations). Currently key/value pair set fields are saved using `key="value"` format: @@ -76,19 +101,30 @@ Downward API volumes can expose: * The pod's namespace * The pod's labels * The pod's annotations +* A container's cpu limit +* A container's cpu request +* A container's memory limit +* A container's memory request The downward API volume refreshes its data in step with the kubelet refresh loop. When labels will be modifiable on the fly without respawning the pod containers will be able to detect changes through mechanisms such as [inotify](https://en.wikipedia.org/wiki/Inotify). In future, it will be possible to specify a specific annotation or label. -## Example +### Example This is an example of a pod that consumes its labels and annotations via the downward API volume, labels and annotations are dumped in `/etc/labels` and in `/etc/annotations`, respectively: {% include code.html language="yaml" file="volume/dapi-volume.yaml" ghlink="/docs/user-guide/downward-api/volume/dapi-volume.yaml" %} +This is an example of a pod that consumes its container's resources via the downward API volume. + +{% include code.html language="yaml" file="volume/dapi-volume-resources.yaml" ghlink="/docs/user-guide/downward-api/volume/dapi-volume-resources.yaml" %} Some more thorough examples: * [environment variables](/docs/user-guide/environment-guide/) * [downward API](/docs/user-guide/downward-api/) + +## Default values for container resource limits + +If cpu and memory limits are not specified for a container, the downward API will default to node's cpu and memory capacities. diff --git a/docs/user-guide/downward-api/volume/dapi-volume-resources.yaml b/docs/user-guide/downward-api/volume/dapi-volume-resources.yaml new file mode 100644 index 0000000000..f28bb99e3f --- /dev/null +++ b/docs/user-guide/downward-api/volume/dapi-volume-resources.yaml @@ -0,0 +1,40 @@ +apiVersion: v1 +kind: Pod +metadata: + name: kubernetes-downwardapi-volume-example +spec: + containers: + - name: client-container + image: gcr.io/google_containers/busybox:1.24 + command: ["sh", "-c", "while true; do echo; if [[ -e /etc/cpu_limit ]]; then cat /etc/cpu_limit; fi; if [[ -e /etc/cpu_request ]]; then cat /etc/cpu_request; fi; if [[ -e /etc/mem_limit ]]; then cat /etc/mem_limit; fi; if [[ -e /etc/mem_request ]]; then cat /etc/mem_request; fi; sleep 5; done"] + resources: + requests: + memory: "32Mi" + cpu: "125m" + limits: + memory: "64Mi" + cpu: "250m" + volumeMounts: + - name: podinfo + mountPath: /etc + readOnly: false + volumes: + - name: podinfo + downwardAPI: + items: + - path: "cpu_limit" + resourceFieldRef: + containerName: client-container + resource: limits.cpu + - path: "cpu_request" + resourceFieldRef: + containerName: client-container + resource: requests.cpu + - path: "mem_limit" + resourceFieldRef: + containerName: client-container + resource: limits.memory + - path: "mem_request" + resourceFieldRef: + containerName: client-container + resource: requests.memory diff --git a/docs/user-guide/downward-api/volume/index.md b/docs/user-guide/downward-api/volume/index.md index 3cfdc2c2d9..46f54d0776 100644 --- a/docs/user-guide/downward-api/volume/index.md +++ b/docs/user-guide/downward-api/volume/index.md @@ -2,7 +2,7 @@ --- Following this example, you will create a pod with a downward API volume. -A downward API volume is a k8s volume plugin with the ability to save some pod information in a plain text file. The pod information can be for example some [metadata](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/devel/api-conventions.md#metadata). +A downward API volume is a k8s volume plugin with the ability to save some pod information in a plain text file. The pod information can be for example some [metadata](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/devel/api-conventions.md#metadata) or a container's [resources](/docs/user-guide/compute-resources). Supported metadata fields: @@ -11,6 +11,13 @@ Supported metadata fields: 3. `metadata.name` 4. `metadata.labels` +Supported container's resources: + +1. `limits.cpu` +2. `limits.memory` +3. `requests.cpu` +4. `requests.memory` + ### Step Zero: Prerequisites This example assumes you have a Kubernetes cluster installed and running, and the `kubectl` command line tool somewhere in your path. Please see the [gettingstarted](/docs/getting-started-guides/) for installation instructions for your platform. @@ -64,3 +71,48 @@ drwxrwxrwt 3 0 0 120 Jun 1 19:55 .. ``` The file `labels` is stored in a temporary directory (`..6986_01_06_15_55_10.473583074` in the example above) which is symlinked to by `..data`. Symlinks for annotations and labels in `/etc` point to files containing the actual metadata through the `..data` indirection.  This structure allows for dynamic atomic refresh of the metadata: updates are written to a new temporary directory, and the `..data` symlink is updated atomically using `rename(2)`. + +## Example of downward API volume with container resources + +Use the `docs/user-guide/downward-api/volume/dapi-volume-resources.yaml` file to create a Pod with a downward API volume which stores its container's limits and requests in /etc. + +```shell +$ kubectl create -f docs/user-guide/downward-api/volume/dapi-volume-resources.yaml +``` + +### Examine pod/container output + +In pod's `/etc` directory one may find the files created by the plugin: + +```shell +$ kubectl exec kubernetes-downwardapi-volume-example -i -t -- sh +/ # ls -alR /etc +/etc: +total 4 +drwxrwxrwt 3 0 0 160 Jun 1 19:47 . +drwxr-xr-x 17 0 0 4096 Jun 1 19:48 .. +drwxr-xr-x 2 0 0 120 Jun 1 19:47 ..6986_01_06_15_47_23.076909525 +lrwxrwxrwx 1 0 0 31 Jun 1 19:47 ..data -> ..6986_01_06_15_47_23.076909525 +lrwxrwxrwx 1 0 0 16 Jun 1 19:47 cpu_limit -> ..data/cpu_limit +lrwxrwxrwx 1 0 0 18 Jun 1 19:47 cpu_request -> ..data/cpu_request +lrwxrwxrwx 1 0 0 16 Jun 1 19:47 mem_limit -> ..data/mem_limit +lrwxrwxrwx 1 0 0 18 Jun 1 19:47 mem_request -> ..data/mem_request + +/etc/..6986_01_06_15_47_23.076909525: +total 16 +drwxr-xr-x 2 0 0 120 Jun 1 19:47 . +drwxrwxrwt 3 0 0 160 Jun 1 19:47 .. +-rw-r--r-- 1 0 0 1 Jun 1 19:47 cpu_limit +-rw-r--r-- 1 0 0 1 Jun 1 19:47 cpu_request +-rw-r--r-- 1 0 0 8 Jun 1 19:47 mem_limit +-rw-r--r-- 1 0 0 8 Jun 1 19:47 mem_request + +/ # cat /etc/cpu_limit +1 +/ # cat /etc/mem_limit +67108864 +/ # cat /etc/cpu_request +1 +/ # cat /etc/mem_request +33554432 +``` From f982e21d5b0004e30b05495b7ce996f88fd2c8de Mon Sep 17 00:00:00 2001 From: derekwaynecarr Date: Mon, 6 Jun 2016 14:57:50 -0400 Subject: [PATCH 112/348] Update ResourceQuota documentation for current code --- docs/admin/resourcequota/index.md | 185 ++++++++++++++++++++---------- 1 file changed, 122 insertions(+), 63 deletions(-) diff --git a/docs/admin/resourcequota/index.md b/docs/admin/resourcequota/index.md index 40559f058e..a3a4dab246 100644 --- a/docs/admin/resourcequota/index.md +++ b/docs/admin/resourcequota/index.md @@ -4,29 +4,26 @@ When several users or teams share a cluster with a fixed number of nodes, there is a concern that one team could use more than its fair share of resources. -Resource quotas are a tool for administrators to address this concern. Resource quotas -work like this: +Resource quotas are a tool for administrators to address this concern. + +A resource quota, defined by a `ResourceQuota` object, provides constraints that limit +aggregate resource consumption per namespace. It can limit the quantity of objects that can +be created in a namespace by type, as well as the total amount of compute resources that may +be consumed by resources in that project. + +Resource quotas work like this: - Different teams work in different namespaces. Currently this is voluntary, but support for making this mandatory via ACLs is planned. -- The administrator creates a Resource Quota for each namespace. -- Users put compute resource requests on their pods. The sum of all resource requests across - all pods in the same namespace must not exceed any hard resource limit in any Resource Quota - document for the namespace. Note that we used to verify Resource Quota by taking the sum of - resource limits of the pods, but this was altered to use resource requests. Backwards compatibility - for those pods previously created is preserved because pods that only specify a resource limit have - their resource requests defaulted to match their defined limits. The user is only charged for the - resources they request in the Resource Quota versus their limits because the request is the minimum - amount of resource guaranteed by the cluster during scheduling. For more information on over commit, - see [compute-resources](/docs/user-guide/compute-resources). -- If creating a pod would cause the namespace to exceed any of the limits specified in the - the Resource Quota for that namespace, then the request will fail with HTTP status - code `403 FORBIDDEN`. -- If quota is enabled in a namespace and the user does not specify *requests* on the pod for each - of the resources for which quota is enabled, then the POST of the pod will fail with HTTP - status code `403 FORBIDDEN`. Hint: Use the LimitRange admission controller to force default - values of *limits* (then resource *requests* would be equal to *limits* by default, see - [admission controller](/docs/admin/admission-controllers)) before the quota is checked to avoid this problem. +- The administrator creates one or more Resource Quota objects for each namespace. +- Users create resources (pods, services, etc.) in the namespace, and the quota system + tracks usage to ensure it does not exceed hard resource limits defined in a Resource Quota. +- If creating or updating a resource violates a quota constraint, the request will fail with HTTP + status code `403 FORBIDDEN` with a message explaining the constraint that would have been violated. +- If quota is enabled in a namespace for compute resources like `cpu` and `memory`, users must specify + requests or limits for those values; otherwise, the quota system may reject pod creation. Hint: Use + the LimitRange admission controller to force defaults for pods that make no compute resource requirements. + See the [walkthrough](/docs/admin/resourcequota/walkthrough.md) for an example to avoid this problem. Examples of policies that could be created using namespaces and quotas are: @@ -38,7 +35,7 @@ Examples of policies that could be created using namespaces and quotas are: In the case where the total capacity of the cluster is less than the sum of the quotas of the namespaces, there may be contention for resources. This is handled on a first-come-first-served basis. -Neither contention nor changes to quota will affect already-running pods. +Neither contention nor changes to quota will affect already created resources. ## Enabling Resource Quota @@ -57,11 +54,12 @@ in a namespace can be limited. The following compute resource types are support | ResourceName | Description | | ------------ | ----------- | -| cpu | Total cpu requests of containers | -| memory | Total memory requests of containers - -For example, `cpu` quota sums up the `resources.requests.cpu` fields of every -container of every pod in the namespace, and enforces a maximum on that sum. +| cpu | Across all pods in a non-terminal state, the sum of CPU requests cannot exceed this value. | +| limits.cpu | Across all pods in a non-terminal state, the sum of CPU limits cannot exceed this value. | +| limits.memory | Across all pods in a non-terminal state, the sum of memory limits cannot exceed this value. | +| memory | Across all pods in a non-terminal state, the sum of memory requests cannot exceed this value. | +| requests.cpu | Across all pods in a non-terminal state, the sum of CPU requests cannot exceed this value. | +| requests.memory | Across all pods in a non-terminal state, the sum of memory requests cannot exceed this value. | ## Object Count Quota @@ -70,12 +68,13 @@ are supported: | ResourceName | Description | | ------------ | ----------- | -| pods | Total number of pods | -| services | Total number of services | -| replicationcontrollers | Total number of replication controllers | -| resourcequotas | Total number of [resource quotas](/docs/admin/admission-controllers/#resourcequota) | -| secrets | Total number of secrets | -| persistentvolumeclaims | Total number of [persistent volume claims](/docs/user-guide/persistent-volumes/#persistentvolumeclaims) | +| configmaps | The total number of config maps that can exist in the namespace. | +| persistentvolumeclaims | The total number of [persistent volume claims](/docs/user-guide/persistent-volumes/#persistentvolumeclaims) that can exist in the namespace. | +| pods | The total number of pods in a non-terminal state that can exist in the namespace. A pod is in a terminal state if `status.phase in (Failed, Succeeded)` is true. | +| replicationcontrollers | The total number of replication controllers that can exist in the namespace. | +| resourcequotas | The total number of [resource quotas](/docs/admin/admission-controllers/#resourcequota) that can exist in the namespace. | +| services | The total number of services that can exist in the namespace. | +| secrets | The total number of secrets that can exist in the namespace. | For example, `pods` quota counts and enforces a maximum on the number of `pods` created in a single namespace. @@ -84,45 +83,105 @@ You might want to set a pods quota on a namespace to avoid the case where a user creates many small pods and exhausts the cluster's supply of Pod IPs. +## Quota Scopes + +Each quota can have an associated set of scopes. A quota will only measure usage for a resource if it matches +the intersection of enumerated scopes. + +When a scope is added to the quota, it limits the number of resources it supports to those that pertain to the scope. +Resources specified on the quota outside of the allowed set results in a validation error. + +| Scope | Description | +| ----- | ----------- | +| Terminating | Match pods where `spec.activeDeadlineSeconds >= 0` | +| NotTerminating | Match pods where `spec.activeDeadlineSeconds is nil` | +| BestEffort | Match pods that have best effort quality of service. | +| NotBestEffort | Match pods that do not have best effort quality of service. | + +The `BestEffort` scope restricts a quota to tracking the following resources: +* pods + +The `Terminating`, `NotTerminating`, and `NotBestEffort` scopes restrict a quota to tracking the following resources: +* cpu +* limits.cpu +* limits.memory +* memory +* pods +* requests.cpu +* requests.memory + +## Requests vs Limits + +When allocating compute resources, each container may specify a request and a limit value for either CPU or memory. +The quota can be configured to quota either value. + +If the quota has a value specified for `requests.cpu` or `requests.memory`, then it requires that every incoming +container makes an explicit request for those resources. If the quota has a value specified for `limits.cpu` or `limits.memory`, +then it requires that every incoming container specifies an explict limit for those resources. + ## Viewing and Setting Quotas Kubectl supports creating, updating, and viewing quotas: ```shell -$ kubectl namespace myspace -$ cat < quota.json -{ - "apiVersion": "v1", - "kind": "ResourceQuota", - "metadata": { - "name": "quota" - }, - "spec": { - "hard": { - "memory": "1Gi", - "cpu": "20", - "pods": "10", - "services": "5", - "replicationcontrollers":"20", - "resourcequotas":"1" - } - } -} +$ kubectl create namespace myspace + +$ cat < compute-resources.yaml +apiVersion: v1 +kind: ResourceQuota +metadata: + name: compute-resources +spec: + hard: + pods: "4" + requests.cpu: "1" + requests.memory: 1Gi + limits.cpu: "2" + limits.memory: 2Gi EOF -$ kubectl create -f ./quota.json -$ kubectl get quota -NAME -quota -$ kubectl describe quota quota -Name: quota +$ kubectl create -f ./compute-resources.yaml --namespace=myspace + +$ cat < object-counts.yaml +apiVersion: v1 +kind: ResourceQuota +metadata: + name: object-counts +spec: + hard: + configmaps: "10" + persistentvolumeclaims: "4" + replicationcontrollers: "20" + secrets: "10" + services: "10" +EOF +$ kubectl create -f ./object-counts.yaml --namespace=myspace + +$ kubectl get quota --namespace=myspace +NAME AGE +compute-resources 30s +object-counts 32s + +$ kubectl describe quota compute-resources --namespace=myspace +Name: compute-resources +Namespace: myspace +Resource Used Hard +-------- ---- ---- +limits.cpu 0 2 +limits.memory 0 2Gi +pods 0 4 +requests.cpu 0 1 +requests.memory 0 1Gi + +$ kubectl describe quota object-counts --namespace=myspace +Name: object-counts +Namespace: myspace Resource Used Hard -------- ---- ---- -cpu 0m 20 -memory 0 1Gi -pods 5 10 -replicationcontrollers 5 20 -resourcequotas 1 1 -services 3 5 +configmaps 0 10 +persistentvolumeclaims 0 4 +replicationcontrollers 0 20 +secrets 1 10 +services 0 10 ``` ## Quota and Cluster Capacity From 53e888aa9c78254ed6b3598ac29b713df98e5147 Mon Sep 17 00:00:00 2001 From: Yifan Gu Date: Fri, 24 Jun 2016 11:04:09 -0700 Subject: [PATCH 113/348] rkt: Update the getting-started-guide and known issues. --- _data/guides.yml | 4 +- docs/getting-started-guides/rkt/index.md | 217 ++++++++++------------- docs/getting-started-guides/rkt/notes.md | 131 +++++++------- 3 files changed, 153 insertions(+), 199 deletions(-) diff --git a/_data/guides.yml b/_data/guides.yml index f5654a6efd..6f21a5749e 100644 --- a/_data/guides.yml +++ b/_data/guides.yml @@ -190,9 +190,9 @@ toc: path: /docs/getting-started-guides/fedora/fedora-calico/ - title: rkt section: - - title: Running Kubernetes on rkt + - title: Running Kubernetes with rkt path: /docs/getting-started-guides/rkt/ - - title: Notes on Different UX with rkt Container Runtime + - title: Known Issues when Using rkt path: /docs/getting-started-guides/rkt/notes/ - title: Kubernetes on Mesos path: /docs/getting-started-guides/mesos/ diff --git a/docs/getting-started-guides/rkt/index.md b/docs/getting-started-guides/rkt/index.md index e644b52462..764ed2e0e5 100644 --- a/docs/getting-started-guides/rkt/index.md +++ b/docs/getting-started-guides/rkt/index.md @@ -2,108 +2,100 @@ --- This document describes how to run Kubernetes using [rkt](https://github.com/coreos/rkt) as a container runtime. -We still have [a bunch of work](http://issue.k8s.io/8262) to do to make the experience with rkt wonderful, please stay tuned! ### **Prerequisite** -- [systemd](http://www.freedesktop.org/wiki/Software/systemd/) should be installed on the machine and should be enabled. The minimum version required at this moment (2015/09/01) is 219 +- [systemd](http://www.freedesktop.org/wiki/Software/systemd/) should be installed on the machine and should be enabled. + The minimum version required for Kubernetes 1.3 is `219`. *(Note that systemd is not required by rkt itself, we are using it here to monitor and manage the pods launched by kubelet.)* - Install the latest rkt release according to the instructions [here](https://github.com/coreos/rkt). - The minimum version required for now is [v0.8.0](https://github.com/coreos/rkt/releases/tag/v0.8.0). + The minimum version required is [v1.9.1](https://github.com/coreos/rkt/releases/tag/v1.9.1). -- Note that for rkt version later than v0.7.0, `metadata service` is not required for running pods in private networks. So now rkt pods will not register the metadata service be default. +- The [rkt API service](http://coreos.com/rkt/docs/latest/subcommands/api-service.html) must be running on the node. -- Since release [v1.2.0-alpha.5](https://github.com/kubernetes/kubernetes/releases/tag/v1.2.0-alpha.5), -the [rkt API service](https://github.com/coreos/rkt/blob/master/api/v1alpha/README.md) -must be running on the node. +### Setup network -### Network Setup +You can configure the Kubernetes networking using its own `kubenet` and `CNI` [network +plugins](http://kubernetes.io/docs/admin/network-plugins/) by setting the kubelet's `--network-plugin` and `--network-plugin-dir` flag. +In addition, rkt supports using rkt's [Contained Networking](https://coreos.com/rkt/docs/latest/networking.html#contained-mode). -rkt uses the [Container Network Interface (CNI)](https://github.com/appc/cni) -to manage container networking. By default, all pods attempt to join a network -called `rkt.kubernetes.io`, which is currently defined [in -`rkt.go`](https://github.com/kubernetes/kubernetes/blob/v1.2.0-alpha.6/pkg/kubelet/rkt/rkt.go#L91). -In order for pods to get correct IP addresses, the CNI config file must be -edited to add this `rkt.kubernetes.io` network: +##### Use rkt's Contained Networking -#### Using flannel +In this mode, rkt will attempt to join pods into a network named `rkt.kubernetes.io`. +To use rkt's contained networking, you can leave the `--network-plugin` to empty, and put a network config file under one of the rkt's [config directories](https://github.com/coreos/rkt/blob/master/Documentation/configuration.md#command-line-flags), for example: -In addition to the basic prerequisites above, each node must be running -a [flannel](https://github.com/coreos/flannel) daemon. This implies -that a flannel-supporting etcd service must be available to the cluster -as well, apart from the Kubernetes etcd, which will not yet be -available at flannel configuration time. Once it's running, flannel can -be set up with a CNI config like: - -```shell -$ cat </etc/rkt/net.d/k8s_cluster.conf +```shell +$ cat </etc/rkt/net.d/k8s_network_example.conf { - "name": "rkt.kubernetes.io", - "type": "flannel" -} -EOF -``` - -While `k8s_cluster.conf` is a rather arbitrary name for the config file itself, -and can be adjusted to suit local conventions, the keys and values should be exactly -as shown above. `name` must be `rkt.kubernetes.io` and `type` should be `flannel`. -More details about the flannel CNI plugin can be found -[in the CNI documentation](https://github.com/appc/cni/blob/master/Documentation/flannel.md). - -#### On GCE - -Each VM on GCE has an additional 256 IP addresses routed to it, so -it is possible to forego flannel in smaller clusters. This makes the -necessary CNI config file a bit more verbose: - -```shell -$ cat </etc/rkt/net.d/k8s_cluster.conf -{ - "name": "rkt.kubernetes.io", - "type": "bridge", - "bridge": "cbr0", - "isGateway": true, - "ipam": { - "type": "host-local", - "subnet": "10.255.228.1/24", - "gateway": "10.255.228.1" - }, + "name": "rkt.kubernetes.io", + "type": "bridge", + "bridge": "mybridge", + "mtu": 1460, + "addIf": "true", + "isGateway": true, + "ipMasq": true, + "ipam": { + "type": "host-local", + "subnet": "10.22.0.0/16", + "gateway": "10.22.0.1", "routes": [ { "dst": "0.0.0.0/0" } ] + } } EOF ``` -This example creates a `bridge` plugin configuration for the CNI network, specifying -the bridge name `cbr0`. It also specifies the CIDR, in the `ipam` field. +However, there are a small number of caveats you should be aware of when using rkt's networking: -Creating these files for any moderately-sized cluster is at best inconvenient. -Work is in progress to -[enable Kubernetes to use the CNI by default] -(https://github.com/kubernetes/kubernetes/pull/18795/files). -As that work matures, such manual CNI config munging will become unnecessary -for primary use cases. For early adopters, an initial example shows one way to -[automatically generate these CNI configurations] -(https://gist.github.com/yifan-gu/fbb911db83d785915543) -for rkt. +* You must create an appropriate CNI configuration file with a network name of `rkt.kubernetes.io`. +* The downwards API and environment variable substitution will not contain the pod IP. +* The `/etc/hosts` file will not contain your own hostname (though `/etc/hostname` is populated). -### Local cluster +##### Use flannel + +While it's recommended that you configure flannel using kubernetes' CNI support, you can also configure it using rkt's contained networking. +An example flannel/CNI config file looks like this: + +```shell +$ cat </etc/rkt/net.d/k8s_flannel_example.conf +{ + "name": "rkt.kubernetes.io", + "type": "flannel", + "delegate": { + "isDefaultGateway": true + } +} +EOF +``` + +For more information on flannel configuration, please read [CNI/flannel README](https://github.com/containernetworking/cni/blob/master/Documentation/flannel.md). + +##### Use Google Compute Engine (GCE) network + +Each VM on GCE has an additional 256 IP addresses routed to it, so it is possible to forego flannel in smaller clusters. +This can most easily be done by using the builtin kubenet plugin. +This can be done by setting `--network-plugin=kubenet`. + +### Launch a local cluster To use rkt as the container runtime, we need to supply the following flags to kubelet: -- `--container-runtime=rkt` chooses the container runtime to use. Possible values: 'docker', 'rkt'. Default: 'docker'. -- `--rkt-path=$PATH_TO_RKT_BINARY` sets the path of rkt binary. Leave empty to use the first rkt in $PATH. -- `--rkt-stage1-image` sets the path of the stage1 image. Local paths and http/https URLs are supported. Leave empty to use the 'stage1.aci' that locates in the same directory as the rkt binary. +- `--container-runtime=rkt` chooses the container runtime to use. +- `--rkt-api-endpoint=HOST:PORT` sets the endpoint of the rkt API service. + Leave empty to use the default one (`localhost:15441`). +- `--rkt-path=$PATH_TO_RKT_BINARY` sets the path of rkt binary. + Leave empty to use the first rkt in $PATH. +- `--rkt-stage1-image` sets the name of the stage1 image, e.g. coreos.com/rkt/stage1-coreos. + Leave empty to use the default stage1 image in the rkt's configuration. -If you are using the [hack/local-up-cluster.sh](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/hack/local-up-cluster.sh) script to launch the local cluster, then you can edit the environment variable `CONTAINER_RUNTIME`, `RKT_PATH` and `RKT_STAGE1_IMAGE` to -set these flags: +If you are using the [hack/local-up-cluster.sh](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/hack/local-up-cluster.sh) script to launch the local cluster, then you can edit the environment variable `CONTAINER_RUNTIME`, `RKT_PATH` and `RKT_STAGE1_IMAGE` to set these flags, the `RKT_PATH` and `RKT_STAGE1_IMAGE` are optional if you have `rkt` in your `$PATH` with appropriate configuration. ```shell $ export CONTAINER_RUNTIME=rkt $ export RKT_PATH=$PATH_TO_RKT_BINARY -$ export RKT_STAGE1_IMAGE=PATH=$PATH_TO_STAGE1_IMAGE +$ export RKT_STAGE1_IMAGE=$NAME_OF_THE_STAGE1_IMAGE ``` Then we can launch the local cluster using the script: @@ -112,21 +104,26 @@ Then we can launch the local cluster using the script: $ hack/local-up-cluster.sh ``` -### CoreOS cluster on Google Compute Engine (GCE) +We are also working on set up rkt as the container runtime for [minikube](https://github.com/kubernetes/minikube/issues/168). -To use rkt as the container runtime for your CoreOS cluster on GCE, you need to specify the OS distribution, project, image: +### Launch a CoreOS/rkt cluster on Google Compute Engine (GCE) + +Here we provide instruction on how to use the `kube-up` script to launch a CoreOS/rkt cluster on GCE. +In order to do that, you need to specify the OS distribution, project, image: ```shell $ export KUBE_OS_DISTRIBUTION=coreos -$ export KUBE_GCE_NODE_IMAGE= +$ export KUBE_GCE_MASTER_PROJECT=coreos-cloud +$ export KUBE_GCE_MASTER_IMAGE= $ export KUBE_GCE_NODE_PROJECT=coreos-cloud +$ export KUBE_GCE_NODE_IMAGE= $ export KUBE_CONTAINER_RUNTIME=rkt ``` You can optionally choose the version of rkt used by setting `KUBE_RKT_VERSION`: ```shell -$ export KUBE_RKT_VERSION=0.15.0 +$ export KUBE_RKT_VERSION=1.9.1 ``` Then you can launch the cluster by: @@ -135,52 +132,25 @@ Then you can launch the cluster by: $ cluster/kube-up.sh ``` -Note that we are still working on making all containerized the master components run smoothly in rkt. Before that we are not able to run the master node with rkt yet. +### Launch a CoreOS/rkt cluster on AWS -### CoreOS cluster on AWS +`kube-up` for AWS is currently unsupported. +Instead, we recommend you to refer the [Kubernetes on AWS guide](https://coreos.com/kubernetes/docs/latest/kubernetes-on-aws.html) to launch a CoreOS/rkt cluster on AWS. -To use rkt as the container runtime for your CoreOS cluster on AWS, you need to specify the provider and OS distribution: +### Deploy apps to your cluster -```shell -$ export KUBERNETES_PROVIDER=aws -$ export KUBE_OS_DISTRIBUTION=coreos -$ export KUBE_CONTAINER_RUNTIME=rkt -``` +After you created the cluster, you can start deploying apps to the cluster. For example here is how you can [deploy a simgle nginx app](/docs/user-guide/simple-nginx). +More examples can be found in the [examples directory](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/). -You can optionally choose the version of rkt used by setting `KUBE_RKT_VERSION`: +### Known Issues and Differences -```shell -$ export KUBE_RKT_VERSION=0.8.0 -``` - -You can optionally choose the CoreOS channel by setting `COREOS_CHANNEL`: - -```shell -$ export COREOS_CHANNEL=stable -``` - -Then you can launch the cluster by: - -```shell -$ kube-up.sh -``` - -Note: CoreOS is not supported as the master using the automated launch -scripts. The master node is always Ubuntu. - -### Getting started with your cluster - -See [a simple nginx example](/docs/user-guide/simple-nginx) to try out your new cluster. - -For more complete applications, please look in the [examples directory](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/). - -### Different UX with rkt container runtime - -rkt and Docker have very different designs, as well as ACI and Docker image format. Users might experience some different experience when switching from one to the other. More information can be found [here](/docs/getting-started-guides/rkt/notes/). +rkt and Docker have very different designs, as well as ACI and Docker image format. +Users might experience some different experience when switching from one to the other. +More information can be found [here](/docs/getting-started-guides/rkt/notes/). ### Debugging -Here are several tips for you when you run into any issues. +Here are several tips in case you run into any issues. ##### Check logs @@ -191,29 +161,20 @@ If the cluster is using salt, we can edit the [logging.sls](https://github.com/k ##### Check rkt pod status To check the pods' status, we can use rkt command, such as `rkt list`, `rkt status`, `rkt image list`, etc. -More information about rkt command line can be found [here](https://github.com/coreos/rkt/blob/master/Documentation/commands.md) +More information about rkt command line can be found [here](https://github.com/coreos/rkt/blob/master/Documentation/commands.md). ##### Check journal logs -As we use systemd to launch rkt pods(by creating service files which will run `rkt run-prepared`, we can check the pods' log -using `journalctl`: +As we use systemd to launch/manage rkt pods, we can check the pods' log using `journalctl`: - Check the running state of the systemd service: ```shell -$ sudo journalctl -u $SERVICE_FILE +$ sudo journalctl -u ${SERVICE_NAME} ``` -where `$SERVICE_FILE` is the name of the service file created for the pod, you can find it in the kubelet logs. - -##### Check the log of the container in the pod: - -```shell -$ sudo journalctl -M rkt-$UUID -u $CONTAINER_NAME -``` - -where `$UUID` is the rkt pod's UUID, which you can find via `rkt list --full`, and `$CONTAINER_NAME` is the container's name. +where `${SERVICE_NAME}` is the name of the service file created for the pod, typically the format is `k8s_${RKT_UUID}`. ##### Check Kubernetes events, logs. -Besides above tricks, Kubernetes also provides us handy tools for debugging the pods. More information can be found [here](/docs/user-guide/application-troubleshooting) +Kubernetes also provides various tools for debugging. More information can be found [here](/docs/user-guide/application-troubleshooting). diff --git a/docs/getting-started-guides/rkt/notes.md b/docs/getting-started-guides/rkt/notes.md index 44eb57f90e..61e9676ba4 100644 --- a/docs/getting-started-guides/rkt/notes.md +++ b/docs/getting-started-guides/rkt/notes.md @@ -1,99 +1,92 @@ --- --- -# Notes on Different UX with rkt container runtime +# Known issues -### Doesn't support ENTRYPOINT + CMD feature +The following features either are not supported or have large caveats when using the rkt container runtime. -To run a Docker image, rkt will convert it into [App Container Image (ACI) format](https://github.com/appc/spec/blob/master/SPEC.md) first. -However, during the conversion, the `ENTRYPOINT` and `CMD` are concatentated to construct ACI's `Exec` field. -This means after the conversion, we are not able to replace only `ENTRYPOINT` or `CMD` without touching the other part. -So for now, users are recommended to specify the **executable path** in `Command` and **arguments** in `Args`. -(This has the same effect if users specify the **executable path + arguments** in `Command` or `Args` alone). +##### Non-existent host volume paths -For example: +When mounting a host volume path that does not exist, rkt will error out. Under the Docker runtime, an empty directory will be created at the referenced path. + +An example of a pod which will error out: ```yaml apiVersion: v1 kind: Pod metadata: - name: nginx labels: - name: nginx + name: mount-dne + name: mount-dne spec: + volumes: + - name: does-not-exist + hostPath: + path: /does/not/exist containers: - - name: nginx - image: nginx - ports: - - containerPort: 80 + - name: exit + image: busybox + command: ["sh", "-c", "ls /test; sleep 60"] + volumeMounts: + - mountPath: /test + name: does-not-exist ``` -The above pod yaml file is valid as it's not specifying `Command` or `Args`, so the default `ENTRYPOINT` and `CMD` of the image will be used. +##### Kubectl attach + +The `kubectl attach` command does not work under the rkt container runtime. +And because of this, some flags in `kubectl run` are not supported, including: + +- --attach=true +- --leave-stdin-open=true +- --rm=true + +##### Init containers + +The alpha [init container](https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/container-init.md) feature is currently not supported. + +##### Container restart back-off + +Exponential restart back-off for a failing container is currently not supported. + +##### Experimental NVIDIA GPU support + +The `--experimental-nvidia-gpus` flag, and related [GPU features](https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/gpu-support.md) are not supported. + +##### QoS Classes + +Under rkt, QoS classes do not result in the `OOM Score` of containers being adjusted as occurs under Docker. + +##### HostPID and HostIPC namespaces + +Setting a the hostPID or hostIPC flag on a pod is not supported. + +For example, the following pod will not run correctly: ```yaml apiVersion: v1 kind: Pod metadata: - name: busybox labels: - name: busybox + name: host-ipc-pid + name: host-ipc-pid spec: + hostIPC: true + hostPID: true containers: - - name: busybox - image: busybox - command: - - /bin/sleep - - 1000 + ... ``` -```yaml -apiVersion: v1 -kind: Pod -metadata: - name: busybox - labels: - name: busybox -spec: - containers: - - name: busybox - image: busybox - command: - - /bin/sleep - args: - - 1000 -``` +On the other hand, when running the pod with [stage1-fly](https://coreos.com/rkt/docs/latest/running-fly-stage1.html), the pod will be run in the host namespace. -```yaml -apiVersion: v1 -kind: Pod -metadata: - name: busybox - labels: - name: busybox -spec: - containers: - - name: busybox - image: busybox - args: - - /bin/sleep - - 1000 -``` -All the three examples above are valid as they contain both the executable path and the arguments. +##### Container image updates (patch) -```yaml -apiVersion: v1 -kind: Pod -metadata: - name: busybox - labels: - name: busybox -spec: - containers: - - name: busybox - image: busybox - args: - - 1000 -``` +Patching a pod to change the image will result in the entire pod restarting, not just the container that was changed. -The last example is invalid, as we cannot override just the `CMD` of the image alone. +##### Volume mounts specifying a subPath + +The [subPath](https://github.com/kubernetes/kubernetes/pull/22575) feature does not work correctly under rkt. +In addition, the above-issue of Non-existent host volume paths being invalid would make many common use-cases for subPaths fail in that way as well. + +In some cases, this issue can be worked around by creating and using subdirectories from within the container rather than relying on Kubernetes to do so. From be2d47401decd0b1277ccffb4d8a1ebdad86d60b Mon Sep 17 00:00:00 2001 From: Yifan Gu Date: Fri, 24 Jun 2016 11:30:14 -0700 Subject: [PATCH 114/348] rkt: Minor refactoring. --- docs/getting-started-guides/rkt/index.md | 9 ++++----- docs/getting-started-guides/rkt/notes.md | 10 ++++------ 2 files changed, 8 insertions(+), 11 deletions(-) diff --git a/docs/getting-started-guides/rkt/index.md b/docs/getting-started-guides/rkt/index.md index 764ed2e0e5..ac79c3ac7b 100644 --- a/docs/getting-started-guides/rkt/index.md +++ b/docs/getting-started-guides/rkt/index.md @@ -3,7 +3,7 @@ This document describes how to run Kubernetes using [rkt](https://github.com/coreos/rkt) as a container runtime. -### **Prerequisite** +### Prerequisite - [systemd](http://www.freedesktop.org/wiki/Software/systemd/) should be installed on the machine and should be enabled. The minimum version required for Kubernetes 1.3 is `219`. @@ -25,7 +25,7 @@ In addition, rkt supports using rkt's [Contained Networking](https://coreos.com/ In this mode, rkt will attempt to join pods into a network named `rkt.kubernetes.io`. To use rkt's contained networking, you can leave the `--network-plugin` to empty, and put a network config file under one of the rkt's [config directories](https://github.com/coreos/rkt/blob/master/Documentation/configuration.md#command-line-flags), for example: -```shell +```shell $ cat </etc/rkt/net.d/k8s_network_example.conf { "name": "rkt.kubernetes.io", @@ -75,8 +75,7 @@ For more information on flannel configuration, please read [CNI/flannel README]( ##### Use Google Compute Engine (GCE) network Each VM on GCE has an additional 256 IP addresses routed to it, so it is possible to forego flannel in smaller clusters. -This can most easily be done by using the builtin kubenet plugin. -This can be done by setting `--network-plugin=kubenet`. +This can most easily be done by using the builtin kubenet plugin, by setting the kubelet flag `--network-plugin=kubenet`. ### Launch a local cluster @@ -104,7 +103,7 @@ Then we can launch the local cluster using the script: $ hack/local-up-cluster.sh ``` -We are also working on set up rkt as the container runtime for [minikube](https://github.com/kubernetes/minikube/issues/168). +We are also working on setting up rkt as the container runtime for [minikube](https://github.com/kubernetes/minikube/issues/168). ### Launch a CoreOS/rkt cluster on Google Compute Engine (GCE) diff --git a/docs/getting-started-guides/rkt/notes.md b/docs/getting-started-guides/rkt/notes.md index 61e9676ba4..8e53481046 100644 --- a/docs/getting-started-guides/rkt/notes.md +++ b/docs/getting-started-guides/rkt/notes.md @@ -1,8 +1,6 @@ --- --- -# Known issues - The following features either are not supported or have large caveats when using the rkt container runtime. ##### Non-existent host volume paths @@ -37,9 +35,9 @@ spec: The `kubectl attach` command does not work under the rkt container runtime. And because of this, some flags in `kubectl run` are not supported, including: -- --attach=true -- --leave-stdin-open=true -- --rm=true +- `--attach=true` +- `--leave-stdin-open=true` +- `--rm=true` ##### Init containers @@ -84,7 +82,7 @@ On the other hand, when running the pod with [stage1-fly](https://coreos.com/rkt Patching a pod to change the image will result in the entire pod restarting, not just the container that was changed. -##### Volume mounts specifying a subPath +##### Volume mounts specifying a subPath The [subPath](https://github.com/kubernetes/kubernetes/pull/22575) feature does not work correctly under rkt. In addition, the above-issue of Non-existent host volume paths being invalid would make many common use-cases for subPaths fail in that way as well. From 523a36e622e1af7b72733f916cefe05d367c8960 Mon Sep 17 00:00:00 2001 From: johndmulhausen Date: Fri, 24 Jun 2016 12:55:58 -0700 Subject: [PATCH 115/348] Create _travis.yml --- _travis.yml | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) create mode 100644 _travis.yml diff --git a/_travis.yml b/_travis.yml new file mode 100644 index 0000000000..5dbf8c972a --- /dev/null +++ b/_travis.yml @@ -0,0 +1,22 @@ +language: ruby +rvm: +- 2.1 +branches: + only: + - master +script: +- cd $HOME +- git config --global user.email ${GIT_EMAIL} +- git config --global user.name "${GIT_NAME}" +- git clone https://${GIT_USERNAME}:${GH_TOKEN}@github.com/kubernetes-v1-2/kubernetes-v1-2.github.io.git +- cd kubernetes-v1-2.github.io +- git remote add --fetch --track master homebase "https://${GIT_USERNAME}:${GH_TOKEN}@github.com/kubernetes/kubernetes.github.io.git" +- git merge -s recursive -X theirs homebase/master -m "Sync from homebase" +- git push +env: + global: + - secure: Fd6wlE2mjPb1fAACxklQcJumpJWycYkaJQBfKRcjGCFlmw1XWVFGhpUC7Ni/MOyzTolqOvtb2rXnYpaujMlJP1UXqVFJ+zPbwur2lc8unQF8PcqPezl8DbPsr6HdceOjRdut/dN8zTw6+hZRDzw/mG4Rf8IVaozlYycAOnWZdAZsLXdbBpAvBp0WYHP9+8wn9xiet1L/QpSiawa/Q35Q9UM1tciPmfFBL4Fkq6Mm8/w6ECaxHyA2EX+eH5ea9EtykrzB5cMA/odJLptjnmfzsPeGS5F1MxuTqrH2Z09emZcqjfXW+fNNzioiZdD4BQe+rqOA1Ktpw645szJVJJenIMzM4P8dDiW+9cJyddGdTC9A7APgLlLVdG/tuoYcjiY1F068SWx8sUWW/xc7YenCzGj2nXIeHjUuWsjqjMorpBKe6Y1QSGfi7HsLq1DpJDR1xQZbPgM/FOaovVIPIozgAIBpS1ukdNNwadmrCw55tUKfr4s2SNv3SUmeprL37QEtIxtBpPDjL1z+qTLjc6JzRr7J57guPBdNbj6Ukg/uxW+z1CwDYb3uAIIa5e6eDxfxGQsUEj8NmkIHBLLGTnKb1WAyPXAIf9ZvZHUwQQzHvOWXGZRd32enRh00uvuwHEy3OuGEqTAOIs/31q5WAamZ1uqzW7KBTMU/nq4QA9W2yfM= + - secure: Lvj1MT/I2zaJp9mQi46us64O4JfeT9cz1m+nqKY/nuC828ULfqMYLlpw08jvrc25JhAaqHNawAwdtaxHTQqHztsmq1ixHFYA6G/1oll+YPhym29toFT9oIkXThf/L77FTJfM1gVFMyiHFbz11Ob8R15tRB1WF1+Uu1RBmtudJD7HSy0u6Uc6lPxpqtycWVRCyPdvZqF3e0KIZaTDRkRzJpcVMHa/5ZJDDcMtuJyjJYXZqS1WR3QHC1z44LlnqyB5ZM2PU4H3LyWnY8wHF2mutF0QtDDVdEuBqILBHiFuKHMxpLY92UgHm2n51RR63MxFjjEzE+iu5f9ComEm5JC0N/cc8sunIiol+d8SRC30/00Vs0tvmeAjRX9IMCExiP3mv7Tz6mqEVk6PyrVlg675hxRg0eVdaNNv92gWzCSIecZ5TuCaRG4JaWO0P8lQC4NreONt6gbnwMD350hMQZLpUUE0QoSEdafpudaD+agl4ZzTFVTOOcSOz2Sa/+RT96Msazq2YlddYXaEKZeyYzqHVkk60PiQzQcAuwCMFrAagqMe3bNI2aCFEWbc9CoR13K/wwRDwAeSzBq/UkylZ8AayJLnIpewr/iYBOQasrXLrorW969Rfr8d/nwhDN5VpRgItU0arDzngaJRoZEAtHx4zaCvZF/H/nvbORcD2gsFP9Q= + - secure: WGmgvSnT1OeHPYtsm/JzcRDnZYp+SeelgbSVNbtwOXGImkeqDo/sDIRH80HxVC3zkUu5wvRO/NkMA7N3LXN2BqENacWBFQ+j+Hf3lnDw9yETBaBpLQPO/MElKGd48GoSiukCipfaITUfwtjAQ3wx0W/z06dYC9BdUYvUM73OtsGtlrY1MEQnBfff/neDGZbzfvTdgDwg1n6d69WP+WSUJt5Aysz0ZoAbYO2rzBrKNwcL7KEhoI/ketXXQ4xEW0nnxU/qfxqhVfJbck1HpSz+HMsVGXF4tT3zdRXmU+P47KGbfYn0GmlviGrRSWCu9/elKTlX9fIqRnR1/UvdTkflwLsk4i3hWnEUflKpIT7soJrKjNQseyd06KI8qvVRXRcll92vCtsYDuOO1AlemxIqp2dIFduCf+5FWH06PbS8oHU8NBe5aVYaoBY8nK14EB3A98saJ8Un7ziB8moRaWZX9bMjfJWfxDF526NdHIusqLQrxvUthtBjf5aRULE2PeIyqYVY8/IKmBEfsy9IUENmlM5KItvgZZt4xdk8o//kceLrKabudHGVC1M1RLZo9ejkdLySz2i8nlJyJ03Hn30H9DfDjW/OMQTn/b1HU4CYSbd/yGR+6+5+tdPeKy/Z6gI6+EjHRYaaQrBLo0RXm2620cb7uDTBIsKVs2TetyhTq54= + - secure: lqboLQKFpfa6F+UkSCs9+NPv027EnSWZA4/cmAXj0hZvxc4W+qMFIlPKytcq+bbZrp7kx3cZgRd5NrgmOTqlJ9merVK1cgMsDegMv/YImHjhGVBH+3ATvqMDb7I7dLg4x/DjhKt/ogyZpoh8vsS77T9MQXaIJM8JMM3ISxYI6MMloCh0z/hbT2qv3IrPMFjpHohHdF1yorxKvSefNgPx6Y4wC+t1t+u8wS9RBhvrmOkVynlU2NgbSiMPyz3syJmnszHDTXXJlqciGT5Lc8lzpzt2OGOnZmAupEYVGok7Jx4aAxU9O8/bnoDwOcGGVrd/pGHTI5+rvT4TPy+WG1Aec55dgva4XZfrJQ88snHLvq0VrvdiI/fHJEPF18QnrBwKSni1Vd1jOzsuPIF6HL0Wwv8tklO8OX3D5wqVsXpeJd4Cj4HPPAtzxOVsV6Fz3On9vuLdV7StC/ZafQL40koaytqkNO0gX8zeUiaaVQtMCf/2MVTbX9x5m51gkGpwT0JBvqQpSMOlIM5S1fiN4X9DLBLqmsARYeZw1Jeiq7Zm39Sy1QeCgfUXS8+6t62BjWx70iftYGIkoljXtD7x/3pjKymdpwjcyUcS8KQ5W6vbMXFxmZe2phtgGTCxOIDuASPspD8zwm/ckIGArHj4qkrg94/mUpgDDIbVsaUpWXjYvSc= + - secure: bPZMNH79Lx0Wb2SCxTwyZek28w/keKxEAlfad8RDMwJrct6Bi2B0o4KjkrwFS5DyCUU7Ndk192145XnUKOWX2YVs6cQ8ge+5LvtgYhWLgX8g5Ycro+JyzOBskn+o1gQjvi8+3b42X31efcUmTEhfRdKVrrUpONEIcjG1NpLGk/mQJ6AM6hGWO2xdNfAezeWq5ISnpK0b6VUWZyTEDg3NivrTfCL2juWWCnjYm1BWSHUblXwRQ/Rl7Tcldl6cMMUVsalUQ1iG0h8YZDrxNz0cm3XTZZZJKuSHYeTCLd57RBeHD5/iMxjCmfzfq6ETNLONWLmtvA8yhWMQZ0DPFtLZzbqFIfOR6P1feBZFqP7/X5KZeFKBufN15JbcXIqHE8homLY9mS0LhyNffOs5G/P/x8ChE1DJaYiZIWCZ60umvpqibScZB3z5uFPTxLk9rJSOtT7hCWjcmg3EdJ+R4ExOiBDd62ZS5jH72WU4uysPXORRofUCL+zHycJoJxsFWQSW49GAGrohllrW45jnTgpalErxvjJFSKy8JW9w634eetz9ct2fObep7m8bfVMl8U2H3ITIoXHm+f8ooHUiNzHFLSl4wtcoAOtrCAvGtJPNfv8T2eTkznzj7Tk+XzTvFg03u+J99TmnC58Bs516Oc7E40NarmsZceOD2sN4BA3X9tE= From c65d7400fd13ccbbb4ab75c90effaa566f519e4a Mon Sep 17 00:00:00 2001 From: Lantao Liu Date: Mon, 20 Jun 2016 16:24:35 -0700 Subject: [PATCH 116/348] Add document for node problem detector. --- _data/guides.yml | 2 + docs/admin/node-problem.md | 245 +++++++++++++++++++++++++++++++++++++ 2 files changed, 247 insertions(+) create mode 100644 docs/admin/node-problem.md diff --git a/_data/guides.yml b/_data/guides.yml index f5654a6efd..91ce1bef50 100644 --- a/_data/guides.yml +++ b/_data/guides.yml @@ -265,3 +265,5 @@ toc: path: /docs/admin/garbage-collection/ - title: Configuring Kubernetes with Salt path: /docs/admin/salt/ + - title: Monitoring Node Health + path: /docs/admin/node-problem/ diff --git a/docs/admin/node-problem.md b/docs/admin/node-problem.md new file mode 100644 index 0000000000..5dc2d4cb52 --- /dev/null +++ b/docs/admin/node-problem.md @@ -0,0 +1,245 @@ +--- +--- + +* TOC +{:toc} + +## Node Problem Detector + +*Node problem detector* is a [DaemonSet](/docs/admin/daemons/) monitoring the +node health. It collects node problems from various daemons and reports them +to the apiserver as [NodeCondition](/docs/admin/node/#node-condition) and +[Event](/docs/api-reference/v1/definitions/#_v1_event). + +It supports some known kernel issue detection now, and will detect more and +more node problems over time. + +Currently Kubernetes won't take any action on the node conditions and events +generated by node problem detector. In the future, a remedy system could be +introduced to deal with node problems. + +See more information +[here](https://github.com/kubernetes/node-problem-detector). + +## Limitations + +* The kernel issue detection of node problem detector only supports file based +kernel log now. It doesn't support log tools like journald. + +* The kernel issue detection of node problem detector has assumption on kernel +log format, now it only works on Ubuntu and Debian. However, it is easy to extend +it to [support other log format](/docs/admin/node-problem/#support-other-log-format). + +## Enable/Disable in GCE cluster + +Node problem detector is running as a cluster +[addon](docs/admin/cluster-large/#addon-resources) enabled by default in the +gce cluster. + +You can enable/disable it by setting the environment variable +`KUBE_ENABLE_NODE_PROBLEM_DETECTOR` before `kube-up.sh`. + +## Use in Other Environment + +To enable node problem detector in other environment outside of GCE, you can use +either `kubectl` or addon pod. + +### Kubectl + +This is the recommanded way to start node problem detector outside of GCE. It +provides more flexible management, such as overwriting the default +configuration to fit it into your environment or detect +customized node problems. + +* **Step 1:** Create `node-problem-detector.yaml`: + +```yaml +apiVersion: extensions/v1beta1 +kind: DaemonSet +metadata: + name: node-problem-detector-v0.1 + namespace: kube-system + labels: + k8s-app: node-problem-detector + version: v0.1 + kubernetes.io/cluster-service: "true" +spec: + template: + metadata: + labels: + k8s-app: node-problem-detector + version: v0.1 + kubernetes.io/cluster-service: "true" + spec: + hostNetwork: true + containers: + - name: node-problem-detector + image: gcr.io/google_containers/node-problem-detector:v0.1 + securityContext: + privileged: true + resources: + limits: + cpu: "200m" + memory: "100Mi" + requests: + cpu: "20m" + memory: "20Mi" + volumeMounts: + - name: log + mountPath: /log + readOnly: true + volumes: + - name: log + hostPath: + path: /var/log/ +``` + +***Notice that you should make sure the system log directory is right for your +OS distro.*** + +* **Step 2:** Start node problem detector with `kubectl`: + +```shell +kubectl create -f node-problem-detector.yaml +``` + +### Addon Pod + +This is for those who have their own cluster bootstrap solution, and don't need +to overwrite the default configuration. They could leverage the addon pod to +further automate the deployment. + +Just create `node-problem-detector.yaml`, and put it under the addon pods directory +`/etc/kubernetes/addons/node-problem-detector` on master node. + +## Overwrite the Configuration + +The [default configuration](https://github.com/kubernetes/node-problem-detector/tree/v0.1/config) +is embedded when building the docker image of node problem detector. + +However, you can use [ConfigMap](/docs/user-guide/configmap/) to overwrite it +following the steps: + +* **Step 1:** Change the config files in `config/`. +* **Step 2:** Create the ConfigMap `node-problem-detector-config` with `kubectl create configmap +node-problem-detector-config --from-file=config/`. +* **Step 3:** Change the `node-problem-detector.yaml` to use the ConfigMap: + +```yaml +apiVersion: extensions/v1beta1 +kind: DaemonSet +metadata: + name: node-problem-detector-v0.1 + namespace: kube-system + labels: + k8s-app: node-problem-detector + version: v0.1 + kubernetes.io/cluster-service: "true" +spec: + template: + metadata: + labels: + k8s-app: node-problem-detector + version: v0.1 + kubernetes.io/cluster-service: "true" + spec: + hostNetwork: true + containers: + - name: node-problem-detector + image: gcr.io/google_containers/node-problem-detector:v0.1 + securityContext: + privileged: true + resources: + limits: + cpu: "200m" + memory: "100Mi" + requests: + cpu: "20m" + memory: "20Mi" + volumeMounts: + - name: log + mountPath: /log + readOnly: true + - name: config # Overwrite the config/ directory with ConfigMap volume + mountPath: /config + readOnly: true + volumes: + - name: log + hostPath: + path: /var/log/ + - name: config # Define ConfigMap volume + configMap: + name: node-problem-detector-config +``` + +* **Step 4:** Re-create the node problem detector with the new yaml file: + +```shell +kubectl delete -f node-problem-detector.yaml # If you have a node-problem-detector running +kubectl create -f node-problem-detector.yaml +``` + +***Notice that this approach only applies to node problem detector started with `kubectl`.*** + +For node problem detector running as cluster addon, because addon manager doesn't support +ConfigMap, configuration overwriting is not supported now. + +## Kernel Monitor + +*Kernel Monitor* is a problem daemon in node problem detector. It monitors kernel log +and detects known kernel issues following predefined rules. + +The Kernel Monitor matches kernel issues according to a set of predefined rule list in +[`config/kernel-monitor.json`](https://github.com/kubernetes/node-problem-detector/blob/v0.1/config/kernel-monitor.json). +The rule list is extensible, you can always extend it by [overwriting the +configuration](/docs/admin/node-problem/#overwrite-the-configuration). + +### Add New NodeConditions + +To support new node conditions, you can extend the `conditions` field in +`config/kernel-monitor.json` with new condition definition: + +```json +{ + "type": "NodeConditionType", + "reason": "CamelCaseDefaultNodeConditionReason", + "message": "arbitrary default node condition message" +} +``` + +### Detect New Problems + +To detect new problems, you can extend the `rules` field in `config/kernel-monitor.json` +with new rule definition: + +```json +{ + "type": "temporary/permanent", + "condition": "NodeConditionOfPermanentIssue", + "reason": "CamelCaseShortReason", + "message": "regexp matching the issue in the kernel log" +} +``` + +### Change Log Path + +Kernel log in different OS distros may locate in different path. The `log` +field in `config/kernel-monitor.json` is the log path inside the container. +You can always configure it to match your OS distro. + +### Support Other Log Format + +Kernel monitor uses [`Translator`](https://github.com/kubernetes/node-problem-detector/blob/v0.1/pkg/kernelmonitor/translator/translator.go) +plugin to translate kernel log the internal data structure. It is easy to +implement a new translator for a new log format. + +## Caveats + +It is recommanded to run the node problem detector in your cluster to monitor +the node health. However, you should be aware that this will introduce extra +resource overhead on each node. Usually this is fine, because: + +* The kernel log is generated relatively slowly. +* Resource limit is set for node problem detector. +* Even under high load, the resource usage is acceptable. +(see [benchmark result](https://github.com/kubernetes/node-problem-detector/issues/2#issuecomment-220255629)) From a6ba113db9c0a4e4c9dfa8c0c5c3f3e6c1c2e287 Mon Sep 17 00:00:00 2001 From: Phillip Wittrock Date: Fri, 24 Jun 2016 15:27:38 -0700 Subject: [PATCH 117/348] More udpates to how to use templates. No longer recommend docsv2. --- docs/templatedemos/index.md | 190 ++++++++++++++++++++---------------- 1 file changed, 105 insertions(+), 85 deletions(-) diff --git a/docs/templatedemos/index.md b/docs/templatedemos/index.md index a202795fd9..73167fc445 100644 --- a/docs/templatedemos/index.md +++ b/docs/templatedemos/index.md @@ -10,129 +10,121 @@ $(function() { Check out the kubernetes/kubernetes.github.io repo and the docsv2 branch. -## Step 1: Fork and Clone the repo +### Step 1: Fork and Clone the repo - Fork [kubernetes/kubernetes.github.io](https://github.com/kubernetes/kubernetes.github.io) - [Setup your GitHub authentication using ssh](https://help.github.com/articles/generating-an-ssh-key/) -- Clone the repo under go/src/k8s.io +- Clone the repo under ~/go/src/k8s.io -```sh +```shell cd ~/go/src/k8s.io git clone git@github.com:/kubernetes.github.io cd kubernetes.github.io git remote add upstream https://github.com/kubernetes/kubernetes.github.io.git ``` -## Step 2: Move to the docsv2 branch - -Docs v2 development is being performed in the docsv2 branch. This is the branch +### Step 2: Switch to the docsv2 branch +Docs v2 development is being performed in the `docsv2` branch. This is the branch you want to be working from. -From ~/go/src/k8s.io/kubernetes.github.io +From ~/go/src/k8s.io/kubernetes.github.io: ```shell -git checkout -b docsv2 # switch to the docsv2 branch -git fetch upstream # get the latest upstream changes -git reset --hard upstream/docsv2 # sync the branch to upstream +git checkout -b docsv2 +git fetch upstream +git reset --hard upstream/docsv2 ``` -## Step 3: Make sure you can serve rendered docs locally from your dev box +### Step 3: Make sure you can serve rendered docs locally from your dev box - [Follow the editdocs instructions](http://kubernetes.io/editdocs/) # Writing Docs Using Templates - From 198666e83ff0d81c22de36e6b012d01adbf42de7 Mon Sep 17 00:00:00 2001 From: Janet Kuo Date: Tue, 12 Jul 2016 11:01:07 -0700 Subject: [PATCH 191/348] Add one-line installation for kubectl in minikube guide --- docs/getting-started-guides/minikube.md | 40 ++++++++++++------------- 1 file changed, 19 insertions(+), 21 deletions(-) diff --git a/docs/getting-started-guides/minikube.md b/docs/getting-started-guides/minikube.md index 96fd62ae66..7e422e65bc 100644 --- a/docs/getting-started-guides/minikube.md +++ b/docs/getting-started-guides/minikube.md @@ -51,39 +51,37 @@ Minikube requires one of the following: * The latest [Virtualbox](https://www.virtualbox.org/wiki/Downloads). * The latest version of [VMWare Fusion](https://www.vmware.com/products/fusion). -### Installation +### Install `minikube` See the [latest Minikube release](https://github.com/kubernetes/minikube/releases) for installation instructions. -### Download `kubectl` +### Install `kubectl` -You will need to download the kubectl client binary for `${K8S_VERSION}` (in this example: `{{page.version}}.0`) +You will need to download and install the kubectl client binary for `${K8S_VERSION}` (in this example: `{{page.version}}.0`) to run commands against the cluster. -Downloads: - - - `linux/amd64`: http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/amd64/kubectl - - `linux/386`: http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/386/kubectl - - `linux/arm`: http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/arm/kubectl - - `linux/arm64`: http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/arm64/kubectl - - `linux/ppc64le`: http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/ppc64le/kubectl - - `OS X/amd64`: http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/darwin/amd64/kubectl - - `OS X/386`: http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/darwin/386/kubectl - - `windows/amd64`: http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/windows/amd64/kubectl.exe - - `windows/386`: http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/windows/386/kubectl.exe +```shell +# linux/amd64 +curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/darwin/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ +# linux/386 +curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/386/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ +# linux/arm +curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/arm/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ +# linux/arm64 +curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/arm64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ +#linux/ppc64le +curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/ppc64le/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ +# OS X/amd64 +curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/darwin/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ +# OS X/386 +curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/darwin/386/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ +``` The generic download path is: ``` http://storage.googleapis.com/kubernetes-release/release/${K8S_VERSION}/bin/${GOOS}/${GOARCH}/${K8S_BINARY} ``` -An example install with `linux/amd64`: - -``` -curl -sSL "http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/amd64/kubectl" > /usr/bin/kubectl -chmod +x /usr/bin/kubectl -``` - ### Starting the cluster To start a cluster, run the command: From 6fede2cf0f2934d6693f5663e92a20fa42201796 Mon Sep 17 00:00:00 2001 From: Manuel de Brito Fontes Date: Tue, 12 Jul 2016 15:03:47 -0400 Subject: [PATCH 192/348] Update nginx-slim image version and remove default command --- docs/user-guide/petset.yaml | 7 +------ docs/user-guide/petset/bootstrapping/petset_peers.yaml | 2 +- 2 files changed, 2 insertions(+), 7 deletions(-) diff --git a/docs/user-guide/petset.yaml b/docs/user-guide/petset.yaml index 017a65746c..5c29237c48 100644 --- a/docs/user-guide/petset.yaml +++ b/docs/user-guide/petset.yaml @@ -31,15 +31,10 @@ spec: terminationGracePeriodSeconds: 0 containers: - name: nginx - image: gcr.io/google_containers/nginx-slim:0.7 + image: gcr.io/google_containers/nginx-slim:0.8 ports: - containerPort: 80 name: web - command: - - nginx - args: - - -g - - "daemon off;" volumeMounts: - name: www mountPath: /usr/share/nginx/html diff --git a/docs/user-guide/petset/bootstrapping/petset_peers.yaml b/docs/user-guide/petset/bootstrapping/petset_peers.yaml index e12988dee2..f8393b5c2c 100644 --- a/docs/user-guide/petset/bootstrapping/petset_peers.yaml +++ b/docs/user-guide/petset/bootstrapping/petset_peers.yaml @@ -71,7 +71,7 @@ spec: terminationGracePeriodSeconds: 0 containers: - name: nginx - image: gcr.io/google_containers/nginx-slim:0.7 + image: gcr.io/google_containers/nginx-slim:0.8 ports: - containerPort: 80 name: web From 8240c9259af745866692a6fe21e480960180ed55 Mon Sep 17 00:00:00 2001 From: Josh Hoak Date: Tue, 12 Jul 2016 13:54:36 -0700 Subject: [PATCH 193/348] Capitalize replica set -> Replica Set for consistency. --- docs/user-guide/replicasets.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/user-guide/replicasets.md b/docs/user-guide/replicasets.md index b2b9b02ee3..f51629bda4 100644 --- a/docs/user-guide/replicasets.md +++ b/docs/user-guide/replicasets.md @@ -22,16 +22,16 @@ instead. Also, the imperative whereas Deployments are declarative, so we recommend using Deployments through the [`rollout`](/docs/user-guide/kubectl/kubectl_rollout/) command. -While replica sets can be used independently, today it's mainly used by +While Replica Sets can be used independently, today it's mainly used by [Deployments](/docs/user-guide/deployments/) as a mechanism to orchestrate pod creation, deletion and updates. When you use Deployments you don't have to worry -about managing the replica sets that they create. Deployments own and manage -their replica sets. +about managing the Replica Sets that they create. Deployments own and manage +their Replica Sets. ## When to use a Replica Set? A Replica Set ensures that a specified number of pod “replicas” are running at any given -time. However, a Deployment is a higher-level concept that manages replica sets and +time. However, a Deployment is a higher-level concept that manages Replica Sets and provides declarative updates to pods along with a lot of other useful features. Therefore, we recommend using Deployments instead of directly using Replica Sets, unless you require custom update orchestration or don't require updates at all. @@ -44,7 +44,7 @@ use directly a Deployment and define your application in the spec section. {% include code.html language="yaml" file="replicasets/frontend.yaml" ghlink="/docs/user-guide/replicasets/frontend.yaml" %} Saving this config into `frontend.yaml` and submitting it to a Kubernetes cluster should -create the defined replica set and the pods that it manages. +create the defined Replica Set and the pods that it manages. ```shell $ kubectl create -f frontend.yaml From 6208bcf9efaf297de970245387388dad12f9a71f Mon Sep 17 00:00:00 2001 From: Mike Spreitzer Date: Wed, 13 Jul 2016 11:52:17 -0400 Subject: [PATCH 194/348] Added missing list structure in example NetworkPolicy Fixes #801 --- docs/user-guide/networkpolicies.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/user-guide/networkpolicies.md b/docs/user-guide/networkpolicies.md index e550657df6..52ecdf6c99 100644 --- a/docs/user-guide/networkpolicies.md +++ b/docs/user-guide/networkpolicies.md @@ -61,9 +61,9 @@ spec: role: db ingress: - from: - podSelector: - matchLabels: - role: frontend + - podSelector: + matchLabels: + role: frontend ports: - protocol: tcp port: 6379 From ffcc0baadb619796598554587701a8bd446d8275 Mon Sep 17 00:00:00 2001 From: Quinton Hoole Date: Fri, 8 Jul 2016 18:01:07 -0700 Subject: [PATCH 195/348] User Guide for Federated Services. --- .../federation/federated-services.md | 378 ++++++++++++++++++ 1 file changed, 378 insertions(+) create mode 100644 docs/user-guide/federation/federated-services.md diff --git a/docs/user-guide/federation/federated-services.md b/docs/user-guide/federation/federated-services.md new file mode 100644 index 0000000000..16eb967f0e --- /dev/null +++ b/docs/user-guide/federation/federated-services.md @@ -0,0 +1,378 @@ +--- +--- +# Kubernetes Federated Services and Cross-Cluster Service Discovery + +This guide explains how to use Kubernetes Federated Services to deploy +a common Service across multiple Kubernetes clusters. This makes it +easy to achieve cross-cluster service discovery and availibility zone +fault tolerance for your Kubernetes applications. + + +* TOC +{:toc} + +## Prerequisites + +This guide assumes that you have a running Kubernetes Cluster +Federation installation. If not, then head over to the +[federation admin guide](/docs/admin/federation/) to learn how to +bring up a cluster federation (or have your cluster administrator do +this for you). Other tutorials, for example +[this one](https://github.com/kelseyhightower/kubernetes-cluster-federation) +by Kelsey Hightower, are also available to help you. + +You are also expected to have a basic +[working knowledge of Kubernetes](/docs/getting-started-guides/) in +general, and [Services](/docs/user-guide/services/) in particular. + +## Overview + +Federated Services are created in much that same way as traditional +[Kubernetes Services](/docs/user-guide/services/) by making an API +call which specifies the desired properties of your service. In the +case of Federated Services, this API call is directed to the +Federation API endpoint, rather than a Kubernetes cluster API +endpoint. The API for Federated Services is 100% compatible with the +API for traditional Kubernetes Services. + +Once created, the Federated Service automatically: + +1. creates matching Kubernetes Services in every cluster underlying your Cluster Federation, +2. monitors the health of those service "shards" (and the clusters in which they reside), and +3. manages a set of DNS records in a public DNS provder (like Google Cloud DNS, or AWS Route 53), thus ensuring that clients +of your federated service can seamlessly locate an appropriate healthy service endpoint at all times, even in the event of cluster, +availability zone or regional outages. + +Clients inside your federated Kubernetes clusters (i.e. Pods) will +automatically find the local shard of the Federated Service in their +cluster if it exists and is healthy, or the closest healthy shard in a +different cluster if it does not. + +## Hybrid Cloud Capabilities + +Federations of Kubernetes Clusters can include clusters running in +different cloud providers (e.g. Google Cloud, AWS), and on-premises +(e.g. on OpenStack). Simply create all of the clusters that you +require, in the appropriate cloud providers and/or locations, and +register each cluster's API endpoint and credentials with your +Federation API Server (See the +[federation admin guide](/docs/admin/federation/) for details). + +Thereafter, your applications and services can span different clusters +and cloud providers as described in more detail below. + +## Creating a Federated Service + +This is done in the usual way, for example: + +``` +kubectl --context=federation-cluster create -f services/nginx.yaml +``` + +The '--context=federation-cluster' flag tells kubectl to submit the +request to the Federation API endpoint, with the appropriate +credentials. If you have not yet configured such a context, visit the +[federation admin guide](/docs/admin/federation/) or one of the +[administration tutorials](https://github.com/kelseyhightower/kubernetes-cluster-federation) +to find out how to do so. + +As described above, the Federated Service will automatically create +and maintain matching Kubernetes services in all of the clusters +underlying your federation. + +You can verify this by checking in each of the underlying clusters, for example: + +``` +kubectl --context=gce-asia-east1a get services nginx +NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE +nginx 10.63.250.98 104.199.136.89 80/TCP 9m +``` + +The above assumes that you have a context named 'gce-asia-east1a' +configured in your client for your cluster in that zone. The name and +namespace of the underlying services will automatically match those of +the Federated Service that you created above (and if you happen to +have had services of the same name and namespace already existing in +any of those clusters, they will be automatically adopted by the +Federation and updated to conform with the specification of your +Federated Service - either way, the end result will be the same). + +The status of your Federated Service will automatically reflect the +real-time status of the underlying Kubernetes services, for example: + +``` +kubectl --context=federation-cluster describe services nginx +``` + +``` +Name: nginx +Namespace: default +Labels: run=nginx +Selector: run=nginx +Type: LoadBalancer +IP: +LoadBalancer Ingress: 104.197.246.190, 130.211.57.243, 104.196.14.231, 104.199.136.89, ... +Port: http 80/TCP +Endpoints: +Session Affinity: None +No events. +``` + +Note the 'LoadBalancer Ingress' addresses of your Federated Service +correspond with the 'LoadBalancer Ingress' addresses of all of the +underlying Kubernetes services (once these have been allocated - this +may take a few seconds). For inter-cluster and inter-cloud-provider +networking between service shards to work correctly, your services +need to have an externally visible IP address. Service Type: +Loadbalancer is typically used for this. + +Note also that we have not yet provisioned any backend Pods to receive +the network traffic directed to these addresses (i.e. 'Service +Endpoints'), so the Federated Service does not yet consider these to +be healthy service shards, and has accordingly not yet added their +addresses to the DNS records for this Federated Service (more on this +aspect later). + +## Adding Backend Pods + +To render the underlying service shards healthy, we need to add +backend Pods behind them. This is currently done directly against the +API endpoints of the underlying clusters (although in future the +Federation server will be able to do all this for you with a single +command, to save you the trouble). For example, to create backend Pods +in 13 underlying clusters: + +``` +for CLUSTER in asia-east1-c asia-east1-a asia-east1-b \ + europe-west1-d europe-west1-c europe-west1-b \ + us-central1-f us-central1-a us-central1-b us-central1-c \ + us-east1-d us-east1-c us-east1-b +do + kubectl --context=$CLUSTER run nginx --image=nginx:1.11.1-alpine --port=80 +done +``` +Note that `kubectl run` automatically adds the `run=nginx` labels required to associate the backend pods with their services. + +## Verifying Public DNS Records + +Once the above Pods have successfully started and have begun listening +for connections, Kubernetes in each cluster (via automatic health +checks) will report them as healthy endpoints of the service in that +cluster. The Cluster Federation will in turn consider each of these +service 'shards' to be healthy, and place them in serving by +automatically configuring corresponding public DNS records. You can +use your preferred interface to your configured DNS provider to verify +this. For example, if your Federation is configured to use Google +Cloud DNS, and a managed DNS domain 'example.com': + +``` +$ gcloud dns managed-zones describe example-dot-com +creationTime: '2016-06-26T18:18:39.229Z' +description: Example domain for Kubernetes Cluster Federation +dnsName: example.com. +id: '3229332181334243121' +kind: dns#managedZone +name: example-dot-com +nameServers: +- ns-cloud-a1.googledomains.com. +- ns-cloud-a2.googledomains.com. +- ns-cloud-a3.googledomains.com. +- ns-cloud-a4.googledomains.com. +``` + +``` +$ gcloud dns record-sets list --zone example-dot-com +NAME TYPE TTL DATA +example.com. NS 21600 ns-cloud-e1.googledomains.com., ns-cloud-e2.googledomains.com. +example.com. SOA 21600 ns-cloud-e1.googledomains.com. cloud-dns-hostmaster.google.com. 1 21600 3600 1209600 300 +nginx.mynamespace.myfederation.svc.example.com. A 180 104.197.246.190, 130.211.57.243, 104.196.14.231, 104.199.136.89,... +nginx.mynamespace.myfederation.svc.us-central1-a.example.com. A 180 104.197.247.191 +nginx.mynamespace.myfederation.svc.us-central1-b.example.com. A 180 104.197.244.180 +nginx.mynamespace.myfederation.svc.us-central1-c.example.com. A 180 104.197.245.170 +nginx.mynamespace.myfederation.svc.us-central1-f.example.com. CNAME 180 nginx.mynamespace.myfederation.svc.us-central1.example.com. +nginx.mynamespace.myfederation.svc.us-central1.example.com. A 180 104.197.247.191, 104.197.244.180, 104.197.245.170 +nginx.mynamespace.myfederation.svc.asia-east1-a.example.com. A 180 130.211.57.243 +nginx.mynamespace.myfederation.svc.asia-east1-b.example.com. CNAME 180 nginx.mynamespace.myfederation.svc.asia-east1.example.com. +nginx.mynamespace.myfederation.svc.asia-east1-c.example.com. A 180 130.211.56.221 +nginx.mynamespace.myfederation.svc.asia-east1.example.com. A 180 130.211.57.243, 130.211.56.221 +nginx.mynamespace.myfederation.svc.europe-west1.example.com. CNAME 180 nginx.mynamespace.myfederation.svc.example.com. +nginx.mynamespace.myfederation.svc.europe-west1-d.example.com. CNAME 180 nginx.mynamespace.myfederation.svc.europe-west1.example.com. +... etc. +``` + +Note: If your Federation is configured to use AWS Route53, you can use one of the equivalent AWS tools, for example: + +``` +$aws route53 list-hosted-zones +``` +and +``` +$aws route53 list-resource-record-sets --hosted-zone-id Z3ECL0L9QLOVBX +``` + +Whatever DNS provider you use, any DNS query tool (for example 'dig' +or 'nslookup') will of course also allow you to see the records +created by the Federation for you. Note that you should either point +these tools directly at your DNS provider (e.g. `dig +@ns-cloud-e1.googledomains.com...`) or expect delays in the order of +your configured TTL (180 seconds, by default) before seeing updates, +due to caching by intermediate DNS servers. + +### Some notes about the above example + +1. Notice that there is a normal ('A') record for each service shard that has at least one healthy backend endpoint. For example in us-central1-a, 104.197.247.191 is the external IP address of the service shard in that zone, and in asia-east1-a the address is 130.211.56.221. +2. Similarly, there are regional 'A' records which include all healthy shards in that region. For example, 'us-central1'. These regional records are useful for clients which do not have a particular zone preference, and as a building block for the automated locality and failover mechanism described below. +2. For zones where there are currently no healthy backend endpoints, a CNAME ('Canonical Name') record is used to alias (automatically redirect) those queries to the next closest healthy zone. In the example, the service shard in us-central1-f currently has no healthy backend endpoints (i.e. Pods), so a CNAME record has been created to automatically redirect queries to other shards in that region (us-central1 in this case). +3. Similarly, if no healthy shards exist in the enclosing region, the search progresses further afield. In the europe-west1-d availability zone, there are no healthy backends, so queries are redirected to the broader europe-west1 region (which also has no healthy backends), and onward to the global set of healthy addresses (' nginx.mynamespace.myfederation.svc.example.com.') + +The above set of DNS records is automatically kept in sync with the +current state of health of all service shards globally by the +Federated Service system. DNS resolver libraries (which are invoked by +all clients) automatically traverse the hiearchy of 'CNAME' and 'A' +records to return the correct set of healthy IP addresses. Clients can +then select any one of the returned addresses to initiate a network +connection (and fail over automatically to one of the other equivalent +addresses if required). + +## Discovering a Federated Service + +### From Pods Inside your Federated Clusters + +By default, Kubernetes clusters come pre-configured with a +cluster-local DNS server ('KubeDNS'), as well as an intelligently +constructed DNS search path which together ensure that DNS queries +like "myservice", "myservice.mynamespace", +"bobsservice.othernamespace" etc issued by your software running +inside Pods are automatically expanded and resolved correctly to the +appropriate service IP of services running in the local cluster. + +With the introduction of Federated Services and Cross-Cluster Service +Discovery, this concept is extended to cover Kubernetes services +running in any other cluster across your Cluster Federation, globally. +To take advantage of this extended range, you use a slightly different +DNS name (of the form "..", +e.g. myservice.mynamespace.myfederation) to resolve Federated +Services. Using a different DNS name also avoids having your existing +applications accidentally traversing cross-zone or cross-region +networks and you incurring perhaps unwanted network charges or +latency, without you explicitly opting in to this behavior. + +So, using our NGINX example service above, and the Federated Service +DNS name form just described, let's consider an example: A Pod in a +cluster in the `us-central1-f` availability zone needs to contact our +NGINX service. Rather than use the service's traditional cluster-local +DNS name (```"nginx.mynamespace"```, which is automatically expanded +to ```"nginx.mynamespace.svc.cluster.local"```) it can now use the +service's Federated DNS name, which is +```"nginx.mynamespace.myfederation"```. This will be automatically +expanded and resolved to the closest healthy shard of my NGINX +service, wherever in the world that may be. If a healthy shard exists +in the local cluster, that service's cluster-local (typically +10.x.y.z) IP address will be returned (by the cluster-local KubeDNS). +This is almost exactly equivalent to non-federated service resolution +(almost because KubeDNS actually returns both a CNAME and an A record +for local federated services, but applications will be oblivious +to this minor technical difference). + +But if the service does not exist in the local cluster (or it exists +but has no healthy backend pods), the DNS query is automatically +expanded to +```"nginx.mynamespace.myfederation.svc.us-central1-f.example.com"``` +(i.e. logically "find the external IP of one of the shards closest to +my availability zone"). This expansion is performed automatically by +KubeDNS, which returns the associated CNAME record. This results in +automatic traversal of the hierarchy of DNS records in the above +example, and ends up at one of the external IP's of the Federated +Service in the local us-central1 region (i.e. 104.197.247.191, +104.197.244.180 or 104.197.245.170). + +It is of course possible to explicitly target service shards in +availability zones and regions other than the ones local to a Pod by +specifying the appropriate DNS names explicitly, and not relying on +automatic DNS expansion. For example, +"nginx.mynamespace.myfederation.svc.europe-west1.example.com" will +resolve to all of the currently healthy service shards in Europe, even +if the Pod issuing the lookup is located in the U.S., and irrespective +of whether or not there are healthy shards of the service in the U.S. +This is useful for remote monitoring and other similar applications. + +### From Other Clients Outside your Federated Clusters + +Much of the above discussion applies equally to external clients, +except that the automatic DNS expansion described is no longer +possible. So external clients need to specify one of the fully +qualified DNS names of the Federated Service, be that a zonal, +regional or global name. For convenience reasons, it is often a good +idea to manually configure additional static CNAME records in your +service, for example: + +``` +eu.nginx.acme.com CNAME nginx.mynamespace.myfederation.svc.europe-west1.example.com. +us.nginx.acme.com CNAME nginx.mynamespace.myfederation.svc.us-central1.example.com. +nginx.acme.com CNAME nginx.mynamespace.myfederation.svc.example.com. +``` +That way your clients can always use the short form on the left, and +always be automatcally routed to the closest healthy shard on their +home continent. All of the required failover is handled for you +automatically by Kubernetes Cluster Federation. Future releases will +improve upon this even further. + +## Handling Failures of Backend Pods and Whole Clusters + +Standard Kubernetes service cluster-IP's already ensure that +non-responsive individual Pod endpoints are automatically taken out of +service with low latency (a few seconds). In addition, as alluded +above, the Kubernetes Cluster Federation system automatically monitors +the health of clusters and the endpoints behind all of the shards of +your Federated Service, taking shards in and out of service as +required (e.g. when all of the endpoints behind a service, or perhaps +the entire cluster or availability zone go down, or conversely recover +from an outage). Due to the latency inherent in DNS caching (the cache +timeout, or TTL for Federated Service DNS records is configured to 3 +minutes, by default, but can be adjusted), it may take up to that long +for all clients to completely fail over to an alternative cluster in +the case of catastrophic failure. However, given the number of +discrete IP addresses which can be returned for each regional service +endpoint (see e.g. us-central1 above, which has three alternatives) +many clients will fail over automatically to one of the alternative +IP's in less time than that given appropriate configuration. + +## Troubleshooting + +#### I cannot connect to my Cluster Federation API +Check that your + +1. Client (typically kubectl) is correctly configured (including API endpoints and login credentials), and +2. Cluster Federation API server is running and network-reachable. + +See the [federation admin guide](/docs/admin/federation/) to learn +how to bring up a cluster federation correctly (or have your cluster administrator do this for you), and how to correctly configure your client. + +#### I can create a Federated Service successfully against the Cluster Federation API, but no matching services are created in my underlying clusters +Check that: + +1. Your clusters are correctly registered in the Cluster Federation API (`kubectl describe clusters`) +2. Your clusters are all 'Active'. This means that the cluster Federation system was able to connect and authenticate against the clusters' endpoints. If not, consult the logs of the federation-controller-manager pod to ascertain what the failure might be. (`kubectl --namespace=federation logs $(kubectl get pods --namespace=federation -l module=federation-controller-manager -oname`) +3. That the login credentials provided to the Cluster Federation API for the clusters have the correct authorization and quota to create services in the relevant namespace in the clusters. Again you should see associated error messages providing more detail in the above log file if this is not the case. +4. Whether any other error is preventing the service creation operation from succeeding (look for `service-controller` errors in the output of `kubectl logs federation-controller-manager --namespace federation`). + +#### I can create a Federated Service successfully, but no matching DNS records are created in my DNS provider. +Check that: + +1. Your federation name, DNS provider, DNS domain name are configured correctly. Consult the [federation admin guide](/docs/admin/federation/) or [tutorial](https://github.com/kelseyhightower/kubernetes-cluster-federation) to learn +how to configure your Cluster Federation system's DNS provider (or have your cluster administrator do this for you). +2. Confirm that the Cluster Federation's service-controller is successfully connecting to and authenticating against your selected DNS provider (look for `service-controller` errors or successes in the output of `kubectl logs federation-controller-manager --namespace federation`) +3. Confirm that the Cluster Federation's service-controller is successfully creating DNS records in your DNS provider (or outputting errors in it's logs explaining in more detail what's failing). + +#### Matching DNS records are created in my DNS provider, but clients are unable to resolve against those names +Check that: + +1. The DNS registrar that manages your federation DNS domain has been correctly configured to point to your configured DNS provider's nameservers. See for example [Google Domains Documentation](https://support.google.com/domains/answer/3290309?hl=en&ref_topic=3251230) and [Google Cloud DNS Documentation](https://cloud.google.com/dns/update-name-servers), or equivalent guidance from your domain registrar and DNS provider. + +#### This troubleshooting guide did not help me solve my problem + +1. Please use one of our [support channels](http://kubernetes.io/docs/troubleshooting/) to seek assistance. + +## For more information + + * [Federation proposal](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/proposals/federation.md) details use cases that motivated this work. From 4e81449bb087494438780650bb638d0c4264d6f2 Mon Sep 17 00:00:00 2001 From: Allan Lei Date: Thu, 14 Jul 2016 11:11:43 +0800 Subject: [PATCH 196/348] Incorrect binary download link for linux/amd64 Link pointed to darwin/amd64 instead of linux/amd64 --- docs/getting-started-guides/minikube.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting-started-guides/minikube.md b/docs/getting-started-guides/minikube.md index 7e422e65bc..735d397a7d 100644 --- a/docs/getting-started-guides/minikube.md +++ b/docs/getting-started-guides/minikube.md @@ -62,7 +62,7 @@ to run commands against the cluster. ```shell # linux/amd64 -curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/darwin/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ +curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ # linux/386 curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/386/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/ # linux/arm From dfa61ed831c5ff245f66f4144531e321fd9da10b Mon Sep 17 00:00:00 2001 From: Ronald Petty Date: Wed, 13 Jul 2016 20:58:42 -0700 Subject: [PATCH 197/348] CMD syntax is invalid Argument to 'cat' command was lost. Merged 'cat' and args for exec command syntax. --- docs/user-guide/configmap/index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/user-guide/configmap/index.md b/docs/user-guide/configmap/index.md index 17cc9c154b..9a26dc429d 100644 --- a/docs/user-guide/configmap/index.md +++ b/docs/user-guide/configmap/index.md @@ -362,7 +362,7 @@ spec: containers: - name: test-container image: gcr.io/google_containers/busybox - command: [ "/bin/sh", "cat", "/etc/config/special.how" ] + command: [ "/bin/sh", "cat /etc/config/special.how" ] volumeMounts: - name: config-volume mountPath: /etc/config @@ -390,7 +390,7 @@ spec: containers: - name: test-container image: gcr.io/google_containers/busybox - command: [ "/bin/sh", "cat", "/etc/config/path/to/special-key" ] + command: [ "/bin/sh", "cat /etc/config/path/to/special-key" ] volumeMounts: - name: config-volume mountPath: /etc/config From d516a86a7aadc1faf173d1a03848487d4b263947 Mon Sep 17 00:00:00 2001 From: Pete Miller Date: Thu, 14 Jul 2016 15:03:27 +0100 Subject: [PATCH 198/348] fix links to volume examples --- docs/user-guide/volumes.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/user-guide/volumes.md b/docs/user-guide/volumes.md index 433deae10f..58e1ca070c 100644 --- a/docs/user-guide/volumes.md +++ b/docs/user-guide/volumes.md @@ -236,7 +236,7 @@ writers simultaneously. __Important: You must have your own NFS server running with the share exported before you can use it__ -See the [NFS example](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/nfs/) for more details. +See the [NFS example](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/volumes/nfs) for more details. ### iscsi @@ -255,7 +255,7 @@ and then serve it in parallel from as many pods as you need. Unfortunately, iSCSI volumes can only be mounted by a single consumer in read-write mode - no simultaneous writers allowed. -See the [iSCSI example](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/iscsi/) for more details. +See the [iSCSI example](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/volumes/iscsi) for more details. ### flocker @@ -270,7 +270,7 @@ can be "handed off" between pods as required. __Important: You must have your own Flocker installation running before you can use it__ -See the [Flocker example](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/flocker/) for more details. +See the [Flocker example](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/volumes/flocker) for more details. ### glusterfs @@ -285,7 +285,7 @@ simultaneously. __Important: You must have your own GlusterFS installation running before you can use it__ -See the [GlusterFS example](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/glusterfs/) for more details. +See the [GlusterFS example](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/volumes/glusterfs) for more details. ### rbd @@ -305,7 +305,7 @@ and then serve it in parallel from as many pods as you need. Unfortunately, RBD volumes can only be mounted by a single consumer in read-write mode - no simultaneous writers allowed. -See the [RBD example](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/rbd/) for more details. +See the [RBD example](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/volumes/rbd) for more details. ### gitRepo @@ -371,14 +371,14 @@ A `FlexVolume` enables users to mount vendor volumes into a pod. It expects vend drivers are installed in the volume plugin path on each kubelet node. This is an alpha feature and may change in future. -More details are in [here](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/flexvolume/README.md) +More details are in [here](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/volumes/flexvolume/README.md) ### AzureFileVolume A `AzureFileVolume` is used to mount a Microsoft Azure File Volume (SMB 2.1 and 3.0) into a Pod. -More details can be found [here](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/azure_file/README.md) +More details can be found [here](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/volumes/azure_file/README.md) ### vsphereVirtualDisk @@ -429,4 +429,4 @@ pods. In the future, we expect that `emptyDir` and `hostPath` volumes will be able to request a certain amount of space using a [resource](/docs/user-guide/compute-resources) specification, and to select the type of media to use, for clusters that have -several media types. \ No newline at end of file +several media types. From 25dc768bdce9f9d094601dbe95a118d90a345728 Mon Sep 17 00:00:00 2001 From: Matthew Wong Date: Thu, 14 Jul 2016 16:08:41 -0400 Subject: [PATCH 199/348] Add section on label selectors to PVC doc --- docs/user-guide/persistent-volumes/index.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/docs/user-guide/persistent-volumes/index.md b/docs/user-guide/persistent-volumes/index.md index ae465eea00..b8c73113c1 100644 --- a/docs/user-guide/persistent-volumes/index.md +++ b/docs/user-guide/persistent-volumes/index.md @@ -139,6 +139,11 @@ spec: resources: requests: storage: 8Gi + selector: + matchLabels: + release: "stable" + matchExpressions: + - {key: environment, operator: In, values: [dev]} ``` ### Access Modes @@ -149,6 +154,15 @@ Claims use the same conventions as volumes when requesting storage with specific Claims, like pods, can request specific quantities of a resource. In this case, the request is for storage. The same [resource model](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/resources.md) applies to both volumes and claims. +### Selector + +Claims can specify a [label selector](/docs/user-guide/labels/#label-selectors) to further filter the set of volumes. Only the volumes whose labels match the selector can be bound to the claim. The selector can consist of two fields: + +* matchLabels - the volume must have a label with this value +* matchExpressions - a list of requirements made by specifying key, list of values, and operator that relates the key and values. Valid operators include In, NotIn, Exists, and DoesNotExist. + +All of the requirements, from both `matchLabels` and `matchExpressions` are ANDed together – they must all be satisfied in order to match. + ## Claims As Volumes Pods access storage by using the claim as a volume. Claims must exist in the same namespace as the pod using the claim. The cluster finds the claim in the pod's namespace and uses it to get the `PersistentVolume` backing the claim. The volume is then mounted to the host and into the pod. From 3eb90da6d417c6080c2753855e0eb69c43bd6d35 Mon Sep 17 00:00:00 2001 From: Johanan Lieberman Date: Fri, 15 Jul 2016 00:30:29 +0300 Subject: [PATCH 200/348] Update aws.md Full S3 access is required in order to successfully run `kube-up.sh`. --- docs/getting-started-guides/aws.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting-started-guides/aws.md b/docs/getting-started-guides/aws.md index 319393c03c..3fb100dce6 100644 --- a/docs/getting-started-guides/aws.md +++ b/docs/getting-started-guides/aws.md @@ -8,7 +8,7 @@ 1. You need an AWS account. Visit [http://aws.amazon.com](http://aws.amazon.com) to get started 2. Install and configure [AWS Command Line Interface](http://aws.amazon.com/cli) -3. You need an AWS [instance profile and role](http://docs.aws.amazon.com/IAM/latest/UserGuide/instance-profiles.html) with EC2 full access. +3. You need an AWS [instance profile and role](http://docs.aws.amazon.com/IAM/latest/UserGuide/instance-profiles.html) with EC2 full access and S3 full access. NOTE: This script use the 'default' AWS profile by default. You may explicitly set AWS profile to use using the `AWS_DEFAULT_PROFILE` environment variable: From ff50399b6f09f78611fc631a2ebc78782cc81e14 Mon Sep 17 00:00:00 2001 From: Johanan Lieberman Date: Fri, 15 Jul 2016 02:48:52 +0300 Subject: [PATCH 201/348] Add missing closing parantheses --- docs/admin/multiple-zones.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/admin/multiple-zones.md b/docs/admin/multiple-zones.md index 68383ea003..630fc21b63 100644 --- a/docs/admin/multiple-zones.md +++ b/docs/admin/multiple-zones.md @@ -6,7 +6,7 @@ Kubernetes 1.2 adds support for running a single cluster in multiple failure zones (GCE calls them simply "zones", AWS calls them "availability zones", here we'll refer to them as "zones"). This is a lightweight version of a broader Cluster Federation feature (previously referred to by the affectionate -nickname ["Ubernetes"](https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/federation.md). +nickname ["Ubernetes"](https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/federation.md)). Full Cluster Federation allows combining separate Kubernetes clusters running in different regions or cloud providers (or on-premise data centers). However, many From ef8005c11231ddd842f41d56cccabf72fc39bdac Mon Sep 17 00:00:00 2001 From: Matt Dupre Date: Thu, 14 Jul 2016 17:23:30 -0700 Subject: [PATCH 202/348] Correct hyphenation of net.beta.kubernetes.io/network-policy --- docs/user-guide/networkpolicies.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/user-guide/networkpolicies.md b/docs/user-guide/networkpolicies.md index 52ecdf6c99..5fb8008eff 100644 --- a/docs/user-guide/networkpolicies.md +++ b/docs/user-guide/networkpolicies.md @@ -15,13 +15,13 @@ You must enable the `extensions/v1beta/networkpolicies` runtime config in your a You must also be using a networking solution which supports `NetworkPolicy` - simply creating the resource without a controller to implement it will have no effect. -## Configuring Namespace Isolation Policy +## Configuring Namespace Isolation Policy Isolation can be configured on a per-namespace basis. Once isolation is configured on a namespace it will be applied to all pods in that namespace. Currently, only isolation policy on inbound traffic (ingress) can be defined. -The following ingress isolation types being supported: +The following ingress isolation types being supported: -- `DefaultDeny`: Pods in the namespace will be inaccessible from any source except the pod's local node. +- `DefaultDeny`: Pods in the namespace will be inaccessible from any source except the pod's local node. Ingress isolation can be enabled using an annotation on the Namespace. @@ -41,7 +41,7 @@ metadata: To configure the annotation via `kubectl`: ```shell{% raw %} -kubectl annotate ns "net.beta.kubernetes.io/networkpolicy={\"ingress\": {\"isolation\": \"DefaultDeny\"}}" +kubectl annotate ns "net.beta.kubernetes.io/network-policy={\"ingress\": {\"isolation\": \"DefaultDeny\"}}" {% endraw %}``` ## The `NetworkPolicy` Resource @@ -75,6 +75,6 @@ __Mandatory Fields__: As with all other Kubernetes config, a `NetworkPolicy` nee __spec__: `NetworkPolicy` [spec](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/devel/api-conventions.md#spec-and-status) has all the information needed to define a network isolation policy in the deployed controller. -__podSelector__: Each `NetworkPolicy` includes a `podSelector` which selects the grouping of pods to which the `ingress` rules in the policy apply. +__podSelector__: Each `NetworkPolicy` includes a `podSelector` which selects the grouping of pods to which the `ingress` rules in the policy apply. -__ingress__: Each `NetworkPolicy` includes a list of whitelist `ingress` rules. Each rule allows traffic which matches both the `from` and `ports` sections. +__ingress__: Each `NetworkPolicy` includes a list of whitelist `ingress` rules. Each rule allows traffic which matches both the `from` and `ports` sections. From f1d7b1c9c5406dad6ceaf77f33b98a7deeb3d953 Mon Sep 17 00:00:00 2001 From: Jimmy Cuadra Date: Thu, 14 Jul 2016 19:37:39 -0700 Subject: [PATCH 203/348] Use correct API group name for RBAC. --- docs/admin/authorization.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/admin/authorization.md b/docs/admin/authorization.md index 6d555fbad7..312bbb1d05 100644 --- a/docs/admin/authorization.md +++ b/docs/admin/authorization.md @@ -175,7 +175,7 @@ As of 1.3 RBAC mode is in alpha and considered experimental. To use RBAC, you must both enable the authorization module with `--authorization-mode=RBAC`, and [enable the API version]( docs/admin/cluster-management.md/#Turn-on-or-off-an-api-version-for-your-cluster), -with a `--runtime-config=` that includes `rbac.authorization/v1alpha1`. +with a `--runtime-config=` that includes `rbac.authorization.k8s.io/v1alpha1`. ### Roles, RolesBindings, ClusterRoles, and ClusterRoleBindings From 905703bfcd4f27660d33494b84fd906e031f58bc Mon Sep 17 00:00:00 2001 From: jatins Date: Fri, 15 Jul 2016 17:49:46 +0530 Subject: [PATCH 204/348] documentation as suggested #338 --- docs/getting-started-guides/minikube.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/getting-started-guides/minikube.md b/docs/getting-started-guides/minikube.md index 735d397a7d..3562de56df 100644 --- a/docs/getting-started-guides/minikube.md +++ b/docs/getting-started-guides/minikube.md @@ -123,15 +123,15 @@ kubectl get nodes ``` Minikube contains a built-in Docker daemon that for running containers. -If you use another Docker daemon for building your containers, you will have to publish them to a registry before minikube can pull them. +If you use another Docker daemon for building your containers, you will have to publish them to a registry before minikube can pull them. You can use minikube's built in Docker daemon to avoid this extra step of pushing your images. Use the built-in Docker daemon with: ```shell eval $(minikube docker-env) ``` - -This command sets up the Docker environment variables so a Docker client can communicate with the minikube Docker daemon: +This command sets up the Docker environment variables so a Docker client can communicate with the minikube Docker daemon. +Minikube currently supports only docker version 1.11.1 on the server, which is what is supported by Kubernetes 1.3. With a newer docker version you'll get this [issue](https://github.com/kubernetes/minikube/issues/338). ```shell docker ps From 7053058632883f3753357be8437b1fa4ccd1c7a9 Mon Sep 17 00:00:00 2001 From: Mike Spreitzer Date: Fri, 15 Jul 2016 12:00:34 -0400 Subject: [PATCH 205/348] Fixed rkt/cni typo and noted use of ca-key.pem for worker Fixed a typo and an oversight in the instructions for deploying Kubernetes with Calico on Ubuntu. Fixes #614 Fixes #616 --- docs/getting-started-guides/ubuntu-calico.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/getting-started-guides/ubuntu-calico.md b/docs/getting-started-guides/ubuntu-calico.md index fb12cae859..4ebb225388 100644 --- a/docs/getting-started-guides/ubuntu-calico.md +++ b/docs/getting-started-guides/ubuntu-calico.md @@ -175,7 +175,7 @@ The following steps should be run on each Kubernetes node. ### Configure TLS Worker nodes require three keys: `ca.pem`, `worker.pem`, and `worker-key.pem`. We've already generated -`ca.pem` for use on the Master. The worker public/private keypair should be generated for each Kubernetes node. +`ca.pem` and `ca-key.pem` for use on the Master. The worker public/private keypair should be generated for each Kubernetes node. 1. Create the file `worker-openssl.cnf` with the following contents. @@ -192,7 +192,7 @@ Worker nodes require three keys: `ca.pem`, `worker.pem`, and `worker-key.pem`. IP.1 = $ENV::WORKER_IP ``` -2. Generate the necessary TLS assets for this worker. This relies on the worker's IP address, and the `ca.pem` file generated earlier in the guide. +2. Generate the necessary TLS assets for this worker. This relies on the worker's IP address, and the `ca.pem` and `ca-key.pem` files generated earlier in the guide. ```shell # Export this worker's IP address. @@ -304,7 +304,7 @@ On your compute nodes, it is important that you install Calico before Kubernetes mkdir -p /etc/cni/net.d # Make the network configuration file - cat >/etc/rkt/net.d/10-calico.conf </etc/cni/net.d/10-calico.conf < Date: Sun, 10 Jul 2016 17:52:03 -0400 Subject: [PATCH 206/348] Update AWS docs for 1.3 * We are using Debian by default * New instance sizes * We never used OVS * Add link to kops --- docs/getting-started-guides/aws.md | 30 ++++++++++++++++------------ docs/getting-started-guides/index.md | 3 ++- 2 files changed, 19 insertions(+), 14 deletions(-) diff --git a/docs/getting-started-guides/aws.md b/docs/getting-started-guides/aws.md index 3fb100dce6..717590ba38 100644 --- a/docs/getting-started-guides/aws.md +++ b/docs/getting-started-guides/aws.md @@ -7,11 +7,11 @@ ## Prerequisites 1. You need an AWS account. Visit [http://aws.amazon.com](http://aws.amazon.com) to get started -2. Install and configure [AWS Command Line Interface](http://aws.amazon.com/cli) -3. You need an AWS [instance profile and role](http://docs.aws.amazon.com/IAM/latest/UserGuide/instance-profiles.html) with EC2 full access and S3 full access. +2. Install and configure the [AWS Command Line Interface](http://aws.amazon.com/cli) +3. We recommend installing using an account which has full access to the AWS APIs. NOTE: This script use the 'default' AWS profile by default. -You may explicitly set AWS profile to use using the `AWS_DEFAULT_PROFILE` environment variable: +You may explicitly set the AWS profile to use using the `AWS_DEFAULT_PROFILE` environment variable: ```shell export AWS_DEFAULT_PROFILE=myawsprofile @@ -36,7 +36,7 @@ This process takes about 5 to 10 minutes. Once the cluster is up, the IP address as well as information about the default services running in the cluster (monitoring, logging, dns). User credentials and security tokens are written in `~/.kube/config`, they will be necessary to use the CLI or the HTTP Basic Auth. -By default, the script will provision a new VPC and a 4 node k8s cluster in us-west-2a (Oregon) with EC2 instances running on Ubuntu. +By default, the script will provision a new VPC and a 4 node k8s cluster in us-west-2a (Oregon) with EC2 instances running on Debian. You can override the variables defined in [config-default.sh](http://releases.k8s.io/{{page.githubbranch}}/cluster/aws/config-default.sh) to change this behavior as follows: ```shell @@ -52,11 +52,11 @@ export INSTANCE_PREFIX=k8s If you don't specify master and minion sizes, the scripts will attempt to guess the correct size of the master and worker nodes based on `${NUM_NODES}`. In -version 1.2 these default are: +version 1.3 these default are: -* For the master, for clusters of less than 150 nodes it will use an - `m3.medium`, for clusters of greater than 150 nodes it will use an - `m3.large`. +* For the master, for clusters of less than 5 nodes it will use an + `m3.medium`, for 6-10 nodes it will use an `m3.large`; + for 11-100 nodes it will use an `m3.xlarge`. * For worker nodes, for clusters less than 50 nodes it will use a `t2.micro`, for clusters between 50 and 150 nodes it will use a `t2.small` and for @@ -95,7 +95,10 @@ NOTE: If using an existing keypair named "kubernetes" then you must set the `AWS ### Alternatives -CoreOS maintains [a CLI tool](https://coreos.com/kubernetes/docs/latest/kubernetes-on-aws.html), `kube-aws` that will create and manage a Kubernetes cluster based on [CoreOS](http://www.coreos.com), using AWS tools: EC2, CloudFormation and Autoscaling. +* [kops](https://github.com/kubernetes/kops) "kubernetes-ops" is a complete Kubernetes cluster lifecycle management tool, + that supports AWS. + +* CoreOS maintains [a CLI tool](https://coreos.com/kubernetes/docs/latest/kubernetes-on-aws.html), `kube-aws` that will create and manage a Kubernetes cluster based on [CoreOS](http://www.coreos.com), using AWS tools: EC2, CloudFormation and Autoscaling. ## Getting started with your cluster @@ -139,10 +142,11 @@ cluster/kube-down.sh ## Support Level -IaaS Provider | Config. Mgmt | OS | Networking | Docs | Conforms | Support Level --------------------- | ------------ | ------ | ---------- | --------------------------------------------- | ---------| ---------------------------- -AWS | CoreOS | CoreOS | flannel | [docs](/docs/getting-started-guides/aws) | | Community -AWS | Saltstack | Ubuntu | OVS | [docs](/docs/getting-started-guides/aws) | | Community ([@justinsb](https://github.com/justinsb)) +IaaS Provider | Config. Mgmt | OS | Networking | Docs | Conforms | Support Level +-------------------- | ------------ | ------------- | ---------- | --------------------------------------------- | ---------| ---------------------------- +AWS | Saltstack | Debian/Ubuntu | k8s (VPC) | [docs](/docs/getting-started-guides/aws) | | Community ([@justinsb](https://github.com/justinsb)) +AWS | kops | Debian | k8s (VPC) | [docs](https://github.com/kubernetes/kops) | | Community ([@justinsb](https://github.com/justinsb)) +AWS | CoreOS | CoreOS | flannel | [docs](/docs/getting-started-guides/aws) | | Community For support level information on all solutions, see the [Table of solutions](/docs/getting-started-guides/#table-of-solutions) chart. diff --git a/docs/getting-started-guides/index.md b/docs/getting-started-guides/index.md index d8d70b5be8..c9eab02521 100644 --- a/docs/getting-started-guides/index.md +++ b/docs/getting-started-guides/index.md @@ -144,7 +144,8 @@ Bare-metal | custom | CentOS | _none_ | [docs](/docs/gettin AWS | Juju | Ubuntu | flannel | [docs](/docs/getting-started-guides/juju) | | [Community](https://github.com/whitmo/bundle-kubernetes) ( [@whit](https://github.com/whitmo), [@matt](https://github.com/mbruzek), [@chuck](https://github.com/chuckbutler) ) OpenStack/HPCloud | Juju | Ubuntu | flannel | [docs](/docs/getting-started-guides/juju) | | [Community](https://github.com/whitmo/bundle-kubernetes) ( [@whit](https://github.com/whitmo), [@matt](https://github.com/mbruzek), [@chuck](https://github.com/chuckbutler) ) Joyent | Juju | Ubuntu | flannel | [docs](/docs/getting-started-guides/juju) | | [Community](https://github.com/whitmo/bundle-kubernetes) ( [@whit](https://github.com/whitmo), [@matt](https://github.com/mbruzek), [@chuck](https://github.com/chuckbutler) ) -AWS | Saltstack | Ubuntu | OVS | [docs](/docs/getting-started-guides/aws) | | Community ([@justinsb](https://github.com/justinsb)) +AWS | Saltstack | Debian | AWS | [docs](/docs/getting-started-guides/aws) | | Community ([@justinsb](https://github.com/justinsb)) +AWS | kops | Debian | AWS | [docs](https://github.com/kubernetes/kops) | | Community ([@justinsb](https://github.com/justinsb)) Bare-metal | custom | Ubuntu | Calico | [docs](/docs/getting-started-guides/ubuntu-calico) | | Community ([@djosborne](https://github.com/djosborne)) Bare-metal | custom | Ubuntu | flannel | [docs](/docs/getting-started-guides/ubuntu) | | Community ([@resouer](https://github.com/resouer), [@WIZARD-CXY](https://github.com/WIZARD-CXY)) libvirt/KVM | CoreOS | CoreOS | libvirt/KVM | [docs](/docs/getting-started-guides/libvirt-coreos) | | Community ([@lhuard1A](https://github.com/lhuard1A)) From b122a8ddea2bb4df5fdb85b102dfd1a3716897a5 Mon Sep 17 00:00:00 2001 From: Quinton Hoole Date: Fri, 15 Jul 2016 10:05:03 -0700 Subject: [PATCH 207/348] Address review feedback. --- _data/guides.yml | 2 + docs/user-guide/connecting-applications.md | 7 +++ .../federation/federated-services.md | 58 +++++++++---------- 3 files changed, 38 insertions(+), 29 deletions(-) diff --git a/_data/guides.yml b/_data/guides.yml index 732a09fb98..01501b6256 100644 --- a/_data/guides.yml +++ b/_data/guides.yml @@ -82,6 +82,8 @@ toc: path: /docs/user-guide/load-balancer/ - title: Configuring Your Cloud Provider's Firewalls path: /docs/user-guide/services-firewalls/ + - title: Cross-cluster Service Discovery using Federated Services + path: /docs/user-guide/federation/federated-services/ - title: Containers and Pods section: diff --git a/docs/user-guide/connecting-applications.md b/docs/user-guide/connecting-applications.md index 9854a05a16..f4ada5393e 100644 --- a/docs/user-guide/connecting-applications.md +++ b/docs/user-guide/connecting-applications.md @@ -286,6 +286,13 @@ $ kubectl describe service my-nginx LoadBalancer Ingress: a320587ffd19711e5a37606cf4a74574-1142138393.us-east-1.elb.amazonaws.com ... ``` +## Further reading + +Kubernetes also supports Federated Services, which can span multiple +clusters and cloud providers, to provide increased availability, +bettern fault tolerance and greater scalability for your services. See +the [Federated Services User Guide](/docs/user-guide/federation/federated-services/) +for further information. ## What's next? diff --git a/docs/user-guide/federation/federated-services.md b/docs/user-guide/federation/federated-services.md index 16eb967f0e..2a49d625ef 100644 --- a/docs/user-guide/federation/federated-services.md +++ b/docs/user-guide/federation/federated-services.md @@ -1,6 +1,5 @@ --- --- -# Kubernetes Federated Services and Cross-Cluster Service Discovery This guide explains how to use Kubernetes Federated Services to deploy a common Service across multiple Kubernetes clusters. This makes it @@ -48,7 +47,7 @@ automatically find the local shard of the Federated Service in their cluster if it exists and is healthy, or the closest healthy shard in a different cluster if it does not. -## Hybrid Cloud Capabilities +## Hybrid cloud capabilities Federations of Kubernetes Clusters can include clusters running in different cloud providers (e.g. Google Cloud, AWS), and on-premises @@ -61,11 +60,11 @@ Federation API Server (See the Thereafter, your applications and services can span different clusters and cloud providers as described in more detail below. -## Creating a Federated Service +## Creating a federated service This is done in the usual way, for example: -``` +``` shell kubectl --context=federation-cluster create -f services/nginx.yaml ``` @@ -82,7 +81,7 @@ underlying your federation. You can verify this by checking in each of the underlying clusters, for example: -``` +``` shell kubectl --context=gce-asia-east1a get services nginx NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE nginx 10.63.250.98 104.199.136.89 80/TCP 9m @@ -100,11 +99,9 @@ Federated Service - either way, the end result will be the same). The status of your Federated Service will automatically reflect the real-time status of the underlying Kubernetes services, for example: -``` -kubectl --context=federation-cluster describe services nginx -``` +``` shell +$kubectl --context=federation-cluster describe services nginx -``` Name: nginx Namespace: default Labels: run=nginx @@ -123,8 +120,10 @@ correspond with the 'LoadBalancer Ingress' addresses of all of the underlying Kubernetes services (once these have been allocated - this may take a few seconds). For inter-cluster and inter-cloud-provider networking between service shards to work correctly, your services -need to have an externally visible IP address. Service Type: -Loadbalancer is typically used for this. +need to have an externally visible IP address. [Service Type: +Loadbalancer](/docs/user-guide/services/#type-loadbalancer) +is typically used for this, although other options +(e.g. [External IP's](/docs/user-guide/services/#external-ips)) exist. Note also that we have not yet provisioned any backend Pods to receive the network traffic directed to these addresses (i.e. 'Service @@ -133,7 +132,7 @@ be healthy service shards, and has accordingly not yet added their addresses to the DNS records for this Federated Service (more on this aspect later). -## Adding Backend Pods +## Adding backend pods To render the underlying service shards healthy, we need to add backend Pods behind them. This is currently done directly against the @@ -142,7 +141,7 @@ Federation server will be able to do all this for you with a single command, to save you the trouble). For example, to create backend Pods in 13 underlying clusters: -``` +``` shell for CLUSTER in asia-east1-c asia-east1-a asia-east1-b \ europe-west1-d europe-west1-c europe-west1-b \ us-central1-f us-central1-a us-central1-b us-central1-c \ @@ -151,21 +150,22 @@ do kubectl --context=$CLUSTER run nginx --image=nginx:1.11.1-alpine --port=80 done ``` + Note that `kubectl run` automatically adds the `run=nginx` labels required to associate the backend pods with their services. -## Verifying Public DNS Records +## Verifying public DNS records Once the above Pods have successfully started and have begun listening -for connections, Kubernetes in each cluster (via automatic health -checks) will report them as healthy endpoints of the service in that -cluster. The Cluster Federation will in turn consider each of these +for connections, Kubernetes will report them as healthy endpoints of +the service in that cluster (via automatic health checks). The Cluster +Federation will in turn consider each of these service 'shards' to be healthy, and place them in serving by automatically configuring corresponding public DNS records. You can use your preferred interface to your configured DNS provider to verify this. For example, if your Federation is configured to use Google Cloud DNS, and a managed DNS domain 'example.com': -``` +``` shell $ gcloud dns managed-zones describe example-dot-com creationTime: '2016-06-26T18:18:39.229Z' description: Example domain for Kubernetes Cluster Federation @@ -180,7 +180,7 @@ nameServers: - ns-cloud-a4.googledomains.com. ``` -``` +``` shell $ gcloud dns record-sets list --zone example-dot-com NAME TYPE TTL DATA example.com. NS 21600 ns-cloud-e1.googledomains.com., ns-cloud-e2.googledomains.com. @@ -202,11 +202,11 @@ nginx.mynamespace.myfederation.svc.europe-west1-d.example.com. CNAME 180 Note: If your Federation is configured to use AWS Route53, you can use one of the equivalent AWS tools, for example: -``` +``` shell $aws route53 list-hosted-zones ``` and -``` +``` shell $aws route53 list-resource-record-sets --hosted-zone-id Z3ECL0L9QLOVBX ``` @@ -234,9 +234,9 @@ then select any one of the returned addresses to initiate a network connection (and fail over automatically to one of the other equivalent addresses if required). -## Discovering a Federated Service +## Discovering a federated service -### From Pods Inside your Federated Clusters +### From pods inside your federated clusters By default, Kubernetes clusters come pre-configured with a cluster-local DNS server ('KubeDNS'), as well as an intelligently @@ -296,7 +296,7 @@ if the Pod issuing the lookup is located in the U.S., and irrespective of whether or not there are healthy shards of the service in the U.S. This is useful for remote monitoring and other similar applications. -### From Other Clients Outside your Federated Clusters +### From other clients outside your federated clusters Much of the above discussion applies equally to external clients, except that the automatic DNS expansion described is no longer @@ -306,7 +306,7 @@ regional or global name. For convenience reasons, it is often a good idea to manually configure additional static CNAME records in your service, for example: -``` +``` shell eu.nginx.acme.com CNAME nginx.mynamespace.myfederation.svc.europe-west1.example.com. us.nginx.acme.com CNAME nginx.mynamespace.myfederation.svc.us-central1.example.com. nginx.acme.com CNAME nginx.mynamespace.myfederation.svc.example.com. @@ -317,7 +317,7 @@ home continent. All of the required failover is handled for you automatically by Kubernetes Cluster Federation. Future releases will improve upon this even further. -## Handling Failures of Backend Pods and Whole Clusters +## Handling failures of backend pods and whole clusters Standard Kubernetes service cluster-IP's already ensure that non-responsive individual Pod endpoints are automatically taken out of @@ -339,7 +339,7 @@ IP's in less time than that given appropriate configuration. ## Troubleshooting -#### I cannot connect to my Cluster Federation API +#### I cannot connect to my cluster federation API Check that your 1. Client (typically kubectl) is correctly configured (including API endpoints and login credentials), and @@ -348,7 +348,7 @@ Check that your See the [federation admin guide](/docs/admin/federation/) to learn how to bring up a cluster federation correctly (or have your cluster administrator do this for you), and how to correctly configure your client. -#### I can create a Federated Service successfully against the Cluster Federation API, but no matching services are created in my underlying clusters +#### I can create a federated service successfully against the cluster federation API, but no matching services are created in my underlying clusters Check that: 1. Your clusters are correctly registered in the Cluster Federation API (`kubectl describe clusters`) @@ -356,7 +356,7 @@ Check that: 3. That the login credentials provided to the Cluster Federation API for the clusters have the correct authorization and quota to create services in the relevant namespace in the clusters. Again you should see associated error messages providing more detail in the above log file if this is not the case. 4. Whether any other error is preventing the service creation operation from succeeding (look for `service-controller` errors in the output of `kubectl logs federation-controller-manager --namespace federation`). -#### I can create a Federated Service successfully, but no matching DNS records are created in my DNS provider. +#### I can create a federated service successfully, but no matching DNS records are created in my DNS provider. Check that: 1. Your federation name, DNS provider, DNS domain name are configured correctly. Consult the [federation admin guide](/docs/admin/federation/) or [tutorial](https://github.com/kelseyhightower/kubernetes-cluster-federation) to learn From 180866b0dd043a1d7bdd26089a830ed23673b3a6 Mon Sep 17 00:00:00 2001 From: jatins Date: Fri, 15 Jul 2016 23:05:16 +0530 Subject: [PATCH 208/348] list Minikube under "local-machine solutions" List Minikube under "local-machine solutions" as the recommended solution. --- docs/getting-started-guides/index.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/getting-started-guides/index.md b/docs/getting-started-guides/index.md index d8d70b5be8..89c4ded154 100644 --- a/docs/getting-started-guides/index.md +++ b/docs/getting-started-guides/index.md @@ -31,7 +31,8 @@ But their size and availability is limited to that of a single machine. The local-machine solutions are: -- [Local Docker-based](/docs/getting-started-guides/docker) (recommended starting point) +- [Minikube](/docs/getting-started-guides/minikube/) (recommended) +- [Local Docker-based](/docs/getting-started-guides/docker) - [No-VM local cluster](/docs/getting-started-guides/locally) (Linux only) - [Vagrant](/docs/getting-started-guides/vagrant) From d9853f020ad465a8ac4e2b05971373db7555824f Mon Sep 17 00:00:00 2001 From: Dalton Hubble Date: Fri, 15 Jul 2016 14:52:20 -0700 Subject: [PATCH 209/348] Add official CoreOS bare metal guide --- docs/getting-started-guides/coreos/index.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/getting-started-guides/coreos/index.md b/docs/getting-started-guides/coreos/index.md index 46d058603d..59fe04a1f2 100644 --- a/docs/getting-started-guides/coreos/index.md +++ b/docs/getting-started-guides/coreos/index.md @@ -16,6 +16,10 @@ Guide and CLI tool for setting up a multi-node cluster on AWS. CloudFormation is
+[**Bare Metal Multi-Node**](https://coreos.com/kubernetes/docs/latest/kubernetes-on-baremetal.html#automated-provisioning) + +Guide and HTTP/API service for PXE booting and provisioning a multi-node cluster on bare metal. [Ignition](https://coreos.com/ignition/docs/latest/) is used to provision a master and multiple workers on the first boot from disk. + [**Vagrant Multi-Node**](https://coreos.com/kubernetes/docs/latest/kubernetes-on-vagrant.html) Guide to setting up a multi-node cluster on Vagrant. The deployer can independently configure the number of etcd nodes, master nodes, and worker nodes to bring up a fully HA control plane. From c927ce104e4e039b08a80cacc5c275559ee0f8c1 Mon Sep 17 00:00:00 2001 From: Mayank Kumar Date: Sat, 16 Jul 2016 00:08:56 -0700 Subject: [PATCH 210/348] fix doc bug 64Mb is 2^26 --- docs/user-guide/compute-resources.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/compute-resources.md b/docs/user-guide/compute-resources.md index 0d90e117e2..c737486159 100644 --- a/docs/user-guide/compute-resources.md +++ b/docs/user-guide/compute-resources.md @@ -41,7 +41,7 @@ type is the sum of the resource requests/limits of that type for each container unset values treated as zero (or equal to default values in some cluster configurations). The following pod has two containers. Each has a request of 0.25 core of cpu and 64MiB -(220 bytes) of memory and a limit of 0.5 core of cpu and 128MiB of memory. The pod can +(226 bytes) of memory and a limit of 0.5 core of cpu and 128MiB of memory. The pod can be said to have a request of 0.5 core and 128 MiB of memory and a limit of 1 core and 256MiB of memory. From 2fcf8d31c06efc29ade94ee507f10fcd6a9e1882 Mon Sep 17 00:00:00 2001 From: Michael Wasser Date: Sat, 16 Jul 2016 14:35:45 -0700 Subject: [PATCH 211/348] Update jobs future work See https://github.com/kubernetes/kubernetes/issues/2156#issuecomment-223385808 --- docs/user-guide/jobs.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/jobs.md b/docs/user-guide/jobs.md index 3f54116f75..ab193f6f50 100644 --- a/docs/user-guide/jobs.md +++ b/docs/user-guide/jobs.md @@ -372,4 +372,4 @@ object, but complete control over what pods are created and how work is assigned ## Future work -Support for creating Jobs at specified times/dates (i.e. cron) is expected in [1.3](https://github.com/kubernetes/kubernetes/pull/11980). +Support for creating Jobs at specified times/dates (i.e. cron) is expected in [1.4](https://github.com/kubernetes/kubernetes/pull/11980). From 3676d73665ded55b55e357104914860fa5950766 Mon Sep 17 00:00:00 2001 From: Eyal Levin Date: Sun, 17 Jul 2016 15:16:44 +0300 Subject: [PATCH 212/348] Add line break between commands Currently the second command seems to "disappear" between the two outputs. This change tries to mitigate it. --- docs/user-guide/secrets/index.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/user-guide/secrets/index.md b/docs/user-guide/secrets/index.md index 3ab0ca8e38..822cd1399b 100644 --- a/docs/user-guide/secrets/index.md +++ b/docs/user-guide/secrets/index.md @@ -66,6 +66,7 @@ You can check that the secret was created like this: $ kubectl get secrets NAME TYPE DATA AGE db-user-pass Opaque 2 51s + $ kubectl describe secrets/db-user-pass Name: db-user-pass Namespace: default From 5e9ef0f5931966323dcd13116ec6146b0f9e12cf Mon Sep 17 00:00:00 2001 From: Eyal Levin Date: Sun, 17 Jul 2016 15:39:54 +0300 Subject: [PATCH 213/348] Fix decode flag I'm on Ubuntu 14.04 and `-D` is invalid. `-d` gets the required output. --- docs/user-guide/secrets/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/secrets/index.md b/docs/user-guide/secrets/index.md index 3ab0ca8e38..1f0e0a236a 100644 --- a/docs/user-guide/secrets/index.md +++ b/docs/user-guide/secrets/index.md @@ -154,7 +154,7 @@ type: Opaque Decode the password field: ```shell -$ echo "MWYyZDFlMmU2N2Rm" | base64 -D +$ echo "MWYyZDFlMmU2N2Rm" | base64 -d 1f2d1e2e67df ``` From 22c41d2dfcb5a9377a3511a771c4b55d4316175c Mon Sep 17 00:00:00 2001 From: Eyal Levin Date: Sun, 17 Jul 2016 16:03:13 +0300 Subject: [PATCH 214/348] Remove redundant line The link refers to itself. --- docs/user-guide/secrets/index.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/docs/user-guide/secrets/index.md b/docs/user-guide/secrets/index.md index 3ab0ca8e38..97b5ab89f5 100644 --- a/docs/user-guide/secrets/index.md +++ b/docs/user-guide/secrets/index.md @@ -212,8 +212,6 @@ own `volumeMounts` block, but only one `spec.volumes` is needed per secret. You can package many files into one secret, or use many secrets, whichever is convenient. -See another example of creating a secret and a pod that consumes that secret in a volume [here](/docs/user-guide/secrets/). - **Projection of secret keys to specific paths** We can also control the paths within the volume where Secret keys are projected. From 1c58c914015d432ce5625b344fcadcb0dd00f8a8 Mon Sep 17 00:00:00 2001 From: Eyal Levin Date: Sun, 17 Jul 2016 17:32:05 +0300 Subject: [PATCH 215/348] Remove redundant word --- docs/user-guide/secrets/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/secrets/index.md b/docs/user-guide/secrets/index.md index 3ab0ca8e38..826e5f99a7 100644 --- a/docs/user-guide/secrets/index.md +++ b/docs/user-guide/secrets/index.md @@ -580,7 +580,7 @@ one called, say, `prod-user` with the `prod-db-secret`, and one called, say, ### Use-case: Dotfiles in secret volume In order to make piece of data 'hidden' (ie, in a file whose name begins with a dot character), simply -make that key begin with a dot. For example, when the following secret secret is mounted into a volume: +make that key begin with a dot. For example, when the following secret is mounted into a volume: ```json { From 5f2ba4c2cd28517ec8c0a3960fe37005a1e6d6a8 Mon Sep 17 00:00:00 2001 From: "Matt T. Proud" Date: Sun, 10 Jul 2016 12:58:56 +0200 Subject: [PATCH 216/348] docs: convert curl downloads from HTTP to HTTPS Several command literals in the installation documents for downloading used `curl` with non-HTTPS protocol. They have been converted to HTTPS for both man-in-the-middle risk reduction and consistency with the rest of the documentation corpus. --- docs/getting-started-guides/docker.md | 2 +- docs/getting-started-guides/minikube.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/getting-started-guides/docker.md b/docs/getting-started-guides/docker.md index a1807ce6fd..0b65719cf8 100644 --- a/docs/getting-started-guides/docker.md +++ b/docs/getting-started-guides/docker.md @@ -90,7 +90,7 @@ http://storage.googleapis.com/kubernetes-release/release/${K8S_VERSION}/bin/${GO An example install with `linux/amd64`: ``` -curl -sSL "http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/amd64/kubectl" > /usr/bin/kubectl +curl -sSL "https://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/amd64/kubectl" > /usr/bin/kubectl chmod +x /usr/bin/kubectl ``` diff --git a/docs/getting-started-guides/minikube.md b/docs/getting-started-guides/minikube.md index 735d397a7d..5185244c37 100644 --- a/docs/getting-started-guides/minikube.md +++ b/docs/getting-started-guides/minikube.md @@ -79,7 +79,7 @@ curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/{{page The generic download path is: ``` -http://storage.googleapis.com/kubernetes-release/release/${K8S_VERSION}/bin/${GOOS}/${GOARCH}/${K8S_BINARY} +https://storage.googleapis.com/kubernetes-release/release/${K8S_VERSION}/bin/${GOOS}/${GOARCH}/${K8S_BINARY} ``` ### Starting the cluster From e27902eb02de695f1cbad98edb29193a53692c58 Mon Sep 17 00:00:00 2001 From: Gert Leenders Date: Tue, 19 Jul 2016 07:20:11 +0200 Subject: [PATCH 217/348] Changed deployment definition as suggested --- docs/getting-started-guides/meanstack.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting-started-guides/meanstack.md b/docs/getting-started-guides/meanstack.md index c02c7c6440..b4e501946a 100644 --- a/docs/getting-started-guides/meanstack.md +++ b/docs/getting-started-guides/meanstack.md @@ -20,7 +20,7 @@ Before we jump in and start kube’ing it up, it’s important to understand som * Containers: These are the Docker, rtk, AppC, or whatever Container you are running. You can think of these like subatomic particles; everything is made up of them, but you rarely (if ever) interact with them directly. * Pods: Pods are the basic component of Kubernetes. They are a group of Containers that are scheduled, live, and die together. Why would you want to have a group of containers instead of just a single container? Let’s say you had a log processor, a web server, and a database. If you couldn't use Pods, you would have to bundle the log processor in the web server and database containers, and each time you updated one you would have to update the other. With Pods, you can just reuse the same log processor for both the web server and database. -* Deployments: A Deployment provides declarative updates for Pods and ReplicaSets. You only need to describe the desired state in a Deployment object, and the deployment controller will change the actual state to the desired state at a controlled rate for you. You can define Deployments to create new resources, or replace existing ones by new ones. +* Deployments: A Deployment provides declarative updates for Pods. You can define Deployments to create new Pods, or replace existing Pods. You only need to describe the desired state in a Deployment object, and the deployment controller will change the actual state to the desired state at a controlled rate for you. You can define Deployments to create new resources, or replace existing ones by new ones. * Services: A service is the single point of contact for a group of Pods. For example, let’s say you have a Deployment that creates four copies of a web server pod. A Service will split the traffic to each of the four copies. Services are "permanent" while the pods behind them can come and go, so it’s a good idea to use Services. From ffffd27457dfd727cf3e0f7dd3a709c5200c3fc1 Mon Sep 17 00:00:00 2001 From: Alex Robinson Date: Thu, 7 Jul 2016 23:00:54 +0000 Subject: [PATCH 218/348] Seriously simplify the elasticsearch logging getting started guide. --- .../logging-elasticsearch.md | 226 ++++-------------- 1 file changed, 41 insertions(+), 185 deletions(-) diff --git a/docs/getting-started-guides/logging-elasticsearch.md b/docs/getting-started-guides/logging-elasticsearch.md index d65ff7b9bd..515e981ab7 100644 --- a/docs/getting-started-guides/logging-elasticsearch.md +++ b/docs/getting-started-guides/logging-elasticsearch.md @@ -1,12 +1,18 @@ --- --- -On the Google Compute Engine (GCE) platform the default cluster level logging support targets -[Google Cloud Logging](https://cloud.google.com/logging/docs/) as described at the [Logging](/docs/getting-started-guides/logging) getting -started page. Here we describe how to set up a cluster to ingest logs into Elasticsearch and view them using Kibana as an -alternative to Google Cloud Logging. +On the Google Compute Engine (GCE) platform, the default logging support targets +[Google Cloud Logging](https://cloud.google.com/logging/) as described in the +[Logging](/docs/getting-started-guides/logging) getting-started guide. Here we +describe how to set up a cluster to ingest logs into +[Elasticsearch](https://github.com/elastic/elasticsearch) and view +them using [Kibana](https://github.com/elastic/kibana) as an alternative to +Google Cloud Logging when running on GCE (note that this will not work as +written for Google Container Engine). -To use Elasticsearch and Kibana for cluster logging you should set the following environment variable as shown below: +To use Elasticsearch and Kibana for cluster logging, you should set the +following environment variable as shown below when creating your cluster with +kube-up.sh: ```shell KUBE_LOGGING_DESTINATION=elasticsearch @@ -14,8 +20,8 @@ KUBE_LOGGING_DESTINATION=elasticsearch You should also ensure that `KUBE_ENABLE_NODE_LOGGING=true` (which is the default for the GCE platform). -Now when you create a cluster a message will indicate that the Fluentd node-level log collectors -will target Elasticsearch: +Now, when you create a cluster, a message will indicate that the Fluentd log +collection daemons that run on each node will target Elasticsearch: ```shell $ cluster/kube-up.sh @@ -37,8 +43,9 @@ Created [https://www.googleapis.com/compute/v1/projects/kubernetes-satnam/region +++ Logging using Fluentd to elasticsearch ``` -The node level Fluentd collector pods and the Elasticsearch pods used to ingest cluster logs and the pod for the Kibana -viewer should be running in the kube-system namespace soon after the cluster comes to life. +The per-node Fluentd pods, the Elasticsearch pods, and the Kibana pods should +all be running in the kube-system namespace soon after the cluster comes to +life. ```shell $ kubectl get pods --namespace=kube-system @@ -55,189 +62,38 @@ monitoring-heapster-v4-yl332 1/1 Running 1 2h monitoring-influx-grafana-v1-o79xf 2/2 Running 0 2h ``` -Here we see that for a four node cluster there is a `fluent-elasticsearch` pod running which gathers -the Docker container logs and sends them to Elasticsearch. The Fluentd collector communicates to -a Kubernetes service that maps requests to specific Elasticsearch pods. Similarly, Kibana can also be -accessed via a Kubernetes service definition. +The `fluentd-elasticsearch` pods gather logs from each node and send them to +the `elasticsearch-logging` pods, which are part of a +[service](/docs/user-guide/services/) named `elasticsearch-logging`. These +Elasticsearch pods store the logs and expose them via a REST API. +The `kibana-logging` pod provides a web UI for reading the logs stored in +Elasticsearch, and is part of a service named `kibana-logging`. +The Elasticsearch and Kibana services are both in the `kube-system` namespace +and are not directly exposed via a publicly reachable IP address. To reach them, +follow the instructions for [Accessing services running in a cluster](/docs/user-guide/accessing-the-cluster/#accessing-services-running-on-the-cluster). -```shell -$ kubectl get services --namespace=kube-system -NAME LABELS SELECTOR IP(S) PORT(S) -elasticsearch-logging k8s-app=elasticsearch-logging,kubernetes.io/cluster-service=true,kubernetes.io/name=Elasticsearch k8s-app=elasticsearch-logging 10.0.222.57 9200/TCP -kibana-logging k8s-app=kibana-logging,kubernetes.io/cluster-service=true,kubernetes.io/name=Kibana k8s-app=kibana-logging 10.0.193.226 5601/TCP -kube-dns k8s-app=kube-dns,kubernetes.io/cluster-service=true,kubernetes.io/name=KubeDNS k8s-app=kube-dns 10.0.0.10 53/UDP - 53/TCP -kubernetes component=apiserver,provider=kubernetes 10.0.0.1 443/TCP -monitoring-grafana kubernetes.io/cluster-service=true,kubernetes.io/name=Grafana k8s-app=influxGrafana 10.0.167.139 80/TCP -monitoring-heapster kubernetes.io/cluster-service=true,kubernetes.io/name=Heapster k8s-app=heapster 10.0.208.221 80/TCP -monitoring-influxdb kubernetes.io/cluster-service=true,kubernetes.io/name=InfluxDB k8s-app=influxGrafana 10.0.188.57 8083/TCP -``` - -By default two Elasticsearch replicas are created and one Kibana replica is created. - -```shell -$ kubectl get rc --namespace=kube-system -CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS -elasticsearch-logging-v1 elasticsearch-logging gcr.io/google_containers/elasticsearch:1.4 k8s-app=elasticsearch-logging,version=v1 2 -kibana-logging-v1 kibana-logging gcr.io/google_containers/kibana:1.3 k8s-app=kibana-logging,version=v1 1 -kube-dns-v3 etcd gcr.io/google_containers/etcd:2.0.9 k8s-app=kube-dns,version=v3 1 - kube2sky gcr.io/google_containers/kube2sky:1.9 - skydns gcr.io/google_containers/skydns:2015-03-11-001 -monitoring-heapster-v4 heapster gcr.io/google_containers/heapster:v0.14.3 k8s-app=heapster,version=v4 1 -monitoring-influx-grafana-v1 influxdb gcr.io/google_containers/heapster_influxdb:v0.3 k8s-app=influxGrafana,version=v1 1 - grafana gcr.io/google_containers/heapster_grafana:v0.7 -``` - -The Elasticsearch and Kibana services are not directly exposed via a publicly reachable IP address. Instead, -they can be accessed via the service proxy running at the master. The URLs for accessing Elasticsearch -and Kibana via the service proxy can be found using the `kubectl cluster-info` command. - -```shell -$ kubectl cluster-info -Kubernetes master is running at https://146.148.94.154 -Elasticsearch is running at https://146.148.94.154/api/v1/proxy/namespaces/kube-system/services/elasticsearch-logging -Kibana is running at https://146.148.94.154/api/v1/proxy/namespaces/kube-system/services/kibana-logging -KubeDNS is running at https://146.148.94.154/api/v1/proxy/namespaces/kube-system/services/kube-dns -KubeUI is running at https://146.148.94.154/api/v1/proxy/namespaces/kube-system/services/kube-ui -Grafana is running at https://146.148.94.154/api/v1/proxy/namespaces/kube-system/services/monitoring-grafana -Heapster is running at https://146.148.94.154/api/v1/proxy/namespaces/kube-system/services/monitoring-heapster -InfluxDB is running at https://146.148.94.154/api/v1/proxy/namespaces/kube-system/services/monitoring-influxdb -``` - -Before accessing the logs ingested into Elasticsearch using a browser and the service proxy URL we need to find out -the `admin` password for the cluster using `kubectl config view`. - -```shell -$ kubectl config view -... -- name: kubernetes-satnam_kubernetes-basic-auth - user: - password: 7GlspJ9Q43OnGIJO - username: admin -... -``` - -The first time you try to access the cluster from a browser a dialog box appears asking for the username and password. -Use the username `admin` and provide the basic auth password reported by `kubectl config view` for the -cluster you are trying to connect to. Connecting to the Elasticsearch URL should then give the -status page for Elasticsearch. +If you try accessing the `elasticsearch-logging` service in your browser, you'll +see a status page that looks something like this: ![Elasticsearch Status](/images/docs/es-browser.png) -You can now type Elasticsearch queries directly into the browser. Alternatively you can query Elasticsearch -from your local machine using `curl` but first you need to know what your bearer token is: +You can now type Elasticsearch queries directly into the browser, if you'd +like. See [Elasticsearch's documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-uri-request.html) +for more details on how to do so. -```shell -$ kubectl config view --minify -``` +Alternatively, you can view your cluster's logs using Kibana (again using the +[instructions for accessing a service running in the cluster](/docs/user-guide/accessing-the-cluster/#accessing-services-running-on-the-cluster)). +The first time you visit the Kibana URL you will be presented with a page that +asks you to configure your view of the ingested logs. Select the option for +timeseries values and select `@timestamp`. On the following page select the +`Discover` tab and then you should be able to see the ingested logs. +You can set the refresh interval to 5 seconds to have the logs +regulary refreshed. -```conf -apiVersion: v1 -clusters: -- cluster: - certificate-authority-data: REDACTED - server: https://146.148.94.154 - name: kubernetes-satnam_kubernetes -contexts: -- context: - cluster: kubernetes-satnam_kubernetes - user: kubernetes-satnam_kubernetes - name: kubernetes-satnam_kubernetes -current-context: kubernetes-satnam_kubernetes -kind: Config -preferences: {} -users: -- name: kubernetes-satnam_kubernetes - user: - client-certificate-data: REDACTED - client-key-data: REDACTED - token: JsUe2Z3cXqa17UQqQ8qWGGf4nOSLwSnp -``` - -Now you can issue requests to Elasticsearch: - -```shell -$ curl --header "Authorization: Bearer JsUe2Z3cXqa17UQqQ8qWGGf4nOSLwSnp" --insecure https://146.148.94.154/api/v1/proxy/namespaces/kube-system/services/elasticsearch-logging/ -``` - -```json -{ - "status" : 200, - "name" : "Vance Astrovik", - "cluster_name" : "kubernetes-logging", - "version" : { - "number" : "1.5.2", - "build_hash" : "62ff9868b4c8a0c45860bebb259e21980778ab1c", - "build_timestamp" : "2015-04-27T09:21:06Z", - "build_snapshot" : false, - "lucene_version" : "4.10.4" - }, - "tagline" : "You Know, for Search" -} -``` - -Note that you need the trailing slash at the end of the service proxy URL. Here is an example of a search: - -```shell -$ curl --header "Authorization: Bearer JsUe2Z3cXqa17UQqQ8qWGGf4nOSLwSnp" --insecure https://146.148.94.154/api/v1/proxy/namespaces/kube-system/services/elasticsearch-logging/_search?pretty=true -``` - -```json -{ - "took" : 7, - "timed_out" : false, - "_shards" : { - "total" : 6, - "successful" : 6, - "failed" : 0 - }, - "hits" : { - "total" : 123711, - "max_score" : 1.0, - "hits" : [ { - "_index" : ".kibana", - "_type" : "config", - "_id" : "4.0.2", - "_score" : 1.0, - "_source":{"buildNum":6004,"defaultIndex":"logstash-*"} - }, { -... - "_index" : "logstash-2015.06.22", - "_type" : "fluentd", - "_id" : "AU4c_GvFZL5p_gZ8dxtx", - "_score" : 1.0, - "_source":{"log":"synthetic-logger-10lps-pod: 31: 2015-06-22 20:35:33.597918073+00:00\n","stream":"stdout","tag":"kubernetes.synthetic-logger-10lps-pod_default_synth-lgr","@timestamp":"2015-06-22T20:35:33+00:00"} - }, { - "_index" : "logstash-2015.06.22", - "_type" : "fluentd", - "_id" : "AU4c_GvFZL5p_gZ8dxt2", - "_score" : 1.0, - "_source":{"log":"synthetic-logger-10lps-pod: 36: 2015-06-22 20:35:34.108780133+00:00\n","stream":"stdout","tag":"kubernetes.synthetic-logger-10lps-pod_default_synth-lgr","@timestamp":"2015-06-22T20:35:34+00:00"} - } ] - } -} -``` - -The Elasticsearch website contains information about [URI search queries](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-uri-request.html) which can be used to extract the required logs. - -Alternatively you can view the ingested logs using Kibana. The first time you visit the Kibana URL you will be -presented with a page that asks you to configure your view of the ingested logs. Select the option for -timeseries values and select `@timestamp`. On the following page select the `Discover` tab and then you -should be able to see the ingested logs. You can set the refresh interval to 5 seconds to have the logs -regulary refreshed. Here is a typical view of ingested logs from the Kibana viewer. +Here is a typical view of ingested logs from the Kibana viewer: ![Kibana logs](/images/docs/kibana-logs.png) -Another way to access Elasticsearch and Kibana in the cluster is to use `kubectl proxy` which will serve -a local proxy to the remote master: - -```shell -$ kubectl proxy -Starting to serve on localhost:8001 -``` - -Now you can visit the URL [http://localhost:8001/api/v1/proxy/namespaces/kube-system/services/elasticsearch-logging](http://localhost:8001/api/v1/proxy/namespaces/kube-system/services/elasticsearch-logging) to contact Elasticsearch and [http://localhost:8001/api/v1/proxy/namespaces/kube-system/services/kibana-logging](http://localhost:8001/api/v1/proxy/namespaces/kube-system/services/kibana-logging) to access the Kibana viewer. - - - +Kibana opens up all sorts of powerful options for exploring your logs! For some +ideas on how to dig into it, check out [Kibana's documentation](https://www.elastic.co/guide/en/kibana/current/discover.html). From d7e0faefcf4fc1ddd2b427881fb85c55d36b1290 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=81ukasz=20Ole=C5=9B?= Date: Wed, 20 Jul 2016 15:09:57 +0200 Subject: [PATCH 219/348] Update DaemonSet docs about node affinity After merging kubernetes/kubernetes#28803 DaemonSets can also be scheduled using node affinity. --- docs/admin/daemons.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/admin/daemons.md b/docs/admin/daemons.md index f2b5463e8d..2e50d8ef9b 100644 --- a/docs/admin/daemons.md +++ b/docs/admin/daemons.md @@ -71,9 +71,11 @@ a node for testing. If you specify a `.spec.template.spec.nodeSelector`, then the DaemonSet controller will create pods on nodes which match that [node -selector](https://github.com/kubernetes/kubernetes.github.io/tree/{{page.docsbranch}}/docs/user-guide/node-selection). +selector](https://github.com/kubernetes/kubernetes.github.io/tree/{{page.docsbranch}}/docs/user-guide/node-selection). +If you specify a `scheduler.alpha.kubernetes.io/affinity` annotation in `.spec.template.metadata.annotations`, +then DaemonSet controller will create pods on nodes which match that [node affinity](../../user-guide/node-selection/#alpha-feature-in-kubernetes-v12-node-affinity). -If you do not specify a `.spec.template.spec.nodeSelector`, then the DaemonSet controller will +If you do not specify a `.spec.template.spec.nodeSelector` nor `node affinity`, then the DaemonSet controller will create pods on all nodes. ## How Daemon Pods are Scheduled From d58ba7b075bb4848349a2c920caaa08ff3773d70 Mon Sep 17 00:00:00 2001 From: Dmitry Shulyak Date: Wed, 20 Jul 2016 17:05:19 +0300 Subject: [PATCH 220/348] Add example mounting ConfigMap as a single file into /etc dir ref: https://github.com/kubernetes/kubernetes/pull/22575 --- docs/user-guide/configmap/README.md | 3 +++ docs/user-guide/configmap/mount-file-pod.yaml | 21 +++++++++++++++++++ 2 files changed, 24 insertions(+) create mode 100644 docs/user-guide/configmap/mount-file-pod.yaml diff --git a/docs/user-guide/configmap/README.md b/docs/user-guide/configmap/README.md index e6bf6f52a5..2df57176dd 100644 --- a/docs/user-guide/configmap/README.md +++ b/docs/user-guide/configmap/README.md @@ -101,3 +101,6 @@ This pod runs a `cat` command to print the value of one of the keys in the volum $ kubectl logs config-volume-test-pod value-1 ``` + +Alternatively you can use [`mount-file-pod.yaml`](mount-file-pod.yaml) file to mount +only a file from ConfigMap, preserving original content of /etc directory. diff --git a/docs/user-guide/configmap/mount-file-pod.yaml b/docs/user-guide/configmap/mount-file-pod.yaml new file mode 100644 index 0000000000..7efd9b4003 --- /dev/null +++ b/docs/user-guide/configmap/mount-file-pod.yaml @@ -0,0 +1,21 @@ +apiVersion: v1 +kind: Pod +metadata: + name: config-single-file-volume-pod +spec: + containers: + - name: test-container + image: gcr.io/google_containers/busybox + command: [ "/bin/sh", "-c", "cat /etc/special-key" ] + volumeMounts: + - name: config-volume + mountPath: /etc/special-key + subPath: path/to/special-key + volumes: + - name: config-volume + configMap: + name: test-configmap + items: + - key: data-1 + path: path/to/special-key + restartPolicy: Never From 00ca10945d7207984b6805c56262c2ee71c61f09 Mon Sep 17 00:00:00 2001 From: Anirudh Date: Mon, 18 Jul 2016 12:28:00 -0700 Subject: [PATCH 221/348] Reviewed examples and petset docs, minor errors fixed. --- docs/user-guide/petset.md | 2 +- docs/user-guide/petset/bootstrapping/index.md | 15 ++++++++++++--- 2 files changed, 13 insertions(+), 4 deletions(-) diff --git a/docs/user-guide/petset.md b/docs/user-guide/petset.md index 77b132dea2..f3d5d4c4fa 100644 --- a/docs/user-guide/petset.md +++ b/docs/user-guide/petset.md @@ -161,7 +161,7 @@ web-1 And the hostname is linked to the in-cluster DNS address: ```shell -$ kubectl run -it --image busybox dns-test --restart=Never /bin/sh +$ kubectl run -i --tty --image busybox dns-test --restart=Never /bin/sh dns-test # nslookup web-0.nginx Server: 10.0.0.10 Address 1: 10.0.0.10 kube-dns.kube-system.svc.cluster.local diff --git a/docs/user-guide/petset/bootstrapping/index.md b/docs/user-guide/petset/bootstrapping/index.md index 2462c43691..8a4924e303 100644 --- a/docs/user-guide/petset/bootstrapping/index.md +++ b/docs/user-guide/petset/bootstrapping/index.md @@ -32,7 +32,7 @@ Since Pet Set already gives each Pet a consistent identity, all we need is a way Download [this](petset_vm.yaml) petset into a file called petset_vm.yaml, and create it: ```shell -$ kubectl create -f ./petset_vm.yaml +$ kubectl create -f petset_vm.yaml service "ub" created petset "vm" created ``` @@ -49,7 +49,7 @@ vm-1 1/1 Running 0 2m We can exec into one and install nginx ```shell -$ kubectl exec vm-0 /bin/sh +$ kubectl exec -it vm-0 /bin/sh vm-0 # apt-get update ... vm-0 # apt-get install nginx -y @@ -79,6 +79,9 @@ And access it from anywhere in the cluster (and because this is an example that ```shell $ kubectl exec -it vm-1 /bin/sh +vm-1 # apt-get update +... +vm-1 # apt-get install netcat -y vm-1 # printf "GET / HTTP/1.0\r\n\r\n" | netcat vm-0.ub 80 ``` @@ -139,7 +142,10 @@ The role of the peer finder: You can invoke the peer finder inside the Pets we created in the last example: ```shell -$ kubectl exec -it vm-0 /bin/bash +$ kubectl exec -it vm-0 /bin/sh +vm-0 # apt-get update +... +vm-0 # apt-get install curl -y vm-0 # curl -sSL -o /peer-finder https://storage.googleapis.com/kubernetes-release/pets/peer-finder vm-0 # chmod -c 755 peer-finder @@ -196,6 +202,9 @@ web-0 If you scale the cluster, the new pods parent themselves to the same master. To test this you can `kubectl edit` the petset and change the `replicas` field to 5: ```shell +$ kubectl edit petset web +... + $ kubectl get po -l app=nginx NAME READY STATUS RESTARTS AGE web-0 1/1 Running 0 2h From 4c4dd13f3631cc98af5fe3f67a9a8701de8c96d2 Mon Sep 17 00:00:00 2001 From: Scott Stensland Date: Wed, 20 Jul 2016 14:30:50 -0400 Subject: [PATCH 222/348] I fixed typo on command line If you actually issue these commands you will see the error I am fixing old : kubectl get pods/original -o yaml > /tmp/current.yaml Error from server: pods "original" not found new : kubectl get pods/foo -o yaml > /tmp/current.yaml previously the pod name was wrong --- docs/user-guide/working-with-resources.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/working-with-resources.md b/docs/user-guide/working-with-resources.md index 7dc3025daf..243aa72fc5 100644 --- a/docs/user-guide/working-with-resources.md +++ b/docs/user-guide/working-with-resources.md @@ -28,7 +28,7 @@ spec: EOF $ kubectl create -f /tmp/original.yaml pods/original -$ kubectl get pods/original -o yaml > /tmp/current.yaml +$ kubectl get pods/foo -o yaml > /tmp/current.yaml pods/original $ wc -l /tmp/original.yaml /tmp/current.yaml 51 /tmp/current.yaml From 55d549283f243dd3b4f0249412b0abdb7f2c375f Mon Sep 17 00:00:00 2001 From: Jared Date: Wed, 20 Jul 2016 15:32:50 -0700 Subject: [PATCH 223/348] Remove commas at the end of "containerPort" "containerPort": 8000, should be "containerPort": 8000 --- docs/user-guide/pods/pod-sample.json | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/user-guide/pods/pod-sample.json b/docs/user-guide/pods/pod-sample.json index 25d0373204..7cab452b64 100644 --- a/docs/user-guide/pods/pod-sample.json +++ b/docs/user-guide/pods/pod-sample.json @@ -14,7 +14,7 @@ "image": "redis", "ports": [ { - "containerPort": 6379, + "containerPort": 6379 } ] }, @@ -23,7 +23,7 @@ "image": "django", "ports": [ { - "containerPort": 8000, + "containerPort": 8000 } ] } From febeb5825b711527331fa901fe24efa4473bc5a0 Mon Sep 17 00:00:00 2001 From: George G Date: Fri, 15 Jul 2016 13:34:58 +0100 Subject: [PATCH 224/348] scratch.md: kubernetes/cluster/images/etcd/Makefile no longer uses ETCD_VERSION --- docs/getting-started-guides/scratch.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting-started-guides/scratch.md b/docs/getting-started-guides/scratch.md index a40b4ec546..658a1de190 100644 --- a/docs/getting-started-guides/scratch.md +++ b/docs/getting-started-guides/scratch.md @@ -179,7 +179,7 @@ For etcd, you can: We recommend that you use the etcd version which is provided in the Kubernetes binary distribution. The Kubernetes binaries in the release were tested extensively with this version of etcd and not with any other version. -The recommended version number can also be found as the value of `ETCD_VERSION` in `kubernetes/cluster/images/etcd/Makefile`. +The recommended version number can also be found as the value of `TAG` in `kubernetes/cluster/images/etcd/Makefile`. The remainder of the document assumes that the image identifiers have been chosen and stored in corresponding env vars. Examples (replace with latest tags and appropriate registry): From c2f5bf2cc650f933fadf715562d419f214e163ea Mon Sep 17 00:00:00 2001 From: George G Date: Fri, 15 Jul 2016 13:38:14 +0100 Subject: [PATCH 225/348] scratch.md: Don't duplicate CA instructions --- docs/getting-started-guides/scratch.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/docs/getting-started-guides/scratch.md b/docs/getting-started-guides/scratch.md index 658a1de190..34d75a79d8 100644 --- a/docs/getting-started-guides/scratch.md +++ b/docs/getting-started-guides/scratch.md @@ -208,11 +208,10 @@ You need to prepare several certs: - The kubelets optionally need certs to identify themselves as clients of the master, and when serving its own API over HTTPS. -Unless you plan to have a real CA generate your certs, you will need to generate a root cert and use that to sign the master, kubelet, and kubectl certs. - -- see function `create-certs` in `cluster/common.sh` -- see also `cluster/saltbase/salt/generate-cert/make-ca-cert.sh` and - `cluster/saltbase/salt/generate-cert/make-cert.sh` +Unless you plan to have a real CA generate your certs, you will need +to generate a root cert and use that to sign the master, kubelet, and +kubectl certs. How to do this is described in the [authentication +documentation](/docs/admin/authentication/#creating-certificates). You will end up with the following files (we will use these variables later on) From b678c151f5f573096f53f10994ae9c69b9db34c0 Mon Sep 17 00:00:00 2001 From: George G Date: Fri, 15 Jul 2016 14:35:58 +0100 Subject: [PATCH 226/348] scratch.md: Remove extra = --- docs/getting-started-guides/scratch.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting-started-guides/scratch.md b/docs/getting-started-guides/scratch.md index 34d75a79d8..a571e21845 100644 --- a/docs/getting-started-guides/scratch.md +++ b/docs/getting-started-guides/scratch.md @@ -183,7 +183,7 @@ The recommended version number can also be found as the value of `TAG` in `kuber The remainder of the document assumes that the image identifiers have been chosen and stored in corresponding env vars. Examples (replace with latest tags and appropriate registry): - - `HYPERKUBE_IMAGE==gcr.io/google_containers/hyperkube:$TAG` + - `HYPERKUBE_IMAGE=gcr.io/google_containers/hyperkube:$TAG` - `ETCD_IMAGE=gcr.io/google_containers/etcd:$ETCD_VERSION` ### Security Models From 489892d65d24face65093fc8e5a459b4c2c28ac0 Mon Sep 17 00:00:00 2001 From: George G Date: Fri, 15 Jul 2016 14:37:44 +0100 Subject: [PATCH 227/348] scratch.md: Clarify CA_CERT_BASE64_ENCODED / certificate-authority --- docs/getting-started-guides/scratch.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/getting-started-guides/scratch.md b/docs/getting-started-guides/scratch.md index a571e21845..d9e1a561d1 100644 --- a/docs/getting-started-guides/scratch.md +++ b/docs/getting-started-guides/scratch.md @@ -288,6 +288,10 @@ contexts: current-context: service-account-context ``` +Where CA_CERT_BASE64_ENCODED can be generated like `< ca.crt base64 +-w0`. Alternatively you can refer to a file (e.g. `certificate-authority: +/etc/kubernetes/ca.crt`). + Put the kubeconfig(s) on every node. The examples later in this guide assume that there are kubeconfigs in `/var/lib/kube-proxy/kubeconfig` and `/var/lib/kubelet/kubeconfig`. From 3e55a5ba5af6ef1c7d2f7614ed13fb5254a8302e Mon Sep 17 00:00:00 2001 From: George G Date: Fri, 15 Jul 2016 15:36:54 +0100 Subject: [PATCH 228/348] scratch.md: Fix formatting --- docs/getting-started-guides/scratch.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/getting-started-guides/scratch.md b/docs/getting-started-guides/scratch.md index d9e1a561d1..84f789a679 100644 --- a/docs/getting-started-guides/scratch.md +++ b/docs/getting-started-guides/scratch.md @@ -106,6 +106,7 @@ be active at once. Note that you can grow the end of this range, but you cannot move it without disrupting the services and pods that already use it. Also, you need to pick a static IP for master node. + - Call this `MASTER_IP`. - Open any firewalls to allow access to the apiserver ports 80 and/or 443. - Enable ipv4 forwarding sysctl, `net.ipv4.ip_forward = 1` From 5b91603d693cb9fa3a160d758c1e8048d3c310a5 Mon Sep 17 00:00:00 2001 From: George G Date: Fri, 15 Jul 2016 15:47:01 +0100 Subject: [PATCH 229/348] scratch.md: kubeconfig should not embed the CA cert --- docs/getting-started-guides/scratch.md | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/docs/getting-started-guides/scratch.md b/docs/getting-started-guides/scratch.md index 84f789a679..f1b222ae7e 100644 --- a/docs/getting-started-guides/scratch.md +++ b/docs/getting-started-guides/scratch.md @@ -280,7 +280,7 @@ users: clusters: - name: local cluster: - certificate-authority-data: ${CA_CERT_BASE64_ENCODED} + certificate-authority: /srv/kubernetes/ca.crt contexts: - context: cluster: local @@ -289,10 +289,6 @@ contexts: current-context: service-account-context ``` -Where CA_CERT_BASE64_ENCODED can be generated like `< ca.crt base64 --w0`. Alternatively you can refer to a file (e.g. `certificate-authority: -/etc/kubernetes/ca.crt`). - Put the kubeconfig(s) on every node. The examples later in this guide assume that there are kubeconfigs in `/var/lib/kube-proxy/kubeconfig` and `/var/lib/kubelet/kubeconfig`. From 0859bcfad15516df5d4ca6e79bd17badd7e9e20c Mon Sep 17 00:00:00 2001 From: George G Date: Fri, 15 Jul 2016 18:07:02 +0100 Subject: [PATCH 230/348] scratch.md: Add missing newline --- docs/getting-started-guides/scratch.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/getting-started-guides/scratch.md b/docs/getting-started-guides/scratch.md index f1b222ae7e..bf31a5422d 100644 --- a/docs/getting-started-guides/scratch.md +++ b/docs/getting-started-guides/scratch.md @@ -600,6 +600,7 @@ If you are following the firewall-only security approach, then use these argumen - `--advertise-address=$MASTER_IP` If you are using the HTTPS approach, then set: + - `--client-ca-file=/srv/kubernetes/ca.crt` - `--token-auth-file=/srv/kubernetes/known_tokens.csv` - `--basic-auth-file=/srv/kubernetes/basic_auth.csv` From 3195bc7db94a6faeef62a7f2d318c38b112742fc Mon Sep 17 00:00:00 2001 From: George G Date: Wed, 20 Jul 2016 12:01:25 +0100 Subject: [PATCH 231/348] scratch.md: Whitespace --- docs/getting-started-guides/scratch.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting-started-guides/scratch.md b/docs/getting-started-guides/scratch.md index bf31a5422d..d9817d4098 100644 --- a/docs/getting-started-guides/scratch.md +++ b/docs/getting-started-guides/scratch.md @@ -662,7 +662,7 @@ Complete this template for the scheduler pod: ], "livenessProbe": { "httpGet": { - "host" : "127.0.0.1", + "host": "127.0.0.1", "path": "/healthz", "port": 10251 }, From 097e93b28740169401eac7c9523f99455cb9fb5e Mon Sep 17 00:00:00 2001 From: Matthias Schmitz Date: Tue, 7 Jun 2016 11:01:53 +0200 Subject: [PATCH 232/348] Fix commands and output of "Running an example job" * The examples in the jobs user-guide are not longer working because the ``labels app=pi`` got removed in commit f16668b45f5a8ca4d734f0356c72d63451e8a6bb * The exammples now use the automatic added label ``job-name`` as selector for kubectl. --- docs/user-guide/jobs.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/user-guide/jobs.md b/docs/user-guide/jobs.md index ab193f6f50..e78f121121 100644 --- a/docs/user-guide/jobs.md +++ b/docs/user-guide/jobs.md @@ -38,11 +38,11 @@ $ kubectl describe jobs/pi Name: pi Namespace: default Image(s): perl -Selector: app in (pi) +Selector: controller-uid=b1db589a-2c8d-11e6-b324-0209dc45a495 Parallelism: 1 Completions: 1 -Start Time: Mon, 11 Jan 2016 15:35:52 -0800 -Labels: app=pi +Start Time: Tue, 07 Jun 2016 10:56:16 +0200 +Labels: controller-uid=b1db589a-2c8d-11e6-b324-0209dc45a495,job-name=pi Pods Statuses: 0 Running / 1 Succeeded / 0 Failed No volumes. Events: @@ -56,7 +56,7 @@ To view completed pods of a job, use `kubectl get pods --show-all`. The `--show To list all the pods that belong to job in a machine readable form, you can use a command like this: ```shell -$ pods=$(kubectl get pods --selector=app=pi --output=jsonpath={.items..metadata.name}) +$ pods=$(kubectl get pods --selector=job-name=pi --output=jsonpath={.items..metadata.name}) echo $pods pi-aiw0a ``` @@ -190,7 +190,7 @@ The job object also remains after it is completed so that you can view its statu old jobs after noting their status. Delete the job with `kubectl` (e.g. `kubectl delete jobs/pi` or `kubectl delete -f ./job.yaml`). When you delete the job using `kubectl`, all the pods it created are deleted too. If a Job's pods are failing repeatedly, the Job will keep creating new pods forever, by default. -Retrying forever can be a useful pattern. If an external dependency of the Job's +Retrying forever can be a useful pattern. If an external dependency of the Job's pods is missing (for example an input file on a networked storage volume is not present), then the Job will keep trying Pods, and when you later resolve the external dependency (for example, creating the missing file) the Job will then complete without any further action. From 9266e7145cb89fc441b61324ff9661bc258c6dfa Mon Sep 17 00:00:00 2001 From: Gurvinder Singh Date: Thu, 21 Jul 2016 14:59:00 +0200 Subject: [PATCH 233/348] Update networkpolicies.md FIxed typo in api group path --- docs/user-guide/networkpolicies.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/networkpolicies.md b/docs/user-guide/networkpolicies.md index 52ecdf6c99..d2a0653dd2 100644 --- a/docs/user-guide/networkpolicies.md +++ b/docs/user-guide/networkpolicies.md @@ -10,7 +10,7 @@ A network policy is a specification of how selections of pods are allowed to com ## Prerequisites -You must enable the `extensions/v1beta/networkpolicies` runtime config in your apiserver to enable this resource. +You must enable the `extensions/v1beta1/networkpolicies` runtime config in your apiserver to enable this resource. You must also be using a networking solution which supports `NetworkPolicy` - simply creating the resource without a controller to implement it will have no effect. From c6ad7e587f849273518be260a8aab957225e8055 Mon Sep 17 00:00:00 2001 From: George G Date: Thu, 21 Jul 2016 18:42:46 +0100 Subject: [PATCH 234/348] scratch.md: More consistent/explicit livenessProbes --- docs/getting-started-guides/scratch.md | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/docs/getting-started-guides/scratch.md b/docs/getting-started-guides/scratch.md index d9817d4098..8ebc6c81d9 100644 --- a/docs/getting-started-guides/scratch.md +++ b/docs/getting-started-guides/scratch.md @@ -553,8 +553,10 @@ For each of these components, the steps to start them running are similar: ], "livenessProbe": { "httpGet": { - "path": "/healthz", - "port": 8080 + "scheme": "HTTP", + "host": "127.0.0.1", + "port": 8080, + "path": "/healthz" }, "initialDelaySeconds": 15, "timeoutSeconds": 15 @@ -662,9 +664,10 @@ Complete this template for the scheduler pod: ], "livenessProbe": { "httpGet": { + "scheme": "HTTP", "host": "127.0.0.1", - "path": "/healthz", - "port": 10251 + "port": 10251, + "path": "/healthz" }, "initialDelaySeconds": 15, "timeoutSeconds": 15 @@ -717,9 +720,10 @@ Template for controller manager pod: ], "livenessProbe": { "httpGet": { + "scheme": "HTTP", "host": "127.0.0.1", - "path": "/healthz", - "port": 10252 + "port": 10252, + "path": "/healthz" }, "initialDelaySeconds": 15, "timeoutSeconds": 15 From e5f590a539f7bd984e2ea850762f4b07d54c258f Mon Sep 17 00:00:00 2001 From: Vladimir Rutsky Date: Sat, 23 Jul 2016 00:25:26 +0400 Subject: [PATCH 235/348] fix typo: "etc" -> "etcd" --- docs/getting-started-guides/scratch.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting-started-guides/scratch.md b/docs/getting-started-guides/scratch.md index a40b4ec546..e4862b3218 100644 --- a/docs/getting-started-guides/scratch.md +++ b/docs/getting-started-guides/scratch.md @@ -480,7 +480,7 @@ You will need to run one or more instances of etcd. by durable storage (RAID, GCE PD) - Alternative: run 3 or 5 etcd instances. - Log can be written to non-durable storage because storage is replicated. - - run a single apiserver which connects to one of the etc nodes. + - run a single apiserver which connects to one of the etcd nodes. See [cluster-troubleshooting](/docs/admin/cluster-troubleshooting) for more discussion on factors affecting cluster availability. From 696c4ed0abd8dadd47b8277581ee122b4137c9c9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Lucas=20K=C3=A4ldstr=C3=B6m?= Date: Sat, 23 Jul 2016 00:52:00 +0300 Subject: [PATCH 236/348] Remove the old docker-multinode in favor for the new one --- _data/guides.yml | 10 +- .../docker-multinode.md | 118 +++++++++ .../docker-multinode/deployDNS.md | 36 --- .../docker-multinode/index.md | 99 ------- .../docker-multinode/kube-system.yaml | 4 - .../docker-multinode/master.md | 244 ------------------ .../docker-multinode/master.sh | 243 ----------------- .../docker-multinode/skydns.yaml.in | 136 ---------- .../docker-multinode/testing.md | 68 ----- .../docker-multinode/worker.md | 179 ------------- .../docker-multinode/worker.sh | 231 ----------------- 11 files changed, 120 insertions(+), 1248 deletions(-) create mode 100644 docs/getting-started-guides/docker-multinode.md delete mode 100644 docs/getting-started-guides/docker-multinode/deployDNS.md delete mode 100644 docs/getting-started-guides/docker-multinode/index.md delete mode 100644 docs/getting-started-guides/docker-multinode/kube-system.yaml delete mode 100644 docs/getting-started-guides/docker-multinode/master.md delete mode 100755 docs/getting-started-guides/docker-multinode/master.sh delete mode 100644 docs/getting-started-guides/docker-multinode/skydns.yaml.in delete mode 100644 docs/getting-started-guides/docker-multinode/testing.md delete mode 100644 docs/getting-started-guides/docker-multinode/worker.md delete mode 100755 docs/getting-started-guides/docker-multinode/worker.sh diff --git a/_data/guides.yml b/_data/guides.yml index e2c32f7f2f..9db5e93a2f 100644 --- a/_data/guides.yml +++ b/_data/guides.yml @@ -163,14 +163,6 @@ toc: path: /docs/getting-started-guides/azure/ - title: Running Kubernetes on CenturyLink Cloud path: /docs/getting-started-guides/clc/ - - title: Portable Multi-Node Clusters - section: - - title: Installing a Kubernetes Master Node via Docker - path: /docs/getting-started-guides/docker-multinode/master/ - - title: Adding a Kubernetes Worker Node via Docker - path: /docs/getting-started-guides/docker-multinode/worker/ - - title: Deploying DNS - path: /docs/getting-started-guides/docker-multinode/deployDNS/ - title: Running Kubernetes on Custom Solutions section: - title: Creating a Custom Cluster from Scratch @@ -235,6 +227,8 @@ toc: path: /docs/getting-started-guides/coreos/bare_metal_calico/ - title: Ubuntu Nodes with Calico path: /docs/getting-started-guides/ubuntu-calico/ + - title: Portable Multi-Node Cluster + path: /docs/getting-started-guides/docker-multinode/ - title: Building Large Clusters path: /docs/admin/cluster-large/ - title: Running in Multiple Zones diff --git a/docs/getting-started-guides/docker-multinode.md b/docs/getting-started-guides/docker-multinode.md new file mode 100644 index 0000000000..0f366c5f22 --- /dev/null +++ b/docs/getting-started-guides/docker-multinode.md @@ -0,0 +1,118 @@ +--- +--- + +* TOC +{:toc} + + +## Prerequisites + +The only thing you need is a linux machine with **Docker 1.10.0 or higher** + +## Overview + +This guide will set up a 2-node Kubernetes cluster, consisting of a _master_ node which hosts the API server and orchestrates work +and a _worker_ node which receives work from the master. You can repeat the process of adding worker nodes an arbitrary number of +times to create larger clusters. + +Here's a diagram of what the final result will look like: +![Kubernetes Single Node on Docker](k8s-docker.png) + +### Bootstrap Docker + +This guide uses a pattern of running two instances of the Docker daemon: + 1) A _bootstrap_ Docker instance which is used to start `etcd` and `flanneld`, on which the Kubernetes components depend + 2) A _main_ Docker instance which is used for the Kubernetes infrastructure and user's scheduled containers + +This pattern is necessary because the `flannel` daemon is responsible for setting up and managing the network that interconnects +all of the Docker containers created by Kubernetes. To achieve this, it must run outside of the _main_ Docker daemon. However, +it is still useful to use containers for deployment and management, so we create a simpler _bootstrap_ daemon to achieve this. + +### Versions supported + +v1.2.x and v1.3.x are supported versions for this deployment. +v1.3.0 alphas and betas might work, but be sure you know what you're doing if you're trying them out. + +### Multi-arch solution + +Yeah, it's true. You may run this deployment setup seamlessly on `amd64`, `arm`, `arm64` and `ppc64le` hosts. +See this tracking issue for more details: https://github.com/kubernetes/kubernetes/issues/17981 + +v1.3.0 ships with support for amd64, arm and arm64. ppc64le isn't supported, due to a bug in the Go runtime, `hyperkube` (only!) isn't built for the stable v1.3.0 release, and therefore this guide can't run it. But you may still run Kubernetes on ppc64le via custom deployments. + +hyperkube was pushed for ppc64le at versions `v1.3.0-alpha.3` and `v1.3.0-alpha.4`, feel free to try them out, but there might be some unexpected bugs. + +### Options/configuration + +The scripts will output something like this when starting: + +```shell ++++ [0611 12:50:12] K8S_VERSION is set to: v1.3.0 ++++ [0611 12:50:12] ETCD_VERSION is set to: 2.2.5 ++++ [0611 12:50:12] FLANNEL_VERSION is set to: 0.5.5 ++++ [0611 12:50:12] FLANNEL_IPMASQ is set to: true ++++ [0611 12:50:12] FLANNEL_NETWORK is set to: 10.1.0.0/16 ++++ [0611 12:50:12] FLANNEL_BACKEND is set to: udp ++++ [0611 12:50:12] RESTART_POLICY is set to: unless-stopped ++++ [0611 12:50:12] MASTER_IP is set to: 192.168.1.50 ++++ [0611 12:50:12] ARCH is set to: amd64 ++++ [0611 12:50:12] NET_INTERFACE is set to: eth0 +``` + +Each of these options are overridable by `export`ing the values before running the script. + +## Setup the master node + +The first step in the process is to initialize the master node. + +Clone the `kube-deploy` repo, and run [master.sh](master.sh) on the master machine _with root_: + +```shell +$ git clone https://github.com/kubernetes/kube-deploy +$ cd docker-multinode +$ ./master.sh +``` + +First, the `bootstrap` docker daemon is started, then `etcd` and `flannel` are started as containers in the bootstrap daemon. +Then, the main docker daemon is restarted, and this is an OS/distro-specific tasks, so if it doesn't work for your distro, feel free to contribute! + +Lastly, it launches `kubelet` in the main docker daemon, and the `kubelet` in turn launches the control plane (apiserver, controller-manager and scheduler) as static pods. + +## Adding a worker node + +Once your master is up and running you can add one or more workers on different machines. + +Clone the `kube-deploy` repo, and run [worker.sh](worker.sh) on the worker machine _with root_: + +```shell +$ git clone https://github.com/kubernetes/kube-deploy +$ cd docker-multinode +$ export MASTER_IP=${SOME_IP} +$ ./worker.sh +``` + +First, the `bootstrap` docker daemon is started, then `flannel` is started as a container in the bootstrap daemon, in order to set up the overlay network. +Then, the main docker daemon is restarted and lastly `kubelet` is launched as a container in the main docker daemon. + +## Addons + +kube-dns and the dashboard are deployed automatically with v1.3.0 + +### Deploy DNS manually for v1.2.x + +Just specify the architecture, and deploy via these commands: + +```shell +# Possible options: amd64, arm, arm64 and ppc64le +$ export ARCH=amd64 + +# If the kube-system namespace isn't already created, create it +$ kubectl get ns +$ kubectl create namespace kube-system + +$ sed -e "s/ARCH/${ARCH}/g;" skydns.yaml | kubectl create -f - +``` + +### Test if DNS works + +Follow [this link](https://releases.k8s.io/release-1.2/cluster/addons/dns#how-do-i-test-if-it-is-working) to check it out. diff --git a/docs/getting-started-guides/docker-multinode/deployDNS.md b/docs/getting-started-guides/docker-multinode/deployDNS.md deleted file mode 100644 index e970807e48..0000000000 --- a/docs/getting-started-guides/docker-multinode/deployDNS.md +++ /dev/null @@ -1,36 +0,0 @@ ---- ---- - -### Get the template file - -First of all, download the dns template - -[skydns template](/docs/getting-started-guides/docker-multinode/skydns.yaml.in) - -### Set environment variables - -Then you need to set `DNS_REPLICAS`, `DNS_DOMAIN` and `DNS_SERVER_IP` envs - -```shell -$ export DNS_REPLICAS=1 - -$ export DNS_DOMAIN=cluster.local # specify in startup parameter `--cluster-domain` for containerized kubelet - -$ export DNS_SERVER_IP=10.0.0.10 # specify in startup parameter `--cluster-dns` for containerized kubelet -``` - -### Replace the corresponding value in the template and create the pod - -```shell{% raw %} -$ sed -e "s/{{ pillar\['dns_replicas'\] }}/${DNS_REPLICAS}/g;s/{{ pillar\['dns_domain'\] }}/${DNS_DOMAIN}/g;s/{{ pillar\['dns_server'\] }}/${DNS_SERVER_IP}/g" skydns.yaml.in > ./skydns.yaml - -# If the kube-system namespace isn't already created, create it -$ kubectl get ns -$ kubectl create namespace kube-system - -$ kubectl create -f ./skydns.yaml{% endraw %} -``` - -### Test if DNS works - -Follow [this link](https://releases.k8s.io/{{page.githubbranch}}/cluster/addons/dns#how-do-i-test-if-it-is-working) to check it out. diff --git a/docs/getting-started-guides/docker-multinode/index.md b/docs/getting-started-guides/docker-multinode/index.md deleted file mode 100644 index ec7e033e1b..0000000000 --- a/docs/getting-started-guides/docker-multinode/index.md +++ /dev/null @@ -1,99 +0,0 @@ ---- ---- - -_Note_: -These instructions are somewhat significantly more advanced than the [single node](/docs/getting-started-guides/docker) instructions. If you are -interested in just starting to explore Kubernetes, we recommend that you start there. - -* TOC -{:toc} - -## Prerequisites - -The only thing you need is a machine with **Docker 1.7.1 or higher** - -## Overview - -This guide will set up a 2-node Kubernetes cluster, consisting of a _master_ node which hosts the API server and orchestrates work -and a _worker_ node which receives work from the master. You can repeat the process of adding worker nodes an arbitrary number of -times to create larger clusters. - -Here's a diagram of what the final result will look like: - -![Kubernetes Single Node on Docker](/images/docs/k8s-docker.png) - -### Bootstrap Docker - -This guide also uses a pattern of running two instances of the Docker daemon - 1) A _bootstrap_ Docker instance which is used to start system daemons like `flanneld` and `etcd` - 2) A _main_ Docker instance which is used for the Kubernetes infrastructure and user's scheduled containers - -This pattern is necessary because the `flannel` daemon is responsible for setting up and managing the network that interconnects -all of the Docker containers created by Kubernetes. To achieve this, it must run outside of the _main_ Docker daemon. However, -it is still useful to use containers for deployment and management, so we create a simpler _bootstrap_ daemon to achieve this. - -You can specify the version on every node before install: - -```shell -export K8S_VERSION= -export ETCD_VERSION= -export FLANNEL_VERSION= -export FLANNEL_IFACE= -export FLANNEL_IPMASQ= -``` - -Otherwise, we'll use latest `hyperkube` image as default k8s version. - -## Master Node - -The first step in the process is to initialize the master node. - -The MASTER_IP step here is optional, it defaults to the first value of `hostname -I`. -Clone the Kubernetes repo, and run [master.sh](/docs/getting-started-guides/docker-multinode/master.sh) on the master machine _with root_: - -```shell -$ export MASTER_IP= -$ cd kubernetes/docs/getting-started-guides/docker-multinode/ -$ ./master.sh -``` - -`Master done!` - -See [here](/docs/getting-started-guides/docker-multinode/master) for detailed instructions explanation. - -## Adding a worker node - -Once your master is up and running you can add one or more workers on different machines. - -Clone the Kubernetes repo, and run [worker.sh](/docs/getting-started-guides/docker-multinode/worker.sh) on the worker machine _with root_: - -```shell -$ export MASTER_IP= -$ cd kubernetes/docs/getting-started-guides/docker-multinode/ -$ ./worker.sh -``` - -`Worker done!` - -See [here](/docs/getting-started-guides/docker-multinode/worker) for a detailed explanation. - -## Deploy a DNS - -See [here](/docs/getting-started-guides/docker-multinode/deployDNS) for instructions. - -## Testing your cluster - -Once your cluster has been created you can [test it out](/docs/getting-started-guides/docker-multinode/testing) - -For more complete applications, please look in the [examples directory](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/) - -## Support Level - - -IaaS Provider | Config. Mgmt | OS | Networking | Docs | Conforms | Support Level --------------------- | ------------ | ------ | ---------- | --------------------------------------------- | ---------| ---------------------------- -Docker Multi Node | custom | N/A | flannel | [docs](/docs/getting-started-guides/docker-multinode) | | Project ([@brendandburns](https://github.com/brendandburns)) - - -For support level information on all solutions, see the [Table of solutions](/docs/getting-started-guides/#table-of-solutions) chart. - diff --git a/docs/getting-started-guides/docker-multinode/kube-system.yaml b/docs/getting-started-guides/docker-multinode/kube-system.yaml deleted file mode 100644 index 986f4b4822..0000000000 --- a/docs/getting-started-guides/docker-multinode/kube-system.yaml +++ /dev/null @@ -1,4 +0,0 @@ -apiVersion: v1 -kind: Namespace -metadata: - name: kube-system diff --git a/docs/getting-started-guides/docker-multinode/master.md b/docs/getting-started-guides/docker-multinode/master.md deleted file mode 100644 index 4b297458bf..0000000000 --- a/docs/getting-started-guides/docker-multinode/master.md +++ /dev/null @@ -1,244 +0,0 @@ ---- ---- - -We'll begin by setting up the master node. For the purposes of illustration, we'll assume that the IP of this machine -is `${MASTER_IP}`. We'll need to run several versioned Kubernetes components, so we'll assume that the version we want -to run is `${K8S_VERSION}`, which should hold a released version of Kubernetes >= "1.2.1" - -Environment variables used: - -```shell -export MASTER_IP= -export K8S_VERSION= -export ETCD_VERSION= -export FLANNEL_VERSION= -export FLANNEL_IFACE= -export FLANNEL_IPMASQ= -``` - -There are two main phases to installing the master: - - * [Setting up `flanneld` and `etcd`](#setting-up-flanneld-and-etcd) - * [Starting the Kubernetes master components](#starting-the-kubernetes-master) - - -## Setting up flanneld and etcd - -_Note_: -This guide expects **Docker 1.7.1 or higher**. - -### Setup Docker Bootstrap - -We're going to use `flannel` to set up networking between Docker daemons. Flannel itself (and etcd on which it relies) will run inside of -Docker containers themselves. To achieve this, we need a separate "bootstrap" instance of the Docker daemon. This daemon will be started with -`--iptables=false` so that it can only run containers with `--net=host`. That's sufficient to bootstrap our system. - -Run: - -```shell -sudo sh -c 'docker -d -H unix:///var/run/docker-bootstrap.sock -p /var/run/docker-bootstrap.pid --iptables=false --ip-masq=false --bridge=none --graph=/var/lib/docker-bootstrap 2> /var/log/docker-bootstrap.log 1> /dev/null &' -``` - -_If you have Docker 1.8.0 or higher run this instead_ - -```shell -sudo sh -c 'docker daemon -H unix:///var/run/docker-bootstrap.sock -p /var/run/docker-bootstrap.pid --iptables=false --ip-masq=false --bridge=none --graph=/var/lib/docker-bootstrap 2> /var/log/docker-bootstrap.log 1> /dev/null &' -``` - -_Important Note_: -If you are running this on a long running system, rather than experimenting, you should run the bootstrap Docker instance under something like SysV init, upstart or systemd so that it is restarted -across reboots and failures. - - -### Startup etcd for flannel and the API server to use - -Run: - -```shell -sudo docker -H unix:///var/run/docker-bootstrap.sock run -d \ - --net=host \ - gcr.io/google_containers/etcd-amd64:${ETCD_VERSION} \ - /usr/local/bin/etcd \ - --listen-client-urls=http://127.0.0.1:4001,http://${MASTER_IP}:4001 \ - --advertise-client-urls=http://${MASTER_IP}:4001 \ - --data-dir=/var/etcd/data -``` - -Next, you need to set a CIDR range for flannel. This CIDR should be chosen to be non-overlapping with any existing network you are using: - -```shell -sudo docker -H unix:///var/run/docker-bootstrap.sock run \ - --net=host \ - gcr.io/google_containers/etcd-amd64:${ETCD_VERSION} \ - etcdctl set /coreos.com/network/config '{ "Network": "10.1.0.0/16" }' -``` - - -### Set up Flannel on the master node - -Flannel is a network abstraction layer build by CoreOS, we will use it to provide simplified networking between our Pods of containers. - -Flannel re-configures the bridge that Docker uses for networking. As a result we need to stop Docker, reconfigure its networking, and then restart Docker. - -#### Bring down Docker - -To re-configure Docker to use flannel, we need to take docker down, run flannel and then restart Docker. - -Turning down Docker is system dependent, it may be: - -```shell -sudo /etc/init.d/docker stop -``` - -or - -```shell -sudo systemctl stop docker -``` - -or - -```shell -sudo service docker stop -``` - -or it may be something else. - -#### Run flannel - -Now run flanneld itself: - -```shell -sudo docker -H unix:///var/run/docker-bootstrap.sock run -d \ - --net=host \ - --privileged \ - -v /dev/net:/dev/net \ - quay.io/coreos/flannel:${FLANNEL_VERSION} \ - /opt/bin/flanneld \ - --ip-masq=${FLANNEL_IPMASQ} \ - --iface=${FLANNEL_IFACE} -``` - -The previous command should have printed a really long hash, the container id, copy this hash. - -Now get the subnet settings from flannel: - -```shell -sudo docker -H unix:///var/run/docker-bootstrap.sock exec cat /run/flannel/subnet.env -``` - -#### Edit the docker configuration - -You now need to edit the docker configuration to activate new flags. Again, this is system specific. - -This may be in `/etc/default/docker` or `/etc/systemd/service/docker.service` or it may be elsewhere. - -Regardless, you need to add the following to the docker command line: - -```shell ---bip=${FLANNEL_SUBNET} --mtu=${FLANNEL_MTU} -``` - -#### Remove the existing Docker bridge - -Docker creates a bridge named `docker0` by default. You need to remove this: - -```shell -sudo /sbin/ifconfig docker0 down -sudo brctl delbr docker0 -``` - -You may need to install the `bridge-utils` package for the `brctl` binary. - -#### Restart Docker - -Again this is system dependent, it may be: - -```shell -sudo /etc/init.d/docker start -``` - -it may be: - -```shell -systemctl start docker -``` - -## Starting the Kubernetes Master - -Ok, now that your networking is set up, you can startup Kubernetes, this is the same as the single-node case, we will use the "main" instance of the Docker daemon for the Kubernetes components. - -```shell -sudo docker run \ - --volume=/:/rootfs:ro \ - --volume=/sys:/sys:ro \ - --volume=/var/lib/docker/:/var/lib/docker:rw \ - --volume=/var/lib/kubelet/:/var/lib/kubelet:rw \ - --volume=/var/run:/var/run:rw \ - --net=host \ - --privileged=true \ - --pid=host \ - -d \ - gcr.io/google_containers/hyperkube-amd64:v${K8S_VERSION} \ - /hyperkube kubelet \ - --allow-privileged=true \ - --api-servers=http://localhost:8080 \ - --v=2 \ - --address=0.0.0.0 \ - --enable-server \ - --hostname-override=127.0.0.1 \ - --config=/etc/kubernetes/manifests-multi \ - --containerized \ - --cluster-dns=10.0.0.10 \ - --cluster-domain=cluster.local -``` - -> Note that `--cluster-dns` and `--cluster-domain` is used to deploy dns, feel free to discard them if dns is not needed. - -### Test it out - -At this point, you should have a functioning 1-node cluster. Let's test it out! - -Download the kubectl binary for `${K8S_VERSION}` ({{page.version}}) and make it available by editing your PATH environment variable. -([OS X/amd64](http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0-alpha.7/bin/darwin/amd64/kubectl)) -([OS X/386](http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0-alpha.7/bin/darwin/386/kubectl)) -([linux/amd64](http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0-alpha.7/bin/linux/amd64/kubectl)) -([linux/386](http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0-alpha.7/bin/linux/386/kubectl)) -([linux/arm](http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0-alpha.7/bin/linux/arm/kubectl)) - -For example, OS X: - -```shell -$ wget http://storage.googleapis.com/kubernetes-release/release/v${K8S_VERSION}/bin/darwin/amd64/kubectl -$ chmod 755 kubectl -$ PATH=$PATH:`pwd` -``` - -Linux: - -```shell -$ wget http://storage.googleapis.com/kubernetes-release/release/v${K8S_VERSION}/bin/linux/amd64/kubectl -$ chmod 755 kubectl -$ PATH=$PATH:`pwd` -``` - -Now you can list the nodes: - -```shell -kubectl get nodes -``` - -This should print something like: - -```shell -NAME LABELS STATUS -127.0.0.1 kubernetes.io/hostname=127.0.0.1 Ready -``` - -If the status of the node is `NotReady` or `Unknown` please check that all of the containers you created are successfully running. -If all else fails, ask questions on [Slack](/docs/troubleshooting/#slack). - - -### Next steps - -Move on to [adding one or more workers](/docs/getting-started-guides/docker-multinode/worker/) or [deploy a dns](/docs/getting-started-guides/docker-multinode/deployDNS/) diff --git a/docs/getting-started-guides/docker-multinode/master.sh b/docs/getting-started-guides/docker-multinode/master.sh deleted file mode 100755 index 63465a288a..0000000000 --- a/docs/getting-started-guides/docker-multinode/master.sh +++ /dev/null @@ -1,243 +0,0 @@ -#!/bin/bash - -# Copyright 2015 The Kubernetes Authors All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# A script to setup the k8s master in docker containers. -# Authors @wizard_cxy @resouer - -set -e - -# Make sure docker daemon is running -if ( ! ps -ef | grep "/usr/bin/docker" | grep -v 'grep' &> /dev/null ); then - echo "Docker is not running on this machine!" - exit 1 -fi - -# Make sure k8s version env is properly set -K8S_VERSION=${K8S_VERSION:-"1.2.2} -ETCD_VERSION=${ETCD_VERSION:-"2.3.1"} -FLANNEL_VERSION=${FLANNEL_VERSION:-"0.5.5"} -FLANNEL_IPMASQ=${FLANNEL_IPMASQ:-"true"} -FLANNEL_IFACE=${FLANNEL_IFACE:-"eth0"} -ARCH=${ARCH:-"amd64"} - -# Run as root -if [ "$(id -u)" != "0" ]; then - echo >&2 "Please run as root" - exit 1 -fi - -# Make sure master ip is properly set -if [ -z ${MASTER_IP} ]; then - MASTER_IP=$(hostname -I | awk '{print $1}') -fi - -echo "K8S_VERSION is set to: ${K8S_VERSION}" -echo "ETCD_VERSION is set to: ${ETCD_VERSION}" -echo "FLANNEL_VERSION is set to: ${FLANNEL_VERSION}" -echo "FLANNEL_IFACE is set to: ${FLANNEL_IFACE}" -echo "FLANNEL_IPMASQ is set to: ${FLANNEL_IPMASQ}" -echo "MASTER_IP is set to: ${MASTER_IP}" -echo "ARCH is set to: ${ARCH}" - -# Check if a command is valid -command_exists() { - command -v "$@" > /dev/null 2>&1 -} - -lsb_dist="" - -# Detect the OS distro, we support ubuntu, debian, mint, centos, fedora dist -detect_lsb() { - # TODO: remove this when ARM support is fully merged - case "$(uname -m)" in - *64) - ;; - *) - echo "Error: We currently only support 64-bit platforms." - exit 1 - ;; - esac - - if command_exists lsb_release; then - lsb_dist="$(lsb_release -si)" - fi - if [ -z ${lsb_dist} ] && [ -r /etc/lsb-release ]; then - lsb_dist="$(. /etc/lsb-release && echo "$DISTRIB_ID")" - fi - if [ -z ${lsb_dist} ] && [ -r /etc/debian_version ]; then - lsb_dist='debian' - fi - if [ -z ${lsb_dist} ] && [ -r /etc/fedora-release ]; then - lsb_dist='fedora' - fi - if [ -z ${lsb_dist} ] && [ -r /etc/os-release ]; then - lsb_dist="$(. /etc/os-release && echo "$ID")" - fi - - lsb_dist="$(echo ${lsb_dist} | tr '[:upper:]' '[:lower:]')" - - case "${lsb_dist}" in - amzn|centos|debian|ubuntu|fedora) - ;; - *) - echo "Error: We currently only support ubuntu|debian|amzn|centos|fedora." - exit 1 - ;; - esac -} - - -# Start the bootstrap daemon -# TODO: do not start docker-bootstrap if it's already running -bootstrap_daemon() { - # Detecting docker version so we could run proper docker_daemon command - [[ $(eval "docker --version") =~ ([0-9][.][0-9][.][0-9]*) ]] && version="${BASH_REMATCH[1]}" - local got=$(echo -e "${version}\n1.8.0" | sed '/^$/d' | sort -nr | head -1) - if [[ "${got}" = "${version}" ]]; then - docker_daemon="docker -d" - else - docker_daemon="docker daemon" - fi - ${docker_daemon} \ - -H unix:///var/run/docker-bootstrap.sock \ - -p /var/run/docker-bootstrap.pid \ - --iptables=false \ - --ip-masq=false \ - --bridge=none \ - --graph=/var/lib/docker-bootstrap \ - 2> /var/log/docker-bootstrap.log \ - 1> /dev/null & - - sleep 5 -} - -# Start k8s components in containers -DOCKER_CONF="" - -start_k8s(){ - # Start etcd - docker -H unix:///var/run/docker-bootstrap.sock run \ - --restart=on-failure \ - --net=host \ - -d \ - gcr.io/google_containers/etcd-${ARCH}:${ETCD_VERSION} \ - /usr/local/bin/etcd \ - --listen-client-urls=http://127.0.0.1:4001,http://${MASTER_IP}:4001 \ - --advertise-client-urls=http://${MASTER_IP}:4001 \ - --data-dir=/var/etcd/data - - sleep 5 - # Set flannel net config - docker -H unix:///var/run/docker-bootstrap.sock run \ - --net=host gcr.io/google_containers/etcd:${ETCD_VERSION} \ - etcdctl \ - set /coreos.com/network/config \ - '{ "Network": "10.1.0.0/16", "Backend": {"Type": "vxlan"}}' - - # iface may change to a private network interface, eth0 is for default - flannelCID=$(docker -H unix:///var/run/docker-bootstrap.sock run \ - --restart=on-failure \ - -d \ - --net=host \ - --privileged \ - -v /dev/net:/dev/net \ - quay.io/coreos/flannel:${FLANNEL_VERSION} \ - /opt/bin/flanneld \ - --ip-masq="${FLANNEL_IPMASQ}" \ - --iface="${FLANNEL_IFACE}") - - sleep 8 - - # Copy flannel env out and source it on the host - docker -H unix:///var/run/docker-bootstrap.sock \ - cp ${flannelCID}:/run/flannel/subnet.env . - source subnet.env - - # Configure docker net settings, then restart it - case "${lsb_dist}" in - amzn) - DOCKER_CONF="/etc/sysconfig/docker" - echo "OPTIONS=\"\$OPTIONS --mtu=${FLANNEL_MTU} --bip=${FLANNEL_SUBNET}\"" | tee -a ${DOCKER_CONF} - ifconfig docker0 down - yum -y -q install bridge-utils && brctl delbr docker0 && service docker restart - ;; - centos|fedora) - DOCKER_CONF="/etc/sysconfig/docker" - sed -i "/^OPTIONS=/ s|\( --mtu=.*\)\?'$| --mtu=${FLANNEL_MTU} --bip=${FLANNEL_SUBNET}'|" ${DOCKER_CONF} - if ! command_exists ifconfig; then - yum -y -q install net-tools - fi - ifconfig docker0 down - yum -y -q install bridge-utils && brctl delbr docker0 && systemctl restart docker - ;; - ubuntu|debian) - DOCKER_CONF="/etc/default/docker" - echo "DOCKER_OPTS=\"\$DOCKER_OPTS --mtu=${FLANNEL_MTU} --bip=${FLANNEL_SUBNET}\"" | tee -a ${DOCKER_CONF} - ifconfig docker0 down - apt-get install bridge-utils - brctl delbr docker0 - service docker stop - while [ `ps aux | grep /usr/bin/docker | grep -v grep | wc -l` -gt 0 ]; do - echo "Waiting for docker to terminate" - sleep 1 - done - service docker start - ;; - *) - echo "Unsupported operations system ${lsb_dist}" - exit 1 - ;; - esac - - # sleep a little bit - sleep 5 - - # Start kubelet and then start master components as pods - docker run \ - --net=host \ - --pid=host \ - --privileged \ - --restart=on-failure \ - -d \ - -v /sys:/sys:ro \ - -v /var/run:/var/run:rw \ - -v /:/rootfs:ro \ - -v /var/lib/docker/:/var/lib/docker:rw \ - -v /var/lib/kubelet/:/var/lib/kubelet:rw \ - gcr.io/google_containers/hyperkube-${ARCH}:v${K8S_VERSION} \ - /hyperkube kubelet \ - --address=0.0.0.0 \ - --allow-privileged=true \ - --enable-server \ - --api-servers=http://localhost:8080 \ - --config=/etc/kubernetes/manifests-multi \ - --cluster-dns=10.0.0.10 \ - --cluster-domain=cluster.local \ - --containerized \ - --v=2 - -} - -echo "Detecting your OS distro ..." -detect_lsb - -echo "Starting bootstrap docker ..." -bootstrap_daemon - -echo "Starting k8s ..." -start_k8s - -echo "Master done!" diff --git a/docs/getting-started-guides/docker-multinode/skydns.yaml.in b/docs/getting-started-guides/docker-multinode/skydns.yaml.in deleted file mode 100644 index 098b85ddf3..0000000000 --- a/docs/getting-started-guides/docker-multinode/skydns.yaml.in +++ /dev/null @@ -1,136 +0,0 @@ -apiVersion: v1 -kind: ReplicationController -metadata: - name: kube-dns-v10 - namespace: kube-system - labels: - k8s-app: kube-dns - version: v10 - kubernetes.io/cluster-service: "true" -spec: - replicas: {{ pillar['dns_replicas'] }} - selector: - k8s-app: kube-dns - version: v10 - template: - metadata: - labels: - k8s-app: kube-dns - version: v10 - kubernetes.io/cluster-service: "true" - spec: - containers: - - name: etcd - image: gcr.io/google_containers/etcd-amd64:2.2.1 - resources: - # keep request = limit to keep this container in guaranteed class - limits: - cpu: 100m - memory: 50Mi - requests: - cpu: 100m - memory: 50Mi - command: - - /usr/local/bin/etcd - - -data-dir - - /var/etcd/data - - -listen-client-urls - - http://127.0.0.1:2379,http://127.0.0.1:4001 - - -advertise-client-urls - - http://127.0.0.1:2379,http://127.0.0.1:4001 - - -initial-cluster-token - - skydns-etcd - volumeMounts: - - name: etcd-storage - mountPath: /var/etcd/data - - name: kube2sky - image: gcr.io/google_containers/kube2sky:1.12 - resources: - # keep request = limit to keep this container in guaranteed class - limits: - cpu: 100m - memory: 50Mi - requests: - cpu: 100m - memory: 50Mi - args: - # command = "/kube2sky" - - --domain={{ pillar['dns_domain'] }} - - name: skydns - image: gcr.io/google_containers/skydns:2015-10-13-8c72f8c - resources: - # keep request = limit to keep this container in guaranteed class - limits: - cpu: 100m - memory: 50Mi - requests: - cpu: 100m - memory: 50Mi - args: - # command = "/skydns" - - -machines=http://127.0.0.1:4001 - - -addr=0.0.0.0:53 - - -ns-rotate=false - - -domain={{ pillar['dns_domain'] }}. - ports: - - containerPort: 53 - name: dns - protocol: UDP - - containerPort: 53 - name: dns-tcp - protocol: TCP - livenessProbe: - httpGet: - path: /healthz - port: 8080 - scheme: HTTP - initialDelaySeconds: 30 - timeoutSeconds: 5 - readinessProbe: - httpGet: - path: /healthz - port: 8080 - scheme: HTTP - initialDelaySeconds: 1 - timeoutSeconds: 5 - - name: healthz - image: gcr.io/google_containers/exechealthz:1.0 - resources: - # keep request = limit to keep this container in guaranteed class - limits: - cpu: 10m - memory: 20Mi - requests: - cpu: 10m - memory: 20Mi - args: - - -cmd=nslookup kubernetes.default.svc.{{ pillar['dns_domain'] }} 127.0.0.1 >/dev/null - - -port=8080 - ports: - - containerPort: 8080 - protocol: TCP - volumes: - - name: etcd-storage - emptyDir: {} - dnsPolicy: Default # Don't use cluster DNS. ---- -apiVersion: v1 -kind: Service -metadata: - name: kube-dns - namespace: kube-system - labels: - k8s-app: kube-dns - kubernetes.io/cluster-service: "true" - kubernetes.io/name: "KubeDNS" -spec: - selector: - k8s-app: kube-dns - clusterIP: {{ pillar['dns_server'] }} - ports: - - name: dns - port: 53 - protocol: UDP - - name: dns-tcp - port: 53 - protocol: TCP diff --git a/docs/getting-started-guides/docker-multinode/testing.md b/docs/getting-started-guides/docker-multinode/testing.md deleted file mode 100644 index c1492bf814..0000000000 --- a/docs/getting-started-guides/docker-multinode/testing.md +++ /dev/null @@ -1,68 +0,0 @@ ---- ---- - -To validate that your node(s) have been added, run: - -```shell -kubectl get nodes -``` - -That should show something like: - -```shell -NAME LABELS STATUS -10.240.99.26 kubernetes.io/hostname=10.240.99.26 Ready -127.0.0.1 kubernetes.io/hostname=127.0.0.1 Ready -``` - -If the status of any node is `Unknown` or `NotReady` your cluster is broken, double check that all containers are running properly, and if all else fails, contact us on [Slack](/docs/troubleshooting/#slack). - -### Run an application - -```shell -kubectl run nginx --image=nginx --port=80 -``` - -now run `docker ps` you should see nginx running. You may need to wait a few minutes for the image to get pulled. - -### Expose it as a service - -```shell -kubectl expose rc nginx --port=80 -``` - -Run the following command to obtain the IP of this service we just created. There are two IPs, the first one is internal (CLUSTER_IP), and the second one is the external load-balanced IP. - -```shell -kubectl get svc nginx -``` - -Alternatively, you can obtain only the first IP (CLUSTER_IP) by running: - -```shell -{% raw %}kubectl get svc nginx --template={{.spec.clusterIP}}{% endraw %} -``` - -Hit the webserver with the first IP (CLUSTER_IP): - -```shell -curl -``` - -Note that you will need run this curl command on your boot2docker VM if you are running on OS X. - -### Scaling - -Now try to scale up the nginx you created before: - -```shell -kubectl scale rc nginx --replicas=3 -``` - -And list the pods - -```shell -kubectl get pods -``` - -You should see pods landing on the newly added machine. diff --git a/docs/getting-started-guides/docker-multinode/worker.md b/docs/getting-started-guides/docker-multinode/worker.md deleted file mode 100644 index 799da471d4..0000000000 --- a/docs/getting-started-guides/docker-multinode/worker.md +++ /dev/null @@ -1,179 +0,0 @@ ---- ---- - -These instructions are very similar to the master set-up above, but they are duplicated for clarity. -You need to repeat these instructions for each node you want to join the cluster. -We will assume that you have the IP address of the master in `${MASTER_IP}` that you created in the [master instructions](/docs/getting-started-guides/docker-multinode/master/). We'll need to run several versioned Kubernetes components, so we'll assume that the version we want -to run is `${K8S_VERSION}`, which should hold a released version of Kubernetes >= "1.2.0-alpha.6" - -Enviroinment variables used: - -```shell -export MASTER_IP= -export K8S_VERSION= -export FLANNEL_VERSION= -export FLANNEL_IFACE= -export FLANNEL_IPMASQ= -``` - -For each worker node, there are three steps: - - * [Set up `flanneld` on the worker node](#set-up-flanneld-on-the-worker-node) - * [Start Kubernetes on the worker node](#start-kubernetes-on-the-worker-node) - * [Add the worker to the cluster](#add-the-node-to-the-cluster) - -### Set up Flanneld on the worker node - -As before, the Flannel daemon is going to provide network connectivity. - -_Note_: -This guide expects **Docker 1.7.1 or higher**. - - -#### Set up a bootstrap docker - -As previously, we need a second instance of the Docker daemon running to bootstrap the flannel networking. - -Run: - -```shell -sudo sh -c 'docker -d -H unix:///var/run/docker-bootstrap.sock -p /var/run/docker-bootstrap.pid --iptables=false --ip-masq=false --bridge=none --graph=/var/lib/docker-bootstrap 2> /var/log/docker-bootstrap.log 1> /dev/null &' -``` - -_If you have Docker 1.8.0 or higher run this instead_ - -```shell -sudo sh -c 'docker daemon -H unix:///var/run/docker-bootstrap.sock -p /var/run/docker-bootstrap.pid --iptables=false --ip-masq=false --bridge=none --graph=/var/lib/docker-bootstrap 2> /var/log/docker-bootstrap.log 1> /dev/null &' -``` - -_Important Note_: -If you are running this on a long running system, rather than experimenting, you should run the bootstrap Docker instance under something like SysV init, upstart or systemd so that it is restarted -across reboots and failures. - -#### Bring down Docker - -To re-configure Docker to use flannel, we need to take docker down, run flannel and then restart Docker. - -Turning down Docker is system dependent, it may be: - -```shell -sudo /etc/init.d/docker stop -``` - -or - -```shell -sudo systemctl stop docker -``` - -or it may be something else. - -#### Run flannel - -Now run flanneld itself, this call is slightly different from the above, since we point it at the etcd instance on the master. - -```shell -sudo docker -H unix:///var/run/docker-bootstrap.sock run -d \ - --net=host \ - --privileged \ - -v /dev/net:/dev/net \ - quay.io/coreos/flannel:${FLANNEL_VERSION} \ - /opt/bin/flanneld \ - --ip-masq=${FLANNEL_IPMASQ} \ - --etcd-endpoints=http://${MASTER_IP}:4001 \ - --iface=${FLANNEL_IFACE} -``` - -The previous command should have printed a really long hash, the container id, copy this hash. - -Now get the subnet settings from flannel: - -```shell -sudo docker -H unix:///var/run/docker-bootstrap.sock exec cat /run/flannel/subnet.env -``` - - -#### Edit the docker configuration - -You now need to edit the docker configuration to activate new flags. Again, this is system specific. - -This may be in `/etc/default/docker` or `/etc/systemd/service/docker.service` or it may be elsewhere. - -Regardless, you need to add the following to the docker command line: - -```shell ---bip=${FLANNEL_SUBNET} --mtu=${FLANNEL_MTU} -``` - -#### Remove the existing Docker bridge - -Docker creates a bridge named `docker0` by default. You need to remove this: - -```shell -sudo /sbin/ifconfig docker0 down -sudo brctl delbr docker0 -``` - -You may need to install the `bridge-utils` package for the `brctl` binary. - -#### Restart Docker - -Again this is system dependent, it may be: - -```shell -sudo /etc/init.d/docker start -``` - -or it may be: - -```shell -systemctl start docker -``` - -### Start Kubernetes on the worker node - -#### Run the kubelet - -Again this is similar to the above, but the `--api-servers` now points to the master we set up in the beginning. - -```shell -sudo docker run \ - --volume=/:/rootfs:ro \ - --volume=/sys:/sys:ro \ - --volume=/dev:/dev \ - --volume=/var/lib/docker/:/var/lib/docker:rw \ - --volume=/var/lib/kubelet/:/var/lib/kubelet:rw \ - --volume=/var/run:/var/run:rw \ - --net=host \ - --privileged=true \ - --pid=host \ - -d \ - gcr.io/google_containers/hyperkube-amd64:v${K8S_VERSION} \ - /hyperkube kubelet \ - --allow-privileged=true \ - --api-servers=http://${MASTER_IP}:8080 \ - --v=2 \ - --address=0.0.0.0 \ - --enable-server \ - --containerized \ - --cluster-dns=10.0.0.10 \ - --cluster-domain=cluster.local -``` - -#### Run the service proxy - -The service proxy provides load-balancing between groups of containers defined by Kubernetes `Services` - -```shell -sudo docker run -d \ - --net=host \ - --privileged \ - gcr.io/google_containers/hyperkube-amd64:v${K8S_VERSION} \ - /hyperkube proxy \ - --master=http://${MASTER_IP}:8080 \ - --v=2 -``` - -### Next steps - -Move on to [testing your cluster](/docs/getting-started-guides/docker-multinode/testing/) or [add another node](#adding-a-kubernetes-worker-node-via-docker) diff --git a/docs/getting-started-guides/docker-multinode/worker.sh b/docs/getting-started-guides/docker-multinode/worker.sh deleted file mode 100755 index 3989b5f09f..0000000000 --- a/docs/getting-started-guides/docker-multinode/worker.sh +++ /dev/null @@ -1,231 +0,0 @@ -#!/bin/bash - -# Copyright 2015 The Kubernetes Authors All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# A script to the k8s worker in docker containers. -# Authors @wizard_cxy @resouer - -set -e - -# Make sure docker daemon is running -if ( ! ps -ef | grep "/usr/bin/docker" | grep -v 'grep' &> /dev/null ); then - echo "Docker is not running on this machine!" - exit 1 -fi - -# Make sure k8s version env is properly set -K8S_VERSION=${K8S_VERSION:-"1.2.0-alpha.7"} -FLANNEL_VERSION=${FLANNEL_VERSION:-"0.5.5"} -FLANNEL_IFACE=${FLANNEL_IFACE:-"eth0"} -FLANNEL_IPMASQ=${FLANNEL_IPMASQ:-"true"} -ARCH=${ARCH:-"amd64"} - -# Run as root -if [ "$(id -u)" != "0" ]; then - echo >&2 "Please run as root" - exit 1 -fi - -# Make sure master ip is properly set -if [ -z ${MASTER_IP} ]; then - echo "Please export MASTER_IP in your env" - exit 1 -fi - -echo "K8S_VERSION is set to: ${K8S_VERSION}" -echo "FLANNEL_VERSION is set to: ${FLANNEL_VERSION}" -echo "FLANNEL_IFACE is set to: ${FLANNEL_IFACE}" -echo "FLANNEL_IPMASQ is set to: ${FLANNEL_IPMASQ}" -echo "MASTER_IP is set to: ${MASTER_IP}" -echo "ARCH is set to: ${ARCH}" - -# Check if a command is valid -command_exists() { - command -v "$@" > /dev/null 2>&1 -} - -lsb_dist="" - -# Detect the OS distro, we support ubuntu, debian, mint, centos, fedora dist -detect_lsb() { - case "$(uname -m)" in - *64) - ;; - *) - echo "Error: We currently only support 64-bit platforms." - exit 1 - ;; - esac - - if command_exists lsb_release; then - lsb_dist="$(lsb_release -si)" - fi - if [ -z ${lsb_dist} ] && [ -r /etc/lsb-release ]; then - lsb_dist="$(. /etc/lsb-release && echo "$DISTRIB_ID")" - fi - if [ -z ${lsb_dist} ] && [ -r /etc/debian_version ]; then - lsb_dist='debian' - fi - if [ -z ${lsb_dist} ] && [ -r /etc/fedora-release ]; then - lsb_dist='fedora' - fi - if [ -z ${lsb_dist} ] && [ -r /etc/os-release ]; then - lsb_dist="$(. /etc/os-release && echo "$ID")" - fi - - lsb_dist="$(echo ${lsb_dist} | tr '[:upper:]' '[:lower:]')" - - case "${lsb_dist}" in - amzn|centos|debian|ubuntu) - ;; - *) - echo "Error: We currently only support ubuntu|debian|amzn|centos." - exit 1 - ;; - esac -} - - -# Start the bootstrap daemon -bootstrap_daemon() { - # Detecting docker version so we could run proper docker_daemon command - [[ $(eval "docker --version") =~ ([0-9][.][0-9][.][0-9]*) ]] && version="${BASH_REMATCH[1]}" - local got=$(echo -e "${version}\n1.8.0" | sed '/^$/d' | sort -nr | head -1) - if [[ "${got}" = "${version}" ]]; then - docker_daemon="docker -d" - else - docker_daemon="docker daemon" - fi - ${docker_daemon} \ - -H unix:///var/run/docker-bootstrap.sock \ - -p /var/run/docker-bootstrap.pid \ - --iptables=false \ - --ip-masq=false \ - --bridge=none \ - --graph=/var/lib/docker-bootstrap \ - 2> /var/log/docker-bootstrap.log \ - 1> /dev/null & - - sleep 5 -} - -DOCKER_CONF="" - -# Start k8s components in containers -start_k8s() { - # Start flannel - flannelCID=$(docker -H unix:///var/run/docker-bootstrap.sock run \ - -d \ - --restart=on-failure \ - --net=host \ - --privileged \ - -v /dev/net:/dev/net \ - quay.io/coreos/flannel:${FLANNEL_VERSION} \ - /opt/bin/flanneld \ - --ip-masq="${FLANNEL_IPMASQ}" \ - --etcd-endpoints=http://${MASTER_IP}:4001 \ - --iface="${FLANNEL_IFACE}") - - sleep 10 - - # Copy flannel env out and source it on the host - docker -H unix:///var/run/docker-bootstrap.sock \ - cp ${flannelCID}:/run/flannel/subnet.env . - source subnet.env - - # Configure docker net settings, then restart it - case "${lsb_dist}" in - centos) - DOCKER_CONF="/etc/sysconfig/docker" - sed -i "/^OPTIONS=/ s|\( --mtu=.*\)\?'$| --mtu=${FLANNEL_MTU} --bip=${FLANNEL_SUBNET}'|" ${DOCKER_CONF} - if ! command_exists ifconfig; then - yum -y -q install net-tools - fi - ifconfig docker0 down - yum -y -q install bridge-utils && brctl delbr docker0 && systemctl restart docker - ;; - amzn) - DOCKER_CONF="/etc/sysconfig/docker" - echo "OPTIONS=\"\$OPTIONS --mtu=${FLANNEL_MTU} --bip=${FLANNEL_SUBNET}\"" | tee -a ${DOCKER_CONF} - ifconfig docker0 down - yum -y -q install bridge-utils && brctl delbr docker0 && service docker restart - ;; - ubuntu|debian) # TODO: today ubuntu uses systemd. Handle that too - DOCKER_CONF="/etc/default/docker" - echo "DOCKER_OPTS=\"\$DOCKER_OPTS --mtu=${FLANNEL_MTU} --bip=${FLANNEL_SUBNET}\"" | tee -a ${DOCKER_CONF} - ifconfig docker0 down - apt-get install bridge-utils - brctl delbr docker0 - service docker stop - while [ `ps aux | grep /usr/bin/docker | grep -v grep | wc -l` -gt 0 ]; do - echo "Waiting for docker to terminate" - sleep 1 - done - service docker start - ;; - *) - echo "Unsupported operations system ${lsb_dist}" - exit 1 - ;; - esac - - # sleep a little bit - sleep 5 - - # Start kubelet & proxy in container - # TODO: Use secure port for communication - docker run \ - --net=host \ - --pid=host \ - --privileged \ - --restart=on-failure \ - -d \ - -v /sys:/sys:ro \ - -v /var/run:/var/run:rw \ - -v /:/rootfs:ro \ - -v /var/lib/docker/:/var/lib/docker:rw \ - -v /var/lib/kubelet/:/var/lib/kubelet:rw \ - gcr.io/google_containers/hyperkube-${ARCH}:v${K8S_VERSION} \ - /hyperkube kubelet \ - --allow-privileged=true \ - --api-servers=http://${MASTER_IP}:8080 \ - --address=0.0.0.0 \ - --enable-server \ - --cluster-dns=10.0.0.10 \ - --cluster-domain=cluster.local \ - --containerized \ - --v=2 - - docker run \ - -d \ - --net=host \ - --privileged \ - --restart=on-failure \ - gcr.io/google_containers/hyperkube-${ARCH}:v${K8S_VERSION} \ - /hyperkube proxy \ - --master=http://${MASTER_IP}:8080 \ - --v=2 -} - -echo "Detecting your OS distro ..." -detect_lsb - -echo "Starting bootstrap docker ..." -bootstrap_daemon - -echo "Starting k8s ..." -start_k8s - -echo "Worker done!" From 8bf9235a60a4ae3166e62e25ee742b34c7bda90d Mon Sep 17 00:00:00 2001 From: Carter Morgan Date: Fri, 22 Jul 2016 15:09:15 -0700 Subject: [PATCH 237/348] Update hellonode.md to use the kubectl set image command Updated to use new command (which should simplify user experience) and update references from 1.2 to 1.3 --- docs/hellonode.md | 70 ++++------------------------------------------- 1 file changed, 6 insertions(+), 64 deletions(-) diff --git a/docs/hellonode.md b/docs/hellonode.md index 30e016745d..e399d43b8d 100755 --- a/docs/hellonode.md +++ b/docs/hellonode.md @@ -141,7 +141,7 @@ It’s now time to deploy your own containerized application to the Kubernetes c $ gcloud container clusters get-credentials hello-world ``` -**The rest of this document requires both the Kubernetes client and server version to be 1.2. Run `kubectl version` to see your current versions.** For 1.1 see [this document](https://github.com/kubernetes/kubernetes.github.io/blob/release-1.1/docs/hellonode.md). +**The rest of this document requires both the Kubernetes client and server version to be 1.3. Run `kubectl version` to see your current versions.** For 1.2 see [this document](https://github.com/kubernetes/kubernetes.github.io/blob/release-1.2/docs/hellonode.md). ## Create your pod @@ -210,7 +210,7 @@ From our development machine we can expose the pod to the public internet using kubectl expose deployment hello-node --type="LoadBalancer" ``` -**If this fails, make sure your client and server are both version 1.2. See the [Create your cluster](#create-your-cluster) section for details.** +**If this fails, make sure your client and server are both version 1.3. See the [Create your cluster](#create-your-cluster) section for details.** The flag used in this command specifies that we’ll be using the load-balancer provided by the underlying infrastructure (in this case the [Compute Engine load balancer](https://cloud.google.com/compute/docs/load-balancing/)). Note that we expose the deployment, and not the pod directly. This will cause the resulting service to load balance traffic across all pods managed by the deployment (in this case only 1 pod, but we will add more replicas later). @@ -291,69 +291,11 @@ Building and pushing this updated image should be much quicker as we take full a We’re now ready for Kubernetes to smoothly update our deployment to the new version of the application. In order to change the image label for our running container, we will need to edit the existing *hello-node deployment* and change the image from -`gcr.io/PROJECT_ID/hello-node:v1` to `gcr.io/PROJECT_ID/hello-node:v2`. To do this, we will use the `kubectl edit` command. -This will open up a text editor displaying the full deployment yaml [configuration](/docs/user-guide/configuring-containers/). It isn't necessary to understand the full yaml config -right now, instead just understand that by updating the `spec.template.spec.containers.image` field in the config we are telling -the deployment to update the pods to use the new image. +`gcr.io/PROJECT_ID/hello-node:v1` to `gcr.io/PROJECT_ID/hello-node:v2`. To do this, we will use the `kubectl set image` command. ```shell -kubectl edit deployment hello-node -``` - -```yaml -# Please edit the object below. Lines beginning with a '#' will be ignored, -# and an empty file will abort the edit. If an error occurs while saving this file will be -# reopened with the relevant failures. -# -apiVersion: extensions/v1beta1 -kind: Deployment -metadata: - annotations: - deployment.kubernetes.io/revision: "1" - creationTimestamp: 2016-03-24T17:55:28Z - generation: 3 - labels: - run: hello-node - name: hello-node - namespace: default - resourceVersion: "151017" - selfLink: /apis/extensions/v1beta1/namespaces/default/deployments/hello-node - uid: 981fe302-f1e9-11e5-9a78-42010af00005 -spec: - replicas: 4 - selector: - matchLabels: - run: hello-node - strategy: - rollingUpdate: - maxSurge: 1 - maxUnavailable: 1 - type: RollingUpdate - template: - metadata: - creationTimestamp: null - labels: - run: hello-node - spec: - containers: - - image: gcr.io/PROJECT_ID/hello-node:v1 # Update this line - imagePullPolicy: IfNotPresent - name: hello-node - ports: - - containerPort: 8080 - protocol: TCP - resources: {} - terminationMessagePath: /dev/termination-log - dnsPolicy: ClusterFirst - restartPolicy: Always - securityContext: {} - terminationGracePeriodSeconds: 30 -``` - -After making the change save and close the file. - -``` -deployment "hello-node" edited +$ kubectl set image deployment/hello-node hello-node=gcr.io/PROJECT_ID/hello-node:v2 +deployment "hello-node" image updated ``` This updates the deployment with the new image, causing new pods to be created with the new image and old pods to be deleted. @@ -370,7 +312,7 @@ Hopefully with these deployment, scaling and update features you’ll agree that ## Observe the Kubernetes Web UI (optional) -With Kubernetes 1.2, a graphical web user interface (dashboard) has been introduced. It is enabled by default for 1.2 clusters. +Kubernetes comes with a graphical web user interface that is enabled by default with your clusters. This user interface allows you to get started quickly and enables some of the functionality found in the CLI as a more approachable and discoverable way of interacting with the system. Enjoy the Kubernetes graphical dashboard and use it for deploying containerized applications, as well as for monitoring and managing your clusters! From c0100848acd5cce321ed7a0c5e0e9e0ec1dabf44 Mon Sep 17 00:00:00 2001 From: Chris Seto Date: Sat, 23 Jul 2016 18:03:33 -0400 Subject: [PATCH 238/348] s/creae/create/ --- docs/user-guide/petset.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/petset.md b/docs/user-guide/petset.md index f3d5d4c4fa..ec66d1079a 100644 --- a/docs/user-guide/petset.md +++ b/docs/user-guide/petset.md @@ -93,7 +93,7 @@ We'll create a basic Pet Set to demonstrate how Pets are assigned unique and "st {% include code.html language="yaml" file="petset.yaml" ghlink="/docs/user-guide/petset.yaml" %} -Saving this config into `petset.yaml` and submitting it to a Kubernetes cluster should creae the defined Pet Set and Pets it manages: +Saving this config into `petset.yaml` and submitting it to a Kubernetes cluster should create the defined Pet Set and Pets it manages: ```shell $ kubectl create -f petset.yaml From 4179f7034350aac30fda484299aa370283975928 Mon Sep 17 00:00:00 2001 From: Ben Dang Date: Mon, 25 Jul 2016 15:42:43 -0700 Subject: [PATCH 239/348] fix quotation when specifying a specific attribute --- docs/user-guide/walkthrough/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/walkthrough/index.md b/docs/user-guide/walkthrough/index.md index 69f395b433..3dfa0e00b4 100644 --- a/docs/user-guide/walkthrough/index.md +++ b/docs/user-guide/walkthrough/index.md @@ -56,7 +56,7 @@ On most providers, the pod IPs are not externally accessible. The easiest way to Provided the pod IP is accessible, you should be able to access its http endpoint with wget on port 80: ```shell{% raw %} -$ kubectl run busybox --image=busybox --restart=Never --tty -i --generator=run-pod/v1 --env "POD_IP=$(kubectl get pod nginx -o go-template={{.status.podIP}})" +$ kubectl run busybox --image=busybox --restart=Never --tty -i --generator=run-pod/v1 --env "POD_IP=$(kubectl get pod nginx -o go-template='{{.status.podIP}}')" u@busybox$ wget -qO- http://$POD_IP # Run in the busybox container u@busybox$ exit # Exit the busybox container $ kubectl delete pod busybox # Clean up the pod we created with "kubectl run" From 1ea60a21d25bbde18cf3eb611bf78207761f4727 Mon Sep 17 00:00:00 2001 From: Andrey Kurilin Date: Wed, 18 May 2016 17:13:53 +0300 Subject: [PATCH 240/348] Add diagram for `kubectl rolling-update` cmd Related issue: #528 --- images/docs/kubectl_rollingupdate.svg | 2 ++ 1 file changed, 2 insertions(+) create mode 100644 images/docs/kubectl_rollingupdate.svg diff --git a/images/docs/kubectl_rollingupdate.svg b/images/docs/kubectl_rollingupdate.svg new file mode 100644 index 0000000000..73c85d0ba7 --- /dev/null +++ b/images/docs/kubectl_rollingupdate.svg @@ -0,0 +1,2 @@ + +
$ kubectl rolling-update foo [foo-new flags]
[Not supported by viewer]
Not found
Not found<br>
get old replication controller for foo
[Not supported by viewer]
foo-new  or
--image option is specified
[Not supported by viewer]
Rename
<b>Rename</b>
list all RCs to find RC foo(check case when we're in the middle of a rename)
[Not supported by viewer]
Not Found
Not Found
EXIT
[Not supported by viewer]
--image
option is
 specified
[Not supported by viewer]
Create  new RC based on current one and update oldRc  with pointers to the 'next' controller
[Not supported by viewer]
--dryrun
option is
 specified
[Not supported by viewer]
Print oldRC and newRC
Print oldRC and newRC<br>
--rollback
option is
 specified
[Not supported by viewer]
Swap the controllers and update  annotations
[Not supported by viewer]
True
True
True
True
True
True
True
True
True
True
return error
return error<br>
True
True
return error
return error<br>
Print error and EXIT
<b>Print error and EXIT</b>
return error
return error<br> \ No newline at end of file From c85d8150e97e0284de6c3d12ef912a731f6b9ec5 Mon Sep 17 00:00:00 2001 From: kevinll Date: Tue, 26 Jul 2016 14:47:56 -0700 Subject: [PATCH 241/348] Update hellonode.md updating instructions for people using Docker Toolbox on OSX. https://github.com/boot2docker/boot2docker/issues/412#issuecomment-145242855 --- docs/hellonode.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/hellonode.md b/docs/hellonode.md index 30e016745d..eac2972c53 100755 --- a/docs/hellonode.md +++ b/docs/hellonode.md @@ -104,7 +104,11 @@ $ curl http://localhost:8080 Hello World! ``` -**If you recieve a `Connection refused` message from Docker for Mac, ensure you are using the latest version of Docker (1.12 or later).** +**If you recieve a `Connection refused` message from Docker for Mac, ensure you are using the latest version of Docker (1.12 or later). Alternatively, if you are using Docker Toolbox on OSX, make sure you are using the VM's IP and not localhost :** + +```shell +$ curl "http://$(docker-machine ip YOUR-VM-MACHINE-NAME):8080" +``` Let’s now stop the container. In this example, our app was running as Docker process `2c66d0efcbd4`, which we looked up with `docker ps`: From b56eff2384adf948cbcbc55cc47603df5ad18f06 Mon Sep 17 00:00:00 2001 From: Casey Davenport Date: Tue, 26 Jul 2016 16:47:16 -0700 Subject: [PATCH 242/348] Update Calico blurb --- docs/admin/networking.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/docs/admin/networking.md b/docs/admin/networking.md index 3557d7da49..4defb64471 100644 --- a/docs/admin/networking.md +++ b/docs/admin/networking.md @@ -177,10 +177,13 @@ complicated way to build an overlay network. This is endorsed by several of the "Big Shops" for networking. -### Calico +### Project Calico -[Calico](https://github.com/projectcalico/calico-containers) uses BGP to enable real container -IPs. +[Project Calico](https://github.com/projectcalico/calico-containers/blob/master/docs/cni/kubernetes/README.md) is an open source container networking provider and network policy engine. + +Calico provides a highly scalable networking and network policy solution for connecting Kubernetes pods based on the same IP networking principles as the internet. Calico can be deployed without encapsulation or overlays to provide high-performance, high-scale data center networking. Calico also provides fine-grained, intent based network security policy for Kubernetes pods via its distributed firewall. + +Calico can also be run in policy enforcement mode in conjunction with other networking solutions such as Flannel, aka [canal](https://github.com/tigera/canal), or native GCE networking. ### Romana From 30df8d7f369490fc65150f540279d1caf5e8e0a2 Mon Sep 17 00:00:00 2001 From: Age Mooij Date: Thu, 28 Jul 2016 11:40:57 +0200 Subject: [PATCH 243/348] Fixes bad label reference in "Creating a Service" section of connecting-applications.md My assumption here is that the linked service yaml file (`nginx-svc.yaml`) is correct and the text that refers to the contents of that file is incorrect. If that assumption is false, the `nginx-svc.yaml` file should be changed instead. But I suspect it is linked into multiple other docs and that this issue is the result of someone updating the file without updating all docs that reference it. --- docs/user-guide/connecting-applications.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/connecting-applications.md b/docs/user-guide/connecting-applications.md index f4ada5393e..b6c70366cb 100644 --- a/docs/user-guide/connecting-applications.md +++ b/docs/user-guide/connecting-applications.md @@ -59,7 +59,7 @@ This is equivalent to `kubectl create -f` the following yaml: {% include code.html language="yaml" file="nginx-svc.yaml" ghlink="/docs/user-guide/nginx-svc.yaml" %} -This specification will create a Service which targets TCP port 80 on any Pod with the `app=nginx` label, and expose it on an abstracted Service port (`targetPort`: is the port the container accepts traffic on, `port`: is the abstracted Service port, which can be any port other pods use to access the Service). View [service API object](/docs/api-reference/v1/definitions/#_v1_service) to see the list of supported fields in service definition. +This specification will create a Service which targets TCP port 80 on any Pod with the `run: my-nginx` label, and expose it on an abstracted Service port (`targetPort`: is the port the container accepts traffic on, `port`: is the abstracted Service port, which can be any port other pods use to access the Service). View [service API object](/docs/api-reference/v1/definitions/#_v1_service) to see the list of supported fields in service definition. Check your Service: ```shell From 3695094224e96307cfd08ab72e52473e7bfcca1f Mon Sep 17 00:00:00 2001 From: Ihor Dvoretskyi Date: Thu, 28 Jul 2016 13:16:14 +0300 Subject: [PATCH 244/348] Weave links fixes Updated links referring to weave documentation. --- docs/admin/networking.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/admin/networking.md b/docs/admin/networking.md index 3557d7da49..b4502f5ad7 100644 --- a/docs/admin/networking.md +++ b/docs/admin/networking.md @@ -156,9 +156,9 @@ Lars Kellogg-Stedman. ### Weave Net from Weaveworks -[Weave Net](https://www.weave.works/documentation/net-1-5-0-introducing-weave/) is a +[Weave Net](https://www.weave.works/documentation/net-1-6-0-introducing-weave/) is a resilient and simple to use network for Kubernetes and its hosted applications. -Weave Net runs as a [CNI plug-in](https://www.weave.works/documentation/net-1-5-0-cni-plugin/) +Weave Net runs as a [CNI plug-in](https://www.weave.works/docs/net/latest/cni-plugin/) or stand-alone. In either version, it doesn’t require any configuration or extra code to run, and in both cases, the network provides one IP address per pod - as is standard for Kubernetes. From 8f76449b13fb14d9d12f6d5a5d6f936a13cc0310 Mon Sep 17 00:00:00 2001 From: Matt Bruzek Date: Wed, 27 Jul 2016 17:55:34 -0500 Subject: [PATCH 245/348] Updating the juju docs with new commands and links. --- docs/getting-started-guides/juju.md | 185 ++++++++++++++++++---------- 1 file changed, 120 insertions(+), 65 deletions(-) diff --git a/docs/getting-started-guides/juju.md b/docs/getting-started-guides/juju.md index 313f10176d..ba70a6a5e1 100644 --- a/docs/getting-started-guides/juju.md +++ b/docs/getting-started-guides/juju.md @@ -1,10 +1,12 @@ --- --- -[Juju](https://jujucharms.com/docs/stable/about-juju) makes it easy to deploy -Kubernetes by provisioning, installing and configuring all the systems in -the cluster. Once deployed the cluster can easily scale up with one command -to increase the cluster size. +[Juju](https://jujucharms.com/docs/2.0/about-juju) encapsulates the +operational knowledge of provisioning, installing, and securing a Kubernetes +cluster into one step. Juju allows you to consistently deploy a cluster on +different cloud providers with a consistent repeatable open source operations. +Once deployed the cluster can easily scale up with one command to increase the +cluster size. The Juju Kubernetes work is curated by a very small group of community members. Let us know how we are doing. If you find any problems please open an @@ -23,7 +25,7 @@ and tag the issue with "juju" so we can find them. ### On Ubuntu -[Install the Juju client](https://jujucharms.com/get-started) +[Install the Juju client](https://jujucharms.com/docs/2.0/getting-started-general) > This documentation focuses on the Juju 2.0 release which will be > promoted to stable during the April 2016 release cycle. @@ -33,48 +35,55 @@ To paraphrase, on your local Ubuntu system: ```shell sudo add-apt-repository ppa:juju/devel sudo apt-get update -sudo apt-get install juju2 +sudo apt-get install juju ``` If you are using another distro/platform - please consult the -[getting started guide](https://jujucharms.com/get-started) to install the -Juju dependencies for your platform. +[getting started guide](https://jujucharms.com/docs/2.0/getting-started-general) +to install the Juju dependencies for your platform. ### With Docker -If you are not using Ubuntu or prefer the isolation of Docker, you may -run the following: +If you prefer the isolation of Docker, you can run the Juju client in a +container. Create a local directory to store the Juju configuration, then +volume mount the container: + +```shell +mkdir -p $HOME/.local/share/juju +docker run --rm -ti \ + -v $HOME/.local/share/juju:/home/ubuntu/.local/share/juju \ + jujusolutions/charmbox:devel +``` > While this is a common target, the charmbox flavors of images are > unofficial, and should be treated as experimental. If you encounter any issues > turning up the Kubernetes cluster with charmbox, please file a bug on the > [charmbox issue tracker](https://github.com/juju-solutions/charmbox/issues). -```shell -mkdir ~/.juju2 -sudo docker run -v ~/.juju2:/home/ubuntu/.local/share/juju -ti jujusolutions/charmbox:devel -``` - -### Configure Juju to point a cloud +### Configure Juju to your favorite cloud provider At this point you have access to the Juju client. Before you can deploy a -cluster you have to configure the credentials for the Juju cloud provider. +cluster you have to configure Juju with the +[cloud credentials](https://jujucharms.com/docs/2.0/credentials) for each +cloud provider you would like to use. -Juju [supports a wide variety of public clouds](#Cloud-compatibility) to set +Juju [supports a wide variety of public clouds](#cloud-compatibility) to set up the credentials for your chosen cloud see the -[cloud setup page](https://jujucharms.com/docs/devel/getting-started#2.-choose-a-cloud). +[cloud setup page](https://jujucharms.com/docs/devel/getting-started-general#2.-choose-a-cloud). After configuration is complete test your setup with a `juju bootstrap` -command: -`juju bootstrap $cloudname $cloudtype` you are ready to launch the -Kubernetes cluster. +command: `juju bootstrap $controllername $cloudtype` you are ready to launch +the Kubernetes cluster. ## Launch a Kubernetes cluster -You will need to export the `KUBERNETES_PROVIDER` environment variable before -bringing up the cluster. +You can deploy a Kubernetes cluster with Juju from the `kubernetes` directory of +the [kubernetes github project](https://github.com/kubernetes/kubernetes.git). +Clone the repository on your local system. Export the `KUBERNETES_PROVIDER` +environment variable before bringing up the cluster. ```shell +cd kubernetes export KUBERNETES_PROVIDER=juju cluster/kube-up.sh ``` @@ -82,47 +91,60 @@ cluster/kube-up.sh If this is your first time running the `kube-up.sh` script, it will attempt to install the required dependencies to get started with Juju. -Next it will deploy two nodes of Kubernetes, 1 unit of etcd, and network +The script will deploy two nodes of kubernetes, 1 unit of etcd, and network the units so containers on different hosts can communicate with each other. ## Exploring the cluster -The `juju status` command provides information about each unit in the cluster: +The `juju status` command provides information about each unit in the cluster: ```shell $ juju status +MODEL CONTROLLER CLOUD/REGION VERSION +default windows azure/centralus 2.0-beta13 -... (omitted for brevity) +APP VERSION STATUS EXPOSED ORIGIN CHARM REV OS +etcd active false jujucharms etcd 3 ubuntu +kubernetes active true jujucharms kubernetes 5 ubuntu -[Units] -ID WORKLOAD-STATE AGENT-STATE VERSION MACHINE PORTS PUBLIC-ADDRESS MESSAGE -etcd/0 active idle 2.0-beta2 1 54.146.50.29 Etcd leader running -kubernetes/0 active idle 2.0-beta2 2 6443/tcp,8088/tcp 54.205.204.227 Kubernetes follower running -kubernetes/1 active idle 2.0-beta2 3 6443/tcp,8088/tcp 54.145.57.114 Kubernetes leader running +RELATION PROVIDES CONSUMES TYPE +cluster etcd etcd peer +etcd etcd kubernetes regular +certificates kubernetes kubernetes peer -... (omitted for brevity) +UNIT WORKLOAD AGENT MACHINE PORTS PUBLIC-ADDRESS MESSAGE +etcd/0 active idle 0 2379/tcp 13.67.217.11 (leader) cluster is healthy +kubernetes/0 active idle 1 8088/tcp 13.67.219.76 Kubernetes running. +kubernetes/1 active idle 2 6443/tcp 13.67.219.182 (master) Kubernetes running. + +MACHINE STATE DNS INS-ID SERIES AZ +0 started 13.67.217.11 machine-0 trusty +1 started 13.67.219.76 machine-1 trusty +2 started 13.67.219.182 machine-2 trusty ``` ## Run some containers! -The `kubectl` file, the TLS certificates along with the configuration are -all available on the Kubernetes leader unit. Fetch the kubectl package so you +The `kubectl` file, and the TLS certificates along with the configuration are +all available on the Kubernetes master unit. Fetch the kubectl package so you can run commands on the new Kuberntetes cluster. -Use the `juju status` command to figure out which Kubernetes unit is the leader -and copy the file from the leader: +Use the `juju status` command to figure out which unit is the master. In the +example above the "kubernetes/1" unit is the master. Use the `juju scp` +command to copy the file from the unit: ```shell juju scp kubernetes/1:kubectl_package.tar.gz . tar xvfz kubectl_package.tar.gz -kubectl --kubeconfig config get pods +./kubectl --kubeconfig kubeconfig get pods ``` If you are not on a Linux amd64 host system, you will need to find or build a kubectl binary package for your architecture. -Put the config file in the home directory so you don't have to specify it on -the command line each time. The default location is `${HOME}/.kube/config`. +Copy the `kubeconfig` file to the home directory so you don't have to specify +it on the command line each time. The default location is +`${HOME}/.kube/config`. No pods will be available before starting a container: @@ -202,6 +224,7 @@ Finally delete the pod: juju ssh kubernetes/0 kubectl delete pods hello ``` + ## Scale up cluster Want larger Kubernetes nodes? It is easy to request different sizes of cloud @@ -209,12 +232,12 @@ resources from Juju by using **constraints**. You can increase the amount of CPU or memory (RAM) in any of the systems requested by Juju. This allows you to fine tune th Kubernetes cluster to fit your workload. Use flags on the bootstrap command or as a separate `juju constraints` command. Look to the -[Juju documentation for machine constraints](https://jujucharms.com/docs/devel/charms-constraints) +[Juju documentation for machine](https://jujucharms.com/docs/2.0/charms-constraints) details. ## Scale out cluster -Need more clusters? Juju makes it easy to add units of a charm: +Need more workers? Juju makes it easy to add units of a charm: ```shell juju add-unit kubernetes @@ -226,21 +249,28 @@ Or multiple units at one time: juju add-unit -n3 kubernetes ``` +You can also scale the etcd charm for more fault tolerant key/value storage: + +```shell +juju add-unit -n2 etcd +``` + ## Tear down cluster -We recommend that you use the `kube-down.sh` command when you are done using +We recommend that you use the `kube-down.sh` script when you are done using the cluster, as it properly brings down the cloud and removes some of the build directories. ```shell -./kube-down.sh +./cluster/kube-down.sh ``` -Alternately if you want stop the servers you can destroy your current Juju -environment. Use the `juju env` command to get the current environment name: +Alternately if you want stop the servers you can destroy the Juju model or the +controller. Use the `juju switch` command to get the current controller name: ```shell -juju kill-controller `juju env` +juju switch +juju destroy-controller $controllername ``` ## More Info @@ -260,33 +290,58 @@ project on github.com: ### Cloud compatibility -Juju runs natively against a variety of public cloud providers. Juju currently -works with [Amazon Web Service](https://jujucharms.com/docs/stable/config-aws), -[Windows Azure](https://jujucharms.com/docs/stable/config-azure), -[DigitalOcean](https://jujucharms.com/docs/stable/config-digitalocean), -[Google Compute Engine](https://jujucharms.com/docs/stable/config-gce), -[HP Public Cloud](https://jujucharms.com/docs/stable/config-hpcloud), -[Joyent](https://jujucharms.com/docs/stable/config-joyent), -[LXC](https://jujucharms.com/docs/stable/config-LXC), any -[OpenStack](https://jujucharms.com/docs/stable/config-openstack) deployment, -[Vagrant](https://jujucharms.com/docs/stable/config-vagrant), and -[Vmware vSphere](https://jujucharms.com/docs/stable/config-vmware). +Juju is cloud agnostic and gives you a consistent experience across different +cloud providers. Juju supports a variety of public cloud providers: [Amazon Web Service](https://jujucharms.com/docs/2.0/help-aws), +[Microsoft Azure](https://jujucharms.com/docs/2.0/help-azure), +[Google Compute Engine](https://jujucharms.com/docs/2.0/help-google), +[Joyent](https://jujucharms.com/docs/2.0/help-joyent), +[Rackspace](https://jujucharms.com/docs/2.0/help-rackspace), any +[OpenStack cloud](https://jujucharms.com/docs/2.0/clouds#specifying-additional-clouds), +and +[Vmware vSphere](https://jujucharms.com/docs/2.0/config-vmware). If you do not see your favorite cloud provider listed many clouds with ssh access can be configured for -[manual provisioning](https://jujucharms.com/docs/stable/config-manual). +[manual provisioning](https://jujucharms.com/docs/2.0/clouds-manual). + +There are three special types of clouds: MAAS, LXD and Manual. + +- [LXD](https://jujucharms.com/docs/2.0/clouds-LXD): Is a system level + container technology that is extremely fast and less overhead than + traditional virutalization. + [LXD is a container hypervisor](http://www.ubuntu.com/cloud/lxd) + that runs on any Linux host, providing the ability to spin up system + containers on the host machine. **Note** that at this time nested + containerization of Docker inside LXD is not supported, so you can not spin + up a Kubernetes cluster on LXD. For more details on using LXD, please see the + [LXD documentation](https://jujucharms.com/docs/2.0/clouds-LXD). + +- [MAAS](https://jujucharms.com/docs/2.0/clouds-maas): An acronym of Metal As + A Service, MAAS lets you treat physical servers like virtual machines in the + cloud. Rather than having to manage each server individually, MAAS turns your + bare metal into an elastic cloud-like resource. There is more information on + MAAS at the [MAAS website](http://maas.io/), and detailed instructions on + [using MAAS with Juju here](https://jujucharms.com/docs/2.0/clouds-maas). + + - [Manual](https://jujucharms.com/docs/2.0/clouds-manual): There may be + occasions where you can bring up machines for Juju to use which are not part + of a recognized public cloud or do not support other protocols used by Juju. + As long as you have SSH access to these machines, you can get part of the + Juju magic and deploy applications. + +To change to a different cloud you can use the `juju switch` command and set +up the credentials for that cloud provider and continue to use the `kubeup.sh` +script. ## Support Level IaaS Provider | Config. Mgmt | OS | Networking | Docs | Conforms | Support Level -------------------- | ------------ | ------ | ---------- | --------------------------------------------- | ---------| ---------------------------- -AWS | Juju | Ubuntu | flannel | [docs](/docs/getting-started-guides/juju) | | [Community](https://github.com/whitmo/bundle-kubernetes) ( [@whit](https://github.com/whitmo), [@matt](https://github.com/mbruzek), [@chuck](https://github.com/chuckbutler) ) -OpenStack/HPCloud | Juju | Ubuntu | flannel | [docs](/docs/getting-started-guides/juju) | | [Community](https://github.com/whitmo/bundle-kubernetes) ( [@whit](https://github.com/whitmo), [@matt](https://github.com/mbruzek), [@chuck](https://github.com/chuckbutler) ) -Joyent | Juju | Ubuntu | flannel | [docs](/docs/getting-started-guides/juju) | | [Community](https://github.com/whitmo/bundle-kubernetes) ( [@whit](https://github.com/whitmo), [@matt](https://github.com/mbruzek), [@chuck](https://github.com/chuckbutler) ) - +Amazon Web Services (AWS) | Juju | Ubuntu | flannel | [docs](/docs/getting-started-guides/juju) | | [Community](https://github.com/juju-solutions/bundle-kubernetes-core) ( [@mbruzek](https://github.com/mbruzek), [@chuckbutler](https://github.com/chuckbutler) ) +OpenStack | Juju | Ubuntu | flannel | [docs](/docs/getting-started-guides/juju) | | [Community](https://github.com/juju-solutions/bundle-kubernetes-core) ( [@mbruzek](https://github.com/mbruzek), [@chuckbutler](https://github.com/chuckbutler) ) +Microsoft Azure | Juju | Ubuntu | flannel | [docs](/docs/getting-started-guides/juju) | | [Community](https://github.com/juju-solutions/bundle-kubernetes-core) ( [@mbruzek](https://github.com/mbruzek), [@chuckbutler](https://github.com/chuckbutler) ) +Google Compute Engine (GCE) | Juju | Ubuntu | flannel | [docs](/docs/getting-started-guides/juju) | | [Community](https://github.com/juju-solutions/bundle-kubernetes-core) ( [@mbruzek](https://github.com/mbruzek), [@chuckbutler](https://github.com/chuckbutler) ) For support level information on all solutions, see the [Table of solutions](/docs/getting-started-guides/#table-of-solutions) chart. - - From 1ee8600ed526a36b0d5ad7ea1f719abc23115d28 Mon Sep 17 00:00:00 2001 From: Kelsey Hightower Date: Thu, 28 Jul 2016 11:50:47 -0700 Subject: [PATCH 246/348] Fix formatting of options and file paths --- docs/getting-started-guides/fedora/fedora_manual_config.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting-started-guides/fedora/fedora_manual_config.md b/docs/getting-started-guides/fedora/fedora_manual_config.md index 35735fb9c5..9872275b6c 100644 --- a/docs/getting-started-guides/fedora/fedora_manual_config.md +++ b/docs/getting-started-guides/fedora/fedora_manual_config.md @@ -30,7 +30,7 @@ fed-node = 192.168.121.65 * Install Kubernetes on all hosts - fed-{master,node}. This will also pull in docker. Also install etcd on fed-master. This guide has been tested with kubernetes-0.18 and beyond. * The [--enablerepo=updates-testing](https://fedoraproject.org/wiki/QA:Updates_Testing) directive in the yum command below will ensure that the most recent Kubernetes version that is scheduled for pre-release will be installed. This should be a more recent version than the Fedora "stable" release for Kubernetes that you would get without adding the directive. * If you want the very latest Kubernetes release [you can download and yum install the RPM directly from Fedora Koji](http://koji.fedoraproject.org/koji/packageinfo?packageID=19202) instead of using the yum install command below. -* Running on AWS EC2 with RHEL 7.2, you need to enable extras repository for yum by editing /etc/yum.repos.d/redhat-rhui.repo and changing the changing the enable=0 to enable=1 for extras. +* Running on AWS EC2 with RHEL 7.2, you need to enable "extras" repository for yum by editing `/etc/yum.repos.d/redhat-rhui.repo` and changing the changing the `enable=0` to `enable=1` for extras. ```shell yum -y install --enablerepo=updates-testing kubernetes From 6b8b1cf2b225cac79ecceab61420a8162a4a4dc8 Mon Sep 17 00:00:00 2001 From: Matt Bruzek Date: Thu, 28 Jul 2016 13:59:02 -0500 Subject: [PATCH 247/348] Addressing review comments from @chuckbutler and @kelseyhightower --- docs/getting-started-guides/juju.md | 42 ++++++----------------------- 1 file changed, 8 insertions(+), 34 deletions(-) diff --git a/docs/getting-started-guides/juju.md b/docs/getting-started-guides/juju.md index ba70a6a5e1..ac7ceef8d0 100644 --- a/docs/getting-started-guides/juju.md +++ b/docs/getting-started-guides/juju.md @@ -3,14 +3,13 @@ [Juju](https://jujucharms.com/docs/2.0/about-juju) encapsulates the operational knowledge of provisioning, installing, and securing a Kubernetes -cluster into one step. Juju allows you to consistently deploy a cluster on -different cloud providers with a consistent repeatable open source operations. -Once deployed the cluster can easily scale up with one command to increase the -cluster size. +cluster into one step. Juju allows you to deploy a Kubernetes cluster on +different cloud providers with a consistent, repeatable user experience. +Once deployed the cluster can easily scale up with one command. -The Juju Kubernetes work is curated by a very small group of community members. -Let us know how we are doing. If you find any problems please open an -[issue at the kubernetes project](https://github.com/kubernetes/kubernetes/issues) +The Juju Kubernetes work is curated by a dedicated team of community members, +let us know how we are doing. If you find any problems please open an +[issue on the kubernetes project](https://github.com/kubernetes/kubernetes/issues) and tag the issue with "juju" so we can find them. @@ -91,7 +90,7 @@ cluster/kube-up.sh If this is your first time running the `kube-up.sh` script, it will attempt to install the required dependencies to get started with Juju. -The script will deploy two nodes of kubernetes, 1 unit of etcd, and network +The script will deploy two nodes of Kubernetes, 1 unit of etcd, and network the units so containers on different hosts can communicate with each other. ## Exploring the cluster @@ -270,7 +269,7 @@ controller. Use the `juju switch` command to get the current controller name: ```shell juju switch -juju destroy-controller $controllername +juju destroy-controller $controllername --destroy-all-models ``` ## More Info @@ -304,31 +303,6 @@ If you do not see your favorite cloud provider listed many clouds with ssh access can be configured for [manual provisioning](https://jujucharms.com/docs/2.0/clouds-manual). -There are three special types of clouds: MAAS, LXD and Manual. - -- [LXD](https://jujucharms.com/docs/2.0/clouds-LXD): Is a system level - container technology that is extremely fast and less overhead than - traditional virutalization. - [LXD is a container hypervisor](http://www.ubuntu.com/cloud/lxd) - that runs on any Linux host, providing the ability to spin up system - containers on the host machine. **Note** that at this time nested - containerization of Docker inside LXD is not supported, so you can not spin - up a Kubernetes cluster on LXD. For more details on using LXD, please see the - [LXD documentation](https://jujucharms.com/docs/2.0/clouds-LXD). - -- [MAAS](https://jujucharms.com/docs/2.0/clouds-maas): An acronym of Metal As - A Service, MAAS lets you treat physical servers like virtual machines in the - cloud. Rather than having to manage each server individually, MAAS turns your - bare metal into an elastic cloud-like resource. There is more information on - MAAS at the [MAAS website](http://maas.io/), and detailed instructions on - [using MAAS with Juju here](https://jujucharms.com/docs/2.0/clouds-maas). - - - [Manual](https://jujucharms.com/docs/2.0/clouds-manual): There may be - occasions where you can bring up machines for Juju to use which are not part - of a recognized public cloud or do not support other protocols used by Juju. - As long as you have SSH access to these machines, you can get part of the - Juju magic and deploy applications. - To change to a different cloud you can use the `juju switch` command and set up the credentials for that cloud provider and continue to use the `kubeup.sh` script. From 76788af3253ec74a8473b9ed4d20ad5e93d1d743 Mon Sep 17 00:00:00 2001 From: Kelsey Hightower Date: Thu, 28 Jul 2016 12:58:49 -0700 Subject: [PATCH 248/348] fix typo --- docs/admin/admission-controllers.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/admin/admission-controllers.md b/docs/admin/admission-controllers.md index 3c5ba70c9a..9072522ac2 100644 --- a/docs/admin/admission-controllers.md +++ b/docs/admin/admission-controllers.md @@ -145,5 +145,5 @@ For Kubernetes >= 1.2.0, we strongly recommend running the following set of admi For Kubernetes >= 1.0.0, we strongly recommend running the following set of admission control plug-ins (order matters): ```shell ---admission-control=NamespaceLifecycle,LimitRanger,SecurityContextDeny,ServiceAccount,PersistentVolumeLabel,ResoureQuota +--admission-control=NamespaceLifecycle,LimitRanger,SecurityContextDeny,ServiceAccount,PersistentVolumeLabel,ResourceQuota ``` From 3ec08ff940a22d5e5afdfb10ecdd0a9b907bdc3a Mon Sep 17 00:00:00 2001 From: Christoph Held Date: Mon, 25 Jul 2016 15:11:50 +0200 Subject: [PATCH 249/348] described needed steps for shared mount propagation --- docs/getting-started-guides/docker.md | 102 +++++++++++++++----------- 1 file changed, 61 insertions(+), 41 deletions(-) diff --git a/docs/getting-started-guides/docker.md b/docs/getting-started-guides/docker.md index 56a19d2be3..0fc999c1b6 100644 --- a/docs/getting-started-guides/docker.md +++ b/docs/getting-started-guides/docker.md @@ -16,58 +16,79 @@ Here's a diagram of what the final result will look like: **Note: These steps have not been tested with the [Docker For Mac or Docker For Windows beta programs](https://blog.docker.com/2016/03/docker-for-mac-windows-beta/).** -1. You need to have docker installed on one machine. -2. Decide what Kubernetes version to use. Set the `${K8S_VERSION}` variable to - a released version of Kubernetes >= "v1.2.0". If you'd like to use the current stable version of Kubernetes, run the following: +1. You need to have Docker version >= "1.10" installed on the machine. +2. Enable mount propagation. Hyperkube is running in a container which has to mount volumes for other containers, for example in case of persistent storage. The required steps depend on the init system. -```sh -export K8S_VERSION=$(curl -sS https://storage.googleapis.com/kubernetes-release/release/stable.txt) -``` -and for the latest available version (including unstable releases): + In case of **systemd**, change MountFlags in the Docker unit file to shared. + ```shell + DOCKER_CONF=$(systemctl cat docker | head -1 | awk '{print $2}') + sed -i.bak 's/^\(MountFlags=\).*/\1shared/' $DOCKER_CONF + systemctl daemon-reload + systemctl restart docker + ``` + + **Otherwise**, manually set the mount point used by Hyperkube to be shared: + ```shell + mkdir -p /var/lib/kubelet + mount --bind /var/lib/kubelet /var/lib/kubelet + mount --make-shared /var/lib/kubelet + ``` -```sh -export K8S_VERSION=$(curl -sS https://storage.googleapis.com/kubernetes-release/release/latest.txt) -``` ### Run it -```shell -export ARCH=amd64 -docker run -d \ - --volume=/:/rootfs:ro \ - --volume=/sys:/sys:rw \ - --volume=/var/lib/docker/:/var/lib/docker:rw \ - --volume=/var/lib/kubelet/:/var/lib/kubelet:rw,shared \ - --volume=/var/run:/var/run:rw \ - --net=host \ - --pid=host \ - --privileged \ - gcr.io/google_containers/hyperkube-${ARCH}:${K8S_VERSION} \ - /hyperkube kubelet \ - --containerized \ - --hostname-override=127.0.0.1 \ - --api-servers=http://localhost:8080 \ - --config=/etc/kubernetes/manifests \ - --cluster-dns=10.0.0.10 \ - --cluster-domain=cluster.local \ - --allow-privileged --v=2 -``` +1. Decide which Kubernetes version to use. Set the `${K8S_VERSION}` variable to a version of Kubernetes >= "v1.2.0". -> Note that `--cluster-dns` and `--cluster-domain` is used to deploy dns, feel free to discard them if dns is not needed. -> If you would like to mount an external device as a volume, add `--volume=/dev:/dev` to the command above. It may however, cause some problems described in [#18230](https://github.com/kubernetes/kubernetes/issues/18230) + If you'd like to use the current **stable** version of Kubernetes, run the following: -> Architectures other than `amd64` are experimental and sometimes unstable, but feel free to try them out! Valid values: `arm`, `arm64` and `ppc64le`. ARM is available with Kubernetes version `v1.3.0-alpha.2` and higher. ARM 64-bit and PowerPC 64 little-endian are available with `v1.3.0-alpha.3` and higher. Track progress on multi-arch support [here](https://github.com/kubernetes/kubernetes/issues/17981) + ```sh + export K8S_VERSION=$(curl -sS https://storage.googleapis.com/kubernetes-release/release/stable.txt) + ``` -> If you are behind a proxy, you need to pass the proxy setup to curl in the containers to pull the certificates. Create a .curlrc under /root folder (because the containers are running as root) with the following line: -``` -proxy = : -``` + and for the **latest** available version (including unstable releases): -This actually runs the kubelet, which in turn runs a [pod](/docs/user-guide/pods/) that contains the other master components. + ```sh + export K8S_VERSION=$(curl -sS https://storage.googleapis.com/kubernetes-release/release/latest.txt) + ``` -** **SECURITY WARNING** ** services exposed via Kubernetes using Hyperkube are available on the host node's public network interface / IP address. Because of this, this guide is not suitable for any host node/server that is directly internet accessible. Refer to [#21735](https://github.com/kubernetes/kubernetes/issues/21735) for addtional info. +2. Start Hyperkube + + ```shell + export ARCH=amd64 + docker run -d \ + --volume=/sys:/sys:rw \ + --volume=/var/lib/docker/:/var/lib/docker:rw \ + --volume=/var/lib/kubelet/:/var/lib/kubelet:rw,shared \ + --volume=/var/run:/var/run:rw \ + --net=host \ + --pid=host \ + --privileged \ + gcr.io/google_containers/hyperkube-${ARCH}:${K8S_VERSION} \ + /hyperkube kubelet \ + --hostname-override=127.0.0.1 \ + --api-servers=http://localhost:8080 \ + --config=/etc/kubernetes/manifests \ + --cluster-dns=10.0.0.10 \ + --cluster-domain=cluster.local \ + --allow-privileged --v=2 + ``` + + > Note that `--cluster-dns` and `--cluster-domain` is used to deploy dns, feel free to discard them if dns is not needed. + + > If you would like to mount an external device as a volume, add `--volume=/dev:/dev` to the command above. It may however, cause some problems described in [#18230](https://github.com/kubernetes/kubernetes/issues/18230) + + > Architectures other than `amd64` are experimental and sometimes unstable, but feel free to try them out! Valid values: `arm`, `arm64` and `ppc64le`. ARM is available with Kubernetes version `v1.3.0-alpha.2` and higher. ARM 64-bit and PowerPC 64 little-endian are available with `v1.3.0-alpha.3` and higher. Track progress on multi-arch support [here](https://github.com/kubernetes/kubernetes/issues/17981) + + > If you are behind a proxy, you need to pass the proxy setup to curl in the containers to pull the certificates. Create a .curlrc under /root folder (because the containers are running as root) with the following line: + ``` + proxy = : + ``` + + This actually runs the kubelet, which in turn runs a [pod](/docs/user-guide/pods/) that contains the other master components. + + ** **SECURITY WARNING** ** services exposed via Kubernetes using Hyperkube are available on the host node's public network interface / IP address. Because of this, this guide is not suitable for any host node/server that is directly internet accessible. Refer to [#21735](https://github.com/kubernetes/kubernetes/issues/21735) for addtional info. ### Download `kubectl` @@ -229,4 +250,3 @@ For support level information on all solutions, see the [Table of solutions](/do Please see the [Kubernetes docs](/docs/) for more details on administering and using a Kubernetes cluster. - From 3e8d78fd5cecd3118d4a61f60e282af4b801a09b Mon Sep 17 00:00:00 2001 From: Sha Date: Fri, 29 Jul 2016 16:42:32 +0800 Subject: [PATCH 250/348] Update index.md typo fix `etcd get foo` => `etcdctl get foo` --- docs/admin/high-availability/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/admin/high-availability/index.md b/docs/admin/high-availability/index.md index 01ded6eb05..d7932e47ca 100644 --- a/docs/admin/high-availability/index.md +++ b/docs/admin/high-availability/index.md @@ -111,7 +111,7 @@ and etcdctl cluster-health ``` -You can also validate that this is working with `etcdctl set foo bar` on one node, and `etcd get foo` +You can also validate that this is working with `etcdctl set foo bar` on one node, and `etcdctl get foo` on a different node. ### Even more reliable storage From ff12cf1a0ba18ae613063fbea135dfc2ae96d5f6 Mon Sep 17 00:00:00 2001 From: Ilya Dmitrichenko Date: Mon, 4 Apr 2016 17:35:06 +0100 Subject: [PATCH 251/348] coreos/azure: The provisioning code has been moved into an external repository --- .../coreos/azure/addons/skydns-rc.yaml | 99 ------ .../coreos/azure/addons/skydns-svc.yaml | 20 -- .../coreos/azure/azure-login.js | 3 - .../kubernetes-cluster-etcd-node-template.yml | 19 -- .../coreos/azure/create-kubernetes-cluster.js | 15 - .../coreos/azure/destroy-cluster.js | 7 - .../coreos/azure/expose_guestbook_app_port.sh | 29 -- .../coreos/azure/lib/azure_wrapper.js | 286 ------------------ .../coreos/azure/lib/cloud_config.js | 58 ---- .../azure/lib/deployment_logic/kubernetes.js | 77 ----- .../coreos/azure/lib/util.js | 33 -- .../coreos/azure/scale-kubernetes-cluster.js | 10 - 12 files changed, 656 deletions(-) delete mode 100644 docs/getting-started-guides/coreos/azure/addons/skydns-rc.yaml delete mode 100644 docs/getting-started-guides/coreos/azure/addons/skydns-svc.yaml delete mode 100755 docs/getting-started-guides/coreos/azure/azure-login.js delete mode 100644 docs/getting-started-guides/coreos/azure/cloud_config_templates/kubernetes-cluster-etcd-node-template.yml delete mode 100755 docs/getting-started-guides/coreos/azure/create-kubernetes-cluster.js delete mode 100755 docs/getting-started-guides/coreos/azure/destroy-cluster.js delete mode 100755 docs/getting-started-guides/coreos/azure/expose_guestbook_app_port.sh delete mode 100644 docs/getting-started-guides/coreos/azure/lib/azure_wrapper.js delete mode 100644 docs/getting-started-guides/coreos/azure/lib/cloud_config.js delete mode 100644 docs/getting-started-guides/coreos/azure/lib/deployment_logic/kubernetes.js delete mode 100644 docs/getting-started-guides/coreos/azure/lib/util.js delete mode 100755 docs/getting-started-guides/coreos/azure/scale-kubernetes-cluster.js diff --git a/docs/getting-started-guides/coreos/azure/addons/skydns-rc.yaml b/docs/getting-started-guides/coreos/azure/addons/skydns-rc.yaml deleted file mode 100644 index ee31c5107c..0000000000 --- a/docs/getting-started-guides/coreos/azure/addons/skydns-rc.yaml +++ /dev/null @@ -1,99 +0,0 @@ -apiVersion: v1 -kind: ReplicationController -metadata: - name: kube-dns-v9 - namespace: kube-system - labels: - k8s-app: kube-dns - version: v9 - kubernetes.io/cluster-service: "true" -spec: - replicas: 3 - selector: - k8s-app: kube-dns - version: v9 - template: - metadata: - labels: - k8s-app: kube-dns - version: v9 - kubernetes.io/cluster-service: "true" - spec: - containers: - - name: etcd - image: gcr.io/google_containers/etcd:2.0.9 - resources: - limits: - cpu: 100m - memory: 50Mi - command: - - /usr/local/bin/etcd - - -data-dir - - /var/etcd/data - - -listen-client-urls - - http://127.0.0.1:2379,http://127.0.0.1:4001 - - -advertise-client-urls - - http://127.0.0.1:2379,http://127.0.0.1:4001 - - -initial-cluster-token - - skydns-etcd - volumeMounts: - - name: etcd-storage - mountPath: /var/etcd/data - - name: kube2sky - image: gcr.io/google_containers/kube2sky:1.11 - resources: - limits: - cpu: 100m - memory: 50Mi - args: - # command = "/kube2sky" - - -domain=kube.local - - -kube_master_url=http://kube-00:8080 - - name: skydns - image: gcr.io/google_containers/skydns:2015-03-11-001 - resources: - limits: - cpu: 100m - memory: 50Mi - args: - # command = "/skydns" - - -machines=http://localhost:4001 - - -addr=0.0.0.0:53 - - -domain=kube.local - ports: - - containerPort: 53 - name: dns - protocol: UDP - - containerPort: 53 - name: dns-tcp - protocol: TCP - livenessProbe: - httpGet: - path: /healthz - port: 8080 - scheme: HTTP - initialDelaySeconds: 30 - timeoutSeconds: 5 - readinessProbe: - httpGet: - path: /healthz - port: 8080 - scheme: HTTP - initialDelaySeconds: 1 - timeoutSeconds: 5 - - name: healthz - image: gcr.io/google_containers/exechealthz:1.0 - resources: - limits: - cpu: 10m - memory: 20Mi - args: - - -cmd=nslookup kubernetes.default.svc.kube.local localhost >/dev/null - - -port=8080 - ports: - - containerPort: 8080 - protocol: TCP - volumes: - - name: etcd-storage - emptyDir: {} - dnsPolicy: Default # Don't use cluster DNS. diff --git a/docs/getting-started-guides/coreos/azure/addons/skydns-svc.yaml b/docs/getting-started-guides/coreos/azure/addons/skydns-svc.yaml deleted file mode 100644 index c15822d6bc..0000000000 --- a/docs/getting-started-guides/coreos/azure/addons/skydns-svc.yaml +++ /dev/null @@ -1,20 +0,0 @@ -apiVersion: v1 -kind: Service -metadata: - name: kube-dns - namespace: kube-system - labels: - k8s-app: kube-dns - kubernetes.io/cluster-service: "true" - kubernetes.io/name: "KubeDNS" -spec: - selector: - k8s-app: kube-dns - clusterIP: 10.16.0.3 - ports: - - name: dns - port: 53 - protocol: UDP - - name: dns-tcp - port: 53 - protocol: TCP diff --git a/docs/getting-started-guides/coreos/azure/azure-login.js b/docs/getting-started-guides/coreos/azure/azure-login.js deleted file mode 100755 index 624916b2b5..0000000000 --- a/docs/getting-started-guides/coreos/azure/azure-login.js +++ /dev/null @@ -1,3 +0,0 @@ -#!/usr/bin/env node - -require('child_process').fork('node_modules/azure-cli/bin/azure', ['login'].concat(process.argv)); diff --git a/docs/getting-started-guides/coreos/azure/cloud_config_templates/kubernetes-cluster-etcd-node-template.yml b/docs/getting-started-guides/coreos/azure/cloud_config_templates/kubernetes-cluster-etcd-node-template.yml deleted file mode 100644 index 4cbb480e53..0000000000 --- a/docs/getting-started-guides/coreos/azure/cloud_config_templates/kubernetes-cluster-etcd-node-template.yml +++ /dev/null @@ -1,19 +0,0 @@ -## This file is used as input to deployment script, which amends it as needed. -## More specifically, we need to add peer hosts for each but the elected peer. - -coreos: - units: - - name: etcd2.service - enable: true - command: start - etcd2: - name: '%H' - initial-cluster-token: 'etcd-cluster' - initial-advertise-peer-urls: 'http://%H:2380' - listen-peer-urls: 'http://%H:2380' - listen-client-urls: 'http://0.0.0.0:2379,http://0.0.0.0:4001' - advertise-client-urls: 'http://%H:2379,http://%H:4001' - initial-cluster-state: 'new' - update: - group: stable - reboot-strategy: off diff --git a/docs/getting-started-guides/coreos/azure/create-kubernetes-cluster.js b/docs/getting-started-guides/coreos/azure/create-kubernetes-cluster.js deleted file mode 100755 index 70248c596c..0000000000 --- a/docs/getting-started-guides/coreos/azure/create-kubernetes-cluster.js +++ /dev/null @@ -1,15 +0,0 @@ -#!/usr/bin/env node - -var azure = require('./lib/azure_wrapper.js'); -var kube = require('./lib/deployment_logic/kubernetes.js'); - -azure.create_config('kube', { 'etcd': 3, 'kube': 3 }); - -azure.run_task_queue([ - azure.queue_default_network(), - azure.queue_storage_if_needed(), - azure.queue_machines('etcd', 'stable', - kube.create_etcd_cloud_config), - azure.queue_machines('kube', 'stable', - kube.create_node_cloud_config), -]); diff --git a/docs/getting-started-guides/coreos/azure/destroy-cluster.js b/docs/getting-started-guides/coreos/azure/destroy-cluster.js deleted file mode 100755 index ce441e538a..0000000000 --- a/docs/getting-started-guides/coreos/azure/destroy-cluster.js +++ /dev/null @@ -1,7 +0,0 @@ -#!/usr/bin/env node - -var azure = require('./lib/azure_wrapper.js'); - -azure.destroy_cluster(process.argv[2]); - -console.log('The cluster had been destroyed, you can delete the state file now.'); diff --git a/docs/getting-started-guides/coreos/azure/expose_guestbook_app_port.sh b/docs/getting-started-guides/coreos/azure/expose_guestbook_app_port.sh deleted file mode 100755 index 65dfaf5d3a..0000000000 --- a/docs/getting-started-guides/coreos/azure/expose_guestbook_app_port.sh +++ /dev/null @@ -1,29 +0,0 @@ -#!/bin/bash - -# Copyright 2014 The Kubernetes Authors All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -set -e - -[ ! -z $1 ] || (echo Usage: $0 ssh_conf; exit 1) - -fe_port=$(ssh -F $1 kube-00 \ - "/opt/bin/kubectl get -o template --template='{{(index .spec.ports 0).nodePort}}' services frontend -L name=frontend" \ -) - -echo "Guestbook app is on port $fe_port, will map it to port 80 on kube-00" - -./node_modules/.bin/azure vm endpoint create kube-00 80 $fe_port - -./node_modules/.bin/azure vm endpoint show kube-00 tcp-80-${fe_port} diff --git a/docs/getting-started-guides/coreos/azure/lib/azure_wrapper.js b/docs/getting-started-guides/coreos/azure/lib/azure_wrapper.js deleted file mode 100644 index 93402c10ed..0000000000 --- a/docs/getting-started-guides/coreos/azure/lib/azure_wrapper.js +++ /dev/null @@ -1,286 +0,0 @@ -var _ = require('underscore'); - -var fs = require('fs'); -var cp = require('child_process'); - -var yaml = require('js-yaml'); - -var openssl = require('openssl-wrapper'); - -var clr = require('colors'); -var inspect = require('util').inspect; - -var util = require('./util.js'); - -var coreos_image_ids = { - 'stable': '2b171e93f07c4903bcad35bda10acf22__CoreOS-Stable-835.12.0', // untested - 'beta': '2b171e93f07c4903bcad35bda10acf22__CoreOS-Beta-899.6.0', - 'alpha': '2b171e93f07c4903bcad35bda10acf22__CoreOS-Alpha-942.0.0' // untested -}; - -var conf = {}; - -var hosts = { - collection: [], - ssh_port_counter: 2200, -}; - -var task_queue = []; - -exports.run_task_queue = function (dummy) { - var tasks = { - todo: task_queue, - done: [], - }; - - var pop_task = function() { - console.log(clr.yellow('azure_wrapper/task:'), clr.grey(inspect(tasks))); - var ret = {}; - ret.current = tasks.todo.shift(); - ret.remaining = tasks.todo.length; - return ret; - }; - - (function iter (task) { - if (task.current === undefined) { - if (conf.destroying === undefined) { - create_ssh_conf(); - save_state(); - } - return; - } else { - if (task.current.length !== 0) { - console.log(clr.yellow('azure_wrapper/exec:'), clr.blue(inspect(task.current))); - cp.fork('node_modules/azure-cli/bin/azure', task.current) - .on('exit', function (code, signal) { - tasks.done.push({ - code: code, - signal: signal, - what: task.current.join(' '), - remaining: task.remaining, - }); - if (code !== 0 && conf.destroying === undefined) { - console.log(clr.red('azure_wrapper/fail: Exiting due to an error.')); - save_state(); - console.log(clr.cyan('azure_wrapper/info: You probably want to destroy and re-run.')); - process.abort(); - } else { - iter(pop_task()); - } - }); - } else { - iter(pop_task()); - } - } - })(pop_task()); -}; - -var save_state = function () { - var file_name = util.join_output_file_path(conf.name, 'deployment.yml'); - try { - conf.hosts = hosts.collection; - fs.writeFileSync(file_name, yaml.safeDump(conf)); - console.log(clr.yellow('azure_wrapper/info: Saved state into `%s`'), file_name); - } catch (e) { - console.log(clr.red(e)); - } -}; - -var load_state = function (file_name) { - try { - conf = yaml.safeLoad(fs.readFileSync(file_name, 'utf8')); - console.log(clr.yellow('azure_wrapper/info: Loaded state from `%s`'), file_name); - return conf; - } catch (e) { - console.log(clr.red(e)); - } -}; - -var create_ssh_key = function (prefix) { - var opts = { - x509: true, - nodes: true, - newkey: 'rsa:2048', - subj: '/O=Weaveworks, Inc./L=London/C=GB/CN=weave.works', - keyout: util.join_output_file_path(prefix, 'ssh.key'), - out: util.join_output_file_path(prefix, 'ssh.pem'), - }; - openssl.exec('req', opts, function (err, buffer) { - if (err) console.log(clr.red(err)); - openssl.exec('rsa', { in: opts.keyout, out: opts.keyout }, function (err, buffer) { - if (err) console.log(clr.red(err)); - fs.chmod(opts.keyout, '0600', function (err) { - if (err) console.log(clr.red(err)); - }); - }); - }); - return { - key: opts.keyout, - pem: opts.out, - } -} - -var create_ssh_conf = function () { - var file_name = util.join_output_file_path(conf.name, 'ssh_conf'); - var ssh_conf_head = [ - "Host *", - "\tHostname " + conf.resources['service'] + ".cloudapp.net", - "\tUser core", - "\tCompression yes", - "\tLogLevel FATAL", - "\tStrictHostKeyChecking no", - "\tUserKnownHostsFile /dev/null", - "\tIdentitiesOnly yes", - "\tIdentityFile " + conf.resources['ssh_key']['key'], - "\n", - ]; - - fs.writeFileSync(file_name, ssh_conf_head.concat(_.map(hosts.collection, function (host) { - return _.template("Host <%= name %>\n\tPort <%= port %>\n")(host); - })).join('\n')); - console.log(clr.yellow('azure_wrapper/info:'), clr.green('Saved SSH config, you can use it like so: `ssh -F ', file_name, '`')); - console.log(clr.yellow('azure_wrapper/info:'), clr.green('The hosts in this deployment are:\n'), _.map(hosts.collection, function (host) { return host.name; })); -}; - -var get_location = function () { - if (process.env['AZ_AFFINITY']) { - return '--affinity-group=' + process.env['AZ_AFFINITY']; - } else if (process.env['AZ_LOCATION']) { - return '--location=' + process.env['AZ_LOCATION']; - } else { - return '--location=West Europe'; - } -} -var get_vm_size = function () { - if (process.env['AZ_VM_SIZE']) { - return '--vm-size=' + process.env['AZ_VM_SIZE']; - } else { - return '--vm-size=Small'; - } -} - -var get_subscription= function () { - if (process.env['AZ_SUBSCRIPTION']) { - return '--subscription=' + process.env['AZ_SUBSCRIPTION']; - } -} - -exports.queue_default_network = function () { - task_queue.push([ - 'network', 'vnet', 'create', - get_location(), - '--address-space=172.16.0.0', - get_subscription(), - conf.resources['vnet'], - ]); -} - -exports.queue_storage_if_needed = function() { - if (!process.env['AZURE_STORAGE_ACCOUNT']) { - conf.resources['storage_account'] = util.rand_suffix; - task_queue.push([ - 'storage', 'account', 'create', - '--type=LRS', - get_location(), - get_subscription(), - conf.resources['storage_account'], - ]); - process.env['AZURE_STORAGE_ACCOUNT'] = conf.resources['storage_account']; - } else { - // Preserve it for resizing, so we don't create a new one by accident, - // when the environment variable is unset - conf.resources['storage_account'] = process.env['AZURE_STORAGE_ACCOUNT']; - } -}; - -exports.queue_machines = function (name_prefix, coreos_update_channel, cloud_config_creator) { - var x = conf.nodes[name_prefix]; - var vm_create_base_args = [ - 'vm', 'create', - get_location(), - get_vm_size(), - '--connect=' + conf.resources['service'], - '--virtual-network-name=' + conf.resources['vnet'], - '--no-ssh-password', - '--ssh-cert=' + conf.resources['ssh_key']['pem'], - get_subscription(), - ]; - - var cloud_config = cloud_config_creator(x, conf); - - var next_host = function (n) { - hosts.ssh_port_counter += 1; - var host = { name: util.hostname(n, name_prefix), port: hosts.ssh_port_counter }; - if (cloud_config instanceof Array) { - host.cloud_config_file = cloud_config[n]; - } else { - host.cloud_config_file = cloud_config; - } - hosts.collection.push(host); - return _.map([ - "--vm-name=<%= name %>", - "--ssh=<%= port %>", - "--custom-data=<%= cloud_config_file %>", - ], function (arg) { return _.template(arg)(host); }); - }; - - task_queue = task_queue.concat(_(x).times(function (n) { - if (conf.resizing && n < conf.old_size) { - return []; - } else { - if (process.env['AZ_VM_COREOS_CHANNEL']) { - coreos_update_channel = process.env['AZ_VM_COREOS_CHANNEL'] - } - return vm_create_base_args.concat(next_host(n), [ - coreos_image_ids[coreos_update_channel], 'core', - ]); - } - })); -}; - -exports.create_config = function (name, nodes) { - conf = { - name: name, - nodes: nodes, - weave_salt: util.rand_string(), - resources: { - vnet: [name, 'internal-vnet', util.rand_suffix].join('-'), - service: [name, util.rand_suffix].join('-'), - ssh_key: create_ssh_key(name), - } - }; - -}; - -exports.destroy_cluster = function (state_file) { - load_state(state_file); - if (conf.hosts === undefined) { - console.log(clr.red('azure_wrapper/fail: Nothing to delete.')); - process.abort(); - } - - conf.destroying = true; - task_queue = _.map(conf.hosts, function (host) { - return ['vm', 'delete', '--quiet', '--blob-delete', host.name, get_subscription()]; - }); - - task_queue.push(['network', 'vnet', 'delete', '--quiet', conf.resources['vnet'], get_subscription()]); - task_queue.push(['storage', 'account', 'delete', '--quiet', conf.resources['storage_account'], get_subscription()]); - - exports.run_task_queue(); -}; - -exports.load_state_for_resizing = function (state_file, node_type, new_nodes) { - load_state(state_file); - if (conf.hosts === undefined) { - console.log(clr.red('azure_wrapper/fail: Nothing to look at.')); - process.abort(); - } - conf.resizing = true; - conf.old_size = conf.nodes[node_type]; - conf.old_state_file = state_file; - conf.nodes[node_type] += new_nodes; - hosts.collection = conf.hosts; - hosts.ssh_port_counter += conf.hosts.length; - process.env['AZURE_STORAGE_ACCOUNT'] = conf.resources['storage_account']; -} diff --git a/docs/getting-started-guides/coreos/azure/lib/cloud_config.js b/docs/getting-started-guides/coreos/azure/lib/cloud_config.js deleted file mode 100644 index d08b3f06ae..0000000000 --- a/docs/getting-started-guides/coreos/azure/lib/cloud_config.js +++ /dev/null @@ -1,58 +0,0 @@ -var _ = require('underscore'); -var fs = require('fs'); -var yaml = require('js-yaml'); -var colors = require('colors/safe'); - -var write_cloud_config_from_object = function (data, output_file) { - try { - fs.writeFileSync(output_file, [ - '#cloud-config', - yaml.safeDump(data), - ].join("\n")); - return output_file; - } catch (e) { - console.log(colors.red(e)); - } -}; - -exports.generate_environment_file_entry_from_object = function (hostname, environ) { - var data = { - hostname: hostname, - environ_array: _.map(environ, function (value, key) { - return [key.toUpperCase(), JSON.stringify(value.toString())].join('='); - }), - }; - - return { - permissions: '0600', - owner: 'root', - content: _.template("<%= environ_array.join('\\n') %>\n")(data), - path: _.template("/etc/weave.<%= hostname %>.env")(data), - }; -}; - -exports.process_template = function (input_file, output_file, processor) { - var data = {}; - try { - data = yaml.safeLoad(fs.readFileSync(input_file, 'utf8')); - } catch (e) { - console.log(colors.red(e)); - } - return write_cloud_config_from_object(processor(_.clone(data)), output_file); -}; - -exports.write_files_from = function (local_dir, remote_dir) { - try { - return _.map(fs.readdirSync(local_dir), function (fn) { - return { - path: [remote_dir, fn].join('/'), - owner: 'root', - permissions: '0640', - encoding: 'base64', - content: fs.readFileSync([local_dir, fn].join('/')).toString('base64'), - }; - }); - } catch (e) { - console.log(colors.red(e)); - } -}; diff --git a/docs/getting-started-guides/coreos/azure/lib/deployment_logic/kubernetes.js b/docs/getting-started-guides/coreos/azure/lib/deployment_logic/kubernetes.js deleted file mode 100644 index 2002b43a53..0000000000 --- a/docs/getting-started-guides/coreos/azure/lib/deployment_logic/kubernetes.js +++ /dev/null @@ -1,77 +0,0 @@ -var _ = require('underscore'); -_.mixin(require('underscore.string').exports()); - -var util = require('../util.js'); -var cloud_config = require('../cloud_config.js'); - - -etcd_initial_cluster_conf_self = function (conf) { - var port = '2380'; - - var data = { - nodes: _(conf.nodes.etcd).times(function (n) { - var host = util.hostname(n, 'etcd'); - return [host, [host, port].join(':')].join('=http://'); - }), - }; - - return { - 'name': 'etcd2.service', - 'drop-ins': [{ - 'name': '50-etcd-initial-cluster.conf', - 'content': _.template("[Service]\nEnvironment=ETCD_INITIAL_CLUSTER=<%= nodes.join(',') %>\n")(data), - }], - }; -}; - -etcd_initial_cluster_conf_kube = function (conf) { - var port = '4001'; - - var data = { - nodes: _(conf.nodes.etcd).times(function (n) { - var host = util.hostname(n, 'etcd'); - return 'http://' + [host, port].join(':'); - }), - }; - - return { - 'name': 'kube-apiserver.service', - 'drop-ins': [{ - 'name': '50-etcd-initial-cluster.conf', - 'content': _.template("[Service]\nEnvironment=ETCD_SERVERS=--etcd-servers=<%= nodes.join(',') %>\n")(data), - }], - }; -}; - -exports.create_etcd_cloud_config = function (node_count, conf) { - var input_file = './cloud_config_templates/kubernetes-cluster-etcd-node-template.yml'; - var output_file = util.join_output_file_path('kubernetes-cluster-etcd-nodes', 'generated.yml'); - - return cloud_config.process_template(input_file, output_file, function(data) { - data.coreos.units.push(etcd_initial_cluster_conf_self(conf)); - return data; - }); -}; - -exports.create_node_cloud_config = function (node_count, conf) { - var elected_node = 0; - - var input_file = './cloud_config_templates/kubernetes-cluster-main-nodes-template.yml'; - var output_file = util.join_output_file_path('kubernetes-cluster-main-nodes', 'generated.yml'); - - var make_node_config = function (n) { - return cloud_config.generate_environment_file_entry_from_object(util.hostname(n, 'kube'), { - weave_password: conf.weave_salt, - weave_peers: n === elected_node ? "" : util.hostname(elected_node, 'kube'), - breakout_route: util.ipv4([10, 2, 0, 0], 16), - bridge_address_cidr: util.ipv4([10, 2, n, 1], 24), - }); - }; - - var write_files_extra = cloud_config.write_files_from('addons', '/etc/kubernetes/addons'); - return cloud_config.process_template(input_file, output_file, function(data) { - data.write_files = data.write_files.concat(_(node_count).times(make_node_config), write_files_extra); - data.coreos.units.push(etcd_initial_cluster_conf_kube(conf)); - return data; - }); -}; diff --git a/docs/getting-started-guides/coreos/azure/lib/util.js b/docs/getting-started-guides/coreos/azure/lib/util.js deleted file mode 100644 index 2c88b8cff3..0000000000 --- a/docs/getting-started-guides/coreos/azure/lib/util.js +++ /dev/null @@ -1,33 +0,0 @@ -var _ = require('underscore'); -_.mixin(require('underscore.string').exports()); - -exports.ipv4 = function (ocets, prefix) { - return { - ocets: ocets, - prefix: prefix, - toString: function () { - return [ocets.join('.'), prefix].join('/'); - } - } -}; - -exports.hostname = function hostname (n, prefix) { - return _.template("<%= pre %>-<%= seq %>")({ - pre: prefix || 'core', - seq: _.pad(n, 2, '0'), - }); -}; - -exports.rand_string = function () { - var crypto = require('crypto'); - var shasum = crypto.createHash('sha256'); - shasum.update(crypto.randomBytes(256)); - return shasum.digest('hex'); -}; - - -exports.rand_suffix = exports.rand_string().substring(50); - -exports.join_output_file_path = function(prefix, suffix) { - return './output/' + [prefix, exports.rand_suffix, suffix].join('_'); -}; diff --git a/docs/getting-started-guides/coreos/azure/scale-kubernetes-cluster.js b/docs/getting-started-guides/coreos/azure/scale-kubernetes-cluster.js deleted file mode 100755 index f606898874..0000000000 --- a/docs/getting-started-guides/coreos/azure/scale-kubernetes-cluster.js +++ /dev/null @@ -1,10 +0,0 @@ -#!/usr/bin/env node - -var azure = require('./lib/azure_wrapper.js'); -var kube = require('./lib/deployment_logic/kubernetes.js'); - -azure.load_state_for_resizing(process.argv[2], 'kube', parseInt(process.argv[3] || 1)); - -azure.run_task_queue([ - azure.queue_machines('kube', 'stable', kube.create_node_cloud_config), -]); From 2b1f051de5c9d7e393b63281342483f38852cdfe Mon Sep 17 00:00:00 2001 From: Ilya Dmitrichenko Date: Mon, 4 Apr 2016 17:37:34 +0100 Subject: [PATCH 252/348] coreos/azure: Convert to UNIX EOL, `git clone` external repo (which provides v1.2 now) --- .../coreos/azure/index.md | 493 +++++++++--------- 1 file changed, 246 insertions(+), 247 deletions(-) diff --git a/docs/getting-started-guides/coreos/azure/index.md b/docs/getting-started-guides/coreos/azure/index.md index 7101ae7ee7..589cf81fcc 100644 --- a/docs/getting-started-guides/coreos/azure/index.md +++ b/docs/getting-started-guides/coreos/azure/index.md @@ -1,247 +1,246 @@ ---- ---- - -* TOC -{:toc} - - -In this guide I will demonstrate how to deploy a Kubernetes cluster to Azure cloud. You will be using CoreOS with Weave, which implements simple and secure networking, in a transparent, yet robust way. The purpose of this guide is to provide an out-of-the-box implementation that can ultimately be taken into production with little change. It will demonstrate how to provision a dedicated Kubernetes master and etcd nodes, and show how to scale the cluster with ease. - -### Prerequisites - -1. You need an Azure account. - -## Let's go! - -To get started, you need to checkout the code: - -```shell -git clone https://github.com/kubernetes/kubernetes -cd kubernetes/docs/getting-started-guides/coreos/azure/ -``` - -You will need to have [Node.js installed](http://nodejs.org/download/) on you machine. If you have previously used Azure CLI, you should have it already. - -First, you need to install some of the dependencies with - -```shell -npm install -``` - -Now, all you need to do is: - -```shell -./azure-login.js -u -./create-kubernetes-cluster.js -``` - -This script will provision a cluster suitable for production use, where there is a ring of 3 dedicated etcd nodes: 1 kubernetes master and 2 kubernetes nodes. The `kube-00` VM will be the master, your work loads are only to be deployed on the nodes, `kube-01` and `kube-02`. Initially, all VMs are single-core, to ensure a user of the free tier can reproduce it without paying extra. I will show how to add more bigger VMs later. -If you need to pass Azure specific options for the creation script you can do this via additional environment variables e.g. - -```shell -AZ_SUBSCRIPTION= AZ_LOCATION="East US" ./create-kubernetes-cluster.js -# or -AZ_VM_COREOS_CHANNEL=beta ./create-kubernetes-cluster.js -``` - -![VMs in Azure](/images/docs/initial_cluster.png) - -Once the creation of Azure VMs has finished, you should see the following: - -```shell -... -azure_wrapper/info: Saved SSH config, you can use it like so: `ssh -F ./output/kube_1c1496016083b4_ssh_conf ` -azure_wrapper/info: The hosts in this deployment are: - [ 'etcd-00', 'etcd-01', 'etcd-02', 'kube-00', 'kube-01', 'kube-02' ] -azure_wrapper/info: Saved state into `./output/kube_1c1496016083b4_deployment.yml` -``` - -Let's login to the master node like so: - -```shell -ssh -F ./output/kube_1c1496016083b4_ssh_conf kube-00 -``` - -> Note: config file name will be different, make sure to use the one you see. - -Check there are 2 nodes in the cluster: - -```shell -core@kube-00 ~ $ kubectl get nodes -NAME LABELS STATUS -kube-01 kubernetes.io/hostname=kube-01 Ready -kube-02 kubernetes.io/hostname=kube-02 Ready -``` - -## Deploying the workload - -Let's follow the Guestbook example now: - -```shell -kubectl create -f ~/guestbook-example -``` - -You need to wait for the pods to get deployed, run the following and wait for `STATUS` to change from `Pending` to `Running`. - -```shell -kubectl get pods --watch -``` - -> Note: the most time it will spend downloading Docker container images on each of the nodes. - -Eventually you should see: - -```shell -NAME READY STATUS RESTARTS AGE -frontend-0a9xi 1/1 Running 0 4m -frontend-4wahe 1/1 Running 0 4m -frontend-6l36j 1/1 Running 0 4m -redis-master-talmr 1/1 Running 0 4m -redis-slave-12zfd 1/1 Running 0 4m -redis-slave-3nbce 1/1 Running 0 4m -``` - -## Scaling - -Two single-core nodes are certainly not enough for a production system of today. Let's scale the cluster by adding a couple of bigger nodes. - -You will need to open another terminal window on your machine and go to the same working directory (e.g. `~/Workspace/kubernetes/docs/getting-started-guides/coreos/azure/`). - -First, lets set the size of new VMs: - -```shell -export AZ_VM_SIZE=Large -``` - -Now, run scale script with state file of the previous deployment and number of nodes to add: - -```shell -core@kube-00 ~ $ ./scale-kubernetes-cluster.js ./output/kube_1c1496016083b4_deployment.yml 2 -... -azure_wrapper/info: Saved SSH config, you can use it like so: `ssh -F ./output/kube_8f984af944f572_ssh_conf ` -azure_wrapper/info: The hosts in this deployment are: - [ 'etcd-00', - 'etcd-01', - 'etcd-02', - 'kube-00', - 'kube-01', - 'kube-02', - 'kube-03', - 'kube-04' ] -azure_wrapper/info: Saved state into `./output/kube_8f984af944f572_deployment.yml` -``` - -> Note: this step has created new files in `./output`. - -Back on `kube-00`: - -```shell -core@kube-00 ~ $ kubectl get nodes -NAME LABELS STATUS -kube-01 kubernetes.io/hostname=kube-01 Ready -kube-02 kubernetes.io/hostname=kube-02 Ready -kube-03 kubernetes.io/hostname=kube-03 Ready -kube-04 kubernetes.io/hostname=kube-04 Ready -``` - -You can see that two more nodes joined happily. Let's scale the number of Guestbook instances now. - -First, double-check how many replication controllers there are: - -```shell -core@kube-00 ~ $ kubectl get rc -ONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS -frontend php-redis kubernetes/example-guestbook-php-redis:v2 name=frontend 3 -redis-master master redis name=redis-master 1 -redis-slave worker kubernetes/redis-slave:v2 name=redis-slave 2 -``` - -As there are 4 nodes, let's scale proportionally: - -```shell -core@kube-00 ~ $ kubectl scale --replicas=4 rc redis-slave ->>>>>>> coreos/azure: Updates for 1.0 -scaled -core@kube-00 ~ $ kubectl scale --replicas=4 rc frontend -scaled -``` - -Check what you have now: - -```shell -core@kube-00 ~ $ kubectl get rc -CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS -frontend php-redis kubernetes/example-guestbook-php-redis:v2 name=frontend 4 -redis-master master redis name=redis-master 1 -redis-slave worker kubernetes/redis-slave:v2 name=redis-slave 4 -``` - -You now will have more instances of front-end Guestbook apps and Redis slaves; and, if you look up all pods labeled `name=frontend`, you should see one running on each node. - -```shell -core@kube-00 ~/guestbook-example $ kubectl get pods -l name=frontend -NAME READY STATUS RESTARTS AGE -frontend-0a9xi 1/1 Running 0 22m -frontend-4wahe 1/1 Running 0 22m -frontend-6l36j 1/1 Running 0 22m -frontend-z9oxo 1/1 Running 0 41s -``` - -## Exposing the app to the outside world - -There is no native Azure load-balancer support in Kubernetes 1.0, however here is how you can expose the Guestbook app to the Internet. - -```shell -./expose_guestbook_app_port.sh ./output/kube_1c1496016083b4_ssh_conf -Guestbook app is on port 31605, will map it to port 80 on kube-00 -info: Executing command vm endpoint create -+ Getting virtual machines -+ Reading network configuration -+ Updating network configuration -info: vm endpoint create command OK -info: Executing command vm endpoint show -+ Getting virtual machines -data: Name : tcp-80-31605 -data: Local port : 31605 -data: Protcol : tcp -data: Virtual IP Address : 137.117.156.164 -data: Direct server return : Disabled -info: vm endpoint show command OK -``` - -You then should be able to access it from anywhere via the Azure virtual IP for `kube-00` displayed above, i.e. `http://137.117.156.164/` in my case. - -## Next steps - -You now have a full-blow cluster running in Azure, congrats! - -You should probably try deploy other [example apps](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/) or write your own ;) - -## Tear down... - -If you don't wish care about the Azure bill, you can tear down the cluster. It's easy to redeploy it, as you can see. - -```shell -./destroy-cluster.js ./output/kube_8f984af944f572_deployment.yml -``` - -> Note: make sure to use the _latest state file_, as after scaling there is a new one. - -By the way, with the scripts shown, you can deploy multiple clusters, if you like :) - -## Support Level - - -IaaS Provider | Config. Mgmt | OS | Networking | Docs | Conforms | Support Level --------------------- | ------------ | ------ | ---------- | --------------------------------------------- | ---------| ---------------------------- -Azure | CoreOS | CoreOS | Weave | [docs](/docs/getting-started-guides/coreos/azure/) | | Community ([@errordeveloper](https://github.com/errordeveloper), [@squillace](https://github.com/squillace), [@chanezon](https://github.com/chanezon), [@crossorigin](https://github.com/crossorigin)) - - -For support level information on all solutions, see the [Table of solutions](/docs/getting-started-guides/#table-of-solutions) chart. - - -## Further reading - -Please see the [Kubernetes docs](/docs/) for more details on administering -and using a Kubernetes cluster - +--- +--- + +* TOC +{:toc} + + +In this guide I will demonstrate how to deploy a Kubernetes cluster to Azure cloud. You will be using CoreOS with Weave, which implements simple and secure networking, in a transparent, yet robust way. The purpose of this guide is to provide an out-of-the-box implementation that can ultimately be taken into production with little change. It will demonstrate how to provision a dedicated Kubernetes master and etcd nodes, and show how to scale the cluster with ease. + +### Prerequisites + +1. You need an Azure account. + +## Let's go! + +To get started, you need to checkout the code: + +```shell +https://github.com/weaveworks-guides/weave-kubernetes-coreos-azure +cd weave-kubernetes-coreos-azure +``` + +You will need to have [Node.js installed](http://nodejs.org/download/) on you machine. If you have previously used Azure CLI, you should have it already. + +First, you need to install some of the dependencies with + +```shell +npm install +``` + +Now, all you need to do is: + +```shell +./azure-login.js -u +./create-kubernetes-cluster.js +``` + +This script will provision a cluster suitable for production use, where there is a ring of 3 dedicated etcd nodes: 1 kubernetes master and 2 kubernetes nodes. The `kube-00` VM will be the master, your work loads are only to be deployed on the nodes, `kube-01` and `kube-02`. Initially, all VMs are single-core, to ensure a user of the free tier can reproduce it without paying extra. I will show how to add more bigger VMs later. +If you need to pass Azure specific options for the creation script you can do this via additional environment variables e.g. + +```shell +AZ_SUBSCRIPTION= AZ_LOCATION="East US" ./create-kubernetes-cluster.js +# or +AZ_VM_COREOS_CHANNEL=beta ./create-kubernetes-cluster.js +``` + +![VMs in Azure](/images/docs/initial_cluster.png) + +Once the creation of Azure VMs has finished, you should see the following: + +```shell +... +azure_wrapper/info: Saved SSH config, you can use it like so: `ssh -F ./output/kube_1c1496016083b4_ssh_conf ` +azure_wrapper/info: The hosts in this deployment are: + [ 'etcd-00', 'etcd-01', 'etcd-02', 'kube-00', 'kube-01', 'kube-02' ] +azure_wrapper/info: Saved state into `./output/kube_1c1496016083b4_deployment.yml` +``` + +Let's login to the master node like so: + +```shell +ssh -F ./output/kube_1c1496016083b4_ssh_conf kube-00 +``` + +> Note: config file name will be different, make sure to use the one you see. + +Check there are 2 nodes in the cluster: + +```shell +core@kube-00 ~ $ kubectl get nodes +NAME LABELS STATUS +kube-01 kubernetes.io/hostname=kube-01 Ready +kube-02 kubernetes.io/hostname=kube-02 Ready +``` + +## Deploying the workload + +Let's follow the Guestbook example now: + +```shell +kubectl create -f ~/guestbook-example +``` + +You need to wait for the pods to get deployed, run the following and wait for `STATUS` to change from `Pending` to `Running`. + +```shell +kubectl get pods --watch +``` + +> Note: the most time it will spend downloading Docker container images on each of the nodes. + +Eventually you should see: + +```shell +NAME READY STATUS RESTARTS AGE +frontend-0a9xi 1/1 Running 0 4m +frontend-4wahe 1/1 Running 0 4m +frontend-6l36j 1/1 Running 0 4m +redis-master-talmr 1/1 Running 0 4m +redis-slave-12zfd 1/1 Running 0 4m +redis-slave-3nbce 1/1 Running 0 4m +``` + +## Scaling + +Two single-core nodes are certainly not enough for a production system of today. Let's scale the cluster by adding a couple of bigger nodes. + +You will need to open another terminal window on your machine and go to the same working directory (e.g. `~/Workspace/kubernetes/docs/getting-started-guides/coreos/azure/`). + +First, lets set the size of new VMs: + +```shell +export AZ_VM_SIZE=Large +``` + +Now, run scale script with state file of the previous deployment and number of nodes to add: + +```shell +core@kube-00 ~ $ ./scale-kubernetes-cluster.js ./output/kube_1c1496016083b4_deployment.yml 2 +... +azure_wrapper/info: Saved SSH config, you can use it like so: `ssh -F ./output/kube_8f984af944f572_ssh_conf ` +azure_wrapper/info: The hosts in this deployment are: + [ 'etcd-00', + 'etcd-01', + 'etcd-02', + 'kube-00', + 'kube-01', + 'kube-02', + 'kube-03', + 'kube-04' ] +azure_wrapper/info: Saved state into `./output/kube_8f984af944f572_deployment.yml` +``` + +> Note: this step has created new files in `./output`. + +Back on `kube-00`: + +```shell +core@kube-00 ~ $ kubectl get nodes +NAME LABELS STATUS +kube-01 kubernetes.io/hostname=kube-01 Ready +kube-02 kubernetes.io/hostname=kube-02 Ready +kube-03 kubernetes.io/hostname=kube-03 Ready +kube-04 kubernetes.io/hostname=kube-04 Ready +``` + +You can see that two more nodes joined happily. Let's scale the number of Guestbook instances now. + +First, double-check how many replication controllers there are: + +```shell +core@kube-00 ~ $ kubectl get rc +ONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS +frontend php-redis kubernetes/example-guestbook-php-redis:v2 name=frontend 3 +redis-master master redis name=redis-master 1 +redis-slave worker kubernetes/redis-slave:v2 name=redis-slave 2 +``` + +As there are 4 nodes, let's scale proportionally: + +```shell +core@kube-00 ~ $ kubectl scale --replicas=4 rc redis-slave +scaled +core@kube-00 ~ $ kubectl scale --replicas=4 rc frontend +scaled +``` + +Check what you have now: + +```shell +core@kube-00 ~ $ kubectl get rc +CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS +frontend php-redis kubernetes/example-guestbook-php-redis:v2 name=frontend 4 +redis-master master redis name=redis-master 1 +redis-slave worker kubernetes/redis-slave:v2 name=redis-slave 4 +``` + +You now will have more instances of front-end Guestbook apps and Redis slaves; and, if you look up all pods labeled `name=frontend`, you should see one running on each node. + +```shell +core@kube-00 ~/guestbook-example $ kubectl get pods -l name=frontend +NAME READY STATUS RESTARTS AGE +frontend-0a9xi 1/1 Running 0 22m +frontend-4wahe 1/1 Running 0 22m +frontend-6l36j 1/1 Running 0 22m +frontend-z9oxo 1/1 Running 0 41s +``` + +## Exposing the app to the outside world + +There is no native Azure load-balancer support in Kubernetes 1.0, however here is how you can expose the Guestbook app to the Internet. + +```shell +./expose_guestbook_app_port.sh ./output/kube_1c1496016083b4_ssh_conf +Guestbook app is on port 31605, will map it to port 80 on kube-00 +info: Executing command vm endpoint create ++ Getting virtual machines ++ Reading network configuration ++ Updating network configuration +info: vm endpoint create command OK +info: Executing command vm endpoint show ++ Getting virtual machines +data: Name : tcp-80-31605 +data: Local port : 31605 +data: Protcol : tcp +data: Virtual IP Address : 137.117.156.164 +data: Direct server return : Disabled +info: vm endpoint show command OK +``` + +You then should be able to access it from anywhere via the Azure virtual IP for `kube-00` displayed above, i.e. `http://137.117.156.164/` in my case. + +## Next steps + +You now have a full-blown cluster running in Azure, congrats! + +You should probably try deploy other [example apps](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/) or write your own ;) + +## Tear down... + +If you don't wish care about the Azure bill, you can tear down the cluster. It's easy to redeploy it, as you can see. + +```shell +./destroy-cluster.js ./output/kube_8f984af944f572_deployment.yml +``` + +> Note: make sure to use the _latest state file_, as after scaling there is a new one. + +By the way, with the scripts shown, you can deploy multiple clusters, if you like :) + +## Support Level + + +IaaS Provider | Config. Mgmt | OS | Networking | Docs | Conforms | Support Level +-------------------- | ------------ | ------ | ---------- | --------------------------------------------- | ---------| ---------------------------- +Azure | CoreOS | CoreOS | Weave | [docs](/docs/getting-started-guides/coreos/azure/) | | Community ([@errordeveloper](https://github.com/errordeveloper), [@squillace](https://github.com/squillace), [@chanezon](https://github.com/chanezon), [@crossorigin](https://github.com/crossorigin)) + + +For support level information on all solutions, see the [Table of solutions](/docs/getting-started-guides/#table-of-solutions) chart. + + +## Further reading + +Please see the [Kubernetes docs](/docs/) for more details on administering +and using a Kubernetes cluster + From b7bc56e1fe2c4a9a4dac11563e7477b66a8a4db4 Mon Sep 17 00:00:00 2001 From: Kelsey Hightower Date: Fri, 29 Jul 2016 08:10:34 -0700 Subject: [PATCH 253/348] use the expose command to expose the nginx service --- docs/user-guide/quick-start.md | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/docs/user-guide/quick-start.md b/docs/user-guide/quick-start.md index 00dc999d53..423e701de7 100644 --- a/docs/user-guide/quick-start.md +++ b/docs/user-guide/quick-start.md @@ -11,14 +11,20 @@ This guide will help you get oriented to Kubernetes and running your first conta Once your application is packaged into a container and pushed to an image registry, you're ready to deploy it to Kubernetes. Through integration with some cloud providers (for example Google Compute Engine and AWS EC2), Kubernetes also enables you to request it to provision a public IP address for your application. -For example, [nginx](http://wiki.nginx.org/Main) is a popular HTTP server, with a [pre-built container on Docker hub](https://registry.hub.docker.com/_/nginx/). The [`kubectl run`](/docs/user-guide/kubectl/kubectl_run) command below will create two nginx replicas, listening on port 80, and a public IP address for your application. +For example, [nginx](http://wiki.nginx.org/Main) is a popular HTTP server, with a [pre-built container on Docker hub](https://registry.hub.docker.com/_/nginx/). The [`kubectl run`](/docs/user-guide/kubectl/kubectl_run) commands below will create two nginx replicas, listening on port 80, and a public IP address for your application. ```shell -$ kubectl run my-nginx --image=nginx --replicas=2 --port=80 --expose --service-overrides='{ "spec": { "type": "LoadBalancer" } }' -service "my-nginx" created +$ kubectl run my-nginx --image=nginx --replicas=2 --port=80 deployment "my-nginx" created ``` +To expose your service to the public internet, run: + +```shell +$ kubectl expose rc my-nginx --target-port=80 --type=LoadBalancer +service "my-nginx" exposed +``` + You can see that they are running by: ```shell @@ -33,7 +39,7 @@ Kubernetes will ensure that your application keeps running, by automatically res To find the public IP address assigned to your application, execute: ```shell -$ kubectl get service/my-nginx +$ kubectl get service my-nginx NAME CLUSTER_IP EXTERNAL_IP PORT(S) AGE my-nginx 10.179.240.1 25.1.2.3 80/TCP 8s ``` From 32aeae5684ec94df2282af57fa642bb6594a63aa Mon Sep 17 00:00:00 2001 From: Rudi Chiarito Date: Thu, 17 Mar 2016 12:04:04 -0400 Subject: [PATCH 254/348] Document revisionHistoryLimit in Deployment spec I'm not sure the field should be optional, but that ship has already sailed. It should definitely be a lot more prominent than hidden in the API docs, though. --- docs/user-guide/deployments.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/deployments.md b/docs/user-guide/deployments.md index a76dbff5e7..38a417e198 100644 --- a/docs/user-guide/deployments.md +++ b/docs/user-guide/deployments.md @@ -558,7 +558,7 @@ To learn more about when a Pod is considered ready, see [Container Probes](/docs ### Revision History Limit -`.spec.revisionHistoryLimit` is an optional field that specifies the number of old Replica Sets to retain to allow rollback. All old Replica Sets will be kept by default, if this field is not set. The configuration of each Deployment revision is stored in its Replica Sets; therefore, once an old Replica Set is deleted, you lose the ability to rollback to that revision of Deployment. +`.spec.revisionHistoryLimit` is an optional field that specifies the number of old Replica Sets to retain to allow rollback. Its ideal value depends on the frequency and stability of new deployments. All old Replica Sets will be kept by default, consuming resources in `etcd` and crowding the output of `kubectl get rs`, if this field is not set. The configuration of each Deployment revision is stored in its Replica Sets; therefore, once an old Replica Set is deleted, you lose the ability to rollback to that revision of Deployment. More specifically, setting this field to zero means that a new deployment revision can be undone only while still in progress. ### Paused From fed59d5d50c208a70c7b277e996d4fb766e906e3 Mon Sep 17 00:00:00 2001 From: Janet Kuo Date: Fri, 29 Jul 2016 11:12:36 -0700 Subject: [PATCH 255/348] Further update docs about revisionHistoryLimit --- docs/user-guide/deployments.md | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/docs/user-guide/deployments.md b/docs/user-guide/deployments.md index 38a417e198..3f18a56354 100644 --- a/docs/user-guide/deployments.md +++ b/docs/user-guide/deployments.md @@ -384,6 +384,11 @@ Events: 29m 2m 2 {deployment-controller } Normal ScalingReplicaSet Scaled up replica set nginx-deployment-1564180365 to 3 ``` +### Clean up Policy + +You can set `.spec.revisionHistoryLimit` field to specify how much revision history of this deployment you want to keep. By default, +all revision history will be kept; explicitly setting this field to `0` disallows a deployment being rolled back. + ## Pausing and Resuming a Deployment You can also pause a Deployment mid-way and then resume it. A use case is to support canary deployment. @@ -558,7 +563,12 @@ To learn more about when a Pod is considered ready, see [Container Probes](/docs ### Revision History Limit -`.spec.revisionHistoryLimit` is an optional field that specifies the number of old Replica Sets to retain to allow rollback. Its ideal value depends on the frequency and stability of new deployments. All old Replica Sets will be kept by default, consuming resources in `etcd` and crowding the output of `kubectl get rs`, if this field is not set. The configuration of each Deployment revision is stored in its Replica Sets; therefore, once an old Replica Set is deleted, you lose the ability to rollback to that revision of Deployment. More specifically, setting this field to zero means that a new deployment revision can be undone only while still in progress. +A deployment's revision history is stored in the replica sets it controls. + +`.spec.revisionHistoryLimit` is an optional field that specifies the number of old Replica Sets to retain to allow rollback. Its ideal value depends on the frequency and stability of new deployments. All old Replica Sets will be kept by default, consuming resources in `etcd` and crowding the output of `kubectl get rs`, if this field is not set. The configuration of each Deployment revision is stored in its Replica Sets; therefore, once an old Replica Set is deleted, you lose the ability to rollback to that revision of Deployment. + +More specifically, setting this field to zero means that all old replica sets with 0 replica will be cleaned up. +In this case, a new deployment rollout cannot be undone, since its revision history is cleaned up. ### Paused From 87e8f37dfffdda1772c26b9fd946e430a5217173 Mon Sep 17 00:00:00 2001 From: Ace Nassri Date: Fri, 29 Jul 2016 12:19:38 -0700 Subject: [PATCH 256/348] Convert PROJECT_ID to a variable in shell snippets --- docs/hellonode.md | 38 +++++++++++++++++++++----------------- 1 file changed, 21 insertions(+), 17 deletions(-) diff --git a/docs/hellonode.md b/docs/hellonode.md index 78acac80d8..146e6452f7 100755 --- a/docs/hellonode.md +++ b/docs/hellonode.md @@ -22,7 +22,11 @@ If you don't already have a Google Account (Gmail or Google Apps), you must [cre ![image](/images/hellonode/image_3.png) -Remember the project ID; it will be referred to later in this codelab as `PROJECT_ID`. +Remember the project ID; it will be referred to later in this codelab as `$PROJECT_ID`. It may be helpful to store your project ID into a variable: + + ```shell + export $PROJECT_ID="your-project-id" + ``` Next, [enable billing](https://console.developers.google.com/billing) in the Developers Console in order to use Google Cloud resources and [enable the Container Engine API](https://console.developers.google.com/project/_/kubernetes/list). @@ -83,17 +87,17 @@ CMD node server.js This "recipe" for the Docker image will start from the official Node.js LTS image found on the Docker registry, expose port 8080, copy our `server.js` file to the image and start the Node server. -Now build an image of your container by running `docker build`, tagging the image with the Google Container Registry repo for your `PROJECT_ID`: +Now build an image of your container by running `docker build`, tagging the image with the Google Container Registry repo for your `$PROJECT_ID`: ```shell -docker build -t gcr.io/PROJECT_ID/hello-node:v1 . +docker build -t gcr.io/$PROJECT_ID/hello-node:v1 . ``` Now there is a trusted source for getting an image of your containerized app. Let's try your image out with Docker: ```shell -$ docker run -d -p 8080:8080 gcr.io/PROJECT_ID/hello-node:v1 +$ docker run -d -p 8080:8080 gcr.io/$PROJECT_ID/hello-node:v1 325301e6b2bffd1d0049c621866831316d653c0b25a496d04ce0ec6854cb7998 ``` @@ -115,16 +119,16 @@ Let’s now stop the container. In this example, our app was running as Docker p ```shell docker ps CONTAINER ID IMAGE COMMAND -2c66d0efcbd4 gcr.io/PROJECT_ID/hello-node:v1 "/bin/sh -c 'node +2c66d0efcbd4 gcr.io/$PROJECT_ID/hello-node:v1 "/bin/sh -c 'node docker stop 2c66d0efcbd4 2c66d0efcbd4 ``` -Now that the image works as intended and is all tagged with your `PROJECT_ID`, we can push it to the [Google Container Registry](https://cloud.google.com/tools/container-registry/), a private repository for your Docker images accessible from every Google Cloud project (but also from outside Google Cloud Platform) : +Now that the image works as intended and is all tagged with your `$PROJECT_ID`, we can push it to the [Google Container Registry](https://cloud.google.com/tools/container-registry/), a private repository for your Docker images accessible from every Google Cloud project (but also from outside Google Cloud Platform) : ```shell -gcloud docker push gcr.io/PROJECT_ID/hello-node:v1 +gcloud docker push gcr.io/$PROJECT_ID/hello-node:v1 ``` If all goes well, you should be able to see the container image listed in the console: *Compute > Container Engine > Container Registry*. We now have a project-wide Docker image available which Kubernetes can access and orchestrate. @@ -154,7 +158,7 @@ A Kubernetes **[pod](/docs/user-guide/pods/)** is a group of containers, tied to Create a pod with the `kubectl run` command: ```shell -$ kubectl run hello-node --image=gcr.io/PROJECT_ID/hello-node:v1 --port=8080 +$ kubectl run hello-node --image=gcr.io/$PROJECT_ID/hello-node:v1 --port=8080 deployment "hello-node" created ``` @@ -287,18 +291,18 @@ First, let’s modify the application. On the development machine, edit server.j We can now build and publish a new container image to the registry with an incremented tag: ```shell -docker build -t gcr.io/PROJECT_ID/hello-node:v2 . -gcloud docker push gcr.io/PROJECT_ID/hello-node:v2 +docker build -t gcr.io/$PROJECT_ID/hello-node:v2 . +gcloud docker push gcr.io/$PROJECT_ID/hello-node:v2 ``` Building and pushing this updated image should be much quicker as we take full advantage of the Docker cache. We’re now ready for Kubernetes to smoothly update our deployment to the new version of the application. In order to change the image label for our running container, we will need to edit the existing *hello-node deployment* and change the image from -`gcr.io/PROJECT_ID/hello-node:v1` to `gcr.io/PROJECT_ID/hello-node:v2`. To do this, we will use the `kubectl set image` command. +`gcr.io/$PROJECT_ID/hello-node:v1` to `gcr.io/$PROJECT_ID/hello-node:v2`. To do this, we will use the `kubectl set image` command. ```shell -$ kubectl set image deployment/hello-node hello-node=gcr.io/PROJECT_ID/hello-node:v2 +$ kubectl set image deployment/hello-node hello-node=gcr.io/$PROJECT_ID/hello-node:v2 deployment "hello-node" image updated ``` @@ -345,7 +349,7 @@ The following clusters will be deleted. Do you want to continue (Y/n)? Deleting cluster hello-world...done. -Deleted [https://container.googleapis.com/v1/projects//zones/us-central1-f/clusters/hello-world]. +Deleted [https://container.googleapis.com/v1/projects/<$PROJECT_ID>/zones/us-central1-f/clusters/hello-world]. ``` This deletes the Google Compute Engine instances that are running the cluster. @@ -354,9 +358,9 @@ Finally delete the Docker registry storage bucket hosting your image(s) : ```shell $ gsutil ls -gs://artifacts..appspot.com/ -$ gsutil rm -r gs://artifacts..appspot.com/ -Removing gs://artifacts..appspot.com/... +gs://artifacts.<$PROJECT_ID>.appspot.com/ +$ gsutil rm -r gs://artifacts.<$PROJECT_ID>.appspot.com/ +Removing gs://artifacts.<$PROJECT_ID>.appspot.com/... ``` -Of course, you can also delete the entire project but note that you must first disable billing on the project. Additionally, deleting a project will only happen after the current billing cycle ends. +Of course, you can also delete the entire project but note that you must first disable billing on the project. Additionally, deleting a project will only happen after the current billing cycle ends. \ No newline at end of file From 1c3e499e79dcc2ca6c665dc2bc19a9d1bb17abff Mon Sep 17 00:00:00 2001 From: Eric Tune Date: Wed, 30 Mar 2016 09:41:26 -0700 Subject: [PATCH 257/348] Update images.md --- docs/user-guide/images.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/docs/user-guide/images.md b/docs/user-guide/images.md index 49f80a6500..26d65a7ab1 100644 --- a/docs/user-guide/images.md +++ b/docs/user-guide/images.md @@ -81,6 +81,21 @@ The kubelet will fetch and periodically refresh ECR credentials. It needs the f - `ecr:ListImages` - `ecr:BatchGetImage` +Requirements: + +- You must be using kubelet version `v1.2.0` or newer. (e.g. run `/usr/bin/kubelet --version=true`). +- Your nodes must be in the same region as the registry you are using +- ECR must be offered in your region + +Troubleshooting: + +- Verify all requirements above. +- Get $REGION (e.g. `us-west-2`) credentials on your workstation. SSH into the host and run Docker manually with those creds. Does it work? +- Verify kubelet is running with `--cloud-provider=aws`. +- Check kubelet logs (e.g. `journalctl -t kubelet`) for log lines like: + - `plugins.go:56] Registering credential provider: aws-ecr-key` + - `provider.go:91] Refreshing cache for provider: *aws_credentials.ecrProvider` + ### Configuring Nodes to Authenticate to a Private Repository **Note:** if you are running on Google Container Engine (GKE), there will already be a `.dockercfg` on each node From 674f52b54149e6066dd603ec2ba6fb179c62ebb5 Mon Sep 17 00:00:00 2001 From: Naveen Date: Sun, 31 Jul 2016 09:07:51 -0400 Subject: [PATCH 258/348] Fixed the command line Fixed the command line which was causing an error. --- docs/user-guide/configmap/index.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/user-guide/configmap/index.md b/docs/user-guide/configmap/index.md index ccd3b74ebb..110dfd7c97 100644 --- a/docs/user-guide/configmap/index.md +++ b/docs/user-guide/configmap/index.md @@ -1,3 +1,4 @@ + --- --- Many applications require configuration via some combination of config files, command line @@ -362,7 +363,7 @@ spec: containers: - name: test-container image: gcr.io/google_containers/busybox - command: ["cat", "/etc/config/special.how" ] + command: ["/bin/sh","-c", "cat", "/etc/config/special.how" ] volumeMounts: - name: config-volume mountPath: /etc/config @@ -390,7 +391,7 @@ spec: containers: - name: test-container image: gcr.io/google_containers/busybox - command: [ "/bin/sh", "cat", "/etc/config/path/to/special-key" ] + command: [ "/bin/sh", "-c", "cat", "/etc/config/path/to/special-key" ] volumeMounts: - name: config-volume mountPath: /etc/config From 55c944b83b74f8496600da90886089d1ae0e3649 Mon Sep 17 00:00:00 2001 From: Naveen Date: Sun, 31 Jul 2016 09:13:03 -0400 Subject: [PATCH 259/348] Fixed the conflicts --- docs/user-guide/configmap/index.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/docs/user-guide/configmap/index.md b/docs/user-guide/configmap/index.md index 110dfd7c97..59374851ac 100644 --- a/docs/user-guide/configmap/index.md +++ b/docs/user-guide/configmap/index.md @@ -1,4 +1,3 @@ - --- --- Many applications require configuration via some combination of config files, command line @@ -363,7 +362,7 @@ spec: containers: - name: test-container image: gcr.io/google_containers/busybox - command: ["/bin/sh","-c", "cat", "/etc/config/special.how" ] + command: [ "/bin/sh", "-c" , "cat /etc/config/special.how" ] volumeMounts: - name: config-volume mountPath: /etc/config @@ -391,7 +390,7 @@ spec: containers: - name: test-container image: gcr.io/google_containers/busybox - command: [ "/bin/sh", "-c", "cat", "/etc/config/path/to/special-key" ] + command: [ "/bin/sh", "-c" , "cat /etc/config/path/to/special-key" ] volumeMounts: - name: config-volume mountPath: /etc/config From d037bdd4bf1eaef6da3ac99e570b728fb736d400 Mon Sep 17 00:00:00 2001 From: Scott Fleckenstein Date: Mon, 1 Aug 2016 20:09:17 -0700 Subject: [PATCH 260/348] Update vsphere tutorial to reflect non-free requirement The guide updated in this commit does not work for the free version of ESXi due to the fact that any write operation fails with the error message "Current license or ESXi version prohibits execution of the requested operation." See http://www.doublecloud.org/2011/01/free-esxi-and-apicli-support/ for discussion on the limitations of vSphere's API. --- docs/getting-started-guides/vsphere.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting-started-guides/vsphere.md b/docs/getting-started-guides/vsphere.md index 4816cf5599..0e98968bda 100644 --- a/docs/getting-started-guides/vsphere.md +++ b/docs/getting-started-guides/vsphere.md @@ -11,7 +11,7 @@ convenient). ### Prerequisites -1. You need administrator credentials to an ESXi machine or vCenter instance. +1. You need administrator credentials to an ESXi machine or vCenter instance with write mode api access enabled (not available on the free ESXi license). 2. You must have Go (see [here](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/devel/development.md#go-versions) for supported versions) installed: [www.golang.org](http://www.golang.org). 3. You must have your `GOPATH` set up and include `$GOPATH/bin` in your `PATH`. From bca79cfdcbb121a6ca543b9cd1ca704d5ee87a11 Mon Sep 17 00:00:00 2001 From: Tolleiv Nietsch Date: Tue, 2 Aug 2016 16:44:04 +0200 Subject: [PATCH 261/348] Undo the changes within ubuntu.md --- docs/getting-started-guides/ubuntu.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/getting-started-guides/ubuntu.md b/docs/getting-started-guides/ubuntu.md index ac1abbf233..c47a15dc9d 100644 --- a/docs/getting-started-guides/ubuntu.md +++ b/docs/getting-started-guides/ubuntu.md @@ -80,7 +80,7 @@ First configure the cluster information in cluster/ubuntu/config-default.sh, fol ```shell export nodes="vcap@10.10.103.250 vcap@10.10.103.162 vcap@10.10.103.223" -export roles="ai i i" +export role="ai i i" export NUM_NODES=${NUM_NODES:-3} @@ -92,7 +92,7 @@ export FLANNEL_NET=172.16.0.0/16 The first variable `nodes` defines all your cluster nodes, master node comes first and separated with blank space like ` ` -Then the `roles` variable defines the role of above machine in the same order, "ai" stands for machine +Then the `role` variable defines the role of above machine in the same order, "ai" stands for machine acts as both master and node, "a" stands for master, "i" stands for node. The `NUM_NODES` variable defines the total number of nodes. From e34b834815862cc4b023f32632708a9f7a620873 Mon Sep 17 00:00:00 2001 From: markturansky Date: Tue, 2 Aug 2016 13:16:48 -0400 Subject: [PATCH 262/348] added named port to example --- .../liveness/http-liveness-named-port.yaml | 25 +++++++++++++++++++ docs/user-guide/liveness/index.md | 7 ++++++ 2 files changed, 32 insertions(+) create mode 100644 docs/user-guide/liveness/http-liveness-named-port.yaml diff --git a/docs/user-guide/liveness/http-liveness-named-port.yaml b/docs/user-guide/liveness/http-liveness-named-port.yaml new file mode 100644 index 0000000000..80b17e06db --- /dev/null +++ b/docs/user-guide/liveness/http-liveness-named-port.yaml @@ -0,0 +1,25 @@ +apiVersion: v1 +kind: Pod +metadata: + labels: + test: liveness + name: liveness-http +spec: + containers: + - args: + - /server + image: gcr.io/google_containers/liveness + ports: + - name: liveness-port + containerPort: 8080 + hostPort: 8080 + livenessProbe: + httpGet: + path: /healthz + port: liveness-port + httpHeaders: + - name: X-Custom-Header + value: Awesome + initialDelaySeconds: 15 + timeoutSeconds: 1 + name: liveness diff --git a/docs/user-guide/liveness/index.md b/docs/user-guide/liveness/index.md index 4ef1641764..1b580be234 100644 --- a/docs/user-guide/liveness/index.md +++ b/docs/user-guide/liveness/index.md @@ -24,6 +24,13 @@ The [http-liveness.yaml](/docs/user-guide/liveness/http-liveness.yaml) demonstra The Kubelet sends an HTTP request to the specified path and port to perform the health check. If you take a look at image/server.go, you will see the server starts to respond with an error code 500 after 10 seconds, so the check fails. The Kubelet sends probes to the container's IP address, unless overridden by the optional `host` field in httpGet. If the container listens on `127.0.0.1` and `hostNetwork` is `true` (i.e., it does not use the pod-specific network), then `host` should be specified as `127.0.0.1`. Be warned that, outside of less common cases like that, `host` does probably not result in what you would expect. If you set it to a non-existing hostname (or your competitor's!), probes will never reach the pod, defeating the whole point of health checks. If your pod relies on e.g. virtual hosts, which is probably the more common case, you should not use `host`, but rather set the `Host` header in `httpHeaders`. +### Using a named port for liveness probes + +You can also use a named `ContainerPort` for HTTP liveness checks. + +The [http-liveness-named-port.yaml](/docs/user-guide/liveness/http-liveness-named-port.yaml) demonstrates the named-port HTTP check. +{% include code.html language="yaml" file="http-liveness-named-port.yaml" ghlink="/docs/user-guide/liveness/http-liveness-named-port.yaml" %} + This [guide](/docs/user-guide/walkthrough/k8s201/#health-checking) has more information on health checks. ## Get your hands dirty From 369a3f6e722e133a144e3942bab2874e3e658b52 Mon Sep 17 00:00:00 2001 From: Alex Robinson Date: Tue, 2 Aug 2016 18:38:27 +0000 Subject: [PATCH 263/348] Add ToC and fix markdown formatting in cluster management doc. --- docs/admin/cluster-management.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/admin/cluster-management.md b/docs/admin/cluster-management.md index 051c4b44c9..f133f06983 100644 --- a/docs/admin/cluster-management.md +++ b/docs/admin/cluster-management.md @@ -1,6 +1,9 @@ --- --- +* TOC +{:toc} + This document describes several topics related to the lifecycle of a cluster: creating a new cluster, upgrading your cluster's master and worker nodes, performing node maintenance (e.g. kernel upgrades), and upgrading the Kubernetes API version of a @@ -102,6 +105,7 @@ On GKE you configure cluster autoscaler either on cluster creation or update or to the corresponding `gcloud` commands. Examples: + ```shell gcloud container clusters create mytestcluster --zone=us-central1-b --enable-autoscaling=true --min-nodes=3 --max-nodes=10 --num-nodes=5 ``` From 1a8e00108aca8cad77b2a2364fe1f4ac49f5b679 Mon Sep 17 00:00:00 2001 From: Phillip Wittrock Date: Tue, 2 Aug 2016 13:59:07 -0700 Subject: [PATCH 264/348] fix issue with calico --- docs/getting-started-guides/ubuntu-calico.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/getting-started-guides/ubuntu-calico.md b/docs/getting-started-guides/ubuntu-calico.md index 86f0cb7e85..cffa8ffd2e 100644 --- a/docs/getting-started-guides/ubuntu-calico.md +++ b/docs/getting-started-guides/ubuntu-calico.md @@ -436,7 +436,9 @@ kubectl create -f https://raw.githubusercontent.com/projectcalico/calico-cni/k8s Note: The Kubernetes UI addon is deprecated and has been replaced with Kubernetes dashboard. You can install it by running: -```kubectl create -f https://rawgit.com/kubernetes/dashboard/master/src/deploy/kubernetes-dashboard.yaml``` +```shell +kubectl create -f https://rawgit.com/kubernetes/dashboard/master/src/deploy/kubernetes-dashboard.yaml +``` You can find the docs at [Kubernetes Dashboard](https://github.com/kubernetes/dashboard) From ce198ad723f9f10daf52ff897e6acff567ccf3d3 Mon Sep 17 00:00:00 2001 From: Jimmy Cuadra Date: Tue, 2 Aug 2016 16:37:38 -0700 Subject: [PATCH 265/348] Authz: Explain how to determine request verbs. --- docs/admin/authorization.md | 64 ++++++++++++++++++++++++++----------- 1 file changed, 46 insertions(+), 18 deletions(-) diff --git a/docs/admin/authorization.md b/docs/admin/authorization.md index 312bbb1d05..7f19b61995 100644 --- a/docs/admin/authorization.md +++ b/docs/admin/authorization.md @@ -18,7 +18,7 @@ The following implementations are available, and are selected by flag: need authorization. - `--authorization-mode=ABAC` allows for a simple local-file-based user-configured authorization policy. ABAC stands for Attribute-Based Access Control. -authorization policy. +authorization policy. - `--authorization-mode=RBAC` is an experimental implementation which allows for authorization to be driven by the Kubernetes API. RBAC stands for Roles-Based Access Control. @@ -43,17 +43,25 @@ A request has the following attributes that can be considered for authorization: - group (the list of group names the authenticated user is a member of). - whether the request is for an API resource. - the request path. - - allows authorizing access to miscellaneous endpoints like `/api` or -`/healthz` (see [kubectl](#kubectl)). + - allows authorizing access to miscellaneous non-resource endpoints like `/api` or `/healthz` (see [kubectl](#kubectl)). - the request verb. - - API verbs like `get`, `list`, `create`, `update`, `watch`, `delete`, and -`deletecollection` are used for API requests - - HTTP verbs like `get`, `post`, `put`, and `delete` are used for non-API + - API verbs `get`, `list`, `create`, `update`, `watch`, `delete`, and `deletecollection` are used for resource requests + - HTTP verbs `get`, `post`, `put`, and `delete` are used for non-resource requests - - what resource is being accessed (for API requests only) - - the namespace of the object being accessed (for namespaced API requests + - what resource is being accessed (for resource requests only) + - the namespace of the object being accessed (for namespaced resource requests only) - - the API group being accessed (for API requests only) + - the API group being accessed (for resource requests only) + +The request verb for a resource API endpoint can be determined by the HTTP verb used and whether or not the request acts on an individual resource or a collection of resources: + +HTTP verb | request verb +----------|--------------- +POST | create +GET, HEAD | get (for individual resources), list (for collections) +PUT | update +PATCH | patch +DELETE | delete (for individual resources), deletecollection (for collections) We anticipate adding more attributes to allow finer grained access control and to assist in policy management. @@ -97,17 +105,17 @@ A request has attributes which correspond to the properties of a policy object. When a request is received, the attributes are determined. Unknown attributes are set to the zero value of its type (e.g. empty string, 0, false). -A property set to "*" will match any value of the corresponding attribute. +A property set to `"*"` will match any value of the corresponding attribute. The tuple of attributes is checked for a match against every policy in the policy file. If at least one line matches the request attributes, then the request is authorized (but may fail later validation). To permit any user to do something, write a policy with the user property set to -"*". +`"*"`. To permit a user to do anything, write a policy with the apiGroup, namespace, -resource, and nonResourcePath properties set to "*". +resource, and nonResourcePath properties set to `"*"`. ### Kubectl @@ -130,11 +138,31 @@ up the verbosity: ### Examples - 1. Alice can do anything to all resources: `{"apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": {"user": "alice", "namespace": "*", "resource": "*", "apiGroup": "*"}}` - 2. Kubelet can read any pods: `{"apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": {"user": "kubelet", "namespace": "*", "resource": "pods", "readonly": true}}` - 3. Kubelet can read and write events: `{"apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": {"user": "kubelet", "namespace": "*", "resource": "events"}}` - 4. Bob can just read pods in namespace "projectCaribou": `{"apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": {"user": "bob", "namespace": "projectCaribou", "resource": "pods", "readonly": true}}` - 5. Anyone can make read-only requests to all non-API paths: `{"apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": {"user": "*", "readonly": true, "nonResourcePath": "*"}}` + 1. Alice can do anything to all resources: + + ```json + {"apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": {"user": "alice", "namespace": "*", "resource": "*", "apiGroup": "*"}} + ``` + 2. Kubelet can read any pods: + + ```json + {"apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": {"user": "kubelet", "namespace": "*", "resource": "pods", "readonly": true}} + ``` + 3. Kubelet can read and write events: + + ```json + {"apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": {"user": "kubelet", "namespace": "*", "resource": "events"}} + ``` + 4. Bob can just read pods in namespace "projectCaribou": + + ```json + {"apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": {"user": "bob", "namespace": "projectCaribou", "resource": "pods", "readonly": true}} + ``` + 5. Anyone can make read-only requests to all non-resource paths: + + ```json + {"apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": {"user": "*", "readonly": true, "nonResourcePath": "*"}} + ``` [Complete file example](http://releases.k8s.io/{{page.githubbranch}}/pkg/auth/authorizer/abac/example_policy_file.jsonl) @@ -147,7 +175,7 @@ according to the naming convention: system:serviceaccount:: ``` Creating a new namespace also causes a new service account to be created, of -this form:* +this form: ```shell system:serviceaccount::default From 9232f7f2fa0c53ce388b178889515027bf4c480b Mon Sep 17 00:00:00 2001 From: Vishnu Kannan Date: Wed, 30 Mar 2016 17:01:19 -0700 Subject: [PATCH 266/348] Fix curl command. Make cleanup easier. Signed-off-by: Vishnu Kannan --- docs/getting-started-guides/docker.md | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/docs/getting-started-guides/docker.md b/docs/getting-started-guides/docker.md index 828f162bfb..a6893f27e7 100644 --- a/docs/getting-started-guides/docker.md +++ b/docs/getting-started-guides/docker.md @@ -65,6 +65,7 @@ Here's a diagram of what the final result will look like: --net=host \ --pid=host \ --privileged \ + --name=kubelet \ gcr.io/google_containers/hyperkube-${ARCH}:${K8S_VERSION} \ /hyperkube kubelet \ --hostname-override=127.0.0.1 \ @@ -172,7 +173,11 @@ ip=$(kubectl get svc nginx --template={{.spec.clusterIP}}) echo $ip {% endraw %}``` -On Linux the IP is directly accessible via a web browser. +Hit the webserver with this IP: + +```shell{% raw %} +curl $ip +{% endraw %}``` On OS X, since docker is running inside a VM, run the following command instead: @@ -196,10 +201,10 @@ kubectl delete service,deployments nginx 2\. Delete all the containers including the kubelet: -Many of these containers run under the management of the `kubelet` binary, which attempts to keep containers running, even if they fail. -So, in order to turn down the cluster, you need to first kill the kubelet container, and then any other containers. - -You may use `docker rm -f $(docker ps -aq)`, note this removes _all_ containers running under Docker, so use with caution. +```shell +docker rm -f kubelet +docker rm -f `docker ps | grep k8s | awk '{print $1}'` +``` 3\. Cleanup the filesystem: From 66f28bb8201bafc68b87ed8752e5e9609dc72844 Mon Sep 17 00:00:00 2001 From: foxish Date: Fri, 29 Jul 2016 10:36:25 -0700 Subject: [PATCH 267/348] Adding OWNERS for docs. --- OWNERS | 7 + docs/admin/OWNERS | 4 + docs/admin/accessing-the-api.md | 5 + docs/admin/admission-controllers.md | 8 + docs/admin/authentication.md | 5 + docs/admin/authorization.md | 4 + docs/admin/cluster-components.md | 3 + docs/admin/cluster-large.md | 4 + docs/admin/cluster-management.md | 4 + docs/admin/cluster-troubleshooting.md | 3 + docs/admin/daemons.md | 3 + docs/admin/dns.md | 5 + docs/admin/etcd.md | 3 + docs/admin/federation/OWNERS | 4 + docs/admin/federation/index.md | 4 + docs/admin/garbage-collection.md | 4 + docs/admin/high-availability/OWNERS | 4 + docs/admin/high-availability/index.md | 6 +- docs/admin/index.md | 4 + docs/admin/kube-apiserver.md | 4 + docs/admin/kube-controller-manager.md | 4 + docs/admin/kube-proxy.md | 3 + docs/admin/kube-scheduler.md | 4 + docs/admin/kubelet.md | 5 + docs/admin/limitrange/OWNERS | 4 + docs/admin/limitrange/index.md | 180 ++++++------- docs/admin/master-node-communication.md | 5 + docs/admin/multi-cluster.md | 3 + docs/admin/multiple-schedulers.md | 4 + docs/admin/multiple-schedulers/OWNERS | 4 + docs/admin/multiple-zones.md | 5 + docs/admin/namespaces/OWNERS | 4 + docs/admin/namespaces/index.md | 4 + docs/admin/namespaces/walkthrough.md | 4 + docs/admin/network-plugins.md | 5 + docs/admin/networking.md | 4 + docs/admin/node-problem.md | 4 + docs/admin/node.md | 5 + docs/admin/out-of-resource.md | 5 + docs/admin/ovs-networking.md | 4 + docs/admin/resourcequota/OWNERS | 3 + docs/admin/resourcequota/index.md | 3 + docs/admin/resourcequota/walkthrough.md | 4 + docs/admin/salt.md | 34 +-- docs/admin/service-accounts-admin.md | 6 + docs/admin/static-pods.md | 3 + docs/api-reference/OWNERS | 5 + docs/api-reference/autoscaling/OWNERS | 6 + docs/api-reference/batch/OWNERS | 6 + docs/api-reference/extensions/OWNERS | 6 + docs/api-reference/v1/OWNERS | 6 + docs/api-reference/v1/definitions.md | 4 + docs/api-reference/v1/operations.md | 4 + docs/api.md | 5 + docs/getting-started-guides/OWNERS | 4 + docs/getting-started-guides/aws.md | 5 + docs/getting-started-guides/azure.md | 4 + docs/getting-started-guides/binary_release.md | 4 + docs/getting-started-guides/centos/OWNERS | 5 + .../centos/centos_manual_config.md | 5 + docs/getting-started-guides/clc.md | 4 + docs/getting-started-guides/cloudstack.md | 4 + docs/getting-started-guides/coreos/OWNERS | 5 + .../coreos/bare_metal_calico.md | 3 + .../coreos/bare_metal_offline.md | 5 + .../coreos/coreos_multinode_cluster.md | 4 + docs/getting-started-guides/coreos/index.md | 4 + docs/getting-started-guides/dcos.md | 3 + .../docker-multinode/OWNERS | 5 + docs/getting-started-guides/docker.md | 5 + docs/getting-started-guides/fedora/OWNERS | 5 + .../fedora/fedora-calico.md | 5 + .../fedora/fedora_ansible_config.md | 4 + .../fedora/fedora_manual_config.md | 5 + .../fedora/flannel_multi_node_cluster.md | 5 + docs/getting-started-guides/gce.md | 6 + docs/getting-started-guides/index.md | 5 + docs/getting-started-guides/juju.md | 5 + docs/getting-started-guides/libvirt-coreos.md | 5 + docs/getting-started-guides/locally.md | 5 + .../logging-elasticsearch.md | 4 + docs/getting-started-guides/logging.md | 4 + docs/getting-started-guides/meanstack.md | 4 + docs/getting-started-guides/mesos-docker.md | 4 + docs/getting-started-guides/mesos/OWNERS | 7 +- docs/getting-started-guides/mesos/index.md | 5 + docs/getting-started-guides/minikube.md | 5 + docs/getting-started-guides/openstack-heat.md | 7 + docs/getting-started-guides/ovirt.md | 4 + .../photon-controller.md | 4 + docs/getting-started-guides/rackspace.md | 4 + docs/getting-started-guides/rkt/OWNERS | 5 + docs/getting-started-guides/rkt/index.md | 5 + docs/getting-started-guides/rkt/notes.md | 4 + docs/getting-started-guides/scratch.md | 5 + docs/getting-started-guides/ubuntu-calico.md | 4 + docs/getting-started-guides/ubuntu.md | 5 + docs/getting-started-guides/vagrant.md | 5 + docs/getting-started-guides/vsphere.md | 4 + docs/hellonode.md | 5 + docs/index.md | 4 + docs/reference.md | 3 + docs/reporting-security-issues.md | 4 + docs/samples.md | 4 + docs/templatedemos/OWNERS | 3 + docs/templatedemos/blankkubectl.md | 3 + docs/templatedemos/index.md | 3 + docs/templatedemos/kubectl.md | 3 + docs/troubleshooting.md | 4 + docs/user-guide/OWNERS | 5 + docs/user-guide/accessing-the-cluster.md | 74 +++--- docs/user-guide/annotations.md | 4 + .../user-guide/application-troubleshooting.md | 4 + docs/user-guide/compute-resources.md | 4 + docs/user-guide/config-best-practices.md | 3 + docs/user-guide/configmap/OWNERS | 4 + docs/user-guide/configmap/index.md | 4 + docs/user-guide/configuring-containers.md | 4 + docs/user-guide/connecting-applications.md | 5 + ...connecting-to-applications-port-forward.md | 4 + .../connecting-to-applications-proxy.md | 20 +- docs/user-guide/container-environment.md | 4 + docs/user-guide/containers.md | 3 + ...ugging-pods-and-replication-controllers.md | 3 + docs/user-guide/debugging-services.md | 5 + docs/user-guide/deploying-applications.md | 5 + docs/user-guide/deployments.md | 4 + docs/user-guide/docker-cli-to-kubectl.md | 5 + docs/user-guide/downward-api/OWNERS | 4 + docs/user-guide/downward-api/index.md | 4 + docs/user-guide/environment-guide/OWNERS | 3 + docs/user-guide/environment-guide/index.md | 3 + docs/user-guide/federation/OWNERS | 3 + .../federation/federated-services.md | 4 + docs/user-guide/garbage-collector.md | 34 +-- docs/user-guide/getting-into-containers.md | 48 ++-- .../horizontal-pod-autoscaling/OWNERS | 5 + .../horizontal-pod-autoscaling/index.md | 4 + .../horizontal-pod-autoscaling/walkthrough.md | 5 + docs/user-guide/identifiers.md | 4 + docs/user-guide/images.md | 4 + docs/user-guide/index.md | 3 + docs/user-guide/ingress.md | 3 + .../user-guide/introspection-and-debugging.md | 4 + docs/user-guide/jobs.md | 4 + docs/user-guide/jobs/OWNERS | 5 + docs/user-guide/jsonpath.md | 3 + docs/user-guide/kubeconfig-file.md | 248 +++++++++--------- docs/user-guide/kubectl-cheatsheet.md | 5 + docs/user-guide/kubectl-conventions.md | 4 + docs/user-guide/kubectl-overview.md | 4 + docs/user-guide/kubectl/OWNERS | 6 + docs/user-guide/kubectl/kubectl.md | 5 + docs/user-guide/kubectl/kubectl_annotate.md | 6 + .../kubectl/kubectl_api-versions.md | 4 + docs/user-guide/kubectl/kubectl_apply.md | 4 + docs/user-guide/kubectl/kubectl_attach.md | 4 + docs/user-guide/kubectl/kubectl_autoscale.md | 5 + .../kubectl/kubectl_cluster-info.md | 4 + docs/user-guide/kubectl/kubectl_config.md | 4 + .../kubectl/kubectl_config_current-context.md | 4 + .../kubectl/kubectl_config_set-cluster.md | 4 + .../kubectl/kubectl_config_set-context.md | 4 + .../kubectl/kubectl_config_set-credentials.md | 4 + docs/user-guide/kubectl/kubectl_config_set.md | 4 + .../kubectl/kubectl_config_unset.md | 4 + .../kubectl/kubectl_config_use-context.md | 4 + .../user-guide/kubectl/kubectl_config_view.md | 6 + docs/user-guide/kubectl/kubectl_convert.md | 4 + docs/user-guide/kubectl/kubectl_cordon.md | 4 + docs/user-guide/kubectl/kubectl_create.md | 5 + .../kubectl/kubectl_create_configmap.md | 4 + .../kubectl/kubectl_create_namespace.md | 4 + .../kubectl/kubectl_create_secret.md | 4 + .../kubectl_create_secret_docker-registry.md | 4 + .../kubectl/kubectl_create_secret_generic.md | 4 + .../kubectl/kubectl_create_serviceaccount.md | 6 + docs/user-guide/kubectl/kubectl_delete.md | 4 + docs/user-guide/kubectl/kubectl_describe.md | 4 + docs/user-guide/kubectl/kubectl_drain.md | 4 + docs/user-guide/kubectl/kubectl_edit.md | 4 + docs/user-guide/kubectl/kubectl_exec.md | 4 + docs/user-guide/kubectl/kubectl_explain.md | 4 + docs/user-guide/kubectl/kubectl_expose.md | 6 + docs/user-guide/kubectl/kubectl_get.md | 5 + docs/user-guide/kubectl/kubectl_label.md | 6 + docs/user-guide/kubectl/kubectl_logs.md | 4 + docs/user-guide/kubectl/kubectl_namespace.md | 5 + docs/user-guide/kubectl/kubectl_patch.md | 4 + .../kubectl/kubectl_port-forward.md | 4 + docs/user-guide/kubectl/kubectl_proxy.md | 4 + docs/user-guide/kubectl/kubectl_replace.md | 6 + .../kubectl/kubectl_rolling-update.md | 6 + docs/user-guide/kubectl/kubectl_rollout.md | 4 + .../kubectl/kubectl_rollout_history.md | 4 + .../kubectl/kubectl_rollout_pause.md | 4 + .../kubectl/kubectl_rollout_resume.md | 4 + .../kubectl/kubectl_rollout_undo.md | 4 + docs/user-guide/kubectl/kubectl_run.md | 6 + docs/user-guide/kubectl/kubectl_scale.md | 4 + docs/user-guide/kubectl/kubectl_stop.md | 5 + docs/user-guide/kubectl/kubectl_uncordon.md | 4 + docs/user-guide/kubectl/kubectl_version.md | 4 + docs/user-guide/labels.md | 3 + docs/user-guide/liveness/OWNERS | 3 + docs/user-guide/liveness/index.md | 4 + docs/user-guide/load-balancer.md | 3 + docs/user-guide/logging-demo/OWNERS | 3 + docs/user-guide/logging.md | 29 +- docs/user-guide/managing-deployments.md | 5 + docs/user-guide/monitoring.md | 3 + docs/user-guide/namespaces.md | 5 + docs/user-guide/networkpolicies.md | 6 + docs/user-guide/nginx/OWNERS | 3 + docs/user-guide/node-selection/OWNERS | 4 + docs/user-guide/node-selection/index.md | 5 + docs/user-guide/persistent-volumes/OWNERS | 6 + docs/user-guide/persistent-volumes/index.md | 6 + .../persistent-volumes/walkthrough.md | 4 + docs/user-guide/petset.md | 5 + docs/user-guide/petset/OWNERS | 5 + docs/user-guide/pod-states.md | 3 + docs/user-guide/pod-templates.md | 3 + docs/user-guide/pods/OWNERS | 4 + docs/user-guide/pods/index.md | 2 + docs/user-guide/pods/multi-container.md | 3 + docs/user-guide/pods/single-container.md | 4 + docs/user-guide/prereqs.md | 4 + docs/user-guide/production-pods.md | 5 + docs/user-guide/quick-start.md | 4 + docs/user-guide/replicasets.md | 6 + docs/user-guide/replicasets/OWNERS | 5 + docs/user-guide/replication-controller/OWNERS | 7 + .../replication-controller/index.md | 5 + .../replication-controller/operations.md | 3 + .../resizing-a-replication-controller.md | 3 + docs/user-guide/rolling-updates.md | 4 + docs/user-guide/secrets/OWNERS | 4 + docs/user-guide/secrets/index.md | 4 + docs/user-guide/secrets/walkthrough.md | 2 + docs/user-guide/security-context.md | 5 + docs/user-guide/service-accounts.md | 6 + docs/user-guide/services-firewalls.md | 4 + docs/user-guide/services/OWNERS | 6 + docs/user-guide/services/index.md | 5 + docs/user-guide/services/operations.md | 4 + docs/user-guide/sharing-clusters.md | 4 + docs/user-guide/simple-nginx.md | 3 + docs/user-guide/simple-yaml.md | 4 + docs/user-guide/ui.md | 87 +++--- docs/user-guide/update-demo/OWNERS | 4 + docs/user-guide/update-demo/index.md | 18 +- docs/user-guide/volumes.md | 6 + docs/user-guide/walkthrough/OWNERS | 5 + docs/user-guide/walkthrough/index.md | 5 + docs/user-guide/walkthrough/k8s201.md | 4 + docs/user-guide/working-with-resources.md | 4 + docs/whatisk8s.md | 4 + 258 files changed, 1477 insertions(+), 369 deletions(-) create mode 100644 OWNERS create mode 100644 docs/admin/OWNERS create mode 100644 docs/admin/federation/OWNERS create mode 100644 docs/admin/high-availability/OWNERS create mode 100644 docs/admin/limitrange/OWNERS create mode 100644 docs/admin/multiple-schedulers/OWNERS create mode 100644 docs/admin/namespaces/OWNERS create mode 100644 docs/admin/resourcequota/OWNERS create mode 100644 docs/api-reference/OWNERS create mode 100644 docs/api-reference/autoscaling/OWNERS create mode 100644 docs/api-reference/batch/OWNERS create mode 100644 docs/api-reference/extensions/OWNERS create mode 100644 docs/api-reference/v1/OWNERS create mode 100644 docs/getting-started-guides/OWNERS create mode 100644 docs/getting-started-guides/centos/OWNERS create mode 100644 docs/getting-started-guides/coreos/OWNERS create mode 100644 docs/getting-started-guides/docker-multinode/OWNERS create mode 100644 docs/getting-started-guides/fedora/OWNERS create mode 100644 docs/getting-started-guides/rkt/OWNERS create mode 100644 docs/templatedemos/OWNERS create mode 100644 docs/user-guide/OWNERS create mode 100644 docs/user-guide/configmap/OWNERS create mode 100644 docs/user-guide/downward-api/OWNERS create mode 100644 docs/user-guide/environment-guide/OWNERS create mode 100644 docs/user-guide/federation/OWNERS create mode 100644 docs/user-guide/horizontal-pod-autoscaling/OWNERS create mode 100644 docs/user-guide/jobs/OWNERS create mode 100644 docs/user-guide/kubectl/OWNERS create mode 100644 docs/user-guide/liveness/OWNERS create mode 100644 docs/user-guide/logging-demo/OWNERS create mode 100644 docs/user-guide/nginx/OWNERS create mode 100644 docs/user-guide/node-selection/OWNERS create mode 100644 docs/user-guide/persistent-volumes/OWNERS create mode 100644 docs/user-guide/petset/OWNERS create mode 100644 docs/user-guide/pods/OWNERS create mode 100644 docs/user-guide/replicasets/OWNERS create mode 100644 docs/user-guide/replication-controller/OWNERS create mode 100644 docs/user-guide/secrets/OWNERS create mode 100644 docs/user-guide/services/OWNERS create mode 100644 docs/user-guide/update-demo/OWNERS create mode 100644 docs/user-guide/walkthrough/OWNERS diff --git a/OWNERS b/OWNERS new file mode 100644 index 0000000000..247d39ea5e --- /dev/null +++ b/OWNERS @@ -0,0 +1,7 @@ +assignees: +- lavalamp +- smarterclayton +- janetkuo +- pwittrock +- kelseyhightower +- jaredbhatti diff --git a/docs/admin/OWNERS b/docs/admin/OWNERS new file mode 100644 index 0000000000..498d076a5f --- /dev/null +++ b/docs/admin/OWNERS @@ -0,0 +1,4 @@ +assignees: +- derekwaynecarr +- mikedanese + diff --git a/docs/admin/accessing-the-api.md b/docs/admin/accessing-the-api.md index d18d30bfd1..92380ded3f 100644 --- a/docs/admin/accessing-the-api.md +++ b/docs/admin/accessing-the-api.md @@ -1,4 +1,9 @@ --- +assignees: +- bgrant0607 +- erictune +- lavalamp + --- This document describes how access to the Kubernetes API is controlled. diff --git a/docs/admin/admission-controllers.md b/docs/admin/admission-controllers.md index 9072522ac2..a99b44f339 100644 --- a/docs/admin/admission-controllers.md +++ b/docs/admin/admission-controllers.md @@ -1,4 +1,12 @@ --- +assignees: +- bprashanth +- davidopp +- derekwaynecarr +- erictune +- janetkuo +- thockin + --- * TOC diff --git a/docs/admin/authentication.md b/docs/admin/authentication.md index 52b5bf925d..d003aabb61 100644 --- a/docs/admin/authentication.md +++ b/docs/admin/authentication.md @@ -1,4 +1,9 @@ --- +assignees: +- erictune +- lavalamp +- yifan-gu + --- Kubernetes uses client certificates, tokens, or http basic auth to authenticate users for API calls. diff --git a/docs/admin/authorization.md b/docs/admin/authorization.md index 312bbb1d05..e683dc1deb 100644 --- a/docs/admin/authorization.md +++ b/docs/admin/authorization.md @@ -1,4 +1,8 @@ --- +assignees: +- erictune +- lavalamp + --- In Kubernetes, authorization happens as a separate step from authentication. diff --git a/docs/admin/cluster-components.md b/docs/admin/cluster-components.md index 2e7e663f42..c1bcae8577 100644 --- a/docs/admin/cluster-components.md +++ b/docs/admin/cluster-components.md @@ -1,4 +1,7 @@ --- +assignees: +- lavalamp + --- This document outlines the various binary components that need to run to diff --git a/docs/admin/cluster-large.md b/docs/admin/cluster-large.md index be05cb627f..6df6bca474 100644 --- a/docs/admin/cluster-large.md +++ b/docs/admin/cluster-large.md @@ -1,4 +1,8 @@ --- +assignees: +- davidopp +- lavalamp + --- diff --git a/docs/admin/cluster-management.md b/docs/admin/cluster-management.md index f133f06983..1be4fc8a02 100644 --- a/docs/admin/cluster-management.md +++ b/docs/admin/cluster-management.md @@ -1,4 +1,8 @@ --- +assignees: +- lavalamp +- thockin + --- * TOC diff --git a/docs/admin/cluster-troubleshooting.md b/docs/admin/cluster-troubleshooting.md index 22d461cd70..8bab089ce6 100644 --- a/docs/admin/cluster-troubleshooting.md +++ b/docs/admin/cluster-troubleshooting.md @@ -1,4 +1,7 @@ --- +assignees: +- davidopp + --- This doc is about cluster troubleshooting; we assume you have already ruled out your application as the root cause of the diff --git a/docs/admin/daemons.md b/docs/admin/daemons.md index 2e50d8ef9b..be3137bc93 100644 --- a/docs/admin/daemons.md +++ b/docs/admin/daemons.md @@ -1,4 +1,7 @@ --- +assignees: +- erictune + --- * TOC diff --git a/docs/admin/dns.md b/docs/admin/dns.md index 9bf60e8bba..bdd0848cf9 100644 --- a/docs/admin/dns.md +++ b/docs/admin/dns.md @@ -1,4 +1,9 @@ --- +assignees: +- ArtfulCoder +- davidopp +- lavalamp + --- ## Introduction diff --git a/docs/admin/etcd.md b/docs/admin/etcd.md index f17b555cd8..14b36a33be 100644 --- a/docs/admin/etcd.md +++ b/docs/admin/etcd.md @@ -1,4 +1,7 @@ --- +assignees: +- lavalamp + --- diff --git a/docs/admin/federation/OWNERS b/docs/admin/federation/OWNERS new file mode 100644 index 0000000000..321cd31a2c --- /dev/null +++ b/docs/admin/federation/OWNERS @@ -0,0 +1,4 @@ +assignees: +- mml +- nikhiljindal + diff --git a/docs/admin/federation/index.md b/docs/admin/federation/index.md index fc077ec839..76e08cf90b 100644 --- a/docs/admin/federation/index.md +++ b/docs/admin/federation/index.md @@ -1,4 +1,8 @@ --- +assignees: +- mml +- nikhiljindal + --- This guide explains how to set up cluster federation that lets us control multiple Kubernetes clusters. diff --git a/docs/admin/garbage-collection.md b/docs/admin/garbage-collection.md index 81aeb772c0..d608687400 100644 --- a/docs/admin/garbage-collection.md +++ b/docs/admin/garbage-collection.md @@ -1,4 +1,8 @@ --- +assignees: +- dalanlan +- mikedanese + --- * TOC diff --git a/docs/admin/high-availability/OWNERS b/docs/admin/high-availability/OWNERS new file mode 100644 index 0000000000..0ebc2894ee --- /dev/null +++ b/docs/admin/high-availability/OWNERS @@ -0,0 +1,4 @@ +assignees: +- davidopp +- lavalamp + diff --git a/docs/admin/high-availability/index.md b/docs/admin/high-availability/index.md index d7932e47ca..1074bdeb17 100644 --- a/docs/admin/high-availability/index.md +++ b/docs/admin/high-availability/index.md @@ -1,4 +1,8 @@ ---- +--- +assignees: +- mwhahaha +- stp-ip + --- ## Introduction diff --git a/docs/admin/index.md b/docs/admin/index.md index 1fc64186ea..4df7fb3375 100644 --- a/docs/admin/index.md +++ b/docs/admin/index.md @@ -1,4 +1,8 @@ --- +assignees: +- davidopp +- lavalamp + --- The cluster admin guide is for anyone creating or administering a Kubernetes cluster. diff --git a/docs/admin/kube-apiserver.md b/docs/admin/kube-apiserver.md index 43fe66d37c..350727c61f 100644 --- a/docs/admin/kube-apiserver.md +++ b/docs/admin/kube-apiserver.md @@ -1,4 +1,8 @@ --- +assignees: +- bgrant0607 +- nikhiljindal + --- ## kube-apiserver diff --git a/docs/admin/kube-controller-manager.md b/docs/admin/kube-controller-manager.md index 70a1e576d4..02648408da 100644 --- a/docs/admin/kube-controller-manager.md +++ b/docs/admin/kube-controller-manager.md @@ -1,4 +1,8 @@ --- +assignees: +- bprashanth +- mqliang + --- ## kube-controller-manager diff --git a/docs/admin/kube-proxy.md b/docs/admin/kube-proxy.md index ff2a5af980..feb9143a8f 100644 --- a/docs/admin/kube-proxy.md +++ b/docs/admin/kube-proxy.md @@ -1,4 +1,7 @@ --- +assignees: +- thockin + --- ## kube-proxy diff --git a/docs/admin/kube-scheduler.md b/docs/admin/kube-scheduler.md index e0f6e35fa6..3470241d88 100644 --- a/docs/admin/kube-scheduler.md +++ b/docs/admin/kube-scheduler.md @@ -1,4 +1,8 @@ --- +assignees: +- davidopp +- mikedanese + --- ## kube-scheduler diff --git a/docs/admin/kubelet.md b/docs/admin/kubelet.md index ac88581046..e065c696b9 100644 --- a/docs/admin/kubelet.md +++ b/docs/admin/kubelet.md @@ -1,4 +1,9 @@ --- +assignees: +- bprashanth +- derekwaynecarr +- mikedanese + --- ## kubelet diff --git a/docs/admin/limitrange/OWNERS b/docs/admin/limitrange/OWNERS new file mode 100644 index 0000000000..1dac6f1017 --- /dev/null +++ b/docs/admin/limitrange/OWNERS @@ -0,0 +1,4 @@ +assignees: +- derekwaynecarr +- janetkuo + diff --git a/docs/admin/limitrange/index.md b/docs/admin/limitrange/index.md index 7c89b3058b..f38737981d 100644 --- a/docs/admin/limitrange/index.md +++ b/docs/admin/limitrange/index.md @@ -1,5 +1,9 @@ ---- ---- +--- +assignees: +- derekwaynecarr +- janetkuo + +--- By default, pods run with unbounded CPU and memory limits. This means that any pod in the system will be able to consume as much CPU and memory on the node that executes the pod. @@ -40,32 +44,32 @@ This example will work in a custom namespace to demonstrate the concepts involve Let's create a new namespace called limit-example: -```shell -$ kubectl create namespace limit-example -namespace "limit-example" created -``` - -Note that `kubectl` commands will print the type and name of the resource created or mutated, which can then be used in subsequent commands: - -```shell +```shell +$ kubectl create namespace limit-example +namespace "limit-example" created +``` + +Note that `kubectl` commands will print the type and name of the resource created or mutated, which can then be used in subsequent commands: + +```shell $ kubectl get namespaces -NAME STATUS AGE -default Active 51s -limit-example Active 45s -``` - +NAME STATUS AGE +default Active 51s +limit-example Active 45s +``` + ## Step 2: Apply a limit to the namespace Let's create a simple limit in our namespace. -```shell +```shell $ kubectl create -f docs/admin/limitrange/limits.yaml --namespace=limit-example limitrange "mylimits" created -``` - +``` + Let's describe the limits that we have imposed in our namespace. -```shell +```shell $ kubectl describe limits mylimits --namespace=limit-example Name: mylimits Namespace: limit-example @@ -75,8 +79,8 @@ Pod cpu 200m 2 - - Pod memory 6Mi 1Gi - - - Container cpu 100m 2 200m 300m - Container memory 3Mi 1Gi 100Mi 200Mi - -``` - +``` + In this scenario, we have said the following: 1. If a max constraint is specified for a resource (2 CPU and 1Gi memory in this case), then a limit @@ -103,108 +107,108 @@ of creation explaining why. Let's first spin up a [Deployment](/docs/user-guide/deployments) that creates a single container Pod to demonstrate how default values are applied to each pod. -```shell +```shell $ kubectl run nginx --image=nginx --replicas=1 --namespace=limit-example deployment "nginx" created -``` - -Note that `kubectl run` creates a Deployment named "nginx" on Kubernetes cluster >= v1.2. If you are running older versions, it creates replication controllers instead. -If you want to obtain the old behavior, use `--generator=run/v1` to create replication controllers. See [`kubectl run`](/docs/user-guide/kubectl/kubectl_run/) for more details. -The Deployment manages 1 replica of single container Pod. Let's take a look at the Pod it manages. First, find the name of the Pod: - -```shell +``` + +Note that `kubectl run` creates a Deployment named "nginx" on Kubernetes cluster >= v1.2. If you are running older versions, it creates replication controllers instead. +If you want to obtain the old behavior, use `--generator=run/v1` to create replication controllers. See [`kubectl run`](/docs/user-guide/kubectl/kubectl_run/) for more details. +The Deployment manages 1 replica of single container Pod. Let's take a look at the Pod it manages. First, find the name of the Pod: + +```shell $ kubectl get pods --namespace=limit-example -NAME READY STATUS RESTARTS AGE -nginx-2040093540-s8vzu 1/1 Running 0 11s -``` - -Let's print this Pod with yaml output format (using `-o yaml` flag), and then `grep` the `resources` field. Note that your pod name will be different. - -``` shell +NAME READY STATUS RESTARTS AGE +nginx-2040093540-s8vzu 1/1 Running 0 11s +``` + +Let's print this Pod with yaml output format (using `-o yaml` flag), and then `grep` the `resources` field. Note that your pod name will be different. + +``` shell $ kubectl get pods nginx-2040093540-s8vzu --namespace=limit-example -o yaml | grep resources -C 8 - resourceVersion: "57" - selfLink: /api/v1/namespaces/limit-example/pods/nginx-2040093540-ivimu - uid: 67b20741-f53b-11e5-b066-64510658e388 -spec: - containers: - - image: nginx - imagePullPolicy: Always - name: nginx - resources: - limits: - cpu: 300m - memory: 200Mi - requests: - cpu: 200m - memory: 100Mi - terminationMessagePath: /dev/termination-log - volumeMounts: -``` - + resourceVersion: "57" + selfLink: /api/v1/namespaces/limit-example/pods/nginx-2040093540-ivimu + uid: 67b20741-f53b-11e5-b066-64510658e388 +spec: + containers: + - image: nginx + imagePullPolicy: Always + name: nginx + resources: + limits: + cpu: 300m + memory: 200Mi + requests: + cpu: 200m + memory: 100Mi + terminationMessagePath: /dev/termination-log + volumeMounts: +``` + Note that our nginx container has picked up the namespace default cpu and memory resource *limits* and *requests*. Let's create a pod that exceeds our allowed limits by having it have a container that requests 3 cpu cores. -```shell +```shell $ kubectl create -f docs/admin/limitrange/invalid-pod.yaml --namespace=limit-example Error from server: error when creating "docs/admin/limitrange/invalid-pod.yaml": Pod "invalid-pod" is forbidden: [Maximum cpu usage per Pod is 2, but limit is 3., Maximum cpu usage per Container is 2, but limit is 3.] -``` - +``` + Let's create a pod that falls within the allowed limit boundaries. -```shell +```shell $ kubectl create -f docs/admin/limitrange/valid-pod.yaml --namespace=limit-example pod "valid-pod" created -``` - -Now look at the Pod's resources field: - -```shell +``` + +Now look at the Pod's resources field: + +```shell $ kubectl get pods valid-pod --namespace=limit-example -o yaml | grep -C 6 resources - uid: 3b1bfd7a-f53c-11e5-b066-64510658e388 -spec: - containers: - - image: gcr.io/google_containers/serve_hostname - imagePullPolicy: Always - name: kubernetes-serve-hostname - resources: - limits: - cpu: "1" - memory: 512Mi - requests: - cpu: "1" - memory: 512Mi -``` - + uid: 3b1bfd7a-f53c-11e5-b066-64510658e388 +spec: + containers: + - image: gcr.io/google_containers/serve_hostname + imagePullPolicy: Always + name: kubernetes-serve-hostname + resources: + limits: + cpu: "1" + memory: 512Mi + requests: + cpu: "1" + memory: 512Mi +``` + Note that this pod specifies explicit resource *limits* and *requests* so it did not pick up the namespace default values. Note: The *limits* for CPU resource are enforced in the default Kubernetes setup on the physical node that runs the container unless the administrator deploys the kubelet with the folllowing flag: -```shell +```shell $ kubelet --help Usage of kubelet .... --cpu-cfs-quota[=true]: Enable CPU CFS quota enforcement for containers that specify CPU limits $ kubelet --cpu-cfs-quota=false ... -``` - +``` + ## Step 4: Cleanup To remove the resources used by this example, you can just delete the limit-example namespace. -```shell +```shell $ kubectl delete namespace limit-example namespace "limit-example" deleted $ kubectl get namespaces -NAME STATUS AGE -default Active 12m -``` - +NAME STATUS AGE +default Active 12m +``` + ## Summary Cluster operators that want to restrict the amount of resources a single container or pod may consume are able to define allowable ranges per Kubernetes namespace. In the absence of any explicit assignments, the Kubernetes system is able to apply default resource *limits* and *requests* if desired in order to -constrain the amount of resource a pod consumes on a node. +constrain the amount of resource a pod consumes on a node. diff --git a/docs/admin/master-node-communication.md b/docs/admin/master-node-communication.md index 1d40b265f6..8ed0e3dc82 100644 --- a/docs/admin/master-node-communication.md +++ b/docs/admin/master-node-communication.md @@ -1,4 +1,9 @@ --- +assignees: +- dchen1107 +- laushinka +- roberthbailey + --- * TOC diff --git a/docs/admin/multi-cluster.md b/docs/admin/multi-cluster.md index e549a4d3ca..6359782409 100644 --- a/docs/admin/multi-cluster.md +++ b/docs/admin/multi-cluster.md @@ -1,4 +1,7 @@ --- +assignees: +- davidopp + --- You may want to set up multiple Kubernetes clusters, both to diff --git a/docs/admin/multiple-schedulers.md b/docs/admin/multiple-schedulers.md index a481831e50..8ba152ac04 100644 --- a/docs/admin/multiple-schedulers.md +++ b/docs/admin/multiple-schedulers.md @@ -1,4 +1,8 @@ --- +assignees: +- davidopp +- madhusudancs + --- Kubernetes ships with a default scheduler that is described [here](/docs/admin/kube-scheduler/). diff --git a/docs/admin/multiple-schedulers/OWNERS b/docs/admin/multiple-schedulers/OWNERS new file mode 100644 index 0000000000..3941099ffc --- /dev/null +++ b/docs/admin/multiple-schedulers/OWNERS @@ -0,0 +1,4 @@ +assignees: +- davidopp +- madhusudancs + diff --git a/docs/admin/multiple-zones.md b/docs/admin/multiple-zones.md index f7a6223b50..e5d1367862 100644 --- a/docs/admin/multiple-zones.md +++ b/docs/admin/multiple-zones.md @@ -1,4 +1,9 @@ --- +assignees: +- jlowdermilk +- justinsb +- quinton-hoole + --- ## Introduction diff --git a/docs/admin/namespaces/OWNERS b/docs/admin/namespaces/OWNERS new file mode 100644 index 0000000000..1dac6f1017 --- /dev/null +++ b/docs/admin/namespaces/OWNERS @@ -0,0 +1,4 @@ +assignees: +- derekwaynecarr +- janetkuo + diff --git a/docs/admin/namespaces/index.md b/docs/admin/namespaces/index.md index c9f51280ad..95780559e3 100644 --- a/docs/admin/namespaces/index.md +++ b/docs/admin/namespaces/index.md @@ -1,4 +1,8 @@ --- +assignees: +- derekwaynecarr +- janetkuo + --- A Namespace is a mechanism to partition resources created by users into diff --git a/docs/admin/namespaces/walkthrough.md b/docs/admin/namespaces/walkthrough.md index e3b87c7f65..2a3e6298ea 100644 --- a/docs/admin/namespaces/walkthrough.md +++ b/docs/admin/namespaces/walkthrough.md @@ -1,4 +1,8 @@ --- +assignees: +- derekwaynecarr +- janetkuo + --- Kubernetes _namespaces_ help different projects, teams, or customers to share a Kubernetes cluster. diff --git a/docs/admin/network-plugins.md b/docs/admin/network-plugins.md index 5b41c0e310..ee85126961 100644 --- a/docs/admin/network-plugins.md +++ b/docs/admin/network-plugins.md @@ -1,4 +1,9 @@ --- +assignees: +- dcbw +- freehan +- thockin + --- * TOC diff --git a/docs/admin/networking.md b/docs/admin/networking.md index 8032bdf601..cd8aaa09cf 100644 --- a/docs/admin/networking.md +++ b/docs/admin/networking.md @@ -1,4 +1,8 @@ --- +assignees: +- lavalamp +- thockin + --- Kubernetes approaches networking somewhat differently than Docker does by diff --git a/docs/admin/node-problem.md b/docs/admin/node-problem.md index 5dc2d4cb52..ab09a78dfc 100644 --- a/docs/admin/node-problem.md +++ b/docs/admin/node-problem.md @@ -1,4 +1,8 @@ --- +assignees: +- Random-Liu +- dchen1107 + --- * TOC diff --git a/docs/admin/node.md b/docs/admin/node.md index afae4454d6..1f1c89b4cb 100644 --- a/docs/admin/node.md +++ b/docs/admin/node.md @@ -1,4 +1,9 @@ --- +assignees: +- caesarxuchao +- dchen1107 +- lavalamp + --- * TOC diff --git a/docs/admin/out-of-resource.md b/docs/admin/out-of-resource.md index 2b85f486d5..16b8cc9e1e 100644 --- a/docs/admin/out-of-resource.md +++ b/docs/admin/out-of-resource.md @@ -1,4 +1,9 @@ --- +assignees: +- derekwaynecarr +- vishh +- timstclair + --- * TOC diff --git a/docs/admin/ovs-networking.md b/docs/admin/ovs-networking.md index f5a279339a..7a8f89506c 100644 --- a/docs/admin/ovs-networking.md +++ b/docs/admin/ovs-networking.md @@ -1,4 +1,8 @@ --- +assignees: +- lavalamp +- thockin + --- This document describes how OpenVSwitch is used to setup networking between pods across nodes. diff --git a/docs/admin/resourcequota/OWNERS b/docs/admin/resourcequota/OWNERS new file mode 100644 index 0000000000..dc809362b2 --- /dev/null +++ b/docs/admin/resourcequota/OWNERS @@ -0,0 +1,3 @@ +assignees: +- derekwaynecarr + diff --git a/docs/admin/resourcequota/index.md b/docs/admin/resourcequota/index.md index af96bc538d..d3073848ba 100644 --- a/docs/admin/resourcequota/index.md +++ b/docs/admin/resourcequota/index.md @@ -1,4 +1,7 @@ --- +assignees: +- derekwaynecarr + --- When several users or teams share a cluster with a fixed number of nodes, diff --git a/docs/admin/resourcequota/walkthrough.md b/docs/admin/resourcequota/walkthrough.md index dcae893d40..4712f7f25a 100644 --- a/docs/admin/resourcequota/walkthrough.md +++ b/docs/admin/resourcequota/walkthrough.md @@ -1,4 +1,8 @@ --- +assignees: +- derekwaynecarr +- janetkuo + --- This example demonstrates a typical setup to control for resource usage in a namespace. diff --git a/docs/admin/salt.md b/docs/admin/salt.md index 6210e59055..2cb634d7c6 100644 --- a/docs/admin/salt.md +++ b/docs/admin/salt.md @@ -1,5 +1,9 @@ ---- ---- +--- +assignees: +- davidopp +- lavalamp + +--- The Kubernetes cluster can be configured using Salt. @@ -13,11 +17,11 @@ The **salt-minion** service runs on the kubernetes-master and each kubernetes-no Each salt-minion service is configured to interact with the **salt-master** service hosted on the kubernetes-master via the **master.conf** file [(except on GCE)](#standalone-salt-configuration-on-gce). -```shell +```shell [root@kubernetes-master] $ cat /etc/salt/minion.d/master.conf master: kubernetes-master -``` - +``` + The salt-master is contacted by each salt-minion and depending upon the machine information presented, the salt-master will provision the machine as either a kubernetes-master or kubernetes-node with all the required capabilities needed to run Kubernetes. If you are running the Vagrant based environment, the **salt-api** service is running on the kubernetes-master. It is configured to enable the vagrant user to introspect the salt cluster in order to find out about machines in the Vagrant environment via a REST API. @@ -34,27 +38,27 @@ All remaining sections that refer to master/minion setups should be ignored for Security is not enabled on the salt-master, and the salt-master is configured to auto-accept incoming requests from minions. It is not recommended to use this security configuration in production environments without deeper study. (In some environments this isn't as bad as it might sound if the salt master port isn't externally accessible and you trust everyone on your network.) -```shell +```shell [root@kubernetes-master] $ cat /etc/salt/master.d/auto-accept.conf open_mode: True auto_accept: True -``` - +``` + ## Salt minion configuration Each minion in the salt cluster has an associated configuration that instructs the salt-master how to provision the required resources on the machine. An example file is presented below using the Vagrant based environment. -```shell +```shell [root@kubernetes-master] $ cat /etc/salt/minion.d/grains.conf grains: etcd_servers: $MASTER_IP cloud: vagrant roles: - kubernetes-master -``` - +``` + Each hosting environment has a slightly different grains.conf file that is used to build conditional logic where required in the Salt files. The following enumerates the set of defined key/value pairs that are supported today. If you add new ones, please make sure to update this list. @@ -77,16 +81,16 @@ These keys may be leveraged by the Salt sls files to branch behavior. In addition, a cluster may be running a Debian based operating system or Red Hat based operating system (Centos, Fedora, RHEL, etc.). As a result, it's important to sometimes distinguish behavior based on operating system using if branches like the following. -```liquid +```liquid {% raw %} {% if grains['os_family'] == 'RedHat' %} // something specific to a RedHat environment (Centos, Fedora, RHEL) where you may use yum, systemd, etc. {% else %} // something specific to Debian environment (apt-get, initd) -{% endif %} +{% endif %} {% endraw %} -``` - +``` + ## Best Practices 1. When configuring default arguments for processes, it's best to avoid the use of EnvironmentFiles (Systemd in Red Hat environments) or init.d files (Debian distributions) to hold default values that should be common across operating system environments. This helps keep our Salt template files easy to understand for editors who may not be familiar with the particulars of each distribution. diff --git a/docs/admin/service-accounts-admin.md b/docs/admin/service-accounts-admin.md index 146f91310b..810f4d7515 100644 --- a/docs/admin/service-accounts-admin.md +++ b/docs/admin/service-accounts-admin.md @@ -1,4 +1,10 @@ --- +assignees: +- bprashanth +- davidopp +- lavalamp +- liggitt + --- *This is a Cluster Administrator guide to service accounts. It assumes knowledge of diff --git a/docs/admin/static-pods.md b/docs/admin/static-pods.md index a341376c8a..ea9468f31c 100644 --- a/docs/admin/static-pods.md +++ b/docs/admin/static-pods.md @@ -1,4 +1,7 @@ --- +assignees: +- jsafrane + --- **If you are running clustered Kubernetes and are using static pods to run a pod on every node, you should probably be using a [DaemonSet](/docs/admin/daemons/)!** diff --git a/docs/api-reference/OWNERS b/docs/api-reference/OWNERS new file mode 100644 index 0000000000..cca5435e84 --- /dev/null +++ b/docs/api-reference/OWNERS @@ -0,0 +1,5 @@ +assignees: +- bgrant0607 +- caesarxuchao +- pmorie + diff --git a/docs/api-reference/autoscaling/OWNERS b/docs/api-reference/autoscaling/OWNERS new file mode 100644 index 0000000000..815e01e661 --- /dev/null +++ b/docs/api-reference/autoscaling/OWNERS @@ -0,0 +1,6 @@ +assignees: +- bgrant0607 +- caesarxuchao +- fgrzadkowski +- smarterclayton + diff --git a/docs/api-reference/batch/OWNERS b/docs/api-reference/batch/OWNERS new file mode 100644 index 0000000000..859974e212 --- /dev/null +++ b/docs/api-reference/batch/OWNERS @@ -0,0 +1,6 @@ +assignees: +- bgrant0607 +- caesarxuchao +- gmarek +- smarterclayton + diff --git a/docs/api-reference/extensions/OWNERS b/docs/api-reference/extensions/OWNERS new file mode 100644 index 0000000000..f0dea3903c --- /dev/null +++ b/docs/api-reference/extensions/OWNERS @@ -0,0 +1,6 @@ +assignees: +- bgrant0607 +- caesarxuchao +- madhusudancs +- pmorie + diff --git a/docs/api-reference/v1/OWNERS b/docs/api-reference/v1/OWNERS new file mode 100644 index 0000000000..71582ee6a4 --- /dev/null +++ b/docs/api-reference/v1/OWNERS @@ -0,0 +1,6 @@ +assignees: +- bgrant0607 +- caesarxuchao +- nikhiljindal +- pmorie + diff --git a/docs/api-reference/v1/definitions.md b/docs/api-reference/v1/definitions.md index 05dfc063cf..dddff20bf7 100644 --- a/docs/api-reference/v1/definitions.md +++ b/docs/api-reference/v1/definitions.md @@ -1,3 +1,7 @@ --- +assignees: +- bgrant0607 +- caesarxuchao + --- {% include v1.3/v1-definitions.html %} diff --git a/docs/api-reference/v1/operations.md b/docs/api-reference/v1/operations.md index 32b4629e25..e0241125ba 100644 --- a/docs/api-reference/v1/operations.md +++ b/docs/api-reference/v1/operations.md @@ -1,3 +1,7 @@ --- +assignees: +- bgrant0607 +- caesarxuchao + --- {% include v1.3/v1-operations.html %} diff --git a/docs/api.md b/docs/api.md index ab8a6a3953..aa9aea1d7f 100644 --- a/docs/api.md +++ b/docs/api.md @@ -1,4 +1,9 @@ --- +assignees: +- bgrant0607 +- erictune +- lavalamp + --- Primary system and API concepts are documented in the [User guide](/docs/user-guide/). diff --git a/docs/getting-started-guides/OWNERS b/docs/getting-started-guides/OWNERS new file mode 100644 index 0000000000..a1e101ce43 --- /dev/null +++ b/docs/getting-started-guides/OWNERS @@ -0,0 +1,4 @@ +assignees: +- errordeveloper +- pires + diff --git a/docs/getting-started-guides/aws.md b/docs/getting-started-guides/aws.md index 4b5f642ad8..20a310f57f 100644 --- a/docs/getting-started-guides/aws.md +++ b/docs/getting-started-guides/aws.md @@ -1,4 +1,9 @@ --- +assignees: +- JamesDeFabia +- justinsb +- lavalamp + --- * TOC diff --git a/docs/getting-started-guides/azure.md b/docs/getting-started-guides/azure.md index 47050db1c9..bb01c8641d 100644 --- a/docs/getting-started-guides/azure.md +++ b/docs/getting-started-guides/azure.md @@ -1,4 +1,8 @@ --- +assignees: +- colemickens +- jeffmendoza + --- * TOC diff --git a/docs/getting-started-guides/binary_release.md b/docs/getting-started-guides/binary_release.md index 7280e7b89f..d3b2886f87 100644 --- a/docs/getting-started-guides/binary_release.md +++ b/docs/getting-started-guides/binary_release.md @@ -1,4 +1,8 @@ --- +assignees: +- david-mcmahon +- jbeda + --- You can either build a release from sources or download a pre-built release. If you do not plan on developing Kubernetes itself, we suggest a pre-built release. diff --git a/docs/getting-started-guides/centos/OWNERS b/docs/getting-started-guides/centos/OWNERS new file mode 100644 index 0000000000..3ff2ad25b0 --- /dev/null +++ b/docs/getting-started-guides/centos/OWNERS @@ -0,0 +1,5 @@ +assignees: +- coolsvap +- lavalamp +- thockin + diff --git a/docs/getting-started-guides/centos/centos_manual_config.md b/docs/getting-started-guides/centos/centos_manual_config.md index f2bc49099f..eeca62036c 100644 --- a/docs/getting-started-guides/centos/centos_manual_config.md +++ b/docs/getting-started-guides/centos/centos_manual_config.md @@ -1,4 +1,9 @@ --- +assignees: +- coolsvap +- lavalamp +- thockin + --- * TOC diff --git a/docs/getting-started-guides/clc.md b/docs/getting-started-guides/clc.md index 3e9b3f0858..637d2707e5 100644 --- a/docs/getting-started-guides/clc.md +++ b/docs/getting-started-guides/clc.md @@ -1,4 +1,8 @@ --- +assignees: +- JamesDeFabia +- ckleban + --- * TOC {: toc} diff --git a/docs/getting-started-guides/cloudstack.md b/docs/getting-started-guides/cloudstack.md index 313c97de65..2361a4fd79 100644 --- a/docs/getting-started-guides/cloudstack.md +++ b/docs/getting-started-guides/cloudstack.md @@ -1,4 +1,8 @@ --- +assignees: +- lavalamp +- thockin + --- CloudStack is a software to build public and private clouds based on hardware virtualization principles (traditional IaaS). To deploy Kubernetes on CloudStack there are several possibilities depending on the Cloud being used and what images are made available. [Exoscale](http://exoscale.ch) for instance makes a [CoreOS](http://coreos.com) template available, therefore instructions to deploy Kubernetes on coreOS can be used. CloudStack also has a vagrant plugin available, hence Vagrant could be used to deploy Kubernetes either using the existing shell provisioner or using new Salt based recipes. diff --git a/docs/getting-started-guides/coreos/OWNERS b/docs/getting-started-guides/coreos/OWNERS new file mode 100644 index 0000000000..a6c2c1b164 --- /dev/null +++ b/docs/getting-started-guides/coreos/OWNERS @@ -0,0 +1,5 @@ +assignees: +- AntonioMeireles +- errordeveloper +- pires + diff --git a/docs/getting-started-guides/coreos/bare_metal_calico.md b/docs/getting-started-guides/coreos/bare_metal_calico.md index 06ca2697a7..2447fe8f90 100644 --- a/docs/getting-started-guides/coreos/bare_metal_calico.md +++ b/docs/getting-started-guides/coreos/bare_metal_calico.md @@ -1,4 +1,7 @@ --- +assignees: +- caseydavenport + --- This document describes how to deploy Kubernetes with Calico networking on _bare metal_ CoreOS. For more information on Project Calico, visit [projectcalico.org](http://projectcalico.org) and the [calico-containers repository](https://github.com/projectcalico/calico-containers). diff --git a/docs/getting-started-guides/coreos/bare_metal_offline.md b/docs/getting-started-guides/coreos/bare_metal_offline.md index 9f2bec9320..8c71617163 100644 --- a/docs/getting-started-guides/coreos/bare_metal_offline.md +++ b/docs/getting-started-guides/coreos/bare_metal_offline.md @@ -1,4 +1,9 @@ --- +assignees: +- erictune +- jeffbean +- thockin + --- Deploy a CoreOS running Kubernetes environment. This particular guide is made to help those in an OFFLINE system, wither for testing a POC before the real deal, or you are restricted to be totally offline for your applications. diff --git a/docs/getting-started-guides/coreos/coreos_multinode_cluster.md b/docs/getting-started-guides/coreos/coreos_multinode_cluster.md index cc786e3ec4..9939a6960c 100644 --- a/docs/getting-started-guides/coreos/coreos_multinode_cluster.md +++ b/docs/getting-started-guides/coreos/coreos_multinode_cluster.md @@ -1,4 +1,8 @@ --- +assignees: +- dchen1107 +- pires + --- Use the [master.yaml](/docs/getting-started-guides/coreos/cloud-configs/master.yaml) and [node.yaml](/docs/getting-started-guides/coreos/cloud-configs/node.yaml) cloud-configs to provision a multi-node Kubernetes cluster. diff --git a/docs/getting-started-guides/coreos/index.md b/docs/getting-started-guides/coreos/index.md index 59fe04a1f2..6eab664c1c 100644 --- a/docs/getting-started-guides/coreos/index.md +++ b/docs/getting-started-guides/coreos/index.md @@ -1,4 +1,8 @@ --- +assignees: +- JamesDeFabia +- johscheuer + --- * TOC diff --git a/docs/getting-started-guides/dcos.md b/docs/getting-started-guides/dcos.md index 442c485bf9..43c6d28e36 100644 --- a/docs/getting-started-guides/dcos.md +++ b/docs/getting-started-guides/dcos.md @@ -1,4 +1,7 @@ --- +assignees: +- karlkfi + --- This guide will walk you through installing [Kubernetes-Mesos](https://github.com/mesosphere/kubernetes-mesos) on [Datacenter Operating System (DCOS)](https://mesosphere.com/product/) with the [DCOS CLI](https://github.com/mesosphere/dcos-cli) and operating Kubernetes with the [DCOS Kubectl plugin](https://github.com/mesosphere/dcos-kubectl). diff --git a/docs/getting-started-guides/docker-multinode/OWNERS b/docs/getting-started-guides/docker-multinode/OWNERS new file mode 100644 index 0000000000..687339adf0 --- /dev/null +++ b/docs/getting-started-guides/docker-multinode/OWNERS @@ -0,0 +1,5 @@ +assignees: +- dalanlan +- dchen1107 +- resouer + diff --git a/docs/getting-started-guides/docker.md b/docs/getting-started-guides/docker.md index a6893f27e7..4046045559 100644 --- a/docs/getting-started-guides/docker.md +++ b/docs/getting-started-guides/docker.md @@ -1,4 +1,9 @@ --- +assignees: +- asridharan +- brendandburns +- fgrzadkowski + --- **Stop. This guide has been superseded by [Minikube](../minikube/) which is the recommended method of running Kubernetes on your local machine.** diff --git a/docs/getting-started-guides/fedora/OWNERS b/docs/getting-started-guides/fedora/OWNERS new file mode 100644 index 0000000000..26456cebf2 --- /dev/null +++ b/docs/getting-started-guides/fedora/OWNERS @@ -0,0 +1,5 @@ +assignees: +- aveshagarwal +- eparis +- thockin + diff --git a/docs/getting-started-guides/fedora/fedora-calico.md b/docs/getting-started-guides/fedora/fedora-calico.md index a73b7e23bc..d8f983ec18 100644 --- a/docs/getting-started-guides/fedora/fedora-calico.md +++ b/docs/getting-started-guides/fedora/fedora-calico.md @@ -1,4 +1,9 @@ --- +assignees: +- alexhersh +- caesarxuchao +- caseydavenport + --- This guide will walk you through the process of getting a Kubernetes Fedora cluster running on Digital Ocean with networking powered by Calico networking. diff --git a/docs/getting-started-guides/fedora/fedora_ansible_config.md b/docs/getting-started-guides/fedora/fedora_ansible_config.md index 57532a4fa5..aa439f07b0 100644 --- a/docs/getting-started-guides/fedora/fedora_ansible_config.md +++ b/docs/getting-started-guides/fedora/fedora_ansible_config.md @@ -1,4 +1,8 @@ --- +assignees: +- aveshagarwal +- erictune + --- Configuring Kubernetes on Fedora via Ansible offers a simple way to quickly create a clustered environment with little effort. diff --git a/docs/getting-started-guides/fedora/fedora_manual_config.md b/docs/getting-started-guides/fedora/fedora_manual_config.md index 9872275b6c..238498d18c 100644 --- a/docs/getting-started-guides/fedora/fedora_manual_config.md +++ b/docs/getting-started-guides/fedora/fedora_manual_config.md @@ -1,4 +1,9 @@ --- +assignees: +- aveshagarwal +- eparis +- thockin + --- * TOC diff --git a/docs/getting-started-guides/fedora/flannel_multi_node_cluster.md b/docs/getting-started-guides/fedora/flannel_multi_node_cluster.md index 8de2e9dc4e..4c89d85e90 100644 --- a/docs/getting-started-guides/fedora/flannel_multi_node_cluster.md +++ b/docs/getting-started-guides/fedora/flannel_multi_node_cluster.md @@ -1,4 +1,9 @@ --- +assignees: +- dchen1107 +- erictune +- thockin + --- * TOC {:toc} diff --git a/docs/getting-started-guides/gce.md b/docs/getting-started-guides/gce.md index e3bbb8f6e5..a8b3a38a5e 100644 --- a/docs/getting-started-guides/gce.md +++ b/docs/getting-started-guides/gce.md @@ -1,4 +1,10 @@ --- +assignees: +- brendandburns +- jbeda +- mikedanese +- thockin + --- diff --git a/docs/getting-started-guides/index.md b/docs/getting-started-guides/index.md index 404191e62d..59857a03f2 100644 --- a/docs/getting-started-guides/index.md +++ b/docs/getting-started-guides/index.md @@ -1,4 +1,9 @@ --- +assignees: +- brendandburns +- erictune +- mikedanese + --- Kubernetes can run on a range of platforms, from your laptop, to VMs on a cloud provider, to rack of diff --git a/docs/getting-started-guides/juju.md b/docs/getting-started-guides/juju.md index ac7ceef8d0..d85c4b99c5 100644 --- a/docs/getting-started-guides/juju.md +++ b/docs/getting-started-guides/juju.md @@ -1,4 +1,9 @@ --- +assignees: +- caesarxuchao +- erictune +- mbruzek + --- [Juju](https://jujucharms.com/docs/2.0/about-juju) encapsulates the diff --git a/docs/getting-started-guides/libvirt-coreos.md b/docs/getting-started-guides/libvirt-coreos.md index 0b4d677a1f..eb16ee8a01 100644 --- a/docs/getting-started-guides/libvirt-coreos.md +++ b/docs/getting-started-guides/libvirt-coreos.md @@ -1,4 +1,9 @@ --- +assignees: +- erictune +- idvoretskyi +- lhuard1A + --- * TOC diff --git a/docs/getting-started-guides/locally.md b/docs/getting-started-guides/locally.md index cad7f6b586..afa71ea936 100644 --- a/docs/getting-started-guides/locally.md +++ b/docs/getting-started-guides/locally.md @@ -1,4 +1,9 @@ --- +assignees: +- erictune +- mikedanese +- thockin + --- **Stop. This guide has been superseded by [Minikube](../minikube/) which is the recommended method of running Kubernetes on your local machine.** diff --git a/docs/getting-started-guides/logging-elasticsearch.md b/docs/getting-started-guides/logging-elasticsearch.md index 515e981ab7..2141cfbc6d 100644 --- a/docs/getting-started-guides/logging-elasticsearch.md +++ b/docs/getting-started-guides/logging-elasticsearch.md @@ -1,4 +1,8 @@ --- +assignees: +- lavalamp +- satnam6502 + --- On the Google Compute Engine (GCE) platform, the default logging support targets diff --git a/docs/getting-started-guides/logging.md b/docs/getting-started-guides/logging.md index 84801d45fa..47bb5236b9 100644 --- a/docs/getting-started-guides/logging.md +++ b/docs/getting-started-guides/logging.md @@ -1,4 +1,8 @@ --- +assignees: +- lavalamp +- satnam6502 + --- A Kubernetes cluster will typically be humming along running many system and application pods. How does the system administrator collect, manage and query the logs of the system pods? How does a user query the logs of their application which is composed of many pods which may be restarted or automatically generated by the Kubernetes system? These questions are addressed by the Kubernetes **cluster level logging** services. diff --git a/docs/getting-started-guides/meanstack.md b/docs/getting-started-guides/meanstack.md index c758ed00fd..c3dbdf984a 100644 --- a/docs/getting-started-guides/meanstack.md +++ b/docs/getting-started-guides/meanstack.md @@ -1,4 +1,8 @@ --- +assignees: +- glnds +- paulzim + --- **By: Sandeep Dinesh** - _July 29, 2015_ diff --git a/docs/getting-started-guides/mesos-docker.md b/docs/getting-started-guides/mesos-docker.md index 5067c0ba72..eb9ccbc0ad 100644 --- a/docs/getting-started-guides/mesos-docker.md +++ b/docs/getting-started-guides/mesos-docker.md @@ -1,4 +1,8 @@ --- +assignees: +- jdef +- karlkfi + --- diff --git a/docs/getting-started-guides/mesos/OWNERS b/docs/getting-started-guides/mesos/OWNERS index 7cdb4cdb49..769585a3f1 100644 --- a/docs/getting-started-guides/mesos/OWNERS +++ b/docs/getting-started-guides/mesos/OWNERS @@ -1,3 +1,6 @@ assignees: - - jdef - - karlkfi +- jdef +- nak3 +- sttts +- thockin + diff --git a/docs/getting-started-guides/mesos/index.md b/docs/getting-started-guides/mesos/index.md index 059122c497..b5c6f9932f 100644 --- a/docs/getting-started-guides/mesos/index.md +++ b/docs/getting-started-guides/mesos/index.md @@ -1,4 +1,9 @@ --- +assignees: +- JamesDeFabia +- jdef +- mikebrow + --- * TOC diff --git a/docs/getting-started-guides/minikube.md b/docs/getting-started-guides/minikube.md index 5e61caa967..b2c9b8a776 100644 --- a/docs/getting-started-guides/minikube.md +++ b/docs/getting-started-guides/minikube.md @@ -1,4 +1,9 @@ --- +assignees: +- dlorenc +- janetkuo +- jlowdermilk + --- * TOC diff --git a/docs/getting-started-guides/openstack-heat.md b/docs/getting-started-guides/openstack-heat.md index ae1684bb81..f4b259a2b5 100644 --- a/docs/getting-started-guides/openstack-heat.md +++ b/docs/getting-started-guides/openstack-heat.md @@ -1,4 +1,11 @@ --- +assignees: +- elsonrodriguez +- idvoretskyi +- ohtake +- xsgordon +- zreigz + --- * TOC diff --git a/docs/getting-started-guides/ovirt.md b/docs/getting-started-guides/ovirt.md index e1ebe7e67a..d22b36fe0f 100644 --- a/docs/getting-started-guides/ovirt.md +++ b/docs/getting-started-guides/ovirt.md @@ -1,4 +1,8 @@ --- +assignees: +- caesarxuchao +- erictune + --- * TOC diff --git a/docs/getting-started-guides/photon-controller.md b/docs/getting-started-guides/photon-controller.md index c3482dc654..9878ca898a 100644 --- a/docs/getting-started-guides/photon-controller.md +++ b/docs/getting-started-guides/photon-controller.md @@ -1,4 +1,8 @@ --- +assignees: +- AlainRoy +- bprashanth + --- The example below creates a Kubernetes cluster using VMware's Photon diff --git a/docs/getting-started-guides/rackspace.md b/docs/getting-started-guides/rackspace.md index a91187275e..d71438bb1e 100644 --- a/docs/getting-started-guides/rackspace.md +++ b/docs/getting-started-guides/rackspace.md @@ -1,4 +1,8 @@ --- +assignees: +- doublerr +- erictune + --- * Supported Version: v0.18.1 diff --git a/docs/getting-started-guides/rkt/OWNERS b/docs/getting-started-guides/rkt/OWNERS new file mode 100644 index 0000000000..578ee5b700 --- /dev/null +++ b/docs/getting-started-guides/rkt/OWNERS @@ -0,0 +1,5 @@ +assignees: +- joshix +- lavalamp +- yifan-gu + diff --git a/docs/getting-started-guides/rkt/index.md b/docs/getting-started-guides/rkt/index.md index 00d5764c83..8bc25d07e3 100644 --- a/docs/getting-started-guides/rkt/index.md +++ b/docs/getting-started-guides/rkt/index.md @@ -1,4 +1,9 @@ --- +assignees: +- joshix +- lavalamp +- yifan-gu + --- This document describes how to run Kubernetes using [rkt](https://github.com/coreos/rkt) as the container runtime. diff --git a/docs/getting-started-guides/rkt/notes.md b/docs/getting-started-guides/rkt/notes.md index 30a022dc24..658d5de2a9 100644 --- a/docs/getting-started-guides/rkt/notes.md +++ b/docs/getting-started-guides/rkt/notes.md @@ -1,4 +1,8 @@ --- +assignees: +- dchen1107 +- yifan-gu + --- The following features either are not supported or have large caveats when using the rkt container runtime. Increasing support for these items and others, including reasonable feature parity with the default container engine, is planned through future releases. diff --git a/docs/getting-started-guides/scratch.md b/docs/getting-started-guides/scratch.md index eb59ff061e..a4fd03e643 100644 --- a/docs/getting-started-guides/scratch.md +++ b/docs/getting-started-guides/scratch.md @@ -1,4 +1,9 @@ --- +assignees: +- erictune +- lavalamp +- thockin + --- This guide is for people who want to craft a custom Kubernetes cluster. If you diff --git a/docs/getting-started-guides/ubuntu-calico.md b/docs/getting-started-guides/ubuntu-calico.md index 678b9b8388..bfa92be2bb 100644 --- a/docs/getting-started-guides/ubuntu-calico.md +++ b/docs/getting-started-guides/ubuntu-calico.md @@ -1,4 +1,8 @@ --- +assignees: +- alexhersh +- caseydavenport + --- This document describes how to deploy Kubernetes with Calico networking from scratch on _bare metal_ Ubuntu. For more information on Project Calico, visit [projectcalico.org](http://projectcalico.org) and the [calico-containers repository](https://github.com/projectcalico/calico-containers). diff --git a/docs/getting-started-guides/ubuntu.md b/docs/getting-started-guides/ubuntu.md index 1643841d53..236a6d9e6e 100644 --- a/docs/getting-started-guides/ubuntu.md +++ b/docs/getting-started-guides/ubuntu.md @@ -1,4 +1,9 @@ --- +assignees: +- WIZARD-CXY +- dalanlan +- thockin + --- This document describes how to deploy kubernetes on ubuntu nodes, 1 master and 3 nodes involved diff --git a/docs/getting-started-guides/vagrant.md b/docs/getting-started-guides/vagrant.md index b72649b581..09bd4a20c9 100644 --- a/docs/getting-started-guides/vagrant.md +++ b/docs/getting-started-guides/vagrant.md @@ -1,4 +1,9 @@ --- +assignees: +- brendandburns +- derekwaynecarr +- jbeda + --- **Stop. This guide has been superseded by [Minikube](../minikube/) which is the recommended method of running Kubernetes on your local machine.** diff --git a/docs/getting-started-guides/vsphere.md b/docs/getting-started-guides/vsphere.md index 0e98968bda..3b2d34d324 100644 --- a/docs/getting-started-guides/vsphere.md +++ b/docs/getting-started-guides/vsphere.md @@ -1,4 +1,8 @@ --- +assignees: +- erictune +- jbeda + --- The example below creates a Kubernetes cluster with 4 worker node Virtual diff --git a/docs/hellonode.md b/docs/hellonode.md index 146e6452f7..41da09d274 100755 --- a/docs/hellonode.md +++ b/docs/hellonode.md @@ -1,4 +1,9 @@ --- +assignees: +- dchen1107 +- juretta +- pwittrock + --- * TOC diff --git a/docs/index.md b/docs/index.md index da6febb315..84a2f5c08a 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,4 +1,8 @@ --- +assignees: +- bgrant0607 +- thockin + ---
-

Concept Overviews

+## Types of Templates +- Concept Template + - Introduce K8s Api Objects e.g. Pod +- Task Template + - Step-by-step guide for "Doing X". + - Useful for breaking down various ways of configuring Concepts into sub-topics +- Landing Pages Template + - Collection of click-able cards on a grid + - Useful for directing users to actual content from a visual Table of Contents + +# Concept Overview Template Details
{% capture instructions %} A concept overview covers the most essential, important information about core Kubernetes concepts and features. Examples of Concepts include `Pod`, `Deployment`, `Service`, etc. -### Example +## Reference Examples -- [Source Link](https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/master/docs/templatedemos/filledout.md) -- [Rendered Link](filledout/) +- [Link to Example Template](https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/master/docs/templatedemos/filledout.md) +- [Link to Rendered Template](filledout/) -### Usage +## Usage -To create a new concept overview page, create a new file under -`docs/your-concept-name/index.md`. +### Creating the file -The file should start with the set of tags to apply to the concept and end -with an `include templates/concept-overview.md` block. +To create a new concept overview page, create a new directory with the concept +name under the docs directory and an index.md file. +e.g. `docs/your-concept-name/index.md`. -The file needs to have the following `capture` sections: +### Adding the Template sections -- concept -- when_to_use -- when_not_to_use -- status -- usage +- tags ```liquid{% raw %} --- -add +include tags -here -e.g. -pod --- -{% capture concept %} -Name of the Concept goes here (e.g. Pod) -{% endcapture %} +``` -{% capture what_is %} -1 sentence description of the Concept that describes the function it performs. -If needed: -- add -- clarifying -- bullet points +- concept: the concept name e.g. Pod -![Also add a diagram](/images/docs/file.svg){: style="max-width: 25%" } +``` # +{% capture concept %}{% endcapture %} +``` -{% comment %} -Include a comment indicating where an editable image exists: -https://drive.google.com/open?id=1pQe4-s76fqyrzB8f3xoJo4MPLNVoBlsE1tT9MyLNINg -{% endcomment %} -{% endcapture %} +- what_is: one sentence description the function / role of the concept. Diagrams are helpful. -{% capture when_to_use %} -Clarify precisely when and how to use this Concept. Disambiguate when to use -this instead of similar concepts. e.g. Secret vs ConfigMap, -Deployment vs Job vs other controllers, Service vs Ingress. +``` # +{% capture what_is %}{% endcapture %} +``` -| Tables | Can Be | -|------------|----------------| -| Helpful | When | -| Comparing | Functionality | -| Across | Concepts | -{% endcapture %} +- when_to_use: disambiguate when to use this vs alternatives -{% capture when_not_to_use %} -Call out common anti-patterns to be avoided. -{% endcapture %} +``` # +{% capture when_to_use %}{% endcapture %} +``` -{% capture status %} -Describe how to get the current status of an instance in a cluster using kubectl +- when_not_to_use: highlight common anti-patterns -Use a table to clarify the response. +``` # +{% capture when_not_to_use %}{% endcapture %} +``` -Consider status from kubectl describe as well. -{% endcapture %} +- status: how to get the status for this object using kubectl -{% capture usage %} -If the concept interacts with or is embedded in other Concepts, clarify that here. -For Services this should call out the meaning of the label selectors, for Volumes -this should call out that they are defined within a Pod. +``` # +{% capture status %}{% endcapture %} +``` -Include the example yaml here -{% endcapture %} +- usage: example yaml +``` # +{% capture usage %}{% endcapture %} + +``` + +- template: include the template at the end + +``` # {% include templates/concept-overview.md %} {% endraw %}``` -### Adding page to navigation +### Adding the page to navigation Once your page is saved, somewhere in the `/docs/` directory, add a reference to the `concepts.yml` file under `/_data/` so that it will appear in the left-hand navigation of the site. This is also where you add a title to the page. @@ -141,46 +133,74 @@ Once your page is saved, somewhere in the `/docs/` directory, add a reference to
- -

Task

+# Task Template Details
{% capture instructions %} A task page offers step-by-step instructions for completing a task with Kubernetes. **A task page should be narrowly focused on task completion and not delve into concepts or reference information.** -### Demos - -- [Blank](blanktask/) ([Source](https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/master/docs/templatedemos/blanktask.md)) -- [Filled Out](task/) ([Source](https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/master/docs/templatedemos/task.md)) +## Example +- [Link to Example Template](https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/master/docs/templatedemos/task.md) +- [Link to Rendered Page](task/) ### Usage +## Creating the file + +To create a new task page, create a file under docs/tasks/task-name. +e.g. `docs/tasks/your-task-name`. + +### Adding the Template sections + +- tags + ```liquid{% raw %} --- +tags --- +``` + +- purpose: one sentence description of the task and motivation + +``` # {% capture purpose %}{% endcapture %} +``` + +- recommended_background: List of Concepts referenced or other Tasks, Tutorials that provide needed context + +``` # {% capture recommended_background %}{% endcapture %} +``` + +- set_by_step: Add multiple sections. 1 per step in the task. + +``` # {% capture step_by_step %}{% endcapture %} +``` + +- template: include the template at the end + +``` # {% include templates/task.md %} {% endraw %}``` -### Adding page to navigation +### Adding the page to navigation -Once your page is saved, somewhere in the `/docs/` directory, add a reference to the `tasks.yml` file under `/_data/` so that it will appear in the left-hand navigation of the site. This is also where you add a title to the page. +Add a reference to the `tasks.yml` file under `/_data/` so that it will appear in the left-hand navigation of the site. This is also where you add a title to the page. {% endcapture %} {{ instructions | markdownify }}
-

Landing Pages

+# Landing Pages
{% capture instructions %} Landing pages are a set of clickable "cards" arranged in a grid. Each card has a heading and description, and optioninall, a thumbnail image. They are meant to be index pages that quickly forward users on to deeper content. ### Demos -- [Blank](blanklanding/) ([Source](https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/master/docs/templatedemos/blanklanding.md)) -- [Filled Out](landingpage/) ([Source](https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/master/docs/templatedemos/landingpage.md)) +- [Link to Example Landing Page](https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/master/docs/templatedemos/landingpage.md) +- [Link to Rendered Landing Page](landingpage/) ### Usage @@ -209,14 +229,14 @@ Once your page is saved, somewhere in the `/docs/` directory, add a reference to
-

kubectl yaml

+# kubectl yaml
{% capture instructions %} You probably shouldn't be using this, but we also have templates which consume YAML files that are generated by the Kubernetes authors. These are turned into pages which display the reference information for the various CLI tools. ### Demos -- [Blank](blankkubectl/) ([Source](https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/master/docs/templatedemos/blankkubectl.md)) -- [Filled Out](kubectl/) ([Source](https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/master/docs/templatedemos/kubectl.md)) +- [Link to Example Template](https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/master/docs/templatedemos/kubectl.md) +- [Link to Rendered Template](kubectl/) ### Adding page to navigation From ba0fe6e81cf851dfbe8268ff70daced42448719b Mon Sep 17 00:00:00 2001 From: Phillip Wittrock Date: Fri, 24 Jun 2016 16:34:44 -0700 Subject: [PATCH 118/348] clean up template how to docs --- docs/templatedemos/index.md | 70 ++++++------------------------------- 1 file changed, 11 insertions(+), 59 deletions(-) diff --git a/docs/templatedemos/index.md b/docs/templatedemos/index.md index 73167fc445..d3d31d9104 100644 --- a/docs/templatedemos/index.md +++ b/docs/templatedemos/index.md @@ -73,54 +73,23 @@ e.g. `docs/your-concept-name/index.md`. ### Adding the Template sections - tags +- concept: the concept name e.g. Pod +- what_is: one sentence description the function / role of the concept. Diagrams are helpful. +- when_to_use: disambiguate when to use this vs alternatives +- when_not_to_use: highlight common anti-patterns +- status: how to get the status for this object using kubectl +- usage: example yaml +- template: include the template at the end ```liquid{% raw %} --- -include -tags --- -``` - -- concept: the concept name e.g. Pod - -``` # {% capture concept %}{% endcapture %} -``` - -- what_is: one sentence description the function / role of the concept. Diagrams are helpful. - -``` # {% capture what_is %}{% endcapture %} -``` - -- when_to_use: disambiguate when to use this vs alternatives - -``` # {% capture when_to_use %}{% endcapture %} -``` - -- when_not_to_use: highlight common anti-patterns - -``` # {% capture when_not_to_use %}{% endcapture %} -``` - -- status: how to get the status for this object using kubectl - -``` # {% capture status %}{% endcapture %} -``` - -- usage: example yaml - -``` # {% capture usage %}{% endcapture %} - -``` - -- template: include the template at the end - -``` # {% include templates/concept-overview.md %} {% endraw %}``` @@ -152,34 +121,17 @@ e.g. `docs/tasks/your-task-name`. ### Adding the Template sections - tags +- purpose: one sentence description of the task and motivation +- recommended_background: List of Concepts referenced or other Tasks, Tutorials that provide needed context +- set_by_step: Add multiple sections. 1 per step in the task. +- template: include the template at the end ```liquid{% raw %} --- -tags --- -``` - -- purpose: one sentence description of the task and motivation - -``` # {% capture purpose %}{% endcapture %} -``` - -- recommended_background: List of Concepts referenced or other Tasks, Tutorials that provide needed context - -``` # {% capture recommended_background %}{% endcapture %} -``` - -- set_by_step: Add multiple sections. 1 per step in the task. - -``` # {% capture step_by_step %}{% endcapture %} -``` - -- template: include the template at the end - -``` # {% include templates/task.md %} {% endraw %}``` From c6a64fce1c4ddcffc5b7d4249948814fc3d7f3ef Mon Sep 17 00:00:00 2001 From: Phillip Wittrock Date: Fri, 24 Jun 2016 18:39:14 -0700 Subject: [PATCH 119/348] Add metadata section to example template bodies --- docs/templatedemos/index.md | 41 ++++++++++++++++++++++++++++--------- 1 file changed, 31 insertions(+), 10 deletions(-) diff --git a/docs/templatedemos/index.md b/docs/templatedemos/index.md index d3d31d9104..bc63f807d9 100644 --- a/docs/templatedemos/index.md +++ b/docs/templatedemos/index.md @@ -83,13 +83,23 @@ e.g. `docs/your-concept-name/index.md`. ```liquid{% raw %} --- +objects: +- list_of_concepts +- fields_and_tools +- that_are_the_focus +concepts: +- list_of_concepts: 10 +- fields_and_tools: 20 +- that_are_used: 30 +- with_a_sort_order: 15 +advanced: false --- -{% capture concept %}{% endcapture %} -{% capture what_is %}{% endcapture %} -{% capture when_to_use %}{% endcapture %} -{% capture when_not_to_use %}{% endcapture %} -{% capture status %}{% endcapture %} -{% capture usage %}{% endcapture %} +{% capture concept %} concept-name-here {% endcapture %} +{% capture what_is %} description-of-concept-here {% endcapture %} +{% capture when_to_use %} when-to-user-here {% endcapture %} +{% capture when_not_to_use %} anti-patterns-here {% endcapture %} +{% capture status %} how-to-get-with-kubectl-here {% endcapture %} +{% capture usage %} yaml-config-usage-here {% endcapture %} {% include templates/concept-overview.md %} {% endraw %}``` @@ -120,7 +130,7 @@ e.g. `docs/tasks/your-task-name`. ### Adding the Template sections -- tags +- metadata: structured description of the doc content - purpose: one sentence description of the task and motivation - recommended_background: List of Concepts referenced or other Tasks, Tutorials that provide needed context - set_by_step: Add multiple sections. 1 per step in the task. @@ -128,10 +138,21 @@ e.g. `docs/tasks/your-task-name`. ```liquid{% raw %} --- +synopsis: "one sentence description of task." +objects: +- list_of_concepts +- fields_and_tools +- that_are_the_focus +concepts: +- list_of_concepts: 10 +- fields_and_tools: 20 +- that_are_used: 30 +- with_a_sort_order: 15 +advanced: false --- -{% capture purpose %}{% endcapture %} -{% capture recommended_background %}{% endcapture %} -{% capture step_by_step %}{% endcapture %} +{% capture purpose %} task-description-here {% endcapture %} +{% capture recommended_background %} prereq-reading-here {% endcapture %} +{% capture step_by_step %} single-step-here {% endcapture %} {% include templates/task.md %} {% endraw %}``` From 429028dc54920e124187502e42a80c195f7bf1b5 Mon Sep 17 00:00:00 2001 From: Tolleiv Nietsch Date: Sat, 25 Jun 2016 13:04:54 +0200 Subject: [PATCH 120/348] Add some missing PVs and a capability overview --- docs/user-guide/persistent-volumes/index.md | 24 ++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/docs/user-guide/persistent-volumes/index.md b/docs/user-guide/persistent-volumes/index.md index ae465eea00..1c4bb150f5 100644 --- a/docs/user-guide/persistent-volumes/index.md +++ b/docs/user-guide/persistent-volumes/index.md @@ -51,10 +51,15 @@ The reclaim policy for a `PersistentVolume` tells the cluster what to do with th * GCEPersistentDisk * AWSElasticBlockStore +* AzureFile +* FC (Fibre Channel) * NFS * iSCSI * RBD (Ceph Block Device) +* CephFS +* Cinder (OpenStack block storage) * Glusterfs +* VsphereVolume * HostPath (single node testing only -- local storage is not supported in any way and WILL NOT WORK in a multi-node cluster) @@ -86,7 +91,7 @@ Currently, storage size is the only resource that can be set or requested. Futu ### Access Modes -A `PersistentVolume` can be mounted on a host in any way supported by the resource provider. Providers will have different capabilities and each PV's access modes are set to the specific modes supported by that particular volume. For example, NFS can support multiple read/write clients, but a specific NFS PV might be exported on the server as read-only. Each PV gets its own set of access modes describing that specific PV's capabilities. +A `PersistentVolume` can be mounted on a host in any way supported by the resource provider. As shown in the table below, providers will have different capabilities and each PV's access modes are set to the specific modes supported by that particular volume. For example, NFS can support multiple read/write clients, but a specific NFS PV might be exported on the server as read-only. Each PV gets its own set of access modes describing that specific PV's capabilities. The access modes are: @@ -103,6 +108,23 @@ In the CLI, the access modes are abbreviated to: > __Important!__ A volume can only be mounted using one access mode at a time, even if it supports many. For example, a GCEPersistentDisk can be mounted as ReadWriteOnce by a single node or ReadOnlyMany by many nodes, but not at the same time. +| | ReadWriteOnce| ReadOnlyMany| ReadWriteMany| +| :--- | :---: | :---: | :---: | +| AWSElasticBlockStore | x | - | - | +| AzureFile | x | x | x | +| CephFS | x | x | x | +| Cinder | x | - | - | +| FC | x | x | - | +| FlexVolume | x | x | - | +| GCEPersistentDisk | x | x | - | +| Glusterfs | x | x | x | +| HostPath | x | - | - | +| iSCSI | x | x | - | +| NFS | x | x | x | +| RDB | x | x | - | +| VsphereVolume | x | - | - | + + ### Recycling Policy Current recycling policies are: From ac34a08d665fbd8fb2b37d7106a47675d8d792fe Mon Sep 17 00:00:00 2001 From: Johannes Scheuermann Date: Wed, 15 Jun 2016 09:43:00 +0200 Subject: [PATCH 121/348] Add Foreman CoreOS --- docs/getting-started-guides/coreos/index.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/getting-started-guides/coreos/index.md b/docs/getting-started-guides/coreos/index.md index e56ebd16e0..46d058603d 100644 --- a/docs/getting-started-guides/coreos/index.md +++ b/docs/getting-started-guides/coreos/index.md @@ -82,6 +82,12 @@ Guide to running an HA etcd cluster with a single master on Azure. Uses the Azur Configure a single master, single worker cluster on VMware ESXi. +
+ +[**Single/Multi-node cluster using cloud-config, CoreOS and Foreman**](https://github.com/johscheuer/theforeman-coreos-kubernetes) + +Configure a standalone Kubernetes or a Kubernetes cluster with [Foreman](https://theforeman.org). + ## Support Level From 5f448b86fe058792fe06e7856cf1985ac55e13af Mon Sep 17 00:00:00 2001 From: John Mulhausen Date: Sun, 26 Jun 2016 00:17:59 -0700 Subject: [PATCH 122/348] Refactor of template demo instructions --- _data/support.yml | 2 + docs/sitemap.md | 114 +++++++++++++++++++++++ docs/templatedemos/index.md | 176 ++++++++++++++++++++---------------- 3 files changed, 215 insertions(+), 77 deletions(-) create mode 100644 docs/sitemap.md diff --git a/_data/support.yml b/_data/support.yml index 4ca3eeff0a..4ca3b339c8 100644 --- a/_data/support.yml +++ b/_data/support.yml @@ -40,3 +40,5 @@ toc: path: https://github.com/kubernetes/kubernetes/milestones/ - title: Contributing to Kubernetes Documentation path: /editdocs/ + - title: New Template Instructions + path: /docs/templatedemos/ diff --git a/docs/sitemap.md b/docs/sitemap.md new file mode 100644 index 0000000000..a65bd4f01b --- /dev/null +++ b/docs/sitemap.md @@ -0,0 +1,114 @@ +--- +--- + + + +Click tags or use the drop downs to filter. Click table headers to sort or reverse sort. + +

+Filter by Concept:
+Filter by Object:
+Filter by Command: +

+ +
diff --git a/docs/templatedemos/index.md b/docs/templatedemos/index.md index bc63f807d9..bbb883fe9b 100644 --- a/docs/templatedemos/index.md +++ b/docs/templatedemos/index.md @@ -1,28 +1,28 @@ --- --- - -# Before you Begin: Get the docs code checked out locally +* TOC +{:toc} + +## Before you Begin: Get the docs code checked out locally Check out the kubernetes/kubernetes.github.io repo and the docsv2 branch. ### Step 1: Fork and Clone the repo + - Fork [kubernetes/kubernetes.github.io](https://github.com/kubernetes/kubernetes.github.io) - [Setup your GitHub authentication using ssh](https://help.github.com/articles/generating-an-ssh-key/) - Clone the repo under ~/go/src/k8s.io ```shell cd ~/go/src/k8s.io -git clone git@github.com:/kubernetes.github.io +git clone git@github.com:/kubernetes.github.io cd kubernetes.github.io git remote add upstream https://github.com/kubernetes/kubernetes.github.io.git ``` ### Step 2: Switch to the docsv2 branch + Docs v2 development is being performed in the `docsv2` branch. This is the branch you want to be working from. @@ -38,9 +38,19 @@ git reset --hard upstream/docsv2 - [Follow the editdocs instructions](http://kubernetes.io/editdocs/) -# Writing Docs Using Templates +Or, just use Docker! Run this from within your `kubernetes.github.io` directory and you should be good: + +```shell +docker run -ti --rm -v "$PWD":/k8sdocs -p 4000:4000 johndmulhausen/k8sdocs +``` + +The site will then be viewable at [http://localhost:4000/](http://localhost:4000/). + + +## Writing Docs Using Templates + +### Types of Templates -## Types of Templates - Concept Template - Introduce K8s Api Objects e.g. Pod - Task Template @@ -50,19 +60,18 @@ git reset --hard upstream/docsv2 - Collection of click-able cards on a grid - Useful for directing users to actual content from a visual Table of Contents -# Concept Overview Template Details -
{% capture instructions %} +## Concept Overview Template Details A concept overview covers the most essential, important information about core Kubernetes concepts and features. Examples of Concepts include `Pod`, `Deployment`, `Service`, etc. -## Reference Examples +### Reference Examples -- [Link to Example Template](https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/master/docs/templatedemos/filledout.md) -- [Link to Rendered Template](filledout/) +- [Link to Example Template: Source](https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/docsv2/docs/pod/index.md) +- [Link to Example Template: Rendered](http://k8sdocs.github.io/docs/pod/) -## Usage +### Usage ### Creating the file @@ -70,9 +79,12 @@ To create a new concept overview page, create a new directory with the concept name under the docs directory and an index.md file. e.g. `docs/your-concept-name/index.md`. +### Adding the page to navigation + +Once your page is saved, somewhere in the `/docs/` directory, add a reference to the `concepts.yml` file under `/_data/` so that it will appear in the left-hand navigation of the site. This is also where you add a title to the page. + ### Adding the Template sections -- tags - concept: the concept name e.g. Pod - what_is: one sentence description the function / role of the concept. Diagrams are helpful. - when_to_use: disambiguate when to use this vs alternatives @@ -81,53 +93,66 @@ e.g. `docs/your-concept-name/index.md`. - usage: example yaml - template: include the template at the end +### Tags structure + +- `object_rankings:` associates the page with API objects/functions. +- `concept_rankings:` associates the page with Kubernetes concepts. +- `command_rankings:` associates the page with CLI commands + +In each case, the association is ranked. If ranked "1," the topic will surface as a "Core Topic" (of high importance) on various associated pages. If ranked "2," the topic will be grouped under "Advanced Topics," which are deemed less essential. + +Only ranks 1 and 2 are supported. + +Tags are mandatory and should be thorough; they are the connective tissue of the site. To see them in action, [visit our sitemap](http://k8sdocs.github.io/docs/sitemap/). + + ```liquid{% raw %} --- -objects: -- list_of_concepts -- fields_and_tools -- that_are_the_focus -concepts: -- list_of_concepts: 10 -- fields_and_tools: 20 -- that_are_used: 30 -- with_a_sort_order: 15 -advanced: false +glossary: A pod is the vehicle for running containers in Kubernetes. +object_rankings: +- object: pod + rank: 1 +concept_rankings: +- concept: pod + rank: 1 +command_rankings: +- command: kubect describe + rank: 1 +- command: kubectl get + rank: 1 --- {% capture concept %} concept-name-here {% endcapture %} {% capture what_is %} description-of-concept-here {% endcapture %} -{% capture when_to_use %} when-to-user-here {% endcapture %} +{% capture when_to_use %} when-to-use-here {% endcapture %} {% capture when_not_to_use %} anti-patterns-here {% endcapture %} {% capture status %} how-to-get-with-kubectl-here {% endcapture %} {% capture usage %} yaml-config-usage-here {% endcapture %} {% include templates/concept-overview.md %} {% endraw %}``` -### Adding the page to navigation -Once your page is saved, somewhere in the `/docs/` directory, add a reference to the `concepts.yml` file under `/_data/` so that it will appear in the left-hand navigation of the site. This is also where you add a title to the page. - -{% endcapture %} -{{ instructions | markdownify }} - -
- -# Task Template Details -
{% capture instructions %} +## Task Template Details A task page offers step-by-step instructions for completing a task with Kubernetes. **A task page should be narrowly focused on task completion and not delve into concepts or reference information.** -## Example -- [Link to Example Template](https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/master/docs/templatedemos/task.md) -- [Link to Rendered Page](task/) +### Example + +- [Link to Example Template: Source](https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/docsv2/docs/tasks/explicitly-scheduling-pod-node.md) +- [Link to Example Template: Rendered](http://k8sdocs.github.io/docs/tasks/explicitly-scheduling-pod-node/) ### Usage -## Creating the file +### Creating the file To create a new task page, create a file under docs/tasks/task-name. e.g. `docs/tasks/your-task-name`. +Task filenames should match the title, chaining words with dashes in all lowercase, omitting articles and prepositions. For example, the topic "Explictly Scheduling a Pod on a Node" is stored in file `/docs/tasks/explicitly-scheduling-pod-node.md`. + +### Adding the page to navigation + +Add a reference to the `tasks.yml` file under `/_data/` so that it will appear in the left-hand navigation of the site. This is also where you add a title to the page. + ### Adding the Template sections - metadata: structured description of the doc content @@ -136,19 +161,36 @@ e.g. `docs/tasks/your-task-name`. - set_by_step: Add multiple sections. 1 per step in the task. - template: include the template at the end +### Tags structure + +- `object_rankings:` associates the page with API objects/functions. +- `concept_rankings:` associates the page with Kubernetes concepts. +- `command_rankings:` associates the page with CLI commands + +In each case, the association is ranked. If ranked "1," the topic will surface as a "Core Topic" (of high importance) on various associated pages. If ranked "2," the topic will be grouped under "Advanced Topics," which are deemed less essential. + +Only ranks 1 and 2 are supported. + +Tags are mandatory and should be thorough; they are the connective tissue of the site. To see them in action, [visit our sitemap](http://k8sdocs.github.io/docs/sitemap/). + + ```liquid{% raw %} --- -synopsis: "one sentence description of task." -objects: -- list_of_concepts -- fields_and_tools -- that_are_the_focus -concepts: -- list_of_concepts: 10 -- fields_and_tools: 20 -- that_are_used: 30 -- with_a_sort_order: 15 -advanced: false +object_rankings: +- object: nodeAffinity + rank: 1 +- object: nodeSelector + rank: 2 +concept_rankings: +- concept: node + rank: 1 +- concept: pod + rank: 1 +command_rankings: +- command: kubectl label + rank: 1 +- command: kubectl get + rank: 2 --- {% capture purpose %} task-description-here {% endcapture %} {% capture recommended_background %} prereq-reading-here {% endcapture %} @@ -156,17 +198,8 @@ advanced: false {% include templates/task.md %} {% endraw %}``` -### Adding the page to navigation -Add a reference to the `tasks.yml` file under `/_data/` so that it will appear in the left-hand navigation of the site. This is also where you add a title to the page. - -{% endcapture %} -{{ instructions | markdownify }} - -
- -# Landing Pages -
{% capture instructions %} +## Landing Pages Landing pages are a set of clickable "cards" arranged in a grid. Each card has a heading and description, and optioninall, a thumbnail image. They are meant to be index pages that quickly forward users on to deeper content. @@ -196,27 +229,16 @@ cards: Once your page is saved, somewhere in the `/docs/` directory, add a reference to the appropriate .yml file under `/_data/` so that it will appear in the left-hand navigation of the site. This is also where you add a title to the page. -{% endcapture %} -{{ instructions | markdownify }} -
+## kubectl yaml - -# kubectl yaml -
{% capture instructions %} You probably shouldn't be using this, but we also have templates which consume YAML files that are generated by the Kubernetes authors. These are turned into pages which display the reference information for the various CLI tools. ### Demos -- [Link to Example Template](https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/master/docs/templatedemos/kubectl.md) -- [Link to Rendered Template](kubectl/) +- [Link to Example Template](https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/docsv2/docs/kubectl/kubectl_api-versions.md) +- [Link to Rendered Template](http://k8sdocs.github.io/docs/kubectl/kubectl_api-versions/) ### Adding page to navigation -Once your page is saved, somewhere in the `/docs/` directory, add a reference to the `concepts.yml` file under `/_data/` so that it will appear in the left-hand navigation of the site. This is also where you add a title to the page. - -{% endcapture %} -{{ instructions | markdownify }} - -
-
+Once your page is saved, somewhere in the `/docs/` directory, add a reference to the `reference.yml` file under `/_data/` so that it will appear in the left-hand navigation of the site. This is also where you add a title to the page. \ No newline at end of file From a7eb673326f0cb17878904711a17528da0033544 Mon Sep 17 00:00:00 2001 From: John Mulhausen Date: Sun, 26 Jun 2016 00:29:02 -0700 Subject: [PATCH 123/348] Nips/tucks to templatedemos --- docs/templatedemos/index.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/templatedemos/index.md b/docs/templatedemos/index.md index bbb883fe9b..7bb0a564bd 100644 --- a/docs/templatedemos/index.md +++ b/docs/templatedemos/index.md @@ -34,11 +34,11 @@ git fetch upstream git reset --hard upstream/docsv2 ``` -### Step 3: Make sure you can serve rendered docs locally from your dev box +### Step 3: Make sure you can serve rendered docs -- [Follow the editdocs instructions](http://kubernetes.io/editdocs/) +One option is to simply rename your fork's repo on GitHub.com to `yourusername.github.io`, which will auto-stage your commits at that URL. -Or, just use Docker! Run this from within your `kubernetes.github.io` directory and you should be good: +Or, just use Docker! Run this from within your local `kubernetes.github.io` directory and you should be good: ```shell docker run -ti --rm -v "$PWD":/k8sdocs -p 4000:4000 johndmulhausen/k8sdocs @@ -46,6 +46,7 @@ docker run -ti --rm -v "$PWD":/k8sdocs -p 4000:4000 johndmulhausen/k8sdocs The site will then be viewable at [http://localhost:4000/](http://localhost:4000/). +Or, you can [follow the instructions](/editdocs/) for running a from-scratch staging server, which is both the most performant option and the biggest pain to get set up. ## Writing Docs Using Templates @@ -95,6 +96,7 @@ Once your page is saved, somewhere in the `/docs/` directory, add a reference to ### Tags structure +- `glossary:` a brief (~140 character) definition of what this concept is. - `object_rankings:` associates the page with API objects/functions. - `concept_rankings:` associates the page with Kubernetes concepts. - `command_rankings:` associates the page with CLI commands From 35052454087d01ecefe7c34ff92ff18d0a6b0e38 Mon Sep 17 00:00:00 2001 From: Elson Rodriguez Date: Sun, 26 Jun 2016 20:21:27 -0400 Subject: [PATCH 124/348] Adding notices regarding Kubernetes 1.3 --- docs/getting-started-guides/openstack-heat.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/getting-started-guides/openstack-heat.md b/docs/getting-started-guides/openstack-heat.md index 5cc19d85ee..44bffa2f13 100644 --- a/docs/getting-started-guides/openstack-heat.md +++ b/docs/getting-started-guides/openstack-heat.md @@ -88,12 +88,12 @@ Please see the contents of these files for documentation regarding each variable ## Starting a cluster -Once you've installed the OpenStack CLI tools and have set your OpenStack environment variables, issue this command: +Once Kubernetes version 1.3 is released, and you've installed the OpenStack CLI tools and have set your OpenStack environment variables, issue this command: ```sh export KUBERNETES_PROVIDER=openstack-heat; curl -sS https://get.k8s.io | bash ``` -Alternatively, you can download a [Kubernetes release](https://github.com/kubernetes/kubernetes/releases) and extract the archive. To start your cluster, open a shell and run: +Alternatively, you can download a [Kubernetes release](https://github.com/kubernetes/kubernetes/releases) of version 1.3 or higher and extract the archive. To start your cluster, open a shell and run: ```sh cd kubernetes # Or whichever path you have extracted the release to From 99f6734342aae44b888b666ea801db678a94089d Mon Sep 17 00:00:00 2001 From: Prashanth Balasubramanian Date: Tue, 21 Jun 2016 10:25:46 -0700 Subject: [PATCH 125/348] Petset 1.3 user guide --- _data/reference.yml | 2 + docs/user-guide/petset.md | 281 ++++++++++++++++++++++++++++++++++++ docs/user-guide/petset.yaml | 56 +++++++ 3 files changed, 339 insertions(+) create mode 100644 docs/user-guide/petset.md create mode 100644 docs/user-guide/petset.yaml diff --git a/_data/reference.yml b/_data/reference.yml index 3b9b864cc7..e2439cc78b 100644 --- a/_data/reference.yml +++ b/_data/reference.yml @@ -224,6 +224,8 @@ toc: path: /docs/admin/resource-quota/ - title: Replica Sets path: /docs/user-guide/replicasets/ + - title: Pet Sets + path: /docs/user-guide/petset/ - title: Kubernetes Design Docs section: diff --git a/docs/user-guide/petset.md b/docs/user-guide/petset.md new file mode 100644 index 0000000000..324e1f1764 --- /dev/null +++ b/docs/user-guide/petset.md @@ -0,0 +1,281 @@ +--- +--- + +* TOC +{:toc} + +__Terminology__ + +Throughout this doc you will see a few terms that are sometimes used interchangeably elsewhere, that might cause confusion. This section attempts to clarify them. + +* Node: A single virtual or physical machine in a Kubernetes cluster. +* Cluster: A group of nodes in a single failure domain, unless mentioned otherwise. +* Persistent Volume Claim (PVC): A request for storage, typically a [persistent volume](/docs/user-guide/persistent-volumes/walkthrough/). +* Host name: The hostname attached to the UTS namespace of the pod, i.e the output of `hostname` in the pod. +* DNS/Domain name: A *cluster local* domain name resolvable using standard methods (eg: [gethostbyname](http://linux.die.net/man/3/gethostbyname)). +* Ordinality: the proprety of being "ordinal", or occupying a position in a sequence. +* Pet: a single member of a Pet Set; more generally, a stateful application. +* Peer: a process running a server, capable of communicating with other such processes. + +__Prerequisites__ + +This doc assumes familiarity with the following Kubernetes concepts: + +* [Pods](/docs/user-guide/pods/single-container/) +* [Cluster DNS](/docs/admin/dns/) +* [Headless Services](/docs/user-guide/services/#headless-services) +* [Persistent Volumes](/docs/user-guide/volumes/) +* [Dynamic volume provisioning](http://releases.k8s.io/{{page.githubbranch}}/examples/experimental/persistent-volume-provisioning/README.md) + +You need a working Kubernetes cluster at version >= 1.3, with a healthy DNS [cluster addon](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/README.md) at version >= 15. You cannot use Pet Set on a hosted Kubernetes provider that has disabled `alpha` resources. + +## What is a Pet Set? + +In Kubernetes, most pod management abstractions group them into disposable units of work that compose a micro service. Replication controllers for example, are designed with a weak guarantee - that there should be N replicas of a particular pod template. The pods are treated as stateless units, if one of them is unhealthy or superseded by a newer version, the system just disposes it. + +``` + foo.default.svc.cluster.local + |service| + / \ + | pod-asdf | | pod-zxcv | +``` + +A Pet Set, in contrast, is a group of stateful pods that require a stronger notion of identity. The document refers to these as "clustered applications". + +``` + *.foo.default.svc.cluster.local + | mysql-0 | <-> | mysql-1 | + [pv 0] [pv 1] +``` + +The co-ordinated deployment of clustered applications is notoriously hard. They require stronger notions of identity and membership, which they use in opaque internal protocols, and are especially prone to race conditions and deadlock. Traditionally administrators have deployed these applications by leveraging nodes as stable, long-lived entities with persistent storage and static ips. + +The goal of Pet Set is to decouple this dependency by assigning identities to individual instances of an application that are not anchored to the underlying physical infrastructure. For the rest of this document we will refer to these entities as "Pets". Our use of this term is predated by the "Pets vs Cattle" analogy. + +__Relationship between Pets and Pods__: PetSet requires there be {0..N-1} Pets. Each Pet has a deterministic name - PetSetName-Ordinal, and a unique identity. Each Pet has at most one pod, and each Pet Set has at most one Pet with a given identity. + +## When to use Pet Set? + +A Pet Set ensures that a specified number of "pets" with unique identities are running at any given time. The identity of a Pet is comprised of: + +* a stable hostname, available in DNS +* an ordinal index +* stable storage: linked to the ordinal & hostname + +These properties are useful in deploying stateful applications. However most stateful applications are also clustered, meaning they form groups with strict membership requirements that rely on stored state. Pet Set also helps with the 2 most common problems encountered managing such clustered applications: + +* discovery of peers for quorum +* startup/teardown ordering + +Only use Pet Set if your application requires some or all of these properties. Managing pods as stateless replicas is vastly easier. + +Example workloads for Pet Set: + +* Databases like MySQL or PostgreSQL that require a single instance attached to a NFS persistent volume at any time +* Clustered software like Zookeeper, Etcd, or Elasticsearch that require stable membership. + +## Alpha limitations + +Before you start deploying applications as Pet Sets, there are a few limitations you should understand. + +* Pet Set is an *alpha* resource, not available in any Kubernetes release prior to 1.3. +* As with all alpha/beta resources, it can be disable through the `--runtime-config` option passed to the apiserver, and in fact most likely will be disabled on hosted offerings of Kubernetes. +* The only updatable field on a Pet Set is `replicas` +* The storage for a given pet must either be provisioned by a [dynamic storage provisioner](http://releases.k8s.io/{{page.githubbranch}}/examples/experimental/persistent-volume-provisioning/README.md) based on the requested `storage class`, or pre-provisioned by an admin. Note that dynamic volume provisioning is also currently in alpha. +* Deleting the Pet Set *will not* delete any pets. You will either have to manually scale it down to 0 pets first, or delete the pets yourself. +* Deleting and/or scaling a Pet Set down will *not* delete the volumes associated with the Pet Set. This is done to ensure safety first, your data is more valuable than an auto purge of all related Pet Set resources. **Deleting the Persistent Volume Claims will result in a deletion of the associated volumes**. +* All Pet Sets currently require a "governing service", or a Service responsible for the network identity of the pets. The user is responsible for this Service. +* Updating an existing Pet Set is currently a manual process, meaning you either need to deploy a new Pet Set with the new image version, or orphan Pets one by one, update their image, and join them back to the cluster. + +## Example Pet Set + +We'll create a basic Pet Set to demonstrate how Pets are assigned unique and "sticky" identities. + +{% include code.html language="yaml" file="petset.yaml" ghlink="/docs/user-guide/petset.yaml" %} + +Saving this config into `petset.yaml` and submitting it to a Kubernetes cluster should creae the defined Pet Set and Pets it manages: + +```shell +$ kubectl create -f petset.yaml +service "nginx" created +petset "nginx" deleted +``` + +## Pet Identity + +The identity of a Pet sticks to it, regardless of which node it's (re) scheduled on. We can examine the identity of the pets we just created. + +### Ordinal index + +you should see 2 pods with predictable names formatted thus: `$(petset name)-$(ordinal index assigned by petset controller)` + +```shell +$ kubectl get po +NAME READY STATUS RESTARTS AGE +web-0 1/1 Running 0 10m +web-1 1/1 Running 0 10m +``` + +### Stable storage + +2 persistent volumes, one per pod. This is auto created by the Pet Set based on the `volumeTemplate` field + +```shell +$ kubectl get pv +NAME CAPACITY ACCESSMODES STATUS CLAIM REASON AGE +pvc-90234946-3717-11e6-a46e-42010af00002 1Gi RWO Bound default/www-web-0 11m +pvc-902733c2-3717-11e6-a46e-42010af00002 1Gi RWO Bound default/www-web-1 11m +``` + +### Network identity + +The network identity has 2 parts. First, we created a headless Service that controls the domain within which we create Pets. The domain managed by this Service takes the form: `$(service name).$(namespace).svc.cluster.local`, where "cluster.local" is the [cluster domain](http://releases.k8s.io/{{page.githubbranch}}/build/kube-dns/README.md#how-do-i-configure-it). As each pet is created, it gets a matching DNS subdomain, taking the form: `$(petname).$(governing service domain)`, where the governing service is defined by the `serviceName` field on the Pet Set. + +Cluster Domain | Service (ns/name) | Pet Set (ns/name) | Pet Set Domain | Pet DNS | Pet Hostname | +-------------- | ----------------- | ----------------- | -------------- | ------- | ------------ | + cluster.local | default/nginx | default/web | nginx.default.svc.cluster.local | web-{0..N-1}.nginx.default.svc.cluster.local | web-{0..N-1} | + cluster.local | foo/nginx | foo/web | nginx.foo.svc.cluster.local | web-{0..N-1}.nginx.foo.svc.cluster.local | web-{0..N-1} | + kube.local | foo/nginx | foo/web | nginx.foo.svc.kube.local | web-{0..N-1}.nginx.foo.svc.kube.local | web-{0..N-1} | + +Lets verify this assertion with a simple test. + +```shell +$ kubectl get svc +NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE +nginx None 80/TCP 12m +``` + +The containers are running nginx webservers, which by default will look for an index.html file in `/usr/share/nginx/html/index.html`. That directory is backed by a PersistentVolume created by the Pet Set. So lets write our hostname there (remember the Pet Set gives us a stable hostname): + +```shell +$ for i in 0 1; do + kubectl exec web-$i -- sh -c 'echo $(hostname) > /usr/share/nginx/html/index.html'; +done +``` + +Now delete all pods in the petset: + +```shell +$ kubectl delete po -l app=nginx +pod "web-0" deleted +pod "web-1" deleted +``` + +Wait for them to come back up, and try to retrieve the previously written hostname through the DNS name of the peer (remember the Pet Set also gives us stable storage, and that the hostname is linked to the DNS name). + +```shell +$ kubectl exec -it web-1 -- curl web-0.nginx +web-0 +$ kubectl exec -it web-0 -- curl web-1.nginx +web-1 +``` + +## Peer discovery + +A pet can piece together its own identity: + +1. Use the [downward api](/docs/user-guide/downward-api/) to find its pod name +2. Run `hostname` to find its DNS name +3. Run `mount` or `df` to find its volumes (usually this is unnecessary) + +It's not necessary to "discover" the governing Service of a Pet Set, since it's known at creation time you can simply pass it down through an [environment variable](/docs/user-guide/environment-guide). + +Usually pets also need to find their peers. In the previous nginx example, we just used `kubectl` to get the names of existing pods, and as humans, we could tell which ones belonged to a given Pet Set. Another way to find peers is by contacting the API server, just like `kubectl`, but that has several disadvantages (you end up implementing a Kubernetes specific init system that runs as pid 1 in your application container). + +Pet Set gives you a way to disover your peers using DNS records. To illustrate this we can use the previous example (note: one usually doesn't `apt-get` in a container). + +```shell +$ kubectl exec -it web-0 /bin/sh +web-0 # apt-get update && apt-get install -y dnsutils +... + +web-0 # nslookup -type=srv nginx.default +Server: 10.0.0.10 +Address: 10.0.0.10#53 + +nginx.default.svc.cluster.local service = 10 50 0 web-1.ub.default.svc.cluster.local. +nginx.default.svc.cluster.local service = 10 50 0 web-0.ub.default.svc.cluster.local. +``` + +## Updating a Pet Set + +You cannot update any field of the PetSet except `spec.replicas`. You can update the replicas field using standard kubectl update commands like [patch](/docs/user-guide/kubectl/kubectl_patch) and [edit](/docs/user-guide/kubectl/kubectl_edit). Pet Set currently *does not* support image upgrade as noted in the section on [limitations](#alpha-limitations). + +## Scaling a Pet Set + +You can scale a Pet Set by updating the "replicas" field. Note however that the controller will only: + +1. Create one pet at a time, in order from {0..N-1}, and wait till each one is in [Running and Ready](/docs/user-guide/pod-states) before creating the next +2. Delete one pet at a time, in reverse order from {N-1..0}, and wait till each one is completely shutdown (past its [terminationGracePeriodSeconds](/docs/user-guide/pods/index#termination-of-pods)) before deleting the next + +## Deleting a Pet Set + +Cleaning up a Pet Set is somewhat manual, as noted in the [limitations section](#alpha-limitations). You can delete a Pet Set using Kubectl, but this will *not* scale it down to 0: + +```shell +$ kubectl delete -f petset.yaml +petset "web" deleted + +$ kubectl get po -l app=nginx +NAME READY STATUS RESTARTS AGE +web-0 1/1 Running 0 21h +web-1 1/1 Running 0 21h + +$ kubectl delete po -l app=nginx +pod "web-0" deleted +pod "web-1" deleted +``` + +Deleting the pods will *not* delete the volumes. Until we finalize the recycle policy for these volumes they will have to get cleaned up by an admin. This is to ensure that you have the chance to copy data off the volume before deleting it. Simply deleting the PVC after the pods have left the [terminating state](/docs/user-guide/pods/index#termination-of-pods) should trigger deletion of the backing Persistent Volumes. + +**Note: you will lose all your data once the PVC is deleted, do this with caution.** + +```shell +$ kubectl get po -l app=nginx +$ kubectl get pvc -l app=nginx +NAME STATUS VOLUME CAPACITY ACCESSMODES AGE +www-web-0 Bound pvc-62d271cd-3822-11e6-b1b7-42010af00002 0 21h +www-web-1 Bound pvc-62d6750e-3822-11e6-b1b7-42010af00002 0 21h + +$ kubectl delete pvc -l app=nginx +$ kubectl get pv +``` + +If you simply want to clean everything: + +```shell{% raw %} +$ grace=$(k get po web-0 --template '{{.spec.terminationGracePeriodSeconds}}') +$ kubectl delete petset,po -l app=nginx +$ sleep $grace +$ kubectl delete pvc -l app=nginx +{% endraw %}``` + +## Troubleshooting + +You might have noticed an `annotations` field in all the Pet Sets shown above. + +```yaml +annotations: + pod.alpha.kubernetes.io/initialized: "true" +``` + +This field is a debugging hook. It pauses any scale up/down operations on the entire Pet Set. If you'd like to pause a petset after each pet, set it to `false` in the template, wait for each pet to come up, verify it has initialized correctly, and then set it to `true` using `kubectl edit` on the pet (setting it to `false` on *any pet* is enough to pause the Pet Set). If you don't need it, create the Pet Set with it set to `true` as shown. This is surprisingly useful in debugging bootstrapping race conditions. + +## Future Work + +There are a LOT of planned improvements since Pet Set is still in alpha. + +* Data gravity and local storage +* Richer notification events +* Public network identities +* WAN cluster deployments (multi-AZ/region/cloud provider) +* Image and node upgrades + +This list goes on, if you have examples, ideas or thoughts, please contribute. + +## Alternatives + +Deploying one RC of size 1/Service per pod is a popular alternative, as is simply deploying a DaemonSet that utilizes the identity of a Node. + + + diff --git a/docs/user-guide/petset.yaml b/docs/user-guide/petset.yaml new file mode 100644 index 0000000000..017a65746c --- /dev/null +++ b/docs/user-guide/petset.yaml @@ -0,0 +1,56 @@ +# A headless service to create DNS records +apiVersion: v1 +kind: Service +metadata: + name: nginx + labels: + app: nginx +spec: + ports: + - port: 80 + name: web + # *.nginx.default.svc.cluster.local + clusterIP: None + selector: + app: nginx +--- +apiVersion: apps/v1alpha1 +kind: PetSet +metadata: + name: web +spec: + serviceName: "nginx" + replicas: 2 + template: + metadata: + labels: + app: nginx + annotations: + pod.alpha.kubernetes.io/initialized: "true" + spec: + terminationGracePeriodSeconds: 0 + containers: + - name: nginx + image: gcr.io/google_containers/nginx-slim:0.7 + ports: + - containerPort: 80 + name: web + command: + - nginx + args: + - -g + - "daemon off;" + volumeMounts: + - name: www + mountPath: /usr/share/nginx/html + volumeClaimTemplates: + - metadata: + name: www + annotations: + volume.alpha.kubernetes.io/storage-class: anything + spec: + accessModes: [ "ReadWriteOnce" ] + resources: + requests: + storage: 1Gi + From e4afede634184527540d964e3712ed6be5ba08e8 Mon Sep 17 00:00:00 2001 From: Clayton Coleman Date: Wed, 15 Jun 2016 15:35:20 -0400 Subject: [PATCH 126/348] Add description of protobuf and clarify versioning policy --- docs/api.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/api.md b/docs/api.md index e466962df2..ab8a6a3953 100644 --- a/docs/api.md +++ b/docs/api.md @@ -25,13 +25,15 @@ Complete API details are documented using [Swagger v1.2](http://swagger.io/). Th We also host a version of the [latest API documentation UI](http://kubernetes.io/kubernetes/third_party/swagger-ui/). This is updated with the latest release, so if you are using a different version of Kubernetes you will want to use the spec from your apiserver. +Kubernetes implements an alternative Protobuf based serialization format for the API that is primarily intended for intra-cluster communication, documented in the [design proposal](https://github.com/kubernetes/kubernetes/blob/{{ page.githubbranch }}/docs/proposals/protobuf.md) and the IDL files for each schema are located in the Go packages that define the API objects. + ## API versioning To make it easier to eliminate fields or restructure resource representations, Kubernetes supports multiple API versions, each at a different API path, such as `/api/v1` or `/apis/extensions/v1beta1`. -We chose to version at the API level rather than at the resource or field level to ensure that the API presents a clear, consistent view of system resources and behavior, and to enable controlling access to end-of-lifed and/or experimental APIs. +We chose to version at the API level rather than at the resource or field level to ensure that the API presents a clear, consistent view of system resources and behavior, and to enable controlling access to end-of-lifed and/or experimental APIs. The JSON and Protobuf serialization schemas follow the same guidelines for schema changes - all descriptions below cover both formats. Note that API versioning and Software versioning are only indirectly related. The [API and release versioning proposal](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/versioning.md) describes the relationship between API versioning and From 03dddefd01244c3077b62baa36d31f44566c2f76 Mon Sep 17 00:00:00 2001 From: derekwaynecarr Date: Thu, 23 Jun 2016 11:51:35 -0400 Subject: [PATCH 127/348] Update 1.3 example for quota --- docs/admin/resourcequota/best-effort.yaml | 9 + .../resourcequota/compute-resources.yaml | 11 + docs/admin/resourcequota/index.md | 62 +-- docs/admin/resourcequota/not-best-effort.yaml | 13 + docs/admin/resourcequota/object-counts.yaml | 9 + docs/admin/resourcequota/quota.yaml | 14 - docs/admin/resourcequota/walkthrough.md | 373 +++++++++++++----- 7 files changed, 358 insertions(+), 133 deletions(-) create mode 100644 docs/admin/resourcequota/best-effort.yaml create mode 100644 docs/admin/resourcequota/compute-resources.yaml create mode 100644 docs/admin/resourcequota/not-best-effort.yaml create mode 100644 docs/admin/resourcequota/object-counts.yaml delete mode 100644 docs/admin/resourcequota/quota.yaml diff --git a/docs/admin/resourcequota/best-effort.yaml b/docs/admin/resourcequota/best-effort.yaml new file mode 100644 index 0000000000..5bd6efdcfd --- /dev/null +++ b/docs/admin/resourcequota/best-effort.yaml @@ -0,0 +1,9 @@ +apiVersion: v1 +kind: ResourceQuota +metadata: + name: best-effort +spec: + hard: + pods: "10" + scopes: + - BestEffort diff --git a/docs/admin/resourcequota/compute-resources.yaml b/docs/admin/resourcequota/compute-resources.yaml new file mode 100644 index 0000000000..9757018f19 --- /dev/null +++ b/docs/admin/resourcequota/compute-resources.yaml @@ -0,0 +1,11 @@ +apiVersion: v1 +kind: ResourceQuota +metadata: + name: compute-resources +spec: + hard: + pods: "4" + requests.cpu: "1" + requests.memory: 1Gi + limits.cpu: "2" + limits.memory: 2Gi diff --git a/docs/admin/resourcequota/index.md b/docs/admin/resourcequota/index.md index 59be97e38e..af96bc538d 100644 --- a/docs/admin/resourcequota/index.md +++ b/docs/admin/resourcequota/index.md @@ -23,7 +23,7 @@ Resource quotas work like this: - If quota is enabled in a namespace for compute resources like `cpu` and `memory`, users must specify requests or limits for those values; otherwise, the quota system may reject pod creation. Hint: Use the LimitRange admission controller to force defaults for pods that make no compute resource requirements. - See the [walkthrough](/docs/admin/resourcequota/walkthrough.md) for an example to avoid this problem. + See the [walkthrough](/docs/admin/resourcequota/walkthrough/) for an example to avoid this problem. Examples of policies that could be created using namespaces and quotas are: @@ -52,31 +52,31 @@ Resource Quota is enforced in a particular namespace when there is a The total sum of [compute resources](/docs/user-guide/compute-resources) requested by pods in a namespace can be limited. The following compute resource types are supported: -| ResourceName | Description | +| Resource Name | Description | | ------------ | ----------- | -| cpu | Across all pods in a non-terminal state, the sum of CPU requests cannot exceed this value. | -| limits.cpu | Across all pods in a non-terminal state, the sum of CPU limits cannot exceed this value. | -| limits.memory | Across all pods in a non-terminal state, the sum of memory limits cannot exceed this value. | -| memory | Across all pods in a non-terminal state, the sum of memory requests cannot exceed this value. | -| requests.cpu | Across all pods in a non-terminal state, the sum of CPU requests cannot exceed this value. | -| requests.memory | Across all pods in a non-terminal state, the sum of memory requests cannot exceed this value. | +| `cpu` | Across all pods in a non-terminal state, the sum of CPU requests cannot exceed this value. | +| `limits.cpu` | Across all pods in a non-terminal state, the sum of CPU limits cannot exceed this value. | +| `limits.memory` | Across all pods in a non-terminal state, the sum of memory limits cannot exceed this value. | +| `memory` | Across all pods in a non-terminal state, the sum of memory requests cannot exceed this value. | +| `requests.cpu` | Across all pods in a non-terminal state, the sum of CPU requests cannot exceed this value. | +| `requests.memory` | Across all pods in a non-terminal state, the sum of memory requests cannot exceed this value. | ## Object Count Quota The number of objects of a given type can be restricted. The following types are supported: -| ResourceName | Description | +| Resource Name | Description | | ------------ | ----------- | -| configmaps | The total number of config maps that can exist in the namespace. | -| persistentvolumeclaims | The total number of [persistent volume claims](/docs/user-guide/persistent-volumes/#persistentvolumeclaims) that can exist in the namespace. | -| pods | The total number of pods in a non-terminal state that can exist in the namespace. A pod is in a terminal state if `status.phase in (Failed, Succeeded)` is true. | -| replicationcontrollers | The total number of replication controllers that can exist in the namespace. | -| resourcequotas | The total number of [resource quotas](/docs/admin/admission-controllers/#resourcequota) that can exist in the namespace. | -| services | The total number of services that can exist in the namespace. | -| services.loadbalancers | The total number of services of type load balancer that can exist in the namespace. | -| services.nodeports | The total number of services of type node port that can exist in the namespace. | -| secrets | The total number of secrets that can exist in the namespace. | +| `configmaps` | The total number of config maps that can exist in the namespace. | +| `persistentvolumeclaims` | The total number of [persistent volume claims](/docs/user-guide/persistent-volumes/#persistentvolumeclaims) that can exist in the namespace. | +| `pods` | The total number of pods in a non-terminal state that can exist in the namespace. A pod is in a terminal state if `status.phase in (Failed, Succeeded)` is true. | +| `replicationcontrollers` | The total number of replication controllers that can exist in the namespace. | +| `resourcequotas` | The total number of [resource quotas](/docs/admin/admission-controllers/#resourcequota) that can exist in the namespace. | +| `services` | The total number of services that can exist in the namespace. | +| `services.loadbalancers` | The total number of services of type load balancer that can exist in the namespace. | +| `services.nodeports` | The total number of services of type node port that can exist in the namespace. | +| `secrets` | The total number of secrets that can exist in the namespace. | For example, `pods` quota counts and enforces a maximum on the number of `pods` created in a single namespace. @@ -95,22 +95,22 @@ Resources specified on the quota outside of the allowed set results in a validat | Scope | Description | | ----- | ----------- | -| Terminating | Match pods where `spec.activeDeadlineSeconds >= 0` | -| NotTerminating | Match pods where `spec.activeDeadlineSeconds is nil` | -| BestEffort | Match pods that have best effort quality of service. | -| NotBestEffort | Match pods that do not have best effort quality of service. | +| `Terminating` | Match pods where `spec.activeDeadlineSeconds >= 0` | +| `NotTerminating` | Match pods where `spec.activeDeadlineSeconds is nil` | +| `BestEffort` | Match pods that have best effort quality of service. | +| `NotBestEffort` | Match pods that do not have best effort quality of service. | -The `BestEffort` scope restricts a quota to tracking the following resources: -* pods +The `BestEffort` scope restricts a quota to tracking the following resource: `pods` The `Terminating`, `NotTerminating`, and `NotBestEffort` scopes restrict a quota to tracking the following resources: -* cpu -* limits.cpu -* limits.memory -* memory -* pods -* requests.cpu -* requests.memory + +* `cpu` +* `limits.cpu` +* `limits.memory` +* `memory` +* `pods` +* `requests.cpu` +* `requests.memory` ## Requests vs Limits diff --git a/docs/admin/resourcequota/not-best-effort.yaml b/docs/admin/resourcequota/not-best-effort.yaml new file mode 100644 index 0000000000..82666058e1 --- /dev/null +++ b/docs/admin/resourcequota/not-best-effort.yaml @@ -0,0 +1,13 @@ +apiVersion: v1 +kind: ResourceQuota +metadata: + name: not-best-effort +spec: + hard: + pods: "4" + requests.cpu: "1" + requests.memory: 1Gi + limits.cpu: "2" + limits.memory: 2Gi + scopes: + - NotBestEffort diff --git a/docs/admin/resourcequota/object-counts.yaml b/docs/admin/resourcequota/object-counts.yaml new file mode 100644 index 0000000000..604b909614 --- /dev/null +++ b/docs/admin/resourcequota/object-counts.yaml @@ -0,0 +1,9 @@ +apiVersion: v1 +kind: ResourceQuota +metadata: + name: object-counts +spec: + hard: + persistentvolumeclaims: "2" + services.loadbalancers: "2" + services.nodeports: "0" diff --git a/docs/admin/resourcequota/quota.yaml b/docs/admin/resourcequota/quota.yaml deleted file mode 100644 index 6885efc6d1..0000000000 --- a/docs/admin/resourcequota/quota.yaml +++ /dev/null @@ -1,14 +0,0 @@ -apiVersion: v1 -kind: ResourceQuota -metadata: - name: quota -spec: - hard: - cpu: "20" - memory: 1Gi - persistentvolumeclaims: "10" - pods: "10" - replicationcontrollers: "20" - resourcequotas: "1" - secrets: "10" - services: "5" diff --git a/docs/admin/resourcequota/walkthrough.md b/docs/admin/resourcequota/walkthrough.md index eeacbddabf..651f8ae15d 100644 --- a/docs/admin/resourcequota/walkthrough.md +++ b/docs/admin/resourcequota/walkthrough.md @@ -1,12 +1,30 @@ --- --- -This example demonstrates how [resource quota](/docs/admin/admission-controllers/#resourcequota) and -[limitsranger](/docs/admin/admission-controllers/#limitranger) can be applied to a Kubernetes namespace. -See [ResourceQuota design doc](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/admission_control_resource_quota.md) for more information. +This example demonstrates a typical setup to control for resource usage in a namespace. + +It demonstrates using the following resources: + +* [Namespace](/docs/admin/namespaces) +* [Resource Quota](/docs/admin/resourcequota/) +* [Limit Range](/docs/admin/limitrange/) This example assumes you have a functional Kubernetes setup. +## Scenario + +The cluster-admin is operating a cluster on behalf of a user population and the cluster-admin +wants to control the amount of resources that can be consumed in a particular namespace to promote +fair sharing of the cluster and control cost. + +The cluster-admin has the following goals: + +* Limit the amount of compute resource for running pods +* Limit the number of persistent volume claims to control access to storage +* Limit the number of load balancers to control cost +* Prevent the use of node ports to preserve scarce resources +* Provide default compute resource requests to enable better scheduling decisions + ## Step 1: Create a namespace This example will work in a custom namespace to demonstrate the concepts involved. @@ -14,151 +32,330 @@ This example will work in a custom namespace to demonstrate the concepts involve Let's create a new namespace called quota-example: ```shell -$ kubectl create namespace quota-example +$ kubectl create -f docs/admin/resourcequota/namespace.yaml namespace "quota-example" created -``` - -Note that `kubectl` commands will print the type and name of the resource created or mutated, which can then be used in subsequent commands: - -```shell $ kubectl get namespaces NAME STATUS AGE -default Active 50m -quota-example Active 2s +default Active 2m +kube-system Active 2m +quota-example Active 39s ``` -## Step 2: Apply a quota to the namespace +## Step 2: Apply an object-count quota to the namespace -By default, a pod will run with unbounded CPU and memory requests/limits. This means that any pod in the -system will be able to consume as much CPU and memory on the node that executes the pod. +The cluster-admin wants to control the following resources: -Users may want to restrict how much of the cluster resources a given namespace may consume -across all of its pods in order to manage cluster usage. To do this, a user applies a quota to -a namespace. A quota lets the user set hard limits on the total amount of node resources (cpu, memory) -and API resources (pods, services, etc.) that a namespace may consume. In term of resources, Kubernetes -checks the total resource *requests*, not resource *limits* of all containers/pods in the namespace. +* persistent volume claims +* load balancers +* node ports -Let's create a simple quota in our namespace: +Let's create a simple quota that controls object counts for those resource types in this namespace. ```shell -$ kubectl create -f docs/admin/resourcequota/quota.yaml --namespace=quota-example -resourcequota "quota" created +$ kubectl create -f docs/admin/resourcequota/object-counts.yaml --namespace=quota-example +resourcequota "object-counts" created ``` -Once your quota is applied to a namespace, the system will restrict any creation of content -in the namespace until the quota usage has been calculated. This should happen quickly. +The quota system will observe that a quota has been created, and will calculate consumption +in the namespace in response. This should happen quickly. -You can describe your current quota usage to see what resources are being consumed in your -namespace. +Let's describe the quota to see what is currently being consumed in this namespace: ```shell -$ kubectl describe quota quota --namespace=quota-example -Name: quota -Namespace: quota-example -Resource Used Hard --------- ---- ---- -cpu 0 20 -memory 0 1Gi -persistentvolumeclaims 0 10 -pods 0 10 -replicationcontrollers 0 20 -resourcequotas 1 1 -secrets 1 10 -services 0 5 +$ kubectl describe quota object-counts --namespace=quota-example +Name: object-counts +Namespace: quota-example +Resource Used Hard +-------- ---- ---- +persistentvolumeclaims 0 2 +services.loadbalancers 0 2 +services.nodeports 0 0 ``` -## Step 3: Applying default resource requests and limits +The quota system will now prevent users from creating more than the specified amount for each resource. + +## Step 3: Apply a compute-resource quota to the namespace + +To limit the amount of compute resource that can be consumed in this namespace, +let's create a quota that tracks compute resources. + +```shell +$ kubectl create -f docs/admin/resourcequota/compute-resources.yaml --namespace=quota-example +resourcequota "compute-resources" created +``` + +Let's describe the quota to see what is currently being consumed in this namespace: + +```shell +$ kubectl describe quota compute-resources --namespace=quota-example +Name: compute-resources +Namespace: quota-example +Resource Used Hard +-------- ---- ---- +limits.cpu 0 2 +limits.memory 0 2Gi +pods 0 4 +requests.cpu 0 1 +requests.memory 0 1Gi +``` + +The quota system will now prevent the namespace from having more than 4 non-terminal pods. In +addition, it will enforce that each container in a pod makes a `request` and defines a `limit` for +`cpu` and `memory`. + +## Step 4: Applying default resource requests and limits Pod authors rarely specify resource requests and limits for their pods. Since we applied a quota to our project, let's see what happens when an end-user creates a pod that has unbounded cpu and memory by creating an nginx container. -To demonstrate, lets create a Deployment that runs nginx: +To demonstrate, lets create a deployment that runs nginx: ```shell $ kubectl run nginx --image=nginx --replicas=1 --namespace=quota-example deployment "nginx" created ``` -This creates a Deployment "nginx" with its underlying resource, a ReplicaSet, which handles the creation and deletion of Pod replicas. Now let's look at the pods that were created. +Now let's look at the pods that were created. ```shell $ kubectl get pods --namespace=quota-example -NAME READY STATUS RESTARTS AGE ``` -What happened? I have no pods! Let's describe the ReplicaSet managed by the nginx Deployment to get a view of what is happening. -Note that `kubectl describe rs` works only on kubernetes cluster >= v1.2. If you are running older versions, use `kubectl describe rc` instead. -If you want to obtain the old behavior, use `--generator=run/v1` to create replication controllers. See [`kubectl run`](/docs/user-guide/kubectl/kubectl_run/) for more details. +What happened? I have no pods! Let's describe the deployment to get a view of what is happening. ```shell -$ kubectl describe rs -l run=nginx --namespace=quota-example -Name: nginx-2040093540 -Namespace: quota-example -Image(s): nginx -Selector: pod-template-hash=2040093540,run=nginx -Labels: pod-template-hash=2040093540,run=nginx -Replicas: 0 current / 1 desired -Pods Status: 0 Running / 0 Waiting / 0 Succeeded / 0 Failed -No volumes. -Events: - FirstSeen LastSeen Count From SubobjectPath Type Reason Message - --------- -------- ----- ---- ------------- -------- ------ ------- - 48s 26s 4 {replicaset-controller } Warning FailedCreate Error creating: pods "nginx-2040093540-" is forbidden: Failed quota: quota: must specify cpu,memory +$ kubectl describe deployment nginx --namespace=quota-example +Name: nginx +Namespace: quota-example +CreationTimestamp: Mon, 06 Jun 2016 16:11:37 -0400 +Labels: run=nginx +Selector: run=nginx +Replicas: 0 updated | 1 total | 0 available | 1 unavailable +StrategyType: RollingUpdate +MinReadySeconds: 0 +RollingUpdateStrategy: 1 max unavailable, 1 max surge +OldReplicaSets: +NewReplicaSet: nginx-3137573019 (0/1 replicas created) +... ``` -The Kubernetes API server is rejecting the ReplicaSet requests to create a pod because our pods -do not specify any memory usage *request*. +A deployment created a corresponding replica set and attempted to size it to create a single pod. -So let's set some default values for the amount of cpu and memory a pod can consume: +Let's look at the replica set to get more detail. + +```shell +$ kubectl describe rs nginx-3137573019 --namespace=quota-example +Name: nginx-3137573019 +Namespace: quota-example +Image(s): nginx +Selector: pod-template-hash=3137573019,run=nginx +Labels: pod-template-hash=3137573019 + run=nginx +Replicas: 0 current / 1 desired +Pods Status: 0 Running / 0 Waiting / 0 Succeeded / 0 Failed +No volumes. +Events: + FirstSeen LastSeen Count From SubobjectPath Type Reason Message + --------- -------- ----- ---- ------------- -------- ------ ------- + 4m 7s 11 {replicaset-controller } Warning FailedCreate Error creating: pods "nginx-3137573019-" is forbidden: Failed quota: compute-resources: must specify limits.cpu,limits.memory,requests.cpu,requests.memory +``` + +The Kubernetes API server is rejecting the replica set requests to create a pod because our pods +do not specify `requests` or `limits` for `cpu` and `memory`. + +So let's set some default values for the amount of `cpu` and `memory` a pod can consume: ```shell $ kubectl create -f docs/admin/resourcequota/limits.yaml --namespace=quota-example limitrange "limits" created $ kubectl describe limits limits --namespace=quota-example -Name: limits -Namespace: quota-example -Type Resource Min Max Default Request Default Limit Max Limit/Request Ratio ----- -------- --- --- --------------- ------------- ----------------------- -Container cpu - - 100m 200m - -Container memory - - 256Mi 512Mi - +Name: limits +Namespace: quota-example +Type Resource Min Max Default Request Default Limit Max Limit/Request Ratio +---- -------- --- --- --------------- ------------- ----------------------- +Container memory - - 256Mi 512Mi - +Container cpu - - 100m 200m - ``` -Now any time a pod is created in this namespace, if it has not specified any resource request/limit, the default -amount of cpu and memory per container will be applied, and the request will be used as part of admission control. +If the Kubernetes API server observes a request to create a pod in this namespace, and the containers +in that pod do not make any compute resource requests, a default request and default limit will be applied +as part of admission control. -Now that we have applied default resource *request* for our namespace, our Deployment should be able to -create its pods. +In this example, each pod created will have compute resources equivalent to the following: + +```shell +$ kubectl run nginx \ + --image=nginx \ + --replicas=1 \ + --requests=cpu=100m,memory=256Mi \ + --limits=cpu=200m,memory=512Mi \ + --namespace=quota-example +``` + +Now that we have applied default compute resources for our namespace, our replica set should be able to create +its pods. ```shell $ kubectl get pods --namespace=quota-example NAME READY STATUS RESTARTS AGE -nginx-2040093540-miohp 1/1 Running 0 5s +nginx-3137573019-fvrig 1/1 Running 0 6m ``` And if we print out our quota usage in the namespace: ```shell -$ kubectl describe quota quota --namespace=quota-example -Name: quota -Namespace: quota-example -Resource Used Hard --------- ---- ---- -cpu 100m 20 -memory 256Mi 1Gi -persistentvolumeclaims 0 10 -pods 1 10 -replicationcontrollers 1 20 -resourcequotas 1 1 -secrets 1 10 -services 0 5 +$ kubectl describe quota --namespace=quota-example +Name: compute-resources +Namespace: quota-example +Resource Used Hard +-------- ---- ---- +limits.cpu 200m 2 +limits.memory 512Mi 2Gi +pods 1 4 +requests.cpu 100m 1 +requests.memory 256Mi 1Gi + + +Name: object-counts +Namespace: quota-example +Resource Used Hard +-------- ---- ---- +persistentvolumeclaims 0 2 +services.loadbalancers 0 2 +services.nodeports 0 0 ``` -You can now see the pod that was created is consuming explicit amounts of resources (specified by resource *request*), and the usage is being tracked by the Kubernetes system properly. +As you can see, the pod that was created is consuming explict amounts of compute resources, and the usage is being +tracked by Kubernetes properly. + +## Step 5: Advanced quota scopes + +Let's imagine you did not want to specify default compute resource consumption in your namespace. + +Instead, you want to let users run a specific number of `BestEffort` pods in their namespace to take +advantage of slack compute resources, and then require that users make an explicit resource request for +pods that require a higher quality of service. + +Let's create a new namespace with two quotas to demonstrate this behavior: + +```shell +$ kubectl create namespace quota-scopes +namespace "quota-scopes" created +$ kubectl create -f docs/admin/resourcequota/best-effort.yaml --namespace=quota-scopes +resourcequota "best-effort" created +$ kubectl create -f docs/admin/resourcequota/not-best-effort.yaml --namespace=quota-scopes +resourcequota "not-best-effort" created +$ kubectl describe quota --namespace=quota-scopes +Name: best-effort +Namespace: quota-scopes +Scopes: BestEffort + * Matches all pods that have best effort quality of service. +Resource Used Hard +-------- ---- ---- +pods 0 10 + + +Name: not-best-effort +Namespace: quota-scopes +Scopes: NotBestEffort + * Matches all pods that do not have best effort quality of service. +Resource Used Hard +-------- ---- ---- +limits.cpu 0 2 +limits.memory 0 2Gi +pods 0 4 +requests.cpu 0 1 +requests.memory 0 1Gi +``` + +In this scenario, a pod that makes no compute resource requests will be tracked by the `best-effort` quota. + +A pod that does make compute resource requests will be tracked by the `not-best-effort` quota. + +Let's demonstrate this by creating two deployments: + +```shell +$ kubectl run best-effort-nginx --image=nginx --replicas=8 --namespace=quota-scopes +deployment "best-effort-nginx" created +$ kubectl run not-best-effort-nginx \ + --image=nginx \ + --replicas=2 \ + --requests=cpu=100m,memory=256Mi \ + --limits=cpu=200m,memory=512Mi \ + --namespace=quota-scopes +deployment "not-best-effort-nginx" created +``` + +Even though no default limits were specified, the `best-effort-nginx` deployment will create +all 8 pods. This is because it is tracked by the `best-effort` quota, and the `not-best-effort` +quota will just ignore it. The `not-best-effort` quota will track the `not-best-effort-nginx` +deployment since it creates pods with `Burstable` quality of service. + +Let's list the pods in the namespace: + +```shell +$ kubectl get pods --namespace=quota-scopes +NAME READY STATUS RESTARTS AGE +best-effort-nginx-3488455095-2qb41 1/1 Running 0 51s +best-effort-nginx-3488455095-3go7n 1/1 Running 0 51s +best-effort-nginx-3488455095-9o2xg 1/1 Running 0 51s +best-effort-nginx-3488455095-eyg40 1/1 Running 0 51s +best-effort-nginx-3488455095-gcs3v 1/1 Running 0 51s +best-effort-nginx-3488455095-rq8p1 1/1 Running 0 51s +best-effort-nginx-3488455095-udhhd 1/1 Running 0 51s +best-effort-nginx-3488455095-zmk12 1/1 Running 0 51s +not-best-effort-nginx-2204666826-7sl61 1/1 Running 0 23s +not-best-effort-nginx-2204666826-ke746 1/1 Running 0 23s +``` + +As you can see, all 10 pods have been allowed to be created. + +Let's describe current quota usage in the namespace: + +```shell +$ kubectl describe quota --namespace=quota-scopes +Name: best-effort +Namespace: quota-scopes +Scopes: BestEffort + * Matches all pods that have best effort quality of service. +Resource Used Hard +-------- ---- ---- +pods 8 10 + + +Name: not-best-effort +Namespace: quota-scopes +Scopes: NotBestEffort + * Matches all pods that do not have best effort quality of service. +Resource Used Hard +-------- ---- ---- +limits.cpu 400m 2 +limits.memory 1Gi 2Gi +pods 2 4 +requests.cpu 200m 1 +requests.memory 512Mi 1Gi +``` + +As you can see, the `best-effort` quota has tracked the usage for the 8 pods we created in +the `best-effort-nginx` deployment, and the `not-best-effort` quota has tracked the usage for +the 2 pods we created in the `not-best-effort-nginx` quota. + +Scopes provide a mechanism to subdivide the set of resources that are tracked by +any quota document to allow greater flexibility in how operators deploy and track resource +consumption. + +In addition to `BestEffort` and `NotBestEffort` scopes, there are scopes to restrict +long-running versus time-bound pods. The `Terminating` scope will match any pod +where `spec.activeDeadlineSeconds is not nil`. The `NotTerminating` scope will match any pod +where `spec.activeDeadlineSeconds is nil`. These scopes allow you to quota pods based on their +anticipated permanence on a node in your cluster. ## Summary -Actions that consume node resources for cpu and memory can be subject to hard quota limits defined by the namespace quota. The resource consumption is measured by resource *request* in pod specification. +Actions that consume node resources for cpu and memory can be subject to hard quota limits defined by the namespace quota. Any action that consumes those resources can be tweaked, or can pick up namespace level defaults to meet your end goal. + +Quota can be apportioned based on quality of service and anticipated permanence on a node in your cluster. \ No newline at end of file From 0b58359e0282282a5ba3785206c2caf2b2b886a7 Mon Sep 17 00:00:00 2001 From: derekwaynecarr Date: Mon, 6 Jun 2016 17:01:22 -0400 Subject: [PATCH 128/348] Improve and update ResourceQuota example --- docs/admin/resourcequota/best-effort.yaml | 9 + .../resourcequota/compute-resources.yaml | 11 + docs/admin/resourcequota/index.md | 58 +-- docs/admin/resourcequota/not-best-effort.yaml | 13 + docs/admin/resourcequota/object-counts.yaml | 7 + docs/admin/resourcequota/quota.yaml | 14 - docs/admin/resourcequota/walkthrough.md | 365 +++++++++++++----- 7 files changed, 345 insertions(+), 132 deletions(-) create mode 100644 docs/admin/resourcequota/best-effort.yaml create mode 100644 docs/admin/resourcequota/compute-resources.yaml create mode 100644 docs/admin/resourcequota/not-best-effort.yaml create mode 100644 docs/admin/resourcequota/object-counts.yaml delete mode 100644 docs/admin/resourcequota/quota.yaml diff --git a/docs/admin/resourcequota/best-effort.yaml b/docs/admin/resourcequota/best-effort.yaml new file mode 100644 index 0000000000..5bd6efdcfd --- /dev/null +++ b/docs/admin/resourcequota/best-effort.yaml @@ -0,0 +1,9 @@ +apiVersion: v1 +kind: ResourceQuota +metadata: + name: best-effort +spec: + hard: + pods: "10" + scopes: + - BestEffort diff --git a/docs/admin/resourcequota/compute-resources.yaml b/docs/admin/resourcequota/compute-resources.yaml new file mode 100644 index 0000000000..9757018f19 --- /dev/null +++ b/docs/admin/resourcequota/compute-resources.yaml @@ -0,0 +1,11 @@ +apiVersion: v1 +kind: ResourceQuota +metadata: + name: compute-resources +spec: + hard: + pods: "4" + requests.cpu: "1" + requests.memory: 1Gi + limits.cpu: "2" + limits.memory: 2Gi diff --git a/docs/admin/resourcequota/index.md b/docs/admin/resourcequota/index.md index a3a4dab246..f642da55f0 100644 --- a/docs/admin/resourcequota/index.md +++ b/docs/admin/resourcequota/index.md @@ -23,7 +23,7 @@ Resource quotas work like this: - If quota is enabled in a namespace for compute resources like `cpu` and `memory`, users must specify requests or limits for those values; otherwise, the quota system may reject pod creation. Hint: Use the LimitRange admission controller to force defaults for pods that make no compute resource requirements. - See the [walkthrough](/docs/admin/resourcequota/walkthrough.md) for an example to avoid this problem. + See the [walkthrough](/docs/admin/resourcequota/walkthrough/) for an example to avoid this problem. Examples of policies that could be created using namespaces and quotas are: @@ -52,29 +52,29 @@ Resource Quota is enforced in a particular namespace when there is a The total sum of [compute resources](/docs/user-guide/compute-resources) requested by pods in a namespace can be limited. The following compute resource types are supported: -| ResourceName | Description | +| Resource Name | Description | | ------------ | ----------- | -| cpu | Across all pods in a non-terminal state, the sum of CPU requests cannot exceed this value. | -| limits.cpu | Across all pods in a non-terminal state, the sum of CPU limits cannot exceed this value. | -| limits.memory | Across all pods in a non-terminal state, the sum of memory limits cannot exceed this value. | -| memory | Across all pods in a non-terminal state, the sum of memory requests cannot exceed this value. | -| requests.cpu | Across all pods in a non-terminal state, the sum of CPU requests cannot exceed this value. | -| requests.memory | Across all pods in a non-terminal state, the sum of memory requests cannot exceed this value. | +| `cpu` | Across all pods in a non-terminal state, the sum of CPU requests cannot exceed this value. | +| `limits.cpu` | Across all pods in a non-terminal state, the sum of CPU limits cannot exceed this value. | +| `limits.memory` | Across all pods in a non-terminal state, the sum of memory limits cannot exceed this value. | +| `memory` | Across all pods in a non-terminal state, the sum of memory requests cannot exceed this value. | +| `requests.cpu` | Across all pods in a non-terminal state, the sum of CPU requests cannot exceed this value. | +| `requests.memory` | Across all pods in a non-terminal state, the sum of memory requests cannot exceed this value. | ## Object Count Quota The number of objects of a given type can be restricted. The following types are supported: -| ResourceName | Description | +| Resource Name | Description | | ------------ | ----------- | -| configmaps | The total number of config maps that can exist in the namespace. | -| persistentvolumeclaims | The total number of [persistent volume claims](/docs/user-guide/persistent-volumes/#persistentvolumeclaims) that can exist in the namespace. | -| pods | The total number of pods in a non-terminal state that can exist in the namespace. A pod is in a terminal state if `status.phase in (Failed, Succeeded)` is true. | -| replicationcontrollers | The total number of replication controllers that can exist in the namespace. | -| resourcequotas | The total number of [resource quotas](/docs/admin/admission-controllers/#resourcequota) that can exist in the namespace. | -| services | The total number of services that can exist in the namespace. | -| secrets | The total number of secrets that can exist in the namespace. | +| `configmaps` | The total number of config maps that can exist in the namespace. | +| `persistentvolumeclaims` | The total number of [persistent volume claims](/docs/user-guide/persistent-volumes/#persistentvolumeclaims) that can exist in the namespace. | +| `pods` | The total number of pods in a non-terminal state that can exist in the namespace. A pod is in a terminal state if `status.phase in (Failed, Succeeded)` is true. | +| `replicationcontrollers` | The total number of replication controllers that can exist in the namespace. | +| `resourcequotas` | The total number of [resource quotas](/docs/admin/admission-controllers/#resourcequota) that can exist in the namespace. | +| `services` | The total number of services that can exist in the namespace. | +| `secrets` | The total number of secrets that can exist in the namespace. | For example, `pods` quota counts and enforces a maximum on the number of `pods` created in a single namespace. @@ -93,22 +93,22 @@ Resources specified on the quota outside of the allowed set results in a validat | Scope | Description | | ----- | ----------- | -| Terminating | Match pods where `spec.activeDeadlineSeconds >= 0` | -| NotTerminating | Match pods where `spec.activeDeadlineSeconds is nil` | -| BestEffort | Match pods that have best effort quality of service. | -| NotBestEffort | Match pods that do not have best effort quality of service. | +| `Terminating` | Match pods where `spec.activeDeadlineSeconds >= 0` | +| `NotTerminating` | Match pods where `spec.activeDeadlineSeconds is nil` | +| `BestEffort` | Match pods that have best effort quality of service. | +| `NotBestEffort` | Match pods that do not have best effort quality of service. | -The `BestEffort` scope restricts a quota to tracking the following resources: -* pods +The `BestEffort` scope restricts a quota to tracking the following resource: `pods` The `Terminating`, `NotTerminating`, and `NotBestEffort` scopes restrict a quota to tracking the following resources: -* cpu -* limits.cpu -* limits.memory -* memory -* pods -* requests.cpu -* requests.memory + +* `cpu` +* `limits.cpu` +* `limits.memory` +* `memory` +* `pods` +* `requests.cpu` +* `requests.memory` ## Requests vs Limits diff --git a/docs/admin/resourcequota/not-best-effort.yaml b/docs/admin/resourcequota/not-best-effort.yaml new file mode 100644 index 0000000000..82666058e1 --- /dev/null +++ b/docs/admin/resourcequota/not-best-effort.yaml @@ -0,0 +1,13 @@ +apiVersion: v1 +kind: ResourceQuota +metadata: + name: not-best-effort +spec: + hard: + pods: "4" + requests.cpu: "1" + requests.memory: 1Gi + limits.cpu: "2" + limits.memory: 2Gi + scopes: + - NotBestEffort diff --git a/docs/admin/resourcequota/object-counts.yaml b/docs/admin/resourcequota/object-counts.yaml new file mode 100644 index 0000000000..73ea172581 --- /dev/null +++ b/docs/admin/resourcequota/object-counts.yaml @@ -0,0 +1,7 @@ +apiVersion: v1 +kind: ResourceQuota +metadata: + name: object-counts +spec: + hard: + persistentvolumeclaims: "2" diff --git a/docs/admin/resourcequota/quota.yaml b/docs/admin/resourcequota/quota.yaml deleted file mode 100644 index 6885efc6d1..0000000000 --- a/docs/admin/resourcequota/quota.yaml +++ /dev/null @@ -1,14 +0,0 @@ -apiVersion: v1 -kind: ResourceQuota -metadata: - name: quota -spec: - hard: - cpu: "20" - memory: 1Gi - persistentvolumeclaims: "10" - pods: "10" - replicationcontrollers: "20" - resourcequotas: "1" - secrets: "10" - services: "5" diff --git a/docs/admin/resourcequota/walkthrough.md b/docs/admin/resourcequota/walkthrough.md index eeacbddabf..0502655fb4 100644 --- a/docs/admin/resourcequota/walkthrough.md +++ b/docs/admin/resourcequota/walkthrough.md @@ -1,12 +1,28 @@ --- --- -This example demonstrates how [resource quota](/docs/admin/admission-controllers/#resourcequota) and -[limitsranger](/docs/admin/admission-controllers/#limitranger) can be applied to a Kubernetes namespace. -See [ResourceQuota design doc](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/admission_control_resource_quota.md) for more information. +This example demonstrates a typical setup to control for resource usage in a namespace. + +It demonstrates using the following resources: + +* [Namespace](/docs/admin/namespaces/) +* [Resource Quota](/docs/admin/resourcequota/) +* [Limit Range](/docs/admin/limitrange/) This example assumes you have a functional Kubernetes setup. +## Scenario + +The cluster-admin is operating a cluster on behalf of a user population and the cluster-admin +wants to control the amount of resources that can be consumed in a particular namespace to promote +fair sharing of the cluster and control cost. + +The cluster-admin has the following goals: + +* Limit the amount of compute resources for running pods +* Limit the number of persistent volume claims to control access to storage +* Provide default compute resource requests to enable better scheduling decisions + ## Step 1: Create a namespace This example will work in a custom namespace to demonstrate the concepts involved. @@ -14,151 +30,322 @@ This example will work in a custom namespace to demonstrate the concepts involve Let's create a new namespace called quota-example: ```shell -$ kubectl create namespace quota-example +$ kubectl create -f docs/admin/resourcequota/namespace.yaml namespace "quota-example" created -``` - -Note that `kubectl` commands will print the type and name of the resource created or mutated, which can then be used in subsequent commands: - -```shell $ kubectl get namespaces NAME STATUS AGE -default Active 50m -quota-example Active 2s +default Active 2m +kube-system Active 2m +quota-example Active 39s ``` -## Step 2: Apply a quota to the namespace +## Step 2: Apply an object-count quota to the namespace -By default, a pod will run with unbounded CPU and memory requests/limits. This means that any pod in the -system will be able to consume as much CPU and memory on the node that executes the pod. +The cluster-admin wants to control the amount of persistent volume claims. -Users may want to restrict how much of the cluster resources a given namespace may consume -across all of its pods in order to manage cluster usage. To do this, a user applies a quota to -a namespace. A quota lets the user set hard limits on the total amount of node resources (cpu, memory) -and API resources (pods, services, etc.) that a namespace may consume. In term of resources, Kubernetes -checks the total resource *requests*, not resource *limits* of all containers/pods in the namespace. - -Let's create a simple quota in our namespace: +Let's create a simple quota that controls object counts for those resource types in this namespace. ```shell -$ kubectl create -f docs/admin/resourcequota/quota.yaml --namespace=quota-example -resourcequota "quota" created +$ kubectl create -f docs/admin/resourcequota/object-counts.yaml --namespace=quota-example +resourcequota "object-counts" created ``` -Once your quota is applied to a namespace, the system will restrict any creation of content -in the namespace until the quota usage has been calculated. This should happen quickly. +The quota system will observe that a quota has been created, and will calculate consumption +in the namespace in response. This should happen quickly. -You can describe your current quota usage to see what resources are being consumed in your -namespace. +Let's describe the quota to see what is currently being consumed in this namespace: ```shell -$ kubectl describe quota quota --namespace=quota-example -Name: quota -Namespace: quota-example -Resource Used Hard --------- ---- ---- -cpu 0 20 -memory 0 1Gi -persistentvolumeclaims 0 10 -pods 0 10 -replicationcontrollers 0 20 -resourcequotas 1 1 -secrets 1 10 -services 0 5 +$ kubectl describe quota object-counts --namespace=quota-example +Name: object-counts +Namespace: quota-example +Resource Used Hard +-------- ---- ---- +persistentvolumeclaims 0 2 ``` -## Step 3: Applying default resource requests and limits +The quota system will now prevent users from creating more than the specified amount for each resource. + +## Step 3: Apply a compute-resource quota to the namespace + +To limit the amount of compute resource that can be consumed in this namespace, +let's create a quota that tracks compute resources. + +```shell +$ kubectl create -f docs/admin/resourcequota/compute-resources.yaml --namespace=quota-example +resourcequota "compute-resources" created +``` + +Let's describe the quota to see what is currently being consumed in this namespace: + +```shell +$ kubectl describe quota compute-resources --namespace=quota-example +Name: compute-resources +Namespace: quota-example +Resource Used Hard +-------- ---- ---- +limits.cpu 0 2 +limits.memory 0 2Gi +pods 0 4 +requests.cpu 0 1 +requests.memory 0 1Gi +``` + +The quota system will now prevent the namespace from having more than 4 non-terminal pods. In +addition, it will enforce that each container in a pod makes a `request` and defines a `limit` for +`cpu` and `memory`. + +## Step 4: Applying default resource requests and limits Pod authors rarely specify resource requests and limits for their pods. Since we applied a quota to our project, let's see what happens when an end-user creates a pod that has unbounded cpu and memory by creating an nginx container. -To demonstrate, lets create a Deployment that runs nginx: +To demonstrate, lets create a deployment that runs nginx: ```shell $ kubectl run nginx --image=nginx --replicas=1 --namespace=quota-example deployment "nginx" created ``` -This creates a Deployment "nginx" with its underlying resource, a ReplicaSet, which handles the creation and deletion of Pod replicas. Now let's look at the pods that were created. +Now let's look at the pods that were created. ```shell $ kubectl get pods --namespace=quota-example -NAME READY STATUS RESTARTS AGE ``` -What happened? I have no pods! Let's describe the ReplicaSet managed by the nginx Deployment to get a view of what is happening. -Note that `kubectl describe rs` works only on kubernetes cluster >= v1.2. If you are running older versions, use `kubectl describe rc` instead. -If you want to obtain the old behavior, use `--generator=run/v1` to create replication controllers. See [`kubectl run`](/docs/user-guide/kubectl/kubectl_run/) for more details. +What happened? I have no pods! Let's describe the deployment to get a view of what is happening. ```shell -$ kubectl describe rs -l run=nginx --namespace=quota-example -Name: nginx-2040093540 -Namespace: quota-example -Image(s): nginx -Selector: pod-template-hash=2040093540,run=nginx -Labels: pod-template-hash=2040093540,run=nginx -Replicas: 0 current / 1 desired -Pods Status: 0 Running / 0 Waiting / 0 Succeeded / 0 Failed -No volumes. -Events: - FirstSeen LastSeen Count From SubobjectPath Type Reason Message - --------- -------- ----- ---- ------------- -------- ------ ------- - 48s 26s 4 {replicaset-controller } Warning FailedCreate Error creating: pods "nginx-2040093540-" is forbidden: Failed quota: quota: must specify cpu,memory +$ kubectl describe deployment nginx --namespace=quota-example +Name: nginx +Namespace: quota-example +CreationTimestamp: Mon, 06 Jun 2016 16:11:37 -0400 +Labels: run=nginx +Selector: run=nginx +Replicas: 0 updated | 1 total | 0 available | 1 unavailable +StrategyType: RollingUpdate +MinReadySeconds: 0 +RollingUpdateStrategy: 1 max unavailable, 1 max surge +OldReplicaSets: +NewReplicaSet: nginx-3137573019 (0/1 replicas created) +... ``` -The Kubernetes API server is rejecting the ReplicaSet requests to create a pod because our pods -do not specify any memory usage *request*. +A deployment created a corresponding replica set and attempted to size it to create a single pod. -So let's set some default values for the amount of cpu and memory a pod can consume: +Let's look at the replica set to get more detail. + +```shell +$ kubectl describe rs nginx-3137573019 --namespace=quota-example +Name: nginx-3137573019 +Namespace: quota-example +Image(s): nginx +Selector: pod-template-hash=3137573019,run=nginx +Labels: pod-template-hash=3137573019 + run=nginx +Replicas: 0 current / 1 desired +Pods Status: 0 Running / 0 Waiting / 0 Succeeded / 0 Failed +No volumes. +Events: + FirstSeen LastSeen Count From SubobjectPath Type Reason Message + --------- -------- ----- ---- ------------- -------- ------ ------- + 4m 7s 11 {replicaset-controller } Warning FailedCreate Error creating: pods "nginx-3137573019-" is forbidden: Failed quota: compute-resources: must specify limits.cpu,limits.memory,requests.cpu,requests.memory +``` + +The Kubernetes API server is rejecting the replica set's request to create a pod because our pods +do not specify `requests` or `limits` for `cpu` and `memory`. + +So let's set some default values for the amount of `cpu` and `memory` a pod can consume: ```shell $ kubectl create -f docs/admin/resourcequota/limits.yaml --namespace=quota-example limitrange "limits" created $ kubectl describe limits limits --namespace=quota-example -Name: limits -Namespace: quota-example -Type Resource Min Max Default Request Default Limit Max Limit/Request Ratio ----- -------- --- --- --------------- ------------- ----------------------- -Container cpu - - 100m 200m - -Container memory - - 256Mi 512Mi - +Name: limits +Namespace: quota-example +Type Resource Min Max Default Request Default Limit Max Limit/Request Ratio +---- -------- --- --- --------------- ------------- ----------------------- +Container memory - - 256Mi 512Mi - +Container cpu - - 100m 200m - ``` -Now any time a pod is created in this namespace, if it has not specified any resource request/limit, the default -amount of cpu and memory per container will be applied, and the request will be used as part of admission control. +If the Kubernetes API server observes a request to create a pod in this namespace, and the containers +in that pod do not make any compute resource requests, a default request and default limit will be applied +as part of admission control. -Now that we have applied default resource *request* for our namespace, our Deployment should be able to -create its pods. +In this example, each pod created will have compute resources equivalent to the following: + +```shell +$ kubectl run nginx \ + --image=nginx \ + --replicas=1 \ + --requests=cpu=100m,memory=256Mi \ + --limits=cpu=200m,memory=512Mi \ + --namespace=quota-example +``` + +Now that we have applied default compute resources for our namespace, our replica set should be able to create +its pods. ```shell $ kubectl get pods --namespace=quota-example NAME READY STATUS RESTARTS AGE -nginx-2040093540-miohp 1/1 Running 0 5s +nginx-3137573019-fvrig 1/1 Running 0 6m ``` And if we print out our quota usage in the namespace: ```shell -$ kubectl describe quota quota --namespace=quota-example -Name: quota -Namespace: quota-example -Resource Used Hard --------- ---- ---- -cpu 100m 20 -memory 256Mi 1Gi -persistentvolumeclaims 0 10 -pods 1 10 -replicationcontrollers 1 20 -resourcequotas 1 1 -secrets 1 10 -services 0 5 +$ kubectl describe quota --namespace=quota-example +Name: compute-resources +Namespace: quota-example +Resource Used Hard +-------- ---- ---- +limits.cpu 200m 2 +limits.memory 512Mi 2Gi +pods 1 4 +requests.cpu 100m 1 +requests.memory 256Mi 1Gi + + +Name: object-counts +Namespace: quota-example +Resource Used Hard +-------- ---- ---- +persistentvolumeclaims 0 2 ``` -You can now see the pod that was created is consuming explicit amounts of resources (specified by resource *request*), and the usage is being tracked by the Kubernetes system properly. +As you can see, the pod that was created is consuming explict amounts of compute resources, and the usage is being +tracked by Kubernetes properly. + +## Step 5: Advanced quota scopes + +Let's imagine you did not want to specify default compute resource consumption in your namespace. + +Instead, you want to let users run a specific number of `BestEffort` pods in their namespace to take +advantage of slack compute resources, and then require that users make an explicit resource request for +pods that require a higher quality of service. + +Let's create a new namespace with two quotas to demonstrate this behavior: + +```shell +$ kubectl create namespace quota-scopes +namespace "quota-scopes" created +$ kubectl create -f docs/admin/resourcequota/best-effort.yaml --namespace=quota-scopes +resourcequota "best-effort" created +$ kubectl create -f docs/admin/resourcequota/not-best-effort.yaml --namespace=quota-scopes +resourcequota "not-best-effort" created +$ kubectl describe quota --namespace=quota-scopes +Name: best-effort +Namespace: quota-scopes +Scopes: BestEffort + * Matches all pods that have best effort quality of service. +Resource Used Hard +-------- ---- ---- +pods 0 10 + + +Name: not-best-effort +Namespace: quota-scopes +Scopes: NotBestEffort + * Matches all pods that do not have best effort quality of service. +Resource Used Hard +-------- ---- ---- +limits.cpu 0 2 +limits.memory 0 2Gi +pods 0 4 +requests.cpu 0 1 +requests.memory 0 1Gi +``` + +In this scenario, a pod that makes no compute resource requests will be tracked by the `best-effort` quota. + +A pod that does make compute resource requests will be tracked by the `not-best-effort` quota. + +Let's demonstrate this by creating two deployments: + +```shell +$ kubectl run best-effort-nginx --image=nginx --replicas=8 --namespace=quota-scopes +deployment "best-effort-nginx" created +$ kubectl run not-best-effort-nginx \ + --image=nginx \ + --replicas=2 \ + --requests=cpu=100m,memory=256Mi \ + --limits=cpu=200m,memory=512Mi \ + --namespace=quota-scopes +deployment "not-best-effort-nginx" created +``` + +Even though no default limits were specified, the `best-effort-nginx` deployment will create +all 8 pods. This is because it is tracked by the `best-effort` quota, and the `not-best-effort` +quota will just ignore it. The `not-best-effort` quota will track the `not-best-effort-nginx` +deployment since it creates pods with `Burstable` quality of service. + +Let's list the pods in the namespace: + +```shell +$ kubectl get pods --namespace=quota-scopes +NAME READY STATUS RESTARTS AGE +best-effort-nginx-3488455095-2qb41 1/1 Running 0 51s +best-effort-nginx-3488455095-3go7n 1/1 Running 0 51s +best-effort-nginx-3488455095-9o2xg 1/1 Running 0 51s +best-effort-nginx-3488455095-eyg40 1/1 Running 0 51s +best-effort-nginx-3488455095-gcs3v 1/1 Running 0 51s +best-effort-nginx-3488455095-rq8p1 1/1 Running 0 51s +best-effort-nginx-3488455095-udhhd 1/1 Running 0 51s +best-effort-nginx-3488455095-zmk12 1/1 Running 0 51s +not-best-effort-nginx-2204666826-7sl61 1/1 Running 0 23s +not-best-effort-nginx-2204666826-ke746 1/1 Running 0 23s +``` + +As you can see, all 10 pods have been allowed to be created. + +Let's describe current quota usage in the namespace: + +```shell +$ kubectl describe quota --namespace=quota-scopes +Name: best-effort +Namespace: quota-scopes +Scopes: BestEffort + * Matches all pods that have best effort quality of service. +Resource Used Hard +-------- ---- ---- +pods 8 10 + + +Name: not-best-effort +Namespace: quota-scopes +Scopes: NotBestEffort + * Matches all pods that do not have best effort quality of service. +Resource Used Hard +-------- ---- ---- +limits.cpu 400m 2 +limits.memory 1Gi 2Gi +pods 2 4 +requests.cpu 200m 1 +requests.memory 512Mi 1Gi +``` + +As you can see, the `best-effort` quota has tracked the usage for the 8 pods we created in +the `best-effort-nginx` deployment, and the `not-best-effort` quota has tracked the usage for +the 2 pods we created in the `not-best-effort-nginx` quota. + +Scopes provide a mechanism to subdivide the set of resources that are tracked by +any quota document to allow greater flexibility in how operators deploy and track resource +consumption. + +In addition to `BestEffort` and `NotBestEffort` scopes, there are scopes to restrict +long-running versus time-bound pods. The `Terminating` scope will match any pod +where `spec.activeDeadlineSeconds is not nil`. The `NotTerminating` scope will match any pod +where `spec.activeDeadlineSeconds is nil`. These scopes allow you to quota pods based on their +anticipated permanence on a node in your cluster. ## Summary -Actions that consume node resources for cpu and memory can be subject to hard quota limits defined by the namespace quota. The resource consumption is measured by resource *request* in pod specification. +Actions that consume node resources for cpu and memory can be subject to hard quota limits defined by the namespace quota. Any action that consumes those resources can be tweaked, or can pick up namespace level defaults to meet your end goal. + +Quota can be apportioned based on quality of service and anticipated permanence on a node in your cluster. \ No newline at end of file From d1898b59df52520284d9727974ba9797d20ac45d Mon Sep 17 00:00:00 2001 From: dlorenc Date: Mon, 27 Jun 2016 14:22:41 -0700 Subject: [PATCH 129/348] Add minikube docs to the "Downloading or Building Kubernetes" page. --- docs/getting-started-guides/binary_release.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/getting-started-guides/binary_release.md b/docs/getting-started-guides/binary_release.md index 667b5f025d..9429b3a7df 100644 --- a/docs/getting-started-guides/binary_release.md +++ b/docs/getting-started-guides/binary_release.md @@ -3,6 +3,8 @@ You can either build a release from sources or download a pre-built release. If you do not plan on developing Kubernetes itself, we suggest a pre-built release. +If you just want to run Kubernetes locally, we recommend using Minikube. You can download Minikube [here](https://github.com/kubernetes/minikube/releases/latest). + * TOC {:toc} From 8b3359f75d2884ed03aa05b498212419ec2d90db Mon Sep 17 00:00:00 2001 From: Girish Kalele Date: Mon, 27 Jun 2016 16:13:11 -0700 Subject: [PATCH 130/348] Fix DNS documents to remove references to DNS as an optional addon. --- docs/admin/dns.md | 29 ++++++++++++++--------------- 1 file changed, 14 insertions(+), 15 deletions(-) diff --git a/docs/admin/dns.md b/docs/admin/dns.md index 2e37cdaf83..d173e6813e 100644 --- a/docs/admin/dns.md +++ b/docs/admin/dns.md @@ -1,8 +1,8 @@ --- --- -As of Kubernetes 0.8, DNS is offered as a [cluster add-on](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/README.md). -If enabled, a DNS Pod and Service will be scheduled on the cluster, and the kubelets will be +As of Kubernetes 1.3, DNS is a built-in service launched automatically using the addon manager [cluster add-on](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/README.md). +A DNS Pod and Service will be scheduled on the cluster, and the kubelets will be configured to tell individual containers to use the DNS Service's IP to resolve DNS names. Every Service defined in the cluster (including the DNS server itself) will be @@ -15,25 +15,24 @@ in namespace `bar` can look up this service by simply doing a DNS query for `foo`. A Pod running in namespace `quux` can look up this service by doing a DNS query for `foo.bar`. -The cluster DNS server ([SkyDNS](https://github.com/skynetservices/skydns)) -supports forward lookups (A records) and service lookups (SRV records). +The Kubernetes cluster DNS server (based off the [SkyDNS](https://github.com/skynetservices/skydns) library) +supports forward lookups (A records), service lookups (SRV records) and reverse IP address lookups (PTR records). + ## How it Works -The running DNS pod holds 4 containers - skydns, etcd (a private instance which skydns uses), -a Kubernetes-to-skydns bridge called kube2sky, and a health check called healthz. The kube2sky process -watches the Kubernetes master for changes in Services, and then writes the -information to etcd, which skydns reads. This etcd instance is not linked to -any other etcd clusters that might exist, including the Kubernetes master. +The running Kubernetes DNS pod holds 3 containers - kubedns, dnsmasq and a health check called healthz. +The kubedns process watches the Kubernetes master for changes in Services and Endpoints, and maintains +in-memory lookup structures to service DNS requests. The dnsmasq container adds DNS caching to improve +performance. The healthz container provides a single health check endpoint while performing dual healthchecks +(for dnsmasq and kubedns). ## Issues -The skydns service is reachable directly from Kubernetes nodes (outside -of any container) and DNS resolution works if the skydns service is targeted -explicitly. However, nodes are not configured to use the cluster DNS service or -to search the cluster's DNS domain by default. This may be resolved at a later -time. +The 1.3 release introduced Federation (Ubernetes) support for multisite Kubernetes installations. There are +DNS changes introduced that will allow the lookup of services using a six part DNS name. +See [Federation docs](https://kubernetes.github.io/docs/admin/multiple-zones.md) ## For more information -See [the docs for the DNS cluster addon](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/dns/README.md). \ No newline at end of file +See [the docs for the DNS cluster addon](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/dns/README.md). From 0e49d214f907db101deaa1626fca687c81a0c583 Mon Sep 17 00:00:00 2001 From: Girish Kalele Date: Mon, 27 Jun 2016 16:14:21 -0700 Subject: [PATCH 131/348] Revert "Fix DNS documents to remove references to DNS as an optional addon." This reverts commit 8b3359f75d2884ed03aa05b498212419ec2d90db. --- docs/admin/dns.md | 29 +++++++++++++++-------------- 1 file changed, 15 insertions(+), 14 deletions(-) diff --git a/docs/admin/dns.md b/docs/admin/dns.md index d173e6813e..2e37cdaf83 100644 --- a/docs/admin/dns.md +++ b/docs/admin/dns.md @@ -1,8 +1,8 @@ --- --- -As of Kubernetes 1.3, DNS is a built-in service launched automatically using the addon manager [cluster add-on](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/README.md). -A DNS Pod and Service will be scheduled on the cluster, and the kubelets will be +As of Kubernetes 0.8, DNS is offered as a [cluster add-on](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/README.md). +If enabled, a DNS Pod and Service will be scheduled on the cluster, and the kubelets will be configured to tell individual containers to use the DNS Service's IP to resolve DNS names. Every Service defined in the cluster (including the DNS server itself) will be @@ -15,24 +15,25 @@ in namespace `bar` can look up this service by simply doing a DNS query for `foo`. A Pod running in namespace `quux` can look up this service by doing a DNS query for `foo.bar`. -The Kubernetes cluster DNS server (based off the [SkyDNS](https://github.com/skynetservices/skydns) library) -supports forward lookups (A records), service lookups (SRV records) and reverse IP address lookups (PTR records). - +The cluster DNS server ([SkyDNS](https://github.com/skynetservices/skydns)) +supports forward lookups (A records) and service lookups (SRV records). ## How it Works -The running Kubernetes DNS pod holds 3 containers - kubedns, dnsmasq and a health check called healthz. -The kubedns process watches the Kubernetes master for changes in Services and Endpoints, and maintains -in-memory lookup structures to service DNS requests. The dnsmasq container adds DNS caching to improve -performance. The healthz container provides a single health check endpoint while performing dual healthchecks -(for dnsmasq and kubedns). +The running DNS pod holds 4 containers - skydns, etcd (a private instance which skydns uses), +a Kubernetes-to-skydns bridge called kube2sky, and a health check called healthz. The kube2sky process +watches the Kubernetes master for changes in Services, and then writes the +information to etcd, which skydns reads. This etcd instance is not linked to +any other etcd clusters that might exist, including the Kubernetes master. ## Issues -The 1.3 release introduced Federation (Ubernetes) support for multisite Kubernetes installations. There are -DNS changes introduced that will allow the lookup of services using a six part DNS name. -See [Federation docs](https://kubernetes.github.io/docs/admin/multiple-zones.md) +The skydns service is reachable directly from Kubernetes nodes (outside +of any container) and DNS resolution works if the skydns service is targeted +explicitly. However, nodes are not configured to use the cluster DNS service or +to search the cluster's DNS domain by default. This may be resolved at a later +time. ## For more information -See [the docs for the DNS cluster addon](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/dns/README.md). +See [the docs for the DNS cluster addon](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/dns/README.md). \ No newline at end of file From 469f48ae21009a877767621bb0cca0c1e2180679 Mon Sep 17 00:00:00 2001 From: Takeshi Takizawa Date: Tue, 28 Jun 2016 19:02:23 +0900 Subject: [PATCH 132/348] Fix can't start service etcd is missing in centos-master KUBE_MASTER is not localhost in centos-minion --- .../centos/centos_manual_config.md | 34 +++++++++++++------ 1 file changed, 24 insertions(+), 10 deletions(-) diff --git a/docs/getting-started-guides/centos/centos_manual_config.md b/docs/getting-started-guides/centos/centos_manual_config.md index 9e09cabfcf..f2bc49099f 100644 --- a/docs/getting-started-guides/centos/centos_manual_config.md +++ b/docs/getting-started-guides/centos/centos_manual_config.md @@ -20,6 +20,8 @@ The Kubernetes package provides a few services: kube-apiserver, kube-scheduler, Hosts: +Please replace host IP with your environment. + ```conf centos-master = 192.168.121.9 centos-minion = 192.168.121.65 @@ -27,7 +29,7 @@ centos-minion = 192.168.121.65 **Prepare the hosts:** -* Create a virt7-docker-common-release repo on all hosts - centos-{master,minion} with following information. +* Create a /etc/yum.repos.d/virt7-docker-common-release.repo on all hosts - centos-{master,minion} with following information. ```conf [virt7-docker-common-release] @@ -36,10 +38,10 @@ baseurl=http://cbs.centos.org/repos/virt7-docker-common-release/x86_64/os/ gpgcheck=0 ``` -* Install Kubernetes on all hosts - centos-{master,minion}. This will also pull in etcd, docker, and cadvisor. +* Install Kubernetes and etcd on all hosts - centos-{master,minion}. This will also pull in docker and cadvisor. ```shell -yum -y install --enablerepo=virt7-docker-common-release kubernetes +yum -y install --enablerepo=virt7-docker-common-release kubernetes etcd ``` * Add master and node to /etc/hosts on all machines (not needed if hostnames already in DNS) @@ -63,6 +65,9 @@ KUBE_LOG_LEVEL="--v=0" # Should this cluster be allowed to run privileged docker containers KUBE_ALLOW_PRIV="--allow-privileged=false" + +# How the replication controller and scheduler find the kube-apiserver +KUBE_MASTER="--master=http://centos-master:8080" ``` * Disable the firewall on both the master and node, as docker does not play well with other firewall rule managers @@ -74,6 +79,18 @@ systemctl stop iptables-services firewalld **Configure the Kubernetes services on the master.** +* Edit /etc/etcd/etcd.conf to appear as such: + +```shell +# [member] +ETCD_NAME=default +ETCD_DATA_DIR="/var/lib/etcd/default.etcd" +ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379" + +#[cluster] +ETCD_ADVERTISE_CLIENT_URLS="http://0.0.0.0:2379" +``` + * Edit /etc/kubernetes/apiserver to appear as such: ```shell @@ -83,9 +100,6 @@ KUBE_API_ADDRESS="--address=0.0.0.0" # The port on the local server to listen on. KUBE_API_PORT="--port=8080" -# How the replication controller and scheduler find the kube-apiserver -KUBE_MASTER="--master=http://centos-master:8080" - # Port kubelets listen on KUBELET_PORT="--kubelet-port=10250" @@ -99,10 +113,10 @@ KUBE_API_ARGS="" * Start the appropriate services on master: ```shell -for SERVICES in etcd kube-apiserver kube-controller-manager kube-scheduler; do +for SERVICES in etcd kube-apiserver kube-controller-manager kube-scheduler; do systemctl restart $SERVICES systemctl enable $SERVICES - systemctl status $SERVICES + systemctl status $SERVICES done ``` @@ -132,10 +146,10 @@ KUBELET_ARGS="" * Start the appropriate services on node (centos-minion). ```shell -for SERVICES in kube-proxy kubelet docker; do +for SERVICES in kube-proxy kubelet docker; do systemctl restart $SERVICES systemctl enable $SERVICES - systemctl status $SERVICES + systemctl status $SERVICES done ``` From 486d8411bbbdbd5ec863833e37aac067297ebabe Mon Sep 17 00:00:00 2001 From: Mike Brown Date: Tue, 28 Jun 2016 09:00:35 -0500 Subject: [PATCH 133/348] promote slack use for novice and dev channels Signed-off-by: Mike Brown --- docs/troubleshooting.md | 39 ++++++++++++++++++++++++++++----------- 1 file changed, 28 insertions(+), 11 deletions(-) diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index feb6d1cf69..0fcdcb40af 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -3,20 +3,24 @@ ## Troubleshooting -Sometimes things go wrong. This guide is aimed at making them right. It has two sections: +Sometimes things go wrong. This guide is aimed at making them right. It has +two sections: * [Troubleshooting your application](/docs/user-guide/application-troubleshooting) - Useful for users who are deploying code into Kubernetes and wondering why it is not working. * [Troubleshooting your cluster](/docs/admin/cluster-troubleshooting) - Useful for cluster administrators and people whose Kubernetes cluster is unhappy. -You should also check the known issues for the [release](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG.md) you're using. +You should also check the known issues for the [release](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG.md) +you're using. ### Getting help -If your problem isn't answered by any of the guides above, there are variety of ways for you to get help from the Kubernetes team. +If your problem isn't answered by any of the guides above, there are variety of +ways for you to get help from the Kubernetes team. ### Questions -If you aren't familiar with it, many of your questions may be answered by the [user guide](/docs/user-guide/). +If you aren't familiar with it, many of your questions may be answered by the +[user guide](/docs/user-guide/). We also have a number of FAQ pages: @@ -33,11 +37,23 @@ You may also find the Stack Overflow topics relevant: ### Stack Overflow -Someone else from the community may have already asked a similar question or may be able to help with your problem. The Kubernetes team will also monitor [posts tagged kubernetes](http://stackoverflow.com/questions/tagged/kubernetes). If there aren't any existing questions that help, please [ask a new one](http://stackoverflow.com/questions/ask?tags=kubernetes)! +Someone else from the community may have already asked a similar question or may +be able to help with your problem. The Kubernetes team will also monitor +[posts tagged kubernetes](http://stackoverflow.com/questions/tagged/kubernetes). +If there aren't any existing questions that help, please [ask a new one](http://stackoverflow.com/questions/ask?tags=kubernetes)! ### Slack -The Kubernetes team hangs out on Slack in the `#kubernetes-users` channel. You can participate in the Kubernetes team [here](https://kubernetes.slack.com). Slack requires registration, but the Kubernetes team is open invitation to anyone to register [here](http://slack.kubernetes.io). Feel free to come and ask any and all questions. +The Kubernetes team hangs out on Slack in the `#kubernetes-users` channel. You +can participate in discussion with the Kubernetes team [here](https://kubernetes.slack.com). +Slack requires registration, but the Kubernetes team is open invitation to +anyone to register [here](http://slack.kubernetes.io). Feel free to come and ask +any and all questions. + +Once registered, browse the growing list of channels for various subjects of +interest. For example, people new to Kubernetes may also want to join the +`#kubernetes-novice` channel. As another example, developers should join the +`#kubernetes-dev` channel. ### Mailing List @@ -45,14 +61,15 @@ The Google Container Engine mailing list is [google-containers@googlegroups.com] ### Bugs and Feature requests -If you have what looks like a bug, or you would like to make a feature request, please use the [Github issue tracking system](https://github.com/kubernetes/kubernetes/issues). +If you have what looks like a bug, or you would like to make a feature request, +please use the [Github issue tracking system](https://github.com/kubernetes/kubernetes/issues). -Before you file an issue, please search existing issues to see if your issue is already covered. +Before you file an issue, please search existing issues to see if your issue is +already covered. -If filing a bug, please include detailed information about how to reproduce the problem, such as: +If filing a bug, please include detailed information about how to reproduce the +problem, such as: * Kubernetes version: `kubectl version` * Cloud provider, OS distro, network configuration, and Docker version * Steps to reproduce the problem - - From c1206d7b5398eca1c3b036deff0adee29403a708 Mon Sep 17 00:00:00 2001 From: Casey Davenport Date: Sat, 4 Jun 2016 10:23:57 -0700 Subject: [PATCH 134/348] First commit of network policy user guide --- docs/user-guide/networkpolicies.md | 82 ++++++++++++++++++++++++++++++ 1 file changed, 82 insertions(+) create mode 100644 docs/user-guide/networkpolicies.md diff --git a/docs/user-guide/networkpolicies.md b/docs/user-guide/networkpolicies.md new file mode 100644 index 0000000000..06824c692c --- /dev/null +++ b/docs/user-guide/networkpolicies.md @@ -0,0 +1,82 @@ +--- +--- + +* TOC +{:toc} + +## What is a _Network Policy_? + +A Network Policy is a specification of how groupings of pods are allowed to communicate with each other and other network endpoints. + +NetworkPolicy resources use labels to select pods and define whitelist rules which allow traffic to the selected pods in addition to what is allowed by the ingress isolation policy for a given namespace. + +## Prerequisites +Before you start using the NetworkPolicy resource, there are a few things to understand. The NetworkPolicy resource is a beta resource and is +not available in any Kubernetes release prior to 1.3. + +You must enable the `extensions/v1beta/networkpolicies` runtime config in your apiserver to enable this resource. + +You must also be using a networking solution which supports Network Policy - simply creating the +resource without a controller to implement it will have no effect. + +## Configuring Namespace Isolation Policy +Ingress isolation can be configured on a per-namespace basis. Once ingress isolation is configured on a namespace it will be applied to all pods in that namespace. + +Currently the following ingress isolation types are supported: + +- _DefaultDeny_: Pods in the namespace will be inaccessible from any source except the pod's local node. + +Ingress isolation can be enabled using an annotation on the Namespace. + +```yaml +kind: Namespace +apiVersion: v1 +metadata: + annotations: + net.beta.kubernetes.io/network-policy: | + { + "ingress": { + "isolation": "DefaultDeny" + } + } +``` + +To configure the annotation via `kubectl`: + +``` +kubectl annotate ns "net.beta.kubernetes.io/networkpolicy={\"ingress\": {\"isolation\": \"DefaultDeny\"}}" +``` + +## The NetworkPolicy Resource +A minimal `NetworkPolicy` might look like this: + +```yaml +01. apiVersion: extensions/v1beta1 +02. kind: NetworkPolicy +03. metadata: +04. name: test-network-policy +05. spec: +06. podSelector: +07. matchLabels: +08. role: db +09. ingress: +10. - from: +11. podSelector: +12. matchLabels: +13. role: frontend +14. ports: +15. - protocol: tcp +16. port: 6379 +``` + +*POSTing this to the API server will have no effect unless your chosen networking solution supports network policy.* + +__Lines 1-4__: As with all other Kubernetes config, a NetworkPolicy needs `apiVersion`, `kind`, and `metadata` fields. For general information about working with config files, see [here](/docs/user-guide/simple-yaml), [here](/docs/user-guide/configuring-containers), and [here](/docs/user-guide/working-with-resources). + +__Lines 5-9__: NetworkPolicy [spec](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/devel/api-conventions.md#spec-and-status) has all the information needed to configure a loadbalancer or proxy server. Most importantly, it contains a list of rules matched against all incoming requests. Currently the Ingress resource only supports http rules. + +__Lines 6-8__: Each NetworkPolicy includes a `podSelector` which selects the grouping of pods to which the `ingress` rules in the policy apply. + +__Lines 9-16__: Each NetworkPolicy includes a list of whitelist `ingress` rules. Each rule allows traffic which matches both the `from` and `ports` sections. + +__Complete Specification__: See the [api-reference](https://kubernetes.github.io/docs/api-reference/extensions/v1beta1/definitions/#_v1beta1_networkpolicy) for a full definition of the resource. From cb6f85b484f88308300a191c93bdaa857beb96c7 Mon Sep 17 00:00:00 2001 From: Casey Davenport Date: Mon, 27 Jun 2016 15:58:08 -0700 Subject: [PATCH 135/348] Code review comments --- _data/reference.yml | 2 + docs/user-guide/networkpolicies.md | 68 +++++++++++++++--------------- 2 files changed, 35 insertions(+), 35 deletions(-) diff --git a/_data/reference.yml b/_data/reference.yml index e2439cc78b..f337e158b1 100644 --- a/_data/reference.yml +++ b/_data/reference.yml @@ -226,6 +226,8 @@ toc: path: /docs/user-guide/replicasets/ - title: Pet Sets path: /docs/user-guide/petset/ + - title: NetworkPolicy Resources + path: /docs/user-guide/networkpolicies/ - title: Kubernetes Design Docs section: diff --git a/docs/user-guide/networkpolicies.md b/docs/user-guide/networkpolicies.md index 06824c692c..7050b9df81 100644 --- a/docs/user-guide/networkpolicies.md +++ b/docs/user-guide/networkpolicies.md @@ -4,27 +4,24 @@ * TOC {:toc} -## What is a _Network Policy_? +A network policy is a specification of how selections of pods are allowed to communicate with each other and other network endpoints. -A Network Policy is a specification of how groupings of pods are allowed to communicate with each other and other network endpoints. - -NetworkPolicy resources use labels to select pods and define whitelist rules which allow traffic to the selected pods in addition to what is allowed by the ingress isolation policy for a given namespace. +`NetworkPolicy` resources use labels to select pods and define whitelist rules which allow traffic to the selected pods in addition to what is allowed by the isolation policy for a given namespace. ## Prerequisites -Before you start using the NetworkPolicy resource, there are a few things to understand. The NetworkPolicy resource is a beta resource and is -not available in any Kubernetes release prior to 1.3. You must enable the `extensions/v1beta/networkpolicies` runtime config in your apiserver to enable this resource. -You must also be using a networking solution which supports Network Policy - simply creating the +You must also be using a networking solution which supports `NetworkPolicy` - simply creating the resource without a controller to implement it will have no effect. ## Configuring Namespace Isolation Policy -Ingress isolation can be configured on a per-namespace basis. Once ingress isolation is configured on a namespace it will be applied to all pods in that namespace. -Currently the following ingress isolation types are supported: +Isolation can be configured on a per-namespace basis. Once isolation is configured on a namespace it will be applied to all pods in that namespace. Currently, only isolation policy on inbound traffic (ingress) can be defined. -- _DefaultDeny_: Pods in the namespace will be inaccessible from any source except the pod's local node. +The following ingress isolation types being supported: + +- `DefaultDeny`: Pods in the namespace will be inaccessible from any source except the pod's local node. Ingress isolation can be enabled using an annotation on the Namespace. @@ -43,40 +40,41 @@ metadata: To configure the annotation via `kubectl`: -``` +```shell{% raw %} kubectl annotate ns "net.beta.kubernetes.io/networkpolicy={\"ingress\": {\"isolation\": \"DefaultDeny\"}}" -``` +{% endraw %}``` + +## The `NetworkPolicy` Resource + +See the [api-reference](docs/api-reference/extensions/v1beta1/definitions/#_v1beta1_networkpolicy) for a full definition of the resource. -## The NetworkPolicy Resource A minimal `NetworkPolicy` might look like this: ```yaml -01. apiVersion: extensions/v1beta1 -02. kind: NetworkPolicy -03. metadata: -04. name: test-network-policy -05. spec: -06. podSelector: -07. matchLabels: -08. role: db -09. ingress: -10. - from: -11. podSelector: -12. matchLabels: -13. role: frontend -14. ports: -15. - protocol: tcp -16. port: 6379 +apiVersion: extensions/v1beta1 +kind: NetworkPolicy +metadata: + name: test-network-policy +spec: + podSelector: + matchLabels: + role: db + ingress: + - from: + podSelector: + matchLabels: + role: frontend + ports: + - protocol: tcp + port: 6379 ``` *POSTing this to the API server will have no effect unless your chosen networking solution supports network policy.* -__Lines 1-4__: As with all other Kubernetes config, a NetworkPolicy needs `apiVersion`, `kind`, and `metadata` fields. For general information about working with config files, see [here](/docs/user-guide/simple-yaml), [here](/docs/user-guide/configuring-containers), and [here](/docs/user-guide/working-with-resources). +__Mandatory Fields__: As with all other Kubernetes config, a `NetworkPolicy` needs `apiVersion`, `kind`, and `metadata` fields. For general information about working with config files, see [here](/docs/user-guide/simple-yaml), [here](/docs/user-guide/configuring-containers), and [here](/docs/user-guide/working-with-resources). -__Lines 5-9__: NetworkPolicy [spec](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/devel/api-conventions.md#spec-and-status) has all the information needed to configure a loadbalancer or proxy server. Most importantly, it contains a list of rules matched against all incoming requests. Currently the Ingress resource only supports http rules. +__spec__: `NetworkPolicy` [spec](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/docs/devel/api-conventions.md#spec-and-status) has all the information needed to define a network isolation policy in the deployed controller. -__Lines 6-8__: Each NetworkPolicy includes a `podSelector` which selects the grouping of pods to which the `ingress` rules in the policy apply. +__podSelector__: Each `NetworkPolicy` includes a `podSelector` which selects the grouping of pods to which the `ingress` rules in the policy apply. -__Lines 9-16__: Each NetworkPolicy includes a list of whitelist `ingress` rules. Each rule allows traffic which matches both the `from` and `ports` sections. - -__Complete Specification__: See the [api-reference](https://kubernetes.github.io/docs/api-reference/extensions/v1beta1/definitions/#_v1beta1_networkpolicy) for a full definition of the resource. +__ingress__: Each `NetworkPolicy` includes a list of whitelist `ingress` rules. Each rule allows traffic which matches both the `from` and `ports` sections. From ad53cd7423f5f98f1ee3a4fd4f404b58345cd6bf Mon Sep 17 00:00:00 2001 From: Casey Davenport Date: Tue, 28 Jun 2016 13:23:30 -0700 Subject: [PATCH 136/348] Fix title --- _data/reference.yml | 2 +- docs/user-guide/networkpolicies.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/_data/reference.yml b/_data/reference.yml index f337e158b1..a11214565a 100644 --- a/_data/reference.yml +++ b/_data/reference.yml @@ -226,7 +226,7 @@ toc: path: /docs/user-guide/replicasets/ - title: Pet Sets path: /docs/user-guide/petset/ - - title: NetworkPolicy Resources + - title: Network Policies path: /docs/user-guide/networkpolicies/ - title: Kubernetes Design Docs diff --git a/docs/user-guide/networkpolicies.md b/docs/user-guide/networkpolicies.md index 7050b9df81..e550657df6 100644 --- a/docs/user-guide/networkpolicies.md +++ b/docs/user-guide/networkpolicies.md @@ -46,7 +46,7 @@ kubectl annotate ns "net.beta.kubernetes.io/networkpolicy={\"ingress ## The `NetworkPolicy` Resource -See the [api-reference](docs/api-reference/extensions/v1beta1/definitions/#_v1beta1_networkpolicy) for a full definition of the resource. +See the [api-reference](/docs/api-reference/extensions/v1beta1/definitions/#_v1beta1_networkpolicy) for a full definition of the resource. A minimal `NetworkPolicy` might look like this: From c634053ec17c603b774688059253e7b976a6a572 Mon Sep 17 00:00:00 2001 From: Girish Kalele Date: Mon, 27 Jun 2016 16:13:11 -0700 Subject: [PATCH 137/348] Fix DNS documents to remove references to DNS as an optional addon. --- docs/admin/dns.md | 36 +++++++++++++++++++----------------- 1 file changed, 19 insertions(+), 17 deletions(-) diff --git a/docs/admin/dns.md b/docs/admin/dns.md index 2e37cdaf83..df34fbf8a9 100644 --- a/docs/admin/dns.md +++ b/docs/admin/dns.md @@ -1,8 +1,10 @@ --- --- -As of Kubernetes 0.8, DNS is offered as a [cluster add-on](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/README.md). -If enabled, a DNS Pod and Service will be scheduled on the cluster, and the kubelets will be +## Introduction + +As of Kubernetes 1.3, DNS is a built-in service launched automatically using the addon manager [cluster add-on](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/README.md). +A DNS Pod and Service will be scheduled on the cluster, and the kubelets will be configured to tell individual containers to use the DNS Service's IP to resolve DNS names. Every Service defined in the cluster (including the DNS server itself) will be @@ -15,25 +17,25 @@ in namespace `bar` can look up this service by simply doing a DNS query for `foo`. A Pod running in namespace `quux` can look up this service by doing a DNS query for `foo.bar`. -The cluster DNS server ([SkyDNS](https://github.com/skynetservices/skydns)) -supports forward lookups (A records) and service lookups (SRV records). +The Kubernetes cluster DNS server (based off the [SkyDNS](https://github.com/skynetservices/skydns) library) +supports forward lookups (A records), service lookups (SRV records) and reverse IP address lookups (PTR records). + ## How it Works -The running DNS pod holds 4 containers - skydns, etcd (a private instance which skydns uses), -a Kubernetes-to-skydns bridge called kube2sky, and a health check called healthz. The kube2sky process -watches the Kubernetes master for changes in Services, and then writes the -information to etcd, which skydns reads. This etcd instance is not linked to -any other etcd clusters that might exist, including the Kubernetes master. +The running Kubernetes DNS pod holds 3 containers - kubedns, dnsmasq and a health check called healthz. +The kubedns process watches the Kubernetes master for changes in Services and Endpoints, and maintains +in-memory lookup structures to service DNS requests. The dnsmasq container adds DNS caching to improve +performance. The healthz container provides a single health check endpoint while performing dual healthchecks +(for dnsmasq and kubedns). -## Issues +## Kubernetes Federation (Multiple Zone support) -The skydns service is reachable directly from Kubernetes nodes (outside -of any container) and DNS resolution works if the skydns service is targeted -explicitly. However, nodes are not configured to use the cluster DNS service or -to search the cluster's DNS domain by default. This may be resolved at a later -time. +The 1.3 release introduced Federation (Ubernetes) support for multisite Kubernetes installations. There are +DNS changes introduced that will allow the lookup of services using a six part DNS name. +See [Federation docs](/docs/admin/multiple-zones/) for more details on multiple site support. -## For more information +## References + +- [Docs for the DNS cluster addon](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/dns/README.md) -See [the docs for the DNS cluster addon](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/dns/README.md). \ No newline at end of file From dcc4ccefac189f29b76c8154dda7664284c70426 Mon Sep 17 00:00:00 2001 From: dlorenc Date: Tue, 28 Jun 2016 14:50:02 -0700 Subject: [PATCH 138/348] Add more details about minikube, per feedback. --- docs/getting-started-guides/binary_release.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/getting-started-guides/binary_release.md b/docs/getting-started-guides/binary_release.md index 9429b3a7df..7280e7b89f 100644 --- a/docs/getting-started-guides/binary_release.md +++ b/docs/getting-started-guides/binary_release.md @@ -3,7 +3,8 @@ You can either build a release from sources or download a pre-built release. If you do not plan on developing Kubernetes itself, we suggest a pre-built release. -If you just want to run Kubernetes locally, we recommend using Minikube. You can download Minikube [here](https://github.com/kubernetes/minikube/releases/latest). +If you just want to run Kubernetes locally for development, we recommend using Minikube. You can download Minikube [here](https://github.com/kubernetes/minikube/releases/latest). +Minikube sets up a local VM that runs a Kubernetes cluster securely, and makes it easy to work with that cluster. * TOC {:toc} From b012f045012a42b82798969f378d97d3328556e4 Mon Sep 17 00:00:00 2001 From: johndmulhausen Date: Tue, 28 Jun 2016 15:00:34 -0700 Subject: [PATCH 139/348] Update authorization.md --- docs/admin/authorization.md | 125 +++++++++++++----------------------- 1 file changed, 44 insertions(+), 81 deletions(-) diff --git a/docs/admin/authorization.md b/docs/admin/authorization.md index 3b521b5df1..48cd6e65db 100644 --- a/docs/admin/authorization.md +++ b/docs/admin/authorization.md @@ -13,35 +13,31 @@ policies. An API call must be allowed by some policy in order to proceed. The following implementations are available, and are selected by flag: - - `--authorization-mode=AlwaysDeny` blocks all requests (used in tests). - - `--authorization-mode=AlwaysAllow` allows all requests; use if you don't -need authorization. - - `--authorization-mode=ABAC`allows for user-configured authorization policy. -ABAC stands for - Attribute-Based Access Control. - - `--authorization-mode=Webhook` allows for authorization to be driven by a -remote service using REST. + - `--authorization-mode=AlwaysDeny` + - `--authorization-mode=AlwaysAllow` + - `--authorization-mode=ABAC` + - `--authorization-mode=Webhook` + +`AlwaysDeny` blocks all requests (used in tests). +`AlwaysAllow` allows all requests; use if you don't need authorization. +`ABAC` allows for user-configured authorization policy. ABAC stands for Attribute-Based Access Control. +`Webhook` allows for authorization to be driven by a remote service using REST. ## ABAC Mode ### Request Attributes A request has the following attributes that can be considered for authorization: - - user (the user-string which a user was authenticated as). - group (the list of group names the authenticated user is a member of). - whether the request is for an API resource. - the request path. - - allows authorizing access to miscellaneous endpoints like `/api` or -`/healthz` (see [kubectl](#kubectl)). + - allows authorizing access to miscellaneous endpoints like `/api` or `/healthz` (see [kubectl](#kubectl)). - the request verb. - - API verbs like `get`, `list`, `create`, `update`, `watch`, `delete`, and -`deletecollection` are used for API requests - - HTTP verbs like `get`, `post`, `put`, and `delete` are used for non-API -requests + - API verbs like `get`, `list`, `create`, `update`, `watch`, `delete`, and `deletecollection` are used for API requests + - HTTP verbs like `get`, `post`, `put`, and `delete` are used for non-API requests - what resource is being accessed (for API requests only) - - the namespace of the object being accessed (for namespaced API requests -only) + - the namespace of the object being accessed (for namespaced API requests only) - the API group being accessed (for API requests only) We anticipate adding more attributes to allow finer grained access control and @@ -51,33 +47,33 @@ to assist in policy management. For mode `ABAC`, also specify `--authorization-policy-file=SOME_FILENAME`. -The file format is [one JSON object per line](http://jsonlines.org/). There -should be no enclosing list or map, just one map per line. - -Each line is a "policy object". A policy object is a map with the following -properties: +The file format is [one JSON object per line](http://jsonlines.org/). There should be no enclosing list or map, just +one map per line. +Each line is a "policy object". A policy object is a map with the following properties: - Versioning properties: - `apiVersion`, type string; valid values are "abac.authorization.kubernetes.io/v1beta1". Allows versioning and conversion of the policy format. - `kind`, type string: valid values are "Policy". Allows versioning and conversion of the policy format. + - `spec` property set to a map with the following properties: - Subject-matching properties: - `user`, type string; the user-string from `--token-auth-file`. If you specify `user`, it must match the username of the authenticated user. `*` matches all requests. - `group`, type string; if you specify `group`, it must match one of the groups of the authenticated user. `*` matches all requests. + - `readonly`, type boolean, when true, means that the policy only applies to get, list, and watch operations. + - Resource-matching properties: - `apiGroup`, type string; an API group, such as `extensions`. `*` matches all API groups. - `namespace`, type string; a namespace string. `*` matches all resource requests. - `resource`, type string; a resource, such as `pods`. `*` matches all resource requests. + - Non-resource-matching properties: - `nonResourcePath`, type string; matches the non-resource request paths (like `/version` and `/apis`). `*` matches all non-resource requests. `/foo/*` matches `/foo/` and all of its subpaths. -An unset property is the same as a property set to the zero value for its type -(e.g. empty string, 0, false). However, unset should be preferred for -readability. +An unset property is the same as a property set to the zero value for its type (e.g. empty string, 0, false). +However, unset should be preferred for readability. -In the future, policies may be expressed in a JSON format, and managed via a -REST interface. +In the future, policies may be expressed in a JSON format, and managed via a REST interface. ### Authorization Algorithm @@ -88,32 +84,23 @@ are set to the zero value of its type (e.g. empty string, 0, false). A property set to "*" will match any value of the corresponding attribute. -The tuple of attributes is checked for a match against every policy in the -policy file. If at least one line matches the request attributes, then the -request is authorized (but may fail later validation). +The tuple of attributes is checked for a match against every policy in the policy file. +If at least one line matches the request attributes, then the request is authorized (but may fail later validation). -To permit any user to do something, write a policy with the user property set to -"*". - -To permit a user to do anything, write a policy with the apiGroup, namespace, -resource, and nonResourcePath properties set to "*". +To permit any user to do something, write a policy with the user property set to "*". +To permit a user to do anything, write a policy with the apiGroup, namespace, resource, and nonResourcePath properties set to "*". ### Kubectl -Kubectl uses the `/api` and `/apis` endpoints of api-server to negotiate -client/server versions. To validate objects sent to the API by create/update -operations, kubectl queries certain swagger resources. For API version `v1` -those would be `/swaggerapi/api/v1` & `/swaggerapi/experimental/v1`. +Kubectl uses the `/api` and `/apis` endpoints of api-server to negotiate client/server versions. To validate objects sent to the API by create/update operations, kubectl queries certain swagger resources. For API version `v1` those would be `/swaggerapi/api/v1` & `/swaggerapi/experimental/v1`. -When using ABAC authorization, those special resources have to be explicitly -exposed via the `nonResourcePath` property in a policy (see [examples](#examples) below): +When using ABAC authorization, those special resources have to be explicitly exposed via the `nonResourcePath` property in a policy (see [examples](#examples) below): * `/api`, `/api/*`, `/apis`, and `/apis/*` for API version negotiation. * `/version` for retrieving the server version via `kubectl version`. * `/swaggerapi/*` for create/update operations. -To inspect the HTTP calls involved in a specific kubectl operation you can turn -up the verbosity: +To inspect the HTTP calls involved in a specific kubectl operation you can turn up the verbosity: kubectl --v=8 version @@ -129,42 +116,34 @@ up the verbosity: ### A quick note on service accounts -A service account automatically generates a user. The user's name is generated -according to the naming convention: +A service account automatically generates a user. The user's name is generated according to the naming convention: ```shell system:serviceaccount:: ``` -Creating a new namespace also causes a new service account to be created, of -this form:* +Creating a new namespace also causes a new service account to be created, of this form:* ```shell system:serviceaccount::default ``` -For example, if you wanted to grant the default service account in the -kube-system full privilege to the API, you would add this line to your policy -file: +For example, if you wanted to grant the default service account in the kube-system full privilege to the API, you would add this line to your policy file: ```json -{"apiVersion":"abac.authorization.kubernetes.io/v1beta1","kind":"Policy","user":"system:serviceaccount:kube-system:default","namespace":"*","resource":"*","apiGroup":"*"} +{"apiVersion":"abac.authorization.kubernetes.io/v1beta1","kind":"Policy","spec":{"user":"system:serviceaccount:kube-system:default","namespace":"*","resource":"*","apiGroup":"*"}} ``` The apiserver will need to be restarted to pickup the new policy lines. ## Webhook Mode -When specified, mode `Webhook` causes Kubernetes to query an outside REST -service when determining user privileges. +When specified, mode `Webhook` causes Kubernetes to query an outside REST service when determining user privileges. ### Configuration File Format -Mode `Webhook` requires a file for HTTP configuration, specify by the -`--authorization-webhook-config-file=SOME_FILENAME` flag. +Mode `Webhook` requires a file for HTTP configuration, specify by the `--authorization-webhook-config-file=SOME_FILENAME` flag. -The configuration file uses the [kubeconfig](/docs/user-guide/kubeconfig-file/) -file format. Within the file "users" refers to the API Server webhook and -"clusters" refers to the remote service. +The configuration file uses the [kubeconfig](/docs/user-guide/kubeconfig-file/) file format. Within the file "users" refers to the API Server webhook and "clusters" refers to the remote service. A configuration example which uses HTTPS client auth: @@ -194,17 +173,9 @@ contexts: ### Request Payloads -When faced with an authorization decision, the API Server POSTs a JSON -serialized api.authorization.v1beta1.SubjectAccessReview object describing the -action. This object contains fields describing the user attempting to make the -request, and either details about the resource being accessed or requests -attributes. +When faced with an authorization decision, the API Server POSTs a JSON serialized api.authorization.v1beta1.SubjectAccessReview object describing the action. This object contains fields describing the user attempting to make the request, and either details about the resource being accessed or requests attributes. -Note that webhook API objects are subject to the same [versioning compatibility rules](/docs/api/) -as other Kubernetes API objects. Implementers should be aware of loser -compatibility promises for beta objects and check the "apiVersion" field of the -request to ensure correct deserialization. Additionally, the API Server must -enable the `authorization.k8s.io/v1beta1` API extensions group (`--runtime-config=authorization.k8s.io/v1beta1=true`). +Note that webhook API objects are subject to the same [versioning compatibility rules](/docs/api/) as other Kubernetes API objects. Implementers should be aware of loser compatibility promises for beta objects and check the "apiVersion" field of the request to ensure correct deserialization. Additionally, the API Server must enable the `authorization.k8s.io/v1beta1` API extensions group (`--runtime-config=authorization.k8s.io/v1beta1=true`). An example request body: @@ -228,9 +199,7 @@ An example request body: } ``` -The remote service is expected to fill the SubjectAccessReviewStatus field of -the request and respond to either allow or disallow access. The response body's -"spec" field is ignored and may be omitted. A permissive response would return: +The remote service is expected to fill the SubjectAccessReviewStatus field of the request and respond to either allow or disallow access. The response body's "spec" field is ignored and may be omitted. A permissive response would return: ```json { @@ -275,15 +244,9 @@ Access to non-resource paths are sent as: } ``` -Non-resource paths include: `/api`, `/apis`, `/metrics`, `/resetMetrics`, -`/logs`, `/debug`, `/healthz`, `/swagger-ui/`, `/swaggerapi/`, `/ui`, and -`/version.` Clients require access to `/api`, `/api/*/`, `/apis/`, `/apis/*`, -`/apis/*/*`, and `/version` to discover what resources and versions are present -on the server. Access to other non-resource paths can be disallowed without -restricting access to the REST api. +Non-resource paths include: `/api`, `/apis`, `/metrics`, `/resetMetrics`, `/logs`, `/debug`, `/healthz`, `/swagger-ui/`, `/swaggerapi/`, `/ui`, and `/version.` Clients require access to `/api`, `/api/*/`, `/apis/`, `/apis/*`, `/apis/*/*`, and `/version` to discover what resources and versions are present on the server. Access to other non-resource paths can be disallowed without restricting access to the REST api. -For further documentation refer to the authorization.v1beta1 API objects and -plugin/pkg/auth/authorizer/webhook/webhook.go. +For further documentation refer to the authorization.v1beta1 API objects and plugin/pkg/auth/authorizer/webhook/webhook.go. ## Plugin Development @@ -304,5 +267,5 @@ Authorization plugin code goes in `pkg/auth/authorizer/$MODULENAME`. An authorization module can be completely implemented in go, or can call out to a remote authorization service. Authorization modules can implement their own caching to reduce the cost of repeated authorization calls with the -same or similar arguments. Developers should then consider the interaction -between caching and revocation of permissions. +same or similar arguments. Developers should then consider the interaction between +caching and revocation of permissions. From bc1cf1106554db0984aad34f353e403894159f16 Mon Sep 17 00:00:00 2001 From: John Mulhausen Date: Tue, 28 Jun 2016 15:05:21 -0700 Subject: [PATCH 140/348] Revert "Update authorization.md" This reverts commit b012f045012a42b82798969f378d97d3328556e4. --- docs/admin/authorization.md | 125 +++++++++++++++++++++++------------- 1 file changed, 81 insertions(+), 44 deletions(-) diff --git a/docs/admin/authorization.md b/docs/admin/authorization.md index 48cd6e65db..3b521b5df1 100644 --- a/docs/admin/authorization.md +++ b/docs/admin/authorization.md @@ -13,31 +13,35 @@ policies. An API call must be allowed by some policy in order to proceed. The following implementations are available, and are selected by flag: - - `--authorization-mode=AlwaysDeny` - - `--authorization-mode=AlwaysAllow` - - `--authorization-mode=ABAC` - - `--authorization-mode=Webhook` - -`AlwaysDeny` blocks all requests (used in tests). -`AlwaysAllow` allows all requests; use if you don't need authorization. -`ABAC` allows for user-configured authorization policy. ABAC stands for Attribute-Based Access Control. -`Webhook` allows for authorization to be driven by a remote service using REST. + - `--authorization-mode=AlwaysDeny` blocks all requests (used in tests). + - `--authorization-mode=AlwaysAllow` allows all requests; use if you don't +need authorization. + - `--authorization-mode=ABAC`allows for user-configured authorization policy. +ABAC stands for + Attribute-Based Access Control. + - `--authorization-mode=Webhook` allows for authorization to be driven by a +remote service using REST. ## ABAC Mode ### Request Attributes A request has the following attributes that can be considered for authorization: + - user (the user-string which a user was authenticated as). - group (the list of group names the authenticated user is a member of). - whether the request is for an API resource. - the request path. - - allows authorizing access to miscellaneous endpoints like `/api` or `/healthz` (see [kubectl](#kubectl)). + - allows authorizing access to miscellaneous endpoints like `/api` or +`/healthz` (see [kubectl](#kubectl)). - the request verb. - - API verbs like `get`, `list`, `create`, `update`, `watch`, `delete`, and `deletecollection` are used for API requests - - HTTP verbs like `get`, `post`, `put`, and `delete` are used for non-API requests + - API verbs like `get`, `list`, `create`, `update`, `watch`, `delete`, and +`deletecollection` are used for API requests + - HTTP verbs like `get`, `post`, `put`, and `delete` are used for non-API +requests - what resource is being accessed (for API requests only) - - the namespace of the object being accessed (for namespaced API requests only) + - the namespace of the object being accessed (for namespaced API requests +only) - the API group being accessed (for API requests only) We anticipate adding more attributes to allow finer grained access control and @@ -47,33 +51,33 @@ to assist in policy management. For mode `ABAC`, also specify `--authorization-policy-file=SOME_FILENAME`. -The file format is [one JSON object per line](http://jsonlines.org/). There should be no enclosing list or map, just -one map per line. +The file format is [one JSON object per line](http://jsonlines.org/). There +should be no enclosing list or map, just one map per line. + +Each line is a "policy object". A policy object is a map with the following +properties: -Each line is a "policy object". A policy object is a map with the following properties: - Versioning properties: - `apiVersion`, type string; valid values are "abac.authorization.kubernetes.io/v1beta1". Allows versioning and conversion of the policy format. - `kind`, type string: valid values are "Policy". Allows versioning and conversion of the policy format. - - `spec` property set to a map with the following properties: - Subject-matching properties: - `user`, type string; the user-string from `--token-auth-file`. If you specify `user`, it must match the username of the authenticated user. `*` matches all requests. - `group`, type string; if you specify `group`, it must match one of the groups of the authenticated user. `*` matches all requests. - - `readonly`, type boolean, when true, means that the policy only applies to get, list, and watch operations. - - Resource-matching properties: - `apiGroup`, type string; an API group, such as `extensions`. `*` matches all API groups. - `namespace`, type string; a namespace string. `*` matches all resource requests. - `resource`, type string; a resource, such as `pods`. `*` matches all resource requests. - - Non-resource-matching properties: - `nonResourcePath`, type string; matches the non-resource request paths (like `/version` and `/apis`). `*` matches all non-resource requests. `/foo/*` matches `/foo/` and all of its subpaths. -An unset property is the same as a property set to the zero value for its type (e.g. empty string, 0, false). -However, unset should be preferred for readability. +An unset property is the same as a property set to the zero value for its type +(e.g. empty string, 0, false). However, unset should be preferred for +readability. -In the future, policies may be expressed in a JSON format, and managed via a REST interface. +In the future, policies may be expressed in a JSON format, and managed via a +REST interface. ### Authorization Algorithm @@ -84,23 +88,32 @@ are set to the zero value of its type (e.g. empty string, 0, false). A property set to "*" will match any value of the corresponding attribute. -The tuple of attributes is checked for a match against every policy in the policy file. -If at least one line matches the request attributes, then the request is authorized (but may fail later validation). +The tuple of attributes is checked for a match against every policy in the +policy file. If at least one line matches the request attributes, then the +request is authorized (but may fail later validation). -To permit any user to do something, write a policy with the user property set to "*". -To permit a user to do anything, write a policy with the apiGroup, namespace, resource, and nonResourcePath properties set to "*". +To permit any user to do something, write a policy with the user property set to +"*". + +To permit a user to do anything, write a policy with the apiGroup, namespace, +resource, and nonResourcePath properties set to "*". ### Kubectl -Kubectl uses the `/api` and `/apis` endpoints of api-server to negotiate client/server versions. To validate objects sent to the API by create/update operations, kubectl queries certain swagger resources. For API version `v1` those would be `/swaggerapi/api/v1` & `/swaggerapi/experimental/v1`. +Kubectl uses the `/api` and `/apis` endpoints of api-server to negotiate +client/server versions. To validate objects sent to the API by create/update +operations, kubectl queries certain swagger resources. For API version `v1` +those would be `/swaggerapi/api/v1` & `/swaggerapi/experimental/v1`. -When using ABAC authorization, those special resources have to be explicitly exposed via the `nonResourcePath` property in a policy (see [examples](#examples) below): +When using ABAC authorization, those special resources have to be explicitly +exposed via the `nonResourcePath` property in a policy (see [examples](#examples) below): * `/api`, `/api/*`, `/apis`, and `/apis/*` for API version negotiation. * `/version` for retrieving the server version via `kubectl version`. * `/swaggerapi/*` for create/update operations. -To inspect the HTTP calls involved in a specific kubectl operation you can turn up the verbosity: +To inspect the HTTP calls involved in a specific kubectl operation you can turn +up the verbosity: kubectl --v=8 version @@ -116,34 +129,42 @@ To inspect the HTTP calls involved in a specific kubectl operation you can turn ### A quick note on service accounts -A service account automatically generates a user. The user's name is generated according to the naming convention: +A service account automatically generates a user. The user's name is generated +according to the naming convention: ```shell system:serviceaccount:: ``` -Creating a new namespace also causes a new service account to be created, of this form:* +Creating a new namespace also causes a new service account to be created, of +this form:* ```shell system:serviceaccount::default ``` -For example, if you wanted to grant the default service account in the kube-system full privilege to the API, you would add this line to your policy file: +For example, if you wanted to grant the default service account in the +kube-system full privilege to the API, you would add this line to your policy +file: ```json -{"apiVersion":"abac.authorization.kubernetes.io/v1beta1","kind":"Policy","spec":{"user":"system:serviceaccount:kube-system:default","namespace":"*","resource":"*","apiGroup":"*"}} +{"apiVersion":"abac.authorization.kubernetes.io/v1beta1","kind":"Policy","user":"system:serviceaccount:kube-system:default","namespace":"*","resource":"*","apiGroup":"*"} ``` The apiserver will need to be restarted to pickup the new policy lines. ## Webhook Mode -When specified, mode `Webhook` causes Kubernetes to query an outside REST service when determining user privileges. +When specified, mode `Webhook` causes Kubernetes to query an outside REST +service when determining user privileges. ### Configuration File Format -Mode `Webhook` requires a file for HTTP configuration, specify by the `--authorization-webhook-config-file=SOME_FILENAME` flag. +Mode `Webhook` requires a file for HTTP configuration, specify by the +`--authorization-webhook-config-file=SOME_FILENAME` flag. -The configuration file uses the [kubeconfig](/docs/user-guide/kubeconfig-file/) file format. Within the file "users" refers to the API Server webhook and "clusters" refers to the remote service. +The configuration file uses the [kubeconfig](/docs/user-guide/kubeconfig-file/) +file format. Within the file "users" refers to the API Server webhook and +"clusters" refers to the remote service. A configuration example which uses HTTPS client auth: @@ -173,9 +194,17 @@ contexts: ### Request Payloads -When faced with an authorization decision, the API Server POSTs a JSON serialized api.authorization.v1beta1.SubjectAccessReview object describing the action. This object contains fields describing the user attempting to make the request, and either details about the resource being accessed or requests attributes. +When faced with an authorization decision, the API Server POSTs a JSON +serialized api.authorization.v1beta1.SubjectAccessReview object describing the +action. This object contains fields describing the user attempting to make the +request, and either details about the resource being accessed or requests +attributes. -Note that webhook API objects are subject to the same [versioning compatibility rules](/docs/api/) as other Kubernetes API objects. Implementers should be aware of loser compatibility promises for beta objects and check the "apiVersion" field of the request to ensure correct deserialization. Additionally, the API Server must enable the `authorization.k8s.io/v1beta1` API extensions group (`--runtime-config=authorization.k8s.io/v1beta1=true`). +Note that webhook API objects are subject to the same [versioning compatibility rules](/docs/api/) +as other Kubernetes API objects. Implementers should be aware of loser +compatibility promises for beta objects and check the "apiVersion" field of the +request to ensure correct deserialization. Additionally, the API Server must +enable the `authorization.k8s.io/v1beta1` API extensions group (`--runtime-config=authorization.k8s.io/v1beta1=true`). An example request body: @@ -199,7 +228,9 @@ An example request body: } ``` -The remote service is expected to fill the SubjectAccessReviewStatus field of the request and respond to either allow or disallow access. The response body's "spec" field is ignored and may be omitted. A permissive response would return: +The remote service is expected to fill the SubjectAccessReviewStatus field of +the request and respond to either allow or disallow access. The response body's +"spec" field is ignored and may be omitted. A permissive response would return: ```json { @@ -244,9 +275,15 @@ Access to non-resource paths are sent as: } ``` -Non-resource paths include: `/api`, `/apis`, `/metrics`, `/resetMetrics`, `/logs`, `/debug`, `/healthz`, `/swagger-ui/`, `/swaggerapi/`, `/ui`, and `/version.` Clients require access to `/api`, `/api/*/`, `/apis/`, `/apis/*`, `/apis/*/*`, and `/version` to discover what resources and versions are present on the server. Access to other non-resource paths can be disallowed without restricting access to the REST api. +Non-resource paths include: `/api`, `/apis`, `/metrics`, `/resetMetrics`, +`/logs`, `/debug`, `/healthz`, `/swagger-ui/`, `/swaggerapi/`, `/ui`, and +`/version.` Clients require access to `/api`, `/api/*/`, `/apis/`, `/apis/*`, +`/apis/*/*`, and `/version` to discover what resources and versions are present +on the server. Access to other non-resource paths can be disallowed without +restricting access to the REST api. -For further documentation refer to the authorization.v1beta1 API objects and plugin/pkg/auth/authorizer/webhook/webhook.go. +For further documentation refer to the authorization.v1beta1 API objects and +plugin/pkg/auth/authorizer/webhook/webhook.go. ## Plugin Development @@ -267,5 +304,5 @@ Authorization plugin code goes in `pkg/auth/authorizer/$MODULENAME`. An authorization module can be completely implemented in go, or can call out to a remote authorization service. Authorization modules can implement their own caching to reduce the cost of repeated authorization calls with the -same or similar arguments. Developers should then consider the interaction between -caching and revocation of permissions. +same or similar arguments. Developers should then consider the interaction +between caching and revocation of permissions. From b6531ec3e6078c0ee03f1550cf9fa43351991450 Mon Sep 17 00:00:00 2001 From: johndmulhausen Date: Tue, 28 Jun 2016 15:07:34 -0700 Subject: [PATCH 141/348] Manually flying in Tech-Reviewed changes From PR #309, which I am shutting down due to non-CLA --- docs/admin/authorization.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/admin/authorization.md b/docs/admin/authorization.md index 3b521b5df1..ed8652fe47 100644 --- a/docs/admin/authorization.md +++ b/docs/admin/authorization.md @@ -147,7 +147,7 @@ kube-system full privilege to the API, you would add this line to your policy file: ```json -{"apiVersion":"abac.authorization.kubernetes.io/v1beta1","kind":"Policy","user":"system:serviceaccount:kube-system:default","namespace":"*","resource":"*","apiGroup":"*"} +{"apiVersion":"abac.authorization.kubernetes.io/v1beta1","kind":"Policy","spec":{"user":"system:serviceaccount:kube-system:default","namespace":"*","resource":"*","apiGroup":"*"}} ``` The apiserver will need to be restarted to pickup the new policy lines. From b12e7f5cdca4bf60ebef00d780b1063bbd896807 Mon Sep 17 00:00:00 2001 From: runseb Date: Wed, 29 Jun 2016 14:07:07 +0200 Subject: [PATCH 142/348] Fixit: #675, wrong ling to PV walkthrough --- docs/user-guide/persistent-volumes/index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/user-guide/persistent-volumes/index.md b/docs/user-guide/persistent-volumes/index.md index ae465eea00..a9e9934084 100644 --- a/docs/user-guide/persistent-volumes/index.md +++ b/docs/user-guide/persistent-volumes/index.md @@ -14,7 +14,7 @@ A `PersistentVolume` (PV) is a piece of networked storage in the cluster that ha A `PersistentVolumeClaim` (PVC) is a request for storage by a user. It is similar to a pod. Pods consume node resources and PVCs consume PV resources. Pods can request specific levels of resources (CPU and Memory). Claims can request specific size and access modes (e.g, can be mounted once read/write or many times read-only). -Please see the [detailed walkthrough with working examples](/docs/user-guide/persistent-volumes/). +Please see the [detailed walkthrough with working examples](/docs/user-guide/persistent-volumes/walkthrough/). ## Lifecycle of a volume and claim @@ -169,4 +169,4 @@ spec: - name: mypd persistentVolumeClaim: claimName: myclaim -``` \ No newline at end of file +``` From e8c103a3bf146eae16c1d328ffbeef89fe76edfd Mon Sep 17 00:00:00 2001 From: Angus Salkeld Date: Wed, 29 Jun 2016 14:19:56 +0200 Subject: [PATCH 143/348] Add CephFS volume type --- docs/user-guide/volumes.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/docs/user-guide/volumes.md b/docs/user-guide/volumes.md index 433deae10f..48343e4b41 100644 --- a/docs/user-guide/volumes.md +++ b/docs/user-guide/volumes.md @@ -64,6 +64,7 @@ Kubernetes supports several types of Volumes: * `flocker` * `glusterfs` * `rbd` + * `cephfs` * `gitRepo` * `secret` * `persistentVolumeClaim` @@ -307,6 +308,20 @@ simultaneous writers allowed. See the [RBD example](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/rbd/) for more details. +### cephfs + +A `cephfs` volume allows an existing CephFS volume to be +mounted into your pod. Unlike `emptyDir`, which is erased when a Pod is +removed, the contents of a `cephfs` volume are preserved and the volume is merely +unmounted. This means that a CephFS volume can be pre-populated with data, and +that data can be "handed off" between pods. CephFS can be mounted by multiple +writers simultaneously. + +__Important: You must have your own Ceph server running with the share exported +before you can use it__ + +See the [CephFS example](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/cephfs/) for more details. + ### gitRepo A `gitRepo` volume is an example of what can be done as a volume plugin. It From 9cc2a24ada27a9f4360358087b06575470a934e5 Mon Sep 17 00:00:00 2001 From: runseb Date: Wed, 29 Jun 2016 14:53:05 +0200 Subject: [PATCH 144/348] fixit-730: wrong link and missing newline in index.md --- docs/user-guide/jobs.md | 6 +++--- docs/user-guide/jobs/expansions/index.md | 3 ++- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/docs/user-guide/jobs.md b/docs/user-guide/jobs.md index 0d60220873..3f54116f75 100644 --- a/docs/user-guide/jobs.md +++ b/docs/user-guide/jobs.md @@ -267,9 +267,9 @@ Here, `W` is the number of work items. | Pattern | `.spec.completions` | `.spec.parallelism` | | -------------------------------------------------------------------- |:-------------------:|:--------------------:| -| [Job Template Expansion](/docs/user-guide/job/expansions/) | 1 | should be 1 | -| [Queue with Pod Per Work Item](/docs/user-guide/job/work-queue-1/) | W | any | -| [Queue with Variable Pod Count](/docs/user-guide/job/work-queue-2/) | 1 | any | +| [Job Template Expansion](/docs/user-guide/jobs/expansions/) | 1 | should be 1 | +| [Queue with Pod Per Work Item](/docs/user-guide/jobs/work-queue-1/) | W | any | +| [Queue with Variable Pod Count](/docs/user-guide/jobs/work-queue-2/) | 1 | any | | Single Job with Static Work Assignment | W | any | diff --git a/docs/user-guide/jobs/expansions/index.md b/docs/user-guide/jobs/expansions/index.md index 892dff0df3..91e916844f 100644 --- a/docs/user-guide/jobs/expansions/index.md +++ b/docs/user-guide/jobs/expansions/index.md @@ -1,4 +1,5 @@ ---- --- +--- +--- * TOC {:toc} From 3ba2887476934190bf8a467f81d7e89bffa42eac Mon Sep 17 00:00:00 2001 From: runseb Date: Wed, 29 Jun 2016 15:12:16 +0200 Subject: [PATCH 145/348] fixit-575: typo in link to k8petstore --- _data/samples.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_data/samples.yml b/_data/samples.yml index 3ad519c6be..074da2230a 100644 --- a/_data/samples.yml +++ b/_data/samples.yml @@ -76,4 +76,4 @@ toc: - title: Nodejs + Mongo path: https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/nodesjs-mongodb - title: Petstore - path: https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/k8spetstore/ + path: https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/k8petstore/ From df20c0a7ff3acb07da93f991468bbcff5cdc0d6d Mon Sep 17 00:00:00 2001 From: Minhan Xia Date: Thu, 23 Jun 2016 13:16:12 -0700 Subject: [PATCH 146/348] explain network-plugin-dir in case of kubenet --- docs/admin/network-plugins.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/admin/network-plugins.md b/docs/admin/network-plugins.md index f5e2d27192..38b28bd153 100644 --- a/docs/admin/network-plugins.md +++ b/docs/admin/network-plugins.md @@ -39,7 +39,7 @@ The Linux-only kubenet plugin provides functionality similar to the `--configure The plugin requires a few things: -* The standard CNI `bridge` and `host-local` plugins to be placed in `/opt/cni/bin`. +* The standard CNI `bridge` and `host-local` plugins are required. Kubenet will first search for them in `/opt/cni/bin`. Specify `network-plugin-dir` to supply additional search path. The first found match will take effect. * Kubelet must be run with the `--network-plugin=kubenet` argument to enable the plugin * Kubelet must also be run with the `--reconcile-cidr` argument to ensure the IP subnet assigned to the node by configuration or the controller-manager is propagated to the plugin * The node must be assigned an IP subnet through either the `--pod-cidr` kubelet command-line option or the `--allocate-node-cidrs=true --cluster-cidr=` controller-manager command-line options. From 7f5008d1b99534d5e4f69ba01f70da9d35e213e7 Mon Sep 17 00:00:00 2001 From: nikhiljindal Date: Mon, 13 Jun 2016 21:13:22 -0700 Subject: [PATCH 147/348] Adding a doc with instructions to setup federation --- _data/guides.yml | 2 + docs/admin/federation/index.md | 194 +++++++++++++++++++++++++++++++++ docs/admin/multi-cluster.md | 9 +- 3 files changed, 199 insertions(+), 6 deletions(-) create mode 100644 docs/admin/federation/index.md diff --git a/_data/guides.yml b/_data/guides.yml index f5654a6efd..6ba05ee6a0 100644 --- a/_data/guides.yml +++ b/_data/guides.yml @@ -243,6 +243,8 @@ toc: path: /docs/admin/cluster-components/ - title: Configuring Kubernetes Use of etcd path: /docs/admin/etcd/ + - title: Federating Clusters + path: /docs/admin/federation/ - title: Using Multiple Clusters path: /docs/admin/multi-cluster/ - title: Changing Cluster Size diff --git a/docs/admin/federation/index.md b/docs/admin/federation/index.md new file mode 100644 index 0000000000..dee50e33b4 --- /dev/null +++ b/docs/admin/federation/index.md @@ -0,0 +1,194 @@ +--- +--- +This guide explains how to set up cluster federation that lets us control multiple Kubernetes clusters. + + +* TOC +{:toc} + +## Prerequisites + +This guide assumes that we have a running Kubernetes cluster. +If not, then head over to the [getting started guides](/docs/getting-started-guides/) to bring up a cluster. + +This guide also assumes that we have the Kubernetes source code that can be +[downloaded from here](/docs/getting-started-guides/binary_release/). + +## Setting up a federation control plane + +Setting up federation requires running the federation control plane which +consists of etcd, federation-apiserver and federation-controller-manager. +We can run these binaries as pods on an existing Kubernetes cluster. + +### Getting images + +To run these as pods, we first need images for all the components. We can use +official release images or we can build from HEAD. + +#### Using official release images + +As part of every release, images are pushed to `gcr.io/google_containers`. To use +these images, we set env var `FEDERATION_PUSH_REPO_BASE=gcr.io/google_containers` +This will always use the latest image. +To use federation-apiserver and federation-controller-manager images from a specific release, we can set `FEDERATION_IMAGE_TAG`. + +#### Building and pushing images from HEAD + +To run the code from HEAD, we need to build and push our own images. +We can build the images using the following command: + +```shell +$ FEDERATION=true KUBE_RELEASE_RUN_TESTS=n make quick-release +``` + +Next, we need to push these images to a registry such as Google Container Registry or Docker Hub, so that our cluster can pull them. +If Kubernetes cluster is running on Google Compute Engine (GCE), then we can push the images to `gcr.io/`. +The command to push the images will look like: + +```shell +$ FEDERATION=true FEDERATION_PUSH_REPO_BASE=gcr.io/ ./build/push-federation-images.sh +``` + +### Running the federation control plane + +Once we have the images, we can run these as pods on our existing kubernetes cluster. +The command to run these pods on an existing GCE cluster will look like: + +```shell +$ KUBERNETES_PROVIDER=gce FEDERATION_DNS_PROVIDER=google-clouddns FEDERATION_NAME=myfederation DNS_ZONE_NAME=myfederation.example FEDERATION_PUSH_REPO_BASE=gcr.io/google_containers ./federation/cluster/federation-up.sh +``` + +`KUBERNETES_PROVIDER` is the cloud provider. + +`FEDERATION_DNS_PROVIDER` can be `google-clouddns` or `aws-route53`. It will be +set appropriately if it is missing and `KUBERNETES_PROVIDER` is one of `gce`, `gke` and `aws`. +This is used to resolve DNS requests for federation services. The service +controller keeps DNS records with the provider updated as services/pods are +updated in underlying kubernetes clusters. + +`FEDERATION_NAME` is a name we can choose for our federation. This is the name that will appear in DNS routes. + +`DNS_ZONE_NAME` is the domain to be used for DNS records. This is a domain that we +need to buy and then configure it such that DNS queries for that domain are +routed to the appropriate provider as per `FEDERATION_DNS_PROVIDER`. + +Running that command creates a namespace `federation` and creates 2 deployments: `federation-apiserver` and `federation-controller-manager`. +We can verify that the pods are available by running the following command: + +```shell +$ kubectl get deployments --namespace=federation +NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE +federation-apiserver 1 1 1 1 1m +federation-controller-manager 1 1 1 1 1m +``` + +Running `federation-up.sh` also creates a new record in our kubeconfig for us +to be able to talk to federation apiserver. We can view this by running +`kubectl config view`. + +Note: `federation-up.sh` creates the federation-apiserver pod with an etcd container that is not backed by any +persistent volume, so it is possible to lose data. You can edit +`federation/manifests/federation-apiserver-deployment.yaml` to suit your needs, +if required. + +## Registering Kubernetes clusters for federation + +Now that we have the federation control plane up and running, we can start registering Kubernetes clusters. + +First of all, we need to create a secret containing kubeconfig for that Kubernetes cluster, which federation control plane will use to talk to that Kubernetes cluster. +For now, we create this secret in the host Kubernetes cluster (that hosts federation control plane). When we start supporting secrets in federation control plane, we will create this secret there. +Suppose that our kubeconfig for Kubernetes cluster is at `/cluster1/kubeconfig`, we can run the following command to create the secret: + +```shell +$ kubectl create secret generic cluster1 --namespace=federation --from-file=/cluster1/kubeconfig +``` + +Note that the file name should be `kubeconfig` since file name determines the name of the key in the secret. + +Now that the secret is created, we are ready to register the cluster. The YAML file for cluster will look like: + +```yaml +apiVersion: v1beta1 +kind: Cluster +metadata: + name: cluster1 +spec: + serverAddressByClientCIDRs: + - clientCIDR: + serverAddress: + secretRef: + name: +``` + +We need to insert the appropriate values for ``, `` and ``. +`` here is name of the secret that we just created. +serverAddressByClientCIDRs contains the various server addresses that clients +can use as per their CIDR. We can set the server's public IP address with CIDR +`"0.0.0.0/0"` which all clients will match. In addition, if we want internal +clients to use server's clusterIP, we can set that as serverAddress. The client +CIDR in that case will be a CIDR that only matches IPs of pods running in that +cluster. + +Assuming our YAML file is located at `/cluster1/cluster.yaml`, we can run the following command to register this cluster: + +```shell +$ kubectl create -f /cluster1/cluster.yaml --cluster=federation-cluster + +``` + +By specifying `--cluster=federation-cluster`, we direct the request to federation apiserver. +we can ensure that the cluster registration was successful by running: + +```shell +$ kubectl get clusters --context=federation-cluster +NAME STATUS VERSION AGE +cluster1 Ready 3m +``` + +### Updating KubeDNS + +Once the cluster is registered with the federation, we are all ready to use it. +But for the cluster to be able to route federation service requests, we need to restart +KubeDNS and pass it a `--federations` flag which tells it about valid federation DNS hostnames. +Format of the flag is like this: + +``` +--federations=${FEDERATION_NAME}=${DNS_DOMAIN_NAME} +``` + +To update KubeDNS with federations flag, we can edit the existing kubedns replication controller to +include that flag in pod template spec and then delete the existing pod. Replication controller will +recreate the pod with updated template. + +To find the name of existing kubedns replication controller, run + +```shell +$ kubectl get rc --namespace=kube-system +``` + +This will list all the replication controllers. Name of the kube-dns replication +controller will look like `kube-dns-v18`. You can then edit it by running: + +```shell +$ kubectl edit rc --namespace=kube-system +``` +Add the `--federations` flag as args to kube-dns container in the YAML file that +pops up after running the above command. + +To delete the existing kube dns pod, you can first find it by running: + +```shell +$ kubectl get pods --namespace=kube-system +``` + +And then delete it by running: + +```shell +$ kubectl delete pods --namespace=kube-system +``` + +We are now all set to start using federation. + +## For more information + + * [Federation proposal](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/proposals/federation.md) details use cases that motivated this work. diff --git a/docs/admin/multi-cluster.md b/docs/admin/multi-cluster.md index 47e433e29b..e549a4d3ca 100644 --- a/docs/admin/multi-cluster.md +++ b/docs/admin/multi-cluster.md @@ -5,9 +5,7 @@ You may want to set up multiple Kubernetes clusters, both to have clusters in different regions to be nearer to your users, and to tolerate failures and/or invasive maintenance. This document describes some of the issues to consider when making a decision about doing so. -Note that at present, -Kubernetes does not offer a mechanism to aggregate multiple clusters into a single virtual cluster. However, -we [plan to do this in the future](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/proposals/federation.md). +If you decide to have multiple clusters, kubernetes provides a way to [federate them](/docs/admin/federation/) ## Scope of a single cluster @@ -56,11 +54,10 @@ users in the event of a cluster failure), then you need to have `R * (U + 1)` cl (`U + 1` in each of `R` regions). In any case, try to put each cluster in a different zone. Finally, if any of your clusters would need more than the maximum recommended number of nodes for a Kubernetes cluster, then -you may need even more clusters. Kubernetes v1.0 currently supports clusters up to 100 nodes in size, but we are targeting -1000-node clusters by early 2016. +you may need even more clusters. Kubernetes v1.3 supports clusters up to 1000 nodes in size. ## Working with multiple clusters When you have multiple clusters, you would typically create services with the same config in each cluster and put each of those service instances behind a load balancer (AWS Elastic Load Balancer, GCE Forwarding Rule or HTTP Load Balancer) spanning all of them, so that -failures of a single cluster are not visible to end users. \ No newline at end of file +failures of a single cluster are not visible to end users. From 7230d84e597c5d19ff8926a7c0ebed24b21b87cb Mon Sep 17 00:00:00 2001 From: Suraj Deshmukh Date: Thu, 30 Jun 2016 12:06:01 +0530 Subject: [PATCH 148/348] Added fedora in docker-multinode master script Fedora was considered in the initialization scripts but when actual checks were done fedora was missing so added fedora at required positions. Fixes #758 --- docs/getting-started-guides/docker-multinode/master.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/getting-started-guides/docker-multinode/master.sh b/docs/getting-started-guides/docker-multinode/master.sh index 0bc88c1075..63465a288a 100755 --- a/docs/getting-started-guides/docker-multinode/master.sh +++ b/docs/getting-started-guides/docker-multinode/master.sh @@ -90,10 +90,10 @@ detect_lsb() { lsb_dist="$(echo ${lsb_dist} | tr '[:upper:]' '[:lower:]')" case "${lsb_dist}" in - amzn|centos|debian|ubuntu) + amzn|centos|debian|ubuntu|fedora) ;; *) - echo "Error: We currently only support ubuntu|debian|amzn|centos." + echo "Error: We currently only support ubuntu|debian|amzn|centos|fedora." exit 1 ;; esac @@ -174,7 +174,7 @@ start_k8s(){ ifconfig docker0 down yum -y -q install bridge-utils && brctl delbr docker0 && service docker restart ;; - centos) + centos|fedora) DOCKER_CONF="/etc/sysconfig/docker" sed -i "/^OPTIONS=/ s|\( --mtu=.*\)\?'$| --mtu=${FLANNEL_MTU} --bip=${FLANNEL_SUBNET}'|" ${DOCKER_CONF} if ! command_exists ifconfig; then From ad83d18db4acb3c1f784839cf5230aa32002b27b Mon Sep 17 00:00:00 2001 From: Tolleiv Nietsch Date: Thu, 30 Jun 2016 12:32:40 +0200 Subject: [PATCH 149/348] Add header to the empty column --- docs/getting-started-guides/ubuntu.md | 4 ++-- docs/user-guide/persistent-volumes/index.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/getting-started-guides/ubuntu.md b/docs/getting-started-guides/ubuntu.md index c47a15dc9d..ac1abbf233 100644 --- a/docs/getting-started-guides/ubuntu.md +++ b/docs/getting-started-guides/ubuntu.md @@ -80,7 +80,7 @@ First configure the cluster information in cluster/ubuntu/config-default.sh, fol ```shell export nodes="vcap@10.10.103.250 vcap@10.10.103.162 vcap@10.10.103.223" -export role="ai i i" +export roles="ai i i" export NUM_NODES=${NUM_NODES:-3} @@ -92,7 +92,7 @@ export FLANNEL_NET=172.16.0.0/16 The first variable `nodes` defines all your cluster nodes, master node comes first and separated with blank space like ` ` -Then the `role` variable defines the role of above machine in the same order, "ai" stands for machine +Then the `roles` variable defines the role of above machine in the same order, "ai" stands for machine acts as both master and node, "a" stands for master, "i" stands for node. The `NUM_NODES` variable defines the total number of nodes. diff --git a/docs/user-guide/persistent-volumes/index.md b/docs/user-guide/persistent-volumes/index.md index 1c4bb150f5..27fb34f14f 100644 --- a/docs/user-guide/persistent-volumes/index.md +++ b/docs/user-guide/persistent-volumes/index.md @@ -108,7 +108,7 @@ In the CLI, the access modes are abbreviated to: > __Important!__ A volume can only be mounted using one access mode at a time, even if it supports many. For example, a GCEPersistentDisk can be mounted as ReadWriteOnce by a single node or ReadOnlyMany by many nodes, but not at the same time. -| | ReadWriteOnce| ReadOnlyMany| ReadWriteMany| +| Volume Plugin | ReadWriteOnce| ReadOnlyMany| ReadWriteMany| | :--- | :---: | :---: | :---: | | AWSElasticBlockStore | x | - | - | | AzureFile | x | x | x | From 3be608750f035448f36e556ea0704a85432c81e5 Mon Sep 17 00:00:00 2001 From: Malet Date: Thu, 30 Jun 2016 13:29:48 +0100 Subject: [PATCH 150/348] Fix "Sharing Clusters" docs error --- docs/user-guide/sharing-clusters.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/docs/user-guide/sharing-clusters.md b/docs/user-guide/sharing-clusters.md index 9db2460ed2..c641f4ed6d 100644 --- a/docs/user-guide/sharing-clusters.md +++ b/docs/user-guide/sharing-clusters.md @@ -70,7 +70,10 @@ $ kubectl config set-credentials $USER_NICK \ --kubeconfig=/path/to/standalone/.kube/config # create context entry -$ kubectl config set-context $CONTEXT_NAME --cluster=$CLUSTER_NICKNAME --user=$USER_NICK +$ kubectl config set-context $CONTEXT_NAME \ + --cluster=$CLUSTER_NICK \ + --user=$USER_NICK \ + --kubeconfig=/path/to/standalone/.kube/config ``` Notes: @@ -116,4 +119,4 @@ $ scp host2:/path/to/home2/.kube/config /path/to/other/.kube/config $ export $KUBECONFIG=/path/to/other/.kube/config ``` -Detailed examples and explanation of `kubeconfig` loading/merging rules can be found in [kubeconfig-file](/docs/user-guide/kubeconfig-file). \ No newline at end of file +Detailed examples and explanation of `kubeconfig` loading/merging rules can be found in [kubeconfig-file](/docs/user-guide/kubeconfig-file). From 2b7788211290e0ae33576b0e286d81c81c770760 Mon Sep 17 00:00:00 2001 From: runseb Date: Thu, 30 Jun 2016 18:53:05 +0200 Subject: [PATCH 151/348] fixit-701: correct expression to get externalIP --- docs/user-guide/kubectl-cheatsheet.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/kubectl-cheatsheet.md b/docs/user-guide/kubectl-cheatsheet.md index d7eda822d3..6897b61106 100644 --- a/docs/user-guide/kubectl-cheatsheet.md +++ b/docs/user-guide/kubectl-cheatsheet.md @@ -86,7 +86,7 @@ $ kubectl get pods --sort-by=.status.containerStatuses[0].restartCount $ kubectl get pods --selector=app=cassandra rc -o 'jsonpath={.items[*].metadata.labels.version}' # Get ExternalIPs of all nodes -$ kubectl get nodes -o jsonpath='{.items[*].status.addresses[?(@.type=ExternalIP)].address}' +$ kubectl get nodes -o jsonpath='{.items[*].status.addresses[?(@.type=="ExternalIP")].address}' # List Names of Pods that belong to Particular RC # "jq" command useful for transformations that are too complex for jsonpath From 99c200eba052a03c298e6e4577f7ba098ba19c79 Mon Sep 17 00:00:00 2001 From: Jeff Lowdermilk Date: Thu, 30 Jun 2016 11:12:28 -0700 Subject: [PATCH 152/348] Add kubectl download instructions to minikube.md https://github.com/kubernetes/kubernetes/issues/28171 --- docs/getting-started-guides/minikube.md | 47 ++++++++++++++++++++++++- 1 file changed, 46 insertions(+), 1 deletion(-) diff --git a/docs/getting-started-guides/minikube.md b/docs/getting-started-guides/minikube.md index e4633dfca9..4a58c3f51b 100644 --- a/docs/getting-started-guides/minikube.md +++ b/docs/getting-started-guides/minikube.md @@ -55,6 +55,51 @@ Minikube requires one of the following: See the [latest Minikube release](https://github.com/kubernetes/minikube/releases) for installation instructions. +### Download `kubectl` + +You will need to download the kubectl client binary for `${K8S_VERSION}` (in this example: `{{page.version}}.0`) +to run commands against the cluster. + +Downloads: + + - `linux/amd64`: http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/amd64/kubectl + - `linux/386`: http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/386/kubectl + - `linux/arm`: http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/arm/kubectl + - `linux/arm64`: http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/arm64/kubectl + - `linux/ppc64le`: http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/ppc64le/kubectl + - `OS X/amd64`: http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/darwin/amd64/kubectl + - `OS X/386`: http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/darwin/386/kubectl + - `windows/amd64`: http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/windows/amd64/kubectl.exe + - `windows/386`: http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/windows/386/kubectl.exe + +The generic download path is: +``` +http://storage.googleapis.com/kubernetes-release/release/${K8S_VERSION}/bin/${GOOS}/${GOARCH}/${K8S_BINARY} +``` + +An example install with `linux/amd64`: + +``` +curl -sSL "http://storage.googleapis.com/kubernetes-release/release/{{page.version}}.0/bin/linux/amd64/kubectl" > /usr/bin/kubectl +chmod +x /usr/bin/kubectl +``` + +On OS X, to make the API server accessible locally, setup a ssh tunnel. + +```shell +docker-machine ssh `docker-machine active` -N -L 8080:localhost:8080 +``` + +Setting up a ssh tunnel is applicable to remote docker hosts as well. + +(Optional) Create kubernetes cluster configuration: + +```shell +kubectl config set-cluster minikube --server=http://localhost:8080 +kubectl config set-context minikube --cluster=test-doc +kubectl config use-context minikube +``` + ### Starting the cluster To start a cluster, run the command: @@ -113,4 +158,4 @@ CONTAINER ID IMAGE 475db7659edf gcr.io/google_containers/pause-amd64:3.0 "/pause" 3 minutes ago Up 3 minutes k8s_POD.2225036b_kubernetes-dashboard-9brhv_kube-system_5062dd0b-370b-11e6-84b6-5eab1f51187f_e76d8136 e9096501addf gcr.io/google-containers/kube-addon-manager-amd64:v2 "/opt/kube-addons.sh" 3 minutes ago Up 3 minutes k8s_kube-addon-manager.a1c58ca2_kube-addon-manager-127.0.0.1_kube-system_48abed82af93bb0b941173334110923f_82655b7d 64748893cf7c gcr.io/google_containers/pause-amd64:3.0 "/pause" 4 minutes ago Up 4 minutes k8s_POD.d8dbe16c_kube-addon-manager-127.0.0.1_kube-system_48abed82af93bb0b941173334110923f_c67701c3 -``` \ No newline at end of file +``` From 456daa13911688da8a8c16ba17c74154c35974cd Mon Sep 17 00:00:00 2001 From: Josh Wood Date: Mon, 27 Jun 2016 15:22:21 -0700 Subject: [PATCH 153/348] docs/getting-started/rkt: Add stage1 section, update, rewrite Update the rkt getting started guide: Add information about rkt stage1 images. Fix broken/outdated links. Fix the heading hierarchy. Edit language/clarity. Reformat markdown source for plaintext legibility. Addresses prior reviews on and supersedes #725. --- docs/getting-started-guides/rkt/index.md | 203 ++++++++++++++--------- docs/getting-started-guides/rkt/notes.md | 50 +++--- 2 files changed, 152 insertions(+), 101 deletions(-) diff --git a/docs/getting-started-guides/rkt/index.md b/docs/getting-started-guides/rkt/index.md index ac79c3ac7b..00d5764c83 100644 --- a/docs/getting-started-guides/rkt/index.md +++ b/docs/getting-started-guides/rkt/index.md @@ -1,29 +1,38 @@ --- --- -This document describes how to run Kubernetes using [rkt](https://github.com/coreos/rkt) as a container runtime. +This document describes how to run Kubernetes using [rkt](https://github.com/coreos/rkt) as the container runtime. -### Prerequisite +* TOC +{:toc} -- [systemd](http://www.freedesktop.org/wiki/Software/systemd/) should be installed on the machine and should be enabled. - The minimum version required for Kubernetes 1.3 is `219`. - *(Note that systemd is not required by rkt itself, we are using it here to monitor and manage the pods launched by kubelet.)* +## Prerequisites -- Install the latest rkt release according to the instructions [here](https://github.com/coreos/rkt). - The minimum version required is [v1.9.1](https://github.com/coreos/rkt/releases/tag/v1.9.1). +* [Systemd](http://www.freedesktop.org/wiki/Software/systemd/) must be installed and enabled. The minimum systemd version required for Kubernetes v1.3 is `219`. Systemd is used to monitor and manage the pods on each node. -- The [rkt API service](http://coreos.com/rkt/docs/latest/subcommands/api-service.html) must be running on the node. +* [Install the latest rkt release](https://coreos.com/rkt/docs/latest/trying-out-rkt.html). The minimum rkt version required is [v1.9.1](https://github.com/coreos/rkt/releases/tag/v1.9.1). The [CoreOS Linux alpha channel](https://coreos.com/releases/) ships with a recent rkt release, and you can easily [upgrade rkt on CoreOS](https://coreos.com/rkt/docs/latest/install-rkt-in-coreos.html), if necessary. -### Setup network +* The [rkt API service](https://coreos.com/rkt/docs/latest/subcommands/api-service.html) must be running on the node. -You can configure the Kubernetes networking using its own `kubenet` and `CNI` [network -plugins](http://kubernetes.io/docs/admin/network-plugins/) by setting the kubelet's `--network-plugin` and `--network-plugin-dir` flag. -In addition, rkt supports using rkt's [Contained Networking](https://coreos.com/rkt/docs/latest/networking.html#contained-mode). +## Pod networking in rktnetes -##### Use rkt's Contained Networking +### Kubernetes CNI networking -In this mode, rkt will attempt to join pods into a network named `rkt.kubernetes.io`. -To use rkt's contained networking, you can leave the `--network-plugin` to empty, and put a network config file under one of the rkt's [config directories](https://github.com/coreos/rkt/blob/master/Documentation/configuration.md#command-line-flags), for example: +You can configure Kubernetes pod networking with the usual Container Network Interface (CNI) [network plugins](/docs/admin/network-plugins/) by setting the kubelet's `--network-plugin` and `--network-plugin-dir` options appropriately. Configured in this fashion, the rkt container engine will be unaware of network details, and expects to connect pods to the provided subnet. + +#### kubenet: Google Compute Engine (GCE) network + +The `kubenet` plugin can be selected with the kubelet option `--network-plugin=kubenet`. This plugin is currently only supported on GCE. When using kubenet, Kubernetes CNI creates and manages the network, and rkt is provided with a subnet from a bridge device connected to the GCE network. + +### rkt contained network + +Rather than delegating pod networking to Kubernetes, rkt can configure connectivity directly with its own [*contained network*](https://coreos.com/rkt/docs/latest/networking/overview.html#contained-mode) on a subnet provided by a bridge device, the flannel SDN, or another CNI plugin. Configured this way, rkt looks in its [config directories](https://coreos.com/rkt/docs/latest/configuration.html#command-line-flags), usually `/etc/rkt/net.d`, to discover the CNI configuration and invoke the appropriate plugins to create the pod network. + +#### rkt contained network with bridge + +The *contained network* is rkt's default, so you can leave the kubelet's `--network-plugin` option empty to select this network. The contained network can be backed by any CNI plugin. With the *contained network*, rkt will attempt to join pods to a network named `rkt.kubernetes.io`, so this network name must be used for whatever desired CNI configuration. + +When using the contained network, create a network configuration file beneath the rkt network config directory that defines how to create this `rkt.kubernetes.io` network in your environment. This example sets up a bridge device with the `bridge` CNI plugin: ```shell $ cat </etc/rkt/net.d/k8s_network_example.conf @@ -47,16 +56,9 @@ $ cat </etc/rkt/net.d/k8s_network_example.conf EOF ``` -However, there are a small number of caveats you should be aware of when using rkt's networking: +#### rkt contained network with flannel -* You must create an appropriate CNI configuration file with a network name of `rkt.kubernetes.io`. -* The downwards API and environment variable substitution will not contain the pod IP. -* The `/etc/hosts` file will not contain your own hostname (though `/etc/hostname` is populated). - -##### Use flannel - -While it's recommended that you configure flannel using kubernetes' CNI support, you can also configure it using rkt's contained networking. -An example flannel/CNI config file looks like this: +While it is recommended to operate flannel through the Kubernetes CNI support, you can alternatively configure the flannel plugin directly to provide the subnet for rkt's contained network. An example CNI/flannel config file looks like this: ```shell $ cat </etc/rkt/net.d/k8s_flannel_example.conf @@ -70,45 +72,46 @@ $ cat </etc/rkt/net.d/k8s_flannel_example.conf EOF ``` -For more information on flannel configuration, please read [CNI/flannel README](https://github.com/containernetworking/cni/blob/master/Documentation/flannel.md). +For more information on flannel configuration, see the [CNI/flannel README](https://github.com/containernetworking/cni/blob/master/Documentation/flannel.md). -##### Use Google Compute Engine (GCE) network +#### Contained network caveats: -Each VM on GCE has an additional 256 IP addresses routed to it, so it is possible to forego flannel in smaller clusters. -This can most easily be done by using the builtin kubenet plugin, by setting the kubelet flag `--network-plugin=kubenet`. +* You must create an appropriate CNI configuration file with a network name of `rkt.kubernetes.io`. +* The downwards API and environment variable substitution will not contain the pod IP address. +* The `/etc/hosts` file will not contain the pod's own hostname, although `/etc/hostname` is populated. -### Launch a local cluster +## Running rktnetes -To use rkt as the container runtime, we need to supply the following flags to kubelet: +### Spin up a local Kubernetes cluster with the rkt runtime -- `--container-runtime=rkt` chooses the container runtime to use. -- `--rkt-api-endpoint=HOST:PORT` sets the endpoint of the rkt API service. - Leave empty to use the default one (`localhost:15441`). -- `--rkt-path=$PATH_TO_RKT_BINARY` sets the path of rkt binary. - Leave empty to use the first rkt in $PATH. -- `--rkt-stage1-image` sets the name of the stage1 image, e.g. coreos.com/rkt/stage1-coreos. - Leave empty to use the default stage1 image in the rkt's configuration. +To use rkt as the container runtime in a local Kubernetes cluster, supply the following flags to the kubelet: -If you are using the [hack/local-up-cluster.sh](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/hack/local-up-cluster.sh) script to launch the local cluster, then you can edit the environment variable `CONTAINER_RUNTIME`, `RKT_PATH` and `RKT_STAGE1_IMAGE` to set these flags, the `RKT_PATH` and `RKT_STAGE1_IMAGE` are optional if you have `rkt` in your `$PATH` with appropriate configuration. +* `--container-runtime=rkt` Set the node's container runtime to rkt. +* `--rkt-api-endpoint=HOST:PORT` Set the endpoint of the rkt API service. Default: `localhost:15441`. +* `--rkt-path=PATH_TO_RKT_BINARY` Set the path of the rkt binary. Optional. If empty, look for `rkt` in `$PATH`. +* `--rkt-stage1-image=STAGE1` Set the name of the stage1 image, e.g. `coreos.com/rkt/stage1-coreos`. Optional. If not set, the default Linux kernel software isolation stage1 is used. + +If you are using the [hack/local-up-cluster.sh](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/hack/local-up-cluster.sh) script to launch the cluster, you can edit the environment variables `CONTAINER_RUNTIME`, `RKT_PATH`, and `RKT_STAGE1_IMAGE` to set these flags. `RKT_PATH` and `RKT_STAGE1_IMAGE` are optional if `rkt` is in your $PATH` with appropriate configuration. ```shell $ export CONTAINER_RUNTIME=rkt -$ export RKT_PATH=$PATH_TO_RKT_BINARY -$ export RKT_STAGE1_IMAGE=$NAME_OF_THE_STAGE1_IMAGE +$ export RKT_PATH= +$ export RKT_STAGE1_IMAGE= ``` -Then we can launch the local cluster using the script: +Now you can launch the cluster using the `local-up-cluster.sh` script: ```shell $ hack/local-up-cluster.sh ``` -We are also working on setting up rkt as the container runtime for [minikube](https://github.com/kubernetes/minikube/issues/168). +We are also working on getting rkt working as the container runtime in [minikube](https://github.com/kubernetes/minikube/issues/168). -### Launch a CoreOS/rkt cluster on Google Compute Engine (GCE) +### Launch a rktnetes cluster on Google Compute Engine (GCE) -Here we provide instruction on how to use the `kube-up` script to launch a CoreOS/rkt cluster on GCE. -In order to do that, you need to specify the OS distribution, project, image: +This section outlines using the `kube-up` script to launch a CoreOS/rkt cluster on GCE. + +Specify the OS distribution, the GCE distributor's master project, and the instance images for the Kubernetes master and nodes. Set the `KUBE_CONTAINER_RUNTIME` to `rkt`: ```shell $ export KUBE_OS_DISTRIBUTION=coreos @@ -119,61 +122,101 @@ $ export KUBE_GCE_NODE_IMAGE= $ export KUBE_CONTAINER_RUNTIME=rkt ``` -You can optionally choose the version of rkt used by setting `KUBE_RKT_VERSION`: +Optionally, set the version of rkt by setting `KUBE_RKT_VERSION`: ```shell $ export KUBE_RKT_VERSION=1.9.1 ``` -Then you can launch the cluster by: +Optionally, select an alternative [stage1 isolator](#modular-isolation-with-interchangeable-stage1-images) for the container runtime by setting `KUBE_RKT_STAGE1_IMAGE`: + +```shell +$ export KUBE_RKT_STAGE1_IMAGE= +``` + +Then you can launch the cluster with: ```shell $ cluster/kube-up.sh ``` -### Launch a CoreOS/rkt cluster on AWS +### Launch a rktnetes cluster on AWS -`kube-up` for AWS is currently unsupported. -Instead, we recommend you to refer the [Kubernetes on AWS guide](https://coreos.com/kubernetes/docs/latest/kubernetes-on-aws.html) to launch a CoreOS/rkt cluster on AWS. +The `kube-up` script is not yet supported on AWS. Instead, we recommend following the [Kubernetes on AWS guide](https://coreos.com/kubernetes/docs/latest/kubernetes-on-aws.html) to launch a CoreOS Kubernetes cluster on AWS, then setting kubelet options as above. -### Deploy apps to your cluster +### Deploy apps to the cluster -After you created the cluster, you can start deploying apps to the cluster. For example here is how you can [deploy a simgle nginx app](/docs/user-guide/simple-nginx). -More examples can be found in the [examples directory](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/). +After creating the cluster, you can start deploying applications. For an introductory example, [deploy a simple nginx web server](/docs/user-guide/simple-nginx). Note that this example did not have to be modified for use with a "rktnetes" cluster. More examples can be found in the [Kubernetes examples directory](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/). -### Known Issues and Differences +## Modular isolation with interchangeable stage1 images -rkt and Docker have very different designs, as well as ACI and Docker image format. -Users might experience some different experience when switching from one to the other. -More information can be found [here](/docs/getting-started-guides/rkt/notes/). +rkt executes containers in an interchangeable isolation environment. This facility is called the [*stage1* image](https://coreos.com/rkt/docs/latest/devel/architecture.html#stage-1). There are currently three supported rkt stage1 images: -### Debugging +* `systemd-nspawn` stage1, the default. Isolates running containers with Linux kernel namespaces and cgroups in a manner similar to the default container runtime. +* [`KVM` stage1](https://coreos.com/rkt/docs/latest/running-lkvm-stage1.html), runs containers inside a KVM hypervisor-managed virtual machine. Experimental in the Kubernetes v1.3 release. +* [`fly stage1`](https://coreos.com/rkt/docs/latest/running-fly-stage1.html), which isolates containers with only a `chroot`, giving host-level access to mount and network namespaces for specially-privileged utilities. -Here are several tips in case you run into any issues. +In addition to the three provided stage1 images, you can [create your own](https://coreos.com/rkt/docs/latest/devel/stage1-implementors-guide.html) for specific isolation requirements. If no configuration is set, the [default stage1](https://coreos.com/rkt/docs/latest/build-configure.html#parameters-for-setting-up-default-stage1-image) is used. There are two ways to select a different stage1; either per-node, or per-pod: -##### Check logs +* Set the kubelet's `--rkt-stage1-image` flag, which tells the kubelet the stage1 image to use for every pod on the node. For example, `--rkt-stage1-image=coreos/rkt/stage1-coreos` selects the default systemd-nspawn stage1. +* Set the annotation `rkt.alpha.kubernetes.io/stage1-name-override` to override the stage1 used to execute a given pod. This allows for mixing different container isolation mechanisms on the same cluster or on the same node. For example, the following (shortened) pod manifest will run its pod with the `fly stage1` to give the application -- the `kubelet` in this case -- access to the host's namespace: -By default, the log verbose level is 2. In order to see more logs related to rkt, we can set the verbose level to 4. -For local cluster, we can set the environment variable: `LOG_LEVEL=4`. -If the cluster is using salt, we can edit the [logging.sls](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/cluster/saltbase/pillar/logging.sls) in the saltbase. - -##### Check rkt pod status - -To check the pods' status, we can use rkt command, such as `rkt list`, `rkt status`, `rkt image list`, etc. -More information about rkt command line can be found [here](https://github.com/coreos/rkt/blob/master/Documentation/commands.md). - -##### Check journal logs - -As we use systemd to launch/manage rkt pods, we can check the pods' log using `journalctl`: - -- Check the running state of the systemd service: - -```shell -$ sudo journalctl -u ${SERVICE_NAME} +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: kubelet + namespace: kube-system + labels: + k8s-app: kubelet + annotations: + rkt.alpha.kubernetes.io/stage1-name-override: coreos.com/rkt/stage1-fly +spec: + containers: + - name: kubelet + image: quay.io/coreos/hyperkube:v1.3.0-beta.2_coreos.0 + command: + - kubelet + - --api-servers=127.0.0.1:8080 + - --config=/etc/kubernetes/manifests + - --allow-privileged + - --kubeconfig=/etc/kubernetes/kubeconfig + securityContext: + privileged: true +[...] ``` -where `${SERVICE_NAME}` is the name of the service file created for the pod, typically the format is `k8s_${RKT_UUID}`. +### Notes on using different stage1 images -##### Check Kubernetes events, logs. +Setting the stage1 annotation could potentially give the pod root privileges. Because of this, the `privileged` boolean in the pod's `securityContext` must be set to `true`. -Kubernetes also provides various tools for debugging. More information can be found [here](/docs/user-guide/application-troubleshooting). +Use rkt's [*contained network*](#rkt-contained-network) with the KVM stage1, because the CNI plugin driver does not yet fully support the hypervisor-based runtime. + +## Known issues and differences between rkt and Docker + +rkt and the default node container engine have very different designs, as do rkt's native ACI and the Docker container image format. Users may experience different behaviors when switching from one container engine to the other. More information can be found [in the Kubernetes rkt notes](/docs/getting-started-guides/rkt/notes/). + +## Troubleshooting + +Here are a few tips for troubleshooting Kubernetes with the rkt container engine: + +### Check rkt pod status + +To check the status of running pods, use the rkt subcommands [`rkt list`](https://coreos.com/rkt/docs/latest/subcommands/list.html), [`rkt status`](https://coreos.com/rkt/docs/latest/subcommands/status.html), and [`rkt image list`](https://coreos.com/rkt/docs/latest/subcommands/image.html#rkt-image-list). See the [rkt commands documentation](https://coreos.com/rkt/docs/latest/commands.html) for more information about rkt subcommands. + +### Check journal logs + +Check a pod's log using `journalctl` on the node. Pods are managed and named as systemd units. The pod's unit name is formed by concatenating a `k8s_` prefix with the pod UUID, in a format like `k8s_${RKT_UUID}`. Find the pod's UUID with `rkt list` to assemble its service name, then ask journalctl for the logs: + + +```shell +$ sudo journalctl -u k8s_ad623346 +``` + +#### Log verbosity + +By default, the log verbosity level is 2. In order to see more log messages related to rkt, set this level to 4 or above. For a local cluster, set the environment variable: `LOG_LEVEL=4`. + +### Check Kubernetes events and logs. + +Kubernetes provides various tools for troubleshooting and examination. More information can be found [in the app troubleshooting guide](/docs/user-guide/application-troubleshooting). diff --git a/docs/getting-started-guides/rkt/notes.md b/docs/getting-started-guides/rkt/notes.md index 8e53481046..30a022dc24 100644 --- a/docs/getting-started-guides/rkt/notes.md +++ b/docs/getting-started-guides/rkt/notes.md @@ -1,9 +1,9 @@ --- --- -The following features either are not supported or have large caveats when using the rkt container runtime. +The following features either are not supported or have large caveats when using the rkt container runtime. Increasing support for these items and others, including reasonable feature parity with the default container engine, is planned through future releases. -##### Non-existent host volume paths +## Non-existent host volume paths When mounting a host volume path that does not exist, rkt will error out. Under the Docker runtime, an empty directory will be created at the referenced path. @@ -30,34 +30,46 @@ spec: name: does-not-exist ``` -##### Kubectl attach +## Kubectl attach The `kubectl attach` command does not work under the rkt container runtime. -And because of this, some flags in `kubectl run` are not supported, including: +Because of this, some flags in `kubectl run` are not supported, including: -- `--attach=true` -- `--leave-stdin-open=true` -- `--rm=true` +* `--attach=true` +* `--leave-stdin-open=true` +* `--rm=true` -##### Init containers +## Port forwarding for kvm and fly stage1s + +`kubectl port-forward` is not supported for pods that are executed with `stage1-kvm` or `stage1-fly`. + +## Volume relabeling + +Currently rkt supports only *per-pod* volume relabeling. After relabeling, the mounted volume is shared by all containers in the pod. There is not yet a way to make the relabeled volume accessible to only one, or some subset, of containers in the pod. [Kubernetes issue # 28187](https://github.com/kubernetes/kubernetes/issues/28187) has the details. + +## kubectl get logs + +Under rktnetes, `kubectl get logs` currently cannot get logs from applications that write them to directly to `/dev/stdout`. Currently such log messages are printed on the node's console. + +## Init containers The alpha [init container](https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/container-init.md) feature is currently not supported. -##### Container restart back-off +## Container restart back-off Exponential restart back-off for a failing container is currently not supported. -##### Experimental NVIDIA GPU support +## Experimental NVIDIA GPU support The `--experimental-nvidia-gpus` flag, and related [GPU features](https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/gpu-support.md) are not supported. -##### QoS Classes +## QoS Classes -Under rkt, QoS classes do not result in the `OOM Score` of containers being adjusted as occurs under Docker. +Under rkt, QoS classes do not adjust the `OOM Score` of containers as occurs under Docker. -##### HostPID and HostIPC namespaces +## HostPID and HostIPC namespaces -Setting a the hostPID or hostIPC flag on a pod is not supported. +Setting the hostPID or hostIPC flags on a pod is not supported. For example, the following pod will not run correctly: @@ -77,14 +89,10 @@ spec: On the other hand, when running the pod with [stage1-fly](https://coreos.com/rkt/docs/latest/running-fly-stage1.html), the pod will be run in the host namespace. - -##### Container image updates (patch) +## Container image updates (patch) Patching a pod to change the image will result in the entire pod restarting, not just the container that was changed. -##### Volume mounts specifying a subPath +## Volume mounts specifying a subPath -The [subPath](https://github.com/kubernetes/kubernetes/pull/22575) feature does not work correctly under rkt. -In addition, the above-issue of Non-existent host volume paths being invalid would make many common use-cases for subPaths fail in that way as well. - -In some cases, this issue can be worked around by creating and using subdirectories from within the container rather than relying on Kubernetes to do so. +The [subPath](https://github.com/kubernetes/kubernetes/pull/22575) feature does not work correctly under rkt. In addition, the issue of non-existent host volume paths being invalid, mentioned above, would cause many common use cases for subPaths to fail in the same way. In some cases, this issue can be worked around by creating and using subdirectories from within the container, rather than relying on Kubernetes to do so. From 6b9412830a8ff5fc928d65f34d2d853a519bf6cf Mon Sep 17 00:00:00 2001 From: Matt Liggett Date: Thu, 30 Jun 2016 13:45:13 -0700 Subject: [PATCH 154/348] Update details now that we try to make a PV for etcd. --- docs/admin/federation/index.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/admin/federation/index.md b/docs/admin/federation/index.md index dee50e33b4..fc077ec839 100644 --- a/docs/admin/federation/index.md +++ b/docs/admin/federation/index.md @@ -86,8 +86,9 @@ Running `federation-up.sh` also creates a new record in our kubeconfig for us to be able to talk to federation apiserver. We can view this by running `kubectl config view`. -Note: `federation-up.sh` creates the federation-apiserver pod with an etcd container that is not backed by any -persistent volume, so it is possible to lose data. You can edit +Note: `federation-up.sh` creates the federation-apiserver pod with an etcd +container that is backed by a persistent volume, so as to persist data. This +currently works only on AWS, GKE, and GCE. You can edit `federation/manifests/federation-apiserver-deployment.yaml` to suit your needs, if required. From b06a313ab9103162dc523d1fbf90528bd2403e73 Mon Sep 17 00:00:00 2001 From: Jeff Lowdermilk Date: Thu, 30 Jun 2016 14:28:18 -0700 Subject: [PATCH 155/348] remove inapplicable bits --- docs/getting-started-guides/minikube.md | 16 ---------------- 1 file changed, 16 deletions(-) diff --git a/docs/getting-started-guides/minikube.md b/docs/getting-started-guides/minikube.md index 4a58c3f51b..96fd62ae66 100644 --- a/docs/getting-started-guides/minikube.md +++ b/docs/getting-started-guides/minikube.md @@ -84,22 +84,6 @@ curl -sSL "http://storage.googleapis.com/kubernetes-release/release/{{page.versi chmod +x /usr/bin/kubectl ``` -On OS X, to make the API server accessible locally, setup a ssh tunnel. - -```shell -docker-machine ssh `docker-machine active` -N -L 8080:localhost:8080 -``` - -Setting up a ssh tunnel is applicable to remote docker hosts as well. - -(Optional) Create kubernetes cluster configuration: - -```shell -kubectl config set-cluster minikube --server=http://localhost:8080 -kubectl config set-context minikube --cluster=test-doc -kubectl config use-context minikube -``` - ### Starting the cluster To start a cluster, run the command: From abd3ee50f5ffa3bc61383e0fa7cb66ed32e5d6e4 Mon Sep 17 00:00:00 2001 From: Alain Roy Date: Mon, 27 Jun 2016 16:32:59 -0700 Subject: [PATCH 156/348] This includes two things: 1. Documentation for kube-up integration with VMware's Photon Controller 2. Minor tweaks to vSphere documentation: - Changed maintainer - Added small note on vSphere parameters --- _data/guides.yml | 4 +- docs/getting-started-guides/index.md | 6 +- .../photon-controller.md | 239 ++++++++++++++++++ docs/getting-started-guides/vsphere.md | 7 +- 4 files changed, 252 insertions(+), 4 deletions(-) create mode 100644 docs/getting-started-guides/photon-controller.md diff --git a/_data/guides.yml b/_data/guides.yml index cc1d2b8a9a..c34d1c719e 100644 --- a/_data/guides.yml +++ b/_data/guides.yml @@ -187,8 +187,10 @@ toc: path: /docs/getting-started-guides/coreos/ - title: Cloudstack path: /docs/getting-started-guides/cloudstack/ - - title: VMWare + - title: VMware vSphere path: /docs/getting-started-guides/vsphere/ + - title: VMware Photon Controller + path: /docs/getting-started-guides/photon-controller/ - title: Juju path: /docs/getting-started-guides/juju/ - title: DCOS diff --git a/docs/getting-started-guides/index.md b/docs/getting-started-guides/index.md index b3eb4289e1..d8d70b5be8 100644 --- a/docs/getting-started-guides/index.md +++ b/docs/getting-started-guides/index.md @@ -81,7 +81,8 @@ These solutions are combinations of cloud provider and OS not covered by the abo - [Vagrant](/docs/getting-started-guides/coreos) (uses CoreOS and flannel) - [CloudStack](/docs/getting-started-guides/cloudstack) (uses Ansible, CoreOS and flannel) -- [Vmware](/docs/getting-started-guides/vsphere) (uses Debian) +- [Vmware vSphere](/docs/getting-started-guides/vsphere) (uses Debian) +- [Vmware Photon Controller](/docs/getting-started-guides/photon-controller) (uses Debian) - [juju.md](/docs/getting-started-guides/juju) (uses Juju, Ubuntu and flannel) - [Vmware](/docs/getting-started-guides/coreos) (uses CoreOS and flannel) - [libvirt-coreos.md](/docs/getting-started-guides/libvirt-coreos) (uses CoreOS) @@ -137,7 +138,8 @@ Vagrant | CoreOS | CoreOS | flannel | [docs](/docs/gettin Bare-metal (Offline) | CoreOS | CoreOS | flannel | [docs](/docs/getting-started-guides/coreos/bare_metal_offline) | | Community ([@jeffbean](https://github.com/jeffbean)) Bare-metal | CoreOS | CoreOS | Calico | [docs](/docs/getting-started-guides/coreos/bare_metal_calico) | | Community ([@caseydavenport](https://github.com/caseydavenport)) CloudStack | Ansible | CoreOS | flannel | [docs](/docs/getting-started-guides/cloudstack) | | Community ([@runseb](https://github.com/runseb)) -Vmware | | Debian | OVS | [docs](/docs/getting-started-guides/vsphere) | | Community ([@pietern](https://github.com/pietern)) +Vmware vSphere | Saltstack | Debian | OVS | [docs](/docs/getting-started-guides/vsphere) | | Community ([@imkin](https://github.com/imkin)) +Vmware Photon | Saltstack | Debian | OVS | [docs](/docs/getting-started-guides/photon-controller) | | Community ([@alainroy](https://github.com/alainroy)) Bare-metal | custom | CentOS | _none_ | [docs](/docs/getting-started-guides/centos/centos_manual_config) | | Community ([@coolsvap](https://github.com/coolsvap)) AWS | Juju | Ubuntu | flannel | [docs](/docs/getting-started-guides/juju) | | [Community](https://github.com/whitmo/bundle-kubernetes) ( [@whit](https://github.com/whitmo), [@matt](https://github.com/mbruzek), [@chuck](https://github.com/chuckbutler) ) OpenStack/HPCloud | Juju | Ubuntu | flannel | [docs](/docs/getting-started-guides/juju) | | [Community](https://github.com/whitmo/bundle-kubernetes) ( [@whit](https://github.com/whitmo), [@matt](https://github.com/mbruzek), [@chuck](https://github.com/chuckbutler) ) diff --git a/docs/getting-started-guides/photon-controller.md b/docs/getting-started-guides/photon-controller.md new file mode 100644 index 0000000000..c3482dc654 --- /dev/null +++ b/docs/getting-started-guides/photon-controller.md @@ -0,0 +1,239 @@ +--- +--- + +The example below creates a Kubernetes cluster using VMware's Photon +Controller. The cluster will have one Kubernetes master and three +Kubernetes nodes. + +* TOC +{:toc} + +### Prerequisites + +1. You need administrator access to a [VMware Photon +Controller](http://vmware.github.io/photon-controller/) +deployment. (Administrator access is only required for the initial +setup: the actual creation of the cluster can be done by anyone.) + +2. The [Photon Controller CLI](https://github.com/vmware/photon-controller-cli) +needs to be installed on the machine on which you'll be running kube-up. If you +have go installed, this can be easily installed with: + +```shell +go get github.com/vmware/photon-controller-cli/photon +``` + +3. `mkisofs` needs to be installed. The installation process creates a +CD-ROM ISO image to bootstrap the VMs with cloud-init. If you are on a +Mac, you can install this with [brew](http://brew.sh/): + +```shell +brew install cdrtools +``` + +4. Several common tools need to be installed: `ssh`, `scp`, `openssl` + +5. You should have an ssh public key installed. This will be used to +give you access to the VM's user account, `kube`. + +6. Get or build a [binary release](/docs/getting-started-guides/binary_release) + +### Download VM Image + +Download a prebuilt Debian 8.2 VMDK that we'll use as a base image: + +```shell +curl --remote-name-all https://s3.amazonaws.com/photon-platform/artifacts/OS/debian/debian-8.2.vmdk +``` + +This is a base Debian 8.2 image with the addition of: + +* openssh-server +* open-vm-tools +* cloud-init + +### Configure Photon Controller: + +In order to deploy Kubernetes, you need to configure Photon Controller +with: + +* A tenant, with associated resource ticket +* A project within that tenant +* VM and disk flavors, to describe the VM characteristics +* An image: we'll use the one above + +When you do this, you'll need to configure the +`cluster/photon-controller/config-common.sh` file with the names of +the tenant, project, flavors, and image. + +If you prefer, you can use the provided `cluster/photon-controller/setup-prereq.sh` +script to create these. Assuming the IP address of your Photon +Controller is 192.0.2.2 (change as appropriate) and the downloaded image is +kube.vmdk, you can run: + +```shell +photon target set https://192.0.2.2 +photon target login ...credentials... +cluster/photon-controller/setup-prereq.sh https://192.0.2.2 kube.vmdk +``` + +The `setup-prereq.sh` script will create the tenant, project, flavors, +and image based on the same configuration file used by kube-up: +`cluster/photon-controller/config-common.sh`. Note that it will create +a resource ticket which limits how many VMs a tenant can create. You +will want to change the resource ticket configuration in +`config-common.sh` based on your actual Photon Controller deployment. + +### Configure kube-up + +There are two files used to configure kube-up's interaction with +Photon Controller: + +1. `cluster/photon-controller/config-common.sh` has the most common +parameters, including the names of the tenant, project, and image. + +2. `cluster/photon-controller/config-default.sh` has more advanced +parameters including the IP subnets to use, the number of nodes to +create and which Kubernetes components to configure. + +Both files have documentation to explain the different parameters. + +### Creating your Kubernetes cluster + +To create your Kubernetes cluster we will run the standard `kube-up` +command. As described above, the parameters that control kube-up's +interaction with Photon Controller are specified in files, not on the +command-line. + +The time to deploy varies based on the number of nodes you create as +well as the specifications of your Photon Controller hosts and +network. Times vary from 10 - 30 minutes for a ten node cluster. + +```shell +KUBERNETES_PROVIDER=photon-controller cluster/kube-up.sh +``` + +Once you have successfully reached this point, your Kubernetes cluster +works just like any other. + +Note that kube-up created a Kubernetes configuration file for you in +`~/.kube/config`. This file will allow you to use the `kubectl` +command. It contains the IP address of the Kubernetes master as well +as the password for the `admin` user. If you wish to use the +Kubernetes web-based user interface you will need this password. In +the config file you'll see a section that look like the following: you +use the password there. (Note that the output has been trimmed: the +certificate data is much lengthier) + +```yaml +- name: photon-kubernetes + user: + client-certificate-data: Q2Vyd... + client-key-data: LS0tL... + password: PASSWORD-HERE + username: admin +``` + +### Removing your Kubernetes cluster + +The recommended way to remove your Kubernetes cluster is with the +`kube-down` command: + +```shell +KUBERNETES_PROVIDER=photon-controller cluster/kube-down.sh +``` + +Your Kubernetes cluster is just a set of VMs: you can manually remove +them if you need to. + +### Making services publicly accessible + +There are multiple ways to make services publicly accessible in Kubernetes. +Currently, the photon-controller support does not yet include built-in +support for the LoadBalancer option. + +#### Option 1: NodePort + +One option is to use the NodePort option with a manually deployed +balancer. Specifically: + +Configure your service with the NodePort option. For example, this +service uses the NodePort option. All Kubernetes nodes will listen on +a port and forward network traffic to any pods in the service. In this +case, Kubernets will choose a random port, but it will be the same +port on all nodes. + +```yaml +apiVersion: v1 +kind: Service +metadata: + name: nginx-demo-service + labels: + app: nginx-demo +spec: + type: NodePort + ports: + - port: 80 + protocol: TCP + name: http + selector: + app: nginx-demo +``` + +Next, create a new standalone VM (or VMs, for high availability) to act +as a load balancer. For example, if you use haproxy, you could make a +configuration similar to the one below. Note that this example assumes there +are three Kubernetes nodes: you would adjust the configuration to reflect the +actual nodes you have. Also note that port 30144 should be replaced +with whatever NodePort was assigned by Kubernetes. + +```yaml +frontend nginx-demo + bind *:30144 + mode http + default_backend nodes +backend nodes + mode http + balance roundrobin + option forwardfor + http-request set-header X-Forwarded-Port %[dst_port] + http-request add-header X-Forwarded-Proto https if { ssl_fc } + option httpchk HEAD / HTTP/1.1\r\nHost:localhost + server web0 192.0.2.2:30144 check + server web1 192.0.2.3:30144 check + server web2 192.0.2.4:30144 check +``` + +#### Option 2: Ingress Controller + +Using an [ingress controller](docs/user-guide/ingress) may also be an +appropriate solution. Note that it in a production environment it will +also require an external load balancer. However, it may be simpler to +manage because it will not require you to manually update the load +balancer configuration, as above. + +### Details + +#### Logging into VMs + +When the VMs are created, a `kube` user is created (using +cloud-init). The password for the kube user is the same as the +administrator password for your Kubernetes master and can be found in +your Kubernetes configuration file: see above to find it. The kube user +will also authorize your ssh public key to log in. This is used during +installation to avoid the need for passwords. + +The VMs do have a root user, but ssh to the root user is disabled. + +### Networking + +The Kubernetes cluster uses `kube-proxy` to configure the overlay +network with iptables. Currently we do not support other overlay +networks such as Weave or Calico. + +## Support Level + + +IaaS Provider | Config. Mgmt | OS | Networking | Docs | Conforms | Support Level +-------------------- | ------------ | ------ | ---------- | --------------------------------------------- | ---------| ---------------------------- +Vmware Photon | Saltstack | Debian | OVS | [docs](/docs/getting-started-guides/photon-controller) | | Community ([@alainroy](https://github.com/alainroy)) diff --git a/docs/getting-started-guides/vsphere.md b/docs/getting-started-guides/vsphere.md index 3c68891b1f..4816cf5599 100644 --- a/docs/getting-started-guides/vsphere.md +++ b/docs/getting-started-guides/vsphere.md @@ -62,6 +62,11 @@ govc datastore.ls ./kube/ Take a look at the file `cluster/vsphere/config-common.sh` fill in the required parameters. The guest login for the image that you imported is `kube:kube`. +Also take a look at the file `cluster/vsphere/config-default.sh` and +make any needed changes. You can configure the number of nodes +as well as the IP subnets you have made available to Kubernetes, pods, +and services. + ### Starting a cluster Now, let's continue with deploying Kubernetes. @@ -100,7 +105,7 @@ going on (find yourself authorized with your SSH key, or use the password IaaS Provider | Config. Mgmt | OS | Networking | Docs | Conforms | Support Level -------------------- | ------------ | ------ | ---------- | --------------------------------------------- | ---------| ---------------------------- -Vmware | | Debian | OVS | [docs](/docs/getting-started-guides/vsphere) | | Community ([@pietern](https://github.com/pietern)) +Vmware vSphere | Saltstack | Debian | OVS | [docs](/docs/getting-started-guides/vsphere) | | Community ([@imkin](https://github.com/imkin)) For support level information on all solutions, see the [Table of solutions](/docs/getting-started-guides/#table-of-solutions) chart. From f8a42ad1d4993d507117add55122ea75759c03b9 Mon Sep 17 00:00:00 2001 From: Tomasz Kopczynski Date: Sun, 3 Jul 2016 21:09:02 +0200 Subject: [PATCH 157/348] Fix a typo in logging-elasticsearch.md --- docs/getting-started-guides/logging-elasticsearch.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting-started-guides/logging-elasticsearch.md b/docs/getting-started-guides/logging-elasticsearch.md index 00202e964a..d65ff7b9bd 100644 --- a/docs/getting-started-guides/logging-elasticsearch.md +++ b/docs/getting-started-guides/logging-elasticsearch.md @@ -37,7 +37,7 @@ Created [https://www.googleapis.com/compute/v1/projects/kubernetes-satnam/region +++ Logging using Fluentd to elasticsearch ``` -The node level Fluentd collector pods and the Elasticsearech pods used to ingest cluster logs and the pod for the Kibana +The node level Fluentd collector pods and the Elasticsearch pods used to ingest cluster logs and the pod for the Kibana viewer should be running in the kube-system namespace soon after the cluster comes to life. ```shell From 03ba417287f55c7ebb2cd4f40d78d536a6d3466b Mon Sep 17 00:00:00 2001 From: Jisoo Park Date: Mon, 4 Jul 2016 21:27:39 +0900 Subject: [PATCH 158/348] Fix the link url to deployments doc --- docs/user-guide/walkthrough/k8s201.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/walkthrough/k8s201.md b/docs/user-guide/walkthrough/k8s201.md index a2d8619a87..3867171f02 100644 --- a/docs/user-guide/walkthrough/k8s201.md +++ b/docs/user-guide/walkthrough/k8s201.md @@ -102,7 +102,7 @@ Delete the Deployment by name: kubectl delete deployment nginx-deployment ``` -For more information, such as how to rollback Deployment changes to a previous version, see [_Deployments_](/docs/user-guide/deployment/). +For more information, such as how to rollback Deployment changes to a previous version, see [_Deployments_](/docs/user-guide/deployments/). ## Services From 1fcbfa3d314b23db4b820bf7c1013fef2423dc12 Mon Sep 17 00:00:00 2001 From: Lee Calcote Date: Mon, 4 Jul 2016 09:37:41 -0500 Subject: [PATCH 159/348] docs/kubelet.md Corrected grammar and added bulleted list. --- docs/admin/kubelet.md | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/docs/admin/kubelet.md b/docs/admin/kubelet.md index e52da6d5c8..ac88581046 100644 --- a/docs/admin/kubelet.md +++ b/docs/admin/kubelet.md @@ -14,16 +14,14 @@ that describes a pod. The kubelet takes a set of PodSpecs that are provided thro various mechanisms (primarily through the apiserver) and ensures that the containers described in those PodSpecs are running and healthy. -Other than from an PodSpec from the apiserver, there are three ways that a container -manifest can be provided to the Kubelet. +Other than a PodSpec from the apiserver, there are three ways that a container +manifest can be provided to the Kubelet: -File: Path passed as a flag on the command line. This file is rechecked every 20 +* File: Path passed as a flag on the command line. This file is rechecked every 20 seconds (configurable with a flag). - -HTTP endpoint: HTTP endpoint passed as a parameter on the command line. This endpoint +* HTTP endpoint: HTTP endpoint passed as a parameter on the command line. This endpoint is checked every 20 seconds (also configurable with a flag). - -HTTP server: The kubelet can also listen for HTTP and respond to a simple API +* HTTP server: The kubelet can also listen for HTTP and respond to a simple API (underspec'd currently) to submit a new manifest. ``` @@ -121,4 +119,4 @@ kubelet --volume-stats-agg-period=1m0s: Specifies interval for kubelet to calculate and cache the volume disk usage for all pods and volumes. To disable volume calculations, set to 0. Default: '1m' ``` -###### Auto generated by spf13/cobra on 15-Mar-2016 \ No newline at end of file +###### Auto generated by spf13/cobra on 15-Mar-2016 From 8513dfb9db1e56cf95362f0502e0e87d63eb31b8 Mon Sep 17 00:00:00 2001 From: derekwaynecarr Date: Thu, 23 Jun 2016 16:57:05 -0400 Subject: [PATCH 160/348] Out of resource handling for 1.3 --- _data/guides.yml | 2 + docs/admin/out-of-resource.md | 220 ++++++++++++++++++++++++++++++++++ 2 files changed, 222 insertions(+) create mode 100644 docs/admin/out-of-resource.md diff --git a/_data/guides.yml b/_data/guides.yml index f5654a6efd..82ffea11da 100644 --- a/_data/guides.yml +++ b/_data/guides.yml @@ -263,5 +263,7 @@ toc: path: /docs/admin/static-pods/ - title: Configuring Garbage Collection path: /docs/admin/garbage-collection/ + - title: Configuring Out Of Resource Handling + path: /docs/admin/out-of-resource/ - title: Configuring Kubernetes with Salt path: /docs/admin/salt/ diff --git a/docs/admin/out-of-resource.md b/docs/admin/out-of-resource.md new file mode 100644 index 0000000000..2b85f486d5 --- /dev/null +++ b/docs/admin/out-of-resource.md @@ -0,0 +1,220 @@ +--- +--- + +* TOC +{:toc} + +The `kubelet` needs to preserve node stability when available compute resources are low. + +This is especially important when dealing with incompressible resources such as memory or disk. + +If either resource is exhausted, the node would become unstable. + +## Eviction Policy + +The `kubelet` can pro-actively monitor for and prevent against total starvation of a compute resource. In +cases where it could appear to occur, the `kubelet` can pro-actively fail one or more pods in order to reclaim +the starved resource. When the `kubelet` fails a pod, it terminates all containers in the pod, and the `PodPhase` +is transitioned to `Failed`. + +### Eviction Signals + +The `kubelet` can support the ability to trigger eviction decisions on the signals described in the +table below. The value of each signal is described in the description column based on the `kubelet` +summary API. + +| Eviction Signal | Description | +|------------------|---------------------------------------------------------------------------------| +| `memory.available` | `memory.available` := `node.status.capacity[memory]` - `node.stats.memory.workingSet` | + +In future releases, the `kubelet` will support the ability to trigger eviction decisions based on disk pressure. + +Until that time, it is recommended users take advantage of [garbage collection](/docs/admin/garbage-collection/). + +### Eviction Thresholds + +The `kubelet` supports the ability to specify eviction thresholds that trigger the `kubelet` to reclaim resources. + +Each threshold is of the following form: + +`` + +* valid `eviction-signal` tokens as defined above. +* valid `operator` tokens are `<` +* valid `quantity` tokens must match the quantity representation used by Kubernetes + +#### Soft Eviction Thresholds + +A soft eviction threshold pairs an eviction threshold with a required +administrator specified grace period. No action is taken by the `kubelet` +to reclaim resources associated with the eviction signal until that grace +period has been exceeded. If no grace period is provided, the `kubelet` will +error on startup. + +In addition, if a soft eviction threshold has been met, an operator can +specify a maximum allowed pod termination grace period to use when evicting +pods from the node. If specified, the `kubelet` will use the lesser value among +the `pod.Spec.TerminationGracePeriodSeconds` and the max allowed grace period. +If not specified, the `kubelet` will kill pods immediately with no graceful +termination. + +To configure soft eviction thresholds, the following flags are supported: + +* `eviction-soft` describes a set of eviction thresholds (e.g. `memory.available<1.5Gi`) that if met over a +corresponding grace period would trigger a pod eviction. +* `eviction-soft-grace-period` describes a set of eviction grace periods (e.g. `memory.available=1m30s`) that +correspond to how long a soft eviction threshold must hold before triggering a pod eviction. +* `eviction-max-pod-grace-period` describes the maximum allowed grace period (in seconds) to use when terminating +pods in response to a soft eviction threshold being met. + +#### Hard Eviction Thresholds + +A hard eviction threshold has no grace period, and if observed, the `kubelet` +will take immediate action to reclaim the associated starved resource. If a +hard eviction threshold is met, the `kubelet` will kill the pod immediately +with no graceful termination. + +To configure hard eviction thresholds, the following flag is supported: + +* `eviction-hard` describes a set of eviction thresholds (e.g. `memory.available<1Gi`) that if met +would trigger a pod eviction. + +### Eviction Monitoring Interval + +The `kubelet` evaluates eviction thresholds per its configured housekeeping interval. + +* `housekeeping-interval` is the interval between container housekeepings. + +### Node Conditions + +The `kubelet` will map one or more eviction signals to a corresponding node condition. + +If a hard eviction threshold has been met, or a soft eviction threshold has been met +independent of its associated grace period, the `kubelet` will report a condition that +reflects the node is under pressure. + +The following node conditions are defined that correspond to the specified eviction signal. + +| Node Condition | Eviction Signal | Description | +|----------------|------------------|------------------------------------------------------------------| +| `MemoryPressure` | `memory.available` | Available memory on the node has satisfied an eviction threshold | + +The `kubelet` will continue to report node status updates at the frequency specified by +`--node-status-update-frequency` which defaults to `10s`. + +### Oscillation of node conditions + +If a node is oscillating above and below a soft eviction threshold, but not exceeding +its associated grace period, it would cause the corresponding node condition to +constantly oscillate between true and false, and could cause poor scheduling decisions +as a consequence. + +To protect against this oscillation, the following flag is defined to control how +long the `kubelet` must wait before transitioning out of a pressure condition. + +* `eviction-pressure-transition-period` is the duration for which the `kubelet` has +to wait before transitioning out of an eviction pressure condition. + +The `kubelet` would ensure that it has not observed an eviction threshold being met +for the specified pressure condition for the period specified before toggling the +condition back to `false`. + +### Eviction of Pods + +If an eviction threshold has been met and the grace period has passed, +the `kubelet` will initiate the process of evicting pods until it has observed +the signal has gone below its defined threshold. + +The `kubelet` ranks pods for eviction 1) by their quality of service, +2) and among those with the same quality of service by the consumption of the +starved compute resource relative to the pods scheduling request. + +* `BestEffort` pods that consume the most of the starved resource are failed +first. +* `Burstable` pods that consume the greatest amount of the starved resource +relative to their request for that resource are killed first. If no pod +has exceeded its request, the strategy targets the largest consumer of the +starved resource. +* `Guaranteed` pods that consume the greatest amount of the starved resource +relative to their request are killed first. If no pod has exceeded its request, +the strategy targets the largest consumer of the starved resource. + +A `Guaranteed` pod is guaranteed to never be evicted because of another pod's +resource consumption. If a system daemon (i.e. `kubelet`, `docker`, `journald`, etc.) +is consuming more resources than were reserved via `system-reserved` or `kube-reserved` allocations, +and the node only has `Guaranteed` pod(s) remaining, then the node must choose to evict a +`Guaranteed` pod in order to preserve node stability, and to limit the impact +of the unexpected consumption to other `Guaranteed` pod(s). + +### Scheduler + +The node will report a condition when a compute resource is under pressure. The +scheduler views that condition as a signal to dissuade placing additional +pods on the node. + +| Node Condition | Scheduler Behavior | +| ---------------- | ------------------------------------------------ | +| `MemoryPressure` | `BestEffort` pods are not scheduled to the node. | + +## Node OOM Behavior + +If the node experiences a system OOM (out of memory) event prior to the `kubelet` is able to reclaim memory, +the node depends on the [oom_killer](https://lwn.net/Articles/391222/) to respond. + +The `kubelet` sets a `oom_score_adj` value for each container based on the quality of service for the pod. + +| Quality of Service | oom_score_adj | +| ----------------- | ------------- | +| `Guaranteed` | -998 | +| `BestEffort` | 1000 | +| `Burstable` | min(max(2, 1000 - (1000 * memoryRequestBytes) / machineMemoryCapacityBytes), 999) | + +If the `kubelet` is unable to reclaim memory prior to a node experiencing system OOM, the `oom_killer` will calculate +an `oom_score` based on the percentage of memory its using on the node, and then add the `oom_score_adj` to get an +effective `oom_score` for the container, and then kills the container with the highest score. + +The intended behavior should be that containers with the lowest quality of service that +are consuming the largest amount of memory relative to the scheduling request should be killed first in order +to reclaim memory. + +Unlike pod eviction, if a pod container is OOM killed, it may be restarted by the `kubelet` based on its `RestartPolicy`. + +## Best Practices + +### Schedulable resources and eviction policies + +Let's imagine the following scenario: + +* Node memory capacity: `10Gi` +* Operator wants to reserve 10% of memory capacity for system daemons (kernel, `kubelet`, etc.) +* Operator wants to evict pods at 95% memory utilization to reduce thrashing and incidence of system OOM. + +To facilitate this scenario, the `kubelet` would be launched as follows: + +``` +--eviction-hard=memory.available<500Mi +--system-reserved=memory=1.5Gi +``` + +Implicit in this configuration is the understanding that "System reserved" should include the amount of memory +covered by the eviction threshold. + +To reach that capacity, either some pod is using more than its request, or the system is using more than `500Mi`. + +This configuration will ensure that the scheduler does not place pods on a node that immediately induce memory pressure +and trigger eviction assuming those pods use less than their configured request. + +### DaemonSet + +It is never desired for a `kubelet` to evict a pod that was derived from +a `DaemonSet` since the pod will immediately be recreated and rescheduled +back to the same node. + +At the moment, the `kubelet` has no ability to distinguish a pod created +from `DaemonSet` versus any other object. If/when that information is +available, the `kubelet` could pro-actively filter those pods from the +candidate set of pods provided to the eviction strategy. + +In general, it is strongly recommended that `DaemonSet` not +create `BestEffort` pods to avoid being identified as a candidate pod +for eviction. Instead `DaemonSet` should ideally launch `Guaranteed` pods. From 41370f3c040e55003e8fc7d7f2867b59208dec12 Mon Sep 17 00:00:00 2001 From: Stefan Gangefors Date: Thu, 17 Mar 2016 14:30:20 +0100 Subject: [PATCH 161/348] Add proxy environment section to Vagrant guide This commit readds the proxy environment section that was removed in previous commits. --- docs/getting-started-guides/vagrant.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/docs/getting-started-guides/vagrant.md b/docs/getting-started-guides/vagrant.md index 12f02b593f..6683319e10 100644 --- a/docs/getting-started-guides/vagrant.md +++ b/docs/getting-started-guides/vagrant.md @@ -361,6 +361,24 @@ export KUBERNETES_MASTER_MEMORY=1536 export KUBERNETES_NODE_MEMORY=2048 ``` +#### I want to set proxy settings for my Kubernetes cluster boot strapping! + +If you are behind a proxy, you need to install vagrant proxy plugin and set the proxy settings by + +```sh +vagrant plugin install vagrant-proxyconf +export VAGRANT_HTTP_PROXY=http://username:password@proxyaddr:proxyport +export VAGRANT_HTTPS_PROXY=https://username:password@proxyaddr:proxyport +``` + +Optionally you can specify addresses to not proxy, for example + +```sh +export VAGRANT_NO_PROXY=127.0.0.1 +``` + +If you are using sudo to make kubernetes build for example make quick-release, you need run `sudo -E make quick-release` to pass the environment variables. + #### I ran vagrant suspend and nothing works! `vagrant suspend` seems to mess up the network. This is not supported at this time. From 96a939d885d2011da08d97cb07a653e433eede15 Mon Sep 17 00:00:00 2001 From: Tim Brockman Date: Wed, 6 Jul 2016 14:45:47 -0400 Subject: [PATCH 162/348] address issue #404 in older docker for mac More succinctly addressed issue #404 by directing the user to check for to the latest release candidate of Docker for Mac. --- docs/hellonode.md | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/docs/hellonode.md b/docs/hellonode.md index 401de6c4c0..c0702c9eed 100755 --- a/docs/hellonode.md +++ b/docs/hellonode.md @@ -104,14 +104,7 @@ $ curl http://localhost:8080 Hello World! ``` -**If you get a `Connection refused` message, your `DOCKER_HOST` address may be the address of your Docker VM, not the localhost address.** Use the `docker-machine ip default` command to find the Docker VM address. - -```shell -$ docker-machine ip default -192.168.1.100 -$ curl 192.168.1.100:8080 -Hello World! -``` +**If you recieve a `Connection refused` message from Docker for Mac, ensure you are using the latest version of Docker (1.12 or later).** Let’s now stop the container. In this example, our app was running as Docker process `2c66d0efcbd4`, which we looked up with `docker ps`: From 24578658a2c392936643ffddffa60b83fd3b0cf0 Mon Sep 17 00:00:00 2001 From: Quinton Hoole Date: Wed, 6 Jul 2016 14:14:25 -0700 Subject: [PATCH 163/348] Deprecate the term "Ubernetes" in documentation. --- docs/admin/dns.md | 10 +++++++--- docs/admin/multiple-zones.md | 12 ++++++------ 2 files changed, 13 insertions(+), 9 deletions(-) diff --git a/docs/admin/dns.md b/docs/admin/dns.md index df34fbf8a9..c56fc5b89b 100644 --- a/docs/admin/dns.md +++ b/docs/admin/dns.md @@ -31,9 +31,13 @@ performance. The healthz container provides a single health check endpoint while ## Kubernetes Federation (Multiple Zone support) -The 1.3 release introduced Federation (Ubernetes) support for multisite Kubernetes installations. There are -DNS changes introduced that will allow the lookup of services using a six part DNS name. -See [Federation docs](/docs/admin/multiple-zones/) for more details on multiple site support. +Release 1.3 introduced Cluster Federation support for multi-site +Kubernetes installations. This required some minor +(backward-compatible) changes to the way +the Kubernetes cluster DNS server processes DNS queries, to facilitate +the lookup of federated services (which span multiple Kubernetes clusters). +See the [Cluster Federation Administrators' Guide](/docs/admin/federation/index.md) for more +details on Cluster Federation and multi-site support. ## References diff --git a/docs/admin/multiple-zones.md b/docs/admin/multiple-zones.md index f3378069d6..68383ea003 100644 --- a/docs/admin/multiple-zones.md +++ b/docs/admin/multiple-zones.md @@ -5,14 +5,14 @@ Kubernetes 1.2 adds support for running a single cluster in multiple failure zones (GCE calls them simply "zones", AWS calls them "availability zones", here we'll refer to them as "zones"). -This is a lightweight version of a broader effort for federating multiple -Kubernetes clusters together (sometimes referred to by the affectionate +This is a lightweight version of a broader Cluster Federation feature (previously referred to by the affectionate nickname ["Ubernetes"](https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/federation.md). -Full federation will allow combining separate -Kubernetes clusters running in different regions or clouds. However, many +Full Cluster Federation allows combining separate +Kubernetes clusters running in different regions or cloud providers +(or on-premise data centers). However, many users simply want to run a more available Kubernetes cluster in multiple zones -of their cloud provider, and this is what the multizone support in 1.2 allows -(we nickname this "Ubernetes Lite"). +of their single cloud provider, and this is what the multizone support in 1.2 allows +(this previously went by the nickname "Ubernetes Lite"). Multizone support is deliberately limited: a single Kubernetes cluster can run in multiple zones, but only within the same region (and cloud provider). Only From 0c8bf7b16056bcb08986bbe97d779ff11160d27d Mon Sep 17 00:00:00 2001 From: Minhan Xia Date: Tue, 28 Jun 2016 17:08:07 -0700 Subject: [PATCH 164/348] add explaination of hairpin problem --- docs/user-guide/debugging-services.md | 57 +++++++++++++++++++++++++++ 1 file changed, 57 insertions(+) diff --git a/docs/user-guide/debugging-services.md b/docs/user-guide/debugging-services.md index b4f68e7e0c..39388793c4 100644 --- a/docs/user-guide/debugging-services.md +++ b/docs/user-guide/debugging-services.md @@ -555,6 +555,63 @@ There are three things to check: Engine doesn't do port remapping, so if your application serves on 8080, the `containerPort` field needs to be 8080. +### A Pod cannot reach itself via Service IP + +This mostly happens when `kube-proxy` is running in `iptables` mode and Pods +are connected with bridge network. The `Kubelet` exposes a `hairpin-mode` +[flag](http://kubernetes.io/docs/admin/kubelet/) that allows endpoints of a Service to loadbalance back to themselves +if they try to access their own Service VIP. The `hairpin-mode` flag must either be +set to `haripin-veth` or `promiscuous-bridge`. + +The common steps to trouble shoot this are as follows: + +* Confirm `hairpin-mode` is set to `haripin-veth` or `promiscuous-bridge`. +You should see something like the below. `hairpin-mode` is set to +`promiscuous-bridge` in the following example. + +```shell +u@node$ ps auxw|grep kubelet +root 3392 1.1 0.8 186804 65208 ? Sl 00:51 11:11 /usr/local/bin/kubelet --enable-debugging-handlers=true --config=/etc/kubernetes/manifests --allow-privileged=True --v=4 --cluster-dns=10.0.0.10 --cluster-domain=cluster.local --configure-cbr0=true --cgroup-root=/ --system-cgroups=/system --hairpin-mode=promiscuous-bridge --runtime-cgroups=/docker-daemon --kubelet-cgroups=/kubelet --babysit-daemons=true --max-pods=110 --serialize-image-pulls=false --outofdisk-transition-frequency=0 + +``` + +* Confirm the effective `hairpin-mode`. To do this, you'll have to look at +kubelet log. Accessing the logs depends on your Node OS. On some OSes it +is a file, such as /var/log/kubelet.log, while other OSes use `journalctl` +to access logs. Please be noted that the effective hairpin mode may not +match `--hairpin-mode` flag due to compatibility. Check if there is any log +lines with key word `hairpin` in kubelet.log. There should be log lines +indicating the effective hairpin mode, like something below. + +```shell +I0629 00:51:43.648698 3252 kubelet.go:380] Hairpin mode set to "promiscuous-bridge" +``` + +* If the effective hairpin mode is `hairpin-veth`, ensure the `Kubelet` has +the permission to operate in `/sys` on node. If everything works properly, +you should see something like: + +```shell +u@node$ for intf in /sys/devices/virtual/net/cbr0/brif/*; do cat $intf/hairpin_mode; done +1 +1 +1 +1 +``` + +* If the effective hairpin mode is `promiscuous-bridge`, ensure `Kubelet` +has the permission to manipulate linux bridge on node. If cbr0` bridge is +used and configured properly, you should see: + +```shell +u@node$ ifconfig cbr0 |grep PROMISC +UP BROADCAST RUNNING PROMISC MULTICAST MTU:1460 Metric:1 + +``` + +* Seek help if none of above works out. + + ## Seek help If you get this far, something very strange is happening. Your `Service` is From 35bd8b893fb1be7af811cec660bda2435030ec19 Mon Sep 17 00:00:00 2001 From: Alex Robinson Date: Wed, 6 Jul 2016 22:47:02 +0000 Subject: [PATCH 165/348] Remove guide that doesn't exist from the index. --- _data/guides.yml | 2 -- 1 file changed, 2 deletions(-) diff --git a/_data/guides.yml b/_data/guides.yml index cc1d2b8a9a..b41ddb4082 100644 --- a/_data/guides.yml +++ b/_data/guides.yml @@ -122,8 +122,6 @@ toc: path: /docs/getting-started-guides/logging/ - title: Logging with Elasticsearch and Kibana path: /docs/getting-started-guides/logging-elasticsearch/ - - title: Elasticsearch/Kibana Logging Demo - path: /docs/user-guide/logging-demo - title: Running Commands in a Container with kubectl exec path: /docs/user-guide/getting-into-containers/ - title: Connect with Proxies From 37510560b6ab0fdf87946c7d88d3f01378b716d8 Mon Sep 17 00:00:00 2001 From: Prashanth Balasubramanian Date: Tue, 21 Jun 2016 10:25:46 -0700 Subject: [PATCH 166/348] Petset 1.3 user guide --- docs/user-guide/ingress.md | 2 ++ docs/user-guide/petset.md | 53 +++++++++++++++++++++++++++++++++----- 2 files changed, 49 insertions(+), 6 deletions(-) diff --git a/docs/user-guide/ingress.md b/docs/user-guide/ingress.md index 78a0d201a9..61bae15d59 100644 --- a/docs/user-guide/ingress.md +++ b/docs/user-guide/ingress.md @@ -221,6 +221,8 @@ spec: servicePort: 80 ``` +Please refer to documentation on [nginx](https://github.com/kubernetes/contrib/tree/master/ingress/controllers/nginx#https), [GCE](https://github.com/kubernetes/contrib/tree/master/ingress/controllers/gce#tls), or any other platform specific Ingress controller to understand how TLS works in your environment. + ### Loadbalancing An Ingress controller is bootstrapped with some loadbalancing policy settings that it applies to all Ingress, such as the loadbalancing algorithm, backend weight scheme etc. More advanced loadbalancing concepts (eg: persistent sessions, dynamic weights) are not yet exposed through the Ingress. You can still get these features through the [service loadbalancer](https://github.com/kubernetes/contrib/tree/master/service-loadbalancer). With time, we plan to distill loadbalancing patterns that are applicable cross platform into the Ingress resource. diff --git a/docs/user-guide/petset.md b/docs/user-guide/petset.md index 324e1f1764..1a14f4ee26 100644 --- a/docs/user-guide/petset.md +++ b/docs/user-guide/petset.md @@ -98,7 +98,7 @@ Saving this config into `petset.yaml` and submitting it to a Kubernetes cluster ```shell $ kubectl create -f petset.yaml service "nginx" created -petset "nginx" deleted +petset "nginx" created ``` ## Pet Identity @@ -131,21 +131,53 @@ pvc-902733c2-3717-11e6-a46e-42010af00002 1Gi RWO Bound de The network identity has 2 parts. First, we created a headless Service that controls the domain within which we create Pets. The domain managed by this Service takes the form: `$(service name).$(namespace).svc.cluster.local`, where "cluster.local" is the [cluster domain](http://releases.k8s.io/{{page.githubbranch}}/build/kube-dns/README.md#how-do-i-configure-it). As each pet is created, it gets a matching DNS subdomain, taking the form: `$(petname).$(governing service domain)`, where the governing service is defined by the `serviceName` field on the Pet Set. +Here are some examples of choices for Cluster Domain, Service name, Pet Set name, and how that affects the DNS names for the Pets and the hostnames in the Pet's pods: + Cluster Domain | Service (ns/name) | Pet Set (ns/name) | Pet Set Domain | Pet DNS | Pet Hostname | -------------- | ----------------- | ----------------- | -------------- | ------- | ------------ | cluster.local | default/nginx | default/web | nginx.default.svc.cluster.local | web-{0..N-1}.nginx.default.svc.cluster.local | web-{0..N-1} | cluster.local | foo/nginx | foo/web | nginx.foo.svc.cluster.local | web-{0..N-1}.nginx.foo.svc.cluster.local | web-{0..N-1} | kube.local | foo/nginx | foo/web | nginx.foo.svc.kube.local | web-{0..N-1}.nginx.foo.svc.kube.local | web-{0..N-1} | -Lets verify this assertion with a simple test. +Note that Cluster Domain will be set to `cluster.local` unless [otherwise configured](http://releases.k8s.io/{{page.githubbranch}}/build/kube-dns/README.md#how-do-i-configure-it). + +Lets verify our assertion with a simple test. ```shell $ kubectl get svc NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE nginx None 80/TCP 12m +... ``` -The containers are running nginx webservers, which by default will look for an index.html file in `/usr/share/nginx/html/index.html`. That directory is backed by a PersistentVolume created by the Pet Set. So lets write our hostname there (remember the Pet Set gives us a stable hostname): +First, the PetSet gives provides a stable hostname: + +```shell +$ for i in 0 1; do kubectl exec web-$i -- sh -c 'hostname'; done +web-0 +web-1 +``` + +And the hostname is linked to the in-cluster DNS address: + +```shell +$ kubectl run -it --image busybox dns-test --restart=Never /bin/sh +dns-test # nslookup web-0.nginx +Server: 10.0.0.10 +Address 1: 10.0.0.10 kube-dns.kube-system.svc.cluster.local + +Name: web-0.nginx +Address 1: 10.180.3.5 + +dns-test # nslookup web-1.nginx +Server: 10.0.0.10 +Address 1: 10.0.0.10 kube-dns.kube-system.svc.cluster.local + +Name: web-1.nginx +Address 1: 10.180.0.9 +``` + +The containers are running nginx webservers, which by default will look for an index.html file in `/usr/share/nginx/html/index.html`. That directory is backed by a `PersistentVolume` created by the Pet Set. So lets write our hostname there: ```shell $ for i in 0 1; do @@ -153,6 +185,14 @@ $ for i in 0 1; do done ``` +And verify each webserver serves its own hostname: + +```shell +$ for i in 0 1; do kubectl exec -it web-$i -- curl localhost; done +web-0 +web-1 +``` + Now delete all pods in the petset: ```shell @@ -161,7 +201,7 @@ pod "web-0" deleted pod "web-1" deleted ``` -Wait for them to come back up, and try to retrieve the previously written hostname through the DNS name of the peer (remember the Pet Set also gives us stable storage, and that the hostname is linked to the DNS name). +Wait for them to come back up, and try to retrieve the previously written hostname through the DNS name of the peer. They match, because the storage, DNS name, and hostname stick to the Pet no matter where it gets scheduled: ```shell $ kubectl exec -it web-1 -- curl web-0.nginx @@ -250,7 +290,7 @@ $ sleep $grace $ kubectl delete pvc -l app=nginx {% endraw %}``` -## Troubleshooting +## Troubelshooting You might have noticed an `annotations` field in all the Pet Sets shown above. @@ -277,5 +317,6 @@ This list goes on, if you have examples, ideas or thoughts, please contribute. Deploying one RC of size 1/Service per pod is a popular alternative, as is simply deploying a DaemonSet that utilizes the identity of a Node. +## Next steps - +The deployment and maintenance of stateful applications is a vast topic. The next step is to explore cluster bootstrapping and initialization, [here](/docs/user-guide/petset/bootstrapping/). From 4df7d96369964a267466bdffc73f7d343c77584f Mon Sep 17 00:00:00 2001 From: Prashanth Balasubramanian Date: Thu, 23 Jun 2016 22:43:11 -0700 Subject: [PATCH 167/348] Petset bootstrapping --- _data/guides.yml | 2 + docs/user-guide/petset.md | 2 +- docs/user-guide/petset/bootstrapping/index.md | 234 ++++++++++++++++++ .../petset/bootstrapping/petset_peers.yaml | 103 ++++++++ .../petset/bootstrapping/petset_vm.yaml | 102 ++++++++ 5 files changed, 442 insertions(+), 1 deletion(-) create mode 100644 docs/user-guide/petset/bootstrapping/index.md create mode 100644 docs/user-guide/petset/bootstrapping/petset_peers.yaml create mode 100644 docs/user-guide/petset/bootstrapping/petset_vm.yaml diff --git a/_data/guides.yml b/_data/guides.yml index 46a988505f..108359f4fe 100644 --- a/_data/guides.yml +++ b/_data/guides.yml @@ -104,6 +104,8 @@ toc: path: /docs/user-guide/downward-api/volume - title: Persistent Volumes Walkthrough path: /docs/user-guide/persistent-volumes/walkthrough/ + - title: Bootstrapping Pet Sets + path: /docs/user-guide/petset/bootstrapping/ - title: Monitoring, Logging, and Debugging Containers section: diff --git a/docs/user-guide/petset.md b/docs/user-guide/petset.md index 1a14f4ee26..77b132dea2 100644 --- a/docs/user-guide/petset.md +++ b/docs/user-guide/petset.md @@ -290,7 +290,7 @@ $ sleep $grace $ kubectl delete pvc -l app=nginx {% endraw %}``` -## Troubelshooting +## Troubleshooting You might have noticed an `annotations` field in all the Pet Sets shown above. diff --git a/docs/user-guide/petset/bootstrapping/index.md b/docs/user-guide/petset/bootstrapping/index.md new file mode 100644 index 0000000000..2462c43691 --- /dev/null +++ b/docs/user-guide/petset/bootstrapping/index.md @@ -0,0 +1,234 @@ +--- +--- + +* TOC +{:toc} + +## Overview + +This purpose of this guide is to help you become familiar with the runtime initialization of [Pet Sets](/docs/user-guide/petset). This guide assumes the same prerequisites, and uses the same terminology as the [Pet Set user document](/docs/user-guide/petset). + +The most common way to initialize the runtime in a containerized environment, is through a custom [entrypoint](https://docs.docker.com/engine/reference/builder/#entrypoint). While this is not necessarily bad, making your application pid 1, and treating containers as processes in general is good for a few reasons outside the scope of this document. Doing so allows you to run docker images from third-party vendors without modification. We will not be writing custom entrypoints for this example, but using a feature called [init containers](http://releases.k8s.io/{{page.githubbranch}}/docs/proposals/container-init.md), to explain 2 common patterns that come up deploying Pet Sets. + +1. Transferring state across Pet restart, so that a future Pet is initialized with the computations of its past incarnation +2. Initializing the runtime environment of a Pet based on existing conditions, like a list of currently healthy peers + +## Example I: transferring state across Pet restart + +This example shows you how to "carry over" runtime state across Pet restart by simulating virtual machines with a Pet Set. + +### Background + +Applications that incrementally build state usually need strong guarantees that they will not restart for extended durations. This is tricky to achieve with containers, so instead, we will ensure that the results of previous computations are trasferred to future pets. Doing so is straight-forward using vanilla Persistent Volumes (which Pet Set already gives you), unless the volume mount point itself needs to be initialized for the Pet to start. This is exactly the case with "virtual machine" docker images, like those based on ubuntu or fedora. Such images embed the entier rootfs of the distro, including package managers like `apt-get` that assume a certain layout of the filesystem. Meaning: + +* If you mount an empty volume under `/usr`, you won't be able to `apt-get` +* If you mount an empty volume under `/lib`, all your `apt-gets` will fail because there are no system libraries +* If you clobber either of those, previous `apt-get` results will be dysfunctional + +### Simulating Virtual Machines + +Since Pet Set already gives each Pet a consistent identity, all we need is a way to initialize the user environment before allowing tools like `kubectl exec` to enter the application container. + +Download [this](petset_vm.yaml) petset into a file called petset_vm.yaml, and create it: + +```shell +$ kubectl create -f ./petset_vm.yaml +service "ub" created +petset "vm" created +``` + +This should give you 2 pods. + +```shell +$ kubectl get po +NAME READY STATUS RESTARTS AGE +vm-0 1/1 Running 0 37s +vm-1 1/1 Running 0 2m +``` + +We can exec into one and install nginx + +```shell +$ kubectl exec vm-0 /bin/sh +vm-0 # apt-get update +... +vm-0 # apt-get install nginx -y +``` + +On killing this pod we need it to come back with all the Pet Set properties, as well as the installed nginx packages. + +```shell +$ kubectl delete po vm-0 +pod "vm-0" deleted + +$ kubectl get po +NAME READY STATUS RESTARTS AGE +vm-0 1/1 Running 0 1m +vm-1 1/1 Running 0 4m +``` + +Now you can exec back into vm-0 and start nginx + +```shell +$ kubectl exec -it vm-0 /bin/sh +vm-0 # mkdir -p /var/log/nginx /var/lib/nginx; nginx -g 'daemon off;' + +``` + +And access it from anywhere in the cluster (and because this is an example that simulates vms, we're going to apt-get install netcat too) + +```shell +$ kubectl exec -it vm-1 /bin/sh +vm-1 # printf "GET / HTTP/1.0\r\n\r\n" | netcat vm-0.ub 80 +``` + +It's worth exploring what just happened. Init containers run sequentially *before* the application container. In this example we used the init container to copy shared libraries from the rootfs, while preserving user installed packages across container restart. + +```yaml +pod.alpha.kubernetes.io/init-containers: '[ + { + "name": "rootfs", + "image": "ubuntu:15.10", + "command": [ + "/bin/sh", + "-c", + "for d in usr lib etc; do cp -vnpr /$d/* /${d}mnt; done;" + ], + "volumeMounts": [ + { + "name": "usr", + "mountPath": "/usrmnt" + }, + { + "name": "lib", + "mountPath": "/libmnt" + }, + { + "name": "etc", + "mountPath": "/etcmnt" + } + ] + } +]' +``` + +**It's important to note that the init container, when used this way, must be idempotent, or it'll end up clobbering data stored by a previous incarnation.** + + +## Example II: initializing state based on environment + +In this example we are going to setup a cluster of nginx servers, just like we did in the Pet Set [user guide](/docs/user-guide/petset), but make one of them a master. All the other nginx servers will simply proxy requests to the master. This is a common deployment pattern for databases like Mysql, but we're going to replace the database with a stateless webserver to simplify the problem. + +### Background + +Most clustered applications, such as mysql, require an admin to create a config file based on the current state of the world. The most common dynamic variable in such config files is a list of peers, or other Pets running similar database servers that are currently serving requests. The Pet Set user guide already [touched on this topic](/docs/user-guide/petset#peer-discovery), we'll explore it in greater depth in the context of writing a config file with a list of peers. + +Here's a tiny peer finder helper script that handles peer discovery, [available here](https://github.com/kubernetes/contrib/tree/master/pets/peer-finder). The peer finder takes 3 important arguments: + +* A DNS domain +* An `on-start` script to run with the initial constituency of the given domain as input +* An `on-change` script to run everytime the constituency of the given domain changes + +The role of the peer finder: + +* Poll DNS for SRV records of a given domain till the `hostname` of the pod it's running in shows up as a subdomain +* Pipe the sorted list of subdomains to the script specified by its `--on-start` argument +* Exit with the appropriate error code if no `--on-change` script is specified +* Loop invoking `--on-change` for every change + +You can invoke the peer finder inside the Pets we created in the last example: + +```shell +$ kubectl exec -it vm-0 /bin/bash +vm-0 # curl -sSL -o /peer-finder https://storage.googleapis.com/kubernetes-release/pets/peer-finder +vm-0 # chmod -c 755 peer-finder + +vm-0 # ./peer-finder +2016/06/23 21:25:46 Incomplete args, require -on-change and/or -on-start, -service and -ns or an env var for POD_NAMESPACE. + +vm-0 # ./peer-finder -on-start 'tee' -service ub -ns default + +2016/06/23 21:30:21 Peer list updated +was [] +now [vm-0.ub.default.svc.cluster.local vm-1.ub.default.svc.cluster.local] +2016/06/23 21:30:21 execing: tee with stdin: vm-0.ub.default.svc.cluster.local +vm-1.ub.default.svc.cluster.local +2016/06/23 21:30:21 vm-0.ub.default.svc.cluster.local +vm-1.ub.default.svc.cluster.local +2016/06/23 21:30:22 Peer finder exiting +``` + +### Nginx master/slave cluster + +Lets create a Pet Set that writes out its own config based on a list of peers at initalization time, as described above. + +Download and create [this](petset_peers.yaml) petset. It will setup 2 nginx webservers, but the second one will proxy all requests to the first: + +```shell +$ kubectl create -f petset_peers.yaml +service "nginx" created +petset "web" created + +$ kubectl get po --watch-only +NAME READY STATUS RESTARTS AGE +web-0 0/1 Pending 0 7s +web-0 0/1 Init:0/1 0 18s +web-0 0/1 PodInitializing 0 20s +web-0 1/1 Running 0 21s +web-1 0/1 Pending 0 0s +web-1 0/1 Init:0/1 0 0s +web-1 0/1 PodInitializing 0 20s +web-1 1/1 Running 0 21s + +$ kubectl get po +NAME READY STATUS RESTARTS AGE +web-0 1/1 Running 0 1m +web-1 1/1 Running 0 47s +``` + +web-1 will redirect all requests to it's "master": + +```shell +$ kubectl exec -it web-1 -- curl localhost +web-0 +``` + +If you scale the cluster, the new pods parent themselves to the same master. To test this you can `kubectl edit` the petset and change the `replicas` field to 5: + +```shell +$ kubectl get po -l app=nginx +NAME READY STATUS RESTARTS AGE +web-0 1/1 Running 0 2h +web-1 1/1 Running 0 2h +web-2 1/1 Running 0 1h +web-3 1/1 Running 0 1h +web-4 1/1 Running 0 1h + +$ for i in $(seq 0 4); do kubectl exec -it web-$i -- curl localhost; done | sort | uniq +web-0 +``` + +Understanding how we generated the nginx config is important, we did so by passing an init script to the peer finder: + +```shell +echo ` +readarray PEERS; +if [ 1 = ${#PEERS[@]} ]; then + echo \"events{} http { server{ } }\"; +else + echo \"events{} http { server{ location / { proxy_pass http://${PEERS[0]}; } } }\"; +fi;` > /conf/nginx.conf +``` + +All that does is: + +* read in a list of peers from stdin +* if there's only 1, promote it to master +* if there's more than 1, proxy requests to the 0th member of the list +* write the config to a `hostPath` volume shared with the parent PetSet + +**It's important to note that in practice all Pets should query their peers for the current master, instead of making assumptions based on the index.** + +## Next Steps + +You can deploy some example Pet Sets found [here](https://github.com/kubernetes/kubernetes/tree/master/test/e2e/testing-manifests/petset), or write your own. diff --git a/docs/user-guide/petset/bootstrapping/petset_peers.yaml b/docs/user-guide/petset/bootstrapping/petset_peers.yaml new file mode 100644 index 0000000000..e12988dee2 --- /dev/null +++ b/docs/user-guide/petset/bootstrapping/petset_peers.yaml @@ -0,0 +1,103 @@ +# A headless service to create DNS records +apiVersion: v1 +kind: Service +metadata: + annotations: + service.alpha.kubernetes.io/tolerate-unready-endpoints: "true" + name: nginx + labels: + app: nginx +spec: + ports: + - port: 80 + name: web + # *.nginx.default.svc.cluster.local + clusterIP: None + selector: + app: nginx +--- +apiVersion: apps/v1alpha1 +kind: PetSet +metadata: + name: web +spec: + serviceName: "nginx" + replicas: 2 + template: + metadata: + labels: + app: nginx + annotations: + pod.alpha.kubernetes.io/initialized: "true" + pod.alpha.kubernetes.io/init-containers: '[ + { + "name": "peerfinder", + "image": "gcr.io/google_containers/peer-finder:0.1", + "args": [ + "-on-start=\"\"echo ` + readarray PEERS; + echo $(hostname) > /usr/share/nginx/html/index.html; + if [ 1 = ${#PEERS[@]} ]; then + echo \"events{} http { server{ } }\"; + else + echo \"events{} http { server{ location / { proxy_pass http://${PEERS[0]}; } } }\"; + fi;` > /conf/nginx.conf\"\"", + "-service=nginx" + ], + "env": [ + { + "name": "POD_NAMESPACE", + "valueFrom": { + "fieldRef": { + "apiVersion": "v1", + "fieldPath": "metadata.namespace" + } + } + } + ], + "volumeMounts": [ + { + "name": "www", + "mountPath": "/usr/share/nginx/html" + }, + { + "name": "conf", + "mountPath": "/conf" + } + ] + } + ]' + spec: + terminationGracePeriodSeconds: 0 + containers: + - name: nginx + image: gcr.io/google_containers/nginx-slim:0.7 + ports: + - containerPort: 80 + name: web + command: + - nginx + args: + - -g + - "daemon off;" + - -c + - "/conf/nginx.conf" + volumeMounts: + - name: www + mountPath: /usr/share/nginx/html + - name: conf + mountPath: /conf + volumes: + - name: conf + emptyDir: {} + volumeClaimTemplates: + - metadata: + name: www + annotations: + volume.alpha.kubernetes.io/storage-class: anything + spec: + accessModes: [ "ReadWriteOnce" ] + resources: + requests: + storage: 1Gi + diff --git a/docs/user-guide/petset/bootstrapping/petset_vm.yaml b/docs/user-guide/petset/bootstrapping/petset_vm.yaml new file mode 100644 index 0000000000..c506a1bac6 --- /dev/null +++ b/docs/user-guide/petset/bootstrapping/petset_vm.yaml @@ -0,0 +1,102 @@ +# A headless service to create DNS records +apiVersion: v1 +kind: Service +metadata: + name: ub + labels: + app: ub +spec: + ports: + - port: 80 + name: web + # *.ub.default.svc.cluster.local + clusterIP: None + selector: + app: ub +--- +apiVersion: apps/v1alpha1 +kind: PetSet +metadata: + name: vm +spec: + serviceName: "ub" + replicas: 2 + template: + metadata: + labels: + app: ub + annotations: + pod.alpha.kubernetes.io/initialized: "true" + pod.alpha.kubernetes.io/init-containers: '[ + { + "name": "rootfs", + "image": "ubuntu:15.10", + "command": [ + "/bin/sh", + "-c", + "for d in usr lib etc; do cp -vnpr /$d/* /${d}mnt; done;" + ], + "volumeMounts": [ + { + "name": "usr", + "mountPath": "/usrmnt" + }, + { + "name": "lib", + "mountPath": "/libmnt" + }, + { + "name": "etc", + "mountPath": "/etcmnt" + } + ] + } + ]' + spec: + terminationGracePeriodSeconds: 0 + containers: + - name: ub + image: ubuntu:15.10 + ports: + - containerPort: 80 + name: web + command: + - /bin/sh + - -c + - 'while true; do sleep 10; done' + volumeMounts: + - name: usr + mountPath: /usr + - name: lib + mountPath: /lib + - name: etc + mountPath: /etc + volumeClaimTemplates: + - metadata: + name: usr + annotations: + volume.alpha.kubernetes.io/storage-class: anything + spec: + accessModes: [ "ReadWriteOnce" ] + resources: + requests: + storage: 300Mi + - metadata: + name: lib + annotations: + volume.alpha.kubernetes.io/storage-class: anything + spec: + accessModes: [ "ReadWriteOnce" ] + resources: + requests: + storage: 1Gi + - metadata: + name: etc + annotations: + volume.alpha.kubernetes.io/storage-class: anything + spec: + accessModes: [ "ReadWriteOnce" ] + resources: + requests: + storage: 150Mi + From c85c543a8e9347ca5a272a21f45be43238c1657b Mon Sep 17 00:00:00 2001 From: Josh Wood Date: Wed, 6 Jul 2016 16:53:25 -0700 Subject: [PATCH 168/348] docs/getting-started/rkt/: Pick rktnetes docs forward from release-1.3 branch in #741 --- docs/getting-started-guides/rkt/index.md | 309 ++++++++++++----------- docs/getting-started-guides/rkt/notes.md | 137 +++++----- 2 files changed, 224 insertions(+), 222 deletions(-) diff --git a/docs/getting-started-guides/rkt/index.md b/docs/getting-started-guides/rkt/index.md index e644b52462..00d5764c83 100644 --- a/docs/getting-started-guides/rkt/index.md +++ b/docs/getting-started-guides/rkt/index.md @@ -1,219 +1,222 @@ --- --- -This document describes how to run Kubernetes using [rkt](https://github.com/coreos/rkt) as a container runtime. -We still have [a bunch of work](http://issue.k8s.io/8262) to do to make the experience with rkt wonderful, please stay tuned! +This document describes how to run Kubernetes using [rkt](https://github.com/coreos/rkt) as the container runtime. -### **Prerequisite** +* TOC +{:toc} -- [systemd](http://www.freedesktop.org/wiki/Software/systemd/) should be installed on the machine and should be enabled. The minimum version required at this moment (2015/09/01) is 219 - *(Note that systemd is not required by rkt itself, we are using it here to monitor and manage the pods launched by kubelet.)* +## Prerequisites -- Install the latest rkt release according to the instructions [here](https://github.com/coreos/rkt). - The minimum version required for now is [v0.8.0](https://github.com/coreos/rkt/releases/tag/v0.8.0). +* [Systemd](http://www.freedesktop.org/wiki/Software/systemd/) must be installed and enabled. The minimum systemd version required for Kubernetes v1.3 is `219`. Systemd is used to monitor and manage the pods on each node. -- Note that for rkt version later than v0.7.0, `metadata service` is not required for running pods in private networks. So now rkt pods will not register the metadata service be default. +* [Install the latest rkt release](https://coreos.com/rkt/docs/latest/trying-out-rkt.html). The minimum rkt version required is [v1.9.1](https://github.com/coreos/rkt/releases/tag/v1.9.1). The [CoreOS Linux alpha channel](https://coreos.com/releases/) ships with a recent rkt release, and you can easily [upgrade rkt on CoreOS](https://coreos.com/rkt/docs/latest/install-rkt-in-coreos.html), if necessary. -- Since release [v1.2.0-alpha.5](https://github.com/kubernetes/kubernetes/releases/tag/v1.2.0-alpha.5), -the [rkt API service](https://github.com/coreos/rkt/blob/master/api/v1alpha/README.md) -must be running on the node. +* The [rkt API service](https://coreos.com/rkt/docs/latest/subcommands/api-service.html) must be running on the node. -### Network Setup +## Pod networking in rktnetes -rkt uses the [Container Network Interface (CNI)](https://github.com/appc/cni) -to manage container networking. By default, all pods attempt to join a network -called `rkt.kubernetes.io`, which is currently defined [in -`rkt.go`](https://github.com/kubernetes/kubernetes/blob/v1.2.0-alpha.6/pkg/kubelet/rkt/rkt.go#L91). -In order for pods to get correct IP addresses, the CNI config file must be -edited to add this `rkt.kubernetes.io` network: +### Kubernetes CNI networking -#### Using flannel +You can configure Kubernetes pod networking with the usual Container Network Interface (CNI) [network plugins](/docs/admin/network-plugins/) by setting the kubelet's `--network-plugin` and `--network-plugin-dir` options appropriately. Configured in this fashion, the rkt container engine will be unaware of network details, and expects to connect pods to the provided subnet. -In addition to the basic prerequisites above, each node must be running -a [flannel](https://github.com/coreos/flannel) daemon. This implies -that a flannel-supporting etcd service must be available to the cluster -as well, apart from the Kubernetes etcd, which will not yet be -available at flannel configuration time. Once it's running, flannel can -be set up with a CNI config like: +#### kubenet: Google Compute Engine (GCE) network + +The `kubenet` plugin can be selected with the kubelet option `--network-plugin=kubenet`. This plugin is currently only supported on GCE. When using kubenet, Kubernetes CNI creates and manages the network, and rkt is provided with a subnet from a bridge device connected to the GCE network. + +### rkt contained network + +Rather than delegating pod networking to Kubernetes, rkt can configure connectivity directly with its own [*contained network*](https://coreos.com/rkt/docs/latest/networking/overview.html#contained-mode) on a subnet provided by a bridge device, the flannel SDN, or another CNI plugin. Configured this way, rkt looks in its [config directories](https://coreos.com/rkt/docs/latest/configuration.html#command-line-flags), usually `/etc/rkt/net.d`, to discover the CNI configuration and invoke the appropriate plugins to create the pod network. + +#### rkt contained network with bridge + +The *contained network* is rkt's default, so you can leave the kubelet's `--network-plugin` option empty to select this network. The contained network can be backed by any CNI plugin. With the *contained network*, rkt will attempt to join pods to a network named `rkt.kubernetes.io`, so this network name must be used for whatever desired CNI configuration. + +When using the contained network, create a network configuration file beneath the rkt network config directory that defines how to create this `rkt.kubernetes.io` network in your environment. This example sets up a bridge device with the `bridge` CNI plugin: ```shell -$ cat </etc/rkt/net.d/k8s_cluster.conf +$ cat </etc/rkt/net.d/k8s_network_example.conf { - "name": "rkt.kubernetes.io", - "type": "flannel" -} -EOF -``` - -While `k8s_cluster.conf` is a rather arbitrary name for the config file itself, -and can be adjusted to suit local conventions, the keys and values should be exactly -as shown above. `name` must be `rkt.kubernetes.io` and `type` should be `flannel`. -More details about the flannel CNI plugin can be found -[in the CNI documentation](https://github.com/appc/cni/blob/master/Documentation/flannel.md). - -#### On GCE - -Each VM on GCE has an additional 256 IP addresses routed to it, so -it is possible to forego flannel in smaller clusters. This makes the -necessary CNI config file a bit more verbose: - -```shell -$ cat </etc/rkt/net.d/k8s_cluster.conf -{ - "name": "rkt.kubernetes.io", - "type": "bridge", - "bridge": "cbr0", - "isGateway": true, - "ipam": { - "type": "host-local", - "subnet": "10.255.228.1/24", - "gateway": "10.255.228.1" - }, + "name": "rkt.kubernetes.io", + "type": "bridge", + "bridge": "mybridge", + "mtu": 1460, + "addIf": "true", + "isGateway": true, + "ipMasq": true, + "ipam": { + "type": "host-local", + "subnet": "10.22.0.0/16", + "gateway": "10.22.0.1", "routes": [ { "dst": "0.0.0.0/0" } ] + } } EOF ``` -This example creates a `bridge` plugin configuration for the CNI network, specifying -the bridge name `cbr0`. It also specifies the CIDR, in the `ipam` field. +#### rkt contained network with flannel -Creating these files for any moderately-sized cluster is at best inconvenient. -Work is in progress to -[enable Kubernetes to use the CNI by default] -(https://github.com/kubernetes/kubernetes/pull/18795/files). -As that work matures, such manual CNI config munging will become unnecessary -for primary use cases. For early adopters, an initial example shows one way to -[automatically generate these CNI configurations] -(https://gist.github.com/yifan-gu/fbb911db83d785915543) -for rkt. +While it is recommended to operate flannel through the Kubernetes CNI support, you can alternatively configure the flannel plugin directly to provide the subnet for rkt's contained network. An example CNI/flannel config file looks like this: -### Local cluster +```shell +$ cat </etc/rkt/net.d/k8s_flannel_example.conf +{ + "name": "rkt.kubernetes.io", + "type": "flannel", + "delegate": { + "isDefaultGateway": true + } +} +EOF +``` -To use rkt as the container runtime, we need to supply the following flags to kubelet: +For more information on flannel configuration, see the [CNI/flannel README](https://github.com/containernetworking/cni/blob/master/Documentation/flannel.md). -- `--container-runtime=rkt` chooses the container runtime to use. Possible values: 'docker', 'rkt'. Default: 'docker'. -- `--rkt-path=$PATH_TO_RKT_BINARY` sets the path of rkt binary. Leave empty to use the first rkt in $PATH. -- `--rkt-stage1-image` sets the path of the stage1 image. Local paths and http/https URLs are supported. Leave empty to use the 'stage1.aci' that locates in the same directory as the rkt binary. +#### Contained network caveats: -If you are using the [hack/local-up-cluster.sh](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/hack/local-up-cluster.sh) script to launch the local cluster, then you can edit the environment variable `CONTAINER_RUNTIME`, `RKT_PATH` and `RKT_STAGE1_IMAGE` to -set these flags: +* You must create an appropriate CNI configuration file with a network name of `rkt.kubernetes.io`. +* The downwards API and environment variable substitution will not contain the pod IP address. +* The `/etc/hosts` file will not contain the pod's own hostname, although `/etc/hostname` is populated. + +## Running rktnetes + +### Spin up a local Kubernetes cluster with the rkt runtime + +To use rkt as the container runtime in a local Kubernetes cluster, supply the following flags to the kubelet: + +* `--container-runtime=rkt` Set the node's container runtime to rkt. +* `--rkt-api-endpoint=HOST:PORT` Set the endpoint of the rkt API service. Default: `localhost:15441`. +* `--rkt-path=PATH_TO_RKT_BINARY` Set the path of the rkt binary. Optional. If empty, look for `rkt` in `$PATH`. +* `--rkt-stage1-image=STAGE1` Set the name of the stage1 image, e.g. `coreos.com/rkt/stage1-coreos`. Optional. If not set, the default Linux kernel software isolation stage1 is used. + +If you are using the [hack/local-up-cluster.sh](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/hack/local-up-cluster.sh) script to launch the cluster, you can edit the environment variables `CONTAINER_RUNTIME`, `RKT_PATH`, and `RKT_STAGE1_IMAGE` to set these flags. `RKT_PATH` and `RKT_STAGE1_IMAGE` are optional if `rkt` is in your $PATH` with appropriate configuration. ```shell $ export CONTAINER_RUNTIME=rkt -$ export RKT_PATH=$PATH_TO_RKT_BINARY -$ export RKT_STAGE1_IMAGE=PATH=$PATH_TO_STAGE1_IMAGE +$ export RKT_PATH= +$ export RKT_STAGE1_IMAGE= ``` -Then we can launch the local cluster using the script: +Now you can launch the cluster using the `local-up-cluster.sh` script: ```shell $ hack/local-up-cluster.sh ``` -### CoreOS cluster on Google Compute Engine (GCE) +We are also working on getting rkt working as the container runtime in [minikube](https://github.com/kubernetes/minikube/issues/168). -To use rkt as the container runtime for your CoreOS cluster on GCE, you need to specify the OS distribution, project, image: +### Launch a rktnetes cluster on Google Compute Engine (GCE) + +This section outlines using the `kube-up` script to launch a CoreOS/rkt cluster on GCE. + +Specify the OS distribution, the GCE distributor's master project, and the instance images for the Kubernetes master and nodes. Set the `KUBE_CONTAINER_RUNTIME` to `rkt`: ```shell $ export KUBE_OS_DISTRIBUTION=coreos -$ export KUBE_GCE_NODE_IMAGE= +$ export KUBE_GCE_MASTER_PROJECT=coreos-cloud +$ export KUBE_GCE_MASTER_IMAGE= $ export KUBE_GCE_NODE_PROJECT=coreos-cloud +$ export KUBE_GCE_NODE_IMAGE= $ export KUBE_CONTAINER_RUNTIME=rkt ``` -You can optionally choose the version of rkt used by setting `KUBE_RKT_VERSION`: +Optionally, set the version of rkt by setting `KUBE_RKT_VERSION`: ```shell -$ export KUBE_RKT_VERSION=0.15.0 +$ export KUBE_RKT_VERSION=1.9.1 ``` -Then you can launch the cluster by: +Optionally, select an alternative [stage1 isolator](#modular-isolation-with-interchangeable-stage1-images) for the container runtime by setting `KUBE_RKT_STAGE1_IMAGE`: + +```shell +$ export KUBE_RKT_STAGE1_IMAGE= +``` + +Then you can launch the cluster with: ```shell $ cluster/kube-up.sh ``` -Note that we are still working on making all containerized the master components run smoothly in rkt. Before that we are not able to run the master node with rkt yet. +### Launch a rktnetes cluster on AWS -### CoreOS cluster on AWS +The `kube-up` script is not yet supported on AWS. Instead, we recommend following the [Kubernetes on AWS guide](https://coreos.com/kubernetes/docs/latest/kubernetes-on-aws.html) to launch a CoreOS Kubernetes cluster on AWS, then setting kubelet options as above. -To use rkt as the container runtime for your CoreOS cluster on AWS, you need to specify the provider and OS distribution: +### Deploy apps to the cluster -```shell -$ export KUBERNETES_PROVIDER=aws -$ export KUBE_OS_DISTRIBUTION=coreos -$ export KUBE_CONTAINER_RUNTIME=rkt +After creating the cluster, you can start deploying applications. For an introductory example, [deploy a simple nginx web server](/docs/user-guide/simple-nginx). Note that this example did not have to be modified for use with a "rktnetes" cluster. More examples can be found in the [Kubernetes examples directory](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/). + +## Modular isolation with interchangeable stage1 images + +rkt executes containers in an interchangeable isolation environment. This facility is called the [*stage1* image](https://coreos.com/rkt/docs/latest/devel/architecture.html#stage-1). There are currently three supported rkt stage1 images: + +* `systemd-nspawn` stage1, the default. Isolates running containers with Linux kernel namespaces and cgroups in a manner similar to the default container runtime. +* [`KVM` stage1](https://coreos.com/rkt/docs/latest/running-lkvm-stage1.html), runs containers inside a KVM hypervisor-managed virtual machine. Experimental in the Kubernetes v1.3 release. +* [`fly stage1`](https://coreos.com/rkt/docs/latest/running-fly-stage1.html), which isolates containers with only a `chroot`, giving host-level access to mount and network namespaces for specially-privileged utilities. + +In addition to the three provided stage1 images, you can [create your own](https://coreos.com/rkt/docs/latest/devel/stage1-implementors-guide.html) for specific isolation requirements. If no configuration is set, the [default stage1](https://coreos.com/rkt/docs/latest/build-configure.html#parameters-for-setting-up-default-stage1-image) is used. There are two ways to select a different stage1; either per-node, or per-pod: + +* Set the kubelet's `--rkt-stage1-image` flag, which tells the kubelet the stage1 image to use for every pod on the node. For example, `--rkt-stage1-image=coreos/rkt/stage1-coreos` selects the default systemd-nspawn stage1. +* Set the annotation `rkt.alpha.kubernetes.io/stage1-name-override` to override the stage1 used to execute a given pod. This allows for mixing different container isolation mechanisms on the same cluster or on the same node. For example, the following (shortened) pod manifest will run its pod with the `fly stage1` to give the application -- the `kubelet` in this case -- access to the host's namespace: + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: kubelet + namespace: kube-system + labels: + k8s-app: kubelet + annotations: + rkt.alpha.kubernetes.io/stage1-name-override: coreos.com/rkt/stage1-fly +spec: + containers: + - name: kubelet + image: quay.io/coreos/hyperkube:v1.3.0-beta.2_coreos.0 + command: + - kubelet + - --api-servers=127.0.0.1:8080 + - --config=/etc/kubernetes/manifests + - --allow-privileged + - --kubeconfig=/etc/kubernetes/kubeconfig + securityContext: + privileged: true +[...] ``` -You can optionally choose the version of rkt used by setting `KUBE_RKT_VERSION`: +### Notes on using different stage1 images + +Setting the stage1 annotation could potentially give the pod root privileges. Because of this, the `privileged` boolean in the pod's `securityContext` must be set to `true`. + +Use rkt's [*contained network*](#rkt-contained-network) with the KVM stage1, because the CNI plugin driver does not yet fully support the hypervisor-based runtime. + +## Known issues and differences between rkt and Docker + +rkt and the default node container engine have very different designs, as do rkt's native ACI and the Docker container image format. Users may experience different behaviors when switching from one container engine to the other. More information can be found [in the Kubernetes rkt notes](/docs/getting-started-guides/rkt/notes/). + +## Troubleshooting + +Here are a few tips for troubleshooting Kubernetes with the rkt container engine: + +### Check rkt pod status + +To check the status of running pods, use the rkt subcommands [`rkt list`](https://coreos.com/rkt/docs/latest/subcommands/list.html), [`rkt status`](https://coreos.com/rkt/docs/latest/subcommands/status.html), and [`rkt image list`](https://coreos.com/rkt/docs/latest/subcommands/image.html#rkt-image-list). See the [rkt commands documentation](https://coreos.com/rkt/docs/latest/commands.html) for more information about rkt subcommands. + +### Check journal logs + +Check a pod's log using `journalctl` on the node. Pods are managed and named as systemd units. The pod's unit name is formed by concatenating a `k8s_` prefix with the pod UUID, in a format like `k8s_${RKT_UUID}`. Find the pod's UUID with `rkt list` to assemble its service name, then ask journalctl for the logs: + ```shell -$ export KUBE_RKT_VERSION=0.8.0 +$ sudo journalctl -u k8s_ad623346 ``` -You can optionally choose the CoreOS channel by setting `COREOS_CHANNEL`: +#### Log verbosity -```shell -$ export COREOS_CHANNEL=stable -``` +By default, the log verbosity level is 2. In order to see more log messages related to rkt, set this level to 4 or above. For a local cluster, set the environment variable: `LOG_LEVEL=4`. -Then you can launch the cluster by: +### Check Kubernetes events and logs. -```shell -$ kube-up.sh -``` - -Note: CoreOS is not supported as the master using the automated launch -scripts. The master node is always Ubuntu. - -### Getting started with your cluster - -See [a simple nginx example](/docs/user-guide/simple-nginx) to try out your new cluster. - -For more complete applications, please look in the [examples directory](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/examples/). - -### Different UX with rkt container runtime - -rkt and Docker have very different designs, as well as ACI and Docker image format. Users might experience some different experience when switching from one to the other. More information can be found [here](/docs/getting-started-guides/rkt/notes/). - -### Debugging - -Here are several tips for you when you run into any issues. - -##### Check logs - -By default, the log verbose level is 2. In order to see more logs related to rkt, we can set the verbose level to 4. -For local cluster, we can set the environment variable: `LOG_LEVEL=4`. -If the cluster is using salt, we can edit the [logging.sls](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/cluster/saltbase/pillar/logging.sls) in the saltbase. - -##### Check rkt pod status - -To check the pods' status, we can use rkt command, such as `rkt list`, `rkt status`, `rkt image list`, etc. -More information about rkt command line can be found [here](https://github.com/coreos/rkt/blob/master/Documentation/commands.md) - -##### Check journal logs - -As we use systemd to launch rkt pods(by creating service files which will run `rkt run-prepared`, we can check the pods' log -using `journalctl`: - -- Check the running state of the systemd service: - -```shell -$ sudo journalctl -u $SERVICE_FILE -``` - -where `$SERVICE_FILE` is the name of the service file created for the pod, you can find it in the kubelet logs. - -##### Check the log of the container in the pod: - -```shell -$ sudo journalctl -M rkt-$UUID -u $CONTAINER_NAME -``` - -where `$UUID` is the rkt pod's UUID, which you can find via `rkt list --full`, and `$CONTAINER_NAME` is the container's name. - -##### Check Kubernetes events, logs. - -Besides above tricks, Kubernetes also provides us handy tools for debugging the pods. More information can be found [here](/docs/user-guide/application-troubleshooting) +Kubernetes provides various tools for troubleshooting and examination. More information can be found [in the app troubleshooting guide](/docs/user-guide/application-troubleshooting). diff --git a/docs/getting-started-guides/rkt/notes.md b/docs/getting-started-guides/rkt/notes.md index 44eb57f90e..30a022dc24 100644 --- a/docs/getting-started-guides/rkt/notes.md +++ b/docs/getting-started-guides/rkt/notes.md @@ -1,99 +1,98 @@ --- --- -# Notes on Different UX with rkt container runtime +The following features either are not supported or have large caveats when using the rkt container runtime. Increasing support for these items and others, including reasonable feature parity with the default container engine, is planned through future releases. -### Doesn't support ENTRYPOINT + CMD feature +## Non-existent host volume paths -To run a Docker image, rkt will convert it into [App Container Image (ACI) format](https://github.com/appc/spec/blob/master/SPEC.md) first. -However, during the conversion, the `ENTRYPOINT` and `CMD` are concatentated to construct ACI's `Exec` field. -This means after the conversion, we are not able to replace only `ENTRYPOINT` or `CMD` without touching the other part. -So for now, users are recommended to specify the **executable path** in `Command` and **arguments** in `Args`. -(This has the same effect if users specify the **executable path + arguments** in `Command` or `Args` alone). +When mounting a host volume path that does not exist, rkt will error out. Under the Docker runtime, an empty directory will be created at the referenced path. -For example: +An example of a pod which will error out: ```yaml apiVersion: v1 kind: Pod metadata: - name: nginx labels: - name: nginx + name: mount-dne + name: mount-dne spec: + volumes: + - name: does-not-exist + hostPath: + path: /does/not/exist containers: - - name: nginx - image: nginx - ports: - - containerPort: 80 + - name: exit + image: busybox + command: ["sh", "-c", "ls /test; sleep 60"] + volumeMounts: + - mountPath: /test + name: does-not-exist ``` -The above pod yaml file is valid as it's not specifying `Command` or `Args`, so the default `ENTRYPOINT` and `CMD` of the image will be used. +## Kubectl attach + +The `kubectl attach` command does not work under the rkt container runtime. +Because of this, some flags in `kubectl run` are not supported, including: + +* `--attach=true` +* `--leave-stdin-open=true` +* `--rm=true` + +## Port forwarding for kvm and fly stage1s + +`kubectl port-forward` is not supported for pods that are executed with `stage1-kvm` or `stage1-fly`. + +## Volume relabeling + +Currently rkt supports only *per-pod* volume relabeling. After relabeling, the mounted volume is shared by all containers in the pod. There is not yet a way to make the relabeled volume accessible to only one, or some subset, of containers in the pod. [Kubernetes issue # 28187](https://github.com/kubernetes/kubernetes/issues/28187) has the details. + +## kubectl get logs + +Under rktnetes, `kubectl get logs` currently cannot get logs from applications that write them to directly to `/dev/stdout`. Currently such log messages are printed on the node's console. + +## Init containers + +The alpha [init container](https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/container-init.md) feature is currently not supported. + +## Container restart back-off + +Exponential restart back-off for a failing container is currently not supported. + +## Experimental NVIDIA GPU support + +The `--experimental-nvidia-gpus` flag, and related [GPU features](https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/gpu-support.md) are not supported. + +## QoS Classes + +Under rkt, QoS classes do not adjust the `OOM Score` of containers as occurs under Docker. + +## HostPID and HostIPC namespaces + +Setting the hostPID or hostIPC flags on a pod is not supported. + +For example, the following pod will not run correctly: ```yaml apiVersion: v1 kind: Pod metadata: - name: busybox labels: - name: busybox + name: host-ipc-pid + name: host-ipc-pid spec: + hostIPC: true + hostPID: true containers: - - name: busybox - image: busybox - command: - - /bin/sleep - - 1000 + ... ``` -```yaml -apiVersion: v1 -kind: Pod -metadata: - name: busybox - labels: - name: busybox -spec: - containers: - - name: busybox - image: busybox - command: - - /bin/sleep - args: - - 1000 -``` +On the other hand, when running the pod with [stage1-fly](https://coreos.com/rkt/docs/latest/running-fly-stage1.html), the pod will be run in the host namespace. -```yaml -apiVersion: v1 -kind: Pod -metadata: - name: busybox - labels: - name: busybox -spec: - containers: - - name: busybox - image: busybox - args: - - /bin/sleep - - 1000 -``` +## Container image updates (patch) -All the three examples above are valid as they contain both the executable path and the arguments. +Patching a pod to change the image will result in the entire pod restarting, not just the container that was changed. -```yaml -apiVersion: v1 -kind: Pod -metadata: - name: busybox - labels: - name: busybox -spec: - containers: - - name: busybox - image: busybox - args: - - 1000 -``` +## Volume mounts specifying a subPath -The last example is invalid, as we cannot override just the `CMD` of the image alone. +The [subPath](https://github.com/kubernetes/kubernetes/pull/22575) feature does not work correctly under rkt. In addition, the issue of non-existent host volume paths being invalid, mentioned above, would cause many common use cases for subPaths to fail in the same way. In some cases, this issue can be worked around by creating and using subdirectories from within the container, rather than relying on Kubernetes to do so. From 1c47cd800cf1122ab8de8ccdbfe1b77c8351c0fd Mon Sep 17 00:00:00 2001 From: Prashanth Balasubramanian Date: Wed, 6 Jul 2016 17:16:59 -0700 Subject: [PATCH 169/348] Update ingress docs --- docs/user-guide/ingress.md | 20 ++++++++------------ 1 file changed, 8 insertions(+), 12 deletions(-) diff --git a/docs/user-guide/ingress.md b/docs/user-guide/ingress.md index 61bae15d59..09728d8501 100644 --- a/docs/user-guide/ingress.md +++ b/docs/user-guide/ingress.md @@ -44,20 +44,12 @@ Before you start using the Ingress resource, there are a few things you should u On GCE/GKE there should be a [L7 cluster addon](https://github.com/kubernetes/contrib/blob/master/ingress/controllers/gce/README.md), deployed into the `kube-system` namespace: ```shell -$ kubectl get pods --namespace=kube-system -l name=glbc +$ kubectl get pods --namespace=kube-system -l k8s-app=glbc NAME READY STATUS RESTARTS AGE l7-lb-controller-v0.6.0-chnan 2/2 Running 0 1d ``` -Make sure you review the [beta limitations](https://github.com/kubernetes/contrib/tree/master/ingress/controllers/gce/BETA_LIMITATIONS.md) of this controller. In particular, you need to create a single firewall-rule on your cloudprovider, to allow health checks. On GKE this would be: - -```shell -$ gcloud compute firewall-rules create allow-130-211-0-0-22 \ - --source-ranges 130.211.0.0/22 \ - --allow tcp:30000-32767 -``` - -In environments other than GCE/GKE, you need to [deploy a controller](https://github.com/kubernetes/contrib/tree/master/ingress/controllers) as a pod. +Make sure you review the [beta limitations](https://github.com/kubernetes/contrib/tree/master/ingress/controllers/gce/BETA_LIMITATIONS.md) of this controller. In environments other than GCE/GKE, you need to [deploy a controller](https://github.com/kubernetes/contrib/tree/master/ingress/controllers) as a pod. ## The Ingress Resource @@ -94,6 +86,10 @@ __Global Parameters__: For the sake of simplicity the example Ingress has no glo In order for the Ingress resource to work, the cluster must have an Ingress controller running. This is unlike other types of controllers, which typically run as part of the `kube-controller-manager` binary, and which are typically started automatically as part of cluster creation. You need to choose the ingress controller implementation that is the best fit for your cluster, or implement one. Examples and instructions can be found [here](https://github.com/kubernetes/contrib/tree/master/ingress/controllers). +## Before you begin + +The following document describes a set of cross platform features exposed through the Ingress resource. Ideally, all Ingress controllers should fulfill this specification, but we're not there yet. The docs for the GCE and nginx controllers are [here](https://github.com/kubernetes/contrib/blob/master/ingress/controllers/gce/README.md) and [here](https://github.com/kubernetes/contrib/blob/master/ingress/controllers/nginx/README.md) respectively. **Make sure you review controller specific docs so you understand the caveats of each one**. + ## Types of Ingress ### Single Service Ingress @@ -221,13 +217,13 @@ spec: servicePort: 80 ``` -Please refer to documentation on [nginx](https://github.com/kubernetes/contrib/tree/master/ingress/controllers/nginx#https), [GCE](https://github.com/kubernetes/contrib/tree/master/ingress/controllers/gce#tls), or any other platform specific Ingress controller to understand how TLS works in your environment. +Note that there is a gap between TLS features supported by various Ingress controllers. Please refer to documentation on [nginx](https://github.com/kubernetes/contrib/tree/master/ingress/controllers/nginx#https), [GCE](https://github.com/kubernetes/contrib/tree/master/ingress/controllers/gce#tls), or any other platform specific Ingress controller to understand how TLS works in your environment. ### Loadbalancing An Ingress controller is bootstrapped with some loadbalancing policy settings that it applies to all Ingress, such as the loadbalancing algorithm, backend weight scheme etc. More advanced loadbalancing concepts (eg: persistent sessions, dynamic weights) are not yet exposed through the Ingress. You can still get these features through the [service loadbalancer](https://github.com/kubernetes/contrib/tree/master/service-loadbalancer). With time, we plan to distill loadbalancing patterns that are applicable cross platform into the Ingress resource. -It's also worth noting that even though health checks are not exposed directly through the Ingress, there exist parallel concepts in Kubernetes such as [readiness probes](https://github.com/kubernetes/kubernetes/blob/release-1.0/docs/user-guide/production-pods.md#liveness-and-readiness-probes-aka-health-checks) which allow you to achieve the same end result. +It's also worth noting that even though health checks are not exposed directly through the Ingress, there exist parallel concepts in Kubernetes such as [readiness probes](https://github.com/kubernetes/kubernetes/blob/release-1.0/docs/user-guide/production-pods.md#liveness-and-readiness-probes-aka-health-checks) which allow you to achieve the same end result. Please review the controller specific docs to see how they handle health checks ([nginx](https://github.com/kubernetes/contrib/blob/master/ingress/controllers/nginx/README.md), [GCE](https://github.com/kubernetes/contrib/blob/master/ingress/controllers/gce/README.md#health-checks)). ## Updating an Ingress From 62677a7f8f28c973e33bc60b60e12c593f3d15ea Mon Sep 17 00:00:00 2001 From: Minhan Xia Date: Tue, 7 Jun 2016 17:46:46 -0700 Subject: [PATCH 170/348] add docs for loadBalancerSourceRange field --- docs/user-guide/services-firewalls.md | 46 ++++++++++++++++++- .../services/load-balancer-sample.json | 6 ++- .../services/load-balancer-sample.yaml | 6 ++- docs/user-guide/services/operations.md | 10 +++- docs/user-guide/services/service-sample.yaml | 3 +- 5 files changed, 64 insertions(+), 7 deletions(-) diff --git a/docs/user-guide/services-firewalls.md b/docs/user-guide/services-firewalls.md index b7f48116cd..805acdde96 100644 --- a/docs/user-guide/services-firewalls.md +++ b/docs/user-guide/services-firewalls.md @@ -6,6 +6,50 @@ exposure to the internet. When exposing a service to the external world, you ma one or more ports in these firewalls to serve traffic. This document describes this process, as well as any provider specific details that may be necessary. +### Restrict Access For LoadBlancer Service + + When using a Service with `spec.type: LoadBalancer`, you can specify the IP ranges that are allowed to access the load balancer + by using `spec.loadBalancerSourceRanges`. This field takes a list of IP CIDR ranges, which Kubernetes will use to configure firewall exceptions. + This feature is currently supported on Google Compute Engine, Google Container Engine and AWS. This field will be ignored if the cloud provider does not support the feature. + + Assuming 10.0.0.0/8 is the internal subnet. In the following example, a load blancer will be created that is only accessible to cluster internal ips. + This will not allow clients from outside of your Kubernetes cluster to access the load blancer. + +```yaml +apiVersion: v1 +kind: Service +metadata: + name: myapp +spec: + ports: + - port: 8765 + targetPort: 9376 + selector: + app: example + type: LoadBalancer + loadBalancerSourceRanges: + - 10.0.0.0/8 +``` + + In the following example, a load blancer will be created that is only accessible to clients with IP addresses from 130.211.204.1 and 130.211.204.2. + +```yaml +apiVersion: v1 +kind: Service +metadata: + name: myapp +spec: + ports: + - port: 8765 + targetPort: 9376 + selector: + app: example + type: LoadBalancer + loadBalancerSourceRanges: + - 130.211.204.1/32 + - 130.211.204.2/32 +``` + ### Google Compute Engine When using a Service with `spec.type: LoadBalancer`, the firewall will be @@ -48,4 +92,4 @@ This will be fixed in an upcoming release of Kubernetes. ### Other cloud providers -Coming soon. \ No newline at end of file +Coming soon. diff --git a/docs/user-guide/services/load-balancer-sample.json b/docs/user-guide/services/load-balancer-sample.json index 847f21dbb6..e12d1e5d24 100644 --- a/docs/user-guide/services/load-balancer-sample.json +++ b/docs/user-guide/services/load-balancer-sample.json @@ -12,6 +12,10 @@ "selector": { "app": "example" }, - "type": "LoadBalancer" + "type": "LoadBalancer", + "loadBalancerSourceRanges": [ + "10.180.0.0/16", + "10.245.0.0/24" + ] } } diff --git a/docs/user-guide/services/load-balancer-sample.yaml b/docs/user-guide/services/load-balancer-sample.yaml index b66c5ea231..28b1dd3014 100644 --- a/docs/user-guide/services/load-balancer-sample.yaml +++ b/docs/user-guide/services/load-balancer-sample.yaml @@ -4,9 +4,11 @@ metadata: name: myapp spec: ports: - - - port: 8765 + - port: 8765 targetPort: 9376 selector: app: example type: LoadBalancer + loadBalancerSourceRanges: + - 10.180.0.0/16 + - 10.245.0.0/24 diff --git a/docs/user-guide/services/operations.md b/docs/user-guide/services/operations.md index 64551d47a4..f9f5778ee8 100644 --- a/docs/user-guide/services/operations.md +++ b/docs/user-guide/services/operations.md @@ -51,7 +51,11 @@ YAML or as JSON, and supports the following fields: "selector": { string: string }, - "type": "LoadBalancer" + "type": "LoadBalancer", + "loadBalancerSourceRanges": [ + "10.180.0.0/16", + "10.245.0.0/24" + ] } } ``` @@ -71,6 +75,10 @@ Required fields are: * `type`: Optional. If the type is `LoadBalancer`, sets up a [network load balancer](/docs/user-guide/load-balancer/) for your service. This provides an externally-accessible IP address that sends traffic to the correct port on your cluster nodes. + * `loadBalancerSourceRanges:`: Optional. Must use with `LoadBalancer` type. + If specified and supported by the cloud provider, this will restrict traffic + such that the load balancer will be accessible only to clients from the specified IP ranges. + This field will be ignored if the cloud-provider does not support the feature. For the full `service` schema see the [Kubernetes api reference](/docs/api-reference/v1/definitions/#_v1_service). diff --git a/docs/user-guide/services/service-sample.yaml b/docs/user-guide/services/service-sample.yaml index b9c214226f..c819df7ac5 100644 --- a/docs/user-guide/services/service-sample.yaml +++ b/docs/user-guide/services/service-sample.yaml @@ -4,8 +4,7 @@ metadata: name: myapp spec: ports: - - - port: 8765 + - port: 8765 targetPort: 9376 selector: app: example From 6c9ce7bd357f5a7693e8e38faf728a80322a0737 Mon Sep 17 00:00:00 2001 From: johndmulhausen Date: Thu, 7 Jul 2016 02:06:24 -0700 Subject: [PATCH 171/348] Delete .travis.yml --- .travis.yml | 22 ---------------------- 1 file changed, 22 deletions(-) delete mode 100644 .travis.yml diff --git a/.travis.yml b/.travis.yml deleted file mode 100644 index df4d6c8710..0000000000 --- a/.travis.yml +++ /dev/null @@ -1,22 +0,0 @@ -language: ruby -rvm: -- 2.1 -branches: - only: - - release-1.3 -script: -- cd $HOME -- git config --global user.email ${GIT_EMAIL} -- git config --global user.name "${GIT_NAME}" -- git clone https://${GIT_USERNAME}:${GH_TOKEN}@github.com/kubernetes-v1-3/kubernetes-v1-3.github.io.git -- cd kubernetes-v1-3.github.io -- git remote add --fetch --track release-1.3 homebase "https://${GIT_USERNAME}:${GH_TOKEN}@github.com/kubernetes/kubernetes.github.io.git" -- git merge -s recursive -X theirs homebase/release-1.3 -m "Sync from homebase" -- git push -env: - global: - - secure: Fd6wlE2mjPb1fAACxklQcJumpJWycYkaJQBfKRcjGCFlmw1XWVFGhpUC7Ni/MOyzTolqOvtb2rXnYpaujMlJP1UXqVFJ+zPbwur2lc8unQF8PcqPezl8DbPsr6HdceOjRdut/dN8zTw6+hZRDzw/mG4Rf8IVaozlYycAOnWZdAZsLXdbBpAvBp0WYHP9+8wn9xiet1L/QpSiawa/Q35Q9UM1tciPmfFBL4Fkq6Mm8/w6ECaxHyA2EX+eH5ea9EtykrzB5cMA/odJLptjnmfzsPeGS5F1MxuTqrH2Z09emZcqjfXW+fNNzioiZdD4BQe+rqOA1Ktpw645szJVJJenIMzM4P8dDiW+9cJyddGdTC9A7APgLlLVdG/tuoYcjiY1F068SWx8sUWW/xc7YenCzGj2nXIeHjUuWsjqjMorpBKe6Y1QSGfi7HsLq1DpJDR1xQZbPgM/FOaovVIPIozgAIBpS1ukdNNwadmrCw55tUKfr4s2SNv3SUmeprL37QEtIxtBpPDjL1z+qTLjc6JzRr7J57guPBdNbj6Ukg/uxW+z1CwDYb3uAIIa5e6eDxfxGQsUEj8NmkIHBLLGTnKb1WAyPXAIf9ZvZHUwQQzHvOWXGZRd32enRh00uvuwHEy3OuGEqTAOIs/31q5WAamZ1uqzW7KBTMU/nq4QA9W2yfM= - - secure: Lvj1MT/I2zaJp9mQi46us64O4JfeT9cz1m+nqKY/nuC828ULfqMYLlpw08jvrc25JhAaqHNawAwdtaxHTQqHztsmq1ixHFYA6G/1oll+YPhym29toFT9oIkXThf/L77FTJfM1gVFMyiHFbz11Ob8R15tRB1WF1+Uu1RBmtudJD7HSy0u6Uc6lPxpqtycWVRCyPdvZqF3e0KIZaTDRkRzJpcVMHa/5ZJDDcMtuJyjJYXZqS1WR3QHC1z44LlnqyB5ZM2PU4H3LyWnY8wHF2mutF0QtDDVdEuBqILBHiFuKHMxpLY92UgHm2n51RR63MxFjjEzE+iu5f9ComEm5JC0N/cc8sunIiol+d8SRC30/00Vs0tvmeAjRX9IMCExiP3mv7Tz6mqEVk6PyrVlg675hxRg0eVdaNNv92gWzCSIecZ5TuCaRG4JaWO0P8lQC4NreONt6gbnwMD350hMQZLpUUE0QoSEdafpudaD+agl4ZzTFVTOOcSOz2Sa/+RT96Msazq2YlddYXaEKZeyYzqHVkk60PiQzQcAuwCMFrAagqMe3bNI2aCFEWbc9CoR13K/wwRDwAeSzBq/UkylZ8AayJLnIpewr/iYBOQasrXLrorW969Rfr8d/nwhDN5VpRgItU0arDzngaJRoZEAtHx4zaCvZF/H/nvbORcD2gsFP9Q= - - secure: WGmgvSnT1OeHPYtsm/JzcRDnZYp+SeelgbSVNbtwOXGImkeqDo/sDIRH80HxVC3zkUu5wvRO/NkMA7N3LXN2BqENacWBFQ+j+Hf3lnDw9yETBaBpLQPO/MElKGd48GoSiukCipfaITUfwtjAQ3wx0W/z06dYC9BdUYvUM73OtsGtlrY1MEQnBfff/neDGZbzfvTdgDwg1n6d69WP+WSUJt5Aysz0ZoAbYO2rzBrKNwcL7KEhoI/ketXXQ4xEW0nnxU/qfxqhVfJbck1HpSz+HMsVGXF4tT3zdRXmU+P47KGbfYn0GmlviGrRSWCu9/elKTlX9fIqRnR1/UvdTkflwLsk4i3hWnEUflKpIT7soJrKjNQseyd06KI8qvVRXRcll92vCtsYDuOO1AlemxIqp2dIFduCf+5FWH06PbS8oHU8NBe5aVYaoBY8nK14EB3A98saJ8Un7ziB8moRaWZX9bMjfJWfxDF526NdHIusqLQrxvUthtBjf5aRULE2PeIyqYVY8/IKmBEfsy9IUENmlM5KItvgZZt4xdk8o//kceLrKabudHGVC1M1RLZo9ejkdLySz2i8nlJyJ03Hn30H9DfDjW/OMQTn/b1HU4CYSbd/yGR+6+5+tdPeKy/Z6gI6+EjHRYaaQrBLo0RXm2620cb7uDTBIsKVs2TetyhTq54= - - secure: lqboLQKFpfa6F+UkSCs9+NPv027EnSWZA4/cmAXj0hZvxc4W+qMFIlPKytcq+bbZrp7kx3cZgRd5NrgmOTqlJ9merVK1cgMsDegMv/YImHjhGVBH+3ATvqMDb7I7dLg4x/DjhKt/ogyZpoh8vsS77T9MQXaIJM8JMM3ISxYI6MMloCh0z/hbT2qv3IrPMFjpHohHdF1yorxKvSefNgPx6Y4wC+t1t+u8wS9RBhvrmOkVynlU2NgbSiMPyz3syJmnszHDTXXJlqciGT5Lc8lzpzt2OGOnZmAupEYVGok7Jx4aAxU9O8/bnoDwOcGGVrd/pGHTI5+rvT4TPy+WG1Aec55dgva4XZfrJQ88snHLvq0VrvdiI/fHJEPF18QnrBwKSni1Vd1jOzsuPIF6HL0Wwv8tklO8OX3D5wqVsXpeJd4Cj4HPPAtzxOVsV6Fz3On9vuLdV7StC/ZafQL40koaytqkNO0gX8zeUiaaVQtMCf/2MVTbX9x5m51gkGpwT0JBvqQpSMOlIM5S1fiN4X9DLBLqmsARYeZw1Jeiq7Zm39Sy1QeCgfUXS8+6t62BjWx70iftYGIkoljXtD7x/3pjKymdpwjcyUcS8KQ5W6vbMXFxmZe2phtgGTCxOIDuASPspD8zwm/ckIGArHj4qkrg94/mUpgDDIbVsaUpWXjYvSc= - - secure: bPZMNH79Lx0Wb2SCxTwyZek28w/keKxEAlfad8RDMwJrct6Bi2B0o4KjkrwFS5DyCUU7Ndk192145XnUKOWX2YVs6cQ8ge+5LvtgYhWLgX8g5Ycro+JyzOBskn+o1gQjvi8+3b42X31efcUmTEhfRdKVrrUpONEIcjG1NpLGk/mQJ6AM6hGWO2xdNfAezeWq5ISnpK0b6VUWZyTEDg3NivrTfCL2juWWCnjYm1BWSHUblXwRQ/Rl7Tcldl6cMMUVsalUQ1iG0h8YZDrxNz0cm3XTZZZJKuSHYeTCLd57RBeHD5/iMxjCmfzfq6ETNLONWLmtvA8yhWMQZ0DPFtLZzbqFIfOR6P1feBZFqP7/X5KZeFKBufN15JbcXIqHE8homLY9mS0LhyNffOs5G/P/x8ChE1DJaYiZIWCZ60umvpqibScZB3z5uFPTxLk9rJSOtT7hCWjcmg3EdJ+R4ExOiBDd62ZS5jH72WU4uysPXORRofUCL+zHycJoJxsFWQSW49GAGrohllrW45jnTgpalErxvjJFSKy8JW9w634eetz9ct2fObep7m8bfVMl8U2H3ITIoXHm+f8ooHUiNzHFLSl4wtcoAOtrCAvGtJPNfv8T2eTkznzj7Tk+XzTvFg03u+J99TmnC58Bs516Oc7E40NarmsZceOD2sN4BA3X9tE= From 6ffd9d6b1caff9cdb28fc6ee7c701ccfee4f1605 Mon Sep 17 00:00:00 2001 From: johndmulhausen Date: Thu, 7 Jul 2016 02:06:45 -0700 Subject: [PATCH 172/348] Delete _travis.yml --- _travis.yml | 22 ---------------------- 1 file changed, 22 deletions(-) delete mode 100644 _travis.yml diff --git a/_travis.yml b/_travis.yml deleted file mode 100644 index 5dbf8c972a..0000000000 --- a/_travis.yml +++ /dev/null @@ -1,22 +0,0 @@ -language: ruby -rvm: -- 2.1 -branches: - only: - - master -script: -- cd $HOME -- git config --global user.email ${GIT_EMAIL} -- git config --global user.name "${GIT_NAME}" -- git clone https://${GIT_USERNAME}:${GH_TOKEN}@github.com/kubernetes-v1-2/kubernetes-v1-2.github.io.git -- cd kubernetes-v1-2.github.io -- git remote add --fetch --track master homebase "https://${GIT_USERNAME}:${GH_TOKEN}@github.com/kubernetes/kubernetes.github.io.git" -- git merge -s recursive -X theirs homebase/master -m "Sync from homebase" -- git push -env: - global: - - secure: Fd6wlE2mjPb1fAACxklQcJumpJWycYkaJQBfKRcjGCFlmw1XWVFGhpUC7Ni/MOyzTolqOvtb2rXnYpaujMlJP1UXqVFJ+zPbwur2lc8unQF8PcqPezl8DbPsr6HdceOjRdut/dN8zTw6+hZRDzw/mG4Rf8IVaozlYycAOnWZdAZsLXdbBpAvBp0WYHP9+8wn9xiet1L/QpSiawa/Q35Q9UM1tciPmfFBL4Fkq6Mm8/w6ECaxHyA2EX+eH5ea9EtykrzB5cMA/odJLptjnmfzsPeGS5F1MxuTqrH2Z09emZcqjfXW+fNNzioiZdD4BQe+rqOA1Ktpw645szJVJJenIMzM4P8dDiW+9cJyddGdTC9A7APgLlLVdG/tuoYcjiY1F068SWx8sUWW/xc7YenCzGj2nXIeHjUuWsjqjMorpBKe6Y1QSGfi7HsLq1DpJDR1xQZbPgM/FOaovVIPIozgAIBpS1ukdNNwadmrCw55tUKfr4s2SNv3SUmeprL37QEtIxtBpPDjL1z+qTLjc6JzRr7J57guPBdNbj6Ukg/uxW+z1CwDYb3uAIIa5e6eDxfxGQsUEj8NmkIHBLLGTnKb1WAyPXAIf9ZvZHUwQQzHvOWXGZRd32enRh00uvuwHEy3OuGEqTAOIs/31q5WAamZ1uqzW7KBTMU/nq4QA9W2yfM= - - secure: Lvj1MT/I2zaJp9mQi46us64O4JfeT9cz1m+nqKY/nuC828ULfqMYLlpw08jvrc25JhAaqHNawAwdtaxHTQqHztsmq1ixHFYA6G/1oll+YPhym29toFT9oIkXThf/L77FTJfM1gVFMyiHFbz11Ob8R15tRB1WF1+Uu1RBmtudJD7HSy0u6Uc6lPxpqtycWVRCyPdvZqF3e0KIZaTDRkRzJpcVMHa/5ZJDDcMtuJyjJYXZqS1WR3QHC1z44LlnqyB5ZM2PU4H3LyWnY8wHF2mutF0QtDDVdEuBqILBHiFuKHMxpLY92UgHm2n51RR63MxFjjEzE+iu5f9ComEm5JC0N/cc8sunIiol+d8SRC30/00Vs0tvmeAjRX9IMCExiP3mv7Tz6mqEVk6PyrVlg675hxRg0eVdaNNv92gWzCSIecZ5TuCaRG4JaWO0P8lQC4NreONt6gbnwMD350hMQZLpUUE0QoSEdafpudaD+agl4ZzTFVTOOcSOz2Sa/+RT96Msazq2YlddYXaEKZeyYzqHVkk60PiQzQcAuwCMFrAagqMe3bNI2aCFEWbc9CoR13K/wwRDwAeSzBq/UkylZ8AayJLnIpewr/iYBOQasrXLrorW969Rfr8d/nwhDN5VpRgItU0arDzngaJRoZEAtHx4zaCvZF/H/nvbORcD2gsFP9Q= - - secure: WGmgvSnT1OeHPYtsm/JzcRDnZYp+SeelgbSVNbtwOXGImkeqDo/sDIRH80HxVC3zkUu5wvRO/NkMA7N3LXN2BqENacWBFQ+j+Hf3lnDw9yETBaBpLQPO/MElKGd48GoSiukCipfaITUfwtjAQ3wx0W/z06dYC9BdUYvUM73OtsGtlrY1MEQnBfff/neDGZbzfvTdgDwg1n6d69WP+WSUJt5Aysz0ZoAbYO2rzBrKNwcL7KEhoI/ketXXQ4xEW0nnxU/qfxqhVfJbck1HpSz+HMsVGXF4tT3zdRXmU+P47KGbfYn0GmlviGrRSWCu9/elKTlX9fIqRnR1/UvdTkflwLsk4i3hWnEUflKpIT7soJrKjNQseyd06KI8qvVRXRcll92vCtsYDuOO1AlemxIqp2dIFduCf+5FWH06PbS8oHU8NBe5aVYaoBY8nK14EB3A98saJ8Un7ziB8moRaWZX9bMjfJWfxDF526NdHIusqLQrxvUthtBjf5aRULE2PeIyqYVY8/IKmBEfsy9IUENmlM5KItvgZZt4xdk8o//kceLrKabudHGVC1M1RLZo9ejkdLySz2i8nlJyJ03Hn30H9DfDjW/OMQTn/b1HU4CYSbd/yGR+6+5+tdPeKy/Z6gI6+EjHRYaaQrBLo0RXm2620cb7uDTBIsKVs2TetyhTq54= - - secure: lqboLQKFpfa6F+UkSCs9+NPv027EnSWZA4/cmAXj0hZvxc4W+qMFIlPKytcq+bbZrp7kx3cZgRd5NrgmOTqlJ9merVK1cgMsDegMv/YImHjhGVBH+3ATvqMDb7I7dLg4x/DjhKt/ogyZpoh8vsS77T9MQXaIJM8JMM3ISxYI6MMloCh0z/hbT2qv3IrPMFjpHohHdF1yorxKvSefNgPx6Y4wC+t1t+u8wS9RBhvrmOkVynlU2NgbSiMPyz3syJmnszHDTXXJlqciGT5Lc8lzpzt2OGOnZmAupEYVGok7Jx4aAxU9O8/bnoDwOcGGVrd/pGHTI5+rvT4TPy+WG1Aec55dgva4XZfrJQ88snHLvq0VrvdiI/fHJEPF18QnrBwKSni1Vd1jOzsuPIF6HL0Wwv8tklO8OX3D5wqVsXpeJd4Cj4HPPAtzxOVsV6Fz3On9vuLdV7StC/ZafQL40koaytqkNO0gX8zeUiaaVQtMCf/2MVTbX9x5m51gkGpwT0JBvqQpSMOlIM5S1fiN4X9DLBLqmsARYeZw1Jeiq7Zm39Sy1QeCgfUXS8+6t62BjWx70iftYGIkoljXtD7x/3pjKymdpwjcyUcS8KQ5W6vbMXFxmZe2phtgGTCxOIDuASPspD8zwm/ckIGArHj4qkrg94/mUpgDDIbVsaUpWXjYvSc= - - secure: bPZMNH79Lx0Wb2SCxTwyZek28w/keKxEAlfad8RDMwJrct6Bi2B0o4KjkrwFS5DyCUU7Ndk192145XnUKOWX2YVs6cQ8ge+5LvtgYhWLgX8g5Ycro+JyzOBskn+o1gQjvi8+3b42X31efcUmTEhfRdKVrrUpONEIcjG1NpLGk/mQJ6AM6hGWO2xdNfAezeWq5ISnpK0b6VUWZyTEDg3NivrTfCL2juWWCnjYm1BWSHUblXwRQ/Rl7Tcldl6cMMUVsalUQ1iG0h8YZDrxNz0cm3XTZZZJKuSHYeTCLd57RBeHD5/iMxjCmfzfq6ETNLONWLmtvA8yhWMQZ0DPFtLZzbqFIfOR6P1feBZFqP7/X5KZeFKBufN15JbcXIqHE8homLY9mS0LhyNffOs5G/P/x8ChE1DJaYiZIWCZ60umvpqibScZB3z5uFPTxLk9rJSOtT7hCWjcmg3EdJ+R4ExOiBDd62ZS5jH72WU4uysPXORRofUCL+zHycJoJxsFWQSW49GAGrohllrW45jnTgpalErxvjJFSKy8JW9w634eetz9ct2fObep7m8bfVMl8U2H3ITIoXHm+f8ooHUiNzHFLSl4wtcoAOtrCAvGtJPNfv8T2eTkznzj7Tk+XzTvFg03u+J99TmnC58Bs516Oc7E40NarmsZceOD2sN4BA3X9tE= From d1074bc3c24dba41cf46c8bfd6edd14f77ad6f63 Mon Sep 17 00:00:00 2001 From: johndmulhausen Date: Thu, 7 Jul 2016 02:18:44 -0700 Subject: [PATCH 173/348] Fixing any remaining 1.2 hardcodes --- _data/guides.yml | 4 +- _data/overrides.yml | 8 +-- _data/reference.yml | 10 ++-- _data/samples.yml | 54 +++++++++---------- .../extensions-v1beta1-definitions.html | 0 .../extensions-v1beta1-operations.html | 0 _includes/{v1.2 => v1.3}/v1-definitions.html | 0 _includes/{v1.2 => v1.3}/v1-operations.html | 0 .../extensions/v1beta1/definitions.md | 2 +- .../extensions/v1beta1/operations.md | 2 +- docs/api-reference/v1/definitions.md | 2 +- docs/api-reference/v1/operations.md | 2 +- 12 files changed, 42 insertions(+), 42 deletions(-) rename _includes/{v1.2 => v1.3}/extensions-v1beta1-definitions.html (100%) rename _includes/{v1.2 => v1.3}/extensions-v1beta1-operations.html (100%) rename _includes/{v1.2 => v1.3}/v1-definitions.html (100%) rename _includes/{v1.2 => v1.3}/v1-operations.html (100%) diff --git a/_data/guides.yml b/_data/guides.yml index 8e956b4e39..732a09fb98 100644 --- a/_data/guides.yml +++ b/_data/guides.yml @@ -133,7 +133,7 @@ toc: - title: Connect with Port Forwarding path: /docs/user-guide/connecting-to-applications-port-forward/ - title: Using Explorer to Examine the Runtime Environment - path: https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/explorer + path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/explorer - title: Creating a Cluster section: @@ -273,7 +273,7 @@ toc: - title: Using DNS Pods and Services path: /docs/admin/dns/ - title: Setting Up and Configuring DNS - path: https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/cluster-dns + path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/cluster-dns - title: Master <-> Node Communication path: /docs/admin/master-node-communication/ - title: Network Plugins diff --git a/_data/overrides.yml b/_data/overrides.yml index 7fd399130c..e4ca98e613 100644 --- a/_data/overrides.yml +++ b/_data/overrides.yml @@ -6,10 +6,10 @@ overrides: - path: docs/admin/kube-proxy.md - path: docs/admin/kube-scheduler.md - path: docs/admin/kubelet.md -- changedpath: docs/api-reference/extensions/v1beta1/definitions.html _includes/v1.2/extensions-v1beta1-definitions.html -- changedpath: docs/api-reference/extensions/v1beta1/operations.html _includes/v1.2/extensions-v1beta1-operations.html -- changedpath: docs/api-reference/v1/definitions.html _includes/v1.2/v1-definitions.html -- changedpath: docs/api-reference/v1/operations.html _includes/v1.2/v1-operations.html +- changedpath: docs/api-reference/extensions/v1beta1/definitions.html _includes/v1.3/extensions-v1beta1-definitions.html +- changedpath: docs/api-reference/extensions/v1beta1/operations.html _includes/v1.3/extensions-v1beta1-operations.html +- changedpath: docs/api-reference/v1/definitions.html _includes/v1.3/v1-definitions.html +- changedpath: docs/api-reference/v1/operations.html _includes/v1.3/v1-operations.html - copypath: k8s/cluster/saltbase/salt/fluentd-gcp/fluentd-gcp.yaml docs/getting-started-guides/fluentd-gcp.yaml - copypath: k8s/examples/blog-logging/counter-pod.yaml docs/getting-started-guides/counter-pod.yaml - copypath: k8s/examples/blog-logging/counter-pod.yaml docs/user-guide/counter-pod.yaml diff --git a/_data/reference.yml b/_data/reference.yml index a11214565a..939081a971 100644 --- a/_data/reference.yml +++ b/_data/reference.yml @@ -232,14 +232,14 @@ toc: - title: Kubernetes Design Docs section: - title: Kubernetes Architecture - path: https://github.com/kubernetes/kubernetes/blob/release-1.2/docs/design/architecture.md + path: https://github.com/kubernetes/kubernetes/blob/release-1.3/docs/design/architecture.md - title: Kubernetes Design Overview - path: https://github.com/kubernetes/kubernetes/blob/release-1.2/docs/design/ + path: https://github.com/kubernetes/kubernetes/blob/release-1.3/docs/design/ - title: Security in Kubernetes - path: https://github.com/kubernetes/kubernetes/blob/release-1.2/docs/design/security.md + path: https://github.com/kubernetes/kubernetes/blob/release-1.3/docs/design/security.md - title: Kubernetes Identity and Access Management - path: https://github.com/kubernetes/kubernetes/blob/release-1.2/docs/design/access.md + path: https://github.com/kubernetes/kubernetes/blob/release-1.3/docs/design/access.md - title: Security Contexts - path: https://github.com/kubernetes/kubernetes/blob/release-1.2/docs/design/security_context.md + path: https://github.com/kubernetes/kubernetes/blob/release-1.3/docs/design/security_context.md - title: Kubernetes OpenVSwitch GRE/VxLAN networking path: /docs/admin/ovs-networking/ diff --git a/_data/samples.yml b/_data/samples.yml index 074da2230a..7852666a78 100644 --- a/_data/samples.yml +++ b/_data/samples.yml @@ -7,73 +7,73 @@ toc: - title: Storage / Database / KV section: - title: Apache Cassandra Database - path: https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/cassandra + path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/cassandra - title: Ceph - path: https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/rbd/ + path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/rbd/ - title: CephFS - path: https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/cephfs/ + path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/cephfs/ - title: GlusterFS - path: /https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/glusterfs/ + path: /https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/glusterfs/ - title: Hazelcast - path: https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/hazelcast + path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/hazelcast - title: iSCSI - path: https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/iscsi/ + path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/iscsi/ - title: MySQL Galera - path: https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/mysql-galera + path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/mysql-galera - title: NFS - path: https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/nfs/ + path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/nfs/ - title: Redis - path: https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/redis/ + path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/redis/ - title: RethinkDB - path: https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/rethinkdb/ + path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/rethinkdb/ - title: Vitess - path: https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/vitess/ + path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/vitess/ - title: Big Data section: - title: Apache Spark - path: https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/spark + path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/spark - title: Apache Storm - path: https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/storm + path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/storm - title: Messaging / Queueing section: - title: Celery + RabbitMQ - path: https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/celery-rabbitmq + path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/celery-rabbitmq - title: Hazelcast - path: https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/hazelcast + path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/hazelcast - title: Miscellaneous section: - title: Meteor Applications - path: https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/meteor/ + path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/meteor/ - title: Elasticsearch - path: https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/elasticsearch/ + path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/elasticsearch/ - title: OpenShift Origin - path: https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/openshift-origin/ + path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/openshift-origin/ - title: Selenium - path: https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/selenium/ + path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/selenium/ - title: Monitoring and Logging section: - title: Elasticsearch/Kibana Logging Demonstration - path: https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/logging-demo/ + path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/logging-demo/ - title: NewRelic - path: https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/newrelic + path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/newrelic - title: Multi-tier Applications section: - title: Guestbook - Go Server - path: https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/guestbook-go/ + path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/guestbook-go/ - title: GuestBook - PHP Server - path: https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/guestbook/ + path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/guestbook/ - title: MEAN stack on Google Cloud Platform path: /docs/getting-started-guides/meanstack/ - title: MySQL + Wordpress - path: https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/mysql-wordpress-pd/ + path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/mysql-wordpress-pd/ - title: MySQL + Phabricator Server - path: https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/phabricator/ + path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/phabricator/ - title: Nodejs + Mongo - path: https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/nodesjs-mongodb + path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/nodesjs-mongodb - title: Petstore - path: https://github.com/kubernetes/kubernetes/tree/release-1.2/examples/k8petstore/ + path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/k8petstore/ diff --git a/_includes/v1.2/extensions-v1beta1-definitions.html b/_includes/v1.3/extensions-v1beta1-definitions.html similarity index 100% rename from _includes/v1.2/extensions-v1beta1-definitions.html rename to _includes/v1.3/extensions-v1beta1-definitions.html diff --git a/_includes/v1.2/extensions-v1beta1-operations.html b/_includes/v1.3/extensions-v1beta1-operations.html similarity index 100% rename from _includes/v1.2/extensions-v1beta1-operations.html rename to _includes/v1.3/extensions-v1beta1-operations.html diff --git a/_includes/v1.2/v1-definitions.html b/_includes/v1.3/v1-definitions.html similarity index 100% rename from _includes/v1.2/v1-definitions.html rename to _includes/v1.3/v1-definitions.html diff --git a/_includes/v1.2/v1-operations.html b/_includes/v1.3/v1-operations.html similarity index 100% rename from _includes/v1.2/v1-operations.html rename to _includes/v1.3/v1-operations.html diff --git a/docs/api-reference/extensions/v1beta1/definitions.md b/docs/api-reference/extensions/v1beta1/definitions.md index 624570626a..bd9bd7bf2e 100644 --- a/docs/api-reference/extensions/v1beta1/definitions.md +++ b/docs/api-reference/extensions/v1beta1/definitions.md @@ -1,3 +1,3 @@ --- --- -{% include v1.2/extensions-v1beta1-definitions.html %} +{% include v1.3/extensions-v1beta1-definitions.html %} diff --git a/docs/api-reference/extensions/v1beta1/operations.md b/docs/api-reference/extensions/v1beta1/operations.md index 19debfc6da..6425157a58 100644 --- a/docs/api-reference/extensions/v1beta1/operations.md +++ b/docs/api-reference/extensions/v1beta1/operations.md @@ -1,3 +1,3 @@ --- --- -{% include v1.2/extensions-v1beta1-operations.html %} +{% include v1.3/extensions-v1beta1-operations.html %} diff --git a/docs/api-reference/v1/definitions.md b/docs/api-reference/v1/definitions.md index de7a131dcb..05dfc063cf 100644 --- a/docs/api-reference/v1/definitions.md +++ b/docs/api-reference/v1/definitions.md @@ -1,3 +1,3 @@ --- --- -{% include v1.2/v1-definitions.html %} +{% include v1.3/v1-definitions.html %} diff --git a/docs/api-reference/v1/operations.md b/docs/api-reference/v1/operations.md index 23023f6665..32b4629e25 100644 --- a/docs/api-reference/v1/operations.md +++ b/docs/api-reference/v1/operations.md @@ -1,3 +1,3 @@ --- --- -{% include v1.2/v1-operations.html %} +{% include v1.3/v1-operations.html %} From 2730cadf8d6a4f6f248ffb3cac8aed8f1faae85a Mon Sep 17 00:00:00 2001 From: johndmulhausen Date: Thu, 7 Jul 2016 02:25:26 -0700 Subject: [PATCH 174/348] Update README.md --- README.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/README.md b/README.md index 31824d6f36..d2ad5c1682 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,5 @@ ## Instructions for Contributing to the Docs/Website -**Note: This branch is for pre-release documentation for 1.3 of Kubernetes. Changes to this branch are automatically served as a website at [http://kubernetes-v1-3.github.io/](http://kubernetes-v1-3.github.io/). See the "master" branch to edit the current stable release, which is served at http://kubernetes.io.** - Welcome! We are very pleased you want to contribute to the documentation and/or website for Kubernetes. You can click the "Fork" button in the upper-right area of the screen to create a copy of our site on your GitHub account called a "fork." Make any changes you want in your fork, and when you are ready to send those changes to us, go to the index page for your fork and click "New Pull Request" to let us know about it. From 8f65764acbbd168bf95aca0539715a4fee97c01c Mon Sep 17 00:00:00 2001 From: Minhan Xia Date: Thu, 7 Jul 2016 10:52:18 -0700 Subject: [PATCH 175/348] add usage summary for network plugins --- docs/admin/network-plugins.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/admin/network-plugins.md b/docs/admin/network-plugins.md index 38b28bd153..5b41c0e310 100644 --- a/docs/admin/network-plugins.md +++ b/docs/admin/network-plugins.md @@ -43,3 +43,9 @@ The plugin requires a few things: * Kubelet must be run with the `--network-plugin=kubenet` argument to enable the plugin * Kubelet must also be run with the `--reconcile-cidr` argument to ensure the IP subnet assigned to the node by configuration or the controller-manager is propagated to the plugin * The node must be assigned an IP subnet through either the `--pod-cidr` kubelet command-line option or the `--allocate-node-cidrs=true --cluster-cidr=` controller-manager command-line options. + +## Usage Summary + +* `--network-plugin=exec` specifies that we use the `exec` plugin, with executables located in `--network-plugin-dir`. +* `--network-plugin=cni` specifies that we use the `cni` network plugin with actual CNI plugin binaries located in `/opt/cni/bin` and CNI plugin configuration located in `network-plugin-dir`, config location defaults to `/etc/cni/net.d`. +* `--network-plugin=kubenet` specifies that we use the `kubenet` network plugin with CNI `bridge` and `host-local` plugins placed in `/opt/cni/bin` or `network-plugin-dir`. \ No newline at end of file From bbd9067586c94019b96bf66a38829dfb9fb71873 Mon Sep 17 00:00:00 2001 From: Alex Robinson Date: Thu, 7 Jul 2016 17:50:47 +0000 Subject: [PATCH 176/348] Remove dead link to elasticsearch logging example and move the normal elasticsearch example into the logging section. --- _data/samples.yml | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/_data/samples.yml b/_data/samples.yml index 7852666a78..356175c632 100644 --- a/_data/samples.yml +++ b/_data/samples.yml @@ -47,8 +47,6 @@ toc: section: - title: Meteor Applications path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/meteor/ - - title: Elasticsearch - path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/elasticsearch/ - title: OpenShift Origin path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/openshift-origin/ - title: Selenium @@ -56,8 +54,8 @@ toc: - title: Monitoring and Logging section: - - title: Elasticsearch/Kibana Logging Demonstration - path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/logging-demo/ + - title: Elasticsearch + path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/elasticsearch/ - title: NewRelic path: https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/newrelic From c787d16479c93364027b1a9368d2012dd0b66a24 Mon Sep 17 00:00:00 2001 From: Ivan Babrou Date: Fri, 8 Jul 2016 20:52:55 +0100 Subject: [PATCH 177/348] Fix quotes in yaml --- docs/user-guide/configuring-containers.md | 2 +- docs/user-guide/rolling-updates.md | 8 ++++---- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/user-guide/configuring-containers.md b/docs/user-guide/configuring-containers.md index 99eb9221bc..551c2ffcef 100644 --- a/docs/user-guide/configuring-containers.md +++ b/docs/user-guide/configuring-containers.md @@ -24,7 +24,7 @@ spec: # specification of the pod's contents containers: - name: hello image: "ubuntu:14.04" - command: ["/bin/echo","hello'?,'?world"] + command: ["/bin/echo", "hello", "world"] ``` The value of `metadata.name`, `hello-world`, will be the name of the pod resource created, and must be unique within the cluster, whereas `containers[0].name` is just a nickname for the container within that pod. `image` is the name of the Docker image, which Kubernetes expects to be able to pull from a registry, the [Docker Hub](https://registry.hub.docker.com/) by default. diff --git a/docs/user-guide/rolling-updates.md b/docs/user-guide/rolling-updates.md index ce3444ebf9..f1db6dc46b 100644 --- a/docs/user-guide/rolling-updates.md +++ b/docs/user-guide/rolling-updates.md @@ -8,8 +8,8 @@ To update a service without an outage, `kubectl` supports what is called ['rolling update'](/docs/user-guide/kubectl/kubectl_rolling-update), which updates one pod at a time, rather than taking down the entire service at the same time. See the [rolling update design document](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/simple-rolling-update.md) and the [example of rolling update](/docs/user-guide/update-demo/) for more information. -Note that `kubectl rolling-update` only supports Replication Controllers. However, if you deploy applications with Replication Controllers, -consider switching them to [Deployments](/docs/user-guide/deployments/). A Deployments is a higher-level controller that automates rolling updates +Note that `kubectl rolling-update` only supports Replication Controllers. However, if you deploy applications with Replication Controllers, +consider switching them to [Deployments](/docs/user-guide/deployments/). A Deployments is a higher-level controller that automates rolling updates of applications declaratively, and therefore is recommended. If you still want to keep your Replication Controllers and use `kubectl rolling-update`, keep reading: A rolling update applies changes to the configuration of pods being managed by @@ -73,7 +73,7 @@ The update will fail if `IMAGE:TAG` is identical to the current value. For this reason, we recommend the use of versioned tags as opposed to values such as `:latest`. Doing a rolling update from `image:latest` to a new `image:latest` will fail, even if the image at that tag has changed. -Moreover, the use of `:latest` is not recommended, see +Moreover, the use of `:latest` is not recommended, see [Best Practices for Configuration](/docs/user-guide/config-best-practices/#container-images) for more information. ### Examples @@ -215,7 +215,7 @@ spec: containers: - name: nginx image: nginx:1.9.2 - args: [“nginx”,”-T”] + args: ["nginx", "-T"] ports: - containerPort: 80 ``` From dfb8da7e54cacde99610643d86f549cd97663f6c Mon Sep 17 00:00:00 2001 From: Alena Prokharchyk Date: Fri, 8 Jul 2016 14:15:48 -0700 Subject: [PATCH 178/348] Broken link fix on dns doc page --- docs/admin/dns.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/admin/dns.md b/docs/admin/dns.md index c56fc5b89b..9bf60e8bba 100644 --- a/docs/admin/dns.md +++ b/docs/admin/dns.md @@ -41,5 +41,5 @@ details on Cluster Federation and multi-site support. ## References -- [Docs for the DNS cluster addon](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/dns/README.md) +- [Docs for the DNS cluster addon](http://releases.k8s.io/{{page.githubbranch}}/build/kube-dns/README.md) From 5c0f338c3e876275ff3c61280b4e1cf98c8a2e45 Mon Sep 17 00:00:00 2001 From: Ryan Eschinger Date: Sat, 9 Jul 2016 07:44:09 -0400 Subject: [PATCH 179/348] fix LoadBalancer spelling in services-firewalls.md --- docs/user-guide/services-firewalls.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/services-firewalls.md b/docs/user-guide/services-firewalls.md index 805acdde96..05f81f29df 100644 --- a/docs/user-guide/services-firewalls.md +++ b/docs/user-guide/services-firewalls.md @@ -6,7 +6,7 @@ exposure to the internet. When exposing a service to the external world, you ma one or more ports in these firewalls to serve traffic. This document describes this process, as well as any provider specific details that may be necessary. -### Restrict Access For LoadBlancer Service +### Restrict Access For LoadBalancer Service When using a Service with `spec.type: LoadBalancer`, you can specify the IP ranges that are allowed to access the load balancer by using `spec.loadBalancerSourceRanges`. This field takes a list of IP CIDR ranges, which Kubernetes will use to configure firewall exceptions. From f9fed7fe497b4e88b41b07cef905a1f23d75af13 Mon Sep 17 00:00:00 2001 From: Eric Tune Date: Mon, 2 May 2016 13:14:05 -0700 Subject: [PATCH 180/348] Remove reference to long-gon readonly port --- docs/admin/accessing-the-api.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/docs/admin/accessing-the-api.md b/docs/admin/accessing-the-api.md index 27ee393ac3..d836cd63d8 100644 --- a/docs/admin/accessing-the-api.md +++ b/docs/admin/accessing-the-api.md @@ -24,6 +24,7 @@ By default the Kubernetes APIserver serves HTTP on 2 ports: - defaults IP is localhost, change with `--insecure-bind-address` flag. - no authentication or authorization checks in HTTP - protected by need to have host access + 2. `Secure Port`: - default is port 6443, change with `--secure-port` flag. @@ -31,9 +32,6 @@ By default the Kubernetes APIserver serves HTTP on 2 ports: - serves HTTPS. Set cert with `--tls-cert-file` and key with `--tls-private-key-file` flag. - uses token-file or client-certificate based [authentication](/docs/admin/authentication). - uses policy-based [authorization](/docs/admin/authorization). - 3. Removed: `ReadOnly Port` - - - For security reasons, this had to be removed. Use the [service account](/docs/user-guide/service-accounts) feature instead. ## Proxies and Firewall rules From 554e5e0def4a5212e237ffe16fb3dfc361acb03a Mon Sep 17 00:00:00 2001 From: Eric Tune Date: Mon, 2 May 2016 13:28:20 -0700 Subject: [PATCH 181/348] Remove reference to old nginx proxy, and port 6443 Port 6443 is still a default, but not commonly used byt setup scripts. --- docs/admin/accessing-the-api.md | 21 ++++++--------------- 1 file changed, 6 insertions(+), 15 deletions(-) diff --git a/docs/admin/accessing-the-api.md b/docs/admin/accessing-the-api.md index d836cd63d8..b759558eeb 100644 --- a/docs/admin/accessing-the-api.md +++ b/docs/admin/accessing-the-api.md @@ -33,27 +33,18 @@ By default the Kubernetes APIserver serves HTTP on 2 ports: - uses token-file or client-certificate based [authentication](/docs/admin/authentication). - uses policy-based [authorization](/docs/admin/authorization). -## Proxies and Firewall rules - -Additionally, in some configurations there is a proxy (nginx) running -on the same machine as the apiserver process. The proxy serves HTTPS protected -by Basic Auth on port 443, and proxies to the apiserver on localhost:8080. In -these configurations the secure port is typically set to 6443. - -A firewall rule is typically configured to allow external HTTPS access to port -443. - -The above are defaults and reflect how Kubernetes is deployed to Google Compute -Engine using `kube-up.sh.` Other cloud providers may vary. +When the cluster is created by `kube-up.sh`, on Google Compute Engine (GCE), +and on several other cloud providers, the API server serves on port 443. On +GCE, a firewall rule is configured on the project to allow external HTTPS +access to the API. Other cluster setup methods vary. ## Use Cases vs IP:Ports There are differently configured serving ports to serve a variety of uses cases: 1. Clients outside of a Kubernetes cluster, such as human running `kubectl` -on a desktop machine. Currently, accesses the Localhost Port via a proxy (nginx) -running on the `kubernetes-master` machine. The proxy can use cert-based -authentication or token-based authentication. +on a desktop machine. These access the Kubernetes cluster in however is specified +in `$USER/.kube/config`. On GCE, this is via port 443 on the host of the apiserver. 2. Processes running in Containers on Kubernetes that need to read from the apiserver. Currently, these can use a [service account](/docs/user-guide/service-accounts). 3. Scheduler and Controller-manager processes, which need to do read-write From 20fed78fc7fcd7ecd4ec5cf45762b132424e8904 Mon Sep 17 00:00:00 2001 From: Eric Tune Date: Mon, 2 May 2016 14:09:37 -0700 Subject: [PATCH 182/348] Remove not useful text from accessing-the-api --- docs/admin/accessing-the-api.md | 21 --------------------- 1 file changed, 21 deletions(-) diff --git a/docs/admin/accessing-the-api.md b/docs/admin/accessing-the-api.md index b759558eeb..6630e20d40 100644 --- a/docs/admin/accessing-the-api.md +++ b/docs/admin/accessing-the-api.md @@ -38,24 +38,3 @@ and on several other cloud providers, the API server serves on port 443. On GCE, a firewall rule is configured on the project to allow external HTTPS access to the API. Other cluster setup methods vary. -## Use Cases vs IP:Ports - -There are differently configured serving ports to serve a variety of uses cases: - - 1. Clients outside of a Kubernetes cluster, such as human running `kubectl` -on a desktop machine. These access the Kubernetes cluster in however is specified -in `$USER/.kube/config`. On GCE, this is via port 443 on the host of the apiserver. - 2. Processes running in Containers on Kubernetes that need to read from -the apiserver. Currently, these can use a [service account](/docs/user-guide/service-accounts). - 3. Scheduler and Controller-manager processes, which need to do read-write -API operations, using service accounts to avoid the need to be co-located. - 4. Kubelets, which need to do read-write API operations and are necessarily -on different machines than the apiserver. Kubelet uses the Secure Port -to get their pods, to find the services that a pod can see, and to -write events. Credentials are distributed to kubelets at cluster -setup time. Kubelet and kube-proxy can use cert-based authentication or -token-based authentication. - -## Expected changes - - - Policy will limit the actions kubelets can do via the authed port. From a9f8a8eb56b5c7dc14e5c7b3fd84f779a59bed27 Mon Sep 17 00:00:00 2001 From: Eric Tune Date: Mon, 2 May 2016 17:24:28 -0700 Subject: [PATCH 183/348] authentication, authorization, admission overview Added diagram too. --- docs/admin/accessing-the-api.md | 142 ++++++++++++++++-- docs/admin/authorization.md | 12 +- images/docs/admin/access-control-overview.svg | 4 + 3 files changed, 137 insertions(+), 21 deletions(-) create mode 100644 images/docs/admin/access-control-overview.svg diff --git a/docs/admin/accessing-the-api.md b/docs/admin/accessing-the-api.md index 6630e20d40..d18d30bfd1 100644 --- a/docs/admin/accessing-the-api.md +++ b/docs/admin/accessing-the-api.md @@ -1,37 +1,149 @@ --- --- -This document describes what ports the Kubernetes apiserver -may serve on and how to reach them. The audience is -cluster administrators who want to customize their cluster -or understand the details. +This document describes how access to the Kubernetes API is controlled. -Most questions about accessing the cluster are covered -in [Accessing the cluster](/docs/user-guide/accessing-the-cluster). +## Overview + +Users [access the API](/docs/user-guide/accessing-the-cluster) using `kubectl`, +client libraries, or by making REST requests. Both human users and +[Kubernetes service accounts](/docs/user-guide/service-accounts/) can be +authorized for API access. +When a request reaches the API, it goes through several stages, illustrated in the +following diagram: + +![Diagram of request handling steps for Kubernetes API request](/images/docs/admin/access-control-overview.svg) + +## Transport Security + +In a typical Kubernetes cluster, the API served on port 443. A TLS connection is +established. The API server presents a certificate. This certificate is +often self-signed, so `$USER/.kube/config` on the user's machine typically +contains the root certficate for the API server's certificate, which when specified +is used in place of the system default root certificates. This certificate is typically +automatically written into your `$USER/.kube/config` when you create a cluster yourself +using `kube-up.sh`. If the cluster has multiple users, then the creator needs to share +the certificate with other users. + +## Authentication + +Once TLS is established, the HTTP request moves to the Authentication step. +This is shown as step **1** in the diagram. +The cluster creation script or cluster admin configures the API server to run +one or more Authenticator Modules. +Authenticators are described in more detail [here](/docs/admin/authentication/). + +The input to the authentication step is the entire HTTP request, however, it typically +just examines the headers and/or or client certificate. + +Authentication modules include Client Certificates, Password, and Plain Tokens, +and JWT Tokens (used for service accounts). + +Multiple authentication modules can be specified, in which case each one is tried in sequence, +until one of them succeeds. + +On GCE, Client Certificates, Password, Plain Tokens, and JWT Tokens are all enabled. + +If the request cannot be authenticated, it is rejected with HTTP status code 401. +Otherwise, the user is authenticated as a specific `username`, and the user name +is available to subsequent steps to use in their decisions. Some authenticators +may also provide the group memberships of the user, while other authenticators +do not (and expect the authorizer to determine these). + +While Kubernetes uses "usernames" for access control decisions and in request logging, +it does not have a `user` object nor does it store usernames or other information about +users in its object store. + +## Authorization + +Once the request is authenticated as coming from a specific user, +it moves to a generic authorization step. This is shown as step **2** in the +diagram. + +The input to the Authorization step are attributes of the REST request, including: + - the username determined by the Authentication step. + - a `verb` associated with the API request. Most object support these common operations: `list, watch, create, update, patch, delete`. Some objects have "special verbs"; for example pods and services can be `proxy`-ed. + - any subresource associated with the API request (e.g. `status`). + - the Group, Version, and Kind of the API resource (e.g. `v1 pod`, or `batch/v1 job`) being + operated on. + - the name and namespace of the object. + +There are multiple supported Authorization Modules. The cluster creator configures the API +server with which Authorization Modules should be used. When multiple Authorization Modules +are configured, each is checked in sequence, and if any Module authorizes the request, +then the request can proceed. If all deny the request, then the request is denied (HTTP status +code 403). + +The [Authorization Modules](/docs/admin/authorization) page describes what authorization modules +are available and how to configure them. + +For version 1.2, clusters created by `kube-up.sh` are configured so that no authorization is +required for any request. + +As of version 1.3, clusters created by `kube-up.sh` are configured so that the ABAC authorization +modules is enabled. However, its input file is initially set to allow all users to do all +operations. The cluster administrator needs to edit that file, or configure a different authorizer +to restrict what users can do. -## Ports and IPs Served On +The Authorization step is designed to operate on attributes that are likely to be common to most +REST requests, such as object name, kind, etc. This is intended to facilitate interation with +existing organization-wide or cloud-provider-wide access control systems (which may handle +other APIs besides the Kubernetes API. -The Kubernetes API is served by the Kubernetes apiserver process. Typically, -there is one of these running on a single kubernetes-master node. +Access controls and policies that depend on specific fields of specific Kinds of objects +are handled by Admission Controllers. + +## Admission Control + +Admission Control Modules are software modules that can modify or reject requests. +In addition to the attributes available to Authorization Modules, Admission +Control Modules can access the contents of the object that is being created or updated. +They act on objects being created, deleted, updated or connected (proxy), but not reads. + +Multiple admission controllers can be configured. Each is called in order. + +This is shown as step **3** in the diagram. + +Unlike Authentication and Authorization Modules, if any admission controller module +rejects, then the request is immediately rejected. + +In addition to rejecting objects, admission controllers can also set complex defaults for +fields. + +The available Admission Control Modules are described [here](docs/admin/admission-controllers/). + +Once a request passes all admission controllers, it is validated using the validation routines +for the corresponding API object, and then written to the object store (shown as step **4**). + + +## API Server Ports and IPs + +The previous discussion applies to requests sent to the secure port of the API server +(the typical case). The API server can actually serve on 2 ports: By default the Kubernetes APIserver serves HTTP on 2 ports: 1. `Localhost Port`: - - serves HTTP + - is intended for testing and bootstrap, and for other components of the master node + (scheduler, controller-manager) to talk to the API + - no TLS - default is port 8080, change with `--insecure-port` flag. - defaults IP is localhost, change with `--insecure-bind-address` flag. - - no authentication or authorization checks in HTTP + - request **bypasses** authentication and authorization modules. + - request handled by admission control module(s). - protected by need to have host access 2. `Secure Port`: - + + - use whenever possible + - uses TLS. Set cert with `--tls-cert-file` and key with `--tls-private-key-file` flag. - default is port 6443, change with `--secure-port` flag. - default IP is first non-localhost network interface, change with `--bind-address` flag. - - serves HTTPS. Set cert with `--tls-cert-file` and key with `--tls-private-key-file` flag. - - uses token-file or client-certificate based [authentication](/docs/admin/authentication). - - uses policy-based [authorization](/docs/admin/authorization). + - request handled by authentication and authorization modules. + - request handled by admission control module(s). + - authentication and authoriation modules run. When the cluster is created by `kube-up.sh`, on Google Compute Engine (GCE), and on several other cloud providers, the API server serves on port 443. On diff --git a/docs/admin/authorization.md b/docs/admin/authorization.md index efd93fb497..c56088e8d1 100644 --- a/docs/admin/authorization.md +++ b/docs/admin/authorization.md @@ -2,8 +2,8 @@ --- In Kubernetes, authorization happens as a separate step from authentication. -See the [authentication documentation](/docs/admin/authentication) for an -overview of authentication. +See the [Accessing Control Overview](/docs/admin/accessing-the-api/) for an +overview of how authentication and authorization are applied to requests. Authorization applies to all HTTP accesses on the main (secure) apiserver port. @@ -16,9 +16,9 @@ The following implementations are available, and are selected by flag: - `--authorization-mode=AlwaysDeny` blocks all requests (used in tests). - `--authorization-mode=AlwaysAllow` allows all requests; use if you don't need authorization. - - `--authorization-mode=ABAC`allows for user-configured authorization policy. -ABAC stands for - Attribute-Based Access Control. + - `--authorization-mode=ABAC` allows for a simple local-file-based user-configured +authorization policy. ABAC stands for Attribute-Based Access Control. +authorization policy. - `--authorization-mode=RBAC` is an experimental implementation which allows for authorization to be driven by the Kubernetes API. RBAC stands for Roles-Based Access Control. @@ -444,7 +444,7 @@ restricting access to the REST api. For further documentation refer to the authorization.v1beta1 API objects and plugin/pkg/auth/authorizer/webhook/webhook.go. -## Plugin Development +## Module Development Other implementations can be developed fairly easily. The APIserver calls the Authorizer interface: diff --git a/images/docs/admin/access-control-overview.svg b/images/docs/admin/access-control-overview.svg new file mode 100644 index 0000000000..684fd24f44 --- /dev/null +++ b/images/docs/admin/access-control-overview.svg @@ -0,0 +1,4 @@ + + + + From 389c77b6ac7df37b0b2dbf2ecd79398ca20007bd Mon Sep 17 00:00:00 2001 From: Eric Tune Date: Fri, 8 Jul 2016 10:24:10 -0700 Subject: [PATCH 184/348] Describe CPU and Memory resources. --- docs/user-guide/compute-resources.md | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/docs/user-guide/compute-resources.md b/docs/user-guide/compute-resources.md index bd8fd278f2..71fa9ca56e 100644 --- a/docs/user-guide/compute-resources.md +++ b/docs/user-guide/compute-resources.md @@ -40,6 +40,32 @@ about pod resource requests/limits. A *pod resource request/limit* for a partic type is the sum of the resource requests/limits of that type for each container in the pod, with unset values treated as zero (or equal to default values in some cluster configurations). +### Meaning of CPU +Limits and requests for `cpu` are measured in cpus. +One cpu, in Kubernetes, is equivalent to: + +- 1 AWS vCPU +- 1 GCP Core +- 1 Azure vCore +- 1 *Hyperthread* on a bare-metal Intel processor with Hyperthreading + +Fractional requests are allowed. A container with `spec.container[].resources.requests.cpu` of `0.5` will +be guaranteed half as much CPU as one that asks for `1`. The expression `0.1` is equivalent to the expression +`100m`, which can be read as "one hundred millicpu" (some may say "one hundred millicores", and this is understood +to mean the same thing when talking about Kubernetes). A request with a decimal point, like `0.1` is converted to +`100m` by the API, and precision finer than `1m` is not allowed. For this reason, the form `100m` may be preferred. + +CPU is always requested as an absolute quantity, never as a relative quantity; 0.1 is the same amount of cpu on a single +core, dual core, or 48 core machine. + +# Meaning of Memory + +Limits and requests for `memory` are measured in bytes. +Memory can be expressed a plain integer or as fixed-point integers with one of these SI suffixes (E, P, T, G, M, K) +or their power-of-two equivalents (Ei, Pi, Ti, Gi, Mi, Ki). For example, the following represent roughly the same value: +`128974848`, `129e6`, `129M` , `123Mi`. + +### Example The following pod has two containers. Each has a request of 0.25 core of cpu and 64MiB (220 bytes) of memory and a limit of 0.5 core of cpu and 128MiB of memory. The pod can be said to have a request of 0.5 core and 128 MiB of memory and a limit of 1 core and 256MiB of From bda17741b3454550a1ba8f0cb3d33b5527b1f354 Mon Sep 17 00:00:00 2001 From: Eric Tune Date: Sun, 10 Jul 2016 08:09:46 -0700 Subject: [PATCH 185/348] Mention runtime-config flag for RBAC. --- docs/admin/authorization.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/admin/authorization.md b/docs/admin/authorization.md index efd93fb497..2643702dfc 100644 --- a/docs/admin/authorization.md +++ b/docs/admin/authorization.md @@ -172,6 +172,11 @@ Kubernetes API. As of 1.3 RBAC mode is in alpha and considered experimental. +To use RBAC, you must both enable the authorization module with `--authorization-mode=RBAC`, +and [enable the API version]( +docs/admin/cluster-management.md/#Turn-on-or-off-an-api-version-for-your-cluster), +with a `--runtime-config=` that includes `rbac.authorization/v1alpha1`. + ### Roles, RolesBindings, ClusterRoles, and ClusterRoleBindings The RBAC API Group declares four top level types which will be covered in this From 10cd664b3e123886dac2ca69b33b287c296fe97d Mon Sep 17 00:00:00 2001 From: keglevich3 Date: Sun, 10 Jul 2016 19:23:05 +0300 Subject: [PATCH 186/348] typo --- docs/getting-started-guides/coreos/bare_metal_offline.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting-started-guides/coreos/bare_metal_offline.md b/docs/getting-started-guides/coreos/bare_metal_offline.md index f243caf81d..9f2bec9320 100644 --- a/docs/getting-started-guides/coreos/bare_metal_offline.md +++ b/docs/getting-started-guides/coreos/bare_metal_offline.md @@ -1,7 +1,7 @@ --- --- -Deploy a CoreOS running Kubernetes environment. This particular guild is made to help those in an OFFLINE system, wither for testing a POC before the real deal, or you are restricted to be totally offline for your applications. +Deploy a CoreOS running Kubernetes environment. This particular guide is made to help those in an OFFLINE system, wither for testing a POC before the real deal, or you are restricted to be totally offline for your applications. * TOC {:toc} From 272814cdb2f917702b877f5f89b5dcb58d801061 Mon Sep 17 00:00:00 2001 From: Justin Santa Barbara Date: Sun, 10 Jul 2016 17:36:55 -0400 Subject: [PATCH 187/348] Two minor fixes to multizone doc --- docs/admin/multiple-zones.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/admin/multiple-zones.md b/docs/admin/multiple-zones.md index e420bda304..3dd135b046 100644 --- a/docs/admin/multiple-zones.md +++ b/docs/admin/multiple-zones.md @@ -117,7 +117,7 @@ kubernetes-minion-a12q Ready 6m beta.kubernetes.io Let's add another set of nodes to the existing cluster, reusing the existing master, running in a different zone (us-central1-b or us-west-2b). -We run kube-up again, but by specifying `KUBE_USE_EXISTING_MASTER=1` +We run kube-up again, but by specifying `KUBE_USE_EXISTING_MASTER=true` kube-up will not create a new master, but will reuse one that was previously created instead. @@ -183,7 +183,7 @@ EOF The PV is also labeled with the zone & region it was created in. For version 1.2, dynamic persistent volumes are always created in the zone -of the cluster master (here us-centaral1-a / us-west-2a); this will +of the cluster master (here us-central1-a / us-west-2a); this will be improved in a future version (issue [#23330](https://github.com/kubernetes/kubernetes/issues/23330).) ```shell From aa43a90b0d6ac4fc13b99071002a700515ecded8 Mon Sep 17 00:00:00 2001 From: Adam Kaplan Date: Mon, 11 Jul 2016 14:11:28 -0400 Subject: [PATCH 188/348] Clarify AWS ECR Instructions Adding additional notes on using Kubernetes with AWS EC2 Container Registry. Like GKE, Kubernetes admins in AWS should not use `imagePullSecrets` to store ECR credentials. The kubelets should manage that on your behalf, since Amazon rotates ECR credentials on a regular basis. --- docs/user-guide/images.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/docs/user-guide/images.md b/docs/user-guide/images.md index 4bf4748355..49f80a6500 100644 --- a/docs/user-guide/images.md +++ b/docs/user-guide/images.md @@ -32,6 +32,9 @@ Credentials can be provided in several ways: - Per-cluster - automatically configured on Google Compute Engine or Google Container Engine - all pods can read the project's private registry + - Using AWS EC2 Container Registry (ECR) + - use IAM roles and policies to control access to ECR repositories + - automatically refreshes ECR login credentials - Configuring Nodes to Authenticate to a Private Registry - all pods can read any configured private registries - requires node configuration by cluster administrator @@ -60,7 +63,7 @@ so it can pull from the project's GCR, but not push. ### Using AWS EC2 Container Registry Kubernetes has native support for the [AWS EC2 Container -Registry](https://aws.amazon.com/ecr/), when nodes are AWS instances. +Registry](https://aws.amazon.com/ecr/), when nodes are AWS EC2 instances. Simply use the full image name (e.g. `ACCOUNT.dkr.ecr.REGION.amazonaws.com/imagename:tag`) in the Pod definition. @@ -83,6 +86,9 @@ The kubelet will fetch and periodically refresh ECR credentials. It needs the f **Note:** if you are running on Google Container Engine (GKE), there will already be a `.dockercfg` on each node with credentials for Google Container Registry. You cannot use this approach. +**Note:** if you are running on AWS EC2 and are using the EC2 Container Registry (ECR), the kubelet on each node will +manage and update the ECR login credentials. You cannot use this approach. + **Note:** this approach is suitable if you can control node configuration. It will not work reliably on GCE, and any other cloud provider that does automatic node replacement. From 3a151822bce532fd56b3e1052646f592f3077108 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=81ukasz=20Ole=C5=9B?= Date: Fri, 8 Jul 2016 23:23:04 +0200 Subject: [PATCH 189/348] Fix commands in Connect with Port Forwarding guide fixes: * added -f flag to command create * changed not existent flag -t to --template --- docs/user-guide/connecting-to-applications-port-forward.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/user-guide/connecting-to-applications-port-forward.md b/docs/user-guide/connecting-to-applications-port-forward.md index 9c331f5918..a02790b117 100644 --- a/docs/user-guide/connecting-to-applications-port-forward.md +++ b/docs/user-guide/connecting-to-applications-port-forward.md @@ -6,7 +6,7 @@ kubectl port-forward forwards connections to a local port to a port on a pod. It ## Creating a Redis master ```shell -$ kubectl create examples/redis/redis-master.yaml +$ kubectl create -f examples/redis/redis-master.yaml pods/redis-master ``` @@ -23,7 +23,7 @@ redis-master 2/2 Running 0 41s The Redis master is listening on port 6379, to verify this, ```shell{% raw %} -$ kubectl get pods redis-master -t='{{(index (index .spec.containers 0).ports 0).containerPort}}{{"\n"}}' +$ kubectl get pods redis-master --template='{{(index (index .spec.containers 0).ports 0).containerPort}}{{"\n"}}' 6379{% endraw %} ``` From b1ccad5099126cb8d2f45d0a5dc7bb50d00cdbae Mon Sep 17 00:00:00 2001 From: Adam J Date: Tue, 12 Jul 2016 19:13:07 +0200 Subject: [PATCH 190/348] Fix for Cassandra example link I've fixed invalid link to Cassandra example. --- docs/samples.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/samples.md b/docs/samples.md index d608ab7cbd..62c85f0932 100644 --- a/docs/samples.md +++ b/docs/samples.md @@ -83,10 +83,10 @@ h2, h3, h4 {
Guestbook + Redis - +
Cloud Native Cassandra
WordPress + MySQL -
\ No newline at end of file +