From dee98660da95b368516f512b5c1013a422a950f1 Mon Sep 17 00:00:00 2001 From: Quentin Bourgeois Date: Sat, 24 Oct 2020 10:06:14 +0000 Subject: [PATCH 001/190] Fix links MD syntax typo in FR documentation --- content/fr/docs/setup/learning-environment/minikube.md | 6 ++---- content/fr/docs/tasks/tools/install-minikube.md | 2 +- 2 files changed, 3 insertions(+), 5 deletions(-) diff --git a/content/fr/docs/setup/learning-environment/minikube.md b/content/fr/docs/setup/learning-environment/minikube.md index 77be61831f..2ab0b3ae5a 100644 --- a/content/fr/docs/setup/learning-environment/minikube.md +++ b/content/fr/docs/setup/learning-environment/minikube.md @@ -48,10 +48,10 @@ Suivez les étapes ci-dessous pour commencer et explorer Minikube. Starting local Kubernetes cluster... ``` - Pour plus d'informations sur le démarrage de votre cluster avec une version spécifique de Kubernetes, une machine virtuelle ou un environnement de conteneur, voir [Démarrage d'un cluster].(#starting-a-cluster). + Pour plus d'informations sur le démarrage de votre cluster avec une version spécifique de Kubernetes, une machine virtuelle ou un environnement de conteneur, voir [Démarrage d'un cluster](#starting-a-cluster). 2. Vous pouvez maintenant interagir avec votre cluster à l'aide de kubectl. - Pour plus d'informations, voir [Interagir avec votre cluster.](#interacting-with-your-cluster). + Pour plus d'informations, voir [Interagir avec votre cluster](#interacting-with-your-cluster). Créons un déploiement Kubernetes en utilisant une image existante nommée `echoserver`, qui est un serveur HTTP, et exposez-la sur le port 8080 à l’aide de `--port`. @@ -529,5 +529,3 @@ Les contributions, questions et commentaires sont les bienvenus et sont encourag Les développeurs de minikube sont dans le canal #minikube du [Slack](https://kubernetes.slack.com) de Kubernetes (recevoir une invitation [ici](http://slack.kubernetes.io/)). Nous avons également la liste de diffusion [kubernetes-dev Google Groupes](https://groups.google.com/forum/#!forum/kubernetes-dev). Si vous publiez sur la liste, veuillez préfixer votre sujet avec "minikube:". - - diff --git a/content/fr/docs/tasks/tools/install-minikube.md b/content/fr/docs/tasks/tools/install-minikube.md index b079eba6ec..745f1e4c22 100644 --- a/content/fr/docs/tasks/tools/install-minikube.md +++ b/content/fr/docs/tasks/tools/install-minikube.md @@ -84,7 +84,7 @@ Vous pouvez télécharger les packages `.deb` depuis [Docker](https://www.docker {{< caution >}} Le pilote VM `none` peut entraîner des problèmes de sécurité et de perte de données. -Avant d'utiliser `--driver=none`, consultez [cette documentation] (https://minikube.sigs.k8s.io/docs/reference/drivers/none/) pour plus d'informations. +Avant d'utiliser `--driver=none`, consultez [cette documentation](https://minikube.sigs.k8s.io/docs/reference/drivers/none/) pour plus d'informations. {{}} Minikube prend également en charge un `vm-driver=podman` similaire au pilote Docker. Podman est exécuté en tant que superutilisateur (utilisateur root), c'est le meilleur moyen de garantir que vos conteneurs ont un accès complet à toutes les fonctionnalités disponibles sur votre système. From 8df84e27081523d29865df6255be87b22cfe1118 Mon Sep 17 00:00:00 2001 From: Quentin Bourgeois Date: Sat, 24 Oct 2020 13:41:56 +0000 Subject: [PATCH 002/190] Fix links MD syntax typo in FR documentation --- content/fr/docs/setup/learning-environment/minikube.md | 6 ++---- content/fr/docs/tasks/tools/install-minikube.md | 2 +- 2 files changed, 3 insertions(+), 5 deletions(-) diff --git a/content/fr/docs/setup/learning-environment/minikube.md b/content/fr/docs/setup/learning-environment/minikube.md index 77be61831f..2ab0b3ae5a 100644 --- a/content/fr/docs/setup/learning-environment/minikube.md +++ b/content/fr/docs/setup/learning-environment/minikube.md @@ -48,10 +48,10 @@ Suivez les étapes ci-dessous pour commencer et explorer Minikube. Starting local Kubernetes cluster... ``` - Pour plus d'informations sur le démarrage de votre cluster avec une version spécifique de Kubernetes, une machine virtuelle ou un environnement de conteneur, voir [Démarrage d'un cluster].(#starting-a-cluster). + Pour plus d'informations sur le démarrage de votre cluster avec une version spécifique de Kubernetes, une machine virtuelle ou un environnement de conteneur, voir [Démarrage d'un cluster](#starting-a-cluster). 2. Vous pouvez maintenant interagir avec votre cluster à l'aide de kubectl. - Pour plus d'informations, voir [Interagir avec votre cluster.](#interacting-with-your-cluster). + Pour plus d'informations, voir [Interagir avec votre cluster](#interacting-with-your-cluster). Créons un déploiement Kubernetes en utilisant une image existante nommée `echoserver`, qui est un serveur HTTP, et exposez-la sur le port 8080 à l’aide de `--port`. @@ -529,5 +529,3 @@ Les contributions, questions et commentaires sont les bienvenus et sont encourag Les développeurs de minikube sont dans le canal #minikube du [Slack](https://kubernetes.slack.com) de Kubernetes (recevoir une invitation [ici](http://slack.kubernetes.io/)). Nous avons également la liste de diffusion [kubernetes-dev Google Groupes](https://groups.google.com/forum/#!forum/kubernetes-dev). Si vous publiez sur la liste, veuillez préfixer votre sujet avec "minikube:". - - diff --git a/content/fr/docs/tasks/tools/install-minikube.md b/content/fr/docs/tasks/tools/install-minikube.md index b079eba6ec..745f1e4c22 100644 --- a/content/fr/docs/tasks/tools/install-minikube.md +++ b/content/fr/docs/tasks/tools/install-minikube.md @@ -84,7 +84,7 @@ Vous pouvez télécharger les packages `.deb` depuis [Docker](https://www.docker {{< caution >}} Le pilote VM `none` peut entraîner des problèmes de sécurité et de perte de données. -Avant d'utiliser `--driver=none`, consultez [cette documentation] (https://minikube.sigs.k8s.io/docs/reference/drivers/none/) pour plus d'informations. +Avant d'utiliser `--driver=none`, consultez [cette documentation](https://minikube.sigs.k8s.io/docs/reference/drivers/none/) pour plus d'informations. {{}} Minikube prend également en charge un `vm-driver=podman` similaire au pilote Docker. Podman est exécuté en tant que superutilisateur (utilisateur root), c'est le meilleur moyen de garantir que vos conteneurs ont un accès complet à toutes les fonctionnalités disponibles sur votre système. From 01262ab3c24d9d7f0932dd7eab390d94c9c7f1d8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Collignon-Ducret=20R=C3=A9mi?= Date: Mon, 16 Nov 2020 15:49:00 +0100 Subject: [PATCH 003/190] Fix formatting issue MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit https://kubernetes.io/fr/docs/concepts/workloads/controllers/deployment/#mise-%C3%A0-jour-d-un-d%C3%A9ploiement Obtenez les détails de votre déploiement -> La sortie est similaire à ceci --- content/fr/docs/concepts/workloads/controllers/deployment.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/fr/docs/concepts/workloads/controllers/deployment.md b/content/fr/docs/concepts/workloads/controllers/deployment.md index e12c3ffbab..212d23227a 100644 --- a/content/fr/docs/concepts/workloads/controllers/deployment.md +++ b/content/fr/docs/concepts/workloads/controllers/deployment.md @@ -285,7 +285,7 @@ Obtenez plus de détails sur votre déploiement mis à jour: La sortie est similaire à ceci: - ```text + ```text Name: nginx-deployment Namespace: default CreationTimestamp: Thu, 30 Nov 2017 10:56:25 +0000 From 845a8855318d3b483fec8c87041cd5fec300d154 Mon Sep 17 00:00:00 2001 From: "salva.gglez" Date: Sun, 6 Dec 2020 17:02:01 +0000 Subject: [PATCH 004/190] Add a caution message on CronJob Controller workload if using startingDeadlineSeconds with a value less than 10s --- content/en/docs/concepts/workloads/controllers/cron-jobs.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/content/en/docs/concepts/workloads/controllers/cron-jobs.md b/content/en/docs/concepts/workloads/controllers/cron-jobs.md index af12bcba25..cdc736aaa5 100644 --- a/content/en/docs/concepts/workloads/controllers/cron-jobs.md +++ b/content/en/docs/concepts/workloads/controllers/cron-jobs.md @@ -61,6 +61,11 @@ If `startingDeadlineSeconds` is set to a large value or left unset (the default) and if `concurrencyPolicy` is set to `Allow`, the jobs will always run at least once. +{{< caution >}} +If `startingDeadlineSeconds` is set to a value less than 10 seconds, then the CronJob may not be scheduled. This is because Cronjob Controller checks things every 10 seconds by default. +{{< /caution >}} + + For every CronJob, the CronJob {{< glossary_tooltip term_id="controller" >}} checks how many schedules it missed in the duration from its last scheduled time until now. If there are more than 100 missed schedules, then it does not start the job and logs the error ```` From d38c74124a89f1978e5d5f74989f439ca0cc8196 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Salvador=20Gonz=C3=A1lez=20Gonz=C3=A1lez?= Date: Sun, 6 Dec 2020 20:08:30 +0000 Subject: [PATCH 005/190] Update content/en/docs/concepts/workloads/controllers/cron-jobs.md Co-authored-by: bl-ue <54780737+bl-ue@users.noreply.github.com> --- content/en/docs/concepts/workloads/controllers/cron-jobs.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/content/en/docs/concepts/workloads/controllers/cron-jobs.md b/content/en/docs/concepts/workloads/controllers/cron-jobs.md index cdc736aaa5..11e30b203d 100644 --- a/content/en/docs/concepts/workloads/controllers/cron-jobs.md +++ b/content/en/docs/concepts/workloads/controllers/cron-jobs.md @@ -62,7 +62,7 @@ and if `concurrencyPolicy` is set to `Allow`, the jobs will always run at least once. {{< caution >}} -If `startingDeadlineSeconds` is set to a value less than 10 seconds, then the CronJob may not be scheduled. This is because Cronjob Controller checks things every 10 seconds by default. +If `startingDeadlineSeconds` is set to a value less than 10 seconds, the CronJob may not be scheduled. This is because the CronJob controller checks things every 10 seconds. {{< /caution >}} @@ -96,4 +96,3 @@ documents the format of CronJob `schedule` fields. For instructions on creating and working with cron jobs, and for an example of CronJob manifest, see [Running automated tasks with cron jobs](/docs/tasks/job/automated-tasks-with-cron-jobs). - From 18b8846b9be9a1f78f432b1867ddf4792143ce44 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Salvador=20Gonz=C3=A1lez=20Gonz=C3=A1lez?= Date: Tue, 8 Dec 2020 11:00:18 +0000 Subject: [PATCH 006/190] Update cron-jobs.md Mention about v2 CronJob controller. --- content/en/docs/concepts/workloads/controllers/cron-jobs.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/concepts/workloads/controllers/cron-jobs.md b/content/en/docs/concepts/workloads/controllers/cron-jobs.md index 11e30b203d..a7e1f03097 100644 --- a/content/en/docs/concepts/workloads/controllers/cron-jobs.md +++ b/content/en/docs/concepts/workloads/controllers/cron-jobs.md @@ -62,7 +62,7 @@ and if `concurrencyPolicy` is set to `Allow`, the jobs will always run at least once. {{< caution >}} -If `startingDeadlineSeconds` is set to a value less than 10 seconds, the CronJob may not be scheduled. This is because the CronJob controller checks things every 10 seconds. +If `startingDeadlineSeconds` is set to a value less than 10 seconds, the CronJob may not be scheduled. This is because the CronJob controller checks things every 10 seconds. As an alpha feature, there is a V2 controller implementation with different behavior. The version 2 controller provides experimental performance improvements. {{< /caution >}} From 06ad25e38d8977c45aa6bbedfb123c071195ceb0 Mon Sep 17 00:00:00 2001 From: Karen Bradshaw Date: Wed, 28 Oct 2020 14:30:33 -0400 Subject: [PATCH 007/190] remove stackdriver, elastic task pages clean up links and redirects --- .../cluster-administration/logging.md | 132 +++++++++--------- .../events-stackdriver.md | 94 ------------- .../logging-elasticsearch-kibana.md | 126 ----------------- .../stateful-application/zookeeper.md | 95 ++++++------- static/_redirects | 2 - 5 files changed, 111 insertions(+), 338 deletions(-) delete mode 100644 content/en/docs/tasks/debug-application-cluster/events-stackdriver.md delete mode 100644 content/en/docs/tasks/debug-application-cluster/logging-elasticsearch-kibana.md diff --git a/content/en/docs/concepts/cluster-administration/logging.md b/content/en/docs/concepts/cluster-administration/logging.md index 80c610f963..d8c0a0615c 100644 --- a/content/en/docs/concepts/cluster-administration/logging.md +++ b/content/en/docs/concepts/cluster-administration/logging.md @@ -9,23 +9,22 @@ weight: 60 -Application logs can help you understand what is happening inside your application. The logs are particularly useful for debugging problems and monitoring cluster activity. Most modern applications have some kind of logging mechanism; as such, most container engines are likewise designed to support some kind of logging. The easiest and most embraced logging method for containerized applications is to write to the standard output and standard error streams. +Application logs can help you understand what is happening inside your application. The logs are particularly useful for debugging problems and monitoring cluster activity. Most modern applications have some kind of logging mechanism. Likewise, container engines are designed to support logging. The easiest and most adopted logging method for containerized applications is writing to standard output and standard error streams. -However, the native functionality provided by a container engine or runtime is usually not enough for a complete logging solution. For example, if a container crashes, a pod is evicted, or a node dies, you'll usually still want to access your application's logs. As such, logs should have a separate storage and lifecycle independent of nodes, pods, or containers. This concept is called _cluster-level-logging_. Cluster-level logging requires a separate backend to store, analyze, and query logs. Kubernetes provides no native storage solution for log data, but you can integrate many existing logging solutions into your Kubernetes cluster. +However, the native functionality provided by a container engine or runtime is usually not enough for a complete logging solution. +For example, you may want access your application's logs if a container crashes; a pod gets evicted; or a node dies. +In a cluster, logs should have a separate storage and lifecycle independent of nodes, pods, or containers. This concept is called _cluster-level logging_. -Cluster-level logging architectures are described in assumption that -a logging backend is present inside or outside of your cluster. If you're -not interested in having cluster-level logging, you might still find -the description of how logs are stored and handled on the node to be useful. +Cluster-level logging architectures require a separate backend to store, analyze, and query logs. Kubernetes +does not provide a native storage solution for log data. Instead, there are many logging solutions that +integrate with Kubernetes. The following sections describe how to handle and store logs on nodes. ## Basic logging in Kubernetes -In this section, you can see an example of basic logging in Kubernetes that -outputs data to the standard output stream. This demonstration uses -a pod specification with a container that writes some text to standard output -once per second. +This example uses a `Pod` specification with a container +to write text to the standard output stream once per second. {{< codenew file="debug/counter-pod.yaml" >}} @@ -34,8 +33,10 @@ To run this pod, use the following command: ```shell kubectl apply -f https://k8s.io/examples/debug/counter-pod.yaml ``` + The output is: -``` + +```console pod/counter created ``` @@ -44,73 +45,73 @@ To fetch the logs, use the `kubectl logs` command, as follows: ```shell kubectl logs counter ``` + The output is: -``` + +```console 0: Mon Jan 1 00:00:00 UTC 2001 1: Mon Jan 1 00:00:01 UTC 2001 2: Mon Jan 1 00:00:02 UTC 2001 ... ``` -You can use `kubectl logs` to retrieve logs from a previous instantiation of a container with `--previous` flag, in case the container has crashed. If your pod has multiple containers, you should specify which container's logs you want to access by appending a container name to the command. See the [`kubectl logs` documentation](/docs/reference/generated/kubectl/kubectl-commands#logs) for more details. +You can use `kubectl logs --previous` to retrieve logs from a previous instantiation of a container. If your pod has multiple containers, specify which container's logs you want to access by appending a container name to the command. See the [`kubectl logs` documentation](/docs/reference/generated/kubectl/kubectl-commands#logs) for more details. ## Logging at the node level ![Node level logging](/images/docs/user-guide/logging/logging-node-level.png) -Everything a containerized application writes to `stdout` and `stderr` is handled and redirected somewhere by a container engine. For example, the Docker container engine redirects those two streams to [a logging driver](https://docs.docker.com/engine/admin/logging/overview), which is configured in Kubernetes to write to a file in json format. +A container engine handles and redirects any output generated to a containerized application's `stdout` and `stderr` streams. +For example, the Docker container engine redirects those two streams to [a logging driver](https://docs.docker.com/engine/admin/logging/overview), which is configured in Kubernetes to write to a file in JSON format. {{< note >}} -The Docker json logging driver treats each line as a separate message. When using the Docker logging driver, there is no direct support for multi-line messages. You need to handle multi-line messages at the logging agent level or higher. +The Docker JSON logging driver treats each line as a separate message. When using the Docker logging driver, there is no direct support for multi-line messages. You need to handle multi-line messages at the logging agent level or higher. {{< /note >}} By default, if a container restarts, the kubelet keeps one terminated container with its logs. If a pod is evicted from the node, all corresponding containers are also evicted, along with their logs. An important consideration in node-level logging is implementing log rotation, so that logs don't consume all available storage on the node. Kubernetes -currently is not responsible for rotating logs, but rather a deployment tool +is not responsible for rotating logs, but rather a deployment tool should set up a solution to address that. For example, in Kubernetes clusters, deployed by the `kube-up.sh` script, there is a [`logrotate`](https://linux.die.net/man/8/logrotate) tool configured to run each hour. You can also set up a container runtime to -rotate application's logs automatically, for example by using Docker's `log-opt`. -In the `kube-up.sh` script, the latter approach is used for COS image on GCP, -and the former approach is used in any other environment. In both cases, by -default rotation is configured to take place when log file exceeds 10MB. +rotate an application's logs automatically. As an example, you can find detailed information about how `kube-up.sh` sets up logging for COS image on GCP in the corresponding -[script](https://github.com/kubernetes/kubernetes/blob/{{< param "githubbranch" >}}/cluster/gce/gci/configure-helper.sh). +[`configure-helper` script](https://github.com/kubernetes/kubernetes/blob/{{< param "githubbranch" >}}/cluster/gce/gci/configure-helper.sh). When you run [`kubectl logs`](/docs/reference/generated/kubectl/kubectl-commands#logs) as in the basic logging example, the kubelet on the node handles the request and -reads directly from the log file, returning the contents in the response. +reads directly from the log file. The kubelet returns the content of the log file. {{< note >}} -Currently, if some external system has performed the rotation, +If an external system has performed the rotation, only the contents of the latest log file will be available through -`kubectl logs`. E.g. if there's a 10MB file, `logrotate` performs -the rotation and there are two files, one 10MB in size and one empty, -`kubectl logs` will return an empty response. +`kubectl logs`. For example, if there's a 10MB file, `logrotate` performs +the rotation and there are two files: one file that is 10MB in size and a second file that is empty. +`kubectl logs` returns the latest log file which in this example is an empty response. {{< /note >}} -[cosConfigureHelper]: https://github.com/kubernetes/kubernetes/blob/{{< param "githubbranch" >}}/cluster/gce/gci/configure-helper.sh ### System component logs There are two types of system components: those that run in a container and those that do not run in a container. For example: * The Kubernetes scheduler and kube-proxy run in a container. -* The kubelet and container runtime, for example Docker, do not run in containers. +* The kubelet and container runtime do not run in containers. On machines with systemd, the kubelet and container runtime write to journald. If -systemd is not present, they write to `.log` files in the `/var/log` directory. -System components inside containers always write to the `/var/log` directory, -bypassing the default logging mechanism. They use the [klog](https://github.com/kubernetes/klog) +systemd is not present, the kubelet and container runtime write to `.log` files +in the `/var/log` directory. System components inside containers always write +to the `/var/log` directory, bypassing the default logging mechanism. +They use the [`klog`](https://github.com/kubernetes/klog) logging library. You can find the conventions for logging severity for those components in the [development docs on logging](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-instrumentation/logging.md). -Similarly to the container logs, system component logs in the `/var/log` +Similar to the container logs, system component logs in the `/var/log` directory should be rotated. In Kubernetes clusters brought up by the `kube-up.sh` script, those logs are configured to be rotated by the `logrotate` tool daily or once the size exceeds 100MB. @@ -129,13 +130,14 @@ While Kubernetes does not provide a native solution for cluster-level logging, t You can implement cluster-level logging by including a _node-level logging agent_ on each node. The logging agent is a dedicated tool that exposes logs or pushes logs to a backend. Commonly, the logging agent is a container that has access to a directory with log files from all of the application containers on that node. -Because the logging agent must run on every node, it's common to implement it as either a DaemonSet replica, a manifest pod, or a dedicated native process on the node. However the latter two approaches are deprecated and highly discouraged. +Because the logging agent must run on every node, it is recommended to run the agent +as a `DaemonSet`. -Using a node-level logging agent is the most common and encouraged approach for a Kubernetes cluster, because it creates only one agent per node, and it doesn't require any changes to the applications running on the node. However, node-level logging _only works for applications' standard output and standard error_. +Node-level logging creates only one agent per node and doesn't require any changes to the applications running on the node. -Kubernetes doesn't specify a logging agent, but two optional logging agents are packaged with the Kubernetes release: [Stackdriver Logging](/docs/tasks/debug-application-cluster/logging-stackdriver/) for use with Google Cloud Platform, and [Elasticsearch](/docs/tasks/debug-application-cluster/logging-elasticsearch-kibana/). You can find more information and instructions in the dedicated documents. Both use [fluentd](https://www.fluentd.org/) with custom configuration as an agent on the node. +Containers write stdout and stderr, but with no agreed format. A node-level agent collects these logs and forwards them for aggregation. -### Using a sidecar container with the logging agent +### Using a sidecar container with the logging agent {#sidecar-container-with-logging-agent} You can use a sidecar container in one of the following ways: @@ -146,28 +148,27 @@ You can use a sidecar container in one of the following ways: ![Sidecar container with a streaming container](/images/docs/user-guide/logging/logging-with-streaming-sidecar.png) -By having your sidecar containers stream to their own `stdout` and `stderr` +By having your sidecar containers write to their own `stdout` and `stderr` streams, you can take advantage of the kubelet and the logging agent that already run on each node. The sidecar containers read logs from a file, a socket, -or the journald. Each individual sidecar container prints log to its own `stdout` -or `stderr` stream. +or journald. Each sidecar container prints a log to its own `stdout` or `stderr` stream. This approach allows you to separate several log streams from different parts of your application, some of which can lack support for writing to `stdout` or `stderr`. The logic behind redirecting logs -is minimal, so it's hardly a significant overhead. Additionally, because +is minimal, so it's not a significant overhead. Additionally, because `stdout` and `stderr` are handled by the kubelet, you can use built-in tools like `kubectl logs`. -Consider the following example. A pod runs a single container, and the container -writes to two different log files, using two different formats. Here's a +For example, a pod runs a single container, and the container +writes to two different log files using two different formats. Here's a configuration file for the Pod: {{< codenew file="admin/logging/two-files-counter-pod.yaml" >}} -It would be a mess to have log entries of different formats in the same log +It is not recommended to write log entries with different formats to the same log stream, even if you managed to redirect both components to the `stdout` stream of -the container. Instead, you could introduce two sidecar containers. Each sidecar +the container. Instead, you can create two sidecar containers. Each sidecar container could tail a particular log file from a shared volume and then redirect the logs to its own `stdout` stream. @@ -181,7 +182,10 @@ running the following commands: ```shell kubectl logs counter count-log-1 ``` -``` + +The output is: + +```console 0: Mon Jan 1 00:00:00 UTC 2001 1: Mon Jan 1 00:00:01 UTC 2001 2: Mon Jan 1 00:00:02 UTC 2001 @@ -191,7 +195,10 @@ kubectl logs counter count-log-1 ```shell kubectl logs counter count-log-2 ``` -``` + +The output is: + +```console Mon Jan 1 00:00:00 UTC 2001 INFO 0 Mon Jan 1 00:00:01 UTC 2001 INFO 1 Mon Jan 1 00:00:02 UTC 2001 INFO 2 @@ -202,16 +209,15 @@ The node-level agent installed in your cluster picks up those log streams automatically without any further configuration. If you like, you can configure the agent to parse log lines depending on the source container. -Note, that despite low CPU and memory usage (order of couple of millicores +Note, that despite low CPU and memory usage (order of a couple of millicores for cpu and order of several megabytes for memory), writing logs to a file and then streaming them to `stdout` can double disk usage. If you have -an application that writes to a single file, it's generally better to set -`/dev/stdout` as destination rather than implementing the streaming sidecar +an application that writes to a single file, it's recommended to set +`/dev/stdout` as the destination rather than implement the streaming sidecar container approach. Sidecar containers can also be used to rotate log files that cannot be -rotated by the application itself. An example -of this approach is a small container running logrotate periodically. +rotated by the application itself. An example of this approach is a small container running `logrotate` periodically. However, it's recommended to use `stdout` and `stderr` directly and leave rotation and retention policies to the kubelet. @@ -226,21 +232,17 @@ configured specifically to run with your application. {{< note >}} Using a logging agent in a sidecar container can lead to significant resource consumption. Moreover, you won't be able to access -those logs using `kubectl logs` command, because they are not controlled +those logs using `kubectl logs` because they are not controlled by the kubelet. {{< /note >}} -As an example, you could use [Stackdriver](/docs/tasks/debug-application-cluster/logging-stackdriver/), -which uses fluentd as a logging agent. Here are two configuration files that -you can use to implement this approach. The first file contains -a [ConfigMap](/docs/tasks/configure-pod-container/configure-pod-configmap/) to configure fluentd. +Here are two configuration files that you can use to implement a sidecar container with a logging agent. The first file contains +a [`ConfigMap`](/docs/tasks/configure-pod-container/configure-pod-configmap/) to configure fluentd. {{< codenew file="admin/logging/fluentd-sidecar-config.yaml" >}} {{< note >}} -The configuration of fluentd is beyond the scope of this article. For -information about configuring fluentd, see the -[official fluentd documentation](https://docs.fluentd.org/). +For information about configuring fluentd, see the [fluentd documentation](https://docs.fluentd.org/). {{< /note >}} The second file describes a pod that has a sidecar container running fluentd. @@ -248,18 +250,10 @@ The pod mounts a volume where fluentd can pick up its configuration data. {{< codenew file="admin/logging/two-files-counter-pod-agent-sidecar.yaml" >}} -After some time you can find log messages in the Stackdriver interface. - -Remember, that this is just an example and you can actually replace fluentd -with any logging agent, reading from any source inside an application -container. +In the sample configurations, you can replace fluentd with any logging agent, reading from any source inside an application container. ### Exposing logs directly from the application ![Exposing logs directly from the application](/images/docs/user-guide/logging/logging-from-application.png) -You can implement cluster-level logging by exposing or pushing logs directly from -every application; however, the implementation for such a logging mechanism -is outside the scope of Kubernetes. - - +Cluster-logging that exposes or pushes logs directly from every application is outside the scope of Kubernetes. diff --git a/content/en/docs/tasks/debug-application-cluster/events-stackdriver.md b/content/en/docs/tasks/debug-application-cluster/events-stackdriver.md deleted file mode 100644 index 859c163307..0000000000 --- a/content/en/docs/tasks/debug-application-cluster/events-stackdriver.md +++ /dev/null @@ -1,94 +0,0 @@ ---- -reviewers: -- piosz -- x13n -content_type: concept -title: Events in Stackdriver ---- - - - -Kubernetes events are objects that provide insight into what is happening -inside a cluster, such as what decisions were made by scheduler or why some -pods were evicted from the node. You can read more about using events -for debugging your application in the [Application Introspection and Debugging -](/docs/tasks/debug-application-cluster/debug-application-introspection/) -section. - -Since events are API objects, they are stored in the apiserver on master. To -avoid filling up master's disk, a retention policy is enforced: events are -removed one hour after the last occurrence. To provide longer history -and aggregation capabilities, a third party solution should be installed -to capture events. - -This article describes a solution that exports Kubernetes events to -Stackdriver Logging, where they can be processed and analyzed. - -{{< note >}} -It is not guaranteed that all events happening in a cluster will be -exported to Stackdriver. One possible scenario when events will not be -exported is when event exporter is not running (e.g. during restart or -upgrade). In most cases it's fine to use events for purposes like setting up -[metrics](https://cloud.google.com/logging/docs/logs-based-metrics/) and [alerts](https://cloud.google.com/logging/docs/logs-based-metrics/charts-and-alerts), but you should be aware -of the potential inaccuracy. -{{< /note >}} - - - - - - - -## Deployment - -### Google Kubernetes Engine - -In Google Kubernetes Engine, if cloud logging is enabled, event exporter -is deployed by default to the clusters with master running version 1.7 and -higher. To prevent disturbing your workloads, event exporter does not have -resources set and is in the best effort QOS class, which means that it will -be the first to be killed in the case of resource starvation. If you want -your events to be exported, make sure you have enough resources to facilitate -the event exporter pod. This may vary depending on the workload, but on -average, approximately 100Mb RAM and 100m CPU is needed. - -### Deploying to the Existing Cluster - -Deploy event exporter to your cluster using the following command: - -```shell -kubectl apply -f https://k8s.io/examples/debug/event-exporter.yaml -``` - -Since event exporter accesses the Kubernetes API, it requires permissions to -do so. The following deployment is configured to work with RBAC -authorization. It sets up a service account and a cluster role binding -to allow event exporter to read events. To make sure that event exporter -pod will not be evicted from the node, you can additionally set up resource -requests. As mentioned earlier, 100Mb RAM and 100m CPU should be enough. - -{{< codenew file="debug/event-exporter.yaml" >}} - -## User Guide - -Events are exported to the `GKE Cluster` resource in Stackdriver Logging. -You can find them by selecting an appropriate option from a drop-down menu -of available resources: - -Events location in the Stackdriver Logging interface - -You can filter based on the event object fields using Stackdriver Logging -[filtering mechanism](https://cloud.google.com/logging/docs/view/advanced_filters). -For example, the following query will show events from the scheduler -about pods from deployment `nginx-deployment`: - -``` -resource.type="gke_cluster" -jsonPayload.kind="Event" -jsonPayload.source.component="default-scheduler" -jsonPayload.involvedObject.name:"nginx-deployment" -``` - -{{< figure src="/images/docs/stackdriver-event-exporter-filter.png" alt="Filtered events in the Stackdriver Logging interface" width="500" >}} - - diff --git a/content/en/docs/tasks/debug-application-cluster/logging-elasticsearch-kibana.md b/content/en/docs/tasks/debug-application-cluster/logging-elasticsearch-kibana.md deleted file mode 100644 index 17e2ac45a2..0000000000 --- a/content/en/docs/tasks/debug-application-cluster/logging-elasticsearch-kibana.md +++ /dev/null @@ -1,126 +0,0 @@ ---- -reviewers: -- piosz -- x13n -content_type: concept -title: Logging Using Elasticsearch and Kibana ---- - - - -On the Google Compute Engine (GCE) platform, the default logging support targets -[Stackdriver Logging](https://cloud.google.com/logging/), which is described in detail -in the [Logging With Stackdriver Logging](/docs/tasks/debug-application-cluster/logging-stackdriver). - -This article describes how to set up a cluster to ingest logs into -[Elasticsearch](https://www.elastic.co/products/elasticsearch) and view -them using [Kibana](https://www.elastic.co/products/kibana), as an alternative to -Stackdriver Logging when running on GCE. - -{{< note >}} -You cannot automatically deploy Elasticsearch and Kibana in the Kubernetes cluster hosted on Google Kubernetes Engine. You have to deploy them manually. -{{< /note >}} - - - - - -To use Elasticsearch and Kibana for cluster logging, you should set the -following environment variable as shown below when creating your cluster with -kube-up.sh: - -```shell -KUBE_LOGGING_DESTINATION=elasticsearch -``` - -You should also ensure that `KUBE_ENABLE_NODE_LOGGING=true` (which is the default for the GCE platform). - -Now, when you create a cluster, a message will indicate that the Fluentd log -collection daemons that run on each node will target Elasticsearch: - -```shell -cluster/kube-up.sh -``` -``` -... -Project: kubernetes-satnam -Zone: us-central1-b -... calling kube-up -Project: kubernetes-satnam -Zone: us-central1-b -+++ Staging server tars to Google Storage: gs://kubernetes-staging-e6d0e81793/devel -+++ kubernetes-server-linux-amd64.tar.gz uploaded (sha1 = 6987c098277871b6d69623141276924ab687f89d) -+++ kubernetes-salt.tar.gz uploaded (sha1 = bdfc83ed6b60fa9e3bff9004b542cfc643464cd0) -Looking for already existing resources -Starting master and configuring firewalls -Created [https://www.googleapis.com/compute/v1/projects/kubernetes-satnam/zones/us-central1-b/disks/kubernetes-master-pd]. -NAME ZONE SIZE_GB TYPE STATUS -kubernetes-master-pd us-central1-b 20 pd-ssd READY -Created [https://www.googleapis.com/compute/v1/projects/kubernetes-satnam/regions/us-central1/addresses/kubernetes-master-ip]. -+++ Logging using Fluentd to elasticsearch -``` - -The per-node Fluentd pods, the Elasticsearch pods, and the Kibana pods should -all be running in the kube-system namespace soon after the cluster comes to -life. - -```shell -kubectl get pods --namespace=kube-system -``` -``` -NAME READY STATUS RESTARTS AGE -elasticsearch-logging-v1-78nog 1/1 Running 0 2h -elasticsearch-logging-v1-nj2nb 1/1 Running 0 2h -fluentd-elasticsearch-kubernetes-node-5oq0 1/1 Running 0 2h -fluentd-elasticsearch-kubernetes-node-6896 1/1 Running 0 2h -fluentd-elasticsearch-kubernetes-node-l1ds 1/1 Running 0 2h -fluentd-elasticsearch-kubernetes-node-lz9j 1/1 Running 0 2h -kibana-logging-v1-bhpo8 1/1 Running 0 2h -kube-dns-v3-7r1l9 3/3 Running 0 2h -monitoring-heapster-v4-yl332 1/1 Running 1 2h -monitoring-influx-grafana-v1-o79xf 2/2 Running 0 2h -``` - -The `fluentd-elasticsearch` pods gather logs from each node and send them to -the `elasticsearch-logging` pods, which are part of a -[service](/docs/concepts/services-networking/service/) named `elasticsearch-logging`. These -Elasticsearch pods store the logs and expose them via a REST API. -The `kibana-logging` pod provides a web UI for reading the logs stored in -Elasticsearch, and is part of a service named `kibana-logging`. - -The Elasticsearch and Kibana services are both in the `kube-system` namespace -and are not directly exposed via a publicly reachable IP address. To reach them, -follow the instructions for -[Accessing services running in a cluster](/docs/tasks/access-application-cluster/access-cluster/#accessing-services-running-on-the-cluster). - -If you try accessing the `elasticsearch-logging` service in your browser, you'll -see a status page that looks something like this: - -![Elasticsearch Status](/images/docs/es-browser.png) - -You can now type Elasticsearch queries directly into the browser, if you'd -like. See [Elasticsearch's documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-uri-request.html) -for more details on how to do so. - -Alternatively, you can view your cluster's logs using Kibana (again using the -[instructions for accessing a service running in the cluster](/docs/tasks/access-application-cluster/access-cluster/#accessing-services-running-on-the-cluster)). -The first time you visit the Kibana URL you will be presented with a page that -asks you to configure your view of the ingested logs. Select the option for -timeseries values and select `@timestamp`. On the following page select the -`Discover` tab and then you should be able to see the ingested logs. -You can set the refresh interval to 5 seconds to have the logs -regularly refreshed. - -Here is a typical view of ingested logs from the Kibana viewer: - -![Kibana logs](/images/docs/kibana-logs.png) - - - -## {{% heading "whatsnext" %}} - - -Kibana opens up all sorts of powerful options for exploring your logs! For some -ideas on how to dig into it, check out [Kibana's documentation](https://www.elastic.co/guide/en/kibana/current/discover.html). - - diff --git a/content/en/docs/tutorials/stateful-application/zookeeper.md b/content/en/docs/tutorials/stateful-application/zookeeper.md index c044532e5f..6d517ef229 100644 --- a/content/en/docs/tutorials/stateful-application/zookeeper.md +++ b/content/en/docs/tutorials/stateful-application/zookeeper.md @@ -21,39 +21,37 @@ and [PodAntiAffinity](/docs/concepts/scheduling-eviction/assign-pod-node/#affini ## {{% heading "prerequisites" %}} Before starting this tutorial, you should be familiar with the following -Kubernetes concepts. +Kubernetes concepts: -- [Pods](/docs/concepts/workloads/pods/) -- [Cluster DNS](/docs/concepts/services-networking/dns-pod-service/) -- [Headless Services](/docs/concepts/services-networking/service/#headless-services) -- [PersistentVolumes](/docs/concepts/storage/persistent-volumes/) -- [PersistentVolume Provisioning](https://github.com/kubernetes/examples/tree/{{< param "githubbranch" >}}/staging/persistent-volume-provisioning/) -- [StatefulSets](/docs/concepts/workloads/controllers/statefulset/) -- [PodDisruptionBudgets](/docs/concepts/workloads/pods/disruptions/#pod-disruption-budget) -- [PodAntiAffinity](/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity) -- [kubectl CLI](/docs/reference/kubectl/kubectl/) +- [Pods](/docs/concepts/workloads/pods/) +- [Cluster DNS](/docs/concepts/services-networking/dns-pod-service/) +- [Headless Services](/docs/concepts/services-networking/service/#headless-services) +- [PersistentVolumes](/docs/concepts/storage/volumes/) +- [PersistentVolume Provisioning](https://github.com/kubernetes/examples/tree/{{< param "githubbranch" >}}/staging/persistent-volume-provisioning/) +- [StatefulSets](/docs/concepts/workloads/controllers/statefulset/) +- [PodDisruptionBudgets](/docs/concepts/workloads/pods/disruptions/#pod-disruption-budget) +- [PodAntiAffinity](/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity) +- [kubectl CLI](/docs/reference/kubectl/kubectl/) -You will require a cluster with at least four nodes, and each node requires at least 2 CPUs and 4 GiB of memory. In this tutorial you will cordon and drain the cluster's nodes. **This means that the cluster will terminate and evict all Pods on its nodes, and the nodes will temporarily become unschedulable.** You should use a dedicated cluster for this tutorial, or you should ensure that the disruption you cause will not interfere with other tenants. +You must have a cluster with at least four nodes, and each node requires at least 2 CPUs and 4 GiB of memory. In this tutorial you will cordon and drain the cluster's nodes. **This means that the cluster will terminate and evict all Pods on its nodes, and the nodes will temporarily become unschedulable.** You should use a dedicated cluster for this tutorial, or you should ensure that the disruption you cause will not interfere with other tenants. This tutorial assumes that you have configured your cluster to dynamically provision PersistentVolumes. If your cluster is not configured to do so, you will have to manually provision three 20 GiB volumes before starting this tutorial. - ## {{% heading "objectives" %}} After this tutorial, you will know the following. -- How to deploy a ZooKeeper ensemble using StatefulSet. -- How to consistently configure the ensemble. -- How to spread the deployment of ZooKeeper servers in the ensemble. -- How to use PodDisruptionBudgets to ensure service availability during planned maintenance. - +- How to deploy a ZooKeeper ensemble using StatefulSet. +- How to consistently configure the ensemble. +- How to spread the deployment of ZooKeeper servers in the ensemble. +- How to use PodDisruptionBudgets to ensure service availability during planned maintenance. -### ZooKeeper Basics +### ZooKeeper [Apache ZooKeeper](https://zookeeper.apache.org/doc/current/) is a distributed, open-source coordination service for distributed applications. @@ -68,7 +66,7 @@ The ensemble uses the Zab protocol to elect a leader, and the ensemble cannot wr ZooKeeper servers keep their entire state machine in memory, and write every mutation to a durable WAL (Write Ahead Log) on storage media. When a server crashes, it can recover its previous state by replaying the WAL. To prevent the WAL from growing without bound, ZooKeeper servers will periodically snapshot them in memory state to storage media. These snapshots can be loaded directly into memory, and all WAL entries that preceded the snapshot may be discarded. -## Creating a ZooKeeper Ensemble +## Creating a ZooKeeper ensemble The manifest below contains a [Headless Service](/docs/concepts/services-networking/service/#headless-services), @@ -127,7 +125,7 @@ zk-2 1/1 Running 0 40s The StatefulSet controller creates three Pods, and each Pod has a container with a [ZooKeeper](https://www-us.apache.org/dist/zookeeper/stable/) server. -### Facilitating Leader Election +### Facilitating leader election Because there is no terminating algorithm for electing a leader in an anonymous network, Zab requires explicit membership configuration to perform leader election. Each server in the ensemble needs to have a unique identifier, all servers need to know the global set of identifiers, and each identifier needs to be associated with a network address. @@ -211,7 +209,7 @@ server.2=zk-1.zk-hs.default.svc.cluster.local:2888:3888 server.3=zk-2.zk-hs.default.svc.cluster.local:2888:3888 ``` -### Achieving Consensus +### Achieving consensus Consensus protocols require that the identifiers of each participant be unique. No two participants in the Zab protocol should claim the same unique identifier. This is necessary to allow the processes in the system to agree on which processes have committed which data. If two Pods are launched with the same ordinal, two ZooKeeper servers would both identify themselves as the same server. @@ -260,7 +258,7 @@ server.3=zk-2.zk-hs.default.svc.cluster.local:2888:3888 When the servers use the Zab protocol to attempt to commit a value, they will either achieve consensus and commit the value (if leader election has succeeded and at least two of the Pods are Running and Ready), or they will fail to do so (if either of the conditions are not met). No state will arise where one server acknowledges a write on behalf of another. -### Sanity Testing the Ensemble +### Sanity testing the ensemble The most basic sanity test is to write data to one ZooKeeper server and to read the data from another. @@ -270,6 +268,7 @@ The command below executes the `zkCli.sh` script to write `world` to the path `/ ```shell kubectl exec zk-0 zkCli.sh create /hello world ``` + ``` WATCHER:: @@ -304,7 +303,7 @@ dataLength = 5 numChildren = 0 ``` -### Providing Durable Storage +### Providing durable storage As mentioned in the [ZooKeeper Basics](#zookeeper-basics) section, ZooKeeper commits all entries to a durable WAL, and periodically writes snapshots @@ -445,8 +444,8 @@ The `volumeMounts` section of the `StatefulSet`'s container `template` mounts th ```shell volumeMounts: - - name: datadir - mountPath: /var/lib/zookeeper +- name: datadir + mountPath: /var/lib/zookeeper ``` When a Pod in the `zk` `StatefulSet` is (re)scheduled, it will always have the @@ -454,7 +453,7 @@ same `PersistentVolume` mounted to the ZooKeeper server's data directory. Even when the Pods are rescheduled, all the writes made to the ZooKeeper servers' WALs, and all their snapshots, remain durable. -## Ensuring Consistent Configuration +## Ensuring consistent configuration As noted in the [Facilitating Leader Election](#facilitating-leader-election) and [Achieving Consensus](#achieving-consensus) sections, the servers in a @@ -469,6 +468,7 @@ Get the `zk` StatefulSet. ```shell kubectl get sts zk -o yaml ``` + ``` … command: @@ -497,7 +497,7 @@ command: The command used to start the ZooKeeper servers passed the configuration as command line parameter. You can also use environment variables to pass configuration to the ensemble. -### Configuring Logging +### Configuring logging One of the files generated by the `zkGenConfig.sh` script controls ZooKeeper's logging. ZooKeeper uses [Log4j](https://logging.apache.org/log4j/2.x/), and, by default, @@ -558,13 +558,11 @@ You can view application logs written to standard out or standard error using `k 2016-12-06 19:34:46,230 [myid:1] - INFO [Thread-1142:NIOServerCnxn@1008] - Closed socket connection for client /127.0.0.1:52768 (no session established for client) ``` -Kubernetes supports more powerful, but more complex, logging integrations -with [Stackdriver](/docs/tasks/debug-application-cluster/logging-stackdriver/) -and [Elasticsearch and Kibana](/docs/tasks/debug-application-cluster/logging-elasticsearch-kibana/). -For cluster level log shipping and aggregation, consider deploying a [sidecar](https://kubernetes.io/blog/2015/06/the-distributed-system-toolkit-patterns) -container to rotate and ship your logs. +Kubernetes integrates with many logging solutions. You can choose a logging solution +that best fits your cluster and applications. For cluster-level logging and aggregation, +consider deploying a [sidecar container](/docs/concepts/cluster-administration/logging#sidecar-container-with-logging-agent) to rotate and ship your logs. -### Configuring a Non-Privileged User +### Configuring a non-privileged user The best practices to allow an application to run as a privileged user inside of a container are a matter of debate. If your organization requires @@ -612,7 +610,7 @@ Because the `fsGroup` field of the `securityContext` object is set to 1000, the drwxr-sr-x 3 zookeeper zookeeper 4096 Dec 5 20:45 /var/lib/zookeeper/data ``` -## Managing the ZooKeeper Process +## Managing the ZooKeeper process The [ZooKeeper documentation](https://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_supervision) mentions that "You will want to have a supervisory process that @@ -622,7 +620,7 @@ common pattern. When deploying an application in Kubernetes, rather than using an external utility as a supervisory process, you should use Kubernetes as the watchdog for your application. -### Updating the Ensemble +### Updating the ensemble The `zk` `StatefulSet` is configured to use the `RollingUpdate` update strategy. @@ -631,6 +629,7 @@ You can use `kubectl patch` to update the number of `cpus` allocated to the serv ```shell kubectl patch sts zk --type='json' -p='[{"op": "replace", "path": "/spec/template/spec/containers/0/resources/requests/cpu", "value":"0.3"}]' ``` + ``` statefulset.apps/zk patched ``` @@ -640,6 +639,7 @@ Use `kubectl rollout status` to watch the status of the update. ```shell kubectl rollout status sts/zk ``` + ``` waiting for statefulset rolling update to complete 0 pods at revision zk-5db4499664... Waiting for 1 pods to be ready... @@ -678,7 +678,7 @@ kubectl rollout undo sts/zk statefulset.apps/zk rolled back ``` -### Handling Process Failure +### Handling process failure [Restart Policies](/docs/concepts/workloads/pods/pod-lifecycle/#restart-policy) control how Kubernetes handles process failures for the entry point of the container in a Pod. @@ -731,7 +731,7 @@ that implements the application's business logic, the script must terminate with child process. This ensures that Kubernetes will restart the application's container when the process implementing the application's business logic fails. -### Testing for Liveness +### Testing for liveness Configuring your application to restart failed processes is not enough to keep a distributed system healthy. There are scenarios where @@ -795,7 +795,7 @@ zk-0 0/1 Running 1 1h zk-0 1/1 Running 1 1h ``` -### Testing for Readiness +### Testing for readiness Readiness is not the same as liveness. If a process is alive, it is scheduled and healthy. If a process is ready, it is able to process input. Liveness is @@ -824,7 +824,7 @@ Even though the liveness and readiness probes are identical, it is important to specify both. This ensures that only healthy servers in the ZooKeeper ensemble receive network traffic. -## Tolerating Node Failure +## Tolerating Node failure ZooKeeper needs a quorum of servers to successfully commit mutations to data. For a three server ensemble, two servers must be healthy for @@ -879,10 +879,10 @@ as `zk` in the domain defined by the `topologyKey`. The `topologyKey` different rules, labels, and selectors, you can extend this technique to spread your ensemble across physical, network, and power failure domains. -## Surviving Maintenance +## Surviving maintenance -**In this section you will cordon and drain nodes. If you are using this tutorial -on a shared cluster, be sure that this will not adversely affect other tenants.** +In this section you will cordon and drain nodes. If you are using this tutorial +on a shared cluster, be sure that this will not adversely affect other tenants. The previous section showed you how to spread your Pods across nodes to survive unplanned node failures, but you also need to plan for temporary node failures @@ -1017,6 +1017,7 @@ Continue to watch the Pods of the stateful set, and drain the node on which ```shell kubectl drain $(kubectl get pod zk-2 --template {{.spec.nodeName}}) --ignore-daemonsets --force --delete-local-data ``` + ``` node "kubernetes-node-i4c4" cordoned @@ -1059,6 +1060,7 @@ Use [`kubectl uncordon`](/docs/reference/generated/kubectl/kubectl-commands/#unc ```shell kubectl uncordon kubernetes-node-pb41 ``` + ``` node "kubernetes-node-pb41" uncordoned ``` @@ -1068,6 +1070,7 @@ node "kubernetes-node-pb41" uncordoned ```shell kubectl get pods -w -l app=zk ``` + ``` NAME READY STATUS RESTARTS AGE zk-0 1/1 Running 2 1h @@ -1130,9 +1133,7 @@ You should always allocate additional capacity for critical services so that the ## {{% heading "cleanup" %}} - - Use `kubectl uncordon` to uncordon all the nodes in your cluster. -- You will need to delete the persistent storage media for the PersistentVolumes - used in this tutorial. Follow the necessary steps, based on your environment, - storage configuration, and provisioning method, to ensure that all storage is - reclaimed. +- You must delete the persistent storage media for the PersistentVolumes used in this tutorial. + Follow the necessary steps, based on your environment, storage configuration, + and provisioning method, to ensure that all storage is reclaimed. diff --git a/static/_redirects b/static/_redirects index 97f6a8b80f..e1c2e8b09f 100644 --- a/static/_redirects +++ b/static/_redirects @@ -373,9 +373,7 @@ /docs/user-guide/liveness/ /docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/ 301 /docs/user-guide/load-balancer/ /docs/tasks/access-application-cluster/create-external-load-balancer/ 301 /docs/user-guide/logging/ /docs/concepts/cluster-administration/logging/ 301 -/docs/user-guide/logging/elasticsearch/ /docs/tasks/debug-application-cluster/logging-elasticsearch-kibana/ 301 /docs/user-guide/logging/overview/ /docs/concepts/cluster-administration/logging/ 301 -/docs/user-guide/logging/stackdriver/ /docs/tasks/debug-application-cluster/logging-stackdriver/ 301 /docs/user-guide/managing-deployments/ /docs/concepts/cluster-administration/manage-deployment/ 301 /docs/user-guide/monitoring/ /docs/tasks/debug-application-cluster/resource-usage-monitoring/ 301 /docs/user-guide/namespaces/ /docs/concepts/overview/working-with-objects/namespaces/ 301 From 6ea9106c05ccfb7195cac48b78c6f55a11ce9116 Mon Sep 17 00:00:00 2001 From: Aris Cahyadi Risdianto Date: Thu, 10 Dec 2020 23:19:57 +0800 Subject: [PATCH 008/190] Add special localization documentation for Bahasa Indonesia. --- content/id/docs/contribute/_index.md | 2 +- content/id/docs/contribute/localization_id.md | 178 ++++++++++++++++++ 2 files changed, 179 insertions(+), 1 deletion(-) create mode 100644 content/id/docs/contribute/localization_id.md diff --git a/content/id/docs/contribute/_index.md b/content/id/docs/contribute/_index.md index d793a78967..0105a29791 100644 --- a/content/id/docs/contribute/_index.md +++ b/content/id/docs/contribute/_index.md @@ -75,5 +75,5 @@ terhadap dokumentasi Kubernetes, tetapi daftar ini dapat membantumu memulainya. - Untuk berkontribusi ke komunitas Kubernetes melalui forum-forum daring seperti Twitter atau Stack Overflow, atau mengetahui tentang pertemuan komunitas (_meetup_) lokal dan acara-acara Kubernetes, kunjungi [situs komunitas Kubernetes](/community/). - Untuk mulai berkontribusi ke pengembangan fitur, baca [_cheatseet_ kontributor](https://github.com/kubernetes/community/tree/master/contributors/guide/contributor-cheatsheet). - +- Untuk kontribusi khusus ke halaman Bahansa Indonesia, baca [Dokumentasi Khusus Untuk Translasi Bahasa Indonesia](/docs/contribute/localization_id.md) diff --git a/content/id/docs/contribute/localization_id.md b/content/id/docs/contribute/localization_id.md new file mode 100644 index 0000000000..5a9c491297 --- /dev/null +++ b/content/id/docs/contribute/localization_id.md @@ -0,0 +1,178 @@ +--- +title: Dokumentasi Khusus Untuk Translasi Bahasa Indonesia +content_type: concept +--- + + + +Panduan khusus untuk bergabung ke komunitas SIG DOC Indonesia dan melakukan +kontribusi untuk mentranslasikan dokumentasi Kubernetes ke dalam Bahasa +Indonesia. + + + +## Manajemen _Milestone_ Tim {#manajemen-milestone-tim} + +Secara umum siklus translasi dokumentasi ke Bahasa Indonesia akan dilakukan +3 kali dalam setahun (sekitar setiap 4 bulan). Untuk menentukan dan mengevaluasi +pencapaian atau _milestone_ dalam kurun waktu tersebut [jadwal rapat daring +reguler tim Bahasa Indonesia](https://zoom.us/j/6072809193) dilakukan secara +konsisten setiap dua minggu sekali. Dalam [agenda rapat ini](https://docs.google.com/document/d/1Qrj-WUAMA11V6KmcfxJsXcPeWwMbFsyBGV4RGbrSRXY) +juga dilakukan pemilihan PR _Wrangler_ untuk dua minggu ke depan. Tugas PR +_Wrangler_ tim Bahasa Indonesia serupa dengan PR _Wrangler_ dari proyek +_upstream_. + +Target pencapaian atau _milestone_ tim akan dirilis sebagai +[_issue tracking_ seperti ini](https://github.com/kubernetes/website/issues/22296) +pada Kubernetes GitHub Website setiap 4 bulan. Dan bersama dengan informasi +PR _Wrangler_ yang dipilih setiap dua minggu, keduanya akan diumumkan di Slack +_channel_ [#kubernetes-docs-id](https://kubernetes.slack.com/archives/CJ1LUCUHM) +dari Komunitas Kubernetes. + +## Cara Memulai Translasi + +Untuk menerjemahkan satu halaman Bahasa Inggris ke Bahasa Indonesia, lakukan +langkah-langkah berikut ini: + +* Check halaman _issue_ di GitHub dan pastikan tidak ada orang lain yang sudah +mengklaim halaman kamu dalam daftar periksa atau komentar-komentar sebelumnya. +* Klaim halaman kamu pada _issue_ di GitHub dengan memberikan komentar di bawah +dengan nama halaman yang ingin kamu terjemahkan dan ambillah hanya satu halaman +dalam satu waktu. +* _Fork_ [repo ini](https://github.com/kubernetes/website), buat terjemahan +kamu, dan kirimkan PR (_pull request_) dengan label `language/id` +* Setelah dikirim, pengulas akan memberikan komentar dalam beberapa hari, dan +tolong untuk menjawab semua komentar. Direkomendasikan juga untuk melakukan +[_squash_](https://github.com/wprig/wprig/wiki/How-to-squash-commits) _commit_ +kamu dengan pesan _commit_ yang baik. + + +## Informasi Acuan Untuk Translasi + +Tidak ada panduan gaya khusus untuk menulis translasi ke bahasa Indonesia. +Namun, secara umum kita dapat mengikuti panduan gaya bahasa Inggris dengan +beberapa tambahan untuk kata-kata impor yang dicetak miring. + +Harap berkomitmen dengan terjemahan kamu dan pada saat kamu mendapatkan komentar +dari pengulas, silahkan atasi sebaik-baiknya. Kami berharap halaman yang +diklaim akan diterjemahkan dalam waktu kurang lebih dua minggu. Jika ternyata +kamu tidak dapat berkomitmen lagi, beri tahu para pengulas agar mereka dapat +meberikan halaman tersebut ke orang lain. + +Beberapa acuan tambahan dalam melakukan translasi silahkan lihat informasi +berikut ini: + +### Daftara Glosarium Translasi dari tim SIG DOC Indonesia +Untuk kata-kata selengkapnya silahkan baca glosariumnya +[disini](#glosarium-indonesia) + +### KBBI +Konsultasikan dengan KBBI (Kamus Besar Bahasa Indonesia) +[disini](https://kbbi.web.id/) dari +[Kemendikbud](https://kbbi.kemdikbud.go.id/). + +### RSNI Glosarium dari Ivan Lanin +[RSNI Glosarium](https://github.com/jk8s/sig-docs-id-localization-how-tos/blob/master/resources/RSNI-glossarium.pdf) +dapat digunakan untuk memahami bagaimana menerjemahkan berbagai istilah teknis +dan khusus Kubernetes. + + +## Panduan Penulisan _Source Code_ + +### Mengikuti kode asli dari dokumentasi bahasa Inggris + +Untuk kenyamanan pemeliharaan, ikuti lebar teks asli dalam kode bahasa Inggris. +Dengan kata lain, jika teks asli ditulis dalam baris yang panjang tanpa putus +atu baris, maka teks tersebut ditulis panjang dalam satu baris meskipun dalam +bahasa Indonesia. Jagalah agar tetap serupa. + +### Hapus nama reviewer di kode asli bahasa Inggris + +Terkadang _reviewer_ ditentukan di bagian atas kode di teks asli Bahasa Inggris. +Secara umum, _reviewer-reviewer_ halaman aslinya akan kesulitan untuk meninjau +halaman dalam bahasa Indonesia, jadi hapus kode yang terkait dengan informasi +_reviewer_ dari metadata kode tersebut. + + +## Panduan Penulisan Kata-kata Translasi + +### Panduan umum + +* Gunakan "kamu" daripada "Anda" sebagai subyek agar lebih bersahabat dengan +para pembaca dokumentasi. +* Tulislah miring untuk kata-kata bahasa Inggris yang diimpor jika kamu tidak +dapat menemukan kata-kata tersebut dalam bahasa Indonesia. +*Benar*: _controller_. *Salah*: controller, `controller` + +### Panduan untuk kata-kata API Objek Kubernetes + +Gunakan gaya "CamelCase" untuk menulis objek API Kubernetes, lihat daftar +lengkapnya [di sini](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/). +Sebagai contoh: + +* *Benar*: PersistentVolume. *Salah*: volume persisten, `PersistentVolume`, +persistentVolume +* *Benar*: Pod. *Salah*: pod, `pod`, "pod" + +*Tips* : Biasanya API objek sudah ditulis dalam huruf kapital pada halaman asli +bahasa Inggris. + +### Panduan untuk kata-kata yang sama dengan API Objek Kubernetes + +Ada beberapa kata-kata yang serupa dengan nama API objek dari Kubernetes dan +dapat mengacu ke arti yang lebih umum (tidak selalu dalam konteks Kubernetes). +Sebagai contoh: _service_, _container_, _node_ , dan lain sebagainya. Kata-kata +sebaiknya ditranslasikan ke Bahasa Indonesia sebagai contoh _service_ menjadi +layanan, _container_ menjadi kontainer. + +*Tips* : Biasanya kata-kata yang mengacu ke arti yang lebih umum sudah *tidak* +ditulis dalam huruf kapital pada halaman asli bahasa Inggris. + +### Panduan untuk "Feature Gate" Kubernetes + +Istilah [_functional gate_](https://kubernetes.io/ko/docs/reference/command-line-tools-reference/feature-gates/) +Kubernetes tidak perlu diterjemahkan ke dalam bahasa Indonesia dan tetap +dipertahankan dalam bentuk aslinya. + +Contoh dari _functional gate_ adalah sebagai berikut: + +- Akselerator +- AdvancedAuditing +- AffinityInAnnotations +- AllowExtTrafficLocalEndpoints +- ... + +### Glosarium Indonesia {#glosarium-indonesia} + +Inggris | Tipe Kata | Indonesia | Sumber | Contoh Kalimat +---|---|---|---|--- +cluster | | klaster | | +container | | kontainer | | +node | kata benda | node | | +file | | berkas | | +service | kata benda | layanan | | +set | | sekumpulan | | +resource | | sumber daya | | +default | | bawaan atau standar (tergantung context) | | Secara bawaan, ...; Pada konfigurasi dan instalasi standar, ... +deploy | | menggelar | | +image | | _image_ | | +request | | permintaan | | +object | kata benda | objek | https://kbbi.web.id/objek | +command | | perintah | https://kbbi.web.id/perintah | +view | | tampilan | | +support | | tersedia atau dukungan (tergantung konteks) | "This feature is supported on version X; Fitur ini tersedia pada versi X; Supported by community; Didukung oleh komunitas" +release | kata benda | rilis | https://kbbi.web.id/rilis | +tool | | perangkat | | +deployment | | penggelaran | | +client | | klien | | +reference | | rujukan | | +update | | pembaruan | | The latest update... ; Pembaruan terkini... +state | | _state_ | | +task | | _task_ | | +certificate | | sertifikat | | +install | | instalasi | https://kbbi.web.id/instalasi | +scale | | skala | | +process | kata kerja | memproses | https://kbbi.web.id/proses | +replica | kata benda | replika | https://kbbi.web.id/replika | +flag | | tanda, parameter, argumen | | +event | | _event_ | | \ No newline at end of file From 75c599829d2bdfffe0371032e57ccb4d23361efc Mon Sep 17 00:00:00 2001 From: Denis GERMAIN Date: Mon, 14 Dec 2020 17:21:50 +0100 Subject: [PATCH 009/190] fix: errors in base64 and sed commands * All base64 commands need `-w0` argument or else the base64_encoded_ca bash variable will contain space chars (" ") where newlines were * All sed command are missing final "/" at the end of the expression. Command fails with the following error ```bash /bin/sed: -e expression #1, char 95: unterminated `s' command ``` --- .../docs/tasks/tls/manual-rotation-of-ca-certificates.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/content/en/docs/tasks/tls/manual-rotation-of-ca-certificates.md b/content/en/docs/tasks/tls/manual-rotation-of-ca-certificates.md index 3147ac3a18..1720ab34a2 100644 --- a/content/en/docs/tasks/tls/manual-rotation-of-ca-certificates.md +++ b/content/en/docs/tasks/tls/manual-rotation-of-ca-certificates.md @@ -51,12 +51,12 @@ Configurations with a single API server will experience unavailability while the If any pods are started before new CA is used by API servers, they will get this update and trust both old and new CAs. ```shell - base64_encoded_ca="$(base64 )" + base64_encoded_ca="$(base64 -w0 )" for namespace in $(kubectl get ns --no-headers | awk '{print $1}'); do for token in $(kubectl get secrets --namespace "$namespace" --field-selector type=kubernetes.io/service-account-token -o name); do kubectl get $token --namespace "$namespace" -o yaml | \ - /bin/sed "s/\(ca.crt:\).*/\1 ${base64_encoded_ca}" | \ + /bin/sed "s/\(ca.crt:\).*/\1 ${base64_encoded_ca}/" | \ kubectl apply -f - done done @@ -132,10 +132,10 @@ Configurations with a single API server will experience unavailability while the 1. If your cluster is using bootstrap tokens to join nodes, update the ConfigMap `cluster-info` in the `kube-public` namespace with new CA. ```shell - base64_encoded_ca="$(base64 /etc/kubernetes/pki/ca.crt)" + base64_encoded_ca="$(base64 -w0 /etc/kubernetes/pki/ca.crt)" kubectl get cm/cluster-info --namespace kube-public -o yaml | \ - /bin/sed "s/\(certificate-authority-data:\).*/\1 ${base64_encoded_ca}" | \ + /bin/sed "s/\(certificate-authority-data:\).*/\1 ${base64_encoded_ca}/" | \ kubectl apply -f - ``` From 4bf6c31e4e1e66761a087671c5aa1a0ae59f4b44 Mon Sep 17 00:00:00 2001 From: Pierre Tessier Date: Tue, 22 Dec 2020 21:14:32 -0500 Subject: [PATCH 010/190] remove misleading instructions There is no need to restart the kubelet as part of configuring kubeadm to set the cgroup driver. At this point in the setup instructions, the kubernetes cluster isn't even up and running yet, as kubeadm init hasn't been run. The previous step even says the kubelet is in a crashloop waiting for kubeadm. --- .../tools/kubeadm/install-kubeadm.md | 7 ------- 1 file changed, 7 deletions(-) diff --git a/content/en/docs/setup/production-environment/tools/kubeadm/install-kubeadm.md b/content/en/docs/setup/production-environment/tools/kubeadm/install-kubeadm.md index fc93d46cd2..52e903fecf 100644 --- a/content/en/docs/setup/production-environment/tools/kubeadm/install-kubeadm.md +++ b/content/en/docs/setup/production-environment/tools/kubeadm/install-kubeadm.md @@ -304,13 +304,6 @@ or `/etc/default/kubelet`(`/etc/sysconfig/kubelet` for RPMs), please remove it a (stored in `/var/lib/kubelet/config.yaml` by default). {{< /note >}} -Restarting the kubelet is required: - -```bash -sudo systemctl daemon-reload -sudo systemctl restart kubelet -``` - The automatic detection of cgroup driver for other container runtimes like CRI-O and containerd is work in progress. From 364b9e9f5de88e69ecd86295e3be5a03a399dfc9 Mon Sep 17 00:00:00 2001 From: Mehmet Efe Umit Date: Fri, 25 Dec 2020 16:52:18 +0300 Subject: [PATCH 011/190] Replaced the Old Diagrams in Expose Intro Page --- .../expose/expose-intro.html | 8 +- .../public/images/module_04_labels.svg | 1762 ++++++++++------- 2 files changed, 1054 insertions(+), 716 deletions(-) diff --git a/content/en/docs/tutorials/kubernetes-basics/expose/expose-intro.html b/content/en/docs/tutorials/kubernetes-basics/expose/expose-intro.html index c610b6e9f4..1d8a069984 100644 --- a/content/en/docs/tutorials/kubernetes-basics/expose/expose-intro.html +++ b/content/en/docs/tutorials/kubernetes-basics/expose/expose-intro.html @@ -63,13 +63,7 @@ weight: 10

Services and Labels

- -
-
-

-
-
- +

A Service routes traffic across a set of Pods. Services are the abstraction that allow pods to die and replicate in Kubernetes without impacting your application. Discovery and routing among dependent Pods (such as the frontend and backend components in an application) is handled by Kubernetes Services.

diff --git a/content/en/docs/tutorials/kubernetes-basics/public/images/module_04_labels.svg b/content/en/docs/tutorials/kubernetes-basics/public/images/module_04_labels.svg index 31cd8638a1..781bfa0888 100644 --- a/content/en/docs/tutorials/kubernetes-basics/public/images/module_04_labels.svg +++ b/content/en/docs/tutorials/kubernetes-basics/public/images/module_04_labels.svg @@ -1,710 +1,1054 @@ - - - - - - - - - - - - - - - - - - - - - - - Docker - - Kubelt - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + From e4efeae4b87b585f32772af53b9e1489f389a3f6 Mon Sep 17 00:00:00 2001 From: ryicoh Date: Mon, 28 Dec 2020 02:53:13 +0900 Subject: [PATCH 012/190] [ja] new file: memory-constraint-namespace.md --- .../memory-constraint-namespace.md | 265 ++++++++++++++++++ 1 file changed, 265 insertions(+) create mode 100644 content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md diff --git a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md new file mode 100644 index 0000000000..ea659d7340 --- /dev/null +++ b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -0,0 +1,265 @@ +--- +title: ネームスペースに対する最小および最大メモリ制約の構成 + +content_type: task +weight: 30 +--- + + + + +このページでは、ネームスペースで実行されるコンテナが使用するメモリの最小値と最大値を設定する方法を説明します。 +[LimitRange](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#limitrange-v1-core) で最小値と最大値のメモリ値を指定します。 +ポッドがLimitRangeによって課される制約を満たさない場合、そのネームスペースではポッドを作成できません。 + + +## {{% heading "prerequisites" %}} + + +{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}} + +クラスタ内の各ノードには、少なくとも1GiBのメモリが必要です。 + + + + +## ネームスペースの作成 + +この演習で作成したリソースがクラスタの他の部分から分離されるように、ネームスペースを作成します。 + + +```shell +kubectl create namespace constraints-mem-example +``` + +## LimitRangeとポッドを作成 + +LimitRangeの設定ファイルです。 + +{{< codenew file="admin/resource/memory-constraints.yaml" >}} + +LimitRangeを作成します。 + +```shell +kubectl apply -f https://k8s.io/examples/admin/resource/memory-constraints.yaml --namespace=constraints-mem-example +``` + +LimitRangeの詳細情報を表示します。 + + +```shell +kubectl get limitrange mem-min-max-demo-lr --namespace=constraints-mem-example --output=yaml +``` + +出力されるのは、予想通りメモリ制約の最小値と最大値を示しています。 +しかし、LimitRangeの設定ファイルでデフォルト値を指定していないにもかかわらず、 +自動的に作成されていることに気づきます。 + + +``` + limits: + - default: + memory: 1Gi + defaultRequest: + memory: 1Gi + max: + memory: 1Gi + min: + memory: 500Mi + type: Container +``` + + +constraints-mem-exampleネームスペースにコンテナが作成されるたびに、 +Kubernetesは以下の手順を実行するようになっています。 + +* コンテナが独自のメモリ要求と制限を指定しない場合は、デフォルトのメモリ要求と制限をコンテナに割り当てます。 + +* コンテナに500MiB以上のメモリ要求があることを確認します。 + +* コンテナのメモリ制限が1GiB以下であることを確認します。 + +以下は、1つのコンテナを持つポッドの設定ファイルです。設定ファイルのコンテナ(containers)では、600MiBのメモリ要求と800MiBのメモリ制限が指定されています。これらはLimitRangeによって課される最小と最大のメモリ制約を満たしています。 + + +{{< codenew file="admin/resource/memory-constraints-pod.yaml" >}} + +ポッドの作成 + +```shell +kubectl apply -f https://k8s.io/examples/admin/resource/memory-constraints-pod.yaml --namespace=constraints-mem-example +``` + +ポッドのコンテナが実行されていることを確認します。 + +```shell +kubectl get pod constraints-mem-demo --namespace=constraints-mem-example +``` + +ポッドの詳細情報を見ます + +```shell +kubectl get pod constraints-mem-demo --output=yaml --namespace=constraints-mem-example +``` + +出力は、コンテナが600MiBのメモリ要求と800MiBのメモリ制限になっていることを示しています。これらはLimitRangeによって課される制約を満たしています。 + + +```yaml +resources: + limits: + memory: 800Mi + requests: + memory: 600Mi +``` + +ポッドを消します。 + +```shell +kubectl delete pod constraints-mem-demo --namespace=constraints-mem-example +``` + +## 最大メモリ制約を超えるポッドの作成の試み + +これは、1つのコンテナを持つポッドの設定ファイルです。コンテナは800MiBのメモリ要求と1.5GiBのメモリ制限を指定しています。 + + +{{< codenew file="admin/resource/memory-constraints-pod-2.yaml" >}} + +ポッドを作成してみます。 + +```shell +kubectl apply -f https://k8s.io/examples/admin/resource/memory-constraints-pod-2.yaml --namespace=constraints-mem-example +``` + +出力は、コンテナが大きすぎるメモリ制限を指定しているため、ポッドが作成されないことを示しています。 + + +``` +Error from server (Forbidden): error when creating "examples/admin/resource/memory-constraints-pod-2.yaml": +pods "constraints-mem-demo-2" is forbidden: maximum memory usage per Container is 1Gi, but limit is 1536Mi. +``` + +## 最低限のメモリ要求を満たさないポッドの作成の試み + + +これは、1つのコンテナを持つポッドの設定ファイルです。コンテナは100MiBのメモリ要求と800MiBのメモリ制限を指定しています。 + + +{{< codenew file="admin/resource/memory-constraints-pod-3.yaml" >}} + +ポッドを作成してみます。 + +```shell +kubectl apply -f https://k8s.io/examples/admin/resource/memory-constraints-pod-3.yaml --namespace=constraints-mem-example +``` + +出力は、コンテナが小さすぎるメモリリクエストを指定しているため、ポッドが作成されないことを示しています。 + +``` +Error from server (Forbidden): error when creating "examples/admin/resource/memory-constraints-pod-3.yaml": +pods "constraints-mem-demo-3" is forbidden: minimum memory usage per Container is 500Mi, but request is 100Mi. +``` + +## メモリ要求や制限を指定しないポッドの作成 + + +これは、1つのコンテナを持つポッドの設定ファイルです。コンテナはメモリ要求を指定しておらず、メモリ制限も指定していません。 + +{{< codenew file="admin/resource/memory-constraints-pod-4.yaml" >}} + +ポッドを作成します。 + +```shell +kubectl apply -f https://k8s.io/examples/admin/resource/memory-constraints-pod-4.yaml --namespace=constraints-mem-example +``` + +ポッドの詳細情報を見ます + +``` +kubectl get pod constraints-mem-demo-4 --namespace=constraints-mem-example --output=yaml +``` + +出力を見ると、ポッドのコンテナのメモリ要求は1GiB、メモリ制限は1GiBであることがわかります。 +コンテナはどのようにしてこれらの値を取得したのでしょうか？ + + +``` +resources: + limits: + memory: 1Gi + requests: + memory: 1Gi +``` + +コンテナが独自のメモリ要求と制限を指定していなかったため、LimitRangeから与えられたのです。 +コンテナが独自のメモリ要求と制限を指定していなかったため、LimitRangeから[デフォルトのメモリ要求と制限](/docs/tasks/administer-cluster/manage-resources/memory-default-namespace/)が与えられたのです。 + +この時点で、コンテナは起動しているかもしれませんし、起動していないかもしれません。このタスクの前提条件は、ノードが少なくとも1GiBのメモリを持っていることであることを思い出してください。それぞれのノードが1GiBのメモリしか持っていない場合、どのノードにも1GiBのメモリ要求に対応するのに十分な割り当て可能なメモリがありません。たまたま2GiBのメモリを持つノードを使用しているのであれば、おそらく1GiBのメモリリクエストに対応するのに十分なスペースを持っていることになります。 + + +ポッドを削除します。 + +``` +kubectl delete pod constraints-mem-demo-4 --namespace=constraints-mem-example +``` + +## 最小および最大メモリ制約の強制 + +LimitRangeによってネームスペースに課される最大および最小のメモリ制約は、ポッドが作成または更新されたときにのみ適用されます。LimitRangeを変更しても、以前に作成されたポッドには影響しません。 + + +## 最小・最大メモリ制約の動機 + + +クラスタ管理者としては、ポッドが使用できるメモリ量に制限を課したいと思うかもしれません。 + + +例: + +* クラスタ内の各ノードは2GBのメモリを持っています。クラスタ内のどのノードもその要求をサポートできないため、2GB以上のメモリを要求するポッドは受け入れたくありません。 + + +* クラスタは運用部門と開発部門で共有されています。 本番用のワークロードでは最大8GBのメモリを消費しますが、開発用のワークロードでは512MBに制限したいとします。本番用と開発用に別々のネームスペースを作成し、それぞれのネームスペースにメモリ制限を適用します。 + +## クリーンアップ + +ネームスペースを削除します。 + +```shell +kubectl delete namespace constraints-mem-example +``` + + + +## {{% heading "whatsnext" %}} + + +### クラスタ管理者向け + +* [名前空間に対するデフォルトのメモリ要求と制限の構成](/docs/tasks/administer-cluster/manage-resources/memory-default-namespace/) + +* [名前空間に対するデフォルトのCPU要求と制限の構成](/docs/tasks/administer-cluster/manage-resources/cpu-default-namespace/) + +* [名前空間に対する最小および最大CPU制約の構成](/docs/tasks/administer-cluster/manage-resources/cpu-constraint-namespace/) + +* [名前空間に対するメモリとCPUのクォータの構成](/docs/tasks/administer-cluster/manage-resources/quota-memory-cpu-namespace/) + +* [名前空間に対するポッドクォータの設定](/docs/tasks/administer-cluster/manage-resources/quota-pod-namespace/) + +* [APIオブジェクトのクォータの設定](/docs/tasks/administer-cluster/quota-api-object/) + +### アプリケーション開発者向け + +* [コンテナとポッドへのメモリリソースの割り当て](/docs/tasks/configure-pod-container/assign-memory-resource/) + +* [コンテナとポッドへのCPUリソースの割り当て](/docs/tasks/configure-pod-container/assign-cpu-resource/) + +* [ポッドのQoS(サービス品質)を設定](/docs/tasks/configure-pod-container/quality-service-pod/) + + + + + + + From d02c2701422fcc5d7a0e382e35cc95e0f02f1717 Mon Sep 17 00:00:00 2001 From: ryicoh Date: Mon, 28 Dec 2020 14:14:11 +0900 Subject: [PATCH 013/190] refs: https://github.com/kubernetes/website/pull/25827#discussion_r549204180 --- .../memory-constraint-namespace.md | 70 +++++++++---------- 1 file changed, 35 insertions(+), 35 deletions(-) diff --git a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md index ea659d7340..43b51f8d3a 100644 --- a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md +++ b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -1,5 +1,5 @@ --- -title: ネームスペースに対する最小および最大メモリ制約の構成 +title: Namespaceに対する最小および最大メモリ制約の構成 content_type: task weight: 30 @@ -8,9 +8,9 @@ weight: 30 -このページでは、ネームスペースで実行されるコンテナが使用するメモリの最小値と最大値を設定する方法を説明します。 +このページでは、Namespaceで実行されるコンテナが使用するメモリの最小値と最大値を設定する方法を説明します。 [LimitRange](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#limitrange-v1-core) で最小値と最大値のメモリ値を指定します。 -ポッドがLimitRangeによって課される制約を満たさない場合、そのネームスペースではポッドを作成できません。 +PodがLimitRangeによって課される制約を満たさない場合、そのNamespaceではPodを作成できません。 ## {{% heading "prerequisites" %}} @@ -23,16 +23,16 @@ weight: 30 -## ネームスペースの作成 +## Namespaceの作成 -この演習で作成したリソースがクラスタの他の部分から分離されるように、ネームスペースを作成します。 +この演習で作成したリソースがクラスタの他の部分から分離されるように、Namespaceを作成します。 ```shell kubectl create namespace constraints-mem-example ``` -## LimitRangeとポッドを作成 +## LimitRangeとPodを作成 LimitRangeの設定ファイルです。 @@ -70,7 +70,7 @@ kubectl get limitrange mem-min-max-demo-lr --namespace=constraints-mem-example - ``` -constraints-mem-exampleネームスペースにコンテナが作成されるたびに、 +constraints-mem-exampleNamespaceにコンテナが作成されるたびに、 Kubernetesは以下の手順を実行するようになっています。 * コンテナが独自のメモリ要求と制限を指定しない場合は、デフォルトのメモリ要求と制限をコンテナに割り当てます。 @@ -79,24 +79,24 @@ Kubernetesは以下の手順を実行するようになっています。 * コンテナのメモリ制限が1GiB以下であることを確認します。 -以下は、1つのコンテナを持つポッドの設定ファイルです。設定ファイルのコンテナ(containers)では、600MiBのメモリ要求と800MiBのメモリ制限が指定されています。これらはLimitRangeによって課される最小と最大のメモリ制約を満たしています。 +以下は、1つのコンテナを持つPodの設定ファイルです。設定ファイルのコンテナ(containers)では、600MiBのメモリ要求と800MiBのメモリ制限が指定されています。これらはLimitRangeによって課される最小と最大のメモリ制約を満たしています。 {{< codenew file="admin/resource/memory-constraints-pod.yaml" >}} -ポッドの作成 +Podの作成 ```shell kubectl apply -f https://k8s.io/examples/admin/resource/memory-constraints-pod.yaml --namespace=constraints-mem-example ``` -ポッドのコンテナが実行されていることを確認します。 +Podのコンテナが実行されていることを確認します。 ```shell kubectl get pod constraints-mem-demo --namespace=constraints-mem-example ``` -ポッドの詳細情報を見ます +Podの詳細情報を見ます ```shell kubectl get pod constraints-mem-demo --output=yaml --namespace=constraints-mem-example @@ -113,26 +113,26 @@ resources: memory: 600Mi ``` -ポッドを消します。 +Podを消します。 ```shell kubectl delete pod constraints-mem-demo --namespace=constraints-mem-example ``` -## 最大メモリ制約を超えるポッドの作成の試み +## 最大メモリ制約を超えるPodの作成の試み -これは、1つのコンテナを持つポッドの設定ファイルです。コンテナは800MiBのメモリ要求と1.5GiBのメモリ制限を指定しています。 +これは、1つのコンテナを持つPodの設定ファイルです。コンテナは800MiBのメモリ要求と1.5GiBのメモリ制限を指定しています。 {{< codenew file="admin/resource/memory-constraints-pod-2.yaml" >}} -ポッドを作成してみます。 +Podを作成してみます。 ```shell kubectl apply -f https://k8s.io/examples/admin/resource/memory-constraints-pod-2.yaml --namespace=constraints-mem-example ``` -出力は、コンテナが大きすぎるメモリ制限を指定しているため、ポッドが作成されないことを示しています。 +出力は、コンテナが大きすぎるメモリ制限を指定しているため、Podが作成されないことを示しています。 ``` @@ -140,47 +140,47 @@ Error from server (Forbidden): error when creating "examples/admin/resource/memo pods "constraints-mem-demo-2" is forbidden: maximum memory usage per Container is 1Gi, but limit is 1536Mi. ``` -## 最低限のメモリ要求を満たさないポッドの作成の試み +## 最低限のメモリ要求を満たさないPodの作成の試み -これは、1つのコンテナを持つポッドの設定ファイルです。コンテナは100MiBのメモリ要求と800MiBのメモリ制限を指定しています。 +これは、1つのコンテナを持つPodの設定ファイルです。コンテナは100MiBのメモリ要求と800MiBのメモリ制限を指定しています。 {{< codenew file="admin/resource/memory-constraints-pod-3.yaml" >}} -ポッドを作成してみます。 +Podを作成してみます。 ```shell kubectl apply -f https://k8s.io/examples/admin/resource/memory-constraints-pod-3.yaml --namespace=constraints-mem-example ``` -出力は、コンテナが小さすぎるメモリリクエストを指定しているため、ポッドが作成されないことを示しています。 +出力は、コンテナが小さすぎるメモリリクエストを指定しているため、Podが作成されないことを示しています。 ``` Error from server (Forbidden): error when creating "examples/admin/resource/memory-constraints-pod-3.yaml": pods "constraints-mem-demo-3" is forbidden: minimum memory usage per Container is 500Mi, but request is 100Mi. ``` -## メモリ要求や制限を指定しないポッドの作成 +## メモリ要求や制限を指定しないPodの作成 -これは、1つのコンテナを持つポッドの設定ファイルです。コンテナはメモリ要求を指定しておらず、メモリ制限も指定していません。 +これは、1つのコンテナを持つPodの設定ファイルです。コンテナはメモリ要求を指定しておらず、メモリ制限も指定していません。 {{< codenew file="admin/resource/memory-constraints-pod-4.yaml" >}} -ポッドを作成します。 +Podを作成します。 ```shell kubectl apply -f https://k8s.io/examples/admin/resource/memory-constraints-pod-4.yaml --namespace=constraints-mem-example ``` -ポッドの詳細情報を見ます +Podの詳細情報を見ます ``` kubectl get pod constraints-mem-demo-4 --namespace=constraints-mem-example --output=yaml ``` -出力を見ると、ポッドのコンテナのメモリ要求は1GiB、メモリ制限は1GiBであることがわかります。 +出力を見ると、Podのコンテナのメモリ要求は1GiB、メモリ制限は1GiBであることがわかります。 コンテナはどのようにしてこれらの値を取得したのでしょうか？ @@ -198,7 +198,7 @@ resources: この時点で、コンテナは起動しているかもしれませんし、起動していないかもしれません。このタスクの前提条件は、ノードが少なくとも1GiBのメモリを持っていることであることを思い出してください。それぞれのノードが1GiBのメモリしか持っていない場合、どのノードにも1GiBのメモリ要求に対応するのに十分な割り当て可能なメモリがありません。たまたま2GiBのメモリを持つノードを使用しているのであれば、おそらく1GiBのメモリリクエストに対応するのに十分なスペースを持っていることになります。 -ポッドを削除します。 +Podを削除します。 ``` kubectl delete pod constraints-mem-demo-4 --namespace=constraints-mem-example @@ -206,25 +206,25 @@ kubectl delete pod constraints-mem-demo-4 --namespace=constraints-mem-example ## 最小および最大メモリ制約の強制 -LimitRangeによってネームスペースに課される最大および最小のメモリ制約は、ポッドが作成または更新されたときにのみ適用されます。LimitRangeを変更しても、以前に作成されたポッドには影響しません。 +LimitRangeによってNamespaceに課される最大および最小のメモリ制約は、Podが作成または更新されたときにのみ適用されます。LimitRangeを変更しても、以前に作成されたPodには影響しません。 ## 最小・最大メモリ制約の動機 -クラスタ管理者としては、ポッドが使用できるメモリ量に制限を課したいと思うかもしれません。 +クラスタ管理者としては、Podが使用できるメモリ量に制限を課したいと思うかもしれません。 例: -* クラスタ内の各ノードは2GBのメモリを持っています。クラスタ内のどのノードもその要求をサポートできないため、2GB以上のメモリを要求するポッドは受け入れたくありません。 +* クラスタ内の各ノードは2GBのメモリを持っています。クラスタ内のどのノードもその要求をサポートできないため、2GB以上のメモリを要求するPodは受け入れたくありません。 -* クラスタは運用部門と開発部門で共有されています。 本番用のワークロードでは最大8GBのメモリを消費しますが、開発用のワークロードでは512MBに制限したいとします。本番用と開発用に別々のネームスペースを作成し、それぞれのネームスペースにメモリ制限を適用します。 +* クラスタは運用部門と開発部門で共有されています。 本番用のワークロードでは最大8GBのメモリを消費しますが、開発用のワークロードでは512MBに制限したいとします。本番用と開発用に別々のNamespaceを作成し、それぞれのNamespaceにメモリ制限を適用します。 ## クリーンアップ -ネームスペースを削除します。 +Namespaceを削除します。 ```shell kubectl delete namespace constraints-mem-example @@ -245,17 +245,17 @@ kubectl delete namespace constraints-mem-example * [名前空間に対するメモリとCPUのクォータの構成](/docs/tasks/administer-cluster/manage-resources/quota-memory-cpu-namespace/) -* [名前空間に対するポッドクォータの設定](/docs/tasks/administer-cluster/manage-resources/quota-pod-namespace/) +* [名前空間に対するPodクォータの設定](/docs/tasks/administer-cluster/manage-resources/quota-pod-namespace/) * [APIオブジェクトのクォータの設定](/docs/tasks/administer-cluster/quota-api-object/) ### アプリケーション開発者向け -* [コンテナとポッドへのメモリリソースの割り当て](/docs/tasks/configure-pod-container/assign-memory-resource/) +* [コンテナとPodへのメモリリソースの割り当て](/docs/tasks/configure-pod-container/assign-memory-resource/) -* [コンテナとポッドへのCPUリソースの割り当て](/docs/tasks/configure-pod-container/assign-cpu-resource/) +* [コンテナとPodへのCPUリソースの割り当て](/docs/tasks/configure-pod-container/assign-cpu-resource/) -* [ポッドのQoS(サービス品質)を設定](/docs/tasks/configure-pod-container/quality-service-pod/) +* [PodのQoS(サービス品質)を設定](/docs/tasks/configure-pod-container/quality-service-pod/) From 9c94fb01bcafc3bbe41453828f958f2445ec6830 Mon Sep 17 00:00:00 2001 From: Chintan Thakker <45217805+chintan8saaras@users.noreply.github.com> Date: Tue, 5 Jan 2021 18:20:49 -0800 Subject: [PATCH 014/190] Add EnRoute as an option EnRoute is an actively maintained API Gateway Ingress controller built on [Envoy Proxy](https://envoyproxy.io/community.html). It is an [OSS project](https://github.com/saarasio/enroute) that includes support for advanced rate-limiting in the [community edition](https://getenroute.io/features). --- .../en/docs/concepts/services-networking/ingress-controllers.md | 1 + 1 file changed, 1 insertion(+) diff --git a/content/en/docs/concepts/services-networking/ingress-controllers.md b/content/en/docs/concepts/services-networking/ingress-controllers.md index df0e3ffd22..99072882c6 100644 --- a/content/en/docs/concepts/services-networking/ingress-controllers.md +++ b/content/en/docs/concepts/services-networking/ingress-controllers.md @@ -29,6 +29,7 @@ Kubernetes as a project supports and maintains [AWS](https://github.com/kubernet * The [Citrix ingress controller](https://github.com/citrix/citrix-k8s-ingress-controller#readme) works with Citrix Application Delivery Controller. * [Contour](https://projectcontour.io/) is an [Envoy](https://www.envoyproxy.io/) based ingress controller. +* [EnRoute](https://getenroute.io) is an open source Ingress Controller API Gateway based on [Envoy](https://www.envoyproxy.io) that includes support to automate OpenAPI spec ingestion and advanced rate-limiting. * F5 BIG-IP [Container Ingress Services for Kubernetes](https://clouddocs.f5.com/containers/latest/userguide/kubernetes/) lets you use an Ingress to configure F5 BIG-IP virtual servers. * [Gloo](https://gloo.solo.io) is an open-source ingress controller based on [Envoy](https://www.envoyproxy.io), From 26300785f5f6fb12dbf90ae48d70c64e1bda9290 Mon Sep 17 00:00:00 2001 From: Oliver L Schoenborn Date: Wed, 6 Jan 2021 21:33:07 -0500 Subject: [PATCH 015/190] Clarify binaryData handling by kubectl create and describe --- .../tasks/configure-pod-container/configure-pod-configmap.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/content/en/docs/tasks/configure-pod-container/configure-pod-configmap.md b/content/en/docs/tasks/configure-pod-container/configure-pod-configmap.md index 2824cce642..1d22c2f3e7 100644 --- a/content/en/docs/tasks/configure-pod-container/configure-pod-configmap.md +++ b/content/en/docs/tasks/configure-pod-container/configure-pod-configmap.md @@ -201,6 +201,8 @@ allow.textmode=true how.nice.to.look=fairlyNice ``` +Note that data sources that are not ASCII or UTF-8 will be put in the `binaryData` field of the configmap. Both text and binary data sources can be combined in one configmap, but the `binaryData` field is not displayed (as of kubectl 1.20) by `kubectl describe`. To see the `binaryData` in the configmap, you can use `kubectl get configmap -o json MAP_NAME`. + Use the option `--from-env-file` to create a ConfigMap from an env-file, for example: ```shell From 3d23d1799b4bf7bcd003ddefec9747c30b17c039 Mon Sep 17 00:00:00 2001 From: Konrad Delong Date: Thu, 7 Jan 2021 10:40:49 +0100 Subject: [PATCH 016/190] fixes explanation for --[hpa]-stabilization The current explanation is misleading (for example: for consistently decreasing resource usage, the HPA stabilization does not impact the frequency of its actuation). This has confused users recently ( https://github.com/kubernetes/kubernetes/issues/96671 ) --- .../docs/tasks/run-application/horizontal-pod-autoscale.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/content/en/docs/tasks/run-application/horizontal-pod-autoscale.md b/content/en/docs/tasks/run-application/horizontal-pod-autoscale.md index ff51598f79..4cfbb98f2d 100644 --- a/content/en/docs/tasks/run-application/horizontal-pod-autoscale.md +++ b/content/en/docs/tasks/run-application/horizontal-pod-autoscale.md @@ -221,9 +221,9 @@ the global HPA settings exposed as flags for the `kube-controller-manager` compo Starting from v1.12, a new algorithmic update removes the need for the upscale delay. -- `--horizontal-pod-autoscaler-downscale-stabilization`: The value for this option is a - duration that specifies how long the autoscaler has to wait before another - downscale operation can be performed after the current one has completed. +- `--horizontal-pod-autoscaler-downscale-stabilization`: Specifies the duration of the + downscale stabilization time window. Horizontal Pod Autoscaler remembers + the historical recommended sizes and only acts on the largest size within this time window. The default value is 5 minutes (`5m0s`). {{< note >}} From c32685a23dddd2615470fdcaeb6de7c18525657b Mon Sep 17 00:00:00 2001 From: Jailton Lopes Date: Thu, 7 Jan 2021 23:22:17 -0300 Subject: [PATCH 017/190] Update README-pt.md --- README-pt.md | 173 ++++++++++++++++++++++++++++++++++----------------- 1 file changed, 117 insertions(+), 56 deletions(-) diff --git a/README-pt.md b/README-pt.md index 3154b77bab..3d8d5fb5bc 100644 --- a/README-pt.md +++ b/README-pt.md @@ -1,76 +1,137 @@ # A documentação do Kubernetes -[![Build Status](https://api.travis-ci.org/kubernetes/website.svg?branch=master)](https://travis-ci.org/kubernetes/website) -[![GitHub release](https://img.shields.io/github/release/kubernetes/website.svg)](https://github.com/kubernetes/website/releases/latest) +[![Netlify Status](https://api.netlify.com/api/v1/badges/be93b718-a6df-402a-b4a4-855ba186c97d/deploy-status)](https://app.netlify.com/sites/kubernetes-io-master-staging/deploys) [![GitHub release](https://img.shields.io/github/release/kubernetes/website.svg)](https://github.com/kubernetes/website/releases/latest) -Bem vindos! Este repositório abriga todos os recursos necessários para criar o [site e documentação do Kubernetes](https://kubernetes.io/). Estamos muito satisfeitos por você querer contribuir! +Bem vindos! Este repositório abriga todos os recursos necessários para criar o [website e documentação do Kubernetes](https://kubernetes.io/). Estamos muito satisfeitos por você querer contribuir! -## Contribuindo com os documentos +# Utilizando este repositório -Você pode clicar no botão **Fork** na área superior direita da tela para criar uma cópia desse repositório na sua conta do GitHub. Esta cópia é chamada de *fork*. Faça as alterações desejadas no seu fork e, quando estiver pronto para enviar as alterações para nós, vá até o fork e crie uma nova solicitação de pull para nos informar sobre isso. +Você pode executar o website localmente utilizando o Hugo (versão Extended), ou você pode executa-ló em um container runtime. É altamente recomendável usar um container runtime, pois garante a consistência na implantação do website real. -Depois que seu **pull request** for criado, um revisor do Kubernetes assumirá a responsabilidade de fornecer um feedback claro e objetivo. Como proprietário do pull request, **é sua responsabilidade modificar seu pull request para abordar o feedback que foi fornecido a você pelo revisor do Kubernetes.** Observe também que você pode acabar tendo mais de um revisor do Kubernetes para fornecer seu feedback ou você pode acabar obtendo feedback de um revisor do Kubernetes que é diferente daquele originalmente designado para lhe fornecer feedback. Além disso, em alguns casos, um de seus revisores pode solicitar uma revisão técnica de um [revisor de tecnologia Kubernetes](https://github.com/kubernetes/website/wiki/Tech-reviewers) quando necessário. Os revisores farão o melhor para fornecer feedback em tempo hábil, mas o tempo de resposta pode variar de acordo com as circunstâncias. +## Pré-requisitos + +Para usar este repositório, você precisa instalar: + +- [npm](https://www.npmjs.com/) +- [Go](https://golang.org/) +- [Hugo (versão Extended)](https://gohugo.io/) +- Um container runtime, por exemplo [Docker](https://www.docker.com/). + +Antes de você iniciar, instale as dependências, clone o repositório e navegue até o diretório: + +``` +git clone https://github.com/kubernetes/website.git +cd website +``` + +O website do Kubernetes utiliza o [tema Docsy Hugo](https://github.com/google/docsy#readme). Mesmo se você planeje executar o website em um container, é altamente recomendado baixar os submódulos e outras dependências executando o seguinte comando: + +``` +# pull in the Docsy submodule +git submodule update --init --recursive --depth 1 +``` + +## Executando o website usando um container + +Para executar o build do website em um container, execute o comando abaixo para criar a imagem do container e executa-lá: + +``` +make container-image +make container-serve +``` + +Abra seu navegador em http://localhost:1313 para visualizar o website. Conforme você faz alterações nos arquivos fontes, Hugo atualiza o website e força a atualização do navegador. + +## Executando o website localmente utilizando o Hugo + +Consulte a [documentação oficial do Hugo](https://gohugo.io/getting-started/installing/) para instruções de instalação do Hugo. Certifique-se de instalar a versão do Hugo especificada pela variável de ambiente `HUGO_VERSION` no arquivo [`netlify.toml`](netlify.toml#L9). + +Para executar o build e testar o website localmente, execute: + +```bash +# instalar dependências +npm ci +make serve +``` + +Isso iniciará localmente o Hugo na porta 1313. Abra o seu navegador em http://localhost:1313 para visualizar o website. Conforme você faz alterações nos arquivos fontes, o Hugo atualiza o website e força uma atualização no navegador. + +## Troubleshooting +### error: failed to transform resource: TOCSS: failed to transform "scss/main.scss" (text/x-scss): this feature is not available in your current Hugo version + +Por motivos técnicos, o Hugo é disponibilizado em dois conjuntos de binários. O website atual funciona apenas na versão **Hugo Extended**. Na [página de releases](https://github.com/gohugoio/hugo/releases) procure por arquivos com `extended` no nome. Para confirmar, execute `hugo version` e procure pela palavra `extended`. + +### Troubleshooting macOS for too many open files + +Se você executar o comando `make serve` no macOS e retornar o seguinte erro: + +``` +ERROR 2020/08/01 19:09:18 Error: listen tcp 127.0.0.1:1313: socket: too many open files +make: *** [serve] Error 1 +``` + +Verifique o limite atual para arquivos abertos: + +`launchctl limit maxfiles` + +Em seguida, execute os seguintes comandos (adaptado de https://gist.github.com/tombigel/d503800a282fcadbee14b537735d202c): + +```shell +#!/bin/sh + +# Esse são os links do gist original, vinculados ao meu gists agora. +# curl -O https://gist.githubusercontent.com/a2ikm/761c2ab02b7b3935679e55af5d81786a/raw/ab644cb92f216c019a2f032bbf25e258b01d87f9/limit.maxfiles.plist +# curl -O https://gist.githubusercontent.com/a2ikm/761c2ab02b7b3935679e55af5d81786a/raw/ab644cb92f216c019a2f032bbf25e258b01d87f9/limit.maxproc.plist + +curl -O https://gist.githubusercontent.com/tombigel/d503800a282fcadbee14b537735d202c/raw/ed73cacf82906fdde59976a0c8248cce8b44f906/limit.maxfiles.plist +curl -O https://gist.githubusercontent.com/tombigel/d503800a282fcadbee14b537735d202c/raw/ed73cacf82906fdde59976a0c8248cce8b44f906/limit.maxproc.plist + +sudo mv limit.maxfiles.plist /Library/LaunchDaemons +sudo mv limit.maxproc.plist /Library/LaunchDaemons + +sudo chown root:wheel /Library/LaunchDaemons/limit.maxfiles.plist +sudo chown root:wheel /Library/LaunchDaemons/limit.maxproc.plist + +sudo launchctl load -w /Library/LaunchDaemons/limit.maxfiles.plist +``` + +Esta solução funciona tanto para o MacOS Catalina quanto para o MacOS Mojave. + +# Comunidade, discussão, contribuição e apoio + +Saiba mais sobre a comunidade Kubernetes SIG Docs e reuniões na [página da comunidade](http://kubernetes.io/community/). + +Você também pode entrar em contato com os mantenedores deste projeto em: + +- [Slack](https://kubernetes.slack.com/messages/sig-docs) ([Obter o convide para o este slack](https://slack.k8s.io/)) +- [Mailing List](https://groups.google.com/forum/#!forum/kubernetes-sig-docs) + +# Contribuindo com os documentos + +Você pode clicar no botão **Fork** na área superior direita da tela para criar uma cópia desse repositório na sua conta do GitHub. Esta cópia é chamada de *fork*. Faça as alterações desejadas no seu fork e, quando estiver pronto para enviar as alterações para nós, vá até o fork e crie um novo **pull request** para nos informar sobre isso. + +Depois que seu **pull request** for criado, um revisor do Kubernetes assumirá a responsabilidade de fornecer um feedback claro e objetivo. Como proprietário do pull request, **é sua responsabilidade modificar seu pull request para atender ao feedback que foi fornecido a você pelo revisor do Kubernetes.** + +Observe também que você pode acabar tendo mais de um revisor do Kubernetes para fornecer seu feedback ou você pode acabar obtendo feedback de um outro revisor do Kubernetes diferente daquele originalmente designado para lhe fornecer o feedback. + +Além disso, em alguns casos, um de seus revisores pode solicitar uma revisão técnica de um [revisor técnico do Kubernetes](https://github.com/kubernetes/website/wiki/Tech-reviewers) quando necessário. Os revisores farão o melhor para fornecer feedbacks em tempo hábil, mas o tempo de resposta pode variar de acordo com as circunstâncias. Para mais informações sobre como contribuir com a documentação do Kubernetes, consulte: -* [Comece a contribuir](https://kubernetes.io/docs/contribute/start/) -* [Preparando suas alterações na documentação](http://kubernetes.io/docs/contribute/intermediate#view-your-changes-locally) -* [Usando Modelos de Página](http://kubernetes.io/docs/contribute/style/page-templates/) +* [Contribua com a documentação do Kubernetes](https://kubernetes.io/docs/contribute/) +* [Tipos de conteúdo de página](https://kubernetes.io/docs/contribute/style/page-content-types/) * [Guia de Estilo da Documentação](http://kubernetes.io/docs/contribute/style/style-guide/) * [Localizando documentação do Kubernetes](https://kubernetes.io/docs/contribute/localization/) -Você pode contactar os mantenedores da localização em Português em: +Você pode contatar os mantenedores da localização em Português em: * Felipe ([GitHub - @femrtnz](https://github.com/femrtnz)) * [Slack channel](https://kubernetes.slack.com/messages/kubernetes-docs-pt) -## Executando o site localmente usando o Docker - -A maneira recomendada de executar o site do Kubernetes localmente é executar uma imagem especializada do [Docker](https://docker.com) que inclui o gerador de site estático [Hugo](https://gohugo.io). - -> Se você está rodando no Windows, você precisará de mais algumas ferramentas que você pode instalar com o [Chocolatey](https://chocolatey.org). `choco install make` - -> Se você preferir executar o site localmente sem o Docker, consulte [Executando o site localmente usando o Hugo](#executando-o-site-localmente-usando-o-hugo) abaixo. - -Se você tiver o Docker [em funcionamento](https://www.docker.com/get-started), crie a imagem do Docker do `kubernetes-hugo` localmente: - -```bash -make container-image -``` - -Depois que a imagem foi criada, você pode executar o site localmente: - -```bash -make container-serve -``` - -Abra seu navegador para http://localhost:1313 para visualizar o site. Conforme você faz alterações nos arquivos de origem, Hugo atualiza o site e força a atualização do navegador. - -## Executando o site localmente usando o Hugo - -Veja a [documentação oficial do Hugo](https://gohugo.io/getting-started/installing/) para instruções de instalação do Hugo. Certifique-se de instalar a versão do Hugo especificada pela variável de ambiente `HUGO_VERSION` no arquivo [`netlify.toml`](netlify.toml#L9). - -Para executar o site localmente quando você tiver o Hugo instalado: - -```bash -make serve -``` - -Isso iniciará o servidor Hugo local na porta 1313. Abra o navegador para http://localhost:1313 para visualizar o site. Conforme você faz alterações nos arquivos de origem, Hugo atualiza o site e força a atualização do navegador. - -## Comunidade, discussão, contribuição e apoio - -Aprenda a se envolver com a comunidade do Kubernetes na [página da comunidade](http://kubernetes.io/community/). - -Você pode falar com os mantenedores deste projeto: - -- [Slack](https://kubernetes.slack.com/messages/sig-docs) -- [Mailing List](https://groups.google.com/forum/#!forum/kubernetes-sig-docs) - -### Código de conduta +# Código de conduta A participação na comunidade Kubernetes é regida pelo [Código de Conduta da Kubernetes](code-of-conduct.md). -## Obrigado! +# Obrigado! -O Kubernetes conta com a participação da comunidade e nós realmente agradecemos suas contribuições para o nosso site e nossa documentação! +O Kubernetes conta com a participação da comunidade e nós realmente agradecemos suas contribuições para o nosso website e nossa documentação! \ No newline at end of file From a0b3e728514bed530107bf9c978e321bbd1f6f8c Mon Sep 17 00:00:00 2001 From: Chintan Thakker <45217805+chintan8saaras@users.noreply.github.com> Date: Fri, 8 Jan 2021 09:08:32 -0800 Subject: [PATCH 018/190] Update EnRoute entry --- .../en/docs/concepts/services-networking/ingress-controllers.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/concepts/services-networking/ingress-controllers.md b/content/en/docs/concepts/services-networking/ingress-controllers.md index 99072882c6..80d2aec0fe 100644 --- a/content/en/docs/concepts/services-networking/ingress-controllers.md +++ b/content/en/docs/concepts/services-networking/ingress-controllers.md @@ -29,7 +29,7 @@ Kubernetes as a project supports and maintains [AWS](https://github.com/kubernet * The [Citrix ingress controller](https://github.com/citrix/citrix-k8s-ingress-controller#readme) works with Citrix Application Delivery Controller. * [Contour](https://projectcontour.io/) is an [Envoy](https://www.envoyproxy.io/) based ingress controller. -* [EnRoute](https://getenroute.io) is an open source Ingress Controller API Gateway based on [Envoy](https://www.envoyproxy.io) that includes support to automate OpenAPI spec ingestion and advanced rate-limiting. +* [EnRoute](https://getenroute.io/) is an [Envoy](https://www.envoyproxy.io) based API gateway that can run as an ingress controller. * F5 BIG-IP [Container Ingress Services for Kubernetes](https://clouddocs.f5.com/containers/latest/userguide/kubernetes/) lets you use an Ingress to configure F5 BIG-IP virtual servers. * [Gloo](https://gloo.solo.io) is an open-source ingress controller based on [Envoy](https://www.envoyproxy.io), From 1b70e98626a0e601fad18e388dd13eab27fbe8b5 Mon Sep 17 00:00:00 2001 From: ydFu Date: Thu, 7 Jan 2021 09:30:47 +0800 Subject: [PATCH 019/190] Add the code blocks in authorization.md * Add the code blocks in the Markdown spec to make it easy to read. * Add description that distinguish between **command** and **output** make it easy to read. * Adjust description in Kubernetes components for smoother reading. Signed-off-by: ydFu --- .../reference/access-authn-authz/authorization.md | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/content/en/docs/reference/access-authn-authz/authorization.md b/content/en/docs/reference/access-authn-authz/authorization.md index a084a7e54e..04963e10ee 100644 --- a/content/en/docs/reference/access-authn-authz/authorization.md +++ b/content/en/docs/reference/access-authn-authz/authorization.md @@ -104,6 +104,9 @@ a given action, and works regardless of the authorization mode used. ```bash kubectl auth can-i create deployments --namespace dev ``` + +The output is similar to this: + ``` yes ``` @@ -111,6 +114,9 @@ yes ```shell kubectl auth can-i create deployments --namespace prod ``` + +The output is similar to this: + ``` no ``` @@ -121,6 +127,9 @@ to determine what action other users can perform. ```bash kubectl auth can-i list secrets --namespace dev --as dave ``` + +The output is similar to this: + ``` no ``` @@ -150,7 +159,7 @@ EOF ``` The generated `SelfSubjectAccessReview` is: -``` +```yaml apiVersion: authorization.k8s.io/v1 kind: SelfSubjectAccessReview metadata: From 929bfc95fa3d92756b3aaf7b7e9ebe54ec29a639 Mon Sep 17 00:00:00 2001 From: xiaoping Date: Sun, 10 Jan 2021 13:27:46 +0800 Subject: [PATCH 020/190] Update deployment.md fix error translation --- content/zh/docs/concepts/workloads/controllers/deployment.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/zh/docs/concepts/workloads/controllers/deployment.md b/content/zh/docs/concepts/workloads/controllers/deployment.md index 8969cad235..487bba0bb1 100644 --- a/content/zh/docs/concepts/workloads/controllers/deployment.md +++ b/content/zh/docs/concepts/workloads/controllers/deployment.md @@ -186,7 +186,7 @@ Follow the steps given below to create the above Deployment: --> * `NAME` 列出了集群中 Deployment 的名称。 * `READY` 显示应用程序的可用的 _副本_ 数。显示的模式是“就绪个数/期望个数”。 - * `UP-TO-DATE` 显示为了打到期望状态已经更新的副本数。 + * `UP-TO-DATE` 显示为了达到期望状态已经更新的副本数。 * `AVAILABLE` 显示应用可供用户使用的副本数。 * `AGE` 显示应用程序运行的时间。 From 5accf8f1283ed0d76ef7e154597993b1afe68cd2 Mon Sep 17 00:00:00 2001 From: Margo Crawford Date: Fri, 8 Jan 2021 16:38:51 -0800 Subject: [PATCH 021/190] Rewording of paragraph about provideClusterInfo key Signed-off-by: Margo Crawford --- .../access-authn-authz/authentication.md | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/content/en/docs/reference/access-authn-authz/authentication.md b/content/en/docs/reference/access-authn-authz/authentication.md index c385a15fda..f88d8e2370 100644 --- a/content/en/docs/reference/access-authn-authz/authentication.md +++ b/content/en/docs/reference/access-authn-authz/authentication.md @@ -1004,14 +1004,13 @@ RFC3339 timestamp. Presence or absence of an expiry has the following impact: } } ``` - -The plugin can optionally be called with an environment variable, `KUBERNETES_EXEC_INFO`, -that contains information about the cluster for which this plugin is obtaining -credentials. This information can be used to perform cluster-specific credential -acquisition logic. In order to enable this behavior, the `provideClusterInfo` field must -be set on the exec user field in the -[kubeconfig](/docs/concepts/configuration/organize-cluster-access-kubeconfig/). Here is an -example of the aforementioned `KUBERNETES_EXEC_INFO` environment variable. +To provide the plugin with information about the cluster for which it is +obtaining credentials, the `provideClusterInfo` field can be set on the exec user +field in the [kubeconfig](/docs/concepts/configuration/organize-cluster-access-kubeconfig/). +The plugin will then be supplied with an environment variable, `KUBERNETES_EXEC_INFO`. +Information from this environment variable can be used to perform cluster-specific +credential acquisition logic. Here is an example of the aforementioned +`KUBERNETES_EXEC_INFO` environment variable. ```json { From c398ae4821c05b41dce4c2c216f6d856bb7e55af Mon Sep 17 00:00:00 2001 From: Oliver L Schoenborn Date: Tue, 12 Jan 2021 19:57:21 -0500 Subject: [PATCH 022/190] Update content/en/docs/tasks/configure-pod-container/configure-pod-configmap.md Co-authored-by: Tim Bannister --- .../tasks/configure-pod-container/configure-pod-configmap.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/en/docs/tasks/configure-pod-container/configure-pod-configmap.md b/content/en/docs/tasks/configure-pod-container/configure-pod-configmap.md index 1d22c2f3e7..40987152e8 100644 --- a/content/en/docs/tasks/configure-pod-container/configure-pod-configmap.md +++ b/content/en/docs/tasks/configure-pod-container/configure-pod-configmap.md @@ -201,7 +201,8 @@ allow.textmode=true how.nice.to.look=fairlyNice ``` -Note that data sources that are not ASCII or UTF-8 will be put in the `binaryData` field of the configmap. Both text and binary data sources can be combined in one configmap, but the `binaryData` field is not displayed (as of kubectl 1.20) by `kubectl describe`. To see the `binaryData` in the configmap, you can use `kubectl get configmap -o json MAP_NAME`. +When `kubectl` creates a ConfigMap from inputs that are not ASCII or UTF-8, the tool puts these into the `binaryData` field of the ConfigMap, and not in `data`. Both text and binary data sources can be combined in one ConfigMap. +If you want to view the `binaryData` keys (and their values) in a ConfigMap, you can run `kubectl get configmap -o jsonpath='{.binaryData}' `. Use the option `--from-env-file` to create a ConfigMap from an env-file, for example: @@ -689,4 +690,3 @@ data: * Follow a real world example of [Configuring Redis using a ConfigMap](/docs/tutorials/configuration/configure-redis-using-configmap/). - From 8589a9c76a3803af5bc9187f2dd409740d553e20 Mon Sep 17 00:00:00 2001 From: Anastas Dancha Date: Thu, 14 Jan 2021 16:12:53 +0300 Subject: [PATCH 023/190] Update cheatsheet.md - improve on [pr #26053](https://github.com/kubernetes/website/pull/26053) - use idiomatic secret name in example --- content/en/docs/reference/kubectl/cheatsheet.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/reference/kubectl/cheatsheet.md b/content/en/docs/reference/kubectl/cheatsheet.md index 2ed62c2e2a..8fcaa32551 100644 --- a/content/en/docs/reference/kubectl/cheatsheet.md +++ b/content/en/docs/reference/kubectl/cheatsheet.md @@ -195,7 +195,7 @@ JSONPATH='{range .items[*]}{@.metadata.name}:{range @.status.conditions[*]}{@.ty && kubectl get nodes -o jsonpath="$JSONPATH" | grep "Ready=True" # Output decoded secrets without external tools -kubectl get secret ${secret_name} -o go-template='{{range $k,$v := .data}}{{$k}}={{$v|base64decode}}{{"\n"}}{{end}}' +kubectl get secret my-secret -o go-template='{{range $k,$v := .data}}{{"### "}}{{$k}}{{"\n"}}{{$v|base64decode}}{{"\n\n"}}{{end}}' # List all Secrets currently in use by a pod kubectl get pods -o json | jq '.items[].spec.containers[].env[]?.valueFrom.secretKeyRef.name' | grep -v null | sort | uniq From 09cdaac0c8161a25e5c9336bb3d4092ad1828d6d Mon Sep 17 00:00:00 2001 From: Bob Killen Date: Tue, 12 Jan 2021 19:42:52 -0500 Subject: [PATCH 024/190] Add Kubernetes project values to community page --- content/en/community/_index.html | 23 +++++++++++++-- .../en/community/static/community-values.md | 28 +++++++++++++++++++ content/en/community/values.md | 13 +++++++++ static/_redirects | 1 + 4 files changed, 63 insertions(+), 2 deletions(-) create mode 100644 content/en/community/static/community-values.md create mode 100644 content/en/community/values.md diff --git a/content/en/community/_index.html b/content/en/community/_index.html index e1ebb9e9cb..ad9cab5d94 100644 --- a/content/en/community/_index.html +++ b/content/en/community/_index.html @@ -19,6 +19,7 @@ cid: community
+Community Values      Code of conduct       Videos      Discussions      @@ -41,10 +42,28 @@ cid: community Kubernetes Conference Gallery
Kubernetes Conference Gallery - - +
+
+
+
+

+

+

Community Values

+The Kubernetes Community values are the keystone to the ongoing success of the project.
+These principles guide every aspect of the Kubernetes project. +
+ +

+ + READ MORE + + +
+
+
+
diff --git a/content/en/community/static/community-values.md b/content/en/community/static/community-values.md new file mode 100644 index 0000000000..f6469a3e61 --- /dev/null +++ b/content/en/community/static/community-values.md @@ -0,0 +1,28 @@ + + +# Kubernetes Community Values + +Kubernetes Community culture is frequently cited as a substantial contributor to the meteoric rise of this Open Source project. Below are the distilled values which have evolved over the last many years in our community pushing our project and peers toward constant improvement. + +## Distribution is better than centralization + +The scale of the Kubernetes project is only viable through high-trust and high-visibility distribution of work, which includes delegation of authority, decision making, technical design, code ownership, and documentation. Distributed asynchronous ownership, collaboration, communication and decision making are the cornerstone of our world-wide community. + +## Community over product or company + +We are here as a community first, our allegiance is to the intentional stewardship of the Kubernetes project for the benefit of all its members and users everywhere. We support working together publicly for the common goal of a vibrant interoperable ecosystem providing an excellent experience for our users. Individuals gain status through work, companies gain status through their commitments to support this community and fund the resources necessary for the project to operate. + +## Automation over process + +Large projects have a lot of less exciting, yet, hard work. We value time spent automating repetitive work more highly than toil. Where that work cannot be automated, it is our culture to recognize and reward all types of contributions. However, heroism is not sustainable. + +## Inclusive is better than exclusive + +Broadly successful and useful technology requires different perspectives and skill sets which can only be heard in a welcoming and respectful environment. Community membership is a privilege, not a right. Community Leadership is earned through effort, scope, quality, quantity, and duration of contributions. Our community shows respect for the time and effort put into a discussion regardless of where a contributor is on their growth path. + +## Evolution is better than stagnation + +Openness to new ideas and studied technological evolution make Kubernetes a stronger project. Continual improvement, servant leadership, mentorship and respect are the foundations of the Kubernetes project culture. It is the duty for leaders in the Kubernetes community to find, sponsor, and promote new community members. Leaders should expect to step aside. Community members should expect to step up. + +**"Culture eats strategy for breakfast." --Peter Drucker** diff --git a/content/en/community/values.md b/content/en/community/values.md new file mode 100644 index 0000000000..4ae1fe30b6 --- /dev/null +++ b/content/en/community/values.md @@ -0,0 +1,13 @@ +--- +title: Community +layout: basic +cid: community +css: /css/community.css +--- + +
+ +
+{{< include "/static/community-values.md" >}} +
+
diff --git a/static/_redirects b/static/_redirects index d6626accc0..cdcc8e01b2 100644 --- a/static/_redirects +++ b/static/_redirects @@ -466,6 +466,7 @@ /docs/admin/authorization/ /docs/reference/access-authn-authz/authorization/ 301 /docs/admin/high-availability/building/ /docs/setup/production-environment/tools/kubeadm/high-availability/ 301 /code-of-conduct/ /community/code-of-conduct/ 301 +/values/ /community/values/ 302 /docs/setup/version-skew-policy/ /docs/setup/release/version-skew-policy/ 301 From 298432dc36934f1c13a7592a8a0a8943f4124baa Mon Sep 17 00:00:00 2001 From: andrzejsydor Date: Fri, 15 Jan 2021 10:01:45 +0100 Subject: [PATCH 025/190] Add example of using `xargs` Add example of using `xargs` --- .../en/docs/concepts/cluster-administration/manage-deployment.md | 1 + 1 file changed, 1 insertion(+) diff --git a/content/en/docs/concepts/cluster-administration/manage-deployment.md b/content/en/docs/concepts/cluster-administration/manage-deployment.md index 50ed69ff42..8364b4f81c 100644 --- a/content/en/docs/concepts/cluster-administration/manage-deployment.md +++ b/content/en/docs/concepts/cluster-administration/manage-deployment.md @@ -91,6 +91,7 @@ Because `kubectl` outputs resource names in the same syntax it accepts, it's eas ```shell kubectl get $(kubectl create -f docs/concepts/cluster-administration/nginx/ -o name | grep service) +kubectl create -f docs/concepts/cluster-administration/nginx/ -o name | grep service | xargs -i kubectl get {} ``` ```shell From 06228c06a9d7bd1722a9b85178cf6aee51c9258d Mon Sep 17 00:00:00 2001 From: Qiming Teng Date: Mon, 26 Oct 2020 22:13:24 +0800 Subject: [PATCH 026/190] Fix examples tests This PR updates the go.mod file so that tests of the example manifests are run against the 1.20 branch. The missing test cases for newly added examples are also added. To perform tests on your local machine, run the following command on the root of your local clone: ``` go test k8s.io/website/content/en/examples ``` --- content/en/examples/examples_test.go | 96 +++-- go.mod | 56 +-- go.sum | 527 +++++++++++++++++++++++++++ 3 files changed, 624 insertions(+), 55 deletions(-) diff --git a/content/en/examples/examples_test.go b/content/en/examples/examples_test.go index 012a2acaa7..982ddbd693 100644 --- a/content/en/examples/examples_test.go +++ b/content/en/examples/examples_test.go @@ -148,6 +148,11 @@ func getCodecForObject(obj runtime.Object) (runtime.Codec, error) { } func validateObject(obj runtime.Object) (errors field.ErrorList) { + podValidationOptions := validation.PodValidationOptions{ + AllowMultipleHugePageResources: true, + AllowDownwardAPIHugePages: true, + } + // Enable CustomPodDNS for testing // feature.DefaultFeatureGate.Set("CustomPodDNS=true") switch t := obj.(type) { @@ -182,7 +187,7 @@ func validateObject(obj runtime.Object) (errors field.ErrorList) { opts := validation.PodValidationOptions{ AllowMultipleHugePageResources: true, } - errors = validation.ValidatePod(t, opts) + errors = validation.ValidatePodCreate(t, opts) case *api.PodList: for i := range t.Items { errors = append(errors, validateObject(&t.Items[i])...) @@ -191,12 +196,12 @@ func validateObject(obj runtime.Object) (errors field.ErrorList) { if t.Namespace == "" { t.Namespace = api.NamespaceDefault } - errors = validation.ValidatePodTemplate(t) + errors = validation.ValidatePodTemplate(t, podValidationOptions) case *api.ReplicationController: if t.Namespace == "" { t.Namespace = api.NamespaceDefault } - errors = validation.ValidateReplicationController(t) + errors = validation.ValidateReplicationController(t, podValidationOptions) case *api.ReplicationControllerList: for i := range t.Items { errors = append(errors, validateObject(&t.Items[i])...) @@ -215,7 +220,11 @@ func validateObject(obj runtime.Object) (errors field.ErrorList) { if t.Namespace == "" { t.Namespace = api.NamespaceDefault } - errors = validation.ValidateService(t, true) + // handle clusterIPs, logic copied from service strategy + if len(t.Spec.ClusterIP) > 0 && len(t.Spec.ClusterIPs) == 0 { + t.Spec.ClusterIPs = []string{t.Spec.ClusterIP} + } + errors = validation.ValidateService(t) case *api.ServiceAccount: if t.Namespace == "" { t.Namespace = api.NamespaceDefault @@ -250,12 +259,12 @@ func validateObject(obj runtime.Object) (errors field.ErrorList) { if t.Namespace == "" { t.Namespace = api.NamespaceDefault } - errors = apps_validation.ValidateDaemonSet(t) + errors = apps_validation.ValidateDaemonSet(t, podValidationOptions) case *apps.Deployment: if t.Namespace == "" { t.Namespace = api.NamespaceDefault } - errors = apps_validation.ValidateDeployment(t) + errors = apps_validation.ValidateDeployment(t, podValidationOptions) case *networking.Ingress: if t.Namespace == "" { t.Namespace = api.NamespaceDefault @@ -265,18 +274,30 @@ func validateObject(obj runtime.Object) (errors field.ErrorList) { Version: legacyscheme.Scheme.PrioritizedVersionsForGroup(networking.GroupName)[0].Version, } errors = networking_validation.ValidateIngressCreate(t, gv) + case *networking.IngressClass: + /* + if t.Namespace == "" { + t.Namespace = api.NamespaceDefault + } + gv := schema.GroupVersion{ + Group: networking.GroupName, + Version: legacyscheme.Scheme.PrioritizedVersionsForGroup(networking.GroupName)[0].Version, + } + */ + errors = networking_validation.ValidateIngressClass(t) + case *policy.PodSecurityPolicy: errors = policy_validation.ValidatePodSecurityPolicy(t) case *apps.ReplicaSet: if t.Namespace == "" { t.Namespace = api.NamespaceDefault } - errors = apps_validation.ValidateReplicaSet(t) + errors = apps_validation.ValidateReplicaSet(t, podValidationOptions) case *batch.CronJob: if t.Namespace == "" { t.Namespace = api.NamespaceDefault } - errors = batch_validation.ValidateCronJob(t) + errors = batch_validation.ValidateCronJob(t, podValidationOptions) case *networking.NetworkPolicy: if t.Namespace == "" { t.Namespace = api.NamespaceDefault @@ -287,6 +308,9 @@ func validateObject(obj runtime.Object) (errors field.ErrorList) { t.Namespace = api.NamespaceDefault } errors = policy_validation.ValidatePodDisruptionBudget(t) + case *rbac.ClusterRole: + // clusterole does not accept namespace + errors = rbac_validation.ValidateClusterRole(t) case *rbac.ClusterRoleBinding: // clusterolebinding does not accept namespace errors = rbac_validation.ValidateClusterRoleBinding(t) @@ -414,6 +438,7 @@ func TestExampleObjectSchemas(t *testing.T) { "storagelimits": {&api.LimitRange{}}, }, "admin/sched": { + "clusterrole": {&rbac.ClusterRole{}}, "my-scheduler": {&api.ServiceAccount{}, &rbac.ClusterRoleBinding{}, &rbac.ClusterRoleBinding{}, &apps.Deployment{}}, "pod1": {&api.Pod{}}, "pod2": {&api.Pod{}}, @@ -539,6 +564,7 @@ func TestExampleObjectSchemas(t *testing.T) { "dapi-envars-pod": {&api.Pod{}}, "dapi-volume": {&api.Pod{}}, "dapi-volume-resources": {&api.Pod{}}, + "dependent-envars": {&api.Pod{}}, "envars": {&api.Pod{}}, "pod-multiple-secret-env-variable": {&api.Pod{}}, "pod-secret-envFrom": {&api.Pod{}}, @@ -596,29 +622,41 @@ func TestExampleObjectSchemas(t *testing.T) { "load-balancer-example": {&apps.Deployment{}}, }, "service/access": { - "frontend": {&api.Service{}, &apps.Deployment{}}, - "hello-application": {&apps.Deployment{}}, - "hello-service": {&api.Service{}}, - "hello": {&apps.Deployment{}}, + "backend-deployment": {&apps.Deployment{}}, + "backend-service": {&api.Service{}}, + "frontend-deployment": {&apps.Deployment{}}, + "frontend-service": {&api.Service{}}, + "hello-application": {&apps.Deployment{}}, }, "service/networking": { - "curlpod": {&apps.Deployment{}}, - "custom-dns": {&api.Pod{}}, - "dual-stack-default-svc": {&api.Service{}}, - "dual-stack-ipv4-svc": {&api.Service{}}, - "dual-stack-ipv6-lb-svc": {&api.Service{}}, - "dual-stack-ipv6-svc": {&api.Service{}}, - "hostaliases-pod": {&api.Pod{}}, - "ingress": {&networking.Ingress{}}, - "network-policy-allow-all-egress": {&networking.NetworkPolicy{}}, - "network-policy-allow-all-ingress": {&networking.NetworkPolicy{}}, - "network-policy-default-deny-egress": {&networking.NetworkPolicy{}}, - "network-policy-default-deny-ingress": {&networking.NetworkPolicy{}}, - "network-policy-default-deny-all": {&networking.NetworkPolicy{}}, - "nginx-policy": {&networking.NetworkPolicy{}}, - "nginx-secure-app": {&api.Service{}, &apps.Deployment{}}, - "nginx-svc": {&api.Service{}}, - "run-my-nginx": {&apps.Deployment{}}, + "curlpod": {&apps.Deployment{}}, + "custom-dns": {&api.Pod{}}, + "dual-stack-default-svc": {&api.Service{}}, + "dual-stack-ipfamilies-ipv6": {&api.Service{}}, + "dual-stack-ipv6-svc": {&api.Service{}}, + "dual-stack-prefer-ipv6-lb-svc": {&api.Service{}}, + "dual-stack-preferred-ipfamilies-svc": {&api.Service{}}, + "dual-stack-preferred-svc": {&api.Service{}}, + "external-lb": {&networking.IngressClass{}}, + "example-ingress": {&networking.Ingress{}}, + "hostaliases-pod": {&api.Pod{}}, + "ingress-resource-backend": {&networking.Ingress{}}, + "ingress-wildcard-host": {&networking.Ingress{}}, + "minimal-ingress": {&networking.Ingress{}}, + "name-virtual-host-ingress": {&networking.Ingress{}}, + "name-virtual-host-ingress-no-third-host": {&networking.Ingress{}}, + "network-policy-allow-all-egress": {&networking.NetworkPolicy{}}, + "network-policy-allow-all-ingress": {&networking.NetworkPolicy{}}, + "network-policy-default-deny-egress": {&networking.NetworkPolicy{}}, + "network-policy-default-deny-ingress": {&networking.NetworkPolicy{}}, + "network-policy-default-deny-all": {&networking.NetworkPolicy{}}, + "nginx-policy": {&networking.NetworkPolicy{}}, + "nginx-secure-app": {&api.Service{}, &apps.Deployment{}}, + "nginx-svc": {&api.Service{}}, + "run-my-nginx": {&apps.Deployment{}}, + "simple-fanout-example": {&networking.Ingress{}}, + "test-ingress": {&networking.Ingress{}}, + "tls-example-ingress": {&networking.Ingress{}}, }, "windows": { "configmap-pod": {&api.ConfigMap{}, &api.Pod{}}, diff --git a/go.mod b/go.mod index 30c6741140..b45ff242a4 100644 --- a/go.mod +++ b/go.mod @@ -1,34 +1,38 @@ module k8s.io/website -go 1.14 +go 1.15 require ( - k8s.io/apimachinery v0.18.4 - k8s.io/kubernetes v1.18.4 + k8s.io/apimachinery v0.20.0 + k8s.io/kubernetes v1.20.0 ) replace ( - k8s.io/api => k8s.io/api v0.18.4 - k8s.io/apiextensions-apiserver => k8s.io/apiextensions-apiserver v0.18.4 - k8s.io/apimachinery => k8s.io/apimachinery v0.18.4 - k8s.io/apiserver => k8s.io/apiserver v0.18.4 - k8s.io/cli-runtime => k8s.io/cli-runtime v0.18.4 - k8s.io/client-go => k8s.io/client-go v0.18.4 - k8s.io/cloud-provider => k8s.io/cloud-provider v0.18.4 - k8s.io/cluster-bootstrap => k8s.io/cluster-bootstrap v0.18.4 - k8s.io/code-generator => k8s.io/code-generator v0.18.4 - k8s.io/component-base => k8s.io/component-base v0.18.4 - k8s.io/cri-api => k8s.io/cri-api v0.18.4 - k8s.io/csi-translation-lib => k8s.io/csi-translation-lib v0.18.4 - k8s.io/kube-aggregator => k8s.io/kube-aggregator v0.18.4 - k8s.io/kube-controller-manager => k8s.io/kube-controller-manager v0.18.4 - k8s.io/kube-proxy => k8s.io/kube-proxy v0.18.4 - k8s.io/kube-scheduler => k8s.io/kube-scheduler v0.18.4 - k8s.io/kubectl => k8s.io/kubectl v0.18.4 - k8s.io/kubelet => k8s.io/kubelet v0.18.4 - k8s.io/legacy-cloud-providers => k8s.io/legacy-cloud-providers v0.18.4 - k8s.io/metrics => k8s.io/metrics v0.18.4 - k8s.io/sample-apiserver => k8s.io/sample-apiserver v0.18.4 - k8s.io/sample-cli-plugin => k8s.io/sample-cli-plugin v0.18.4 - k8s.io/sample-controller => k8s.io/sample-controller v0.18.4 + k8s.io/api => k8s.io/api v0.20.0 + k8s.io/apiextensions-apiserver => k8s.io/apiextensions-apiserver v0.20.0 + k8s.io/apimachinery => k8s.io/apimachinery v0.20.0 + k8s.io/apiserver => k8s.io/apiserver v0.20.0 + k8s.io/cli-runtime => k8s.io/cli-runtime v0.20.0 + k8s.io/client-go => k8s.io/client-go v0.20.0 + k8s.io/cloud-provider => k8s.io/cloud-provider v0.20.0 + k8s.io/cluster-bootstrap => k8s.io/cluster-bootstrap v0.20.0 + k8s.io/code-generator => k8s.io/code-generator v0.20.0 + k8s.io/component-base => k8s.io/component-base v0.20.0 + k8s.io/component-helpers => k8s.io/component-helpers v0.20.0 + k8s.io/controller-manager => k8s.io/controller-manager v0.20.0 + k8s.io/cri-api => k8s.io/cri-api v0.20.0 + k8s.io/csi-translation-lib => k8s.io/csi-translation-lib v0.20.0 + k8s.io/kube-aggregator => k8s.io/kube-aggregator v0.20.0 + k8s.io/kube-controller-manager => k8s.io/kube-controller-manager v0.20.0 + k8s.io/kube-proxy => k8s.io/kube-proxy v0.20.0 + k8s.io/kube-scheduler => k8s.io/kube-scheduler v0.20.0 + k8s.io/kubectl => k8s.io/kubectl v0.20.0 + k8s.io/kubelet => k8s.io/kubelet v0.20.0 + k8s.io/kubernetes => k8s.io/kubernetes v1.20.0 + k8s.io/legacy-cloud-providers => k8s.io/legacy-cloud-providers v0.20.0 + k8s.io/metrics => k8s.io/metrics v0.20.0 + k8s.io/mount-utils => k8s.io/mount-utils v0.20.0 + k8s.io/sample-apiserver => k8s.io/sample-apiserver v0.20.0 + k8s.io/sample-cli-plugin => k8s.io/sample-cli-plugin v0.20.0 + k8s.io/sample-controller => k8s.io/sample-controller v0.20.0 ) diff --git a/go.sum b/go.sum index d0f82ad655..723fccaf6a 100644 --- a/go.sum +++ b/go.sum @@ -2,25 +2,66 @@ bitbucket.org/bertimus9/systemstat v0.0.0-20180207000608-0eeff89b0690/go.mod h1: cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw= cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw= cloud.google.com/go v0.38.0/go.mod h1:990N+gfupTy94rShfmMCWGDn0LpTmnzTp2qbd1dvSRU= +cloud.google.com/go v0.44.1/go.mod h1:iSa0KzasP4Uvy3f1mN/7PiObzGgflwredwwASm/v6AU= +cloud.google.com/go v0.44.2/go.mod h1:60680Gw3Yr4ikxnPRS/oxxkBccT6SA1yMk63TGekxKY= +cloud.google.com/go v0.45.1/go.mod h1:RpBamKRgapWJb87xiFSdk4g1CME7QZg3uwTez+TSTjc= +cloud.google.com/go v0.46.3/go.mod h1:a6bKKbmY7er1mI7TEI4lsAkts/mkhTSZK8w33B4RAg0= +cloud.google.com/go v0.50.0/go.mod h1:r9sluTvynVuxRIOHXQEHMFffphuXHOMZMycpNR5e6To= +cloud.google.com/go v0.51.0/go.mod h1:hWtGJ6gnXH+KgDv+V0zFGDvpi07n3z8ZNj3T1RW0Gcw= +cloud.google.com/go v0.52.0/go.mod h1:pXajvRH/6o3+F9jDHZWQ5PbGhn+o8w9qiu/CffaVdO4= +cloud.google.com/go v0.53.0/go.mod h1:fp/UouUEsRkN6ryDKNW/Upv/JBKnv6WDthjR6+vze6M= +cloud.google.com/go v0.54.0/go.mod h1:1rq2OEkV3YMf6n/9ZvGWI3GWw0VoqH/1x2nd8Is/bPc= +cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o= +cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE= +cloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc= +cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE= +cloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk= +cloud.google.com/go/firestore v1.1.0/go.mod h1:ulACoGHTpvq5r8rxGJ4ddJZBZqakUQqClKRT5SZwBmk= +cloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I= +cloud.google.com/go/pubsub v1.1.0/go.mod h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+R3AArQw= +cloud.google.com/go/pubsub v1.2.0/go.mod h1:jhfEVHT8odbXTkndysNHCcx0awwzvfOlguIAii9o8iA= +cloud.google.com/go/storage v1.0.0/go.mod h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiyrjsg+URw= +cloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0ZeosJ0Rtdos= +cloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohlUTyfDhBk= +dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU= github.com/Azure/azure-sdk-for-go v35.0.0+incompatible/go.mod h1:9XXNKU+eRnpl9moKnB4QOLf1HestfXbmab5FXxiDBjc= +github.com/Azure/azure-sdk-for-go v43.0.0+incompatible/go.mod h1:9XXNKU+eRnpl9moKnB4QOLf1HestfXbmab5FXxiDBjc= github.com/Azure/go-ansiterm v0.0.0-20170929234023-d6e3b3328b78/go.mod h1:LmzpDX56iTiv29bbRTIsUNlaFfuhWRQBWjQdVyAevI8= +github.com/Azure/go-autorest v14.2.0+incompatible/go.mod h1:r+4oMnoxhatjLLJ6zxSWATqVooLgysK6ZNox3g/xq24= github.com/Azure/go-autorest/autorest v0.9.0/go.mod h1:xyHB1BMZT0cuDHU7I0+g046+BFDTQ8rEZB0s4Yfa6bI= +github.com/Azure/go-autorest/autorest v0.9.6/go.mod h1:/FALq9T/kS7b5J5qsQ+RSTUdAmGFqi0vUdVNNx8q630= +github.com/Azure/go-autorest/autorest v0.11.1/go.mod h1:JFgpikqFJ/MleTTxwepExTKnFUKKszPS8UavbQYUMuw= github.com/Azure/go-autorest/autorest/adal v0.5.0/go.mod h1:8Z9fGy2MpX0PvDjB1pEgQTmVqjGhiHBW7RJJEciWzS0= +github.com/Azure/go-autorest/autorest/adal v0.8.2/go.mod h1:ZjhuQClTqx435SRJ2iMlOxPYt3d2C/T/7TiQCVZSn3Q= +github.com/Azure/go-autorest/autorest/adal v0.9.0/go.mod h1:/c022QCutn2P7uY+/oQWWNcK9YU+MH96NgK+jErpbcg= +github.com/Azure/go-autorest/autorest/adal v0.9.5/go.mod h1:B7KF7jKIeC9Mct5spmyCB/A8CG/sEz1vwIRGv/bbw7A= github.com/Azure/go-autorest/autorest/date v0.1.0/go.mod h1:plvfp3oPSKwf2DNjlBjWF/7vwR+cUD/ELuzDCXwHUVA= +github.com/Azure/go-autorest/autorest/date v0.2.0/go.mod h1:vcORJHLJEh643/Ioh9+vPmf1Ij9AEBM5FuBIXLmIy0g= +github.com/Azure/go-autorest/autorest/date v0.3.0/go.mod h1:BI0uouVdmngYNUzGWeSYnokU+TrmwEsOqdt8Y6sso74= github.com/Azure/go-autorest/autorest/mocks v0.1.0/go.mod h1:OTyCOPRA2IgIlWxVYxBee2F5Gr4kF2zd2J5cFRaIDN0= github.com/Azure/go-autorest/autorest/mocks v0.2.0/go.mod h1:OTyCOPRA2IgIlWxVYxBee2F5Gr4kF2zd2J5cFRaIDN0= +github.com/Azure/go-autorest/autorest/mocks v0.3.0/go.mod h1:a8FDP3DYzQ4RYfVAxAN3SVSiiO77gL2j2ronKKP0syM= +github.com/Azure/go-autorest/autorest/mocks v0.4.0/go.mod h1:LTp+uSrOhSkaKrUy935gNZuuIPPVsHlr9DSOxSayd+k= +github.com/Azure/go-autorest/autorest/mocks v0.4.1/go.mod h1:LTp+uSrOhSkaKrUy935gNZuuIPPVsHlr9DSOxSayd+k= github.com/Azure/go-autorest/autorest/to v0.2.0/go.mod h1:GunWKJp1AEqgMaGLV+iocmRAJWqST1wQYhyyjXJ3SJc= github.com/Azure/go-autorest/autorest/validation v0.1.0/go.mod h1:Ha3z/SqBeaalWQvokg3NZAlQTalVMtOIAs1aGK7G6u8= github.com/Azure/go-autorest/logger v0.1.0/go.mod h1:oExouG+K6PryycPJfVSxi/koC6LSNgds39diKLz7Vrc= +github.com/Azure/go-autorest/logger v0.2.0/go.mod h1:T9E3cAhj2VqvPOtCYAvby9aBXkZmbF5NWuPV8+WeEW8= github.com/Azure/go-autorest/tracing v0.5.0/go.mod h1:r/s2XiOKccPW3HrqB+W0TQzfbtp2fGCgRFtBroKn4Dk= +github.com/Azure/go-autorest/tracing v0.6.0/go.mod h1:+vhtPC754Xsa23ID7GlGsrdKBpUA79WCAKPPZVC2DeU= github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU= github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo= github.com/GoogleCloudPlatform/k8s-cloud-provider v0.0.0-20190822182118-27a4ced34534/go.mod h1:iroGtC8B3tQiqtds1l+mgk/BBOrxbqjH+eUfFQYRc14= +github.com/GoogleCloudPlatform/k8s-cloud-provider v0.0.0-20200415212048-7901bc822317/go.mod h1:DF8FZRxMHMGv/vP2lQP6h+dYzzjpuRn24VeRiYn3qjQ= github.com/JeffAshton/win_pdh v0.0.0-20161109143554-76bb4ee9f0ab/go.mod h1:3VYc5hodBMJ5+l/7J4xAyMeuM2PNuepvHlGs8yilUCA= github.com/MakeNowJust/heredoc v0.0.0-20170808103936-bb23615498cd/go.mod h1:64YHyfSL2R96J44Nlwm39UHepQbyR5q10x7iYa1ks2E= github.com/Microsoft/go-winio v0.4.14/go.mod h1:qXqCSQ3Xa7+6tgxaGTIe4Kpcdsi+P8jBhyzoq1bpyYA= +github.com/Microsoft/go-winio v0.4.15-0.20190919025122-fc70bd9a86b5/go.mod h1:tTuCMEN+UleMWgg9dVx4Hu52b1bJo+59jBh3ajtinzw= +github.com/Microsoft/go-winio v0.4.15/go.mod h1:tTuCMEN+UleMWgg9dVx4Hu52b1bJo+59jBh3ajtinzw= github.com/Microsoft/hcsshim v0.0.0-20190417211021-672e52e9209d/go.mod h1:Op3hHsoHPAvb6lceZHDtd9OkTew38wNoXnJs8iY7rUg= +github.com/Microsoft/hcsshim v0.8.10-0.20200715222032-5eafd1556990/go.mod h1:ay/0dTb7NsG8QMDfsRfLHgZo/6xAJShLe1+ePPflihk= github.com/NYTimes/gziphandler v0.0.0-20170623195520-56545f4a5d46/go.mod h1:3wb06e3pkSAbeQ52E9H9iFoQsEEwGN64994WTCIhntQ= +github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU= github.com/OpenPeeDeeP/depguard v1.0.0/go.mod h1:7/4sitnI9YlQgTLLk734QlzXT8DuHVnAyztLplQjk+o= github.com/OpenPeeDeeP/depguard v1.0.1/go.mod h1:xsIw86fROiiwelg+jB2uM9PiKihMMmUx/1V+TNhjQvM= github.com/PuerkitoBio/purell v1.0.0/go.mod h1:c11w/QuzBsJSee3cPx9rAFu61PvFxuPbtSwDGJws/X0= @@ -33,15 +74,21 @@ github.com/StackExchange/wmi v0.0.0-20180116203802-5d049714c4a6/go.mod h1:3eOhrU github.com/agnivade/levenshtein v1.0.1/go.mod h1:CURSv5d9Uaml+FovSIICkLbAUZ9S4RqaHDIsdSBg7lM= github.com/ajstarks/svgo v0.0.0-20180226025133-644b8db467af/go.mod h1:K08gAheRH3/J6wwsYMMT4xOr94bZjxIelGM0+d/wbFw= github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= +github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= +github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883/go.mod h1:rCTlJbsFo29Kk6CurOXKm700vrz8f0KW0JNfpkRJY/8= github.com/anmitsu/go-shlex v0.0.0-20161002113705-648efa622239/go.mod h1:2FmKhYUyUczH0OGQWaF5ceTx0UBShxjsH6f8oGKYe2c= github.com/armon/circbuf v0.0.0-20150827004946-bbbad097214e/go.mod h1:3U/XgcO3hCbHZ8TKRvWD2dDTCfh9M9ya+I9JpbB7O8o= github.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6/go.mod h1:grANhF5doyWs3UAsr3K4I6qtAmlQcZDesFNEHPZAzj8= +github.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da/go.mod h1:Q73ZrmVTwzkszR9V5SSuryQ31EELlFMUz1kKyl939pY= +github.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310/go.mod h1:ufUuZ+zHj4x4TnLV4JWEpy2hxWSpsRywHrMgIH9cCH8= github.com/asaskevich/govalidator v0.0.0-20180720115003-f9ffefc3facf/go.mod h1:lB+ZfQJz7igIIfQNfa7Ml4HSf2uFQQRzpGGRXenZAgY= github.com/asaskevich/govalidator v0.0.0-20190424111038-f61b66f89f4a/go.mod h1:lB+ZfQJz7igIIfQNfa7Ml4HSf2uFQQRzpGGRXenZAgY= github.com/auth0/go-jwt-middleware v0.0.0-20170425171159-5493cabe49f7/go.mod h1:LWMyo4iOLWXHGdBki7NIht1kHru/0wM179h+d3g8ATM= +github.com/aws/aws-sdk-go v1.6.10/go.mod h1:ZRmQr0FajVIyZ4ZzBYKG5P3ZqPz9IHG41ZoMu1ADI3k= github.com/aws/aws-sdk-go v1.28.2/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo= +github.com/aws/aws-sdk-go v1.35.24/go.mod h1:tlPOdRjfxPBpNIwqDj61rmsnA85v9jc0Ps9+muhnW+k= github.com/bazelbuild/bazel-gazelle v0.18.2/go.mod h1:D0ehMSbS+vesFsLGiD6JXu3mVEzOlfUl8wNnq+x/9p0= github.com/bazelbuild/bazel-gazelle v0.19.1-0.20191105222053-70208cbdc798/go.mod h1:rPwzNHUqEzngx1iVBfO/2X2npKaT3tqPqqHW6rVsn/A= github.com/bazelbuild/buildtools v0.0.0-20190731111112-f720930ceb60/go.mod h1:5JP0TXzWDHXv8qvxRC4InIazwdyDseBDbzESUMKk1yU= @@ -50,32 +97,70 @@ github.com/bazelbuild/rules_go v0.0.0-20190719190356-6dae44dc5cab/go.mod h1:MC23 github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q= github.com/beorn7/perks v1.0.0 h1:HWo1m869IqiPhD389kmkxeTalrjNbbJTC8LXupb+sl0= github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8= +github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= +github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= github.com/bgentry/speakeasy v0.1.0/go.mod h1:+zsyZBPWlz7T6j88CTgSN5bM796AkVf0kBD4zp0CCIs= github.com/bifurcation/mint v0.0.0-20180715133206-93c51c6ce115/go.mod h1:zVt7zX3K/aDCk9Tj+VM7YymsX66ERvzCJzw8rFCX2JU= +github.com/bketelsen/crypt v0.0.3-0.20200106085610-5cbc8cc4026c/go.mod h1:MKsuJmJgSg28kpZDP6UIiPt0e0Oz0kqKNGyRaWEPv84= +github.com/blang/semver v3.1.0+incompatible/go.mod h1:kRBLl5iJ+tD4TcOOxsy/0fnwebNt5EWlYSAyrTnjyyk= github.com/blang/semver v3.5.0+incompatible h1:CGxCgetQ64DKk7rdZ++Vfnb1+ogGNnB17OJKJXD2Cfs= github.com/blang/semver v3.5.0+incompatible/go.mod h1:kRBLl5iJ+tD4TcOOxsy/0fnwebNt5EWlYSAyrTnjyyk= +github.com/blang/semver v3.5.1+incompatible h1:cQNTCjp13qL8KC3Nbxr/y2Bqb63oX6wdnnjpJbkM4JQ= +github.com/blang/semver v3.5.1+incompatible/go.mod h1:kRBLl5iJ+tD4TcOOxsy/0fnwebNt5EWlYSAyrTnjyyk= github.com/boltdb/bolt v1.3.1/go.mod h1:clJnj/oiGkjum5o1McbSZDSLxVThjynRyGBgiAx27Ps= github.com/bradfitz/go-smtpd v0.0.0-20170404230938-deb6d6237625/go.mod h1:HYsPBTaaSFSlLx/70C2HPIMNZpVV8+vt/A+FMnYP11g= github.com/caddyserver/caddy v1.0.3/go.mod h1:G+ouvOY32gENkJC+jhgl62TyhvqEsFaDiZ4uw0RzP1E= github.com/cenkalti/backoff v2.1.1+incompatible/go.mod h1:90ReRw6GdpyfrHakVjL/QHaoyV4aDUVVkXQJJJ3NXXM= github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU= github.com/cespare/prettybench v0.0.0-20150116022406-03b8cfe5406c/go.mod h1:Xe6ZsFhtM8HrDku0pxJ3/Lr51rwykrzgFwpmTzleatY= +github.com/cespare/xxhash v1.1.0 h1:a6HrQnmkObjyL+Gs60czilIUGqrzKutQD6XZog3p+ko= +github.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc= +github.com/cespare/xxhash/v2 v2.1.1 h1:6MnRN8NT7+YBpUIWxHtefFZOKTAPgGjpQSxqLNn0+qY= +github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= github.com/chai2010/gettext-go v0.0.0-20160711120539-c6fed771bfd5/go.mod h1:/iP1qXHoty45bqomnu2LM+VVyAEdWN+vtSHGlQgyxbw= github.com/checkpoint-restore/go-criu v0.0.0-20181120144056-17b0214f6c48/go.mod h1:TrMrLQfeENAPYPRsJuq3jsqdlRh3lvi6trTZJG8+tho= +github.com/checkpoint-restore/go-criu/v4 v4.0.2/go.mod h1:xUQBLp4RLc5zJtWY++yjOoMoB5lihDt7fai+75m+rGw= +github.com/checkpoint-restore/go-criu/v4 v4.1.0/go.mod h1:xUQBLp4RLc5zJtWY++yjOoMoB5lihDt7fai+75m+rGw= github.com/cheekybits/genny v0.0.0-20170328200008-9127e812e1e9/go.mod h1:+tQajlRqAUrPI7DOSpB0XAqZYtQakVtB7wXkRAgjxjQ= +github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI= +github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI= +github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU= github.com/cilium/ebpf v0.0.0-20191025125908-95b36a581eed/go.mod h1:MA5e5Lr8slmEg9bt0VpxxWqJlO4iwu3FBdHUzV7wQVg= +github.com/cilium/ebpf v0.0.0-20200110133405-4032b1d8aae3/go.mod h1:MA5e5Lr8slmEg9bt0VpxxWqJlO4iwu3FBdHUzV7wQVg= +github.com/cilium/ebpf v0.0.0-20200507155900-a9f01edf17e3/go.mod h1:XT+cAw5wfvsodedcijoh1l9cf7v1x9FlFB/3VmF/O8s= +github.com/cilium/ebpf v0.0.0-20200601085316-9f1617e5c574/go.mod h1:XT+cAw5wfvsodedcijoh1l9cf7v1x9FlFB/3VmF/O8s= +github.com/cilium/ebpf v0.0.0-20200702112145-1c8d4c9ef775/go.mod h1:7cR51M8ViRLIdUjrmSXlK9pkrsDlLHbO8jiB8X8JnOc= github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw= github.com/clusterhq/flocker-go v0.0.0-20160920122132-2b8b7259d313/go.mod h1:P1wt9Z3DP8O6W3rvwCt0REIlshg1InHImaLW0t3ObY0= github.com/cockroachdb/datadriven v0.0.0-20190809214429-80d97fb3cbaa h1:OaNxuTZr7kxeODyLWsRMC+OD03aFUH+mW6r2d+MWa5Y= github.com/cockroachdb/datadriven v0.0.0-20190809214429-80d97fb3cbaa/go.mod h1:zn76sxSg3SzpJ0PPJaLDCu+Bu0Lg3sKTORVIj19EIF8= github.com/codegangsta/negroni v1.0.0/go.mod h1:v0y3T5G7Y1UlFfyxFn/QLRU4a2EuNau2iZY63YTKWo0= github.com/container-storage-interface/spec v1.2.0/go.mod h1:6URME8mwIBbpVyZV93Ce5St17xBiQJQY67NDsuohiy4= +github.com/containerd/cgroups v0.0.0-20200531161412-0dbf7f05ba59/go.mod h1:pA0z1pT8KYB3TCXK/ocprsh7MAkoW8bZVzPdih9snmM= github.com/containerd/console v0.0.0-20170925154832-84eeaae905fa/go.mod h1:Tj/on1eG8kiEhd0+fhSDzsPAFESxzBBvdyEgyryXffw= +github.com/containerd/console v0.0.0-20180822173158-c12b1e7919c1/go.mod h1:Tj/on1eG8kiEhd0+fhSDzsPAFESxzBBvdyEgyryXffw= +github.com/containerd/console v1.0.0/go.mod h1:8Pf4gM6VEbTNRIT26AyyU7hxdQU3MvAvxVI0sc00XBE= github.com/containerd/containerd v1.0.2/go.mod h1:bC6axHOhabU15QhwfG7w5PipXdVtMXFTttgp+kVtyUA= +github.com/containerd/containerd v1.3.2/go.mod h1:bC6axHOhabU15QhwfG7w5PipXdVtMXFTttgp+kVtyUA= +github.com/containerd/containerd v1.3.3/go.mod h1:bC6axHOhabU15QhwfG7w5PipXdVtMXFTttgp+kVtyUA= +github.com/containerd/containerd v1.4.1/go.mod h1:bC6axHOhabU15QhwfG7w5PipXdVtMXFTttgp+kVtyUA= +github.com/containerd/continuity v0.0.0-20190426062206-aaeac12a7ffc/go.mod h1:GL3xCUCBDV3CZiTSEKksMWbLE66hEyuu9qyDOOqM47Y= +github.com/containerd/fifo v0.0.0-20190226154929-a9fb20d87448/go.mod h1:ODA38xgv3Kuk8dQz2ZQXpnv/UZZUHUCL7pnLehbXgQI= +github.com/containerd/go-runc v0.0.0-20180907222934-5a6d9f37cfa3/go.mod h1:IV7qH3hrUgRmyYrtgEeGWJfWbgcHL9CSRruz2Vqcph0= +github.com/containerd/ttrpc v0.0.0-20190828154514-0e0f228740de/go.mod h1:PvCDdDGpgqzQIzDW1TphrGLssLDZp2GuS+X5DkEJB8o= +github.com/containerd/ttrpc v1.0.0/go.mod h1:PvCDdDGpgqzQIzDW1TphrGLssLDZp2GuS+X5DkEJB8o= +github.com/containerd/ttrpc v1.0.2/go.mod h1:UAxOpgT9ziI0gJrmKvgcZivgxOp8iFPSk8httJEt98Y= +github.com/containerd/typeurl v0.0.0-20180627222232-a93fcdb778cd/go.mod h1:Cm3kwCdlkCfMSHURc+r6fwoGH6/F1hH3S4sg0rLFWPc= github.com/containerd/typeurl v0.0.0-20190228175220-2a93cfde8c20/go.mod h1:Cm3kwCdlkCfMSHURc+r6fwoGH6/F1hH3S4sg0rLFWPc= +github.com/containerd/typeurl v1.0.0/go.mod h1:Cm3kwCdlkCfMSHURc+r6fwoGH6/F1hH3S4sg0rLFWPc= +github.com/containerd/typeurl v1.0.1/go.mod h1:TB1hUtrpaiO88KEK56ijojHS1+NeF0izUACaJW2mdXg= github.com/containernetworking/cni v0.7.1/go.mod h1:LGwApLUm2FpoOfxTDEeq8T9ipbpZ61X79hmU3w8FmsY= +github.com/containernetworking/cni v0.8.0/go.mod h1:LGwApLUm2FpoOfxTDEeq8T9ipbpZ61X79hmU3w8FmsY= github.com/coredns/corefile-migration v1.0.6/go.mod h1:OFwBp/Wc9dJt5cAZzHWMNhK1r5L0p0jDwIBc6j8NC8E= +github.com/coredns/corefile-migration v1.0.10/go.mod h1:RMy/mXdeDlYwzt0vdMEJvT2hGJ2I86/eO0UdXmH9XNI= +github.com/coreos/bbolt v1.3.2/go.mod h1:iRUV2dpdMOn7Bo10OQBFzIJO9kkE559Wcmn+qkEiiKk= github.com/coreos/etcd v3.3.10+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE= +github.com/coreos/etcd v3.3.13+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE= github.com/coreos/go-etcd v2.0.0+incompatible/go.mod h1:Jez6KQU2B/sWsbdaef3ED8NzMklzPG4d5KIOhIy30Tk= github.com/coreos/go-oidc v2.1.0+incompatible/go.mod h1:CgnwVTmzoESiwO9qyAFEMiHoZ1nMCKZlZ9V6mm3/LKc= github.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk= @@ -85,10 +170,16 @@ github.com/coreos/go-systemd v0.0.0-20180511133405-39ca1b05acc7/go.mod h1:F5haX7 github.com/coreos/go-systemd v0.0.0-20181012123002-c6f51f82210d/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4= github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e h1:Wf6HqHfScWJN9/ZjdUKyjop4mf3Qdd+1TvvltAvM3m8= github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4= +github.com/coreos/go-systemd/v22 v22.0.0/go.mod h1:xO0FLkIi5MaZafQlIrOotqXZ90ih+1atmu1JpKERPPk= +github.com/coreos/go-systemd/v22 v22.1.0/go.mod h1:xO0FLkIi5MaZafQlIrOotqXZ90ih+1atmu1JpKERPPk= github.com/coreos/pkg v0.0.0-20160727233714-3ac0863d7acf/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA= github.com/coreos/pkg v0.0.0-20180108230652-97fdf19511ea h1:n2Ltr3SrfQlf/9nOna1DoGKxLx3qTSI8Ttl6Xrqp6mw= github.com/coreos/pkg v0.0.0-20180108230652-97fdf19511ea/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA= +github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f h1:lBNOc5arjvs8E5mO2tbpBpLoyyu8B6e44T7hJy6potg= +github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA= github.com/cpuguy83/go-md2man v1.0.10/go.mod h1:SmD6nW6nTyfqj6ABTjUi3V3JVMnlJmwcJI5acqYI6dE= +github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU= +github.com/cpuguy83/go-md2man/v2 v2.0.0/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU= github.com/creack/pty v1.1.7/go.mod h1:lj5s0c3V2DBrqTV7llrYr5NG6My20zk30Fl46Y7DoTY= github.com/cyphar/filepath-securejoin v0.2.2/go.mod h1:FpkQEhXnPnOthhzymB7CGsFk2G9VLXONKD9G7QGMM+4= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= @@ -97,14 +188,20 @@ github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSs github.com/daviddengcn/go-colortext v0.0.0-20160507010035-511bcaf42ccd/go.mod h1:dv4zxwHi5C/8AeI+4gX4dCWOIvNi7I6JCSX0HvlKPgE= github.com/dgrijalva/jwt-go v3.2.0+incompatible h1:7qlOGliEKZXTDg6OTjfoBKDXWrumCAMpl/TFQ4/5kLM= github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ= +github.com/dgryski/go-sip13 v0.0.0-20181026042036-e10d5fee7954/go.mod h1:vAd38F8PWV+bWy6jNmig1y/TA+kYO4g3RSRF0IAv0no= github.com/dnaeon/go-vcr v1.0.1/go.mod h1:aBB1+wY4s93YsC3HHjMBMrwTj2R9FHDzUr9KyGc8n1E= +github.com/docker/distribution v2.7.1-0.20190205005809-0d3efadf0154+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w= github.com/docker/distribution v2.7.1+incompatible h1:a5mlkVzth6W5A4fOsS3D2EO5BUmsJpcB+cRlLU7cSug= github.com/docker/distribution v2.7.1+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w= github.com/docker/docker v0.7.3-0.20190327010347-be7ac8be2ae0/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk= +github.com/docker/docker v1.4.2-0.20200309214505-aa6a9891b09c/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk= +github.com/docker/docker v17.12.0-ce-rc1.0.20200916142827-bd33bbf0497b+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk= github.com/docker/go-connections v0.3.0/go.mod h1:Gbd7IOopHjR8Iph03tsViu4nIes5XhDvyHbTtUxmeec= +github.com/docker/go-connections v0.4.0/go.mod h1:Gbd7IOopHjR8Iph03tsViu4nIes5XhDvyHbTtUxmeec= github.com/docker/go-units v0.3.3/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk= github.com/docker/go-units v0.4.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk= github.com/docker/spdystream v0.0.0-20160310174837-449fdfce4d96/go.mod h1:Qh8CwZgvJUkLughtfhJv5dyTYa91l1fOUCrgjqmcifM= +github.com/docopt/docopt-go v0.0.0-20180111231733-ee0de3bc6815/go.mod h1:WwZ+bS3ebgob9U8Nd0kOddGdZWjyMGR8Wziv+TBNwSE= github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk= github.com/dustin/go-humanize v1.0.0 h1:VSnTsYCnlFHaM2/igO1h6X3HA71jcobQuxemgkq4zYo= github.com/dustin/go-humanize v1.0.0/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk= @@ -115,13 +212,17 @@ github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.m github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c= github.com/euank/go-kmsg-parser v2.0.0+incompatible/go.mod h1:MhmAMZ8V4CYH4ybgdRwPr2TU5ThnS43puaKEMpja1uw= github.com/evanphx/json-patch v4.2.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk= +github.com/evanphx/json-patch v4.9.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk= github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d/go.mod h1:ZZMPRZwes7CROmyNKgQzC3XPs6L/G2EJLHddWejkmf4= github.com/fatih/camelcase v1.0.0/go.mod h1:yN2Sb0lFhZJUdVvtELVWefmrXpuZESvPmqwoZc+/fpc= github.com/fatih/color v1.6.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4= github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4= github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568/go.mod h1:xEzjJPgXI435gkrCt3MPfRiAkVrwSbHsst4LCFVfpJc= github.com/fogleman/gg v1.2.1-0.20190220221249-0403632d5b90/go.mod h1:R/bRT+9gY/C5z7JzPU0zXsXHKM4/ayA+zqcVNZzPa1k= +github.com/form3tech-oss/jwt-go v3.2.2+incompatible/go.mod h1:pbq4aXjuKjdthFRnoDwaVPLA+WlJuPGy+QneDUgJi2k= github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo= +github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ= +github.com/fvbommel/sortorder v1.0.1/go.mod h1:uk88iVf1ovNn1iLfgUVU2F9o5eO30ui720w+kxuqRs0= github.com/ghodss/yaml v0.0.0-20150909031657-73d445a93680/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04= github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04= github.com/gliderlabs/ssh v0.1.1/go.mod h1:U7qILu1NlMHj9FlMhZLlkCdDnU1DBEAqr0aevW3Awn0= @@ -130,10 +231,18 @@ github.com/globalsign/mgo v0.0.0-20181015135952-eeefdecb41b8/go.mod h1:xkRDCp4j0 github.com/go-acme/lego v2.5.0+incompatible/go.mod h1:yzMNe9CasVUhkquNvti5nAtPmG94USbYxYrZfTkIn0M= github.com/go-bindata/go-bindata v3.1.1+incompatible/go.mod h1:xK8Dsgwmeed+BBsSy2XTopBn/8uK2HWuGSnA11C3Joo= github.com/go-critic/go-critic v0.3.5-0.20190526074819-1df300866540/go.mod h1:+sE8vrLDS2M0pZkBk0wy6+nLdKexVDrl/jBqQOTDThA= +github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU= +github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8= +github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8= +github.com/go-ini/ini v1.9.0/go.mod h1:ByCAeIL28uOIIG0E3PJtZPDL8WnHpFKFOtgjp+3Ies8= github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as= +github.com/go-kit/kit v0.9.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as= github.com/go-lintpack/lintpack v0.5.2/go.mod h1:NwZuYi2nUHho8XEIZ6SIxihrnPoqBTDqfpXvXAN0sXM= github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE= +github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk= github.com/go-logr/logr v0.1.0/go.mod h1:ixOQHD9gLJUVQQ2ZOR7zLEifBX6tGkNJF4QyIY7sIas= +github.com/go-logr/logr v0.2.0 h1:QvGt2nLcHH0WK9orKa+ppBPAxREcH364nPUedEpK0TY= +github.com/go-logr/logr v0.2.0/go.mod h1:z6/tIYblkpsD+a4lm/fGIIU9mZ+XfAiaFtq7xTgseGU= github.com/go-ole/go-ole v1.2.1/go.mod h1:7FAglXiTm7HKlQRDeOQ6ZNUHidzCWXuZWq/1dTyBNF8= github.com/go-openapi/analysis v0.0.0-20180825180245-b006789cd277/go.mod h1:k70tL6pCuVxPJOHXQ+wIac1FUrvNkHolPie/cLEU6hI= github.com/go-openapi/analysis v0.17.0/go.mod h1:IowGgpVeD0vNm45So8nr+IcQ3pxVtpRoBWb8PVZO0ik= @@ -195,6 +304,7 @@ github.com/go-toolsmith/strparse v1.0.0/go.mod h1:YI2nUKP9YGZnL/L1/DLFBfixrcjslW github.com/go-toolsmith/typep v1.0.0/go.mod h1:JSQCQMUPdRlMZFswiq3TGpNp1GMktqkR2Ns5AIQkATU= github.com/gobwas/glob v0.2.3/go.mod h1:d3Ez4x06l9bZtSvzIay5+Yzi0fmZzPgnTbPcKjJAkT8= github.com/godbus/dbus v0.0.0-20181101234600-2ff6f7ffd60f/go.mod h1:/YcGZj5zSblfDWMMoOzV4fas9FZnQYTkDnsGvmh2Grw= +github.com/godbus/dbus/v5 v5.0.3/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA= github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ= github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4= github.com/gogo/protobuf v1.3.1 h1:DqDEcV5aeaTmdFBePNpYsp3FlcVH/2ISVVM9Qf8PSls= @@ -204,15 +314,34 @@ github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b h1:VKtxabqXZkF25pY9ekf github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q= github.com/golang/groupcache v0.0.0-20160516000752-02826c3e7903 h1:LbsanbbD6LieFkXbj9YNNBupiGHJgFeLpO0j0Fza1h8= github.com/golang/groupcache v0.0.0-20160516000752-02826c3e7903/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= +github.com/golang/groupcache v0.0.0-20190129154638-5b532d6fd5ef/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= +github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= +github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= +github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= github.com/golang/mock v1.0.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= github.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFUx0Y= +github.com/golang/mock v1.4.0/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw= +github.com/golang/mock v1.4.1/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw= github.com/golang/protobuf v0.0.0-20161109072736-4bd1920723d7/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= github.com/golang/protobuf v1.3.2 h1:6nsPYzhq5kReh6QImI3k5qWzO4PEbvbIW2cwSfR/6xs= github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= +github.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw= +github.com/golang/protobuf v1.3.4/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw= +github.com/golang/protobuf v1.3.5/go.mod h1:6O5/vntMXwX2lRkT1hjjk0nAC1IDOTvTlVgjlRvqsdk= +github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8= +github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA= +github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs= +github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w= +github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0= +github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8= +github.com/golang/protobuf v1.4.2 h1:+Z5KGCizgyZCbGh1KZqA0fcLLkwbsjIzS4aV2v7wJX0= +github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI= +github.com/golang/protobuf v1.4.3 h1:JjCZWpVbqXDqFVmTfYWEVTMIYrL/NPdPSCHPJ0T/raM= +github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI= github.com/golangci/check v0.0.0-20180506172741-cfe4005ccda2/go.mod h1:k9Qvh+8juN+UKMCS/3jFtGICgW8O96FVaZsaxdzDkR4= github.com/golangci/dupl v0.0.0-20180902072040-3e9179ac440a/go.mod h1:ryS0uhF+x9jgbj/N71xsEqODy9BN81/GonCZiOzirOk= github.com/golangci/errcheck v0.0.0-20181223084120-ef45e06d44b6/go.mod h1:DbHgvLiFKX1Sh2T1w8Q/h4NAI8MHIpzCdnBUDTXU3I0= @@ -237,9 +366,17 @@ github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Z github.com/google/btree v1.0.0 h1:0udJVsspx3VBr5FwtLhQQtuAsVc79tTq0ocGIPAU6qo= github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= github.com/google/cadvisor v0.35.0/go.mod h1:1nql6U13uTHaLYB8rLS5x9IJc2qT6Xd/Tr1sTX6NE48= +github.com/google/cadvisor v0.37.0/go.mod h1:OhDE+goNVel0eGY8mR7Ifq1QUI1in5vJBIgIpcajK/I= +github.com/google/cadvisor v0.38.5/go.mod h1:1OFB9sOOMkBdUBGCO/1SArawTnDscgMzTodacVDe8mA= github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M= github.com/google/go-cmp v0.3.0 h1:crn/baboCvb5fXaQ0IJ1SGTsTVrWpDsCWC8EGETZijY= github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= +github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= +github.com/google/go-cmp v0.4.0 h1:xsAVV57WRhGj6kEIi8ReJzQlHHqcBYCElAvkovg3B/4= +github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= +github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= +github.com/google/go-cmp v0.5.2 h1:X2ev0eStA3AbceY54o37/0PQ/UWqKEiiO2dKL5OPaFM= +github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-github v17.0.0+incompatible/go.mod h1:zLgOLi98H3fifZn+44m+umXrS52loVEgC2AApnigrVQ= github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck= github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= @@ -247,56 +384,95 @@ github.com/google/gofuzz v1.1.0 h1:Hsa8mG0dQ46ij8Sl2AYJDUv1oA9/d6Vk+3LG99Oe02g= github.com/google/gofuzz v1.1.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs= github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc= +github.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc= +github.com/google/pprof v0.0.0-20191218002539-d4f498aebedc/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= +github.com/google/pprof v0.0.0-20200212024743-f11f1df84d12/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= +github.com/google/pprof v0.0.0-20200229191704-1ebb73c60ed3/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI= github.com/google/uuid v1.0.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= github.com/google/uuid v1.1.1 h1:Gkbcsh/GbpXz7lPftLA3P6TYMwjCLYm83jiFQZF/3gY= github.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/google/uuid v1.1.2 h1:EVhdT+1Kseyi1/pUmXKaFxYsDNy9RQYkMWRH68J/W7Y= +github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg= +github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk= github.com/googleapis/gnostic v0.0.0-20170729233727-0c5108395e2d/go.mod h1:sJBsCZ4ayReDTBIg8b9dl28c5xFWyhBTVRp3pOg5EKY= github.com/googleapis/gnostic v0.1.0/go.mod h1:sJBsCZ4ayReDTBIg8b9dl28c5xFWyhBTVRp3pOg5EKY= +github.com/googleapis/gnostic v0.4.1 h1:DLJCy1n/vrD4HPjOvYcT8aYQXpPIzoRZONaYwyycI+I= +github.com/googleapis/gnostic v0.4.1/go.mod h1:LRhVm6pbyptWbWbuZ38d1eyptfvIytN3ir6b65WBswg= github.com/gophercloud/gophercloud v0.1.0/go.mod h1:vxM41WHh5uqHVBMZHzuwNOHh8XEoIEcSTewFxm1c5g8= github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY= github.com/gorilla/context v1.1.1/go.mod h1:kBGZzfjB9CEq2AlWe17Uuf7NDRt0dE0s8S51q0aT7Yg= github.com/gorilla/mux v1.7.0/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs= +github.com/gorilla/mux v1.7.3/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs= +github.com/gorilla/mux v1.8.0/go.mod h1:DVbg23sWSpFRCP0SfiEN6jmj59UnW/n46BH5rLB71So= github.com/gorilla/websocket v0.0.0-20170926233335-4201258b820c/go.mod h1:E7qHFY5m1UJ88s3WnNqhKjPHQ0heANvMoAMk2YaljkQ= github.com/gorilla/websocket v1.4.0 h1:WDFjx/TMzVgy9VdMMQi2K2Emtwi2QcUQsztZ/zLaH/Q= github.com/gorilla/websocket v1.4.0/go.mod h1:E7qHFY5m1UJ88s3WnNqhKjPHQ0heANvMoAMk2YaljkQ= +github.com/gorilla/websocket v1.4.2/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= github.com/gostaticanalysis/analysisutil v0.0.0-20190318220348-4088753ea4d3/go.mod h1:eEOZF4jCKGi+aprrirO9e7WKB3beBRtWgqGunKl6pKE= github.com/gostaticanalysis/analysisutil v0.0.3/go.mod h1:eEOZF4jCKGi+aprrirO9e7WKB3beBRtWgqGunKl6pKE= github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7/go.mod h1:FecbI9+v66THATjSRHfNgh1IVFe/9kFxbXtjV0ctIMA= +github.com/grpc-ecosystem/go-grpc-middleware v1.0.0/go.mod h1:FiyG127CGDf3tlThmgyCl78X/SZQqEOJBCDaAfeWzPs= github.com/grpc-ecosystem/go-grpc-middleware v1.0.1-0.20190118093823-f849b5445de4 h1:z53tR0945TRRQO/fLEVPI6SMv7ZflF0TEaTAoU7tOzg= github.com/grpc-ecosystem/go-grpc-middleware v1.0.1-0.20190118093823-f849b5445de4/go.mod h1:FiyG127CGDf3tlThmgyCl78X/SZQqEOJBCDaAfeWzPs= github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0 h1:Ovs26xHkKqVztRpIrF/92BcuyuQ/YW4NSIpoGtfXNho= github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0/go.mod h1:8NvIoxWQoOIhqOTXgfV/d3M/q6VIi02HzZEHgUlZvzk= +github.com/grpc-ecosystem/grpc-gateway v1.9.0/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY= github.com/grpc-ecosystem/grpc-gateway v1.9.5 h1:UImYN5qQ8tuGpGE16ZmjvcTtTw24zw1QAp/SlnNrZhI= github.com/grpc-ecosystem/grpc-gateway v1.9.5/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY= +github.com/hashicorp/consul/api v1.1.0/go.mod h1:VmuI/Lkw1nC05EYQWNKwWGbkg+FbDBtguAZLlVdkD9Q= +github.com/hashicorp/consul/sdk v0.1.1/go.mod h1:VKf9jXwCTEY1QZP2MOLRhb5i/I/ssyNV1vwHyQBF0x8= +github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4= +github.com/hashicorp/go-cleanhttp v0.5.1/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80= +github.com/hashicorp/go-immutable-radix v1.0.0/go.mod h1:0y9vanUI8NX6FsYoO3zeMjhV/C5i9g4Q3DwcSNZ4P60= +github.com/hashicorp/go-msgpack v0.5.3/go.mod h1:ahLV/dePpqEmjfWmKiqvPkv/twdG7iPBM1vqhUKIvfM= +github.com/hashicorp/go-multierror v1.0.0/go.mod h1:dHtQlpGsu+cZNNAkkCN/P3hoUDHhCYQXV3UM06sGGrk= +github.com/hashicorp/go-rootcerts v1.0.0/go.mod h1:K6zTfqpRlCUIjkwsN4Z+hiSfzSTQa6eBIzfwKfwNnHU= +github.com/hashicorp/go-sockaddr v1.0.0/go.mod h1:7Xibr9yA9JjQq1JpNB2Vw7kxv8xerXegt+ozgdvDeDU= github.com/hashicorp/go-syslog v1.0.0/go.mod h1:qPfqrKkXGihmCqbJM2mZgkZGvKG1dFdvsLplgctolz4= +github.com/hashicorp/go-uuid v1.0.0/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro= +github.com/hashicorp/go-uuid v1.0.1/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro= +github.com/hashicorp/go.net v0.0.1/go.mod h1:hjKkEWcCURg++eb33jQU7oqQcI9XDCnUzHA0oac0k90= github.com/hashicorp/golang-lru v0.0.0-20180201235237-0fb14efe8c47/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= github.com/hashicorp/golang-lru v0.5.1 h1:0hERBMJE1eitiLkihrMvRVBYAkpHzc/J3QdDN+dAcgU= github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= github.com/hashicorp/hcl v0.0.0-20180404174102-ef8a98b0bbce/go.mod h1:oZtUIOe8dh44I2q6ScRibXws4Ajl+d+nod3AaR9vL5w= github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ= +github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64= +github.com/hashicorp/mdns v1.0.0/go.mod h1:tL+uN++7HEJ6SQLQ2/p+z2pH24WQKWjBPkE0mNTz8vQ= +github.com/hashicorp/memberlist v0.1.3/go.mod h1:ajVTdAv/9Im8oMAAj5G31PhhMCZJV2pPBoIllUwCN7I= +github.com/hashicorp/serf v0.8.2/go.mod h1:6hOLApaqBFA1NXqRQAsxw9QxuDEvNxSQRwA/JwenrHc= github.com/heketi/heketi v9.0.1-0.20190917153846-c2e2a4ab7ab9+incompatible/go.mod h1:bB9ly3RchcQqsQ9CpyaQwvva7RS5ytVoSoholZQON6o= github.com/heketi/tests v0.0.0-20151005000721-f3775cbcefd6/go.mod h1:xGMAM8JLi7UkZt1i4FQeQy0R2T8GLUwQhOP5M1gBhy4= github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU= +github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc= github.com/imdario/mergo v0.3.5 h1:JboBksRwiiAJWvIYJVo46AfV+IAIKZpfrSzVKj42R4Q= github.com/imdario/mergo v0.3.5/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA= github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8= +github.com/ishidawataru/sctp v0.0.0-20190723014705-7c296d48a2b5/go.mod h1:DM4VvS+hD/kDi1U1QsX2fnZowwBhqD0Dk3bRPKF/Oc8= github.com/jellevandenhooff/dkim v0.0.0-20150330215556-f50fe3d243e1/go.mod h1:E0B/fFc00Y+Rasa88328GlI/XbtyysCtTHZS8h7IrBU= github.com/jimstudt/http-authentication v0.0.0-20140401203705-3eca13d6893a/go.mod h1:wK6yTYYcgjHE1Z1QtXACPDjcFJyBskHEdagmnq3vsP8= +github.com/jmespath/go-jmespath v0.0.0-20160202185014-0b12d6b521d8/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k= github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k= +github.com/jmespath/go-jmespath v0.4.0/go.mod h1:T8mJZnbsbmF+m6zOOFylbeCJqk5+pHWvzYPziyZiYoo= +github.com/jmespath/go-jmespath/internal/testify v1.5.1/go.mod h1:L3OGu8Wl2/fWfCI6z80xFu9LTZmf1ZRjMHUOPmWr69U= github.com/jonboulle/clockwork v0.1.0 h1:VKV+ZcuP6l3yW9doeqz6ziZGgcynBVQO+obU0+0hcPo= github.com/jonboulle/clockwork v0.1.0/go.mod h1:Ii8DK3G1RaLaWxj9trq07+26W01tbo22gdxWY5EU2bo= github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU= github.com/json-iterator/go v1.1.7/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= github.com/json-iterator/go v1.1.8 h1:QiWkFLKq0T7mpzwOTu6BzNDbfTE8OLrYhVKYMLF46Ok= github.com/json-iterator/go v1.1.8/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= +github.com/json-iterator/go v1.1.10 h1:Kz6Cvnvv2wGdaG/V8yMvfkmNiXq9Ya2KUv4rouJJr68= +github.com/json-iterator/go v1.1.10/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU= +github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk= github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU= github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w= github.com/jung-kurt/gofpdf v1.0.3-0.20190309125859-24315acbbda5/go.mod h1:7Id9E/uU8ce6rXgefFLlgrJj/GYY22cpxn+r32jIOes= github.com/karrick/godirwalk v1.7.5/go.mod h1:2c9FRhkDxdIbgkOnCEvnSWs71Bhugbl46shStcFDJ34= +github.com/karrick/godirwalk v1.16.1/go.mod h1:j4mkqPuvaLI8mp1DroR3P6ad7cyYd4c1qeJ3RV7ULlk= github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q= github.com/kisielk/errcheck v1.2.0/go.mod h1:/BMXB+zMLi60iA8Vv6Ksmxu/1UDYcXs4uQLJ+jE2L00= github.com/kisielk/gotool v0.0.0-20161130080628-0de1eaf82fa3/go.mod h1:jxZFDH7ILpTPQTk+E2s+z4CUas9lVNjIuKR4c5/zKgM= @@ -307,9 +483,12 @@ github.com/klauspost/cpuid v0.0.0-20180405133222-e7e905edc00e/go.mod h1:Pj4uuM52 github.com/klauspost/cpuid v1.2.0/go.mod h1:Pj4uuM528wm8OyEC2QMXAi2YiTZ96dNQPGgoMS4s3ek= github.com/konsorten/go-windows-terminal-sequences v1.0.1 h1:mweAR1A6xJ3oS2pRaGiHgQ4OO8tzTaLawm8vnODuwDk= github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= +github.com/konsorten/go-windows-terminal-sequences v1.0.2/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= +github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc= github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI= github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= +github.com/kr/pretty v0.2.0/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI= github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= github.com/kr/pty v1.1.3/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= github.com/kr/pty v1.1.5/go.mod h1:9r2w37qlBe7rQ6e1fg1S/9xpWHSnaqNdHD3WcMdbPDA= @@ -344,18 +523,30 @@ github.com/mattn/go-shellwords v1.0.5/go.mod h1:3xCvwCdWdlDJUrvuMn7Wuy9eWs4pE8vq github.com/mattn/goveralls v0.0.2/go.mod h1:8d1ZMHsd7fW6IRPKQh46F2WRpyib5/X4FOpevwGNQEw= github.com/matttproud/golang_protobuf_extensions v1.0.1 h1:4hp9jkHxhMHkqkrB3Ix0jegS5sx/RkqARlsWZ6pIwiU= github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0= +github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 h1:I0XW9+e1XWDxdcEniV4rQAIOPUGDq67JSCiRCgGCZLI= +github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369/go.mod h1:BSXmuO+STAnVfrANrmjBb36TMTDstsz7MSK+HVaYKv4= github.com/mesos/mesos-go v0.0.9/go.mod h1:kPYCMQ9gsOXVAle1OsoY4I1+9kPu8GHkf88aV59fDr4= github.com/mholt/certmagic v0.6.2-0.20190624175158-6a42ef9fe8c2/go.mod h1:g4cOPxcjV0oFq3qwpjSA30LReKD8AoIfwAY9VvG35NY= +github.com/miekg/dns v1.0.14/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg= github.com/miekg/dns v1.1.3/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg= github.com/miekg/dns v1.1.4/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg= github.com/mindprince/gonvml v0.0.0-20190828220739-9ebdce4bb989/go.mod h1:2eu9pRWp8mo84xCg6KswZ+USQHjwgRhNp06sozOdsTY= github.com/mistifyio/go-zfs v2.1.1+incompatible/go.mod h1:8AuVvqP/mXw1px98n46wfvcGfQ4ci2FwoAjKYxuo3Z4= +github.com/mistifyio/go-zfs v2.1.2-0.20190413222219-f784269be439+incompatible/go.mod h1:8AuVvqP/mXw1px98n46wfvcGfQ4ci2FwoAjKYxuo3Z4= +github.com/mitchellh/cli v1.0.0/go.mod h1:hNIlj7HEI86fIcpObd7a0FcrxTWetlwJDGcceTlRvqc= github.com/mitchellh/go-homedir v1.0.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0= github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0= github.com/mitchellh/go-ps v0.0.0-20170309133038-4fdf99ab2936/go.mod h1:r1VsdOzOPt1ZSrGZWFoNhsAedKnEd6r9Np1+5blZCWk= +github.com/mitchellh/go-testing-interface v1.0.0/go.mod h1:kRemZodwjscx+RGhAo8eIhFbs2+BFgRtFPeD/KE+zxI= github.com/mitchellh/go-wordwrap v1.0.0/go.mod h1:ZXFpozHsX6DPmq2I0TCekCxypsnAUbP2oI0UX1GXzOo= +github.com/mitchellh/gox v0.4.0/go.mod h1:Sd9lOJ0+aimLBi73mGofS1ycjY8lL3uZM3JPS42BGNg= +github.com/mitchellh/iochan v1.0.0/go.mod h1:JwYml1nuB7xOzsp52dPpHFffvOCDupsG0QubkSMEySY= +github.com/mitchellh/mapstructure v0.0.0-20160808181253-ca63d7c062ee/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y= github.com/mitchellh/mapstructure v0.0.0-20180220230111-00c29f56e238/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y= github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y= +github.com/moby/ipvs v1.0.1/go.mod h1:2pngiyseZbIKXNv7hsKj3O9UEz30c53MT9005gt2hxQ= +github.com/moby/sys/mountinfo v0.1.3/go.mod h1:w2t2Avltqx8vE7gX5l+QiBKxODu2TX0+Syr3h52Tw4o= +github.com/moby/term v0.0.0-20200312100748-672ec06f55cd/go.mod h1:DdlQx2hp0Ss5/fLikoLlEeIYiATotOjgB//nb973jeo= github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= @@ -366,6 +557,7 @@ github.com/mohae/deepcopy v0.0.0-20170603005431-491d3605edfb/go.mod h1:TaXosZuwd github.com/morikuni/aec v1.0.0/go.mod h1:BbKIizmSmc5MMPqRYbxO4ZU0S0+P200+tUnFx7PXmsc= github.com/mozilla/tls-observatory v0.0.0-20180409132520-8791a200eb40/go.mod h1:SrKMQvPiws7F7iqYp8/TX+IhxCYhzr6N/1yb8cwHsGk= github.com/mrunalp/fileutils v0.0.0-20171103030105-7d4729fb3618/go.mod h1:x8F1gnqOkIEiO4rqoeEEEqQbo7HjGMTvyoq3gej4iT0= +github.com/mrunalp/fileutils v0.0.0-20200520151820-abd8a0e76976/go.mod h1:x8F1gnqOkIEiO4rqoeEEEqQbo7HjGMTvyoq3gej4iT0= github.com/munnerz/goautoneg v0.0.0-20120707110453-a547fc61f48d/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= github.com/mvdan/xurls v1.1.0/go.mod h1:tQlNn3BED8bE/15hnSL2HLkDeLWpNPAwtw7wkEq44oU= @@ -375,6 +567,7 @@ github.com/naoina/go-stringutil v0.1.0/go.mod h1:XJ2SJL9jCtBh+P9q5btrd/Ylo8XwT/h github.com/naoina/toml v0.1.1/go.mod h1:NBIhNtsFMo3G2szEBne+bO4gS192HuIYRqfvOWb4i1E= github.com/nbutton23/zxcvbn-go v0.0.0-20160627004424-a22cb81b2ecd/go.mod h1:o96djdrsSGy3AWPyBgZMAGfxZNfgntdJG+11KU4QvbU= github.com/nbutton23/zxcvbn-go v0.0.0-20171102151520-eafdab6b0663/go.mod h1:o96djdrsSGy3AWPyBgZMAGfxZNfgntdJG+11KU4QvbU= +github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U= github.com/olekukonko/tablewriter v0.0.0-20170122224234-a0225b3f23b5/go.mod h1:vsDQFd/mU46D+Z4whnwzcISnGGzXWMclvtLoiIKAKIo= github.com/onsi/ginkgo v0.0.0-20170829012221-11459a886d9c/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= @@ -385,12 +578,26 @@ github.com/onsi/gomega v1.4.2/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1Cpa github.com/onsi/gomega v1.4.3/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY= github.com/onsi/gomega v1.5.0/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY= github.com/onsi/gomega v1.7.0/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY= +github.com/opencontainers/go-digest v0.0.0-20180430190053-c9281466c8b2/go.mod h1:cMLVZDEM3+U2I4VmLI6N8jQYUd2OVphdqWwCJHrFt2s= github.com/opencontainers/go-digest v1.0.0-rc1 h1:WzifXhOVOEOuFYOJAW6aQqW0TooG2iki3E3Ii+WN7gQ= github.com/opencontainers/go-digest v1.0.0-rc1/go.mod h1:cMLVZDEM3+U2I4VmLI6N8jQYUd2OVphdqWwCJHrFt2s= +github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U= +github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM= github.com/opencontainers/image-spec v1.0.1/go.mod h1:BtxoFyWECRxE4U/7sNtV5W15zMzWCbyJoFRP3s7yZA0= +github.com/opencontainers/runc v0.0.0-20190115041553-12f6a991201f/go.mod h1:qT5XzbpPznkRYVz/mWwUaVBUv2rmF59PVA73FjuZG0U= github.com/opencontainers/runc v1.0.0-rc10/go.mod h1:qT5XzbpPznkRYVz/mWwUaVBUv2rmF59PVA73FjuZG0U= +github.com/opencontainers/runc v1.0.0-rc90.0.20200616040943-82d2fa4eb069/go.mod h1:3Sm6Dt7OT8z88EbdQqqcRN2oCT54jbi72tT/HqgflT8= +github.com/opencontainers/runc v1.0.0-rc91.0.20200707015106-819fcc687efb/go.mod h1:ZuXhqlr4EiRYgDrBDNfSbE4+n9JX4+V107NwAmF7sZA= +github.com/opencontainers/runc v1.0.0-rc92/go.mod h1:X1zlU4p7wOlX4+WRCz+hvlRv8phdL7UqbYD+vQwNMmE= github.com/opencontainers/runtime-spec v1.0.0/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0= +github.com/opencontainers/runtime-spec v1.0.2/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0= +github.com/opencontainers/runtime-spec v1.0.3-0.20200520003142-237cc4f519e2/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0= +github.com/opencontainers/runtime-spec v1.0.3-0.20200728170252-4d89ac9fbff6/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0= github.com/opencontainers/selinux v1.3.1-0.20190929122143-5215b1806f52/go.mod h1:+BLncwf63G4dgOzykXAxcmnFlUaOlkDdmw/CqsW6pjs= +github.com/opencontainers/selinux v1.5.1/go.mod h1:yTcKuYAh6R95iDpefGLQaPaRwJFwyzAJufJyiTt7s0g= +github.com/opencontainers/selinux v1.5.2/go.mod h1:yTcKuYAh6R95iDpefGLQaPaRwJFwyzAJufJyiTt7s0g= +github.com/opencontainers/selinux v1.6.0/go.mod h1:VVGKuOLlE7v4PJyT6h7mNWvq1rzqiriPsEqVhc+svHE= +github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc= github.com/pborman/uuid v1.2.0/go.mod h1:X/NO0urCmaxf9VXbdlT7C2Yzkj2IKimNn4k+gtPdI/k= github.com/pelletier/go-toml v1.1.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic= github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic= @@ -398,25 +605,43 @@ github.com/peterbourgon/diskv v2.0.1+incompatible/go.mod h1:uqqh8zWWbv1HBMNONnaR github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pkg/errors v0.8.1 h1:iURUrRGxPUNPdy5/HRSm+Yj6okJ6UtLINN0Q9M4+h3I= github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= +github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/posener/complete v1.1.1/go.mod h1:em0nMJCgc9GFtwrmVmEMR/ZL6WyhyjMBndrE9hABlRI= github.com/pquerna/cachecontrol v0.0.0-20171018203845-0dec1b30a021/go.mod h1:prYjPmNq4d1NPVmpShWobRqXY3q7Vp+80DqgxxUrUIA= github.com/pquerna/ffjson v0.0.0-20180717144149-af8b230fcd20/go.mod h1:YARuvh7BUWHNhzDq2OM5tzR2RiCcN2D7sapiKyCel/M= github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw= +github.com/prometheus/client_golang v0.9.3/go.mod h1:/TN21ttK/J9q6uSwhBd54HahCDft0ttaMvbicHlPoso= github.com/prometheus/client_golang v1.0.0 h1:vrDKnkGzuGvhNAL56c7DBz29ZL+KxnoR0x7enabFceM= github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo= +github.com/prometheus/client_golang v1.7.1 h1:NTGy1Ja9pByO+xAeH/qiWnLrKtr3hJPNjaVUwnjpdpA= +github.com/prometheus/client_golang v1.7.1/go.mod h1:PY5Wy2awLA44sXw4AOSfFBetzPP4j5+D6mVACh+pe2M= github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo= github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= github.com/prometheus/client_model v0.2.0 h1:uq5h0d+GuxiXLJLNABMgp2qUWDPiLvgCzz2dUR+/W/M= github.com/prometheus/client_model v0.2.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= +github.com/prometheus/common v0.0.0-20181113130724-41aa239b4cce/go.mod h1:daVV7qP5qjZbuso7PdcryaAu0sAZbrN9i7WWcTMWvro= +github.com/prometheus/common v0.4.0/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4= github.com/prometheus/common v0.4.1 h1:K0MGApIoQvMw27RTdJkPbr3JZ7DNbtxQNyi5STVM6Kw= github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4= +github.com/prometheus/common v0.10.0 h1:RyRA7RzGXQZiW+tGMr7sxa85G1z0yOpM1qq5c8lNawc= +github.com/prometheus/common v0.10.0/go.mod h1:Tlit/dnDKsSWFlCLTWaA1cyBgKHSMdTB80sz/V91rCo= +github.com/prometheus/procfs v0.0.0-20180125133057-cb4147076ac7/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= +github.com/prometheus/procfs v0.0.0-20190507164030-5867b95ac084/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA= +github.com/prometheus/procfs v0.0.0-20190522114515-bc1a522cf7b1/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA= github.com/prometheus/procfs v0.0.2 h1:6LJUbpNm42llc4HRCuvApCSWB/WfhuNo9K98Q9sNGfs= github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA= +github.com/prometheus/procfs v0.1.3 h1:F0+tqvhOksq22sc6iCHF5WGlWjdwj92p0udFh1VFBS8= +github.com/prometheus/procfs v0.1.3/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU= +github.com/prometheus/procfs v0.2.0 h1:wH4vA7pcjKuZzjF7lM8awk4fnuJO6idemZXoKnULUx4= +github.com/prometheus/procfs v0.2.0/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU= +github.com/prometheus/tsdb v0.7.1/go.mod h1:qhTCs0VvXwvX/y3TZrWD7rabWM+ijKTux40TwIPHuXU= github.com/quasilyte/go-consistent v0.0.0-20190521200055-c6f3937de18c/go.mod h1:5STLWrekHfjyYwxBRVRXNOSewLJ3PWfDJd1VyTS21fI= github.com/quobyte/api v0.1.2/go.mod h1:jL7lIHrmqQ7yh05OJ+eEEdHr0u/kmT1Ff9iHd+4H6VI= +github.com/quobyte/api v0.1.8/go.mod h1:jL7lIHrmqQ7yh05OJ+eEEdHr0u/kmT1Ff9iHd+4H6VI= github.com/remyoudompheng/bigfft v0.0.0-20170806203942-52369c62f446/go.mod h1:uYEyJGbgTkfkS4+E/PavXkNJcbFIpEtjt2B0KDQ5+9M= github.com/robfig/cron v1.1.0 h1:jk4/Hud3TTdcrJgUOBgsqrZBarcxl6ADIjSC2iniwLY= github.com/robfig/cron v1.1.0/go.mod h1:JGuDeoQd7Z6yL4zQhZ3OPEVHB7fL6Ka6skscFHfmt2k= @@ -424,27 +649,34 @@ github.com/rogpeppe/fastuuid v0.0.0-20150106093220-6724a57986af/go.mod h1:XWv6So github.com/rogpeppe/go-internal v1.1.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4= github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4= github.com/rubiojr/go-vhd v0.0.0-20160810183302-0bfd3b39853c/go.mod h1:DM5xW0nvfNNm2uytzsvhI3OnX8uzaRAg8UX/CnDqbto= +github.com/rubiojr/go-vhd v0.0.0-20200706105327-02e210299021/go.mod h1:DM5xW0nvfNNm2uytzsvhI3OnX8uzaRAg8UX/CnDqbto= github.com/russross/blackfriday v0.0.0-20170610170232-067529f716f4/go.mod h1:JO/DiYxRf+HjHt06OyowR9PTA263kcR/rfWxYHBV53g= github.com/russross/blackfriday v1.5.2/go.mod h1:JO/DiYxRf+HjHt06OyowR9PTA263kcR/rfWxYHBV53g= +github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= +github.com/ryanuber/columnize v0.0.0-20160712163229-9b3edd62028f/go.mod h1:sm1tb6uqfes/u+d4ooFouqFdy9/2g9QGwK3SQygK0Ts= github.com/ryanuber/go-glob v0.0.0-20170128012129-256dc444b735/go.mod h1:807d1WSdnB0XRJzKNil9Om6lcp/3a0v4qIHxIXzX/Yc= github.com/satori/go.uuid v1.2.0/go.mod h1:dA0hQrYB0VpLJoorglMZABFdXlWrHn1NEOzdhQKdks0= +github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529/go.mod h1:DxrIzT+xaE7yg65j358z/aeFdxmN0P9QXhEzd20vsDc= github.com/seccomp/libseccomp-golang v0.9.1/go.mod h1:GbW5+tmTXfcxTToHLXlScSlAvWlF4P2Ca7zGrPiEpWo= github.com/sergi/go-diff v1.0.0/go.mod h1:0CfEIISq7TuYL3j771MWULgwwjU+GofnZX9QAmXWZgo= github.com/shirou/gopsutil v0.0.0-20180427012116-c95755e4bcd7/go.mod h1:5b4v6he4MtMOwMlS0TUMTu2PcXUg8+E1lC7eC3UO/RA= github.com/shirou/w32 v0.0.0-20160930032740-bb4de0191aa4/go.mod h1:qsXQc7+bwAM3Q1u/4XEfrquwF8Lw7D7y5cD8CuHnfIc= github.com/shurcooL/go v0.0.0-20180423040247-9e1955d9fb6e/go.mod h1:TDJrrUr11Vxrven61rcy3hJMUqaf/CLWYhHNPmT14Lk= github.com/shurcooL/go-goon v0.0.0-20170922171312-37c2f522c041/go.mod h1:N5mDOmsrJOB+vfqUK+7DmDyjhSLIIBnXo9lvZJj3MWQ= +github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc= github.com/sirupsen/logrus v1.0.5/go.mod h1:pMByvHTf9Beacp5x1UXfOR9xyW/9antXMhjMPG0dEzc= github.com/sirupsen/logrus v1.0.6/go.mod h1:pMByvHTf9Beacp5x1UXfOR9xyW/9antXMhjMPG0dEzc= github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo= github.com/sirupsen/logrus v1.4.1/go.mod h1:ni0Sbl8bgC9z8RoU9G6nDWqqs/fq4eDPysMBDgk/93Q= github.com/sirupsen/logrus v1.4.2 h1:SPIRibHv4MatM3XXNO2BJeFLZwZ2LvZgfQ5+UNI2im4= github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE= +github.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrfsX/uA88= github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc= github.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA= github.com/soheilhy/cmux v0.1.4 h1:0HKaf1o97UwFjHH9o5XsHUOF+tqmdA7KEzXLpiyaw0E= github.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4kGIyLM= github.com/sourcegraph/go-diff v0.5.1/go.mod h1:j2dHj3m8aZgQO8lMTcTnBcXkRRRqi34cd2MNlA9u1mE= +github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA= github.com/spf13/afero v1.1.0/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ= github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ= github.com/spf13/afero v1.2.2 h1:5jhuqJyZCZf2JRofRvN/nIFgIWNzPa3/Vz8mYylgbWc= @@ -454,6 +686,8 @@ github.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkU github.com/spf13/cobra v0.0.2/go.mod h1:1l0Ry5zgKvJasoi3XT1TypsSe7PqH0Sj9dhYf7v3XqQ= github.com/spf13/cobra v0.0.3/go.mod h1:1l0Ry5zgKvJasoi3XT1TypsSe7PqH0Sj9dhYf7v3XqQ= github.com/spf13/cobra v0.0.5/go.mod h1:3K3wKZymM7VvHMDS9+Akkh4K60UwM26emMESw8tLCHU= +github.com/spf13/cobra v1.0.0/go.mod h1:/6GTrnGXV9HjY+aR4k0oJ5tcvakLuG6EuKReYlHNrgE= +github.com/spf13/cobra v1.1.1/go.mod h1:WnodtKOvamDL/PwE2M4iKs8aMDBZ5Q5klgD3qfVJQMI= github.com/spf13/jwalterweatherman v0.0.0-20180109140146-7c0cea34c8ec/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo= github.com/spf13/jwalterweatherman v1.0.0/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo= github.com/spf13/jwalterweatherman v1.1.0/go.mod h1:aNWZUN0dPAAO/Ljvb5BEdw96iTZ0EXowPYD95IqWIGo= @@ -464,7 +698,10 @@ github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA= github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= github.com/spf13/viper v1.0.2/go.mod h1:A8kyI5cUJhb8N+3pkfONlcEcZbueH6nhAm0Fq7SrnBM= github.com/spf13/viper v1.3.2/go.mod h1:ZiWeW+zYFKm7srdB9IoDzzZXaJaI5eL9QjNiN/DMA2s= +github.com/spf13/viper v1.4.0/go.mod h1:PTJ7Z/lr49W6bUbkmS1V3by4uWynFiR9p7+dSq/yZzE= +github.com/spf13/viper v1.7.0/go.mod h1:8WkrPz2fc9jxqZNCJI/76HCieCp4Q8HaLFoCha5qpdg= github.com/storageos/go-api v0.0.0-20180912212459-343b3eff91fc/go.mod h1:ZrLn+e0ZuF3Y65PNF6dIwbJPZqfmtCXxFm9ckv0agOY= +github.com/storageos/go-api v2.2.0+incompatible/go.mod h1:ZrLn+e0ZuF3Y65PNF6dIwbJPZqfmtCXxFm9ckv0agOY= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/objx v0.2.0/go.mod h1:qt09Ya8vawLte6SNmTgCsAVtYtaKzEcn8ATUoHMkEqE= @@ -472,6 +709,8 @@ github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXf github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= github.com/stretchr/testify v1.4.0 h1:2E4SXV/wtOkTonXsotYi4li6zVWxYlZuYNCXe9XRJyk= github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= +github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= +github.com/subosito/gotenv v1.2.0/go.mod h1:N0PQaV/YGNqwC0u51sEeR/aUtSLEXKX9iv69rRypqCw= github.com/syndtr/gocapability v0.0.0-20180916011248-d98352740cb2/go.mod h1:hkRG7XYTFWNJGYcbNJQlaLq0fg1yr4J4t/NcTQtrfww= github.com/tarm/serial v0.0.0-20180830185346-98f6abe2eb07/go.mod h1:kDXzergiv9cbyO7IOYJZWg1U88JhDg3PB6klq9Hg2pA= github.com/thecodeteam/goscaleio v0.1.0/go.mod h1:68sdkZAsK8bvEwBlbQnlLS+xU+hvLYM/iQ8KXej1AwM= @@ -479,10 +718,14 @@ github.com/tidwall/pretty v1.0.0/go.mod h1:XNkn88O1ChpSDQmQeStsy+sBenx6DDtFZJxhV github.com/timakin/bodyclose v0.0.0-20190721030226-87058b9bfcec/go.mod h1:Qimiffbc6q9tBWlVV6x0P9sat/ao1xEkREYPPj9hphk= github.com/tmc/grpc-websocket-proxy v0.0.0-20170815181823-89b8d40f7ca8 h1:ndzgwNDnKIqyCvHTXaCqh9KlOWKvBry6nuXMJmonVsE= github.com/tmc/grpc-websocket-proxy v0.0.0-20170815181823-89b8d40f7ca8/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U= +github.com/tmc/grpc-websocket-proxy v0.0.0-20190109142713-0ad062ec5ee5/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U= +github.com/ugorji/go v1.1.4/go.mod h1:uQMGLiO92mf5W77hV/PUCpI3pbzQx3CRekS0kk+RGrc= github.com/ugorji/go/codec v0.0.0-20181204163529-d75b2dcb6bc8/go.mod h1:VFNgLljTbGfSG7qAOspJ7OScBnGdDN/yBr0sguwnwf0= github.com/ultraware/funlen v0.0.1/go.mod h1:Dp4UiAus7Wdb9KUZsYWZEWiRzGuM2kXM1lPbfaF6xhA= github.com/ultraware/funlen v0.0.2/go.mod h1:Dp4UiAus7Wdb9KUZsYWZEWiRzGuM2kXM1lPbfaF6xhA= github.com/urfave/cli v1.20.0/go.mod h1:70zkFmudgCuE/ngEzBv17Jvp/497gISqfk5gWijbERA= +github.com/urfave/cli v1.22.1/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0= +github.com/urfave/cli v1.22.2/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0= github.com/urfave/negroni v1.0.0/go.mod h1:Meg73S6kFm/4PpbYdq35yYWoCZ9mS/YSx+lKnmiohz4= github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc= github.com/valyala/fasthttp v1.2.0/go.mod h1:4vX61m6KN+xDduDNwXrhIAVZaZaZiQ1luJk8LWSxF3s= @@ -490,22 +733,39 @@ github.com/valyala/quicktemplate v1.1.1/go.mod h1:EH+4AkTd43SvgIbQHYu59/cJyxDoOV github.com/valyala/tcplisten v0.0.0-20161114210144-ceec8f93295a/go.mod h1:v3UYOV9WzVtRmSR+PDvWpU/qWl4Wa5LApYYX4ZtKbio= github.com/vektah/gqlparser v1.1.2/go.mod h1:1ycwN7Ij5njmMkPPAOaRFY4rET2Enx7IkVv3vaXspKw= github.com/vishvananda/netlink v1.0.0/go.mod h1:+SR5DhBJrl6ZM7CoCKvpw5BKroDKQ+PJqOg65H/2ktk= +github.com/vishvananda/netlink v1.1.0/go.mod h1:cTgwzPIzzgDAYoQrMm0EdrjRUBkTqKYppBueQtXaqoE= github.com/vishvananda/netns v0.0.0-20171111001504-be1fbeda1936/go.mod h1:ZjcWmFBXmLKZu9Nxj3WKYEafiSqer2rnvPr0en9UNpI= +github.com/vishvananda/netns v0.0.0-20191106174202-0a2b9b5464df/go.mod h1:JP3t17pCcGlemwknint6hfoeCVQrEMVwxRLRjXpq+BU= +github.com/vishvananda/netns v0.0.0-20200520041808-52d707b772fe/go.mod h1:DD4vA1DwXk04H54A1oHXtwZmA0grkVMdPxx/VGLCah0= +github.com/vishvananda/netns v0.0.0-20200728191858-db3c7e526aae/go.mod h1:DD4vA1DwXk04H54A1oHXtwZmA0grkVMdPxx/VGLCah0= github.com/vmware/govmomi v0.20.3/go.mod h1:URlwyTFZX72RmxtxuaFL2Uj3fD1JTvZdx59bHWk6aFU= +github.com/willf/bitset v1.1.11-0.20200630133818-d5bec3311243/go.mod h1:RjeCKbqT1RxIR/KWY6phxZiaY1IyutSBfGjNPySAYV4= github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2 h1:eY9dn8+vbi4tKz5Qo6v2eYzo7kUS51QINcR5jNpbZS8= github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU= github.com/xlab/handysort v0.0.0-20150421192137-fb3537ed64a1/go.mod h1:QcJo0QPSfTONNIgpN5RA8prR7fF8nkF6cTWTcNerRO8= github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q= +github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= +go.etcd.io/bbolt v1.3.2/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU= go.etcd.io/bbolt v1.3.3 h1:MUGmc65QhB3pIlaQ5bB4LwqSj6GIonVJXpZiaKNyaKk= go.etcd.io/bbolt v1.3.3/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU= +go.etcd.io/bbolt v1.3.5/go.mod h1:G5EMThwa9y8QZGBClrRx5EY+Yw9kAhnjy3bSjsnlVTQ= go.etcd.io/etcd v0.0.0-20191023171146-3cf2f69b5738 h1:VcrIfasaLFkyjk6KNlXQSzO+B0fZcnECiDrKJsfxka0= go.etcd.io/etcd v0.0.0-20191023171146-3cf2f69b5738/go.mod h1:dnLIgRNXwCJa5e+c6mIZCrds/GIG4ncV9HhK5PX7jPg= +go.etcd.io/etcd v0.5.0-alpha.5.0.20200819165624-17cef6e3e9d5 h1:Gqga3zA9tdAcfqobUGjSoCob5L3f8Dt5EuOp3ihNZko= +go.etcd.io/etcd v0.5.0-alpha.5.0.20200819165624-17cef6e3e9d5/go.mod h1:skWido08r9w6Lq/w70DO5XYIKMu4QFu1+4VsqLQuJy8= +go.etcd.io/etcd v0.5.0-alpha.5.0.20200910180754-dd1b699fc489 h1:1JFLBqwIgdyHN1ZtgjTBwO+blA6gVOmZurpiMEsETKo= +go.etcd.io/etcd v0.5.0-alpha.5.0.20200910180754-dd1b699fc489/go.mod h1:yVHk9ub3CSBatqGNg7GRmsnfLWtoW60w4eDYfh7vHDg= go.mongodb.org/mongo-driver v1.0.3/go.mod h1:u7ryQJ+DOzQmeO7zB6MHyr8jkEQvC8vH7qLUO4lqsUM= go.mongodb.org/mongo-driver v1.1.1/go.mod h1:u7ryQJ+DOzQmeO7zB6MHyr8jkEQvC8vH7qLUO4lqsUM= go.mongodb.org/mongo-driver v1.1.2/go.mod h1:u7ryQJ+DOzQmeO7zB6MHyr8jkEQvC8vH7qLUO4lqsUM= go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU= +go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8= +go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= +go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= go.uber.org/atomic v1.3.2 h1:2Oa65PReHzfn29GpvgsYwloV9AVFHPDk8tYxt2c2tr4= go.uber.org/atomic v1.3.2/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE= +go.uber.org/atomic v1.4.0 h1:cxzIVoETapQEqDhQu3QfnvXAV4AlzcvUCxkVUFw3+EU= +go.uber.org/atomic v1.4.0/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE= go.uber.org/multierr v1.1.0 h1:HoEmRHQPVSqub6w2z2d2EOVs2fjyFRGyofhKuyDq0QI= go.uber.org/multierr v1.1.0/go.mod h1:wR5kodmAFQ0UK8QlbwjlSNy0Z68gJhDJUG5sjR94q/0= go.uber.org/zap v1.10.0 h1:ORx85nbTijNz8ljznvCMR1ZBIPKFn3jQrag10X2AsuM= @@ -514,6 +774,7 @@ go4.org v0.0.0-20180809161055-417644f6feb5/go.mod h1:MkTOUMDaeVYJUOUsaDXIhWPZYa1 golang.org/x/build v0.0.0-20190927031335-2835ba2e683f/go.mod h1:fYw7AShPAhGMdXqA9gRadk/CcMsvLlClpE5oBwnS3dM= golang.org/x/crypto v0.0.0-20180426230345-b49d69b5da94/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= +golang.org/x/crypto v0.0.0-20181029021203-45a5f77698d3/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= golang.org/x/crypto v0.0.0-20181203042331-505ab145d0a9/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= golang.org/x/crypto v0.0.0-20190123085648-057139ce5d2b/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= golang.org/x/crypto v0.0.0-20190211182817-74369b46fc67/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= @@ -523,24 +784,52 @@ golang.org/x/crypto v0.0.0-20190313024323-a1f597ede03a/go.mod h1:djNgcEr1/C05ACk golang.org/x/crypto v0.0.0-20190320223903-b7391e95e576/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= golang.org/x/crypto v0.0.0-20190424203555-c05e17bb3b2d/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= +golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20190611184440-5c40567a22f8/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20190617133340-57b3e21c3d56/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= +golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= +golang.org/x/crypto v0.0.0-20191206172530-e9b2fee46413/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= golang.org/x/crypto v0.0.0-20200220183623-bac4c82f6975 h1:/Tl7pH94bvbAAHBdZJT947M/+gp0+CqQXDtMRC0fseo= golang.org/x/crypto v0.0.0-20200220183623-bac4c82f6975/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= +golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9 h1:psW17arqaxU48Z5kZ0CQnkZWQJsqcURM6tKiBApRjXI= +golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= +golang.org/x/crypto v0.0.0-20201002170205-7f63de1d35b0 h1:hb9wdF1z5waM+dSIICn1l0DkLVDT3hqhhQsDNUmHPRE= +golang.org/x/crypto v0.0.0-20201002170205-7f63de1d35b0/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= golang.org/x/exp v0.0.0-20180321215751-8460e604b9de/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20180807140117-3d87b88a115f/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20190125153040-c74c464bbbf2/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= +golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20190312203227-4b39c73a6495/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8= +golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8= +golang.org/x/exp v0.0.0-20190829153037-c13cbed26979/go.mod h1:86+5VVa7VpoJ4kLfm080zCjGlMRFzhUhsZKEZO7MGek= +golang.org/x/exp v0.0.0-20191030013958-a1ab85dbe136/go.mod h1:JXzH8nQsPlswgeRAPE3MuO9GYsAcnJvJ4vnMwN/5qkY= +golang.org/x/exp v0.0.0-20191129062945-2f5052295587/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4= +golang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4= +golang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4= +golang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM= +golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU= golang.org/x/image v0.0.0-20180708004352-c73c2afc3b81/go.mod h1:ux5Hcp/YLpHSI86hEcLt0YII63i6oz57MZXIpbrjZUs= golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js= +golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0= golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE= golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU= golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE= golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= golang.org/x/lint v0.0.0-20190409202823-959b441ac422/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= +golang.org/x/lint v0.0.0-20190909230951-414d861bb4ac/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= +golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= +golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRuDixDT3tpyyb+LUpUlRWLxfhWrs= +golang.org/x/lint v0.0.0-20200130185559-910be7a94367/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY= +golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY= golang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE= +golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o= golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc= +golang.org/x/mod v0.1.0/go.mod h1:0QHyrYULN0/3qlju5TqG8bIK38QM8yzMo5ekMj3DlcY= +golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg= +golang.org/x/mod v0.1.1-0.20191107180719-034126e5016b/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg= +golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= +golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/net v0.0.0-20170114055629-f2499483f923/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20170915142106-8351a756f30f/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= @@ -548,8 +837,10 @@ golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73r golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180911220305-26e67e76b6c3/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20181005035420-146acd28ed58/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= +golang.org/x/net v0.0.0-20181023162649-9b4f9f5ad519/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20181102091132-c10e9556a7bc/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= +golang.org/x/net v0.0.0-20181201002055-351d144fa1fc/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20181220203305-927f97764cc3/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20190125091013-d26f9f9a57f3/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= @@ -558,19 +849,38 @@ golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn golang.org/x/net v0.0.0-20190320064053-1272bf9dcd53/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190328230028-74de082e2cca/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= +golang.org/x/net v0.0.0-20190501004415-9ce7a6920f09/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190502183928-7f726cade0ab/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= +golang.org/x/net v0.0.0-20190522155817-f3200d17e092/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks= +golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks= golang.org/x/net v0.0.0-20190613194153-d28f0bde5980/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20190813141303-74dc4d7220e7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20190827160401-ba9fcec4b297/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20191004110552-13f9640d40b9 h1:rjwSpXsdiK0dV8/Naq3kAw9ymfAeJIyd0upUIElB+lI= golang.org/x/net v0.0.0-20191004110552-13f9640d40b9/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20200222125558-5a598a2470a0/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20200301022130-244492dfa37a/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20200324143707-d3edc9973b7e/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= +golang.org/x/net v0.0.0-20200707034311-ab3426394381 h1:VXak5I6aEWmAXeQjA+QSZzlgNrpq9mjcfDemuexIKsU= +golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= +golang.org/x/net v0.0.0-20201110031124-69a78807bb2b h1:uwuIcX0g4Yl1NC5XAz37xsr2lTtcqevgzYNVt49waME= +golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= golang.org/x/oauth2 v0.0.0-20190402181905-9f3314589c9a/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45 h1:SVwTIAaPC2U/AvvLNZ2a7OVsmBpC8L5BlwK1whH3hm0= golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= +golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6 h1:pE8b58s1HRDMi8RDc79m0HISf9D4TzseP40cEA6IGfs= +golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= +golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d h1:TzXSXBo42m9gQenoE3b9BGiEpg5IG2JkU5FkPIawgtw= +golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= golang.org/x/perf v0.0.0-20180704124530-6e6d33e29852/go.mod h1:JLpeXjPJfIyPr5TlbXLkXWLhP8nz10XfvxElABhCtcw= golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= @@ -580,9 +890,11 @@ golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJ golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sys v0.0.0-20170830134202-bb24a47a89ea/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20171026204733-164713f0dfce/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20181026203630-95b1ffbd15a5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20181107165924-66b7b1311ac8/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20181205085412-a5c9d58dba9a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= @@ -595,23 +907,65 @@ golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7w golang.org/x/sys v0.0.0-20190321052220-f7bb7a8bee54/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190502175342-a43fa875dd82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190606203320-7fc4e5ec1444/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190616124812-15dcb6c0061f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190813064441-fde4db37ae7a/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190826190057-c7b8b68b1456/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190916202348-b4ddaad3f8a3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191022100944-742c48ecaeb7 h1:HmbHVPwrPEKPGLAcHSrMe6+hqSUlvZU0rab6x5EXfGU= golang.org/x/sys v0.0.0-20191022100944-742c48ecaeb7/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20191115151921-52ab43148777/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20191120155948-bd437916bb0e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200106162015-b016eb3dc98e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200120151820-655fe14d7479/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200124204421-9fbb57f87de9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200217220822-9197077df867/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200327173247-9dae0f8f5775/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200615200032-f1bc736245b1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200622214017-ed371f2e16b4 h1:5/PjkGUjvEU5Gl6BxmvKRPpqo2uNMv4rcHBMwzk/st8= +golang.org/x/sys v0.0.0-20200622214017-ed371f2e16b4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200728102440-3e129f6d46b1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20201110211018-35f3e6cf4a65/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20201112073958-5cba982894dd h1:5CtCZbICpIOFdgO940moixOPjc0178IU44m4EjOO5IY= +golang.org/x/sys v0.0.0-20201112073958-5cba982894dd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/text v0.0.0-20160726164857-2910a502d2bf/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= +golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.0.0-20170915090833-1cbadb444a80/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.2 h1:tW2bmiBqwgJj/UpqtC8EpXEZVYOwU0yG4iWbprSVAcs= golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk= +golang.org/x/text v0.3.3 h1:cokOdA+Jmi5PJGXLlLllQSgYigAEfHXJAERHVMaCc2k= +golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= +golang.org/x/text v0.3.4 h1:0YWbFKbhXG/wIiuHDSKpS0Iy7FSA+u45VtBMfQcFTTc= +golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20190308202827-9d24e82272b4 h1:SvFZT6jyqRaOeXpc5h/JSfZenJ2O330aBsf7JfSUXmQ= golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= +golang.org/x/time v0.0.0-20191024005414-555d28b269f0 h1:/5xXl8Y5W96D+TtHSlonuFqGHIWVuyCkGJLwGh9JJFs= +golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= +golang.org/x/time v0.0.0-20200630173020-3af7569d3a1e h1:EHBhcS0mlXEAVwNyO2dLfjToGsyY4j24pTs2ScHnX7s= +golang.org/x/time v0.0.0-20200630173020-3af7569d3a1e/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/tools v0.0.0-20170915040203-e531a2a1c15f/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20180221164845-07fd8470d635/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20180525024113-a5b4c53f6e8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= @@ -636,12 +990,39 @@ golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBn golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= golang.org/x/tools v0.0.0-20190521203540-521d6ed310dd/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= +golang.org/x/tools v0.0.0-20190606124116-d0a3d012864b/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= golang.org/x/tools v0.0.0-20190614205625-5aca471b1d59/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= golang.org/x/tools v0.0.0-20190617190820-da514acc4774/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= +golang.org/x/tools v0.0.0-20190624222133-a101b041ded4/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= +golang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= +golang.org/x/tools v0.0.0-20190816200558-6889da9d5479/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20190909030654-5b82db07426d/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20190911174233-4f2ddba30aff/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20190920225731-5eefd052ad72/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20191012152004-8de300cfc20a/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20191112195655-aa38f8e97acc/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20191113191852-77e3bb0ad9e7/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20191115202509-3a792d9c32b2/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20191125144606-a911d9008d1f/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20191130070609-6e064ea0cf2d/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20191216173652-a0e659d51361/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20191227053925-7b8e75db28f4/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200117161641-43d50277825c/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200122220014-bf1340f18c4a/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200204074204-1cc6d1ef6c74/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200207183749-b753a1ba74fa/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200212150539-ea181f53ac56/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200224181240-023911ca70b2/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200304193943-95d2e580d8eb/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw= +golang.org/x/tools v0.0.0-20200505023115-26f46d2f7ef8/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= +golang.org/x/tools v0.0.0-20200616133436-c1934b75d054/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= gonum.org/v1/gonum v0.0.0-20180816165407-929014505bf4/go.mod h1:Y+Yx5eoAFn32cQvJDxZx5Dpnq+c3wtXuadVZAcxbbBo= gonum.org/v1/gonum v0.0.0-20190331200053-3d26580ed485/go.mod h1:2ltnJ7xHfj0zHS40VVPYEAAMTa3ZGguvHGBSJeRWqE0= gonum.org/v1/gonum v0.6.2/go.mod h1:9mxDZsDKxgMAuccQkewq682L+0eCu4dCN2yonUJTCLU= @@ -650,27 +1031,77 @@ gonum.org/v1/netlib v0.0.0-20190331212654-76723241ea4e/go.mod h1:kS+toOQn6AQKjmK gonum.org/v1/plot v0.0.0-20190515093506-e2840ee46a6b/go.mod h1:Wt8AAjI+ypCyYX3nZBvf6cAIx93T+c/OS2HFAYskSZc= google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE= google.golang.org/api v0.6.1-0.20190607001116-5213b8090861/go.mod h1:btoxGiFvQNVUZQ8W08zLtrVS08CNpINPEfxXxgJL1Q4= +google.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M= +google.golang.org/api v0.8.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg= +google.golang.org/api v0.9.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg= +google.golang.org/api v0.13.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= +google.golang.org/api v0.14.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= +google.golang.org/api v0.15.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= +google.golang.org/api v0.15.1-0.20200106000736-b8fc810ca6b5/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= +google.golang.org/api v0.15.1/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= +google.golang.org/api v0.17.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= +google.golang.org/api v0.18.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= +google.golang.org/api v0.20.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM= google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= google.golang.org/appengine v1.5.0 h1:KxkO13IPW4Lslp2bz+KHP2E3gtFlrIGNThxkZQ3g+4c= google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= +google.golang.org/appengine v1.6.1/go.mod h1:i06prIuMbXzDqacNJfV5OdTW448YApPu5ww/cMBSeb0= +google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc= google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc= google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= google.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= +google.golang.org/genproto v0.0.0-20190425155659-357c62f0e4bb/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= google.golang.org/genproto v0.0.0-20190502173448-54afdca5d873/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= +google.golang.org/genproto v0.0.0-20190801165951-fa694d86fc64/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc= google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55 h1:gSJIx1SDwno+2ElGhA4+qG2zF97qiUzTM+rQ0klBOcE= google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc= +google.golang.org/genproto v0.0.0-20190911173649-1774047e7e51/go.mod h1:IbNlFCBrqXvoKpeg0TB2l7cyZUmoaFKYIwrEpbDKLA8= +google.golang.org/genproto v0.0.0-20191108220845-16a3f7862a1a/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= +google.golang.org/genproto v0.0.0-20191115194625-c23dd37a84c9/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= +google.golang.org/genproto v0.0.0-20191216164720-4f79533eabd1/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= +google.golang.org/genproto v0.0.0-20191230161307-f3c370f40bfb/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= +google.golang.org/genproto v0.0.0-20200115191322-ca5a22157cba/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= +google.golang.org/genproto v0.0.0-20200117163144-32f20d992d24/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= +google.golang.org/genproto v0.0.0-20200122232147-0452cf42e150/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= +google.golang.org/genproto v0.0.0-20200204135345-fa8e72b47b90/go.mod h1:GmwEX6Z4W5gMy59cAlVYjN9JhxgbQH6Gn+gFDQe2lzA= +google.golang.org/genproto v0.0.0-20200212174721-66ed5ce911ce/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= +google.golang.org/genproto v0.0.0-20200224152610-e50cd9704f63/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= +google.golang.org/genproto v0.0.0-20200305110556-506484158171/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= +google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013 h1:+kGHl1aib/qcwaRi1CbqBZ1rk19r85MNUf8HaBghugY= +google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo= +google.golang.org/genproto v0.0.0-20201110150050-8816d57aaa9a h1:pOwg4OoaRYScjmR4LlLgdtnyoHYTSAVhhqe5uPdpII8= +google.golang.org/genproto v0.0.0-20201110150050-8816d57aaa9a/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c= google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38= +google.golang.org/grpc v1.21.0/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM= +google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM= google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg= google.golang.org/grpc v1.23.1/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg= google.golang.org/grpc v1.26.0 h1:2dTRdpdFEEhJYQD8EMLB61nnrzSCTbG38PhqdhvOltg= google.golang.org/grpc v1.26.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk= +google.golang.org/grpc v1.27.0 h1:rRYRFMVgRv6E0D70Skyfsr28tDXIuuPZyWGMPdMcnXg= +google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk= +google.golang.org/grpc v1.27.1 h1:zvIju4sqAGvwKspUQOhwnpcqSbzi7/H6QomNNjTL4sk= +google.golang.org/grpc v1.27.1/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk= +google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8= +google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0= +google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM= +google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE= +google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo= +google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= +google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= +google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= +google.golang.org/protobuf v1.24.0 h1:UhZDfRO8JRQru4/+LlLE0BRKGF8L+PICnvYZmx/fEGA= +google.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGjtUeSXeh4= +google.golang.org/protobuf v1.25.0 h1:Ejskq+SyPohKW+1uil0JJMtmHCgJPJ/qWTxr8qp+R4c= +google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c= gopkg.in/airbrake/gobrake.v2 v2.0.9/go.mod h1:/h5ZAUhDkGaJfjzjKLSjv6zCL6O0LLBxU4K+aSYdM/U= gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 h1:qIbj1fsPNlZgppZ+VLlY7N33q108Sa+fhmuc+sWQYwY= gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/cheggaaa/pb.v1 v1.0.25/go.mod h1:V/YB90LKu/1FcN3WVnfiiE5oMCibMjukxqG/qStrOgw= gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI= gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys= @@ -678,6 +1109,7 @@ gopkg.in/gcfg.v1 v1.2.0/go.mod h1:yesOnuUOFQAhST5vPY4nbZsb/huCgGGXlipJsBn0b3o= gopkg.in/gemnasium/logrus-airbrake-hook.v2 v2.1.2/go.mod h1:Xk6kEKp8OKb+X14hQBKWaSkCsqBpgog8nAV2xsGOxlo= gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc= gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw= +gopkg.in/ini.v1 v1.51.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k= gopkg.in/mcuadros/go-syslog.v2 v2.2.1/go.mod h1:l5LPIyOOyIdQquNg+oU6Z3524YwrcqEm0aKH+5zpt2U= gopkg.in/natefinch/lumberjack.v2 v2.0.0/go.mod h1:l0ndWWf7gzL7RNwBG7wST/UCcT4T24xpD6X8LsfU/+k= gopkg.in/resty.v1 v1.12.0/go.mod h1:mDo4pnntr5jdWRML875a/NmxYqAlA73dVijT2AXvQQo= @@ -688,57 +1120,141 @@ gopkg.in/yaml.v2 v2.0.0-20170812160011-eb3733d160e7/go.mod h1:JAlM8MvJe8wmxCU4Bl gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= +gopkg.in/yaml.v2 v2.2.5/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.8 h1:obN1ZagJSUGI0Ek/LBmuj4SNLPfIny3KsKFopxRdj10= gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= +gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gotest.tools v2.1.0+incompatible/go.mod h1:DsYFclhRJ6vuDpmuTbkuFWG+y2sxOXAzmJt81HFBacw= gotest.tools v2.2.0+incompatible/go.mod h1:DsYFclhRJ6vuDpmuTbkuFWG+y2sxOXAzmJt81HFBacw= gotest.tools/gotestsum v0.3.5/go.mod h1:Mnf3e5FUzXbkCfynWBGOwLssY7gTQgCHObK9tMpAriY= +gotest.tools/v3 v3.0.2/go.mod h1:3SzNCllyD9/Y+b5r9JIKQ474KzkZyqLqEfYqMsX94Bk= grpc.go4.org v0.0.0-20170609214715-11d0a25b4919/go.mod h1:77eQGdRu53HpSqPFJFmuJdjuHRquDANNeA4x7B8WQ9o= honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.1-2019.2.2/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg= +honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg= +honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k= k8s.io/api v0.18.4 h1:8x49nBRxuXGUlDlwlWd3RMY1SayZrzFfxea3UZSkFw4= k8s.io/api v0.18.4/go.mod h1:lOIQAKYgai1+vz9J7YcDZwC26Z0zQewYOGWdyIPUUQ4= +k8s.io/api v0.19.0 h1:XyrFIJqTYZJ2DU7FBE/bSPz7b1HvbVBuBf07oeo6eTc= +k8s.io/api v0.19.0/go.mod h1:I1K45XlvTrDjmj5LoM5LuP/KYrhWbjUKT/SoPG0qTjw= +k8s.io/api v0.20.0 h1:WwrYoZNM1W1aQEbyl8HNG+oWGzLpZQBlcerS9BQw9yI= +k8s.io/api v0.20.0/go.mod h1:HyLC5l5eoS/ygQYl1BXBgFzWNlkHiAuyNAbevIn+FKg= k8s.io/apiextensions-apiserver v0.18.4/go.mod h1:NYeyeYq4SIpFlPxSAB6jHPIdvu3hL0pc36wuRChybio= +k8s.io/apiextensions-apiserver v0.19.0/go.mod h1:znfQxNpjqz/ZehvbfMg5N6fvBJW5Lqu5HVLTJQdP4Fs= +k8s.io/apiextensions-apiserver v0.20.0/go.mod h1:ZH+C33L2Bh1LY1+HphoRmN1IQVLTShVcTojivK3N9xg= k8s.io/apimachinery v0.18.4 h1:ST2beySjhqwJoIFk6p7Hp5v5O0hYY6Gngq/gUYXTPIA= k8s.io/apimachinery v0.18.4/go.mod h1:OaXp26zu/5J7p0f92ASynJa1pZo06YlV9fG7BoWbCko= +k8s.io/apimachinery v0.19.0 h1:gjKnAda/HZp5k4xQYjL0K/Yb66IvNqjthCb03QlKpaQ= +k8s.io/apimachinery v0.19.0/go.mod h1:DnPGDnARWFvYa3pMHgSxtbZb7gpzzAZ1pTfaUNDVlmA= +k8s.io/apimachinery v0.20.0 h1:jjzbTJRXk0unNS71L7h3lxGDH/2HPxMPaQY+MjECKL8= +k8s.io/apimachinery v0.20.0/go.mod h1:WlLqWAHZGg07AeltaI0MV5uk1Omp8xaN0JGLY6gkRpU= k8s.io/apiserver v0.18.4 h1:pn1jSQkfboPSirZopkVpEdLW4FcQLnYMaIY8LFxxj30= k8s.io/apiserver v0.18.4/go.mod h1:q+zoFct5ABNnYkGIaGQ3bcbUNdmPyOCoEBcg51LChY8= +k8s.io/apiserver v0.19.0 h1:jLhrL06wGAADbLUUQm8glSLnAGP6c7y5R3p19grkBoY= +k8s.io/apiserver v0.19.0/go.mod h1:XvzqavYj73931x7FLtyagh8WibHpePJ1QwWrSJs2CLk= +k8s.io/apiserver v0.20.0 h1:0MwO4xCoqZwhoLbFyyBSJdu55CScp4V4sAgX6z4oPBY= +k8s.io/apiserver v0.20.0/go.mod h1:6gRIWiOkvGvQt12WTYmsiYoUyYW0FXSiMdNl4m+sxY8= k8s.io/cli-runtime v0.18.4/go.mod h1:9/hS/Cuf7NVzWR5F/5tyS6xsnclxoPLVtwhnkJG1Y4g= +k8s.io/cli-runtime v0.19.0/go.mod h1:tun9l0eUklT8IHIM0jors17KmUjcrAxn0myoBYwuNuo= +k8s.io/cli-runtime v0.20.0/go.mod h1:C5tewU1SC1t09D7pmkk83FT4lMAw+bvMDuRxA7f0t2s= k8s.io/client-go v0.18.4 h1:un55V1Q/B3JO3A76eS0kUSywgGK/WR3BQ8fHQjNa6Zc= k8s.io/client-go v0.18.4/go.mod h1:f5sXwL4yAZRkAtzOxRWUhA/N8XzGCb+nPZI8PfobZ9g= +k8s.io/client-go v0.19.0 h1:1+0E0zfWFIWeyRhQYWzimJOyAk2UT7TiARaLNwJCf7k= +k8s.io/client-go v0.19.0/go.mod h1:H9E/VT95blcFQnlyShFgnFT9ZnJOAceiUHM3MlRC+mU= +k8s.io/client-go v0.20.0 h1:Xlax8PKbZsjX4gFvNtt4F5MoJ1V5prDvCuoq9B7iax0= +k8s.io/client-go v0.20.0/go.mod h1:4KWh/g+Ocd8KkCwKF8vUNnmqgv+EVnQDK4MBF4oB5tY= k8s.io/cloud-provider v0.18.4/go.mod h1:JdI6cuSFPSPANEciv0v5qfwztkeyFCVc1S3krLYrw0E= +k8s.io/cloud-provider v0.19.0 h1:Ae09nHr6BVPEzmAWbZedYC0gjsIPbt7YsIY0V/NHGr0= +k8s.io/cloud-provider v0.19.0/go.mod h1:TYh7b7kQ6wiqF7Ftb+u3lN4IwvgOPbBrcvC3TDAW4cw= +k8s.io/cloud-provider v0.20.0 h1:CVPQ66iyfNgeGomUq2jE/TWrfzE77bdCpemhFS8955U= +k8s.io/cloud-provider v0.20.0/go.mod h1:Lz/luSVD5BrHDDhtVdjFh0C2qQCRYdf0b9BHQ9L+bXc= k8s.io/cluster-bootstrap v0.18.4/go.mod h1:hNG705ec9SMN2BGlJ81R2CnyJjNKfROtAxvI9JXZdiM= +k8s.io/cluster-bootstrap v0.19.0/go.mod h1:kBn1DKyqoM245wzz+AAnGkuysJ+9GqVbPYveTo4KiaA= +k8s.io/cluster-bootstrap v0.20.0/go.mod h1:6WZaNIBvcvL7MkPzSRKrZDIr4u+ePW2oIWoRsEFMjmE= k8s.io/code-generator v0.18.4/go.mod h1:TgNEVx9hCyPGpdtCWA34olQYLkh3ok9ar7XfSsr8b6c= +k8s.io/code-generator v0.19.0/go.mod h1:moqLn7w0t9cMs4+5CQyxnfA/HV8MF6aAVENF+WZZhgk= +k8s.io/code-generator v0.20.0/go.mod h1:UsqdF+VX4PU2g46NC2JRs4gc+IfrctnwHb76RNbWHJg= k8s.io/component-base v0.18.4 h1:Kr53Fp1iCGNsl9Uv4VcRvLy7YyIqi9oaJOQ7SXtKI98= k8s.io/component-base v0.18.4/go.mod h1:7jr/Ef5PGmKwQhyAz/pjByxJbC58mhKAhiaDu0vXfPk= +k8s.io/component-base v0.19.0 h1:OueXf1q3RW7NlLlUCj2Dimwt7E1ys6ZqRnq53l2YuoE= +k8s.io/component-base v0.19.0/go.mod h1:dKsY8BxkA+9dZIAh2aWJLL/UdASFDNtGYTCItL4LM7Y= +k8s.io/component-base v0.20.0 h1:BXGL8iitIQD+0NgW49UsM7MraNUUGDU3FBmrfUAtmVQ= +k8s.io/component-base v0.20.0/go.mod h1:wKPj+RHnAr8LW2EIBIK7AxOHPde4gme2lzXwVSoRXeA= +k8s.io/component-helpers v0.20.0/go.mod h1:nx6NOtfSfGOxnSZsDJxpGbnsVuUA1UXpwDvZIrtigNk= +k8s.io/controller-manager v0.20.0/go.mod h1:nD4qym/pmCz2v1tpqvlEBVlHW9CAZwedloM8GrJTLpg= k8s.io/cri-api v0.18.4/go.mod h1:OJtpjDvfsKoLGhvcc0qfygved0S0dGX56IJzPbqTG1s= +k8s.io/cri-api v0.19.0/go.mod h1:UN/iU9Ua0iYdDREBXNE9vqCJ7MIh/FW3VIL0d8pw7Fw= +k8s.io/cri-api v0.20.0/go.mod h1:2JRbKt+BFLTjtrILYVqQK5jqhI+XNdF6UiGMgczeBCI= k8s.io/csi-translation-lib v0.18.4/go.mod h1:FTci2m8/3oN8E+8OyblBXei8w4mwbiH4boNPeob4piE= +k8s.io/csi-translation-lib v0.19.0/go.mod h1:zGS1YqV8U2So/t4Hz8SoRXMx5y5/KSKnA6BXXxGuo4A= +k8s.io/csi-translation-lib v0.20.0/go.mod h1:M4CdD66GxEI6ev8aTtsA2NkK9kIF9K5VZQMcw/SsoLs= k8s.io/gengo v0.0.0-20190128074634-0689ccc1d7d6/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8IAqLxYwwyPxAX1Pzy0ii0= k8s.io/gengo v0.0.0-20200114144118-36b2048a9120/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8IAqLxYwwyPxAX1Pzy0ii0= +k8s.io/gengo v0.0.0-20200413195148-3a45101e95ac/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8IAqLxYwwyPxAX1Pzy0ii0= +k8s.io/gengo v0.0.0-20200428234225-8167cfdcfc14/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8IAqLxYwwyPxAX1Pzy0ii0= +k8s.io/gengo v0.0.0-20201113003025-83324d819ded/go.mod h1:FiNAH4ZV3gBg2Kwh89tzAEV2be7d5xI0vBa/VySYy3E= k8s.io/heapster v1.2.0-beta.1/go.mod h1:h1uhptVXMwC8xtZBYsPXKVi8fpdlYkTs6k949KozGrM= k8s.io/klog v0.0.0-20181102134211-b9b56d5dfc92/go.mod h1:Gq+BEi5rUBO/HRz0bTSXDUcqjScdoY3a9IHpCEIOOfk= k8s.io/klog v0.3.0/go.mod h1:Gq+BEi5rUBO/HRz0bTSXDUcqjScdoY3a9IHpCEIOOfk= k8s.io/klog v1.0.0 h1:Pt+yjF5aB1xDSVbau4VsWe+dQNzA0qv1LlXdC2dF6Q8= k8s.io/klog v1.0.0/go.mod h1:4Bi6QPql/J/LkTDqv7R/cd3hPo4k2DG6Ptcz060Ez5I= +k8s.io/klog/v2 v2.0.0/go.mod h1:PBfzABfn139FHAV07az/IF9Wp1bkk3vpT2XSJ76fSDE= +k8s.io/klog/v2 v2.2.0 h1:XRvcwJozkgZ1UQJmfMGpvRthQHOvihEhYtDfAaxMz/A= +k8s.io/klog/v2 v2.2.0/go.mod h1:Od+F08eJP+W3HUb4pSrPpgp9DGU4GzlpG/TmITuYh/Y= +k8s.io/klog/v2 v2.4.0 h1:7+X0fUguPyrKEC4WjH8iGDg3laWgMo5tMnRTIGTTxGQ= +k8s.io/klog/v2 v2.4.0/go.mod h1:Od+F08eJP+W3HUb4pSrPpgp9DGU4GzlpG/TmITuYh/Y= k8s.io/kube-aggregator v0.18.4/go.mod h1:xOVy4wqhpivXCt07Diwdms2gonG+SONVx+1e7O+GfC0= +k8s.io/kube-aggregator v0.19.0/go.mod h1:1Ln45PQggFAG8xOqWPIYMxUq8WNtpPnYsbUJ39DpF/A= +k8s.io/kube-aggregator v0.20.0/go.mod h1:3Is/gzzWmhhG/rA3CpA1+eVye87lreBQDFGcAGT7gzo= k8s.io/kube-controller-manager v0.18.4/go.mod h1:GrY1S0F7zA0LQlt0ApOLt4iMpphKTk3mFrQl1+usrfs= +k8s.io/kube-controller-manager v0.19.0/go.mod h1:uGZyiHK73NxNEN5EZv/Esm3fbCOzeq4ndttMexVZ1L0= +k8s.io/kube-controller-manager v0.20.0/go.mod h1:Pmli7dnwIVpwKJVeab97yBt35QEFdw65oqT5ti0ikUs= k8s.io/kube-openapi v0.0.0-20200410145947-61e04a5be9a6/go.mod h1:GRQhZsXIAJ1xR0C9bd8UpWHZ5plfAS9fzPjJuQ6JL3E= +k8s.io/kube-openapi v0.0.0-20200805222855-6aeccd4b50c6/go.mod h1:UuqjUnNftUyPE5H64/qeyjQoUZhGpeFDVdxjTeEVN2o= +k8s.io/kube-openapi v0.0.0-20201113171705-d219536bb9fd/go.mod h1:WOJ3KddDSol4tAGcJo0Tvi+dK12EcqSLqcWsryKMpfM= k8s.io/kube-proxy v0.18.4/go.mod h1:h2c+ckQC1XpybDs53mWhLCvvM6txduWVLPQwwvGqR9M= +k8s.io/kube-proxy v0.19.0/go.mod h1:7NoJCFgsWb7iiMB1F6bW1St5rEXC+ir2aWiJehASmTU= +k8s.io/kube-proxy v0.20.0/go.mod h1:R97oobM6zSh3ZqFMXi5DzCH/qJXNzua/UzcDmuQRexM= k8s.io/kube-scheduler v0.18.4/go.mod h1:vRFb/8Yi7hh670beaPrXttMpjt7H8EooDkgwFm8ts4k= +k8s.io/kube-scheduler v0.19.0/go.mod h1:1XGjJUgstM0/0x8to+bSGSyCs3Dp3dbCEr3Io/mvd4s= +k8s.io/kube-scheduler v0.20.0/go.mod h1:cRTGsJU3TfQvbMJBmpoPgq9rBF5cQLpLKoOafKwdZnI= k8s.io/kubectl v0.18.4/go.mod h1:EzB+nfeUWk6fm6giXQ8P4Fayw3dsN+M7Wjy23mTRtB0= +k8s.io/kubectl v0.19.0/go.mod h1:gPCjjsmE6unJzgaUNXIFGZGafiUp5jh0If3F/x7/rRg= +k8s.io/kubectl v0.20.0/go.mod h1:8x5GzQkgikz7M2eFGGuu6yOfrenwnw5g4RXOUgbjR1M= k8s.io/kubelet v0.18.4/go.mod h1:D0V9JYaTJRF+ry+9JfnM4uyg3ySRLQ02XjfQ5f2u4CM= +k8s.io/kubelet v0.19.0/go.mod h1:cGds22piF/LnFzfAaIT+efvOYBHVYdunqka6NVuNw9g= +k8s.io/kubelet v0.20.0/go.mod h1:lMdjO1NA+JZXSYtxb48pQmNERmC+vVIXIYkJIugVhl0= k8s.io/kubernetes v1.18.4 h1:AYtJ24PIT91P1K8ekCrvay8LK8WctWhC5+NI0HZ8sqE= k8s.io/kubernetes v1.18.4/go.mod h1:Efg82S+Ti02A/Mww53bxroc7IgzX2bgPsf6hT8gAs3M= +k8s.io/kubernetes v1.19.0 h1:ir53YuXsfsuVABmtYHCTUa3xjD41Htxv3o+xoQjJdUo= +k8s.io/kubernetes v1.19.0/go.mod h1:yhT1/ltQajQsha3tnYc9QPFYSumGM45nlZdjf7WqE1A= +k8s.io/kubernetes v1.20.0 h1:mnc69esJC3PJgSptxNJomGz2gBthyGLSEy18WiyRH4U= +k8s.io/kubernetes v1.20.0/go.mod h1:/xrHGNfoQphtkhZvyd5bA1lRmz+QkDVmBZu+O8QMoek= +k8s.io/kubernetes v1.20.2 h1:EsQROw+yFsDMfjEHp52cKs4JVI6lAHA2SHGAF88cK7s= k8s.io/legacy-cloud-providers v0.18.4/go.mod h1:Mnxtra7DxVrODfGZHPsrkLi22lwmZOlWkjyyO3vW+WM= +k8s.io/legacy-cloud-providers v0.19.0/go.mod h1:Q5czDCPnStdpFohMpcbnqL+MLR75kUhIDIsnmwEm0/o= +k8s.io/legacy-cloud-providers v0.20.0/go.mod h1:1jEkaU7h9+b1EYdfWDBvhFAr+QpRfUjQfK+dGhxPGfA= k8s.io/metrics v0.18.4/go.mod h1:luze4fyI9JG4eLDZy0kFdYEebqNfi0QrG4xNEbPkHOs= +k8s.io/metrics v0.19.0/go.mod h1:WykpW8B60OeAJx1imdwUgyOID2kDljr/Q+1zrPJ98Wo= +k8s.io/metrics v0.20.0/go.mod h1:9yiRhfr8K8sjdj2EthQQE9WvpYDvsXIV3CjN4Ruq4Jw= +k8s.io/mount-utils v0.20.0/go.mod h1:Jv9NRZ5L2LF87A17GaGlArD+r3JAJdZFvo4XD1cG4Kc= k8s.io/repo-infra v0.0.1-alpha.1/go.mod h1:wO1t9WaB99V80ljbeENTnayuEEwNZt7gECYh/CEyOJ8= k8s.io/sample-apiserver v0.18.4/go.mod h1:j5XH5FUmMd/ztoz+9ch0+hL+lsvWdgxnTV7l3P3Ijoo= +k8s.io/sample-apiserver v0.19.0/go.mod h1:Bq9UulNoKnT72JqlkWF2JS14cXxJqcmvLtb5+EcwiNA= +k8s.io/sample-apiserver v0.20.0/go.mod h1:tScvbz/BcUG46IOsu2YLt4EjBP7XeUuMzMbQt2tQYWw= k8s.io/system-validators v1.0.4/go.mod h1:HgSgTg4NAGNoYYjKsUyk52gdNi2PVDswQ9Iyn66R7NI= +k8s.io/system-validators v1.1.2/go.mod h1:bPldcLgkIUK22ALflnsXk8pvkTEndYdNuaHH6gRrl0Q= +k8s.io/system-validators v1.2.0/go.mod h1:bPldcLgkIUK22ALflnsXk8pvkTEndYdNuaHH6gRrl0Q= k8s.io/utils v0.0.0-20200324210504-a9aa75ae1b89 h1:d4vVOjXm687F1iLSP2q3lyPPuyvTUt3aVoBpi2DqRsU= k8s.io/utils v0.0.0-20200324210504-a9aa75ae1b89/go.mod h1:sZAwmy6armz5eXlNoLmJcl4F1QuKu7sr+mFQ0byX7Ew= +k8s.io/utils v0.0.0-20200414100711-2df71ebbae66/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA= +k8s.io/utils v0.0.0-20200729134348-d5654de09c73 h1:uJmqzgNWG7XyClnU/mLPBWwfKKF1K8Hf8whTseBgJcg= +k8s.io/utils v0.0.0-20200729134348-d5654de09c73/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA= +k8s.io/utils v0.0.0-20201110183641-67b214c5f920 h1:CbnUZsM497iRC5QMVkHwyl8s2tB3g7yaSHkYPkpgelw= +k8s.io/utils v0.0.0-20201110183641-67b214c5f920/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA= modernc.org/cc v1.0.0/go.mod h1:1Sk4//wdnYJiUIxnW8ddKpaOJCF37yAdqYnkxUpaYxw= modernc.org/golex v1.0.0/go.mod h1:b/QX9oBD/LhixY6NDh+IdGv17hgB+51fET1i2kPSmvk= modernc.org/mathutil v1.0.0/go.mod h1:wU0vUrJsVWBZ4P6e7xtFJEhFSNsfRLJ8H458uRjg03k= @@ -747,13 +1263,24 @@ modernc.org/xc v1.0.0/go.mod h1:mRNCo0bvLjGhHO9WsyuKVU4q0ceiDDDoEeWDJHrNx8I= mvdan.cc/interfacer v0.0.0-20180901003855-c20040233aed/go.mod h1:Xkxe497xwlCKkIaQYRfC7CSLworTXY9RMqwhhCm+8Nc= mvdan.cc/lint v0.0.0-20170908181259-adc824a0674b/go.mod h1:2odslEg/xrtNQqCYg2/jCoyKnw3vv5biOc3JnIcYfL4= mvdan.cc/unparam v0.0.0-20190209190245-fbb59629db34/go.mod h1:H6SUd1XjIs+qQCyskXg5OFSrilMRUkD8ePJpHKDPaeY= +rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8= rsc.io/pdf v0.1.1/go.mod h1:n8OzWcQ6Sp37PL01nO98y4iUCRdTGarVfzxY20ICaU4= +rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0= +rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA= sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.0.7 h1:uuHDyjllyzRyCIvvn0OBjiRB0SgBZGqHNYAmjR7fO50= sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.0.7/go.mod h1:PHgbrJT7lCHcxMU+mDHEm+nx46H4zuuHZkDP6icnhu0= +sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.0.9 h1:rusRLrDhjBp6aYtl9sGEvQJr6faoHoDLd0YcUBTZguI= +sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.0.9/go.mod h1:dzAXnQbTRyDlZPJX2SUPEqvnB+j7AJjtlox7PEwigU0= +sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.0.14 h1:TihvEz9MPj2u0KWds6E2OBUXfwaL4qRJ33c7HGiJpqk= +sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.0.14/go.mod h1:LEScyzhFmoF5pso/YSeBstl57mOzx9xlU9n85RGrDQg= sigs.k8s.io/kustomize v2.0.3+incompatible/go.mod h1:MkjgH3RdOWrievjo6c9T245dYlB5QeXV4WCbnt/PEpU= sigs.k8s.io/structured-merge-diff/v3 v3.0.0-20200116222232-67a7b8c61874/go.mod h1:PlARxl6Hbt/+BC80dRLi1qAmnMqwqDg62YvvVkZjemw= sigs.k8s.io/structured-merge-diff/v3 v3.0.0 h1:dOmIZBMfhcHS09XZkMyUgkq5trg3/jRyJYFZUiaOp8E= sigs.k8s.io/structured-merge-diff/v3 v3.0.0/go.mod h1:PlARxl6Hbt/+BC80dRLi1qAmnMqwqDg62YvvVkZjemw= +sigs.k8s.io/structured-merge-diff/v4 v4.0.1 h1:YXTMot5Qz/X1iBRJhAt+vI+HVttY0WkSqqhKxQ0xVbA= +sigs.k8s.io/structured-merge-diff/v4 v4.0.1/go.mod h1:bJZC9H9iH24zzfZ/41RGcq60oK1F7G282QMXDPYydCw= +sigs.k8s.io/structured-merge-diff/v4 v4.0.2 h1:YHQV7Dajm86OuqnIR6zAelnDWBRjo+YhYV9PmGrh1s8= +sigs.k8s.io/structured-merge-diff/v4 v4.0.2/go.mod h1:bJZC9H9iH24zzfZ/41RGcq60oK1F7G282QMXDPYydCw= sigs.k8s.io/yaml v1.1.0/go.mod h1:UJmg0vDUVViEyp3mgSv9WPwZCDxu4rQW1olrI1uml+o= sigs.k8s.io/yaml v1.2.0 h1:kr/MCeFWJWTwyaHoR9c8EjH9OumOmoF9YGiZd7lFm/Q= sigs.k8s.io/yaml v1.2.0/go.mod h1:yfXDCHCao9+ENCvLSE62v9VSji2MKu5jeNfTrofGhJc= From 9cc4fde6678f8684e0467758e1a520e8ed2ca0ba Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Salvador=20Gonz=C3=A1lez=20Gonz=C3=A1lez?= Date: Sun, 17 Jan 2021 09:36:18 +0000 Subject: [PATCH 027/190] Remove v2 controller mention I removed mention to v2 controller behaviour. --- content/en/docs/concepts/workloads/controllers/cron-jobs.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/concepts/workloads/controllers/cron-jobs.md b/content/en/docs/concepts/workloads/controllers/cron-jobs.md index a7e1f03097..11e30b203d 100644 --- a/content/en/docs/concepts/workloads/controllers/cron-jobs.md +++ b/content/en/docs/concepts/workloads/controllers/cron-jobs.md @@ -62,7 +62,7 @@ and if `concurrencyPolicy` is set to `Allow`, the jobs will always run at least once. {{< caution >}} -If `startingDeadlineSeconds` is set to a value less than 10 seconds, the CronJob may not be scheduled. This is because the CronJob controller checks things every 10 seconds. As an alpha feature, there is a V2 controller implementation with different behavior. The version 2 controller provides experimental performance improvements. +If `startingDeadlineSeconds` is set to a value less than 10 seconds, the CronJob may not be scheduled. This is because the CronJob controller checks things every 10 seconds. {{< /caution >}} From a12da52b4ddf1976bab5dc2f61fed22704e26d38 Mon Sep 17 00:00:00 2001 From: Ryuichiroh Ikeuchi <37844673+ryicoh@users.noreply.github.com> Date: Tue, 19 Jan 2021 12:48:55 +0900 Subject: [PATCH 028/190] Update content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md Co-authored-by: nasa9084 --- .../manage-resources/memory-constraint-namespace.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md index 43b51f8d3a..452bbb5838 100644 --- a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md +++ b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -1,5 +1,5 @@ --- -title: Namespaceに対する最小および最大メモリ制約の構成 +title: Namespaceに対する最小および最大メモリー制約の構成 content_type: task weight: 30 @@ -262,4 +262,3 @@ kubectl delete namespace constraints-mem-example - From d8f38a2e1a5a7b2a22fd2e394ac9b36f2b20d908 Mon Sep 17 00:00:00 2001 From: Ryuichiroh Ikeuchi <37844673+ryicoh@users.noreply.github.com> Date: Tue, 19 Jan 2021 12:49:05 +0900 Subject: [PATCH 029/190] Update content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md Co-authored-by: nasa9084 --- .../manage-resources/memory-constraint-namespace.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md index 452bbb5838..5cfeea4c0e 100644 --- a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md +++ b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -8,7 +8,7 @@ weight: 30 -このページでは、Namespaceで実行されるコンテナが使用するメモリの最小値と最大値を設定する方法を説明します。 +このページでは、Namespaceで実行されるコンテナが使用するメモリーの最小値と最大値を設定する方法を説明します。 [LimitRange](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#limitrange-v1-core) で最小値と最大値のメモリ値を指定します。 PodがLimitRangeによって課される制約を満たさない場合、そのNamespaceではPodを作成できません。 @@ -261,4 +261,3 @@ kubectl delete namespace constraints-mem-example - From ed5895073c27729503fc1fb0a31217efc96b95bc Mon Sep 17 00:00:00 2001 From: Ryuichiroh Ikeuchi <37844673+ryicoh@users.noreply.github.com> Date: Tue, 19 Jan 2021 12:49:12 +0900 Subject: [PATCH 030/190] Update content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md Co-authored-by: nasa9084 --- .../manage-resources/memory-constraint-namespace.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md index 5cfeea4c0e..e12f410253 100644 --- a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md +++ b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -9,7 +9,7 @@ weight: 30 このページでは、Namespaceで実行されるコンテナが使用するメモリーの最小値と最大値を設定する方法を説明します。 -[LimitRange](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#limitrange-v1-core) で最小値と最大値のメモリ値を指定します。 +[LimitRange](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#limitrange-v1-core) で最小値と最大値のメモリー値を指定します。 PodがLimitRangeによって課される制約を満たさない場合、そのNamespaceではPodを作成できません。 @@ -260,4 +260,3 @@ kubectl delete namespace constraints-mem-example - From d78a000b98821afc1c8796371bf528fdcbdf87ae Mon Sep 17 00:00:00 2001 From: Ryuichiroh Ikeuchi <37844673+ryicoh@users.noreply.github.com> Date: Tue, 19 Jan 2021 12:49:19 +0900 Subject: [PATCH 031/190] Update content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md Co-authored-by: nasa9084 --- .../manage-resources/memory-constraint-namespace.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md index e12f410253..8dcb9d4a3c 100644 --- a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md +++ b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -25,7 +25,7 @@ PodがLimitRangeによって課される制約を満たさない場合、そのN ## Namespaceの作成 -この演習で作成したリソースがクラスタの他の部分から分離されるように、Namespaceを作成します。 +この演習で作成したリソースがクラスターの他の部分から分離されるように、Namespaceを作成します。 ```shell @@ -259,4 +259,3 @@ kubectl delete namespace constraints-mem-example - From 8835ad4b7e705d799188d21f19f8e8b583535c04 Mon Sep 17 00:00:00 2001 From: Ryuichiroh Ikeuchi <37844673+ryicoh@users.noreply.github.com> Date: Tue, 19 Jan 2021 12:49:27 +0900 Subject: [PATCH 032/190] Update content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md Co-authored-by: nasa9084 --- .../manage-resources/memory-constraint-namespace.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md index 8dcb9d4a3c..e389adbd8a 100644 --- a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md +++ b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -220,7 +220,7 @@ LimitRangeによってNamespaceに課される最大および最小のメモリ * クラスタ内の各ノードは2GBのメモリを持っています。クラスタ内のどのノードもその要求をサポートできないため、2GB以上のメモリを要求するPodは受け入れたくありません。 -* クラスタは運用部門と開発部門で共有されています。 本番用のワークロードでは最大8GBのメモリを消費しますが、開発用のワークロードでは512MBに制限したいとします。本番用と開発用に別々のNamespaceを作成し、それぞれのNamespaceにメモリ制限を適用します。 +* クラスターは運用部門と開発部門で共有されています。 本番用のワークロードでは最大8GBのメモリーを消費しますが、開発用のワークロードでは512MBに制限したいとします。本番用と開発用に別々のNamespaceを作成し、それぞれのNamespaceにメモリー制限を適用します。 ## クリーンアップ @@ -258,4 +258,3 @@ kubectl delete namespace constraints-mem-example * [PodのQoS(サービス品質)を設定](/docs/tasks/configure-pod-container/quality-service-pod/) - From 2d832a7b9040d09e148ef62968601846f52ded54 Mon Sep 17 00:00:00 2001 From: Ryuichiroh Ikeuchi <37844673+ryicoh@users.noreply.github.com> Date: Tue, 19 Jan 2021 12:49:34 +0900 Subject: [PATCH 033/190] Update content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md Co-authored-by: nasa9084 --- .../manage-resources/memory-constraint-namespace.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md index e389adbd8a..b784588400 100644 --- a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md +++ b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -251,10 +251,9 @@ kubectl delete namespace constraints-mem-example ### アプリケーション開発者向け -* [コンテナとPodへのメモリリソースの割り当て](/docs/tasks/configure-pod-container/assign-memory-resource/) +* [コンテナとPodへのメモリーリソースの割り当て](/docs/tasks/configure-pod-container/assign-memory-resource/) * [コンテナとPodへのCPUリソースの割り当て](/docs/tasks/configure-pod-container/assign-cpu-resource/) * [PodのQoS(サービス品質)を設定](/docs/tasks/configure-pod-container/quality-service-pod/) - From 447d69d2c10a8b0047a01b3bd32185f80d38d10e Mon Sep 17 00:00:00 2001 From: Ryuichiroh Ikeuchi <37844673+ryicoh@users.noreply.github.com> Date: Tue, 19 Jan 2021 12:49:40 +0900 Subject: [PATCH 034/190] Update content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md Co-authored-by: nasa9084 --- .../manage-resources/memory-constraint-namespace.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md index b784588400..d68d4aad87 100644 --- a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md +++ b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -235,7 +235,7 @@ kubectl delete namespace constraints-mem-example ## {{% heading "whatsnext" %}} -### クラスタ管理者向け +### クラスター管理者向け * [名前空間に対するデフォルトのメモリ要求と制限の構成](/docs/tasks/administer-cluster/manage-resources/memory-default-namespace/) @@ -256,4 +256,3 @@ kubectl delete namespace constraints-mem-example * [コンテナとPodへのCPUリソースの割り当て](/docs/tasks/configure-pod-container/assign-cpu-resource/) * [PodのQoS(サービス品質)を設定](/docs/tasks/configure-pod-container/quality-service-pod/) - From bbad30174bda34474a23ee63ad890edbd50d6e77 Mon Sep 17 00:00:00 2001 From: Ryuichiroh Ikeuchi <37844673+ryicoh@users.noreply.github.com> Date: Tue, 19 Jan 2021 12:49:46 +0900 Subject: [PATCH 035/190] Update content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md Co-authored-by: nasa9084 --- .../manage-resources/memory-constraint-namespace.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md index d68d4aad87..1d3b26dfd4 100644 --- a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md +++ b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -243,7 +243,7 @@ kubectl delete namespace constraints-mem-example * [名前空間に対する最小および最大CPU制約の構成](/docs/tasks/administer-cluster/manage-resources/cpu-constraint-namespace/) -* [名前空間に対するメモリとCPUのクォータの構成](/docs/tasks/administer-cluster/manage-resources/quota-memory-cpu-namespace/) +* [名前空間に対するメモリーとCPUのクォータの構成](/docs/tasks/administer-cluster/manage-resources/quota-memory-cpu-namespace/) * [名前空間に対するPodクォータの設定](/docs/tasks/administer-cluster/manage-resources/quota-pod-namespace/) From 6cde1b4a7928126b4cf7a61ebd0586ba25dd1373 Mon Sep 17 00:00:00 2001 From: Ryuichiroh Ikeuchi <37844673+ryicoh@users.noreply.github.com> Date: Tue, 19 Jan 2021 12:49:53 +0900 Subject: [PATCH 036/190] Update content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md Co-authored-by: nasa9084 --- .../manage-resources/memory-constraint-namespace.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md index 1d3b26dfd4..dd99671f34 100644 --- a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md +++ b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -237,7 +237,7 @@ kubectl delete namespace constraints-mem-example ### クラスター管理者向け -* [名前空間に対するデフォルトのメモリ要求と制限の構成](/docs/tasks/administer-cluster/manage-resources/memory-default-namespace/) +* [名前空間に対するデフォルトのメモリー要求と制限の構成](/docs/tasks/administer-cluster/manage-resources/memory-default-namespace/) * [名前空間に対するデフォルトのCPU要求と制限の構成](/docs/tasks/administer-cluster/manage-resources/cpu-default-namespace/) From 62d607b8e1a568dd46087c76f7e03f7b7e60b98a Mon Sep 17 00:00:00 2001 From: Ryuichiroh Ikeuchi <37844673+ryicoh@users.noreply.github.com> Date: Tue, 19 Jan 2021 12:50:03 +0900 Subject: [PATCH 037/190] Update content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md Co-authored-by: nasa9084 --- .../manage-resources/memory-constraint-namespace.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md index dd99671f34..0125f68851 100644 --- a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md +++ b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -18,7 +18,7 @@ PodがLimitRangeによって課される制約を満たさない場合、そのN {{< include "task-tutorial-prereqs.md" >}} {{< version-check >}} -クラスタ内の各ノードには、少なくとも1GiBのメモリが必要です。 +クラスター内の各ノードには、少なくとも1GiBのメモリーが必要です。 From a1be937428f08a003536ccbb7bcb3877ec7d90a7 Mon Sep 17 00:00:00 2001 From: Ryuichiroh Ikeuchi <37844673+ryicoh@users.noreply.github.com> Date: Tue, 19 Jan 2021 12:50:24 +0900 Subject: [PATCH 038/190] Update content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md Co-authored-by: nasa9084 --- .../manage-resources/memory-constraint-namespace.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md index 0125f68851..2f17a46acf 100644 --- a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md +++ b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -217,7 +217,7 @@ LimitRangeによってNamespaceに課される最大および最小のメモリ 例: -* クラスタ内の各ノードは2GBのメモリを持っています。クラスタ内のどのノードもその要求をサポートできないため、2GB以上のメモリを要求するPodは受け入れたくありません。 +* クラスター内の各ノードは2GBのメモリーを持っています。クラスタ内のどのノードもその要求をサポートできないため、2GB以上のメモリーを要求するPodは受け入れたくありません。 * クラスターは運用部門と開発部門で共有されています。 本番用のワークロードでは最大8GBのメモリーを消費しますが、開発用のワークロードでは512MBに制限したいとします。本番用と開発用に別々のNamespaceを作成し、それぞれのNamespaceにメモリー制限を適用します。 From 0a6fa5d137d37f2fbfaa19a3cfda72d1854643ea Mon Sep 17 00:00:00 2001 From: Ryuichiroh Ikeuchi <37844673+ryicoh@users.noreply.github.com> Date: Tue, 19 Jan 2021 12:50:37 +0900 Subject: [PATCH 039/190] Update content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md Co-authored-by: nasa9084 --- .../manage-resources/memory-constraint-namespace.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md index 2f17a46acf..9d1aef7d94 100644 --- a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md +++ b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -51,7 +51,7 @@ LimitRangeの詳細情報を表示します。 kubectl get limitrange mem-min-max-demo-lr --namespace=constraints-mem-example --output=yaml ``` -出力されるのは、予想通りメモリ制約の最小値と最大値を示しています。 +出力されるのは、予想通りメモリー制約の最小値と最大値を示しています。 しかし、LimitRangeの設定ファイルでデフォルト値を指定していないにもかかわらず、 自動的に作成されていることに気づきます。 From 0440f3c0f96fc63ae1978f0323c76d4ef3b45385 Mon Sep 17 00:00:00 2001 From: Ryuichiroh Ikeuchi <37844673+ryicoh@users.noreply.github.com> Date: Tue, 19 Jan 2021 12:50:51 +0900 Subject: [PATCH 040/190] Update content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md Co-authored-by: nasa9084 --- .../manage-resources/memory-constraint-namespace.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md index 9d1aef7d94..34b4c4cbec 100644 --- a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md +++ b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -75,7 +75,7 @@ Kubernetesは以下の手順を実行するようになっています。 * コンテナが独自のメモリ要求と制限を指定しない場合は、デフォルトのメモリ要求と制限をコンテナに割り当てます。 -* コンテナに500MiB以上のメモリ要求があることを確認します。 +* コンテナに500MiB以上のメモリー要求があることを確認します。 * コンテナのメモリ制限が1GiB以下であることを確認します。 From fe6fe85dbc25adb5aff02d207f3518ff31ee2b2e Mon Sep 17 00:00:00 2001 From: Ryuichiroh Ikeuchi <37844673+ryicoh@users.noreply.github.com> Date: Tue, 19 Jan 2021 12:51:00 +0900 Subject: [PATCH 041/190] Update content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md Co-authored-by: nasa9084 --- .../manage-resources/memory-constraint-namespace.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md index 34b4c4cbec..14463ed312 100644 --- a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md +++ b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -73,7 +73,7 @@ kubectl get limitrange mem-min-max-demo-lr --namespace=constraints-mem-example - constraints-mem-exampleNamespaceにコンテナが作成されるたびに、 Kubernetesは以下の手順を実行するようになっています。 -* コンテナが独自のメモリ要求と制限を指定しない場合は、デフォルトのメモリ要求と制限をコンテナに割り当てます。 +* コンテナが独自のメモリー要求と制限を指定しない場合は、デフォルトのメモリー要求と制限をコンテナに割り当てます。 * コンテナに500MiB以上のメモリー要求があることを確認します。 From 1b5a7e05619caa663a3616f9403b8cfc69b1bef4 Mon Sep 17 00:00:00 2001 From: Ryuichiroh Ikeuchi <37844673+ryicoh@users.noreply.github.com> Date: Tue, 19 Jan 2021 12:51:12 +0900 Subject: [PATCH 042/190] Update content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md Co-authored-by: nasa9084 --- .../manage-resources/memory-constraint-namespace.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md index 14463ed312..cdc3b4aaf6 100644 --- a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md +++ b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -77,7 +77,7 @@ Kubernetesは以下の手順を実行するようになっています。 * コンテナに500MiB以上のメモリー要求があることを確認します。 -* コンテナのメモリ制限が1GiB以下であることを確認します。 +* コンテナのメモリー制限が1GiB以下であることを確認します。 以下は、1つのコンテナを持つPodの設定ファイルです。設定ファイルのコンテナ(containers)では、600MiBのメモリ要求と800MiBのメモリ制限が指定されています。これらはLimitRangeによって課される最小と最大のメモリ制約を満たしています。 From d4452c25edf8beb37714aefcfcafe9faa30f3f56 Mon Sep 17 00:00:00 2001 From: Ryuichiroh Ikeuchi <37844673+ryicoh@users.noreply.github.com> Date: Tue, 19 Jan 2021 12:51:59 +0900 Subject: [PATCH 043/190] Update content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md Co-authored-by: nasa9084 --- .../manage-resources/memory-constraint-namespace.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md index cdc3b4aaf6..2a3b258bde 100644 --- a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md +++ b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -79,7 +79,7 @@ Kubernetesは以下の手順を実行するようになっています。 * コンテナのメモリー制限が1GiB以下であることを確認します。 -以下は、1つのコンテナを持つPodの設定ファイルです。設定ファイルのコンテナ(containers)では、600MiBのメモリ要求と800MiBのメモリ制限が指定されています。これらはLimitRangeによって課される最小と最大のメモリ制約を満たしています。 +以下は、1つのコンテナを持つPodの設定ファイルです。設定ファイルのコンテナ(containers)では、600MiBのメモリー要求と800MiBのメモリー制限が指定されています。これらはLimitRangeによって課される最小と最大のメモリー制約を満たしています。 {{< codenew file="admin/resource/memory-constraints-pod.yaml" >}} From 20142e2b318f4e2dab290a36e9f1280966679fbf Mon Sep 17 00:00:00 2001 From: Ryuichiroh Ikeuchi <37844673+ryicoh@users.noreply.github.com> Date: Tue, 19 Jan 2021 12:52:14 +0900 Subject: [PATCH 044/190] Update content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md Co-authored-by: nasa9084 --- .../manage-resources/memory-constraint-namespace.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md index 2a3b258bde..7ccec70650 100644 --- a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md +++ b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -102,7 +102,7 @@ Podの詳細情報を見ます kubectl get pod constraints-mem-demo --output=yaml --namespace=constraints-mem-example ``` -出力は、コンテナが600MiBのメモリ要求と800MiBのメモリ制限になっていることを示しています。これらはLimitRangeによって課される制約を満たしています。 +出力は、コンテナが600MiBのメモリ要求と800MiBのメモリー制限になっていることを示しています。これらはLimitRangeによって課される制約を満たしています。 ```yaml From 57a7a9ab81227ed11240d9c3402ba7b8e7cf79f9 Mon Sep 17 00:00:00 2001 From: Ryuichiroh Ikeuchi <37844673+ryicoh@users.noreply.github.com> Date: Tue, 19 Jan 2021 12:52:25 +0900 Subject: [PATCH 045/190] Update content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md Co-authored-by: nasa9084 --- .../manage-resources/memory-constraint-namespace.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md index 7ccec70650..72f884d37a 100644 --- a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md +++ b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -121,7 +121,7 @@ kubectl delete pod constraints-mem-demo --namespace=constraints-mem-example ## 最大メモリ制約を超えるPodの作成の試み -これは、1つのコンテナを持つPodの設定ファイルです。コンテナは800MiBのメモリ要求と1.5GiBのメモリ制限を指定しています。 +これは、1つのコンテナを持つPodの設定ファイルです。コンテナは800MiBのメモリー要求と1.5GiBのメモリー制限を指定しています。 {{< codenew file="admin/resource/memory-constraints-pod-2.yaml" >}} From 214740d14a0508800e2f87e2693c9fc43ba76d1c Mon Sep 17 00:00:00 2001 From: Ryuichiroh Ikeuchi <37844673+ryicoh@users.noreply.github.com> Date: Tue, 19 Jan 2021 12:52:36 +0900 Subject: [PATCH 046/190] Update content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md Co-authored-by: nasa9084 --- .../manage-resources/memory-constraint-namespace.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md index 72f884d37a..d4aa3f8f4c 100644 --- a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md +++ b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -132,7 +132,7 @@ Podを作成してみます。 kubectl apply -f https://k8s.io/examples/admin/resource/memory-constraints-pod-2.yaml --namespace=constraints-mem-example ``` -出力は、コンテナが大きすぎるメモリ制限を指定しているため、Podが作成されないことを示しています。 +出力は、コンテナが大きすぎるメモリー制限を指定しているため、Podが作成されないことを示しています。 ``` From 91af908261cd0bb5f6b6bbd3657dc97bc2278cad Mon Sep 17 00:00:00 2001 From: Ryuichiroh Ikeuchi <37844673+ryicoh@users.noreply.github.com> Date: Tue, 19 Jan 2021 12:52:46 +0900 Subject: [PATCH 047/190] Update content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md Co-authored-by: nasa9084 --- .../manage-resources/memory-constraint-namespace.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md index d4aa3f8f4c..d04d018b79 100644 --- a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md +++ b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -143,7 +143,7 @@ pods "constraints-mem-demo-2" is forbidden: maximum memory usage per Container i ## 最低限のメモリ要求を満たさないPodの作成の試み -これは、1つのコンテナを持つPodの設定ファイルです。コンテナは100MiBのメモリ要求と800MiBのメモリ制限を指定しています。 +これは、1つのコンテナを持つPodの設定ファイルです。コンテナは100MiBのメモリー要求と800MiBのメモリー制限を指定しています。 {{< codenew file="admin/resource/memory-constraints-pod-3.yaml" >}} From ea822c29aa3dad4b11c4e160a7331b2de40b72ab Mon Sep 17 00:00:00 2001 From: Ryuichiroh Ikeuchi <37844673+ryicoh@users.noreply.github.com> Date: Tue, 19 Jan 2021 12:52:59 +0900 Subject: [PATCH 048/190] Update content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md Co-authored-by: nasa9084 --- .../manage-resources/memory-constraint-namespace.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md index d04d018b79..5021136796 100644 --- a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md +++ b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -154,7 +154,7 @@ Podを作成してみます。 kubectl apply -f https://k8s.io/examples/admin/resource/memory-constraints-pod-3.yaml --namespace=constraints-mem-example ``` -出力は、コンテナが小さすぎるメモリリクエストを指定しているため、Podが作成されないことを示しています。 +出力は、コンテナが小さすぎるメモリー要求を指定しているため、Podが作成されないことを示しています。 ``` Error from server (Forbidden): error when creating "examples/admin/resource/memory-constraints-pod-3.yaml": From eb788d096455445b463ad0cb5e62676ddb1f8f18 Mon Sep 17 00:00:00 2001 From: Ryuichiroh Ikeuchi <37844673+ryicoh@users.noreply.github.com> Date: Tue, 19 Jan 2021 12:53:10 +0900 Subject: [PATCH 049/190] Update content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md Co-authored-by: nasa9084 --- .../manage-resources/memory-constraint-namespace.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md index 5021136796..1643d23204 100644 --- a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md +++ b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -164,7 +164,7 @@ pods "constraints-mem-demo-3" is forbidden: minimum memory usage per Container i ## メモリ要求や制限を指定しないPodの作成 -これは、1つのコンテナを持つPodの設定ファイルです。コンテナはメモリ要求を指定しておらず、メモリ制限も指定していません。 +これは、1つのコンテナを持つPodの設定ファイルです。コンテナはメモリー要求を指定しておらず、メモリー制限も指定していません。 {{< codenew file="admin/resource/memory-constraints-pod-4.yaml" >}} From ff8a41b302a729e4d5dde7d7bc0c6b03e997978f Mon Sep 17 00:00:00 2001 From: Ryuichiroh Ikeuchi <37844673+ryicoh@users.noreply.github.com> Date: Tue, 19 Jan 2021 12:53:21 +0900 Subject: [PATCH 050/190] Update content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md Co-authored-by: nasa9084 --- .../manage-resources/memory-constraint-namespace.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md index 1643d23204..efc086c3ee 100644 --- a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md +++ b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -180,7 +180,7 @@ Podの詳細情報を見ます kubectl get pod constraints-mem-demo-4 --namespace=constraints-mem-example --output=yaml ``` -出力を見ると、Podのコンテナのメモリ要求は1GiB、メモリ制限は1GiBであることがわかります。 +出力を見ると、Podのコンテナのメモリ要求は1GiB、メモリー制限は1GiBであることがわかります。 コンテナはどのようにしてこれらの値を取得したのでしょうか？ From 5b9eaf5623971451ba4a458f3b05398cdbd6ae95 Mon Sep 17 00:00:00 2001 From: Ryuichiroh Ikeuchi <37844673+ryicoh@users.noreply.github.com> Date: Tue, 19 Jan 2021 12:53:33 +0900 Subject: [PATCH 051/190] Update content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md Co-authored-by: nasa9084 --- .../manage-resources/memory-constraint-namespace.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md index efc086c3ee..60b76b36e2 100644 --- a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md +++ b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -192,7 +192,7 @@ resources: memory: 1Gi ``` -コンテナが独自のメモリ要求と制限を指定していなかったため、LimitRangeから与えられたのです。 +コンテナが独自のメモリー要求と制限を指定していなかったため、LimitRangeから与えられのです。 コンテナが独自のメモリ要求と制限を指定していなかったため、LimitRangeから[デフォルトのメモリ要求と制限](/docs/tasks/administer-cluster/manage-resources/memory-default-namespace/)が与えられたのです。 この時点で、コンテナは起動しているかもしれませんし、起動していないかもしれません。このタスクの前提条件は、ノードが少なくとも1GiBのメモリを持っていることであることを思い出してください。それぞれのノードが1GiBのメモリしか持っていない場合、どのノードにも1GiBのメモリ要求に対応するのに十分な割り当て可能なメモリがありません。たまたま2GiBのメモリを持つノードを使用しているのであれば、おそらく1GiBのメモリリクエストに対応するのに十分なスペースを持っていることになります。 From 4e9e61ddd7d9e9725caf2a6125417f2945a78db0 Mon Sep 17 00:00:00 2001 From: Ryuichiroh Ikeuchi <37844673+ryicoh@users.noreply.github.com> Date: Tue, 19 Jan 2021 12:53:46 +0900 Subject: [PATCH 052/190] Update content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md Co-authored-by: nasa9084 --- .../manage-resources/memory-constraint-namespace.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md index 60b76b36e2..0554883e89 100644 --- a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md +++ b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -193,7 +193,7 @@ resources: ``` コンテナが独自のメモリー要求と制限を指定していなかったため、LimitRangeから与えられのです。 -コンテナが独自のメモリ要求と制限を指定していなかったため、LimitRangeから[デフォルトのメモリ要求と制限](/docs/tasks/administer-cluster/manage-resources/memory-default-namespace/)が与えられたのです。 +コンテナが独自のメモリー要求と制限を指定していなかったため、LimitRangeから[デフォルトのメモリー要求と制限](/docs/tasks/administer-cluster/manage-resources/memory-default-namespace/)が与えられたのです。 この時点で、コンテナは起動しているかもしれませんし、起動していないかもしれません。このタスクの前提条件は、ノードが少なくとも1GiBのメモリを持っていることであることを思い出してください。それぞれのノードが1GiBのメモリしか持っていない場合、どのノードにも1GiBのメモリ要求に対応するのに十分な割り当て可能なメモリがありません。たまたま2GiBのメモリを持つノードを使用しているのであれば、おそらく1GiBのメモリリクエストに対応するのに十分なスペースを持っていることになります。 From e5f09ba8cdff6516766593dbea8ad3bdd2e7eab8 Mon Sep 17 00:00:00 2001 From: Ryuichiroh Ikeuchi <37844673+ryicoh@users.noreply.github.com> Date: Tue, 19 Jan 2021 12:53:59 +0900 Subject: [PATCH 053/190] Update content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md Co-authored-by: nasa9084 --- .../manage-resources/memory-constraint-namespace.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md index 0554883e89..7889345991 100644 --- a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md +++ b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -195,7 +195,7 @@ resources: コンテナが独自のメモリー要求と制限を指定していなかったため、LimitRangeから与えられのです。 コンテナが独自のメモリー要求と制限を指定していなかったため、LimitRangeから[デフォルトのメモリー要求と制限](/docs/tasks/administer-cluster/manage-resources/memory-default-namespace/)が与えられたのです。 -この時点で、コンテナは起動しているかもしれませんし、起動していないかもしれません。このタスクの前提条件は、ノードが少なくとも1GiBのメモリを持っていることであることを思い出してください。それぞれのノードが1GiBのメモリしか持っていない場合、どのノードにも1GiBのメモリ要求に対応するのに十分な割り当て可能なメモリがありません。たまたま2GiBのメモリを持つノードを使用しているのであれば、おそらく1GiBのメモリリクエストに対応するのに十分なスペースを持っていることになります。 +この時点で、コンテナは起動しているかもしれませんし、起動していないかもしれません。このタスクの前提条件は、ノードが少なくとも1GiBのメモリーを持っていることであることを思い出してください。それぞれのノードが1GiBのメモリーしか持っていない場合、どのノードにも1GiBのメモリー要求に対応するのに十分な割り当て可能なメモリーがありません。たまたま2GiBのメモリーを持つノードを使用しているのであれば、おそらく1GiBのメモリーリクエストに対応するのに十分なスペースを持っていることになります。 Podを削除します。 From 712694865d315d85f307dea4dd945df28a282d35 Mon Sep 17 00:00:00 2001 From: Ryuichiroh Ikeuchi <37844673+ryicoh@users.noreply.github.com> Date: Tue, 19 Jan 2021 12:54:12 +0900 Subject: [PATCH 054/190] Update content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md Co-authored-by: nasa9084 --- .../manage-resources/memory-constraint-namespace.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md index 7889345991..1929b8d349 100644 --- a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md +++ b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -204,7 +204,7 @@ Podを削除します。 kubectl delete pod constraints-mem-demo-4 --namespace=constraints-mem-example ``` -## 最小および最大メモリ制約の強制 +## 最小および最大メモリー制約の強制 LimitRangeによってNamespaceに課される最大および最小のメモリ制約は、Podが作成または更新されたときにのみ適用されます。LimitRangeを変更しても、以前に作成されたPodには影響しません。 From d2d1248dc966de69f7c846e46822e4cd45090a45 Mon Sep 17 00:00:00 2001 From: Ryuichiroh Ikeuchi <37844673+ryicoh@users.noreply.github.com> Date: Tue, 19 Jan 2021 12:54:23 +0900 Subject: [PATCH 055/190] Update content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md Co-authored-by: nasa9084 --- .../manage-resources/memory-constraint-namespace.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md index 1929b8d349..8dea786826 100644 --- a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md +++ b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -206,7 +206,7 @@ kubectl delete pod constraints-mem-demo-4 --namespace=constraints-mem-example ## 最小および最大メモリー制約の強制 -LimitRangeによってNamespaceに課される最大および最小のメモリ制約は、Podが作成または更新されたときにのみ適用されます。LimitRangeを変更しても、以前に作成されたPodには影響しません。 +LimitRangeによってNamespaceに課される最大および最小のメモリー制約は、Podが作成または更新されたときにのみ適用されます。LimitRangeを変更しても、以前に作成されたPodには影響しません。 ## 最小・最大メモリ制約の動機 From 8bc44fa6e19ee2112ac830e623c5c7e2284ffea3 Mon Sep 17 00:00:00 2001 From: Ryuichiroh Ikeuchi <37844673+ryicoh@users.noreply.github.com> Date: Tue, 19 Jan 2021 12:54:33 +0900 Subject: [PATCH 056/190] Update content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md Co-authored-by: nasa9084 --- .../manage-resources/memory-constraint-namespace.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md index 8dea786826..c303e6137b 100644 --- a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md +++ b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -209,7 +209,7 @@ kubectl delete pod constraints-mem-demo-4 --namespace=constraints-mem-example LimitRangeによってNamespaceに課される最大および最小のメモリー制約は、Podが作成または更新されたときにのみ適用されます。LimitRangeを変更しても、以前に作成されたPodには影響しません。 -## 最小・最大メモリ制約の動機 +## 最小・最大メモリー制約の動機 クラスタ管理者としては、Podが使用できるメモリ量に制限を課したいと思うかもしれません。 From 32e9186fd8e6b3634aa88521a35f03726ea6d506 Mon Sep 17 00:00:00 2001 From: Ryuichiroh Ikeuchi <37844673+ryicoh@users.noreply.github.com> Date: Tue, 19 Jan 2021 12:54:44 +0900 Subject: [PATCH 057/190] Update content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md Co-authored-by: nasa9084 --- .../manage-resources/memory-constraint-namespace.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md index c303e6137b..1bc1b7951d 100644 --- a/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md +++ b/content/ja/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace.md @@ -212,7 +212,7 @@ LimitRangeによってNamespaceに課される最大および最小のメモリ ## 最小・最大メモリー制約の動機 -クラスタ管理者としては、Podが使用できるメモリ量に制限を課したいと思うかもしれません。 +クラスター管理者としては、Podが使用できるメモリー量に制限を課したいと思うかもしれません。 例: From 99029b97d7c93373cf20e9609782428f107654c2 Mon Sep 17 00:00:00 2001 From: Paul Czarkowski Date: Mon, 3 Aug 2020 14:50:11 -0500 Subject: [PATCH 058/190] Remove Master/Slave terminology from stateless application tutorial. The stateless application tutorial is littered with unsuitable terminology. This update switches the database technology to a database that has already updated their own terminology as well as removes the terminology from the tutorial itself. The advanced logging tutorial followup is set to Draft as it will take a larger effort to convert that, and I'm not convinced its even in a working state right now. A followup PR to the examples repo to be referenced here will be made. Addresses #22918 Signed-off-by: Paul Czarkowski address comments in PR Signed-off-by: Paul Czarkowski remove confusing comment about dns, we can assume k8s has kube-dns Signed-off-by: Paul Czarkowski --- content/en/docs/tutorials/_index.md | 2 +- .../guestbook-logs-metrics-with-elk.md | 460 ------------------ .../stateless-application/guestbook.md | 188 +++---- .../guestbook/frontend-deployment.yaml | 24 +- .../guestbook/frontend-service.yaml | 10 +- .../guestbook/mongo-deployment.yaml | 31 ++ .../application/guestbook/mongo-service.yaml | 14 + .../guestbook/redis-master-deployment.yaml | 29 -- .../guestbook/redis-master-service.yaml | 17 - .../guestbook/redis-slave-deployment.yaml | 40 -- .../guestbook/redis-slave-service.yaml | 15 - 11 files changed, 124 insertions(+), 706 deletions(-) delete mode 100644 content/en/docs/tutorials/stateless-application/guestbook-logs-metrics-with-elk.md create mode 100644 content/en/examples/application/guestbook/mongo-deployment.yaml create mode 100644 content/en/examples/application/guestbook/mongo-service.yaml delete mode 100644 content/en/examples/application/guestbook/redis-master-deployment.yaml delete mode 100644 content/en/examples/application/guestbook/redis-master-service.yaml delete mode 100644 content/en/examples/application/guestbook/redis-slave-deployment.yaml delete mode 100644 content/en/examples/application/guestbook/redis-slave-service.yaml diff --git a/content/en/docs/tutorials/_index.md b/content/en/docs/tutorials/_index.md index b4f0709a76..630c04f5f6 100644 --- a/content/en/docs/tutorials/_index.md +++ b/content/en/docs/tutorials/_index.md @@ -33,7 +33,7 @@ Before walking through each tutorial, you may want to bookmark the * [Exposing an External IP Address to Access an Application in a Cluster](/docs/tutorials/stateless-application/expose-external-ip-address/) -* [Example: Deploying PHP Guestbook application with Redis](/docs/tutorials/stateless-application/guestbook/) +* [Example: Deploying PHP Guestbook application with MongoDB](/docs/tutorials/stateless-application/guestbook/) ## Stateful Applications diff --git a/content/en/docs/tutorials/stateless-application/guestbook-logs-metrics-with-elk.md b/content/en/docs/tutorials/stateless-application/guestbook-logs-metrics-with-elk.md deleted file mode 100644 index d3a38c4df5..0000000000 --- a/content/en/docs/tutorials/stateless-application/guestbook-logs-metrics-with-elk.md +++ /dev/null @@ -1,460 +0,0 @@ ---- -title: "Example: Add logging and metrics to the PHP / Redis Guestbook example" -reviewers: -- sftim -content_type: tutorial -weight: 21 -card: - name: tutorials - weight: 31 - title: "Example: Add logging and metrics to the PHP / Redis Guestbook example" ---- - - -This tutorial builds upon the [PHP Guestbook with Redis](/docs/tutorials/stateless-application/guestbook) tutorial. Lightweight log, metric, and network data open source shippers, or *Beats*, from Elastic are deployed in the same Kubernetes cluster as the guestbook. The Beats collect, parse, and index the data into Elasticsearch so that you can view and analyze the resulting operational information in Kibana. This example consists of the following components: - -* A running instance of the [PHP Guestbook with Redis tutorial](/docs/tutorials/stateless-application/guestbook) -* Elasticsearch and Kibana -* Filebeat -* Metricbeat -* Packetbeat - -## {{% heading "objectives" %}} - -* Start up the PHP Guestbook with Redis. -* Install kube-state-metrics. -* Create a Kubernetes Secret. -* Deploy the Beats. -* View dashboards of your logs and metrics. - -## {{% heading "prerequisites" %}} - - -{{< include "task-tutorial-prereqs.md" >}} -{{< version-check >}} - -Additionally you need: - -* A running deployment of the [PHP Guestbook with Redis](/docs/tutorials/stateless-application/guestbook) tutorial. - -* A running Elasticsearch and Kibana deployment. You can use [Elasticsearch Service in Elastic Cloud](https://cloud.elastic.co), - run the [downloaded files](https://www.elastic.co/guide/en/elastic-stack-get-started/current/get-started-elastic-stack.html) - on your workstation or servers, or the [Elastic Helm Charts](https://github.com/elastic/helm-charts). - - - -## Start up the PHP Guestbook with Redis - -This tutorial builds on the [PHP Guestbook with Redis](/docs/tutorials/stateless-application/guestbook) tutorial. If you have the guestbook application running, then you can monitor that. If you do not have it running then follow the instructions to deploy the guestbook and do not perform the **Cleanup** steps. Come back to this page when you have the guestbook running. - -## Add a Cluster role binding - -Create a [cluster level role binding](/docs/reference/access-authn-authz/rbac/#rolebinding-and-clusterrolebinding) so that you can deploy kube-state-metrics and the Beats at the cluster level (in kube-system). - -```shell -kubectl create clusterrolebinding cluster-admin-binding \ - --clusterrole=cluster-admin --user= -``` - -## Install kube-state-metrics - -Kubernetes [*kube-state-metrics*](https://github.com/kubernetes/kube-state-metrics) is a simple service that listens to the Kubernetes API server and generates metrics about the state of the objects. Metricbeat reports these metrics. Add kube-state-metrics to the Kubernetes cluster that the guestbook is running in. - -```shell -git clone https://github.com/kubernetes/kube-state-metrics.git kube-state-metrics -kubectl apply -f kube-state-metrics/examples/standard -``` - -### Check to see if kube-state-metrics is running - -```shell -kubectl get pods --namespace=kube-system -l app.kubernetes.io/name=kube-state-metrics -``` - -Output: - -``` -NAME READY STATUS RESTARTS AGE -kube-state-metrics-89d656bf8-vdthm 1/1 Running 0 21s -``` - -## Clone the Elastic examples GitHub repo - -```shell -git clone https://github.com/elastic/examples.git -``` - -The rest of the commands will reference files in the `examples/beats-k8s-send-anywhere` directory, so change dir there: - -```shell -cd examples/beats-k8s-send-anywhere -``` - -## Create a Kubernetes Secret - -A Kubernetes {{< glossary_tooltip text="Secret" term_id="secret" >}} is an object that contains a small amount of sensitive data such as a password, a token, or a key. Such information might otherwise be put in a Pod specification or in an image; putting it in a Secret object allows for more control over how it is used, and reduces the risk of accidental exposure. - -{{< note >}} -There are two sets of steps here, one for *self managed* Elasticsearch and Kibana (running on your servers or using the Elastic Helm Charts), and a second separate set for the *managed service* Elasticsearch Service in Elastic Cloud. Only create the secret for the type of Elasticsearch and Kibana system that you will use for this tutorial. -{{< /note >}} - -{{< tabs name="tab_with_md" >}} -{{% tab name="Self Managed" %}} - -### Self managed - -Switch to the **Managed service** tab if you are connecting to Elasticsearch Service in Elastic Cloud. - -### Set the credentials - -There are four files to edit to create a k8s secret when you are connecting to self managed Elasticsearch and Kibana (self managed is effectively anything other than the managed Elasticsearch Service in Elastic Cloud). The files are: - -1. `ELASTICSEARCH_HOSTS` -1. `ELASTICSEARCH_PASSWORD` -1. `ELASTICSEARCH_USERNAME` -1. `KIBANA_HOST` - -Set these with the information for your Elasticsearch cluster and your Kibana host. Here are some examples (also see [*this configuration*](https://stackoverflow.com/questions/59892896/how-to-connect-from-minikube-to-elasticsearch-installed-on-host-local-developme/59892897#59892897)) - -#### `ELASTICSEARCH_HOSTS` - -1. A nodeGroup from the Elastic Elasticsearch Helm Chart: - - ``` - ["http://elasticsearch-master.default.svc.cluster.local:9200"] - ``` - -1. A single Elasticsearch node running on a Mac where your Beats are running in Docker for Mac: - - ``` - ["http://host.docker.internal:9200"] - ``` - -1. Two Elasticsearch nodes running in VMs or on physical hardware: - - ``` - ["http://host1.example.com:9200", "http://host2.example.com:9200"] - ``` - -Edit `ELASTICSEARCH_HOSTS`: - -```shell -vi ELASTICSEARCH_HOSTS -``` - -#### `ELASTICSEARCH_PASSWORD` - -Just the password; no whitespace, quotes, `<` or `>`: - -``` - -``` - -Edit `ELASTICSEARCH_PASSWORD`: - -```shell -vi ELASTICSEARCH_PASSWORD -``` - -#### `ELASTICSEARCH_USERNAME` - -Just the username; no whitespace, quotes, `<` or `>`: - -``` - -``` - -Edit `ELASTICSEARCH_USERNAME`: - -```shell -vi ELASTICSEARCH_USERNAME -``` - -#### `KIBANA_HOST` - -1. The Kibana instance from the Elastic Kibana Helm Chart. The subdomain `default` refers to the default namespace. If you have deployed the Helm Chart using a different namespace, then your subdomain will be different: - - ``` - "kibana-kibana.default.svc.cluster.local:5601" - ``` - -1. A Kibana instance running on a Mac where your Beats are running in Docker for Mac: - - ``` - "host.docker.internal:5601" - ``` -1. Two Elasticsearch nodes running in VMs or on physical hardware: - - ``` - "host1.example.com:5601" - ``` - -Edit `KIBANA_HOST`: - -```shell -vi KIBANA_HOST -``` - -### Create a Kubernetes Secret - -This command creates a Secret in the Kubernetes system level namespace (`kube-system`) based on the files you just edited: - -```shell -kubectl create secret generic dynamic-logging \ - --from-file=./ELASTICSEARCH_HOSTS \ - --from-file=./ELASTICSEARCH_PASSWORD \ - --from-file=./ELASTICSEARCH_USERNAME \ - --from-file=./KIBANA_HOST \ - --namespace=kube-system -``` - -{{% /tab %}} -{{% tab name="Managed service" %}} - -## Managed service - -This tab is for Elasticsearch Service in Elastic Cloud only, if you have already created a secret for a self managed Elasticsearch and Kibana deployment, then continue with [Deploy the Beats](#deploy-the-beats). - -### Set the credentials - -There are two files to edit to create a Kubernetes Secret when you are connecting to the managed Elasticsearch Service in Elastic Cloud. The files are: - -1. `ELASTIC_CLOUD_AUTH` -1. `ELASTIC_CLOUD_ID` - -Set these with the information provided to you from the Elasticsearch Service console when you created the deployment. Here are some examples: - -#### `ELASTIC_CLOUD_ID` - -``` -devk8s:ABC123def456ghi789jkl123mno456pqr789stu123vwx456yza789bcd012efg345hijj678klm901nop345zEwOTJjMTc5YWQ0YzQ5OThlN2U5MjAwYTg4NTIzZQ== -``` - -#### `ELASTIC_CLOUD_AUTH` - -Just the username, a colon (`:`), and the password, no whitespace or quotes: - -``` -elastic:VFxJJf9Tjwer90wnfTghsn8w -``` - -### Edit the required files: - -```shell -vi ELASTIC_CLOUD_ID -vi ELASTIC_CLOUD_AUTH -``` - -### Create a Kubernetes Secret - -This command creates a Secret in the Kubernetes system level namespace (`kube-system`) based on the files you just edited: - -```shell -kubectl create secret generic dynamic-logging \ - --from-file=./ELASTIC_CLOUD_ID \ - --from-file=./ELASTIC_CLOUD_AUTH \ - --namespace=kube-system -``` - -{{% /tab %}} - -{{< /tabs >}} - -## Deploy the Beats - -Manifest files are provided for each Beat. These manifest files use the secret created earlier to configure the Beats to connect to your Elasticsearch and Kibana servers. - -### About Filebeat - -Filebeat will collect logs from the Kubernetes nodes and the containers running in each pod running on those nodes. Filebeat is deployed as a {{< glossary_tooltip text="DaemonSet" term_id="daemonset" >}}. Filebeat can autodiscover applications running in your Kubernetes cluster. At startup Filebeat scans existing containers and launches the proper configurations for them, then it will watch for new start/stop events. - -Here is the autodiscover configuration that enables Filebeat to locate and parse Redis logs from the Redis containers deployed with the guestbook application. This configuration is in the file `filebeat-kubernetes.yaml`: - -```yaml -- condition.contains: - kubernetes.labels.app: redis - config: - - module: redis - log: - input: - type: docker - containers.ids: - - ${data.kubernetes.container.id} - slowlog: - enabled: true - var.hosts: ["${data.host}:${data.port}"] -``` - -This configures Filebeat to apply the Filebeat module `redis` when a container is detected with a label `app` containing the string `redis`. The redis module has the ability to collect the `log` stream from the container by using the docker input type (reading the file on the Kubernetes node associated with the STDOUT stream from this Redis container). Additionally, the module has the ability to collect Redis `slowlog` entries by connecting to the proper pod host and port, which is provided in the container metadata. - -### Deploy Filebeat: - -```shell -kubectl create -f filebeat-kubernetes.yaml -``` - -#### Verify - -```shell -kubectl get pods -n kube-system -l k8s-app=filebeat-dynamic -``` - -### About Metricbeat - -Metricbeat autodiscover is configured in the same way as Filebeat. Here is the Metricbeat autodiscover configuration for the Redis containers. This configuration is in the file `metricbeat-kubernetes.yaml`: - -```yaml -- condition.equals: - kubernetes.labels.tier: backend - config: - - module: redis - metricsets: ["info", "keyspace"] - period: 10s - - # Redis hosts - hosts: ["${data.host}:${data.port}"] -``` - -This configures Metricbeat to apply the Metricbeat module `redis` when a container is detected with a label `tier` equal to the string `backend`. The `redis` module has the ability to collect the `info` and `keyspace` metrics from the container by connecting to the proper pod host and port, which is provided in the container metadata. - -### Deploy Metricbeat - -```shell -kubectl create -f metricbeat-kubernetes.yaml -``` - -#### Verify - -```shell -kubectl get pods -n kube-system -l k8s-app=metricbeat -``` - -### About Packetbeat - -Packetbeat configuration is different than Filebeat and Metricbeat. Rather than specify patterns to match against container labels the configuration is based on the protocols and port numbers involved. Shown below is a subset of the port numbers. - -{{< note >}} -If you are running a service on a non-standard port add that port number to the appropriate type in `filebeat.yaml` and delete/create the Packetbeat DaemonSet. -{{< /note >}} - -```yaml -packetbeat.interfaces.device: any - -packetbeat.protocols: -- type: dns - ports: [53] - include_authorities: true - include_additionals: true - -- type: http - ports: [80, 8000, 8080, 9200] - -- type: mysql - ports: [3306] - -- type: redis - ports: [6379] - -packetbeat.flows: - timeout: 30s - period: 10s -``` - -#### Deploy Packetbeat - -```shell -kubectl create -f packetbeat-kubernetes.yaml -``` - -#### Verify - -```shell -kubectl get pods -n kube-system -l k8s-app=packetbeat-dynamic -``` - -## View in Kibana - -Open Kibana in your browser and then open the **Dashboard** application. In the search bar type Kubernetes and click on the Metricbeat dashboard for Kubernetes. This dashboard reports on the state of your Nodes, deployments, etc. - -Search for Packetbeat on the Dashboard page, and view the Packetbeat overview. - -Similarly, view dashboards for Apache and Redis. You will see dashboards for logs and metrics for each. The Apache Metricbeat dashboard will be blank. Look at the Apache Filebeat dashboard and scroll to the bottom to view the Apache error logs. This will tell you why there are no metrics available for Apache. - -To enable Metricbeat to retrieve the Apache metrics, enable server-status by adding a ConfigMap including a mod-status configuration file and re-deploy the guestbook. - -## Scale your Deployments and see new pods being monitored - -List the existing Deployments: - -```shell -kubectl get deployments -``` - -The output: - -``` -NAME READY UP-TO-DATE AVAILABLE AGE -frontend 3/3 3 3 3h27m -redis-master 1/1 1 1 3h27m -redis-slave 2/2 2 2 3h27m -``` - -Scale the frontend down to two pods: - -```shell -kubectl scale --replicas=2 deployment/frontend -``` - -The output: - -``` -deployment.extensions/frontend scaled -``` - -Scale the frontend back up to three pods: - -```shell -kubectl scale --replicas=3 deployment/frontend -``` - -## View the changes in Kibana - -See the screenshot, add the indicated filters and then add the columns to the view. You can see the ScalingReplicaSet entry that is marked, following from there to the top of the list of events shows the image being pulled, the volumes mounted, the pod starting, etc. -![Kibana Discover](https://raw.githubusercontent.com/elastic/examples/master/beats-k8s-send-anywhere/scaling-up.png) - -## {{% heading "cleanup" %}} - -Deleting the Deployments and Services also deletes any running Pods. Use labels to delete multiple resources with one command. - -1. Run the following commands to delete all Pods, Deployments, and Services. - - ```shell - kubectl delete deployment -l app=redis - kubectl delete service -l app=redis - kubectl delete deployment -l app=guestbook - kubectl delete service -l app=guestbook - kubectl delete -f filebeat-kubernetes.yaml - kubectl delete -f metricbeat-kubernetes.yaml - kubectl delete -f packetbeat-kubernetes.yaml - kubectl delete secret dynamic-logging -n kube-system - ``` - -1. Query the list of Pods to verify that no Pods are running: - - ```shell - kubectl get pods - ``` - - The response should be this: - - ``` - No resources found. - ``` - -## {{% heading "whatsnext" %}} - -* Learn about [tools for monitoring resources](/docs/tasks/debug-application-cluster/resource-usage-monitoring/) -* Read more about [logging architecture](/docs/concepts/cluster-administration/logging/) -* Read more about [application introspection and debugging](/docs/tasks/debug-application-cluster/) -* Read more about [troubleshoot applications](/docs/tasks/debug-application-cluster/resource-usage-monitoring/) - diff --git a/content/en/docs/tutorials/stateless-application/guestbook.md b/content/en/docs/tutorials/stateless-application/guestbook.md index 2b6eef90fa..3b9737e831 100644 --- a/content/en/docs/tutorials/stateless-application/guestbook.md +++ b/content/en/docs/tutorials/stateless-application/guestbook.md @@ -1,5 +1,5 @@ --- -title: "Example: Deploying PHP Guestbook application with Redis" +title: "Example: Deploying PHP Guestbook application with MongoDB" reviewers: - ahmetb content_type: tutorial @@ -7,22 +7,19 @@ weight: 20 card: name: tutorials weight: 30 - title: "Stateless Example: PHP Guestbook with Redis" + title: "Stateless Example: PHP Guestbook with MongoDB" +min-kubernetes-server-version: v1.14 --- -This tutorial shows you how to build and deploy a simple, multi-tier web application using Kubernetes and [Docker](https://www.docker.com/). This example consists of the following components: +This tutorial shows you how to build and deploy a simple _(not production ready)_, multi-tier web application using Kubernetes and [Docker](https://www.docker.com/). This example consists of the following components: -* A single-instance [Redis](https://redis.io/) master to store guestbook entries -* Multiple [replicated Redis](https://redis.io/topics/replication) instances to serve reads +* A single-instance [MongoDB](https://www.mongodb.com/) to store guestbook entries * Multiple web frontend instances - - ## {{% heading "objectives" %}} -* Start up a Redis master. -* Start up Redis slaves. +* Start up a Mongo database. * Start up the guestbook frontend. * Expose and view the Frontend Service. * Clean up. @@ -39,24 +36,28 @@ This tutorial shows you how to build and deploy a simple, multi-tier web applica -## Start up the Redis Master +## Start up the Mongo Database -The guestbook application uses Redis to store its data. It writes its data to a Redis master instance and reads data from multiple Redis slave instances. +The guestbook application uses MongoDB to store its data. -### Creating the Redis Master Deployment +### Creating the Mongo Deployment -The manifest file, included below, specifies a Deployment controller that runs a single replica Redis master Pod. +The manifest file, included below, specifies a Deployment controller that runs a single replica MongoDB Pod. -{{< codenew file="application/guestbook/redis-master-deployment.yaml" >}} +{{< codenew file="application/guestbook/mongo-deployment.yaml" >}} 1. Launch a terminal window in the directory you downloaded the manifest files. -1. Apply the Redis Master Deployment from the `redis-master-deployment.yaml` file: +1. Apply the MongoDB Deployment from the `mongo-deployment.yaml` file: ```shell - kubectl apply -f https://k8s.io/examples/application/guestbook/redis-master-deployment.yaml + kubectl apply -f https://k8s.io/examples/application/guestbook/mongo-deployment.yaml ``` + -1. Query the list of Pods to verify that the Redis Master Pod is running: +1. Query the list of Pods to verify that the MongoDB Pod is running: ```shell kubectl get pods @@ -66,32 +67,33 @@ The manifest file, included below, specifies a Deployment controller that runs a ```shell NAME READY STATUS RESTARTS AGE - redis-master-1068406935-3lswp 1/1 Running 0 28s + mongo-5cfd459dd4-lrcjb 1/1 Running 0 28s ``` -1. Run the following command to view the logs from the Redis Master Pod: +1. Run the following command to view the logs from the MongoDB Deployment: ```shell - kubectl logs -f POD-NAME + kubectl logs -f deployment/mongo ``` -{{< note >}} -Replace POD-NAME with the name of your Pod. -{{< /note >}} +### Creating the MongoDB Service -### Creating the Redis Master Service +The guestbook application needs to communicate to the MongoDB to write its data. You need to apply a [Service](/docs/concepts/services-networking/service/) to proxy the traffic to the MongoDB Pod. A Service defines a policy to access the Pods. -The guestbook application needs to communicate to the Redis master to write its data. You need to apply a [Service](/docs/concepts/services-networking/service/) to proxy the traffic to the Redis master Pod. A Service defines a policy to access the Pods. +{{< codenew file="application/guestbook/mongo-service.yaml" >}} -{{< codenew file="application/guestbook/redis-master-service.yaml" >}} - -1. Apply the Redis Master Service from the following `redis-master-service.yaml` file: +1. Apply the MongoDB Service from the following `mongo-service.yaml` file: ```shell - kubectl apply -f https://k8s.io/examples/application/guestbook/redis-master-service.yaml + kubectl apply -f https://k8s.io/examples/application/guestbook/mongo-service.yaml ``` -1. Query the list of Services to verify that the Redis Master Service is running: + + +1. Query the list of Services to verify that the MongoDB Service is running: ```shell kubectl get service @@ -102,77 +104,17 @@ The guestbook application needs to communicate to the Redis master to write its ```shell NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.0.0.1 443/TCP 1m - redis-master ClusterIP 10.0.0.151 6379/TCP 8s + mongo ClusterIP 10.0.0.151 6379/TCP 8s ``` {{< note >}} -This manifest file creates a Service named `redis-master` with a set of labels that match the labels previously defined, so the Service routes network traffic to the Redis master Pod. +This manifest file creates a Service named `mongo` with a set of labels that match the labels previously defined, so the Service routes network traffic to the MongoDB Pod. {{< /note >}} -## Start up the Redis Slaves - -Although the Redis master is a single pod, you can make it highly available to meet traffic demands by adding replica Redis slaves. - -### Creating the Redis Slave Deployment - -Deployments scale based off of the configurations set in the manifest file. In this case, the Deployment object specifies two replicas. - -If there are not any replicas running, this Deployment would start the two replicas on your container cluster. Conversely, if there are more than two replicas running, it would scale down until two replicas are running. - -{{< codenew file="application/guestbook/redis-slave-deployment.yaml" >}} - -1. Apply the Redis Slave Deployment from the `redis-slave-deployment.yaml` file: - - ```shell - kubectl apply -f https://k8s.io/examples/application/guestbook/redis-slave-deployment.yaml - ``` - -1. Query the list of Pods to verify that the Redis Slave Pods are running: - - ```shell - kubectl get pods - ``` - - The response should be similar to this: - - ```shell - NAME READY STATUS RESTARTS AGE - redis-master-1068406935-3lswp 1/1 Running 0 1m - redis-slave-2005841000-fpvqc 0/1 ContainerCreating 0 6s - redis-slave-2005841000-phfv9 0/1 ContainerCreating 0 6s - ``` - -### Creating the Redis Slave Service - -The guestbook application needs to communicate to Redis slaves to read data. To make the Redis slaves discoverable, you need to set up a Service. A Service provides transparent load balancing to a set of Pods. - -{{< codenew file="application/guestbook/redis-slave-service.yaml" >}} - -1. Apply the Redis Slave Service from the following `redis-slave-service.yaml` file: - - ```shell - kubectl apply -f https://k8s.io/examples/application/guestbook/redis-slave-service.yaml - ``` - -1. Query the list of Services to verify that the Redis slave service is running: - - ```shell - kubectl get services - ``` - - The response should be similar to this: - - ``` - NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE - kubernetes ClusterIP 10.0.0.1 443/TCP 2m - redis-master ClusterIP 10.0.0.151 6379/TCP 1m - redis-slave ClusterIP 10.0.0.223 6379/TCP 6s - ``` - ## Set up and Expose the Guestbook Frontend -The guestbook application has a web frontend serving the HTTP requests written in PHP. It is configured to connect to the `redis-master` Service for write requests and the `redis-slave` service for Read requests. +The guestbook application has a web frontend serving the HTTP requests written in PHP. It is configured to connect to the `mongo` Service to store Guestbook entries. ### Creating the Guestbook Frontend Deployment @@ -184,6 +126,11 @@ The guestbook application has a web frontend serving the HTTP requests written i kubectl apply -f https://k8s.io/examples/application/guestbook/frontend-deployment.yaml ``` + + 1. Query the list of Pods to verify that the three frontend replicas are running: ```shell @@ -201,12 +148,12 @@ The guestbook application has a web frontend serving the HTTP requests written i ### Creating the Frontend Service -The `redis-slave` and `redis-master` Services you applied are only accessible within the container cluster because the default type for a Service is [ClusterIP](/docs/concepts/services-networking/service/#publishing-services---service-types). `ClusterIP` provides a single IP address for the set of Pods the Service is pointing to. This IP address is accessible only within the cluster. +The `mongo` Services you applied is only accessible within the Kubernetes cluster because the default type for a Service is [ClusterIP](/docs/concepts/services-networking/service/#publishing-services---service-types). `ClusterIP` provides a single IP address for the set of Pods the Service is pointing to. This IP address is accessible only within the cluster. -If you want guests to be able to access your guestbook, you must configure the frontend Service to be externally visible, so a client can request the Service from outside the container cluster. Minikube can only expose Services through `NodePort`. +If you want guests to be able to access your guestbook, you must configure the frontend Service to be externally visible, so a client can request the Service from outside the Kubernetes cluster. However a Kubernetes user you can use `kubectl port-forward` to access the service even though it uses a `ClusterIP`. {{< note >}} -Some cloud providers, like Google Compute Engine or Google Kubernetes Engine, support external load balancers. If your cloud provider supports load balancers and you want to use it, simply delete or comment out `type: NodePort`, and uncomment `type: LoadBalancer`. +Some cloud providers, like Google Compute Engine or Google Kubernetes Engine, support external load balancers. If your cloud provider supports load balancers and you want to use it, simply uncomment `type: LoadBalancer`. {{< /note >}} {{< codenew file="application/guestbook/frontend-service.yaml" >}} @@ -217,6 +164,11 @@ Some cloud providers, like Google Compute Engine or Google Kubernetes Engine, su kubectl apply -f https://k8s.io/examples/application/guestbook/frontend-service.yaml ``` + + 1. Query the list of Services to verify that the frontend Service is running: ```shell @@ -227,29 +179,27 @@ Some cloud providers, like Google Compute Engine or Google Kubernetes Engine, su ``` NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE - frontend NodePort 10.0.0.112 80:31323/TCP 6s + frontend ClusterIP 10.0.0.112 80/TCP 6s kubernetes ClusterIP 10.0.0.1 443/TCP 4m - redis-master ClusterIP 10.0.0.151 6379/TCP 2m - redis-slave ClusterIP 10.0.0.223 6379/TCP 1m + mongo ClusterIP 10.0.0.151 6379/TCP 2m ``` -### Viewing the Frontend Service via `NodePort` +### Viewing the Frontend Service via `kubectl port-forward` -If you deployed this application to Minikube or a local cluster, you need to find the IP address to view your Guestbook. - -1. Run the following command to get the IP address for the frontend Service. +1. Run the following command to forward port `8080` on your local machine to port `80` on the service. ```shell - minikube service frontend --url + kubectl port-forward svc/frontend 8080:80 ``` The response should be similar to this: ``` - http://192.168.99.100:31323 + Forwarding from 127.0.0.1:8080 -> 80 + Forwarding from [::1]:8080 -> 80 ``` -1. Copy the IP address, and load the page in your browser to view your guestbook. +1. load the page [http://localhost:8080](http://localhost:8080) in your browser to view your guestbook. ### Viewing the Frontend Service via `LoadBalancer` @@ -295,9 +245,7 @@ Scaling up or down is easy because your servers are defined as a Service that us frontend-3823415956-k22zn 1/1 Running 0 54m frontend-3823415956-w9gbt 1/1 Running 0 54m frontend-3823415956-x2pld 1/1 Running 0 5s - redis-master-1068406935-3lswp 1/1 Running 0 56m - redis-slave-2005841000-fpvqc 1/1 Running 0 55m - redis-slave-2005841000-phfv9 1/1 Running 0 55m + mongo-1068406935-3lswp 1/1 Running 0 56m ``` 1. Run the following command to scale down the number of frontend Pods: @@ -318,9 +266,7 @@ Scaling up or down is easy because your servers are defined as a Service that us NAME READY STATUS RESTARTS AGE frontend-3823415956-k22zn 1/1 Running 0 1h frontend-3823415956-w9gbt 1/1 Running 0 1h - redis-master-1068406935-3lswp 1/1 Running 0 1h - redis-slave-2005841000-fpvqc 1/1 Running 0 1h - redis-slave-2005841000-phfv9 1/1 Running 0 1h + mongo-1068406935-3lswp 1/1 Running 0 1h ``` @@ -332,20 +278,18 @@ Deleting the Deployments and Services also deletes any running Pods. Use labels 1. Run the following commands to delete all Pods, Deployments, and Services. ```shell - kubectl delete deployment -l app=redis - kubectl delete service -l app=redis - kubectl delete deployment -l app=guestbook - kubectl delete service -l app=guestbook + kubectl delete deployment -l app.kubernetes.io/name=mongo + kubectl delete service -l app.kubernetes.io/name=mongo + kubectl delete deployment -l app.kubernetes.io/name=guestbook + kubectl delete service -l app.kubernetes.io/name=guestbook ``` The responses should be: ``` - deployment.apps "redis-master" deleted - deployment.apps "redis-slave" deleted - service "redis-master" deleted - service "redis-slave" deleted - deployment.apps "frontend" deleted + deployment.apps "mongo" deleted + service "mongo" deleted + deployment.apps "frontend" deleted service "frontend" deleted ``` @@ -365,9 +309,7 @@ Deleting the Deployments and Services also deletes any running Pods. Use labels ## {{% heading "whatsnext" %}} -* Add [ELK logging and monitoring](/docs/tutorials/stateless-application/guestbook-logs-metrics-with-elk/) to your Guestbook application * Complete the [Kubernetes Basics](/docs/tutorials/kubernetes-basics/) Interactive Tutorials * Use Kubernetes to create a blog using [Persistent Volumes for MySQL and Wordpress](/docs/tutorials/stateful-application/mysql-wordpress-persistent-volume/#visit-your-new-wordpress-blog) * Read more about [connecting applications](/docs/concepts/services-networking/connect-applications-service/) * Read more about [Managing Resources](/docs/concepts/cluster-administration/manage-deployment/#using-labels-effectively) - diff --git a/content/en/examples/application/guestbook/frontend-deployment.yaml b/content/en/examples/application/guestbook/frontend-deployment.yaml index 23d64be644..613c654aa9 100644 --- a/content/en/examples/application/guestbook/frontend-deployment.yaml +++ b/content/en/examples/application/guestbook/frontend-deployment.yaml @@ -3,22 +3,24 @@ kind: Deployment metadata: name: frontend labels: - app: guestbook + app.kubernetes.io/name: guestbook + app.kubernetes.io/component: frontend spec: selector: matchLabels: - app: guestbook - tier: frontend + app.kubernetes.io/name: guestbook + app.kubernetes.io/component: frontend replicas: 3 template: metadata: labels: - app: guestbook - tier: frontend + app.kubernetes.io/name: guestbook + app.kubernetes.io/component: frontend spec: containers: - - name: php-redis - image: gcr.io/google-samples/gb-frontend:v4 + - name: guestbook + image: paulczar/gb-frontend:v5 + # image: gcr.io/google-samples/gb-frontend:v4 resources: requests: cpu: 100m @@ -26,13 +28,5 @@ spec: env: - name: GET_HOSTS_FROM value: dns - # Using `GET_HOSTS_FROM=dns` requires your cluster to - # provide a dns service. As of Kubernetes 1.3, DNS is a built-in - # service launched automatically. However, if the cluster you are using - # does not have a built-in DNS service, you can instead - # access an environment variable to find the master - # service's host. To do so, comment out the 'value: dns' line above, and - # uncomment the line below: - # value: env ports: - containerPort: 80 diff --git a/content/en/examples/application/guestbook/frontend-service.yaml b/content/en/examples/application/guestbook/frontend-service.yaml index 6f283f347b..34ad3771d7 100644 --- a/content/en/examples/application/guestbook/frontend-service.yaml +++ b/content/en/examples/application/guestbook/frontend-service.yaml @@ -3,16 +3,14 @@ kind: Service metadata: name: frontend labels: - app: guestbook - tier: frontend + app.kubernetes.io/name: guestbook + app.kubernetes.io/component: frontend spec: - # comment or delete the following line if you want to use a LoadBalancer - type: NodePort # if your cluster supports it, uncomment the following to automatically create # an external load-balanced IP for the frontend service. # type: LoadBalancer ports: - port: 80 selector: - app: guestbook - tier: frontend + app.kubernetes.io/name: guestbook + app.kubernetes.io/component: frontend diff --git a/content/en/examples/application/guestbook/mongo-deployment.yaml b/content/en/examples/application/guestbook/mongo-deployment.yaml new file mode 100644 index 0000000000..04908ce25b --- /dev/null +++ b/content/en/examples/application/guestbook/mongo-deployment.yaml @@ -0,0 +1,31 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: mongo + labels: + app.kubernetes.io/name: mongo + app.kubernetes.io/component: backend +spec: + selector: + matchLabels: + app.kubernetes.io/name: mongo + app.kubernetes.io/component: backend + replicas: 1 + template: + metadata: + labels: + app.kubernetes.io/name: mongo + app.kubernetes.io/component: backend + spec: + containers: + - name: mongo + image: mongo:4.2 + args: + - --bind_ip + - 0.0.0.0 + resources: + requests: + cpu: 100m + memory: 100Mi + ports: + - containerPort: 27017 diff --git a/content/en/examples/application/guestbook/mongo-service.yaml b/content/en/examples/application/guestbook/mongo-service.yaml new file mode 100644 index 0000000000..b9cef607bc --- /dev/null +++ b/content/en/examples/application/guestbook/mongo-service.yaml @@ -0,0 +1,14 @@ +apiVersion: v1 +kind: Service +metadata: + name: mongo + labels: + app.kubernetes.io/name: mongo + app.kubernetes.io/component: backend +spec: + ports: + - port: 27017 + targetPort: 27017 + selector: + app.kubernetes.io/name: mongo + app.kubernetes.io/component: backend diff --git a/content/en/examples/application/guestbook/redis-master-deployment.yaml b/content/en/examples/application/guestbook/redis-master-deployment.yaml deleted file mode 100644 index 478216d1ac..0000000000 --- a/content/en/examples/application/guestbook/redis-master-deployment.yaml +++ /dev/null @@ -1,29 +0,0 @@ -apiVersion: apps/v1 -kind: Deployment -metadata: - name: redis-master - labels: - app: redis -spec: - selector: - matchLabels: - app: redis - role: master - tier: backend - replicas: 1 - template: - metadata: - labels: - app: redis - role: master - tier: backend - spec: - containers: - - name: master - image: k8s.gcr.io/redis:e2e # or just image: redis - resources: - requests: - cpu: 100m - memory: 100Mi - ports: - - containerPort: 6379 diff --git a/content/en/examples/application/guestbook/redis-master-service.yaml b/content/en/examples/application/guestbook/redis-master-service.yaml deleted file mode 100644 index 65cef2191c..0000000000 --- a/content/en/examples/application/guestbook/redis-master-service.yaml +++ /dev/null @@ -1,17 +0,0 @@ -apiVersion: v1 -kind: Service -metadata: - name: redis-master - labels: - app: redis - role: master - tier: backend -spec: - ports: - - name: redis - port: 6379 - targetPort: 6379 - selector: - app: redis - role: master - tier: backend diff --git a/content/en/examples/application/guestbook/redis-slave-deployment.yaml b/content/en/examples/application/guestbook/redis-slave-deployment.yaml deleted file mode 100644 index 1a7b04386a..0000000000 --- a/content/en/examples/application/guestbook/redis-slave-deployment.yaml +++ /dev/null @@ -1,40 +0,0 @@ -apiVersion: apps/v1 -kind: Deployment -metadata: - name: redis-slave - labels: - app: redis -spec: - selector: - matchLabels: - app: redis - role: slave - tier: backend - replicas: 2 - template: - metadata: - labels: - app: redis - role: slave - tier: backend - spec: - containers: - - name: slave - image: gcr.io/google_samples/gb-redisslave:v3 - resources: - requests: - cpu: 100m - memory: 100Mi - env: - - name: GET_HOSTS_FROM - value: dns - # Using `GET_HOSTS_FROM=dns` requires your cluster to - # provide a dns service. As of Kubernetes 1.3, DNS is a built-in - # service launched automatically. However, if the cluster you are using - # does not have a built-in DNS service, you can instead - # access an environment variable to find the master - # service's host. To do so, comment out the 'value: dns' line above, and - # uncomment the line below: - # value: env - ports: - - containerPort: 6379 diff --git a/content/en/examples/application/guestbook/redis-slave-service.yaml b/content/en/examples/application/guestbook/redis-slave-service.yaml deleted file mode 100644 index 238fd63fb6..0000000000 --- a/content/en/examples/application/guestbook/redis-slave-service.yaml +++ /dev/null @@ -1,15 +0,0 @@ -apiVersion: v1 -kind: Service -metadata: - name: redis-slave - labels: - app: redis - role: slave - tier: backend -spec: - ports: - - port: 6379 - selector: - app: redis - role: slave - tier: backend From 3502b36f86fafbf298ef298df1d29c75853cf021 Mon Sep 17 00:00:00 2001 From: Qiming Teng Date: Tue, 19 Jan 2021 15:24:44 +0800 Subject: [PATCH 059/190] Fix outdated content in feature gates Also applied some line wrapping to unreasonable long lines. --- .../feature-gates.md | 488 ++++++++++++------ 1 file changed, 337 insertions(+), 151 deletions(-) diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates.md b/content/en/docs/reference/command-line-tools-reference/feature-gates.md index ed18f36c77..d9754afb56 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates.md @@ -1,6 +1,6 @@ --- -weight: 10 title: Feature Gates +weight: 10 content_type: concept --- @@ -48,13 +48,15 @@ different Kubernetes components. | Feature | Default | Stage | Since | Until | |---------|---------|-------|-------|-------| -| `AnyVolumeDataSource` | `false` | Alpha | 1.18 | | | `APIListChunking` | `false` | Alpha | 1.8 | 1.8 | | `APIListChunking` | `true` | Beta | 1.9 | | | `APIPriorityAndFairness` | `false` | Alpha | 1.17 | 1.19 | | `APIPriorityAndFairness` | `true` | Beta | 1.20 | | -| `APIResponseCompression` | `false` | Alpha | 1.7 | | +| `APIResponseCompression` | `false` | Alpha | 1.7 | 1.15 | +| `APIResponseCompression` | `false` | Beta | 1.16 | | | `APIServerIdentity` | `false` | Alpha | 1.20 | | +| `AllowInsecureBackendProxy` | `true` | Beta | 1.17 | | +| `AnyVolumeDataSource` | `false` | Alpha | 1.18 | | | `AppArmor` | `true` | Beta | 1.4 | | | `BalanceAttachedNodeVolumes` | `false` | Alpha | 1.11 | | | `BoundServiceAccountTokenVolume` | `false` | Alpha | 1.13 | | @@ -77,7 +79,8 @@ different Kubernetes components. | `CSIMigrationGCE` | `false` | Alpha | 1.14 | 1.16 | | `CSIMigrationGCE` | `false` | Beta | 1.17 | | | `CSIMigrationGCEComplete` | `false` | Alpha | 1.17 | | -| `CSIMigrationOpenStack` | `false` | Alpha | 1.14 | | +| `CSIMigrationOpenStack` | `false` | Alpha | 1.14 | 1.17 | +| `CSIMigrationOpenStack` | `true` | Beta | 1.18 | | | `CSIMigrationOpenStackComplete` | `false` | Alpha | 1.17 | | | `CSIMigrationvSphere` | `false` | Beta | 1.19 | | | `CSIMigrationvSphereComplete` | `false` | Beta | 1.19 | | @@ -89,26 +92,23 @@ different Kubernetes components. | `ConfigurableFSGroupPolicy` | `true` | Beta | 1.20 | | | `CronJobControllerV2` | `false` | Alpha | 1.20 | | | `CustomCPUCFSQuotaPeriod` | `false` | Alpha | 1.12 | | -| `CustomResourceDefaulting` | `false` | Alpha| 1.15 | 1.15 | -| `CustomResourceDefaulting` | `true` | Beta | 1.16 | | | `DefaultPodTopologySpread` | `false` | Alpha | 1.19 | 1.19 | | `DefaultPodTopologySpread` | `true` | Beta | 1.20 | | | `DevicePlugins` | `false` | Alpha | 1.8 | 1.9 | | `DevicePlugins` | `true` | Beta | 1.10 | | | `DisableAcceleratorUsageMetrics` | `false` | Alpha | 1.19 | 1.19 | -| `DisableAcceleratorUsageMetrics` | `true` | Beta | 1.20 | 1.22 | +| `DisableAcceleratorUsageMetrics` | `true` | Beta | 1.20 | | | `DownwardAPIHugePages` | `false` | Alpha | 1.20 | | -| `DryRun` | `false` | Alpha | 1.12 | 1.12 | -| `DryRun` | `true` | Beta | 1.13 | | | `DynamicKubeletConfig` | `false` | Alpha | 1.4 | 1.10 | | `DynamicKubeletConfig` | `true` | Beta | 1.11 | | +| `EfficientWatchResumption` | `false` | Alpha | 1.20 | | | `EndpointSlice` | `false` | Alpha | 1.16 | 1.16 | | `EndpointSlice` | `false` | Beta | 1.17 | | | `EndpointSlice` | `true` | Beta | 1.18 | | | `EndpointSliceNodeName` | `false` | Alpha | 1.20 | | | `EndpointSliceProxying` | `false` | Alpha | 1.18 | 1.18 | | `EndpointSliceProxying` | `true` | Beta | 1.19 | | -| `EndpointSliceTerminating` | `false` | Alpha | 1.20 | | +| `EndpointSliceTerminatingCondition` | `false` | Alpha | 1.20 | | | `EphemeralContainers` | `false` | Alpha | 1.16 | | | `ExpandCSIVolumes` | `false` | Alpha | 1.14 | 1.15 | | `ExpandCSIVolumes` | `true` | Beta | 1.16 | | @@ -119,19 +119,22 @@ different Kubernetes components. | `ExperimentalHostUserNamespaceDefaulting` | `false` | Beta | 1.5 | | | `GenericEphemeralVolume` | `false` | Alpha | 1.19 | | | `GracefulNodeShutdown` | `false` | Alpha | 1.20 | | +| `HPAContainerMetrics` | `false` | Alpha | 1.20 | | | `HPAScaleToZero` | `false` | Alpha | 1.16 | | | `HugePageStorageMediumSize` | `false` | Alpha | 1.18 | 1.18 | | `HugePageStorageMediumSize` | `true` | Beta | 1.19 | | -| `HyperVContainer` | `false` | Alpha | 1.10 | | +| `IPv6DualStack` | `false` | Alpha | 1.15 | | | `ImmutableEphemeralVolumes` | `false` | Alpha | 1.18 | 1.18 | | `ImmutableEphemeralVolumes` | `true` | Beta | 1.19 | | -| `IPv6DualStack` | `false` | Alpha | 1.16 | | -| `LegacyNodeRoleBehavior` | `true` | Alpha | 1.16 | | +| `KubeletCredentialProviders` | `false` | Alpha | 1.20 | | +| `KubeletPodResources` | `true` | Alpha | 1.13 | 1.14 | +| `KubeletPodResources` | `true` | Beta | 1.15 | | +| `LegacyNodeRoleBehavior` | `false` | Alpha | 1.16 | 1.18 | +| `LegacyNodeRoleBehavior` | `true` | True | 1.19 | | | `LocalStorageCapacityIsolation` | `false` | Alpha | 1.7 | 1.9 | | `LocalStorageCapacityIsolation` | `true` | Beta | 1.10 | | | `LocalStorageCapacityIsolationFSQuotaMonitoring` | `false` | Alpha | 1.15 | | | `MixedProtocolLBService` | `false` | Alpha | 1.20 | | -| `MountContainers` | `false` | Alpha | 1.9 | | | `NodeDisruptionExclusion` | `false` | Alpha | 1.16 | 1.18 | | `NodeDisruptionExclusion` | `true` | Beta | 1.19 | | | `NonPreemptingPriority` | `false` | Alpha | 1.15 | 1.18 | @@ -143,25 +146,27 @@ different Kubernetes components. | `ProcMountType` | `false` | Alpha | 1.12 | | | `QOSReserved` | `false` | Alpha | 1.11 | | | `RemainingItemCount` | `false` | Alpha | 1.15 | | +| `RemoveSelfLink` | `false` | Alpha | 1.16 | 1.19 | +| `RemoveSelfLink` | `true` | Beta | 1.20 | | | `RootCAConfigMap` | `false` | Alpha | 1.13 | 1.19 | | `RootCAConfigMap` | `true` | Beta | 1.20 | | | `RotateKubeletServerCertificate` | `false` | Alpha | 1.7 | 1.11 | | `RotateKubeletServerCertificate` | `true` | Beta | 1.12 | | | `RunAsGroup` | `true` | Beta | 1.14 | | -| `RuntimeClass` | `false` | Alpha | 1.12 | 1.13 | -| `RuntimeClass` | `true` | Beta | 1.14 | | | `SCTPSupport` | `false` | Alpha | 1.12 | 1.18 | | `SCTPSupport` | `true` | Beta | 1.19 | | | `ServerSideApply` | `false` | Alpha | 1.14 | 1.15 | | `ServerSideApply` | `true` | Beta | 1.16 | | -| `ServiceAccountIssuerDiscovery` | `false` | Alpha | 1.18 | | -| `ServiceLBNodePortControl` | `false` | Alpha | 1.20 | 1.20 | +| `ServiceAccountIssuerDiscovery` | `false` | Alpha | 1.18 | 1.19 | +| `ServiceAccountIssuerDiscovery` | `true` | Beta | 1.20 | | +| `ServiceLBNodePortControl` | `false` | Alpha | 1.20 | | | `ServiceNodeExclusion` | `false` | Alpha | 1.8 | 1.18 | | `ServiceNodeExclusion` | `true` | Beta | 1.19 | | | `ServiceTopology` | `false` | Alpha | 1.17 | | -| `SizeMemoryBackedVolumes` | `false` | Alpha | 1.20 | | | `SetHostnameAsFQDN` | `false` | Alpha | 1.19 | 1.19 | | `SetHostnameAsFQDN` | `true` | Beta | 1.20 | | +| `SizeMemoryBackedVolumes` | `false` | Alpha | 1.20 | | +| `StorageVersionAPI` | `false` | Alpha | 1.20 | | | `StorageVersionHash` | `false` | Alpha | 1.14 | 1.14 | | `StorageVersionHash` | `true` | Beta | 1.15 | | | `Sysctls` | `true` | Beta | 1.11 | | @@ -170,11 +175,11 @@ different Kubernetes components. | `TopologyManager` | `true` | Beta | 1.18 | | | `ValidateProxyRedirects` | `false` | Alpha | 1.12 | 1.13 | | `ValidateProxyRedirects` | `true` | Beta | 1.14 | | -| `WindowsEndpointSliceProxying` | `false` | Alpha | 1.19 | | -| `WindowsGMSA` | `false` | Alpha | 1.14 | | -| `WindowsGMSA` | `true` | Beta | 1.16 | | +| `WarningHeaders` | `true` | Beta | 1.19 | | | `WinDSR` | `false` | Alpha | 1.14 | | -| `WinOverlay` | `false` | Alpha | 1.14 | | +| `WinOverlay` | `false` | Alpha | 1.14 | 1.19 | +| `WinOverlay` | `true` | Beta | 1.20 | | +| `WindowsEndpointSliceProxying` | `false` | Alpha | 1.19 | | {{< /table >}} ### Feature gates for graduated or deprecated features @@ -228,6 +233,9 @@ different Kubernetes components. | `CustomResourceWebhookConversion` | `false` | Alpha | 1.13 | 1.14 | | `CustomResourceWebhookConversion` | `true` | Beta | 1.15 | 1.15 | | `CustomResourceWebhookConversion` | `true` | GA | 1.16 | - | +| `DryRun` | `false` | Alpha | 1.12 | 1.12 | +| `DryRun` | `true` | Beta | 1.13 | 1.18 | +| `DryRun` | `true` | GA | 1.19 | - | | `DynamicAuditing` | `false` | Alpha | 1.13 | 1.18 | | `DynamicAuditing` | - | Deprecated | 1.19 | - | | `DynamicProvisioningScheduling` | `false` | Alpha | 1.11 | 1.11 | @@ -247,23 +255,28 @@ different Kubernetes components. | `HugePages` | `false` | Alpha | 1.8 | 1.9 | | `HugePages` | `true` | Beta| 1.10 | 1.13 | | `HugePages` | `true` | GA | 1.14 | - | +| `HyperVContainer` | `false` | Alpha | 1.10 | 1.19 | +| `HyperVContainer` | `false` | Deprecated | 1.20 | - | | `Initializers` | `false` | Alpha | 1.7 | 1.13 | | `Initializers` | - | Deprecated | 1.14 | - | | `KubeletConfigFile` | `false` | Alpha | 1.8 | 1.9 | | `KubeletConfigFile` | - | Deprecated | 1.10 | - | -| `KubeletCredentialProviders` | `false` | Alpha | 1.20 | 1.20 | | `KubeletPluginsWatcher` | `false` | Alpha | 1.11 | 1.11 | | `KubeletPluginsWatcher` | `true` | Beta | 1.12 | 1.12 | | `KubeletPluginsWatcher` | `true` | GA | 1.13 | - | | `KubeletPodResources` | `false` | Alpha | 1.13 | 1.14 | | `KubeletPodResources` | `true` | Beta | 1.15 | | | `KubeletPodResources` | `true` | GA | 1.20 | | +| `MountContainers` | `false` | Alpha | 1.9 | 1.16 | +| `MountContainers` | `false` | Deprecated | 1.17 | - | | `MountPropagation` | `false` | Alpha | 1.8 | 1.9 | | `MountPropagation` | `true` | Beta | 1.10 | 1.11 | | `MountPropagation` | `true` | GA | 1.12 | - | | `NodeLease` | `false` | Alpha | 1.12 | 1.13 | | `NodeLease` | `true` | Beta | 1.14 | 1.16 | | `NodeLease` | `true` | GA | 1.17 | - | +| `PVCProtection` | `false` | Alpha | 1.9 | 1.9 | +| `PVCProtection` | - | Deprecated | 1.10 | - | | `PersistentLocalVolumes` | `false` | Alpha | 1.7 | 1.9 | | `PersistentLocalVolumes` | `true` | Beta | 1.10 | 1.13 | | `PersistentLocalVolumes` | `true` | GA | 1.14 | - | @@ -276,8 +289,6 @@ different Kubernetes components. | `PodShareProcessNamespace` | `false` | Alpha | 1.10 | 1.11 | | `PodShareProcessNamespace` | `true` | Beta | 1.12 | 1.16 | | `PodShareProcessNamespace` | `true` | GA | 1.17 | - | -| `PVCProtection` | `false` | Alpha | 1.9 | 1.9 | -| `PVCProtection` | - | Deprecated | 1.10 | - | | `RequestManagement` | `false` | Alpha | 1.15 | 1.16 | | `ResourceLimitsPriorityFunction` | `false` | Alpha | 1.9 | 1.18 | | `ResourceLimitsPriorityFunction` | - | Deprecated | 1.19 | - | @@ -398,65 +409,134 @@ A *General Availability* (GA) feature is also referred to as a *stable* feature. Each feature gate is designed for enabling/disabling a specific feature: +- `APIListChunking`: Enable the API clients to retrieve (`LIST` or `GET`) + resources from API server in chunks. +- `APIPriorityAndFairness`: Enable managing request concurrency with + prioritization and fairness at each server. (Renamed from `RequestManagement`) +- `APIResponseCompression`: Compress the API responses for `LIST` or `GET` requests. +- `APIServerIdentity`: Assign each API server an ID in a cluster. - `Accelerators`: Enable Nvidia GPU support when using Docker - `AdvancedAuditing`: Enable [advanced auditing](/docs/tasks/debug-application-cluster/audit/#advanced-audit) -- `AffinityInAnnotations`(*deprecated*): Enable setting [Pod affinity or anti-affinity](/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity). +- `AffinityInAnnotations`(*deprecated*): Enable setting + [Pod affinity or anti-affinity](/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity). - `AllowExtTrafficLocalEndpoints`: Enable a service to route external requests to node local endpoints. +- `AllowInsecureBackendProxy`: Enable the users to skip TLS verification of + kubelets on Pod log requests. - `AnyVolumeDataSource`: Enable use of any custom resource as the `DataSource` of a {{< glossary_tooltip text="PVC" term_id="persistent-volume-claim" >}}. -- `APIListChunking`: Enable the API clients to retrieve (`LIST` or `GET`) resources from API server in chunks. -- `APIPriorityAndFairness`: Enable managing request concurrency with prioritization and fairness at each server. (Renamed from `RequestManagement`) -- `APIResponseCompression`: Compress the API responses for `LIST` or `GET` requests. -- `APIServerIdentity`: Assign each kube-apiserver an ID in a cluster. - `AppArmor`: Enable AppArmor based mandatory access control on Linux nodes when using Docker. - See [AppArmor Tutorial](/docs/tutorials/clusters/apparmor/) for more details. + See [AppArmor Tutorial](/docs/tutorials/clusters/apparmor/) for more details. - `AttachVolumeLimit`: Enable volume plugins to report limits on number of volumes that can be attached to a node. - See [dynamic volume limits](/docs/concepts/storage/storage-limits/#dynamic-volume-limits) for more details. + See [dynamic volume limits](/docs/concepts/storage/storage-limits/#dynamic-volume-limits) for more details. - `BalanceAttachedNodeVolumes`: Include volume count on node to be considered for balanced resource allocation while scheduling. A node which has closer CPU, memory utilization, and volume count is favored by the scheduler while making decisions. - `BlockVolume`: Enable the definition and consumption of raw block devices in Pods. - See [Raw Block Volume Support](/docs/concepts/storage/persistent-volumes/#raw-block-volume-support) - for more details. + See [Raw Block Volume Support](/docs/concepts/storage/persistent-volumes/#raw-block-volume-support) + for more details. - `BoundServiceAccountTokenVolume`: Migrate ServiceAccount volumes to use a projected volume consisting of a - ServiceAccountTokenVolumeProjection. Cluster admins can use metric `serviceaccount_stale_tokens_total` to - monitor workloads that are depending on the extended tokens. If there are no such workloads, turn off - extended tokens by starting `kube-apiserver` with flag `--service-account-extend-token-expiration=false`. - Check [Bound Service Account Tokens](https://github.com/kubernetes/enhancements/blob/master/keps/sig-auth/1205-bound-service-account-tokens/README.md) + ServiceAccountTokenVolumeProjection. Cluster admins can use metric `serviceaccount_stale_tokens_total` to + monitor workloads that are depending on the extended tokens. If there are no such workloads, turn off + extended tokens by starting `kube-apiserver` with flag `--service-account-extend-token-expiration=false`. + Check [Bound Service Account Tokens](https://github.com/kubernetes/enhancements/blob/master/keps/sig-auth/1205-bound-service-account-tokens/README.md) for more details. -- `ConfigurableFSGroupPolicy`: Allows user to configure volume permission change policy for fsGroups when mounting a volume in a Pod. See [Configure volume permission and ownership change policy for Pods](/docs/tasks/configure-pod-container/security-context/#configure-volume-permission-and-ownership-change-policy-for-pods) for more details. -- `CronJobControllerV2`: Use an alternative implementation of the {{< glossary_tooltip text="CronJob" term_id="cronjob" >}} controller. Otherwise, version 1 of the same controller is selected. The version 2 controller provides experimental performance improvements. -- `CPUManager`: Enable container level CPU affinity support, see [CPU Management Policies](/docs/tasks/administer-cluster/cpu-management-policies/). +- `CPUManager`: Enable container level CPU affinity support, see + [CPU Management Policies](/docs/tasks/administer-cluster/cpu-management-policies/). - `CRIContainerLogRotation`: Enable container log rotation for cri container runtime. -- `CSIBlockVolume`: Enable external CSI volume drivers to support block storage. See the [`csi` raw block volume support](/docs/concepts/storage/volumes/#csi-raw-block-volume-support) documentation for more details. -- `CSIDriverRegistry`: Enable all logic related to the CSIDriver API object in csi.storage.k8s.io. +- `CSIBlockVolume`: Enable external CSI volume drivers to support block storage. + See the [`csi` raw block volume support](/docs/concepts/storage/volumes/#csi-raw-block-volume-support) + documentation for more details. +- `CSIDriverRegistry`: Enable all logic related to the CSIDriver API object in + csi.storage.k8s.io. - `CSIInlineVolume`: Enable CSI Inline volumes support for pods. -- `CSIMigration`: Enables shims and translation logic to route volume operations from in-tree plugins to corresponding pre-installed CSI plugins -- `CSIMigrationAWS`: Enables shims and translation logic to route volume operations from the AWS-EBS in-tree plugin to EBS CSI plugin. Supports falling back to in-tree EBS plugin if a node does not have EBS CSI plugin installed and configured. Requires CSIMigration feature flag enabled. -- `CSIMigrationAWSComplete`: Stops registering the EBS in-tree plugin in kubelet and volume controllers and enables shims and translation logic to route volume operations from the AWS-EBS in-tree plugin to EBS CSI plugin. Requires CSIMigration and CSIMigrationAWS feature flags enabled and EBS CSI plugin installed and configured on all nodes in the cluster. -- `CSIMigrationAzureDisk`: Enables shims and translation logic to route volume operations from the Azure-Disk in-tree plugin to AzureDisk CSI plugin. Supports falling back to in-tree AzureDisk plugin if a node does not have AzureDisk CSI plugin installed and configured. Requires CSIMigration feature flag enabled. -- `CSIMigrationAzureDiskComplete`: Stops registering the Azure-Disk in-tree plugin in kubelet and volume controllers and enables shims and translation logic to route volume operations from the Azure-Disk in-tree plugin to AzureDisk CSI plugin. Requires CSIMigration and CSIMigrationAzureDisk feature flags enabled and AzureDisk CSI plugin installed and configured on all nodes in the cluster. -- `CSIMigrationAzureFile`: Enables shims and translation logic to route volume operations from the Azure-File in-tree plugin to AzureFile CSI plugin. Supports falling back to in-tree AzureFile plugin if a node does not have AzureFile CSI plugin installed and configured. Requires CSIMigration feature flag enabled. -- `CSIMigrationAzureFileComplete`: Stops registering the Azure-File in-tree plugin in kubelet and volume controllers and enables shims and translation logic to route volume operations from the Azure-File in-tree plugin to AzureFile CSI plugin. Requires CSIMigration and CSIMigrationAzureFile feature flags enabled and AzureFile CSI plugin installed and configured on all nodes in the cluster. -- `CSIMigrationGCE`: Enables shims and translation logic to route volume operations from the GCE-PD in-tree plugin to PD CSI plugin. Supports falling back to in-tree GCE plugin if a node does not have PD CSI plugin installed and configured. Requires CSIMigration feature flag enabled. -- `CSIMigrationGCEComplete`: Stops registering the GCE-PD in-tree plugin in kubelet and volume controllers and enables shims and translation logic to route volume operations from the GCE-PD in-tree plugin to PD CSI plugin. Requires CSIMigration and CSIMigrationGCE feature flags enabled and PD CSI plugin installed and configured on all nodes in the cluster. -- `CSIMigrationOpenStack`: Enables shims and translation logic to route volume operations from the Cinder in-tree plugin to Cinder CSI plugin. Supports falling back to in-tree Cinder plugin if a node does not have Cinder CSI plugin installed and configured. Requires CSIMigration feature flag enabled. -- `CSIMigrationOpenStackComplete`: Stops registering the Cinder in-tree plugin in kubelet and volume controllers and enables shims and translation logic to route volume operations from the Cinder in-tree plugin to Cinder CSI plugin. Requires CSIMigration and CSIMigrationOpenStack feature flags enabled and Cinder CSI plugin installed and configured on all nodes in the cluster. -- `CSIMigrationvSphere`: Enables shims and translation logic to route volume operations from the vSphere in-tree plugin to vSphere CSI plugin. Supports falling back to in-tree vSphere plugin if a node does not have vSphere CSI plugin installed and configured. Requires CSIMigration feature flag enabled. -- `CSIMigrationvSphereComplete`: Stops registering the vSphere in-tree plugin in kubelet and volume controllers and enables shims and translation logic to route volume operations from the vSphere in-tree plugin to vSphere CSI plugin. Requires CSIMigration and CSIMigrationvSphere feature flags enabled and vSphere CSI plugin installed and configured on all nodes in the cluster. +- `CSIMigration`: Enables shims and translation logic to route volume + operations from in-tree plugins to corresponding pre-installed CSI plugins +- `CSIMigrationAWS`: Enables shims and translation logic to route volume + operations from the AWS-EBS in-tree plugin to EBS CSI plugin. Supports + falling back to in-tree EBS plugin if a node does not have EBS CSI plugin + installed and configured. Requires CSIMigration feature flag enabled. +- `CSIMigrationAWSComplete`: Stops registering the EBS in-tree plugin in + kubelet and volume controllers and enables shims and translation logic to + route volume operations from the AWS-EBS in-tree plugin to EBS CSI plugin. + Requires CSIMigration and CSIMigrationAWS feature flags enabled and EBS CSI + plugin installed and configured on all nodes in the cluster. +- `CSIMigrationAzureDisk`: Enables shims and translation logic to route volume + operations from the Azure-Disk in-tree plugin to AzureDisk CSI plugin. + Supports falling back to in-tree AzureDisk plugin if a node does not have + AzureDisk CSI plugin installed and configured. Requires CSIMigration feature + flag enabled. +- `CSIMigrationAzureDiskComplete`: Stops registering the Azure-Disk in-tree + plugin in kubelet and volume controllers and enables shims and translation + logic to route volume operations from the Azure-Disk in-tree plugin to + AzureDisk CSI plugin. Requires CSIMigration and CSIMigrationAzureDisk feature + flags enabled and AzureDisk CSI plugin installed and configured on all nodes + in the cluster. +- `CSIMigrationAzureFile`: Enables shims and translation logic to route volume + operations from the Azure-File in-tree plugin to AzureFile CSI plugin. + Supports falling back to in-tree AzureFile plugin if a node does not have + AzureFile CSI plugin installed and configured. Requires CSIMigration feature + flag enabled. +- `CSIMigrationAzureFileComplete`: Stops registering the Azure-File in-tree + plugin in kubelet and volume controllers and enables shims and translation + logic to route volume operations from the Azure-File in-tree plugin to + AzureFile CSI plugin. Requires CSIMigration and CSIMigrationAzureFile feature + flags enabled and AzureFile CSI plugin installed and configured on all nodes + in the cluster. +- `CSIMigrationGCE`: Enables shims and translation logic to route volume + operations from the GCE-PD in-tree plugin to PD CSI plugin. Supports falling + back to in-tree GCE plugin if a node does not have PD CSI plugin installed and + configured. Requires CSIMigration feature flag enabled. +- `CSIMigrationGCEComplete`: Stops registering the GCE-PD in-tree plugin in + kubelet and volume controllers and enables shims and translation logic to + route volume operations from the GCE-PD in-tree plugin to PD CSI plugin. + Requires CSIMigration and CSIMigrationGCE feature flags enabled and PD CSI + plugin installed and configured on all nodes in the cluster. +- `CSIMigrationOpenStack`: Enables shims and translation logic to route volume + operations from the Cinder in-tree plugin to Cinder CSI plugin. Supports + falling back to in-tree Cinder plugin if a node does not have Cinder CSI + plugin installed and configured. Requires CSIMigration feature flag enabled. +- `CSIMigrationOpenStackComplete`: Stops registering the Cinder in-tree plugin in + kubelet and volume controllers and enables shims and translation logic to route + volume operations from the Cinder in-tree plugin to Cinder CSI plugin. + Requires CSIMigration and CSIMigrationOpenStack feature flags enabled and Cinder + CSI plugin installed and configured on all nodes in the cluster. +- `CSIMigrationvSphere`: Enables shims and translation logic to route volume operations + from the vSphere in-tree plugin to vSphere CSI plugin. + Supports falling back to in-tree vSphere plugin if a node does not have vSphere + CSI plugin installed and configured. Requires CSIMigration feature flag enabled. +- `CSIMigrationvSphereComplete`: Stops registering the vSphere in-tree plugin in kubelet + and volume controllers and enables shims and translation logic to route volume operations + from the vSphere in-tree plugin to vSphere CSI plugin. Requires CSIMigration and + CSIMigrationvSphere feature flags enabled and vSphere CSI plugin installed and + configured on all nodes in the cluster. - `CSINodeInfo`: Enable all logic related to the CSINodeInfo API object in csi.storage.k8s.io. - `CSIPersistentVolume`: Enable discovering and mounting volumes provisioned through a [CSI (Container Storage Interface)](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/container-storage-interface.md) compatible volume plugin. -- `CSIServiceAccountToken`: Enable CSI drivers to receive the pods' service account token that they mount volumes for. See [Token Requests](https://kubernetes-csi.github.io/docs/token-requests.html). -- `CSIStorageCapacity`: Enables CSI drivers to publish storage capacity information and the Kubernetes scheduler to use that information when scheduling pods. See [Storage Capacity](/docs/concepts/storage/storage-capacity/). +- `CSIServiceAccountToken`: Enable CSI drivers to receive the pods' service account token + that they mount volumes for. See + [Token Requests](https://kubernetes-csi.github.io/docs/token-requests.html). +- `CSIStorageCapacity`: Enables CSI drivers to publish storage capacity information + and the Kubernetes scheduler to use that information when scheduling pods. See + [Storage Capacity](/docs/concepts/storage/storage-capacity/). Check the [`csi` volume type](/docs/concepts/storage/volumes/#csi) documentation for more details. -- `CSIVolumeFSGroupPolicy`: Allows CSIDrivers to use the `fsGroupPolicy` field. This field controls whether volumes created by a CSIDriver support volume ownership and permission modifications when these volumes are mounted. -- `CustomCPUCFSQuotaPeriod`: Enable nodes to change CPUCFSQuotaPeriod. +- `CSIVolumeFSGroupPolicy`: Allows CSIDrivers to use the `fsGroupPolicy` field. + This field controls whether volumes created by a CSIDriver support volume ownership + and permission modifications when these volumes are mounted. +- `ConfigurableFSGroupPolicy`: Allows user to configure volume permission change policy + for fsGroups when mounting a volume in a Pod. See + [Configure volume permission and ownership change policy for Pods](/docs/tasks/configure-pod-container/security-context/#configure-volume-permission-and-ownership-change-policy-for-pods) + for more details. +- `CronJobControllerV2`: Use an alternative implementation of the + {{< glossary_tooltip text="CronJob" term_id="cronjob" >}} controller. Otherwise, + version 1 of the same controller is selected. + The version 2 controller provides experimental performance improvements. +- `CustomCPUCFSQuotaPeriod`: Enable nodes to change `cpuCFSQuotaPeriod` in + [kubelet config](/docs/tasks/administer-cluster/kubelet-config-file/). - `CustomPodDNS`: Enable customizing the DNS settings for a Pod using its `dnsConfig` property. - Check [Pod's DNS Config](/docs/concepts/services-networking/dns-pod-service/#pods-dns-config) - for more details. + Check [Pod's DNS Config](/docs/concepts/services-networking/dns-pod-service/#pods-dns-config) + for more details. - `CustomResourceDefaulting`: Enable CRD support for default values in OpenAPI v3 validation schemas. - `CustomResourcePublishOpenAPI`: Enables publishing of CRD OpenAPI specs. - `CustomResourceSubresources`: Enable `/status` and `/scale` subresources @@ -466,147 +546,253 @@ Each feature gate is designed for enabling/disabling a specific feature: - `CustomResourceWebhookConversion`: Enable webhook-based conversion on resources created from [CustomResourceDefinition](/docs/concepts/extend-kubernetes/api-extension/custom-resources/). troubleshoot a running Pod. -- `DisableAcceleratorUsageMetrics`: [Disable accelerator metrics collected by the kubelet](/docs/concepts/cluster-administration/system-metrics/#disable-accelerator-metrics). -- `DevicePlugins`: Enable the [device-plugins](/docs/concepts/cluster-administration/device-plugins/) - based resource provisioning on nodes. - `DefaultPodTopologySpread`: Enables the use of `PodTopologySpread` scheduling plugin to do [default spreading](/docs/concepts/workloads/pods/pod-topology-spread-constraints/#internal-default-constraints). -- `DownwardAPIHugePages`: Enables usage of hugepages in downward API. +- `DevicePlugins`: Enable the [device-plugins](/docs/concepts/cluster-administration/device-plugins/) + based resource provisioning on nodes. +- `DisableAcceleratorUsageMetrics`: + [Disable accelerator metrics collected by the kubelet](/docs/concepts/cluster-administration/system-metrics/#disable-accelerator-metrics). +- `DownwardAPIHugePages`: Enables usage of hugepages in + [downward API](/docs/tasks/inject-data-application/downward-api-volume-expose-pod-information). - `DryRun`: Enable server-side [dry run](/docs/reference/using-api/api-concepts/#dry-run) requests so that validation, merging, and mutation can be tested without committing. - `DynamicAuditing`(*deprecated*): Used to enable dynamic auditing before v1.19. -- `DynamicKubeletConfig`: Enable the dynamic configuration of kubelet. See [Reconfigure kubelet](/docs/tasks/administer-cluster/reconfigure-kubelet/). -- `DynamicProvisioningScheduling`: Extend the default scheduler to be aware of volume topology and handle PV provisioning. +- `DynamicKubeletConfig`: Enable the dynamic configuration of kubelet. See + [Reconfigure kubelet](/docs/tasks/administer-cluster/reconfigure-kubelet/). +- `DynamicProvisioningScheduling`: Extend the default scheduler to be aware of + volume topology and handle PV provisioning. This feature is superseded by the `VolumeScheduling` feature completely in v1.12. -- `DynamicVolumeProvisioning`(*deprecated*): Enable the [dynamic provisioning](/docs/concepts/storage/dynamic-provisioning/) of persistent volumes to Pods. -- `EnableAggregatedDiscoveryTimeout` (*deprecated*): Enable the five second timeout on aggregated discovery calls. -- `EnableEquivalenceClassCache`: Enable the scheduler to cache equivalence of nodes when scheduling Pods. -- `EphemeralContainers`: Enable the ability to add {{< glossary_tooltip text="ephemeral containers" - term_id="ephemeral-container" >}} to running pods. -- `EvenPodsSpread`: Enable pods to be scheduled evenly across topology domains. See [Pod Topology Spread Constraints](/docs/concepts/workloads/pods/pod-topology-spread-constraints/). -- `ExecProbeTimeout`: Ensure kubelet respects exec probe timeouts. This feature gate exists in case any of your existing workloads depend on a now-corrected fault where Kubernetes ignored exec probe timeouts. See [readiness probes](/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#configure-probes). -- `ExpandInUsePersistentVolumes`: Enable expanding in-use PVCs. See [Resizing an in-use PersistentVolumeClaim](/docs/concepts/storage/persistent-volumes/#resizing-an-in-use-persistentvolumeclaim). -- `ExpandPersistentVolumes`: Enable the expanding of persistent volumes. See [Expanding Persistent Volumes Claims](/docs/concepts/storage/persistent-volumes/#expanding-persistent-volumes-claims). -- `ExperimentalCriticalPodAnnotation`: Enable annotating specific pods as *critical* so that their [scheduling is guaranteed](/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods/). - This feature is deprecated by Pod Priority and Preemption as of v1.13. -- `ExperimentalHostUserNamespaceDefaultingGate`: Enabling the defaulting user - namespace to host. This is for containers that are using other host namespaces, - host mounts, or containers that are privileged or using specific non-namespaced - capabilities (e.g. `MKNODE`, `SYS_MODULE` etc.). This should only be enabled - if user namespace remapping is enabled in the Docker daemon. -- `EndpointSlice`: Enables Endpoint Slices for more scalable and extensible - network endpoints. See [Enabling Endpoint Slices](/docs/tasks/administer-cluster/enabling-endpointslices/). +- `DynamicVolumeProvisioning`(*deprecated*): Enable the + [dynamic provisioning](/docs/concepts/storage/dynamic-provisioning/) of persistent volumes to Pods. +- `EfficientWatchResumption`: Allows for storage-originated bookmark (progress + notify) events to be delivered to the users. This is only applied to watch + operations. +- `EnableAggregatedDiscoveryTimeout` (*deprecated*): Enable the five second + timeout on aggregated discovery calls. +- `EnableEquivalenceClassCache`: Enable the scheduler to cache equivalence of + nodes when scheduling Pods. +- `EndpointSlice`: Enables EndpointSlices for more scalable and extensible + network endpoints. See [Enabling EndpointSlices](/docs/tasks/administer-cluster/enabling-endpointslices/). - `EndpointSliceNodeName`: Enables EndpointSlice `nodeName` field. -- `EndpointSliceTerminating`: Enables EndpointSlice `terminating` and `serving` - condition fields. -- `EndpointSliceProxying`: When this feature gate is enabled, kube-proxy running +- `EndpointSliceProxying`: When enabled, kube-proxy running on Linux will use EndpointSlices as the primary data source instead of Endpoints, enabling scalability and performance improvements. See [Enabling Endpoint Slices](/docs/tasks/administer-cluster/enabling-endpointslices/). -- `WindowsEndpointSliceProxying`: When this feature gate is enabled, kube-proxy - running on Windows will use EndpointSlices as the primary data source instead - of Endpoints, enabling scalability and performance improvements. See - [Enabling Endpoint Slices](/docs/tasks/administer-cluster/enabling-endpointslices/). +- `EndpointSliceTerminatingCondition`: Enables EndpointSlice `terminating` and `serving` + condition fields. +- `EphemeralContainers`: Enable the ability to add + {{< glossary_tooltip text="ephemeral containers" term_id="ephemeral-container" >}} + to running pods. +- `EvenPodsSpread`: Enable pods to be scheduled evenly across topology domains. See + [Pod Topology Spread Constraints](/docs/concepts/workloads/pods/pod-topology-spread-constraints/). +- `ExecProbeTimeout`: Ensure kubelet respects exec probe timeouts. + This feature gate exists in case any of your existing workloads depend on a + now-corrected fault where Kubernetes ignored exec probe timeouts. See + [readiness probes](/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#configure-probes). +- `ExpandCSIVolumes`: Enable the expanding of CSI volumes. +- `ExpandInUsePersistentVolumes`: Enable expanding in-use PVCs. See + [Resizing an in-use PersistentVolumeClaim](/docs/concepts/storage/persistent-volumes/#resizing-an-in-use-persistentvolumeclaim). +- `ExpandPersistentVolumes`: Enable the expanding of persistent volumes. See + [Expanding Persistent Volumes Claims](/docs/concepts/storage/persistent-volumes/#expanding-persistent-volumes-claims). +- `ExperimentalCriticalPodAnnotation`: Enable annotating specific pods as *critical* + so that their [scheduling is guaranteed](/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods/). + This feature is deprecated by Pod Priority and Preemption as of v1.13. +- `ExperimentalHostUserNamespaceDefaulting`: Enabling the defaulting user + namespace to host. This is for containers that are using other host namespaces, + host mounts, or containers that are privileged or using specific non-namespaced + capabilities (e.g. `MKNODE`, `SYS_MODULE` etc.). This should only be enabled + if user namespace remapping is enabled in the Docker daemon. - `GCERegionalPersistentDisk`: Enable the regional PD feature on GCE. -- `GenericEphemeralVolume`: Enables ephemeral, inline volumes that support all features of normal volumes (can be provided by third-party storage vendors, storage capacity tracking, restore from snapshot, etc.). See [Ephemeral Volumes](/docs/concepts/storage/ephemeral-volumes/). -- `GracefulNodeShutdown`: Enables support for graceful shutdown in kubelet. During a system shutdown, kubelet will attempt to detect the shutdown event and gracefully terminate pods running on the node. See [Graceful Node Shutdown](/docs/concepts/architecture/nodes/#graceful-node-shutdown) for more details. -- `HugePages`: Enable the allocation and consumption of pre-allocated [huge pages](/docs/tasks/manage-hugepages/scheduling-hugepages/). -- `HugePageStorageMediumSize`: Enable support for multiple sizes pre-allocated [huge pages](/docs/tasks/manage-hugepages/scheduling-hugepages/). -- `HyperVContainer`: Enable [Hyper-V isolation](https://docs.microsoft.com/en-us/virtualization/windowscontainers/manage-containers/hyperv-container) for Windows containers. -- `HPAScaleToZero`: Enables setting `minReplicas` to 0 for `HorizontalPodAutoscaler` resources when using custom or external metrics. -- `ImmutableEphemeralVolumes`: Allows for marking individual Secrets and ConfigMaps as immutable for better safety and performance. -- `KubeletConfigFile`: Enable loading kubelet configuration from a file specified using a config file. - See [setting kubelet parameters via a config file](/docs/tasks/administer-cluster/kubelet-config-file/) for more details. +- `GenericEphemeralVolume`: Enables ephemeral, inline volumes that support all features + of normal volumes (can be provided by third-party storage vendors, storage capacity tracking, + restore from snapshot, etc.). + See [Ephemeral Volumes](/docs/concepts/storage/ephemeral-volumes/). +- `GracefulNodeShutdown`: Enables support for graceful shutdown in kubelet. + During a system shutdown, kubelet will attempt to detect the shutdown event + and gracefully terminate pods running on the node. See + [Graceful Node Shutdown](/docs/concepts/architecture/nodes/#graceful-node-shutdown) + for more details. +- `HPAContainerMetrics`: Enable the `HorizontalPodAutoscaler` to scale based on + metrics from individual containers in target pods. +- `HPAScaleToZero`: Enables setting `minReplicas` to 0 for `HorizontalPodAutoscaler` + resources when using custom or external metrics. +- `HugePages`: Enable the allocation and consumption of pre-allocated + [huge pages](/docs/tasks/manage-hugepages/scheduling-hugepages/). +- `HugePageStorageMediumSize`: Enable support for multiple sizes pre-allocated + [huge pages](/docs/tasks/manage-hugepages/scheduling-hugepages/). +- `HyperVContainer`: Enable + [Hyper-V isolation](https://docs.microsoft.com/en-us/virtualization/windowscontainers/manage-containers/hyperv-container) + for Windows containers. +- `IPv6DualStack`: Enable [dual stack](/docs/concepts/services-networking/dual-stack/) + support for IPv6. +- `ImmutableEphemeralVolumes`: Allows for marking individual Secrets and ConfigMaps as + immutable for better safety and performance. +- `KubeletConfigFile` (*deprecated*): Enable loading kubelet configuration from + a file specified using a config file. + See [setting kubelet parameters via a config file](/docs/tasks/administer-cluster/kubelet-config-file/) + for more details. - `KubeletCredentialProviders`: Enable kubelet exec credential providers for image pull credentials. - `KubeletPluginsWatcher`: Enable probe-based plugin watcher utility to enable kubelet to discover plugins such as [CSI volume drivers](/docs/concepts/storage/volumes/#csi). -- `KubeletPodResources`: Enable the kubelet's pod resources grpc endpoint. - See [Support Device Monitoring](https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/compute-device-assignment.md) for more details. -- `LegacyNodeRoleBehavior`: When disabled, legacy behavior in service load balancers and node disruption will ignore the `node-role.kubernetes.io/master` label in favor of the feature-specific labels provided by `NodeDisruptionExclusion` and `ServiceNodeExclusion`. -- `LocalStorageCapacityIsolation`: Enable the consumption of [local ephemeral storage](/docs/concepts/configuration/manage-resources-containers/) and also the `sizeLimit` property of an [emptyDir volume](/docs/concepts/storage/volumes/#emptydir). -- `LocalStorageCapacityIsolationFSQuotaMonitoring`: When `LocalStorageCapacityIsolation` is enabled for [local ephemeral storage](/docs/concepts/configuration/manage-resources-containers/) and the backing filesystem for [emptyDir volumes](/docs/concepts/storage/volumes/#emptydir) supports project quotas and they are enabled, use project quotas to monitor [emptyDir volume](/docs/concepts/storage/volumes/#emptydir) storage consumption rather than filesystem walk for better performance and accuracy. -- `MixedProtocolLBService`: Enable using different protocols in the same LoadBalancer type Service instance. -- `MountContainers`: Enable using utility containers on host as the volume mounter. +- `KubeletPodResources`: Enable the kubelet's pod resources GRPC endpoint. See + [Support Device Monitoring](https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/compute-device-assignment.md) + for more details. +- `LegacyNodeRoleBehavior`: When disabled, legacy behavior in service load balancers and + node disruption will ignore the `node-role.kubernetes.io/master` label in favor of the + feature-specific labels provided by `NodeDisruptionExclusion` and `ServiceNodeExclusion`. +- `LocalStorageCapacityIsolation`: Enable the consumption of + [local ephemeral storage](/docs/concepts/configuration/manage-resources-containers/) + and also the `sizeLimit` property of an + [emptyDir volume](/docs/concepts/storage/volumes/#emptydir). +- `LocalStorageCapacityIsolationFSQuotaMonitoring`: When `LocalStorageCapacityIsolation` + is enabled for + [local ephemeral storage](/docs/concepts/configuration/manage-resources-containers/) + and the backing filesystem for [emptyDir volumes](/docs/concepts/storage/volumes/#emptydir) + supports project quotas and they are enabled, use project quotas to monitor + [emptyDir volume](/docs/concepts/storage/volumes/#emptydir) storage consumption rather than + filesystem walk for better performance and accuracy. +- `MixedProtocolLBService`: Enable using different protocols in the same `LoadBalancer` type + Service instance. +- `MountContainers` (*deprecated*): Enable using utility containers on host as + the volume mounter. - `MountPropagation`: Enable sharing volume mounted by one container to other containers or pods. For more details, please see [mount propagation](/docs/concepts/storage/volumes/#mount-propagation). -- `NodeDisruptionExclusion`: Enable use of the node label `node.kubernetes.io/exclude-disruption` which prevents nodes from being evacuated during zone failures. +- `NodeDisruptionExclusion`: Enable use of the Node label `node.kubernetes.io/exclude-disruption` + which prevents nodes from being evacuated during zone failures. - `NodeLease`: Enable the new Lease API to report node heartbeats, which could be used as a node health signal. -- `NonPreemptingPriority`: Enable NonPreempting option for PriorityClass and Pod. +- `NonPreemptingPriority`: Enable `preemptionPolicy` field for PriorityClass and Pod. +- `PVCProtection`: Enable the prevention of a PersistentVolumeClaim (PVC) from + being deleted when it is still used by any Pod. - `PersistentLocalVolumes`: Enable the usage of `local` volume type in Pods. Pod affinity has to be specified if requesting a `local` volume. - `PodDisruptionBudget`: Enable the [PodDisruptionBudget](/docs/tasks/run-application/configure-pdb/) feature. -- `PodOverhead`: Enable the [PodOverhead](/docs/concepts/scheduling-eviction/pod-overhead/) feature to account for pod overheads. -- `PodPriority`: Enable the descheduling and preemption of Pods based on their [priorities](/docs/concepts/configuration/pod-priority-preemption/). +- `PodOverhead`: Enable the [PodOverhead](/docs/concepts/scheduling-eviction/pod-overhead/) + feature to account for pod overheads. +- `PodPriority`: Enable the descheduling and preemption of Pods based on their + [priorities](/docs/concepts/configuration/pod-priority-preemption/). - `PodReadinessGates`: Enable the setting of `PodReadinessGate` field for extending Pod readiness evaluation. See [Pod readiness gate](/docs/concepts/workloads/pods/pod-lifecycle/#pod-readiness-gate) for more details. - `PodShareProcessNamespace`: Enable the setting of `shareProcessNamespace` in a Pod for sharing a single process namespace between containers running in a pod. More details can be found in [Share Process Namespace between Containers in a Pod](/docs/tasks/configure-pod-container/share-process-namespace/). -- `ProcMountType`: Enables control over ProcMountType for containers. -- `PVCProtection`: Enable the prevention of a PersistentVolumeClaim (PVC) from - being deleted when it is still used by any Pod. -- `QOSReserved`: Allows resource reservations at the QoS level preventing pods at lower QoS levels from - bursting into resources requested at higher QoS levels (memory only for now). +- `ProcMountType`: Enables control over the type proc mounts for containers + by setting the `procMount` field of a SecurityContext. +- `QOSReserved`: Allows resource reservations at the QoS level preventing pods + at lower QoS levels from bursting into resources requested at higher QoS levels + (memory only for now). +- `RemainingItemCount`: Allow the API servers to show a count of remaining + items in the response to a + [chunking list request](/docs/reference/using-api/api-concepts/#retrieving-large-results-sets-in-chunks). +- `RemoveSelfLink`: Deprecates and removes `selfLink` from ObjectMeta and + ListMeta. - `ResourceLimitsPriorityFunction` (*deprecated*): Enable a scheduler priority function that assigns a lowest possible score of 1 to a node that satisfies at least one of the input Pod's cpu and memory limits. The intent is to break ties between nodes with same scores. - `ResourceQuotaScopeSelectors`: Enable resource quota scope selectors. -- `RootCAConfigMap`: Configure the kube-controller-manager to publish a {{< glossary_tooltip text="ConfigMap" term_id="configmap" >}} named `kube-root-ca.crt` to every namespace. This ConfigMap contains a CA bundle used for verifying connections to the kube-apiserver. - See [Bound Service Account Tokens](https://github.com/kubernetes/enhancements/blob/master/keps/sig-auth/1205-bound-service-account-tokens/README.md) for more details. +- `RootCAConfigMap`: Configure the `kube-controller-manager` to publish a + {{< glossary_tooltip text="ConfigMap" term_id="configmap" >}} named `kube-root-ca.crt` + to every namespace. This ConfigMap contains a CA bundle used for verifying connections + to the kube-apiserver. See + [Bound Service Account Tokens](https://github.com/kubernetes/enhancements/blob/master/keps/sig-auth/1205-bound-service-account-tokens/README.md) + for more details. - `RotateKubeletClientCertificate`: Enable the rotation of the client TLS certificate on the kubelet. See [kubelet configuration](/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping/#kubelet-configuration) for more details. - `RotateKubeletServerCertificate`: Enable the rotation of the server TLS certificate on the kubelet. - See [kubelet configuration](/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping/#kubelet-configuration) for more details. -- `RunAsGroup`: Enable control over the primary group ID set on the init processes of containers. -- `RuntimeClass`: Enable the [RuntimeClass](/docs/concepts/containers/runtime-class/) feature for selecting container runtime configurations. -- `ScheduleDaemonSetPods`: Enable DaemonSet Pods to be scheduled by the default scheduler instead of the DaemonSet controller. -- `SCTPSupport`: Enables the _SCTP_ `protocol` value in Pod, Service, Endpoints, EndpointSlice, and NetworkPolicy definitions. -- `ServerSideApply`: Enables the [Sever Side Apply (SSA)](/docs/reference/using-api/server-side-apply/) path at the API Server. -- `ServiceAccountIssuerDiscovery`: Enable OIDC discovery endpoints (issuer and JWKS URLs) for the service account issuer in the API server. See [Configure Service Accounts for Pods](/docs/tasks/configure-pod-container/configure-service-account/#service-account-issuer-discovery) for more details. + See [kubelet configuration](/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping/#kubelet-configuration) + for more details. +- `RunAsGroup`: Enable control over the primary group ID set on the init + processes of containers. +- `RuntimeClass`: Enable the [RuntimeClass](/docs/concepts/containers/runtime-class/) feature + for selecting container runtime configurations. +- `ScheduleDaemonSetPods`: Enable DaemonSet Pods to be scheduled by the default scheduler + instead of the DaemonSet controller. +- `SCTPSupport`: Enables the _SCTP_ `protocol` value in Pod, Service, + Endpoints, EndpointSlice, and NetworkPolicy definitions. +- `ServerSideApply`: Enables the [Sever Side Apply (SSA)](/docs/reference/using-api/server-side-apply/) + feature on the API Server. +- `ServiceAccountIssuerDiscovery`: Enable OIDC discovery endpoints (issuer and + JWKS URLs) for the service account issuer in the API server. See + [Configure Service Accounts for Pods](/docs/tasks/configure-pod-container/configure-service-account/#service-account-issuer-discovery) + for more details. - `ServiceAppProtocol`: Enables the `AppProtocol` field on Services and Endpoints. -- `ServiceLBNodePortControl`: Enables the `spec.allocateLoadBalancerNodePorts` field on Services. +- `ServiceLBNodePortControl`: Enables the `spec.allocateLoadBalancerNodePorts` + field on Services. - `ServiceLoadBalancerFinalizer`: Enable finalizer protection for Service load balancers. -- `ServiceNodeExclusion`: Enable the exclusion of nodes from load balancers created by a cloud provider. - A node is eligible for exclusion if labelled with "`alpha.service-controller.kubernetes.io/exclude-balancer`" key or `node.kubernetes.io/exclude-from-external-load-balancers`. -- `ServiceTopology`: Enable service to route traffic based upon the Node topology of the cluster. See [ServiceTopology](/docs/concepts/services-networking/service-topology/) for more details. -- `SizeMemoryBackedVolumes`: Enables kubelet support to size memory backed volumes. See [volumes](docs/concepts/storage/volumes) for more details. -- `SetHostnameAsFQDN`: Enable the ability of setting Fully Qualified Domain Name(FQDN) as hostname of pod. See [Pod's `setHostnameAsFQDN` field](/docs/concepts/services-networking/dns-pod-service/#pod-sethostnameasfqdn-field). -- `StartupProbe`: Enable the [startup](/docs/concepts/workloads/pods/pod-lifecycle/#when-should-you-use-a-startup-probe) probe in the kubelet. +- `ServiceNodeExclusion`: Enable the exclusion of nodes from load balancers + created by a cloud provider. A node is eligible for exclusion if labelled with + "`node.kubernetes.io/exclude-from-external-load-balancers`". +- `ServiceTopology`: Enable service to route traffic based upon the Node + topology of the cluster. See + [ServiceTopology](/docs/concepts/services-networking/service-topology/) + for more details. +- `SizeMemoryBackedVolumes`: Enables kubelet support to size memory backed volumes. + See [volumes](docs/concepts/storage/volumes) for more details. +- `SetHostnameAsFQDN`: Enable the ability of setting Fully Qualified Domain + Name(FQDN) as the hostname of a pod. See + [Pod's `setHostnameAsFQDN` field](/docs/concepts/services-networking/dns-pod-service/#pod-sethostnameasfqdn-field). +- `SizeMemoryBackedVolumes`: Enable kubelets to determine the size limit for + memory-backed volumes (mainly `emptyDir` volumes). +- `StartupProbe`: Enable the + [startup](/docs/concepts/workloads/pods/pod-lifecycle/#when-should-you-use-a-startup-probe) + probe in the kubelet. - `StorageObjectInUseProtection`: Postpone the deletion of PersistentVolume or PersistentVolumeClaim objects if they are still being used. -- `StorageVersionHash`: Allow apiservers to expose the storage version hash in the discovery. +- `StorageVersionAPI`: Enable the + [storage version API](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#storageversion-v1alpha1-internal-apiserver-k8s-io). +- `StorageVersionHash`: Allow API servers to expose the storage version hash in the + discovery. - `StreamingProxyRedirects`: Instructs the API server to intercept (and follow) - redirects from the backend (kubelet) for streaming requests. + redirects from the backend (kubelet) for streaming requests. Examples of streaming requests include the `exec`, `attach` and `port-forward` requests. - `SupportIPVSProxyMode`: Enable providing in-cluster service load balancing using IPVS. See [service proxies](/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies) for more details. - `SupportPodPidsLimit`: Enable the support to limiting PIDs in Pods. -- `SupportNodePidsLimit`: Enable the support to limiting PIDs on the Node. The parameter `pid=` in the `--system-reserved` and `--kube-reserved` options can be specified to ensure that the specified number of process IDs will be reserved for the system as a whole and for Kubernetes system daemons respectively. -- `Sysctls`: Enable support for namespaced kernel parameters (sysctls) that can be set for each pod. - See [sysctls](/docs/tasks/administer-cluster/sysctl-cluster/) for more details. -- `TaintBasedEvictions`: Enable evicting pods from nodes based on taints on nodes and tolerations on Pods. - See [taints and tolerations](/docs/concepts/scheduling-eviction/taint-and-toleration/) for more details. -- `TaintNodesByCondition`: Enable automatic tainting nodes based on [node conditions](/docs/concepts/architecture/nodes/#condition). +- `SupportNodePidsLimit`: Enable the support to limiting PIDs on the Node. + The parameter `pid=` in the `--system-reserved` and `--kube-reserved` + options can be specified to ensure that the specified number of process IDs + will be reserved for the system as a whole and for Kubernetes system daemons + respectively. +- `Sysctls`: Enable support for namespaced kernel parameters (sysctls) that can be + set for each pod. See + [sysctls](/docs/tasks/administer-cluster/sysctl-cluster/) for more details. +- `TTLAfterFinished`: Allow a + [TTL controller](/docs/concepts/workloads/controllers/ttlafterfinished/) + to clean up resources after they finish execution. +- `TaintBasedEvictions`: Enable evicting pods from nodes based on taints on Nodes + and tolerations on Pods. + See [taints and tolerations](/docs/concepts/scheduling-eviction/taint-and-toleration/) + for more details. +- `TaintNodesByCondition`: Enable automatic tainting nodes based on + [node conditions](/docs/concepts/architecture/nodes/#condition). - `TokenRequest`: Enable the `TokenRequest` endpoint on service account resources. -- `TokenRequestProjection`: Enable the injection of service account tokens into - a Pod through the [`projected` volume](/docs/concepts/storage/volumes/#projected). -- `TopologyManager`: Enable a mechanism to coordinate fine-grained hardware resource assignments for different components in Kubernetes. See [Control Topology Management Policies on a node](/docs/tasks/administer-cluster/topology-manager/). -- `TTLAfterFinished`: Allow a [TTL controller](/docs/concepts/workloads/controllers/ttlafterfinished/) to clean up resources after they finish execution. +- `TokenRequestProjection`: Enable the injection of service account tokens into a + Pod through a [`projected` volume](/docs/concepts/storage/volumes/#projected). +- `TopologyManager`: Enable a mechanism to coordinate fine-grained hardware resource + assignments for different components in Kubernetes. See + [Control Topology Management Policies on a node](/docs/tasks/administer-cluster/topology-manager/). - `VolumePVCDataSource`: Enable support for specifying an existing PVC as a DataSource. - `VolumeScheduling`: Enable volume topology aware scheduling and make the PersistentVolumeClaim (PVC) binding aware of scheduling decisions. It also enables the usage of [`local`](/docs/concepts/storage/volumes/#local) volume type when used together with the `PersistentLocalVolumes` feature gate. - `VolumeSnapshotDataSource`: Enable volume snapshot data source support. -- `VolumeSubpathEnvExpansion`: Enable `subPathExpr` field for expanding environment variables into a `subPath`. +- `VolumeSubpathEnvExpansion`: Enable `subPathExpr` field for expanding environment + variables into a `subPath`. +- `WarningHeaders`: Allow sending warning headers in API responses. - `WatchBookmark`: Enable support for watch bookmark events. -- `WindowsGMSA`: Enables passing of GMSA credential specs from pods to container runtimes. -- `WindowsRunAsUserName` : Enable support for running applications in Windows containers with as a non-default user. - See [Configuring RunAsUserName](/docs/tasks/configure-pod-container/configure-runasusername) for more details. - `WinDSR`: Allows kube-proxy to create DSR loadbalancers for Windows. - `WinOverlay`: Allows kube-proxy to run in overlay mode for Windows. +- `WindowsGMSA`: Enables passing of GMSA credential specs from pods to container runtimes. +- `WindowsRunAsUserName` : Enable support for running applications in Windows containers + with as a non-default user. See + [Configuring RunAsUserName](/docs/tasks/configure-pod-container/configure-runasusername) + for more details. +- `WindowsEndpointSliceProxying`: When enabled, kube-proxy running on Windows + will use EndpointSlices as the primary data source instead of Endpoints, + enabling scalability and performance improvements. See + [Enabling Endpoint Slices](/docs/tasks/administer-cluster/enabling-endpointslices/). ## {{% heading "whatsnext" %}} From b10decb87caac82046da85f77cf11438a41aa542 Mon Sep 17 00:00:00 2001 From: Sergey Kanzhelev Date: Fri, 15 Jan 2021 22:16:26 +0000 Subject: [PATCH 060/190] migrating from dockershim --- .../migrating-from-dockershim/_index.md | 21 +++++ ...k-if-dockershim-deprecation-affects-you.md | 85 +++++++++++++++++++ ...migrating-telemetry-and-security-agents.md | 80 +++++++++++++++++ 3 files changed, 186 insertions(+) create mode 100755 content/en/docs/tasks/administer-cluster/migrating-from-dockershim/_index.md create mode 100644 content/en/docs/tasks/administer-cluster/migrating-from-dockershim/check-if-dockershim-deprecation-affects-you.md create mode 100644 content/en/docs/tasks/administer-cluster/migrating-from-dockershim/migrating-telemetry-and-security-agents.md diff --git a/content/en/docs/tasks/administer-cluster/migrating-from-dockershim/_index.md b/content/en/docs/tasks/administer-cluster/migrating-from-dockershim/_index.md new file mode 100755 index 0000000000..6a186e3766 --- /dev/null +++ b/content/en/docs/tasks/administer-cluster/migrating-from-dockershim/_index.md @@ -0,0 +1,21 @@ +--- +title: "Migrating from dockershim" +weight: 10 +content_type: task +--- + + + +This section presents information you need to know when migrating from +dockershim to other container runtimes. + +Since the announcement of [dockershim deprecation](/blog/2020/12/08/kubernetes-1-20-release-announcement/#dockershim-deprecation) +in Kubernetes 1.20, there were questions on how this will affect various workloads and Kubernetes +installations. You can find this blog post useful to understand the problem better: [Dockershim Deprecation FAQ](/blog/2020/12/02/dockershim-faq/) + +It is recommended to migrate from dockershim to alternative container runtimes. +Check out [container runtimes](/docs/setup/production-environment/container-runtimes/) +section to know your options. Make sure to +[report issues](https://github.com/kubernetes/kubernetes/issues) you encountered +with the migration. So the issue can be fixed in a timely manner and your cluster would be +ready for dockershim removal. diff --git a/content/en/docs/tasks/administer-cluster/migrating-from-dockershim/check-if-dockershim-deprecation-affects-you.md b/content/en/docs/tasks/administer-cluster/migrating-from-dockershim/check-if-dockershim-deprecation-affects-you.md new file mode 100644 index 0000000000..766db38485 --- /dev/null +++ b/content/en/docs/tasks/administer-cluster/migrating-from-dockershim/check-if-dockershim-deprecation-affects-you.md @@ -0,0 +1,85 @@ +--- +title: Check whether Dockershim deprecation affects you +content_type: task +reviewers: +- SergeyKanzhelev +weight: 20 +--- + + + +The `dockershim` component of Kubernetes allows to use Docker as a Kubernetes's +{{< glossary_tooltip text="container runtime" term_id="container-runtime" >}}. +Kubernetes' built-in `dockershim` component was deprecated in release v1.20. + +This page explains how your cluster could be using Docker as a container runtime, +provides details on the role that `dockershim` plays when in use, and shows steps +you can take to check whether any workloads could be affected by `dockershim` deprecation. + +## Finding if your app has a dependencies on Docker {#find-docker-dependencies} + +If you are using Docker for building your application containers, you can still +run these containers on any container runtime. This use of Docker does not count +as a dependency on Docker as a container runtime. + +When alternative container runtime is used, executing Docker commands may either +not work or yield unexpected output. This is how you can find whether you have a +dependency on Docker: + +1. Make sure no privileged Pods execute Docker commands. +2. Check that scripts and apps running on nodes outside of Kubernetes + infrastructure do not execute Docker commands. It might be: + - SSH to nodes to troubleshoot; + - Node startup scripts; + - Monitoring and security agents installed on nodes directly. +3. Third-party tools that perform above mentioned privileged operations. See + [Migrating telemetry and security agents from dockershim](/docs/tasks/administer-cluster/migrating-from-dockershim/migrating-telemetry-and-security-agents) + for more information. +4. Make sure there is no indirect dependencies on dockershim behavior. + This is an edge case and unlikely to affect your application. Some tooling may be configured + to react to Docker-specific behaviors, for example, raise alert on specific metrics or search for + a specific log message as part of troubleshooting instructions. + If you have such tooling configured, test the behavior on test + cluster before migration. + +## Dependency on Docker explained {#role-of-dockershim} + +A [container runtime](/docs/concepts/containers/#container-runtimes) is software that can +execute the containers that make up a Kubernetes pod. Kubernetes is responsible for orchestration +and scheduling of Pods; on each node, the {{< glossary_tooltip text="kubelet" term_id="kubelet" >}} +uses the container runtime interface as an abstraction so that you can use any compatible +container runtime. + +In its earliest releases, Kubernetes offered compatibility with just one container runtime: Docker. +Later in the Kubernetes project's history, cluster operators wanted to adopt additional container runtimes. +The CRI was designed to allow this kind of flexibility - and the kubelet began supporting CRI. However, +because Docker existed before the CRI specification was invented, the Kubernetes project created an +adapter component, `dockershim`. The dockershim adapter allows the kubelet to interact with Docker as +if Docker were a CRI compatible runtime. + +You can read about it in [Kubernetes Containerd integration goes GA](/blog/2018/05/24/kubernetes-containerd-integration-goes-ga/) blog post. + +![Dockershim vs. CRI with Containerd](/images/blog/2018-05-24-kubernetes-containerd-integration-goes-ga/cri-containerd.png) + +Switching to Containerd as a container runtime eliminates the middleman. All the +same containers can be run by container runtimes like Containerd as before. But +now, since containers schedule directly with the container runtime, they are not visible to Docker. +So any Docker tooling or fancy UI you might have used +before to check on these containers is no longer available. + +You cannot get container information using `docker ps` or `docker inspect` +commands. As you cannot list containers, you cannot get logs, stop containers, +or execute something inside container using `docker exec`. + +{{< note >}} + +If you're running workloads via Kubernetes, the best way to stop a container is through +the Kubernetes API rather than directly through the container runtime (this advice applies +for all container runtimes, not just Docker). + +{{< /note >}} + +You can still pull images or build them using `docker build` command. But images +built or pulled by Docker would not be visible to container runtime and +Kubernetes. They needed to be pushed to some registry to allow them to be used +by Kubernetes. diff --git a/content/en/docs/tasks/administer-cluster/migrating-from-dockershim/migrating-telemetry-and-security-agents.md b/content/en/docs/tasks/administer-cluster/migrating-from-dockershim/migrating-telemetry-and-security-agents.md new file mode 100644 index 0000000000..dd813754e4 --- /dev/null +++ b/content/en/docs/tasks/administer-cluster/migrating-from-dockershim/migrating-telemetry-and-security-agents.md @@ -0,0 +1,80 @@ +--- +title: Migrating telemetry and security agents from dockershim +content_type: task +reviewers: +- SergeyKanzhelev +weight: 70 +--- + + + +With Kubernetes 1.20 dockershim was deprecated. From the +[Dockershim Deprecation FAQ](/blog/2020/12/02/dockershim-faq/) +you might already know that most apps do not have a direct dependency on runtime hosting +containers. However, there are still a lot of telemetry and security agents +that has a dependency on docker to collect containers metadata, logs and +metrics. This document aggregates information on how to detect these +dependencies and links on how to migrate these agents to use generic tools or +alternative runtimes. + +## Telemetry and security agents + +There are a few ways agents may run on Kubernetes cluster. Agents may run on +nodes directly or as DaemonSets. + +### Why do telemetry agents rely on Docker? + +Historically, Kubernetes was built on top of Docker. Kubernetes is managing +networking and scheduling, Docker was placing and operating containers on a +node. So you can get scheduling-related metadata like a pod name from Kubernetes +and containers state information from Docker. Over time more runtimes were +created to manage containers. Also there are projects and Kubernetes features +that generalize container status information extraction across many runtimes. + +Some agents are tied specifically to the Docker tool. The agents may run +commands like [`docker ps`](https://docs.docker.com/engine/reference/commandline/ps/) +or [`docker top`](https://docs.docker.com/engine/reference/commandline/top/) to list +containers and processes or [docker logs](https://docs.docker.com/engine/reference/commandline/logs/) +to subscribe on docker logs. With the deprecating of Docker as a container runtime, +these commands will not work any longer. + +### Identify DaemonSets that depend on Docker {#identify-docker-dependency} + +If a pod wants to make calls to the `dockerd` running on the node, the pod must either: + +- mount the filesystem containing the Docker daemon's privileged socket, as a + {{< glossary_tooltip text="volume" term_id="volume" >}}; or +- mount the specific path of the Docker daemon's privileged socket directly, also as a volume. + +For example: on COS images, Docker exposes its Unix domain socket at +`/var/run/docker.sock` This means that the pod spec will include a +`hostPath` volume mount of `/var/run/docker.sock`. + +Here's a sample shell script to find Pods that have a mount directly mapping the +Docker socket. This script outputs the namespace and name of the pod. You can +remove the grep `/var/run/docker.sock` to review other mounts. + +```bash +kubectl get pods --all-namespaces \ +-o=jsonpath='{range .items[*]}{"\n"}{.metadata.namespace}{":\t"}{.metadata.name}{":\t"}{range .spec.volumes[*]}{.hostPath.path}{", "}{end}{end}' \ +| sort \ +| grep '/var/run/docker.sock' +``` + +{{< note >}} +There are alternative ways for a pod to access Docker on the host. For instance, the parent +directory `/var/run` may be mounted instead of the full path (like in [this +example](https://gist.github.com/itaysk/7bc3e56d69c4d72a549286d98fd557dd)). +The script above only detects the most common uses. +{{< /note >}} + +### Detecting Docker dependency from node agents + +In case your cluster nodes are customized and install additional security and +telemetry agents on the node, make sure to check with the vendor of the agent whether it has dependency on Docker. + +### Telemetry and security agent vendors + +We keep the work in progress version of migration instructions for various telemetry and security agent vendors +in [Google doc](https://docs.google.com/document/d/1ZFi4uKit63ga5sxEiZblfb-c23lFhvy6RXVPikS8wf0/edit#). +Please contact the vendor to get up to date instructions for migrating from dockershim. From b512f29cef4cf902efd176e7ee43e2504a34fcbb Mon Sep 17 00:00:00 2001 From: Jailton Lopes Date: Wed, 20 Jan 2021 01:15:18 -0300 Subject: [PATCH 061/190] Translate sentence: pull in Docsy submodule --- README-pt.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README-pt.md b/README-pt.md index 3d8d5fb5bc..8367b0247c 100644 --- a/README-pt.md +++ b/README-pt.md @@ -27,7 +27,7 @@ cd website O website do Kubernetes utiliza o [tema Docsy Hugo](https://github.com/google/docsy#readme). Mesmo se você planeje executar o website em um container, é altamente recomendado baixar os submódulos e outras dependências executando o seguinte comando: ``` -# pull in the Docsy submodule +# Baixar o submódulo Docsy git submodule update --init --recursive --depth 1 ``` @@ -134,4 +134,4 @@ A participação na comunidade Kubernetes é regida pelo [Código de Conduta da # Obrigado! -O Kubernetes conta com a participação da comunidade e nós realmente agradecemos suas contribuições para o nosso website e nossa documentação! \ No newline at end of file +O Kubernetes conta com a participação da comunidade e nós realmente agradecemos suas contribuições para o nosso website e nossa documentação! From aec6ea48956933a8f2d83c9957ccabf2cb7e32c3 Mon Sep 17 00:00:00 2001 From: Tom Kivlin <52716470+tomkivlin@users.noreply.github.com> Date: Wed, 20 Jan 2021 10:23:49 +0000 Subject: [PATCH 062/190] Update secret.md Remove prepended URLs and re-word to avoid use of "we". --- content/en/docs/concepts/configuration/secret.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/content/en/docs/concepts/configuration/secret.md b/content/en/docs/concepts/configuration/secret.md index e0512f8b9b..10f6389e69 100644 --- a/content/en/docs/concepts/configuration/secret.md +++ b/content/en/docs/concepts/configuration/secret.md @@ -28,10 +28,10 @@ also creates some Secrets. Kubernetes Secrets are, by default, stored as unencrypted base64-encoded strings. By default they can be retrieved - as plain text - by anyone with API access, or anyone with access to Kubernetes' underlying data store, etcd. In -order to safely use Secrets, we recommend you (at a minimum): +order to safely use Secrets, it is recommended you (at a minimum): -1. [Enable Encryption at Rest](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/) for Secrets. -2. [Enable RBAC rules that restrict reading and writing the Secret](https://kubernetes.io/docs/reference/access-authn-authz/authorization/). Be aware that secrets can be obtained implicitly by anyone with the permission to create a Pod. +1. [Enable Encryption at Rest](/docs/tasks/administer-cluster/encrypt-data/) for Secrets. +2. [Enable or configure RBAC rules](/docs/reference/access-authn-authz/authorization/) that restrict reading and writing the Secret. Be aware that secrets can be obtained implicitly by anyone with the permission to create a Pod. {{< /caution >}} From 9d3d5548a195e62bd2d118b0360e6e344fb81686 Mon Sep 17 00:00:00 2001 From: chienfuchen32 Date: Wed, 20 Jan 2021 23:24:59 +0800 Subject: [PATCH 063/190] modify Design doc ttl-after-finish upstream --- .../en/docs/concepts/workloads/controllers/ttlafterfinished.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/concepts/workloads/controllers/ttlafterfinished.md b/content/en/docs/concepts/workloads/controllers/ttlafterfinished.md index 6b6ad65e0a..65527e5ce2 100644 --- a/content/en/docs/concepts/workloads/controllers/ttlafterfinished.md +++ b/content/en/docs/concepts/workloads/controllers/ttlafterfinished.md @@ -78,5 +78,5 @@ very small. Please be aware of this risk when setting a non-zero TTL. * [Clean up Jobs automatically](/docs/concepts/workloads/controllers/job/#clean-up-finished-jobs-automatically) -* [Design doc](https://github.com/kubernetes/enhancements/blob/master/keps/sig-apps/0026-ttl-after-finish.md) +* [Design doc](https://github.com/kubernetes/enhancements/blob/master/keps/sig-apps/592-ttl-after-finish/README.md) From d56ac69e9f67251b1c8c53b6af42fb66bef69509 Mon Sep 17 00:00:00 2001 From: Jailton Lopes Date: Fri, 22 Jan 2021 23:16:11 -0300 Subject: [PATCH 064/190] Add instruction to access API reference pages URL locally --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index 44dcc7a7ca..2e88a95866 100644 --- a/README.md +++ b/README.md @@ -100,6 +100,8 @@ make container-image make container-serve ``` +Open up your browser to http://localhost:1313/docs/reference/kubernetes-api/ to view the API reference pages. + 6. When all changes of the new contract are reflected into the configuration files `toc.yaml` and `fields.yaml`, create a Pull Request with the newly generated API reference pages. ## Troubleshooting From ff442608d17870909ef392dacb7ffe767eca866c Mon Sep 17 00:00:00 2001 From: "Alexis \"Horgix\" Chotard" Date: Sat, 23 Jan 2021 17:35:27 +0100 Subject: [PATCH 065/190] Ingress: mention TLS instead of SSL in example URL --- content/fr/docs/concepts/services-networking/ingress.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/content/fr/docs/concepts/services-networking/ingress.md b/content/fr/docs/concepts/services-networking/ingress.md index 81018b3ede..1ac3078bdc 100644 --- a/content/fr/docs/concepts/services-networking/ingress.md +++ b/content/fr/docs/concepts/services-networking/ingress.md @@ -328,7 +328,7 @@ metadata: type: kubernetes.io/tls ``` -Référencer ce secret dans un Ingress indiquera au contrôleur d'ingress de sécuriser le canal du client au load-balancer à l'aide de TLS. Vous devez vous assurer que le secret TLS que vous avez créé provenait d'un certificat contenant un CN pour `sslexample.foo.com`. +Référencer ce secret dans un Ingress indiquera au contrôleur d'ingress de sécuriser le canal du client au load-balancer à l'aide de TLS. Vous devez vous assurer que le secret TLS que vous avez créé provenait d'un certificat contenant un CN pour `tlsexample.foo.com`. ```yaml apiVersion: networking.k8s.io/v1 @@ -338,10 +338,10 @@ metadata: spec: tls: - hosts: - - sslexample.foo.com + - tlsexample.foo.com secretName: testsecret-tls rules: - - host: sslexample.foo.com + - host: tlsexample.foo.com http: paths: - path: / From e4d24124a39c2d46d195c4a9087635d152e32a1e Mon Sep 17 00:00:00 2001 From: Philippe Martin Date: Sat, 23 Jan 2021 18:24:09 +0100 Subject: [PATCH 066/190] Redirects "Well-Known Labels, Annotations and Taints" page --- static/_redirects | 2 ++ 1 file changed, 2 insertions(+) diff --git a/static/_redirects b/static/_redirects index 9027eb903d..5bc1ebf38c 100644 --- a/static/_redirects +++ b/static/_redirects @@ -203,6 +203,8 @@ /docs/reference/kubernetes-api/api-index/ /docs/reference 301 +/docs/reference/kubernetes-api/labels-annotations-taints/ /docs/reference/labels-annotations-taints/ 301 + /docs/reporting-security-issues/ /security/ 301 /docs/roadmap/ https://github.com/kubernetes/kubernetes/milestones/ 301 From 3b6be2ea94f16fe36901736de360c857a60954a5 Mon Sep 17 00:00:00 2001 From: Alexis Horgix Chotard Date: Sat, 23 Jan 2021 23:14:30 +0100 Subject: [PATCH 067/190] docs-fr: make Ingress example YAML match the EN version --- .../concepts/services-networking/ingress.md | 22 +++++++++---------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/content/fr/docs/concepts/services-networking/ingress.md b/content/fr/docs/concepts/services-networking/ingress.md index 1ac3078bdc..3f29785902 100644 --- a/content/fr/docs/concepts/services-networking/ingress.md +++ b/content/fr/docs/concepts/services-networking/ingress.md @@ -338,19 +338,19 @@ metadata: spec: tls: - hosts: - - tlsexample.foo.com + - https-example.foo.com secretName: testsecret-tls rules: - - host: tlsexample.foo.com - http: - paths: - - path: / - pathType: Prefix - backend: - service: - name: service1 - port: - number: 80 + - host: https-example.foo.com + http: + paths: + - path: / + pathType: Prefix + backend: + service: + name: service1 + port: + number: 80 ``` {{< note >}} From da12b0790710e8de46efe8a1a18fe3b92dd27c0c Mon Sep 17 00:00:00 2001 From: Alexis Horgix Chotard Date: Sat, 23 Jan 2021 23:14:48 +0100 Subject: [PATCH 068/190] docs-fr: add missing capital on Ingress --- content/fr/docs/concepts/services-networking/ingress.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/fr/docs/concepts/services-networking/ingress.md b/content/fr/docs/concepts/services-networking/ingress.md index 3f29785902..858d56d2e0 100644 --- a/content/fr/docs/concepts/services-networking/ingress.md +++ b/content/fr/docs/concepts/services-networking/ingress.md @@ -328,7 +328,7 @@ metadata: type: kubernetes.io/tls ``` -Référencer ce secret dans un Ingress indiquera au contrôleur d'ingress de sécuriser le canal du client au load-balancer à l'aide de TLS. Vous devez vous assurer que le secret TLS que vous avez créé provenait d'un certificat contenant un CN pour `tlsexample.foo.com`. +Référencer ce secret dans un Ingress indiquera au contrôleur d'Ingress de sécuriser le canal du client au load-balancer à l'aide de TLS. Vous devez vous assurer que le secret TLS que vous avez créé provenait d'un certificat contenant un CN pour `tlsexample.foo.com`. ```yaml apiVersion: networking.k8s.io/v1 From 9f89f710a4c43cb2bd559caae4bcdc8c73491dd6 Mon Sep 17 00:00:00 2001 From: Alexis Horgix Chotard Date: Sat, 23 Jan 2021 23:20:13 +0100 Subject: [PATCH 069/190] docs-fr: also mention FQDN for certificate in Ingress doc --- content/fr/docs/concepts/services-networking/ingress.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/fr/docs/concepts/services-networking/ingress.md b/content/fr/docs/concepts/services-networking/ingress.md index 858d56d2e0..780f003b10 100644 --- a/content/fr/docs/concepts/services-networking/ingress.md +++ b/content/fr/docs/concepts/services-networking/ingress.md @@ -328,7 +328,7 @@ metadata: type: kubernetes.io/tls ``` -Référencer ce secret dans un Ingress indiquera au contrôleur d'Ingress de sécuriser le canal du client au load-balancer à l'aide de TLS. Vous devez vous assurer que le secret TLS que vous avez créé provenait d'un certificat contenant un CN pour `tlsexample.foo.com`. +Référencer ce secret dans un Ingress indiquera au contrôleur d'Ingress de sécuriser le canal du client au load-balancer à l'aide de TLS. Vous devez vous assurer que le secret TLS que vous avez créé provenait d'un certificat contenant un Common Name (CN), aussi appelé nom de domaine pleinement qualifié (FQDN), pour `tlsexample.foo.com`. ```yaml apiVersion: networking.k8s.io/v1 From cfeed030cc30f74e53430d2cec7fa9cf9856201c Mon Sep 17 00:00:00 2001 From: Alexis Horgix Chotard Date: Sat, 23 Jan 2021 23:20:40 +0100 Subject: [PATCH 070/190] docs-fr: make the example URL match the EN version --- content/fr/docs/concepts/services-networking/ingress.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/fr/docs/concepts/services-networking/ingress.md b/content/fr/docs/concepts/services-networking/ingress.md index 780f003b10..ec656ffd59 100644 --- a/content/fr/docs/concepts/services-networking/ingress.md +++ b/content/fr/docs/concepts/services-networking/ingress.md @@ -328,7 +328,7 @@ metadata: type: kubernetes.io/tls ``` -Référencer ce secret dans un Ingress indiquera au contrôleur d'Ingress de sécuriser le canal du client au load-balancer à l'aide de TLS. Vous devez vous assurer que le secret TLS que vous avez créé provenait d'un certificat contenant un Common Name (CN), aussi appelé nom de domaine pleinement qualifié (FQDN), pour `tlsexample.foo.com`. +Référencer ce secret dans un Ingress indiquera au contrôleur d'Ingress de sécuriser le canal du client au load-balancer à l'aide de TLS. Vous devez vous assurer que le secret TLS que vous avez créé provenait d'un certificat contenant un Common Name (CN), aussi appelé nom de domaine pleinement qualifié (FQDN), pour `https-example.foo.com`. ```yaml apiVersion: networking.k8s.io/v1 From bbc1194d5dc77305f30b599eab7cd55424b4fa81 Mon Sep 17 00:00:00 2001 From: Alexis Horgix Chotard Date: Sun, 24 Jan 2021 02:59:21 +0100 Subject: [PATCH 071/190] docs-fr: replace Ingress TLS example block by YAML file ref --- .../concepts/services-networking/ingress.md | 24 ++----------------- 1 file changed, 2 insertions(+), 22 deletions(-) diff --git a/content/fr/docs/concepts/services-networking/ingress.md b/content/fr/docs/concepts/services-networking/ingress.md index ec656ffd59..5a611a6ad7 100644 --- a/content/fr/docs/concepts/services-networking/ingress.md +++ b/content/fr/docs/concepts/services-networking/ingress.md @@ -330,28 +330,8 @@ type: kubernetes.io/tls Référencer ce secret dans un Ingress indiquera au contrôleur d'Ingress de sécuriser le canal du client au load-balancer à l'aide de TLS. Vous devez vous assurer que le secret TLS que vous avez créé provenait d'un certificat contenant un Common Name (CN), aussi appelé nom de domaine pleinement qualifié (FQDN), pour `https-example.foo.com`. -```yaml -apiVersion: networking.k8s.io/v1 -kind: Ingress -metadata: - name: tls-example-ingress -spec: - tls: - - hosts: - - https-example.foo.com - secretName: testsecret-tls - rules: - - host: https-example.foo.com - http: - paths: - - path: / - pathType: Prefix - backend: - service: - name: service1 - port: - number: 80 -``` +{{< codenew file="service/networking/tls-example-ingress.yaml" >}} + {{< note >}} Les fonctionnalités TLS prisent en charge par les différents contrôleurs peuvent être différentes. Veuillez vous référer à la documentation sur From 23465815664c773f2e67c6f4086f9b7d7a922a13 Mon Sep 17 00:00:00 2001 From: Ricardo Pchevuzinske Katz Date: Sun, 24 Jan 2021 16:27:42 -0300 Subject: [PATCH 072/190] Fix some spell checking Signed-off-by: Ricardo Pchevuzinske Katz --- content/en/docs/reference/using-api/api-concepts.md | 2 +- content/en/docs/setup/best-practices/multiple-zones.md | 2 +- .../windows/intro-windows-in-kubernetes.md | 4 ++-- .../docs/tasks/administer-cluster/dns-custom-nameservers.md | 2 +- content/en/docs/tasks/administer-cluster/topology-manager.md | 4 ++-- .../en/docs/tasks/configure-pod-container/configure-gmsa.md | 2 +- 6 files changed, 8 insertions(+), 8 deletions(-) diff --git a/content/en/docs/reference/using-api/api-concepts.md b/content/en/docs/reference/using-api/api-concepts.md index 5f8f8337d1..e8b834c334 100644 --- a/content/en/docs/reference/using-api/api-concepts.md +++ b/content/en/docs/reference/using-api/api-concepts.md @@ -514,7 +514,7 @@ The meaning of the get and list semantics are: - **Any:** Return data at any resource version. The newest available resource version is preferred, but strong consistency is not required; data at any resource version may be served. It is possible for the request to return data at a much older resource version that the client has previously - observed, particularly in high availabiliy configurations, due to partitions or stale + observed, particularly in high availability configurations, due to partitions or stale caches. Clients that cannot tolerate this should not use this semantic. - **Not older than:** Return data at least as new as the provided resourceVersion. The newest available data is preferred, but any data not older than the provided resourceVersion may be diff --git a/content/en/docs/setup/best-practices/multiple-zones.md b/content/en/docs/setup/best-practices/multiple-zones.md index fce6518eb2..107ee2d0f7 100644 --- a/content/en/docs/setup/best-practices/multiple-zones.md +++ b/content/en/docs/setup/best-practices/multiple-zones.md @@ -29,7 +29,7 @@ one zone also impairs services in another zone. ## Control plane behavior All [control plane components](/docs/concepts/overview/components/#control-plane-components) -support running as a pool of interchangable resources, replicated per +support running as a pool of interchangeable resources, replicated per component. When you deploy a cluster control plane, place replicas of diff --git a/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md b/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md index ec06e070bd..25eeb18050 100644 --- a/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md +++ b/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md @@ -273,7 +273,7 @@ The behavior of the flags behave differently as described below: * MemoryPressure Condition is not implemented * There are no OOM eviction actions taken by the kubelet * Kubelet running on the windows node does not have memory restrictions. `--kubelet-reserve` and `--system-reserve` do not set limits on kubelet or processes running on the host. This means kubelet or a process on the host could cause memory resource starvation outside the node-allocatable and scheduler -* An additional flag to set the priority of the kubelet process is availabe on the Windows nodes called `--windows-priorityclass`. This flag allows kubelet process to get more CPU time slices when compared to other processes running on the Windows host. More information on the allowable values and their meaning is available at [Windows Priority Classes](https://docs.microsoft.com/en-us/windows/win32/procthread/scheduling-priorities#priority-class). In order for kubelet to always have enough CPU cycles it is recommended to set this flag to `ABOVE_NORMAL_PRIORITY_CLASS` and above +* An additional flag to set the priority of the kubelet process is available on the Windows nodes called `--windows-priorityclass`. This flag allows kubelet process to get more CPU time slices when compared to other processes running on the Windows host. More information on the allowable values and their meaning is available at [Windows Priority Classes](https://docs.microsoft.com/en-us/windows/win32/procthread/scheduling-priorities#priority-class). In order for kubelet to always have enough CPU cycles it is recommended to set this flag to `ABOVE_NORMAL_PRIORITY_CLASS` and above #### Storage @@ -634,7 +634,7 @@ We have a lot of features in our roadmap. An abbreviated high level list is incl ### Hyper-V isolation -Hyper-V isolation is requried to enable the following use cases for Windows containers in Kubernetes: +Hyper-V isolation is required to enable the following use cases for Windows containers in Kubernetes: * Hypervisor-based isolation between pods for additional security * Backwards compatibility allowing a node to run a newer Windows Server version without requiring containers to be rebuilt diff --git a/content/en/docs/tasks/administer-cluster/dns-custom-nameservers.md b/content/en/docs/tasks/administer-cluster/dns-custom-nameservers.md index 437e58f39e..06de85b305 100644 --- a/content/en/docs/tasks/administer-cluster/dns-custom-nameservers.md +++ b/content/en/docs/tasks/administer-cluster/dns-custom-nameservers.md @@ -255,7 +255,7 @@ To migrate from kube-dns to CoreDNS, a detailed [blog article](https://coredns.io/2018/05/21/migration-from-kube-dns-to-coredns/) is available to help users adapt CoreDNS in place of kube-dns. -You can also migrate using the offical CoreDNS +You can also migrate using the official CoreDNS [deploy script](https://github.com/coredns/deployment/blob/master/kubernetes/deploy.sh). diff --git a/content/en/docs/tasks/administer-cluster/topology-manager.md b/content/en/docs/tasks/administer-cluster/topology-manager.md index 01bb78161c..7d3f017940 100644 --- a/content/en/docs/tasks/administer-cluster/topology-manager.md +++ b/content/en/docs/tasks/administer-cluster/topology-manager.md @@ -24,7 +24,7 @@ _Topology Manager_ is a Kubelet component that aims to co-ordinate the set of co -## {{% heading "prerequisites" %}} +## {{% heading "prerequisites" %}} {{< include "task-tutorial-prereqs.md" >}} {{< version-check >}} @@ -192,7 +192,7 @@ spec: This pod runs in the `Burstable` QoS class because requests are less than limits. -If the selected policy is anything other than `none`, Topology Manager would consider these Pod specifications. The Topology Manager would consult the Hint Providers to get topology hints. In the case of the `static`, the CPU Manager policy would return default topology hint, because these Pods do not have explicity request CPU resources. +If the selected policy is anything other than `none`, Topology Manager would consider these Pod specifications. The Topology Manager would consult the Hint Providers to get topology hints. In the case of the `static`, the CPU Manager policy would return default topology hint, because these Pods do not have explicitly request CPU resources. ```yaml diff --git a/content/en/docs/tasks/configure-pod-container/configure-gmsa.md b/content/en/docs/tasks/configure-pod-container/configure-gmsa.md index 391be52b23..360deada02 100644 --- a/content/en/docs/tasks/configure-pod-container/configure-gmsa.md +++ b/content/en/docs/tasks/configure-pod-container/configure-gmsa.md @@ -254,4 +254,4 @@ If the above command corrects the error, you can automate the step by adding the If you add the `lifecycle` section show above to your Pod spec, the Pod will execute the commands listed to restart the `netlogon` service until the `nltest.exe /query` command exits without error. ## GMSA limitations -When using the [ContainerD runtime for Windows](/docs/setup/production-environment/windows/intro-windows-in-kubernetes/#cri-containerd) accessing restricted network shares via the GMSA domain identity fails. The container will recieve the identity of and calls from `nltest.exe /query` will work. It is recommended to use the [Docker EE runtime](/docs/setup/production-environment/windows/intro-windows-in-kubernetes/#docker-ee) if access to network shares is required. The Windows Server team is working on resolving the issue in the Windows Kernel and will release a patch to resolve this issue in the future. Look for updates on the [Microsoft Windows Containers issue tracker](https://github.com/microsoft/Windows-Containers/issues/44). +When using the [ContainerD runtime for Windows](/docs/setup/production-environment/windows/intro-windows-in-kubernetes/#cri-containerd) accessing restricted network shares via the GMSA domain identity fails. The container will receive the identity of and calls from `nltest.exe /query` will work. It is recommended to use the [Docker EE runtime](/docs/setup/production-environment/windows/intro-windows-in-kubernetes/#docker-ee) if access to network shares is required. The Windows Server team is working on resolving the issue in the Windows Kernel and will release a patch to resolve this issue in the future. Look for updates on the [Microsoft Windows Containers issue tracker](https://github.com/microsoft/Windows-Containers/issues/44). From e239d8089f28119fa08c2c0ef4c7a33702f2ed5f Mon Sep 17 00:00:00 2001 From: ljnaresh Date: Fri, 16 Oct 2020 16:27:27 +0200 Subject: [PATCH 073/190] Replace alpine with busybox image for better response from nslookup --- .../en/docs/tasks/debug-application-cluster/debug-service.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/en/docs/tasks/debug-application-cluster/debug-service.md b/content/en/docs/tasks/debug-application-cluster/debug-service.md index 4ee9d6f490..ba141135fa 100644 --- a/content/en/docs/tasks/debug-application-cluster/debug-service.md +++ b/content/en/docs/tasks/debug-application-cluster/debug-service.md @@ -18,10 +18,10 @@ you to figure out what's going wrong. ## Running commands in a Pod For many steps here you will want to see what a Pod running in the cluster -sees. The simplest way to do this is to run an interactive alpine Pod: +sees. The simplest way to do this is to run an interactive busybox Pod: ```none -kubectl run -it --rm --restart=Never alpine --image=alpine sh +kubectl run -it --rm --restart=Never busybox --image=gcr.io/google-containers/busybox sh ``` {{< note >}} From ac1409edda4c721a42c6555bf6bb95daa2814660 Mon Sep 17 00:00:00 2001 From: Arhell Date: Mon, 25 Jan 2021 00:55:32 +0200 Subject: [PATCH 074/190] [zh] update feature-gates description for TopologyManager --- .../reference/command-line-tools-reference/feature-gates.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/content/zh/docs/reference/command-line-tools-reference/feature-gates.md b/content/zh/docs/reference/command-line-tools-reference/feature-gates.md index 35021d0af0..4a302d4793 100644 --- a/content/zh/docs/reference/command-line-tools-reference/feature-gates.md +++ b/content/zh/docs/reference/command-line-tools-reference/feature-gates.md @@ -210,7 +210,8 @@ different Kubernetes components. | `TokenRequestProjection` | `false` | Alpha | 1.11 | 1.11 | | `TokenRequestProjection` | `true` | Beta | 1.12 | | | `TTLAfterFinished` | `false` | Alpha | 1.12 | | -| `TopologyManager` | `false` | Alpha | 1.16 | | +| `TopologyManager` | `false` | Alpha | 1.16 | 1.17 | +| `TopologyManager` | `true` | Beta | 1.18 | | | `ValidateProxyRedirects` | `false` | Alpha | 1.12 | 1.13 | | `ValidateProxyRedirects` | `true` | Beta | 1.14 | | | `VolumeSnapshotDataSource` | `false` | Alpha | 1.12 | 1.16 | From 9be902bee04fe1bdacf3cd85b201c08c9fb41418 Mon Sep 17 00:00:00 2001 From: Kedar Vijay Kulkarni Date: Mon, 25 Jan 2021 11:17:10 -0500 Subject: [PATCH 075/190] added example linking to hostPath volume definition --- content/en/docs/concepts/storage/persistent-volumes.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/content/en/docs/concepts/storage/persistent-volumes.md b/content/en/docs/concepts/storage/persistent-volumes.md index cf915bfa52..ccb1470e57 100644 --- a/content/en/docs/concepts/storage/persistent-volumes.md +++ b/content/en/docs/concepts/storage/persistent-volumes.md @@ -620,6 +620,11 @@ spec: PersistentVolumes binds are exclusive, and since PersistentVolumeClaims are namespaced objects, mounting claims with "Many" modes (`ROX`, `RWX`) is only possible within one namespace. +### PersistentVolumes typed HostPath + +A hostPath PersistentVolume uses a file or directory on the Node to emulate network-attached storage. +See [an example of HostPath typed volume](/docs/tasks/configure-pod-container/configure-persistent-volume-storage/#create-a-persistentvolume). + ## Raw Block Volume Support {{< feature-state for_k8s_version="v1.18" state="stable" >}} From 8a48273855a8b221b878e490f1b0d847f9ffd441 Mon Sep 17 00:00:00 2001 From: Marc Campbell Date: Mon, 25 Jan 2021 12:57:00 -0800 Subject: [PATCH 076/190] Better description of Job execution Just a small clarification on the description of the responsibilities of a Job. Kubernetes doesn't ensure that the Pods are successful, but the scheduler's responsibility is to continue retrying until success. Updating the docs with a slight wording change to more accurately reflect that a Job won't ensure success, but it will continue retrying until success. --- content/en/docs/concepts/workloads/controllers/job.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/concepts/workloads/controllers/job.md b/content/en/docs/concepts/workloads/controllers/job.md index 14f218ddc8..2c99a704d1 100644 --- a/content/en/docs/concepts/workloads/controllers/job.md +++ b/content/en/docs/concepts/workloads/controllers/job.md @@ -13,7 +13,7 @@ weight: 50 -A Job creates one or more Pods and ensures that a specified number of them successfully terminate. +A Job creates one or more Pods and will continue to retry execution of the Pods until a specified number of them successfully terminate. As pods successfully complete, the Job tracks the successful completions. When a specified number of successful completions is reached, the task (ie, Job) is complete. Deleting a Job will clean up the Pods it created. From 74d0035e3c6a8b64117cf758727e9c36e17c7be3 Mon Sep 17 00:00:00 2001 From: Arhell Date: Tue, 26 Jan 2021 00:22:43 +0200 Subject: [PATCH 077/190] [ja] update feature-gates --- .../reference/command-line-tools-reference/feature-gates.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/content/ja/docs/reference/command-line-tools-reference/feature-gates.md b/content/ja/docs/reference/command-line-tools-reference/feature-gates.md index 2fc9f826f7..11334045fc 100644 --- a/content/ja/docs/reference/command-line-tools-reference/feature-gates.md +++ b/content/ja/docs/reference/command-line-tools-reference/feature-gates.md @@ -137,7 +137,8 @@ content_type: concept | `TokenRequestProjection` | `false` | Alpha | 1.11 | 1.11 | | `TokenRequestProjection` | `true` | Beta | 1.12 | | | `TTLAfterFinished` | `false` | Alpha | 1.12 | | -| `TopologyManager` | `false` | Alpha | 1.16 | | +| `TopologyManager` | `false` | Alpha | 1.16 | 1.17 | +| `TopologyManager` | `true` | Beta | 1.18 | | | `ValidateProxyRedirects` | `false` | Alpha | 1.12 | 1.13 | | `ValidateProxyRedirects` | `true` | Beta | 1.14 | | | `VolumePVCDataSource` | `false` | Alpha | 1.15 | 1.15 | From c06d1320f28fcc2176078002e267c44c1c29b41e Mon Sep 17 00:00:00 2001 From: ydFu Date: Mon, 25 Jan 2021 21:15:22 +0800 Subject: [PATCH 078/190] [zh] Sync concepts pages for service-networking * [zh] Umbrella issue: pages out of sync in concepts section(#26177 Misc Batch 2) ``` [x] content/zh/docs/concepts/services-networking/network-policies.md [x] content/zh/docs/concepts/services-networking/service.md [x] content/zh/docs/concepts/services-networking/dns-pod-service.md ``` Signed-off-by: ydFu ader.ydfu@gmail.com --- .../docs/concepts/services-networking/dns-pod-service.md | 9 +++------ .../concepts/services-networking/network-policies.md | 7 +++++++ content/zh/docs/concepts/services-networking/service.md | 6 +++--- 3 files changed, 13 insertions(+), 9 deletions(-) diff --git a/content/zh/docs/concepts/services-networking/dns-pod-service.md b/content/zh/docs/concepts/services-networking/dns-pod-service.md index fc23f799ae..2672d4e7d0 100644 --- a/content/zh/docs/concepts/services-networking/dns-pod-service.md +++ b/content/zh/docs/concepts/services-networking/dns-pod-service.md @@ -271,17 +271,14 @@ record unless `publishNotReadyAddresses=True` is set on the Service. ### Pod 的 setHostnameAsFQDN 字段 {#pod-sethostnameasfqdn-field} -{{< feature-state for_k8s_version="v1.19" state="alpha" >}} +{{< feature-state for_k8s_version="v1.20" state="beta" >}} **前置条件**：`SetHostnameAsFQDN` diff --git a/content/zh/docs/concepts/services-networking/network-policies.md b/content/zh/docs/concepts/services-networking/network-policies.md index 6ea89e28f4..8b38fa25d4 100644 --- a/content/zh/docs/concepts/services-networking/network-policies.md +++ b/content/zh/docs/concepts/services-networking/network-policies.md @@ -69,6 +69,8 @@ By default, pods are non-isolated; they accept traffic from any source. Pods become isolated by having a NetworkPolicy that selects them. Once there is any NetworkPolicy in a namespace selecting a particular pod, that pod will reject any connections that are not allowed by any NetworkPolicy. (Other pods in the namespace that are not selected by any NetworkPolicy will continue to accept all traffic.) Network policies do not conflict; they are additive. If any policy or policies select a pod, the pod is restricted to what is allowed by the union of those policies' ingress/egress rules. Thus, order of evaluation does not affect the policy result. + +For a network flow between two pods to be allowed, both the egress policy on the source pod and the ingress policy on the destination pod need to allow the traffic. If either the egress policy on the source, or the ingress policy on the destination denies the traffic, the traffic will be denied. --> ## 隔离和非隔离的 Pod {#isolated-and-non-isolated-pods} @@ -83,6 +85,11 @@ Pod 在被某 NetworkPolicy 选中时进入被隔离状态。 如果任何一个或多个策略选择了一个 Pod, 则该 Pod 受限于这些策略的 入站（Ingress）/出站（Egress）规则的并集。因此评估的顺序并不会影响策略的结果。 +为了允许两个 Pods 之间的网络数据流，源端 Pod 上的出站（Egress）规则和 +目标端 Pod 上的入站（Ingress）规则都需要允许该流量。 +如果源端的出站（Egress）规则或目标端的入站（Ingress）规则拒绝该流量， +则流量将被拒绝。 + ### userspace 代理模式 {#proxy-mode-userspace} -这种模式，kube-proxy 会监视 Kubernetes 主控节点对 Service 对象和 Endpoints 对象的添加和移除操作。 +这种模式，kube-proxy 会监视 Kubernetes 控制平面对 Service 对象和 Endpoints 对象的添加和移除操作。 对每个 Service，它会在本地 Node 上打开一个端口（随机选择）。 任何连接到“代理端口”的请求，都会被代理到 Service 的后端 `Pods` 中的某个上面（如 `Endpoints` 所报告的一样）。 使用哪个后端 Pod，是 kube-proxy 基于 `SessionAffinity` 来确定的。 From 29e2644f755603c679b18397277340679719eeb9 Mon Sep 17 00:00:00 2001 From: Qiming Teng Date: Tue, 26 Jan 2021 17:18:29 +0800 Subject: [PATCH 079/190] [zh] sync audit task page --- .../tasks/debug-application-cluster/audit.md | 407 +++++------------- 1 file changed, 107 insertions(+), 300 deletions(-) diff --git a/content/zh/docs/tasks/debug-application-cluster/audit.md b/content/zh/docs/tasks/debug-application-cluster/audit.md index a5f0271128..1e8f12388c 100644 --- a/content/zh/docs/tasks/debug-application-cluster/audit.md +++ b/content/zh/docs/tasks/debug-application-cluster/audit.md @@ -15,14 +15,16 @@ title: Auditing {{< feature-state state="beta" >}} -Kubernetes 审计功能提供了与安全相关的按时间顺序排列的记录集，记录每个用户、管理员 -或系统其他组件影响系统的活动顺序。 -它能帮助集群管理员处理以下问题： +Kubernetes _审计（Auditing）_ 功能提供了与安全相关的、按时间顺序排列的记录集， +记录每个用户、使用 Kubernetes API 的应用以及控制面自身引发的活动。 + +审计功能使得集群管理员能够回答以下问题： -每个请求都可以用相关的 "stage" 记录。已知的 stage 有： +每个请求都可被记录其相关的 _阶段（stage）_。已定义的阶段有： -- `RequestReceived` - 事件的 stage 将在审计处理器接收到请求后，并且在委托给其余处理器之前生成。 -- `ResponseStarted` - 在响应消息的头部发送后，但是响应消息体发送前。 - 这个阶段仅为长时间运行的请求生成（例如 watch）。 +- `RequestReceived` - 此阶段对应审计处理器接收到请求后，并且在委托给 + 其余处理器之前生成的事件。 +- `ResponseStarted` - 在响应消息的头部发送后，响应消息体发送前生成的事件。 + 只有长时间运行的请求（例如 watch）才会生成这个阶段。 - `ResponseComplete` - 当响应消息体完成并且没有更多数据需要传输的时候。 - `Panic` - 当 panic 发生时生成。 {{< note >}} @@ -96,7 +99,7 @@ Audit policy defines rules about what events should be recorded and what data they should include. The audit policy object structure is defined in the [`audit.k8s.io` API group][auditing-api]. When an event is processed, it's compared against the list of rules in order. The first matching rule sets the -"audit level" of the event. The known audit levels are: +_audit level_ of the event. The defined audit levels are: --> ## 审计策略 {#audit-policy} @@ -104,7 +107,7 @@ compared against the list of rules in order. The first matching rule sets the 审计策略对象结构定义在 [`audit.k8s.io` API 组](https://github.com/kubernetes/kubernetes/blob/{{< param "githubbranch" >}}/staging/src/k8s.io/apiserver/pkg/apis/audit/v1/types.go) 处理事件时，将按顺序与规则列表进行比较。第一个匹配规则设置事件的 -"审计级别"。已知的审计级别有： +_审计级别（Audit Level）_。已定义的审计级别有： -管理员构建自己的审计配置文件时，可参考 +如果你在打磨自己的审计配置文件，你可以使用为 Google Container-Optimized OS +设计的审计配置作为出发点。你可以参考 [configure-helper.sh](https://github.com/kubernetes/kubernetes/blob/{{< param "githubbranch" >}}/cluster/gce/gci/configure-helper.sh) -脚本，该脚本生成审计策略文件。你可以直接在脚本中看到审计策略的绝大部份内容。 -[]。 +脚本，该脚本能够生成审计策略文件。你可以直接在脚本中看到审计策略的绝大部份内容。 + ## 审计后端 {#audit-backends} 审计后端实现将审计事件导出到外部存储。`Kube-apiserver` 默认提供两个后端： -- Log 后端，将事件写入到磁盘 -- Webhook 后端，将事件发送到外部 API +- Log 后端，将事件写入到文件系统 +- Webhook 后端，将事件发送到外部 HTTP API 在这两种情况下，审计事件结构均由 `audit.k8s.io` API 组中的 API 定义。 -当前版本的 API 是 +对于 Kubernetes {{< param "fullversion" >}}，该 API 的当前版本是 [`v1`](https://github.com/kubernetes/kubernetes/blob/{{< param "githubbranch" >}}/staging/src/k8s.io/apiserver/pkg/apis/audit/v1/types.go). {{< note >}} -在 patch 请求的情况下，请求的消息体需要是一个 JSON 串指定 patch 操作， +对于 patch 请求，请求的消息体需要是设定 patch 操作的 JSON 所构成的一个串， 而不是一个完整的 Kubernetes API 对象 JSON 串。 例如，以下的示例是一个合法的 patch 请求消息体，该请求对应 `/apis/batch/v1/namespaces/some-namespace/jobs/some-job-name`。 @@ -214,13 +219,16 @@ request to `/apis/batch/v1/namespaces/some-namespace/jobs/some-job-name`. ### Log 后端 -Log 后端将审计事件写入 JSON 格式的文件。你可以使用以下 `kube-apiserver` 标志配置 -Log 审计后端： +Log 后端将审计事件写入 [JSONlines](https://jsonlines.org/) 格式的文件。 +你可以使用以下 `kube-apiserver` 标志配置 Log 审计后端： - `--audit-log-path` 指定用来写入审计事件的日志文件路径。不指定此标志会禁用日志后端。`-` 意味着标准化 -- `--audit-log-maxage` 定义了保留旧审计日志文件的最大天数 -- `--audit-log-maxbackup` 定义了要保留的审计日志文件的最大数量 +- `--audit-log-maxage` 定义保留旧审计日志文件的最大天数 +- `--audit-log-maxbackup` 定义要保留的审计日志文件的最大数量 - `--audit-log-maxsize` 定义审计日志文件的最大大小（兆字节） -如果 `kube-apiserver` 被配置为运行在 Pod 中，请记得将包含策略文件和日志文件的 -位置用 `hostPath` 挂载到 Pod 中。例如， +如果你的集群控制面以 Pod 的形式运行 kube-apiserver，记得要通过 `hostPath` +卷来访问策略文件和日志文件所在的目录，这样审计记录才会持久保存下来。例如： -``` ---audit-policy-file=/etc/kubernetes/audit-policy.yaml ---audit-log-path=/var/log/audit.log +```shell + --audit-policy-file=/etc/kubernetes/audit-policy.yaml + --audit-log-path=/var/log/audit.log ``` 接下来挂载数据卷： -``` +```yaml volumeMounts: - mountPath: /etc/kubernetes/audit-policy.yaml name: audit @@ -262,10 +266,12 @@ volumeMounts: readOnly: false ``` - -下面是 hostPath 卷本身。 + +最后配置 `hostPath`： -``` +```yaml - name: audit hostPath: path: /etc/kubernetes/audit-policy.yaml @@ -275,56 +281,59 @@ volumeMounts: hostPath: path: /var/log/audit.log type: FileOrCreate - ``` ### Webhook 后端 {#webhook-backend} -Webhook 后端将审计事件发送到远程 API，该远程 API 应该暴露与 `kube-apiserver` 相同的API。 -你可以使用如下 kube-apiserver 标志来配置 webhook 审计后端： +Webhook 后端将审计事件发送到远程 Web API，该远程 API 应该暴露与 `kube-apiserver` +形式相同的 API，包括其身份认证机制。你可以使用如下 kube-apiserver 标志来配置 +Webhook 审计后端： -- `--audit-webhook-config-file` webhook 配置文件的路径。Webhook 配置文件实际上是一个 - [kubeconfig 文件](/zh/docs/concepts/configuration/organize-cluster-access-kubeconfig/) +- `--audit-webhook-config-file` 设置 Webhook 配置文件的路径。Webhook 配置文件实际上是一个 + [kubeconfig 文件](/zh/docs/concepts/configuration/organize-cluster-access-kubeconfig/)。 - `--audit-webhook-initial-backoff` 指定在第一次失败后重发请求等待的时间。随后的请求将以指数退避重试。 -webhook 配置文件使用 kubeconfig 格式指定服务的远程地址和用于连接它的凭据。 +Webhook 配置文件使用 kubeconfig 格式指定服务的远程地址和用于连接它的凭据。 -### 批处理 {#batching} +## 事件批处理 {#batching} -log 和 webhook 后端都支持批处理。以 webhook 为例，以下是可用参数列表。要获取 log 后端的同样参数， -请在参数名称中将 `webhook` 替换为 `log`。 -默认情况下，在 `webhook` 中启用批处理，在 `log` 中禁用批处理。 +日志和 Webhook 后端都支持批处理。以 Webhook 为例，以下是可用参数列表。要获取日志 +后端的同样参数，请在参数名称中将 `webhook` 替换为 `log`。 +默认情况下，在 `webhook` 中批处理是被启用的，在 `log` 中批处理是被禁用的。 同样，默认情况下，在 `webhook` 中启用带宽限制，在 `log` 中禁用带宽限制。 + - `--audit-webhook-mode` 定义缓存策略，可选值如下： - `batch` - 以批处理缓存事件和异步的过程。这是默认值。 - `blocking` - 在 API 服务器处理每个单独事件时，阻塞其响应。 @@ -354,25 +363,26 @@ The following flags are used only in the `batch` mode. - `--audit-webhook-batch-throttle-burst` 在达到允许的 QPS 前，同一时刻允许存在的最大 batch 生成数。 -#### 参数调整 +## 参数调整 {#parameter-tuning} -需要设置参数以适应 apiserver 上的负载。 +需要设置参数以适应 API 服务器上的负载。 - -例如，如果 kube-apiserver 每秒收到 100 个请求，并且每个请求仅在 `ResponseStarted` 和 `ResponseComplete` 阶段进行审计，则应该考虑每秒生成约 200 个审计事件。 -假设批处理中最多有 100 个事件，则应将限制级别设置为至少 2 个 QPS。 -假设后端最多需要 5 秒钟来写入事件，你应该设置缓冲区大小以容纳最多 5 秒的事件，即 10 个 batch，即 1000 个事件。 +例如，如果 kube-apiserver 每秒收到 100 个请求，并且每个请求仅在 `ResponseStarted` +和 `ResponseComplete` 阶段进行审计，则应该考虑每秒生成约 200 个审计事件。 +假设批处理中最多有 100 个事件，则应将限制级别设置为每秒至少 2 个查询。 +假设后端最多需要 5 秒钟来写入事件，你应该设置缓冲区大小以容纳最多 5 秒的事件， +即 10 个 batch，即 1000 个事件。 -## 多 API 服务器的配置 +### 日志条目截断 {#truncate} -如果你通过[聚合层](/zh/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation/) -对 Kubernetes API 进行扩展，那么你也可以为聚合的 API 服务器设置审计日志。 -想要这么做，你需要以上述的格式给聚合的 API 服务器配置参数，并且配置日志管道以采用审计日志。 -不同的 API 服务器可以配置不同的审计配置和策略。 +日志后端和 Webhook 后端都支持限制所输出的事件的尺寸。 +例如，下面是可以为日志后端配置的标志列表： -## 日志收集器示例 - -### 使用 fluentd 从日志文件中选择并且分发审计日志 - -[Fluentd](https://www.fluentd.org/) 是一个开源的数据采集器，可以从统一的日志层中采集。 -在以下示例中，我们将使用 fluentd 来按照命名空间划分审计事件。 +- `audit-log-truncate-enabled`：是否弃用事件和批次的截断处理。 +- `audit-log-truncate-max-batch-size`：向下层后端发送的各批次的最大尺寸字节数。 +- `audit-log-truncate-max-event-size`：向下层后端发送的审计事件的最大尺寸字节数。 -{{< note >}} -`fluent-plugin-forest` 和 `fluent-plugin-rewrite-tag-filter` fluentd 的插件。 -你可以在 [fluentd plugin-management](https://docs.fluentd.org/v1.0/articles/plugin-management) -了解安装插件相关的细节。 -{{< /note >}} - - -1. 在 kube-apiserver 节点上安装 [`fluentd`](https://docs.fluentd.org/v1.0/articles/quickstart#step-1:-installing-fluentd)、 -, `fluent-plugin-forest` 和 `fluent-plugin-rewrite-tag-filter`。 - -1. 为 fluentd 创建一个配置文件 - - ```none - $ cat < /etc/fluentd/config - # fluentd 运行在 kube-apiserver 相同的主机上 - - @type tail - # kube-apiserver 审计日志路径 - path /var/log/audit - pos_file /var/log/audit.pos - format json - time_key time - time_format %Y-%m-%dT%H:%M:%S.%N%z - tag audit - - - - #https://github.com/fluent/fluent-plugin-rewrite-tag-filter/issues/13 - type record_transformer - enable_ruby - - namespace ${record["objectRef"].nil? ? "none":(record["objectRef"]["namespace"].nil? ? "none":record["objectRef"]["namespace"])} - - - - - # 根据上下文中的名字空间元素对审计进行路由 - @type rewrite_tag_filter - rewriterule1 namespace ^(.+) ${tag}.$1 - - - - @type record_transformer - remove_keys namespace - - - - @type forest - subtype file - remove_prefix audit - - - ``` - - -3. 启动 fluentd - - ```shell - fluentd -c /etc/fluentd/config -vv - ``` - - -4. 给 kube-apiserver 配置以下参数并启动： - - ```shell - --audit-policy-file=/etc/kubernetes/audit-policy.yaml --audit-log-path=/var/log/kube-audit --audit-log-format=json - ``` - - -5. 在 `/var/log/audit-*.log` 文件中检查不同命名空间的审计事件 - - -### 使用 logstash 采集并分发 webhook 后端的审计事件 - -[Logstash](https://www.elastic.co/products/logstash) 是一个开源的、服务器端的数据处理工具。 -在下面的示例中，我们将使用 logstash 采集 webhook 后端的审计事件，并且将来自不同用户的事件存入不同的文件。 - -1. 安装 [logstash](https://www.elastic.co/guide/en/logstash/current/installing-logstash.html) - -1. 为 logstash 创建配置文件 - - ```none - cat < /etc/logstash/config - input{ - http{ - #TODO, figure out a way to use kubeconfig file to authenticate to logstash - #https://www.elastic.co/guide/en/logstash/current/plugins-inputs-http.html#plugins-inputs-http-ssl - port=>8888 - } - } - filter{ - split{ - # Webhook 审计后端与 EventList 一起发送若干事件 - # 对事件进行分割 - field=>[items] - # 我们只需要 event 子元素，去掉其他元素 - remove_field=>[headers, metadata, apiVersion, "@timestamp", kind, "@version", host] - } - mutate{ - rename => {items=>event} - } - } - output{ - file{ - # 来自不同用户的审计事件会被保存到不同文件中 - path=>"/var/log/kube-audit-%{[event][user][username]}/audit" - } - } - ``` - - -3. 启动 logstash - - ```shell - bin/logstash -f /etc/logstash/config --path.settings /etc/logstash/ - ``` - - -4. 为 kube-apiserver webhook 审计后端创建一个 - [kubeconfig 文件](/zh/docs/tasks/access-application-cluster/configure-access-multiple-clusters/)： - - ```bash - cat < /etc/kubernetes/audit-webhook-kubeconfig - apiVersion: v1 - clusters: - - cluster: - server: http://:8888 - name: logstash - contexts: - - context: - cluster: logstash - user: "" - name: default-context - current-context: default-context - kind: Config - preferences: {} - users: [] - EOF - ``` - - -5. 为 kube-apiserver 配置以下参数并启动： - - ```shell - --audit-policy-file=/etc/kubernetes/audit-policy.yaml --audit-webhook-config-file=/etc/kubernetes/audit-webhook-kubeconfig - ``` - - -6. 在 logstash 节点的 `/var/log/kube-audit-*/audit` 目录中检查审计事件 - -请注意，除了文件输出插件外，logstash 还有其它多种输出可以让用户路由不同的数据。 -例如，用户可以将审计事件发送给支持全文搜索和分析的 elasticsearch 插件。 +默认情况下，截断操作在 `webhook` 和 `log` 后端都是被禁用的，集群管理员需要设置 +`audit-log-truncate-enabled` 或 `audit-webhook-truncate-enabled` 标志来启用此操作。 ## {{% heading "whatsnext" %}} - -了解 [Mutating webhook 审计注解](/zh/docs/reference/access-authn-authz/extensible-admission-controllers/#mutating-webhook-auditing-annotations)。 +* 了解 [Mutating webhook 审计注解](/zh/docs/reference/access-authn-authz/extensible-admission-controllers/#mutating-webhook-auditing-annotations)。 From b60b5926efb8423edb908be56c9fbb4ec51b2ba7 Mon Sep 17 00:00:00 2001 From: chenxuc Date: Tue, 26 Jan 2021 18:14:15 +0800 Subject: [PATCH 080/190] [zh] Sync web page for tasks Sync web page for part of the items in umbrella issue: Misc Batch 2 (S) content/zh/docs/tasks/debug-application-cluster/debug-pod-replication-controller.md content/zh/docs/tasks/manage-kubernetes-objects/update-api-object-kubectl-patch.md content/zh/docs/tasks/access-application-cluster/configure-access-multiple-clusters.md --- .../configure-access-multiple-clusters.md | 2 +- .../debug-pod-replication-controller.md | 2 +- .../update-api-object-kubectl-patch.md | 11 +++++++++++ 3 files changed, 13 insertions(+), 2 deletions(-) diff --git a/content/zh/docs/tasks/access-application-cluster/configure-access-multiple-clusters.md b/content/zh/docs/tasks/access-application-cluster/configure-access-multiple-clusters.md index 56fde5ff39..66c4200ea8 100644 --- a/content/zh/docs/tasks/access-application-cluster/configure-access-multiple-clusters.md +++ b/content/zh/docs/tasks/access-application-cluster/configure-access-multiple-clusters.md @@ -211,7 +211,7 @@ for the pathnames of the certificate files. You need to change these to the actu of certificate files in your environment. Sometimes you may want to use Base64-encoded data embedded here instead of separate -certificate files; in that case you need add the suffix `-data` to the keys, for example, +certificate files; in that case you need to add the suffix `-data` to the keys, for example, `certificate-authority-data`, `client-certificate-data`, `client-key-data`. --> 其中的 `fake-ca-file`、`fake-cert-file` 和 `fake-key-file` 是证书文件路径名的占位符。 diff --git a/content/zh/docs/tasks/debug-application-cluster/debug-pod-replication-controller.md b/content/zh/docs/tasks/debug-application-cluster/debug-pod-replication-controller.md index 195dabb43a..8792bf89fb 100644 --- a/content/zh/docs/tasks/debug-application-cluster/debug-pod-replication-controller.md +++ b/content/zh/docs/tasks/debug-application-cluster/debug-pod-replication-controller.md @@ -79,7 +79,7 @@ case you can try several things: * Add more nodes to the cluster. -* [Terminate unneeded pods](/docs/concepts/workloads/pods/#pod-termination) +* [Terminate unneeded pods](/docs/concepts/workloads/pods/pod-lifecycle/#pod-termination) to make room for pending pods. * Check that the pod is not larger than your nodes. For example, if all diff --git a/content/zh/docs/tasks/manage-kubernetes-objects/update-api-object-kubectl-patch.md b/content/zh/docs/tasks/manage-kubernetes-objects/update-api-object-kubectl-patch.md index 1958924fdb..44cb202930 100644 --- a/content/zh/docs/tasks/manage-kubernetes-objects/update-api-object-kubectl-patch.md +++ b/content/zh/docs/tasks/manage-kubernetes-objects/update-api-object-kubectl-patch.md @@ -657,6 +657,17 @@ and 和 [`kubectl apply`](/docs/reference/generated/kubectl/kubectl-commands/#apply)。 + + +{{< note >}} +定制资源不支持策略性合并 patch。 +{{< /note >}} + + ## {{% heading "whatsnext" %}} + +### 日志清理 + +{{< feature-state for_k8s_version="v1.20" state="alpha" >}} + +{{}} +日志清理可能会导致大量的计算开销，因此不应启用在生产环境中。 +{{< /warning >}} + + + +`--experimental-logging-sanitization` 参数可用来启用 klog 清理过滤器。 +如果启用后，将检查所有日志参数中是否有标记为敏感数据的字段（比如：密码，密钥，令牌），并且将阻止这些字段的记录。 + + + +当前支持日志清理的组件列表： + +* kube-controller-manager +* kube-apiserver +* kube-scheduler +* kubelet + +{{< note >}} +日志清理过滤器不会阻止用户工作负载日志泄漏敏感数据。 +{{< /note >}} + -* 你已拥有一个[配置好的 Kubernetes 集群](/zh/docs/setup/). -* 你已对 Kubernetes 的 _[Pods](/zh/docs/concepts/workloads/pods/)_, - _[Services](/zh/docs/concepts/services-networking/service/)_, 和 - _[Deployments](/zh/docs/concepts/workloads/controllers/deployment/)_ 有基本理解。 +* 你已拥有一个[配置好的 Kubernetes 集群](/zh/docs/setup/)。 +* 你已对 Kubernetes 的 {{< glossary_tooltip text="Pods" term_id="pod" >}} , + {{< glossary_tooltip term_id="service" text="Services" >}} , 和 + {{< glossary_tooltip text="Deployments" term_id="deployment" >}} 有基本理解。 diff --git a/content/zh/docs/tasks/administer-cluster/out-of-resource.md b/content/zh/docs/tasks/administer-cluster/out-of-resource.md index f86beff12f..33646a5e09 100644 --- a/content/zh/docs/tasks/administer-cluster/out-of-resource.md +++ b/content/zh/docs/tasks/administer-cluster/out-of-resource.md @@ -50,6 +50,7 @@ the `kubelet` summary API. | `nodefs.inodesFree` | `nodefs.inodesFree` := `node.stats.fs.inodesFree` | | `imagefs.available` | `imagefs.available` := `node.stats.runtime.imagefs.available` | | `imagefs.inodesFree` | `imagefs.inodesFree` := `node.stats.runtime.imagefs.inodesFree` | +| `pid.available` | `pid.available` := `node.stats.rlimit.maxpid` - `node.stats.rlimit.curproc` | --> | 驱逐信号 | 描述 | |----------------------------|---------------------------------------------------------------------| @@ -58,7 +59,7 @@ the `kubelet` summary API. | `nodefs.inodesFree` | `nodefs.inodesFree` := `node.stats.fs.inodesFree` | | `imagefs.available` | `imagefs.available` := `node.stats.runtime.imagefs.available` | | `imagefs.inodesFree` | `imagefs.inodesFree` := `node.stats.runtime.imagefs.inodesFree` | - +| `pid.available` | `pid.available` := `node.stats.rlimit.maxpid` - `node.stats.rlimit.curproc` | +{{< feature-state for_k8s_version="v1.18" state="stable" >}} @@ -103,57 +104,107 @@ This is the path followed by DNS Queries after NodeLocal DNSCache is enabled: ## Configuration --> ## 配置 + +{{< note >}} +NodeLocal DNSCache 的本地侦听 IP 地址可以是任何地址，只要该地址不和你的集群里现有的 IP 地址发生冲突。 +推荐使用本地范围内的地址，例如，IPv4 链路本地区段 169.254.0.0/16 内的地址， +或者 IPv6 唯一本地地址区段 fd00::/8 内的地址。 +{{< /note >}} -可以使用以下命令启用此功能： - -`KUBE_ENABLE_NODELOCAL_DNS=true go run hack/e2e.go -v --up` +可以使用以下步骤启动此功能： -这适用于在 GCE 上创建 e2e 集群。 -在所有其他环境上，以下步骤将设置 NodeLocal DNSCache ： +* 根据示例 [`nodelocaldns.yaml`](https://github.com/kubernetes/kubernetes/blob/master/cluster/addons/dns/nodelocaldns/nodelocaldns.yaml) + 准备一个清单，把它保存为 `nodelocaldns.yaml`。 + +* 如果使用 IPv6，在使用 IP:Port 格式的时候需要把 CoreDNS 配置文件里的所有 IPv6 地址用方括号包起来。 + 如果你使用上述的示例清单，需要把 [配置行 L70](https://github.com/kubernetes/kubernetes/blob/b2ecd1b3a3192fbbe2b9e348e095326f51dc43dd/cluster/addons/dns/nodelocaldns/nodelocaldns.yaml#L70) + 修改为 `health [__PILLAR__LOCAL__DNS__]:8080`。 + +* 把清单里的变量更改为正确的值： + * kubedns=`kubectl get svc kube-dns -n kube-system -o jsonpath={.spec.clusterIP}` + + * domain=`` + + * localdns=`` + + `` 的默认值是 "cluster.local"。 `` 是 NodeLocal DNSCache 选择的本地侦听 IP 地址。 -* 可以使用 `kubectl create -f` 命令应用类似于[这个](https://github.com/kubernetes/kubernetes/blob/master/cluster/addons/dns/nodelocaldns/nodelocaldns.yaml)的 Yaml 。 + * If kube-proxy is running in IPTABLES mode: + ``` bash + sed -i "s/__PILLAR__LOCAL__DNS__/$localdns/g; s/__PILLAR__DNS__DOMAIN__/$domain/g; s/__PILLAR__DNS__SERVER__/$kubedns/g" nodelocaldns.yaml + ``` + + `__PILLAR__CLUSTER__DNS__` and `__PILLAR__UPSTREAM__SERVERS__` will be populated by the node-local-dns pods. + In this mode, node-local-dns pods listen on both the kube-dns service IP as well as ``, so pods can lookup DNS records using either IP address. +--> + * 如果 kube-proxy 运行在 IPTABLES 模式： + + ``` bash + sed -i "s/__PILLAR__LOCAL__DNS__/$localdns/g; s/__PILLAR__DNS__DOMAIN__/$domain/g; s/__PILLAR__DNS__SERVER__/$kubedns/g" nodelocaldns.yaml + ``` + + node-local-dns Pods 会设置 `__PILLAR__CLUSTER__DNS__` 和 `__PILLAR__UPSTREAM__SERVERS__`。 + 在此模式下, node-local-dns Pods 会同时侦听 kube-dns 服务的 IP 地址和 `` 的地址， + 以便 Pods 可以使用其中任何一个 IP 地址来查询 DNS 记录。 -* 需要修改 kubelet 的 --cluster-dns 标志以使用 NodeLocal DNSCache 正在侦听的 LOCAL_DNS IP（默认为 169.254.20.10 ） + * 如果 kube-proxy 运行在 IPVS 模式： + + ``` bash + sed -i "s/__PILLAR__LOCAL__DNS__/$localdns/g; s/__PILLAR__DNS__DOMAIN__/$domain/g; s/__PILLAR__DNS__SERVER__//g; s/__PILLAR__CLUSTER__DNS__/$kubedns/g" nodelocaldns.yaml + ``` + + 在此模式下，node-local-dns Pods 只会侦听 `` 的地址。 + node-local-dns 接口不能绑定 kube-dns 的集群 IP 地址，因为 IPVS 负载均衡 + 使用的接口已经占用了该地址。 + node-local-dns Pods 会设置 `__PILLAR__UPSTREAM__SERVERS__`。 + + +* 运行 `kubectl create -f nodelocaldns.yaml` +* 如果 kube-proxy 运行在 IPVS 模式，需要修改 kubelet 的 `--cluster-dns` 参数为 NodeLocal DNSCache 正在侦听的 `` 地址。 + 否则，不需要修改 `--cluster-dns` 参数，因为 NodeLocal DNSCache 会同时侦听 kube-dns 服务的 IP 地址和 `` 的地址。 -启用后，node-local-dns Pods 将在每个集群节点上的 kube-system 名称空间中运行。 +启用后，node-local-dns Pods 将在每个集群节点上的 kube-system 名字空间中运行。 此 Pod 在缓存模式下运行 [CoreDNS](https://github.com/coredns/coredns) ，因此每个节点都可以使用不同插件公开的所有 CoreDNS 指标。 - -### 功能可用性 - - -可以在任何 K8s 版本中使用上面指定的 yaml 应用该插件。 -功能支持如下所述： - - -| k8s 版本 | 功能支持 | -| :---------: |:-----------:| -| 1.15 | Beta(默认情况下未启用) | -| 1.13 | Alpha(默认情况下未启用) | - - - +如果要禁用该功能，你可以使用 `kubectl delete -f ` 来删除 DaemonSet。你还应该恢复你对 kubelet 配置所做的所有改动。 From 04cbb6b47a6ffffa10684eebaa99e424e588281b Mon Sep 17 00:00:00 2001 From: chenxuc Date: Wed, 27 Jan 2021 14:56:44 +0800 Subject: [PATCH 085/190] [zh] Sync web page for config service account This is for part of the items in the following umbrella issue: [zh] Umbrella issue: pages out of sync in tasks section #26178 Service Account (L) content/zh/docs/tasks/configure-pod-container/configure-service-account.md --- .../configure-service-account.md | 55 +++++++++++++++---- 1 file changed, 44 insertions(+), 11 deletions(-) diff --git a/content/zh/docs/tasks/configure-pod-container/configure-service-account.md b/content/zh/docs/tasks/configure-pod-container/configure-service-account.md index 22cc4d2a92..4c33eee483 100644 --- a/content/zh/docs/tasks/configure-pod-container/configure-service-account.md +++ b/content/zh/docs/tasks/configure-pod-container/configure-service-account.md @@ -125,6 +125,14 @@ You can list this and any other serviceAccount resources in the namespace with t ```shell kubectl get serviceAccounts +``` + + +输出类似于： + +``` NAME SECRETS AGE default 1 1d ``` @@ -141,9 +149,15 @@ kind: ServiceAccount metadata: name: build-robot EOF -serviceaccount/build-robot created ``` + +ServiceAccount 对象的名字必须是一个有效的 +[DNS 子域名](/zh/docs/concepts/overview/working-with-objects/names#dns-subdomain-names). + @@ -151,6 +165,14 @@ If you get a complete dump of the service account object, like this: ```shell kubectl get serviceaccounts/build-robot -o yaml +``` + + +输出类似于： + +```yaml apiVersion: v1 kind: ServiceAccount metadata: @@ -229,6 +251,14 @@ Any tokens for non-existent service accounts will be cleaned up by the token con ```shell kubectl describe secrets/build-robot-secret +``` + + +输出类似于： + +``` Name: build-robot-secret Namespace: default Labels: @@ -384,23 +414,26 @@ myregistrykey --> ## 服务帐户令牌卷投射 {#service-account-token-volume-projection} -{{< feature-state for_k8s_version="v1.12" state="beta" >}} +{{< feature-state for_k8s_version="v1.20" state="stable" >}} {{< note >}} -ServiceAccountTokenVolumeProjection 在 1.12 版本中是 __beta__ 阶段， -可以通过向 API 服务器传递以下所有参数来启用它： +为了启用令牌请求投射，你必须为 `kube-apiserver` 设置以下命令行参数： * `--service-account-issuer` +* `--service-account-key-file` * `--service-account-signing-key-file` -* `--service-account-api-audiences` +* `--api-audiences` + {{< /note >}} @@ -455,7 +489,7 @@ The application is responsible for reloading the token when it rotates. Periodic --> ## 发现服务账号分发者 -{{< feature-state for_k8s_version="v1.18" state="alpha" >}} +{{< feature-state for_k8s_version="v1.20" state="beta" >}} -Creating a minimum viable Kubernetes cluster that conforms to best practices. In fact, you can use `kubeadm` to set up a cluster that will pass the [Kubernetes Conformance tests](https://kubernetes.io/blog/2017/10/software-conformance-certification). +Using `kubeadm`, you can create a minimum viable Kubernetes cluster that conforms to best practices. In fact, you can use `kubeadm` to set up a cluster that will pass the [Kubernetes Conformance tests](https://kubernetes.io/blog/2017/10/software-conformance-certification). `kubeadm` also supports other cluster lifecycle functions, such as [bootstrap tokens](/docs/reference/access-authn-authz/bootstrap-tokens/) and cluster upgrades. From 36170e41931625769de515bd40daa6f2ff121367 Mon Sep 17 00:00:00 2001 From: Brock R Date: Wed, 27 Jan 2021 14:27:43 -0700 Subject: [PATCH 088/190] Fixed a small field name spelling mistake. `resourceName` is incorrect and fails upon apply/install. `resourceNames` is the correct file name. --- .../access/certificate-signing-request/clusterrole-sign.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/examples/access/certificate-signing-request/clusterrole-sign.yaml b/content/en/examples/access/certificate-signing-request/clusterrole-sign.yaml index 29bbc6a9cd..6d1a2f7882 100644 --- a/content/en/examples/access/certificate-signing-request/clusterrole-sign.yaml +++ b/content/en/examples/access/certificate-signing-request/clusterrole-sign.yaml @@ -21,7 +21,7 @@ rules: - certificates.k8s.io resources: - signers - resourceName: + resourceNames: - example.com/my-signer-name # example.com/* can be used to authorize for all signers in the 'example.com' domain verbs: - sign From 74ecaf12ea6e6a1a069bb59ae9eef566d826554d Mon Sep 17 00:00:00 2001 From: Alexis Horgix Chotard Date: Thu, 28 Jan 2021 01:42:22 +0100 Subject: [PATCH 089/190] docs-fr: add TLS example ingress file --- .../networking/tls-example-ingress.yaml | 20 +++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100644 content/fr/examples/service/networking/tls-example-ingress.yaml diff --git a/content/fr/examples/service/networking/tls-example-ingress.yaml b/content/fr/examples/service/networking/tls-example-ingress.yaml new file mode 100644 index 0000000000..fe5d52a0cb --- /dev/null +++ b/content/fr/examples/service/networking/tls-example-ingress.yaml @@ -0,0 +1,20 @@ +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + name: tls-example-ingress +spec: + tls: + - hosts: + - https-example.foo.com + secretName: testsecret-tls + rules: + - host: https-example.foo.com + http: + paths: + - path: / + pathType: Prefix + backend: + service: + name: service1 + port: + number: 80 From 52b80e0dc17daf45e821d2fec84e057844fc10e9 Mon Sep 17 00:00:00 2001 From: chymy Date: Thu, 28 Jan 2021 10:04:36 +0800 Subject: [PATCH 090/190] Fix net-conf.json fommat err Signed-off-by: chymy --- .../tasks/administer-cluster/kubeadm/adding-windows-nodes.md | 2 +- .../tasks/administer-cluster/kubeadm/adding-windows-nodes.md | 2 +- .../tasks/administer-cluster/kubeadm/adding-windows-nodes.md | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/content/en/docs/tasks/administer-cluster/kubeadm/adding-windows-nodes.md b/content/en/docs/tasks/administer-cluster/kubeadm/adding-windows-nodes.md index 3897d6566e..2f23379400 100644 --- a/content/en/docs/tasks/administer-cluster/kubeadm/adding-windows-nodes.md +++ b/content/en/docs/tasks/administer-cluster/kubeadm/adding-windows-nodes.md @@ -72,7 +72,7 @@ Once you have a Linux-based Kubernetes control-plane node you are ready to choos "Network": "10.244.0.0/16", "Backend": { "Type": "vxlan", - "VNI" : 4096, + "VNI": 4096, "Port": 4789 } } diff --git a/content/ko/docs/tasks/administer-cluster/kubeadm/adding-windows-nodes.md b/content/ko/docs/tasks/administer-cluster/kubeadm/adding-windows-nodes.md index ce453789cb..16c84d451c 100644 --- a/content/ko/docs/tasks/administer-cluster/kubeadm/adding-windows-nodes.md +++ b/content/ko/docs/tasks/administer-cluster/kubeadm/adding-windows-nodes.md @@ -69,7 +69,7 @@ VXLAN/오버레이 네트워킹을 사용하는 경우 [KB4489899](https://suppo "Network": "10.244.0.0/16", "Backend": { "Type": "vxlan", - "VNI" : 4096, + "VNI": 4096, "Port": 4789 } } diff --git a/content/zh/docs/tasks/administer-cluster/kubeadm/adding-windows-nodes.md b/content/zh/docs/tasks/administer-cluster/kubeadm/adding-windows-nodes.md index 8cc9ec2368..1fd811f789 100644 --- a/content/zh/docs/tasks/administer-cluster/kubeadm/adding-windows-nodes.md +++ b/content/zh/docs/tasks/administer-cluster/kubeadm/adding-windows-nodes.md @@ -108,7 +108,7 @@ Once you have a Linux-based Kubernetes control-plane node you are ready to choos "Network": "10.244.0.0/16", "Backend": { "Type": "vxlan", - "VNI" : 4096, + "VNI": 4096, "Port": 4789 } } @@ -136,7 +136,7 @@ Once you have a Linux-based Kubernetes control-plane node you are ready to choos "Network": "10.244.0.0/16", "Backend": { "Type": "vxlan", - "VNI" : 4096, + "VNI": 4096, "Port": 4789 } } From 7ba4f09b2260534a6b9e334ac180980074d94379 Mon Sep 17 00:00:00 2001 From: Zhang Yong Date: Thu, 28 Jan 2021 11:17:58 +0800 Subject: [PATCH 091/190] Use https for link to contributor covenant --- content/ja/community/static/cncf-code-of-conduct.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ja/community/static/cncf-code-of-conduct.md b/content/ja/community/static/cncf-code-of-conduct.md index f61005e387..61255d92d9 100644 --- a/content/ja/community/static/cncf-code-of-conduct.md +++ b/content/ja/community/static/cncf-code-of-conduct.md @@ -25,7 +25,7 @@ CNCF コミュニティ行動規範 v1.0 Kubernetesで虐待的、嫌がらせ、または許されない行動があった場合には、から[Kubernetes Code of Conduct Committee](https://git.k8s.io/community/committee-code-of-conduct)（行動規範委員会）にご連絡ください。その他のプロジェクトにつきましては、CNCFプロジェクト管理者または仲介者にご連絡ください。 -本行動規範は、コントリビューターの合意 (http://contributor-covenant.org) バージョン 1.2.0 http://contributor-covenant.org/version/1/2/0/ から適応されています。 +本行動規範は、コントリビューターの合意 (https://contributor-covenant.org) バージョン 1.2.0 https://contributor-covenant.org/version/1/2/0/ から適応されています。 ### CNCF イベント行動規範 From bb808687d05b0ecae82867f81b9c89b8a41d3dfe Mon Sep 17 00:00:00 2001 From: Zhang Yong Date: Thu, 28 Jan 2021 11:23:13 +0800 Subject: [PATCH 092/190] Use https for link to contributor covenant --- content/fr/community/static/cncf-code-of-conduct.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/fr/community/static/cncf-code-of-conduct.md b/content/fr/community/static/cncf-code-of-conduct.md index 8d67a9c21a..93ac779477 100644 --- a/content/fr/community/static/cncf-code-of-conduct.md +++ b/content/fr/community/static/cncf-code-of-conduct.md @@ -24,7 +24,7 @@ Ce Code de conduite s’applique à la fois dans le cadre du projet et dans le c Des cas de conduite abusive, de harcèlement ou autre pratique inacceptable ayant cours sur Kubernetes peuvent être signalés en contactant le [comité pour le code de conduite de Kubernetes](https://git.k8s.io/community/committee-code-of-conduct) via l'adresse . Pour d'autres projets, bien vouloir contacter un responsable de projet CNCF ou notre médiateur, Mishi Choudhary à l'adresse . -Ce Code de conduite est inspiré du « Contributor Covenant » (http://contributor-covenant.org) version 1.2.0, disponible à l’adresse http://contributor-covenant.org/version/1/2/0/. +Ce Code de conduite est inspiré du « Contributor Covenant » (https://contributor-covenant.org) version 1.2.0, disponible à l’adresse https://contributor-covenant.org/version/1/2/0/. ### Code de conduite pour les événements de la CNCF From 2ddacf2ea2904762a72be3b619ab620ff28acd0f Mon Sep 17 00:00:00 2001 From: Zhang Yong Date: Thu, 28 Jan 2021 11:31:31 +0800 Subject: [PATCH 093/190] Use https for link to contributor covenant --- content/vi/community/static/cncf-code-of-conduct.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/vi/community/static/cncf-code-of-conduct.md b/content/vi/community/static/cncf-code-of-conduct.md index 9d7008e902..12c5472142 100644 --- a/content/vi/community/static/cncf-code-of-conduct.md +++ b/content/vi/community/static/cncf-code-of-conduct.md @@ -23,8 +23,8 @@ Quy tắc ứng xử này áp dụng cả trong không gian dự án và trong k Các trường hợp lạm dụng, quấy rối hoặc hành vi không thể chấp nhận được trong Kubernetes có thể được báo cáo bằng cách liên hệ với [Ủy ban Quy tắc ứng xử Kubernetes](https://git.k8s.io/community/committee-code-of-conduct) thông qua . Đối với các dự án khác, vui lòng liên hệ với người bảo trì dự án CNCF hoặc hòa giải viên của chúng tôi, Mishi Choudhary . -Quy tắc ứng xử này được điều chỉnh từ Giao ước cộng tác viên (http://contributor-covenant.org), phiên bản 1.2.0, có sẵn tại -http://contributor-covenant.org/version/1/2/0/ +Quy tắc ứng xử này được điều chỉnh từ Giao ước cộng tác viên (https://contributor-covenant.org), phiên bản 1.2.0, có sẵn tại +https://contributor-covenant.org/version/1/2/0/ ### Quy tắc ứng xử sự kiện CNCF From b7997b4a6d6e06149a02a6d26d9e527be8ef5ebe Mon Sep 17 00:00:00 2001 From: Zhang Yong Date: Thu, 28 Jan 2021 11:39:43 +0800 Subject: [PATCH 094/190] Use https for link to contributor covenant --- content/it/community/static/cncf-code-of-conduct.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/it/community/static/cncf-code-of-conduct.md b/content/it/community/static/cncf-code-of-conduct.md index 0622b9edab..046cc20584 100644 --- a/content/it/community/static/cncf-code-of-conduct.md +++ b/content/it/community/static/cncf-code-of-conduct.md @@ -21,7 +21,7 @@ I responsabili di progetto hanno il diritto e la responsabilità di rimuovere, m Casi di comportamenti abusivi, molestie o altri comportamenti inaccettabili in Kubernetes potranno essere denunciati contattando [Il comitato del codice di condotta Kubernetes (CNCF)](https://git.k8s.io/community/committee-code-of-conduct) attraverso . Per altri progetti, contattare il Project manager del CNCF o il nostro mediatore, Mishi Choudhary . -Il codice di condotta è stato adattato dal Contributor Covenant (http://contributor-covenant.org), versione 1.2.0, disponibile su http://contributor-covenant.org/version/1/2/0/ +Il codice di condotta è stato adattato dal Contributor Covenant (https://contributor-covenant.org), versione 1.2.0, disponibile su https://contributor-covenant.org/version/1/2/0/ ### CNCF Codice di condotta negli eventi From 146e0a8aeb51e2368fd3865887c6c2061c662126 Mon Sep 17 00:00:00 2001 From: chymy Date: Thu, 28 Jan 2021 15:13:45 +0800 Subject: [PATCH 095/190] Fix typo for zh\docs\reference\command-line-tools-reference\kubelet.md Signed-off-by: chymy --- .../zh/docs/reference/command-line-tools-reference/kubelet.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/content/zh/docs/reference/command-line-tools-reference/kubelet.md b/content/zh/docs/reference/command-line-tools-reference/kubelet.md index 32daddda68..317a3a581e 100644 --- a/content/zh/docs/reference/command-line-tools-reference/kubelet.md +++ b/content/zh/docs/reference/command-line-tools-reference/kubelet.md @@ -118,7 +118,7 @@ log to standard error as well as files Enables anonymous requests to the Kubelet server. Requests that are not rejected by another authentication method are treated as anonymous requests. Anonymous requests have a username of `system:anonymous`, and a group name of `system:unauthenticated`. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's `--config` flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.) --> 设置为 true 表示 kubelet 服务器可以接受匿名请求。未被任何认证组件拒绝的请求将被视为匿名请求。 -匿名请求的用户名为 system:anonymous，用户组为 system:unauthenticated。 +匿名请求的用户名为 system:anonymous，用户组为 system:unauthenticated。 已弃用：应在 --config 所给的配置文件中进行设置。 （进一步了解） @@ -1353,7 +1353,6 @@ Content type of requests sent to apiserver. (default "application/vnd.kubernetes 已弃用：应在 --config 所给的配置文件中进行设置。 （进一步了解） - From 59ae06054d26293de3158108c60bfbcab27c338e Mon Sep 17 00:00:00 2001 From: Arhell Date: Thu, 28 Jan 2021 10:18:32 +0200 Subject: [PATCH 096/190] [zh] fix design doc ttl-after-finish upstream link --- .../docs/concepts/workloads/controllers/ttlafterfinished.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/zh/docs/concepts/workloads/controllers/ttlafterfinished.md b/content/zh/docs/concepts/workloads/controllers/ttlafterfinished.md index 7aecf4dc00..91483c5513 100644 --- a/content/zh/docs/concepts/workloads/controllers/ttlafterfinished.md +++ b/content/zh/docs/concepts/workloads/controllers/ttlafterfinished.md @@ -136,8 +136,8 @@ very small. Please be aware of this risk when setting a non-zero TTL. * [自动清理 Job](/zh/docs/concepts/workloads/controllers/job/#clean-up-finished-jobs-automatically) -* [设计文档](https://github.com/kubernetes/enhancements/blob/master/keps/sig-apps/0026-ttl-after-finish.md) +* [设计文档](https://github.com/kubernetes/enhancements/blob/master/keps/sig-apps/592-ttl-after-finish/README.md) From 237a449ca33de1eb96f659f39d911d4e3be0fcde Mon Sep 17 00:00:00 2001 From: Ed Bartosh Date: Thu, 28 Jan 2021 10:40:12 +0200 Subject: [PATCH 097/190] get rid of running 'containerd config default' with sudo 'containerd config default' doesn't require root privileges, so it doesn't make sense to run it with sudo --- .../docs/setup/production-environment/container-runtimes.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/content/en/docs/setup/production-environment/container-runtimes.md b/content/en/docs/setup/production-environment/container-runtimes.md index 15bd4a131d..59725188d8 100644 --- a/content/en/docs/setup/production-environment/container-runtimes.md +++ b/content/en/docs/setup/production-environment/container-runtimes.md @@ -122,7 +122,7 @@ sudo apt-get update && sudo apt-get install -y containerd.io ```shell # Configure containerd sudo mkdir -p /etc/containerd -sudo containerd config default | sudo tee /etc/containerd/config.toml +containerd config default | sudo tee /etc/containerd/config.toml ``` ```shell @@ -140,7 +140,7 @@ sudo apt-get update && sudo apt-get install -y containerd ```shell # Configure containerd sudo mkdir -p /etc/containerd -sudo containerd config default | sudo tee /etc/containerd/config.toml +containerd config default | sudo tee /etc/containerd/config.toml ``` ```shell @@ -210,7 +210,7 @@ sudo yum update -y && sudo yum install -y containerd.io ```shell ## Configure containerd sudo mkdir -p /etc/containerd -sudo containerd config default | sudo tee /etc/containerd/config.toml +containerd config default | sudo tee /etc/containerd/config.toml ``` ```shell From d1707c87f8a33c75cbcb3d4739b21d44cb1e1fea Mon Sep 17 00:00:00 2001 From: Qiming Teng Date: Wed, 27 Jan 2021 08:46:28 +0800 Subject: [PATCH 098/190] [zh] resync kubeadm-upgrade --- .../kubeadm/kubeadm-upgrade.md | 615 +++++++----------- 1 file changed, 245 insertions(+), 370 deletions(-) diff --git a/content/zh/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade.md b/content/zh/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade.md index bfe410fe65..905d236d6d 100644 --- a/content/zh/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade.md +++ b/content/zh/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade.md @@ -2,7 +2,6 @@ title: 升级 kubeadm 集群 content_type: task weight: 20 -min-kubernetes-server-version: 1.19 --- -本页介绍如何将 `kubeadm` 创建的 Kubernetes 集群从 1.18.x 版本升级到 1.19.x 版本， -或者从版本 1.19.x 升级到 1.19.y ，其中 `y > x`。 +本页介绍如何将 `kubeadm` 创建的 Kubernetes 集群从 {{< skew latestVersionAddMinor -1 >}}.x 版本 +升级到 {{< skew latestVersion >}}.x 版本以及从 {{< skew latestVersion >}}.x +升级到 {{< skew latestVersion >}}.y（其中 `y > x`）。略过次版本号的升级是 +不被支持的。 @@ -57,60 +60,55 @@ The upgrade workflow at high level is the following: ## {{% heading "prerequisites" %}} -- 你需要有一个由 `kubeadm` 创建并运行着 1.18.0 或更高版本的 Kubernetes 集群。 -- [禁用交换分区](https://serverfault.com/questions/684771/best-way-to-disable-swap-in-linux)。 -- 集群应使用静态的控制平面和 etcd Pod 或者 外部 etcd。 - 务必仔细认真阅读[发行说明]({{< latest-release-notes >}})。 +- 集群应使用静态的控制平面和 etcd Pod 或者外部 etcd。 - 务必备份所有重要组件，例如存储在数据库中应用层面的状态。 `kubeadm upgrade` 不会影响你的工作负载，只会涉及 Kubernetes 内部的组件，但备份终究是好的。 +- [必须禁用交换分区](https://serverfault.com/questions/684771/best-way-to-disable-swap-in-linux)。 ### 附加信息 +- 在对 kubelet 作次版本升级时需要[腾空节点](/zh/docs/tasks/administer-cluster/safely-drain-node/)。 + 对于控制面节点，其上可能运行着 CoreDNS Pods 或者其它非常重要的负载。 - 升级后，因为容器规约的哈希值已更改，所有容器都会被重新启动。 -- 你只能从一个次版本升级到下一个次版本，或者在次版本相同时升级补丁版本。 - 也就是说，升级时不可以跳过次版本。 - 例如，你只能从 1.y 升级到 1.y+1，而不能从 from 1.y 升级到 1.y+2。 ## 确定要升级到哪个版本 -找到最新的稳定版 1.19： +使用操作系统的包管理器找到最新的稳定 {{< skew latestVersion >}}： {{< tabs name="k8s_install_versions" >}} {{% tab name="Ubuntu、Debian 或 HypriotOS" %}} ``` apt update apt-cache policy kubeadm -# 在列表中查找最新的 1.19 版本 -# 它看起来应该是 1.19.x-00 ，其中 x 是最新的补丁 +# 在列表中查找最新的 {{< skew latestVersion >}} 版本 +# 它看起来应该是 {{< skew latestVersion >}}.x-00，其中 x 是最新的补丁版本 ``` {{% /tab %}} {{% tab name="CentOS、RHEL 或 Fedora" %}} ``` yum list --showduplicates kubeadm --disableexcludes=kubernetes -# 在列表中查找最新的 1.19 版本 -# 它看起来应该是 1.19.x-0 ，其中 x 是最新的补丁版本 +# 在列表中查找最新的 {{< skew latestVersion >}} 版本 +# 它看起来应该是 {{< skew latestVersion >}}.x-0，其中 x 是最新的补丁版本 ``` {{% /tab %}} {{< /tabs >}} @@ -118,44 +116,53 @@ yum list --showduplicates kubeadm --disableexcludes=kubernetes ## 升级控制平面节点 -### 升级第一个控制面节点 +控制面节点上的升级过程应该每次处理一个节点。 +首先选择一个要先行升级的控制面节点。该节点上必须拥有 +`/etc/kubernetes/admin.conf` 文件。 + +### 执行 "kubeadm upgrade" -- 在第一个控制平面节点上，升级 kubeadm : + +**升级第一个控制面节点** + + +- 升级 kubeadm： {{< tabs name="k8s_install_kubeadm_first_cp" >}} {{% tab name="Ubuntu、Debian 或 HypriotOS" %}} ```shell -# 用最新的修补程序版本替换 1.19.x-00 中的 x +# 用最新的补丁版本号替换 {{< skew latestVersion >}}.x-00 中的 x apt-mark unhold kubeadm && \ -apt-get update && apt-get install -y kubeadm=1.19.x-00 && \ +apt-get update && apt-get install -y kubeadm={{< skew latestVersion >}}.x-00 && \ apt-mark hold kubeadm - # 从 apt-get 1.1 版本起，你也可以使用下面的方法 apt-get update && \ -apt-get install -y --allow-change-held-packages kubeadm=1.19.x-00 +apt-get install -y --allow-change-held-packages kubeadm={{< skew latestVersion >}}.x-00 ``` {{% /tab %}} {{% tab name="CentOS、RHEL 或 Fedora" %}} ```shell -# 用最新的修补程序版本替换 1.19.x-0 中的 x -yum install -y kubeadm-1.19.x-0 --disableexcludes=kubernetes +# 用最新的补丁版本号替换 {{< skew latestVersion >}}.x-0 中的 x +yum install -y kubeadm-{{< skew latestVersion >}}.x-0 --disableexcludes=kubernetes ``` {{% /tab %}} {{< /tabs >}} - 验证下载操作正常，并且 kubeadm 版本正确： @@ -164,203 +171,65 @@ yum install -y kubeadm-1.19.x-0 --disableexcludes=kubernetes ``` -- 腾空控制平面节点： +- 验证升级计划： ```shell - # 将 替换为你自己的控制面节点名称 - kubectl drain --ignore-daemonsets - ``` - - -- 在控制面节点上，运行: - - ```shell - sudo kubeadm upgrade plan - ``` - - - 你应该可以看到与下面类似的输出： - - ```none - [upgrade/config] Making sure the configuration is correct: - [upgrade/config] Reading configuration from the cluster... - [upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml' - [preflight] Running pre-flight checks. - [upgrade] Running cluster health checks - [upgrade] Fetching available versions to upgrade to - [upgrade/versions] Cluster version: v1.18.4 - [upgrade/versions] kubeadm version: v1.19.0 - [upgrade/versions] Latest stable version: v1.19.0 - [upgrade/versions] Latest version in the v1.18 series: v1.18.4 - - Components that must be upgraded manually after you have upgraded the control plane with 'kubeadm upgrade apply': - COMPONENT CURRENT AVAILABLE - Kubelet 1 x v1.18.4 v1.19.0 - - Upgrade to the latest version in the v1.18 series: - - COMPONENT CURRENT AVAILABLE - API Server v1.18.4 v1.19.0 - Controller Manager v1.18.4 v1.19.0 - Scheduler v1.18.4 v1.19.0 - Kube Proxy v1.18.4 v1.19.0 - CoreDNS 1.6.7 1.7.0 - Etcd 3.4.3-0 3.4.7-0 - - You can now apply the upgrade by executing the following command: - - kubeadm upgrade apply v1.19.0 - - _____________________________________________________________________ - - The table below shows the current state of component configs as understood by this version of kubeadm. - Configs that have a "yes" mark in the "MANUAL UPGRADE REQUIRED" column require manual config upgrade or - resetting to kubeadm defaults before a successful upgrade can be performed. The version to manually - upgrade to is denoted in the "PREFERRED VERSION" column. - - API GROUP CURRENT VERSION PREFERRED VERSION MANUAL UPGRADE REQUIRED - kubeproxy.config.k8s.io v1alpha1 v1alpha1 no - kubelet.config.k8s.io v1beta1 v1beta1 no - _____________________________________________________________________ + kubeadm upgrade plan ``` - 此命令检查你的集群是否可以升级，并可以获取到升级的版本。 - 其中也显示了组件配置版本状态的表格。 + 此命令检查你的集群是否可被升级，并取回你要升级的目标版本。 + 命令也会显示一个包含组件配置版本状态的表格。 - -{{< note >}} -`kubeadm upgrade` 也会自动对它在此节点上管理的证书进行续约。 -如果选择不对证书进行续约，可以使用标志 `--certificate-renewal=false`。 -关于更多细节信息，可参见[证书管理指南](/zh/docs/tasks/administer-cluster/kubeadm/kubeadm-certs)。 -{{}} + {{< note >}} + + `kubeadm upgrade` 也会自动对 kubeadm 在节点上所管理的证书执行续约操作。 + 如果需要略过证书续约操作，可以使用标志 `--certificate-renewal=false`。 + 更多的信息，可参阅[证书管理指南](/zh/docs/tasks/administer-cluster/kubeadm/kubeadm-certs)。 + {{}} -{{< note >}} - -如果 `kubeadm upgrade plan` 显示有任何组件配置需要手动升级，则用户必须 -通过命令行参数 `--config` 给 `kubeadm upgrade apply` 操作 -提供带有替换配置的配置文件。 -{{}} + {{< note >}} + + 如果 `kubeadm upgrade plan` 给出任何需要手动升级的组件配置，用户必须 + 通过 `--config` 命令行标志向 `kubeadm upgrade apply` 命令提供替代的配置文件。 + 如果不这样做，`kubeadm upgrade apply` 会出错并退出，不再执行升级操作。 + {{}} -- 选择要升级到的版本，然后运行相应的命令。例如: +选择要升级到的目标版本，运行合适的命令。例如： ```shell - # 将 x 替换为你为此次升级所选的补丁版本号 - sudo kubeadm upgrade apply v1.19.x + # 将 x 替换为你为此次升级所选择的补丁版本号 + sudo kubeadm upgrade apply v{{< skew latestVersion >}}.x ``` - 你应该可以看见与下面类似的输出： + 一旦该命令结束，你应该会看到： ``` - [upgrade/config] Making sure the configuration is correct: - [upgrade/config] Reading configuration from the cluster... - [upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml' - [preflight] Running pre-flight checks. - [upgrade] Running cluster health checks - [upgrade/version] You have chosen to change the cluster version to "v1.19.0" - [upgrade/versions] Cluster version: v1.18.4 - [upgrade/versions] kubeadm version: v1.19.0 - [upgrade/confirm] Are you sure you want to proceed with the upgrade? [y/N]: y - [upgrade/prepull] Pulling images required for setting up a Kubernetes cluster - [upgrade/prepull] This might take a minute or two, depending on the speed of your internet connection - [upgrade/prepull] You can also perform this action in beforehand using 'kubeadm config images pull' - [upgrade/apply] Upgrading your Static Pod-hosted control plane to version "v1.19.0"... - Static pod: kube-apiserver-kind-control-plane hash: b4c8effe84b4a70031f9a49a20c8b003 - Static pod: kube-controller-manager-kind-control-plane hash: 9ac092f0ca813f648c61c4d5fcbf39f2 - Static pod: kube-scheduler-kind-control-plane hash: 7da02f2c78da17af7c2bf1533ecf8c9a - [upgrade/etcd] Upgrading to TLS for etcd - Static pod: etcd-kind-control-plane hash: 171c56cd0e81c0db85e65d70361ceddf - [upgrade/staticpods] Preparing for "etcd" upgrade - [upgrade/staticpods] Renewing etcd-server certificate - [upgrade/staticpods] Renewing etcd-peer certificate - [upgrade/staticpods] Renewing etcd-healthcheck-client certificate - [upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/etcd.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests-2020-07-13-16-24-16/etcd.yaml" - [upgrade/staticpods] Waiting for the kubelet to restart the component - [upgrade/staticpods] This might take a minute or longer depending on the component/version gap (timeout 5m0s) - Static pod: etcd-kind-control-plane hash: 171c56cd0e81c0db85e65d70361ceddf - Static pod: etcd-kind-control-plane hash: 171c56cd0e81c0db85e65d70361ceddf - Static pod: etcd-kind-control-plane hash: 59e40b2aab1cd7055e64450b5ee438f0 - [apiclient] Found 1 Pods for label selector component=etcd - [upgrade/staticpods] Component "etcd" upgraded successfully! - [upgrade/etcd] Waiting for etcd to become available - [upgrade/staticpods] Writing new Static Pod manifests to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests999800980" - [upgrade/staticpods] Preparing for "kube-apiserver" upgrade - [upgrade/staticpods] Renewing apiserver certificate - [upgrade/staticpods] Renewing apiserver-kubelet-client certificate - [upgrade/staticpods] Renewing front-proxy-client certificate - [upgrade/staticpods] Renewing apiserver-etcd-client certificate - [upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-apiserver.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests-2020-07-13-16-24-16/kube-apiserver.yaml" - [upgrade/staticpods] Waiting for the kubelet to restart the component - [upgrade/staticpods] This might take a minute or longer depending on the component/version gap (timeout 5m0s) - Static pod: kube-apiserver-kind-control-plane hash: b4c8effe84b4a70031f9a49a20c8b003 - Static pod: kube-apiserver-kind-control-plane hash: b4c8effe84b4a70031f9a49a20c8b003 - Static pod: kube-apiserver-kind-control-plane hash: b4c8effe84b4a70031f9a49a20c8b003 - Static pod: kube-apiserver-kind-control-plane hash: b4c8effe84b4a70031f9a49a20c8b003 - Static pod: kube-apiserver-kind-control-plane hash: f717874150ba572f020dcd89db8480fc - [apiclient] Found 1 Pods for label selector component=kube-apiserver - [upgrade/staticpods] Component "kube-apiserver" upgraded successfully! - [upgrade/staticpods] Preparing for "kube-controller-manager" upgrade - [upgrade/staticpods] Renewing controller-manager.conf certificate - [upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-controller-manager.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests-2020-07-13-16-24-16/kube-controller-manager.yaml" - [upgrade/staticpods] Waiting for the kubelet to restart the component - [upgrade/staticpods] This might take a minute or longer depending on the component/version gap (timeout 5m0s) - Static pod: kube-controller-manager-kind-control-plane hash: 9ac092f0ca813f648c61c4d5fcbf39f2 - Static pod: kube-controller-manager-kind-control-plane hash: b155b63c70e798b806e64a866e297dd0 - [apiclient] Found 1 Pods for label selector component=kube-controller-manager - [upgrade/staticpods] Component "kube-controller-manager" upgraded successfully! - [upgrade/staticpods] Preparing for "kube-scheduler" upgrade - [upgrade/staticpods] Renewing scheduler.conf certificate - [upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-scheduler.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests-2020-07-13-16-24-16/kube-scheduler.yaml" - [upgrade/staticpods] Waiting for the kubelet to restart the component - [upgrade/staticpods] This might take a minute or longer depending on the component/version gap (timeout 5m0s) - Static pod: kube-scheduler-kind-control-plane hash: 7da02f2c78da17af7c2bf1533ecf8c9a - Static pod: kube-scheduler-kind-control-plane hash: 260018ac854dbf1c9fe82493e88aec31 - [apiclient] Found 1 Pods for label selector component=kube-scheduler - [upgrade/staticpods] Component "kube-scheduler" upgraded successfully! - [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace - [kubelet] Creating a ConfigMap "kubelet-config-1.19" in namespace kube-system with the configuration for the kubelets in the cluster - [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" - [bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to get nodes - [bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials - [bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token - [bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster - W0713 16:26:14.074656 2986 dns.go:282] the CoreDNS Configuration will not be migrated due to unsupported version of CoreDNS. The existing CoreDNS Corefile configuration and deployment has been retained. - [addons] Applied essential addon: CoreDNS - [addons] Applied essential addon: kube-proxy - - [upgrade/successful] SUCCESS! Your cluster was upgraded to "v1.19.0". Enjoy! + [upgrade/successful] SUCCESS! Your cluster was upgraded to "v{{< skew latestVersion >}}.x". Enjoy! [upgrade/kubelet] Now that your control plane is upgraded, please proceed with upgrading your kubelets if you haven't already done so. ``` @@ -368,96 +237,127 @@ Failing to do so will cause `kubeadm upgrade apply` to exit with an error and no - 手动升级你的 CNI 驱动插件。 你的容器网络接口（CNI）驱动应该提供了程序自身的升级说明。 - 参阅[插件](/zh/docs/concepts/cluster-administration/addons/)页面查找你 CNI 所提供的程序， + 参阅[插件](/zh/docs/concepts/cluster-administration/addons/)页面查找你的 CNI 驱动， 并查看是否需要其他升级步骤。 - 如果 CNI 提供程序作为 DaemonSet 运行，则在其他控制平面节点上不需要此步骤。 + 如果 CNI 驱动作为 DaemonSet 运行，则在其他控制平面节点上不需要此步骤。 -- 取消对控制面节点的保护 - - ```shell - # 将 替换为你的控制面节点名称 - kubectl uncordon - ``` +**对于其它控制面节点** -### 升级其他控制面节点 - -与第一个控制面节点类似，不过使用下面的命令： +与第一个控制面节点相同，但是使用： ``` sudo kubeadm upgrade node ``` - + 而不是： ``` sudo kubeadm upgrade apply ``` - -同时，也不需要执行 `sudo kubeadm upgrade plan`。 + +此外，不需要执行 `kubeadm upgrade plan` 和更新 CNI 驱动插件的操作。 + + +### 腾空节点 + +- 通过将节点标记为不可调度并腾空节点为节点作升级准备： + + ```shell + # 将 替换为你要腾空的控制面节点名称 + kubectl drain --ignore-daemonsets + ``` ### 升级 kubelet 和 kubectl -{{< tabs name="k8s_install_kubelet" >}} -{{% tab name="Ubuntu、Debian 或 HypriotOS" %}} -```shell -# 用最新的补丁版本替换 1.19.x-00 中的 x -apt-mark unhold kubelet kubectl && \ -apt-get update && apt-get install -y kubelet=1.19.x-00 kubectl=1.19.x-00 && \ -apt-mark hold kubelet kubectl +- 升级 kubelet 和 kubectl -# 从 apt-get 的 1.1 版本开始，你也可以使用下面的方法： + {{< tabs name="k8s_install_kubelet" >}} + {{% tab name="Ubuntu、Debian 或 HypriotOS" %}} -apt-get update && \ -apt-get install -y --allow-change-held-packages kubelet=1.19.x-00 kubectl=1.19.x-00 -``` -{{% /tab %}} -{{% tab name="CentOS、RHEL 或 Fedora" %}} - -用最新的补丁版本替换 1.19.x-00 中的 x - -```shell -yum install -y kubelet-1.19.x-0 kubectl-1.19.x-0 --disableexcludes=kubernetes -``` -{{% /tab %}} -{{< /tabs >}} + 
+  # 用最新的补丁版本替换 {{< skew latestVersion >}}.x-00 中的 x
+  apt-mark unhold kubelet kubectl && \
+  apt-get update && apt-get install -y kubelet={{< skew latestVersion >}}.x-00 kubectl={{< skew latestVersion >}}.x-00 && \
+  apt-mark hold kubelet kubectl
+  - 
+  # 从 apt-get 的 1.1 版本开始，你也可以使用下面的方法：
+  apt-get update && \
+  apt-get install -y --allow-change-held-packages kubelet={{< skew latestVersion >}}.x-00 kubectl={{< skew latestVersion >}}.x-00
+
+ {{% /tab %}} + {{% tab name="CentOS、RHEL 或 Fedora" %}} + + 
 
+  # 用最新的补丁版本号替换 {{< skew latestVersion >}}.x-00 中的 x
+  yum install -y kubelet-{{< skew latestVersion >}}.x-0 kubectl-{{< skew latestVersion >}}.x-0 --disableexcludes=kubernetes
+
+ {{% /tab %}} + {{< /tabs >}} -重启 kubelet +- 重启 kubelet -```shell -sudo systemctl daemon-reload -sudo systemctl restart kubelet -``` + ```shell + sudo systemctl daemon-reload + sudo systemctl restart kubelet + ``` + + +### 解除节点的保护 + +- 通过将节点标记为可调度，让其重新上线： + + ```shell + # 将 替换为你的节点名称 + kubectl uncordon + ``` -- 在所有工作节点升级 kubeadm: - -{{< tabs name="k8s_install_kubeadm_worker_nodes" >}} -{{% tab name="Ubuntu、Debian 或 HypriotOS" %}} - -```shell -# 将 1.19.x-00 中的 x 替换为最新的补丁版本 -apt-mark unhold kubeadm && \ -apt-get update && apt-get install -y kubeadm=1.19.x-00 && \ -apt-mark hold kubeadm - -# 从 apt-get 的 1.1 版本开始，你也可以使用下面的方法： - -apt-get update && \ -apt-get install -y --allow-change-held-packages kubeadm=1.19.x-00 -``` - -{{% /tab %}} -{{% tab name="CentOS、RHEL 或 Fedora" %}} - -```shell -# 用最新的补丁版本替换 1.19.x-00 中的 x -yum install -y kubeadm-1.19.x-0 --disableexcludes=kubernetes -``` -{{% /tab %}} -{{< /tabs >}} - - -### 腾空节点 - - -- 通过将节点标记为不可调度并逐出工作负载，为维护做好准备。运行： +- 升级 kubeadm： + {{< tabs name="k8s_install_kubeadm_worker_nodes" >}} + {{% tab name="Ubuntu、Debian 或 HypriotOS" %}} + ```shell - # 将 替换为你正在腾空的节点的名称 - kubectl drain --ignore-daemonsets + # 将 {{< skew latestVersion >}}.x-00 中的 x 替换为最新的补丁版本号 + apt-mark unhold kubeadm && \ + apt-get update && apt-get install -y kubeadm={{< skew latestVersion >}}.x-00 && \ + apt-mark hold kubeadm + - + # 从 apt-get 的 1.1 版本开始，你也可以使用下面的方法： + apt-get update && \ + apt-get install -y --allow-change-held-packages kubeadm={{< skew latestVersion >}}.x-00 ``` - - - 你应该可以看见与下面类似的输出： - + {{% /tab %}} + {{% tab name="CentOS、RHEL 或 Fedora" %}} + ```shell - node/ip-172-31-85-18 cordoned - WARNING: ignoring DaemonSet-managed Pods: kube-system/kube-proxy-dj7d7, kube-system/weave-net-z65qx - node/ip-172-31-85-18 drained + # 用最新的补丁版本替换 {{< skew latestVersion >}}.x-00 中的 x + yum install -y kubeadm-{{< skew latestVersion >}}.x-0 --disableexcludes=kubernetes ``` + {{% /tab %}} + {{< /tabs >}} -### 升级 kubelet 配置 +### Call "kubeadm upgrade" - -- 升级 kubelet 配置: +### 执行 "kubeadm upgrade" + +- 对于工作节点，下面的命令会升级本地的 kubelet 配置： ```shell sudo kubeadm upgrade node ``` -### 升级 kubelet 与 kubectl +### 腾空节点 + +- 将节点标记为不可调度并驱逐所有负载，准备节点的维护： + + ```shell + # 将 替换为你正在腾空的节点的名称 + kubectl drain --ignore-daemonsets + ``` -- 在所有工作节点上升级 kubelet 和 kubectl： +### 升级 kubelet 和 kubectl -{{< tabs name="k8s_kubelet_and_kubectl" >}} -{{% tab name="Ubuntu、Debian 或 HypriotOS" %}} + +- 升级 kubelet 和 kubectl： -```shell -# 将 1.19.x-00 中的 x 替换为最新的补丁版本 -apt-mark unhold kubelet kubectl && \ -apt-get update && apt-get install -y kubelet=1.19.x-00 kubectl=1.19.x-00 && \ -apt-mark hold kubelet kubectl - -# 从 apt-get 的 1.1 版本开始，你也可以使用下面的方法： - -apt-get update && \ -apt-get install -y --allow-change-held-packages kubelet=1.19.x-00 kubectl=1.19.x-00 -``` - -{{% /tab %}} -{{% tab name="CentOS, RHEL or Fedora" %}} - -```shell -# 将 1.18.x-00 中的 x 替换为最新的补丁版本 -yum install -y kubelet-1.19.x-0 kubectl-1.19.x-0 --disableexcludes=kubernetes -``` - -{{% /tab %}} -{{< /tabs >}} + {{< tabs name="k8s_kubelet_and_kubectl" >}} + {{% tab name="Ubuntu、Debian 或 HypriotOS" %}} + + ```shell + # 将 {{< skew latestVersion >}}.x-00 中的 x 替换为最新的补丁版本 + apt-mark unhold kubelet kubectl && \ + apt-get update && apt-get install -y kubelet={{< skew latestVersion >}}.x-00 kubectl={{< skew latestVersion >}}.x-00 && \ + apt-mark hold kubelet kubectl + + # 从 apt-get 的 1.1 版本开始，你也可以使用下面的方法： + + apt-get update && \ + apt-get install -y --allow-change-held-packages kubelet={{< skew latestVersion >}}.x-00 kubectl={{< skew latestVersion >}}.x-00 + ``` + + {{% /tab %}} + {{% tab name="CentOS, RHEL or Fedora" %}} + + ```shell + # 将 {{< skew latestVersion >}}.x-0 x 替换为最新的补丁版本 + yum install -y kubelet-{{< skew latestVersion >}}.x-0 kubectl-{{< skew latestVersion >}}.x-0 --disableexcludes=kubernetes + ``` + {{% /tab %}} + {{< /tabs >}} ## 验证集群的状态 -在所有节点上升级 kubelet 后，通过从 kubectl 可以访问集群的任何位置运行以下命令，验证所有节点是否再次可用： +在所有节点上升级 kubelet 后，通过从 kubectl 可以访问集群的任何位置运行以下命令， +验证所有节点是否再次可用： ```shell kubectl get nodes @@ -659,13 +534,14 @@ The `STATUS` column should show `Ready` for all your nodes, and the version numb If `kubeadm upgrade` fails and does not roll back, for example because of an unexpected shutdown during execution, you can run `kubeadm upgrade` again. This command is idempotent and eventually makes sure that the actual state is the desired state you declare. -To recover from a bad state, you can also run `kubeadm upgrade --force` without changing the version that your cluster is running. +To recover from a bad state, you can also run `kubeadm upgrade--force` without changing the version that your cluster is running. --> ## 从故障状态恢复 -如果 `kubeadm upgrade` 失败并且没有回滚，例如由于执行期间意外关闭，你可以再次运行 `kubeadm upgrade`。 -此命令是幂等的，并最终确保实际状态是你声明的所需状态。 -要从故障状态恢复，你还可以运行 `kubeadm upgrade --force` 而不去更改集群正在运行的版本。 +如果 `kubeadm upgrade` 失败并且没有回滚，例如由于执行期间节点意外关闭， +你可以再次运行 `kubeadm upgrade`。 +此命令是幂等的，并最终确保实际状态是你声明的期望状态。 +要从故障状态恢复，你还可以运行 `kubeadm upgrade --force` 而无需更改集群正在运行的版本。 `kubeadm upgrade node` 在工作节点上完成以下工作： - 从集群取回 kubeadm `ClusterConfiguration`。 -- 为本节点升级 kubelet 配置 +- 为本节点升级 kubelet 配置。 From 05da4dd6691868ba438797b490c1b72c7570b2dd Mon Sep 17 00:00:00 2001 From: seokho-son Date: Mon, 18 Jan 2021 15:46:35 +0900 Subject: [PATCH 099/190] Ko: Third Korean l10n work for release-1.20 - Update outdated files in the dev-1.20-ko.3 (1) (#26131) - Update outdated files in the dev-1.20-ko.3 branch (2) (#26122) Co-authored-by: seokho-son Co-authored-by: Jerry Park --- ...-12-02-dont-panic-kubernetes-and-docker.md | 12 +-- .../ko/docs/concepts/architecture/nodes.md | 3 +- .../cluster-administration/certificates.md | 10 +-- .../docs/concepts/configuration/overview.md | 4 + .../ko/docs/concepts/configuration/secret.md | 2 +- .../docs/concepts/containers/runtime-class.md | 13 +-- .../compute-storage-net/device-plugins.md | 3 +- .../compute-storage-net/network-plugins.md | 2 +- .../scheduling-eviction/assign-pod-node.md | 30 ++++--- .../scheduling-eviction/pod-overhead.md | 4 +- .../resource-bin-packing.md | 9 +- .../ingress-controllers.md | 3 +- .../services-networking/network-policies.md | 6 ++ .../concepts/services-networking/service.md | 18 ++-- .../concepts/storage/persistent-volumes.md | 14 ++- .../storage/volume-snapshot-classes.md | 9 +- .../controllers/garbage-collection.md | 9 +- .../workloads/controllers/replicaset.md | 6 +- .../controllers/replicationcontroller.md | 8 +- .../ko/docs/concepts/workloads/pods/_index.md | 19 +++-- .../concepts/workloads/pods/disruptions.md | 6 +- .../workloads/pods/init-containers.md | 7 +- .../concepts/workloads/pods/pod-lifecycle.md | 12 ++- .../feature-gates.md | 3 +- .../ko/docs/reference/kubectl/cheatsheet.md | 5 ++ .../ko/docs/reference/kubectl/conventions.md | 39 +++++---- .../kubectl/docker-cli-to-kubectl.md | 8 +- .../container-runtimes.md | 83 ++++++++++++++++-- .../connecting-frontend-backend.md | 85 +++++++++++-------- .../kustomization.md | 9 +- .../deploy-app/deploy-intro.html | 4 +- .../ko/examples/application/deployment.yaml | 2 +- .../guestbook/frontend-deployment.yaml | 2 +- .../guestbook/redis-master-deployment.yaml | 2 +- .../guestbook/redis-slave-deployment.yaml | 2 +- .../application/mysql/mysql-deployment.yaml | 2 +- .../wordpress/mysql-deployment.yaml | 2 +- .../wordpress/wordpress-deployment.yaml | 2 +- content/ko/examples/service/access/Dockerfile | 2 +- .../{hello.yaml => backend-deployment.yaml} | 0 ...ello-service.yaml => backend-service.yaml} | 0 .../service/access/frontend-deployment.yaml | 27 ++++++ .../service/access/frontend-nginx.conf | 14 +++ .../service/access/frontend-service.yaml | 15 ++++ .../ko/examples/service/access/frontend.conf | 11 --- .../ko/examples/service/access/frontend.yaml | 39 --------- 46 files changed, 354 insertions(+), 213 deletions(-) rename content/ko/examples/service/access/{hello.yaml => backend-deployment.yaml} (100%) rename content/ko/examples/service/access/{hello-service.yaml => backend-service.yaml} (100%) create mode 100644 content/ko/examples/service/access/frontend-deployment.yaml create mode 100644 content/ko/examples/service/access/frontend-nginx.conf create mode 100644 content/ko/examples/service/access/frontend-service.yaml delete mode 100644 content/ko/examples/service/access/frontend.conf delete mode 100644 content/ko/examples/service/access/frontend.yaml diff --git a/content/ko/blog/_posts/2020-12-02-dont-panic-kubernetes-and-docker.md b/content/ko/blog/_posts/2020-12-02-dont-panic-kubernetes-and-docker.md index de7d44ecad..b7550a3d83 100644 --- a/content/ko/blog/_posts/2020-12-02-dont-panic-kubernetes-and-docker.md +++ b/content/ko/blog/_posts/2020-12-02-dont-panic-kubernetes-and-docker.md @@ -70,7 +70,7 @@ containerd가 정말 필요로 하는 것들을 확보하기 위해서 도커심 컨테이너 런타임을 도커에서 지원되는 다른 컨테이너 런타임으로 변경하기만 하면 됩니다. 참고할 사항 한 가지: 현재 클러스터 내 워크플로의 일부가 기본 도커 소켓 -(/var/run/docker.sock)에 의존하고 있는 경우, 다른 +(`/var/run/docker.sock`)에 의존하고 있는 경우, 다른 런타임으로 전환하는 것은 해당 워크플로의 사용에 문제를 일으킵니다. 이 패턴은 종종 도커 내의 도커라고 합니다. 이런 특정 유스케이스에 대해서 [kaniko](https://github.com/GoogleContainerTools/kaniko), @@ -82,11 +82,11 @@ containerd가 정말 필요로 하는 것들을 확보하기 위해서 도커심 이 변경 사항은 사람들(folks)이 보통 도커와 상호 작용하는 데 사용하는 것과는 다른 환경을 제시합니다. 개발에 사용하는 도커의 설치는 쿠버네티스 클러스터 내의 -도커 런타임과 관련이 없습니다. 혼란스럽죠, 우리도 알고 있습니다. 개발자에게 -도커는 이 변경 사항이 발표되기 전과 마찬가지로 여전히 +도커 런타임과 관련이 없습니다. 혼란스럽죠, 우리도 알고 있습니다. +개발자에게 도커는 이 변경 사항이 발표되기 전과 마찬가지로 여전히 유용합니다. 도커가 생성하는 이미지는 실제로는 도커에만 특정된 이미지가 아니라 OCI([Open Container Initiative](https://opencontainers.org/)) 이미지입니다. -모든 OCI 호환 이미지는 해당 이미지를 빌드하는 데 사용하는 도구에 관계없이 +모든 OCI 호환 이미지는 해당 이미지를 빌드하는 데 사용하는 도구에 관계없이 쿠버네티스에서 동일하게 보입니다. [containerd](https://containerd.io/)와 [CRI-O](https://cri-o.io/)는 모두 해당 이미지를 가져와 실행하는 방법을 알고 있습니다. 이것이 컨테이너가 어떤 모습이어야 하는지에 대한 표준이 있는 이유입니다. @@ -98,7 +98,7 @@ containerd가 정말 필요로 하는 것들을 확보하기 위해서 도커심 혼란스럽더라도 괜찮습니다. 이에 대해서 많은 일이 진행되고 있습니다. 쿠버네티스에는 변화되는 부분이 많이 있고, 이에 대해 100% 전문가는 없습니다. 경험 수준이나 복잡성에 관계없이 어떤 질문이든 하시기 바랍니다! 우리의 목표는 -모든 사람이 다가오는 변경 사항에 대해 최대한 많은 교육을 받을 수 있도록 하는 것입니다. `<3` 이 글이 -여러분이 가지는 대부분의 질문에 대한 답이 되었고, 불안을 약간은 진정시켰기를 바랍니다! +모든 사람이 다가오는 변경 사항에 대해 최대한 많은 교육을 받을 수 있도록 하는 것입니다. 이 글이 +여러분이 가지는 대부분의 질문에 대한 답이 되었고, 불안을 약간은 진정시켰기를 바랍니다! ❤️ 더 많은 답변을 찾고 계신가요? 함께 제공되는 [도커심 사용 중단 FAQ](/blog/2020/12/02/dockershim-faq/)를 확인하세요. diff --git a/content/ko/docs/concepts/architecture/nodes.md b/content/ko/docs/concepts/architecture/nodes.md index 18133b6577..6872096814 100644 --- a/content/ko/docs/concepts/architecture/nodes.md +++ b/content/ko/docs/concepts/architecture/nodes.md @@ -239,7 +239,7 @@ NodeStatus의 NodeReady 컨디션을 ConditionUnknown으로 업데이트 하는 쿠버네티스 노드에서 보내는 하트비트는 노드의 가용성을 결정하는데 도움이 된다. 하트비트의 두 가지 형태는 `NodeStatus` 와 -[리스(Lease) 오브젝트](/docs/reference/generated/kubernetes-api/{{< latest-version >}}/#lease-v1-coordination-k8s-io) 이다. +[리스(Lease) 오브젝트](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#lease-v1-coordination-k8s-io)이다. 각 노드에는 `kube-node-lease` 라는 {{< glossary_tooltip term_id="namespace" text="네임스페이스">}} 에 관련된 리스 오브젝트가 있다. 리스는 경량 리소스로, 클러스터가 확장될 때 @@ -355,4 +355,3 @@ Kubelet은 노드가 종료되는 동안 파드가 일반 [파드 종료 프로 * 아키텍처 디자인 문서의 [노드](https://git.k8s.io/community/contributors/design-proposals/architecture/architecture.md#the-kubernetes-node) 섹션을 읽어본다. * [테인트와 톨러레이션](/ko/docs/concepts/scheduling-eviction/taint-and-toleration/)을 읽어본다. - diff --git a/content/ko/docs/concepts/cluster-administration/certificates.md b/content/ko/docs/concepts/cluster-administration/certificates.md index d7051e4145..7b71b9c344 100644 --- a/content/ko/docs/concepts/cluster-administration/certificates.md +++ b/content/ko/docs/concepts/cluster-administration/certificates.md @@ -34,7 +34,7 @@ weight: 20 이름을 설정한다. `MASTER_CLUSTER_IP` 는 일반적으로 API 서버와 컨트롤러 관리자 컴포넌트에 대해 `--service-cluster-ip-range` 인수로 지정된 서비스 CIDR의 첫 번째 IP이다. `--days` 인수는 인증서가 만료되는 - 일 수를 설정하는데 사용된다. + 일 수를 설정하는데 사용된다. 또한, 아래 샘플은 기본 DNS 이름으로 `cluster.local` 을 사용한다고 가정한다. @@ -130,11 +130,11 @@ weight: 20 사용 중인 하드웨어 아키텍처 및 cfssl 버전에 따라 샘플 명령을 조정해야 할 수도 있다. - curl -L https://github.com/cloudflare/cfssl/releases/download/v1.4.1/cfssl_1.4.1_linux_amd64 -o cfssl + curl -L https://github.com/cloudflare/cfssl/releases/download/v1.5.0/cfssl_1.5.0_linux_amd64 -o cfssl chmod +x cfssl - curl -L https://github.com/cloudflare/cfssl/releases/download/v1.4.1/cfssljson_1.4.1_linux_amd64 -o cfssljson + curl -L https://github.com/cloudflare/cfssl/releases/download/v1.5.0/cfssljson_1.5.0_linux_amd64 -o cfssljson chmod +x cfssljson - curl -L https://github.com/cloudflare/cfssl/releases/download/v1.4.1/cfssl-certinfo_1.4.1_linux_amd64 -o cfssl-certinfo + curl -L https://github.com/cloudflare/cfssl/releases/download/v1.5.0/cfssl-certinfo_1.5.0_linux_amd64 -o cfssl-certinfo chmod +x cfssl-certinfo 1. 아티팩트(artifact)를 보유할 디렉터리를 생성하고 cfssl을 초기화한다. @@ -248,5 +248,3 @@ done. `certificates.k8s.io` API를 사용해서 [여기](/docs/tasks/tls/managing-tls-in-a-cluster)에 설명된 대로 인증에 사용할 x509 인증서를 프로비전 할 수 있다. - - diff --git a/content/ko/docs/concepts/configuration/overview.md b/content/ko/docs/concepts/configuration/overview.md index 5e2047b367..6bcec1d6c0 100644 --- a/content/ko/docs/concepts/configuration/overview.md +++ b/content/ko/docs/concepts/configuration/overview.md @@ -1,4 +1,6 @@ --- + + title: 구성 모범 사례 content_type: concept weight: 10 @@ -67,6 +69,8 @@ DNS 서버는 새로운 `서비스`를 위한 쿠버네티스 API를 Watch하며 오브젝트의 의도한 상태는 디플로이먼트에 의해 기술되며, 만약 그 스펙에 대한 변화가 _적용될_ 경우, 디플로이먼트 컨트롤러는 일정한 비율로 실제 상태를 의도한 상태로 변화시킨다. +- 일반적인 활용 사례인 경우 [쿠버네티스 공통 레이블](/ko/docs/concepts/overview/working-with-objects/common-labels/)을 사용한다. 이 표준화된 레이블은 `kubectl` 및 [대시보드](/ko/docs/tasks/access-application-cluster/web-ui-dashboard)와 같은 도구들이 상호 운용이 가능한 방식으로 동작할 수 있도록 메타데이터를 향상시킨다. + - 디버깅을 위해 레이블을 조작할 수 있다. (레플리카셋과 같은) 쿠버네티스 컨트롤러와 서비스는 셀렉터 레이블을 사용해 파드를 선택하기 때문에, 관련된 레이블을 파드에서 삭제하는 것은 컨트롤러로부터 관리되거나 서비스로부터 트래픽을 전달받는 것을 중단시킨다. 만약 이미 존재하는 파드의 레이블을 삭제한다면, 파드의 컨트롤러는 그 자리를 대신할 새로운 파드를 생성한다. 이것은 이전에 "살아 있는" 파드를 "격리된" 환경에서 디버그할 수 있는 유용한 방법이다. 레이블을 상호적으로 추가하고 삭제하기 위해서, [`kubectl label`](/docs/reference/generated/kubectl/kubectl-commands#label)를 사용할 수 있다. ## 컨테이너 이미지 diff --git a/content/ko/docs/concepts/configuration/secret.md b/content/ko/docs/concepts/configuration/secret.md index f120b0002d..e5466b8dec 100644 --- a/content/ko/docs/concepts/configuration/secret.md +++ b/content/ko/docs/concepts/configuration/secret.md @@ -767,7 +767,7 @@ immutable: true `imagePullSecrets` 필드는 동일한 네임스페이스의 시크릿에 대한 참조 목록이다. `imagePullSecretsDocker` 를 사용하여 도커(또는 다른 컨테이너) 이미지 레지스트리 비밀번호가 포함된 시크릿을 kubelet에 전달할 수 있다. kubelet은 이 정보를 사용해서 파드를 대신하여 프라이빗 이미지를 가져온다. -`imagePullSecrets` 필드에 대한 자세한 정보는 [PodSpec API](/docs/reference/generated/kubernetes-api/{{< latest-version >}}/#podspec-v1-core)를 참고한다. +`imagePullSecrets` 필드에 대한 자세한 정보는 [PodSpec API](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podspec-v1-core)를 참고한다. #### imagePullSecret 수동으로 지정하기 diff --git a/content/ko/docs/concepts/containers/runtime-class.md b/content/ko/docs/concepts/containers/runtime-class.md index 674f47b32a..b31cabe888 100644 --- a/content/ko/docs/concepts/containers/runtime-class.md +++ b/content/ko/docs/concepts/containers/runtime-class.md @@ -1,4 +1,7 @@ --- + + + title: 런타임클래스(RuntimeClass) content_type: concept weight: 20 @@ -34,10 +37,10 @@ weight: 20 런타임클래스 기능 게이트가 활성화(기본값)된 것을 확인한다. 기능 게이트 활성화에 대한 설명은 [기능 게이트](/ko/docs/reference/command-line-tools-reference/feature-gates/)를 -참고한다. `RuntimeClass` 기능 게이트는 apiservers _및_ kubelets에서 활성화되어야 한다. +참고한다. `RuntimeClass` 기능 게이트는 API 서버 _및_ kubelets에서 활성화되어야 한다. -1. CRI 구현(implementation)을 노드에 설정(런타임에 따라서) -2. 상응하는 런타임클래스 리소스 생성 +1. CRI 구현(implementation)을 노드에 설정(런타임에 따라서). +2. 상응하는 런타임클래스 리소스 생성. ### 1. CRI 구현을 노드에 설정 @@ -48,7 +51,7 @@ weight: 20 {{< note >}} 런타임클래스는 기본적으로 클러스터 전체에 걸쳐 동질의 노드 설정 (모든 노드가 컨테이너 런타임에 준하는 동일한 방식으로 설정되었음을 의미)을 가정한다. -이종의(heterogenous) 노드 설정을 지원하기 위해서는, 아래 [스케줄](#스케줄)을 참고한다. +이종의(heterogeneous) 노드 설정을 지원하기 위해서는, 아래 [스케줄](#스케줄)을 참고한다. {{< /note >}} 해당 설정은 상응하는 `handler` 이름을 가지며, 이는 런타임클래스에 의해서 참조된다. @@ -95,7 +98,7 @@ spec: # ... ``` -이것은 Kubelet이 지명된 런타임클래스를 사용하여 해당 파드를 실행하도록 지시할 것이다. +이것은 kubelet이 지명된 런타임클래스를 사용하여 해당 파드를 실행하도록 지시할 것이다. 만약 지명된 런타임클래스가 없거나, CRI가 상응하는 핸들러를 실행할 수 없는 경우, 파드는 `Failed` 터미널 [단계](/ko/docs/concepts/workloads/pods/pod-lifecycle/#파드의-단계-phase)로 들어간다. 에러 메시지에 상응하는 [이벤트](/docs/tasks/debug-application-cluster/debug-application-introspection/)를 diff --git a/content/ko/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins.md b/content/ko/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins.md index 56230e5911..5f58604cd7 100644 --- a/content/ko/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins.md +++ b/content/ko/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins.md @@ -1,4 +1,5 @@ --- + title: 장치 플러그인 description: GPU, NIC, FPGA, InfiniBand 및 공급 업체별 설정이 필요한 유사한 리소스를 위한 플러그인을 구현하는데 쿠버네티스 장치 플러그인 프레임워크를 사용한다. content_type: concept @@ -199,7 +200,7 @@ gRPC 서비스는 `/var/lib/kubelet/pod-resources/kubelet.sock` 의 유닉스 장치 플러그인 리소스에 대한 모니터링 에이전트는 데몬 또는 데몬셋으로 배포할 수 있다. 표준 디렉터리 `/var/lib/kubelet/pod-resources` 에는 특권을 가진 접근이 필요하므로, 모니터링 에이전트는 특권을 가진 ​​보안 컨텍스트에서 실행해야 한다. 장치 모니터링 에이전트가 -데몬셋으로 실행 중인 경우, 플러그인의 [PodSpec](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podspec-v1-core)에서 +데몬셋으로 실행 중인 경우, 해당 장치 모니터링 에이전트의 [PodSpec](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podspec-v1-core)에서 `/var/lib/kubelet/pod-resources` 를 {{< glossary_tooltip text="볼륨" term_id="volume" >}}으로 마운트해야 한다. diff --git a/content/ko/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins.md b/content/ko/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins.md index 76c2bdd003..7542ac0add 100644 --- a/content/ko/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins.md +++ b/content/ko/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins.md @@ -155,5 +155,5 @@ AWS에서 `eth0` MTU는 일반적으로 9001이므로, `--network-plugin-mtu=900 ## 용법 요약 * `--network-plugin=cni` 는 `--cni-bin-dir`(기본값 `/opt/cni/bin`)에 있는 실제 CNI 플러그인 바이너리와 `--cni-conf-dir`(기본값 `/etc/cni/net.d`)에 있는 CNI 플러그인 구성과 함께 `cni` 네트워크 플러그인을 사용하도록 지정한다. -* `--network-plugin=kubenet` 은 `/opt/cni/bin` 또는 `cni-bin-dir` 에 있는 CNI `bridge` 및 `host-local` 플러그인과 함께 kubenet 네트워크 플러그인을 사용하도록 지정한다. +* `--network-plugin=kubenet` 은 `/opt/cni/bin` 또는 `cni-bin-dir` 에 있는 CNI `bridge`, `lo` 및 `host-local` 플러그인과 함께 `kubenet` 네트워크 플러그인을 사용하도록 지정한다. * 현재 kubenet 네트워크 플러그인에서만 사용하는 `--network-plugin-mtu=9001` 은 사용할 MTU를 지정한다. diff --git a/content/ko/docs/concepts/scheduling-eviction/assign-pod-node.md b/content/ko/docs/concepts/scheduling-eviction/assign-pod-node.md index edbe7daf39..ebbc00f91e 100644 --- a/content/ko/docs/concepts/scheduling-eviction/assign-pod-node.md +++ b/content/ko/docs/concepts/scheduling-eviction/assign-pod-node.md @@ -1,4 +1,8 @@ --- + + + + title: 노드에 파드 할당하기 content_type: concept weight: 50 @@ -62,7 +66,7 @@ spec: {{< codenew file="pods/pod-nginx.yaml" >}} 그런 다음에 `kubectl apply -f https://k8s.io/examples/pods/pod-nginx.yaml` 을 -실행하면, 레이블이 붙여진 노드에 파드가 스케줄 된다. +실행하면, 레이블이 붙여진 노드에 파드가 스케줄된다. `kubectl get pods -o wide` 를 실행해서 파드가 할당된 "NODE" 를 보면 작동하는지 검증할 수 있다. @@ -100,7 +104,7 @@ spec: 1. 어피니티/안티-어피니티 언어가 더 표현적이다. 언어는 논리 연산자인 AND 연산으로 작성된 정확한 매칭 항목 이외에 더 많은 매칭 규칙을 제공한다. 2. 규칙이 엄격한 요구 사항이 아니라 "유연한(soft)"/"선호(preference)" 규칙을 나타낼 수 있기에 스케줄러가 규칙을 만족할 수 없더라도, - 파드가 계속 스케줄 되도록 한다. + 파드가 계속 스케줄되도록 한다. 3. 노드 자체에 레이블을 붙이기보다는 노드(또는 다른 토폴로지 도메인)에서 실행 중인 다른 파드의 레이블을 제한할 수 있다. 이를 통해 어떤 파드가 함께 위치할 수 있는지와 없는지에 대한 규칙을 적용할 수 있다. @@ -115,7 +119,7 @@ spec: 스케줄할 수 있는 노드를 제한할 수 있다. 여기에 현재 `requiredDuringSchedulingIgnoredDuringExecution` 와 `preferredDuringSchedulingIgnoredDuringExecution` 로 부르는 -두 가지 종류의 노드 어피니티가 있다. 전자는 파드가 노드에 스케줄 되도록 *반드시* +두 가지 종류의 노드 어피니티가 있다. 전자는 파드가 노드에 스케줄되도록 *반드시* 규칙을 만족해야 하는 것(`nodeSelector` 와 같으나 보다 표현적인 구문을 사용해서)을 지정하고, 후자는 스케줄러가 시도하려고는 하지만, 보증하지 않는 *선호(preferences)* 를 지정한다는 점에서 이를 각각 "엄격함(hard)" 과 "유연함(soft)" 으로 생각할 수 있다. @@ -143,14 +147,14 @@ spec: `NotIn` 과 `DoesNotExist` 를 사용해서 안티-어피니티를 수행하거나, 특정 노드에서 파드를 쫓아내는 [노드 테인트(taint)](/ko/docs/concepts/scheduling-eviction/taint-and-toleration/)를 설정할 수 있다. -`nodeSelector` 와 `nodeAffinity` 를 모두 지정한다면 파드가 후보 노드에 스케줄 되기 위해서는 +`nodeSelector` 와 `nodeAffinity` 를 모두 지정한다면 파드가 후보 노드에 스케줄되기 위해서는 *둘 다* 반드시 만족해야 한다. -`nodeAffinity` 유형과 연관된 `nodeSelectorTerms` 를 지정하면, 파드는 `nodeSelectorTerms` 를 **모두** 만족하는 노드에만 스케줄할 수 있다. +`nodeAffinity` 유형과 연관된 `nodeSelectorTerms` 를 지정하면, `nodeSelectorTerms` 중 **하나라도** 만족시키는 노드에 파드가 스케줄된다. -`nodeSelectorTerms` 와 연관된 여러 `matchExpressions` 를 지정하면, 파드는 `matchExpressions` 이 지정된 것 중 **한 가지**라도 만족하는 노드에만 스케줄할 수 있다. +`nodeSelectorTerms` 와 연관된 여러 `matchExpressions` 를 지정하면, 파드는 `matchExpressions` 를 **모두** 만족하는 노드에만 스케줄된다. -파드가 스케줄 된 노드의 레이블을 지우거나 변경해도 파드는 제거되지 않는다. 다시 말해서 어피니티 선택은 파드를 스케줄링 하는 시점에만 작동한다. +파드가 스케줄된 노드의 레이블을 지우거나 변경해도 파드는 제거되지 않는다. 다시 말해서 어피니티 선택은 파드를 스케줄링 하는 시점에만 작동한다. `preferredDuringSchedulingIgnoredDuringExecution` 의 `weight` 필드의 범위는 1-100이다. 모든 스케줄링 요구 사항 (리소스 요청, RequiredDuringScheduling 어피니티 표현식 등)을 만족하는 각 노드들에 대해 스케줄러는 이 필드의 요소들을 반복해서 합계를 계산하고 노드가 MatchExpressions 에 일치하는 경우 합계에 "가중치(weight)"를 추가한다. 이후에 이 점수는 노드에 대한 다른 우선순위 함수의 점수와 합쳐진다. 전체 점수가 가장 높은 노드를 가장 선호한다. @@ -158,9 +162,9 @@ spec: {{< feature-state for_k8s_version="v1.20" state="beta" >}} -여러 [스케줄링 프로파일](/ko/docs/reference/scheduling/config/#여러-프로파일)을 구성할 때 +여러 [스케줄링 프로파일](/ko/docs/reference/scheduling/config/#여러-프로파일)을 구성할 때 노드 어피니티가 있는 프로파일을 연결할 수 있는데, 이는 프로파일이 특정 노드 집합에만 적용되는 경우 유용하다. -이렇게 하려면 [스케줄러 구성](/ko/docs/reference/scheduling/config/)에 있는 +이렇게 하려면 [스케줄러 구성](/ko/docs/reference/scheduling/config/)에 있는 [`NodeAffinity` 플러그인](/ko/docs/reference/scheduling/config/#스케줄링-플러그인-1)의 인수에 `addedAffinity`를 추가한다. 예를 들면 ```yaml @@ -183,15 +187,15 @@ profiles: - foo ``` -`addedAffinity`는 `.spec.schedulerName`을 `foo-scheduler`로 설정하는 모든 파드에 적용되며 +`addedAffinity`는 `.spec.schedulerName`을 `foo-scheduler`로 설정하는 모든 파드에 적용되며 PodSpec에 지정된 NodeAffinity도 적용된다. 즉, 파드를 매칭시키려면, 노드가 `addedAffinity`와 파드의 `.spec.NodeAffinity`를 충족해야 한다. -`addedAffinity`는 엔드 유저에게 표시되지 않으므로, 예상치 못한 동작이 일어날 수 있다. 프로파일의 +`addedAffinity`는 엔드 유저에게 표시되지 않으므로, 예상치 못한 동작이 일어날 수 있다. 프로파일의 스케줄러 이름과 명확한 상관 관계가 있는 노드 레이블을 사용하는 것이 좋다. {{< note >}} -[데몬셋용 파드를 생성](/ko/docs/concepts/workloads/controllers/daemonset/#기본-스케줄러로-스케줄)하는 데몬셋 컨트롤러는 +[데몬셋용 파드를 생성](/ko/docs/concepts/workloads/controllers/daemonset/#기본-스케줄러로-스케줄)하는 데몬셋 컨트롤러는 스케줄링 프로파일을 인식하지 못한다. 따라서 `addedAffinity`없이 `default-scheduler`와 같은 스케줄러 프로파일을 유지하는 것이 좋다. 그런 다음 데몬셋의 파드 템플릿이 스케줄러 이름을 사용해야 한다. 그렇지 않으면, 데몬셋 컨트롤러에 의해 생성된 일부 파드가 스케줄되지 않은 상태로 유지될 수 있다. @@ -429,5 +433,3 @@ spec: 파드가 노드에 할당되면 kubelet은 파드를 실행하고 노드의 로컬 리소스를 할당한다. [토폴로지 매니저](/docs/tasks/administer-cluster/topology-manager/)는 노드 수준의 리소스 할당 결정에 참여할 수 있다. - - diff --git a/content/ko/docs/concepts/scheduling-eviction/pod-overhead.md b/content/ko/docs/concepts/scheduling-eviction/pod-overhead.md index 29ab845af2..0f474338d9 100644 --- a/content/ko/docs/concepts/scheduling-eviction/pod-overhead.md +++ b/content/ko/docs/concepts/scheduling-eviction/pod-overhead.md @@ -26,7 +26,7 @@ _파드 오버헤드_ 는 컨테이너 리소스 요청과 상한 위에서 파 이 수행될 때 지정된다. 파드 오버헤드가 활성화 되면, 파드를 노드에 스케줄링 할 때 컨테이너 리소스 요청의 합에 -파드의 오버헤드를 추가해서 스케줄링을 고려한다. 마찬가지로, Kubelet은 파드의 cgroups 크기를 변경하거나 +파드의 오버헤드를 추가해서 스케줄링을 고려한다. 마찬가지로, kubelet은 파드의 cgroups 크기를 변경하거나 파드의 축출 등급을 부여할 때에도 파드의 오버헤드를 포함하여 고려한다. ## 파드 오버헤드 활성화하기 {#set-up} @@ -190,4 +190,4 @@ sudo crictl inspectp -o=json $POD_ID | grep cgroupsPath * [런타임클래스](/ko/docs/concepts/containers/runtime-class/) -* [파드오버헤드 디자인](https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/20190226-pod-overhead.md) +* [파드오버헤드 디자인](https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/688-pod-overhead) diff --git a/content/ko/docs/concepts/scheduling-eviction/resource-bin-packing.md b/content/ko/docs/concepts/scheduling-eviction/resource-bin-packing.md index 124f009020..f23396d936 100644 --- a/content/ko/docs/concepts/scheduling-eviction/resource-bin-packing.md +++ b/content/ko/docs/concepts/scheduling-eviction/resource-bin-packing.md @@ -1,4 +1,8 @@ --- + + + + title: 확장된 리소스를 위한 리소스 빈 패킹(bin packing) content_type: concept weight: 50 @@ -8,8 +12,9 @@ weight: 50 {{< feature-state for_k8s_version="v1.16" state="alpha" >}} -kube-scheduler는 `RequestedToCapacityRatioResourceAllocation` 우선 순위 기능을 사용해서 -확장된 리소스와 함께 리소스의 빈 패킹이 가능하도록 구성할 수 있다. 우선 순위 기능을 사용해서 맞춤 요구에 따라 +kube-scheduler는 `RequestedToCapacityRatioResourceAllocation` +우선 순위 기능을 사용해서 확장된 리소스와 함께 리소스의 빈 패킹이 가능하도록 +구성할 수 있다. 우선 순위 기능을 사용해서 맞춤 요구에 따라 kube-scheduler를 미세 조정할 수 있다. diff --git a/content/ko/docs/concepts/services-networking/ingress-controllers.md b/content/ko/docs/concepts/services-networking/ingress-controllers.md index 7690669757..3af939488e 100644 --- a/content/ko/docs/concepts/services-networking/ingress-controllers.md +++ b/content/ko/docs/concepts/services-networking/ingress-controllers.md @@ -17,7 +17,6 @@ kube-controller-manager 바이너리의 일부로 실행되는 컨트롤러의 [nginx](https://git.k8s.io/ingress-nginx/README.md#readme) 인그레스 컨트롤러를 지원하고 유지한다. - ## 추가 컨트롤러 @@ -27,6 +26,7 @@ kube-controller-manager 바이너리의 일부로 실행되는 컨트롤러의 * [AKS 애플리케이션 게이트웨이 인그레스 컨트롤러] (https://azure.github.io/application-gateway-kubernetes-ingress/)는 [Azure 애플리케이션 게이트웨이](https://docs.microsoft.com)를 구성하는 인그레스 컨트롤러다. * [Ambassador](https://www.getambassador.io/) API 게이트웨이는 [Envoy](https://www.envoyproxy.io) 기반 인그레스 컨트롤러다. +* [Avi 쿠버네티스 오퍼레이터](https://github.com/vmware/load-balancer-and-ingress-services-for-kubernetes)는 [VMware NSX Advanced Load Balancer](https://avinetworks.com/)을 사용하는 L4-L7 로드 밸런싱을 제공한다. * [Citrix 인그레스 컨트롤러](https://github.com/citrix/citrix-k8s-ingress-controller#readme)는 Citrix 애플리케이션 딜리버리 컨트롤러에서 작동한다. * [Contour](https://projectcontour.io/)는 [Envoy](https://www.envoyproxy.io/) 기반 인그레스 컨트롤러다. @@ -73,4 +73,3 @@ kube-controller-manager 바이너리의 일부로 실행되는 컨트롤러의 * [인그레스](/ko/docs/concepts/services-networking/ingress/)에 대해 자세히 알아보기. * [NGINX 컨트롤러로 Minikube에서 인그레스를 설정하기](/docs/tasks/access-application-cluster/ingress-minikube). - diff --git a/content/ko/docs/concepts/services-networking/network-policies.md b/content/ko/docs/concepts/services-networking/network-policies.md index be145f78c9..d7872b1d92 100644 --- a/content/ko/docs/concepts/services-networking/network-policies.md +++ b/content/ko/docs/concepts/services-networking/network-policies.md @@ -1,4 +1,8 @@ --- + + + + title: 네트워크 정책 content_type: concept weight: 50 @@ -31,6 +35,8 @@ pod- 또는 namespace- 기반의 네트워크폴리시를 정의할 때, {{< glo 네트워크 정책은 충돌하지 않으며, 추가된다. 만약 어떤 정책 또는 정책들이 파드를 선택하면, 해당 정책의 인그레스(수신)/이그레스(송신) 규칙을 통합하여 허용되는 범위로 파드가 제한된다. 따라서 평가 순서는 정책 결과에 영향을 미치지 않는다. +두 파드간 네트워크 흐름을 허용하기 위해서는, 소스 파드의 이그레스 정책과 목적지 파드의 인그레스 정책 모두가 해당 트래픽을 허용해야 한다. 만약 소스의 이그레스 정책이나 목적지의 인그레스 정책 중 한쪽이라도 트래픽을 거절하게 되어 있다면, 해당 트래픽은 거절될 것이다. + ## 네트워크폴리시 리소스 {#networkpolicy-resource} 리소스에 대한 전체 정의에 대한 참조는 [네트워크폴리시](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#networkpolicy-v1-networking-k8s-io) 를 본다. diff --git a/content/ko/docs/concepts/services-networking/service.md b/content/ko/docs/concepts/services-networking/service.md index 3270c498aa..da9d353d6f 100644 --- a/content/ko/docs/concepts/services-networking/service.md +++ b/content/ko/docs/concepts/services-networking/service.md @@ -1,4 +1,6 @@ --- + + title: 서비스 feature: title: 서비스 디스커버리와 로드 밸런싱 @@ -204,10 +206,10 @@ API 리소스이다. 개념적으로 엔드포인트와 매우 유사하지만, {{< feature-state for_k8s_version="v1.20" state="stable" >}} `appProtocol` 필드는 각 서비스 포트에 대한 애플리케이션 프로토콜을 지정하는 방법을 제공한다. -이 필드의 값은 해당 엔드포인트와 엔드포인트슬라이스 +이 필드의 값은 해당 엔드포인트와 엔드포인트슬라이스 오브젝트에 의해 미러링된다. -이 필드는 표준 쿠버네티스 레이블 구문을 따른다. 값은 +이 필드는 표준 쿠버네티스 레이블 구문을 따른다. 값은 [IANA 표준 서비스 이름](http://www.iana.org/assignments/service-names) 또는 `mycompany.com/my-custom-protocol`과 같은 도메인 접두사 이름 중 하나여야 한다. @@ -236,7 +238,7 @@ DNS 레코드를 구성하고, 라운드-로빈 이름 확인 방식을 ### 유저 스페이스(User space) 프록시 모드 {#proxy-mode-userspace} -이 모드에서는, kube-proxy는 쿠버네티스 마스터의 서비스, 엔드포인트 오브젝트의 +이 모드에서는, kube-proxy는 쿠버네티스 컨트롤 플레인의 서비스 및 엔드포인트 오브젝트의 추가와 제거를 감시한다. 각 서비스는 로컬 노드에서 포트(임의로 선택됨)를 연다. 이 "프록시 포트"에 대한 모든 연결은 (엔드포인트를 통해 보고된 대로) 서비스의 백엔드 파드 중 하나로 프록시된다. @@ -602,8 +604,8 @@ status: {{< feature-state for_k8s_version="v1.20" state="alpha" >}} -기본적으로 로드밸런서 서비스 유형의 경우 둘 이상의 포트가 정의되어 있을 때 모든 -포트는 동일한 프로토콜을 가져야 하며 프로토콜은 클라우드 공급자가 +기본적으로 로드밸런서 서비스 유형의 경우 둘 이상의 포트가 정의되어 있을 때 모든 +포트는 동일한 프로토콜을 가져야 하며 프로토콜은 클라우드 공급자가 지원하는 프로토콜이어야 한다. kube-apiserver에 대해 기능 게이트 `MixedProtocolLBService`가 활성화된 경우 둘 이상의 포트가 정의되어 있을 때 다른 프로토콜을 사용할 수 있다. @@ -618,7 +620,7 @@ kube-apiserver에 대해 기능 게이트 `MixedProtocolLBService`가 활성화 {{< feature-state for_k8s_version="v1.20" state="alpha" >}} -v1.20부터는 `spec.allocateLoadBalancerNodePorts`필드를 `false`로 설정하여 서비스 Type=LoadBalancer에 +v1.20부터는 `spec.allocateLoadBalancerNodePorts` 필드를 `false`로 설정하여 서비스 Type=LoadBalancer에 대한 노드 포트 할당을 선택적으로 비활성화 할 수 있다. 노드 포트를 사용하는 대신 트래픽을 파드로 직접 라우팅하는 로드 밸런서 구현에만 사용해야 한다. 기본적으로 `spec.allocateLoadBalancerNodePorts`는 `true`이며 로드밸런서 서비스 유형은 계속해서 노드 포트를 할당한다. @@ -1210,8 +1212,8 @@ IPVS는 로드 밸런싱을 위해 설계되었고 커널-내부 해시 테이 {{< feature-state for_k8s_version="v1.20" state="stable" >}} -SCTP 트래픽을 지원하는 네트워크 플러그인을 사용하는 경우 대부분의 서비스에 SCTP를 사용할 수 있다. -type=LoadBalancer 서비스의 경우 SCTP 지원은 이 기능을 제공하는 +SCTP 트래픽을 지원하는 네트워크 플러그인을 사용하는 경우 대부분의 서비스에 SCTP를 사용할 수 있다. +type=LoadBalancer 서비스의 경우 SCTP 지원은 이 기능을 제공하는 클라우드 공급자에 따라 다르다. (대부분 그렇지 않음) #### 경고 {#caveat-sctp-overview} diff --git a/content/ko/docs/concepts/storage/persistent-volumes.md b/content/ko/docs/concepts/storage/persistent-volumes.md index 2b421c7fb2..9ce1eba6cf 100644 --- a/content/ko/docs/concepts/storage/persistent-volumes.md +++ b/content/ko/docs/concepts/storage/persistent-volumes.md @@ -1,4 +1,10 @@ --- + + + + + + title: 퍼시스턴트 볼륨 feature: title: 스토리지 오케스트레이션 @@ -225,7 +231,7 @@ spec: * Azure Disk * Portworx * FlexVolumes -* CSI +* {{< glossary_tooltip text="CSI" term_id="csi" >}} 스토리지 클래스의 `allowVolumeExpansion` 필드가 true로 설정된 경우에만 PVC를 확장할 수 있다. @@ -317,14 +323,14 @@ EBS 볼륨 확장은 시간이 많이 걸리는 작업이다. 또한 6시간마 * [`gcePersistentDisk`](/ko/docs/concepts/storage/volumes/#gcepersistentdisk) - GCE Persistent Disk * [`glusterfs`](/ko/docs/concepts/storage/volumes/#glusterfs) - Glusterfs 볼륨 * [`hostPath`](/ko/docs/concepts/storage/volumes/#hostpath) - HostPath 볼륨 - (단일 노드 테스트 전용. 다중-노드 클러스터에서 작동하지 않음. + (단일 노드 테스트 전용. 다중-노드 클러스터에서 작동하지 않음. 대신 `로컬` 볼륨 사용 고려) * [`iscsi`](/ko/docs/concepts/storage/volumes/#iscsi) - iSCSI (SCSI over IP) 스토리지 -* [`local`](/ko/docs/concepts/storage/volumes/#local) - 노드에 마운트된 +* [`local`](/ko/docs/concepts/storage/volumes/#local) - 노드에 마운트된 로컬 스토리지 디바이스 * [`nfs`](/ko/docs/concepts/storage/volumes/#nfs) - 네트워크 파일 시스템 (NFS) 스토리지 * `photonPersistentDisk` - Photon 컨트롤러 퍼시스턴트 디스크. - (이 볼륨 유형은 해당 클라우드 공급자가 없어진 이후 더 이상 + (이 볼륨 유형은 해당 클라우드 공급자가 없어진 이후 더 이상 작동하지 않는다.) * [`portworxVolume`](/ko/docs/concepts/storage/volumes/#portworxvolume) - Portworx 볼륨 * [`quobyte`](/ko/docs/concepts/storage/volumes/#quobyte) - Quobyte 볼륨 diff --git a/content/ko/docs/concepts/storage/volume-snapshot-classes.md b/content/ko/docs/concepts/storage/volume-snapshot-classes.md index 11708eeec5..e5b6002e6e 100644 --- a/content/ko/docs/concepts/storage/volume-snapshot-classes.md +++ b/content/ko/docs/concepts/storage/volume-snapshot-classes.md @@ -1,4 +1,11 @@ --- + + + + + + + title: 볼륨 스냅샷 클래스 content_type: concept weight: 30 @@ -65,7 +72,7 @@ parameters: ### 삭제정책(DeletionPolicy) -볼륨 스냅샷 클래스는 삭제정책을 가지고 있다. 바인딩된 볼륨스냅샷 오브젝트를 삭제할 때 VolumeSnapshotContent의 상황을 구성할 수 있다. 볼륨 스냅삿의 삭제정책은 `Retain` 또는 `Delete` 일 수 있다. 이 필드는 반드시 지정해야 한다. +볼륨 스냅샷 클래스는 삭제정책을 가지고 있다. 바인딩된 볼륨스냅샷 오브젝트를 삭제할 때 VolumeSnapshotContent의 상황을 구성할 수 있다. 볼륨 스냅샷 클래스의 삭제정책은 `Retain` 또는 `Delete` 일 수 있다. 이 필드는 반드시 지정해야 한다. 삭제정책이 `Delete` 인 경우 기본 스토리지 스냅샷이 VolumeSnapshotContent 오브젝트와 함께 삭제된다. 삭제정책이 `Retain` 인 경우 기본 스냅샷과 VolumeSnapshotContent 모두 유지된다. diff --git a/content/ko/docs/concepts/workloads/controllers/garbage-collection.md b/content/ko/docs/concepts/workloads/controllers/garbage-collection.md index a93b455948..a3ed4ea68d 100644 --- a/content/ko/docs/concepts/workloads/controllers/garbage-collection.md +++ b/content/ko/docs/concepts/workloads/controllers/garbage-collection.md @@ -150,9 +150,12 @@ curl -X DELETE localhost:8080/apis/apps/v1/namespaces/default/replicasets/my-rep ``` kubectl도 캐스케이딩 삭제를 지원한다. -kubectl을 사용해서 종속 항목을 자동으로 삭제하려면 `--cascade` 를 true로 설정한다. 종속 항목을 -분리하기 위해서는 `--cascade` 를 false로 설정한다. `--cascade` 의 기본값은 -true 이다. + +kubectl을 사용해서 포어그라운드의 종속 항목을 삭제하려면 `--cascade=foreground` 를 설정한다. 종속 항목을 +분리하기 위해서는 `--cascade=orphan` 를 설정한다. + +기본 동작은 백그라운드의 종속 항목을 삭제하는 것이며, +이는 `--cascade` 를 생략하거나 명시적으로 `background` 를 설정한 경우의 동작에 해당한다. 여기에 레플리카셋의 종속 항목을 분리로 만드는 예시가 있다. diff --git a/content/ko/docs/concepts/workloads/controllers/replicaset.md b/content/ko/docs/concepts/workloads/controllers/replicaset.md index c5a6832fd9..8966029620 100644 --- a/content/ko/docs/concepts/workloads/controllers/replicaset.md +++ b/content/ko/docs/concepts/workloads/controllers/replicaset.md @@ -1,4 +1,8 @@ --- + + + + title: 레플리카셋 content_type: concept weight: 20 @@ -279,7 +283,7 @@ curl -X DELETE 'localhost:8080/apis/apps/v1/namespaces/default/replicasets/fron ### 레플리카셋만 삭제하기 -레플리카셋을 `--cascade=false` 옵션과 함께 [`kubectl delete`](/docs/reference/generated/kubectl/kubectl-commands#delete)를 사용하면 연관 파드에 영향을 주지 않고 삭제할 수 있다. +레플리카셋을 `--cascade=orphan` 옵션과 함께 [`kubectl delete`](/docs/reference/generated/kubectl/kubectl-commands#delete)를 사용하면 연관 파드에 영향을 주지 않고 삭제할 수 있다. REST API 또는 `client-go` 라이브러리를 이용할 때는 `propagationPolicy`에 `Orphan`을 설정해야 한다. 예시: ```shell diff --git a/content/ko/docs/concepts/workloads/controllers/replicationcontroller.md b/content/ko/docs/concepts/workloads/controllers/replicationcontroller.md index 5c5b5c750f..9c3450851a 100644 --- a/content/ko/docs/concepts/workloads/controllers/replicationcontroller.md +++ b/content/ko/docs/concepts/workloads/controllers/replicationcontroller.md @@ -1,4 +1,7 @@ --- + + + title: 레플리케이션 컨트롤러 feature: title: 자가 치유 @@ -51,15 +54,17 @@ kubectl 명령에서 숏컷으로 사용된다. ```shell kubectl apply -f https://k8s.io/examples/controllers/replication.yaml ``` +출력 결과는 다음과 같다. ``` replicationcontroller/nginx created ``` -다음 명령을 사용하여 레플리케이션 컨트롤러의 상태를 확인하라. +다음 명령을 사용하여 레플리케이션 컨트롤러의 상태를 확인하자. ```shell kubectl describe replicationcontrollers/nginx ``` +출력 결과는 다음과 같다. ``` Name: nginx Namespace: default @@ -98,6 +103,7 @@ Pods Status: 3 Running / 0 Waiting / 0 Succeeded / 0 Failed pods=$(kubectl get pods --selector=app=nginx --output=jsonpath={.items..metadata.name}) echo $pods ``` +출력 결과는 다음과 같다. ``` nginx-3ntk0 nginx-4ok8v nginx-qrm3m ``` diff --git a/content/ko/docs/concepts/workloads/pods/_index.md b/content/ko/docs/concepts/workloads/pods/_index.md index 265f971816..8645053bef 100644 --- a/content/ko/docs/concepts/workloads/pods/_index.md +++ b/content/ko/docs/concepts/workloads/pods/_index.md @@ -15,8 +15,7 @@ card: _파드(Pod)_ 는 쿠버네티스에서 생성하고 관리할 수 있는 배포 가능한 가장 작은 컴퓨팅 단위이다. _파드_ (고래 떼(pod of whales)나 콩꼬투리(pea pod)와 마찬가지로)는 하나 이상의 -[컨테이너](/ko/docs/concepts/containers/)의 그룹이다. -이 그룹은 스토리지/네트워크를 공유하고, 해당 컨테이너를 구동하는 방식에 대한 명세를 갖는다. 파드의 콘텐츠는 항상 함께 배치되고, +[컨테이너](/ko/docs/concepts/containers/)의 그룹이다. 이 그룹은 스토리지 및 네트워크를 공유하고, 해당 컨테이너를 구동하는 방식에 대한 명세를 갖는다. 파드의 콘텐츠는 항상 함께 배치되고, 함께 스케줄되며, 공유 콘텍스트에서 실행된다. 파드는 애플리케이션 별 "논리 호스트"를 모델링한다. 여기에는 상대적으로 밀접하게 결합된 하나 이상의 애플리케이션 컨테이너가 포함된다. @@ -217,7 +216,8 @@ spec: 허용한다. 1. 지정되지 않은 필드를 양수로 설정; - 1. 필드의 양수를 음수가 아닌 더 작은 숫자로 갱신. + 1. 필드의 양수를 음수가 아닌 더 작은 숫자로 + 갱신. ## 리소스 공유와 통신 @@ -294,9 +294,10 @@ kubelet은 자동으로 각 정적 파드에 대한 쿠버네티스 API 서버 오브젝트 정의는 오브젝트를 상세히 설명한다. * [분산 시스템 툴킷: 컴포지트 컨테이너에 대한 패턴](https://kubernetes.io/blog/2015/06/the-distributed-system-toolkit-patterns)은 둘 이상의 컨테이너가 있는 파드의 일반적인 레이아웃을 설명한다. -쿠버네티스가 다른 리소스({{< glossary_tooltip text="스테이트풀셋" term_id="statefulset" >}}이나 {{< glossary_tooltip text="디플로이먼트" term_id="deployment" >}}와 같은)에서 공통 파드 API를 래핑하는 이유에 대한 콘텍스트를 이해하기 위해서, 다음을 포함한 선행 기술에 대해 읽어볼 수 있다. - * [Aurora](http://aurora.apache.org/documentation/latest/reference/configuration/#job-schema) - * [Borg](https://research.google.com/pubs/pub43438.html) - * [Marathon](https://mesosphere.github.io/marathon/docs/rest-api.html) - * [Omega](https://research.google/pubs/pub41684/) - * [Tupperware](https://engineering.fb.com/data-center-engineering/tupperware/). +쿠버네티스가 다른 리소스({{< glossary_tooltip text="스테이트풀셋" term_id="statefulset" >}}이나 {{< glossary_tooltip text="디플로이먼트" term_id="deployment" >}}와 같은)에서 공통 파드 API를 래핑하는 이유에 대한 콘텍스트를 이해하기 위해서, 다음과 같은 선행 기술에 대해 읽어볼 수 있다. + +* [Aurora](https://aurora.apache.org/documentation/latest/reference/configuration/#job-schema) +* [Borg](https://research.google.com/pubs/pub43438.html) +* [Marathon](https://mesosphere.github.io/marathon/docs/rest-api.html) +* [Omega](https://research.google/pubs/pub41684/) +* [Tupperware](https://engineering.fb.com/data-center-engineering/tupperware/). diff --git a/content/ko/docs/concepts/workloads/pods/disruptions.md b/content/ko/docs/concepts/workloads/pods/disruptions.md index 5f13dddef9..02647adb70 100644 --- a/content/ko/docs/concepts/workloads/pods/disruptions.md +++ b/content/ko/docs/concepts/workloads/pods/disruptions.md @@ -1,4 +1,8 @@ --- + + + + title: 중단(disruption) content_type: concept weight: 60 @@ -45,7 +49,7 @@ weight: 60 - 복구 또는 업그레이드를 위한 [노드 드레이닝](/docs/tasks/administer-cluster/safely-drain-node/). - 클러스터의 스케일 축소를 위한 - 노드 드레이닝([클러스터 오토스케일링](/ko/docs/tasks/administer-cluster/cluster-management/#클러스터-오토스케일링)에 대해 알아보기 + 노드 드레이닝([클러스터 오토스케일링](https://github.com/kubernetes/autoscaler/#readme)에 대해 알아보기 ). - 노드에 다른 무언가를 추가하기 위해 파드를 제거. diff --git a/content/ko/docs/concepts/workloads/pods/init-containers.md b/content/ko/docs/concepts/workloads/pods/init-containers.md index c1e3943178..56f59c1d30 100644 --- a/content/ko/docs/concepts/workloads/pods/init-containers.md +++ b/content/ko/docs/concepts/workloads/pods/init-containers.md @@ -133,6 +133,7 @@ spec: ```shell kubectl apply -f myapp.yaml ``` +출력 결과는 다음과 같다. ``` pod/myapp-pod created ``` @@ -141,6 +142,7 @@ pod/myapp-pod created ```shell kubectl get -f myapp.yaml ``` +출력 결과는 다음과 같다. ``` NAME READY STATUS RESTARTS AGE myapp-pod 0/1 Init:0/2 0 6m @@ -150,6 +152,7 @@ myapp-pod 0/1 Init:0/2 0 6m ```shell kubectl describe -f myapp.yaml ``` +출력 결과는 다음과 같다. ``` Name: myapp-pod Namespace: default @@ -224,6 +227,7 @@ spec: ```shell kubectl apply -f services.yaml ``` +출력 결과는 다음과 같다. ``` service/myservice created service/mydb created @@ -235,6 +239,7 @@ service/mydb created ```shell kubectl get -f myapp.yaml ``` +출력 결과는 다음과 같다. ``` NAME READY STATUS RESTARTS AGE myapp-pod 1/1 Running 0 9m @@ -257,7 +262,7 @@ myapp-pod 1/1 Running 0 9m 파드는 모든 초기화 컨테이너가 성공되기 전까지 `Ready` 될 수 없다. 초기화 컨테이너의 포트는 서비스 하에 합쳐지지 않는다. 초기화 중인 파드는 `Pending` 상태이지만 -`Initialized` 가 참이 되는 조건을 가져야 한다. +`Initialized` 가 거짓이 되는 조건을 가져야 한다. 만약 파드가 [재시작](#파드-재시작-이유)되었다면, 모든 초기화 컨테이너는 반드시 다시 실행된다. diff --git a/content/ko/docs/concepts/workloads/pods/pod-lifecycle.md b/content/ko/docs/concepts/workloads/pods/pod-lifecycle.md index 0c4ea5960c..1066e3eb83 100644 --- a/content/ko/docs/concepts/workloads/pods/pod-lifecycle.md +++ b/content/ko/docs/concepts/workloads/pods/pod-lifecycle.md @@ -85,6 +85,13 @@ UID로 정의된 특정 파드는 다른 노드로 절대 "다시 스케줄"되 `Failed` | 파드에 있는 모든 컨테이너가 종료되었고, 적어도 하나 이상의 컨테이너가 실패로 종료되었다. 즉, 해당 컨테이너는 non-zero 상태로 빠져나왔거나(exited) 시스템에 의해서 종료(terminated)되었다. `Unknown` | 어떤 이유에 의해서 파드의 상태를 얻을 수 없다. 이 단계는 일반적으로 파드가 실행되어야 하는 노드와의 통신 오류로 인해 발생한다. +{{< note >}} +파드가 삭제될 때, 일부 kubectl 커맨드에서 `Terminating` 이 표시된다. +이 `Terminating` 상태는 파드의 단계에 해당하지 않는다. +파드에는 그레이스풀하게(gracefully) 종료되도록 기간이 부여되며, 그 기본값은 30초이다. +[강제로 파드를 종료](/ko/docs/concepts/workloads/pods/pod-lifecycle/#pod-termination-forced)하려면 `--force` 플래그를 설정하면 된다. +{{< /note >}} + 노드가 죽거나 클러스터의 나머지와의 연결이 끊어지면, 쿠버네티스는 손실된 노드의 모든 파드의 `phase` 를 Failed로 설정하는 정책을 적용한다. @@ -142,8 +149,7 @@ Never이다. 기본값은 Always이다. 동일한 노드에서 kubelet에 의한 컨테이너 재시작만을 의미한다. 파드의 컨테이너가 종료된 후, kubelet은 5분으로 제한되는 지수 백오프 지연(10초, 20초, 40초, …)으로 컨테이너를 재시작한다. 컨테이너가 10분 동안 아무런 문제없이 실행되면, -kubelet은 해당 컨테이너의 재시작 백오프 타이머를 -재설정한다. +kubelet은 해당 컨테이너의 재시작 백오프 타이머를 재설정한다. ## 파드의 조건 @@ -326,7 +332,7 @@ kubelet은 실행 중인 컨테이너들에 대해서 선택적으로 세 가지 컨테이너가 보통 `initialDelaySeconds + failureThreshold × periodSeconds` 이후에 기동된다면, 스타트업 프로브가 활성화 프로브와 같은 엔드포인트를 확인하도록 지정해야 한다. -`periodSeconds`의 기본값은 30s 이다. 이 때 컨테이너가 활성화 프로브의 +`periodSeconds`의 기본값은 10s 이다. 이 때 컨테이너가 활성화 프로브의 기본값 변경 없이 기동되도록 하려면, `failureThreshold` 를 충분히 높게 설정해주어야 한다. 그래야 데드락(deadlocks)을 방지하는데 도움이 된다. diff --git a/content/ko/docs/reference/command-line-tools-reference/feature-gates.md b/content/ko/docs/reference/command-line-tools-reference/feature-gates.md index ad9c9a45de..6fa2c58a56 100644 --- a/content/ko/docs/reference/command-line-tools-reference/feature-gates.md +++ b/content/ko/docs/reference/command-line-tools-reference/feature-gates.md @@ -166,7 +166,8 @@ kubelet과 같은 컴포넌트의 기능 게이트를 설정하려면, 기능 | `StorageVersionHash` | `true` | 베타 | 1.15 | | | `Sysctls` | `true` | 베타 | 1.11 | | | `TTLAfterFinished` | `false` | 알파 | 1.12 | | -| `TopologyManager` | `false` | 알파 | 1.16 | | +| `TopologyManager` | `false` | 알파 | 1.16 | 1.17 | +| `TopologyManager` | `true` | 베타 | 1.18 | | | `ValidateProxyRedirects` | `false` | 알파 | 1.12 | 1.13 | | `ValidateProxyRedirects` | `true` | 베타 | 1.14 | | | `WindowsEndpointSliceProxying` | `false` | 알파 | 1.19 | | diff --git a/content/ko/docs/reference/kubectl/cheatsheet.md b/content/ko/docs/reference/kubectl/cheatsheet.md index 7db5ce3bdb..2ac5f6076c 100644 --- a/content/ko/docs/reference/kubectl/cheatsheet.md +++ b/content/ko/docs/reference/kubectl/cheatsheet.md @@ -190,6 +190,9 @@ kubectl get pods --show-labels JSONPATH='{range .items[*]}{@.metadata.name}:{range @.status.conditions[*]}{@.type}={@.status};{end}{end}' \ && kubectl get nodes -o jsonpath="$JSONPATH" | grep "Ready=True" +# 외부 도구 없이 디코딩된 시크릿 출력 +kubectl get secret ${secret_name} -o go-template='{{range $k,$v := .data}}{{$k}}={{$v|base64decode}}{{"\n"}}{{end}}' + # 파드에 의해 현재 사용되고 있는 모든 시크릿 목록 조회 kubectl get pods -o json | jq '.items[].spec.containers[].env[]?.valueFrom.secretKeyRef.name' | grep -v null | sort | uniq @@ -311,6 +314,7 @@ kubectl exec my-pod -- ls / # 기존 파드에서 명령 kubectl exec --stdin --tty my-pod -- /bin/sh # 실행 중인 파드로 대화형 셸 액세스(1 컨테이너 경우) kubectl exec my-pod -c my-container -- ls / # 기존 파드에서 명령 실행(멀티-컨테이너 경우) kubectl top pod POD_NAME --containers # 특정 파드와 해당 컨테이너에 대한 메트릭 표시 +kubectl top pod POD_NAME --sort-by=cpu # 지정한 파드에 대한 메트릭을 표시하고 'cpu' 또는 'memory'별로 정렬 ``` ## 노드, 클러스터와 상호 작용 @@ -388,6 +392,7 @@ Kubectl 로그 상세 레벨(verbosity)은 `-v` 또는`--v` 플래그와 로그 `--v=2` | 서비스와 시스템의 중요한 변화와 관련이있는 중요한 로그 메시지에 대한 유용한 정상 상태 정보. 이는 대부분의 시스템에서 권장되는 기본 로그 수준이다. `--v=3` | 변경 사항에 대한 확장 정보. `--v=4` | 디버그 수준 상세화. +`--v=5` | 트레이스 수준 상세화. `--v=6` | 요청한 리소스를 표시. `--v=7` | HTTP 요청 헤더를 표시. `--v=8` | HTTP 요청 내용을 표시. diff --git a/content/ko/docs/reference/kubectl/conventions.md b/content/ko/docs/reference/kubectl/conventions.md index b18f64b5ef..aa76c0c8bf 100644 --- a/content/ko/docs/reference/kubectl/conventions.md +++ b/content/ko/docs/reference/kubectl/conventions.md @@ -1,7 +1,7 @@ --- + + title: kubectl 사용 규칙 - - content_type: concept --- @@ -36,24 +36,23 @@ content_type: concept {{< /note >}} #### 생성기 -`kubectl create --dry-run -o yaml`라는 kubectl 커맨드를 통해 다음과 같은 리소스를 생성 할 수 있다. -``` - clusterrole 클러스터롤(ClusterRole)를 생성한다. - clusterrolebinding 특정 클러스터롤에 대한 클러스터롤바인딩(ClusterRoleBinding)을 생성한다. - configmap 로컬 파일, 디렉토리 또는 문자 그대로의 값으로 컨피그맵(ConfigMap)을 생성한다. - cronjob 지정된 이름으로 크론잡(CronJob)을 생성한다. - deployment 지정된 이름으로 디플로이먼트(Deployment)를 생성한다. - job 지정된 이름으로 잡(Job)을 생성한다. - namespace 지정된 이름으로 네임스페이스(Namespace)를 생성한다. - poddisruptionbudget 지정된 이름으로 pod disruption budget을 생성한다. - priorityclass 지정된 이름으로 프라이어리티클래스(PriorityClass)을 생성한다. - quota 지정된 이름으로 쿼터(Quota)를 생성한다. - role 단일 규칙으로 롤(Role)을 생성한다. - rolebinding 특정 롤 또는 클러스터롤에 대한 롤바인딩(RoleBinding)을 생성한다. - secret 지정된 하위 커맨드를 사용하여 시크릿(Secret)을 생성한다. - service 지정된 하위 커맨드를 사용하여 서비스(Service)를 생성한다. - serviceaccount 지정된 이름으로 서비스어카운트(ServiceAccount)을 생성한다. -``` +`kubectl create --dry-run -o yaml`라는 kubectl 커맨드를 통해 다음과 같은 리소스를 생성할 수 있다. + +* `clusterrole`: 클러스터롤(ClusterRole)를 생성한다. +* `clusterrolebinding`: 특정 클러스터롤에 대한 클러스터롤바인딩(ClusterRoleBinding)을 생성한다. +* `configmap`: 로컬 파일, 디렉토리 또는 문자 그대로의 값으로 컨피그맵(ConfigMap)을 생성한다. +* `cronjob`: 지정된 이름으로 크론잡(CronJob)을 생성한다. +* `deployment`: 지정된 이름으로 디플로이먼트(Deployment)를 생성한다. +* `job`: 지정된 이름으로 잡(Job)을 생성한다. +* `namespace`: 지정된 이름으로 네임스페이스(Namespace)를 생성한다. +* `poddisruptionbudget`: 지정된 이름으로 PodDisruptionBudget을 생성한다. +* `priorityclass`: 지정된 이름으로 프라이어리티클래스(PriorityClass)을 생성한다. +* `quota`: 지정된 이름으로 쿼터(Quota)를 생성한다. +* `role`: 단일 규칙으로 롤(Role)을 생성한다. +* `rolebinding`: 특정 롤 또는 클러스터롤에 대한 롤바인딩(RoleBinding)을 생성한다. +* `secret`: 지정된 하위 커맨드를 사용하여 시크릿(Secret)을 생성한다. +* `service`: 지정된 하위 커맨드를 사용하여 서비스(Service)를 생성한다. +* `serviceaccount`: 지정된 이름으로 서비스어카운트(ServiceAccount)을 생성한다. ### `kubectl apply` diff --git a/content/ko/docs/reference/kubectl/docker-cli-to-kubectl.md b/content/ko/docs/reference/kubectl/docker-cli-to-kubectl.md index e81dd65b4a..1679059871 100644 --- a/content/ko/docs/reference/kubectl/docker-cli-to-kubectl.md +++ b/content/ko/docs/reference/kubectl/docker-cli-to-kubectl.md @@ -37,16 +37,11 @@ kubectl: # nginx 실행하는 파드를 시작한다 kubectl create deployment --image=nginx nginx-app ``` - -```shell -# nginx-app 에 env를 추가한다 -kubectl set env deployment/nginx-app DOMAIN=cluster -``` ``` deployment.apps/nginx-app created ``` -``` +```shell # nginx-app 에 env를 추가한다 kubectl set env deployment/nginx-app DOMAIN=cluster ``` @@ -369,4 +364,3 @@ Grafana is running at https://203.0.113.141/api/v1/namespaces/kube-system/servic Heapster is running at https://203.0.113.141/api/v1/namespaces/kube-system/services/monitoring-heapster/proxy InfluxDB is running at https://203.0.113.141/api/v1/namespaces/kube-system/services/monitoring-influxdb/proxy ``` - diff --git a/content/ko/docs/setup/production-environment/container-runtimes.md b/content/ko/docs/setup/production-environment/container-runtimes.md index 8bdd90800b..4b749ab7e5 100644 --- a/content/ko/docs/setup/production-environment/container-runtimes.md +++ b/content/ko/docs/setup/production-environment/container-runtimes.md @@ -143,6 +143,44 @@ sudo mkdir -p /etc/containerd sudo containerd config default | sudo tee /etc/containerd/config.toml ``` +```shell +# containerd 재시작 +sudo systemctl restart containerd +``` +{{% /tab %}} +{{% tab name="Debian 9+" %}} + +```shell +# (containerd 설치) +## 리포지터리 설정 +### HTTPS를 통해 리포지터리를 사용할 수 있도록 패키지 설치 +sudo apt-get update && sudo apt-get install -y apt-transport-https ca-certificates curl software-properties-common +``` + +```shell +## 도커의 공식 GPG 키 추가 +curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key --keyring /etc/apt/trusted.gpg.d/docker.gpg add - +``` + +```shell +## 도커 apt 리포지터리 추가 +sudo add-apt-repository \ + "deb [arch=amd64] https://download.docker.com/linux/debian \ + $(lsb_release -cs) \ + stable" +``` + +```shell +## containerd 설치 +sudo apt-get update && sudo apt-get install -y containerd.io +``` + +```shell +# 기본 containerd 구성 설정 +sudo mkdir -p /etc/containerd +containerd config default | sudo tee /etc/containerd/config.toml +``` + ```shell # containerd 재시작 sudo systemctl restart containerd @@ -230,12 +268,18 @@ kubeadm을 사용하는 경우, {{< note >}} CRI-O 메이저와 마이너 버전은 쿠버네티스 메이저와 마이너 버전이 일치해야 한다. -더 자세한 정보는 [CRI-O 호환 매트릭스](https://github.com/cri-o/cri-o)를 본다. +더 자세한 정보는 [CRI-O 호환 매트릭스](https://github.com/cri-o/cri-o#compatibility-matrix-cri-o--kubernetes)를 본다. {{< /note >}} 필수 구성 요소를 설치하고 구성한다. ```shell +# .conf 파일을 만들어 부팅 시 모듈을 로드한다 +cat < 그런 다음, `$VERSION` 을 사용자의 쿠버네티스 버전과 일치하는 CRI-O 버전으로 설정한다. -예를 들어, CRI-O 1.18을 설치하려면, `VERSION=1.18` 로 설정한다. +예를 들어, CRI-O 1.20을 설치하려면, `VERSION=1.20` 로 설정한다. 사용자의 설치를 특정 릴리스에 고정할 수 있다. -버전 1.18.3을 설치하려면, `VERSION=1.18:1.18.3` 을 설정한다. +버전 1.20.0을 설치하려면, `VERSION=1.20:1.20.0` 을 설정한다.
그런 다음, 아래를 실행한다. @@ -298,9 +342,9 @@ sudo apt-get install cri-o cri-o-runc
그런 다음, `$VERSION` 을 사용자의 쿠버네티스 버전과 일치하는 CRI-O 버전으로 설정한다. -예를 들어, CRI-O 1.18을 설치하려면, `VERSION=1.18` 로 설정한다. +예를 들어, CRI-O 1.20을 설치하려면, `VERSION=1.20` 로 설정한다. 사용자의 설치를 특정 릴리스에 고정할 수 있다. -버전 1.18.3을 설치하려면, `VERSION=1.18:1.18.3` 을 설정한다. +버전 1.20.0을 설치하려면, `VERSION=1.20:1.20.0` 을 설정한다.
그런 다음, 아래를 실행한다. @@ -332,9 +376,9 @@ apt-get install cri-o cri-o-runc
그런 다음, `$VERSION` 을 사용자의 쿠버네티스 버전과 일치하는 CRI-O 버전으로 설정한다. -예를 들어, CRI-O 1.18을 설치하려면, `VERSION=1.18` 로 설정한다. +예를 들어, CRI-O 1.20을 설치하려면, `VERSION=1.20` 로 설정한다. 사용자의 설치를 특정 릴리스에 고정할 수 있다. -버전 1.18.3을 설치하려면, `VERSION=1.18:1.18.3` 을 설정한다. +버전 1.20.0을 설치하려면, `VERSION=1.20:1.20.0` 을 설정한다.
그런 다음, 아래를 실행한다. @@ -355,7 +399,7 @@ sudo zypper install cri-o {{% tab name="Fedora" %}} `$VERSION` 을 사용자의 쿠버네티스 버전과 일치하는 CRI-O 버전으로 설정한다. -예를 들어, CRI-O 1.18을 설치하려면, `VERSION=1.18` 로 설정한다. +예를 들어, CRI-O 1.20을 설치하려면, `VERSION=1.20` 로 설정한다. 사용할 수 있는 버전을 찾으려면 다음을 실행한다. ```shell @@ -379,10 +423,26 @@ sudo systemctl daemon-reload sudo systemctl start crio ``` -자세한 사항은 [CRI-O 설치 가이드](https://github.com/kubernetes-sigs/cri-o#getting-started)를 +자세한 사항은 [CRI-O 설치 가이드](https://github.com/cri-o/cri-o/blob/master/install.md)를 참고한다. +#### cgroup 드라이버 + +CRI-O는 기본적으로 systemd cgroup 드라이버를 사용한다. `cgroupfs` cgroup 드라이버로 +전환하려면, `/etc/crio/crio.conf` 를 수정하거나 `/etc/crio/crio.conf.d/02-cgroup-manager.conf` 에 +드롭-인(drop-in) 구성을 배치한다. 예를 들면, 다음과 같다. + +```toml +[crio.runtime] +conmon_cgroup = "pod" +cgroup_manager = "cgroupfs" +``` + +또한 `cgroupfs` 와 함께 CRI-O를 사용할 때 `pod` 값으로 설정해야 하는 +변경된 `conmon_cgroup` 에 유의한다. 일반적으로 kubelet(일반적으로 kubeadm을 통해 수행됨)과 +CRI-O의 cgroup 드라이버 구성을 동기화 상태로 +유지해야 한다. ### 도커 @@ -425,6 +485,11 @@ sudo apt-get update && sudo apt-get install -y \ docker-ce-cli=5:19.03.11~3-0~ubuntu-$(lsb_release -cs) ``` +```shell +## /etc/docker 생성 +sudo mkdir /etc/docker +``` + ```shell # 도커 데몬 설정 cat < -이 작업은 프론트엔드와 마이크로서비스 백엔드를 어떻게 생성하는지를 -설명한다. 백엔드 마이크로서비스는 인사하기(hello greeter)이다. -프론트엔드와 백엔드는 쿠버네티스 {{< glossary_tooltip term_id="service" text="서비스" >}} -오브젝트를 이용해 연결되어 있다. +이 작업은 _프론트엔드_ 와 _백엔드_ 마이크로서비스를 어떻게 생성하는지를 설명한다. 백엔드 +마이크로서비스는 인사하기(hello greeter)이다. 프론트엔드는 nginx 및 쿠버네티스 +{{< glossary_tooltip term_id="service" text="서비스" >}} 오브젝트를 사용해 백엔드를 노출한다. ## {{% heading "objectives" %}} -* {{< glossary_tooltip term_id="deployment" text="디플로이먼트(Deployment)" >}} 오브젝트를 사용해 마이크로서비스를 생성하고 실행한다. -* 프론트엔드를 사용하여 백엔드로 트래픽을 전달한다. -* 프론트엔드 애플리케이션을 백엔드 애플리케이션에 연결하기 위해 - 서비스 오브젝트를 사용한다. +* {{< glossary_tooltip term_id="deployment" text="디플로이먼트(Deployment)" >}} 오브젝트를 사용해 + 샘플 `hello` 백엔드 마이크로서비스를 생성하고 실행한다. +* 서비스 오브젝트를 사용하여 백엔드 마이크로서비스의 여러 복제본으로 트래픽을 보낸다. +* 디플로이먼트 오브젝트를 사용하여 `nginx` 프론트엔드 마이크로서비스를 생성하고 실행한다. +* 트래픽을 백엔드 마이크로서비스로 보내도록 프론트엔드 마이크로서비스를 구성한다. +* `type=LoadBalancer` 의 서비스 오브젝트를 사용해 클러스터 외부에 프론트엔드 마이크로서비스를 + 노출한다. ## {{% heading "prerequisites" %}} @@ -34,24 +36,24 @@ weight: 70 백엔드는 인사하기라는 간단한 마이크로서비스이다. 여기에 백엔드 디플로이먼트 구성 파일이 있다. -{{< codenew file="service/access/hello.yaml" >}} +{{< codenew file="service/access/backend-deployment.yaml" >}} 백엔드 디플로이먼트를 생성한다. ```shell -kubectl apply -f https://k8s.io/examples/service/access/hello.yaml +kubectl apply -f https://k8s.io/examples/service/access/backend-deployment.yaml ``` 백엔드 디플로이먼트에 관한 정보를 본다. ```shell -kubectl describe deployment hello +kubectl describe deployment backend ``` 결과는 아래와 같다. ``` -Name: hello +Name: backend Namespace: default CreationTimestamp: Mon, 24 Oct 2016 14:21:02 -0700 Labels: app=hello @@ -59,7 +61,7 @@ Labels: app=hello track=stable Annotations: deployment.kubernetes.io/revision=1 Selector: app=hello,tier=backend,track=stable -Replicas: 7 desired | 7 updated | 7 total | 7 available | 0 unavailable +Replicas: 3 desired | 3 updated | 3 total | 3 available | 0 unavailable StrategyType: RollingUpdate MinReadySeconds: 0 RollingUpdateStrategy: 1 max unavailable, 1 max surge @@ -80,14 +82,14 @@ Conditions: Available True MinimumReplicasAvailable Progressing True NewReplicaSetAvailable OldReplicaSets: -NewReplicaSet: hello-3621623197 (7/7 replicas created) +NewReplicaSet: hello-3621623197 (3/3 replicas created) Events: ... ``` -## 백엔드 서비스 오브젝트 생성하기 +## `hello` 서비스 오브젝트 생성하기 -프론트엔드와 백엔드를 연결하는 방법은 백엔드 +프론트엔드에서 백엔드로 요청을 보내는 핵심은 백엔드 서비스이다. 서비스는 백엔드 마이크로서비스에 언제든 도달하기 위해 변하지 않는 IP 주소와 DNS 이름 항목을 생성한다. 서비스는 트래픽을 보내는 파드를 찾기 위해 @@ -95,42 +97,51 @@ Events: 먼저, 서비스 구성 파일을 살펴보자. -{{< codenew file="service/access/hello-service.yaml" >}} +{{< codenew file="service/access/backend-service.yaml" >}} -구성 파일에서 서비스가 `app: hello` 과 `tier: backend` 레이블을 갖는 +구성 파일에서 `hello` 라는 이름의 서비스가 `app: hello` 및 `tier: backend` 레이블을 갖는 파드에 트래픽을 보내는 것을 볼 수 있다. -`hello` 서비스를 생성한다. +백엔드 서비스를 생성한다. ```shell -kubectl apply -f https://k8s.io/examples/service/access/hello-service.yaml +kubectl apply -f https://k8s.io/examples/service/access/backend-service.yaml ``` -이 시점에서, 백엔드 디플로이먼트는 Running 중이며, 해당 백엔드로 -트래픽을 보내는 서비스를 갖고 있다. +이 시점에서 `hello` 애플리케이션의 복제본 3개를 실행하는 `backend` +디플로이먼트가 있고, 해당 백엔드로 트래픽을 보내는 서비스가 있다. 그러나, 이 +서비스는 클러스터 외부에서 사용할 수 없거나 확인할 수 없다. ## 프론트엔드 생성하기 -이제 백엔드가 있기 때문에 백엔드와 연결하는 프론트엔드를 생성할 수 있다. -프론트엔드는 백엔드 서비스에서 제공된 DNS 이름을 사용해 백엔드 워커 -파드에 연결할 수 있다. DNS 이름은 "hello" 이고, 앞선 -서비스 구성 파일의 `name` 항목의 값이다. +이제 백엔드를 실행했으므로, 클러스터 외부에서 접근할 수 있는 +프론트엔드를 만들고, 백엔드로의 요청을 프록시하여 백엔드에 연결할 수 있다. -프론트엔드 디플로이먼트 안에 파드는 hello 백엔드 서비스를 찾도록 -구성된 nginx 이미지를 실행한다. 여기에 nginx 구성 파일이 있다. +프론트엔드는 백엔드 서비스에 지정된 DNS 이름을 사용하여 백엔드 +워커 파드에 요청을 보낸다. DNS 이름은 +`examples/service/access/backend-service.yaml` 구성 파일의 +`name` 필드 값인 `hello` 이다. -{{< codenew file="service/access/frontend.conf" >}} +프론트엔드 디플로이먼트 안의 파드는 `hello` 백엔드 서비스에 대한 요청을 +프록시하도록 구성된 nginx 이미지를 실행한다. 다음은 nginx 구성 파일이다. -백엔드와 같이, 프론트엔드는 디플로이먼트와 서비스를 갖고 있다. -서비스의 구성은 `type: LoadBalancer` 이며, 이는 서비스가 -클라우드 공급자의 기본 로드 밸런서를 사용하는 것을 의미한다. +{{< codenew file="service/access/frontend-nginx.conf" >}} -{{< codenew file="service/access/frontend.yaml" >}} +백엔드와 같이, 프론트엔드는 디플로이먼트와 서비스를 갖고 있다. 백엔드 +서비스와 프론트엔드 서비스 간에 주목해야 할 중요한 차이점은 프론트엔드 +서비스의 구성에 `type: LoadBalancer` 가 있다는 것이다. 즉, +서비스가 클라우드 공급자가 프로비저닝한 로드 밸런서를 사용하고 +클러스터 외부에서 접근할 수 있음을 의미한다. + +{{< codenew file="service/access/frontend-service.yaml" >}} + +{{< codenew file="service/access/frontend-deployment.yaml" >}} 프론트엔드 디플로이먼트와 서비스를 생성한다. ```shell -kubectl apply -f https://k8s.io/examples/service/access/frontend.yaml +kubectl apply -f https://k8s.io/examples/service/access/frontend-deployment.yaml +kubectl apply -f https://k8s.io/examples/service/access/frontend-service.yaml ``` 결과는 두 리소스가 생성되었음을 확인한다. @@ -196,17 +207,17 @@ curl http://${EXTERNAL_IP} # 앞의 예에서 본 EXTERNAL-IP로 수정한다 서비스를 삭제하기 위해, 아래 명령어를 입력하자. ```shell -kubectl delete services frontend hello +kubectl delete services frontend backend ``` 백엔드와 프론트엔드 애플리케이션에서 실행 중인 디플로이먼트, 레플리카셋, 파드를 삭제하기 위해, 아래 명령어를 입력하자. ```shell -kubectl delete deployment frontend hello +kubectl delete deployment frontend backend ``` ## {{% heading "whatsnext" %}} * [서비스](/ko/docs/concepts/services-networking/service/)에 대해 더 알아본다. * [컨피그맵](/docs/tasks/configure-pod-container/configure-pod-configmap/)에 대해 더 알아본다. - +* [서비스와 파드용 DNS](/docs/concepts/services-networking/dns-pod-service/)에 대해 더 알아본다. diff --git a/content/ko/docs/tasks/manage-kubernetes-objects/kustomization.md b/content/ko/docs/tasks/manage-kubernetes-objects/kustomization.md index fd59ccfd4f..c9ed347c56 100644 --- a/content/ko/docs/tasks/manage-kubernetes-objects/kustomization.md +++ b/content/ko/docs/tasks/manage-kubernetes-objects/kustomization.md @@ -392,8 +392,8 @@ spec: containers: - name: my-nginx resources: - limits: - memory: 512Mi + limits: + memory: 512Mi EOF cat <./kustomization.yaml @@ -424,11 +424,12 @@ spec: spec: containers: - image: nginx - limits: - memory: 512Mi name: my-nginx ports: - containerPort: 80 + resources: + limits: + memory: 512Mi ``` 모든 리소스 또는 필드가 전략적 병합 패치를 지원하는 것은 아니다. 임의의 리소스 내 임의의 필드의 수정을 지원하기 위해, diff --git a/content/ko/docs/tutorials/kubernetes-basics/deploy-app/deploy-intro.html b/content/ko/docs/tutorials/kubernetes-basics/deploy-app/deploy-intro.html index 6cf468f0b6..5b41fe207a 100644 --- a/content/ko/docs/tutorials/kubernetes-basics/deploy-app/deploy-intro.html +++ b/content/ko/docs/tutorials/kubernetes-basics/deploy-app/deploy-intro.html @@ -103,9 +103,7 @@ weight: 10

- 첫 번째 디플로이먼트로, 도커 컨테이너로 패키지된 Node.js 애플리케이션을 사용해보자. - (Node.js 애플리케이션을 작성하고 컨테이너를 활용해서 배포해보지 않았다면, - Hello Minikube 튜토리얼의 지시를 따른다.)

+ 첫 번째 디플로이먼트로, NGINX를 사용해 모든 요청을 에코(echo)하는 도커 컨테이너로 패키지한 hello-node 애플리케이션을 사용해보자. (아직 hello-node 애플리케이션을 작성하고 컨테이너를 활용해서 배포해보지 않았다면, Hello Minikube 튜토리얼의 지시를 따른다.)

이제 디플로이먼트를 이해했으니, 온라인 튜토리얼을 통해 우리의 첫 번째 애플리케이션을 배포해보자!

diff --git a/content/ko/examples/application/deployment.yaml b/content/ko/examples/application/deployment.yaml index 2cd599218d..dbed8bc72b 100644 --- a/content/ko/examples/application/deployment.yaml +++ b/content/ko/examples/application/deployment.yaml @@ -1,4 +1,4 @@ -apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2 +apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment diff --git a/content/ko/examples/application/guestbook/frontend-deployment.yaml b/content/ko/examples/application/guestbook/frontend-deployment.yaml index 50d6e1f0d4..23d64be644 100644 --- a/content/ko/examples/application/guestbook/frontend-deployment.yaml +++ b/content/ko/examples/application/guestbook/frontend-deployment.yaml @@ -1,4 +1,4 @@ -apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2 +apiVersion: apps/v1 kind: Deployment metadata: name: frontend diff --git a/content/ko/examples/application/guestbook/redis-master-deployment.yaml b/content/ko/examples/application/guestbook/redis-master-deployment.yaml index fc6f418c39..478216d1ac 100644 --- a/content/ko/examples/application/guestbook/redis-master-deployment.yaml +++ b/content/ko/examples/application/guestbook/redis-master-deployment.yaml @@ -1,4 +1,4 @@ -apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2 +apiVersion: apps/v1 kind: Deployment metadata: name: redis-master diff --git a/content/ko/examples/application/guestbook/redis-slave-deployment.yaml b/content/ko/examples/application/guestbook/redis-slave-deployment.yaml index 7dcfb6c263..1a7b04386a 100644 --- a/content/ko/examples/application/guestbook/redis-slave-deployment.yaml +++ b/content/ko/examples/application/guestbook/redis-slave-deployment.yaml @@ -1,4 +1,4 @@ -apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2 +apiVersion: apps/v1 kind: Deployment metadata: name: redis-slave diff --git a/content/ko/examples/application/mysql/mysql-deployment.yaml b/content/ko/examples/application/mysql/mysql-deployment.yaml index 518457777e..419fbe03d3 100644 --- a/content/ko/examples/application/mysql/mysql-deployment.yaml +++ b/content/ko/examples/application/mysql/mysql-deployment.yaml @@ -9,7 +9,7 @@ spec: app: mysql clusterIP: None --- -apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2 +apiVersion: apps/v1 kind: Deployment metadata: name: mysql diff --git a/content/ko/examples/application/wordpress/mysql-deployment.yaml b/content/ko/examples/application/wordpress/mysql-deployment.yaml index 8b92b76f54..c6b221512d 100644 --- a/content/ko/examples/application/wordpress/mysql-deployment.yaml +++ b/content/ko/examples/application/wordpress/mysql-deployment.yaml @@ -25,7 +25,7 @@ spec: requests: storage: 20Gi --- -apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2 +apiVersion: apps/v1 kind: Deployment metadata: name: wordpress-mysql diff --git a/content/ko/examples/application/wordpress/wordpress-deployment.yaml b/content/ko/examples/application/wordpress/wordpress-deployment.yaml index d898474211..c8ed239142 100644 --- a/content/ko/examples/application/wordpress/wordpress-deployment.yaml +++ b/content/ko/examples/application/wordpress/wordpress-deployment.yaml @@ -25,7 +25,7 @@ spec: requests: storage: 20Gi --- -apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2 +apiVersion: apps/v1 kind: Deployment metadata: name: wordpress diff --git a/content/ko/examples/service/access/Dockerfile b/content/ko/examples/service/access/Dockerfile index b7b09d492a..61c8ce2831 100644 --- a/content/ko/examples/service/access/Dockerfile +++ b/content/ko/examples/service/access/Dockerfile @@ -1,4 +1,4 @@ FROM nginx:1.17.3 RUN rm /etc/nginx/conf.d/default.conf -COPY frontend.conf /etc/nginx/conf.d +COPY frontend-nginx.conf /etc/nginx/conf.d diff --git a/content/ko/examples/service/access/hello.yaml b/content/ko/examples/service/access/backend-deployment.yaml similarity index 100% rename from content/ko/examples/service/access/hello.yaml rename to content/ko/examples/service/access/backend-deployment.yaml diff --git a/content/ko/examples/service/access/hello-service.yaml b/content/ko/examples/service/access/backend-service.yaml similarity index 100% rename from content/ko/examples/service/access/hello-service.yaml rename to content/ko/examples/service/access/backend-service.yaml diff --git a/content/ko/examples/service/access/frontend-deployment.yaml b/content/ko/examples/service/access/frontend-deployment.yaml new file mode 100644 index 0000000000..182b0e708e --- /dev/null +++ b/content/ko/examples/service/access/frontend-deployment.yaml @@ -0,0 +1,27 @@ +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: frontend +spec: + selector: + matchLabels: + app: hello + tier: frontend + track: stable + replicas: 1 + template: + metadata: + labels: + app: hello + tier: frontend + track: stable + spec: + containers: + - name: nginx + image: "gcr.io/google-samples/hello-frontend:1.0" + lifecycle: + preStop: + exec: + command: ["/usr/sbin/nginx","-s","quit"] +... \ No newline at end of file diff --git a/content/ko/examples/service/access/frontend-nginx.conf b/content/ko/examples/service/access/frontend-nginx.conf new file mode 100644 index 0000000000..39a911a09a --- /dev/null +++ b/content/ko/examples/service/access/frontend-nginx.conf @@ -0,0 +1,14 @@ +# The identifier Backend is internal to nginx, and used to name this specific upstream +upstream Backend { + # hello is the internal DNS name used by the backend Service inside Kubernetes + server hello; +} + +server { + listen 80; + + location / { + # The following statement will proxy traffic to the upstream named Backend + proxy_pass http://Backend; + } +} diff --git a/content/ko/examples/service/access/frontend-service.yaml b/content/ko/examples/service/access/frontend-service.yaml new file mode 100644 index 0000000000..898a5ed51b --- /dev/null +++ b/content/ko/examples/service/access/frontend-service.yaml @@ -0,0 +1,15 @@ +--- +apiVersion: v1 +kind: Service +metadata: + name: frontend +spec: + selector: + app: hello + tier: frontend + ports: + - protocol: "TCP" + port: 80 + targetPort: 80 + type: LoadBalancer +... \ No newline at end of file diff --git a/content/ko/examples/service/access/frontend.conf b/content/ko/examples/service/access/frontend.conf deleted file mode 100644 index 9a1f5a0ed6..0000000000 --- a/content/ko/examples/service/access/frontend.conf +++ /dev/null @@ -1,11 +0,0 @@ -upstream hello { - server hello; -} - -server { - listen 80; - - location / { - proxy_pass http://hello; - } -} diff --git a/content/ko/examples/service/access/frontend.yaml b/content/ko/examples/service/access/frontend.yaml deleted file mode 100644 index 9f5b6b757f..0000000000 --- a/content/ko/examples/service/access/frontend.yaml +++ /dev/null @@ -1,39 +0,0 @@ -apiVersion: v1 -kind: Service -metadata: - name: frontend -spec: - selector: - app: hello - tier: frontend - ports: - - protocol: "TCP" - port: 80 - targetPort: 80 - type: LoadBalancer ---- -apiVersion: apps/v1 -kind: Deployment -metadata: - name: frontend -spec: - selector: - matchLabels: - app: hello - tier: frontend - track: stable - replicas: 1 - template: - metadata: - labels: - app: hello - tier: frontend - track: stable - spec: - containers: - - name: nginx - image: "gcr.io/google-samples/hello-frontend:1.0" - lifecycle: - preStop: - exec: - command: ["/usr/sbin/nginx","-s","quit"] From bcd2f0ec2a5e01c4b1f0b73d83827c4464fe7cf8 Mon Sep 17 00:00:00 2001 From: Arhell Date: Fri, 29 Jan 2021 01:21:31 +0200 Subject: [PATCH 100/190] [ja] fix ttlafterfinished.md link --- .../ja/docs/concepts/workloads/controllers/ttlafterfinished.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ja/docs/concepts/workloads/controllers/ttlafterfinished.md b/content/ja/docs/concepts/workloads/controllers/ttlafterfinished.md index f863cb4e92..728e0346fe 100644 --- a/content/ja/docs/concepts/workloads/controllers/ttlafterfinished.md +++ b/content/ja/docs/concepts/workloads/controllers/ttlafterfinished.md @@ -52,4 +52,4 @@ Kubernetesにおいてタイムスキューを避けるために、全てのNode * [Jobの自動クリーンアップ](/ja/docs/concepts/workloads/controllers/job/#clean-up-finished-jobs-automatically) -* [設計ドキュメント](https://github.com/kubernetes/enhancements/blob/master/keps/sig-apps/0026-ttl-after-finish.md) +* [設計ドキュメント](https://github.com/kubernetes/enhancements/blob/master/keps/sig-apps/592-ttl-after-finish/README.md) From 6f777c9e64c0a4116866fdde10f66879e1eac170 Mon Sep 17 00:00:00 2001 From: Weiping Cai Date: Wed, 27 Jan 2021 11:03:58 +0800 Subject: [PATCH 101/190] fix pid pressure Signed-off-by: Weiping Cai --- content/en/docs/tasks/administer-cluster/out-of-resource.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/content/en/docs/tasks/administer-cluster/out-of-resource.md b/content/en/docs/tasks/administer-cluster/out-of-resource.md index c45a773c45..f750dd2585 100644 --- a/content/en/docs/tasks/administer-cluster/out-of-resource.md +++ b/content/en/docs/tasks/administer-cluster/out-of-resource.md @@ -117,9 +117,10 @@ The `kubelet` has the following default hard eviction threshold: * `memory.available<100Mi` * `nodefs.available<10%` -* `nodefs.inodesFree<5%` * `imagefs.available<15%` +On a Linux node, the default value also includes `nodefs.inodesFree<5%`. + ### Eviction Monitoring Interval The `kubelet` evaluates eviction thresholds per its configured housekeeping interval. @@ -140,6 +141,7 @@ The following node conditions are defined that correspond to the specified evict |-------------------|---------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------| | `MemoryPressure` | `memory.available` | Available memory on the node has satisfied an eviction threshold | | `DiskPressure` | `nodefs.available`, `nodefs.inodesFree`, `imagefs.available`, or `imagefs.inodesFree` | Available disk space and inodes on either the node's root filesystem or image filesystem has satisfied an eviction threshold | +| `PIDPressure` | `pid.available` | Available processes identifiers on the (Linux) node has fallen below an eviction threshold | | The `kubelet` continues to report node status updates at the frequency specified by `--node-status-update-frequency` which defaults to `10s`. From a25e0c5cc305f7cf80132b3e2964d3f74fb12618 Mon Sep 17 00:00:00 2001 From: chenxuc Date: Fri, 29 Jan 2021 13:46:12 +0800 Subject: [PATCH 102/190] [zh] Sync web page for port forward This is for part of work in the umbrella issue: [zh] Umbrella issue: pages out of sync in tasks section #26178 Portforward (L) content/zh/docs/tasks/access-application-cluster/port-forward-access-application-cluster.md --- ...port-forward-access-application-cluster.md | 54 +++++++++++++++++-- 1 file changed, 49 insertions(+), 5 deletions(-) diff --git a/content/zh/docs/tasks/access-application-cluster/port-forward-access-application-cluster.md b/content/zh/docs/tasks/access-application-cluster/port-forward-access-application-cluster.md index 06791739d7..4219ca4f20 100644 --- a/content/zh/docs/tasks/access-application-cluster/port-forward-access-application-cluster.md +++ b/content/zh/docs/tasks/access-application-cluster/port-forward-access-application-cluster.md @@ -213,9 +213,22 @@ for database debugging. 以上所有命令都应该有效。输出应该类似于： ``` - I0710 14:43:38.274550 3655 portforward.go:225] Forwarding from 127.0.0.1:7000 -> 6379 - I0710 14:43:38.274797 3655 portforward.go:225] Forwarding from [::1]:7000 -> 6379 + Forwarding from 127.0.0.1:7000 -> 6379 + Forwarding from [::1]:7000 -> 6379 ``` + +{{< note >}} + +`kubectl port-forward` 不会返回。你需要打开另一个终端来继续这个练习。 + +{{< /note >}} + - 成功的 ping 请求应该返回 PONG。 + 成功的 ping 请求应该返回： + + ``` + PONG + ``` + +### （可选操作）让 _kubectl_ 来选择本地端口 {#let-kubectl-choose-local-port} + + +如果你不需要指定特定的本地端口，你可以让 `kubectl` 来选择和分配本地端口， +以便你不需要管理本地端口冲突。该命令使用稍微不同的语法： + +```shell +kubectl port-forward deployment/redis-master :6379 +``` + +`kubectl` 工具会找到一个未被使用的本地端口号（避免使用低段位的端口号，因为他们可能会被其他应用程序使用）。输出应该类似于： + +``` +Forwarding from 127.0.0.1:62162 -> 6379 +Forwarding from [::1]:62162 -> 6379 +``` + From 14afc4dd35c2de4a419f873d9dfebc5468bb5624 Mon Sep 17 00:00:00 2001 From: Qiming Teng Date: Wed, 20 Jan 2021 22:11:48 +0800 Subject: [PATCH 103/190] [zh] Translate kubelet-tls-bootstrapping This page, to my surprise, was not localized. --- .../kubelet-tls-bootstrapping.md | 971 +++++++++++++++--- 1 file changed, 830 insertions(+), 141 deletions(-) diff --git a/content/zh/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping.md b/content/zh/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping.md index a70f08bea1..60f3772c07 100644 --- a/content/zh/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping.md +++ b/content/zh/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping.md @@ -1,148 +1,558 @@ --- -approvers: -- ericchiang +title: TLS 启动引导 +content_type: concept +--- + -{{< toc >}} + -## Overview + +在一个 Kubernetes 集群中，工作节点上的组件（kubelet 和 kube-proxy）需要与 +Kubernetes 主控组件通信，尤其是 kube-apiserver。 +为了确保通信本身是私密的、不被干扰，并且确保集群的每个组件都在与另一个 +可信的组件通信，我们强烈建议使用节点上的客户端 TLS 证书。 -This document describes how to set up TLS client certificate bootstrapping for kubelets. -Kubernetes 1.4 introduced an API for requesting certificates from a cluster-level Certificate Authority (CA). The original intent of this API is to enable provisioning of TLS client certificates for kubelets. The proposal can be found [here](https://github.com/kubernetes/kubernetes/pull/20439) -and progress on the feature is being tracked as [feature #43](https://github.com/kubernetes/features/issues/43). + +启动引导这些组件的正常过程，尤其是需要证书来与 kube-apiserver 安全通信的 +工作节点，可能会是一个具有挑战性的过程，因为这一过程通常不受 Kubernetes 控制， +需要不少额外工作。 +这也使得初始化或者扩缩一个集群的操作变得具有挑战性。 + +为了简化这一过程，从 1.4 版本开始，Kubernetes 引入了一个证书请求和签名 +API 以便简化此过程。该提案可在 +[这里](https://github.com/kubernetes/kubernetes/pull/20439)看到。 + +本文档描述节点初始化的过程，如何为 kubelet 配置 TLS 客户端证书启动引导， +以及其背后的工作原理。 + + + + +## 初始化过程 {#initialization-process} + +当工作节点启动时，kubelet 执行以下操作： + + +1. 寻找自己的 `kubeconfig` 文件 +2. 检索 API 服务器的 URL 和凭据，通常是来自 `kubeconfig` 文件中的 + TLS 密钥和已签名证书 +3. 尝试使用这些凭据来与 API 服务器通信 + + +假定 kube-apiserver 成功地认证了 kubelet 的凭据数据，它会将 kubelet 视为 +一个合法的节点并开始将 Pods 分派给该节点。 + +注意，签名的过程依赖于： + +* `kubeconfig` 中包含密钥和本地主机的证书 +* 证书被 kube-apiserver 所信任的一个证书机构（CA）所签名 + + +负责部署和管理集群的人有以下责任： + +1. 创建 CA 密钥和证书 +2. 将 CA 证书发布到 kube-apiserver 运行所在的主控节点上 +3. 为每个 kubelet 创建密钥和证书；强烈建议为每个 kubelet 使用独一无二的、 + CN 取值与众不同的密钥和证书 +4. 使用 CA 密钥对 kubelet 证书签名 +5. 将 kubelet 密钥和签名的证书发布到 kubelet 运行所在的特定节点上 + +本文中描述的 TLS 启动引导过程有意简化甚至完全自动化上述过程，尤其是 +第三步之后的操作，因为这些步骤是初始化或者扩缩集群时最常见的操作。 + + +### 启动引导初始化 {#bootstrap-initialization} + +在启动引导初始化过程中，会发生以下事情： + + +1. kubelet 启动 +2. kubelet 看到自己 *没有* 对应的 `kubeconfig` 文件 +3. kubelet 搜索并发现 `bootstrap-kubeconfig` 文件 +4. kubelet 读取该启动引导文件，从中获得 API 服务器的 URL 和用途有限的 + 一个“令牌（Token）” +5. kubelet 建立与 API 服务器的连接，使用上述令牌执行身份认证 +6. kubelet 现在拥有受限制的凭据来创建和取回证书签名请求（CSR） +7. kubelet 为自己创建一个 CSR，并将其 signerName 设置为 `kubernetes.io/kube-apiserver-client-kubelet` +8. CSR 被以如下两种方式之一批复： + * 如果配置了，kube-controller-manager 会自动批复该 CSR + * 如果配置了，一个外部进程，或者是人，使用 Kubernetes API 或者使用 `kubectl` + 来批复该 CSR +9. kubelet 所需要的证书被创建 + +10. 证书被发放给 kubelet +11. kubelet 取回该证书 +12. kubelet 创建一个合适的 `kubeconfig`，其中包含密钥和已签名的证书 +13. kubelet 开始正常操作 +14. 可选地，如果配置了，kubelet 在证书接近于过期时自动请求更新证书 +15. 更新的证书被批复并发放；取决于配置，这一过程可能是自动的或者手动完成 + + +本文的其余部分描述配置 TLS 启动引导的必要步骤及其局限性。 + + +## 配置 {#configuration} + +要配置 TLS 启动引导及可选的自动批复，你必须配置以下组件的选项： + +* kube-apiserver +* kube-controller-manager +* kubelet +* 集群内的资源：`ClusterRoleBinding` 以及可能需要的 `ClusterRole` + +此外，你需要有 Kubernetes 证书机构（Certificate Authority，CA）。 + + +## 证书机构 {#certificate-authority} + +就像在没有启动引导的情况下，你会需要证书机构（CA）密钥和证书。 +这些数据会被用来对 kubelet 证书进行签名。 +如前所述，将证书机构密钥和证书发布到主控节点是你的责任。 + + +就本文而言，我们假定这些数据被发布到主控节点上的 +`/var/lib/kubernetes/ca.pem`（证书）和 +`/var/lib/kubernetes/ca-key.pem`（密钥）文件中。 +我们将这两个文件称作“Kubernetes CA 证书和密钥”。 +所有 Kubernetes 组件（kubelet、kube-apiserver、kube-controller-manager）都使用 +这些凭据，并假定这里的密钥和证书都是 PEM 编码的。 + + +## kube-apiserver 配置 {#kube-apiserver-configuration} -While any authentication strategy can be used for the kubelet's initial bootstrap credentials, the following two authenticators are recommended for ease of provisioning. +启用 TLS 启动引导对 kube-apiserver 有若干需求： -1. [Bootstrap Tokens](/docs/admin/bootstrap-tokens/) - __alpha__ -2. [Token authentication file](###token-authentication-file) +* 能够识别对客户端证书进行签名的 CA +* 能够对启动引导的 kubelet 执行身份认证，并将其置入 `system:bootstrappers` 组 +* 能够对启动引导的 kubelet 执行鉴权操作，允许其创建证书签名请求（CSR） -Using bootstrap tokens is currently __alpha__ and will simplify the management of bootstrap token management especially in a HA scenario. + +### 识别客户证书 {#recognizing-client-certificates} -`head -c 16 /dev/urandom | od -An -t x | tr -d ' '` +对于所有客户端证书的认证操作而言，这是很常见的。 +如果还没有设置，要为 kube-apiserver 命令添加 `--client-ca-file=FILENAME` +标志来启用客户端证书认证，在标志中引用一个包含用来签名的证书的证书机构包， +例如：`--client-ca-file=/var/lib/kubernetes/ca.pem`。 -will generate tokens that look like `02b50b05283e98dd0fd71db496ef01e8` + +### 初始启动引导认证 {#initial-bootstrap-authentication} + +为了让启动引导的 kubelet 能够连接到 kube-apiserver 并请求证书， +它必须首先在服务器上认证自身身份。你可以使用任何一种能够对 kubelet 执行身份认证的 +[身份认证组件](/zh/docs/reference/access-authn-authz/authentication/)。 + + +尽管所有身份认证策略都可以用来对 kubelet 的初始启动凭据来执行认证， +出于容易准备的因素，建议使用如下两个身份认证组件： + +1. [启动引导令牌（Bootstrap Token）](#bootstrap-tokens) +2. [令牌认证文件](#token-authentication-file) + + +启动引导令牌是一种对 kubelet 进行身份认证的方法，相对简单且容易管理， +且不需要在启动 kube-apiserver 时设置额外的标志。 +启动引导令牌从 Kubernetes 1.12 开始是一种 __Beta__ 功能特性。 + + +无论选择哪种方法，这里的需求是 kubelet 能够被身份认证为某个具有如下权限的用户： + +1. 创建和读取 CSR +2. 在启用了自动批复时，能够在请求节点客户端证书时得到自动批复 + + +使用启动引导令牌执行身份认证的 kubelet 会被认证为 `system:bootstrappers` +组中的用户。这是使用启动引导令牌的一种标准方法。 + + +随着这个功能特性的逐渐成熟，你需要确保令牌绑定到某基于角色的的访问控制（RBAC） +策略上，从而严格限制请求（使用[启动引导令牌](/zh/docs/reference/access-authn-authz/bootstrap-tokens/)） +仅限于客户端申请提供证书。当 RBAC 被配置启用时，可以将令牌限制到某个组，从而 +提高灵活性。例如，你可以在准备节点期间禁止某特定启动引导组的访问。 + + +#### 启动引导令牌 {#bootstrap-tokens} + +启动引导令牌的细节在[这里](/zh/docs/reference/access-authn-authz/bootstrap-tokens/) +详述。启动引导令牌在 Kubernetes 集群中存储为 Secret 对象，被发放给各个 kubelet。 +你可以在整个集群中使用同一个令牌，也可以为每个节点发放单独的令牌。 + + +这一过程有两个方面： + +1. 基于令牌 ID、机密数据和范畴信息创建 Kubernetes Secret +2. 将令牌发放给 kubelet + + +从 kubelet 的角度，所有令牌看起来都很像，没有特别的含义。 +从 kube-apiserver 服务器的角度，启动引导令牌是很特殊的。 +根据其 `type`、`namespace` 和 `name`，kube-apiserver 能够将认作特殊的令牌， +并授予携带该令牌的任何人特殊的启动引导权限，换言之，将其视为 +`system:bootstrappers` 组的成员。这就满足了 TLS 启动引导的基本需求。 + + +关于创建 Secret 的进一步细节可访问[这里](/zh/docs/reference/access-authn-authz/bootstrap-tokens/)。 + +如果你希望使用启动引导令牌，你必须在 kube-apiserver 上使用下面的标志启用之： + +``` +--enable-bootstrap-token-auth=true +``` + + +#### 令牌认证文件 {#token-authentication-file} + +kube-apiserver 能够将令牌视作身份认证依据。 +这些令牌可以是任意数据，但必须表示为基于某安全的随机数生成器而得到的 +至少 128 位混沌数据。这里的随机数生成器可以是现代 Linux 系统上的 +`/dev/urandom`。生成令牌的方式有很多种。例如： + +``` +head -c 16 /dev/urandom | od -An -t x | tr -d ' ' +``` + + +上面的命令会生成类似于 `02b50b05283e98dd0fd71db496ef01e8` 这样的令牌。 + +令牌文件看起来是下面的例子这样，其中前面三个值可以是任何值，用引号括起来 +的组名称则只能用例子中给的值。 ``` 02b50b05283e98dd0fd71db496ef01e8,kubelet-bootstrap,10001,"system:bootstrappers" ``` -Add the `--token-auth-file=FILENAME` flag to the kube-apiserver command (in your systemd unit file perhaps) to enable the token file. -See docs [here](/docs/admin/authentication/#static-token-file) for further details. + +向 kube-apiserver 添加 `--token-auth-file=FILENAME` 标志（或许这要对 systemd +的单元文件作修改）以启用令牌文件。参见 +[这里](/zh/docs/reference/access-authn-authz/authentication/#static-token-file) +的文档以了解进一步的细节。 -### Client certificate CA bundle + +### 授权 kubelet 创建 CSR {#authorize-kubelet-to-create-csr} + +现在启动引导节点被身份认证为 `system:bootstrapping` 组的成员，它需要被 _授权_ +创建证书签名请求（CSR）并在证书被签名之后将其取回。 +幸运的是，Kubernetes 提供了一个 `ClusterRole`，其中精确地封装了这些许可， +`system:node-bootstrapper`。 + +为了实现这一点，你只需要创建 `ClusterRoleBinding`，将 `system:bootstrappers` +组绑定到集群角色 `system:node-bootstrapper`。 + +``` +# 允许启动引导节点创建 CSR +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: create-csrs-for-bootstrapping +subjects: +- kind: Group + name: system:bootstrappers + apiGroup: rbac.authorization.k8s.io +roleRef: + kind: ClusterRole + name: system:node-bootstrapper + apiGroup: rbac.authorization.k8s.io +``` + + +## kube-controller-manager 配置 {#kube-controller-manager-configuration} -The kube-controller-manager flags are: +API 服务器从 kubelet 收到证书请求并对这些请求执行身份认证，但真正负责发放 +签名证书的是控制器管理器。 + + +控制器管理器通过一个证书发放的控制回路来执行此操作。该操作的执行方式是使用磁盘上 +的文件用 [cfssl](https://blog.cloudflare.com/introducing-cfssl/) 本地签名组件 +来完成。目前，所发放的所有证书都有一年的有效期，并设定了默认的一组密钥用法。 + + +为了让控制器管理器对证书签名，它需要： + +* 能够访问你之前所创建并分发的“Kubernetes CA 密钥和证书” +* 启用 CSR 签名 + + +### 访问密钥和证书 {#access-to-key-and-certificate} + +如前所述，你需要创建一个 Kubernetes CA 密钥和证书，并将其发布到主控节点。 +这些数据会被控制器管理器来对 kubelet 证书进行签名。 + + +由于这些被签名的证书反过来会被 kubelet 用来在 kube-apiserver 执行普通的 +kubelet 身份认证，很重要的一点是为控制器管理器所提供的 CA 也被 kube-apiserver +信任用来执行身份认证。CA 密钥和证书是通过 kube-apiserver 的标志 +`--client-ca-file=FILENAME`（例如，`--client-ca-file=/var/lib/kubernetes/ca.pem`)， +来设定的，正如 kube-apiserver 配置节所述。 + +要将 Kubernetes CA 密钥和证书提供给 kube-controller-manager，可使用以下标志： ``` --cluster-signing-cert-file="/etc/path/to/kubernetes/ca/ca.crt" --cluster-signing-key-file="/etc/path/to/kubernetes/ca/ca.key" ``` -### Approval controller - -In 1.7 the experimental "group auto approver" controller is dropped in favor of the new `csrapproving` controller -that ships as part of [kube-controller-manager](/docs/admin/kube-controller-manager/) and is enabled by default. -The controller uses the [`SubjectAccessReview` API](/docs/admin/authorization/#checking-api-access) to determine -if a given user is authorized to request a CSR, then approves based on the authorization outcome. To prevent -conflicts with other approvers, the builtin approver doesn't explicitly deny CSRs, only ignoring unauthorized requests. - -The controller categorizes CSRs into three subresources: - -1. `nodeclient` - a request by a user for a client certificate with `O=system:nodes` and `CN=system:node:(node name)`. -2. `selfnodeclient` - a node renewing a client certificate with the same `O` and `CN`. -3. `selfnodeserver` - a node renewing a serving certificate. (ALPHA, requires feature gate) - -The checks to determine if a CSR is a `selfnodeserver` request is currently tied to the kubelet's credential rotation -implementation, an __alpha__ feature. As such, the definition of `selfnodeserver` will likely change in a future and -requires the `RotateKubeletServerCertificate` feature gate on the controller manager. The feature progress can be -tracked at [kubernetes/features#267](https://github.com/kubernetes/features/issues/267). + +例如： ``` ---feature-gates=RotateKubeletServerCertificate=true +--cluster-signing-cert-file="/var/lib/kubernetes/ca.pem" --cluster-signing-key-file="/var/lib/kubernetes/ca-key.pem" ``` -The following RBAC `ClusterRoles` represent the `nodeclient`, `selfnodeclient`, and `selfnodeserver` capabilities. Similar roles -may be automatically created in future releases. + +所签名的证书的合法期限可以通过下面的标志来配置： -```yml -# A ClusterRole which instructs the CSR approver to approve a user requesting -# node client credentials. -kind: ClusterRole +``` +--cluster-signing-duration +``` + + +### 批复 {#approval} + +为了对 CSR 进行批复，你需要告诉控制器管理器批复这些 CSR 是可接受的。 +这是通过将 RBAC 访问权限授予正确的组来实现的。 + + +许可权限有两组： + +* `nodeclient`：如果节点在为节点创建新的证书，则该节点还没有证书。该节点 + 使用前文所列的令牌之一来执行身份认证，因此是组 `system:bootstrappers` 组 + 的成员。 +* `selfnodeclient`：如果节点在对证书执行续期操作，则该节点已经拥有一个证书。 + 节点持续使用现有的证书将自己认证为 `system:nodes` 组的成员。 + + +要允许 kubelet 请求并接收新的证书，可以创建一个 `ClusterRoleBinding` 将启动 +引导节点所处的组 `system:bootstrappers` 绑定到为其赋予访问权限的 +`ClusterRole` `system:certificates.k8s.io:certificatesigningrequests:nodeclient`： + +```yaml +# 批复 "system:bootstrappers" 组的所有 CSR apiVersion: rbac.authorization.k8s.io/v1 -metadata: - name: approve-node-client-csr -rules: -- apiGroups: ["certificates.k8s.io"] - resources: ["certificatesigningrequests/nodeclient"] - verbs: ["create"] ---- -# A ClusterRole which instructs the CSR approver to approve a node renewing its -# own client credentials. -kind: ClusterRole -apiVersion: rbac.authorization.k8s.io/v1 -metadata: - name: approve-node-client-renewal-csr -rules: -- apiGroups: ["certificates.k8s.io"] - resources: ["certificatesigningrequests/selfnodeclient"] - verbs: ["create"] ---- -# A ClusterRole which instructs the CSR approver to approve a node requesting a -# serving cert matching its client cert. -kind: ClusterRole -apiVersion: rbac.authorization.k8s.io/v1 -metadata: - name: approve-node-server-renewal-csr -rules: -- apiGroups: ["certificates.k8s.io"] - resources: ["certificatesigningrequests/selfnodeserver"] - verbs: ["create"] -``` - -These powers can be granted to credentials, such as bootstrapping tokens. For example, to replicate the behavior -provided by the removed auto-approval flag, of approving all CSRs by a single group: - -``` -# REMOVED: This flag no longer works as of 1.7. ---insecure-experimental-approve-all-kubelet-csrs-for-group="system:bootstrappers" -``` - -An admin would create a `ClusterRoleBinding` targeting that group. - -```yml -# Approve all CSRs for the group "system:bootstrappers" kind: ClusterRoleBinding -apiVersion: rbac.authorization.k8s.io/v1 metadata: name: auto-approve-csrs-for-group subjects: @@ -151,66 +561,345 @@ subjects: apiGroup: rbac.authorization.k8s.io roleRef: kind: ClusterRole - name: approve-node-client-csr + name: system:certificates.k8s.io:certificatesigningrequests:nodeclient apiGroup: rbac.authorization.k8s.io ``` -To let a node renew its own credentials, an admin can construct a `ClusterRoleBinding` targeting -that node's credentials: + +要允许 kubelet 对其客户端证书执行续期操作，可以创建一个 `ClusterRoleBinding` +将正常工作的节点所处的组 `system:nodes` 绑定到为其授予访问许可的 `ClusterRole` +`system:certificates.k8s.io:certificatesigningrequests:selfnodeclient`： -```yml -kind: ClusterRoleBinding +```yaml +# 批复 "system:nodes" 组的 CSR 续约请求 apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding metadata: - name: node1-client-cert-renewal + name: auto-approve-renewals-for-nodes subjects: -- kind: User - name: system:node:node-1 # Let "node-1" renew its client certificate. +- kind: Group + name: system:nodes apiGroup: rbac.authorization.k8s.io roleRef: kind: ClusterRole - name: approve-node-client-renewal-csr + name: system:certificates.k8s.io:certificatesigningrequests:selfnodeclient apiGroup: rbac.authorization.k8s.io ``` -Deleting the binding will prevent the node from renewing its client credentials, effectively -removing it from the cluster once its certificate expires. + +作为 [kube-controller-manager](/zh/docs/reference/generated/kube-controller-manager/) +的一部分的 `csrapproving` 控制器是自动被启用的。 +该控制器使用 [`SubjectAccessReview` API](/docs/reference/access-authn-authz/authorization/#checking-api-access) +来确定是否某给定用户被授权请求 CSR，之后基于鉴权结果执行批复操作。 +为了避免与其它批复组件发生冲突，内置的批复组件不会显式地拒绝任何 CSRs。 +该组件仅是忽略未被授权的请求。 +控制器也会作为垃圾收集的一部分清除已过期的证书。 + +## kubelet 配置 {#kubelet-configuration} + +最后，当主控节点被正确配置并且所有必要的身份认证和鉴权机制都就绪时， +我们可以配置 kubelet。 + + +kubelet 需要以下配置来执行启动引导： + +* 一个用来存储所生成的密钥和证书的路径（可选，可以使用默认配置） +* 一个用来指向尚不存在的 `kubeconfig` 文件的路径；kubelet 会将启动引导 + 配置文件放到这个位置 +* 一个指向启动引导 `kubeconfig` 文件的路径，用来提供 API 服务器的 URL + 和启动引导凭据，例如，启动引导令牌 +* 可选的：轮换证书的指令 + + +启动引导 `kubeconfig` 文件应该放在一个 kubelet 可访问的路径下，例如 +`/var/lib/kubelet/bootstrap-kubeconfig`。 + +其格式与普通的 `kubeconfig` 文件完全相同。实例文件可能看起来像这样： + +```yaml +apiVersion: v1 +kind: Config +clusters: +- cluster: + certificate-authority: /var/lib/kubernetes/ca.pem + server: https://my.server.example.com:6443 + name: bootstrap +contexts: +- context: + cluster: bootstrap + user: kubelet-bootstrap + name: bootstrap +current-context: bootstrap +preferences: {} +users: +- name: kubelet-bootstrap + user: + token: 07401b.f395accd246ae52d ``` -When starting the kubelet, if the file specified by `--kubeconfig` does not exist, the bootstrap kubeconfig is used to request a client certificate from the API server. On approval of the certificate request and receipt back by the kubelet, a kubeconfig file referencing the generated key and obtained certificate is written to the path specified by `--kubeconfig`. The certificate and key file will be placed in the directory specified by `--cert-dir`. + +需要额外注意的一些因素有： + +* `certificate-authority`：指向 CA 文件的路径，用来对 kube-apiserver 所出示 + 的服务器证书进行验证 +* `server`： 用来访问 kube-apiserver 的 URL +* `token`：要使用的令牌 + + +令牌的格式并不重要，只要它与 kube-apiserver 的期望匹配即可。 +在上面的例子中，我们使用的是启动引导令牌。 +如前所述，任何合法的身份认证方法都可以使用，不限于令牌。 + +因为启动引导 `kubeconfig` 文件是一个标准的 `kubeconfig` 文件，你可以使用 +`kubectl` 来生成该文件。要生成上面的示例文件： + +``` +kubectl config --kubeconfig=/var/lib/kubelet/bootstrap-kubeconfig set-cluster bootstrap --server='https://my.server.example.com:6443' --certificate-authority=/var/lib/kubernetes/ca.pem +kubectl config --kubeconfig=/var/lib/kubelet/bootstrap-kubeconfig set-credentials kubelet-bootstrap --token=07401b.f395accd246ae52d +kubectl config --kubeconfig=/var/lib/kubelet/bootstrap-kubeconfig set-context bootstrap --user=kubelet-bootstrap --cluster=bootstrap +kubectl config --kubeconfig=/var/lib/kubelet/bootstrap-kubeconfig use-context bootstrap +``` + + +要指示 kubelet 使用启动引导 `kubeconfig` 文件，可以使用下面的 kubelet 标志： + +``` +--bootstrap-kubeconfig="/var/lib/kubelet/bootstrap-kubeconfig" --kubeconfig="/var/lib/kubelet/kubeconfig" +``` + + +在启动 kubelet 时，如果 `--kubeconfig` 标志所指定的文件并不存在，会使用通过标志 +`--bootstrap-kubeconfig` 所指定的启动引导 kubeconfig 配置来向 API 服务器请求 +客户端证书。在证书请求被批复并被 kubelet 收回时，一个引用所生成的密钥和所获得 +证书的 kubeconfig 文件会被写入到通过 `--kubeconfig` 所指定的文件路径下。 +证书和密钥文件会被放到 `--cert-dir` 所指定的目录中。 + + +### 客户和服务证书 {#client-and-serving-certificates} + +前文所述的内容都与 kubelet _客户端_ 证书相关，尤其是 kubelet 用来向 +kube-apiserver 认证自身身份的证书。 + + +kubelet 也可以使用 _服务（Serving）_ 证书。kubelet 自身向外提供一个 +HTTPS 末端，包含若干功能特性。要保证这些末端的安全性，kubelet 可以执行以下操作 +之一： + +* 使用通过 `--tls-private-key-file` 和 `--tls-cert-file` 所设置的密钥和证书 +* 如果没有提供密钥和证书，则创建自签名的密钥和证书 +* 通过 CSR API 从集群服务器请求服务证书 + + +TLS 启动引导所提供的客户端证书默认被签名为仅用于 `client auth`（客户端认证）， +因此不能作为提供服务的证书，或者 `server auth`。 + +不过，你可以启用服务器证书，至少可以部分地通过证书轮换来实现这点。 + + +### 证书轮换 {#certificate-rotation} + +Kubernetes v1.8 和更高版本的 kubelet 实现了对客户端证书与/或服务证书进行轮换 +这一 Beta 特性。这一特性通过 kubelet 对应的 `RotateKubeletClientCertificate` 和 +`RotateKubeletServerCertificate` 特性门控标志来控制，并且是默认启用的。 + + +`RotateKubeletClientCertificate` 会导致 kubelet 在其现有凭据即将过期时通过 +创建新的 CSR 来轮换其客户端证书。要启用此功能特性，可将下面的标志传递给 +kubelet： + +``` +--rotate-certificates +``` + + +`RotateKubeletServerCertificate` 会让 kubelet 在启动引导其客户端凭据之后请求 +一个服务证书 **且** 对该服务证书执行轮换操作。要启用此功能特性，将下面的标志 +传递给 kubelet： + +``` +--rotate-server-certificates +``` {{< note >}} -The following flags are required to enable this bootstrapping when starting the kubelet: + +Kubernetes 核心中所实现的 CSR 批复控制器出于 +[安全原因](https://github.com/kubernetes/community/pull/1982) +并不会自动批复节点的 _服务_ 证书。 +要使用 `RotateKubeletServerCertificate` 功能特性，集群运维人员需要运行一个 +定制的控制器或者手动批复服务证书的请求。 -``` ---bootstrap-kubeconfig="/path/to/bootstrap/kubeconfig" -``` + +对 kubelet 服务证书的批复过程因集群部署而异，通常应该仅批复如下 CSR： + +1. 由节点发出的请求（确保 `spec.username` 字段形式为 `system:node:` + 且 `spec.groups` 包含 `system:nodes`） +2. 请求中包含服务证书用法（确保 `spec.usages` 中包含 `server auth`，可选地也可 + 包含 `digital signature` 和 `key encipherment`，且不包含其它用法） +3. 仅包含隶属于请求节点的 IP 和 DNS 的 `subjectAltNames`，没有 URI 和 Email + 形式的 `subjectAltNames`（解析 `spec.request` 中的 x509 证书签名请求可以 + 检查 `subjectAltNames`） {{< /note >}} -Additionally, in 1.7 the kubelet implements __alpha__ features for enabling rotation of both its client and/or serving certs. -These can be enabled through the respective `RotateKubeletClientCertificate` and `RotateKubeletServerCertificate` feature -flags on the kubelet, but may change in backward incompatible ways in future releases. + +## 其它身份认证组件 {#other-authenticating-components} -`RotateKubeletClientCertificate` causes the kubelet to rotate its client certificates by creating new CSRs as its existing -credentials expire. `RotateKubeletServerCertificate` causes the kubelet to both request a serving certificate after -bootstrapping its client credentials and rotate the certificate. The serving cert currently does not request DNS or IP -SANs. +本文所描述的所有 TLS 启动引导内容都与 kubelet 相关。不过，其它组件也可能需要 +直接与 kube-apiserver 直接通信。容易想到的是 kube-proxy，同样隶属于 +Kubernetes 的控制面并且运行在所有节点之上，不过也可能包含一些其它负责 +监控或者联网的组件。 + +与 kubelet 类似，这些其它组件也需要一种向 kube-apiserver 认证身份的方法。 +你可以用几种方法来生成这类凭据： + + +* 较老的方式：和 kubelet 在 TLS 启动引导之前所做的一样，用类似的方式 + 创建和分发证书 +* DaemonSet：由于 kubelet 自身被加载到所有节点之上，并且有足够能力来启动基本服务， + 你可以运行将 kube-proxy 和其它特定节点的服务作为 `kube-system` 名字空间中的 + DaemonSet 来执行，而不是独立的进程。由于 DaemonSet 位于集群内部，你可以为其 + 指派一个合适的服务账户，使之具有适当的访问权限来完成其使命。这也许是配置此类 + 服务的最简单的方法。 + + +## kubectl 批复 {#kubectl-approval} + +CSRs 可以在控制器管理其内置的批复工作流之外被批复。 + + +签名控制器并不会立即对所有证书请求执行签名操作。相反，它会等待这些请求被某 +具有适当特权的用户标记为 “Approved（已批准）”状态。 +这一流程有意允许由外部批复控制器来自动执行的批复，或者由控制器管理器内置的 +批复控制器来自动批复。 +不过，集群管理员也可以使用 `kubectl` 来手动批准证书请求。 +管理员可以通过 `kubectl get csr` 来列举所有的 CSR，使用 +`kubectl descsribe csr ` 来描述某个 CSR 的细节。 +管理员可以使用 `kubectl certificate approve ` 来拒绝某 CSR。 + From c6479b8144515ab7496561e3ad9cca335a93431d Mon Sep 17 00:00:00 2001 From: hunomina Date: Fri, 29 Jan 2021 11:38:07 +0100 Subject: [PATCH 104/190] Fix typo --- .../docs/tasks/configure-pod-container/assign-cpu-resource.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/fr/docs/tasks/configure-pod-container/assign-cpu-resource.md b/content/fr/docs/tasks/configure-pod-container/assign-cpu-resource.md index 0845cf2806..284024b425 100644 --- a/content/fr/docs/tasks/configure-pod-container/assign-cpu-resource.md +++ b/content/fr/docs/tasks/configure-pod-container/assign-cpu-resource.md @@ -114,7 +114,7 @@ cpu-demo 974m Souvenez-vous qu'en réglant `-cpu "2"`, vous avez configuré le conteneur pour faire en sorte qu'il utilise 2 CPU, mais que le conteneur ne peut utiliser qu'environ 1 CPU. L'utilisation du CPU du conteneur est entravée, car le conteneur tente d'utiliser plus de ressources CPU que sa limite. {{< note >}} -Une autre explication possible de la la restriction du CPU est que le Nœud pourrait ne pas avoir +Une autre explication possible de la restriction du CPU est que le Nœud pourrait ne pas avoir suffisamment de ressources CPU disponibles. Rappelons que les conditions préalables à cet exercice exigent que chacun de vos Nœuds doit avoir au moins 1 CPU. Si votre conteneur fonctionne sur un nœud qui n'a qu'un seul CPU, le conteneur ne peut pas utiliser plus que 1 CPU, quelle que soit la limite de CPU spécifiée pour le conteneur. {{< /note >}} From 4c6533fe0420c77a011dcd1aec5ec904361a7dda Mon Sep 17 00:00:00 2001 From: Qiming Teng Date: Fri, 29 Jan 2021 21:21:38 +0800 Subject: [PATCH 105/190] [zh] Resync stateful application task --- .../run-replicated-stateful-application.md | 95 ++++++++++--------- 1 file changed, 49 insertions(+), 46 deletions(-) diff --git a/content/zh/docs/tasks/run-application/run-replicated-stateful-application.md b/content/zh/docs/tasks/run-application/run-replicated-stateful-application.md index d41f51a32c..05503836ab 100644 --- a/content/zh/docs/tasks/run-application/run-replicated-stateful-application.md +++ b/content/zh/docs/tasks/run-application/run-replicated-stateful-application.md @@ -21,11 +21,14 @@ weight: 30 本页展示如何使用 [StatefulSet](/zh/docs/concepts/workloads/controllers/statefulset/) -控制器运行一个有状态的应用程序。此例是一主多从、异步复制的 MySQL 集群。 +控制器运行一个有状态的应用程序。此例是多副本的 MySQL 数据库。 +示例应用的拓扑结构有一个主服务器和多个副本，使用异步的基于行（Row-Based） +的数据复制。 这个 ConfigMap 提供 `my.cnf` 覆盖设置，使你可以独立控制 MySQL 主服务器和从服务器的配置。 -在这里，你希望主服务器能够将复制日志提供给从服务器，并且希望从服务器拒绝任何不是通过 +在这里，你希望主服务器能够将复制日志提供给副本服务器，并且希望副本服务器拒绝任何不是通过 复制进行的写操作。 客户端服务称为 `mysql-read`，是一种常规服务，具有其自己的集群 IP。 该集群 IP 在报告就绪的所有MySQL Pod 之间分配连接。 -可能的端点集合包括 MySQL 主节点和所有从节点。 +可能的端点集合包括 MySQL 主节点和所有副本节点。 请注意，只有读查询才能使用负载平衡的客户端服务。 @@ -274,38 +277,37 @@ properties. 而这些 ID 也是需要唯一性、稳定性保证的。 通过将内容复制到 conf.d 中，`init-mysql` 容器中的脚本也可以应用 ConfigMap 中的 -`master.cnf` 或 `slave.cnf`。 -由于示例部署结构由单个 MySQL 主节点和任意数量的从节点组成，因此脚本仅将序数 -`0` 指定为主节点，而将其他所有节点指定为从节点。 +`primary.cnf` 或 `replica.cnf`。 +由于示例部署结构由单个 MySQL 主节点和任意数量的副本节点组成，因此脚本仅将序数 +`0` 指定为主节点，而将其他所有节点指定为副本节点。 与 StatefulSet 控制器的 [部署顺序保证](/zh/docs/concepts/workloads/controllers/statefulset/#deployment-and-scaling-guarantees/) 相结合， -可以确保 MySQL 主服务器在创建从服务器之前已准备就绪，以便它们可以开始复制。 +可以确保 MySQL 主服务器在创建副本服务器之前已准备就绪，以便它们可以开始复制。 ### 克隆现有数据 -通常，当新 Pod 作为从节点加入集合时，必须假定 MySQL 主节点可能已经有数据。 +通常，当新 Pod 作为副本节点加入集合时，必须假定 MySQL 主节点可能已经有数据。 还必须假设复制日志可能不会一直追溯到时间的开始。 -第二个名为 `clone-mysql` 的 Init 容器，第一次在带有空 PersistentVolume 的从属 Pod +第二个名为 `clone-mysql` 的 Init 容器，第一次在带有空 PersistentVolume 的副本 Pod 上启动时，会在从属 Pod 上执行克隆操作。 这意味着它将从另一个运行中的 Pod 复制所有现有数据，使此其本地状态足够一致， 从而可以开始从主服务器复制。 @@ -329,14 +331,14 @@ so its local state is consistent enough to begin replicating from the master. MySQL itself does not provide a mechanism to do this, so the example uses a popular open-source tool called Percona XtraBackup. During the clone, the source MySQL server might suffer reduced performance. -To minimize impact on the MySQL master, the script instructs each Pod to clone +To minimize impact on the primary MySQL server, the script instructs each Pod to clone from the Pod whose ordinal index is one lower. This works because the StatefulSet controller always ensures Pod `N` is Ready before starting Pod `N+1`. --> MySQL 本身不提供执行此操作的机制，因此本示例使用了一种流行的开源工具 Percona XtraBackup。 在克隆期间，源 MySQL 服务器性能可能会受到影响。 -为了最大程度地减少对 MySQL 主节点的影响，该脚本指示每个 Pod 从序号较低的 Pod 中克隆。 +为了最大程度地减少对 MySQL 主服务器的影响，该脚本指示每个 Pod 从序号较低的 Pod 中克隆。 可以这样做的原因是 StatefulSet 控制器始终确保在启动 Pod N + 1 之前 Pod N 已准备就绪。 -`xtrabackup` sidecar 容器查看克隆的数据文件，并确定是否有必要在从服务器上初始化 MySQL 复制。 +`xtrabackup` sidecar 容器查看克隆的数据文件，并确定是否有必要在副本服务器上初始化 MySQL 复制。 如果是这样，它将等待 `mysqld` 准备就绪，然后使用从 XtraBackup 克隆文件中提取的复制参数 执行 `CHANGE MASTER TO` 和 `START SLAVE` 命令。 -一旦从服务器开始复制后，它会记住其 MySQL 主服务器，并且如果服务器重新启动或连接中断也会自动重新连接。 -另外，因为从服务器会以其稳定的 DNS 名称查找主服务器（`mysql-0.mysql`）， -即使由于重新调度而获得新的 Pod IP，他们也会自动找到主服务器。 +一旦副本服务器开始复制后，它会记住其 MySQL 主服务器，并且如果服务器重新启动或 +连接中断也会自动重新连接。 +另外，因为副本服务器会以其稳定的 DNS 名称查找主服务器（`mysql-0.mysql`）， +即使由于重新调度而获得新的 Pod IP，它们也会自动找到主服务器。 @@ -478,13 +481,13 @@ it running in another window so you can see the effects of the following steps. ## 模拟 Pod 和 Node 的宕机时间 -为了证明从从节点缓存而不是单个服务器读取数据的可用性提高，请在使 Pod 退出 Ready +为了证明从副本节点缓存而不是单个服务器读取数据的可用性提高，请在使 Pod 退出 Ready 状态时，保持上述 `SELECT @@server_id` 循环一直运行。 -## 扩展从节点数量 +## 扩展副本节点数量 -使用 MySQL 复制，你可以通过添加从节点来扩展读取查询的能力。 +使用 MySQL 复制，你可以通过添加副本节点来扩展读取查询的能力。 使用 StatefulSet，你可以使用单个命令执行此操作： ```shell From 29833597a5a7df0448acc37a96dc1f7bcb2b5e05 Mon Sep 17 00:00:00 2001 From: Tej-Singh-Rana <58101587+Tej-Singh-Rana@users.noreply.github.com> Date: Fri, 29 Jan 2021 23:47:40 +0530 Subject: [PATCH 106/190] Fix typos in release notes (#25816) * Fixed the typo * Fixed the typo * Update notes.md --- content/en/docs/setup/release/notes.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/content/en/docs/setup/release/notes.md b/content/en/docs/setup/release/notes.md index 34d669e746..adbdb7c48e 100644 --- a/content/en/docs/setup/release/notes.md +++ b/content/en/docs/setup/release/notes.md @@ -92,7 +92,7 @@ We expect this implementation to progress from alpha to beta and GA in coming re ### go1.15.5 -go1.15.5 has been integrated to Kubernets project as of this release, [including other infrastructure related updates on this effort](https://github.com/kubernetes/kubernetes/pull/95776). +go1.15.5 has been integrated to Kubernetes project as of this release, [including other infrastructure related updates on this effort](https://github.com/kubernetes/kubernetes/pull/95776). ### CSI Volume Snapshot graduates to General Availability @@ -190,7 +190,7 @@ Currently, cadvisor_stats_provider provides AcceleratorStats but cri_stats_provi PodSubnet validates against the corresponding cluster "--node-cidr-mask-size" of the kube-controller-manager, it fail if the values are not compatible. kubeadm no longer sets the node-mask automatically on IPv6 deployments, you must check that your IPv6 service subnet mask is compatible with the default node mask /64 or set it accordenly. Previously, for IPv6, if the podSubnet had a mask lower than /112, kubeadm calculated a node-mask to be multiple of eight and splitting the available bits to maximise the number used for nodes. ([#95723](https://github.com/kubernetes/kubernetes/pull/95723), [@aojea](https://github.com/aojea)) [SIG Cluster Lifecycle] -- The deprecated flag --experimental-kustomize is now removed from kubeadm commands. Use --experimental-patches instead, which was introduced in 1.19. Migration infromation available in --help description for --exprimental-patches. ([#94871](https://github.com/kubernetes/kubernetes/pull/94871), [@neolit123](https://github.com/neolit123)) +- The deprecated flag --experimental-kustomize is now removed from kubeadm commands. Use --experimental-patches instead, which was introduced in 1.19. Migration information available in --help description for --experimental-patches. ([#94871](https://github.com/kubernetes/kubernetes/pull/94871), [@neolit123](https://github.com/neolit123)) - Windows hyper-v container featuregate is deprecated in 1.20 and will be removed in 1.21 ([#95505](https://github.com/kubernetes/kubernetes/pull/95505), [@wawa0210](https://github.com/wawa0210)) [SIG Node and Windows] - The kube-apiserver ability to serve on an insecure port, deprecated since v1.10, has been removed. The insecure address flags `--address` and `--insecure-bind-address` have no effect in kube-apiserver and will be removed in v1.24. The insecure port flags `--port` and `--insecure-port` may only be set to 0 and will be removed in v1.24. ([#95856](https://github.com/kubernetes/kubernetes/pull/95856), [@knight42](https://github.com/knight42), [SIG API Machinery, Node, Testing]) - Add dual-stack Services (alpha). This is a BREAKING CHANGE to an alpha API. @@ -2138,4 +2138,4 @@ filename | sha512 hash - github.com/godbus/dbus: [ade71ed](https://github.com/godbus/dbus/tree/ade71ed) - github.com/xlab/handysort: [fb3537e](https://github.com/xlab/handysort/tree/fb3537e) - sigs.k8s.io/structured-merge-diff/v3: v3.0.0 -- vbom.ml/util: db5cfe1 \ No newline at end of file +- vbom.ml/util: db5cfe1 From 15be3b1c7b458823fa83224657223e7c282519c9 Mon Sep 17 00:00:00 2001 From: Qiming Teng Date: Sat, 30 Jan 2021 12:32:43 +0800 Subject: [PATCH 107/190] [zh] Resync connecting-frontend-backend --- .../connecting-frontend-backend.md | 152 +++++++++++------- .../{hello.yaml => backend-deployment.yaml} | 6 +- ...ello-service.yaml => backend-service.yaml} | 2 + .../service/access/frontend-deployment.yaml | 27 ++++ .../service/access/frontend-nginx.conf | 14 ++ .../service/access/frontend-service.yaml | 15 ++ .../zh/examples/service/access/frontend.conf | 11 -- .../zh/examples/service/access/frontend.yaml | 39 ----- 8 files changed, 152 insertions(+), 114 deletions(-) rename content/zh/examples/service/access/{hello.yaml => backend-deployment.yaml} (91%) rename content/zh/examples/service/access/{hello-service.yaml => backend-service.yaml} (95%) create mode 100644 content/zh/examples/service/access/frontend-deployment.yaml create mode 100644 content/zh/examples/service/access/frontend-nginx.conf create mode 100644 content/zh/examples/service/access/frontend-service.yaml delete mode 100644 content/zh/examples/service/access/frontend.conf delete mode 100644 content/zh/examples/service/access/frontend.yaml diff --git a/content/zh/docs/tasks/access-application-cluster/connecting-frontend-backend.md b/content/zh/docs/tasks/access-application-cluster/connecting-frontend-backend.md index 3fa7626c88..a485d9ba3a 100644 --- a/content/zh/docs/tasks/access-application-cluster/connecting-frontend-backend.md +++ b/content/zh/docs/tasks/access-application-cluster/connecting-frontend-backend.md @@ -3,31 +3,40 @@ title: 使用 Service 把前端连接到后端 content_type: tutorial weight: 70 --- + -本任务会描述如何创建前端微服务和后端微服务。后端微服务是一个 hello 欢迎程序。 -前端和后端的连接是通过 Kubernetes {{< glossary_tooltip term_id="service" text="服务" >}} -完成的。 +本任务会描述如何创建前端（Frontend）微服务和后端（Backend）微服务。后端微服务是一个 hello 欢迎程序。 +前端通过 nginx 和一个 Kubernetes {{< glossary_tooltip term_id="service" text="服务" >}} +暴露后端所提供的服务。 ## {{% heading "objectives" %}} -* 使用部署对象（Deployment object）创建并运行一个微服务 -* 从后端将流量路由到前端 -* 使用服务对象把前端应用连接到后端应用 +* 使用部署对象（Deployment object）创建并运行一个 `hello` 后端微服务 +* 使用一个 Service 对象将请求流量发送到后端微服务的多个副本 +* 同样使用一个 Deployment 对象创建并运行一个 `nginx` 前端微服务 +* 配置前端微服务将请求流量发送到后端微服务 +* 使用 `type=LoadBalancer` 的 Service 对象将全段微服务暴露到集群外部 ## {{% heading "prerequisites" %}} @@ -39,8 +48,7 @@ This task uses require a supported environment. If your environment does not support this, you can use a Service of type [NodePort](/docs/concepts/services-networking/service/#nodeport) instead. --> - -本任务使用 [外部负载均衡服务](/zh/docs/tasks/access-application-cluster/create-external-load-balancer/)， +本任务使用[外部负载均衡服务](/zh/docs/tasks/access-application-cluster/create-external-load-balancer/)， 所以需要对应的可支持此功能的环境。如果你的环境不能支持，你可以使用 [NodePort](/zh/docs/concepts/services-networking/service/#nodeport) 类型的服务代替。 @@ -57,7 +65,7 @@ file for the backend Deployment: 后端是一个简单的 hello 欢迎微服务应用。这是后端应用的 Deployment 配置文件： -{{< codenew file="service/access/hello.yaml" >}} +{{< codenew file="service/access/backend-deployment.yaml" >}} 查看后端的 Deployment 信息： -``` +```shell kubectl describe deployment hello ``` @@ -83,7 +91,7 @@ The output is similar to this: 输出类似于： ``` -Name: hello +Name: backend Namespace: default CreationTimestamp: Mon, 24 Oct 2016 14:21:02 -0700 Labels: app=hello @@ -91,7 +99,7 @@ Labels: app=hello track=stable Annotations: deployment.kubernetes.io/revision=1 Selector: app=hello,tier=backend,track=stable -Replicas: 7 desired | 7 updated | 7 total | 7 available | 0 unavailable +Replicas: 3 desired | 3 updated | 3 total | 3 available | 0 unavailable StrategyType: RollingUpdate MinReadySeconds: 0 RollingUpdateStrategy: 1 max unavailable, 1 max surge @@ -112,15 +120,15 @@ Conditions: Available True MinimumReplicasAvailable Progressing True NewReplicaSetAvailable OldReplicaSets: -NewReplicaSet: hello-3621623197 (7/7 replicas created) +NewReplicaSet: hello-3621623197 (3/3 replicas created) Events: ... ``` -### 创建后端服务对象 +### 创建 `hello` Service 对象 -前端连接到后端的关键是 Service（服务）。Service 创建一个固定 IP 和 DNS 解析名入口， -使得后端微服务可达。Service 使用 +将请求从前端发送到到后端的关键是后端 Service。Service 创建一个固定 IP 和 DNS 解析名入口， +使得后端微服务总是可达。Service 使用 {{< glossary_tooltip text="选择算符" term_id="selector" >}} 来寻找目标 Pod。 首先，浏览 Service 的配置文件： -{{< codenew file="service/access/hello-service.yaml" >}} +{{< codenew file="service/access/backend-service.yaml" >}} -配置文件中，你可以看到 Service 将流量路由到包含 `app: hello` 和 `tier: backend` 标签的 Pod。 +配置文件中，你可以看到名为 `hello` 的 Service 将流量路由到包含 `app: hello` +和 `tier: backend` 标签的 Pod。 -创建 `hello` Service： +创建后端 Service： ```shell -kubectl apply -f https://k8s.io/examples/service/access/hello-service.yaml +kubectl apply -f https://k8s.io/examples/service/access/backend-service.yaml ``` -此时，你已经有了一个在运行的后端 Deployment，你也有了一个 Service 用于路由网络流量。 +此时，你已经有了一个运行着 `hello` 应用的三个副本的 `backend` Deployment，你也有了 +一个 Service 用于路由网络流量。不过，这个服务在集群外部无法访问也无法解析。 ### 创建前端应用 -既然你已经有了后端应用，你可以创建一个前端应用连接到后端。前端应用通过 DNS 名连接到后端的工作 Pods。 -DNS 名是 "hello"，也就是 Service 配置文件中 `name` 字段的值。 +现在你已经有了运行中的后端应用，你可以创建一个可在集群外部访问的前端，并通过代理 +前端的请求连接到后端。 -前端 Deployment 中的 Pods 运行一个 nginx 镜像，这个已经配置好镜像去寻找后端的 hello Service。 -只是 nginx 的配置文件： +前端使用被赋予后端 Service 的 DNS 名称将请求发送到后端工作 Pods。这一 DNS +名称为 `hello`，也就是 `examples/service/access/backend-service.yaml` 配置 +文件中 `name` 字段的取值。 -{{< codenew file="service/access/frontend.conf" >}} +前端 Deployment 中的 Pods 运行一个 nginx 镜像，这个已经配置好的镜像会将请求转发 +给后端的 hello Service。下面是 nginx 的配置文件： + +{{< codenew file="service/access/frontend-nginx.conf" >}} -与后端类似，前端用包含一个 Deployment 和一个 Service。Service 的配置文件包含了 `type: LoadBalancer`， -也就是说，Service 会使用你的云服务商的默认负载均衡设备。 +与后端类似，前端用包含一个 Deployment 和一个 Service。后端与前端服务之间的一个 +重要区别是前端 Service 的配置文件包含了 `type: LoadBalancer`，也就是说，Service +会使用你的云服务商的默认负载均衡设备，从而实现从集群外访问的目的。 + +{{< codenew file="service/access/frontend-service.yaml" >}} + +{{< codenew file="service/access/frontend-deployment.yaml" >}} -{{< codenew file="service/access/frontend.yaml" >}} ### 通过前端发送流量 -前端和后端已经完成连接了。你可以使用 curl 命令通过你的前端 Service 的外部 IP 访问服务端点。 +前端和后端已经完成连接了。你可以使用 curl 命令通过你的前端 Service 的外部 +IP 访问服务端点。 ```shell -curl http:// +curl http://${EXTERNAL_IP} # 将 EXTERNAL_P 替换为你之前看到的外部 IP ``` -后端生成的消息输出如下： +输出显示后端生成的消息： ```json {"message":"Hello"} @@ -299,7 +325,7 @@ To delete the Services, enter this command: 要删除服务，输入下面的命令： ```shell -kubectl delete services frontend hello +kubectl delete services frontend backend ``` -* 进一步了解[Service](/zh/docs/concepts/services-networking/service/) -* 进一步了解[ConfigMap](/zh/docs/tasks/configure-pod-container/configure-pod-configmap/) +* 进一步了解 [Service](/zh/docs/concepts/services-networking/service/) +* 进一步了解 [ConfigMap](/zh/docs/tasks/configure-pod-container/configure-pod-configmap/) +* 进一步了解 [Service 和 Pods 的 DNS](/docs/concepts/services-networking/dns-pod-service/) diff --git a/content/zh/examples/service/access/hello.yaml b/content/zh/examples/service/access/backend-deployment.yaml similarity index 91% rename from content/zh/examples/service/access/hello.yaml rename to content/zh/examples/service/access/backend-deployment.yaml index 85dff18ee1..5c95e38a3b 100644 --- a/content/zh/examples/service/access/hello.yaml +++ b/content/zh/examples/service/access/backend-deployment.yaml @@ -1,14 +1,15 @@ +--- apiVersion: apps/v1 kind: Deployment metadata: - name: hello + name: backend spec: selector: matchLabels: app: hello tier: backend track: stable - replicas: 7 + replicas: 3 template: metadata: labels: @@ -22,3 +23,4 @@ spec: ports: - name: http containerPort: 80 +... \ No newline at end of file diff --git a/content/zh/examples/service/access/hello-service.yaml b/content/zh/examples/service/access/backend-service.yaml similarity index 95% rename from content/zh/examples/service/access/hello-service.yaml rename to content/zh/examples/service/access/backend-service.yaml index 71344ecb8b..9262d29bb8 100644 --- a/content/zh/examples/service/access/hello-service.yaml +++ b/content/zh/examples/service/access/backend-service.yaml @@ -1,3 +1,4 @@ +--- apiVersion: v1 kind: Service metadata: @@ -10,3 +11,4 @@ spec: - protocol: TCP port: 80 targetPort: http +... \ No newline at end of file diff --git a/content/zh/examples/service/access/frontend-deployment.yaml b/content/zh/examples/service/access/frontend-deployment.yaml new file mode 100644 index 0000000000..182b0e708e --- /dev/null +++ b/content/zh/examples/service/access/frontend-deployment.yaml @@ -0,0 +1,27 @@ +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: frontend +spec: + selector: + matchLabels: + app: hello + tier: frontend + track: stable + replicas: 1 + template: + metadata: + labels: + app: hello + tier: frontend + track: stable + spec: + containers: + - name: nginx + image: "gcr.io/google-samples/hello-frontend:1.0" + lifecycle: + preStop: + exec: + command: ["/usr/sbin/nginx","-s","quit"] +... \ No newline at end of file diff --git a/content/zh/examples/service/access/frontend-nginx.conf b/content/zh/examples/service/access/frontend-nginx.conf new file mode 100644 index 0000000000..39a911a09a --- /dev/null +++ b/content/zh/examples/service/access/frontend-nginx.conf @@ -0,0 +1,14 @@ +# The identifier Backend is internal to nginx, and used to name this specific upstream +upstream Backend { + # hello is the internal DNS name used by the backend Service inside Kubernetes + server hello; +} + +server { + listen 80; + + location / { + # The following statement will proxy traffic to the upstream named Backend + proxy_pass http://Backend; + } +} diff --git a/content/zh/examples/service/access/frontend-service.yaml b/content/zh/examples/service/access/frontend-service.yaml new file mode 100644 index 0000000000..898a5ed51b --- /dev/null +++ b/content/zh/examples/service/access/frontend-service.yaml @@ -0,0 +1,15 @@ +--- +apiVersion: v1 +kind: Service +metadata: + name: frontend +spec: + selector: + app: hello + tier: frontend + ports: + - protocol: "TCP" + port: 80 + targetPort: 80 + type: LoadBalancer +... \ No newline at end of file diff --git a/content/zh/examples/service/access/frontend.conf b/content/zh/examples/service/access/frontend.conf deleted file mode 100644 index 9a1f5a0ed6..0000000000 --- a/content/zh/examples/service/access/frontend.conf +++ /dev/null @@ -1,11 +0,0 @@ -upstream hello { - server hello; -} - -server { - listen 80; - - location / { - proxy_pass http://hello; - } -} diff --git a/content/zh/examples/service/access/frontend.yaml b/content/zh/examples/service/access/frontend.yaml deleted file mode 100644 index 9f5b6b757f..0000000000 --- a/content/zh/examples/service/access/frontend.yaml +++ /dev/null @@ -1,39 +0,0 @@ -apiVersion: v1 -kind: Service -metadata: - name: frontend -spec: - selector: - app: hello - tier: frontend - ports: - - protocol: "TCP" - port: 80 - targetPort: 80 - type: LoadBalancer ---- -apiVersion: apps/v1 -kind: Deployment -metadata: - name: frontend -spec: - selector: - matchLabels: - app: hello - tier: frontend - track: stable - replicas: 1 - template: - metadata: - labels: - app: hello - tier: frontend - track: stable - spec: - containers: - - name: nginx - image: "gcr.io/google-samples/hello-frontend:1.0" - lifecycle: - preStop: - exec: - command: ["/usr/sbin/nginx","-s","quit"] From 4087e33e618172b82303bf15cf3b0f1c4ffeadbd Mon Sep 17 00:00:00 2001 From: Yang Jiao Date: Sat, 30 Jan 2021 12:59:38 +0800 Subject: [PATCH 108/190] [zh] sync and update fine-parallel-processing-work-queue MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 1. change keyword replicationController to ReplicaSet 2. change word 连续 to 持续 3. other minor changes (e.g: add connection word for readability, add 'shell' for command line input) --- .../tasks/job/fine-parallel-processing-work-queue.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/content/zh/docs/tasks/job/fine-parallel-processing-work-queue.md b/content/zh/docs/tasks/job/fine-parallel-processing-work-queue.md index bd6688374b..4f908c73d8 100644 --- a/content/zh/docs/tasks/job/fine-parallel-processing-work-queue.md +++ b/content/zh/docs/tasks/job/fine-parallel-processing-work-queue.md @@ -85,7 +85,7 @@ of deploying Redis scalably and redundantly. -你可以直接下载如下文件： +你也可以直接下载如下文件： - [`redis-pod.yaml`](/examples/application/job/redis/redis-pod.yaml) - [`redis-service.yaml`](/examples/application/job/redis/redis-service.yaml) @@ -121,7 +121,7 @@ Now hit enter, start the redis CLI, and create a list with some work items in it --> 现在按回车键，启动 redis 命令行界面，然后创建一个存在若干个工作项的列表。 -``` +```shell # redis-cli -h redis redis:6379> rpush job2 "apple" (integer) 1 @@ -214,7 +214,7 @@ your username and push to the Hub with the below commands. Replace ### Push 镜像 对于 [Docker Hub](https://hub.docker.com/)，请先用你的用户名给镜像打上标签， -然后使用下面的命令 push 你的镜像到仓库。请将 `` 替换为你自己的用户名。 +然后使用下面的命令 push 你的镜像到仓库。请将 `` 替换为你自己的 Hub 用户名。 ```shell docker tag job-wq-2 /job-wq-2 @@ -270,7 +270,7 @@ too. --> 在这个例子中，每个 pod 处理了队列中的多个项目，直到队列中没有项目时便退出。 因为是由工作程序自行检测工作队列是否为空，并且 Job 控制器不知道工作队列的存在， -所以依赖于工作程序在完成工作时发出信号。 +这依赖于工作程序在完成工作时发出信号。 工作程序以成功退出的形式发出信号表示工作队列已经为空。 所以，只要有任意一个工作程序成功退出，控制器就知道工作已经完成了，所有的 Pod 将很快会退出。 因此，我们将 Job 的完成计数（Completion Count）设置为 1 。 @@ -364,7 +364,6 @@ consider running your background workers with a `replicationController` instead, and consider running a background processing library such as [https://github.com/resque/resque](https://github.com/resque/resque). --> -如果你有连续的后台处理业务，那么可以考虑使用 `replicationController` 来运行你的后台业务， +如果你有持续的后台处理业务，那么可以考虑使用 `ReplicaSet` 来运行你的后台业务， 和运行一个类似 [https://github.com/resque/resque](https://github.com/resque/resque) 的后台处理库。 - From cb1641b0c74941258c222d543b443b908ad5021d Mon Sep 17 00:00:00 2001 From: Qiming Teng Date: Fri, 29 Jan 2021 17:06:02 +0800 Subject: [PATCH 109/190] [zh] sync install kubectl --- .../zh/docs/tasks/tools/install-kubectl.md | 262 ++++++++++++++---- 1 file changed, 205 insertions(+), 57 deletions(-) diff --git a/content/zh/docs/tasks/tools/install-kubectl.md b/content/zh/docs/tasks/tools/install-kubectl.md index 2e87c83417..e67a7dcb21 100644 --- a/content/zh/docs/tasks/tools/install-kubectl.md +++ b/content/zh/docs/tasks/tools/install-kubectl.md @@ -29,7 +29,7 @@ and view logs. For a complete list of kubectl operations, see --> 使用 Kubernetes 命令行工具 [kubectl](/zh/docs/reference/kubectl/kubectl/)， 你可以在 Kubernetes 上运行命令。 -使用 kubectl，你可以部署应用、检查和管理集群资源、查看日志。 +使用 kubectl，你可以部署应用、检视和管理集群资源、查看日志。 要了解 kubectl 操作的完整列表，请参阅 [kubectl 概览](/zh/docs/reference/kubectl/overview/)。 @@ -60,44 +60,97 @@ Using the latest version of kubectl helps avoid unforeseen issues. --> 1. 使用下面命令下载最新的发行版本： - ``` - curl -LO "https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl" + ```bash + curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" ``` - 要下载特定版本， `$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)` + 要下载特定版本，将命令中的 `$(curl -L -s https://dl.k8s.io/release/stable.txt)` 部分替换为指定版本。 例如，要下载 Linux 上的版本 {{< param "fullversion" >}}，输入： ``` - curl -LO https://storage.googleapis.com/kubernetes-release/release/{{< param "fullversion" >}}/bin/linux/amd64/kubectl + curl -LO https://dl.k8s.io/release/{{< param "fullversion" >}}/bin/linux/amd64/kubectl ``` -2. 标记 kubectl 文件为可执行： +2. 验证可执行文件（可选步骤）： + + 下载 kubectl 校验和文件： + + ```bash + curl -LO "https://dl.k8s.io/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl.sha256" ``` - chmod +x ./kubectl + + + 使用校验和文件检查 kubectl 可执行二进制文件： + + ```bash + echo "$( + 如果合法，则输出为： + + ```bash + kubectl: OK + ``` + + + 如果检查失败，则 `sha256` 退出且状态值非 0 并打印类似如下输出： + + ```bash + kubectl: FAILED + sha256sum: WARNING: 1 computed checksum did NOT match + ``` + + {{< note >}} + + 所下载的二进制可执行文件和校验和文件须是同一版本。 + {{< /note >}} + + -3. 将文件放到 PATH 路径下： +3. 安装 kubectl + ```bash + sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl ``` - sudo mv ./kubectl /usr/local/bin/kubectl + + + 如果你并不拥有目标系统的 root 访问权限，你仍可以将 kubectl 安装到 + `~/.local/bin` 目录下： + + ```bash + mkdir -p ~/.local/bin/kubectl + mv ./kubectl ~/.local/bin/kubectl + # 之后将 ~/.local/bin/kubectl 添加到环境变量 $PATH 中 ``` 4. 测试你所安装的版本是最新的： @@ -118,6 +171,7 @@ echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee -a /etc/ sudo apt-get update sudo apt-get install -y kubectl {{< /tab >}} + {{< tab name="CentOS、RHEL 或 Fedora" codelang="bash" >}} cat < /etc/yum.repos.d/kubernetes.repo [kubernetes] @@ -186,42 +240,95 @@ kubectl version --client 1. 下载最新发行版本： ```bash - curl -LO "https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/darwin/amd64/kubectl" + curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/darwin/amd64/kubectl" ``` - 要下载特定版本，可将上面命令中的`$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)` + 要下载特定版本，可将上面命令中的 `$(curl -L -s https://dl.k8s.io/release/stable.txt)` 部分替换成你想要的版本。 例如，要在 macOS 上安装版本 {{< param "fullversion" >}}，输入： ```bash - curl -LO https://storage.googleapis.com/kubernetes-release/release/{{< param "fullversion" >}}/bin/darwin/amd64/kubectl + curl -LO https://dl.k8s.io/release/{{< param "fullversion" >}}/bin/darwin/amd64/kubectl ``` - - 将二进制文件标记为可执行： + +2. 检查二进制可执行文件（可选操作） + + + 下载 kubectl 校验和文件： + + ```bash + curl -LO "https://dl.k8s.io/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/darwin/amd64/kubectl.sha256" + ``` + + + 使用校验和文件检查 kubectl 二进制可执行文件： + + ```bash + echo "$( + 如果合法，则输出为： + + ```bash + kubectl: OK + ``` + + + 如果检查失败，则 `shasum` 退出且状态值为非 0，并打印类似如下的输出： + + ```bash + kubectl: FAILED + shasum: WARNING: 1 computed checksum did NOT match + ``` + + {{< note >}} + + 下载的二进制可执行文件和校验和文件须为同一版本。 + {{< /note >}} + + +3. 设置 kubectl 二进制文件为可执行模式 ```bash chmod +x ./kubectl ``` + -3. 将二进制文件放入 PATH 目录下： +4. 将 kubectl 二进制文件移动到系统 `PATH` 环境变量中的某个位置： ```bash - sudo mv ./kubectl /usr/local/bin/kubectl + sudo mv ./kubectl /usr/local/bin/kubectl && \ + sudo chown root: /usr/local/bin/kubectl ``` -4. 测试以确保所安装的版本是最新的： +5. 测试以确保所安装的版本是最新的： ```bash kubectl version --client @@ -256,7 +363,7 @@ If you are on macOS and using [Homebrew](https://brew.sh/) package manager, you ``` 2. 测试以确保你安装的版本是最新的： @@ -271,7 +378,8 @@ If you are on macOS and using [Macports](https://macports.org/) package manager, --> ### 在 macOS 上用 Macports 安装 kubectl -如果你使用的是 macOS 系统并使用 [Macports](https://macports.org/) 包管理器，你可以通过 Macports 安装 kubectl。 +如果你使用的是 macOS 系统并使用 [Macports](https://macports.org/) 包管理器， +你可以通过 Macports 安装 kubectl。 @@ -302,36 +410,74 @@ If you are on macOS and using [Macports](https://macports.org/) package manager, ### 在 Windows 上使用 curl 安装 kubectl 二进制文件 -1. 从[此链接](https://storage.googleapis.com/kubernetes-release/release/{{< param "fullversion" >}}/bin/windows/amd64/kubectl.exe) - 下载最新发行版本 {{< param "fullversion" >}}。 +1. 下载[最新发行版本 {{< param "fullversion" >}}](https://dl.k8s.io/release/{{< param "fullversion" >}}/bin/windows/amd64/kubectl.exe)。 或者如何你安装了 `curl`，使用下面的命令： ```bash - curl -LO https://storage.googleapis.com/kubernetes-release/release/{{< param "fullversion" >}}/bin/windows/amd64/kubectl.exe + curl -LO https://dl.k8s.io/release/{{< param "fullversion" >}}/bin/windows/amd64/kubectl.exe ``` - 要了解最新的稳定版本（例如，出于脚本编写目的），可查看 - [https://storage.googleapis.com/kubernetes-release/release/stable.txt](https://storage.googleapis.com/kubernetes-release/release/stable.txt)。 + 要了解哪个是最新的稳定版本（例如，出于脚本编写目的），可查看 + [https://dl.k8s.io/release/stable.txt](https://dl.k8s.io/release/stable.txt)。 -2. 将可执行文件放到 PATH 目录下。 +2. 验证二进制可执行文件（可选操作） + + + 下载 kubectl 校验和文件： + + ```powershell + curl -LO https://dl.k8s.io/{{< param "fullversion" >}}/bin/windows/amd64/kubectl.exe.sha256 + ``` + + + 使用校验和文件验证 kubectl 可执行二进制文件： + + + - 使用命令行提示符（Commmand Prompt）来手动比较 `CertUtil` 的输出与 + 所下载的校验和文件： + + ```cmd + CertUtil -hashfile kubectl.exe SHA256 + type kubectl.exe.sha256 + ``` + + + - 使用 PowerShell 的 `-eq` 操作符来自动完成校验操作，获得 `True` 或 `False` 结果： + + ```powershell + $($(CertUtil -hashfile .\kubectl.exe SHA256)[1] -replace " ", "") -eq $(type .\kubectl.exe.sha256) + ``` -3. 测试以确定所下载的 `kubectl` 版本是正确的的： +3. 将可执行文件放到 PATH 目录下。 - ```bash + +4. 测试以确定所下载的 `kubectl` 版本是正确的的： + + ```cmd kubectl version --client ``` @@ -341,20 +487,20 @@ If you have installed Docker Desktop before, you may need to place your PATH ent --> {{< note >}} [Docker Desktop for Windows](https://docs.docker.com/docker-for-windows/#kubernetes) -会在 PATH 中添加自己的 `kubectl` 程序。 +会将自己的 `kubectl` 程序添加到 PATH 中。 如果你之前安装过 Docker Desktop，你可能需要将新安装的 PATH 项放到 Docker Desktop 安装程序所添加的目录之前，或者干脆删除 Docker Desktop 所安装的 `kubectl`。 {{< /note >}} ## 使用 PowerShell 从 PSGallery 安装 kubectl 如果你使用的是 Windows 系统并使用 [Powershell Gallery](https://www.powershellgallery.com/) -软件包管理器，你可以使用 Powershell 安装和更新 kubectl。 +软件包管理器，你可以使用 PowerShell 安装和更新 kubectl。 {{< note >}} - 如果你没有指定 `DownloadLocation`，那么 `kubectl` 将安装在用户的临时目录中。 + 如果你没有指定 `DownloadLocation`，那么 `kubectl` 将安装在用户的 `temp` 目录中。 {{< /note >}} - - 1. 要在 Windows 上用 [Chocolatey](https://chocolatey.org) 或者 [Scoop](https://scoop.sh) 命令行安装程序安装 kubectl： @@ -421,7 +565,7 @@ Updating the installation is performed by rerunning the two commands listed in s {{< /tabs >}} 2. 测试以确保你安装的版本是最新的： @@ -430,7 +574,7 @@ Updating the installation is performed by rerunning the two commands listed in s ``` 3. 切换到你的 HOME 目录： @@ -440,7 +584,7 @@ Updating the installation is performed by rerunning the two commands listed in s ``` 4. 创建 `.kube` 目录： @@ -449,7 +593,7 @@ Updating the installation is performed by rerunning the two commands listed in s ``` 5. 进入到刚刚创建的 `.kube` 目录： @@ -458,7 +602,7 @@ Updating the installation is performed by rerunning the two commands listed in s ``` 6. 配置 kubectl 以使用远程 Kubernetes 集群： @@ -539,11 +683,13 @@ If you see a URL response, kubectl is correctly configured to access your cluste If you see a message similar to the following, kubectl is not configured correctly or is not able to connect to a Kubernetes cluster. --> -如果你看到一个 URL 被返回，那么 kubectl 已经被正确配置，能够正常访问你的 Kubernetes 集群。 +如果你看到一个 URL 被返回，那么 kubectl 已经被正确配置， +能够正常访问你的 Kubernetes 集群。 -如果你看到类似以下的信息被返回，那么 kubectl 没有被正确配置，无法正常访问你的 Kubernetes 集群。 +如果你看到类似以下的信息被返回，那么 kubectl 没有被正确配置， +无法正常访问你的 Kubernetes 集群。 -```shell +``` The connection to the server was refused - did you specify the right host or port? ``` @@ -552,9 +698,11 @@ For example, if you are intending to run a Kubernetes cluster on your laptop (lo If kubectl cluster-info returns the url response but you can't access your cluster, to check whether it is configured properly, use: --> -例如，如果你打算在笔记本电脑（本地）上运行 Kubernetes 集群，则需要首先安装 minikube 等工具，然后重新运行上述命令。 +例如，如果你打算在笔记本电脑（本地）上运行 Kubernetes 集群，则需要首先安装 +minikube 等工具，然后重新运行上述命令。 -如果 kubectl cluster-info 能够返回 URL 响应，但你无法访问你的集群，可以使用下面的命令检查配置是否正确： +如果 kubectl cluster-info 能够返回 URL 响应，但你无法访问你的集群，可以使用 +下面的命令检查配置是否正确： ```shell kubectl cluster-info dump From 41de0def52b64ba4613cf7f77ee6c3c77aee7a3b Mon Sep 17 00:00:00 2001 From: Suraj Deshmukh Date: Thu, 31 Dec 2020 19:11:22 +0530 Subject: [PATCH 110/190] Reconfigure Kubelet: Update kubelet config link The old link points to the code base of 1.11. Since then there have been several changes to the configuration, either new flags added or removed. Signed-off-by: Suraj Deshmukh --- content/en/docs/tasks/administer-cluster/reconfigure-kubelet.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/tasks/administer-cluster/reconfigure-kubelet.md b/content/en/docs/tasks/administer-cluster/reconfigure-kubelet.md index 7f56e4ec85..4f71db554a 100644 --- a/content/en/docs/tasks/administer-cluster/reconfigure-kubelet.md +++ b/content/en/docs/tasks/administer-cluster/reconfigure-kubelet.md @@ -23,7 +23,7 @@ dynamically, you need a strong understanding of how that change will affect your cluster's behavior. Always carefully test configuration changes on a small set of nodes before rolling them out cluster-wide. Advice on configuring specific fields is available in the inline `KubeletConfiguration` -[type documentation](https://github.com/kubernetes/kubernetes/blob/release-1.11/pkg/kubelet/apis/kubeletconfig/v1beta1/types.go). +[type documentation (for v1.20)](https://github.com/kubernetes/kubernetes/blob/release-1.20/staging/src/k8s.io/kubelet/config/v1beta1/types.go). {{< /warning >}} From f3e5f02604b9d5de2231e18a148bcbaaae7e24bd Mon Sep 17 00:00:00 2001 From: Suraj Deshmukh Date: Sat, 30 Jan 2021 12:54:11 +0530 Subject: [PATCH 111/190] kubeadm: Add backticks and mention Flatcar This commit adds a cosmetic change by adding backticks to show the env var in monospace font when rendered. Mention Flatcar Linux in kubeadm troubleshooting doc, alongwith Fedora CoreOS. Signed-off-by: Suraj Deshmukh --- .../production-environment/tools/kubeadm/install-kubeadm.md | 4 ++-- .../tools/kubeadm/troubleshooting-kubeadm.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/content/en/docs/setup/production-environment/tools/kubeadm/install-kubeadm.md b/content/en/docs/setup/production-environment/tools/kubeadm/install-kubeadm.md index 7c44f6e689..90f80db6ca 100644 --- a/content/en/docs/setup/production-environment/tools/kubeadm/install-kubeadm.md +++ b/content/en/docs/setup/production-environment/tools/kubeadm/install-kubeadm.md @@ -236,8 +236,8 @@ curl -L "https://github.com/containernetworking/plugins/releases/download/${CNI_ Define the directory to download command files {{< note >}} -The DOWNLOAD_DIR variable must be set to a writable directory. -If you are running Flatcar Container Linux, set DOWNLOAD_DIR=/opt/bin. +The `DOWNLOAD_DIR` variable must be set to a writable directory. +If you are running Flatcar Container Linux, set `DOWNLOAD_DIR=/opt/bin`. {{< /note >}} ```bash diff --git a/content/en/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm.md b/content/en/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm.md index ded4250787..717ad579d4 100644 --- a/content/en/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm.md +++ b/content/en/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm.md @@ -363,7 +363,7 @@ kubectl taint nodes NODE_NAME node-role.kubernetes.io/master:NoSchedule- ## `/usr` is mounted read-only on nodes {#usr-mounted-read-only} -On Linux distributions such as Fedora CoreOS, the directory `/usr` is mounted as a read-only filesystem. +On Linux distributions such as Fedora CoreOS or Flatcar Container Linux, the directory `/usr` is mounted as a read-only filesystem. For [flex-volume support](https://github.com/kubernetes/community/blob/ab55d85/contributors/devel/sig-storage/flexvolume.md), Kubernetes components like the kubelet and kube-controller-manager use the default path of `/usr/libexec/kubernetes/kubelet-plugins/volume/exec/`, yet the flex-volume directory _must be writeable_ From f079aa8214edf2bc2a98a6956bbecc241d224b45 Mon Sep 17 00:00:00 2001 From: RainbowMango Date: Sat, 30 Jan 2021 16:09:29 +0800 Subject: [PATCH 112/190] Update validatingadmissionwebhook and mutatingadmissionwebhook docs as they have been promoted to v1 --- .../access-authn-authz/admission-controllers.md | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-) diff --git a/content/en/docs/reference/access-authn-authz/admission-controllers.md b/content/en/docs/reference/access-authn-authz/admission-controllers.md index 0cdcbf2f36..3ff113bb63 100644 --- a/content/en/docs/reference/access-authn-authz/admission-controllers.md +++ b/content/en/docs/reference/access-authn-authz/admission-controllers.md @@ -462,8 +462,6 @@ and the [example of Limit Range](/docs/tasks/administer-cluster/manage-resources ### MutatingAdmissionWebhook {#mutatingadmissionwebhook} -{{< feature-state for_k8s_version="v1.13" state="beta" >}} - This admission controller calls any mutating webhooks which match the request. Matching webhooks are called in serial; each one may modify the object if it desires. @@ -474,7 +472,7 @@ If a webhook called by this has side effects (for example, decrementing quota) i webhooks or validating admission controllers will permit the request to finish. If you disable the MutatingAdmissionWebhook, you must also disable the -`MutatingWebhookConfiguration` object in the `admissionregistration.k8s.io/v1beta1` +`MutatingWebhookConfiguration` object in the `admissionregistration.k8s.io/v1` group/version via the `--runtime-config` flag (both are on by default in versions >= 1.9). @@ -486,8 +484,6 @@ versions >= 1.9). different when read back. * Setting originally unset fields is less likely to cause problems than overwriting fields set in the original request. Avoid doing the latter. - * This is a beta feature. Future versions of Kubernetes may restrict the types of - mutations these webhooks can make. * Future changes to control loops for built-in resources or third-party resources may break webhooks that work well today. Even when the webhook installation API is finalized, not all possible webhook behaviors will be guaranteed to be supported @@ -766,8 +762,6 @@ This admission controller {{< glossary_tooltip text="taints" term_id="taint" >}} ### ValidatingAdmissionWebhook {#validatingadmissionwebhook} -{{< feature-state for_k8s_version="v1.13" state="beta" >}} - This admission controller calls any validating webhooks which match the request. Matching webhooks are called in parallel; if any of them rejects the request, the request fails. This admission controller only runs in the validation phase; the webhooks it calls may not @@ -778,7 +772,7 @@ If a webhook called by this has side effects (for example, decrementing quota) i webhooks or other validating admission controllers will permit the request to finish. If you disable the ValidatingAdmissionWebhook, you must also disable the -`ValidatingWebhookConfiguration` object in the `admissionregistration.k8s.io/v1beta1` +`ValidatingWebhookConfiguration` object in the `admissionregistration.k8s.io/v1` group/version via the `--runtime-config` flag (both are on by default in versions 1.9 and later). From 5e5953fe606f3618477c54cbd22b94c3e78b5783 Mon Sep 17 00:00:00 2001 From: Sungho Hwang Date: Sat, 30 Jan 2021 18:18:55 +0900 Subject: [PATCH 113/190] Update nodes.md --- content/en/docs/concepts/architecture/nodes.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/content/en/docs/concepts/architecture/nodes.md b/content/en/docs/concepts/architecture/nodes.md index 7bd4b355b6..de08e48d8c 100644 --- a/content/en/docs/concepts/architecture/nodes.md +++ b/content/en/docs/concepts/architecture/nodes.md @@ -11,9 +11,10 @@ weight: 10 Kubernetes runs your workload by placing containers into Pods to run on _Nodes_. A node may be a virtual or physical machine, depending on the cluster. Each node -contains the services necessary to run -{{< glossary_tooltip text="Pods" term_id="pod" >}}, managed by the -{{< glossary_tooltip text="control plane" term_id="control-plane" >}}. +is managed by the +{{< glossary_tooltip text="control plane" term_id="control-plane" >}} +and contains the services necessary to run +{{< glossary_tooltip text="Pods" term_id="pod" >}} Typically you have several nodes in a cluster; in a learning or resource-limited environment, you might have just one. From 70d1651f697460c71bb3bcae462797ec3c6b99a9 Mon Sep 17 00:00:00 2001 From: ydFu Date: Sat, 30 Jan 2021 20:29:28 +0800 Subject: [PATCH 114/190] [zh] Sync concepts pages for job.md * sync with english version in job.md (#26242) Signed-off-by: ydFu --- content/zh/docs/concepts/workloads/controllers/job.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/zh/docs/concepts/workloads/controllers/job.md b/content/zh/docs/concepts/workloads/controllers/job.md index fa27f617d1..c6af8954dc 100644 --- a/content/zh/docs/concepts/workloads/controllers/job.md +++ b/content/zh/docs/concepts/workloads/controllers/job.md @@ -22,7 +22,7 @@ weight: 50 -Job 会创建一个或者多个 Pods，并确保指定数量的 Pods 成功终止。 +Job 会创建一个或者多个 Pods，并将继续重试 Pods 的执行，直到指定数量的 Pods 成功终止。 随着 Pods 成功结束，Job 跟踪记录成功完成的 Pods 个数。 当数量达到指定的成功个数阈值时，任务（即 Job）结束。 删除 Job 的操作会清除所创建的全部 Pods。 From b30e798cf5d0c3eb0703a9d7a96ca6470969bbbe Mon Sep 17 00:00:00 2001 From: Yang Jiao Date: Sun, 31 Jan 2021 13:33:02 +0800 Subject: [PATCH 115/190] Update fine-parallel-processing-work-queue.md sync last paragraph using English version from https://github.com/kubernetes/website/edit/master/content/en/docs/tasks/job/fine-parallel-processing-work-queue.md --- .../zh/docs/tasks/job/fine-parallel-processing-work-queue.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/zh/docs/tasks/job/fine-parallel-processing-work-queue.md b/content/zh/docs/tasks/job/fine-parallel-processing-work-queue.md index 4f908c73d8..ee6f71ea69 100644 --- a/content/zh/docs/tasks/job/fine-parallel-processing-work-queue.md +++ b/content/zh/docs/tasks/job/fine-parallel-processing-work-queue.md @@ -360,7 +360,7 @@ want to consider one of the other [job patterns](/docs/concepts/jobs/run-to-comp From adbf6686dcfd501e183e23c5eb183e4af0633485 Mon Sep 17 00:00:00 2001 From: ydFu Date: Sun, 31 Jan 2021 19:54:55 +0800 Subject: [PATCH 116/190] [zh] Sync concepts pages for ingress-controllers.md Add EnRoute as one of the Ingress Controller API Gateway Options #25972 Signed-off-by: ydFu --- .../zh/docs/concepts/services-networking/ingress-controllers.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/content/zh/docs/concepts/services-networking/ingress-controllers.md b/content/zh/docs/concepts/services-networking/ingress-controllers.md index 7a9c4a8a26..4f8e279da3 100644 --- a/content/zh/docs/concepts/services-networking/ingress-controllers.md +++ b/content/zh/docs/concepts/services-networking/ingress-controllers.md @@ -49,6 +49,7 @@ Kubernetes 作为一个项目，目前支持和维护 * The [Citrix ingress controller](https://github.com/citrix/citrix-k8s-ingress-controller#readme) works with Citrix Application Delivery Controller. * [Contour](https://projectcontour.io/) is an [Envoy](https://www.envoyproxy.io/) based ingress controller. +* [EnRoute](https://getenroute.io/) is an [Envoy](https://www.envoyproxy.io) based API gateway that can run as an ingress controller. --> * [AKS 应用程序网关 Ingress 控制器](https://azure.github.io/application-gateway-kubernetes-ingress/) 是一个配置 [Azure 应用程序网关](https://docs.microsoft.com/azure/application-gateway/overview) @@ -62,6 +63,7 @@ Kubernetes 作为一个项目，目前支持和维护 * [Citrix Ingress 控制器](https://github.com/citrix/citrix-k8s-ingress-controller#readme) 可以用来与 Citrix Application Delivery Controller 一起使用。 * [Contour](https://projectcontour.io/) 是一个基于 [Envoy](https://www.envoyproxy.io/) 的 Ingress 控制器。 +* [EnRoute](https://getenroute.io/) 是一个基于 [Envoy](https://www.envoyproxy.io) API 网关能够执行 Ingress 控制器。 有关API优先级和公平性的设计细节的背景信息， 请参阅[增强建议](https://github.com/kubernetes/enhancements/blob/master/keps/sig-api-machinery/20190228-priority-and-fairness.md)。 -你可以通过 [SIG APIMachinery](https://github.com/kubernetes/community/tree/master/sig-api-machinery) -或特性的 [Slack 频道](http://kubernetes.slack.com/messages/api-priority-and-fairness) +你可以通过 [SIG APIMachinery](https://github.com/kubernetes/community/tree/master/sig-api-machinery/) +或特性的 [Slack 频道](https://kubernetes.slack.com/messages/api-priority-and-fairness/) 提出建议和特性请求。 diff --git a/content/zh/docs/concepts/cluster-administration/system-metrics.md b/content/zh/docs/concepts/cluster-administration/system-metrics.md index e5f31b0a54..cc3cb44fa7 100644 --- a/content/zh/docs/concepts/cluster-administration/system-metrics.md +++ b/content/zh/docs/concepts/cluster-administration/system-metrics.md @@ -204,7 +204,7 @@ kubelet 在驱动程序上保持打开状态。这意味着为了执行基础结 现在，收集加速器指标的责任属于供应商，而不是 kubelet。供应商必须提供一个收集指标的容器， 并将其公开给指标服务（例如 Prometheus）。 -[`DisableAcceleratorUsageMetrics` 特性门控](/zh/docs/references/command-line-tools-reference/feature-gates/) +[`DisableAcceleratorUsageMetrics` 特性门控](/zh/docs/reference/command-line-tools-reference/feature-gates/) 禁止由 kubelet 收集的指标。 关于[何时会在默认情况下启用此功能也有一定规划](https://github.com/kubernetes/enhancements/tree/411e51027db842355bd489691af897afc1a41a5e/keps/sig-node/1867-disable-accelerator-usage-metrics#graduation-criteria)。 diff --git a/content/zh/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation.md b/content/zh/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation.md index 5873bdd497..e08fa7690d 100644 --- a/content/zh/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation.md +++ b/content/zh/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation.md @@ -90,6 +90,6 @@ to disable the timeout restriction. This deprecated feature gate will be removed 了解如何在自己的环境中启用聚合器。 * 接下来，了解[安装扩展 API 服务器](/zh/docs/tasks/extend-kubernetes/setup-extension-api-server/)， 开始使用聚合层。 -* 也可以学习怎样[使用自定义资源定义扩展 Kubernetes API](zh/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/)。 +* 也可以学习怎样[使用自定义资源定义扩展 Kubernetes API](/zh/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/)。 * 阅读 [APIService](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#apiservice-v1-apiregistration-k8s-io) 的规范 diff --git a/content/zh/docs/concepts/policy/pod-security-policy.md b/content/zh/docs/concepts/policy/pod-security-policy.md index 2c80607ed4..86d5ac24e7 100644 --- a/content/zh/docs/concepts/policy/pod-security-policy.md +++ b/content/zh/docs/concepts/policy/pod-security-policy.md @@ -1230,7 +1230,7 @@ By default, all safe sysctls are allowed. - Refer to [Pod Security Policy Reference](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podsecuritypolicy-v1beta1-policy) for the api details. --> -- 参阅[Pod 安全标准](zh/docs/concepts/security/pod-security-standards/) +- 参阅[Pod 安全标准](/zh/docs/concepts/security/pod-security-standards/) 了解策略建议。 - 阅读 [Pod 安全策略参考](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podsecuritypolicy-v1beta1-policy)了解 API 细节。 diff --git a/content/zh/docs/concepts/services-networking/connect-applications-service.md b/content/zh/docs/concepts/services-networking/connect-applications-service.md index 3327b43986..bf54a0f2d0 100644 --- a/content/zh/docs/concepts/services-networking/connect-applications-service.md +++ b/content/zh/docs/concepts/services-networking/connect-applications-service.md @@ -316,7 +316,7 @@ The rest of this section will assume you have a Service with a long lived IP 所以可以通过标准做法，使在集群中的任何 Pod 都能与该 Service 通信（例如：`gethostbyname()`）。 如果 CoreDNS 没有在运行，你可以参照 [CoreDNS README](https://github.com/coredns/deployment/tree/master/kubernetes) 或者 -[安装 CoreDNS](/docs/tasks/administer-cluster/coredns/#installing-coredns) 来启用它。 +[安装 CoreDNS](/zh/docs/tasks/administer-cluster/coredns/#installing-coredns) 来启用它。 让我们运行另一个 curl 应用来进行测试： ```shell diff --git a/content/zh/docs/concepts/services-networking/ingress-controllers.md b/content/zh/docs/concepts/services-networking/ingress-controllers.md index 4f8e279da3..a50312b63d 100644 --- a/content/zh/docs/concepts/services-networking/ingress-controllers.md +++ b/content/zh/docs/concepts/services-networking/ingress-controllers.md @@ -63,28 +63,29 @@ Kubernetes 作为一个项目，目前支持和维护 * [Citrix Ingress 控制器](https://github.com/citrix/citrix-k8s-ingress-controller#readme) 可以用来与 Citrix Application Delivery Controller 一起使用。 * [Contour](https://projectcontour.io/) 是一个基于 [Envoy](https://www.envoyproxy.io/) 的 Ingress 控制器。 -* [EnRoute](https://getenroute.io/) 是一个基于 [Envoy](https://www.envoyproxy.io) API 网关能够执行 Ingress 控制器。 +* [EnRoute](https://getenroute.io/) 是一个基于 [Envoy](https://www.envoyproxy.io) API 网关， + 可以作为 Ingress 控制器来执行。 * F5 BIG-IP 的 - [用于 Kubernetes 的容器 Ingress 服务](http://clouddocs.f5.com/products/connectors/k8s-bigip-ctlr/latest) + [用于 Kubernetes 的容器 Ingress 服务](https://clouddocs.f5.com/products/connectors/k8s-bigip-ctlr/latest) 让你能够使用 Ingress 来配置 F5 BIG-IP 虚拟服务器。 * [Gloo](https://gloo.solo.io) 是一个开源的、基于 [Envoy](https://www.envoyproxy.io) 的 Ingress 控制器，能够提供 API 网关功能， -* [HAProxy Ingress](https://haproxy-ingress.github.io/) 针对 [HAProxy](http://www.haproxy.org/#desc) +* [HAProxy Ingress](https://haproxy-ingress.github.io/) 针对 [HAProxy](https://www.haproxy.org/#desc) 的 Ingress 控制器。 * [用于 Kubernetes 的 HAProxy Ingress 控制器](https://github.com/haproxytech/kubernetes-ingress#readme) - 也是一个针对 [HAProxy](http://www.haproxy.org/#desc) 的 Ingress 控制器。 + 也是一个针对 [HAProxy](https://www.haproxy.org/#desc) 的 Ingress 控制器。 * [Istio Ingress](https://istio.io/latest/docs/tasks/traffic-management/ingress/kubernetes-ingress/) 是一个基于 [Istio](https://istio.io/) 的 Ingress 控制器。 * [用于 Kubernetes 的 Kong Ingress 控制器](https://github.com/Kong/kubernetes-ingress-controller#readme) 是一个用来驱动 [Kong Gateway](https://konghq.com/kong/) 的 Ingress 控制器。 @@ -108,7 +109,7 @@ Kubernetes 作为一个项目，目前支持和维护 设计用来作为构造你自己的定制代理的库。 * [Traefik Kubernetes Ingress 提供程序](https://doc.traefik.io/traefik/providers/kubernetes-ingress/) 是一个用于 [Traefik](https://traefik.io/traefik/) 代理的 Ingress 控制器。 -* [Voyager](https://appscode.com/products/voyager) 是一个针对 [HAProxy](http://www.haproxy.org/#desc) +* [Voyager](https://appscode.com/products/voyager) 是一个针对 [HAProxy](https://www.haproxy.org/#desc) 的 Ingress 控制器。 ### 应用程序协议 {#application-protocol} @@ -358,8 +358,8 @@ domain prefixed names such as `mycompany.com/my-custom-protocol`. 此字段的取值会被映射到对应的 Endpoints 和 EndpointSlices 对象。 该字段遵循标准的 Kubernetes 标签语法。 -其值可以是 [IANA 标准服务名称](http://www.iana.org/assignments/service-names)或以域名前缀的名称， -如 `mycompany.com/my-custom-protocol`。 +其值可以是 [IANA 标准服务名称](https://www.iana.org/assignments/service-names) +或以域名为前缀的名称，如 `mycompany.com/my-custom-protocol`。 延迟卷绑定使得调度器在为 PersistentVolumeClaim 选择一个合适的 -PersistentVolume 时能考虑到所有 Pod 的调度限制。 \ No newline at end of file +PersistentVolume 时能考虑到所有 Pod 的调度限制。 From 09c8374fad9707718f8235f355ee0dabe8ea4280 Mon Sep 17 00:00:00 2001 From: MandarJKulkarni <33712629+MandarJKulkarni@users.noreply.github.com> Date: Sun, 31 Jan 2021 18:20:43 +0530 Subject: [PATCH 118/190] Fix typo: simular Fix typo: simular to similar --- .../docs/tutorials/stateful-application/basic-stateful-set.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/tutorials/stateful-application/basic-stateful-set.md b/content/en/docs/tutorials/stateful-application/basic-stateful-set.md index 64073549ef..f43af0e15e 100644 --- a/content/en/docs/tutorials/stateful-application/basic-stateful-set.md +++ b/content/en/docs/tutorials/stateful-application/basic-stateful-set.md @@ -552,7 +552,7 @@ In another terminal, watch the Pods in the StatefulSet: ```shell kubectl get pod -l app=nginx -w ``` -The output is simular to: +The output is similar to: ``` NAME READY STATUS RESTARTS AGE web-0 1/1 Running 0 7m From 9aa16798d3eaf1f0f5c6dcad04e264160c34f6d4 Mon Sep 17 00:00:00 2001 From: ydFu Date: Sun, 31 Jan 2021 21:37:11 +0800 Subject: [PATCH 119/190] [zh] Sync concepts pages for extean-cluster.md Sysc english version in 'Fix links to flexVolume documentation #26214' Signed-off-by: ydFu --- .../extend-kubernetes/extend-cluster.md | 66 ++++++++++--------- 1 file changed, 35 insertions(+), 31 deletions(-) diff --git a/content/zh/docs/concepts/extend-kubernetes/extend-cluster.md b/content/zh/docs/concepts/extend-kubernetes/extend-cluster.md index f70f5df34e..05af5323d9 100644 --- a/content/zh/docs/concepts/extend-kubernetes/extend-cluster.md +++ b/content/zh/docs/concepts/extend-kubernetes/extend-cluster.md @@ -97,7 +97,7 @@ Extensions are software components that extend and deeply integrate with Kuberne They adapt it to support new types and new kinds of hardware. Most cluster administrators will use a hosted or distribution -instance of Kubernetes. As a result, most Kubernetes users will need to +instance of Kubernetes. As a result, most Kubernetes users will not need to install extensions and fewer will need to author new ones. --> ## 扩展程序 {#extension} @@ -105,7 +105,7 @@ install extensions and fewer will need to author new ones. 扩展程序是指对 Kubernetes 进行扩展和深度集成的软件组件。它们适合用于支持新的类型和新型硬件。 大多数集群管理员会使用托管的或统一分发的 Kubernetes 实例。 -因此，大多数 Kubernetes 用户需要安装扩展程序，而且还有少部分用户甚至需要编写新的扩展程序。 +因此，大多数 Kubernetes 用户不需要安装扩展程序，而且还有少部分用户甚至需要编写新的扩展程序。 在 webhook 模型里，Kubernetes 向远程服务发送一个网络请求。 在 *可执行文件插件* 模型里，Kubernetes 执行一个可执行文件（程序）。 可执行文件插件被 kubelet（如 -[Flex 卷插件](https://github.com/kubernetes/community/blob/master/contributors/devel/flexvolume.md)和 -[网络插件](/zh/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/)和 -`kubectl` 所使用。 +[Flex 卷插件](/zh/docs/concepts/storage/volumes/#flexvolume) +和[网络插件](/zh/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/) +和 `kubectl` 所使用。 下图显示了扩展点如何与 Kubernetes 控制平面进行交互。 @@ -184,13 +183,14 @@ This diagram shows the extension points in a Kubernetes system. 1. 用户通常使用 `kubectl` 与 Kubernetes API 进行交互。 @@ -209,9 +209,9 @@ This diagram shows the extension points in a Kubernetes system. 5. Kubernetes 的大部分行为都是由称为控制器（Controllers）的程序实现的，这些程序是 API 服务器的客户端。 控制器通常与自定义资源一起使用。 6. `kubelet` 在主机上运行，并帮助 Pod 看起来就像在集群网络上拥有自己的 IP 的虚拟服务器。 - [网络插件](/zh/docs/concepts/extend-kubernetes/#network-plugins/)让你可以实现不同的 pod 网络。 + [网络插件](/zh/docs/concepts/extend-kubernetes/#network-plugins)让你可以实现不同的 pod 网络。 7. `kubelet` 也负责为容器挂载和卸载卷。新的存储类型可以通过 - [存储插件](/zh/docs/concepts/extend-kubernetes/#storage-plugins/)支持。 + [存储插件](/zh/docs/concepts/extend-kubernetes/#storage-plugins)支持。 ## API 扩展 {#api-extensions} @@ -343,24 +343,23 @@ After a request is authorized, if it is a write operation, it also goes through ### Storage Plugins -[Flex Volumes](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/flexvolume-deployment.md -) allow users to mount volume types without built-in support by having the +[Flex Volumes](/docs/concepts/storage/volumes/#flexvolume) +allow users to mount volume types without built-in support by having the Kubelet call a Binary Plugin to mount the volume. --> ## 基础设施扩展 ### 存储插件 -[Flex Volumes](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/flexvolume-deployment.md -) +[Flex Volumes](/zh/docs/concepts/storage/volumes/#flexvolume) 允许用户挂载无内置插件支持的卷类型，它通过 Kubelet 调用一个可执行文件插件来挂载卷。 ### 设备插件 {#device-plugins} @@ -371,7 +370,8 @@ Plugin](/docs/concepts/cluster-administration/device-plugins/). ### 网络插件 {#network-plugins} @@ -408,17 +408,21 @@ the nodes chosen for a pod. [Webhook](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/scheduling/scheduler_extender.md)， 它允许使用一个 Webhook 后端（调度器扩展程序）为 Pod 筛选节点和确定节点的优先级。 -## {{% heading "whatsnext" %}} +## {{% heading "whatsnext" %}} + + * 详细了解[自定义资源](/zh/docs/concepts/extend-kubernetes/api-extension/custom-resources/) * 了解[动态准入控制](/zh/docs/reference/access-authn-authz/extensible-admission-controllers/) * 详细了解基础设施扩展 From 3625bc5ba59e27d624b6951f85b649e3a0f80c3c Mon Sep 17 00:00:00 2001 From: Sean McGinnis Date: Fri, 29 Jan 2021 16:07:05 -0600 Subject: [PATCH 120/190] Update link to Cinder CSI driver documentation The original link has moved due to docs restructuring in the cloud-provider-openstack repo. Also correcting the capitalization. Signed-off-by: Sean McGinnis --- content/en/docs/concepts/storage/volumes.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/en/docs/concepts/storage/volumes.md b/content/en/docs/concepts/storage/volumes.md index cc5ea38b4e..f8475d6284 100644 --- a/content/en/docs/concepts/storage/volumes.md +++ b/content/en/docs/concepts/storage/volumes.md @@ -210,8 +210,8 @@ spec: The `CSIMigration` feature for Cinder, when enabled, redirects all plugin operations from the existing in-tree plugin to the `cinder.csi.openstack.org` Container -Storage Interface (CSI) Driver. In order to use this feature, the [Openstack Cinder CSI -Driver](https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/using-cinder-csi-plugin.md) +Storage Interface (CSI) Driver. In order to use this feature, the [OpenStack Cinder CSI +Driver](https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/cinder-csi-plugin/using-cinder-csi-plugin.md) must be installed on the cluster and the `CSIMigration` and `CSIMigrationOpenStack` beta features must be enabled. From 72f4f64d04f6e28a7b316a0706286135a0e434c6 Mon Sep 17 00:00:00 2001 From: nasirhm Date: Sun, 31 Jan 2021 18:50:40 +0500 Subject: [PATCH 121/190] :wrench: Modify Kompose page, to remove up Signed-off-by: nasirhm --- .../translate-compose-kubernetes.md | 20 ++----------------- 1 file changed, 2 insertions(+), 18 deletions(-) diff --git a/content/en/docs/tasks/configure-pod-container/translate-compose-kubernetes.md b/content/en/docs/tasks/configure-pod-container/translate-compose-kubernetes.md index 4fadbb3f42..b780baf1f0 100644 --- a/content/en/docs/tasks/configure-pod-container/translate-compose-kubernetes.md +++ b/content/en/docs/tasks/configure-pod-container/translate-compose-kubernetes.md @@ -127,23 +127,7 @@ you need is an existing `docker-compose.yml` file. kompose.service.type: LoadBalancer ``` -2. Run the `kompose up` command to deploy to Kubernetes directly, or skip to - the next step instead to generate a file to use with `kubectl`. - - ```bash - $ kompose up - We are going to create Kubernetes Deployments, Services and PersistentVolumeClaims for your Dockerized application. - If you need different kind of resources, use the 'kompose convert' and 'kubectl apply -f' commands instead. - - INFO Successfully created Service: redis - INFO Successfully created Service: web - INFO Successfully created Deployment: redis - INFO Successfully created Deployment: web - - Your application has been deployed to Kubernetes. You can run 'kubectl get deployment,svc,pods,pvc' for details. - ``` - -3. To convert the `docker-compose.yml` file to files that you can use with +2. To convert the `docker-compose.yml` file to files that you can use with `kubectl`, run `kompose convert` and then `kubectl apply -f `. ```bash @@ -168,7 +152,7 @@ you need is an existing `docker-compose.yml` file. Your deployments are running in Kubernetes. -4. Access your application. +3. Access your application. If you're already using `minikube` for your development process: From b3214813a4ae95128d4a6ffddbd0122bba7b1364 Mon Sep 17 00:00:00 2001 From: Qiming Teng Date: Sun, 31 Jan 2021 21:58:42 +0800 Subject: [PATCH 122/190] Cache scripts locally The current layout partial is using scripts from a site not accessible from behind the dam great firewall. This is a fix to cache the scripts as we do before so that tabs, top menu works for everyone. --- layouts/partials/scripts.html | 28 ++++++++++++++++++++++++++++ static/js/popper-1.14.3.min.js | 5 +++++ 2 files changed, 33 insertions(+) create mode 100644 layouts/partials/scripts.html create mode 100644 static/js/popper-1.14.3.min.js diff --git a/layouts/partials/scripts.html b/layouts/partials/scripts.html new file mode 100644 index 0000000000..2ee8155b11 --- /dev/null +++ b/layouts/partials/scripts.html @@ -0,0 +1,28 @@ + + + + +{{ if .Site.Params.mermaid.enable }} + + +{{ end }} + +{{ $jsBase := resources.Get "js/base.js" }} +{{ $jsAnchor := resources.Get "js/anchor.js" }} +{{ $jsSearch := resources.Get "js/search.js" | resources.ExecuteAsTemplate "js/search.js" .Site.Home }} +{{ $jsMermaid := resources.Get "js/mermaid.js" | resources.ExecuteAsTemplate "js/mermaid.js" . }} +{{ if .Site.Params.offlineSearch }} +{{ $jsSearch = resources.Get "js/offline-search.js" }} +{{ end }} +{{ $js := (slice $jsBase $jsAnchor $jsSearch $jsMermaid) | resources.Concat "js/main.js" }} +{{ if .Site.IsServer }} + +{{ else }} +{{ $js := $js | minify | fingerprint }} + +{{ end }} +{{ with .Site.Params.prism_syntax_highlighting }} + + +{{ end }} +{{ partial "hooks/body-end.html" . }} diff --git a/static/js/popper-1.14.3.min.js b/static/js/popper-1.14.3.min.js new file mode 100644 index 0000000000..79ccbf58b0 --- /dev/null +++ b/static/js/popper-1.14.3.min.js @@ -0,0 +1,5 @@ +/* + Copyright (C) Federico Zivolo 2018 + Distributed under the MIT License (license terms are at http://opensource.org/licenses/MIT). + */(function(e,t){'object'==typeof exports&&'undefined'!=typeof module?module.exports=t():'function'==typeof define&&define.amd?define(t):e.Popper=t()})(this,function(){'use strict';function e(e){return e&&'[object Function]'==={}.toString.call(e)}function t(e,t){if(1!==e.nodeType)return[];var o=getComputedStyle(e,null);return t?o[t]:o}function o(e){return'HTML'===e.nodeName?e:e.parentNode||e.host}function n(e){if(!e)return document.body;switch(e.nodeName){case'HTML':case'BODY':return e.ownerDocument.body;case'#document':return e.body;}var i=t(e),r=i.overflow,p=i.overflowX,s=i.overflowY;return /(auto|scroll|overlay)/.test(r+s+p)?e:n(o(e))}function r(e){return 11===e?re:10===e?pe:re||pe}function p(e){if(!e)return document.documentElement;for(var o=r(10)?document.body:null,n=e.offsetParent;n===o&&e.nextElementSibling;)n=(e=e.nextElementSibling).offsetParent;var i=n&&n.nodeName;return i&&'BODY'!==i&&'HTML'!==i?-1!==['TD','TABLE'].indexOf(n.nodeName)&&'static'===t(n,'position')?p(n):n:e?e.ownerDocument.documentElement:document.documentElement}function s(e){var t=e.nodeName;return'BODY'!==t&&('HTML'===t||p(e.firstElementChild)===e)}function d(e){return null===e.parentNode?e:d(e.parentNode)}function a(e,t){if(!e||!e.nodeType||!t||!t.nodeType)return document.documentElement;var o=e.compareDocumentPosition(t)&Node.DOCUMENT_POSITION_FOLLOWING,n=o?e:t,i=o?t:e,r=document.createRange();r.setStart(n,0),r.setEnd(i,0);var l=r.commonAncestorContainer;if(e!==l&&t!==l||n.contains(i))return s(l)?l:p(l);var f=d(e);return f.host?a(f.host,t):a(e,d(t).host)}function l(e){var t=1=o.clientWidth&&n>=o.clientHeight}),l=0a[e]&&!t.escapeWithReference&&(n=J(f[o],a[e]-('right'===e?f.width:f.height))),ae({},o,n)}};return l.forEach(function(e){var t=-1===['left','top'].indexOf(e)?'secondary':'primary';f=le({},f,m[t](e))}),e.offsets.popper=f,e},priority:['left','right','top','bottom'],padding:5,boundariesElement:'scrollParent'},keepTogether:{order:400,enabled:!0,fn:function(e){var t=e.offsets,o=t.popper,n=t.reference,i=e.placement.split('-')[0],r=Z,p=-1!==['top','bottom'].indexOf(i),s=p?'right':'bottom',d=p?'left':'top',a=p?'width':'height';return o[s]r(n[s])&&(e.offsets.popper[d]=r(n[s])),e}},arrow:{order:500,enabled:!0,fn:function(e,o){var n;if(!q(e.instance.modifiers,'arrow','keepTogether'))return e;var i=o.element;if('string'==typeof i){if(i=e.instance.popper.querySelector(i),!i)return e;}else if(!e.instance.popper.contains(i))return console.warn('WARNING: `arrow.element` must be child of its popper element!'),e;var r=e.placement.split('-')[0],p=e.offsets,s=p.popper,d=p.reference,a=-1!==['left','right'].indexOf(r),l=a?'height':'width',f=a?'Top':'Left',m=f.toLowerCase(),h=a?'left':'top',c=a?'bottom':'right',u=S(i)[l];d[c]-us[c]&&(e.offsets.popper[m]+=d[m]+u-s[c]),e.offsets.popper=g(e.offsets.popper);var b=d[m]+d[l]/2-u/2,y=t(e.instance.popper),w=parseFloat(y['margin'+f],10),E=parseFloat(y['border'+f+'Width'],10),v=b-e.offsets.popper[m]-w-E;return v=$(J(s[l]-u,v),0),e.arrowElement=i,e.offsets.arrow=(n={},ae(n,m,Q(v)),ae(n,h,''),n),e},element:'[x-arrow]'},flip:{order:600,enabled:!0,fn:function(e,t){if(W(e.instance.modifiers,'inner'))return e;if(e.flipped&&e.placement===e.originalPlacement)return e;var o=v(e.instance.popper,e.instance.reference,t.padding,t.boundariesElement,e.positionFixed),n=e.placement.split('-')[0],i=T(n),r=e.placement.split('-')[1]||'',p=[];switch(t.behavior){case he.FLIP:p=[n,i];break;case he.CLOCKWISE:p=z(n);break;case he.COUNTERCLOCKWISE:p=z(n,!0);break;default:p=t.behavior;}return p.forEach(function(s,d){if(n!==s||p.length===d+1)return e;n=e.placement.split('-')[0],i=T(n);var a=e.offsets.popper,l=e.offsets.reference,f=Z,m='left'===n&&f(a.right)>f(l.left)||'right'===n&&f(a.left)f(l.top)||'bottom'===n&&f(a.top)f(o.right),g=f(a.top)f(o.bottom),b='left'===n&&h||'right'===n&&c||'top'===n&&g||'bottom'===n&&u,y=-1!==['top','bottom'].indexOf(n),w=!!t.flipVariations&&(y&&'start'===r&&h||y&&'end'===r&&c||!y&&'start'===r&&g||!y&&'end'===r&&u);(m||b||w)&&(e.flipped=!0,(m||b)&&(n=p[d+1]),w&&(r=G(r)),e.placement=n+(r?'-'+r:''),e.offsets.popper=le({},e.offsets.popper,C(e.instance.popper,e.offsets.reference,e.placement)),e=P(e.instance.modifiers,e,'flip'))}),e},behavior:'flip',padding:5,boundariesElement:'viewport'},inner:{order:700,enabled:!1,fn:function(e){var t=e.placement,o=t.split('-')[0],n=e.offsets,i=n.popper,r=n.reference,p=-1!==['left','right'].indexOf(o),s=-1===['top','left'].indexOf(o);return i[p?'left':'top']=r[o]-(s?i[p?'width':'height']:0),e.placement=T(t),e.offsets.popper=g(i),e}},hide:{order:800,enabled:!0,fn:function(e){if(!q(e.instance.modifiers,'hide','preventOverflow'))return e;var t=e.offsets.reference,o=D(e.instance.modifiers,function(e){return'preventOverflow'===e.name}).boundaries;if(t.bottomo.right||t.top>o.bottom||t.right Date: Sun, 31 Jan 2021 23:48:01 +0800 Subject: [PATCH 123/190] Update job.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 1. sync from en localization 2. minor fix on translation on specific words: graceful, exponential 1. 更正文章英文引用以及翻译 2. 个别词小改动：graceful, exponential --- content/zh/docs/concepts/workloads/controllers/job.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/content/zh/docs/concepts/workloads/controllers/job.md b/content/zh/docs/concepts/workloads/controllers/job.md index fa27f617d1..e5b59620a0 100644 --- a/content/zh/docs/concepts/workloads/controllers/job.md +++ b/content/zh/docs/concepts/workloads/controllers/job.md @@ -22,7 +22,7 @@ weight: 50 -Job 会创建一个或者多个 Pods，并确保指定数量的 Pods 成功终止。 +Job 会创建一个或者多个 Pods，并会持续尝试执行重试直到指定数量的 Pods 成功终止。 随着 Pods 成功结束，Job 跟踪记录成功完成的 Pods 个数。 当数量达到指定的成功个数阈值时，任务（即 Job）结束。 删除 Job 的操作会清除所创建的全部 Pods。 @@ -311,7 +311,7 @@ parallelism, for a variety of reasons: - 如果 Job 控制器因为任何原因（例如，缺少 `ResourceQuota` 或者没有权限）无法创建 Pods。 Pods 个数可能比请求的数目小。 - Job 控制器可能会因为之前同一 Job 中 Pod 失效次数过多而压制新 Pod 的创建。 -- 当 Pod 处于体面终止进程中，需要一定时间才能停止。 +- 当 Pod 处于优雅终止进程中，需要一定时间才能停止。 +### API 服务器标识 + +{{< feature-state state="alpha" for_k8s_version="v1.20" >}} + + +使用 API 服务器标识功能需要启用[特性门控](/zh/docs/reference/command-line-tools-reference/feature-gates/)， +该功能默认不启用。 +你可以在启动 {{< glossary_tooltip text="API 服务器" term_id="kube-apiserver" >}} 的时候启用特性门控 `APIServerIdentity` 来激活 API 服务器标识： + + +```shell +kube-apiserver \ +--feature-gates=APIServerIdentity=true \ + # …其他标记照常 +``` + + +在启动引导过程中，每个 kube-apiserver 会给自己分配一个唯一 ID。 +该 ID 的格式是 `kube-apiserver-{UUID}`。 +每个 kube-apiserver 会在 _kube-system_ {{< glossary_tooltip text="名字空间" term_id="namespace">}} 里创建一个 [`Lease` 对象](/docs/reference/generated/kubernetes-api/{{< param "version" >}}//#lease-v1-coordination-k8s-io)。 + +`Lease` 对象的名字是 kube-apiserver 的唯一 ID。 +`Lease` 对象包含一个标签 `k8s.io/component=kube-apiserver`。 +每个 kube-apiserver 每过 `IdentityLeaseRenewIntervalSeconds`（默认是 10 秒）就会刷新它的 `Lease` 对象。 +每个 kube-apiserver 每过 `IdentityLeaseDurationSeconds`（默认是 3600 秒）也会检查所有 kube-apiserver 的标识 `Lease` 对象， +并且会删除超过 `IdentityLeaseDurationSeconds` 时间还没被刷新的 `Lease` 对象。 +可以在 kube-apiserver 的 `identity-lease-renew-interval-seconds` +和 `identity-lease-duration-seconds` 标记里配置 `IdentityLeaseRenewIntervalSeconds` 和 `IdentityLeaseDurationSeconds`。 + + +启用该功能是使用 HA API 服务器协调相关功能（例如，`StorageVersionAPI` 特性门控）的前提条件。 + -Job 会创建一个或者多个 Pods，并会持续尝试执行重试直到指定数量的 Pods 成功终止。 +Job 会创建一个或者多个 Pods，并会持续重试直到指定数量的 Pods 成功终止。 随着 Pods 成功结束，Job 跟踪记录成功完成的 Pods 个数。 当数量达到指定的成功个数阈值时，任务（即 Job）结束。 删除 Job 的操作会清除所创建的全部 Pods。 @@ -311,7 +311,7 @@ parallelism, for a variety of reasons: - 如果 Job 控制器因为任何原因（例如，缺少 `ResourceQuota` 或者没有权限）无法创建 Pods。 Pods 个数可能比请求的数目小。 - Job 控制器可能会因为之前同一 Job 中 Pod 失效次数过多而压制新 Pod 的创建。 -- 当 Pod 处于优雅终止进程中，需要一定时间才能停止。 +- 当 Pod 处于体面终止进程中，需要一定时间才能停止。 更多信息请参考 [kubernetes-sigs/cri-tools](https://github.com/kubernetes-sigs/cri-tools)。 + +## Docker CLI 和 crictl 的映射 + + +以下的映射表格只适用于 Docker CLI v1.40 和 crictl v1.19.0 版本。 +请注意该表格并不详尽。例如，其中不包含 Docker CLI 的实验性命令。 + + +{{< note >}} +尽管有些命令的输出缺少了一些数据列，CRICTL 的输出格式与 Docker CLI 是类似的。 +如果你的脚本程序需要解析命令的输出，请确认检查该特定命令的输出。 +{{< /note >}} + + +### 获取调试信息 + +{{< table caption="Docker CLI 和 crictl 的映射 - 获取调试信息" >}} + +docker cli | crictl | 描述 | 不支持的功能 +-- | -- | -- | -- +`attach` | `attach` | 连接到一个运行中的容器 | `--detach-keys`, `--sig-proxy` +`exec` | `exec` | 在运行中的容器里运行一个命令 | `--privileged`, `--user`, `--detach-keys` +`images` | `images` | 列举镜像 |   +`info` | `info` | 显示系统级的信息 |   +`inspect` | `inspect`, `inspecti` | 返回容器、镜像或者任务的详细信息 |   +`logs` | `logs` | 获取容器的日志 | `--details` +`ps` | `ps` | 列举容器 |   +`stats` | `stats` | 实时显示容器的资源使用统计信息 | 列：NET/BLOCK I/O, PIDs +`version` | `version` | 显示运行时（Docker、ContainerD、或者其他) 的版本信息 |   +{{< /table >}} + + +### 进行改动 + +{{< table caption="Docker CLI 和 crictl 的映射 - 进行改动" >}} + +docker cli | crictl | 描述 | 不支持的功能 +-- | -- | -- | -- +`create` | `create` | 创建一个新的容器 |   +`kill` | `stop` (timeout=0) | 杀死一个或多个正在运行的容器 | `--signal` +`pull` | `pull` | 从镜像仓库拉取镜像或者代码仓库 | `--all-tags`, `--disable-content-trust` +`rm` | `rm` | 移除一个或多个容器 |   +`rmi` | `rmi` | 移除一个或多个镜像 |   +`run` | `run` | 在新容器里运行一个命令 |   +`start` | `start` | 启动一个或多个停止的容器 | `--detach-keys` +`stop` | `stop` | 停止一个或多个正运行的容器 |   +`update` | `update` | 更新一个或多个容器的配置 | CRI 不支持 `--restart`、`--blkio-weight` 以及一些其他的资源限制选项。 +{{< /table >}} + + +### 仅 crictl 支持 + +{{< table caption="Docker CLI 和 crictl 的映射 - 仅 crictl 支持" >}} + +crictl | 描述 +-- | -- +`imagefsinfo` | 返回镜像的文件系统信息 +`inspectp` | 显示一个或多个 Pod 的状态 +`port-forward` | 转发本地端口到 Pod +`pods` | 列举 Pod +`runp` | 运行一个新的 Pod +`rmp` | 移除一个或多个 Pod +`stopp` | 停止一个或多个正运行的 Pod +{{< /table >}} From 0b09ec82fffbd2b83567cb111adec5bf07d567a9 Mon Sep 17 00:00:00 2001 From: ydFu Date: Mon, 1 Feb 2021 21:41:45 +0800 Subject: [PATCH 133/190] [zh] Sync setup pages for kubeadm/create-cluster-kubeadm.md * sync with english version in 'docs: rephrase sentence missing predicate (#26259)' Signed-off-by: ydFu --- .../tools/kubeadm/create-cluster-kubeadm.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/content/zh/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm.md b/content/zh/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm.md index 81a70abe77..6fa2bb7158 100644 --- a/content/zh/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm.md +++ b/content/zh/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm.md @@ -17,11 +17,12 @@ weight: 30 -创建一个符合最佳实践的最小化 Kubernetes 集群。事实上，你可以使用 `kubeadm` 配置一个通过 [Kubernetes 一致性测试](https://kubernetes.io/blog/2017/10/software-conformance-certification) 的集群。 +使用 `kubeadm`，你 +能创建一个符合最佳实践的最小化 Kubernetes 集群。事实上，你可以使用 `kubeadm` 配置一个通过 [Kubernetes 一致性测试](https://kubernetes.io/blog/2017/10/software-conformance-certification) 的集群。 `kubeadm` 还支持其他集群生命周期功能， 例如 [启动引导令牌](/zh/docs/reference/access-authn-authz/bootstrap-tokens/) 和集群升级。 @@ -265,9 +266,9 @@ kubeadm 不支持将没有 `--control-plane-endpoint` 参数的单个控制平 ### 更多信息 -有关 `kubeadm init` 参数的更多信息，请参见 [kubeadm 参考指南](/zh/docs/reference/setup-tools/kubeadm/kubeadm-init/)。 +有关 `kubeadm init` 参数的更多信息，请参见 [kubeadm 参考指南](/zh/docs/reference/setup-tools/kubeadm/kubeadm/)。 当你部署完 Kubernetes, 即拥有了一个完整的集群。 -{{< glossary_definition term_id="cluster" length="all" prepend="一个 Kubernetes 集群包含">}} +{{< glossary_definition term_id="cluster" length="all" prepend="一个 Kubernetes">}} 本文档概述了交付正常运行的 Kubernetes 集群所需的各种组件。 From bfea67aa82065e600394b41c0d8ce3e75c69b7ed Mon Sep 17 00:00:00 2001 From: Luke Addison Date: Mon, 1 Feb 2021 16:47:44 +0000 Subject: [PATCH 135/190] Clarify PreStop hook behaviour --- .../containers/container-lifecycle-hooks.md | 32 +++++++++---------- 1 file changed, 15 insertions(+), 17 deletions(-) diff --git a/content/en/docs/concepts/containers/container-lifecycle-hooks.md b/content/en/docs/concepts/containers/container-lifecycle-hooks.md index b315ba6f59..c2811fb118 100644 --- a/content/en/docs/concepts/containers/container-lifecycle-hooks.md +++ b/content/en/docs/concepts/containers/container-lifecycle-hooks.md @@ -36,10 +36,12 @@ No parameters are passed to the handler. `PreStop` -This hook is called immediately before a container is terminated due to an API request or management event such as liveness probe failure, preemption, resource contention and others. A call to the preStop hook fails if the container is already in terminated or completed state. -It is blocking, meaning it is synchronous, -so it must complete before the signal to stop the container can be sent. -No parameters are passed to the handler. +This hook is called immediately before a container is terminated due to an API request or management +event such as a liveness/startup probe failure, preemption, resource contention and others. This +hook is not called if the container crashes or exits. The Pod's termination grace period countdown +begins before the `PreStop` hook is executed. Regardless of the outcome of the handler, the +container will eventually terminate within the Pod's termination grace period. No parameters are +passed to the handler. A more detailed description of the termination behavior can be found in [Termination of Pods](/docs/concepts/workloads/pods/pod-lifecycle/#pod-termination). @@ -65,19 +67,15 @@ the Container ENTRYPOINT and hook fire asynchronously. However, if the hook takes too long to run or hangs, the Container cannot reach a `running` state. -`PreStop` hooks are not executed asynchronously from the signal -to stop the Container; the hook must complete its execution before -the signal can be sent. -If a `PreStop` hook hangs during execution, -the Pod's phase will be `Terminating` and remain there until the Pod is -killed after its `terminationGracePeriodSeconds` expires. -This grace period applies to the total time it takes for both -the `PreStop` hook to execute and for the Container to stop normally. -If, for example, `terminationGracePeriodSeconds` is 60, and the hook -takes 55 seconds to complete, and the Container takes 10 seconds to stop -normally after receiving the signal, then the Container will be killed -before it can stop normally, since `terminationGracePeriodSeconds` is -less than the total time (55+10) it takes for these two things to happen. +`PreStop` hooks are not executed asynchronously from the signal to stop the Container; the hook must +complete its execution before the TERM signal can be sent. If a `PreStop` hook hangs during +execution, the Pod's phase will be `Terminating` and remain there until the Pod is killed after its +`terminationGracePeriodSeconds` expires. This grace period applies to the total time it takes for +both the `PreStop` hook to execute and for the Container to stop normally. If, for example, +`terminationGracePeriodSeconds` is 60, and the hook takes 55 seconds to complete, and the Container +takes 10 seconds to stop normally after receiving the signal, then the Container will be killed +before it can stop normally, since `terminationGracePeriodSeconds` is less than the total time +(55+10) it takes for these two things to happen. If either a `PostStart` or `PreStop` hook fails, it kills the Container. From 7d9916af0cc41bf500df7e07412286ee237eda26 Mon Sep 17 00:00:00 2001 From: Karen Bradshaw Date: Mon, 1 Feb 2021 15:14:25 -0500 Subject: [PATCH 136/190] clean up use of word: easy --- .../docs/concepts/cluster-administration/_index.md | 4 ++-- .../cluster-administration/manage-deployment.md | 14 +++++++++++--- .../concepts/cluster-administration/system-logs.md | 10 ++++++---- content/en/docs/concepts/configuration/overview.md | 4 ++-- .../docs/concepts/overview/what-is-kubernetes.md | 2 +- .../overview/working-with-objects/labels.md | 2 +- .../working-with-objects/object-management.md | 4 ++-- .../scheduling-eviction/scheduling-framework.md | 8 +++----- .../workloads/controllers/replicationcontroller.md | 3 ++- .../reference/access-authn-authz/authentication.md | 14 ++++++-------- .../extensible-admission-controllers.md | 4 ++-- content/en/docs/reference/glossary/wg.md | 5 ++--- .../custom-resource-definition-v1.md | 12 ++++++------ .../mutating-webhook-configuration-v1.md | 12 ++++++------ .../validating-webhook-configuration-v1.md | 12 ++++++------ .../setup-tools/kubeadm/implementation-details.md | 2 +- .../reference/setup-tools/kubeadm/kubeadm-join.md | 2 +- content/en/docs/reference/tools.md | 4 ++-- .../docs/reference/using-api/server-side-apply.md | 2 +- .../setup/production-environment/tools/kops.md | 4 ++-- .../windows/intro-windows-in-kubernetes.md | 2 +- .../administer-cluster/configure-upgrade-etcd.md | 2 +- .../limit-storage-consumption.md | 2 +- .../custom-resource-definition-versioning.md | 5 ++--- .../job/coarse-parallel-processing-work-queue.md | 2 +- .../run-application/horizontal-pod-autoscale.md | 2 +- .../docs/tutorials/kubernetes-basics/_index.html | 2 +- .../tutorials/stateless-application/guestbook.md | 2 +- 28 files changed, 74 insertions(+), 69 deletions(-) diff --git a/content/en/docs/concepts/cluster-administration/_index.md b/content/en/docs/concepts/cluster-administration/_index.md index 14b8165ebf..cac156b53b 100644 --- a/content/en/docs/concepts/cluster-administration/_index.md +++ b/content/en/docs/concepts/cluster-administration/_index.md @@ -26,12 +26,12 @@ See the guides in [Setup](/docs/setup/) for examples of how to plan, set up, and Before choosing a guide, here are some considerations: - - Do you just want to try out Kubernetes on your computer, or do you want to build a high-availability, multi-node cluster? Choose distros best suited for your needs. + - Do you want to try out Kubernetes on your computer, or do you want to build a high-availability, multi-node cluster? Choose distros best suited for your needs. - Will you be using **a hosted Kubernetes cluster**, such as [Google Kubernetes Engine](https://cloud.google.com/kubernetes-engine/), or **hosting your own cluster**? - Will your cluster be **on-premises**, or **in the cloud (IaaS)**? Kubernetes does not directly support hybrid clusters. Instead, you can set up multiple clusters. - **If you are configuring Kubernetes on-premises**, consider which [networking model](/docs/concepts/cluster-administration/networking/) fits best. - Will you be running Kubernetes on **"bare metal" hardware** or on **virtual machines (VMs)**? - - Do you **just want to run a cluster**, or do you expect to do **active development of Kubernetes project code**? If the + - Do you **want to run a cluster**, or do you expect to do **active development of Kubernetes project code**? If the latter, choose an actively-developed distro. Some distros only use binary releases, but offer a greater variety of choices. - Familiarize yourself with the [components](/docs/concepts/overview/components/) needed to run a cluster. diff --git a/content/en/docs/concepts/cluster-administration/manage-deployment.md b/content/en/docs/concepts/cluster-administration/manage-deployment.md index 50ed69ff42..f0ccaf61f9 100644 --- a/content/en/docs/concepts/cluster-administration/manage-deployment.md +++ b/content/en/docs/concepts/cluster-administration/manage-deployment.md @@ -70,7 +70,7 @@ deployment.apps "my-nginx" deleted service "my-nginx-svc" deleted ``` -In the case of just two resources, it's also easy to specify both on the command line using the resource/name syntax: +In the case of two resources, you can specify both resources on the command line using the resource/name syntax: ```shell kubectl delete deployments/my-nginx services/my-nginx-svc @@ -87,7 +87,7 @@ deployment.apps "my-nginx" deleted service "my-nginx-svc" deleted ``` -Because `kubectl` outputs resource names in the same syntax it accepts, it's easy to chain operations using `$()` or `xargs`: +Because `kubectl` outputs resource names in the same syntax it accepts, you can chain operations using `$()` or `xargs`: ```shell kubectl get $(kubectl create -f docs/concepts/cluster-administration/nginx/ -o name | grep service) @@ -301,6 +301,7 @@ Sometimes you would want to attach annotations to resources. Annotations are arb kubectl annotate pods my-nginx-v4-9gw19 description='my frontend running nginx' kubectl get pods my-nginx-v4-9gw19 -o yaml ``` + ```shell apiVersion: v1 kind: pod @@ -314,11 +315,12 @@ For more information, please see [annotations](/docs/concepts/overview/working-w ## Scaling your application -When load on your application grows or shrinks, it's easy to scale with `kubectl`. For instance, to decrease the number of nginx replicas from 3 to 1, do: +When load on your application grows or shrinks, use `kubectl` to scale your application. For instance, to decrease the number of nginx replicas from 3 to 1, do: ```shell kubectl scale deployment/my-nginx --replicas=1 ``` + ```shell deployment.apps/my-nginx scaled ``` @@ -328,6 +330,7 @@ Now you only have one pod managed by the deployment. ```shell kubectl get pods -l app=nginx ``` + ```shell NAME READY STATUS RESTARTS AGE my-nginx-2035384211-j5fhi 1/1 Running 0 30m @@ -338,6 +341,7 @@ To have the system automatically choose the number of nginx replicas as needed, ```shell kubectl autoscale deployment/my-nginx --min=1 --max=3 ``` + ```shell horizontalpodautoscaler.autoscaling/my-nginx autoscaled ``` @@ -411,6 +415,7 @@ In some cases, you may need to update resource fields that cannot be updated onc ```shell kubectl replace -f https://k8s.io/examples/application/nginx/nginx-deployment.yaml --force ``` + ```shell deployment.apps/my-nginx deleted deployment.apps/my-nginx replaced @@ -427,14 +432,17 @@ Let's say you were running version 1.14.2 of nginx: ```shell kubectl create deployment my-nginx --image=nginx:1.14.2 ``` + ```shell deployment.apps/my-nginx created ``` with 3 replicas (so the old and new revisions can coexist): + ```shell kubectl scale deployment my-nginx --current-replicas=1 --replicas=3 ``` + ``` deployment.apps/my-nginx scaled ``` diff --git a/content/en/docs/concepts/cluster-administration/system-logs.md b/content/en/docs/concepts/cluster-administration/system-logs.md index 9ccae3adf7..0466837356 100644 --- a/content/en/docs/concepts/cluster-administration/system-logs.md +++ b/content/en/docs/concepts/cluster-administration/system-logs.md @@ -31,22 +31,24 @@ I1025 00:15:15.525108 1 httplog.go:79] GET /api/v1/namespaces/kube-system/ {{< feature-state for_k8s_version="v1.19" state="alpha" >}} -{{}} +{{< warning >}} Migration to structured log messages is an ongoing process. Not all log messages are structured in this version. When parsing log files, you must also handle unstructured log messages. Log formatting and value serialization are subject to change. {{< /warning>}} -Structured logging is a effort to introduce a uniform structure in log messages allowing for easy extraction of information, making logs easier and cheaper to store and process. +Structured logging introduces a uniform structure in log messages allowing for programmatic extraction of information. You can store and process structured logs with less effort and cost. New message format is backward compatible and enabled by default. Format of structured logs: -``` + +```ini "" ="" ="" ... ``` Example: -``` + +```ini I1025 00:15:15.525108 1 controller_utils.go:116] "Pod status updated" pod="kube-system/kubedns" status="ready" ``` diff --git a/content/en/docs/concepts/configuration/overview.md b/content/en/docs/concepts/configuration/overview.md index 2239fe08ac..fce8a0c7a8 100644 --- a/content/en/docs/concepts/configuration/overview.md +++ b/content/en/docs/concepts/configuration/overview.md @@ -59,13 +59,13 @@ DNS server watches the Kubernetes API for new `Services` and creates a set of DN - Avoid using `hostNetwork`, for the same reasons as `hostPort`. -- Use [headless Services](/docs/concepts/services-networking/service/#headless-services) (which have a `ClusterIP` of `None`) for easy service discovery when you don't need `kube-proxy` load balancing. +- Use [headless Services](/docs/concepts/services-networking/service/#headless-services) (which have a `ClusterIP` of `None`) for service discovery when you don't need `kube-proxy` load balancing. ## Using Labels - Define and use [labels](/docs/concepts/overview/working-with-objects/labels/) that identify __semantic attributes__ of your application or Deployment, such as `{ app: myapp, tier: frontend, phase: test, deployment: v3 }`. You can use these labels to select the appropriate Pods for other resources; for example, a Service that selects all `tier: frontend` Pods, or all `phase: test` components of `app: myapp`. See the [guestbook](https://github.com/kubernetes/examples/tree/{{< param "githubbranch" >}}/guestbook/) app for examples of this approach. -A Service can be made to span multiple Deployments by omitting release-specific labels from its selector. [Deployments](/docs/concepts/workloads/controllers/deployment/) make it easy to update a running service without downtime. +A Service can be made to span multiple Deployments by omitting release-specific labels from its selector. When you need to update a running service without downtime, use a [Deployment](/docs/concepts/workloads/controllers/deployment/). A desired state of an object is described by a Deployment, and if changes to that spec are _applied_, the deployment controller changes the actual state to the desired state at a controlled rate. diff --git a/content/en/docs/concepts/overview/what-is-kubernetes.md b/content/en/docs/concepts/overview/what-is-kubernetes.md index f3c4a1da02..b19c4155ce 100644 --- a/content/en/docs/concepts/overview/what-is-kubernetes.md +++ b/content/en/docs/concepts/overview/what-is-kubernetes.md @@ -43,7 +43,7 @@ Each VM is a full machine running all the components, including its own operatin Containers have become popular because they provide extra benefits, such as: * Agile application creation and deployment: increased ease and efficiency of container image creation compared to VM image use. -* Continuous development, integration, and deployment: provides for reliable and frequent container image build and deployment with quick and easy rollbacks (due to image immutability). +* Continuous development, integration, and deployment: provides for reliable and frequent container image build and deployment with quick and efficient rollbacks (due to image immutability). * Dev and Ops separation of concerns: create application container images at build/release time rather than deployment time, thereby decoupling applications from infrastructure. * Observability not only surfaces OS-level information and metrics, but also application health and other signals. * Environmental consistency across development, testing, and production: Runs the same on a laptop as it does in the cloud. diff --git a/content/en/docs/concepts/overview/working-with-objects/labels.md b/content/en/docs/concepts/overview/working-with-objects/labels.md index 560ab23dff..2feec43801 100644 --- a/content/en/docs/concepts/overview/working-with-objects/labels.md +++ b/content/en/docs/concepts/overview/working-with-objects/labels.md @@ -42,7 +42,7 @@ Example labels: * `"partition" : "customerA"`, `"partition" : "customerB"` * `"track" : "daily"`, `"track" : "weekly"` -These are just examples of commonly used labels; you are free to develop your own conventions. Keep in mind that label Key must be unique for a given object. +These are examples of commonly used labels; you are free to develop your own conventions. Keep in mind that label Key must be unique for a given object. ## Syntax and character set diff --git a/content/en/docs/concepts/overview/working-with-objects/object-management.md b/content/en/docs/concepts/overview/working-with-objects/object-management.md index a2dd737e77..b85c622823 100644 --- a/content/en/docs/concepts/overview/working-with-objects/object-management.md +++ b/content/en/docs/concepts/overview/working-with-objects/object-management.md @@ -31,7 +31,7 @@ When using imperative commands, a user operates directly on live objects in a cluster. The user provides operations to the `kubectl` command as arguments or flags. -This is the simplest way to get started or to run a one-off task in +This is the recommended way to get started or to run a one-off task in a cluster. Because this technique operates directly on live objects, it provides no history of previous configurations. @@ -47,7 +47,7 @@ kubectl create deployment nginx --image nginx Advantages compared to object configuration: -- Commands are simple, easy to learn and easy to remember. +- Commands are expressed as a single action word. - Commands require only a single step to make changes to the cluster. Disadvantages compared to object configuration: diff --git a/content/en/docs/concepts/scheduling-eviction/scheduling-framework.md b/content/en/docs/concepts/scheduling-eviction/scheduling-framework.md index 61b6619d01..ae32f840fd 100644 --- a/content/en/docs/concepts/scheduling-eviction/scheduling-framework.md +++ b/content/en/docs/concepts/scheduling-eviction/scheduling-framework.md @@ -10,11 +10,9 @@ weight: 70 {{< feature-state for_k8s_version="v1.15" state="alpha" >}} -The scheduling framework is a pluggable architecture for Kubernetes Scheduler -that makes scheduler customizations easy. It adds a new set of "plugin" APIs to -the existing scheduler. Plugins are compiled into the scheduler. The APIs -allow most scheduling features to be implemented as plugins, while keeping the -scheduling "core" simple and maintainable. Refer to the [design proposal of the +The scheduling framework is a pluggable architecture for the Kubernetes scheduler. +It adds a new set of "plugin" APIs to the existing scheduler. Plugins are compiled into the scheduler. The APIs allow most scheduling features to be implemented as plugins, while keeping the +scheduling "core" lightweight and maintainable. Refer to the [design proposal of the scheduling framework][kep] for more technical information on the design of the framework. diff --git a/content/en/docs/concepts/workloads/controllers/replicationcontroller.md b/content/en/docs/concepts/workloads/controllers/replicationcontroller.md index 36ae4a880a..a6427bedb3 100644 --- a/content/en/docs/concepts/workloads/controllers/replicationcontroller.md +++ b/content/en/docs/concepts/workloads/controllers/replicationcontroller.md @@ -208,7 +208,8 @@ As mentioned above, whether you have 1 pod you want to keep running, or 1000, a ### Scaling -The ReplicationController makes it easy to scale the number of replicas up or down, either manually or by an auto-scaling control agent, by simply updating the `replicas` field. +The ReplicationController scales the number of replicas up or down by setting the `replicas` field. +You can configure the ReplicationController to manage the replicas manually or by an auto-scaling control agent. ### Rolling updates diff --git a/content/en/docs/reference/access-authn-authz/authentication.md b/content/en/docs/reference/access-authn-authz/authentication.md index c385a15fda..83c04069e0 100644 --- a/content/en/docs/reference/access-authn-authz/authentication.md +++ b/content/en/docs/reference/access-authn-authz/authentication.md @@ -68,8 +68,8 @@ when interpreted by an [authorizer](/docs/reference/access-authn-authz/authoriza You can enable multiple authentication methods at once. You should usually use at least two methods: - - service account tokens for service accounts - - at least one other method for user authentication. +- service account tokens for service accounts +- at least one other method for user authentication. When multiple authenticator modules are enabled, the first module to successfully authenticate the request short-circuits evaluation. @@ -321,13 +321,11 @@ sequenceDiagram 9. `kubectl` provides feedback to the user Since all of the data needed to validate who you are is in the `id_token`, Kubernetes doesn't need to -"phone home" to the identity provider. In a model where every request is stateless this provides a very scalable -solution for authentication. It does offer a few challenges: - -1. Kubernetes has no "web interface" to trigger the authentication process. There is no browser or interface to collect credentials which is why you need to authenticate to your identity provider first. -2. The `id_token` can't be revoked, it's like a certificate so it should be short-lived (only a few minutes) so it can be very annoying to have to get a new token every few minutes. -3. There's no easy way to authenticate to the Kubernetes dashboard without using the `kubectl proxy` command or a reverse proxy that injects the `id_token`. +"phone home" to the identity provider. In a model where every request is stateless this provides a very scalable solution for authentication. It does offer a few challenges: +1. Kubernetes has no "web interface" to trigger the authentication process. There is no browser or interface to collect credentials which is why you need to authenticate to your identity provider first. +2. The `id_token` can't be revoked, it's like a certificate so it should be short-lived (only a few minutes) so it can be very annoying to have to get a new token every few minutes. +3. To authenticate to the Kubernetes dashboard, you must the `kubectl proxy` command or a reverse proxy that injects the `id_token`. #### Configuring the API Server diff --git a/content/en/docs/reference/access-authn-authz/extensible-admission-controllers.md b/content/en/docs/reference/access-authn-authz/extensible-admission-controllers.md index a3f4f9c5b9..018195f817 100644 --- a/content/en/docs/reference/access-authn-authz/extensible-admission-controllers.md +++ b/content/en/docs/reference/access-authn-authz/extensible-admission-controllers.md @@ -1093,8 +1093,8 @@ be a layering violation). `host` may also be an IP address. Please note that using `localhost` or `127.0.0.1` as a `host` is risky unless you take great care to run this webhook on all hosts which run an apiserver which might need to make calls to this -webhook. Such installs are likely to be non-portable, i.e., not easy -to turn up in a new cluster. +webhook. Such installations are likely to be non-portable or not readily +run in a new cluster. The scheme must be "https"; the URL must begin with "https://". diff --git a/content/en/docs/reference/glossary/wg.md b/content/en/docs/reference/glossary/wg.md index 2a3b8786f6..89ea85fca7 100755 --- a/content/en/docs/reference/glossary/wg.md +++ b/content/en/docs/reference/glossary/wg.md @@ -12,9 +12,8 @@ tags: --- Facilitates the discussion and/or implementation of a short-lived, narrow, or decoupled project for a committee, {{< glossary_tooltip text="SIG" term_id="sig" >}}, or cross-SIG effort. - + -Working groups are a way of organizing people to accomplish a discrete task, and are relatively easy to create and deprecate when inactive. +Working groups are a way of organizing people to accomplish a discrete task. For more information, see the [kubernetes/community](https://github.com/kubernetes/community) repo and the current list of [SIGs and working groups](https://github.com/kubernetes/community/blob/master/sig-list.md). - diff --git a/content/en/docs/reference/kubernetes-api/extend-resources/custom-resource-definition-v1.md b/content/en/docs/reference/kubernetes-api/extend-resources/custom-resource-definition-v1.md index 3fa3bc9a74..8aa8ed0c78 100644 --- a/content/en/docs/reference/kubernetes-api/extend-resources/custom-resource-definition-v1.md +++ b/content/en/docs/reference/kubernetes-api/extend-resources/custom-resource-definition-v1.md @@ -250,15 +250,15 @@ CustomResourceDefinitionSpec describes how a user wants their resource to appear - **conversion.webhook.clientConfig.url** (string) url gives the location of the webhook, in standard URL form (`scheme://host:port/path`). Exactly one of `url` or `service` must be specified. - + The `host` should not refer to a service running in the cluster; use the `service` field instead. The host might be resolved via external DNS in some apiservers (e.g., `kube-apiserver` cannot resolve in-cluster DNS as that would be a layering violation). `host` may also be an IP address. - - Please note that using `localhost` or `127.0.0.1` as a `host` is risky unless you take great care to run this webhook on all hosts which run an apiserver which might need to make calls to this webhook. Such installs are likely to be non-portable, i.e., not easy to turn up in a new cluster. - + + Please note that using `localhost` or `127.0.0.1` as a `host` is risky unless you take great care to run this webhook on all hosts which run an apiserver which might need to make calls to this webhook. Such installations are likely to be non-portable or not readily run in a new cluster. + The scheme must be "https"; the URL must begin with "https://". - + A path is optional, and if present may be any string permissible in a URL. You may use the path to pass an arbitrary string to the webhook, for example, a cluster identifier. - + Attempting to use a user or basic auth e.g. "user:password@" is not allowed. Fragments ("#...") and query parameters ("?...") are not allowed, either. - **preserveUnknownFields** (boolean) diff --git a/content/en/docs/reference/kubernetes-api/extend-resources/mutating-webhook-configuration-v1.md b/content/en/docs/reference/kubernetes-api/extend-resources/mutating-webhook-configuration-v1.md index ac32c9aaaa..9e17c11ae3 100644 --- a/content/en/docs/reference/kubernetes-api/extend-resources/mutating-webhook-configuration-v1.md +++ b/content/en/docs/reference/kubernetes-api/extend-resources/mutating-webhook-configuration-v1.md @@ -82,15 +82,15 @@ MutatingWebhookConfiguration describes the configuration of and admission webhoo - **webhooks.clientConfig.url** (string) `url` gives the location of the webhook, in standard URL form (`scheme://host:port/path`). Exactly one of `url` or `service` must be specified. - + The `host` should not refer to a service running in the cluster; use the `service` field instead. The host might be resolved via external DNS in some apiservers (e.g., `kube-apiserver` cannot resolve in-cluster DNS as that would be a layering violation). `host` may also be an IP address. - - Please note that using `localhost` or `127.0.0.1` as a `host` is risky unless you take great care to run this webhook on all hosts which run an apiserver which might need to make calls to this webhook. Such installs are likely to be non-portable, i.e., not easy to turn up in a new cluster. - + + Please note that using `localhost` or `127.0.0.1` as a `host` is risky unless you take great care to run this webhook on all hosts which run an apiserver which might need to make calls to this webhook. Such installations are likely to be non-portable or not readily run in a new cluster. + The scheme must be "https"; the URL must begin with "https://". - + A path is optional, and if present may be any string permissible in a URL. You may use the path to pass an arbitrary string to the webhook, for example, a cluster identifier. - + Attempting to use a user or basic auth e.g. "user:password@" is not allowed. Fragments ("#...") and query parameters ("?...") are not allowed, either. - **webhooks.name** (string), required diff --git a/content/en/docs/reference/kubernetes-api/extend-resources/validating-webhook-configuration-v1.md b/content/en/docs/reference/kubernetes-api/extend-resources/validating-webhook-configuration-v1.md index a680b1f7aa..84d26265a3 100644 --- a/content/en/docs/reference/kubernetes-api/extend-resources/validating-webhook-configuration-v1.md +++ b/content/en/docs/reference/kubernetes-api/extend-resources/validating-webhook-configuration-v1.md @@ -82,15 +82,15 @@ ValidatingWebhookConfiguration describes the configuration of and admission webh - **webhooks.clientConfig.url** (string) `url` gives the location of the webhook, in standard URL form (`scheme://host:port/path`). Exactly one of `url` or `service` must be specified. - + The `host` should not refer to a service running in the cluster; use the `service` field instead. The host might be resolved via external DNS in some apiservers (e.g., `kube-apiserver` cannot resolve in-cluster DNS as that would be a layering violation). `host` may also be an IP address. - - Please note that using `localhost` or `127.0.0.1` as a `host` is risky unless you take great care to run this webhook on all hosts which run an apiserver which might need to make calls to this webhook. Such installs are likely to be non-portable, i.e., not easy to turn up in a new cluster. - + + Please note that using `localhost` or `127.0.0.1` as a `host` is risky unless you take great care to run this webhook on all hosts which run an apiserver which might need to make calls to this webhook. Such installations are likely to be non-portable or not readily run in a new cluster. + The scheme must be "https"; the URL must begin with "https://". - + A path is optional, and if present may be any string permissible in a URL. You may use the path to pass an arbitrary string to the webhook, for example, a cluster identifier. - + Attempting to use a user or basic auth e.g. "user:password@" is not allowed. Fragments ("#...") and query parameters ("?...") are not allowed, either. - **webhooks.name** (string), required diff --git a/content/en/docs/reference/setup-tools/kubeadm/implementation-details.md b/content/en/docs/reference/setup-tools/kubeadm/implementation-details.md index 6da8963f17..5a321ec670 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/implementation-details.md +++ b/content/en/docs/reference/setup-tools/kubeadm/implementation-details.md @@ -28,7 +28,7 @@ The cluster that `kubeadm init` and `kubeadm join` set up should be: - lock-down the kubelet API - locking down access to the API for system components like the kube-proxy and CoreDNS - locking down what a Bootstrap Token can access - - **Easy to use**: The user should not have to run anything more than a couple of commands: + - **User-friendly**: The user should not have to run anything more than a couple of commands: - `kubeadm init` - `export KUBECONFIG=/etc/kubernetes/admin.conf` - `kubectl apply -f ` diff --git a/content/en/docs/reference/setup-tools/kubeadm/kubeadm-join.md b/content/en/docs/reference/setup-tools/kubeadm/kubeadm-join.md index 0a39f70927..53ca4a789b 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/kubeadm-join.md +++ b/content/en/docs/reference/setup-tools/kubeadm/kubeadm-join.md @@ -108,7 +108,7 @@ if the `kubeadm init` command was called with `--upload-certs`. control-plane node even if other worker nodes or the network are compromised. - Convenient to execute manually since all of the information required fits - into a single `kubeadm join` command that is easy to copy and paste. + into a single `kubeadm join` command. **Disadvantages:** diff --git a/content/en/docs/reference/tools.md b/content/en/docs/reference/tools.md index ceeadbb279..f5cd81a338 100644 --- a/content/en/docs/reference/tools.md +++ b/content/en/docs/reference/tools.md @@ -20,8 +20,8 @@ Kubernetes contains several built-in tools to help you work with the Kubernetes ## Minikube -[`minikube`](https://minikube.sigs.k8s.io/docs/) is a tool that makes it -easy to run a single-node Kubernetes cluster locally on your workstation for +[`minikube`](https://minikube.sigs.k8s.io/docs/) is a tool that helps +run a single-node Kubernetes cluster locally on your workstation for development and testing purposes. ## Dashboard diff --git a/content/en/docs/reference/using-api/server-side-apply.md b/content/en/docs/reference/using-api/server-side-apply.md index c281eb9400..302ae94d8b 100644 --- a/content/en/docs/reference/using-api/server-side-apply.md +++ b/content/en/docs/reference/using-api/server-side-apply.md @@ -297,7 +297,7 @@ is not what the user wants to happen, even temporarily. There are two solutions: -- (easy) Leave `replicas` in the configuration; when HPA eventually writes to that +- (basic) Leave `replicas` in the configuration; when HPA eventually writes to that field, the system gives the user a conflict over it. At that point, it is safe to remove from the configuration. diff --git a/content/en/docs/setup/production-environment/tools/kops.md b/content/en/docs/setup/production-environment/tools/kops.md index 8394c28faf..13a2474600 100644 --- a/content/en/docs/setup/production-environment/tools/kops.md +++ b/content/en/docs/setup/production-environment/tools/kops.md @@ -39,7 +39,7 @@ kops is an automated provisioning system: #### Installation -Download kops from the [releases page](https://github.com/kubernetes/kops/releases) (it is also easy to build from source): +Download kops from the [releases page](https://github.com/kubernetes/kops/releases) (it is also convenient to build from source): {{< tabs name="kops_installation" >}} {{% tab name="macOS" %}} @@ -147,7 +147,7 @@ You must then set up your NS records in the parent domain, so that records in th you would create NS records in `example.com` for `dev`. If it is a root domain name you would configure the NS records at your domain registrar (e.g. `example.com` would need to be configured where you bought `example.com`). -This step is easy to mess up (it is the #1 cause of problems!) You can double-check that +Verify your route53 domain setup (it is the #1 cause of problems!). You can double-check that your cluster is configured correctly if you have the dig tool by running: `dig NS dev.example.com` diff --git a/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md b/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md index 25eeb18050..03ff264816 100644 --- a/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md +++ b/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md @@ -15,7 +15,7 @@ Windows applications constitute a large portion of the services and applications ## Windows containers in Kubernetes -To enable the orchestration of Windows containers in Kubernetes, simply include Windows nodes in your existing Linux cluster. Scheduling Windows containers in {{< glossary_tooltip text="Pods" term_id="pod" >}} on Kubernetes is as simple and easy as scheduling Linux-based containers. +To enable the orchestration of Windows containers in Kubernetes, include Windows nodes in your existing Linux cluster. Scheduling Windows containers in {{< glossary_tooltip text="Pods" term_id="pod" >}} on Kubernetes is similar to scheduling Linux-based containers. In order to run Windows containers, your Kubernetes cluster must include multiple operating systems, with control plane nodes running Linux and workers running either Windows or Linux depending on your workload needs. Windows Server 2019 is the only Windows operating system supported, enabling [Kubernetes Node](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/architecture/architecture.md#the-kubernetes-node) on Windows (including kubelet, [container runtime](https://docs.microsoft.com/en-us/virtualization/windowscontainers/deploy-containers/containerd), and kube-proxy). For a detailed explanation of Windows distribution channels see the [Microsoft documentation](https://docs.microsoft.com/en-us/windows-server/get-started-19/servicing-channels-19). diff --git a/content/en/docs/tasks/administer-cluster/configure-upgrade-etcd.md b/content/en/docs/tasks/administer-cluster/configure-upgrade-etcd.md index 16fce652cc..72f069d6ac 100644 --- a/content/en/docs/tasks/administer-cluster/configure-upgrade-etcd.md +++ b/content/en/docs/tasks/administer-cluster/configure-upgrade-etcd.md @@ -163,7 +163,7 @@ Backing up an etcd cluster can be accomplished in two ways: etcd built-in snapsh ### Built-in snapshot -etcd supports built-in snapshot, so backing up an etcd cluster is easy. A snapshot may either be taken from a live member with the `etcdctl snapshot save` command or by copying the `member/snap/db` file from an etcd [data directory](https://github.com/coreos/etcd/blob/master/Documentation/op-guide/configuration.md#--data-dir) that is not currently used by an etcd process. Taking the snapshot will normally not affect the performance of the member. +etcd supports built-in snapshot. A snapshot may either be taken from a live member with the `etcdctl snapshot save` command or by copying the `member/snap/db` file from an etcd [data directory](https://github.com/coreos/etcd/blob/master/Documentation/op-guide/configuration.md#--data-dir) that is not currently used by an etcd process. Taking the snapshot will normally not affect the performance of the member. Below is an example for taking a snapshot of the keyspace served by `$ENDPOINT` to the file `snapshotdb`: diff --git a/content/en/docs/tasks/administer-cluster/limit-storage-consumption.md b/content/en/docs/tasks/administer-cluster/limit-storage-consumption.md index 89f130a010..4b4381326e 100644 --- a/content/en/docs/tasks/administer-cluster/limit-storage-consumption.md +++ b/content/en/docs/tasks/administer-cluster/limit-storage-consumption.md @@ -5,7 +5,7 @@ content_type: task -This example demonstrates an easy way to limit the amount of storage consumed in a namespace. +This example demonstrates an straightforward way to limit the amount of storage consumed in a namespace. The following resources are used in the demonstration: [ResourceQuota](/docs/concepts/policy/resource-quotas/), [LimitRange](/docs/tasks/administer-cluster/manage-resources/memory-default-namespace/), diff --git a/content/en/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definition-versioning.md b/content/en/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definition-versioning.md index 71ca43d530..b48d44a078 100644 --- a/content/en/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definition-versioning.md +++ b/content/en/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definition-versioning.md @@ -583,14 +583,13 @@ and can optionally include a custom CA bundle to use to verify the TLS connectio The `host` should not refer to a service running in the cluster; use a service reference by specifying the `service` field instead. The host might be resolved via external DNS in some apiservers -(i.e., `kube-apiserver` cannot resolve in-cluster DNS as that would +(i.e., `kube-apiserver` cannot resolve in-cluster DNS as that would be a layering violation). `host` may also be an IP address. Please note that using `localhost` or `127.0.0.1` as a `host` is risky unless you take great care to run this webhook on all hosts which run an apiserver which might need to make calls to this -webhook. Such installs are likely to be non-portable, i.e., not easy -to turn up in a new cluster. +webhook. Such installations are likely to be non-portable or not readily run in a new cluster. The scheme must be "https"; the URL must begin with "https://". diff --git a/content/en/docs/tasks/job/coarse-parallel-processing-work-queue.md b/content/en/docs/tasks/job/coarse-parallel-processing-work-queue.md index 1bbb49a256..ce7da0b453 100644 --- a/content/en/docs/tasks/job/coarse-parallel-processing-work-queue.md +++ b/content/en/docs/tasks/job/coarse-parallel-processing-work-queue.md @@ -35,7 +35,7 @@ non-parallel, use of [Job](/docs/concepts/workloads/controllers/job/). ## Starting a message queue service -This example uses RabbitMQ, but it should be easy to adapt to another AMQP-type message service. +This example uses RabbitMQ, however, you can adapt the example to use another AMQP-type message service. In practice you could set up a message queue service once in a cluster and reuse it for many jobs, as well as for long-running services. diff --git a/content/en/docs/tasks/run-application/horizontal-pod-autoscale.md b/content/en/docs/tasks/run-application/horizontal-pod-autoscale.md index ff51598f79..a651499e15 100644 --- a/content/en/docs/tasks/run-application/horizontal-pod-autoscale.md +++ b/content/en/docs/tasks/run-application/horizontal-pod-autoscale.md @@ -191,7 +191,7 @@ We can create a new autoscaler using `kubectl create` command. We can list autoscalers by `kubectl get hpa` and get detailed description by `kubectl describe hpa`. Finally, we can delete an autoscaler using `kubectl delete hpa`. -In addition, there is a special `kubectl autoscale` command for easy creation of a Horizontal Pod Autoscaler. +In addition, there is a special `kubectl autoscale` command for creating a HorizontalPodAutoscaler object. For instance, executing `kubectl autoscale rs foo --min=2 --max=5 --cpu-percent=80` will create an autoscaler for replication set *foo*, with target CPU utilization set to `80%` and the number of replicas between 2 and 5. diff --git a/content/en/docs/tutorials/kubernetes-basics/_index.html b/content/en/docs/tutorials/kubernetes-basics/_index.html index fe4b7cb9de..918ff7402c 100644 --- a/content/en/docs/tutorials/kubernetes-basics/_index.html +++ b/content/en/docs/tutorials/kubernetes-basics/_index.html @@ -41,7 +41,7 @@ card:

What can Kubernetes do for you?

-

With modern web services, users expect applications to be available 24/7, and developers expect to deploy new versions of those applications several times a day. Containerization helps package software to serve these goals, enabling applications to be released and updated in an easy and fast way without downtime. Kubernetes helps you make sure those containerized applications run where and when you want, and helps them find the resources and tools they need to work. Kubernetes is a production-ready, open source platform designed with Google's accumulated experience in container orchestration, combined with best-of-breed ideas from the community.

+

With modern web services, users expect applications to be available 24/7, and developers expect to deploy new versions of those applications several times a day. Containerization helps package software to serve these goals, enabling applications to be released and updated quickly without downtime. Kubernetes helps you make sure those containerized applications run where and when you want, and helps them find the resources and tools they need to work. Kubernetes is a production-ready, open source platform designed with Google's accumulated experience in container orchestration, combined with best-of-breed ideas from the community.

diff --git a/content/en/docs/tutorials/stateless-application/guestbook.md b/content/en/docs/tutorials/stateless-application/guestbook.md index 2b6eef90fa..38b638ed1f 100644 --- a/content/en/docs/tutorials/stateless-application/guestbook.md +++ b/content/en/docs/tutorials/stateless-application/guestbook.md @@ -272,7 +272,7 @@ If you deployed the `frontend-service.yaml` manifest with type: `LoadBalancer` y ## Scale the Web Frontend -Scaling up or down is easy because your servers are defined as a Service that uses a Deployment controller. +Scaling up or down is effortless because your servers are defined as a Service that uses a Deployment controller. 1. Run the following command to scale up the number of frontend Pods: From b84e52e2ada5a08f3800f7558369c3ccdfd947b1 Mon Sep 17 00:00:00 2001 From: Jailton Lopes Date: Mon, 1 Feb 2021 18:59:03 -0300 Subject: [PATCH 137/190] Update README.md Co-authored-by: Celeste Horgan --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 2e88a95866..8ec876eef2 100644 --- a/README.md +++ b/README.md @@ -100,7 +100,7 @@ make container-image make container-serve ``` -Open up your browser to http://localhost:1313/docs/reference/kubernetes-api/ to view the API reference pages. +In a web browser, go to http://localhost:1313/docs/reference/kubernetes-api/ to view the API reference. 6. When all changes of the new contract are reflected into the configuration files `toc.yaml` and `fields.yaml`, create a Pull Request with the newly generated API reference pages. From 67a750b5e01fabfeb2c2fb6447471a312a5a2605 Mon Sep 17 00:00:00 2001 From: Margo Crawford Date: Mon, 1 Feb 2021 15:35:49 -0800 Subject: [PATCH 138/190] Incorporated suggestions for provideClusterInfo paragraph Signed-off-by: Margo Crawford --- .../en/docs/reference/access-authn-authz/authentication.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/content/en/docs/reference/access-authn-authz/authentication.md b/content/en/docs/reference/access-authn-authz/authentication.md index f88d8e2370..d7a67f53ef 100644 --- a/content/en/docs/reference/access-authn-authz/authentication.md +++ b/content/en/docs/reference/access-authn-authz/authentication.md @@ -1004,13 +1004,12 @@ RFC3339 timestamp. Presence or absence of an expiry has the following impact: } } ``` -To provide the plugin with information about the cluster for which it is -obtaining credentials, the `provideClusterInfo` field can be set on the exec user +To enable the exec plugin to obtain cluster-specific information, set `provideClusterInfo` on the `user.exec` field in the [kubeconfig](/docs/concepts/configuration/organize-cluster-access-kubeconfig/). The plugin will then be supplied with an environment variable, `KUBERNETES_EXEC_INFO`. Information from this environment variable can be used to perform cluster-specific -credential acquisition logic. Here is an example of the aforementioned -`KUBERNETES_EXEC_INFO` environment variable. +credential acquisition logic. +The following `ExecCredential` manifest describes a cluster information sample. ```json { From 30d887e2552777e5dbcd810ae165a749629d0529 Mon Sep 17 00:00:00 2001 From: Qiming Teng Date: Sun, 31 Jan 2021 12:52:57 +0800 Subject: [PATCH 139/190] [zh] Resync configure-liveness-readiness-startup-probes --- ...igure-liveness-readiness-startup-probes.md | 111 +++++++++++++++++- 1 file changed, 105 insertions(+), 6 deletions(-) diff --git a/content/zh/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes.md b/content/zh/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes.md index 9c3e1cf6fd..a55821b51f 100644 --- a/content/zh/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes.md +++ b/content/zh/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes.md @@ -512,23 +512,71 @@ Defaults to 3. Minimum value is 1. 就绪探测情况下的放弃 Pod 会被打上未就绪的标签。默认值是 3。最小值是 1。 +在 Kubernetes 1.20 版本之前，exec 探针会忽略 `timeoutSeconds`：探针会无限期地 +持续运行，甚至可能超过所配置的限期，直到返回结果为止。 + + +这一缺陷在 Kubernetes v1.20 版本中得到修复。你可能一直依赖于之前错误的探测行为， +甚至你都没有觉察到这一问题的存在，因为默认的超时值是 1 秒钟。 +作为集群管理员，你可以在所有的 kubelet 上禁用 `ExecProbeTimeout` +[特性门控](/zh/docs/reference/command-line-tools-reference/feature-gates/) +（将其设置为 `false`），从而恢复之前版本中的运行行为，之后当集群中所有的 +exec 探针都设置了 `timeoutSeconds` 参数后，移除此标志重载。 +如果你有 Pods 受到此默认 1 秒钟超时值的影响，你应该更新 Pod 对应的探针的 +超时值，这样才能为最终去除该特性门控做好准备。 + + +当此缺陷被修复之后，在使用 `dockershim` 容器运行时的 Kubernetes `1.20+` +版本中，对于 exec 探针而言，容器中的进程可能会因为超时值的设置保持持续运行， +即使探针返回了失败状态。 + +{{< caution >}} + +如果就绪态探针的实现不正确，可能会导致容器中进程的数量不断上升。 +如果不对其采取措施，很可能导致资源枯竭的状况。 +{{< /caution >}} + + +### HTTP 探测 {#http-probes} + [HTTP Probes](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#httpgetaction-v1-core) 可以在 `httpGet` 上配置额外的字段： * `host`：连接使用的主机名，默认是 Pod 的 IP。也可以在 HTTP 头中设置 “Host” 来代替。 * `scheme` ：用于设置连接主机的方式（HTTP 还是 HTTPS）。默认是 HTTP。 -* `path`：访问 HTTP 服务的路径。 +* `path`：访问 HTTP 服务的路径。默认值为 "/"。 * `httpHeaders`：请求中自定义的 HTTP 头。HTTP 头字段允许重复。 * `port`：访问容器的端口号或者端口名。如果数字必须在 1 ～ 65535 之间。 @@ -542,10 +590,6 @@ Here's one scenario where you would set it. Suppose the Container listens on 127 and the Pod's `hostNetwork` field is true. Then `host`, under `httpGet`, should be set to 127.0.0.1. If your pod relies on virtual hosts, which is probably the more common case, you should not use `host`, but rather set the `Host` header in `httpHeaders`. - -For a TCP probe, the kubelet makes the probe connection at the node, not in the pod, which -means that you can not use a service name in the `host` parameter since the kubelet is unable -to resolve it. --> 对于 HTTP 探测，kubelet 发送一个 HTTP 请求到指定的路径和端口来执行检测。 除非 `httpGet` 中的 `host` 字段设置了，否则 kubelet 默认是给 Pod 的 IP 地址发送探测。 @@ -556,6 +600,61 @@ to resolve it. 可能更常见的情况是如果 Pod 依赖虚拟主机，你不应该设置 `host` 字段，而是应该在 `httpHeaders` 中设置 `Host`。 + +针对 HTTP 探针，kubelet 除了必需的 `Host` 头部之外还发送两个请求头部字段： +`User-Agent` 和 `Accept`。这些头部的默认值分别是 `kube-probe/{{ skew latestVersion >}}` +（其中 `{{< skew latestVersion >}}` 是 kubelet 的版本号）和 `*/*`。 + +你可以通过为探测设置 `.httpHeaders` 来重载默认的头部字段值；例如： + +```yaml +livenessProbe: + httpGet: + httpHeaders: + - name: Accept + value: application/json + +startupProbe: + httpGet: + httpHeaders: + - name: User-Agent + value: MyUserAgent +``` + + +你也可以通过将这些头部字段定义为空值，从请求中去掉这些头部字段。 + +```yaml +livenessProbe: + httpGet: + httpHeaders: + - name: Accept + value: "" + +startupProbe: + httpGet: + httpHeaders: + - name: User-Agent + value: "" +``` + + +### TCP 探测 {#tcp-probes} + 对于一次 TCP 探测，kubelet 在节点上（不是在 Pod 里面）建立探测连接， 这意味着你不能在 `host` 参数上配置服务名称，因为 kubelet 不能解析服务名称。 From a0c1266dd33a7691e9a4ce88afd4412d4b024e66 Mon Sep 17 00:00:00 2001 From: littleboy <2283985296@qq.com> Date: Tue, 2 Feb 2021 13:02:29 +0800 Subject: [PATCH 140/190] Update scheduling-gpus.md --- content/zh/docs/tasks/manage-gpus/scheduling-gpus.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/zh/docs/tasks/manage-gpus/scheduling-gpus.md b/content/zh/docs/tasks/manage-gpus/scheduling-gpus.md index 8986b2a6e6..85703130b6 100644 --- a/content/zh/docs/tasks/manage-gpus/scheduling-gpus.md +++ b/content/zh/docs/tasks/manage-gpus/scheduling-gpus.md @@ -22,7 +22,7 @@ Kubernetes includes **experimental** support for managing AMD and NVIDIA GPUs This page describes how users can consume GPUs across different Kubernetes versions and the current limitations. --> -Kubernetes 支持对节点上的 AMD 和 NVIDA GPU （图形处理单元）进行管理，目前处于**实验**状态。 +Kubernetes 支持对节点上的 AMD 和 NVIDIA GPU （图形处理单元）进行管理，目前处于**实验**状态。 本页介绍用户如何在不同的 Kubernetes 版本中使用 GPU，以及当前存在的一些限制。 From 268087e447215034f736c35246ce441c62b1542f Mon Sep 17 00:00:00 2001 From: littleboy <2283985296@qq.com> Date: Tue, 2 Feb 2021 13:30:28 +0800 Subject: [PATCH 141/190] Update service.md --- content/zh/docs/concepts/services-networking/service.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/zh/docs/concepts/services-networking/service.md b/content/zh/docs/concepts/services-networking/service.md index 77fcbbf505..15af8bc000 100644 --- a/content/zh/docs/concepts/services-networking/service.md +++ b/content/zh/docs/concepts/services-networking/service.md @@ -65,7 +65,7 @@ Pod 是非永久性资源。 每个 Pod 都有自己的 IP 地址，但是在 Deployment 中，在同一时刻运行的 Pod 集合可能与稍后运行该应用程序的 Pod 集合不同。 这导致了一个问题： 如果一组 Pod（称为“后端”）为集群内的其他 Pod（称为“前端”）提供功能， -那么前端如何找出并跟踪要连接的 IP 地址，以便前端可以使用工作量的后端部分？ +那么前端如何找出并跟踪要连接的 IP 地址，以便前端可以使用提供工作负载的后端部分？ 进入 _Services_。 From b278dadcedfb75da54f0f7a60dc7aaecb8fffdbe Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Fredrik=20Jons=C3=A9n?= Date: Tue, 2 Feb 2021 12:19:20 +0100 Subject: [PATCH 142/190] Use https for haproxy.org links --- .../concepts/services-networking/ingress-controllers.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/content/en/docs/concepts/services-networking/ingress-controllers.md b/content/en/docs/concepts/services-networking/ingress-controllers.md index 6708b0d229..119958b915 100644 --- a/content/en/docs/concepts/services-networking/ingress-controllers.md +++ b/content/en/docs/concepts/services-networking/ingress-controllers.md @@ -37,9 +37,9 @@ Kubernetes as a project supports and maintains [AWS](https://github.com/kubernet * [Gloo](https://gloo.solo.io) is an open-source ingress controller based on [Envoy](https://www.envoyproxy.io), which offers API gateway functionality. * [HAProxy Ingress](https://haproxy-ingress.github.io/) is an ingress controller for - [HAProxy](http://www.haproxy.org/#desc). + [HAProxy](https://www.haproxy.org/#desc). * The [HAProxy Ingress Controller for Kubernetes](https://github.com/haproxytech/kubernetes-ingress#readme) - is also an ingress controller for [HAProxy](http://www.haproxy.org/#desc). + is also an ingress controller for [HAProxy](https://www.haproxy.org/#desc). * [Istio Ingress](https://istio.io/latest/docs/tasks/traffic-management/ingress/kubernetes-ingress/) is an [Istio](https://istio.io/) based ingress controller. * The [Kong Ingress Controller for Kubernetes](https://github.com/Kong/kubernetes-ingress-controller#readme) @@ -50,7 +50,7 @@ Kubernetes as a project supports and maintains [AWS](https://github.com/kubernet * The [Traefik Kubernetes Ingress provider](https://doc.traefik.io/traefik/providers/kubernetes-ingress/) is an ingress controller for the [Traefik](https://traefik.io/traefik/) proxy. * [Voyager](https://appscode.com/products/voyager) is an ingress controller for - [HAProxy](http://www.haproxy.org/#desc). + [HAProxy](https://www.haproxy.org/#desc). ## Using multiple Ingress controllers From f07f8d5f010a0172c5fd57a2fa9d7767279cd32b Mon Sep 17 00:00:00 2001 From: whoif <3140103604@zju.edu.cn> Date: Wed, 3 Feb 2021 16:57:01 +0800 Subject: [PATCH 143/190] Update controller.md Lowercase 's' to prevent misleading APIS as a whole word. --- content/en/docs/concepts/architecture/controller.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/concepts/architecture/controller.md b/content/en/docs/concepts/architecture/controller.md index 5ab2cb970a..711cf38363 100644 --- a/content/en/docs/concepts/architecture/controller.md +++ b/content/en/docs/concepts/architecture/controller.md @@ -102,7 +102,7 @@ Other control loops can observe that reported data and take their own actions. In the thermostat example, if the room is very cold then a different controller might also turn on a frost protection heater. With Kubernetes clusters, the control plane indirectly works with IP address management tools, storage services, -cloud provider APIS, and other services by +cloud provider APIs, and other services by [extending Kubernetes](/docs/concepts/extend-kubernetes/) to implement that. ## Desired versus current state {#desired-vs-current} From 11ef740edecb41cb2142c38d15cab326bac93137 Mon Sep 17 00:00:00 2001 From: Kedar Vijay Kulkarni Date: Wed, 3 Feb 2021 09:11:00 -0500 Subject: [PATCH 144/190] Added code style to `hostPath` --- content/en/docs/concepts/storage/persistent-volumes.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/content/en/docs/concepts/storage/persistent-volumes.md b/content/en/docs/concepts/storage/persistent-volumes.md index ccb1470e57..379b77d9c0 100644 --- a/content/en/docs/concepts/storage/persistent-volumes.md +++ b/content/en/docs/concepts/storage/persistent-volumes.md @@ -620,10 +620,10 @@ spec: PersistentVolumes binds are exclusive, and since PersistentVolumeClaims are namespaced objects, mounting claims with "Many" modes (`ROX`, `RWX`) is only possible within one namespace. -### PersistentVolumes typed HostPath +### PersistentVolumes typed `hostPath` -A hostPath PersistentVolume uses a file or directory on the Node to emulate network-attached storage. -See [an example of HostPath typed volume](/docs/tasks/configure-pod-container/configure-persistent-volume-storage/#create-a-persistentvolume). +A `hostPath` PersistentVolume uses a file or directory on the Node to emulate network-attached storage. +See [an example of `hostPath` typed volume](/docs/tasks/configure-pod-container/configure-persistent-volume-storage/#create-a-persistentvolume). ## Raw Block Volume Support From c4ef1d4b814ae8d2e55af0c8815b80ab9c70d6b0 Mon Sep 17 00:00:00 2001 From: Karen Bradshaw Date: Wed, 3 Feb 2021 14:28:28 -0500 Subject: [PATCH 145/190] Apply suggestions from code review Thanks for the review. Good suggestions! Co-authored-by: Celeste Horgan --- content/en/docs/reference/tools.md | 5 ++--- .../tasks/administer-cluster/limit-storage-consumption.md | 2 +- content/en/docs/tutorials/kubernetes-basics/_index.html | 2 +- content/en/docs/tutorials/stateless-application/guestbook.md | 3 +-- 4 files changed, 5 insertions(+), 7 deletions(-) diff --git a/content/en/docs/reference/tools.md b/content/en/docs/reference/tools.md index f5cd81a338..41d8ef4d1c 100644 --- a/content/en/docs/reference/tools.md +++ b/content/en/docs/reference/tools.md @@ -20,8 +20,8 @@ Kubernetes contains several built-in tools to help you work with the Kubernetes ## Minikube -[`minikube`](https://minikube.sigs.k8s.io/docs/) is a tool that helps -run a single-node Kubernetes cluster locally on your workstation for +[`minikube`](https://minikube.sigs.k8s.io/docs/) is a tool that +runs a single-node Kubernetes cluster locally on your workstation for development and testing purposes. ## Dashboard @@ -51,4 +51,3 @@ Use Kompose to: * Translate a Docker Compose file into Kubernetes objects * Go from local Docker development to managing your application via Kubernetes * Convert v1 or v2 Docker Compose `yaml` files or [Distributed Application Bundles](https://docs.docker.com/compose/bundles/) - diff --git a/content/en/docs/tasks/administer-cluster/limit-storage-consumption.md b/content/en/docs/tasks/administer-cluster/limit-storage-consumption.md index 4b4381326e..c982a9cb7c 100644 --- a/content/en/docs/tasks/administer-cluster/limit-storage-consumption.md +++ b/content/en/docs/tasks/administer-cluster/limit-storage-consumption.md @@ -5,7 +5,7 @@ content_type: task -This example demonstrates an straightforward way to limit the amount of storage consumed in a namespace. +This example demonstrates how to limit the amount of storage consumed in a namespace. The following resources are used in the demonstration: [ResourceQuota](/docs/concepts/policy/resource-quotas/), [LimitRange](/docs/tasks/administer-cluster/manage-resources/memory-default-namespace/), diff --git a/content/en/docs/tutorials/kubernetes-basics/_index.html b/content/en/docs/tutorials/kubernetes-basics/_index.html index 918ff7402c..afec8f26a8 100644 --- a/content/en/docs/tutorials/kubernetes-basics/_index.html +++ b/content/en/docs/tutorials/kubernetes-basics/_index.html @@ -41,7 +41,7 @@ card:

What can Kubernetes do for you?

-

With modern web services, users expect applications to be available 24/7, and developers expect to deploy new versions of those applications several times a day. Containerization helps package software to serve these goals, enabling applications to be released and updated quickly without downtime. Kubernetes helps you make sure those containerized applications run where and when you want, and helps them find the resources and tools they need to work. Kubernetes is a production-ready, open source platform designed with Google's accumulated experience in container orchestration, combined with best-of-breed ideas from the community.

+

With modern web services, users expect applications to be available 24/7, and developers expect to deploy new versions of those applications several times a day. Containerization helps package software to serve these goals, enabling applications to be released and updated without downtime. Kubernetes helps you make sure those containerized applications run where and when you want, and helps them find the resources and tools they need to work. Kubernetes is a production-ready, open source platform designed with Google's accumulated experience in container orchestration, combined with best-of-breed ideas from the community.

diff --git a/content/en/docs/tutorials/stateless-application/guestbook.md b/content/en/docs/tutorials/stateless-application/guestbook.md index 38b638ed1f..c591e09cea 100644 --- a/content/en/docs/tutorials/stateless-application/guestbook.md +++ b/content/en/docs/tutorials/stateless-application/guestbook.md @@ -272,7 +272,7 @@ If you deployed the `frontend-service.yaml` manifest with type: `LoadBalancer` y ## Scale the Web Frontend -Scaling up or down is effortless because your servers are defined as a Service that uses a Deployment controller. +You can scale up or down as needed because your servers are defined as a Service that uses a Deployment controller. 1. Run the following command to scale up the number of frontend Pods: @@ -370,4 +370,3 @@ Deleting the Deployments and Services also deletes any running Pods. Use labels * Use Kubernetes to create a blog using [Persistent Volumes for MySQL and Wordpress](/docs/tutorials/stateful-application/mysql-wordpress-persistent-volume/#visit-your-new-wordpress-blog) * Read more about [connecting applications](/docs/concepts/services-networking/connect-applications-service/) * Read more about [Managing Resources](/docs/concepts/cluster-administration/manage-deployment/#using-labels-effectively) - From 618dcb3cfa12dc7df61de6d85d3a5b4733de5c79 Mon Sep 17 00:00:00 2001 From: Jason Field <4923914+Xorima@users.noreply.github.com> Date: Wed, 3 Feb 2021 22:23:13 +0000 Subject: [PATCH 146/190] Update content/en/docs/tasks/configure-pod-container/static-pod.md Co-authored-by: Taylor Dolezal --- content/en/docs/tasks/configure-pod-container/static-pod.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/content/en/docs/tasks/configure-pod-container/static-pod.md b/content/en/docs/tasks/configure-pod-container/static-pod.md index a016a875ae..071c1614fc 100644 --- a/content/en/docs/tasks/configure-pod-container/static-pod.md +++ b/content/en/docs/tasks/configure-pod-container/static-pod.md @@ -22,7 +22,7 @@ The kubelet automatically tries to create a {{< glossary_tooltip text="mirror Po on the Kubernetes API server for each static Pod. This means that the Pods running on a node are visible on the API server, but cannot be controlled from there. -The Pods' name will suffixed with the node hostname with a leading hyphen +The Pod names will suffixed with the node hostname with a leading hyphen {{< note >}} If you are running clustered Kubernetes and are using static @@ -238,4 +238,3 @@ CONTAINER ID IMAGE COMMAND CREATED ... e7a62e3427f1 nginx:latest "nginx -g 'daemon of 27 seconds ago ``` - From adf70f8a94e9c8619ba8326757b3899fd53231ac Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Romain=20Tarti=C3=A8re?= Date: Wed, 3 Feb 2021 14:42:29 -1000 Subject: [PATCH 147/190] Fix typo --- .../production-environment/tools/kubeadm/install-kubeadm.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/fr/docs/setup/production-environment/tools/kubeadm/install-kubeadm.md b/content/fr/docs/setup/production-environment/tools/kubeadm/install-kubeadm.md index bcf325df43..06c6d2f53c 100644 --- a/content/fr/docs/setup/production-environment/tools/kubeadm/install-kubeadm.md +++ b/content/fr/docs/setup/production-environment/tools/kubeadm/install-kubeadm.md @@ -40,7 +40,7 @@ Pour plus d'informations sur la création d'un cluster avec kubeadm, une fois qu ## Vérifiez que les adresses MAC et product_uuid sont uniques pour chaque nœud {#verify-mac-address} * Vous pouvez obtenir l'adresse MAC des interfaces réseau en utilisant la commande `ip link` ou` ifconfig -a` -* Le product_uuid peut être vérifié en utilisant la commande `sudo cat/sys/class/dmi/id/product_uuid` +* Le product_uuid peut être vérifié en utilisant la commande `sudo cat /sys/class/dmi/id/product_uuid` Il est très probable que les périphériques matériels aient des adresses uniques, bien que certaines machines virtuelles puissent avoir des valeurs identiques. Kubernetes utilise ces valeurs pour identifier de manière unique les nœuds du cluster. From e54d16c4c6b595b3e27e14bf3cdd5c7034568ebe Mon Sep 17 00:00:00 2001 From: chenxuc Date: Wed, 3 Feb 2021 09:14:29 +0800 Subject: [PATCH 148/190] [zh] Sych web page for topo manager This is part of work in umbrella issue: [zh] Umbrella issue: pages out of sync in tasks section #26178 Topology Manageer (L): content/zh/docs/tasks/administer-cluster/topology-manager.md --- .../administer-cluster/topology-manager.md | 144 +++++++++++++++++- 1 file changed, 139 insertions(+), 5 deletions(-) diff --git a/content/zh/docs/tasks/administer-cluster/topology-manager.md b/content/zh/docs/tasks/administer-cluster/topology-manager.md index d197f5dcf7..fe50dbe606 100644 --- a/content/zh/docs/tasks/administer-cluster/topology-manager.md +++ b/content/zh/docs/tasks/administer-cluster/topology-manager.md @@ -92,25 +92,37 @@ Support for the Topology Manager requires `TopologyManager` [feature gate](/docs 从 Kubernetes 1.18 版本开始，这一特性默认是启用的。 -### 拓扑管理器策略 +### 拓扑管理器作用域和策略 拓扑管理器目前： - - 对所有 QoS 类的 Pod 执行对齐操作 - 针对建议提供者所提供的拓扑建议，对请求的资源进行对齐 如果满足这些条件，则拓扑管理器将对齐请求的资源。 +为了定制如何进行对齐，拓扑管理器提供了两种不同的方式：`scope` 和 `policy`。 + + +`scope` 定义了资源对齐时你所希望使用的粒度（例如，是在 `pod` 还是 `container` 级别）。 +`policy` 定义了对齐时实际使用的策略（例如，`best-effort`、`restricted`、`single-numa-node` 等等）。 + +可以在下文找到现今可用的各种 `scopes` 和 `policies` 的具体信息。 + @@ -120,6 +132,117 @@ To align CPU resources with other requested resources in a Pod Spec, the CPU Man 参看[控制 CPU 管理策略](/zh/docs/tasks/administer-cluster/cpu-management-policies/). {{< /note >}} + +### 拓扑管理器作用域 + +拓扑管理器可以在以下不同的作用域内进行资源对齐： + +* `container` （默认） +* `pod` + +在 kubelet 启动时，可以使用 `--topology-manager-scope` 标志来选择其中任一选项。 + + +### 容器作用域 + +默认使用的是 `container` 作用域。 + + +在该作用域内，拓扑管理器依次进行一系列的资源对齐， +也就是，对每一个容器（包含在一个 Pod 里）计算单独的对齐。 +换句话说，在该特定的作用域内，没有根据特定的 NUMA 节点集来把容器分组的概念。 +实际上，拓扑管理器会把单个容器任意地对齐到 NUMA 节点上。 + + +容器分组的概念是在以下的作用域内特别实现的，也就是 `pod` 作用域。 + + +### Pod 作用域 + +使用命令行选项 `--topology-manager-scope=pod` 来启动 kubelet，就可以选择 `pod` 作用域。 + + +该作用域允许把一个 Pod 里的所有容器作为一个分组，分配到一个共同的 NUMA 节点集。 +也就是，拓扑管理器会把一个 Pod 当成一个整体， +并且试图把整个 Pod（所有容器）分配到一个单个的 NUMA 节点或者一个共同的 NUMA 节点集。 +以下的例子说明了拓扑管理器在不同的场景下使用的对齐方式： + + +* 所有容器可以被分配到一个单一的 NUMA 节点； +* 所有容器可以被分配到一个共享的 NUMA 节点集。 + + +整个 Pod 所请求的某种资源总量是根据 +[有效 request/limit](/zh/docs/concepts/workloads/pods/init-containers/#resources) +公式来计算的， +因此，对某一种资源而言，该总量等于以下数值中的最大值： +* 所有应用容器请求之和； +* 初始容器请求的最大值。 + + +`pod` 作用域与 `single-numa-node` 拓扑管理器策略一起使用， +对于延时敏感的工作负载，或者对于进行 IPC 的高吞吐量应用程序，都是特别有价值的。 +把这两个选项组合起来，你可以把一个 Pod 里的所有容器都放到一个单个的 NUMA 节点， +使得该 Pod 消除了 NUMA 之间的通信开销。 + + +在 `single-numa-node` 策略下，只有当可能的分配方案中存在合适的 NUMA 节点集时，Pod 才会被接受。 +重新考虑上述的例子： + + +* 节点集只包含单个 NUMA 节点时，Pod 就会被接受， +* 然而，节点集包含多个 NUMA 节点时，Pod 就会被拒绝 + （因为满足该分配方案需要两个或以上的 NUMA 节点，而不是单个 NUMA 节点）。 + + +简要地说，拓扑管理器首先计算出 NUMA 节点集，然后使用拓扑管理器策略来测试该集合， +从而决定拒绝或者接受 Pod。 + + +### 拓扑管理器策略 + +{{< note >}} +如果拓扑管理器配置使用 **Pod** 作用域， +那么在策略考量一个容器时，该容器反映的是整个 Pod 的要求， +于是该 Pod 里的每个容器都会得到 **相同的** 拓扑对齐决定。 +{{< /note >}} + -对于 "dev" 名称空间中 "development" 组中的所有服务帐户： +对于 "development" 名称空间中 "dev" 组中的所有服务帐户： ```yaml subjects: From 65b34d02dde0ad7422d7742ca1ca0d0da59b1c8b Mon Sep 17 00:00:00 2001 From: luzg Date: Wed, 3 Feb 2021 15:42:07 +0800 Subject: [PATCH 152/190] [zh] translate Migrating from dockershim make change according to howieyuen and ydFu --- .../migrating-from-dockershim/_index.md | 42 +++++++++++++++++++ 1 file changed, 42 insertions(+) create mode 100755 content/zh/docs/tasks/administer-cluster/migrating-from-dockershim/_index.md diff --git a/content/zh/docs/tasks/administer-cluster/migrating-from-dockershim/_index.md b/content/zh/docs/tasks/administer-cluster/migrating-from-dockershim/_index.md new file mode 100755 index 0000000000..1f3e2b823a --- /dev/null +++ b/content/zh/docs/tasks/administer-cluster/migrating-from-dockershim/_index.md @@ -0,0 +1,42 @@ +--- +title: "从 dockershim 迁移" +weight: 10 +content_type: task +--- + + + + + +本节提供从 dockershim 迁移到其他容器运行时的必备知识。 + + +自从 Kubernetes 1.20 宣布 +[弃用 dockershim](/zh/blog/2020/12/08/kubernetes-1-20-release-announcement/#dockershim-deprecation)， +各类疑问随之而来：这对各类工作负载和 Kubernetes 部署会产生什么影响。 +你会发现这篇博文对于更好地理解此问题非常有用： +[弃用 Dockershim 常见问题](/zh/blog/2020/12/02/dockershim-faq/) + + +建议从 dockershim 迁移到其他替代的容器运行时。 +请参阅[容器运行时](/zh/docs/setup/production-environment/container-runtimes/) +一节以了解可用的备选项。 +当在迁移过程中遇到麻烦，请[上报问题](https://github.com/kubernetes/kubernetes/issues)。 +那么问题就可以及时修复，你的集群也可以进入移除 dockershim 前的就绪状态。 From bed5567056bbccc5637873cee98492dd40995983 Mon Sep 17 00:00:00 2001 From: Bridget Kromhout Date: Thu, 4 Feb 2021 11:59:21 -0600 Subject: [PATCH 153/190] Dual-stack docs correction Signed-off-by: Bridget Kromhout --- content/en/docs/concepts/services-networking/dual-stack.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/concepts/services-networking/dual-stack.md b/content/en/docs/concepts/services-networking/dual-stack.md index 2981bffec8..1c1fe018a4 100644 --- a/content/en/docs/concepts/services-networking/dual-stack.md +++ b/content/en/docs/concepts/services-networking/dual-stack.md @@ -74,7 +74,7 @@ An example of an IPv6 CIDR: `fdXY:IJKL:MNOP:15::/64` (this shows the format but If your cluster has dual-stack enabled, you can create {{< glossary_tooltip text="Services" term_id="service" >}} which can use IPv4, IPv6, or both. -The address family of a Service defaults to the address family of the first service cluster IP range (configured via the `--service-cluster-ip-range` flag to the kube-controller-manager). +The address family of a Service defaults to the address family of the first service cluster IP range (configured via the `--service-cluster-ip-range` flag to the kube-api-server). When you define a Service you can optionally configure it as dual stack. To specify the behavior you want, you set the `.spec.ipFamilyPolicy` field to one of the following values: From ab5b0f97fda8f5e9421a5c3e14955417bbd679e5 Mon Sep 17 00:00:00 2001 From: Bridget Kromhout Date: Thu, 4 Feb 2021 12:30:16 -0600 Subject: [PATCH 154/190] Docs update Signed-off-by: Bridget Kromhout --- content/en/docs/concepts/services-networking/dual-stack.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/concepts/services-networking/dual-stack.md b/content/en/docs/concepts/services-networking/dual-stack.md index 1c1fe018a4..d8b900847f 100644 --- a/content/en/docs/concepts/services-networking/dual-stack.md +++ b/content/en/docs/concepts/services-networking/dual-stack.md @@ -74,7 +74,7 @@ An example of an IPv6 CIDR: `fdXY:IJKL:MNOP:15::/64` (this shows the format but If your cluster has dual-stack enabled, you can create {{< glossary_tooltip text="Services" term_id="service" >}} which can use IPv4, IPv6, or both. -The address family of a Service defaults to the address family of the first service cluster IP range (configured via the `--service-cluster-ip-range` flag to the kube-api-server). +The address family of a Service defaults to the address family of the first service cluster IP range (configured via the `--service-cluster-ip-range` flag to the kube-apiserver). When you define a Service you can optionally configure it as dual stack. To specify the behavior you want, you set the `.spec.ipFamilyPolicy` field to one of the following values: From a0c2963d11e04af10097260e7c628b9d5d7f124a Mon Sep 17 00:00:00 2001 From: Jailton Lopes Date: Thu, 4 Feb 2021 17:57:28 -0300 Subject: [PATCH 155/190] Update README-pt.md - Add translation of Building the API reference pages section - Fix typos - Remove duplicate whitespaces - Remove trailing spaces --- README-pt.md | 61 ++++++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 54 insertions(+), 7 deletions(-) diff --git a/README-pt.md b/README-pt.md index 8367b0247c..0992f6c045 100644 --- a/README-pt.md +++ b/README-pt.md @@ -2,11 +2,11 @@ [![Netlify Status](https://api.netlify.com/api/v1/badges/be93b718-a6df-402a-b4a4-855ba186c97d/deploy-status)](https://app.netlify.com/sites/kubernetes-io-master-staging/deploys) [![GitHub release](https://img.shields.io/github/release/kubernetes/website.svg)](https://github.com/kubernetes/website/releases/latest) -Bem vindos! Este repositório abriga todos os recursos necessários para criar o [website e documentação do Kubernetes](https://kubernetes.io/). Estamos muito satisfeitos por você querer contribuir! +Bem-vindos! Este repositório contém todos os recursos necessários para criar o [website e documentação do Kubernetes](https://kubernetes.io/). Estamos muito satisfeitos por você querer contribuir! # Utilizando este repositório -Você pode executar o website localmente utilizando o Hugo (versão Extended), ou você pode executa-ló em um container runtime. É altamente recomendável usar um container runtime, pois garante a consistência na implantação do website real. +Você pode executar o website localmente utilizando o Hugo (versão Extended), ou você pode executa-ló em um container runtime. É altamente recomendável utilizar um container runtime, pois garante a consistência na implantação do website real. ## Pré-requisitos @@ -40,7 +40,7 @@ make container-image make container-serve ``` -Abra seu navegador em http://localhost:1313 para visualizar o website. Conforme você faz alterações nos arquivos fontes, Hugo atualiza o website e força a atualização do navegador. +Abra seu navegador em http://localhost:1313 para visualizar o website. Conforme você faz alterações nos arquivos fontes, o Hugo atualiza o website e força a atualização do navegador. ## Executando o website localmente utilizando o Hugo @@ -56,10 +56,57 @@ make serve Isso iniciará localmente o Hugo na porta 1313. Abra o seu navegador em http://localhost:1313 para visualizar o website. Conforme você faz alterações nos arquivos fontes, o Hugo atualiza o website e força uma atualização no navegador. +## Construindo a página de referência da API + +A página de referência da API localizada em `content/en/docs/reference/kubernetes-api` é construída a partir da especificação do Swagger utilizando https://github.com/kubernetes-sigs/reference-docs/tree/master/gen-resourcesdocs. + +Siga os passos abaixo para atualizar a página de referência para uma nova versão do Kubernetes: + +OBS: modifique o "v1.20" no exemplo a seguir pela versão a ser atualizada + +1. Obter o submódulo `kubernetes-resources-reference`: + +``` +git submodule update --init --recursive --depth 1 +``` + +2. Criar a nova versão da API no submódulo e adicionar à especificação do Swagger: + +``` +mkdir api-ref-generator/gen-resourcesdocs/api/v1.20 +curl 'https://raw.githubusercontent.com/kubernetes/kubernetes/master/api/openapi-spec/swagger.json' > api-ref-generator/gen-resourcesdocs/api/v1.20/swagger.json +``` + +3. Copiar o sumário e os campos de configuração para a nova versão a partir da versão anterior: + +``` +mkdir api-ref-generator/gen-resourcesdocs/api/v1.20 +cp api-ref-generator/gen-resourcesdocs/api/v1.19/* api-ref-generator/gen-resourcesdocs/api/v1.20/ +``` + +4. Ajustar os arquivos `toc.yaml` e `fields.yaml` para refletir as mudanças entre as duas versões. + +5. Em seguida, gerar as páginas: + +``` +make api-reference +``` + +Você pode validar o resultado localmente gerando e disponibilizando o site a partir da imagem do container: + +``` +make container-image +make container-serve +``` + +Abra o seu navegador em http://localhost:1313/docs/reference/kubernetes-api/ para visualizar a página de referência da API. + +6. Quando todas as mudanças forem refletidas nos arquivos de configuração `toc.yaml` e `fields.yaml`, crie um pull request com a nova página de referência de API. + ## Troubleshooting ### error: failed to transform resource: TOCSS: failed to transform "scss/main.scss" (text/x-scss): this feature is not available in your current Hugo version -Por motivos técnicos, o Hugo é disponibilizado em dois conjuntos de binários. O website atual funciona apenas na versão **Hugo Extended**. Na [página de releases](https://github.com/gohugoio/hugo/releases) procure por arquivos com `extended` no nome. Para confirmar, execute `hugo version` e procure pela palavra `extended`. +Por motivos técnicos, o Hugo é disponibilizado em dois conjuntos de binários. O website atual funciona apenas na versão **Hugo Extended**. Na [página de releases](https://github.com/gohugoio/hugo/releases) procure por arquivos com `extended` no nome. Para confirmar, execute `hugo version` e procure pela palavra `extended`. ### Troubleshooting macOS for too many open files @@ -110,9 +157,9 @@ Você também pode entrar em contato com os mantenedores deste projeto em: Você pode clicar no botão **Fork** na área superior direita da tela para criar uma cópia desse repositório na sua conta do GitHub. Esta cópia é chamada de *fork*. Faça as alterações desejadas no seu fork e, quando estiver pronto para enviar as alterações para nós, vá até o fork e crie um novo **pull request** para nos informar sobre isso. -Depois que seu **pull request** for criado, um revisor do Kubernetes assumirá a responsabilidade de fornecer um feedback claro e objetivo. Como proprietário do pull request, **é sua responsabilidade modificar seu pull request para atender ao feedback que foi fornecido a você pelo revisor do Kubernetes.** +Depois que seu **pull request** for criado, um revisor do Kubernetes assumirá a responsabilidade de fornecer um feedback claro e objetivo. Como proprietário do pull request, **é sua responsabilidade modificar seu pull request para atender ao feedback que foi fornecido a você pelo revisor do Kubernetes.** -Observe também que você pode acabar tendo mais de um revisor do Kubernetes para fornecer seu feedback ou você pode acabar obtendo feedback de um outro revisor do Kubernetes diferente daquele originalmente designado para lhe fornecer o feedback. +Observe também que você pode acabar tendo mais de um revisor do Kubernetes para fornecer seu feedback ou você pode acabar obtendo feedback de um outro revisor do Kubernetes diferente daquele originalmente designado para lhe fornecer o feedback. Além disso, em alguns casos, um de seus revisores pode solicitar uma revisão técnica de um [revisor técnico do Kubernetes](https://github.com/kubernetes/website/wiki/Tech-reviewers) quando necessário. Os revisores farão o melhor para fornecer feedbacks em tempo hábil, mas o tempo de resposta pode variar de acordo com as circunstâncias. @@ -134,4 +181,4 @@ A participação na comunidade Kubernetes é regida pelo [Código de Conduta da # Obrigado! -O Kubernetes conta com a participação da comunidade e nós realmente agradecemos suas contribuições para o nosso website e nossa documentação! +O Kubernetes prospera com a participação da comunidade e nós realmente agradecemos suas contribuições para o nosso website e nossa documentação! \ No newline at end of file From b13328328a3b8d99a8447a886ca1e009f63d1968 Mon Sep 17 00:00:00 2001 From: guidao Date: Fri, 5 Feb 2021 08:32:51 +0800 Subject: [PATCH 156/190] Fix spelling errors --- content/zh/docs/reference/using-api/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/zh/docs/reference/using-api/_index.md b/content/zh/docs/reference/using-api/_index.md index 20bad38482..88b6bce641 100644 --- a/content/zh/docs/reference/using-api/_index.md +++ b/content/zh/docs/reference/using-api/_index.md @@ -76,7 +76,7 @@ Here's a summary of each level: --> 下面是每个级别的摘要： -<-- + 쿠버네티스는 컨테이너를 파드내에 배치하고 _노드_ 에서 실행함으로 워크로드를 구동한다. -노드는 클러스터에 따라 가상 또는 물리적 머신일 수 있다. 각 노드에는 -{{< glossary_tooltip text="컨트롤 플레인" term_id="control-plane" >}}이라는 +노드는 클러스터에 따라 가상 또는 물리적 머신일 수 있다. 각 노드는 +{{< glossary_tooltip text="컨트롤 플레인" term_id="control-plane" >}}에 의해 관리되며 {{< glossary_tooltip text="파드" term_id="pod" >}}를 -실행하는데 필요한 서비스가 포함되어 있다. +실행하는데 필요한 서비스를 포함한다. 일반적으로 클러스터에는 여러 개의 노드가 있으며, 학습 또는 리소스가 제한되는 환경에서는 하나만 있을 수도 있다. From 5465e8ed5b7ec7bfb5a4ae5927878da3a774ccec Mon Sep 17 00:00:00 2001 From: ydFu Date: Fri, 5 Feb 2021 09:30:17 +0800 Subject: [PATCH 158/190] [zh] Sync task pages for administer-cluster/out-of-resource.md * Sync with english version in 'fix default value of nodefs.inodesFree on linux #26256'. Signed-off-by: ydFu --- .../tasks/administer-cluster/out-of-resource.md | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/content/zh/docs/tasks/administer-cluster/out-of-resource.md b/content/zh/docs/tasks/administer-cluster/out-of-resource.md index 33646a5e09..dd6d5afad3 100644 --- a/content/zh/docs/tasks/administer-cluster/out-of-resource.md +++ b/content/zh/docs/tasks/administer-cluster/out-of-resource.md @@ -219,25 +219,28 @@ The `kubelet` has the following default hard eviction threshold: * `memory.available<100Mi` * `nodefs.available<10%` -* `nodefs.inodesFree<5%` * `imagefs.available<15%` + +On a Linux node, the default value also includes `nodefs.inodesFree<5%`. + --> #### 硬驱逐阈值 -硬驱逐阈值没有宽限期，一旦察觉，`kubelet`将立即采取行动回收关联的短缺资源。 -如果满足硬驱逐阈值，`kubelet`将立即结束 pod 而不是优雅终止。 +硬驱逐阈值没有宽限期，一旦察觉，`kubelet` 将立即采取行动回收关联的短缺资源。 +如果满足硬驱逐阈值，`kubelet` 将立即结束 Pod 而不是体面地终止它们。 硬驱逐阈值的配置支持下列标记： -* `eviction-hard` 描述了驱逐阈值的集合（例如 `memory.available<1Gi`），如果满足条件将触发 pod 驱逐。 +* `eviction-hard` 描述了驱逐阈值的集合（例如 `memory.available<1Gi`），如果满足条件将触发 Pod 驱逐。 `kubelet` 有如下所示的默认硬驱逐阈值： * `memory.available<100Mi` * `nodefs.available<10%` -* `nodefs.inodesFree<5%` * `imagefs.available<15%` +在Linux节点上，默认值还包括 `nodefs.inodesFree<5％`。 + + + + + +Kubernetes 的 `dockershim` 组件使得你可以把 Docker 用作 Kubernetes 的 +{{< glossary_tooltip text="容器运行时" term_id="container-runtime" >}}。 +在 Kubernetes v1.20 版本中，内建组件 `dockershim` 被弃用。 + + +本页讲解你的集群把 Docker 用作容器运行时的运作机制， +并提供使用 `dockershim` 时，它所扮演角色的详细信息， +继而展示了一组验证步骤，可用来检查弃用 `dockershim` 对你的工作负载的影响。 + + +## 检查你的应用是否依赖于 Docker {#find-docker-dependencies} + + +虽然你通过 Docker 创建了应用容器，但这些容器却可以运行于所有容器运行时。 +所以这种使用 Docker 容器运行时的方式并不构成对 Docker 的依赖。 + + +当用了替代的容器运行时之后，Docker 命令可能不工作，甚至产生意外的输出。 +这才是判定你是否依赖于 Docker 的方法。 + + +1. 确认没有特权 Pod 执行 docker 命令。 +2. 检查 Kubernetes 基础架构外部节点上的脚本和应用，确认它们没有执行 Docker 命令。可能的命令有： + - SSH 到节点排查故障； + - 节点启动脚本； + - 直接安装在节点上的监视和安全代理。 +3. 检查执行了上述特权操作的第三方工具。详细操作请参考: + [从 dockershim 迁移遥测和安全代理](/zh/docs/tasks/administer-cluster/migrating-from-dockershim/migrating-telemetry-and-security-agents/) +4. 确认没有对 dockershim 行为的间接依赖。这是一种极端情况，不太可能影响你的应用。 + 一些工具很可能被配置为使用了 Docker 特性，比如，基于特定指标发警报，或者在故障排查指令的一个环节中搜索特定的日志信息。 + 如果你有此类配置的工具，需要在迁移之前，在测试集群上完成功能验证。 + + + +## Docker 依赖详解 {#role-of-dockershim} + + +[容器运行时](/zh/docs/concepts/containers/#container-runtimes)是一个软件，用来运行组成 Kubernetes Pod 的容器。 +Kubernetes 负责编排和调度 Pod；在每一个节点上， +{{< glossary_tooltip text="kubelet" term_id="kubelet" >}} +使用抽象的容器运行时接口，所以你可以任意选用兼容的容器运行时。 + + +在早期版本中，Kubernetes 提供的兼容性只支持一个容器运行时：Docker。 +在 Kubernetes 发展历史中，集群运营人员希望采用更多的容器运行时。 +于是 CRI 被设计出来满足这类灵活性需要 - 而 kubelet 亦开始支持 CRI。 +然而，因为 Docker 在 CRI 规范创建之前就已经存在，Kubernetes 就创建了一个适配器组件：`dockershim`。 +dockershim 适配器允许 kubelet 与 Docker交互，就好像 Docker 是一个 CRI 兼容的运行时一样。 + + +你可以阅读博文 +[Kubernetes 容器集成功能的正式发布](/zh/blog/2018/05/24/kubernetes-containerd-integration-goes-ga/) + + +![Dockershim 和 Containerd CRI 的实现对比图](/images/blog/2018-05-24-kubernetes-containerd-integration-goes-ga/cri-containerd.png) + + +切换到容器运行时 Containerd 可以消除掉中间环节。 +所有以前遗留的容器可由 Containerd 这类容器运行时来运行和管理，操作体验也和以前一样。 +但是现在，由于直接用容器运行时调度容器，所以它们对 Docker 来说是不可见的。 +因此，你以前用来检查这些容器的 Docker 工具或漂亮的 UI 都不再可用。 + + +你不能再使用 `docker ps` 或 `docker inspect` 命令来获取容器信息。 +由于你不能列出容器，因此你不能获取日志、停止容器，甚至不能通过 `docker exec` 在容器中执行命令。 + + +{{< note >}} + +如果你用 Kubernetes 运行工作负载，最好通过 Kubernetes API停止容器，而不是通过容器运行时 +（此建议适用于所有容器运行时，不仅仅是针对 Docker）。 + +{{< /note >}} + + +你仍然可以下载镜像，或者用 `docker build` 命令创建它们。 +但用 Docker 创建、下载的镜像，对于容器运行时和 Kubernetes，均不可见。 +为了在 Kubernetes 中使用，需要把镜像推送（push）到某注册中心。 From 6c1ffd397c091d1dfa9dee46f8dcf60b86e4c4b2 Mon Sep 17 00:00:00 2001 From: ydFu Date: Sun, 7 Feb 2021 23:04:40 +0800 Subject: [PATCH 164/190] Fail links in compute-device-assignment.md * Update the links in compute-device-assignment, Fix issue(#26382). Signed-off-by: ydFu --- .../reference/command-line-tools-reference/feature-gates.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates.md b/content/en/docs/reference/command-line-tools-reference/feature-gates.md index d9754afb56..67a8335182 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates.md @@ -634,8 +634,8 @@ Each feature gate is designed for enabling/disabling a specific feature: - `KubeletCredentialProviders`: Enable kubelet exec credential providers for image pull credentials. - `KubeletPluginsWatcher`: Enable probe-based plugin watcher utility to enable kubelet to discover plugins such as [CSI volume drivers](/docs/concepts/storage/volumes/#csi). -- `KubeletPodResources`: Enable the kubelet's pod resources GRPC endpoint. See - [Support Device Monitoring](https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/compute-device-assignment.md) +- `KubeletPodResources`: Enable the kubelet's pod resources gRPC endpoint. See + [Support Device Monitoring](https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/606-compute-device-assignment/README.md) for more details. - `LegacyNodeRoleBehavior`: When disabled, legacy behavior in service load balancers and node disruption will ignore the `node-role.kubernetes.io/master` label in favor of the From d5d4877c2575133ef3c92f3800e0708d8f21733d Mon Sep 17 00:00:00 2001 From: ldsdsy Date: Sat, 6 Feb 2021 00:10:55 +0800 Subject: [PATCH 165/190] translate 2020-10-01 --- .../index.md | 211 ++++++++++++++++++ .../jorge-castro-code-of-conduct.jpg | Bin 0 -> 143155 bytes 2 files changed, 211 insertions(+) create mode 100644 content/zh/blog/_posts/2020-10-01-contributing-to-the-development-guide/index.md create mode 100644 content/zh/blog/_posts/2020-10-01-contributing-to-the-development-guide/jorge-castro-code-of-conduct.jpg diff --git a/content/zh/blog/_posts/2020-10-01-contributing-to-the-development-guide/index.md b/content/zh/blog/_posts/2020-10-01-contributing-to-the-development-guide/index.md new file mode 100644 index 0000000000..01c1e2942f --- /dev/null +++ b/content/zh/blog/_posts/2020-10-01-contributing-to-the-development-guide/index.md @@ -0,0 +1,211 @@ +--- +title: "为开发指南做贡献" +linkTitle: "为开发指南做贡献" +Author: Erik L. Arneson +Description: "一位新的贡献者描述了编写和提交对 Kubernetes 开发指南的修改的经验。" +date: 2020-10-01 +canonicalUrl: https://www.kubernetes.dev/blog/2020/09/28/contributing-to-the-development-guide/ +resources: +- src: "jorge-castro-code-of-conduct.jpg" + title: "Jorge Castro 正在 SIG ContribEx 的周例会上宣布 Kubernetes 的行为准则。" +--- + + + + + + +当大多数人想到为一个开源项目做贡献时，我猜想他们可能想到的是贡献代码修改、新功能和错误修复。作为一个软件工程师和一个长期的开源用户和贡献者，这也正是我的想法。 +虽然我已经在不同的工作流中写了不少文档，但规模庞大的 Kubernetes 社区是一种新型 "客户"。我只是不知道当 Google 要求我和 [Lion's Way](https://lionswaycontent.com/) 的同胞们对 Kubernetes 开发指南进行必要更新时会发生什么。 + +*本文最初出现在 [Kubernetes Contributor Community blog](https://www.kubernetes.dev/blog/2020/09/28/contributing-to-the-development-guide/)。* + + + +## 与社区合作的乐趣 + +作为专业的写手，我们习惯了受雇于他人去书写非常具体的项目。我们专注于技术服务，产品营销，技术培训以及文档编制，范围从相对宽松的营销邮件到针对 IT 和开发人员的深层技术白皮书。 +在这种专业服务下，每一个可交付的项目往往都有可衡量的投资回报。我知道在从事开源文档工作时不会出现这个指标，但我不确定它将如何改变我与项目的关系。 + + + +我们的写作和传统客户之间的关系有一个主要的特点，就是我们在一个公司里面总是有一两个主要的对接人。他们负责审查我们的文稿，并确保文稿内容符合公司的声明且对标于他们正在寻找的受众。 +这随之而来的压力--正好解释了为什么我很高兴我的写作伙伴、鹰眼审稿人同时也是嗜血编辑的 [Joel](https://twitter.com/JoelByronBarker) 处理了大部分的客户联系。 + + + + +在与 Kubernetes 社区合作时，所有与客户接触的压力都消失了，这让我感到惊讶和高兴。 + + + +"我必须得多仔细？如果我搞砸了怎么办？如果我让开发商生气了怎么办？如果我树敌了怎么办？"。 +当我第一次加入 Kubernetes Slack 上的 "#sig-contribex " 频道并宣布我将编写 [开发指南](https://github.com/kubernetes/community/blob/master/contributors/devel/development.md) 时，这些问题都在我脑海中奔腾，让我感觉如履薄冰。 + + + +{{< imgproc jorge-castro-code-of-conduct Fit "800x450" >}} +"Kubernetes 编码准则已经生效，让我们共同勉励。" — Jorge +Castro, SIG ContribEx co-chair +{{< /imgproc >}} + + + +事实上我的担心是多虑的。很快，我就感觉到自己是被欢迎的。我倾向于认为这不仅仅是因为我正在从事一项急需的任务，而是因为 Kubernetes 社区充满了友好、热情的人们。 +在每周的 SIG ContribEx 会议上，我们关于开发指南进展情况的报告会被立即纳入其中。此外，会议的领导会一直强调 [Kubernetes](https://www.kubernetes.dev/resources/code-of-conduct/) 编码准则，我们应该像 Bill 和 Ted 一样，相互进步。 + + + + +## 这并不意味着这一切都很简单 + +开发指南需要一次全面检查。当我们拿到它的时候，它已经捆绑了大量的信息和很多新开发者需要经历的步骤，但随着时间的推移和被忽视，它变得相当陈旧。 +文档的确需要全局观，而不仅仅是点与点的修复。结果，最终我向这个项目提交了一个巨大的 pull 请求。[社区仓库](https://github.com/kubernetes/community)：新增 267 行，删除 88 行。 + + + +pull 请求的周期需要一定数量的 Kubernetes 组织成员审查和批准更改后才能合并。这是一个很好的做法，因为它使文档和代码都保持在相当不错的状态， +但要哄骗合适的人花时间来做这样一个赫赫有名的审查是很难的。 +因此，那次大规模的 PR 从我第一次提交到最后合并，用了 26 天。 但最终，[它是成功的](https://github.com/kubernetes/community/pull/5003). + + + +由于 Kubernetes 是一个发展相当迅速的项目，而且开发人员通常对编写文档并不十分感兴趣，所以我也遇到了一个问题，那就是有时候， +描述 Kubernetes 子系统工作原理的秘密珍宝被深埋在 [天才工程师的迷宫式思维](https://github.com/amwat) 中，而不是用单纯的英文写在 Markdown 文件中。 +当我要更新端到端（e2e）测试的入门文档时，就一头撞上了这个问题。 + + + +这段旅程将我带出了编写文档的领域，进入到一些未完成软件的全新用户角色。最终我花了很多心思与新的 [kubetest2`框架](https://github.com/kubernetes-sigs/kubetest2) 的开发者之一合作， +记录了最新 e2e 测试的启动和运行过程。 +你可以通过查看我的 [已完成的 pull request](https://github.com/kubernetes/community/pull/5045) 来自己判断结果。 + + + +## 没有人是老板，每个人都给出反馈。 + +但当我暗自期待混乱的时候，为 Kubernetes 开发指南做贡献以及与神奇的 Kubernetes 社区互动的过程却非常顺利。 +没有争执，我也没有树敌。每个人都非常友好和热情。这是令人*愉快的*。 + + + +对于一个开源项目，没人是老板。Kubernetes 项目，一个近乎巨大的项目，被分割成许多不同的特殊兴趣小组（SIG）、工作组和社区。 +每个小组都有自己的定期会议、职责分配和主席推选。我的工作与 SIG ContribEx（负责监督并寻求改善贡献者体验）和 SIG Testing（负责测试）的工作有交集。 +事实证明，这两个 SIG 都很容易合作，他们渴望贡献，而且都是非常友好和热情的人。 + + + +在 Kubernetes 这样一个活跃的、有生命力的项目中，文档仍然需要与代码库一起进行维护、修订和测试。 +开发指南将继续对 Kubernetes 代码库的新贡献者起到至关重要的作用，正如我们的努力所显示的那样，该指南必须与 Kubernetes 项目的发展保持同步。 + + + +Joel 和我非常喜欢与 Kubernetes 社区互动并为开发指南做出贡献。我真的很期待，不仅能继续做出更多贡献，还能继续与过去几个月在这个庞大的开源社区中结识的新朋友进行合作。 diff --git a/content/zh/blog/_posts/2020-10-01-contributing-to-the-development-guide/jorge-castro-code-of-conduct.jpg b/content/zh/blog/_posts/2020-10-01-contributing-to-the-development-guide/jorge-castro-code-of-conduct.jpg new file mode 100644 index 0000000000000000000000000000000000000000..aeea042a7a29212690777154ad7f2a457931cf09 GIT binary patch literal 143155 zcmb4qRX|%?6D|@Q(x64#5NL1@E)7~F!Gl9_cPr4+gB5p*6P)1gQVv$!gF{<91Z#`5 zh0~sM`5*4XeY~^hX|Fwdul3EEC40X0-^PEt1R!k4N|sl!Tmw1OOnRBqJjwr>3N)rlO>xq5;x@Xn?dpDk=~Yh?bs#k&%)5(PL&N24*@2 zMuz{rgpl~b20%hVLPEhnLq)^z|1JNG5YUnnRRCIu326z4XbFjF3I7`-;2ImJUUDu2c)k*#d?SMbc0_C}$+a_$YkeaZyzCFov9+82|(V3u6jw;k*Bnh^bP)9+?f0 zGA${9dYq07FvbHQX9g(~qG-XYw(y~;k98~P3K+Z&%~$!q4@Qzz2Ypu~B_>4?4*9~D z2~8J-lH#BqXseU-uKMZN%794mbkDH#ArPS30f z;s)EoFG}H)wyM7H$g$W=_=9B>nQApu<`a;Il#qk~9*Iqdf*)-CWHh0Qunm6-z<6X7 zZ#F2Z~}_;-UrO!A^Ufa1s73JsHb{@AkGQ#G(!qq~&iG8&lIQ$=e8Otu>4aBSQiK zqF;}r*e}$byRrywtFF9)PJ4s9%&xnoGAe3>BhW2o8MutV}a7%8oj?U$#av=kZ7c%EM2|se-D+qXCu2PuPNH^5AJGCPon} zpooS3=k`F&N^&}OP0->pOLD9mF$I`-jxCCUIGQ+$A|^^1tkR`8)dS>8M^jHisBPm# zA6yrVj@Jc}vx8#lI$EHU4;^Xf;$y>ogq5pg0U>G@R}Oeu8F@swN;V9RK6$P>6TyMn zSZduKHL|-M9dS-Kw$~_XOA^E+tPqqWig>A^sU|mem(zD-<=a7$phX&A+j3z^ zLwl51h!%Y&L{Ni_jNp;>Qiw$Z6^-{9)rR(#`04p!4Dm@0l|%0;Phy5R+tA8RV#KgW zEp9~^1rR7r3XoD2%K*VgY{{<)IY*E^q7UsPr>Bpip}o}2;$P+_Hy{TP@!GX~suhFn z2Nxlv09b+yw4|XWG<_Bk2$d7Cm>;M!b~BvjP`4VlC{?~3z9hUZ4OP+i!RzTvMZ`{P z|Hg8(`)^{hK%$y-o^d29WU5hGBk(9q8Ac%WBqeVWDJQG|P49uAj3uMxW5Sj#&T;c$ z*l4gtC%b!KwH`2X*4sIfIATO=){_neR`JzD%J8w%`vYJPag@VLanLSo4I&?pp(jd0 z6smH8S%yG@{d?sAx3Orm#;L{AhE#h$afld%JeuAulg=F}^H32m9OjJF&spAZ)tzAL z$S}n5UzK6PZe{jzUP{KEfWXSIJjj=o!Rq88PeDNfA8>9b_)n6PRNvCECWmQTyOhW2eWDFM_J-3~ED8Oc0!Zn24uaNe2mM20RE z$Jjk+rtwYD>rrX*&4%xmQ>HU}eE@P03n#JkZFA%Pm9qCB}*HF}TxHAV9Z<9GEq*5_VSkm!reIX3EvUeY2iO^lEg4FD@(b?SO;>Q zNc0=2dr7Ki4--Sg)!6+VI(eivjEgum3GglIj3@^rsj%qh_rsL%3!|QKtROmIrDo!Z zo7G$S$_H0sI2mUDAzS~p%PVOTc%6K^`|U(v{S@ioq8V=rAMz&v$h8JyYLD?B4Ra9y zv~lPay_5m1GE|Mgvte;NTW7^H_I;s5!0a@B9C0Qa_Nu4P7xkV;7#tk9eD|w=j;_>x zrlNN);9PyErt@R)_v71uPWS{^bS8BWe;kbS4^pwj+_c_{G3vcsXmKj*Jbtc|_*3_- z7UnNS53axr0?K-PAb$JFGM|L4#4_^;nqW?c*Ccv#iPoO0B&d%#XeF(5x&a#eDL=Ue z2n0oO`GLdK7*9>oGc2a@vrk_@oBA4{$pR8gf5N1tbe#BvF`1AjZ1^MNwBlNstE1g-Z`V zBUR@uwL?zk%jK_}rkN}LYIn8^bP%w9A);qGjGb3cCpTuU4O+5lK>cW3`PkrY^Dgsv zEGI*lQQ`S3;g_ZYR@B@Q)IDdHpMAWZDAnL}3p>-WDel&3!uvV*YOh6CZbAlMpC5nM zbOKa23pLdsdg`s%a5~o3%`agF>Y~pS zS96QkcXE+qXei4tiG0_pMXPSu=G=PS>f%Ij=xLR7hD`k`Dsk6c2p0KFDWmk10r8k z@AsM`ZW0WtMqgby2Kh@4O+Yz}+pwR0&uRtoAO7mWSJM5PoNNX;f zeTAE!kLXZD)5!$~m90Y!umWO?w8|0M5iUUSO_@?KC{iu~(nlDhNW!{g@+`9`KaV0O z@-PZf;PsK9-l#%*EIIK990oXnKQqs=S~DT9kpu_Q#T(XQ%=&OMZaG0NR-s4NX5Nv+ z39*qxkGd3_38ToHzVPJ?KATyS{c<9NTmP|uOOxL`k_pd{Vcefn&p;$VxTQ3MA_I?a z{=r>W2wQX5GlQXOP9xTqyf*%|8LW+$b7x1G4)zHXiDhY7IYp}ZugBNDF)!GRcYZG4 zk@u&(taVYiZ8jJ?s`|L2om9-TcsO=?j+;Mk{J7575wk{SW3VhYA08OA z@VdY7aPI2+C;u#I?mERj$><%{i<5(iJYheDAQg4+G<5R%2v=8HbeKgd#yz}Ly-2Wi zQ~EcWgWikOSH{Ajnj9i4c0ynpP+=ItEMRuhM+cakGfF&?sPo+|j4thZXQ1bKmlAN) zEch6r8-vehErmxpZdJ|=DJ|aq=3&sFy}onX9YDj2OI%OVN~IS@6c&79uK$e>U9s_` zRb!t6W_8Jrr+yCkx%-1{^qs_R=EY(RC#-c&u0E&Naz02HXUO(7Htc?To1+5x9h}hofgl?hlyME+~G$&Yz_{ z)%jQn4fwFJ;Cj<37hd!q0WV(Z?LlW%@mW^v`Oc#zr`t@YGX<&oIG$V$L+@glx@Vbr z&e2i~(}a{*O{f6%xyrRoL-Y)H78?nHU2y_*QH`DTVKEhCRq%qci`z=Ib(0a*KIkKz zMWd6>J>%Y$zqGC-;2C0NGzM!H5T`;QeOb} zl!N7ux_DUsjS)j<)FQ}3Sr+4v% zebEzc*6?viTk5!;yoIXt|qrUhc!|F=k5KYFI3=2Q6V z!8g)RtM`gsu%o$ERfG9KVF389&WU7(_1750gmCI*Eb<=&aLABHNpNl58y~jNg8-d zD!lZ8K7J|Q8zHZezIpSS=P9bmCXVf0Q_(+9KI>)m`F{H+={h_n&p&Y$6aH5*`d^E( z#Oeh6colo6Qg9&7>CoA^0#;DFhM?EBiUjhdnqgTrlKOKn1>4Jt)z!rph&XO-wOB{x zsupmVA+3RwrXi_A6nU~bMO2&~Kd$_p)XA%7&fXbmgG#Rl75 zKs9gFkC+1Qyb}gf@^}3zU6Wz87}kWEpL+))U-|y$cOE)KaWaDkis-#Kw+c6r?Q?`K z+2AK;6oJq4ZldP?QF|_?mk)ejqT+F?Rg=5u6euilJJooXZAx4G9$Mq<49O&hiFNeM zBtb-D(wTh||CaoVWMmXyEeQOK-rbc~UY8=7A~|oGSw-9Y zizM2#z!t)zBVl#&HMhuXVQS^L&dK~cs#*;YaFg1*c6qS+MVy$Yp4o!7Ad6V?Nap67 z!Uw6JN)spgqs_hO~5Fow<$5mFTU#)=oob>^$o@bQ~A) z;BU{5$K`D4bouwRteZq1bsOXic<(HKp_?yI_+xz6Nd;bEO%a-0&M}tetYze*s*jLe zNfF?m?ZI_s?2`|6pQg;9N6pkrkOU!oryRJJUYs|j1p!U;V8z}|iP*6VcT?!_yS{u{d* zRV@}~2!#>n#;!>`L<;T;pG{(^EP}>xLSkMjqE+}31u>l&7!S{Z1y31|0!4~i zx`11gjR5PN{(sa)I2{BV#78s5#eW|t{$5Y3*?!(6$Mmn_Q&?z;mSx(CkqVT%@=$!fPqr};BI(f(?GbK)sKuLhE~QTAv;Dx?JIQ| zVq)mjM$0kuX1T}UzS3V%#oLW2>9yV3?+Us%c%9~S-jUa7% z8p*0`@2T0fkbadds!#^&(ngJ(Yb?E|PLsxhZhKbS7ZHv5ozIbtzEA~#SM$Q6uub$F zv;@)8*LzRhWVKDD2r)WiwkaX?@!|d|BTZ|8SbF^?{Iy?mA-*K@)W3jJ<(}zfw)FVU zTzy*y>Q+5)^ipR9C0FdWz$Yc^fJ}^_bQd2fA{<_#tIUcau|mWq70Y?-HNTEMd?fLu zR#96KX#;>LN5N)0sxKfzE+YzTA4Rmqml4N>zq>Nm`IEWj3Kehxh* zR;Sp)gq3ECG|9<^+RR%+`5g!jvb!wYXS9mO!|tAn=rk(DpDe8N|E7;yPkZj2Na^gt zM2)~I{E8Nl3)KV#<-C42F+XMr9997&-MKAs<@#lJ`E5(sXawve-uKbkt_j&_%;+w} z4R-UO(7it;M~j|NUtj}ncCT(&Nwh6o3Rh<~)%UJF1#c7GaRMvXr}qj*1LHd+{wi~U zJJwk#gD-nW7LU(=AlB`lb9K((d9+U@8E=}pjV@d)u8lkj7-5GpgfGMrdEH~`+5i(O zwjg|Z9`|f&;N548+>hVeH_N|0-u`8Jet_>9hrqMqcgbCuRT*11$y*3^{d#sIBHkw` zX!aR6#8Eg0l(?d#wyr<9OiJULF%Fa-7BZ(%tGL_0OrMG@F|iu&*MvGa<*=KVZB3*o z{FPf7(A@P^>AL*7_-$&A)FI|3fisVKOp{pl`O|`loU40EMV#i{uf*X`y#6e?{)5F& zg>>W*6H6&*iLBxqizID7_!<)PY{sT}?#A_d&^YFHT*tV>T8b@T=QSv5Ml2JPUyXo@WS&6y^@CcHgt*J@EBQ+XPs-~R6Vr)L-~(Y z4yO_Our_HmjSVs^q_bB+VuQ8l5Ygb{Ch;>U?GPP5!G_&1uZr6%@$KG2{Q8V8 z&l}nK%SWZJhQ@l*m8Foxl7w2M>>~_tLuHMNWB`$RsA#~7fpX528IvLtAIsk|J>QFq zZO@V*nOJF|OzCFemwMq*tt>-!Bc~a=xK39!NL@46XnC%}kphp+drX+^lT0NNB3RdZpTi`Yzu#2O8zWO=Wo5vO=4OXe56SE2J; zv4juT<@}XcNGS)`FL*Uy)N^X2e(cTIQ5>m3OmjRIi;O(Z2IOefa;xO`WCKj+OJzmv zP|0z+hGHhZ*ujI-?a{xUe-VC!y;e&5l53z7ML2DwyJL~?Fat{gBLPQ@sT)JKECQpr zfKEA5w~?_1d6R94w@#S1^4J-+5un@L!O`Mgy?LOeUWIG5em-sgQ5#u+a|ZjnCzh#q zX177W!B%{#9kRyP7!cL`e&%h8CL&L@0GAzdT26J+XXEBEM|VpPfsJJfG|b}SOcD5? zJs;0X;oagqmx=NZLsBE!oxkMqq3Hcl)fF}Kf(aeed_bUv&{2Kv+2N_h(VZL_t9qLD z+c?@kK9X+f%kPle`RX%q?MJgej)Q8P{xZA9kKvVER#T}SJ?r{8(N#A&Edez0RplXw z!5UCh!S_Kr8**U2XJQ}3l2sTO+hxzsli>X6%wk?P1%aVVtw$f;qz{0RvJF;!R{TqrXOReuj#^;XVTzJBu#YA)nws55`)#kquwo4R$(bm<>!{j zyK}bqg02E@75;vzxnDX_`BF6z*LXabPeEJzA#`4HbnhX06hlAoA6tZ5F1$6U^tJ9i zE^RY&zJ8q*tfAjwe4OcvAdczOb&bN%166~JGU-vr=}_H-_sb|vBZCQ~jduy+_=|A$ z%m)$amA+peo@tYkCt!9UyqUX>hm!`hnue^lheFM}sF$1^z$WjDP8iB7!oQz6zw0k7BI&`TZnp(6eidVm#;F=utD{)Y5e& zKGO8W{L5m9{7p!ShK-B1gUmhQ*8&Twlr7_{K%eXyG}y^rnLrL{ zurW8J!1X4ixXECJVlH&>^^*ibk~P30Ei4oGwR@puMzGtiR>tqzN=bgR@IDvSwg&qE z8;j%|;RHo-(r|G;36=hY71CQ!3rj+iiMn0O$<`t5@I|jiF7ve}bUCnH69HSb7(rI{ zPmt@^?XunMo}5xWxAHD=v7K#e6rH3G18e6(&e2J8Y;;bo+2z5?&F_7$Z0B)-pCUPf zmJ7`bSs3{xjhuq}4zt@Qz?H(55*4M0_AQ&V8(MX)biil*+`yw}FlYL#4_ zF`S!MDM&XrBG7K;HUOyH+blWnv;-EHixj5=lgl@Kl^Zs(=v2NyvahTcyR`2Ao4FcX zeq^+XFjS@{q>lMUru^7&OXKEq%iUI55&OMsCq>Gr>X!mgZ=*!3d70i-lIflJU5f_LgL2MX@hEZ@n{)ho%{YRu>$q-xoMj+Us5$B;Jh* zT<34=O75?f0Wk;N*e4u{gxk4`X{G#orsGZq?qMV;JLXjB(WT5cm9d}+AT6>D3Zt>;p@vD|By)_rD;JrxdiT4*4T-= z?x^TUm>Uni#g9@|=NZCL*yHafc`O|K`UnSKF-u9CHBjL)SOjFA6~$JMsS zDrdxz%GvoCHx36zZD__Pl1LK_`(0bw#0q6v>^c9yRIW>4e7dA=X0zFlVG@25uN@ZF zrTwU1)gOh49iLJ`=uSu0HjF3KQ)g>Mw*#!cu+PeIr6x7I`HJl)h1|Zoy1km-pH2pj_jesXCIoWr_Ldk8LZ5bL{k49*&_rB~oa-E9 z;&}4N{BCFcc^VwE%+1%!B`M_A-bu6DM#mn>$&f+DORG!Gs48xONZ6rL?fa@_v)^e< zCdMRJw9 zbETCBUYbfLHHpnc#;J8 zk~8s=Kj0&B@2F)z{SdOknUm7d8JX|+dKT++im1)gCq_MUml0CaW)UlPOGThnpgKNc+ijlLA&zU3$&vUUxLn zz90Z)Q4O}T^_{xvU8V4jBCOoXs;F>oS3JICLEXs}SjQp!`25nuKbzKtBYcYYGH$e2 zdb?T=?(?k_7gGSdUsL~n$t%7tPji6o%sM|h>B8Y;?&};X4V}+_Tm=>{btem1ue zk-N&gGd3ukN+FbH@m?yhEofHO8FY@wlGsE$x?bR~8d~%W4E+(BA2oamr@y;s(;H{i zMP!#MW;RF{fNI~@8P>xYNIeZa6I1DJs*FzdK0QJjW3#xAkzm`HcxH$U5E9dOhOnxs zR~Aj==7%LnUHy*sRG6rK&w5OmQ9j@{cT|b;tWbY!P%P8N5Efh{Oz#<;u9)qva4B5$ z`Rwni!k*)Lt5aDzs~cS5n2xuzm7mKNsy16L;urYMDe`UgGf zF0k(1%sc(fDs?Sehhie5XsGSZpk5g(+P#7Ov%Bo0-v7SJ5ZaSyb=z#>>pov!@H5** zcBzS!119!GLwJ&WAZEsnc*dM>(?TlEVVT62>SmE!+cF|@Ki{+oP z`pL-baBsJD>H&(i`=|fBc{44m5&+7Uk)vi+?pmAdR5f6`x&<%1P;3e+_H8A}Hnj*h z;YzS!@{#R6^|C}q6}e`QeOp`4I6Vmcvp8Q$GvF5N*ClssHy}~ADO~KGRL%axt7fmI zl3w`L_n_kHV(afN%$paxJ%M))86IfNr9lG2o3UECx;*UH3_RDM30je~a@8+Dn*Dad z3cS<#5iQE-GC4lrBVrTxTnqD)`)_^AVx-(sbpZOCG_0@nGBDB4oJH=}!$vk!$5iRN z(Vm5_&D~8-ZNSCNrKya!76&ppzO0hXfgHf51C2pS~adIDV22 z0kB|0T5FE)MI~6RmMEDO4SPz-ITEn%!iq)T?AP-jN=F`Mxo95=2FbMmu%2%#fKOE-aAQX}~ z`{~V}%R7H9{}4By!s>uR_f9daNJU55<1dO&itYtUG!O7+cb)nhnR10J)bW$hU0w{o zpXzj#oN*2ok;G89%sj)Gv%Y|qvGtf%`fW8hl&0$Z&}?=ifSXJB@u0z*<2fVjA^UE)yp9zlj+%sU={jZXP94snDzyF`UQ&z8 zwK6N3#7+f#nF`D}Bdu;#Hw(?DORAQ0)~R`}mDMG@XCc`4J|Ar~FN49AR=ekdwnw4LCADw9IZClf7b)5z{8+~d;g0a`7 z`*uBWq@nj#O-cx0TzWfSECpYjYV!jX3=5=cV8ag9;@;{QH^k>)_%ii1A;F2*`P=JY zr(=T!w-l@sWJS>gkQhW0}Sz30?E)LKCJeRq5VKLYKy)1nt#Mv2T6bxV!*T>K2YuAq6vV_Yo z@7Kq7Wi8JIB(E9CIiD}umkuQsgl;|rTLrl!d$gnpI!8W@Z9MzqMTy%Wg4eF8XO8k(39#~A%Mr?$w5TI3W7Ka`$f zEKBeMiAAqlb)j77O7BJcJXR)#&bFun0pn^i`SFp=|BHdN!YvJ_iKoEt57{#J!?BU< z!SfTkti5hJC3bZ+yg?4_gIu^T(uHvTDu3bm@VA%GLUZ)F^yDo=q3Ou>yc20xZObWb z@2I|f(|o z#5xaw43DvU67Rg|&63GlxRlQm@wJ>6co$v7+8ZbH|oA+-#J3tEqC zFe!1h*-!XSP1!NP_5AjB4ue(Z$U{xL9Tv-`FHAyZH)8GMt73J%gHA%j-eiiTYZhM4 zcd144Z&Ux0IK%Z1hOaT6?-bIxK?^j)IAF1k7V)C7`mwO6#OeAYW23sYnWBG8pev?n z_}skU?@3nsE6+3n4{;WoAM8#b1ci2B<@101;i?FKI16MYvfYpzD7dKjRF}W_O+1g5CPw%r#zxDQx6dOqpttWF|g?^wm&)>*eGasb><_m?0cI4bla|SW$4RLHg%A@ zIdo@2xu~~bsfbe8``^ek^mobIw+gG+sSVtkHMDV1_RiPANQ3m`CSs?;Tpf?wD;3a8 zcm9@plQ-8kaW}IjT^3VD2}BQMwX!^ZeR_Zf>?2sEt*+(wphpf5mS3v&gg?eK&MS>< zb6h^YBOTLovQ>mCl>^hL0LYah))OgG(sKzCeRB?c{U&L}8KUgCLEQ9q@4P*9^{j8`;$C2lf|3{&BstNxzehU)wahI2K}% zl`Bh8uj-Qf6ykyl-SE=&kVdrq6B-`9czwPjCpIHiJukS+Z*LxTp@mmxJgN-`d?`m- zo9K%RrrByRZx%`0CLVnCMh($cO^af`q*94z%J3XibvtyGgqSkeeH_*uF=MZ zfBOnHVzWyh7>M`>CMO4PYX+e@XI)dooUtD6rggNg_BPn9o_akZFyQLL8wcOmJib8{ zZ_F!O0ip-C6!>1XB9>mCu$*sL1~~>g%!&%&)PyDgqTqIUBI<6(H~Sa-D)mN@0oj!! z@f~<;MEqQDW7FfVxvxs`io}WzZ0g3adC5l~Kjj{VMI3+Tl6yrrgI7)1xG+5w$ptSk)93&e?zU0bK*BuF>*iD>ZU^c)DETMUrWXnd+D-XFBvJ3=5dsT-TesfiqoV&@JgYwux zn&cEv0Ief$G#v{u1AQY*G$KGVJ^uKJ$pta??_Z}!!}ufTJn!jtD=RBV`UAU_2=G)Z z=2{4^DoVWVvO@m zY{0e>Hmea0c@yFC0Tmxz)D36n(#73M^Tm$>>xCM>{dg=!o;egbWG5;Pg7ZhXYKb#D z*k(0K4cQ?{87bopi(X1SCq3JrJsGYQuhw|8{*+ZlbQvm>2#)-CQ|$r(vqu-Q)>Ko} zqztDIP01Tl0JUbJn~trIRBXz_K(xR-FMH#|GcHG0Epy#P2P-PiCb#4rH|$NrMTHp_ zMO!2C)OkJQ>_tgwgpT=#qL|v=nH9mnLZfZL;$jcKAlW{JP&{t?D0h4GR>TLGlh5tK zKX|D*^XUElomMM=Pt7x96ED^OEq#RkNr5m)t_6v;vH9v+x-mLOqeJbrt2RvFdI!1Qp>OJCoR26I%2hzpwp5zy&6)$zOl7SQ(m1Fyg9g2v{4BU%SK|4U;jr?eI2IAE3s-a&-K-$_mXCQV-4P zxf6NG9XQ(v*%LwM&dxEC8?a!h0As*3O#i^1x`vriOb-uv4M5vbojL|s*r9>r&T|V6 zc!ga=g;7lIQHlp9r#J#p8OkcZ6$+HM*(@Cd#_={Kj}?BV7<+?vi#{1EEa-`3RR$mf zlm@;2_#MiI#icxO112i*xF=zLR>e4pv<}G+sm&5Q;XWCQ_^Z`?Jf4U9nlZ#rp>r^R zga4zu^osfX%1yUoyDnpFT&4WGFQ(zB+GP*5roKR}YAs2KyG6r}hIxK+!X`?~gk4ZJ zzpC%^z@m89giij#Uq$E3ofSe440jM%1qlw^XK$0lmRca^6y^X2M6 zUv(A>dZIX&n4ESFF9-tISi~VW4buV_*&kZb$HsqjJ(oJj*y4@pOPWgZPn`XN$lX`^ zdVl_y$UMR`$;FU$$^TK9E6%t2od5mC-y~b^E?i*sZswo6uYLv1NZGnM(>?D{OpD)o z_efv;+t#vSj}ie)m=`UhC5tI#pGjgPNw9-yAu~RSq3AvXrV(vKm>h_SmRwH8Uy~1K zAWZ%GI2^ZKDqNaA>vfMKSi^Gi@*a8uamP@UR;_1z#7HKbnO@$2mhhx3bdodsS8T8E zr#IC-NVOWCiP0Hjyr!-!2C5zp3fI~`UMpbN*OvtsX-7Gzq10IUIB7wjJc7I9T%!zTBS8f3`~~P@OQF_`*pA-;CcuCo&M#{8QJ5LGnipi zSjvJjF&Ma!ll?4`wD2ZmdHOY4=BHxOtDqy}_Bc$P+hXB&iUyR`yy*wB@Y>S?c9Y)k zVNQN?87NP!^x)|gRAonNaeHAz%Z>xR;c>mb9a~3j68oa_CBNoeG!4?8^m;JYy`lG{ zLU)4o^a^BFx9m7x;3o=pF5<{SrcVt4_{F2d2uFk%NT+ipoY=p_x~1{0&bjYHq8*;n zGNO#4XS2Hzg3hAg`@;U&W+_qU zRUUbNg6r%qkF7zabBtJn04TcN^K%0>h+I05t8ntAhT^^>}8$(wmr zM~f)~JU!;Uo1}ij*-n~b`@rSi)sJ~^VX`g@4ztDM&DK@py4g>%S`BYJDa%)a7TV8` z*VfUb9+HHsjy(Gfo3EN5C$ImVL)W9|e{DGq5w&plv_o15pVk|xq$9@NBcTO5Qd>cG z2A~7X$Mf6%7QY=LF{4R+d=-yYhf~mBWUG0(u7FC8rC*8oo#KHWLE8%bMWJR_J=^BbJvM z`btl`{B1wmt1PSiCZ+G!a}uRT!%aSrlMDVSKAs?X&2j}8zN}ig@CMl7rYRVe!s_Z? zgVc{hIZnu&fhx3ixNN%FZf#!I6m6%P8hilDzdx!8hAr!cY0+|vLlPs+NGIGDqgHjp zIWZYiWSk)Z{~XMGO&$}S^vO?QcID<#rY9jIAdXx8Gw5Yb2%}B3GKwmzW_~!r!>F$c zU0gg&Xo-jQ`=8JBQZFA;PtOOyjDH}tVotgV1nd1dBw9l{(REEp^6$G!rDD)N@{7}Z z7tkPnB*wzJ$Mo<7DA!JJcz)c&@}@t*^;%+GIBDzZ!Y!NW`&OAcE2_><&AxH&pMS$P z7Z8BntyB?2n=qg8dhMbyN(<#*o>F<4#;5Rp4sDJ0KMM|YCm-Sdc32}}HMW(=KLJ0^ zK9Ewt8^V4RN#a9qYKqvTd}QTs8awJ3f!i~VM}`_Q>L7LfCQib9T1~-~L#rA2z+W#E z{v6B)I!t}dtYye|7*5d#`)Ns#IGa)K?%k%jS(ezJr zc65UMh6}>y=8ox5l7?;LTckFT-xGLSabgv&4CCd|0xS^rLn5Rhnl^vEeMuHD9_6~8 z6X(qsDpi~Fk+n*AUXjq@I*7*AG}+E8;2udFNg~=Lp~=dw)NyfjHq+EVPQ5tI$|X#e#l{BZj zsxT&Jr{p8?D}XQV-A=s9wcrRs4QWpZx}{ptGWwz@z_GgGhXtxJhl1@ou!r*&$YF9= z+r=XkB_h5p%PO3K@BDMP`Vq8`J6O!!tCnI;h|%11S%wODQF|>~h04Ecoce!4ri;W82;N=-+Fy*`M*3KaQ&$ zO;uGuO<}b>+Ah@>Ii&DN(-aF&|NY2WY?1F5Ng8n~i!2Ls{P~+dw7A9TA|86Tej(P_ zj=a1KP!{)Ce5t>YD}q(9931#av#c_gzovQavmX|y_eK_d{UnpyVGgHJ0|Lt|k_hlN zszsl9f8~8_k>MdR&;=bp_^SHcD1Zs0{LTgK()P;FV%NzAJ}m`Hik4G2 zbx(=+FcoJNXMPvb7zyvN{HPIWrEDB>|Lanz!l^0#+$PMrXjHfg;xpUl+lbBfl3?|9 zfDzQBz7froIi#Z@Wt5HlWLCH-#@JN!%4S~dIf>E+HrMUOpXLBYwGG)}- zrTSS9t6XMQ%R0wRSWKL*zqUDMLF%d%%J?EPaDKH})idz>i}MwG4Y#V@ zNktZw@~XQ`27gqOXAg)?Qtfg^FBj4iE{WFt@5W_$5{j?PXW`G8r_nY<=;En>N z66PPX7Myh4G^Me=SExmGP9(sg%8ZDfm}DG5x%r|Necu%5Oqsmo6svCt8i@+pgj!G$ zad8T2f1U3}P4biIH=)dzjtbnIYi1o~=KQOsy&`2hiZko8)ZkXiOslWjtNmEb$=IK@ zNYJsqpgeCHhCcjt%pr!B+6#Vx2){;rZu?(r-+a{PJgl!^ijDl}uE}FF1I`qM1b=w zN{43iK6@e;nH3{ol-HOEmnC#G#_2bA4Lqj51X=4HU~FK@&2ur^z}fk z$?zCNLatJ?YA_@i4ZC5<&`r;`yl%Wu)Ur!PdW5Bs*H&OiyVb5b^;v(wL3 z$aEy|k^#f(oaqUpbe#gDe`K5jwE`&ixD)r>IQ zt%(FR{IdQ&vCV9#+EA>vmFd_0*Bth?o95LoefzO22FbTVKfdtu53NEAaITc1%hl^ckWk26~?{?T#%2XtapqZ70)r znm#@E_MRq1!yHKTaUbKU#Q`ePg4;MJRJkm*YePKexMZk=xxVdmGhbB!zp+Y{n0UR( z@@nR(6USm(5tmMD_?s_Al6F?~?4@9GTV&56StV=Hl?#bRPfo$EDf^R+Baawf#|aVk z+6R95%=(Y7B;LCiZBllPxnE@KzkeOAgXjT^S#)KkhU4*bur1^C^u+PLp@)OQO-f;& zgAVIrOs{A&-Ms5-lc!x`QSEg;ew(AleXn?sM4|mNe-GRktmt$_`7n0Rv(5^;<8$22 zGl$T|^OHQ#u0w*YJM4cmuN0Ra(d=@{MJAky6fq&qgc z6;Pzd=v3z8T`6?Mx(4PBpNF`EkyI=T>2Gn^?af^D}NsXI|^Ukt`q@G;N1}lOiOY8$?=@; zWC{GP@h5a~Vx-G*Z4%MaoHJJOqQHz=cDLXwyQ3hnuN>Q8;qrMZ^VD;XXOO*Y-LX%& z-AP5JBt!)krtl~UG;nx^bwPTAT{teQK8emT znKV{r;7R-w0a;Rs<`}|$cQwNVlw+m>lq53oP8Ek`#nwVVGQ$WL#Ao5nq0*02j;(vF z)ljDe>uXLCVqsaL7Wa%X11}$KE31yHBF3Jn)1&IZr3qs3bW+Hns}s2s~XqphOZ&aXhf91`tAH!J?*U14idNS*T?Ba#OnRvTWOdw?}*x~MI z_?(Z(#)#B&g0n0&UTeaWJ%%*J)Z+YL)biN}3--~1zzAefMDn(V5k$P^IlMW#qCNB{^i`!KX6# z?pB`|>Fk|~IhpaW==cPKgCEO<$=WJly4n-)C#p4Ab``A5-2jmY6PBoNE-g}ds25!CeUwfVhZlG7G ztWcup+3vs@Gxqc)w;>@`>7XzIb;cgRPdg20jwH;bKD$~gbcdQNcpS)mdi4Aog7($a zb+h-n(A!;Akc~D8J0M~PWZ>u9$)(H*nq4L1Zw*#HMz^zCl$d1 zLZe|%n$hD4OAk+OXKGY5db;UeTgpUCtVrWI%}%b5oih$muCIT&@cz*q85Qk>QU|G_ zrlmd2hLyU&Vnvy)OV5;5zaRlfEu|HaBgR6v_T;CN_HHcF0-ELD8-`)yL;{=(4cqRP${;hfhhzKkgCq*Vidwk&pYU1}=%)dW;?F zgp5zWI~AMljI*Gc>A5O4uggK}p&LR+$J=}LsCw@eb#>}1vkciungs9NV;O(40X!No zR~2Kej%ynuy6sO1>meNCs7ldkh-;4Gm~4n=5~$5nzN`#2S~Qh2#CS+kTl)Blo;w%_ zQsHC^CtpjXm(L#0t*?$%?u~z*{A#}`DO0}al>r{O=^;j6WW~{#YnXOooO_AQ#rdhhifMeL~;E_iy|1VJ8o@i}^ZaHX$sSKG_#V zIP<`vgC2p7B}f5)#jyGkGinr!k{D}w2IBz|1OT{@vfytg>Tq^FuY`~CUx^bI=8>W9 zXx}hD7I42F(qXc&J@($n3M`)s1$`}Tk}}VPT2R%LcBqxSf1Xx^1|!4x=#cN|PVJ;s z&vkhGRiO0iB)9oe;7Y0o;z&+g)rXs@#!@Jh+)A=5I6biOFIpS`H7Wc4t?L6yhW z58|CQY|uDQp<{{LUPZY0oM*6>DaUc6r4rF@(!A-On#_vKu(M&4u%kQu=qw(PNY@JGC}J==oHDY%8KioHB- zy48P)n)T=SG`D1WN?G&zOC01^)4Qv^#j&ZMKP`Zh+tW=HPYiXueL|xg^jM;%ej+ct zfA@yq_g2t~e^q|{r3Ia*INQPaV>t8MUh{@L-gb|jr?Eso?Tqor^FGE{ZW-;f_TtEr z`2#+HXA138k6n0v6~+-b4v_ z{JPgNi7}@vO88h;({n3F2lF*`VxkaRO=;~Cvi1Dy_EO0hjLLN<*T><9#QGQS({$%& zWkIjM3{RijRC4Il{Xmv?>CQ=@*S#Y*&Th+#m%4j2h&|nvf3?GHLLe=VHlC_&v**8Zg+rpa){t=9odPY!lD8R za2^lI*Zq+{A}p$oZYW!sim^`^9V-PdM$NpRF-3v2C=#o1g^`yRC2vzoGN4sM)igP$RfF!MJduYA9rV6@XYt+{QH^)+Ih^at5KEIzH|t-1-LY=)L*cLB7#CrM`MI+x(eC zaQ769;y ze0p`#bfW1Vp98Q|qS78^tn*S>YkbjAz!uX*@s}lhqqCw@`NS(uMF=2pd5+za{CYzU z1=g5^Y}oVz$tndh1O+n8<~Ue{L^`(Y)<0O~EIkUy7&GvZ_|s8r94XVOswxU9_>V%G z!gh1kzWs6f@>ELAkXmX+D;)gpsE(SCrsdaafI$n9jK1;C*^kEQqhroxlU7|reO*wA zfMN+N%;w#~w}G%8Hm~e*bfuTvsq;e2Dvx3fpiQk5<;6$|DKh;@Jv3&%KG$qIaGehN z*)Uxnabnq*VkXgY&oY@uJ#0-`NsRc9m^Tx6bEqL8>=F^8P;Asj{8P$MbeUw7mFOR% zc+c=rA?SdIE1c=>fGL!a5Zi9xwyI@-h}Pq*r@Kb&s4= zsZG7=m4l9@zY|~IpL&P%M)s>&QT6i3AN74x$sJXcV$f6G6&~rzoWX&|0EC}DmDhJ* z(R5N7mI;AaV6$Xphc_kg@)ME6S?JvH#fhjGaS1enpM*Bj$vp7TB{K?RH4Lc~s1(>r zJaEC$2ZuA$mrOTfQeawRQigv*Jl1%EZrWcC3hT9eok%>d?2P1lp2-Hn6DunmF(QJE zgjf~4&e&j3cpRKH4gXSXSGfE`0Bv&eAXEQG&I^__pFxDDIoMB5B~PmB{STjQgMziP zb>oF52hxdswcs6_y+kt_*O6l>Mg;=h_5hZWUq5p@J*)K0(5Q$Y^7!iH`YP|BzmhE( zN#s%cn^@k)C3}6Ef=GTD#&7b@N$7;A!(R!(J$&=s313i*b9fr`ybD;)VYre1GKKDp zV5F_+iZBI=2XFA1%$p}%PeO&BvO&~wvG4_(tTEC1aIv^pCYkrv)yhIx=9zgn$>s6Q zoV@dVQznHkG2Z1=@H|ScjG0K?d)Go#{nczBWNpMu=OD0I`&CrrSD_R8Q?TZg1&F6K zJ%q+_B=~GB>(TdvjpQS_6bz0}nWyNLD^8P3s3RqVI%7Kh<~EdwkFTj1@p65Ppq0>X zq6L8G$t6*b=GXDv-F-`Ng)vcc&?h07=wa9pd0`uBB*es|ZBe+kZPV}=`c3!C?k+{(>G-45 zQ?geBqzQr9I*{QPbICiC#^9C%fn40UG3Q2!X!q-uPIwK=RKP9xEN|{bpSQV#BPcU3 zA;$YndUx6S&nSQ1+wYS8=HI{==6@7E(CQVWkBw+O^7TyZXY%lKEE>__Csye>xit^ z$#UNG1(vTfLb@>?%c6^b>3Z<(bCexX^lD^eC=8S`=2%WelZ2Razp9n4c@4R`lV-M; z$VWUuyhs7_4y+%Id@5a;*c(43t%LO)ZA&TlqxT{B4{(R(gH=LC8jPP+hQB5;fQB{8 zm}VBTqw|)19O`(2X{At6%~dHgR2Cu{7W@sxa)?K-AzJRNiHu6s+aVi=;^$9zb&J_|5%8RgU={I1jyzK zL*0YUh>M$rj>~K=2^e#9j=(j_E0GGdI}nAS9@WwdW`d)!=A zPD38&*cat~0twR<H7#j=GCuVG~+qO)ar5$LLFs9?@Cb(s_K!Y{GmB>Un}ns$Fh zS%{*#QD5i$yZG)dN`?3INpLsnf?A-=UEBe6y_YLc<~@;#6n9YGqh=`aj!XRSlBmcV z@8c)Shqiew|24~Nxyv@GS5&T`85)nbS%i9(lxt{a-}q&3d0fhaJz7K zJ6#CThxjKSD@Ffxqt5_}jQX#C|JPnmJj44*Tr$-AzW?py|9kOpU(~-3)X8}NM~J?S z{}BT9InC<)|24JY!Qg6ZfD(@=Zl0F|CGLN{!NYQwD~r^?%X;=qNj6diFNvC1oYh-< zH}^@wD_m-NfUXj!P44Ju&-_{$$1|7KW{S_dBc8|QMUBW~`e28wq6NFs)}~QHV_N5C z<2@c?;^{i0LjnT@rnV&umy%DWuK0Q;;0l$7!->4I%i^D0es7$;e7x zDvCe#6IBZ%rQQw z_$;N}djcaf<|Z^L79P}T;!vzQ&Bl1F&(-p(`fzJP71^S4gziZH)IfQGjbRUK` zr=Hhy$L?GNI3co2cN3G}uyX^w#nUcz1dSCJW9|!lMxYYZ6HSFeE4tp5S^7>)t-|X{ z&#A>nhTTo6Vm=wPr{%Y_h2+hSz&Nlt;AI-xG4N(Bi;K2HC8awFr>P;KssLCvHT?{Ym{}9uv<)Y-fsfFP3rceohjhnoS%aVvy2d1sa zc{B)D|I~NSOWT$rzI?7^;j1d8<=Kqv^~}S%^f`qgm1`L}mRIE!WjVzH zr%Z*OPaNW=i^D5GJR}W1Ga~Hm;R-JI+aBe`6?tpg;!Q=WOqHs_m>6fG{+`t(xohT!59SwaUIdIJ-`#zZRaA4^<^+?7vc;u_&TZ%{}(G2b2I@BP4z5~DhVRhzdu0r zKNK(X{ZhwLACLga@P$8w=NHw zj+LBmoM`if1)=W5`J(7s?LdBfoyl*zw1@!%NUFm|j6%t8v^p3ujFC{T9)=Q=1T zMLqT*YCw8}QEo#ui7K?q!TX}#FCw90lRD~o&&UNG zp8O9+Fvc>;(ZI6zQd{3!ICj9eR|Lw$!eY2Cudt1xu{^A zFXTcp?x+&|{;3sEP(((NyQ!r`6ctS70Vp>jRke#+VL-*WU}|Zd@rEHpniBki{Kcms z4aYz(WfIe-9l%=*$!GEC#~>;WR1YL}nHzZ5-WQtD}au%1^Oe_8E0H}H;B{Rhn}Su0iqMB46j zhI~Uv$d79m=UXnzp$E%pGUEeL>i(bj{1B)=)+P-u9Q=CFEecBs6^R4~;hjaEe$`ro z+xIVBSqR=pg&$|6lpacTNltkucw-gA$%?B!wY<;ERhqZ*V9ttF5$t`s*FasK@;>&E z6bYbpO)+M}e?t?2cL)=pdUYQm0OlOE#=|^l}+XbtASZ z)sZ=GCe>za;AX!lJkj0`fu6SRQ|`pHSBMnFi{;ZDQ|as8qKG7*Fy=Z?^Z$~$M?Wc7 z6h4wGZtRbqq=YPzE{IgX3n3xin`-PIX-qp9eBUu;Lh0H0s`U>62453;adPBSyM`0d z-~6omFFzSO0NjBT=T@vnr9$hqg?eFYI8$?7ERKJ^SEX$dndnQJW~-gZGizy-N$Lp|2`WNYrvyF|1tazT zgKQyj2fT0f%};_k6yhHc9(Zu^5QKyK6mwK4fX#J*WflVl<=;nSuhv2i78iIRGW;{T1Blv@<^BF}5-8ARkUo=&NOO)@XGv7bwT^j}X z|H+ZJ?tFJwovr^fsv172$o}0k2g0_Zy>DVBsbMDwqLz|Kv~b(UehpE1W~0QcLLs`I zI0>B`Q=nBIOj+LD4k=?Z($@X3shulDIyVeU{^43}^6O64dVK>$s)HO0X{*^$@O#JH zPbmFp`@4qIw$aPn%ne-swyY0I^^&3RAi*VFy9wXxN|AwB4uaADqE)OQk$v4T%NCJS z%Wjmbpg$-`4fxMhqW%bu_~+$S5|!i%wXJZ0;6m>;E{wLFOs~}8#4mEK>%W(mLbD|7 zpsHamg^76Oit%m)b~;B}(txc*_g8QoDKKOmI8-hKt_oT&j0ezb=RwI@UeD&pJ60W> zoeOnNmZO~Ey9cA@$Q1JhYDkRL;O(w1YXUyO;-}6?9uhN}461N{f9%q8A62*)?Ji0x zV+-;urNH#1*X@E6_9n3Jip!+T(KysZs2Rf3CsE-N&}~`l)dhxgnIL<{^1Fd@J2}el zX-x79IMPYPLG6KVj}kJYM}7xFy0!O)czA<9^3TXrgE!&?6#w%NyU({DNVk1DNO}XZbzVj zejZo1dm%@hqM=-BPYw7EPcS5BNe&)2ht0CTQhWgqq5($=09qzycghLvx60#>h01X8 zF8p!j9+FVY1R#UvJg-d~3mhM1ywq0bgWgn>f`aiApVE6IB+e_fb0n1pyd!{?ho<~a z%A?Q%q#JmaX7aI}-MbBk8VZn%c8TnlIojLGfGVXUp7se+XJBB7c{B$Zj0%z{%#G(^}~X-W72sb-S^b z_~mz(D}Cdgf^`Omhb&;W%pSE3J0Pc4X^qO^yNPs8Hq~$x9CSCoPQ9e0$t!Q_p8^?A zZ|ZBtI#0s2mn#-wwMFy*AA=&{0mv<$X^y@gWED$FL62E^`^|_-+obe~+A|MAm*CY2 z{NZF~^-P)|F3q7y?Lpsw?;9sEf?ZY{%*d!mO|^-C=j z6Ne`a>y@A2I1RdN^ljGkI7?zz*3T^aL>Qm2g_~1ukiW|(%Qlwnfe8xB9+V3-bD&~h zlxK8`vHb$Ftfo3*P7NVsJ2!kFywx(Nq(M?0(waO|<$ZUzrbRe*NL?(69UMTyz5lB7 z)&RZ`ga?uY5eT$$PSNu)5xb#_cc$AB51h^5h0k71`8TnY$S>TFc4;cAS0B20yIBn{ zXxYHZbuCH z2V5DHISQBDUB{i7EBRJb|9aii&eO0`O(M11kuHCmEowjXW={Cb{5D}L!t;G2yNSNu z^S10a`>i$ldO*8HJK9Y!$BOpPnVV>Rs_m(3v$_3Ib4WoGFu=u(M^h+2nuB&IQGiQ6 z!?|XdjqV^$MlXm6dBK7ypXQ$!%fl z|8Mf%XOLXqgzE~CzGN|FlB>HeJ#*o|dX?QjJL8X>L;Us-g%q1QfC5U7TPg&%awSSQ zGS0mOly05Z-sb-~qPZD4V3=zsMox@#7etw<=H{!dlE`6V6_1S55(HTduTXff>CHwh z3Z4U*9d*!U@uq70X&E~a;7%Z}93@Yq5_s?C+ZA`z6)uhtF4olGUo_WM#;v1b5WKOg z(r4B`^TxP}r=_b-bT=#Ap-6rPX6#5}KY6iL*yO^{(p2W~3MB1nuHhnoXrRIZvSoBX z?#)8wrl_9{K>49jVj?w$EK>K=_>=2cjpMDJP|Cv4sDG;;d^V)?*36Mp7wdj9D{DqK zJRHD#?Dv<~IvR7+hC6>^jZtHvh!tx44pzLWZJdv+F7{)8uR$x}e?{l-U117c+1B~u zE7*IAvE5n{%Ai~NLEB~~~CnziR3A%(9TuBw&$6_HVe5;b9_Y2AE~h+bfwJ8@b8ZuzLszpq=p z$yHC8G=UunY#X<-iXPYMmdpq?Qi~f0K#K;ohY1^R7qrxYopfrYZfx7hJn5~RU^oCL zq~W`F94cpL$C?}`=v@55>}S`dVqFn@_N&&kZL*f%gYOz^lFXggW~`k}dO#~phnqmE$*Xe0+gMY@T)*HKWNODJO!|LMDw88IEQX28>!5Um9bMKc}iGswm8^ zB3O7nq#JOvgr2yb9^cGdR5w*b~fwWbN4uY zHaO)jS&MIaGTf{CpXnKoEjf{qBw5T|lkc_4mP#t)YsNC7*i zD+$!^!P7&146Mj!YAl50`s1Ze%Sz^nrK`I`COW3|2zcUX;;!YuJSy@1x3We+A>U$b z+=Wh92Txz*<_94rbT(aC)b~)+yeA*0ft~?_?bE}_?4CO2>*nQ5>{RUqVRso@2loeU zA~DH`CBu4AjubGWV56rG*gR6c{zpTj3}<(*ZshT|yD1n|R^MADlOKw#2yk+uIApIOhO1Q}?3CWim5iS6P49B8>g-E@#sIF3%Hkm1U6Y;KU*N;(cUJNNTXK(^Sf(ZXf7_AeSP3RL?|kiXYV2HJ)?FUts%|3LPda>~C%&e559G)k{T@4;7##c-b=wMsKv<=bBrsg)A+p7c7oNKhi zT2_+sz2U&H3dO`AaupM`G{6cMUw1kDCGHt_rG?Db(r40mfm+;qm_6||1h56e*Dx_; zv{5sAApNa`eQLg-t3Dan;8^MAkf&+-V28QNJR&E$UW}YXo}x%piJV(%Z|f@SvCTc# z*>asJeULfl*|hHjJ79m%z3lhYNjKun)~3l{v>mujzf1CEbLfw=G0v+QmMVy4kDIHN z!HakXqfp3fN`AvHZ}(KT0VO~8wPVRRE#07qqF)T2?gU}ux4_Cu%CF+JB||(hKU75$n&H8Vs(}>Z zXOzoC<*_e2Yc?f1MKZ2y9rwdk)KPA*fm$+QP%ImmRko6KMJj& z>^w0xBvEdS1~%VR$ZizB?K1y;qh^ph&pbPQ%K+G_i4Mykz>!``-l-2~P@npXCOG}+ zj8Xna5ZgU%qBrUd3{N7D?lm|L+Z*Pry) zAs#|FjD=7sSpI+_F`pJhhgQNkTcySb3Z1u+X|x*Mbz=$U41U6!KD;(|GWhZf+TGcf zIto<4+rMa$SfTjI9b+1e4hP()H!h&$Vl%FwWgf|xOnv~Qh;@cvW2F%e4OP-WwgE0$ z%Q)&pb(Cfs88vs?RVP4i3x6Xmn|UZQBzh~dBbUseVXM+9)SKxIGb)RGL{1h5mcdd! zY*C*qi!ZV-g*~ZMgXaA{i?F->Mf=mg_LK-6{s%7YhKWs6y98dYL1d{5%Nu5CM)oZI z;$P<|Ib`v%#+A*)TByU~$rK)nlY{y2+xi}%_C;bhx#C@^DJN48zXpsG_dWr&u-XUGbZI)t0QNce?u**N)QfyOspDHGadi#`TWl zZDqf;Mx#sR1J}Ox^(d>a4aE@h?ar13An!--RoVi||3n|P7eFkNFZT3~A>q30O9=QD z)Hs~0>Zjq9K(SgteoUk5XsSeQv8qQlUAE<5a^kRpe@@B>1mQ=uBUEJ?t0NIjst~Tk zrw}`22~Oeaoz>Jar5(4^I&#)h()kt7j$N0**9K)pIb^_R>Qu}Ecm#tdgL{Th_MPmZ zNT1f*+=#`sDN5Z)GxUu8=k9Y@C}9!idZ&e;Ae?KZlaSLJkLow;3hX*Iuqb_#(8thB zA&gENZ?!0{7dr|SQ~VC)R0$(w%`)uC^Kr{&tXMKsrW^V47$cA zc2qXag13x{K2gC>FW1<8fcvnWt$o&NG2)pQ&M7_|f<9-E@X_PygzO3}REmd*E<{9Ho9snlD3P{00YArPGA4MA;Dql ztUUTeVZqL3!T3Y2$3IeZaF{p}bV7X_xmk!-D=sefIiBjX=P>|=lUaS~V0*RT`E@Xv zxIH{YU==5#HkwO37wktM>So^33==347h}eH3>i|zS&wl-i(1T5_AQ-zS%8$f zp%1UQu-xGt#=ULuK?&40lzq*JWy_p_Y=W_*OiT_ zHy(aD`Wk2#ce%>*_AW}=HF{CMM@NM098Wy;*}|%UuYs!GjC)dAVTmvKlB+>(JQYiWv4%ASDU^wE zfK+@a5NJrx34vT`C&?N`t+0p;X$**T(1(q-)}$vd}L0YXIe zdRSnG>mOIJc9mnl;eJBDmt%QKH-^kCZiDK>Up5J^1?pI(M^ zLe{uAHp)v9-Qpwrw2i5Cr)!Z7E_J!(OYoHUZHd^dAP*e{OxvdJ;>zQev`OvB!6rLB&NJUA=+Az+_|~}=W5h92t8yc;h!~FXb-UBy zUo<^k;==1xr`!fVs9Ms8`p*1Q3oGoG-|~t6Ond$)(n^YM$fv!MSdHJ%9i&2hE;`xD z&+VF3`rXp~7fnf$a8`+256-p+9d}59Pm}0B`4O0Hxxr7Bs9gbZ8JEopCa@`zncjt* zeX0H(*!tb+Gscvw3Mvx+MQb`twhx&8Fkf`2a#DG9G?nW`b(i0y{BriI)d#*?ADoSM z+hWg|;*;N^_7~~tHBK5OIvf>4@xeHgLm6#KNLI4ztc-2t%#A^PKOeQa>u_2=J&8{v z69+IQtYhN5H1-H{7Zuq<$$S1~*gn`yh)ez13vS+}?cCbMA~3)B+sPV;r=U9V+&q^a z@aL|5+@_x$FtMNNr+Z`NGpCxoyt3&UW9sDbM8l!KXl_^4LmCU8kEsve`-07milCY; zu~KtwNZVr{E3Y;}XfgPKs#MT~?W ze=wA~T-M_Kh6S(=!QhJYNOR~sJfd6kXBxxdKD^BDaM~)lG6-yzOb+z#43PPf;Uqwf|F|92_ z*I2bQY6j2HJ7=IzJ~zu(NQCxL>Fzb322WY@>~4{gdTu7?xD}HLdM=4gE-?E%HoQ`lKKa5zeuF@KxkAq=J|{g@`S99$HSMP% z7jmS%`Xu(HeKHTK-YwC?4+nf{H-ZXu6%l{Y1{g<=y8>L{+wZ*9>v-O06ShFZCjiX= z0u*(TrA#f>GFFIlC|1HfJ9c2yyl3oXietKy)2(KF;T5v%UKV zVbxBi4{uMKXSx*=Mc7PU+JgEFF5$A-ms~3Q;25%EQ;w*TL!(s=jz3g&l>|c}t$aMN zK-|7hjfh%X<=+52Kg>~S7R5Y38L5X8lFi4}VD?DUX{OYc+q0CiZb(Nb?ULSGU0Snu zqNy4P&GH_bRpvMNvOlt0v?mwZ0x4oiR7m*YMY1CRQ$8q`O1F={TifbVfr|H!PV4wq znIHW@ept_~zh*b=oe=#VB)Yr2!QoFZRjk-21OZ^Tk>i6X2nugC!@AyKVmD?6HRIVrC*u8N%Jy2(JDHTYNNr=H{uUiWnG=|;$?%v{XOgz>p*DSdtf$8<- zVuw@_zvf{_o04P<^pG8ESCRhmlEWAvY; zSC6n6oq8ze-tP0HzKJ8ugfQSMVui3IO2(2+F_JfIbph2gj*dfIY%RW&m>*s83ym9y zGvmeLZ*bZ^KmYm^_?w-JtYI*MjR^f>M|H>8mKv1uzDqH&`{GqmBO60`n-ZtHVO>dS z&`Yq93IFhkV{sPrOrEb{^^{q6iIyFz=wYMnI9XJ1L8y>%h>e61%oFVBU?gHwi75^a zDgWlNF1;nFJoeJlD^c%WzgseS8Cj({Jai9A?pSN~wjQ+rmAJI6`BfYX>F}_TYUYq4 zEU*=AzwW}58Bz?X!u0?*fD0`fJ?T02?Gf$?c0+#YZROmKeV$yA;pA8LxB$wARph%j zp*I{6H)XRzXNfnTw;P}>qjAL(#c}lsPzQTeI7OGZjuWbyMDkb@km-JdI(xD&uh1oF zGI?q1zp}OcGG3*y&evLN@jI+C(|s$!$kTxx>HzA@|J@{Vsuv)H`K#>E-MrBLt^KkAwCkDM4iGEo~??tT%i5BdNc4_Z7*N`nD7r$9 z<;jyNDhlelfjVmam@nP?k5EN;z78GvTzTwQ42)Ww7UUn~Qv4H#uc%d1@!#i%elPlc z_FJ|(X%F;fsmQ>Ga3kAe0U1a@p*Ksm|1q|6ka9o@4+n-?JE25Rn zy&QqiNrLi{Q6<&c;v?XSl2^0hO-OOES>YLW)gw`g9LQUwf_sHm zuD}&nS)AI8LUnM{qV%K%dvwFZ_mQPO@C8CwL*=cD89z@+@}_@!%Gb-4)zO;~CGml` zjgq=~Oj^X{P`!Tc(JpB-OacN38=GjE-^U!kdilk^17*NvP}cnJyjFFhU?NDQyu{T7wuSV*>RGV7YhL6 zeA?GlJyT#cd$G#>-2I%%m!STny42!J5b<9$0h6h-a1zB1=Sbjwl1kbjGvP>Q5oMDy zu~SUmj=Cb1W+swijJ!v(Ll>c0pQ|zS`&O|}ev-nSln(-H(6TA~ESoSbw;Ve!mNTOk z?{BQiV#?v&pC=)OBZcQ6kU6&HFr3d0WlL0xt?93No|>|Cwg8vE?eDo02VeT94+C4m z4bRd?KEpx?5(5O@VMo5BH$SJDAX*V_rx<^swecd;y|LFT#Ct?Ivlj$3)|@CaC>Lpf zd2Q3HOFZ`F*O#xCDe-oTAcg#fqD}}s&U>zfRt%ddajcR9LS1Z;P|MWkIu0^N+w3a+EO6z`R&Z5JfLN9DqN3@CwI%h%KBnYAP`#RFle+$}@i0tGh+I zD6niIOgmwyWzUvtn|r+_+Y7%$J-a(^3VO1hmrcG28GcDjOn)ToGg+mFzortyud097 zARInC74sUGhk0p{Fgr4NMVC~A^bFbgcIPVK#U2>XD0SrSU#VY&Ui!8K#Tu-Zq#mP?z< zqU}g-c0fLB9KVBeCcneOE;qC|cP(LuwmX}3Eeck8n^bMOGTkO2LDx?A7w!Fwu*HK7 z$cl- zA%-Vg!ybRp2>zmRgk0|VL@sTECyh&C%7U@#2UVYdI#qcu>x2VX*K<>A$WSpD>M4fz z_6j3GCE@3b1jAfsY+Yo3gLBSATO)JT#;cuarj(FCCeESi?Iu<8(}5%Q!EK7xJ3H#5 zqMN^HkMRzsFZeA=wk6WrMRdO(bZ9I}ui27FJlnCDbPUDBvK>OGOAoF2JZK$YF-kk= zLjf#kx0EO9;J)Hs8*dF*Nym_|%19g3U}MhIr?i4asLF3*-^$&zn+El#@0f;t$XxBp zS^;ZM%D>9IQKv31X8Bpewj4=-TOAK>t)|FUCvgLtvFpas<;UAiR{Ke#Cmj( z8MEg5UB=$YyE@yZ!3k5W0|JSrzy*GXk@YXtYu#a^e=hvjo8O5kcO9;%UfNp#o^83q zSiAJ^m?6B55x%;*S^(PPpq7~ffo;u$!Q5#u6mr_zVC;)JWkMp0>1k7ylYYoe>ueCM zSW5L?#B{d=?z9Ugs~_}qDy(QnCRng9H`M$XFTOiv^7kyu!zMTf)_AI@aNF#c7~8!a zGVi=9r0mxju~mvmiBpHH*NEW-5f*um&H-TC{DK1$uA2FQ>tZBqiC?OxzxEdU6*H^7 z8p%-34_8={uth9j$!?FV)g&W*n5CAm2?l|?ydy|l~4jwC4?pcq>Ho#5<-X2 zLtP+9Rgqo>YP`@6~v;-xqUNo^zz^|hAEpMwudF52ss{)$cCvpBR)C6L}>g+wc?jxpD z(Xr>6MaKHM=}Qo3D|N(_lf7M|bCLI)ifnXo=%pcC>YtTJvs;8`fo?6bHGFV~-QZtm zSQ6K|R$jK%8~n@X(_o(`Kcg|ONiTgZMnAP-kjA2CXqcuOH2~w0@Ln7+i`y`~Je} zO8=H$-O!POu1cBCM0UKQ`{JEeD_aTB+mJyKhP<0*H-?vg3|I?NmFiK~jsWv-aqv z(pa&AiX7e=<+Ek!RrmC85euI#$#-A`BQ6+pdB=aobH%a=39mst^GkN?%^0wJ1mciyz*cx zyfUMZD+N>)bm-P>bd0=oBXO76*gLLn>Zn+C$tv6?Q$*3Vng@Hsk0YpNMxO!ujS)@K; z@M?&Vrd)OlvsGgV`s%sAdZtFv1P^GS?xm>S+rGLkZ)Kc7a5F(W?1Wi~7Kb6%3Z&2A zsLV`|^$#<_DSj}Mi>v#Bm8y`3X$}$YKXU%XT)R@|J%A?~voKmgQVH5km#U}*4)X^4 z6pRHAse~-j={|Q9xhZ4E+gH+9b$98wIgxiIlDh$gn0Off-7r~$4B6+=o%2jLEAUWt z?z8;_!J%_Mp7sJ6bZ?xk@J58Q_ee*9U3Irlzy%;XWphe?oRIH48Vt9bEs8Cojx~zL ze>57N=4eRoT-)y3K`L43hCiRMj%1X&59)%H%nJ70nkJWX!DEfZgw%A!qT@`yJ>Yr% z9Bh^Uq)7axUQ4=mLy<8^Dpt=kb8Pd++8i4-$|HIUSf@^jTw3q5%5Wv-tOxu@$VzVI z*XA7ZTNe8^%HyXP{P+c_IO}19bR)y{7=tW+2qfSmr1B`vb5MBL9M%`j`f|R?d0=f{ z=0=u`-Vt&do^>`A(okqx814>e1gx40SH9yT46@jD`LL6?gnW)a(r?CnR=aGc!~-FE z{OqL-Zkf|O1=us8?z^0C@>?4AVqfAxzJbingao4x?z>6xf=^FfZp0Cq7OS$+W(0(C zsYTtgs)f9nYw^;o0mnTz|OqJ!X&(^lL~@|14M8($ec%Vqr014lG0|9z-NVw7P3z7^bd`#rL#F z#)tfUHwasA^;(u9++)u9ES^i7#aQs(kb6iqAN2)=&CUwO=zbLB@e`Z3)J2+&KGJO| z_RSMKY-*UtYf!}=DKp5u!M_DCwaWh~!%{RDwZjSaer|rdo$AZiQXJ`~t}LG={V}&P zhoVDx zx92#nY-&tYtdzjl#04oU_+-B=$qAkl=quq`VpAri(U0WwrtAqsbc|t&3`u-&hrCTr zS~KQ=X7h=6`b@oFnVqPh4zs#K0@+h|5>j*@tt-WgreKiQpphOqH?lzRketm1uX=+# z{ovAkF?n(G;!1oW(^?a-PCWzgMBZR!Fn^DG&Hgji%q#bOplSYBYQNh`zZ=_P7Ml+V z2cZ2)iWrqxD5x_Vpf#H)$agYAn6?R+uGY`=Tz;u${oW=b0GJ-df2NEirT6pE`{ci)4Db_xC94VY498WO_R=V&n^dHS z?a?;KoIn-_XyD)AS=|Daz9qJ+%&Irc;~p1Tu%1VRi1(&_m{9tLOh`TJi!sc3_l&_) zsMz-Ip`eg17f$Rs#Vx3^PvCiWLFK!0GojD>=Ju72(t#y=TaduWYW^A=^a#*P{nMQc z80&~2h=tpPS{}~+G+}k8b15ohzs{z8OI~U%tP3ve-U+N=OUPH6(7|(k@ zgprq~k$Gqlb`X5mei#`Pqmja^QN=L~gR}T!UX9d;KE(bY7o&~H@iz~m`>|%7Vq<1R zxR9N+K}uELaz^~B!|RNYrn8 zNEXv-T`@X4Jzb&F^B_dPe8M(0hUwd^9DGGQR@pl#1llr}3Z-S-<&NG`@~Lh+>&uXD zC;$C*XG~28o7FfU;X|)2!F*Qf1aqTFI4oVV(^l1Ivg-)o>1$35?-`006|{{`O58** zqGZAV_TfX{Efj*ob?e)k7F_lkL1kIi6=gNTNckdY`&B%SL+a7aMH zH><`e?P|>;9s4hLwxW5LNKc-5X}^yBHp25GnN;#UW%#d$`?7?!zB=7=GJsTOCXD!~ zPAbx_+>M_pvDdKY_145YvuNL7XNInZ|6aRTW_12S(fey6ZaRKSd;X6_4ID7~lI|jWi9R%1z!d zrBkYn(YO@L&}7H{ptg;x>XRe%%fb24kI|EygSU>xl#f;GPJ0>T2umr@wRND}sI-49 zcY}3!H_d2DeAw%yE(9jNem64i)tG7HMO)Ui=-Ze8&l>payEb<&!E{tDMc?9cI^p}nxK|J>SKGR-&s)Acul0bQ3O-F;G`ylR zl0IO9C{Zh_E)4%dnz>HG-LfC+Rn>RWYwXH0e|z@Ej@~^n>n~E~px~AqAiRfgBY%9J znB6m!9&iyYn3?F@lsXm9$oT6Fud8kFSXG3Uopf`wr%z%-xZFeH^A-imO7;{d_I)Lu zL-SjnfVoNAmW^v7hS7b)9ZvKXNXT1-%s(H*O5M zt8adwE0IAo@0rO{@zzr)&D!)?yZA|FE98ixd~Je)kCbRFs70=qt1^bG~b+_pY=~UWWQolOQ-G=5xQIvjuly zM*w8<(0$fBTt+=)cKa=#rQ0^)UKLK!Lg{^a-|F(+I>16U$3>Rjk_%AnB7Qd(|8QTc z{6~d=5c|Wx?>ciJT3Y&St^B}1KmoT(esQut9<&s|7r+A?z<+2J82RkeKi3Wc4nPTT zSeaOUG|I>S`-nGs-t?98yK3c9^U|$<9rH8U#HC9gYmd>)dRZo$eteQno+ggFB68%!I&v)$dbThxQc&lQrhNVj*JDjm8`y@ zZ$sRK!81HkkP!Imj1c0B(09r7CmoaZZJv|dOrfcEqmGVSs1kYLJ;hU2icl<+sm_el zF0OnRv0|62;7C8S;!<{^`mSEt-cZ>GY{P_mxs7j=iGBz@eHX{A7WMk;YQ=!n zMM-+NY!i)*&1`s7d9iSU`-R-cdy4N}UG(L@q%OW64Za5^m)(6BX|VP|Z|UM{y8ni* zh5D~E*FGn`G6&+5Ph^`|2MrA^083wn|C1&Ed(orEzxh_=H>15^@`dU{@dYo37Y`b_ zTMmXSbPE{4ga*f{R*y@EOkh<3pEiw$^gYJQsx|OW>)@u!Ck>Mo^}LGqr8oSq5X!4P z8SmwkHk(?n3@tG}S52v;tSS!KmU|y~6gf56RRu;Ig8!MUYZ`KSS z{bTJVf56|i<>&gUtDbvuNoXOS=aYRssLd9m>8AM4zs}Iuyi<;MjEY{@467AsoL7YU z%;KNAvY`U)3(`5$U!6lGS^*9eJtC#&O5RX_;FI^NVVTAdL#`Qi7GE*Pc8pz$GR|*D zv9Z_}&$sR@=;cmdf2b-wB0k`|d+_$xnKZ|?9k=1doZ1~6&8(`>EEA zCPN#K1@!jdTVT(NZQ(3Z{m6oG>D7kHqEWzc@_E&-Gv=S?!qkZ2x5CI`cWZ|w4qRNx zLxGn>i&FRta`k5YSxom7D?~p%gpdbCio{kpZ z;j3@_C|+?Jl5@Mlw&KQo#fVKK(rO{m>(?19*L#|7{-XRj3*M)8svdIPQDNlTX=5nA zZba^D%lG#P!;yNe1nT^fA)so-hs*pr^WMLyS-e8gCSIavvCy_`XQtKBK>dd@U;Yy# z&oZ?fouxZ-BLi(eTgV~T4~(~4EJiM_^#5ISme~`hifnqHSpD4-+XU!_MTC?@D@|S5 z4c52U52uNY`ML>LeO)={@LCqF)Zw=3t&E`#yKJJ2Xt zq#{eub2ZJHq6OZ}ub*%#O}NW;66&?!rDpy`J%#tR~EyIwgDj9@WNb;;icQkYiW7IVCB|lRm1hd z3rnhoE~z>rp5hY3p5g!rUR82qvA&5(_(AHy1^8DFZ?nbhg^Hksoyd>!`SWzioQVkB zXL23y;P}UpfRB2fXmZtX+N*=cRqlZDk;(XjoXz5OKab^_GC5oF$BLzoWe@M*Y-sqZ zvZ~=R#>(Vob1!>C!-wRuHjnp~*|_>KGa+8PH1>d3;o>FVN3yeC?FR@Q5Wg4G97y$6 z<7+{O7UxnvmaGPG^LCiptOhE66|>Ut5MPzZ0hAP;wgG&(!?(>0tQJ;vja|sq4%_(b zMs%GGI^eQUzpgd8_CwjBW%;nK$L6H)y0PGoX!_LY|CpnV$3KO}y7)=Ax)qA1_^%X|WEnDePj@9~!v(w?8$ z6Ejcs338T^13A~!;Cbhy0ydL_+KTjkdVL)dGInfyW$vF02^(uF%1KN!9k>3uIL(>x zzb>X)lI@cXvsTqx)d#3}&=-VR7Qs%u%MK7rsacMN+xCo+qnG;INy=y0FSTDNNGwad z9WuRZMD91+Lm{sQvaH+DY3~QFCw&`hg8*`NInU!=HunUU2la_>U zYDwCn)*+i}^51znUe10yr7Xz%sX$@Hz~`OICwS%X z6-PC~%RvWGbyPhUVnC6bT|yyxOEQOi_iwk3W6T83_*&9-Dz!%Gq$VU9qEnZda zgu;#8B(lZxu;~{hF@2MAXzo+YzOC57LLbJ7U~a*VY~jlAYLtcodvVM4vIghpK7T&$ za%|Ew%JP*p3w>1+!BxCbmr92N)~vI=bXe=%rr6 z$YEI9L}7!6y`q+Xz*I_?x=*TTV;OBC+`E(?7I&SS96r>vxnrYDiGc+=ygT?xoYpRq$2T4rq)$~$~)~n z`Dj#`#2!t4^n1x&apxBIvUBq9cfF#^%sZ+U8spTkrde$F%2ucZ4 z{(201mJr_t znOn4kZk9KyXkNmUjIf6i_3ym`D6)UX$+hk7mQ_8k+~G^)&k}NDeO_wOuRoQYWK<`N zSoq5;Ec%CM>(7lDjVvh_()1Kk*TD9dgUGGne>Nh5MsjtAh(!ehds*x98Jv}lR<&Jy zx1~y1?A2cMI2J;PGdq{sv-_q>m&<8k2dPfw!#j$G0umDWXoba{iz9mz}*_E#$X-7 zvu5UmXs~b7z|my&IdT{w+qjxI^i_9AyrAl7yG_JbtH6qr)YsERA5l7$_m~(Sr^&)3 z4^z25AR$H7$qI`>HtE#iUPgm&i5Dbw9U`UkFy=P(h=q|l1A-L>wK%CwDB>1r5#Okh zBL9Zx{`4pp7eV8}rH)p*NCX|G=cT8Qa9kv>rCYo-`?ORlmbs^WkcbW@7vZel3f(Ji z{N%Anp3lpf@gp~x_Xv_69@PJ^Fd7r*H7k>nH>ZZ*$i5}d;#t-#H(Zx$V&0Duve5KA z{2Gh*5FnOW*e{zFHIgW$2dz62x24!4s#RF3Qw4-u#fxu-Yt zAKnY?vk|{LN$Hmt3P7}y<{}%ua3k$e_j2Y+?pXzf4_2*i*)0{lsQ*${zSi)$OmcA4 z{zfH8=tHT*=*QLe4|w0#?|#nqA1t)|Iy1fgAacO?_P%viU`pYU)%bHW@0TXXyRYUh zdR#BjJ7&LIC8@4Y*C*l|Y*Fvqs}UAvU2seLyhrJK2yzfqCMuX~z7~OSxT`T(n__O^ z{Gvy|5Jk`V>d0MEIU*+HON|wNHsCiv^GV4QY2xkb4^{FoOMb#T8NVe{jqQ2eT58{0 zo|H0V+S4`A4nr}rxfwq0?kMUgYS;PL!#h6ujLXOUC;hxU!^~d`S#dV>1}>mTr&EI~ z_B`;D$J^rb&ukuebQOb7Pj7+Ke-)m7vpEF;ujUFz4r`rA?g3Hz!h!t@@YWVyWmnP& z`<7;{PMQ8HrHZY^OHbMCes*Q!#+;_Y#cLKe3A#mSD-72YGZEIa_=&UX*|(n!8NJR3 z;Fy-&HlBXHHWpYn>@>zPZ!l|_M%d#cHCSC^H^|~$6=^nN?iKBVpq3ZY+#0=44k4?6 zx0uooHyS|pPIdlSK)QH{X#B*ca^)x+oB7ytKeV4!iyc`cASPF|&*4L819cKzaQ{)sHB6P-dk3UzCEV7~=g=^? zK6Dc{GL0_lud!mW6^CdC%yF9BRn&ef$!{$A@SjHx{_Bw*#{armPfX?p<$n~(ofZx8ak+H7176MP z&3FxO)(-K3fr3~gvxMm`lj2waf>{Ibg>@ycWP2Av^A`)nN}*PD*v1rR_TS6O`j_UPGFvw^5XeWt$>>@NPES9L096gfBNWhMN6(^ zeD|H+J+bK52H}0q#CkX79x8S@f%M50JR`PuEveyMc(>V_Y3_qG+k}Ew*J3`+)9#t^ zKgbMd_YUv&?(uzk^|fK_r=n}VnTH8IpNiCGr2kc^&q?wW?6mOCe-?H--IL`9Ak$7E z=cM<5wD5mc|99d4eh>gDd%1xIG_3zWo&?f&(-{RYqXqV=BcO)ys^??d)G<{63T|!p;|W*UD{j2}7>6s*KiS{E$C-V3UK7U2=V> zb|Yz3wx{GiXsqZteVE;nyQ@_^Cx1AdKZ0&)JL|taims>?3(^aVt{;N2bNT6`hT6p~ zU0eP0t15|HiHgSe(4#rQ9*#k|t9#4aKNCOv z?8E3LqeAmsTd(Iy@sE9%9bx1Z2I(z z^}eU}Y}J|Ap>>haa?`M90gBm;uSSQ-S+aA~sgqS!z=0s6>ES-0D&LoJ`*r5qY;Uft z>N7X*UuW{SCpUFn+rG-ntC~&(K+Ws;&8qyZo)QiDAH)omE)~)1#<&_2d2Q`?gbxvc z!y_+?Cf$~JFS|-~@3|fXI!ZWVdE8LUse$O?EZR?X#Z~ zI{7)MQN2ro8p@ZZ90?zJxN^Mg7b!O#y+v*GY!@fxHu3*u9%;dLW1dvcu7IfzK)4$)& z{{Gu<7tR6sG=YSgAm-~VH~;uknpHr<++;-W5llFvl+^_3oY| z)dP{0uN|H8@}#P@dcGaSt0zPlzljHrM)=eop?_9qY&inK90P5Sj-w(=k2#Xo`BOJ^ zbd1*oBApL&pZti4ICpZR(D8y2$g?KQ;XX%v*ZgL1GuoAi_HIVI3F8fBy9AW?xhb?R z$`@EQQ+*80O&A@BcAcgjLpWM{NGtG2Lt7{c!3KP+4B2tw>=o=!rQ}dZ);L69Uf4>g zWGCFQKo=qTnVwKpp;wL;4-K4q?J+DLO_cwDWIDFM~D36xypS*>y;i z-s{9)gFCqmy3v7S5IBa^vbOt+!etk09@*1mKrG5IRZBF)rt|OGn5+&r2hs3!!UoW}$4raA==ZlxfTy~Alt$ym_Og>a@U z2^7dRfnvr@>8WE=r>pBGA&z7R z8iGxh+Aq$>Z^nnkyv(}BgGb6{0ZHkDYY5a?F`ih6K`0umK~&Jw3QjU)d+5YCfael! z{HnL)!bcc%%t?bNZ9dHHj&!;!9D<(Zb10L#3Rh5!LSDurusWz)&yXxQHUuknT9>H> z^m}(VUdStS)IHD3{sCx9PY^qnZ}05_?7kThO5k~W+&`ng;-~cNgWHH&sg&647FvuW zgG^6wB?_ZqkaH|zsC^K~b&frb6)M4cy=SRsJ$Mj%E8kqNxKg zU~DM-rFplHZL}_+65Zv7vjF!!9vC8QK|6X$~R!eB0a%sp6;GcYIgGznld)jO3{P>|pzNbOyE`7%aCYumep z!_-ZHqLad3ROkF|+u`w!HKki6&uRL1(4N|Cxq$(NCIo>Q%=+@g$=Fz0f}xFG^_{du z1A}Irh?VpFC^eG5jyd*o&H%WdF<)&^c^&H)=vqi89EU{p~0GAXxD=(k0eC&g2Zy^(hmu6 zvP$GxKfx`yHKv+Z*jQ9wAJ&RnsHRXUkVVQ?blqY^c4rT1k8j=cngkgAM>mZ2?T7%r z1_K6FrQ_;|OQ%k}1yP39H%Nn7N7TlsRM|9f?qO=tn5*1ijCu;BLkK&Uqp8(-^O%Du zorVB``G79Kummj{XLRhcPAGCbn6eknM2u&`s2N}5p?!;lBC?#7Fuivme6&Wq2rHHZ z6<`Y1L|ndnnGFF#TxN2)1`1ul)H-a9Z0pa!1rm~aXV~zF6c-4s1K4}HLs(zBW7*yu z)ky-HkK^LH6$i2=2qb~*`^j1^^t%!5;3Sfb0F(3e&Mv_$F&->+HeBOfT0GB8N0s;WCl`iyMS zNx%agNX+%XW9HbGsl>AF`+?H)5YEnEsvt0@A3%U!#ECwG<3&Iu9bh0yjU2Ad_4W!f zS}daaL!ne+d`oCTSap&#tta8}I|8t<65u8TI01r2lW+?_R{&Cr)?C8iA#n~Au(0%8 zP`Lzu*ykgr5WIs5My59zjFC|Splt!#IZhM@=rTKLffN&i+hvY1h{&GUYsumK!P6Yv z&xe*d>nA{XmnEz(j##upc!a|uK8h?!S*&dIq`_4F0e}JkEVoh~-C~Kewvj?G4p$#o zG=bo82k|376Gee($VAcH2cCH0v4U2&{?Emn`{(#vTjKU$lCv}=j(Dl7BO$@~YwD=b zszB)^25aZ*e_CUrw_sGv)tLweIfW#p?|kp9%dJe?-=D7k=uqIFFRaaD${F5*SZa*{ zkP!Rmj};r(Q+24U071o!#D`J8Ei#WxAK$2T*SDfOgJHltrqHBFG?+o#U8j_hpvtAs z)3tdCA(|bXXt1x6EI}C9j9dw?g+ae_(AbZigYM-)a9vEH6T#?H6d*<8M9Zb;I11TN z4mCpLb%(Og2_fiUxHrlT{J_QdQlHx=k16+F&uZu3=GHP;i+dj1193ghjh0LP19+J~5-*7WC@-h$38bvD4!3kDWlyi++<4izKx#QJec)9W)U1-{ z!SXvFpFqJ0P%Dq5HMvDO1e=26??^AaiijUc+2LsJY1Uzs>6C?cb_&H^u^Nm9a7wC_ z!NxM4g>xfH;wx#}g?}Yi^Ey(deJ~QLbHw0#m_j(EpbOFYeDQe<_Kg36BX{dWH11~Z z!5QYQu`eeS#Q1IF0A|tUhdN>3c1g3N0{~_W9n|7uj*_xp=$4)fjh}S+%q*DY4Czq$ zkXfL+n|}*BOeL?>?;c zBdkT7eYgf7d%|c;fgewNmm9F`0)y&E?$$?qY0>_G$S8DvIdH%n4^mLNqLulzsD(Wq zRxJ&-{!-=zqjW8_aCWOyjw}i|(G`4Qnq2_uxrc+rcs_Wi#Tm>yE(}DI!Hy3dNnIJh z;u46XVG0z3fn7!`n8Gc6p}qIo{%o?c*(N%0Zq)+>#ox zI_s=aY{rL|0AGpf{%(LiyDatmSj^wh*AoqfBVv$D!DyGx*iS7ZPe(kP$_)T2)>)^i zsdbn*_I3+pUD?&Bel~{I-eC>E!lcuma>J^t(@!qC#u3qB=#UEQ9XLwL@Ltke8Eegs zc8G-3RjJEF$JHtzQ5p?qtPi8?gt(0GNw@s0>UWkxocD3J&VmUzO)=QY(D?X&a<_M7 zM3Z2Y0|&GO>S|)^_qS3ao}xMlI-L)A;Bh20A~BIDECJ@+=s6HtnsD7|{;Ii-sl&F) z&m5;D&pL*}O=`0w98Bt3L<>xBkqBLEc;G^e4TpdbbpAgt*i6OPPkD3E!tn#(b=T(Rp&?@ zA2sScfS;}zBW@5F-h18`WtmP9ptdHzLkUlcHls1mZx~5v%9n7&;URp+RxrD%ZuzuWx^TK00|F z2$hpgbjEeY4vc?6u}HatI(t1>VqQ|Z1^6FU=6+B+4Bx2PYX?sK-3dpoM+f?!?A@`g z+7E64&Wm{oUV$p*5Lk;dOeZ)v&e2()ao70!o+w;RyJDKgr432-2R>87|$Or=4E z$Rl9u9ZeTL&R=KxOtPre`rzbwrOO$`vBz0QUuQK!RMf8NNo}R7fBJQ1Rq6n+JRMbk z;X0ieq(+Ca*tk2mRUZ6Y?gNM9q+@guJlGGRh^2S z?u{$0yD}%c4|k+CEf$YXqJN!9{aMuPmoV9Q6zK*;GX4Czt)_m&zrH>bIRH3>Y(%Q> z`<%F?E(IUy_nZ(G5gCqok@H@TfB=8<-vT@!!UG#)oCDd{KyU~rn5Ae1Cjr4`>7{of`Rj~f*u+2L6FR)!!Mk<&SbWJL{{24! zdF@Le;t-HZWKRQ4o&mBJup0X+#jVZysW(LW%mWi0_$}+*ydPU9?zoZT#rEll;Y2HG zT||d^U*=vwA_JChH$j4H;21WY&^7^28&nmVS`5>ax{XgC|b<*&D949n%C4xwx@8v z-+q_5g-zjO5tvbN*TdySbge~6@MB^GoXzqNi}$Xs0gV&@+6|)|j(5V{QWuE*j8f)* z+PvGkDvIrfYM*sRUXI_nTDTUyKy_yB7v#a8_UUBmxC4m$^yPT*uR8Hp|EWX;Dm@lr z^P#A53toX}b}UudA4p*Y+;{WiZ>eH*;MyU&V6^M}L@>57xz72zZ11){A~t#e%ZTRF zm{-S~$7*%~si8e5yp#I%zZ~+DcC0lQPld$*f&i!|E%Si61q=~X$%Qu(`LKW2VFu`0 zaF;Dj2sW7{AC2ppztQNp$XBAt0k`VqCrHfdz#q)2JOO(l#_OF0XA5`yHrGkPlq-(i)B}#%XcQ@HwVpgA#Pm8J| z0B~h^D=oq3eCBG1JK#c?hSK~;mxE&a6Q8Eb!X;`V@JA09A77nIA|7hNuh{|fZtYgl zAVf3)-2f*UzXjdvb-uO52Uq!m#aq4$2Fe&Ovta*88!s4#aK-~n@ALyUolZOjPS=D( zEV-8`>TAK6mWwPHjK)C{Aae^aB1u-7;FaV~h+A@4r#;MtIU3lf?V7M+c_uJhFA^2d z@8-4P=L%V%z>SECW8GaFXqRoMF5|a^1v9C^aCc*5LBuHdRTh&0Jn06o?3@Y;$uAoL zX*ORJ*C9*;qag;A&Q2ZgW<)YX6>4B@eIH;Mk>lNQ0z?F1$5GQ6Ox*-5w+n#S14>2y z0W|q^_W_{*70B&TFf?vThSnL!$N-Pc%#jfXCW4dG6ywL>scD`DK(25j71i-(qBB(%j>Ns=$+)v8((Penv$)45l_xa|)e7 zpM%GDVUqzI5K0S16kg_oLmzZa-)g%{ieu@#b~OYv33;E<{PN|t%YqF|8Ve89gofxY zf>Qb*z}=?=(_=yqaZZdQ;if`ZKJ_Y#?L60o`1g0ViOjM2{F)O9nh*)jD4i{qK|!H3 zI6*=M?)$P|^yN5}!0$J)qmxYKQ%9{@2V*JQ6hH_fxGrtfyDZo2WfSN^YPVtm`CPm+N*fix%AYT15Vc)3hi$JRat}ZZC>2M`r=sI$Zc2 z#G?y8oiBNvY1%jzK%j{i%4eQ&wofyuLAWN6Ub$$+!OuQy^o91>fZ$LmJwWj{`V!c0iw-}x*aAkgh|EY;%N*aHm}0PL z5jJuZkv)72XpZ7^fCZ93pr+y7qzw|VOBP+EXb72a`U*bO60( zRoYzMA-0W0|L$G1oOA~T?P-p>i}l+2R6+UfULvX=V zvp{uHTy}CmmmQmX(j>eXfNJl0b!L491!N2;G$8w$D+pZXw$iL`*@GoOeBqhLz_MoYNgAvzV0Ggm1E5f7b7JNwI=(>u%!o`+5|EyTUQ7~=r2}V%ol|A^ zS3f|6P4dA0-IP-`qB+5U4%X~R&T;-T^9P8P6H8*iuYV@J+zHm)&BVqdhj${#V5%mq zuP1p6K+m2`Yz_#FAcQpoJ6H}sO8d@~ow zz3_HJz<(h5WsClb%Qmc8G7UKp5)z)|)lQ%CpMZD+qk*NiJqAvAxR85lKA`#)1BKC% zER?oCdloAJ=rFaO3=)`XjSkgc$bv!Coexy9gIkjXVU$kl-A)pY9$F=pH@B@eLKa+5 zw^*vzZ13Pp$bxk`B2$gIRIsFKu(mu-v?9ln!*(Ggta_TpXPH#ziH+Y*vY>#g7@1B% z;agZfgH4qO0N*5TscL1rZtcPc=CdNb+``81i065~>}3J4KIG;EYj@nAN!v&1xr@wg zh1j+t+>*;S-+G>FLU5>L*%v`N2<%wC7tSU$P;ZTiy8}58!3nT-F!Covc?1s79cuWN zVJ@P;E?q60NpLtiGQXO^UXk1Z)pcdodhEPWB(*qma_(E~Pr-Biq?QYX2lq)^^v-EO z?}gj8j0bM*H?r3(5L7Om^3JMv2Aj8~eE}2-3$>~}5GA#k!vtIp?zc>f+UrZ{cK{KS zn*3{mzrm|eQ(;%PPb)Wt)U|%HiX;`3wTDX(b=^scZeH-p`!(J!b72VH4Uwb)q;HNBOrGk zug{G&&X`WZt!eeqI@O}1AZ_M{W42wP55N>mk#0ssD4WUXUP}h4AN+wS0ghw&d_uyl zRsgJU=kzI!zS_cwmN86RwNdND3-TB5MMvR|pVANHOz4QqS6?&O$Wps^^5Cd-O_;aK zc79s)@Nv(l!_oAN{A-9n0WZP6p0sEX+=NXTc(vVXPnYm49E-X62=>?sgzG{NZUr8Y zHQNXOOO)_8$ui--e^jjtXBQpNtkzN{dY|R)E2(e!M$XRCTx(L5WK;hyDa#iSx_XHp z4E_Q5&tGBxQw-agJmt9oTEI?%2M}bUf0qVSn-ZC)trll;;o_twT9#9&SbP4I&ETV5 z>&@Vn`(VhA@cZM|*5Tfj1rm3pHE@u>NdV*KzJn5+sSf@ir_)UDOw)xh5CIDWA}-!P zLOUYpDB)=4$Ez8u;u-PMTtY4(AtX2gVD-mFqi93DnOJKbQf){cJV6IOt;?b{LPF4k z@}bsI5LJG{%=e-F{$6B%>5oMTFu#{*Wnp<|MH(D!V5)RyTL;iJl%FtOw}@bIyeaK8U$X4>6Us_UQLP; zVBFPT{%C)-VrN2RBSHtO>D-6#Ca@tGnJ@$i!0=X`hqX@nVp1O*+8l8^_x|kBT5Fr# zradgR6Y*3E#PXiagun)cQQ*4-7=+U6#DF)6k(~jnCU5@i>Hf+Y^0PxtdMCc;?Y_Da za`tE{JmU2ZMN0`Gc!)K1d?Z5yfyJDhqoT0o)l;e0c%k|X!7f74(R=BDNUPb}B61L@ z0~o5PQImpmkpw*|wi#xfd1`XfO5q_;De#(L0wg*kjtwaW(m^2sG5HA&2%u*_xgvhV z*7o+lt5UlDn`ULhRE+=JfBO!=sa;L_WO+@Xj6dOI;*E(QuyXd%oj)KFr)iu!E+c>} z4p5xiXioj{?G(z zYIYHUb-eA{Fu)X8f?FLInjP+oh=u2F_b-l(r>KA2j;zgK^NriMaW9(9DCL&!QQEIF zx4-nm1t6kT^MMYLYpiopDUqv@7d{G_tH*BL;8wcv^LqdFESJh+5_9xX3~e3IrOw5= zCqzPloOZtIXCpRFK4`JRGJ3bvUPY%zZ()C(;m;req?Y?jz4RyY zxl4asIQz$$-_8JEK`a8&=a^Xu*C97$WHkWc#p%I$K~0lKdF-an)f-T`zkK~uQYmlg z@@9{JvsJEp)wq(Ps*H~+np<$3kr0nY&bVtqvM>mXyDKMyiXe>=M=Opoj-QEyiN ztYCTfwoSWz>{$`@oTmJ^mhTO6_R4o?LN2aeKaI+owEk!-Tl`s3Ve)+@JH=AFM|&;5 za+#&nu~wDjnN55Ems`91re8eeQuKBJPpC&$Vnan@E@f(oOkSGxHy-TpE<+C#uX-?7W*_cZ#{NPj1@qPFxqfTE-#s6TF=Up=KlCCRudb!q> zNXDg@PO%Nf+^VqX)X1qLAf7iBQ&s%blg*(n^@yaoU7gl%A%zq5whxUx3XhuB? z-|OG1)VLXadvj4Bm+@roZQ-LIv`R2zTfyv5ZJHRNosKA?|`d%Rr9G|ppltN5FkAK$Z_5lna zj?e?2$xkDfg~Da0r{8ni^~?okpzneQ+YQ+I16K37BQ0uFanG{;4|i|j6<4$Ljgk-~ z!Gi?`2~Kd=0KqMTyN2NIZV3c;x8UwN*x)X~b#P~JADrBI&N=V*p8KtJ|AD*cwf1hR zy1IAmp4z>4SN)1Q4;E%)?=v=%ZYHIVrvdj5r(Kui!gms;&v+Hf9}L;DfXB92MEEDZ z<f zlS%|sfPv=8Ef?BbnA0%us$pKRx5TET76TsDm$m9+IowQH$(;0@{DpN5);Yr}!#eke zX1|S@3caP@c9J^ZF_d{)kshnTVciQXjtF#M_j`p)cqAUrSzm_pGP@z9B&e@cjvt!T zz4a>1_>GynaWiv3^|PLpWzRV#n>PEXYeRw)j*hSU?oCRp>DQ@UZdhFGY^BV2(G1y3 z#=P}u^pfoHNl^Ie5xtv+tn$k$ed>ofN3|8d!WtT|cnuB_6{!k}N`vYLW7=L_icCf( z8|Pw9Rs8~L3Od%-nNis%8l$4~rxUUuOGeC%fq7S2&WK7mQ&cDdA+=MS-s52!i!to7QkEaS1D}TVYNLzv!AdTQ85+ z8SaaTA67ct>aO4a1pj`m_ZOi-;fIgLA3QTrG)s8k1mWoDw(E7v_336OWV_FJWMY(j z--SK?3GgL&>ym^mU!)$sEz;PQAaifv=JRZx@xYqD!uc`c$UYd>AHjN~5o-XS#S-}^ zM1213w)+IJ(|1Ti&!B77XcQEPWp_s=++|YP&n{6N<<>##=vAIgmM@m{K~+;o(vdM1R=)XZO7O}&c!zEr%4Urj^8o%_pc z^?k{nDeWevJ&d>hA`FibKJ(TsVY>1xWL2(6$)gom8fO^qhO~A>_dyI6*B}w;GvB$hnP%L0(s1*2;t=q$h7Tqi(U z>--X!GmzUDGyDi!7Y$aHr<)><3t@f0&b9g0rSpCu$SmGsC}SyW0a@Ulf8MSM$VS*` zNJZ}LxYXNMksA(wa)O-5;hx?7s-Wr{P(#gh!sD0NzX+F@)^M{viH^59)fz1YYJQo| zxvA=iSL;ZxLisEn&L?Bj-Dy3#>snKKuu-u&*zmzPtF(LV`^lSE=j!5Sgs30O+#3hH z$6YTyZ>%2JRbfOjlYbE+Btj|dvae{F`#-PYo1QmXf`Xez(qd)k$2ceoXRamBbisslG<83;>6S1 z^}5VR;n0p6gO)D3hHu&Xh4GvJ9^6GvtipQNJN;A;m5zxby5=c)4EsBF+V1w|vrot~tt%kIokqH`5q5cRqmoMc7`US*X32 zP*HVtL?deij$V+GF}?p#K=j>8WOVl_!Ggguc(R1%FM`r>pz~}tOhsgjKEKAiaKvSI2#T(Nyud4KlyOmdT zhy{t;CQkeze{}cYjv<&s1)#(>x*z6EhkMV98xgQ>HQ!p`-}t#T5$Ekk5?+EAmIAin zA49!>%ei8uyyz^@g)}xrC4!c|w?%2f(pf}QM5QY=H+M|(C+4tUXeM@eYEBCK(|vYB z6A*(@W|ulqYl4K5yG*(8w23wxLjK6cUq_FLuGk_WT;{m?P-Wz{efSp2#9st;IZ}2I zeFL9U_pq%cP&>wQf{E2Sb(v7-u7N;IXLwbxtEPX4uuix`Q*{}WJf!eO?Oc|Y-tw%+ zHbmp0%#&2GcBZzrZ*NBnP!uYzsZBkFehzqbx2!fYIOdgX)JqerMGBMQ5uq$FNDIEGF!C2vD*UY5zQ%;&|gGu-BS`a_XjG$60QrC>&4Yfl;~XG)Zq zZq^w}b?si};sUY{=FGxLeOdrXDpOfu)@}WHdRF7co$Zh{;>)L9RXr#ERBX~G&*l)j z=-NHMEvblMsul1dt=A^T^qhBw+9ZbXHBTdb?NLzPhymKUqcOV(vyM+EwIyFMB;u}3 zgh>N5`&t(57e+UEGIo2G7sMoFtaxJoFp_J|Ns{m|^5pVbp`^IP$GP<`yNWe)7Pz&G z*JftI;a|OzsRNNIa~+QFouDk(IB~auten;x61mvfZ(v%z!d>TbId5d)N+0rBX~qLd z(N{-cBLDu2fSm^`3uHy(aTT{6qQC%}#P>hIyF0oo62&kwVH&R7Y*@Zjo*Py>JYr4A2omL86 zb}FIn)1W^I&Nb@-M}WcVaXM#k_S?_%nE?E6aB+EKMgkuZpcyU~%R)=c1mQ4+ z=2ps4$-F5+YOfY1Ehovm0f{45-I;#Vyi@$VF+)E>7xtJ|MJvpWynDA+W`fY@rG72@ zx1B5rUgEn`Lb(bGP=cRMkyj$y>AuXEev;4*ad)!l@EwSGx@|^Y2gzJbMT2%Z$vjY- zG02=Q?KdFaBL?jHJ@hdLw#m630O+wF&A%(UPWPf~_%w7jEbRHkTDpvDc2%Zrg|J$z z8(q8IRm!`n{x~r8DnzVa$ac`AK1v1wF5D4oWmtJjCN1Ku0&aJcWBww5SnSAy{5${*}S1K~%S4L#@5}=ouYV z^Dosp9XEkWr7?`&V!s46b8l5e37=VCoGaihrMup(YN2nSbeRkGZ@ld>)6A57br2(} z)P3g=Z5=HkKabo`Pay*AWrWJ>WBXFqO!W+7R^tn+$xn~u0TFv4D@bHL#Ab(=xj(Zq0~3Nb0xVC&1Vd!5 zLqtnlv@)j5$2eDv-Sb@Hu?M*55Vk%o*`So&(T+x-;pzQ?EoLYlJQG zs`pYaS(eMOB+Z%C)2lW*=(TpD&kWW!^cXkk*^bWo@x0$b{7>ifvO$oVCpGDf>`UqW zC4M2n{Tce}%t_GYYSnp-r~PcD+?)H46U_I_8or-dR?9X)IJhr04|IfrI5(pCDyn)U zd@)2xKLF7`5L+vK<}I{x9jcse<@q4O-RNRN-kWR62lKOjV~N!_p3l!ww@8G;GQw}S zx0m?nd=g%)Rw<(|+K0~W;Krf3GRUO1%hia&6PrXYA@lvY)Fx#tjQ@mXQ{%*fAetzSyC%l-)4w=&v1p$E^ce z*V#@|Df$3nz773F*rRo$A?>5g`alp7yrZXZLg>H()#`%2QB8nP8EGSj22CNhkZ)c{ z{4c{+X7?(&c|_Y^1a9Y+xEPKM>`S!mMP1kSMJ@B@7%n3^ z#TFy`@+1tlX};x0Fddy;<5~8118%_b)vAjCB8dL_q75RjWQTTsOgc$7h0O&|Nn$}K zBB5b^iXPsv7q_SW423DRL1$d3+m8h)Gn=vWrv&o@CvwE7(WCc@o?<^`3ol+{$&?%B z-npUV@{@&&U#upVc*VI$uY5#I=aq!R$%GrXA1rs>9J9NbyLjii59WE%%@HK7ryJdU zOb=AU#Fm&*1Ijq)aMpZG7 z6zxYn^a5e{%|v`Fs++XOy_?wrtRHu7 zUNFK{qPvobZhqd}Vwr}#nZp3LJ(#q|ag5CM>Ll~CPn<-~*Qgk*G)6+A_?R!MRJ>?ZG~zYc1+#qb@R@g%#PU7fh6b3<(W?w~+QvQinrZL#b+YpL(f$2jy@ zbuyRElzpjiU*uMK+@Ryasa^Sc2s5H7g`D~vOU=&Z%=d?6Q#0V_B&khn$1W74^B9cf ziF%r_6VsA;=gTWp#n{%`QZCgWOJN|5U3IJPP5?wahj#8{!iSg9-f^nDN;fRB-wdig zeDEHSv|h3Ttt{xHcz55~yZ4hj)1Nx~31(4}s&v#28S_`9!W$Q71e`fZR!EMl0vNLb zd1Kt^Pu;AZqrI3lIl7&+V4^Hv9a z{Do6bu)Y?vd|;#u5c=`TSGS75ma%Ha)LoC|P8(3MLg?)mp%7f&LC~HQFp;w|qa}y3 z8V;AZG9nm2O0sJD`=_hGI?db=V~3cR05gF;ft?`aO1wZoy1)z3Q&E9V!nW8lRlCN? z!1kUcf3n}_Q@d2lT3A##+w`hUE#D^EoV#5c8~l&wQokEahuu+S`fxZcr32EJ*4KgV zfk)dw+8jrR8wLnFZLGt-`urjR-~%#|HzeOFhd3-S0v=|1Mc!8{(z~~k{k(Hgl(--b zsTWKQ_)~RxRxZT?o=4-wot&}+D|DCm!~vEc#|mooA;U{9p89aMkcQba20;XOv1O3nHDH=F_n2A{x?CV&uhmK_ zWqMezkTAsfz8;>r$lstUPOzBZ@bKvs3Xmv?6D z%2~YQ9Sm`mONVJb3%WMx67%i)<&VosmitP?FBDe;-qn%;*@J#u*S^&5T`<>lEhUK! z4#F7&YkFrlZmX1UDKd7RdD_G(@@-b^iqI%Cq#|HuHTY^VIu81Vw_Pv~%W9YY?%Gv{ zDMaissAB}snzsxZeotV*?Zw2vej(w58^@52Em(}30DuauTn{!|g1L)3vie+NzEHG7 zubl`;8L!t0?qP1)OwyiCy+yv1ztWM*_4W03ib@=w<4%jcb&pOTyblhnK+reLQ+1u< zl1{aPWtl;r^Q5Jjw|!gf;PG~lqKSFwZPX%4x_MRE*c5jT3$C;CLoR!kSGEV@ zne_FI&NT;P8))5k3BIs&S8|W9Ay8oWf-_OMnAuwcuEW#{&P+zm@mrAEZo|cj#|&NH zsE~}+P>90x{bN%zy%b)FF10ClCb9 zl``IMk_U`8T#85NDE5qwjg~CCTyU48=Z&KsGN7Swf0Ov>ftWsmx3!PVg_a*KU*u>9 z^_eja`25j34GmXHQ>-h6qJJoHs%7Zufkx2T#@-)_>t_jGZX|snh`Wwa*dh$s_o26Fyl(}^C_sk*C5&4 zH}3YTt@!y6TS&-@-~gSGHtB|$u@Z+7JvR2|ntSWejofJ>s*$EQ>bsZr4udV;b=T)b zdnX|}-;K3oN%s^_E0Ct;grAYSMzRKcLEl<$HCA>qQ|Rek&Ni2?=~xo@L9^I2=hBdD zojdvX779n}MqKM!J6Rxo9kYgUI~ccvd)^#DX%gq2@GW%L=QO=*%6I(aMzfg_Zh8KB z#d2|3<3$McJzb@`4ZW=x8CBEW(a^p7g)S{vZbzRLZLQF!l}J!-klfDGynjaLK|Xn+ zS;m6sXSaqc{EBqHamsQE{_=GE+P*Cl_BejTyuX5?$T>BK(TzFEodT}$QMwJBbYGKL z)y>&FMXVhK-bJ^gQ^Ot}lSx&XLfS}g6-CIwSTfg&GLK;rSR$>CZ_ok=Zhr*tQ~Di@ znbdGq)wVzPJx0Nw50_t_`{$fGXl|m$4Bro}uM+cuPs`ufm(kk~RXoD|#1%bzZ&{>9 zF&RPV)@8nbPM`b^?(FJl-n^?hxzfR$psBXOe#|brtQk=sY$o0Coyy8&5@KZZWb_?$ z{q;Oe3><70^dBwu_a8R%a$iqAPnCk@F74FcvWdMRePHy4`I0ngWkr{(m(9Vg30YwD z)#7GU(EbLtqdihqp+Q+bdCw>h3VC|{2jbSEwS6ZI@Zi_e{=F&<-5&z9aaDV{kVTU! zMKFS*ItpS3Y&EYvseO?9m*sLm#ysIk@D^MgLU*tT??-uw1ixC1Z~8|}J2dVo>abhz zY2(wxSBk}*`k6HkCwC=h4ER0P=#B4N2|tQycI~OS7wa2D!Te({gg9d@ay(GTL!XM& zHt%<0H?XO7)UoR-)+vLBDt`I}tu>TCq|ih?Mqb1NbY+NC#urWNMcW=d>qdc@cEBDu zCT;d?L`b(F_?zuuefbo~W4XqDq+$&IK|qhs5w{2D1n zbIMi0FKfd~d5h4R6No4}!?(tIu&h_ktB~xAi!DJ4$%O{Ei_^HK>A>x(Y0-(_78n?g zUeaP-b~u9ar>gwt0ki?C*GzTlYmn-V=@)7~e=;F^g)_~eNM&r_FF;IS2peHZ7W3Ad zmu7xo<_!owZUW$kM&4>0x&iiYgO+S+c~tL`ZZf~M9#fdr`;n(#=(K56c^UFxF*+$ne`qBwT* zS+v-;%D1-hi1GB(z}^vu;)IXz9NNZK6NyL1S7PX*l}vbuuvy8J>pQOyR!v8-D{KnivW)R z*#i%z&%!z2hchzkHqU=fPyg>4{2$~0DyMt7u#%mS$Z=|q7|)LKf3yFmWwCH@yj3tS z<+;t8DYl-SFysG=`tL5-lm}@fa+}W^?pYpNs`39KtQl}+uQ3iJJobB3zg)scpaNq2 zpF;0l+hYSBg;)QPWI!QV#eY2^8AWC`g7YXi@&3#G=TsbP1Eatjv73{MIp>bUQG;mG z4MzLE%A$S2QD;krY`k~@cIHj$2*ATnXbB9wj#E_TX-qur7{FR(h*WF%Hk{Z4szt7> z@`TfLp-h}h5<9v%fmq8vLeCf1-{(f+l)>G}(P*4k0&!HtB|9*Gue&5tW;?GdoE87s z$=mw3oeiaJ34D z9cmJU1NuvYtf+&th!p1}sbUBeLkVdgC~<%@>cM!X2RdYzVPC&3-8X2r2md-0Ad z%U?T$%Z+;F=2z>6_S;W;=Fz|$N)^tfhtO-zY1yb;p@z6f_J|J6-%RFzs%-}qb)_&e zg-BADW-h)~=5KWH%J=l>iHG#`NWXUE)bPOnCeIVRFbXsAWXv160fW-ctk!YzyX0fY8Q7a zOEWnGKbdeIFu)DL&;ZZdM`|UjiH46UUmMiyLWj8lP<;fIEOZ`v>xAIF;0+nTHfu%)$e8Y+*Awft~ z9Iin5b14zLku(o=yqIAS*Tf(>H@+*CaWe17_ulYv`@Z{Eclm~Wg%yQ%ioVp%=RTKG z#PkTP%1Se>JV2Ub zQ)*rcf9jJAMZyMklvY8`uOJZ`x_3GZEn9@~=~+H1P3Lv^+|juuA!th$MZC!`&8T9y z>hhXl%@dfWZ?id<&1!1(u9-Wz z9b1R&?{eFU?VN?&MW5rc+ee|durLwtg{eD7GQwKW;^*qaMmwfaYrUy+Qr~sS7ZnDL z+TA?4o`vNT%tpib2VJ7EtlpyQR!vc?GHbyq!ChYyjo z(NYl)2jK6S<&0E?;k6UJv@vdUKh`HMRqHJ*))6qzutr9q?l4Oc*gWKAc4Do+R*OWO zT(I`len#?oPV8E=9Gy@Xl(g_KLX4ywLt_{uZf+JF!Dz%`{IPXx^daX3%}R{ts+>ss zRZM`zjzwqOlcPfwIw+qE#~bTgYAim%!W@q3(&*+P~i!!W$j*h zk#iMJ8!)M{4l*0k?{!M1g1E81JIicp|M?;RQmOjzGpj8}kERLy8+EWs5q*@Ni(Oo0 zR3lO^n*Ff1J<-}+OhkW&2dH&XMW&0KnJw+)xrWq!g`YW>tI9DYdei&BIR1flDKHYK zh=mOvKOIwqgEGjF%W@f2!2FH9`@XN`r!UT4tM6M($Sg?R>!anb!Fn!J4OMFd}|YwN?I^lC>7$d}_~ zNW>*P@TKz-|9Y*CsVvu|jTb}8{x8C}+{z-?;>Z<4@hcUzA>xHU!v-7OMouwf%0AgI zDvkMvTDLQ^j^8{ZI+nponVQ#%_JcyYTDFReY_cKKg9fs9{n^f;KAO`L$0yI@y1K!a zJ%`6ASh+YW%%HdD*Wms6HK=@O9n++XF+&1G~^#!L<-U(=mKyH0tJ?0}Y6g zineWF{I6!HWn;rZ*37EHJda~qVIA(=tAJtiQwmqeMJl4FFT98Kd4vn~|)CNlgWN*6{bAMbJs6)snTAwPFN}gpBF1jj1YEVJU9UDb~W;u(HkNj%{7^ z?=lbZ#d&=2b;ZieHf@qTQ9CHtERHk4%R=Q5HvGDKjR|EDKeFN`>(T7z83L-jLyZ#o z=Lv_^PrRfqz9nlK@a{$h5s#BiEfChC8QmMgzey9!g(r$4nhhRbWF0AJkpEOV0fx|g zDRCJuNr~jJu0n0vYX@6M>x$}a-ux-5HJ&Idtc2i+CBM@(G6~+3_XO`{HqhBk6{U;! z#it7zv0RM_R7iG*Rp3ZWuuUC}iRz74^>OKbdjlllK zb-h`be7@?7GAyausI4j4pfd>}m;DaG!?8){Rh{Wsq19HLm-CeiF2W7g`b{=-fwm0$ z<=UWL)kG}dP4KzQqBIAlpDi;^`dQfRhS5mni=J72WTS55T1FlO;KhIEvUh!>_6)<4 zIn{6Gj11ihA!)Visx8RQxNP# zcwG_QNC&Jx`p))e-Wr|_;!0UxtiZlvaCmxMNwZkj(I%k) zbAfp=tTMwu526jLSy_y>zK)&%h*g1poad!?S5j-sb3R`3wLOh2&(^2BwWdH1AM5b7 zzf}j*^sg;2Yo+C5YS~PPbXDfkC_C89#soF%`qff7-1|=7t1iP06iZUru`I)yetS|Y zQQp*T<)uzg$^TXUdGlqZ*$G+Yn^;ntM1)n}yC`fl93-zhoLsmtML%Bi__{&Dy+*<> zqWNh zLI!K|@=M_zD~!K1ls2BS zY4v=yIj4FWo`hKwuy-USrweZ?1Irtaf zV4sbN!o*TXp$2dC0gvQIm+|;{XUGnoE{}b_%j_&?XAsF0&kWn|pDb7EE)rWCGCEi# zmxeOFIeasZtnVq?GByRw4kM<*w|oy&Q<7FqyWO6-|^%21j{rsD0rt z?TED%u(xe%Bi1oa&^uGTzf_N%Q<}07fNrxEXJsEQUOg1hojm8uo}Bu?c{JT+Z$8J+ zz;k%#Ky(%2S-9*A?2u)IP!(TRI%57JSf)HpGiNh_d6>&V*2dd7YAecp6D(zRYk7Hg zV_Bb?x$!P;vHH9`nFCeagoN~6W-CAP=hi}5_Ft_rWY|w1LoFC6OE<>Q#Sd5Soc8jo zo^k4#tZKY@>z@@Tu*zt=42LK#AEBT3(_?nZ^r2e&*CX!ryLW)dKB+HymIRN1;gTZZeZs{@mB9 zk904|)rJ0d%5Y<+YY>p-sA#B)>dLul_h^4~^?U>oFOIsc^+~J^#(Jo!E_h_gGZV3# z=LG&Q1U{89O-73VaG4#eHzAS4IrF-;1xVt#jsKR)w$UAebk;3|g;hm+0 zd9kqxiMGKgcV&m?eHkQ&;=FMR)H~k$d|HN53HXcfXLdw-a*S|Zd%C4vDn?I-LHups zkRCL|DHR4x)okp0v;uhWm5&Yv95&G*BW#_>beryuTFy*`7toiy5-4SX34 z@#&xSWVg=9=MW$~7(u*PmP&uV#vy~oeM}N!skY&6dgZiP+8+zUIZU{@F8UpzEVJ++ z@KNYr1X!B?7$rnd9D04~EAmEK8J(?BnnU#JO+M{HVq>s2BA==xiM`{3yuL5NOP`3d z+FykGn1?rur4+srk(Dhb#yAorq2jx@vt&VEpV?{^?!LREI5OUsClv+#f&Sv%Q?_=2 zuIEkuy!+U_M*5RfBtS5N{>ZG@XabQmh1#qv^X7dvh@wU9d_u4=JhS<`<>Ms`a zYNJ}T7;V8r+C~!lYoIrY7Azx#Yw_2&6wBETN+#VHHIiwa%~YykVo$eB`Ay3H!iT6i z%%qd3j)4pJ3{McTnJq(dcaBBEFh^dNINl##>*W%~EX|aW+wacf1yxuFa#f_2TYxKI zGH)wCu^rjW_ji%N1}kcuX8xScKNiZR>E^>qHeaiROuOA?OWEVB9Yy9|b5Kn=kb)S# zx`CRrk6Jwr_FcYaHdA9fiV|zPCCP74_6GySnun@hEhR|_>a2)A-f|y18ZX(A%zVcQ zu>IFH+X<;%JIo3TSU$-t!#XUdr__YAq5mBk5kxZtvg}O8>Pt(cY*l#dP8mL{sF2{_<9YCpv~3= zF}bBQ1_Le*0hp4svS6VJMmM%Q1a&ej^VsxBoG_VH?LtzUna$sW9fBwOCsEkY=SQi& zJq^=W{#f|;`-ndf3&MGc1;2ND*fZyvMak=El>0x7$=snHKgpUI+v6A9{{5X=r&3k# zUEhIJR+*~z{^Xo>$1{Kue=a$q65z@DHha|oAkqqs+WFs7xdYBmK6!l_E#Jz|{CbiF z`VXxEO>!0OKlOi{rrf1zR?)sRptTbGv>dH-8=O}wmcGQHtegE0IoBN8u*Oz*E8R(y2Q_St@p(3FI8V*K);V3^l`aKSWGtidy;LM_iuZeCWl0+P9b`U4qt~@K}BicWm2C zdK&jXA9*nsxij_W%zonNOq+NiGW=8g7XiI`&-pJxu;vy2UxZ%9e>1zQe4RSfQntr< zN(p|E@AaqI6p2629`;b&X`^#8z`bXE`;_wQMZV|Xgw!{`_C`HeM%#|0e(cmH=1Tzpj5%F#bGq*h68bjrY<3w}tiX%0DXmN0*)P z=j;Edtc~u{0C4}m8v8GmzzJ|w{*RJZ6aGj0yqEfe+1>xN{g*l!TX|m{N=qg&U)Xe% z2CNmBRI+`$3pPnpzA=sAA+R!uKmXyEsqP`+cH?aG`__SJ%Be>B=lC7Fv9~c5OxHCn z<}u=ktQ*ahPq`?XKxSYEn9Lv*%oU@^?Xk%GC^m{dhVg{I`K_tE4*2u<)#(gVqK%2; zqnjqL>dr*z8J#)-sFwF88C4gKK%xFUp4}coXrPypLX+9T$|64+KkM|b+U~HYlTP%% z>hOc||I{vbKqSfeaHjvJS@APV)0F%dMS0kY|6juXVTX&}h1dTtmeKhCh`QWq`={go zH2FU*|7#fk8qYuC;gbIA@IS-&r^){c|F0hZBNa~j&&B+Q^FO2c{}|N&8O^`iedmGq z8#f^3OTblTanMnXJI*UHupI#)$|AfH(N*Z|bp{xU9sZiz3XOp~H zfA8g==a^yA<$d1E`#g76?B7-CbQ~D#UllIi}LBCU6qh=)TohNW+H=Srztf%z#nr+szbmAncHP^BuBxwNeFi5)R%%7~i#%xd9dGK3~GbTbm47t~B@ zor+em-gQscBoAPra<)&^vqVODU?}lTt19MFz%&k)t|sO-?Vqjn4LIWOJdOO&dZ*>^ z^0r6}YA|&O&!TwjXKf~2IIS?)OEN&G`0(dZT`3fl+R-ayBN=jk#FfQh+&ePQgrldb*TJHIrF+@Y!}@Q;D|A;6caKaPw2vJ) zWIpr`f_5BPZ~h{f3|r;dOsmyNbpr;zAZK(wqijPn0LXOBi&$!w8@;{7uSveS51k2S z&b}X%#o)|PF>W*FVDpUs^*Wlfo>VdlM9J>s^I3DS@H=WZn$~CrsdV3PR$oDu zm?YZ^PF>pjdC3r+HZ?$J%$L9)$DIadTM3D6TIHCYBR9{IcVLh`1g%Bik*nj=e1e5- zb&{2km1jcNf80FI7e7*Uj`suqBCN88&aL0mHUr%5eY3}e)(ej%shtcL+eUx%7w;tm$dUmfyS;LEK zCLp>m=JtU$;$KLQ9hY(QN%Y)nC-&5Wi_Q8Vr6gz`$+6}KDc$NFKcl1Lrr!@V>nyKW z!qiSGeR*G9Iaa-;;6o>Q2G)XU>7dQNR(gqikQJ8z z`1nX|_d~Nz3a`@x8sCp{N1CAl4oii?9hDaIDaP$mI!>}RRgSTx>GbE@L&~_P7pBug z2wM8oV@Y%i`%iXo%l4M=s?z$a+%LherP2(BJg{sX`K>dh;pSFirTUPgF@gAGS* zQZ{n@_%rSzx3+s~^PKIA(tn~pUx`o{1rvFE@TmB#c<_b-RSb(F4?9re{l^S~?B@@- zKj2?#u~>xbl(n=?36v!XIPITf5y2x~SlWj|h!=2QI7Mx!5K74nS^lhMKpXys?M5|{ z2Vf&+_H6*vic|DgG5M3WM+F?kt!*mgFTzj6hghQ>_BWv)r*1Hi;Nff72p;}Vfj?4H z;9s%*G^Y{>CsB*3&Uu9S-;0u=QgxG|!sWvIIvR4mAj_Zn`6~(|&Pa@uIIgm@uq>T0 z?Yf<&4KHuJMVuff;X7#N(%NvS&jZNvw&~rM_nY38>m3e4xcL>Cf88F`cu4$!`fG~o83mpQ0H1|^3tX<8p=zC8CiN z{8$#@n>9x$*K`u~Nmm{c4{Nt%rc#$*O*3Ql&L-%wlreAdnVP+c@3<}WWyE;H%~#1PXlU}w$6U`y|K z5sU8No9{bN!Nt&fI|#rgt@N-+nqLC4@=i>e`n7>=tFaKHoL}y++Z@?Si!=Wsh-OJ> zm`(C!_+nM7I#?VLGE`RevkTlUKjCFDCfl`@y(mm<#-kl|2)@7aw=pGR)r;0SepkAS zjA%9PQ01+DHeAtBZX8za-w|P&GHTA%t{)3x;Ced#jLMw6JtTRuToirPk?ol4nOhk= zb0_-6)`F+eaJUia@VJ>gx2|gNWV+oo(}DeTEcX`EFy8;hMs+f) zt*%MAFBI7j#@2mqUAnYcp(R@LHU$8S(ZyhOs@ZStFk%mtpHt!sTF;d|*v}I?TOuTr z8`njynqcO=$D&^I)j}wZ3lnCQzFUT!hHal1BfaG@G+k~Z#a3v~vd+3TUC1`wS6=p? z$<2NM+?Eh~V!BT0y$#VZxtAEd?%ShWMnRiBnkPY)4}t%AEGqCI^v&?g3TcqnKs_3dLEyaCg!X|DT{UIglqt52 zqyIgEWd2A4=$b|)`DChkV<*qe+xGa)sYDd9GCp6lqAIaIzz z4eFG=w}!?M>BH^!Mb#Vc`AaN0mK$#;0>-R1ic?d9_yZcuJDEu;4-~|21&7I7Sq9%k zByxLMd!L+($9O$u_g&;GrSvzyNp9LQ7rXMA_K~>?9;Pj04qTZzpshqmO~Li5?8 zvqb-JJVRZ2qKIqRwsmdU7@Ok#@4+2UDye+D<*b3=^rX)P(IOV+<{9FGS55ZcFF_=q zihc{yEJ#LvD(tRR@kdJS4-cPZ_O{Rd9GSGLR|9~-UiLNFv#HhT-LUv#|$E%U@2gX?oN~xiL2fAQvd$F^y?8CFf3{co!Un!>4 zMsEdv-EKQKyhmM_$NV6hZz!s4c=tj~B!#@5t7Y_HbS_MNN7W6b>uN*E@uQjw+QfBm z2_~P4mA#vXE|r?zoCb)=Ldv;C4P^omE$QmN^&V^Ih;b-Yxq4D;+K|yr)3&U zOP?)cPU4E1uLfZky%0w7VC@>${%ui>vPuVazxyZ5RAw?x%$i zkDT_7I|W2iJ(Z(2=+930T)Xu!NuvR#mgs71wnRVKY|mDnpj^~G1Bpkt7unUumf^6e z#K(avh~F=|Hy%AWh)Rvz?-K%Ks89yU!!F=D?n++zNIs|97yHvN@t|EiDbTFwN)hhr-70~*Lw<()qAlvakqAhz zh{97T%wGg|rA%!kHHiJ@)XyCZaA>3WNsT*WA!ZUUB(6Rsj2ZhSlB#VXXz7y-gn-K{ zq{OS=Mez8x~tHG5*-EhVB4b9~h2x674vltn_pQNcVaP^*ai6Fwz1PR&`Tk688I8RdumHAVkz1Wi;*sH&>qxA2mueTI#6KC`p*${i@dVF_o7=iaVY6f?9F@nPUOk@4a;xYZPJn7colkB ztnEI%J-2;TMp6~!h{L6V0}Hq%3^;k&v@0#Z&$vK!FxYIu+_7?%4pe*3*Jsisvl*U6 zmnhAx!$q-K-4cz-jV8my>E1x~|4{40zl}1H}T-+kOR&zZVRv{U&*rUvgHLjg!gV z^4`3I%URi57!tyywMZA5jG^U5b%(si_$C_28kR>((D~XFeVyV^R|g}i!R6k@WE<_> zt`bX64nP1qtc4cC)kaYaRkU$Z0t(#F{`dgulpyP)Z@M?>wjXf-ge;r=u|{fEs)$ED zmkuBo^I+Ref@7((D<2T)q5i^`7?Z$RMy_|Gz)%rKTXOO|!a6cokzu^(ZvmQX=#Ra>o?F-lR?tWnhxn^G(Gu2y5TRw!+> zsH)wc@AEwW{Bo{4*L{-voGZzdbDeYU&-?RwXSc^M?{umT;68%JIGp1nA26ROxr^as zg_0BfbW>fjPB}4-MeAnC-phR&xAhWZP}c8QJgKT01Lr2Np9Z_UCiHz>huX1sWRNPt zeY3knu*P#Gms4iNxXGf$__1=%RFUL{S-a;FTHG?_0H1O zw|1X=abR@Hgettaqc)J%56!s_?uwYN=Dd3fC44;-S2yFO(=MNTlhCG34ybgvVWMLz z8u{qx#vbND99tY!hh^=BJV;@m)tZ-zp}#e4s58Ab-4iE|YsI>MOG50n5bcyNCNZ+= zS>xR=(&e8-u>XBai-xFxx@3gdF+iW!X+gQ2Qq3We$h#f9$VkD0<%;GMBH!QSbWyC~ zG>+}&njR~US%pk*Wt;yJMus=tOZW(Q$=z*2dWL$aNbl)v6~tU!f-nfEY|N&XsOn*? zwBxQ{NUSXqjfdGPYdg=r)e?)U*R%PZ$d)5eUWPFD95o3Qk%Vbi?h9I7tr^e6JroF2 z)*v+lkk9>Y_{p9QM6Lyc!pNEX&Kh8N9oLY)JE!!AxD^4fw(2lBKR2q*HqEkn1XR1O zfWTvr;vJho3#FVO{cVMHA%rZ)?g*c9vW83Thr$tXwFkP8k6(~@?UL<9=$f*vz1Xk= ze|Iu~>?XBn96fAUSXlcNr|(FMN|XGJ2i)X%?;+2@5_-W#pJ8(ptl$f5e{zH{&G02u zz*vinZ97cwIaH*B=f<oS!0snv{W?u$h+KnWGk5rLpBD;S+aFCUFgq<|Sekf|Petm|)x+OBt3@(5 zi$~~kZc7N}X_}g$%!c|F=SmVNkv`?1i@z(_qSa_KiQ&pP_gT4 z5+ZYT`$jhx%MUf_2D?LQ^EYH3xw6!IZ+gZekF}P7u6!vT%|1U!V`Xmjo;JtrYiBYm zQQ#W$6u#m@yvZ-Ur?6M0bBS!B{D=BDN70>JGs}--4%I=~s`v!yFxWC&ll$c+j28v-Z)}Cyl!G;x?31hQh&JsCRtc?r`8ve{PhVQh6E1EK zJ3ix9V$P&Qb%XkovYKIP0{RfY z+z4%X+OU?Ak5mVe{_s_2l26>2bsZS@l~1aT%GTDf5{UI>SxJ2wAUH>@!}y-{&gQkA zmm>|cOx`A9hPgLYs(vRJMc|#nsXib`%^#?Vdmrm1#MD$YIF z&WH6Y`zwp~6z#5ybcG@b6?1{g!{K6=s@$ig-oRRz(TNCn4caw*V$*gjaH{nYb^Xyz z*$g=+g%wuGRLQ=mZ)M1}Tfcel*V{rvo(I>!w*00+y2Ib23tk|4tOdDw~pzC(d< zg-`m_1$vs32(A@gt5o#;dZ;g5J2jZQVr z>!!-$u?~DM?cs#UoRGxeb!wR^GO_c$;T`p|EJV<_Li6oE(c@7sEswg+?8_fqlX0#w z-RcB6d&&AH_IPAlIqTJ&W2knjTKO;aWaIz~c?D_6IEKNUDGq7>LA{?wuuI#Upu8?v zC{o3-bJ3W}^H7wmzS$d=9JlthO0QqV#eO)QQ*3Sx7_7y#JI0ELuM8F?yI$!f1i=T*3bmW%$3uOfCkZ10eL<)I195x^9roHVpL=+jB z)L{(%ScA1l2-|?hp`uNgfJ|=jnxs&HY}zzaxrBu2Q5)SP64ZiRmk(QQYRTcw6n;FoDE6+`@!6I7L$iV@vM1o7z6G(xT9Ap{9T@kige%IM z6K*Thl8qW*eJH2TAJm7w2{Gz!bG|22KZd3DmyNq%Tl0ubE6)=YeRu#I?C0R6z)q^f zZF!5wU3=c^n}oANa?Th&qeaBh7njXwIm+uwW~k+S_5%mhzL^&eT~Q15wkY_4j>hQi z*EA2@qqMUOXIb`XY21TB^$x8W#_p;{?frvrst1=}6?@(0ka2`AHPCLbPJXQG;P*K) z7=VGxbhaHgAKb<3+SQD}qBZ5_CJQ)k2D3a|x$AJxYdga@n&OZiv8M74EP&9TOwRcL zt4P2oxNdTK+m;BStaqwk;~O-2ZR#Wzm0hmh4>L2y(HCI(0dW)<8muh9!j(J`|A#ux z7oQKrBhKVQlDFteJx>m32Y%CGm1wmnOErb>A|2&wV%}l3kNr@&)Nf2XJ-yOn-!}Khp6EXe>^@*o8U>k=|szm>b{;(-r3<`jsT?xzl74d!QqT;=t=kq@WU3JePx?+D7z|D10i=K zGBdjZV5_&Y;*rG-?GAfCb_6|gYWj6Ov7+x#i*hqV=RiW~S=~IODE$|NQiC7mbn41% zQ9%vZQ@q+>Ms$dM#%H_* zF~L4i^W2%E)=t*0aOrTQ7RpwXr~)?eq&`dFCywbxVyKnHHotMmVDhyGjjKDD8g@h?lOjTge^{-SIb%>p!7W!8QsA4~Dt% zz#fVof!FRM`rD|H$Tj3zRt~;&bcnB7WGD1UWT4jWsg<3kfy|Pw8298=qF{xN?f}Y^ z$5k5pDAtI*P8IZ6B%Ftwt4T9J(^bU7^^_djS?R_JgG;FLm)k5-%W5*Y`LiOrb~F49 zQOXA$au|^8I=d#iboBbR8g>6`UT`46dTtnX3Bb|^Vu+z~daG+!$bv<4Cdo*ErZ!<) zj$w%Q{x|AVf98r)oLXT)R_jNQoYnyN+Tnurqzzc!XOHnRWv4I5OxES57v5*y*-u^% zKvyIV*)c1`2b@a%G9Zo5>ycpnyj_RIukHhFzovcH2H~{ zLVpMmTr>BxS=(3;Z-3(M*p()bYj6ZXPjG3=)_=+Vt76TI@*CigJyi8Wi2^8*<7`sI z5^f&GtRsI6t2yi|grRnZOedw;L*-dbEvy_|D~vR$NJyfG`Oz7@&Cp2}40@RG{{D%` z&2ZZTeEjdpg_8&UqoQk_p}Y!fGEz|X0h%o2y8)Of!o35QL3u@KJ)nb@hRq_@@+JI1 zW_1$DLda3dl0gu)MFjxB$MF4ty$~*uHv}>I0{O!jt`3H6tqmux5QYu+Ig8lGAq+<( zM}yT4`hn5R9P2dl7GyHVz?*tK$+pXIPx7h4e4;hx5;vVG_LSU zeEq)8*Phl)L8g{+k1+8fV);2?A!?tgZb+)D_JgffUO*H=;!jb;{TxRT?&XAEq!K>) z3_-~T;4&$MuV|rRWmDX0!5<;U7+vNO*jYEZ>Le{3U0EwyxgZcjGnsBTV4x=UT~14z zfVe$E=d(M9i{QaF3u!_WAvKWl7oXD#fJom#_=J+&RZLFW)%bY$M*6~t&2qd^=4NJ$VV?F-=BAku&NhhM?-I$ zQl43NMEz#!vUC*AP=GbN3e!1&+ZL^oTD+wBiSP~lOM;#Y)-EjYgG?eMOemN~1ErP- z{?Ki8)iGP6rA;ee`f8Aan#HYpF5$i(Lm%0OVB9@5tYyu_k!nL(ha0v~coCOH>$=vR z>{YbM*lep)=iL^gM`{i%8o(WTuxM`e2oNOshBY!}9>2;}OB9#+6aQjuJ@Z=%tG64w z`>-W)@#r(QUdvy`uN=$F0eqszn~jrm5?;`T^ZxEr1H|u^)fmbnzGYKBbns{tqlbrj z0`Xm&Mj8mqIr;<(I7%$(*%R!k+{~#m;Ni7tZGoRGSW;$%@R<2r$14Syb3FXApt+Px zcf{Yjgw!@r)xAthA`lWIl4ph2fvB=>%%~>otjNoFZ@bH3lNAJDvtglP5=#hGJ)>qXGCk#8b$tSYe zT8`!f8Idv?h2bhvo4j6buof}1z*%^p`urQ3yLle%c$U zxY;V)x^}~amf)WaO-G?lvvVh)6OVBraYte9PouNfX<5n|4Rs z{=3Xypx-mKN^g zt)h$BX4JWPQ(0s(#y=w}TqOC&9kE9|a^d>6+rf&xk^DoA#U9+8b-PSb6>gGd2)lUG z{j^SeAp}yLIGU8cKhn$3aE-SHcT)BtB$inh*9?2o{b)$N6X_MM!=jKU4o+F^RCK-RLOfPT6tMN07c+TcW~ z!tk*drgj>ZQggV~$R@lp;2;gthlt!($h}K#MH`lI0;u)_R{Cz8G0y*JoZi03&?(t? z9|X(~a8_q;n*!zy)g+k-?89^XEgv;Uu+kh!#p>)B^##x6$^^;SOe8)YDxZ@39KYv6 zoslYScdzO;?|Ym^Bs<3k1P~3}FqI8#OtHlCCZq|K1e$g1*%YrG46uXb7`MQ888~|k-WL2|&Ai7Z_JVij0<2?{ zM(@A#(dbcfd9BlY1nVN9LS&2rUl&I6!xHm?IgW1OLpYoj7n%CFohooM`){rEO0r6i zrZfS;{LCT(R5&K&nhq`=7Yl&%4vh|FDQ058>Z`LZJ;kn~ysm*zONetV{Go0_?X;+X zk9Ng&K>0@+N7@}>a*1wkG>Z;HaS!{$c#ooV=O?X6HHVxTF|nC+C|DR|>i4}P&FB<0 z%Bp)2gF0qR6dk%dk$+Mso}+l5s?Zw3V#}DHxNJ+T-A`bW3GPX6g<%cc0n7*pRdYX4 z)UA9p?Irb9hK)0F&o&4rN^N96`k6nYto(3XLo0ivV@5K|vck0>u|x(Dsd%ASYKm8? z=2D}@%C5g_&WF2cDqX9b9^kO4-ek?v%==PQ&O%!P_xb`Kn2WI>UI$yN28#bJ&~03rQv3 z4HZO@x)9enz(tI6T6o7&)HRSoSw*}a9=)#3O-|+OuM*YYar%!+4mreq{)`p40Pc7B}|n30-FX{AKRJ(KChtsmV>{D#17) zkP(g_&d+M6Mr$KYO`N_*i-tYPpSF?FhyeLC@?rF6->H zPA}FTxrlgi+DdJc4kwGQe^rcNH0j%KFVp?=vSC=e==S^4Xd?1 zEB>@Ff@Lha4x&8tLbYUzvyJ~2)XD4az{6#7uDZ^fcc6b%$QhAH#GSPV;uNu#1=o*+ z&7Hq6f;Xd+`@a7JC#u9_&LL(0#{?~59woG7<8j7=;h#f6tCmwQb6Iy0#bq_)g}U)e zbuaiR^M1v)g^YXO++@xbl(|@35v4ci?$?*P3k;{H!1=KpH!-TP<50-Oekf&G;VlhA zEnnkYsmY9YEx9c2TrF>;wXxL}O>0Bn-4*Y`4?#BT{+>m?Ief-Ob;%~Jj#!se>)Zzj zcOG7ZzbnQ-f>|hs!{w|!+&Qce`uU?> z+)A^*JhRI|Q1qOv19Z zHL*Wo6*qQJq2H~ViVU$X_{9)kh%PsV6dy9=?D8|-J_Ukmb0@F2;;Sf@s+`5c`+R=E z0~Cdy-Y!wv*;$D3cV^?gzY9@4VW!`+NC*9J(MY>k9sPZQ$;=h>@*(D$z=i{vOnxT* zQmb6gEu@p2=6@zS5k1%T?5`eHt*`JyG~h0DiFWp7)sP%hpu#l(U=YEBjyPrT(Lkr; zl66}?ld0cSWjEI}wGCD)Z9E`!HH%*fJ+{ebd|JiKr%AP8tZ8d@XY7pZIo++Ka;i{IS}oaPO1WDU z{XgO=VD(I8+4fO255Hp|-Zy zn82_h+Sp?AZ{kNtcoco;hWKbnBdYS@cR`R9l(dIWM3MtI+!oLA#5c?N8~;)QWoo7E zZdBrEn!QwolbQd-(&7TMlK!pi5;AUGe;o=knrS*#*}_Kuo5qE@0T5+YAAo1#B>uC` zjhFn)IFU+@zE)A`gb`3MQo*ix^N0g%aS|ce!peX><<`QtwQ7`(mi!~JV&4%{n5v=t zxao0e5T`|5f+z?+iP}gq;LAzNeoUA9Qp!MRc!J`H{Mx%}@EYulO8|k@yRB;d8?~s! zg2Hyb`(Qi^72nL6p!aEK)iPWrQAA^(LEX&Jy}?8OGKoHZffBh`mVy(`?6tt%h$}ku%D&yNDrMJiz{%M0?xO)8{OzV2NXOss!HOb2rNM z(dcVO>+N-F*R~*4&=E$5e7BDnvnkTC?qT#ADOJhxg;NR=JfZ7dt=s6MT~xAA0j2 z>?~kxm$9!aTs3X`87K=O4zp>n++;YAXo}rWkRi9~P>5&k*bux-ohZ?uBgg@Z2pcEz z&B=|pMEqX?DuIffU~X1#1dP#};kyn?|CFurTKpW2@Y|4M!1YKmX_1O9ERgxoN4I6) z7aq>38YzySJB&80@wL0yOHxcqt_5p55R$ zLIRH;^dk!k6v|1+7{5-#w#Xair9#gu^DCeO6FlV0m9^K3xH)I{BhGSISa|p)qNx@* zD3q%vh=l4v8)i)JB5^Aofgq-F7^`HI&0Y&C@S6n-=zamAU$Kn{6Ap?yW27$FUx{-D z8CGTD-TYnmDbwzT&)vTueS302U8mF`mbO{GH;pT%tf>zVBxvW$Y!5JF(Y#Mlm4vFr zcZ>zbqFidxkBYX4D~pX5Jg zyCK6Mk=4EK>yMRx^wxN|d9AT%KLEP+L42E$j)z?c6R&*CV9EQ?c<=oPjX+$@YTVkM z-)Q$|VLT*s+egr1d=cZ;>gDRpDRC%n-@5psfb_I|^eg}Vt2(|n)DnCrbip9>+J@JBeWh{U_RY%s#g=~L!?K%v6C!LIk=>UPQS;&GcQ~i2`P4b!y z1l%JaI?%WEowtIJs>B)9gyDQfgXXJ8@<^J&-1!r-!QbT$n&Vlf8sFMvSQrL7=>;=q zn=t6j+su?RON?NwfFE{DoXGtx=xH?doOG+KmDOR^CcTZx*%6F`sNP6wfhXnu$RfCd zTB3g?4PZ$#QG5ptDSjnKnW$UqTBc2~Vj!PD#|OL<`a1koMSkt$fjT7BOA?f)UhTN0 zGQk0V7^(LCwMOMcMK0G0*kUJKmSn(^CuC0l6f(||Ako59 z_KBa8hKyu-6R9bukK=b;k1rA5OF2@}S*$f@kU8fKLemlemqy7bL^v;sfsC-NjPO22 zI+g+?2fQF>3-5Re&C{nwj9vj7&Xk~156e92; zXg_I+KSkAxyEKR?Z3N~PZ}c>A_Q+>?BuSL9o84a|h!{wGL4mUR_j z9{PoMax&u!arJ^KQg5;|QnrN`?~eA{ySfziVW4CzoPhmzd?eklKSLR`j7NhfjOwr6 z9bwe+IkA!?Rg!`7Wy)!4>2(I6W4$oFWN1_4=@X^U;bG;{lZOWn0Dx{FLw2T^4vP>6 z)r})f2H7#JRE-^P6|a~o!LSh`K2n1^ll zy(rfF(R;<2-tiUX<&iXYzDkey=WgSc1x5)nxXB0agw6rFIP!+4R;~xNiC@WsRT)Eh zj|89ON~Sc;es6WoH9q2`tJ77e@0?*Ww$3GpXA>}7jea^2YRt#YPf*rp8vDckF8lUB z;_fMDzqw>GqZXWvf1s`t6_l=)ZUM5$<#7>sb^|>MTDSTU&c6S03Y=QImLMK4e=_r?sZuLC#HF#@83VNiA8ZWchcvj5k1`)b^1=&dIJX8{;?&mG zU+FDPgZ9wv2YpauX|g~93dw?4j-OpSLwB!)v(nbebTs|DYM7$ zkHmP*j<@LHM^#ZQrZ)7M&LtDYu+>(mLHYP0=p95ipd`bK)7nzg$V(UEkA5ygNr`u2 zhM>NgrPbtfk@CXtdyf86tQA!l7ica2wgTxx2-<1%R-n~P+4o_zQ7L>yjXezgxh`cp z8VN(fG~I3&=nBq{eiRE`jy5w+dLwtn>A=xtsI53aAr+Au-<*pI#1G!zwele^0Iyzq z1twRZv?mng_k_E&^7Sk0bAydFkaG;LqC1nN6pY}TZM`mt}hp^ z%nH?4DNdCqN6%4OttWW#_h9^=;_KGB?yeWLP^kWpOG+I3Df-p7tDW8}I9yCj$?zRj zvahg3cB9o6Rc^x#>|~R~TCAa9k~*XI+DOV%AP{cpR+TtK{SkSb37^BlZj!l%=*0C^RQRJ66ZGuA7Y$FfiK!yGL) zKYLmDzZvUew^uA#8HbL<<7W%xNk!E?Zw?B|mJhHs$ zieS+p(L8_50F~iq*by^IitD2yk9X|4#o*czGG}b)<|;y%Er^gvUaP(YSiEAp{nP4Y zUWA^(H-Q!q44|^nO4mIpvP3!gD!*W;2@gUOa@;i0Z!n#t9`hgwqH4 z^$k5(I5Hm`tnytz_XB%Dqu?+*v{nZ}49lwhBI1{=9yvh!v{q+81b1{vm+GJEo_x$d zxT5m-D6swbzLK_&WG!P+6r1KS5>LiiY-_|t&a1Ob0px8!tcOvbAqbC%Mnf4}e;LHJ z%G5a*uHKSsMewW4g_!N=T2DOX0gDNN+%jTZ^N?*(i1Ol|(K86N$G=AsKvg2Q1xQ*4-d#1{ZF$d|<9^#w&_juL< zZGD^a0h!HB{Baw53twf}DW^zHNku=J$hKo?3?65FCNkxXRr%9zLSPA+J+S5mX>UBN z^!TnFgiW?4G~e{(i!D@FR(7CbVNMc)LzQlK@^2PIgb17fY`<{t>#Vsp&io%G(#MkP&n6wxH~ct99F{gl5sC_cS-M1%B4x`VApCRcFIvZp)v{uj)kM$ z06YBjfD8j0TB$VP0fV?$Y;T$1@_?C(=|Zr$|8nLN{PSQ@IazRm1rvxWDbFoN=JQTE z90;Ukcr!aY%DerAF<8uNXsiOhk(lt0#DnWA9`wy~7OB6a6X~6>JjrxxsP7}(n49k- zW2NB|D&bG^^kzl`?=O;hDLJEo{dPBg=!=aMoSmv^6bKC7K*1~x<9KNri@C(|m(-hH z5(-K~SEMcgvi%FmN9#7GhRF+q9-d5}s67ugME~gSwq4S-Q@dh8AU-z;=6!7AF`t>P z-b+GctcwnUnON#l{2?UCfeq7)&=vE-?ir&%itg+qc2u%V!45{R!YzI}x@KkRu`CZP zKFWJP>Erk(`aO7U*jAy9hn*onx|+*|V`4xoin`!N^$74)u&0ZI$W3z!DGL1)xi?_x zzljq$Jm^FS+^<3SR9SMo#4;`R&J^fqfWJ|hH_<3DM=uMm+y2+OhMF;)^F<{iT|`%FQ7?a zTgxqa)C5wPf6L=n<`;n`$jRJ-geH$1oi%t8S3<|MsJkzSUn;cxMR(oXb>L?aZgMfX zrX_LpED(k`ll?wo8ZwRxJgxV#-8^&M@iQbYJ(5ws)|R#FFNdKPtuLcQA0b5JoD*x`++&`th1fzQ**40;y!7)%Lp?2F=JK9u37y*d>nsytD=lY)3QT$ z;GN+v8OlZbt2&~{eBPAPNHKZcRpFzJm|P7TiLJj;hI(_b8c)CX)!)Pxo`U2i{j;_@ zw|UgWeFF4=r|AaUf6L{*q{hI#Mf(@>Y}T7zEPOtEP00ZHEz2D4zY4#N3WnEy@ID3p z$TpH)TBncXE{FKOJpHs5_^@K6^u&ELSNJKiaM_VHmTf5Lqo!X)j+fUp$ml}v@K#49 zXpc2&M9wMaiw+Y%xdx8=Z$+j<64~#eP@bm}K`&{I)dKfH3C@jwdg%ejp9ww(;?P>; z!%Hp0iPKx%4>)2{xJ2Z_af=aV)Mr=riiwMnJsA&(*XM-q9Q zUH!cIR-|Lc^^YIkD!JOkAz{?c+%UkF*08v-iuN226d!zZ{)w$9A={BJb1)a`7@2IA zHyepu^-i$gKe4qfwJ4`i!6#vOV%&u9MJQh&3(qTEOqv0ZpB?I_LIb}BGcu)9rO&0% zUt_S-N!}w^17^?NBeM{lFfH~5U%Q=!i%^MAm67sfq9G4`Kk*g@u3s!XNt{1R%Vy#d zx8f-Dy!ECyHNb+ZMd%v3_0ILZI=$y{#e9%iNfd3)=U<@~17F#ztnVxTk$iMQQp~qgpn$dRIXI1 zD1aJgmTE1=s=~}_dRS4(J-hz_Ar`fPSs>nWu`_gs?T*RviGvr^F(VqbTEA4?Qzxj)33vIYsxJa^w5(~o?F9qWcPM0j>%y8 zA3isA&van7i=?H)=P0JX2(vK1)u})J^nNecZM;xsL2^dkX+(WpwrzBErF3lcJuk1e z<`+84(%5IEU&uRri;4{kz0@3jFKP>qIHu({9zRx05oLKHw0sq(s6X##S@$jS=!Bg~ zJ?d^R^?Amvi=Cp#%x=3f>V`*}TY%nx$Wr0T9IeG{v2L+N+w9O33GhHAhmBt$B#%j$ z{prWObprAGW09E#8C%J^AMq%IEsAy|QQFx(daK%VrEy;CrvE>Zq|NKy{<0bL%RoWa zwx_09>j@R%qp&`$A9^#O*kAiq11ja6$Zva}o;#ewC=DTbLS49aPw+`3CGLiWX;rlJ z#YX;t`G|Z0j5k4^77-l$CpDpd)m$j(qDT* zbgF+DpSn3ahrd_PPr*$5BOzH!FJjUdX-@3F2x=aW;QB}M#MA-LdwziX^3|i}QuO3! zX7=fyKXWZV&QdZiBh{jAG>jV$5(DI`59RK@swT=Fb?jmvX&TQ1y$wxfr#8-E5q@?s zr%OiJOe4rbxKF_HTmZ4^(7g3Btp3BF{2y~MThEl=q>F9+TC}aYU;xK2@-t364&!LK z&=eSl%!a$akIYgweL7?VKG%q3uDM&i73nzN1EA4iQxU3}Sw7Et%fZ+7xg|(f;MteG zk>=y~gI=6+AD(Ts#f;psH$oRUQGNw&vlP?9 zcDmP{rFSL0ojsKXHWnQl7a`Ip<&@U`zk`Fs(!!;!`5(VOC-~V~HNEU`cAq$OJTO27 z?*2JlM?#RYo!)+PeSh^^ zQ;u(YKr8MnFg0HRiF@hD=y@mRYt#kN{B^fGPg;3ntk}<~ni^%fDa{mc*P;*abOQ0$_(O*|6y!4dL~GZpoW&~TRW7$B@S_NOngRF0Wq%tF7>c`k)9W?-x* zGnv-?Gqp5h!T@A&4XM*lK7@WTu;4q9xaN11j%*z{;&4KRrQJ>u%I|A7dns;N@Qky? zoima*w9KcxL&Vf1KlG5$ z)DZOkKavBEeY5|=#=M|W9K=4#hZU?ls+f&_Y(muyQYP9;(sZ~I#Ok4#919N?zIb7{YcBespt%kJ%&=gy7UuT8j`64%e{YAIv@DLBD zPrB${MSV?=3-u~%E%Tn^|N?kT?ec7f*2@r<(JTq_e z>UJ%0?wUXmBSIwfpxMNw;n30NY>Lb4N{AQJqf3Ny%z~!g_axNJ;0;Eva0B-2v>ZDK zQv^VR)b&66vV{Pda2JkFuSn^L?Mk@SMkInH_+Dn3W@o4btLqpADRNssMRa-la8Gs&#M z>2DuUum-BU1;#I9Y){RaAQq{5q!@PIznl#Cxw?yuIRF>>8EC>U!LsXPREK7ydHCZQ zVqW-TocCK9Zm+)-(FXhiH;g?OYrcfW$lO1(d7~o-X`MKg@m4|aUc>jF1D(FUPAd4} zG+V|h@=WaW{_9`mzOmxUpW1@2o|Lt2NL@~4w6+no@-IT=@4Xpz$nE`xwf+w2?EG!G zd=l4o7!PIf^Ni`(s+0d{c8YNhw&`WGs?Xkax_rSW@;az&wKCIfX#fRj4#1%v!ym4&HjYEcUUrVpZl39Yod|krX#if zEPDD}&w783ai^WiJ^XB~qjRWX-qhdtlkFhPny>4sinW9gBziVA*BDC=m zSp@H7iuV?l_V<+)d2$@CZ=F=?BnR~&@uJx6{r(+~xh@oIOQY5ppKG zVk|p|E_6`0zLbG8=`S{#p{rE~|B(#mzGGowG(d4K@S>YgV3F!tXCrBjz<(svxc)(@ zeh+=h7_%Bh{RB6x@>Z>E{+i6ziA_OrK+L*9qvhdBs%r(-B0g9{8NpGO3`%A zaOn)A_8CX5ZpZBUp_&7Z6}^*IE)_F%=cgQ&X4$A$f&Ne1E{Dz$WNl~(?eQy|V$j~> z+xcIRz4o5-&z5;2mkFL^Nm`oiUE(`_77lMS=;v66DDGw|Nj;Liv-GIq^HaAL0DD`e zdEMVh6Ku@qC%ZlKPe@nPMg2`;B|Ff@3+V2s*E>lnGcK~NkyduH36gEwiQ=uV?ABtI zX(6Ajy8@is%WZG1(C{5z&ZaK&5zZEs{ANaSvH`upYnx1L4J_=LEW3m9*VHLKd_3l_eq}LO*_GHDfzF?j$7xNT2OM~OJvjv za&);TK$>YuqXmrxSeAhF4V46LR!@!mQ$Yrw;-TdC`tPoE?O+@EJ}QR^Qdn0x8p^-V zr)qtC*k^oJWVe$;XsIRa6A0dj7aS7M4K9x`)NK&sa#C~kcN<_Aw- zgt75Gpl^Ym1QS0=(@7Zu*vxt(hY<0bM?TEF6ZdFS7FxSBpXK{h8v}F(OWG1o||*Gx-4>@BSo@%vd$&>_}8s0%m;6KfR_hTr@OjIb`APb)V<)grGF%t z^xA!_rZ-yHcf{(b>zwVy;X;r60CW46z|PHoB=$;^zlm_4y5&90$s*6tl$@jqTIR!+ z;)Z6Zz!eqRG(=N(T;aJcZX5!gJ?ZBsx46`6F{?Q9iR;}EVTqu>MDz6KySz&IY5IJL z@gIpKiT6BbrC>i%b;q&1r}hs)JD|_BF%6L4f+NV+&i#m2&*@4V_oQkXQp|YEl=YHx z5;;Z6_)sJ>>*?A+JdbXgmBHTsXlPT+gMKa%>=ot&c^|41sRTMUsMz#JPake?zF;gi+KgTioM=|Fm(W}|sj{;>JXJnS8XaBY2I1~Ou_K)O_ z{2!hFNH$M&dv&6dzLSWSD>7W(?wu8QsbwiEBk6mmO7c2L0e6=S@J|?7e5^(T-p=hr zb-GKG7z-BQ53V(Ovp6DrTW22X^sH?rYlUdCb_>2LjhXlf=Fo5aLHyw8@3UpRy&%z& z+V_c9VtD$&Qi9oL~q%vQ{J;R`^0lIiX5I=apn;j25?{aa6F2K zo>LXfJx7+(Pcv6m!2x5{<6(TRyI&<~lE}j%XROeuRrcSftC9>dc7U+Uc9ec7H3@aD zNXa$TCg%~fcxrYok!g@q_@o944X&%+t#f6c)cmH8vI>%V67$f%B^LAT_ozbQSp9Wy z&81^vJ+Gj5k>S+fNNYn2dri-!>UX{nh%<|s+ljfdSi80tkAYpyT~hxenY-Y7JZf^r zG~^tBUj1$H#p0{QidoyQg4MrD=Y&!s%Q2J9`L(U})8?~-Dyl1> zK%OVYOlqx;oYTepMEoN`Xg${Q(0VKw^Qp-Bb6U-xZ|B9A=|SfmrGEqOUOu?Gn{lm8 zb-p~kTGjH?TKx9wdUEm8ddzWtdLH_ZWO*d!*UakB)z9d!*PZ_(=`HOoZU4XWzvurw zzq>wqGcPaxr*vO-|KHhv_5W)BfBk=7``>FX?q6N}Be^-g`uTr*=)P?EpR4OH>n|rS zb1&;-etG;7->dvb;-lqr{UuyE&Lz)Pd%p}_P0RoN{r8~i^7Q`Sn!Ru5N5Ns(`WqKBBiz4DE?tQo?@v=X7}z1mS6w72d>tW(&1dmk zFg$yyo4+5`btwL4njWCFC1l^~WHlhh*ZJPL?*w!o4$gk}m)hWvY(7(3yTqb`O3-k| zV`cS=$F3xQ;ht~Tz6`c?-*27f>u8I!D8p1&QcGT-z2%E%Syf}|iF?xMiEf5lR2;wK zESN{yDa#4Z-VGTMM2yM`PcIoIjGRP_F5$T^MXG*^N1J>}b=q+9t2WeOCXWR<4IDqr z?MdiPB#Nh)EEZZHvmC-X8ha9n@~#npQzd}OSb@&psNgC|LlfpLn1pq-j@$K{VzjGA zb_=DsJuj~?CdPla^SZn3bn;lGTBH;bucy%nBZH^GMxlD1ru9v*8fvHhHMAz@x8nP8 zh2MLdRIHmnDJQAa?LpS!<%Z@Dl~bjRjyzWlL=lIRX}ONo8u9NXihTaG5qA73Dp;$R zlr`{JJf4v8f~~uYwXUU;BKTZ73?$0(q46DI`tC7zdwJy{o6&(+uUvOtQn*#{Wdpp9 z^v0RtBad?EOPnL<_*$P8;4M@Ung5Fgw=YBID|6EylA#MEBf8QiO6+$)kI085gQCl< zlkJcG8-Le>up-8mLlozO9(Pcej4o7k&)qt#_;pifh|UwRSM@E*<$YF%tVlkJ5Toq7 zWO7>k#`W&$^pR$Y75mrx2`ILkzF>tn`PW1O^>_ZBzFqTcZR5rO;^*c& zvS8YfVx-5=X|n6k36Oj_4~DRiwrFogEW_Ker$@*T)1~>=6W(Ow6(0M~hU-Yj-qPnu z`{5_5=y|E)^KvTJhMNNC%bf;q6mt?vr{AzhR4nq!UJsfXVa)i~tvWF+dG4!$e!rAe z;XAQAX>!udYpsP(`pnd&B5nwnzo;mEOt{H~Ghk87*3fz%&bDC-KKjZnEpzEuNJZsF z+ANipzGWf*Rd%x2IK32aaEwKt|0K4RiZSoMnxqUhE*W zw~k-Aey_jaxbJ^8@IQ;X3j)xgJfI5hI6lQ(bhLNqSUfTX8htV{R)UKRZu)2CYLR#- zG&QpRb*$;%z))9P+Fe$2wx!d56(V3RJq6=%lw*+C^*PXom9v|LlQU)|e_YKt;8%Ki zqc8u4s)SbPjPv2kP62=aW%bZ7^NOazsiffmhDhX}-H`T||Bl??8ZMqaF4=2$R~%n3 zlP0F1yz|Vj^41d~F}tT2e2RGWPJ|^pIz&araw6>hitnBmft6&^k=w+hq=wLXt^}+L zbwwL-#k9%kSl{*@!2B;)4z>)1K&21dRXslF=dU*Y!k0?5Wt_dM_uLfnA%)elX>t9) zb0n#tqx(g;BRENk*o9m5hy`O!&qVZ&8F)-z_%xCCtj^&Hsh=`?-1b(r0yUbo?h-{>^*+|_E%nm{{0>Y?>dCYJJpquUhUvM%3`Qw3gEx%S8_k}=Exty0EK7G z{DFA&zr_AOMSbc&;RE{#{3R>+h{rj`75^t^KeR&*TgaE_eE}E$>woUv%0zA3KYAZWW7|QIYe7a5 zPQFb_@2qua^-uS=uh3HER=4h0GkysrMQo!+eYX(1OGiOzdFdOe*Ts!vJgLoHw^d7) zGW_zbfiVCj_U#ec3k(@3BqTjxgl{t)$pQ~Z&r?8Q2jF)Tfaty zIBbR6&&uzm`|S2gC!T&Hn&+(Ts=2o!J479SaX)R3x5j0QzZ7W9dnr*@-BVw+6gn*2 zZ!49S0PoTN!f+uo7pb;dIA?G1YmUJ)T+>FOu8h_s4ee)J&7Q}79QOu7 zF@8yM>x*vYyB6xZBAK=dH?W@&g?-#cO|2c?3Unc8|xGCBE3 zN1tP&d6XL^e?&M=E7O}UWev&~@UH}!{Sx}T0|)hOXO+5}+zlfoDSmiy=B0XyeeVsL zTjTvbRhNv3$k@aK`l#iBI6TAQYi~zdd(V9x+zv%q&+#$wEbduw=4P4KfBd&N#domPro;1y>=}R2}-d! zJa!O&buFdB?FMBo%60D_VTB#>A_IHA9dlWc*SQgpcWJ#|@A=S>1$Lj0743us`JlXu zPZPR|vUgeC73}%7Tk_PhRg`a{ONX>BV(e zyMx_}&Zfi%9-G%)CQk&YzPW&~dmeLQPgNYGK4Q0j&Pc@XA^00INc+zaW5zAbVc)dG zsmG!ji#}h!UDU{o@|DMD=5Tg-b9`rrW{cYC_KM z%v%CKF9ohf+HBMwvjGN#8UEg!wo1|dd>ij2dWa=rO6Ro=$ z27nUhf6M9Ro5zn*_Emg%hWO_a<>LM_op1`8^?gqb)*~c}$12rH-`8!|y_-~08s<81 z_VdvT7wQAoEb5nHmjCnEgnDAt@{!h+q%#cduCrcT>3R5xdo2X+SSBy|GA(WH zsbS5=P-pijL0tcy!pEixjSn=C+Q?9FB;`1fW9Vx{PoXk*nru+E1wZVqkLyjIKX>Q# zq?poDmU&m`{ON)4Z^!nfCAqdxv(7y2Z$2JFv!f7f+=CLTKk0cv@s^=~ikr7*oWAh+ z{3}C@f5dj4aPGEdgbTTCGwZoyd`}#S^e+I7c@t7Byt~t%tk99U(Dv1qe?P0VTx1V) z{5B{LeXRQ$e*xU`TK`4{{^{He&)U(#IIALj)!T~-Kv2x$RT#dbPA_Ss^j1Mtgg^9BwjHU%Xr4KI?*n(@@NC zFjesL-Z1Z-6w9IyHggdoBSU;vTcV~gRBJYj^vZg>lb9E=YjNVZ=@lo(sVn>L0?M=~ zDrWx3&9+6Fh!p!65%D=)^;zecSLS~*%uD*Oq~-%nXM8w1Fk0IoWfgt#a&+2Cp6KV1 ztv(~8g$Lf5gMW1h==dp-L%XhSfibwJXAU3mJ$&Ee_PNGXdELi<7+D^^i$A>D_=u(9 zTT&x9CmsC6vmj5lXsv);N9%>h{qV+h6X7Bvo7x>6QYZg7?WGkzcU5>g#q_I&a%hiF z7N*qkCgW!0pN?mtA-c_zjSz)zT0WbO#3w|gdcCZ}r;O)ErqVyP7Vt^rdm18ce~~)J zm|G1v^M-AwR?XEP52jxl`_-O}mJig<;~GuYqcx0Dv6!wY_4hW)&b{Ai&#~>&8-=gA zHGVfuuVxW*3sM-O-5Q?$wMH2R}Olqf#o9URVVmIQ*}s{UO5^#6a3V*{`DTtikVAqB=ZBSEuN{ z63GE&-jLvI`FbK(s z8edBue^I&U#TwQ(YHT=xY5O_!{I8E#lfa5H*;wsEg~*I6mj>JWVU`HqkQ(gx*g)M; z4bJGd2hP#&ldHW~LJD}yG$q{Nai`w%LJNGYWI<1RKWO!u8Mu0h?p`rpbYj@_qQ{Rt z_R8M+NTr5I)0P#yVPryP_x+u5ySiHnMUZo4j}om%?;Nbvu6pdJX1a>=*Q>Xv9!TWlYdR{z`oZ`D zuOwBK+nc$QKS~3&jwqV+Qup;Pzn-_u;qOYk_O$Xv!g4$|8* zr_8`g>;7f^)rTXOpV0CI;#F(V%um-xUe&+bsPkDxd(ihDKNnrSE*^nke!lWfOIZAk zw1f#aa)MyIlabrf^E9atgLn6;70VrM#BH#D9M!u=Ju=;4TX4j^yn=&ieEZLfWs$W7A#plIe>BYg%1yB{fH~+b#<3{xBd3>OMr8L_4@*{e{CotmB z+~@UudxDEUsI$8R%{`jVxv1%4TUXFuQBQ9o_Xt8giXTTD{Jd6adizbpJC{p|&hsxi zbH|!DwPOy*KUTYT)(E?@2Z33Sj{63 zFhPPZdv|+;!w3iVrJr`LJKKRDxOHG0?{ENlrEQUE;ovZZpQjWLiStj~_|ZP|_O34z zeFyuI&FrED9v99Z`nwQ~k04!nDwC?;t}bk&p4s!~kEE80#%ET3cTP5cy?^p+{<)NM zHP*Q4{p3IWOwHqgQty9J-}* zf;Z2dwY}PSV#~%JI3T{$9yY6l|&3l!@+`z4cwBjhbd#O54ux zmNieQntk@Lp!t9xVef}QEUO6nOUUAb8MjLHAl!obweTZpsHOZynNzQf-7g`tip&zk zBk4nvRP}pQ)&~cb!P$oiIQ<;%w^b>W&1}-p1htPIgAflp#cx0Mb`OkxaB$>U%=bA} z#0j1X7(v=+mVM*`MWJ*+Q|nUt`G;fthx4)5*YB6B2t5-EcF%QO-A%nTJOgva6Wj|# zbQ82Up6G6mBPN>7msa+BDCEov6i@mGAi|u7OR(NSqN2(7nT7LX3i*3mkE#gWl)BQK z{>W;=I!rZ2gCgbGckaPW?yaGWCKRXG^XmFNd5lpOy}A2IXbjCa-PuXmqKb9>?3k?; zyRUFZc;GLsw3qJ$X&akK$B&#(zLT-3%J^|7SnL)?e$evuKLH>@)iq~|+^ph1!BaY? zitY#Md~;ZTy|Z3%38v!5nkolIv8Q~3jt;0?vdO0my-dm7)$bm-Un%=>5v2EMXyay; zqf~vYL(Gq}RYz}}ozTRcy!+hjr-Oz7F@9N;yWPBDqx?ct)$OT>XK>0w_&*WZ+3!as zhjr64zicqN5qS%By6ReJ=RXU<4=f{WyRk73tK^S34#y4M>)h-180o41d;ydoQ5GTC z>Dc;UI=t;f%vW2N9$l=znHuGC%~t-8%$kK1&&<0A+qQeeL$7?HZk5|gi7Joi1 zIbZrx)#%s<-IXCk@{yX^L+6~3*S<-8jR{t)lFUJ_O=%lvlx^JIS%Z`r7u49O|E)f= zBPU@{Xk%UJCYG@JKS6+5d-7`?<)Dm+!mFoQW82N;YkYn6;cuW3kWQlT7-K~9t6T-GT!JzqJTzAs8lR}~PY1rQKeq&@ zCg1oly-@LE-n4GW1hMo~(Mks7Ytj=zW1`pUQ`Eu77BVyLB?v*Q*|qv^UeKGXD6S%$&Dd5nV9xNa!PEn;8U2iRCaZEU&rxJFkSVx zrX0<02-g!qs)wZ!v7o*6?WgQdQR>RWjjQlpKSF;Abz1A!SaC87{N=?|TDvh}A$oA*+bN>HY{%P!|#Wu8*M z($BQ~c{T*~U^H|Bg0%*>a`l z@=5&>s#DA@$%bbtel)obnVG2F)2fmtY$oXo$kGJg$9){D;;WBBJwoq4i_qkl_{*Q;ETY77{%czh}W7GfoRmH`^X`(csNj2$TXneBenWKp*V@fXvC8Gqs;Gm~E;%;p zk8RXqxzq?m~VIHvUcAXC0CE7vz4nShP5-J`B5Xi#&39` zT1a-zjV|~pe%vpiKaZVVr;&odfKxa2#5ICV`RB0xia#Evi~nV$g&&fp38Re9%?(W? z-F00ryk4q9_gJc1A`#w*ewBXlx^T)@lTX$D12y=$WuOF|6nN!`aE z!ZB@R+*=XCpd0m%wSz~z8pE&2+zZvWuzM_-xiSpJ+gb{DWu5&16LcM}E|>dGI4?Sn zsfyl{@(^|&`=H9Y@PKg*;p9i|%H%^o=k1f1T_OaZ9B*6~6WXi5L(i($dsq1H+gAX7 z`@luxGW^?J4=a`9c7`8E4Ua%CJ$ZFtERpi=sBX{8z|4GM1x=l+r~a@}y&FxuX7x(H z<{~X{w>)M;QAi)5^Zw=alVQu8YM(!ng}i%2uj){`o{xSn);z*surh*rQGdrVQ6=8b z$k&cD@5syqu4Ocgy5kQ!0WKOgVnH)xWMDW-)BL_LQV&c0gB7ay)2;q#q`!3F`=7Rd zGSQUU#S7v-f~&Z3f~C6fiM54z=Ly`A>;B(4x6khdi_IPQba5$r!yJdAJ*}7Um9`lS zirlc#2p;iS{7Z;=A6qAzdiAlci}OK`!x#Cn_nx?A91eYR^@qQZ>dK7f(tUFL=w#RS z2Gy)zy?pPf-S!}p4f+KS^-Mbw(T<{jL9rw!aiDuJknnGJT43I1Wjl2lkCQi(4m@4g zno-9eCG;-wwruo!cC>7;2b7Mp9L(@#Y`Xm5`l>_{=E-P{Zz@sQh;;ORin&WW1!uLu41@6kVel)Cq(c1Ki@?PYFp=Lcezqp`jfkuwknbbhH?Tz~Ai2tJ8wNNIGuF*658D#6fGmcG>mTZH2mG=+ zlx*ebGiHjJ{|;iSCd!VBi|)I3?lF6ZVf{>+rwJ%LqyuE=U3=*#gKx< zA6s$qa{O}wpN}!28fNG*4uuy|IU-2~5*~?E>$f9Q;rnCfiga;%>Cok7UkX!um4z<; zaed-t2q6UhHqLH+lBda(9Zt8_y=1hAI6r>Fu^nB)D0v^G|H)~vgxpBK`HJMIc5^jo zj920gB$3FR)P;K-)k;&)rVV(e!w=wru^_VEN*0fVJA9j@)E&3Y>52ym1Sl4KfERjf zjBNht2G|+veQ;QimuF( z^RZpX=t$|G>iT(hxPJU!8#RyKM;#Q%nU?0JU`KRr%OrfiGzO?CWVA7G7i~&Enr-7r zX{q(Cmml3kZO1wCpek8LKpUx(For;U5?YD#QDO_505D*=QFAM;Qp#RYu$0j)m~M7G z#+dl?myisv`{*VUSDNiXfXjfv;9d1)(z2>5W|_x6JY+af zpDa|Ic8fM+DF-`o5TNntcv7QqmdgFm_Jq((O-+|e0juFYMf+n+tcSF^+r;8+Zk0y_ z_v!KST;GmaMkDf^6U#TCGCyp|N1_`rWb}@KzBA zwQafMudZ;TmuM;U@Q(Fxt0nznCy?qKCoP zyi)bOd}Bk*LJ8#f{=kGBW4$7o?g92DxRDXfKH49*i6mN8(p@T_M>w=rhF#dayaV5H zia5+o$3eqiA)A$K(lX97Jt@6(F?AJ_gIB!hRwlAeex!=S^$a;x!!-YZycr3;VWxC&-|? zlEfoA+1qMCCg?q3Zw|?8QNj zy2x`5mo#-R)2_p86*p(nDpWn$KJQl^al+5*mmbqzadxeIKC^Bp28|MjuFG1R$->!q zV<)aVAPW;~0Xns<7R;??UWDqmv}Sg1qKe*!Mhx5-;;vv1XX5Er-;Z?$5-c)t*pzZAQJAeYdjOWf!>A@z|q!m z^un%PR`w35(8d;}rQ8Ve->7r%7L3{2@R1_6b?DL=pYqw$PEMCz(>si-ON2%BTO-~j zPKkSCsop8#HoIz2uPrjVT?7>+WolDmXW2uLh3~BuRW8j&mlz{;1{e+T`Yeb<2y#!Z zYeL%1d3yE4GDRLcXMC1V<#7mbk1!5=mZ=h|v>_ChDIo89$bwpv1)xPqEXA4GMlqxx zw@qH*Jw3%33WP-&wF*UN%$nw}dm!h-lEpB^P}Kr*00oIWgS@ zdLVklrMeZy27^m}%e;&$M4fK^;h5zbH{iVJvV>y7CNIk#3Z0qbZ7wCM72O<*4-uGtq?elsa_Ml9i$%V-Ms` z42f2e`hwgXMK^3>-Njq~>-Wc5A2p4l#a@H>@Q=i}e&n;PGS3np1>m_cM{c(l*_bf8 zO}|O(IE=`zc8xmwJ)ep#ilc*Pq?`Q3p;3kkkgEhZOMjW!a5Ck+k&P z?FSP|HWWag3#0<^mW27^hX-3Ia;dt@W15C*;f($SqdUltyVgAmbu*yg8Jz86a6Qq` zClXqEh{HzN84H;oAU$`FEncJf`YX5z#_;e^9ZCw^WdKTUCb#Qwa+&BjJNBsGq`A^{ zQc2hvI$KLbDYGfZ5SmF!64X#~byW>pDlpp&vDqEGO8?ZXfF`b@YL!gmpC(~XiHzJJ zfw|dS#Pg)ltyJ*g(Q_DZr{QVl7{My|btSTUIkgmGZIE1g=WDKorCy&tgLaH>49FseEWPV( zJP{-F&~9ysv2TcgR8j=t!pkP;b8h;%{^nmfIiYHUtqDwSd|DYSA zzD~z1(|t)EgQfJlKaQqw7nw~ihuuLzF8NS$3$(PgG+s*n5lt((ARkLSN;lR+%|QWu zMcQx0#A|bg^bUcyBbplVboi*F67+X#DS>vneNK1?3k96pM$PujDmSzvLW0wZ z#8REi7uQj)>~s3qWWyk7Z2UWR3Qtk8GGsVa0aYQt&Y>_-&OT?TDHH|QI63x30u#-# zL^XBZWit;lwnVkM*Hm35O6#J>9YFfxFvfw6mN=#t(~S;=&SLf@w=LH-)zhZt!)STa zGklz0_&2ZX8~nMZA>WniG{^YZ^}1x#Bz%})uH&B+L_S?oQ>lv5QxY0*Hss{gifiIp zIABE+weq^OCJ;z_(nkH_ay=k^#+0er06*O3;GxlKnzr2nyaWy43Ac)A+Buw@^}hTd z38dG69-U>|9R$}!SHc2PIHt^yboCv0!;UR%5nj*QuP;V%LdV^77w)}-qR^q{aV!YX zRy?E+g25;<6O|OIB@(^a$$G;Io|xX`Dpin-K{PR}+W11M zw}iDE*~E_=dKsyPe7&j&t=Wo6Ul;2%DWG)hZ!LfYOatRVF3BaNno2Iif)?yI!fACYoXNVoBL2v;5X}U)^hR2@<@@i&$3P7 z4lc+<3VoKpxM|RuZjGRY7ig7y4Mj5J--96vF6!qIWl6K9K^X9;rY)2!HMK9l^xn8K zG51o4x{92Y=|?i%thwH5*{9W#&N~IF5?V&+E{Q~=D2&atyEv6%zihqu9L}0bTuE90 zz2n#x30-+yqBo2w@uCgnJz@+#l^fV^Sv`g(bPj|D7&dlG`$mDK5UAo}^1$pME9cI^ zMmnHsX{02FQnSIvC@cG|>?rW+Hqb}K)icA@3`+=rMv01Y#h_#$y;>>G!CXsCi2)_A zSqt;rXD#eORLil*wDOR3iCm@pda{&+d2O9fS*omg&DoPta%52pNkECtOvLV(M`N++ zrnjTd0|W}p#65~GT6^t?h-*P~L!}jn;#TQfl1qrJ)gz8EM?raJoQBitHZk}Yp$ix9 zU{m`LG{vLPfwT+eV1|p9Py)oFR@BfCY+q`qQ!{EL*38pe-Clirf!tzPZv?x+hn0mfrJLy3%rOc!Eu&_X0N;-} z%8*f1iyz1IR~ow5_PUg>;wp(*3QW8O&t7dx08K_V3I9)(xGX$)V5gA3+_9X@A zqwo=MD5KkWlQxBTk0~nWR%#wnHgH!$bzbJTH79j;_J^*b;|tQGPG8(4O42ygo^o_C z*RQGk{?A+nf8wRM^2*_umz!|pHTEDDThn}kEYs5XkG*85*)X}wOXxE03)Sr6Wq zv>HIhQU;yTrO1PHXmos9eP&DGEF^C}jMEeoMv^Iq1(noIfrwBr#K@q2d1@-Hxz@oq z&>iHP;j<87BaJeG8_jE>)WZq7Yx9BvJiQu}##SwIK#9qSuk-kh3yYx<3w$nD3Oe)Hw5) z6YtX%tj8E0~^VzrVQw2aM8y9p_y%Q;sniu&TnRY%^^u5l1^K430hN(q`KY|1>vgv z=Zv0Y9up`PYM44U8@oWP0y%Z@?GTADuCc>1BE5aEv^0GV)8b2SxK#W zYGj|(s{0T~zNs;cQH@htHrzB(a_{oWXjl9c#|voEXQ;v$qD!rT%b);Rb0mV>Y#9Td z^*XhNbqFpJKi>z$T;M^5>zrD6LpOy{rjD_09+NB)7O<1hPZQ7tM~@2Im@iMieEC9J z1Ncg_`J(Aa(pZYU-rhfG#Cy?8}jEXO{PCmpXQotKJg0ubCs zleB6Q1h5!eBi<-xyxyFx7R%>A%=a79<8iX^M#ME=ee-m<2H^QZf}JGJOI;OY5ZsK# zwYEj-D;#8-6XNXjp-${?N{fR77z>~DI*A4=Eh<05^a4qI{WQXlZ8YhNAf|IU;IU*C z`O~d@!T+YRS>&7;)XY&j^3^>teXN z0wg0P-9sAQl$7=MNXmST-T{K;Z$_<^C!xQP+1C~)&%;>O5RjznW>`MOc@?GG{JWd~ z;hLG;#;xaUuITV*D>@fek4u#-m%f-G>(a50_<3a{+j7?Bn-NdWkzG}}^w!Y3+aW4SNo!Ym)slTU9w1KcBkRZ8Hl8g3d7OmM|#&wu(T-BV>< z_I3|nBoe%0Xk|?|^xKfDK^96qa2Ly#n3&bzg^hq$!uf7$Lu74LB9GHahZ5(nQac$U z)d{X?dNs7SKKe{oj*mP#Y55e-GmeST0qsZAMEr631A*lSGFGlHrNfs$NlLcZfZpQZovPAh|>&wmD(nzEu9m-W2)v0^y0s-|u zD_*8B8wgC8Lp$;&6FphCcxYNik|rXq(m=OeHN&8Fb;3hgr+X2*hL9bi7{ORH)x*ky zuJ(4I-4c1a@nC4Btfk>WOXsd`W3dK8JkuyV`_rI8DEz=CM-f>j91Q5v4*L_*_O=0{ z6PS4(a9|I37UFYh8>838^GP^e6Mba4pIz?Jnx#SVolu_4q*U5zQ&ig@V!-1Fh`Rq# z_nLLBcMg9;Dco|b4WrCM!;hDZjFe@zQFkEWZmLX7Fq9kKq|d~dw>6Ieq8mM?=UpN} z&OE66+E)Rc>+37?e&SY|JMvbHwFC8W>XHr8Nrw1+>RcRFN%U!waM(;G=s9Uir zGewyggOqL#4l+lHAUW5xn@p*CXEYh$m&*WDEeV|laLp=X6nDTETH;2{;a*2HtE<&B|JtVX+R&$MunNDe;Y z3C0*T!yZ4q1rW~2USmo}OOST_2;%T0OP1-{99D;VO&I{fDHuO)qNZTB8B6e^Fh7IS zd>yQq=SuUX0c8*5YtD~;Wnw#fStX1Ca->Iw+K=3%6jQ4FSG&`CwqcWq`l{>*aefvT zHW6`>e|V+p*>({p;^ZcxcK6nQ8^3o08#lI!{#$FVdbv&KBYr3>M~H7Ce(WCRR>f|^ zIPUB2>%b1D9jEY*|JSdGS6y*Er+Z%5@ag!1$D6L=jBB(34Jt9)<^cl`xvDw1?1T=90{y@xR zIlGCz{9yMsxo6WiyA8gizFs|`YcfF4SN9gyN6G5H=98`oXcV}}XFU>;h+J~ws;bPz z#)niGQJAOf6%R_0o}2k%!`lYCqWe~>*Ne?P2`&}h&CW)#lBptjn#Dv2(}^vvGSf89 zsEZzxu3N2}DibV*18vb4mwZmieJ*ro&620R5iXd1#0s|&n68+Im1b;gkz%$Dx3#ws z2e`}JCeRPlrb&JnJk0uEMxXey;`Zt}x{rl`!qjgCqAB3H9MtP|G`U`L(>S+XTyh7_ zlsfsdoM7F|R~$gQlx0RXc+0^pZ8Cs}50<(Q+tVrJMB)*_I|Lvz1!uO=-mVFBrWM(%jtkio^CHMXdGK zfxrLsYRj4JtM<`LM4C8zd#8N;`k?-_fWi_h_i*;nheuc434?XJ*DQuC~6r^+Ts5^bg~yHuzPyXo#lA-FFJW*I@GgOX?^xR=31C|{(! zkFJx@)E!=wA)v8nBbabl=KMhIHkw9Z&U9sa&aMwlGEu%jOqno1VIu2RM=ckfQcHm_ zKm)-S_Ne&qtHTv!yT7ErIQ#h1L(TV_FbsVZR((Fp9^8t#7cb`H6$CGty}5d;2B){> z3<|w4aznVW%DprF`oZ^%g0lunprJ;X|qDd$H<*ty~=mW6}Lt3zVg!l}vBOTfeco zRzq3q920weCb|TWPjpw#Y(F|pt7tq&T(_?#<2l1bA3=1(StVx>)d=EhnbeTE=f-u1 zYQ$mhLtXkt#7z(hj0K;?UqVCSeY;DGBf}Axowgm=OoX=p(Yc$oR`vDwhRsgjT2;Xg zY$3vX3o-b6H?T1iVf+7WbngB>4YYbK7@zo` z1eK#q=mT3V-no<6LfA3DaNPoHlFn}%?6vwQ=jO^34pSV(-SuJz6t!v)pm(VP$2(k4`Xa|356^APR zGq&PS4b(Q(z3x1(#!BAnukn9-WqXPqr58AiMt}YK zHI{O-m+1tIPv8?w^Bx1h5UF(UsZvfwuae^O@8Y0@ZHh>2nx_TVK1$04wUNGAckKs< zn>Enh7?zzJY?lSf(hB}o*WLqe3Ks#bGbB>Y7Ja!uId97(QX2NY3)Ln*VSzdw=6(76C{?#J+8L08U7VI0fI%-Ugs} z=Yg*Vy1!TZ0{_1&^7jmU7E-Tla?>-{t$ivkWXd1-v`t;QS-dd?y=%7%B=OzH*JpaE zLP?Ilgrwlc57A|XGEl93u3oUnoxx<=4SEMw5iMr95Vsdq)w8xhsiP2LSo{j+WmkoR z<7)qGlWweRGQ(~wd4l}23ur0S#bu4%mhf6LZLtHi`7289BQ3+Vn?c`fj7bE|hzD?0Cd)RQWTkvQ458wZ1Kz@mg!vZ)3OB`UnHPbm$@g(O~R~Fwhj% zjO;2d$Qex=K-x>Glj;Z-IvLx!oq?1lrpbKTJG~C<1toQTxWKK}ytuU(YT6&LkObv< zU9ge{5ulp-plcvJh^hRrW|*)Aq^45aYq39@TdXdxgi?l#>w*f!C7Qign$ra+C1oY( zPO%Aav#|7g;?UC~Q4Ugo((`II#FOa2HF5dfREvKJ2|w~A^aJaz36nNvHMuQnX4v0t zxXBq!E!PGD0H^C}+X+qO8gyAa<7PvWEYzvQts%*_FV2tKe^8?zI69Fr9$ZDQDnndr&;WnasOn!&d)KxHd3Ukb4bN- z4x}klmkuM)hp5_YH$@Rw!azONWwjQ89MshP?))zyf9QPDAWrEZ-OzW5W(q4v zvIQKC(-V~Kso`Zh+r%YT-*R{z*T5VAI^?TtC=A1J>#e_|Vu_x38_!%Ee3=>XJ~5Su zY%8umFAJ8%XY%-_K4tzUdI!_`fv%NB%-xS9>P~Zn&sNo3rX!`@aN2n9xQ3$ zyeqZZ=kwwE&9HSewd%9U)t9lhhh`#G*UT5O~~04!lN2p+oJr~uK)YFO{$R3d%}9pot|RV@ye1Igu!cR78V7VCP9o@Ly~ zLpm=WD8OBtSqD)0SXMdC7c(M0PcgY&*|4lMxOT;p_mByj(iG@4HF^3#v}u|l0F|3~ zm?I&I7(ZeTlsz_8!)G5^e-cdDqQO3=HP1<44wu(TNr6|)ViLwoc76O!J4%7Ll{%we z(6zEf+*w{7TWvPlV8yO{F&Wr0ay9m7MJ`t}g{^f_b8NQ|A`A!-xMZ*&vDdjultL4}zDLoY3R24%RD4_!=G7oun^X#Vaxs5e;V@viq z#NHzq+!PBN-Lotw_z@Z>x!%;XeqSeUGj3o;{I~I)O}GL}FuZI%a(s3}Wr5sYdf7%s znM8!q?74p9%hZKw`x5sSzD`d%90u+g2w}qecj(8 zdcIx!ND`ZWwsU2-;PzL|?PL?UNSWc9b!3@*{kjm#Aen3!`w?tI`viEF%lOBqwJ8jy zAN9R1Yv?m3ijFR2<7TGFTy?z0^aP5g$`2O}B`LK_Wv)NP#ieAjRaben#@3Hl*K1aQ z4zD`VJs*Q9r<85t?5m;5m!leTbfl$Ql=1YAnvIluP8wzRQI-81-M!Lsp3BF711bql zlmRarwV$0wvx!S#h>X@<6+>GE!y7Lt{5`_!S??$77|o$H*uNpLTXjk()H1_2*~)n? z8%Oy$Mmffzugl^jTBi}f? z4`%{QlYIEwKB?R5qi~ZJG3cy~rloykcwWRkDgd^iLpk%MO>V60>_zK$l}55QQl!h% zO6gaaxP1ooe(bgQ!v0OjR%)YMV904Qm?wG;Yo?`DcAMcAa*J$eTxsuplkCua8T&ZB zxZgKu-%C!hxt?ckzbhx`)5yNIF+h&juZ#cVnx0sY?>9$rYATXIq1;$x0-BCa3sX1` zCGPl|TKW_Ml439%VbE>c9Y{qo>EEU^P?)!|>(X$K)?{MQ#SQub`AHA6$8+({tybs{ zXJ`;lT|E)X16^wma^xHf)zYPFMf+rG86r0b{1IQImyj|;Bpab43WS-S( zWoq5R$61(M*`#Wl*vmR?xPUtfYSewvOIS1|M2u3&~QCr`v}oly+-er)rsEQN|XpTL|-CNBVqO4Ym`+()DRNA zMQ8Ozq6MqBAbM}{``>)ucmDS{?wvFD%

zcjhh6`@AkGTIDW)XlON}bJotCa&(mh z$U~UkEI>wtr;79LE$7O39#e_^87~YhXzEtH^s;O77&-AhYf55Rn`_FqI9uPk?O>}N z3p~35rV(3GQ{QQT`L4hApZyLVna5Wf0q(VzkMdaDf^1!OCY|Ny;2lGA9}mR`z=`tm z8AnE-Q|bv150#Y_jG=)O8?@nWZ%7B-MYL0oGVB2vMI0w1jUg4bgKg3+mcI$ zt@s{g`I-m1W%5QH5`%(ow|UxI)kpgnET~Ml*z0t+VWzR z)ysUSF}#(TbZ>>htzs7WolyPbU9<8ymn%7Q)gI}W8p<3A3SJ>-rUdYbdQ<U8a+Oun!1+7tjyIt6 zZ;FhLmfc?PQJ&O@TY&jb@R8#F`h{c83L1?daUU>}S$-RnRF;F?$$L|E5_a?9#%ih) z$TGxqXMJDwyd+MfOYQMBY1`-EfRQCw#*(6(OZ8dfl9Zd#lQ+@eS~990jfkeTt5c=$ z-?L9rPeXlKJ;VleauHW9jYk~lracdryiMQ-rKn%8_#feSXo7S0l8W`J>ny}gbB1r# zWdh>(quM-bF_1w>t|PC0vWL45$(*y?&qfTw9AHBj;WUin3eUqji~0pT3>^-K1yN)n`G@4L*_y93EgEIRhwJ`OBpnlVL@?g*~_ z-aPh8Rn%tyoXNnecByf+xGJ*H5tl*2Xpe8jxsdK4tuTr^Ch?weFH4!2BYB@|T8w`r z#}a`}#~gPaNl8PaLobO(otw$HVb7df2j)CL>9Y5HVD~9Y>tFBPJ1gd8!xNKZaLOP! z1qcNQSI0xbZ2Y_;hZH~?kskG80A6P$)4sFO+X261=haKn{0f(I-(;6_k~yvq_{q^O z5sg3&6x=!~rW5kUP}?afTRjGFqZ_VFbIKSq#%1Jb)7*bpJia-&RFupV;NBMQWQ{fq zLaJ^cc7Av`Rq_qW!LicZ?{H>mLFVvR_)SvN{jxu#dHi$I+}-MR?}g7rhx2(LbW*T3 z3F#881FGHoIGE_H63LA-B0i)K*qr;s7f)cleh=TSTh-uwtvMOI?%rAHt3KU0zol4Q zIb1*=)%uUcw+pSRHc2t6KL)%f?>76{7KkEFSpX3b^JcC;#|f~9n&R1g>l%c`XWRp94p-L z)fF^{nnG{3j_+xie8-oMkVhkYt81VCpspam6Fp;f>j=_YZVz+^CpGskba2}%sIw60 zti8*!sqy|Lzv#KHGDyPq=q>$YiJlFk~<1J|O|{8D|@ zv}KRZZ&?`n5m(IqQE^gZbMPUsxRT=`Yoh;J+F$d*Od)5x*&#oO+g zm*2`OIMps)5s_V9?M&*a_qtxxYW5xaFL>Q2R-73xCiez|^ZCFWMkH>OY!SSr@kX zS!yW@JTe80H;XF}An|@R{GYaENQ@KkQ5R4{fvJ3G4sU#!M?pm~dlNb5<=6tu`b>wv zwfZlYS&JLS?iYXf*%kP}wBzEUGf z?(NzEYHf~JMuVLWS4Rk+lo9?!W)_n7it2#HpVgmkF3>G8$nR#^A|UMH_3DW{Exp@Q z`q6x)R7dGJH>TB8d!*iI>ewWX$-n1uL0|K`uM5i$s-kW%fr0nET#aBtpV)wKj@K z)=wN9Fb>P4d5S~!?zQ6&F+IK`eaN|6!Ce`T>%N1)=CMxOtGs~ZxV;sX16$Df_|g*c zk;;<1MdOov^WP6ShvXUZ0*_W;8_D4V$?Kje~0gcE-NnsrKuA@dUJycZkh@(F%j#I7n%s%Jq*ZnTYrF^$< zi3)r+$bb>haxM)VlVhJkXYWOAy#bpMKXJJZ(q$>i&qc6vXOsslEilQ<{oCKKKEOXZ z%)m3k#|c?BcW>b*BOBg?;@x>HI8TnE2=hec<;9+g!QdxzE*&k67$;=PNWX88_taj} zMS#9#H86$YktR36$@@9?Hmn}{C7rBWs%I}of1Ma!GuY7HG)`a;KD}l~*0qBGvP&F} zRKdmfg25Y&9QNCm@~qtL-G4o4?6$?gF^dOtE=!Nth4S5qSWS>O>%eN>S1TWtv`n); zIdhd%7Zu-hsJ5X`zkbqhwFvLl^rBt!wC)qhR?gKU0D|2hBfQhKVq#VA#0WMrX7i~( z)Dp8LI@cc8qU*XFpCyrvKxSQt|17glfcfp7IiNG7ItS0y{8dpHVrf}k zv9AsV?1=3FbK>Iik1yj(C`S&MTpAfNJ0CY*viD{+!W&Ny41tc8e(SR))~)!ZF0M~r zqCS&A2llQM-O;tPNh$djx&Dy!v!k`4;QqEjcj-{pc}*Rm+NcO5LVO0}l1SBihziJD zA7NIqX_VKv9zbnj{5$|!8pvRN>dIBFg<}<9HHSL(rTU{*BVq7Mel@;rb;}KfeuDhk zz}o4RzU_8$jsBlCD&0FZI73`R*S$Y~{bsP$r;=y#;!bZI#@Jl4>}7C2wiWe@LMy1X zL4m0z?;1640;S8%h;@l7!w=&&74RE*0SUl}mCLcXIZm)9o z{Xlr!?`Zk;0}0^Oh5ue^8o(K&5SKzPe;D{3hJYXD=h52Ac-#}8hRVzB)dmK!Jjq3z zuMhOX&w0kkeV@)vSK2+(c6=tpbB%7&C%p z3mh%fxGZsOZ^bP&BHp`yJkZUoWH+3w0za8CmUqd4`|D|0<(#-Bt=s3E+zwQkD=Ja- z`)W2vdU9~cuz+v#pVjX@3cc&7%6RM-u*MlQOUVHM+WIm3NPE43ETqcn$I%c#PEQ#Z zKeWW}phbY`jifGU;-7Vg5)+Pbc^nLK7o+@k4hOEtV|BzXfpF!lTAw;a;#As!n}}RM zVjxjL!UBfppv!EC0L(+QgVjf+J&Ci=@q>r0J|D+6k`In+R){0gBpai5?ua}R{xww9)>HL z!?AQIeH9a))el=4t?B2{0R*EI2LxxF0xU&ls8+{8?>sZn{uCUlll0(!8dfC#CN4e@ zKW_T9jxLKA>ykM-L+HqHPppYY3`3f%e%Z12Gpgbjd565MBfzeoSM{n7`6Q1w5~c`7 zA18b4aM8jXzfnTDvi^uNFZssA+;7Q{DY}iEMs?d<{>&UgF3CF}(8oD&9_~TYe2wLr1Zf^n8|Ub7+`oTIE=qi#5Y+c$zS98v!^#F&T}>rI6yfB9%I>y z$WsHB>zWnY5%!(aK&w=RVOGqBl=+{jPK^Gn{@%DHFgP=o+(;F?T;7dpv^(4C?IW%c z7xp02a;Dz|jx1m)OwW@143j@u2PA6^F($r{$IHqIPO)ACgtJ=ohm`RLQB0SigQ|h8 zwfg0G!M9>FZmNsFC+=hNc^n)NgGc@LwC`?}codgwKx$JwvQOz8|M+VSYJ6B+!yxw% zZL@Y=e|nbjt1JUiS1G<|H2Uu!n{|aJwX~F6sWFZp`U4c$Iv_bGE@L*L+w1!m3&>l1 zCMj5|P@OoS8K~L$KjYmK!8rRti<5QiMG0EDKXQztwGLod$1>zb`?Y-;B^PpyfW(uy z+2V5fwP~qo2;LAmu|!gPDr5T|>0qBCOOWa_jdb4yn5-QHJb~bu5pbMKzB2`hbUDQ2 zW=+51fs47I4FKMn?s(`}`8O+5YUwJPxf3DoCYj_v)JRLez2ULz7rg?~sQa-#dOEoC zD@x2hOrF*G=Af7SASx>TqtvCY#}7`=-vZ=jM>34e7V{! zc<8Ao#d=#%4YT6h#97=c*xCS@B7P_5XUj*9gMj;znLR7I<5&SEZ<(dQ;lPfO4&_xW zPSMsJirF0j{~74?r?78@0WKwr-Lej8t#AHNCQ6ArrV}ltNnk)|2y6MUrGAbYIpfx zMaC^~7jU1|q0R08-nSD}a-VTywZr$fbK*ZN=wHa$zw?{FRTudOo%8>$go3;}7xF6C zd%(5ivvIQHlsdBWH&+i*yTP@W9v{RO|q5JkpR)cgd(o-(;wZ+DI7bGy?x+wW-@6<&3$7iQ!PY0}qKK|kq1HK(4 zk^!qRhHE7uD_EertAEI)pMqwf6|HY7$@VoX!{ZZt#|{6a$^b-w^WUVeXC7zoZXhvt zscvzQ7>WF|^|MaLd|=HEeVm+NnQIBp*u3qBiV^4lkiGSBNEo+OBF-txkMVR3!(82yHQVt?i({AREgRI#9(^Q^sNH17f(V0LeI zFnciDY5x}z)O0Z3Qa62|4||X~i}bzh-hHyZE;A0Z^$i2-f9PyOT|ZQbLwQ3V%?!M& zS#-Xvb_X9(oaQdNRh_Dj&9a?3T1KC(cD8g>W?YL8|=h z_nqZFCL?qUGQsqZFH_YjEEDcI=h?%^_3Zm^+ASJkHyIVXA`zFgM`8bAooY(^>0emw z+(|xXY*@juu>s`3HQH$)E+o7K0$_+!pj@l)Mkd>l&1wu-h+1)VwRhJr5no@7$M-(I zU31}URKg*7P2Sh`zKj-$n#6avt8{ni3KX4n{TBB$J&Hj@zY{UPJ5H8;);mt~g**0u z-6FGHuWg|z=_rovt9L^Je8pp|0dU=W?Ja-<_rLq* zTh-oRnA~d=_GGPMsFp@{QPMVZL4;IG+*sWdNr0A{Bcpz0M;2&KAn>yM+{cf-9SjBDWV-B|c^*T2F z)E1ftn%7hr6ti5m=eXFDHyS~gKdhddX|2Np>^LEb!cSRwkCA7PIK9cbaaKwFOTvXY`$a+wW!fjp1xlPB4^z_;I!>b_TUs?7ZbyvhTO@01(T*+$o zsa^lJ^lg~x(tlXB`L1D0FU*c)$UP)}$I}hzn$Ig9maCzY%5GAwqtv1%6~aDQMr1jP zY~d?{nz z{u5+mmhQwu>H5-|)gYe<{?@+G6(=c3vZR0a`@FLz$o67$Php@8G3OhJ7IShWBn)E< zoku&_Tm^03li%g<04K;@Cy?kQ$ldRo^&8?l=v~|$;CW_`c`{a@5)L z9~SW)JItTyN69Dyu_i;)k`4h~5DggyB!WjQN zDZrBIe#B(6zHu$4!oUInieKfEu*2CRyKUHQ{vGV>@Q8m$cm$G`mR8ve zIJE!$Gc$t$6-ZjTis{{+9-N-HBLIpS{J%$VttwCgAHdM3BLu|clsEtsX^14NZg7hh zhD*yp?-zn0qHh=c@w!tRWZw`1so+wApe_ZNc(qO}xb&^qFCE*%>lTiWs#OTO+JP`P z;O)V;PW`3KTuztd0ec0&Lb7z|odZb(xOb1&Hyf|_0kV<~C4dpa%bb!YDA*4fvGb>B z);aatoJM?C-CxYEc8U_}JS}EB{$`F%wQH;!d4ElA0I(zwI<`{ZSDCt%03Qp`3DT$W z)Me`uK;Hu;mv(c1>DatMQ_Se9o7a+3U+tw|<5fEV1jD!;$-~3-L)TYN6yF1()~yTe zB$3E>7mh>k8vWscgO&J^&b9QaS6S3ryFK*vqv;G>|hf1S7g>;K;y4ekCrevMy#6o`7a3k|+a8<3jwX|tf;0oPFs zT;X%qp8jj#;MCu{*X0!B|F1FNZ&@hMrSmWT(P1QJ4UJ}AUIt{bTc!KM|FG)*!#enJ zc+rhd7!`Knx!=T~Rx^+UTLtX!T2}alLLu#kP06>R*9; zdc_~k@;styL>|N*{H)XO45J*iibZxh-&zWQYgi&FdfpKApZteaTnC{OQZoB^ye~_a zQ8su8&50LY6GHj97u~CL4%weV1@&S3M9_;Ul}fg-ejAkN6tw_NR_Gt90TOo}j_o2@ zU2kyF4tPIjprlw)f5PWKEN$<%zk-U%WiEqsHUrq=d%lVWe7e=wulJYzDuZl$r{&}U zB##L?k`~#upDK#cqUz;eFIdat*MG#yToL{I2SCwvF|L!OMgk9cNoJJT*iI|OzOG=S zn0y?4OjlB&z`m;Bt7ZrRRW>=fqR;X^3L^Ju<}I!2&myj}vhym>ZAUJUGEw2@8T&#n zko!XO=JicZffI+IV#Fj+)z6%7~Zx`=3w3U7+z zb}?psC6F|~LR@abPG7df*Qeg5-ke7khElycTE(c2+5~oLcZ^RUl(ZGcCnN@MRJU0{ zLB~uIv0AZ$(HDin+&K0zDcH7l5&ZD$piNO>=iWa6nf67jTDF5J6J^AAaR!lm`$xvr z)&{=_0*VSZs0ko()`c8JMsFU$GX!`XN;#nm&sBdgNK;24?w*YW)6d&j9;*q_8a_P2 zg^CLbva;Ubyj;O{@IVu?DbW!!)RfUp_wf-%A9Gj>0tIBLy%PG!253v(2x6 z#w=BWrR*Q-MAV9&K9Vvu@9R@2HZQSewPUO~+;Ehzl7B%78yanLajxr?ZQB{PT%{kDKS+a~UCpPsUEigz@B!%Z%a=D8|>Sbu6P zjiQHpvGqu0K9S6(M?d`-#!bpdMEmoVC$>VghH-**Bk5NvcO{A;jTo;7k+~uzIA8ff zR_NC75=RMg2WKJ@5!xSrNlz2gm>9Nj5cdcAvo4!dJay4W+10FhVS2A z%kjj0G>}M0Nc_OjR}#m+)JIz{hc45c>2XsXWYa9u;PecQ&`>osfs?JJ&#_DErv zK4a`L^(a(Gk?%CNkrBl5fya$^wfsnRp?9Rv%TCN8wSNl>`9@%^$!8$57fs#Br#`Ab zYd8v%5Ufw|QK<>d?Q#gwJV?Nq)W8@`4+hk6@~BCyk<25o-wRyeEy|R75IkUv?o|zz zACW%o8l=Se3X!pKm`}9MojQ6_N+$__n(dB8G;V*#BJ-vT9xlscAcnP=-#pyo)X=Es zH!3URb|5X0j4gN(aSgvfBqQT6>GrP&vpe^kB_0-U(kxYs26^kJ@(6DG&Zx$i)BF7j{;+*MY%x3sUc6!f(bYJ6|Xq(P5934fAPot%+c zGfwfMj6j`b?6b0DRDOQ7vgQDdNL~zYNJUXBhXZK8H{ZCNY3aD5mBB-=9`k-89zO|9 zC&gkO_>!Rf%=cM+!&L3`8&h4puSJ8Q^V4(+|xY;Gwix{@*pIZj>u>XoYc@xBbGf})yw#xqkI(`cGe4Gmq>Uqr2NUFzKqsu zFnb~m3f;qVph3%8*Ir&=>BH|Xs zVsRrLKw8@*IB(MCG`HC{DIXD%26TXGzjc!y%%rI)--G5JK1;X0Wdw~wW5kdlz>QuRv1C%4$O*B>|W+CI}6 z&P*JY30G^*Tk;%zmReinmJK7RnRoi4dSxK=2_Dw|x9EEdjmZ}&oWLwfXXT6M=msJ) zZlp^#jGsT^%hzaHT9*T=7aJp>GQRl5i;pig!82S7jzt@rCbNiveij8TP|%9gJd0>b z?T6!dv4P4re27F>Zn=Ph04lQYbK|%VMdd|C>K~8myvn7L)b7UF4&`=4#KAsR5Eh9# z*_!YlrrREcCpYO;*fF$`11BeZ^>zO6F+-U^V$1P0MyF^o zEW4(A++aCX`$zV+4%Q@}`d%&I3|B*O_C1twNR_qT>s|dY_UBR1m7X#tVxUZ6z%rOJ z;Zy%g)2OC-?^P2Mtp2K<_AATthy=as2PVesSk^_F1mE-KRI=qntfcwBd@h{f|3v*T zOeTCXu}y-J42$gNEKz}bahF9@Kn!%8cnNB*kl3=kPt&|g9mMfi2VY)e$ zgNz^u*)<8o;tRef3C{QnE5(P42+##1Rrs)8-8{hUrk69*V+OAjOH`sh{R&VPsN&FK z@A6Ds3w3&`g9v^)@uIrIHIxR22Si-s;HH#~H<@_yqY12SwzhA9*OyZBoDOjmo=Nl| z^?7WjwPIf3@Kwoy*{j3gzcdddUo|~W98YCQ8Yv%c|29iEZ0G%ncu38d`l&wP0(ng_ z2$DJ=$Sd&V8nVlQIl2?5fS!Rge+;2b#@-UOS~rFx(~=u1B2?#zqg6MYX^ACbUyMW# zl8MggJTQDG#DH>XTM_?0wPEJN#?JlVRq(8j(E-?|b!*yS)LJ4fyyK6pS<^+B^9Ct< z{Z_w^gr(2cZ;$On;RK>4t*~Z8>>}&Jb0qgm>%J{=YV2fj8Tl@6 zP(qx@?)&OEVl6wnoLzm4Z&3=X zpk@*`2ZkrZVB=G^KUyF_EClj23ny=iFL-i!P(j03NV5@qD2RFDce`B2~8Ry8WZ#40^DLx{@^}o<^;cO9yMx=o_`o>Cie3^RI=cKMLi_b znD`sf0a3(DL|YeG9N#a8XA9qMLh3cof42TIgHSnxyvuA9cq`A5#}n`Oo3plaCrk{7 zK>KNDs7B6J1;<`_(Gfjz-}J|d)%eo5Xpv;EG!dxs@nz<*@Pysec?PL1Q+ApR&#SZO zSmmeI7B%k>9?ax1E5pK+9U}W2@3eo%5Q+$_;S3MA>0Tf`!s6Xz2?pt9lA$)J zCZ`=557(b{xSmZDMj@QlVKO@yOS0yS0D7N5_28F*epk%&X`AV zrMxwWiLs*mSdd77fPO1bi^;RR231jxQywQNul7h2Tx+Y0@x0X6HjT*#4Hz6}H|I9+ zJG?&?;iWvfjH=cOy6%2YdGoBEZ3^dSxHq)K&;v<>ESxH`$*Mmm=qhZ3LVb@P^yhLA6Ro0CwigWxdyqRe(-i$h1!MA;BH0M4fTU= zLd2?3u!z%e{3WPVmYiJK=jCwv+WxU{ovp8xcEH?KC=q@OEoYy|Vn^!8jmrLwmdG-N zn6lWE)63Ui%oV33pRsU)-D>ocw9*0#UNN-7PletaotNuLP6|;_gi2JqQpF|VQsrvy z3-_-)OBZNG^)o*-R4!r?sE7~K0u3xOzZaH7Oyn}D%#|}@BQ-?{Oac@CU z6oHgx9Eh@=lCu$?uZqJWKMK~wen)GXPB1pJa7Tba`<>A^)>Ey~c|X3>v}=Jf4nbmx zq;?Gso9^j+cq*ZyCDSMrUOa9?iV6eC%zXOuFvrX;N-{!gR^=EW#qfp;0n#7jxU%e3 zLy`>E&}P-@zz-Gr#WhBEsx=zDEp|p9_&_xRF^!zf=sd#UR0t^GFJTZq%C$m~;{|@` zLxGnN`UnB47`H1RiyRJ_<2IV``w!?%E05nX;e@v}4^?GL6v8;h{R^L=UlVyaWv04y zhB8rm$zX_$G+5TVpd(ABuW)5J$QZN4Pz4<;ON0c4y4~F%IY4XF_9h(@ zVaFx-iPH$rEcwwJZE3^(O^)9z&dI3E#xEmK#{=8`CuIiklQ*Mf4=AfBk(Rb1Jc8g- ztCtb#Oxd_quRvaFkN$wfJg{N~Nt(lL(%$t9$Aw5N5EU(IBt)&bOth-0#@R5QdJ>A2 zeArce;$SG2K_o5xcE;&(1P_BIX=Ym1=smlTMW=Xp%bVvhMwEd(a+gR}ry!8^c$;WT zpdy32={~!o80L`q95VMH#_uXyuhu?X4N(JrHIGp5I|6K5 zELR+GB|;^EibKeTK>y zR9Vgw|6!OZD32|PGMvIi1J$b0q9HN-QM?vrO2Q0=j~LglQTOR~#kDcxYP}ZjrIxV` z&WVMjjJo-vIG0z!Gs})y9Ry7&T+4H$Z6=)M3xTk> zlOS9iSF4BdB^Fo21Ot+{A6f}))C{vn)%9fd>7Ub&iw@)Bit<-6gg~$EC=dNm`~&l_ zfqA3zi*R8hcaYUe#>e^L8ZoS2uO6=VQ)#g6Q}kxA;tZgi&B=ODuT<$oAE|Pej8K-@ zGQKtYQ`9@^R+Ps5XmQ!RJEys7c z@5px{0xbVwZT!r)x~6%bq;&GO>lPrB<>tGjp7q|s4hj@csuEJaBv5l$C-S-L(#WhV zPZyiuel4tOB9fEx~$k3tuKb; z{2QEKwIr|NOYXVJEYl&R4mMLw(T&>~NSRnkqx$j^ZkaXOHOYOlu}PXmnZz(pqa&T9 z7os)j;&pMw&&4AFyaMIcX4*3Xus8Ffq_{TCTJjnVzDx<+wSYO01WfHWd`OXFl1@s7 zrY#*&S`6;?il#}(P+Wf((DxD-FM6{q49DXkQ(!ec-3-mEYpvT*rI71P?Z>K}=@hSy zmbW0~!l>cq4B0qRePTMI1wVIBq3u%cMaZ8I{=L_+kuS?VeO@Du6i>l|Vy}4#_L~6) zP9s^ZBTc6A=1gNftr*oH6MZEv6;{fbU7JYRj74izJN6JfNM^Voc^MHN?0}N_@kdb^W;qrh4$ zjA<^7lf+4d_516}VFjX|#jK}smE=UXSyW19#ToVD9`%tGsNF`zvl~5+MO40PPVm=a zHsr%YZ#ia#%3ymKQpO;CpVBXuOo<`C2u!NCqz#mqtYN+)@Eew~@z$DFDp+&vJ3(X<@<@k=x$n%v7EAZp1-9MwNUc^vEd zjmSfw&{Y7fmfn4b(xSd6NW&Lvqb%4}n0+k+*^cdpDpBEn!RsctzlE}EzQ_s(88Nl> z09Mc+>3*EgD9iaoY3vM)g#W`Tb1kc-LbCh^8yxVCT zd(&-cfzPs@`q;|HeV5(FLD7p$6LKBPHC_&dAwimLMX%r$s6c2xy(hJafCnMs7iGt` zeDABD$+oT7!G5|GZmL;O zQ8ycp68RLQ@zu+`8?~=7s-Sp;?_QPFRYJ$|!v^7jp{sACUuzx^)LOtH!o%*ycE@G% z6PY!XBpc{JmHvQ|oCN0G!{2%RP`Hb4<{cvLi)`tj4<9-cwyM-h``J=U~oY+#bk! zZ{yr@9n&r3u`29?)@9n}8F^RimPo{oCzIgfJ3aipAaJ1{RPgjNNc080$`YXWj{LAt z(`z~FL{YyUA6I_=;%~{H=BEl|_qFBXQe2-)R~KFjG>C^8zT7o$b4o7}b{}Z(z0S3b zx=l;)-YVJDqA$wAKvO*3FQW77U*3-OJgOGZptrI|-s^8fR-mNPtn99l6*;96%{=Bm zIulV|VRy=sSiUdIn)iYTU$coMh%zFME}N zehq)y2j804Q!fJwTlQt>bekLzMKPpL^4Yo3aKbmNh~de-K&9ve<`EfjCsKzR5eblU z+;^KaoZz363~IBzgKjytu^Xo!o{TlmEe?5PidsC=KlD;!kl$P0Q+whvwzwMM zTFBCN<3+9)hFb5Spe$@hPoQ6grCZ!iGF-xlt$g~q6P~Q%es8$dr6WyHw|dL?r7Xgg zBTSGfhs7K65RXY$t%U=}Qjm*jGX6^`%*H(n!TU99|F_7CEjPa|w4mSjPy%n-;Vn{F zXkDX%XcC>!JQpm|FrnY@sp*>Gm#={~<(aQ~=={N)UuW_oMZ~En=DpK>3k=fJJ*szG zrHiX%$hSC;%(uH!=_vZ3a9qD(!^mdc6I*~r`YpZiGd%iu|L$rOa}8;JR1X zHPW>1I&0Z|K!Gqm(4_kEbr9&t(k70(n1P~#{_`s3SN^NmFfc>n8f^I{4BiSIGK_iG zoq%2^3yMkci380TMJJ&9UNw)Ge>%PG_5JDWFA(*SX|XgGwZWY*OigU)Zdp%bDlzk3 zUqtK4nMY`%_08|0etN$pZc%@YnM`$Fo==y_#zi?0BiG@6;MIlbDr zRsFMunt=vES>NahOJNOtg+HpRSj`aKZ}FgR|3S?eQcKF8!}163E0&3iWGyVH0PRx_ zh8tt5_h)gE0_z}aFCKYpoDXkER!Oo_UJMVQOj@Dd#Q~*@33d3cVYop-hR-|>LH1u4 zKZq+IGxj4Em>5-FuD#B*^GH9$1&O+y3Of)DD2X)=lA_(GJ$IFy&GDTGmqrmyZu_{BD!vC!^7Z6;YOriFwRL+dV;xdaWB8i!}@Y0uud7J{aApRTuQz99JF9KBAuUW_$G{XK0* zK-*s64;}U6k^WtZl+BPMYoig|E6sJWVmpR%pZd18HY#tta8D9!B541H+#14@S{Yq9 zg>6onDryRff6VJ`tVKHVtRuF?a&68i{CoDzB0(Zez=r3b)}gLiyH+Fj+^JB>mT^H^ zP9&Mb?>1!xAQ?NohG^9$=#odf#EeHlL)Iv2iHRvz9QE9yT=!9=aSq)B@i$Ordr_<& zPv(%8w>0HhC#IX}_mC^QqTZ-qMr$(OX@9WNBg-6^mWGYyhha)GSN@}KVrHp_#ok<1 zFa1?7B%%^nkqA+?y%5||xJ&u^ch(ty3kO3iERg-BBsNU@9{zdD+UNNJ+eKxk`#?8Q zsAkq0toly1;DADh57B7(%Oj1^!3-@nHpy-l+g}h*F(eO}s!^`8r=Pvmo-gd*#I>d0 zv@uVbei9`#JWI=LZd6CCNsCOHcqr|py=ao93$SwG;-NCpCZ8s4%UxNNUy}izQ+4yU z0k%@ZaEhPq<>`m0p;8AASL!xe9$QEqQ+Dt!o>K7&q5QTpU-oOGFD<#@eiM|0@9GC1 zyTnG;e{y~CUNa+0YpxLG;K|68CTu-3^mIdG518SrqDur0ptvauwnL-8o1}HZ>vZH$ z-azAsB~CJHR)5KK*$kiHeOvfRP#Uiw7&HJ5oB)l!G>^*Us;$2I535<=+8l>C-I)QuZAlOHQ`!5%k@VQ+oSZP z5oe;!>pEh|(p`BI-_Xc{KqpbI@H#As5SeakVpKOB(tNDL+9J?87g7mc{v+6Y6=Ac* zm{KFMw}}>LwiUbKMXWo&HXe0pQ9o8097x`#i;n#&p)t%dwN-hGig(hYd4zG8PHz&4 zmXx#+N@T~{p_OCCcGE(Xyuv;7%1BnmS>xK#zFlMcG#N-Xao&`JtVV;-U}$zo8ZP%s zJ=u`xWesUU_9|QV{8tLf3dVwyH0LBExUUPO?Q0HJD?b@p55wq|MM zmHStHRCa#}C$1gp{ezH#4V6$+MNBP~abYW1|0Pw78XFNhEX}B5)Nda1w(vCT*(!s? z2ji{vX9mWze~K31tOQE?GH(u@vP*BHG#!1giNmokGn;^1Pd@SBfN>b*uQ~EMGM;J5 zn1eT)={$9hW}GRDM|aJ!W4wA-BJa=MF|6|kXtm%)QIJ-fSL*(CyYH8TxuGsvF4MmC zyw*(N)==~B5mpgmrP|_%b$ysZn3NSSv!L)I%0o4&9-r+S3)@uX@Vwj~*!O32xZMSefCKwY zMA_DgIz&zM%QhStr{dC!E_;uW`^k#WA@*eHG9P1>RX)0UFpUU}orloh)d#qv0k5pBnxk^KUS!t`lwF*SzPd`B z5P9QH$J!@sAYWU$)sJmO**{u7Z18h4hAe(-YwSjY&2@|3{7j|eh|Ppc80V{sk(`LL zm@O7ln27W1gZpH)nxL2`U+SwerF=eby&+Dv;CeW=#!tixB<@4n**bKv<@)5kG!MNV z{cJ!}`bqo2qI>Y+jrz666`H4I-6rlyV+rBo$ewIu!=5#jd3yLSv7u@uvCrvNC}Ica zo%!bt(fHBN@hpr{lFs!8=+At3v2$5=D~vlCv2ct55EaAWp$MsqlX0Q z{Z!rvPso`iW>2e>!=juDqJP^v&&~ z6Q^0!nlb+x^}K?OMoU+f?jzJbeu{^rY~hRR_9e-+Q(H5K^s`>csTphv-*lPn3bB6v zA2cmKIl+Uq#69vjB}8RWT$K$(f=BOa)WC3%Ecc%^Q$h6}tZ+JdO?&@=@Yq#_qDk8M zF78)2_b#2$U$P0UsK_=Mnj2c2gjWr-+&@$HQjRo>30poQr(I$Ru4ZDFCuq3RqlFj3QDG5@46#|z_FhfKvH!68B{g3DSt$J0eVt|~ zkX&mLO)J;ciABc{3i8ODl#%ub8C90F_oxw8lg9Jmh&_m0A2^F`LJ~uwwJd4M{xEmH zIM~Y9^HR%v6_DI#;hw=^%MfOL2VHKigj+PS&zYo>5gODIsSITZ^mK|WanG`|t@Dv3 zJ!Ru0G&%(fRBN(Z;xt$D5Cpvt5+{BC(a&~mX5e+hgK%n;s3~s48_>6RnjF^1##3?c z9t#tWwFTp01Lh1|&?HC;`%z6xkKpG%m8`+OeKmCu9jMm^w^?;}Z^8MM*mw9KUsNfz zLhf)_ZJsVlO`IcC~9UrB>6E+>G`@niMrCW2V0D(j@2g9*YK= zZM|85HMv`49y$;e61EZ(H=u@$rcY_0Gbd_ve+-tt+ro?-)V@osJ&Hj*d@dj`u^rFZ zyI0Cx8j_Rl^#L?w%von>+VHIPI`*%8q2G8*z-l~hnrw!cEzyhqQp(!ys?68}TURqr zaVW=XMcf12e!p#8ygYUm2R$m%1qFiPUa=2pqPxZK=us4bhGicGnO)F_1+YW%R|`4{OX_80m5Qg+n$Y)n}T zPhh=?tF=~{%Mt^Ref7X)gI|ql4ygP(LiZMmzlBbvyEFeS`ef*aMun$FC~ma5IT`K7 z-s$2Gs7VyJmt?*);}+=4j%92%LsG=O6y=LXjD)T#Ws)&bkmK~<*T%j{Gm-nq)r+AS z7+DhunIrvBwfVDRu^`NvyG1?0Vjv5BpBq=M$p=kKN} zY5ks(Jm4C=WX5@E0ezqORTb;CdwCp%K^WMjCCpChTN=*n9ejApC_@y@Rjr51voaTw zMn~HPD7NeSYK!G)qt`%qvS6ERtuKh*M6e+Jew2(Pxvd|~?^(ADL;xj_`$1vsyI#;Z9 z9m-4Yl*7L8sb@8*ne;VyXLL3iQZa?I9IJ=B#s^TZJTAOL+Xt7uCZ=rMVGN(4NFRR9!3c0D0>sQJO zANGm5n-9cgwIQH#Z;Pr|Y$GnO^SNB!^2LADBN#Cml4HnPRH1vX-bJi8_3q0>%e(6g zvG7v@(aa|Xl!ay#ML%fXmRdfK=8E?#8dH9_0s(XEwQ1kI!XYT8@1Fvi;Qdqtr znciS>ix%D8%{Ot-@6Y6%^@;EYXTv5~hc}Qg9fy;dvybc02M1^!%L|V@<#IpGr3>|X z96@scRlH;()J~;s*!n=S7?6$Jg{X-@ul=r5_su;~b($GWfgk?3#_?(7l8{x>f?qL= z-@KAk!O9}-+URfnhsxC&AN#uVJkc3{tDWGlPPYM|ztf`FEK~B5E8qLTvTtj|vU+>~ zn)=)^f=U%|$5OPNz`Wjtl@D)Vw=RT=W@_CR>{8^_jnIQvq=L_G3>^{?0x23x(c*D&$q+V>V$~Q zcR;!DS5ZQmTtVLX`4k@74Yp=TXon=|mE|?OAL!Oks|qLiInBZWBtkB+=$_S}`Eq{K zaZifbl$ooz8A^5#84P2MV*q^XPQjA-G-(B^`sXX?W6l$ZpU~oIYKFB{{p|Ioi6g}2 zHsQ%%V<8!Ccb^j#pRSDbumiskwyJ)VaQo=|v)6vp`P=5*;qTHzgY;PH7js+_capdq z3tpoyXly9EnPh3P6O^LF>W|vkI8Sht49H7dy;4JQw2kLDQI0-az|Akf%+_$lAxwXjE(T!|oG&lH0Vofk ze);_n4Q4S$yxVQLZ1L7Ks!2@j99r6j5E9SkJ0eqUig7*&80uj%)x@a^96{wghw^N$ z08*VT-Hd_Z80*B1wGb*v5Ro`KXf?htkIfe6%k(hemUN8LpMj^i8-~F zYSS&v^`i$TLK(Z|9IKT`G!hkRnuCL+S`>UzB~b>cg`qA#(; zYT9qlv5z_iE?|{5GWDzoEDV=)00Utn8+tGR#;n}I2t4lms;6;CsB-XbH?a-k|H;

026oE^se5L*KNX-MC|w_Uv(niKX#iw*0nl5^r-yfsSDA@0@r9!a7Cm5`{}N zjx`20ncni-y8l4^huZHfF`OF)wT~buyye-1O&DKfGg**p8&zoGz2fYm&*mLf%iF$^ z%6lsQDxgcw?>Q@XJ_R)QO%>tr(c`~AAEs1q^fetBE?`_NFFLU^~oB(zR z-qVe>SEkLjWEe&xlVxH8KhFz(5tR=sw_!p0)ySDYtz_%o)5jL?j1wA02C5_^3r(wU z-I_8NZK3KS<}D2vYxWLE0AjdfjN8)XgU06IKA~XcVv`oJP>rJrD3_owSaVmNTw5SN zc=7-w92K0`TPqRF#3u=U`tHX)hR@wPPrzq@KZtW)xggdciS+!9?ciX1)>ypg*)g25 ztV#v*KFRD)e3wCJ?1hEO&z1t_z4u8u=L^y=pdKBYT$Yx@h1n4=PuYoIC%zvwN5{Oa zI)U&RvqA86+%9^zMdnmI93p?_ULJZ%C2KmU*AZ|QXln*~>ari(;l}xtvX=l#r|cDq zs)tm(z~D<+9F|XzM#dWG`bhyqglQHE=roI<=n=PwKhveZD1(1d34Rj=)V{~d>Yv0; zRN3rKt`%ffD<6-;L-2P1YL|c^mBaVm!G*5xwPPoywF^P%dJ`~T4>7=EejC3>jM`!Kd^Q2R)eBAZgJ(Jbwm8H-TcqS zMdR01bfaR_=%RJeHT>2<0Xt;f=l5h@7s@qJaF^#t#tVLhDv9ouXgOKW`m7{vgD38cb7GS;k_%(Xy88PoHT;&!jpdrwg8T}pFju4dNZgk^#o)li$ROMAU?5Hs~VD~_RaM`uiLd2}q zw3K~INg`rI<@XDpx`Fr5cEQF^0&(pc0eD64M6BWO=Hm)A$A)ksrD@O6IMR}z$X|FX z2WYnmi#4$HQ&++SSdz_U#KAk2t*=Vx)WHrQ2yqURAJ z7n6d|Qcn{CfCO5sn&dju>kOtQc|>C-Z^hwx&oOO`+Km)OSo?>$uIL=`W=rG?D9OsH z9!hM>0SWa#*#C@~d3$Tty0o>tF+ZF)MUDu4lA4+Fg|SZnx;#matC+o$F zE@i4;pTKB?&NfR5>VUNJM5^|g%Jo?{=GQMk)|PTDGBF;qj>N@FMTeFQvC^|0Imajl z3##{H(@GcJf-Z`}Qsm{$Ep}p6J*kn(#M?I7{85@4Qv4%XFc-|n>s(Ly=}!qdhOVJo9|<3b9quhFozoC21X6P)X8n(v(?pD zmLi`4X~G^!sBt1v^QnIW3qeQXwK=+Cnx&BMmGtk9Q&mJJ52_Nc{XJ8<>_i1*-MU#m zZQ8rRqEfZqxx!WVbyRVE-ki2Tl$RScje+tAX}reL{Y(Nkw{(wxlfy-ECt&0!+m`YM zE1v6fQs9I^CSb)CXBgPBXM$j*QhEZBwVBBNsyw&0=TxcWk__gb<&zkBn`l7>&Tn#< zr6I}7-1W1-TODOQsZ#F!$hkx5n7yu1;oH*!~kWO!RdPMJnAq~V51{#cgeyt4k=%?x0t&0J}dkJl+iN zZ5Rhg^8FdWA7DWmNSQM!_?Xd{8&G~ZsUzAVn*XE>@+99Uzj)D+YTA1DU`QwuF~VofYr zp`3lwJx}MEIPyep0Nzda_}5Rv!pOOpb6FiQWgwhp`_&HG90^N0Emo z(k{U6)m{e(v&^G^STo^uc}y{7-++kDdz0lqK^{Sugogt6h=0--q&}-|&tXr1tOO&1 zV`wi-q)L6MNTEdAv)?10q)0Z&E4&r5m=TkGzf8sKaM!mHFNzeGdkmu<2!i0hUie%d*%v z&u8A7I9@N?X*nhNpD>-?Hzz3kip3aTsssoLo%X!jhqzUj-z`6ZikGV*@A6yd5P2%K z3e8yi_{?N>iXot0b|-l^5jOPV>W5+Jk3-psliGI*N#ju3O~658UkCf6>#eOW;IUvB zuL3AUrx@X$6*1@x06h~90D3KRL{5zT@|=QKsH`)MAvxQPVkO-^Lm#x9>B(LG2N=UX z`Q||phKn7%&Q06sVeZN=M}#7@9WQ)*SKj7@iqcazCp_5N#X|CbOyg>KBIV92qK5lu zQx`?|6?PIn5hk$32V+mnFkL$JzIFXwD)@R`dp`3+rNBYkLvXyO97Uq4p|$X;KtHkE zGQJ?!kwep7kBIY(WcawJj?0({G=K^S%RZFnt%po9HS&8r@rY7|6&9ob1th{T5hPz( zp>ikY`K@}qcRLW#60KG3YUkW0SG4XdA_c6jxYGKPh6b0Y#C5o_-v;sD!Qb&L03PyB z)B!HJFc+kh7?TWhD;UN%fwWQ6g?#i=h6KyH@h%aI2^3+PnY*lLt(nzGhhUml&bDk0 zgcA~(Q!|Ghjn5yO;^E2Nxcy1EPR6)bYBCHeH|$WKkM1zZ9ErfRUSDI@4gbK4${ng; z?OZ7R&(|Rt)cj(ZQo+Zx5+;>%0OHJoFmhpn1_!(UM!4u&eL=bQ0-{_7=1wbo{ZwV} ziy!9{sr@3fi`FiY)bezhNg_{3Wc0t%wZKSC-b^-EP>zd?g_u?WGjO4s5*2ExY4sXn zcj4%1hgWDl@(`;?eHv)U`+OX+g?h>LWI}zF0eIuZ(OkyhW>M(!fFRca53MNrI)zng zX^zgmQ*`ffqKMuRt6S7N6w4ROD?NovrgaGf5vFn;CGoo`5%!o&D{W?kwhP3YDx3si zxGQ`o$}$vn90{W0hDJ}Re_Yhh%qc@y_AVOPR&=I?2>2_2@6SZ$xnIjPao2HZ)pxwO zh&j@m{<;(NZ<7Nr@Ph&L%+8>yu;;u4;#-({dR+7Knt3k}w=LOb!0RxT`jtxKLe^VY z{`p7cX_yo{9^d9g3W9Yh;B}<0PS=mZ7UR8VY}>`cm`k2)n7+r_K(EsO@=DqP8KWbAxYh^1$M&D#vVEbYGDm3P>lG%*GMus_Gr1Bcr6#cGI^k(NZ0XQM zV$1kO5KR9gG%YnJUgeo3YHns58)&1IJxzY_`<~pMO1@DT6wb0)Ij@8nZ^_gWJQng* zsCFgm+=an)+$jdFYjHeW`a0?it%z)K6*uu~h4gZ(*2X0*2MK#XWz_~xAKu+I=J;;B z__!(s@hq6j)!d>%wp6_6u7yRDr|<+9i=npRp%NCqf_~&M7}wBs|5mt+L(cw9?T<)c2FoY@j|uvj z7ydM^mF^0(__$rPNgK8Ms??9*a)kLxR%|&(h+lp-OgyhGZfyzE!%hL5Db(eVOCg+d z?id~g4@vowi22*Ht$}Q@W5xG79`mePIDLreB3Cbm_0Ti^i+?!%Af0N%b%9vz&di!< z{a#IVpYtcWKGLt z-`dG?W!IfAbk$(-gwM_1<6~TDb$UmB-jlg9;cq`mO?;}IL_B0HxQ>8=#)LR9yD^V7 zUjAb_3?emi<28@`=w-cqEx8j?mS1z8iBB|3IbP(Et+A2}7O^!S4Dr>zb?o>T;Wiyn zr~V82W^VMFx3%SgNvHAVQ)-f!SZnS8E~|G7DUa3HX)tX@couTNCR_(^JI z`n1Urg95}LxlrHjh^gyIf7Kv#6-krM_9i7)M2_7Hsy+89N>bmbzh1e!*1q!IT zdSDxQq}I)oxDF$Q)(krIZC=5*rJaaGsRFARe^S=|Ca$aeeXgMQP)CZG$U;vLh-X)la>iq18;rN=<#^YM zY{9a-uIY^mMiGW|6;!e9yfLG$DjdHjV0TqKXW)&KLJ*W@z@}Ch;4kv(QGhb9J#aaS3>w_x>XC zm0J|nq~KhW3!=~MD60@+J+X~$lb=*%6#*9cRbvr!6+xB>AV=W0hb@IoFVg)l!EA0{ z$iu4ChFYOT#t~2dv3$sdZz$RVny8t?Z|eEaC($(6prxle@2s2ZWToVt_5(3#Hq%m< zyuz=ydJexK?rmE=jS8uE*%A{v%Px4uZyw7shWg!!l%0wbDXZt%9?XNDPk?L0XhGA(SXre=aSMfpjZ`<<4SW( z%0?i}xMFiJA-3u6Bs*75__vw)a+Q}>mg~|@RoI(E_^A6-|72D~lu_h)j%$$0J^qQt zC4E3)_qor%QwW{U6b`|MptD1c$kFvr|J{K z{J+mX4+=FTVX^~hA>n505J*S9D@!IKYM1cvg$hf@&{!(fI@>37_cDwGopCVQpyg05TtMW3|k%o=xP`dKR1!5Bi+`c$fM`XQPN znw-zq?^hg!Ej#GpYj&*BALqKfhko( zv~J2?|R?>76d<@xX7ve*NOYd)$f)P@je4|WBDSO5UI^b~5w zRL`bOVdi21+{)~$bGec{5LugH!q(RS)_k#kt%PBmOrQRlp`iZXZ_+{Qt&$T?+wZD> zrRNwrW5Q|mVFT5UQ6#FgAU}Zz3eL31SXu(#g!j!04p&fg zP;NN_P*!JG^N>TqL$7d-rqQgaSvWngXC@_o;;a7MId4eveXsjhIsnTbSa!Y@v!5e! z=lq2yG1=qX!txtXX--#CTb&Dj{1R8Q_Fbn*cXfPNf=6J2R@qR3$e za$mmvzSrG}r&t5;7C=O-psOX*QPeDd29oYwrs}QB72WW0leQTE-mJ2eOOVc;!#wY z^`d($b-TpvZ~|I7a`uGx3f$=-+s@yVlDDgpw5pPGD4bf&Yw}n1*o0uXrq42wRf5d{ z+CAi0L1};r@d)5$i}6SD&CtH*dn~*lSM{LT`S_;z=TE^?HJOKCs`YP$X6+P8GCudE zW!Q?V$y>T8%s>5LCS(RA!sIit?PC(@NyxW(T!iF3O(t+tY2$i_vG+Osb@5c6AjC%hT`M zMds*};VWXAn!A1VxDdk{?p#2aTTC7(TC;;m8RF_T|F7rGo)buoLa<0*oV*i?&MZ*o|a4Q%0W^L>EaCi~;r=OTV&ew=5nie)kY9b4y6u{#jUsykJo z{46mETHJ#w<%Pq1oxhY$`8td0)Wyv;*TsPenwZJjF^4)rmiYd!;+@w$Y?!R%ISalT z5H+S|XGyQ(5;pG$UQE9eg=w;Em zrSK(t%Zjg=4k(=_nt)_O3wg>prc{eMwS3NQ-EE_XTFL*V`+eDwVrnFxDnT`8eBjMLO1p|EhrB$ta;W|8?n#!a%{tvq594o zvR|E!AN?wdqe@k&i8L;&L?f`h@m|ze!M)c3f(M7xTBP-Rc>%6L2e%j`L%QxPqT425d5H?ucw2Kdaz|1Yo7_Aw_hdfr~k#B zb5_m8tMy8i@F5wCe;AgIF|@vT$afn7uYX{^NK*Q-LO$DR>Q{K(Ww~E;HLjiEuN2J|6xci;f65Li zFPgWfibC50!&Vf9G(}j}iXU`j5ar95_Su~qMBnTs$UMoKjcr{{X1uE6izILV2T&w zMZ_py3(Kc&YVQP}?ZKpQ0q{bmoY|z2)FWQb6o)9aABAmS=XFbSJ_852@};ynD4A>} z$%0xT|6?Ld+@^m@Kc-J{{t+&12CcN8)B2Q@(K708dXbP07t`@2;Z9zM$i>} z9jvXstt7jI^dv})yavYr74$HZ@7)NkDep?2U9XH=EPqI70h$T1x3zmyjKOQMeqYl} zEVLr|cLp2oEVd^oOSqaS;URP(G&=1W*9Kd%x$#ua<-;VwX9~`1pEA)c83VH5abBt1 z@>5dMIfMVm$eWE?Re6--E70*64OLHG-|=Ts#nj#uF#fZc0W_pfRki7qWV&sCxjVGV z%s1vieg3R>{IYPM>cjFXfS0k|I$nVy5`*KsYzy2+RE)Q_aVSN2f-jgiC_nee%aB{& zLV7K=YI%g{*j}q--~##{^<5&l8*o={JLF;O*$-IFt3>7415bpX?U5fRU`JiG2%iWF zjur!{u4?w534eS~SVaU_+{=84HA>L&jP0BwZ^TV!X+U4$oY-J`0sFnF?j*KL-XC9J zezzpOK(Qwa;yY?fCD$bh7AWd)Oypqjq*2bw=yZttru1XM!SW=cn`B8L<+Cx>#o7`~ zitU^7OQ`s?bisR`>lIV1s?dlT2QjO^2bbdN)gaz;asNsPv_FnK6eV{k9=XI^IN!-- z&<-dnJbmx7u#@+W&OSCa=uqItO!l4{j2BK^L&|ibr)lvcijNC~`(kBdBSbepLtjb| z=jOh4$VzP7F^bfiyKEk_vVJGe&w>WjaBjUj!;8HD?aNId>t#~yCPJaQ8dj{@C^H)tHuV_tdxqEYJIGkq%H0*`yYt=KxXfk z4bx+zdou+E%4GA+x5P49?l&88$qT^d0@;p1S1UU+5u3QpSe2MXH$X>*jq)C$ik&N4 zLA`J5;NPB4{Pb^=ew8y8sLCx&=1c)8&-K6IQ*DFLOuh;~tJGEZ$c+zRAHay|6DF@h zJv4Y7L%78IEr!|QRAIoB{v#Piw2=L6r!eSK+n_6`i{aRLQKC}VPH!> zE>`da0Q2K1pYi}#{?pC=Cj7@K(C(GAO)C<^Of$`USM^!oCwb5%K0`$*c3s9XFF7ML zk|9WO7HGIa&KurjCcMfLleMxjbPvogGClL^#W}s63D#Bv_PjjB^^ACZ$0yx6BHE683A?INL_G3B@+b7L z`W?1xj1Na&{s_Ru>H-@O?dVz~_i8Wwi9=!u8lVj7{>)`=XObmk2j0;%Yhm6{gD`ef zwv@3|un31DbCnAJKaPL9Nc*p4GUbOqYT1J@P4rhvVP4k6NaRYF<-|)t zL1}Hn<4VQYo>hmL7Qzh&J@jR2m&iHeOafR?uZQ@U2+hp-NUkQ|=la6VGo?E*c1G0c z1Ja&VUIVFf(lFbPfX!lpOuB=iq2>7gmUsRNfsXQh+cmqTY@!7+)RU|%8w1t>HGw1b z3sv>Y)Un56BLnk9bP3E{Rn)g%L(N7v)7Ltt!>H&Z`+W&5Gy6}~+*xkVSHQOYdnI|< zZCiKDGdJY89B6LjQ-qxj^Ic?L`^9bNqYF!=+O1!vQjb1wK=L;8*0*H27fYMPnQf3(x%mm)_7H% z&#S_}$>#e}rKF(p9*P-B!LOZ%{dn@B(2-ryR@{98Qx?TDXz7pPiL*&$pLCl{z}b9x zHnSPzrQ}&_E2DXStWr%P`ioVA{k=%xAf>?exbJ!uVJPV9A%4lQ1`d~3b~UA_MfbOr;88yW7*2@I>nQ$@8q!{ zMLjaKA^z{^wyAQpr^eoN%@*pNySKy65SaI}3MsCz5c0c8Wj$(b;}F8v4J?0^PhhXMqGB;t zGxom?TyujhBEsAY=8qK|8|XFU5C?t4RIrpeD81jFY{cELJ8*%pi1td+B@6WteYF`? zvDR_gp5w7x4uLMbYbek1mYC`wVNV zoKmSpy^tD-Ffwn=NuK>p7BQPV8@&kzy;)tVS}81hbt_CAau%Qel%eUH!~VU;=#3Jn zX#qN>i6}PlKiX4EA$$S(cj`vfZqSU=%Gm&C>;|O#n+NNNh zHC(x?$a{A5(v3Z9=o*{@;||V@H-J zml5+fRgYJEu4ThNb1PJyrKcgEdrr0O+3@#jfgRpj>gCKf$V})}+d8kGa0=PB6hV~S zENd6Ftu84=d^NPbH?@Sradjl2)xOTn*kPm-ZUSmwT&NK-+|OP~Y!mj(mzJANz3)_b zA?2EK6q4L8qkHyjP(3xH71sqasO&Y40#}}LjL}|*>jD0RA2i?!R4G?OXdNA0S9!bg zga+zZ3@X}V#E2A^Y?|&&iQrF#yN7|3k~t>bP@!M||HH%y&RDyGbE*-V&MN#)g0j&) z?qP+m<&dPR@q*i2)l+4~kU{P!VSty`Elnj*g|$kIf%K0ohi-sXc8T*V^!w*7L19uo zp50ot0ijoflySD=@C#NC1YapH`O_`}4M=@q^KqGPa|BNM$IY~+4n(cjPp#(ys0D(2 zM)7JF{_#8DB+JM}$;H3aY)Nu6k$|#LpEP}s$LV4ecI^TMsSQ*_oYOAW5N%mqLX4EpB3M#{%WEuZ9mrL zC30{AL#E(Vrwb$6G&nWPopCbHuPq!e3Vk_P=6og+=-cUuB+DG7OyRJkNchCWr>Xg2 z62!H{dF7d(IIY_W^VS(Z>zk6Z$6@%M*DLY~M{tN+OG^l+@re*5}6 zo^LFvVnVrBo0zF<#Ub$iA}KXJYQ}*|<;=4PCA)X2T5+8Z3kE<~PNl>jI+S8cCWXt2Ulkq%ECo zyfow*G%jmb<;eO16JOL0Q_&R1^WHuar6maAsMT?fz0oqaqJn%g7BAoei0!M#iJG3b+_bfibt^Q;sB`n0b8wjeO!0A^k^HD(zwG@+PsvZKROx zq3VIaJpLuO(ro->9hvEcl4clTK8VCF;w&9nG4y*b)og=SpSf8HNi;|&<6wt=Oxawn zkwlc8cyus~iMNh;6&tttw9z6w)a`tDey1A?sl@i8jPf9n+kVhKl~DV=s;60}52X~2 z<6~oliw1P_Z%9;j+?e?&#|q(pYv1S<=FI){56yESKhsneOq8>*;Br9Cvqpy~Q?~dH zcYEr4cT=3glfv`?+u7w(p7y_SGJ1njwQ7nny zcw%&009IY0*bL$ahYanOLpFiVfso8Sbq12yYOsRMN7k5gCx-y=nhsLwGai0QpRE(oqvKiy<92-7+zZ_ zCE}Y+HMC%X6fqtrzM{CbV*(fsVkWFTV2yAbzSlBVa!yQ3T~kRXFNyNgk!^~+B$j0) z0bz)YSvI5wUTpez^cKVBB9jlb9=t{c8TdZY`VR;Dw$~8Nz-Ht=RnXo z9-EXNNS6^W%8U&n>jUe^Ddw}(6>kORFKl{jFi>IC?i@WVij-#E+i3SGK`rY&`S4rn z+!MiXy-ryy-)ZXhS%}a@RTM@uQ?>w=;;;w7-EmE+geSE+0k=!m0PoeojAQO3nc>{_ zJWo0oAxroQAu#Md4Ajbu6xq0hVJ)CMnkm~X233l#J;$i#C`B=i240*`B35@0_pM%K zl#ezH<)1#b&l-PM(}it)rrSt;<2} za&J=79URWWdVx|@u|+GE_l69Eud6HGtz$xb|Io-UPV(5TkGwC{zLvP#IR3j4j5&oA>cS zB!w!oEXOeun+FpHSG?RdAZZTm2$S*roY_RR?pPPro&3eUE}uJUe(H6N^yvS}lF zqLHDzqIBI+nolUi^FOAQiSFv*;=-x?ov25GP4`uB-6w4EFbtT&-rtGLHwtLEim8DmACCoIHuja`+^I_ zUNKxfNv#tWH$v@KhCSb@hYoGECL|d z5(|5zsofwiGGgxoWw8JO6|!9J3h$o__mc(2~43C(#Hu*469efv<3VVEF<>9JP6+*gU>PduzY3fr*kF&#NUL>YmUb~kryL;22LYb3&*^8 z(u3q05JxbHk96jMSp!MZlkpx|tOR_IKo{ilSdOcRWj$9Z4yoXW?_k{3A|hJk^%O;& zj`uvxTc?iBLK^r@dumL7-u!KW$dc=Lh0VMD#ECQRG!}8>qDi-C_Foi06!PI_t$THE zi}aM#$@ysT0*@A^ZVH*iPdw;?)B7`6-=P-8E!xUTZ{9xNnfyPd){#BFjThIIBA2?R zq3tSLsaH~36!@LDI9bSpX`e~$Ihp~}WfrR3_Cl@&<@qz; zTUIVBFJ{b>=3pccnBA+cHu1({0@e2eFEc}a?3Y@#z77Sn9%%|Jumtd7zEFG-VRCZj zt%x_GaJK(Q*LgXC_9?_Z55*z(@zz$BN5ngW6{`PZVs(ZRz0Yhk#*PmKh_#%n<6i}N z1ZK+XSAbKs7k1!AvR0RaW0fQ=2he7#4jeNfx3XMAfY`j&Bk-HrK`(^1UU^q~c3#|w z^#rBa$`<|%9#a;}@0enb{n}Ttn4dofq#ZLLTIQ^fXu{ovf^WRPD?eGT6}mPz`8Lcn z-Q%#~|MRkhk>V-h?cdj{(6wztmT*DAm=%(wA9_^G|7t%zJ4e~tFSMe7X zl7+E35p>cNADCNnp>pIe4T=Ad5$l_G56*WysEfQa9_y(!X{G6EP;P_6;jj*kh^we# zF!-1`FLIs9cw(psb)@g8jWENVh~N+AK&DT<_$2^4$kqQ0|wq?%;>5oni8Fc~hBq_6fIm@e*=n@@>}t7*}w^$?GX zOtg)rw)aUlnr@zbBMhec7)!^^!_{JS2c5j$ zwlEQdZ$Rm_E&<5TSw$U{ z;&;^J$Z}h~I>}(eKGq58X$478Hn|`kIn-+A2p!Ar*}Ik^)#k6j*%pfOvyZvsG_pRhj&k;U7sDzXm)6?TN5-krbdN}^Si1j5tKrt_?J^$2 z^7+2+C6`EvQ93v)QF#7mmVrTwJ)E3*<;kB(J+z9m-GH_H4WqDc+Qb&UJH1J?g>SNq z?z8npg)^D*ae=OgeXGeuN2sG5T}%RkBwM)`ZrxcTXI@|Jm#T}{WC>YQ_z#86kMq$! zsQ4p`E`GTA6?Ko&No{|w?bYN#SdJzI*36(kK`dpJQmFjl7RZ5gYUP3ZD=8&tm zd}L21IJQZv?a7Kx6$?50PW}?bA4GW0z)`5r%1`;a8m*^8M#j}$2=y(7it|U@>R1)) zohlgnCO3;#R}Ak7jPL+2f4je6j-9zd0AwviIoynf}}p@tRy98UgSbEI;P@RVLNhL4Px9*L(zC= zsr`ZKvB(F!0#flgCi5k6%VJucXy3zzNiNH{@4FU^N6Cp)NpLRx49#eBy9!mBKrT}I z2L;jV?A&s(gkmKz(Ma)K6SIGO$fqPnLy7K8?UZ5t>(A{iC7Z<2>J4UdW-P`NMRPA< zL}($LFN*)JWfHsKe#L2dUJ3H~jTjbp4tlE(uf746oe&u!;tEi#4a>W-W2bv>_#0HK zqh!^>rw5TI<7y@%QCu6Dosm|>yJmyDXz$nbE=(KZ(AuNo^%9C|AZQ@_JctuQQPQ$t zdt=)dEqVo#?fNoXIMiLijZn#|_ zI(rE$&V@RMaOk`+FZ*mKe8qR9SEvsFY#Fi9Ao_`?twer^Bxlkgi^KU#u>NK`-8yuG z2e7}XPKiWZfxJG4-)m%l>@x%V!P7>$s#C3<7_E={JlLwgzr_lEq}lc?`2zcVYh*Xd z!rk5py;tWu&f|7~X)Ll%r|t5{Wm<)qBoZvjbRsgCdrHCA#H+X)>h)n&BPwhLUd;ig zW}P#&R8azI0&re%w@N1`Sll{}Zcs3O3o>^DOQT#%Vs3|tQ5U8vTh;w;Bi3Fit;2+v zX@IJ(S{DwqdppOYJ6Sn81Bir*u2SAl+QD&88scBtCxEzQn8^>Tcr7gKb zDfb$5VX>61m{C|K-Tu}c3Wwj{{eQotx<}6LmnX`l zmOWwv+6%A3HN~)S-=-%?4_bV+)aJk_o- zRq;tT#mLS^HucA|cl<0{r0_wdSJ3#TNzYuF#VxN$$;{ixrFqGvDbHFOuk06%pstQ? zVmU{xQF~769D+Taq%*JoVK7zhl0K{7b^L2EB?9-u$^G^>8 z2L1-0GHs6r&_qkVe^&XcG2-!c^yK>^#*NiK)PSaMcx}^*+g(4KZ?yh-`Q{So_u~`h zztJOqH-F|m+BIjtkRc*`cEv2|_>{SkHv9Xn*YDF4vA>^3{x1K#Uw=~*686_HFL3+c zEQ#lHS(Y;VU9pge(xVZcina5MpugujdVgN(xjy&uM5u50={m=geQ;Ho{+8O`pZ{vO z#QWq@?|;}1c$a{t_-bpA=$JBVt;djq#4-o%de_zx-fIIFSg~=j!Le8JD}! z4eoNKWZuvJL)as#nH?U89Ne+^*tBAQ{Q=UXI@R`(G4iRV{Hq5nGMByDSM?o4GJ%T- zX4Vn6si{KwxD%p*SqnRixe@q|kxDfI#d1&Oh4kMFR#!S#OlXg4sEv<>@44!6c~6Px zQ=crUi=SGr+P>C9exwel* ze!gf6OEW-isI+KM7rO5AO$a4?ueo9M7z*JGzZmu$ zbBOSXt_5?sjjXK+&dT9;^$+-6+g&3TyFv?;UM{K33Ep{v?Bjd(T~B0x>$bzk_UzA< z<8>(VfEr$R^3&BH;_@tM(5#78nJBgF{O^K=eA6<7 zVaUgdD`oQ;hiM@@4*(LM5?VO92*1+NOwF~mZiG8*jtabLU#bw}THtCca|nxt4tx8W zoU>?vPLNtQ-k8fYpR~avz)wd;*xB+UEY^cmHlO4Kxo!-5b{-~_x}mZw*()o(^=(H3 z3yq-sI(^HLKbFG2vRBqBt8m?qz}5D!nKL_xWUbF8(7@V>UG3PLMcj#zOPTfa4O+X# z*IiX~6Bb0mBvF$bZTP8r95G(`1}132TeFhZvp8Cj$;lrk-0MwquoZo;lJ3&swzOgy z!79;JRIKcT`HXPv&gIr1%S%NtF$f-`*<)OOEBwrqW@X`23dSSlT z#qRSqdvlaziu?=Ehu)DE-ZY2($TV9Q_ps-+R0&+6s+^(-)r(^_XrS#5RnheoNT#4F z(pg-p82iPB!`pQC!f~%_rn4_6b=Xn-53<=<2R`U`)gRL-u40N>i6#IWdU-JKa0%Fw zxLQ7daPGQ%LG`tNU0v?WuA_X+*PnDVcMQB+o-Xl|xZ*6?FY29ysSf3z5g*FB8~5`I zFuRA#GK?t()0VuG(b9tDkVX3jQ%#!Z)(=;2gP>56XA+Y|1%=t#Dl)9K7xsAs@gZNB zI7I~}g5*l79O^Sq{Zgo#-F0 zHIdI*EuP$#Mcy2>uok*3@}zR{zvYwKYBuSL4DXWCN`&OGaC0`3)yL1~DZdzJA#WxL z$G(WxdOhx*nj0d*t?x8ue6II%iOpbF?yVShZvX&uPli!vA`JDlQ2+84{Bu0=^q1(| zTd;2#ZI(xjb!aea<4|;y;DP!G_oc-$E7(}#NAw;L^Lv`T^^G|bPhPQdY@h%Mb+4O} zOHeZ?KA^j6^thl~;;FteYe8~zH!GA)2b6{>ok`p;YQLcCq^@7r(Bd*CW;zT^=#7c1 zj)8o65hzv3J}c?VS&&q`qa-=dr^Dldb2kPx5EW`EW?8*&o*ld9EURMbHxzFEKb#z6 z<8y3FF&5!>IwdVJK^%zK7rUkZ037SaN+ozPIWr-4D?ljT106QDSW3|eoP=^H;bIL7 zxnIzrPaWv3ozXu@+8URpSiR;l)E$!*z#H5}gP1(!zzPVMwnoyNq|#hwXz>(d24;g} zGqekbr_wITL@KGKX|Rt0MxV()+?RinDs8qIiekL<79$dxp%ZeoN=jr)jWH37lAh!*dTbZZA4(7YaA(T=5t zt!y`7ePu;ao(?N$Qv(VqOIZklnrbz%4+jFI7=a5GJX;SIr33|`t9TG=X;>cwdN<1<@fC(UxMvo5Go+Z;0fJqV9IJ0*wWU zV^wu44)UpdQHiLbCy|OVX+u`31`4R#T?rPaa48zNr`;i9MidOw?`@^RnT19$AiHyG z6Z6n@q)@%G+gh~?n#y>z7+9+XmU3jqa0*P&jv8n#C3xSvP5Dm16^g7&(t%4}3EDZa z@k>>!2jg5k0a&2KOXW)uMiPPJOb7{4N~JB|Maqk93dZJ+Ybyfv1q-GKOcZ{awN-F} zYJg^HS^@!zjFinepn~ypC4wSySYi+Hl=PSi0`u=~O~YO@3JL^tC<7|hcvj&_cAc)4 zY+a#U_dH-w5V}=y-c*kZF5Ll5k?s?2b8lc^Afk(N5{{re0W4Yo0I*w8on*Pf>|zS0 zWFrjH@-XK4bUg(JYnv7?BKub?=t62J6xE`Yr}XTM6s7D z)Y5ieAp`_4LW5TcXy;0T(i}rlg)AIpMI~m04O(7z%n=4?qzj8kw(OIJf z*aQRgF$Ac}W@O1OTqS-f-L4^se%BuvgvCKc)C!b@0{}hZrx61HK)gYM6>^hvoUjL% z>j*iPO9&hyLKw2mGa6hLzKT;48<<+ewe2%Ds$S}!;ydqbq#k4OoItQp#RIfu%Bo@3 z@RuK$A}VEK+9rb1aSAKnCs$da=wwn6+d~*91i@R6@S+OtRu2=pVUkHpVa>VUH+%~@AW^|f2jFV3cL_8G8tij zATTrRCHRm45TmJX(*2`Jl^81OS)3Ba6lK#ixJ;$g+c3(^!MF+#K+Hp_wq!0HHcud=2FrmqT_IuLDt&|EeAG{@59PF z7hiZ?d6!7BEXLPyUZvG_5h$hKK~VxGQ~-i%Luy@2h#88Bn&s6jBCZU>7jmY{mZ>zp z7%_Su=Ho`mt?3Xba&k2^<{V7Pq&F2um=zQ87`}*P$c%he3Wa58GX@~0E~|!RK#-bN z;!Q6TGa!jj-~KLKzKc8rWA$5Fv{V=#>D`9vjTRLaG97%zthovk<0O7ow`= zLSsrQrN$~(9*Et7Bae6Wj-dmIQ-(DGM{B=I1BHTOIu{+fH4viJL`LynQe@(%twH5b zlu5*lv1K65X|?R6JE$O!Gg#Y#h86(gp1N>p&@Q|)2+|H<+1Pj2$uH28DNwwg)(%D+ z9zNTaIU1#LX<=~W1a6ABXbnVc)~?pCax!TP)&N+wc*Zwe5L^MOYb+(pLfV!E&t~&S zX#6qx0VqTpw!=#%RX1db1Vc(Bi_-61ZQfg62yk z7ff0LmWzix4%G!Hj~3}$PQp1b3g|#!KS2b=R{(0Ygk(3Ef)-msg*9ufM>I?z=>!f` zHC>S_SxjIsEP0${q-~diI(M^&5qBmPCjmlXYLS*4!8d?<)c#bFnoB@4} zL>j-*SVu!a3kPcEy=Mm6#U%>}8BH8*f})yygONd{!($a#L{w&M?N}HY z7Rn%?N6pX*G9*ADVN!s%V_*nqfJH$n8)E@P+haxyfkK1;8fB5FDAyE$MuLs;8dF0k z9HUR;Mm)J?N=@!<(BL!>3v>MhsrNBu(9e_{OC?GoE;MD?F(ty0w?Z0*rKdbZTv24) zhK|s6MpwkLSqeJDXGMz@xUImr<}Tq_UKk19nei*;7NAW*M!G`fO~G+*O?*m+1Q#IK z(4@LblrYtgUkF9Z6)%}gDsEgA6U>BQUKqENQT^Q5PFT8vcE=p(q-afVEyD*;*@Tw? zDZ!M2_{7{tcg4Xr=n@PX4r0d-A8wV~g=4%I_dy3y=-Mq$3}h~txtBF7L}ixZM@uC{ ztb(A>W93N1v%2f%1M_y({-VNQ&3hOGHkm3gSjz!-6{=5dUNEp~Q8BVK)Dl;f*cb}R ztW9U02<4ZJ+P4EV%^X1@PR;>fvabjWNL|DY0Jfqhk^VsJEnBU$4siuXx@%A%Drj`m z4v?hOd8HK#qZ25})SUBHA)JQGW)lDE7AFM?3I0p!!q_EB#i=bn(HjHke4*N=I zvWq}PEi9+SEs3v)DsU~Ui)u1PYzsjjM&}Fi0fNS*`~D~@Ql~i zbkY;{uoPOrP*^Ul%EU8UdB!waW(`bSAY>61d1zy8)GEV^L?A^$q(!1^?p5_aV~!@d zQdAC6LcrE@a+Wm`C>e6;9IDD#KTL7E2zOTE;iMZWM3Jf~0~!@Nj#a`1XN_39&;+vY zAmQFcSSH{c{S5FB7F{xlqYD{)E$0vxmQYd1z71X16%@LmuzzvdDtEb0;+!!Op|CIy zTg^|`0=mK6M;c3dz3-3*{{TI;Du>pajzx#k4I+2APk;C!PjIWas*R4I!zpaR8@W$- zSY65XQ-n3;|DmVtcB5t37P zY`d(+80FMvW-oY+#7u;-gkvz+z@_Rmw;QtVpvDkMib6Nc{v#`vG{{V_L zN_h!OaV+3~#YAZmUS$d4nBF1VM8+^);oBIos0xJbViAhqU34I6A>G`*$P7`3T&&e#f ziIO1R?69b`EusSmRl`A87SK>zDAZj#gEdF+j7{DQ8FJ|2?-oZBZD3M_5i&)IL5s>o zK$t~um{~vzTNV^|A#m`fW)D`wV2^ZEKn-ytmU;cBPHPyB3N>9 zDh7hV$gW)J;$z#72C!BIFub|UcJ{voT2X3?SZKr!#aM%+DGO;!ErR!k7x7iz4q}pN zN_DF3TYRM(uM%2!g|wxY8I4G&+BOP>(7GW@!9-vyH9_!7qK2jbSgX1m@exCdK@=@8 z#ur?~+PAGn)%ME2a~&BvnKd4ZgB&_9p~Ir$n@jzuFvD^4mvf$(jKGO~ab@l-Ot^^< zmn@b`%v4LhAhKVyxTBN?TupMdEG01cOGtBBnqnmq@h&WsrMntwxn~d^k|4BxF&>i8D+=u`7Z_ST zvgT6&ZevK+?pZ`V*pP8~mVBP#yc68YkJb{xy}=}1<|tXPUDT*0+=+5)+)=~zf;IZU zlnGk6+8U^S7kHLJF`Rf}u1_(@Y&$|I0zryuXaeIj4O9?79f@RK74d?oo_FMwFrVLu z%c1>yL){;*>Y4ulx$y=1A87ui_=_Lf_?Jpo0RATsuT=@#_G91TMIw)cO+xraGM>aG z@=g3k2rX~^5LNWVhllBMZuI`8hz#%}jjzTqzq610T>k)x$NJOyhrh#_RDT%f z`0&m6aE8B(v0rZ&{W<3f7<|4fKGYM##PcmdD7(0m|Fn3u>L=7>Zh`-5wJ}+9TWpZMcey zxGQbAo4dFKw493y45z3^XtLuO+-y-M<6aSYxSitWmJXK1MJ0fDYfz+nW1w6@%xtk0 z9GJ(2OO{)xhMlt$M=;gci*rHULG>Z4QnN01GfUX0)+^h@+6v6PS`H;DX4z9+CRDU9 z606Ka4p78RPrL)pJtHmA$%>nXGRUcv%FVSjKlk%yQx>XPD2vP;^U(4|Pb55}j*_u1IO#|$xOq%F zo=Ij&o>~&Ld4%G)4FXJ!<}1V!hLFLiswjX2P1nrD&0C}unqW52RJ>kUx`)(eHAtBNgu!MV({rf}%hG3|}Gc{*sAu58wH1RU9!)=zi znc!E3BdDzx8oIZ^fTs$kDnEix#~Rr=frt`bJQ{VpGjBheWHi;J|;u;J|>sydY1mD+Fbolq-d|)^$sm(pddZq!;Raq5CJ|CHkLeCO=cuKV7T^? zllY&qe-rjk;yd(X{?tQ20&%_t?KG9Wv z=ZS*-uTf2Y=ZUt}`f)4OJb%g)4f?p3P5%G@gb#DS{H5R8`a#GK+lgElX;t=u2lu={ z_x|0YN&B9mRC!_<4Wy|7L~ejfnvK|Cmr|$z8|vNZVhkCgv3RB-CQzweVVQIm0eZaJ z$(ch!PfH1g$QlCPVYOzE0p+a|SE2wshM?48$y|NqDyky1rp@BXai);M#=DR)@l;e{ zOWxz^V#>tuyO{~JS$>@}%Vsmu?qRW*r!}Jvun`Ah=US-4_EQ1{^4z!dFdD-qUBYgy zD1GBlBfXdxcC|mp%<-G;k6YYfFvd8>G(T$&U(mwhzLsE&J&-ixfPY#zVq9jA;6g!L z5l``<{{Rs!q#++m1aD_-1(V}H;t@w$Llh6OmA`>M;xT{dbWMNJ ziT?mXC;Agt;Kr}wLVpPl_|AbY{w%fpS!?(+!hB=@03koXPxwT${0YgU);50@Kk%Ir znGg6$c0s?Z9sLN0^e6a9{ziY9pW|g8;`B%O+5R?vji2LZ_}NGJy%2}=B2VJS{sw=6 zpXXs;@MCxIXZe}_MnCd1{0y{y3~T%d55!}C@L)Uo7{BVz_{@Lj;r{?TKjSg~0GFTS z=lMAQ0Kw1ryv6)j1Nd_D_%r+*{{S~Ke-?j>nEwEWE@$as{{RClaC*jX=wVTRLmcSy zth4-Bw$tN3$ItQe{QU0MtOFtOj$`Ox9iJE*m&lcN2V`|EIL1CH76{=J?hr((Bwwqd z0JI%Tp{!ppa>lc#Vvn1IhW-c|E8&6UEN4N1Ez@eV(k6a*lA&>YiwlNYTLgI$HmogB zVHa?fMXM}f0Cb=?qNTZ*w;POC1lwFfA~@}Qz!2Ta9p#}Bm+r}!NoIIoF3~jKx~Lo7 zmk{cF<*T)CQnPP~3ljZ6O5xZvwT#~KpbK@GovFs0Cd4TMiKY@#D&@#Nk#9F#02J|= zqG_HE2v|MlQ@Jg}h#Div%KTiXcoGPM7>q7qF`+MGpFo^!qRedb3Y`m9VapvQtDvE= zml2)1Od__T310DwnS{!1539;u!KhcKCNWt_X^SagC5;mLto#1}>+ISvvFMcl0MF8U zLpk4c43M0n;x3HiX>#K9a+FD=VANTeOH_Sng`gIIRp23y)%rvCU&MV;N^`mZEmB#G zJ)=nC#^L}lsGtFAfe;Hhj7%60ha{^})RU2+kuoM8I_utXP!s~v&;u$2yNO|lV8SVe zoWj8hN3;kK4|uc+YAW6csA$v`0#ewuS)vtIpsB}VA-EjH!*X_thO8TfUg6R%Y%Vdx z!Esf85NVzW7FIdnfKg^5mPT^Htc3+#jO%%H_*4MkBiB|^j`3}(w1=0KnTsn3MXTCH zm1ujF8FsrIwFfHMPpQgU?UD3 zfF0)BLA2hvMOBqn{lG+K0)+}eHdvy>hBY%+(lRHS6h$U6XtTh?!VMPM&7diBTR`~x zp&Hgp%77l!n5m9JvOBt0UC>J!a63fHQQ6og5^ZfWT2_Q8b0vVV=((Vz(t_oNT4*{# zwO*-62w8Nt_$%;e3>?wUex~isQoUKG9&*BDbef368FtDMuzJiYRCITTj38bjNc9Fu ztSf-ZLNtnD(tVgk_6Scd~7)aa%yL)vkqOk5cs2>1D_N0E6~P_MS4*^8M$u zVasz4VJAVh3Jzh=QKq_X<;#~fn_Rny3XW2fCS64(6q728J7G_jVP28aq5Nm)AT*`N zbKCT+8A{;GosGKVZaBCvFbD(4vdTXBh8MLNJa2)K@}PC&=T-w^ha@}7Ca^-BJA2}Y zsunot+nJFdqZY=CN}#ALq7|uDS#>Kds8Ag+Y&Zw&1p*aQX>q7hix@l(pizRQow0Zj zB?YjtxZ#2M0};uIyZhg#9p4+KLERiK3m z�whag_`dtS1M$($R51tV`C%6SBuIEV=~%3#;2VlP4F35w+ zedsHIu?z)g3|+%EGD%i7DF`jrn~Q2#wRvMg?*uCI>@Y+%~cNF9Gw=_)oV zh;LQ4;GYTADy3!`rQEzjyi(x2fZ#5JfFdi}8cR*gJDH^~MG#S&<#d=ja@?vN<&Q|l zGQrV2=d^$DFG!~{&pjufPrN*){STC3=;bwxxo@jkv}4<*<4&Jh>(f~D{pJ$Tw=>_b z@6$@@k4FilQHvbm9-nD(=u@EF$rXzq_!=0@8Y8`0U0uRJ0XF4ZfqwMu8K5rbL^1$; zgT#Ivo{DA#fE6swUENS_Ey}H7P|9U!S{fG+GRGdv_yJrC<^)TJxGO=shq+-@2w`a; zDTTUNy4bm3lmgAu4?Klx1yGonFPuTx8PLKDm8jt^s}igh2azse8bd(+Eis0w0^}r7 zslsV(gqxP%bq{SqT`Z1xE#&M%+byGsdW8ap#G8Du-Dv2`BLRgph}+tPaZ%+_-KcXb zvke%hfI~MfmRs8|3{thm!~mDLTU?GAN&ukkD+qN5*$Jl-))tzg2ZbY?#AeB14@@gQ zaWbPZA(X!eD>+_$0W6b|He=>v)WTX{%(PP%FD(^P@$1PHxBfk)rTT9@{{ZkV`dTbm zhf56nr^AI+{{X Date: Thu, 4 Feb 2021 16:41:29 -0500 Subject: [PATCH 166/190] clean up use of word: simply --- .../manage-deployment.md | 8 +- .../docs/concepts/configuration/configmap.md | 2 +- .../en/docs/concepts/configuration/secret.md | 2 +- .../api-extension/custom-resources.md | 4 +- .../compute-storage-net/network-plugins.md | 2 +- .../extend-kubernetes/service-catalog.md | 2 +- .../overview/working-with-objects/labels.md | 2 +- .../concepts/policy/pod-security-policy.md | 2 +- .../scheduling-eviction/assign-pod-node.md | 2 +- .../scheduler-perf-tuning.md | 2 +- .../services-networking/dns-pod-service.md | 6 +- .../concepts/services-networking/service.md | 4 +- .../concepts/storage/persistent-volumes.md | 2 +- .../docs/concepts/storage/storage-classes.md | 2 +- .../concepts/storage/volume-pvc-datasource.md | 2 +- .../workloads/controllers/deployment.md | 25 +- .../controllers/replicationcontroller.md | 8 +- .../participate/roles-and-responsibilities.md | 2 +- .../admission-controllers.md | 2 +- .../access-authn-authz/authentication.md | 6 +- .../reference/using-api/server-side-apply.md | 6 +- .../tools/kubeadm/create-cluster-kubeadm.md | 2 +- .../windows/intro-windows-in-kubernetes.md | 2 +- .../windows/user-guide-windows-containers.md | 2 +- .../access-cluster.md | 2 +- .../administer-cluster/access-cluster-api.md | 2 +- .../access-cluster-services.md | 2 +- .../change-default-storage-class.md | 2 +- .../configure-service-account.md | 8 +- .../translate-compose-kubernetes.md | 321 +++++++++++------- .../debug-service.md | 2 +- .../tasks/extend-kubectl/kubectl-plugins.md | 32 +- .../job/parallel-processing-expansion.md | 2 +- .../run-application/delete-stateful-set.md | 2 +- .../run-replicated-stateful-application.md | 2 +- .../en/docs/tutorials/clusters/apparmor.md | 3 +- .../basic-stateful-set.md | 20 +- .../stateless-application/guestbook.md | 2 +- 38 files changed, 283 insertions(+), 218 deletions(-) diff --git a/content/en/docs/concepts/cluster-administration/manage-deployment.md b/content/en/docs/concepts/cluster-administration/manage-deployment.md index d9814a887a..fa31fbd35f 100644 --- a/content/en/docs/concepts/cluster-administration/manage-deployment.md +++ b/content/en/docs/concepts/cluster-administration/manage-deployment.md @@ -45,7 +45,7 @@ kubectl apply -f https://k8s.io/examples/application/nginx/ `kubectl` will read any files with suffixes `.yaml`, `.yml`, or `.json`. -It is a recommended practice to put resources related to the same microservice or application tier into the same file, and to group all of the files associated with your application in the same directory. If the tiers of your application bind to each other using DNS, then you can then simply deploy all of the components of your stack en masse. +It is a recommended practice to put resources related to the same microservice or application tier into the same file, and to group all of the files associated with your application in the same directory. If the tiers of your application bind to each other using DNS, you can deploy all of the components of your stack together. A URL can also be specified as a configuration source, which is handy for deploying directly from configuration files checked into github: @@ -265,7 +265,7 @@ For a more concrete example, check the [tutorial of deploying Ghost](https://git ## Updating labels Sometimes existing pods and other resources need to be relabeled before creating new resources. This can be done with `kubectl label`. -For example, if you want to label all your nginx pods as frontend tier, simply run: +For example, if you want to label all your nginx pods as frontend tier, run: ```shell kubectl label pods -l app=nginx tier=fe @@ -411,7 +411,7 @@ and ## Disruptive updates -In some cases, you may need to update resource fields that cannot be updated once initialized, or you may just want to make a recursive change immediately, such as to fix broken pods created by a Deployment. To change such fields, use `replace --force`, which deletes and re-creates the resource. In this case, you can simply modify your original configuration file: +In some cases, you may need to update resource fields that cannot be updated once initialized, or you may just want to make a recursive change immediately, such as to fix broken pods created by a Deployment. To change such fields, use `replace --force`, which deletes and re-creates the resource. In this case, you can modify your original configuration file: ```shell kubectl replace -f https://k8s.io/examples/application/nginx/nginx-deployment.yaml --force @@ -448,7 +448,7 @@ kubectl scale deployment my-nginx --current-replicas=1 --replicas=3 deployment.apps/my-nginx scaled ``` -To update to version 1.16.1, simply change `.spec.template.spec.containers[0].image` from `nginx:1.14.2` to `nginx:1.16.1`, with the kubectl commands we learned above. +To update to version 1.16.1, change `.spec.template.spec.containers[0].image` from `nginx:1.14.2` to `nginx:1.16.1` using the previous kubectl commands. ```shell kubectl edit deployment/my-nginx diff --git a/content/en/docs/concepts/configuration/configmap.md b/content/en/docs/concepts/configuration/configmap.md index 9a134dfc99..ad735388a0 100644 --- a/content/en/docs/concepts/configuration/configmap.md +++ b/content/en/docs/concepts/configuration/configmap.md @@ -225,7 +225,7 @@ The kubelet checks whether the mounted ConfigMap is fresh on every periodic sync However, the kubelet uses its local cache for getting the current value of the ConfigMap. The type of the cache is configurable using the `ConfigMapAndSecretChangeDetectionStrategy` field in the [KubeletConfiguration struct](https://github.com/kubernetes/kubernetes/blob/{{< param "docsbranch" >}}/staging/src/k8s.io/kubelet/config/v1beta1/types.go). -A ConfigMap can be either propagated by watch (default), ttl-based, or simply redirecting +A ConfigMap can be either propagated by watch (default), ttl-based, or by redirecting all requests directly to the API server. As a result, the total delay from the moment when the ConfigMap is updated to the moment when new keys are projected to the Pod can be as long as the kubelet sync period + cache diff --git a/content/en/docs/concepts/configuration/secret.md b/content/en/docs/concepts/configuration/secret.md index 5a6a0dd092..45cf9297ca 100644 --- a/content/en/docs/concepts/configuration/secret.md +++ b/content/en/docs/concepts/configuration/secret.md @@ -669,7 +669,7 @@ The kubelet checks whether the mounted secret is fresh on every periodic sync. However, the kubelet uses its local cache for getting the current value of the Secret. The type of the cache is configurable using the `ConfigMapAndSecretChangeDetectionStrategy` field in the [KubeletConfiguration struct](https://github.com/kubernetes/kubernetes/blob/{{< param "docsbranch" >}}/staging/src/k8s.io/kubelet/config/v1beta1/types.go). -A Secret can be either propagated by watch (default), ttl-based, or simply redirecting +A Secret can be either propagated by watch (default), ttl-based, or by redirecting all requests directly to the API server. As a result, the total delay from the moment when the Secret is updated to the moment when new keys are projected to the Pod can be as long as the kubelet sync period + cache diff --git a/content/en/docs/concepts/extend-kubernetes/api-extension/custom-resources.md b/content/en/docs/concepts/extend-kubernetes/api-extension/custom-resources.md index dcfef3f6b6..5457dc9204 100644 --- a/content/en/docs/concepts/extend-kubernetes/api-extension/custom-resources.md +++ b/content/en/docs/concepts/extend-kubernetes/api-extension/custom-resources.md @@ -31,7 +31,7 @@ Once a custom resource is installed, users can create and access its objects usi ## Custom controllers -On their own, custom resources simply let you store and retrieve structured data. +On their own, custom resources let you store and retrieve structured data. When you combine a custom resource with a *custom controller*, custom resources provide a true _declarative API_. @@ -120,7 +120,7 @@ Kubernetes provides two ways to add custom resources to your cluster: Kubernetes provides these two options to meet the needs of different users, so that neither ease of use nor flexibility is compromised. -Aggregated APIs are subordinate API servers that sit behind the primary API server, which acts as a proxy. This arrangement is called [API Aggregation](/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation/) (AA). To users, it simply appears that the Kubernetes API is extended. +Aggregated APIs are subordinate API servers that sit behind the primary API server, which acts as a proxy. This arrangement is called [API Aggregation](/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation/) (AA). To users, the Kubernetes API appears extended. CRDs allow users to create new types of resources without adding another API server. You do not need to understand API Aggregation to use CRDs. diff --git a/content/en/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins.md b/content/en/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins.md index 7b53fa326f..0ec8bf81b1 100644 --- a/content/en/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins.md +++ b/content/en/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins.md @@ -24,7 +24,7 @@ Network plugins in Kubernetes come in a few flavors: The kubelet has a single default network plugin, and a default network common to the entire cluster. It probes for plugins when it starts up, remembers what it finds, and executes the selected plugin at appropriate times in the pod lifecycle (this is only true for Docker, as CRI manages its own CNI plugins). There are two Kubelet command line parameters to keep in mind when using plugins: * `cni-bin-dir`: Kubelet probes this directory for plugins on startup -* `network-plugin`: The network plugin to use from `cni-bin-dir`. It must match the name reported by a plugin probed from the plugin directory. For CNI plugins, this is simply "cni". +* `network-plugin`: The network plugin to use from `cni-bin-dir`. It must match the name reported by a plugin probed from the plugin directory. For CNI plugins, this is `cni`. ## Network Plugin Requirements diff --git a/content/en/docs/concepts/extend-kubernetes/service-catalog.md b/content/en/docs/concepts/extend-kubernetes/service-catalog.md index 3aa9675788..af0271d9ab 100644 --- a/content/en/docs/concepts/extend-kubernetes/service-catalog.md +++ b/content/en/docs/concepts/extend-kubernetes/service-catalog.md @@ -26,7 +26,7 @@ Fortunately, there is a cloud provider that offers message queuing as a managed A cluster operator can setup Service Catalog and use it to communicate with the cloud provider's service broker to provision an instance of the message queuing service and make it available to the application within the Kubernetes cluster. The application developer therefore does not need to be concerned with the implementation details or management of the message queue. -The application can simply use it as a service. +The application can access the message queue as a service. ## Architecture diff --git a/content/en/docs/concepts/overview/working-with-objects/labels.md b/content/en/docs/concepts/overview/working-with-objects/labels.md index 2feec43801..7ff6f267a0 100644 --- a/content/en/docs/concepts/overview/working-with-objects/labels.md +++ b/content/en/docs/concepts/overview/working-with-objects/labels.md @@ -98,7 +98,7 @@ For both equality-based and set-based conditions there is no logical _OR_ (`||`) ### _Equality-based_ requirement _Equality-_ or _inequality-based_ requirements allow filtering by label keys and values. Matching objects must satisfy all of the specified label constraints, though they may have additional labels as well. -Three kinds of operators are admitted `=`,`==`,`!=`. The first two represent _equality_ (and are simply synonyms), while the latter represents _inequality_. For example: +Three kinds of operators are admitted `=`,`==`,`!=`. The first two represent _equality_ (and are synonyms), while the latter represents _inequality_. For example: ``` environment = production diff --git a/content/en/docs/concepts/policy/pod-security-policy.md b/content/en/docs/concepts/policy/pod-security-policy.md index 17f30906bf..f355a8f539 100644 --- a/content/en/docs/concepts/policy/pod-security-policy.md +++ b/content/en/docs/concepts/policy/pod-security-policy.md @@ -197,7 +197,7 @@ alias kubectl-user='kubectl --as=system:serviceaccount:psp-example:fake-user -n ### Create a policy and a pod Define the example PodSecurityPolicy object in a file. This is a policy that -simply prevents the creation of privileged pods. +prevents the creation of privileged pods. The name of a PodSecurityPolicy object must be a valid [DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names). diff --git a/content/en/docs/concepts/scheduling-eviction/assign-pod-node.md b/content/en/docs/concepts/scheduling-eviction/assign-pod-node.md index abe4f4b9eb..b6b2bd79a8 100644 --- a/content/en/docs/concepts/scheduling-eviction/assign-pod-node.md +++ b/content/en/docs/concepts/scheduling-eviction/assign-pod-node.md @@ -261,7 +261,7 @@ for performance and security reasons, there are some constraints on topologyKey: and `preferredDuringSchedulingIgnoredDuringExecution`. 2. For pod anti-affinity, empty `topologyKey` is also not allowed in both `requiredDuringSchedulingIgnoredDuringExecution` and `preferredDuringSchedulingIgnoredDuringExecution`. -3. For `requiredDuringSchedulingIgnoredDuringExecution` pod anti-affinity, the admission controller `LimitPodHardAntiAffinityTopology` was introduced to limit `topologyKey` to `kubernetes.io/hostname`. If you want to make it available for custom topologies, you may modify the admission controller, or simply disable it. +3. For `requiredDuringSchedulingIgnoredDuringExecution` pod anti-affinity, the admission controller `LimitPodHardAntiAffinityTopology` was introduced to limit `topologyKey` to `kubernetes.io/hostname`. If you want to make it available for custom topologies, you may modify the admission controller, or disable it. 4. Except for the above cases, the `topologyKey` can be any legal label-key. In addition to `labelSelector` and `topologyKey`, you can optionally specify a list `namespaces` diff --git a/content/en/docs/concepts/scheduling-eviction/scheduler-perf-tuning.md b/content/en/docs/concepts/scheduling-eviction/scheduler-perf-tuning.md index 932e076dfc..7936f9dedc 100644 --- a/content/en/docs/concepts/scheduling-eviction/scheduler-perf-tuning.md +++ b/content/en/docs/concepts/scheduling-eviction/scheduler-perf-tuning.md @@ -107,7 +107,7 @@ value being calculated based on the cluster size. There is also a hardcoded minimum value of 50 nodes. {{< note >}}In clusters with less than 50 feasible nodes, the scheduler still -checks all the nodes, simply because there are not enough feasible nodes to stop +checks all the nodes because there are not enough feasible nodes to stop the scheduler's search early. In a small cluster, if you set a low value for `percentageOfNodesToScore`, your diff --git a/content/en/docs/concepts/services-networking/dns-pod-service.md b/content/en/docs/concepts/services-networking/dns-pod-service.md index 93474f24fa..f02aede0e1 100644 --- a/content/en/docs/concepts/services-networking/dns-pod-service.md +++ b/content/en/docs/concepts/services-networking/dns-pod-service.md @@ -25,9 +25,9 @@ assigned a DNS name. By default, a client Pod's DNS search list will include the Pod's own namespace and the cluster's default domain. This is best illustrated by example: -Assume a Service named `foo` in the Kubernetes namespace `bar`. A Pod running -in namespace `bar` can look up this service by simply doing a DNS query for -`foo`. A Pod running in namespace `quux` can look up this service by doing a +Assume a Service named `foo` in the Kubernetes namespace `bar`. A Pod running +in namespace `bar` can look up this service by querying a DNS service for +`foo`. A Pod running in namespace `quux` can look up this service by doing a DNS query for `foo.bar`. The following sections detail the supported record types and layout that is diff --git a/content/en/docs/concepts/services-networking/service.md b/content/en/docs/concepts/services-networking/service.md index 348c1616ce..28d991e891 100644 --- a/content/en/docs/concepts/services-networking/service.md +++ b/content/en/docs/concepts/services-networking/service.md @@ -430,7 +430,7 @@ Services by their DNS name. For example, if you have a Service called `my-service` in a Kubernetes namespace `my-ns`, the control plane and the DNS Service acting together create a DNS record for `my-service.my-ns`. Pods in the `my-ns` namespace -should be able to find it by simply doing a name lookup for `my-service` +should be able to find the service by doing a name lookup for `my-service` (`my-service.my-ns` would also work). Pods in other namespaces must qualify the name as `my-service.my-ns`. These names @@ -1163,7 +1163,7 @@ rule kicks in, and redirects the packets to the proxy's own port. The "Service proxy" chooses a backend, and starts proxying traffic from the client to the backend. This means that Service owners can choose any port they want without risk of -collision. Clients can simply connect to an IP and port, without being aware +collision. Clients can connect to an IP and port, without being aware of which Pods they are actually accessing. #### iptables diff --git a/content/en/docs/concepts/storage/persistent-volumes.md b/content/en/docs/concepts/storage/persistent-volumes.md index 3423c575db..ef46b7f99a 100644 --- a/content/en/docs/concepts/storage/persistent-volumes.md +++ b/content/en/docs/concepts/storage/persistent-volumes.md @@ -487,7 +487,7 @@ The following volume types support mount options: * VsphereVolume * iSCSI -Mount options are not validated, so mount will simply fail if one is invalid. +Mount options are not validated. If a mount option is invalid, the mount fails. In the past, the annotation `volume.beta.kubernetes.io/mount-options` was used instead of the `mountOptions` attribute. This annotation is still working; however, diff --git a/content/en/docs/concepts/storage/storage-classes.md b/content/en/docs/concepts/storage/storage-classes.md index e6846c7ea4..6834977d70 100644 --- a/content/en/docs/concepts/storage/storage-classes.md +++ b/content/en/docs/concepts/storage/storage-classes.md @@ -149,7 +149,7 @@ mount options specified in the `mountOptions` field of the class. If the volume plugin does not support mount options but mount options are specified, provisioning will fail. Mount options are not validated on either -the class or PV, so mount of the PV will simply fail if one is invalid. +the class or PV. If a mount option is invalid, the PV mount fails. ### Volume Binding Mode diff --git a/content/en/docs/concepts/storage/volume-pvc-datasource.md b/content/en/docs/concepts/storage/volume-pvc-datasource.md index ac8d16041d..8210df661c 100644 --- a/content/en/docs/concepts/storage/volume-pvc-datasource.md +++ b/content/en/docs/concepts/storage/volume-pvc-datasource.md @@ -24,7 +24,7 @@ The {{< glossary_tooltip text="CSI" term_id="csi" >}} Volume Cloning feature add A Clone is defined as a duplicate of an existing Kubernetes Volume that can be consumed as any standard Volume would be. The only difference is that upon provisioning, rather than creating a "new" empty Volume, the back end device creates an exact duplicate of the specified Volume. -The implementation of cloning, from the perspective of the Kubernetes API, simply adds the ability to specify an existing PVC as a dataSource during new PVC creation. The source PVC must be bound and available (not in use). +The implementation of cloning, from the perspective of the Kubernetes API, adds the ability to specify an existing PVC as a dataSource during new PVC creation. The source PVC must be bound and available (not in use). Users need to be aware of the following when using this feature: diff --git a/content/en/docs/concepts/workloads/controllers/deployment.md b/content/en/docs/concepts/workloads/controllers/deployment.md index d8e7646aed..9ba746c2ad 100644 --- a/content/en/docs/concepts/workloads/controllers/deployment.md +++ b/content/en/docs/concepts/workloads/controllers/deployment.md @@ -47,7 +47,7 @@ In this example: * A Deployment named `nginx-deployment` is created, indicated by the `.metadata.name` field. * The Deployment creates three replicated Pods, indicated by the `.spec.replicas` field. * The `.spec.selector` field defines how the Deployment finds which Pods to manage. - In this case, you simply select a label that is defined in the Pod template (`app: nginx`). + In this case, you select a label that is defined in the Pod template (`app: nginx`). However, more sophisticated selection rules are possible, as long as the Pod template itself satisfies the rule. @@ -171,13 +171,15 @@ Follow the steps given below to update your Deployment: ```shell kubectl --record deployment.apps/nginx-deployment set image deployment.v1.apps/nginx-deployment nginx=nginx:1.16.1 ``` - or simply use the following command: - + + or use the following command: + ```shell kubectl set image deployment/nginx-deployment nginx=nginx:1.16.1 --record ``` - The output is similar to this: + The output is similar to: + ``` deployment.apps/nginx-deployment image updated ``` @@ -188,7 +190,8 @@ Follow the steps given below to update your Deployment: kubectl edit deployment.v1.apps/nginx-deployment ``` - The output is similar to this: + The output is similar to: + ``` deployment.apps/nginx-deployment edited ``` @@ -200,10 +203,13 @@ Follow the steps given below to update your Deployment: ``` The output is similar to this: + ``` Waiting for rollout to finish: 2 out of 3 new replicas have been updated... ``` + or + ``` deployment "nginx-deployment" successfully rolled out ``` @@ -212,10 +218,11 @@ Get more details on your updated Deployment: * After the rollout succeeds, you can view the Deployment by running `kubectl get deployments`. The output is similar to this: - ``` - NAME READY UP-TO-DATE AVAILABLE AGE - nginx-deployment 3/3 3 3 36s - ``` + + ```ini + NAME READY UP-TO-DATE AVAILABLE AGE + nginx-deployment 3/3 3 3 36s + ``` * Run `kubectl get rs` to see that the Deployment updated the Pods by creating a new ReplicaSet and scaling it up to 3 replicas, as well as scaling down the old ReplicaSet to 0 replicas. diff --git a/content/en/docs/concepts/workloads/controllers/replicationcontroller.md b/content/en/docs/concepts/workloads/controllers/replicationcontroller.md index a6427bedb3..23d87f81fd 100644 --- a/content/en/docs/concepts/workloads/controllers/replicationcontroller.md +++ b/content/en/docs/concepts/workloads/controllers/replicationcontroller.md @@ -180,16 +180,16 @@ delete`](/docs/reference/generated/kubectl/kubectl-commands#delete). Kubectl wi for it to delete each pod before deleting the ReplicationController itself. If this kubectl command is interrupted, it can be restarted. -When using the REST API or go client library, you need to do the steps explicitly (scale replicas to +When using the REST API or Go client library, you need to do the steps explicitly (scale replicas to 0, wait for pod deletions, then delete the ReplicationController). -### Deleting just a ReplicationController +### Deleting only a ReplicationController You can delete a ReplicationController without affecting any of its pods. Using kubectl, specify the `--cascade=false` option to [`kubectl delete`](/docs/reference/generated/kubectl/kubectl-commands#delete). -When using the REST API or go client library, simply delete the ReplicationController object. +When using the REST API or Go client library, you can delete the ReplicationController object. Once the original is deleted, you can create a new ReplicationController to replace it. As long as the old and new `.spec.selector` are the same, then the new one will adopt the old pods. @@ -240,7 +240,7 @@ Pods created by a ReplicationController are intended to be fungible and semantic ## Responsibilities of the ReplicationController -The ReplicationController simply ensures that the desired number of pods matches its label selector and are operational. Currently, only terminated pods are excluded from its count. In the future, [readiness](https://issue.k8s.io/620) and other information available from the system may be taken into account, we may add more controls over the replacement policy, and we plan to emit events that could be used by external clients to implement arbitrarily sophisticated replacement and/or scale-down policies. +The ReplicationController ensures that the desired number of pods matches its label selector and are operational. Currently, only terminated pods are excluded from its count. In the future, [readiness](https://issue.k8s.io/620) and other information available from the system may be taken into account, we may add more controls over the replacement policy, and we plan to emit events that could be used by external clients to implement arbitrarily sophisticated replacement and/or scale-down policies. The ReplicationController is forever constrained to this narrow responsibility. It itself will not perform readiness nor liveness probes. Rather than performing auto-scaling, it is intended to be controlled by an external auto-scaler (as discussed in [#492](https://issue.k8s.io/492)), which would change its `replicas` field. We will not add scheduling policies (for example, [spreading](https://issue.k8s.io/367#issuecomment-48428019)) to the ReplicationController. Nor should it verify that the pods controlled match the currently specified template, as that would obstruct auto-sizing and other automated processes. Similarly, completion deadlines, ordering dependencies, configuration expansion, and other features belong elsewhere. We even plan to factor out the mechanism for bulk pod creation ([#170](https://issue.k8s.io/170)). diff --git a/content/en/docs/contribute/participate/roles-and-responsibilities.md b/content/en/docs/contribute/participate/roles-and-responsibilities.md index 8ebe7a1303..4e8632ac0b 100644 --- a/content/en/docs/contribute/participate/roles-and-responsibilities.md +++ b/content/en/docs/contribute/participate/roles-and-responsibilities.md @@ -52,7 +52,7 @@ Members can: {{< note >}} Using `/lgtm` triggers automation. If you want to provide non-binding - approval, simply commenting "LGTM" works too! + approval, commenting "LGTM" works too! {{< /note >}} - Use the `/hold` comment to block merging for a pull request diff --git a/content/en/docs/reference/access-authn-authz/admission-controllers.md b/content/en/docs/reference/access-authn-authz/admission-controllers.md index 3ff113bb63..ef1d9a03a5 100644 --- a/content/en/docs/reference/access-authn-authz/admission-controllers.md +++ b/content/en/docs/reference/access-authn-authz/admission-controllers.md @@ -110,7 +110,7 @@ This admission controller allows all pods into the cluster. It is deprecated bec This admission controller modifies every new Pod to force the image pull policy to Always. This is useful in a multitenant cluster so that users can be assured that their private images can only be used by those who have the credentials to pull them. Without this admission controller, once an image has been pulled to a -node, any pod from any user can use it simply by knowing the image's name (assuming the Pod is +node, any pod from any user can use it by knowing the image's name (assuming the Pod is scheduled onto the right node), without any authorization check against the image. When this admission controller is enabled, images are always pulled prior to starting containers, which means valid credentials are required. diff --git a/content/en/docs/reference/access-authn-authz/authentication.md b/content/en/docs/reference/access-authn-authz/authentication.md index 8c2c4fa520..13517002f0 100644 --- a/content/en/docs/reference/access-authn-authz/authentication.md +++ b/content/en/docs/reference/access-authn-authz/authentication.md @@ -206,7 +206,7 @@ spec: Service account bearer tokens are perfectly valid to use outside the cluster and can be used to create identities for long standing jobs that wish to talk to the -Kubernetes API. To manually create a service account, simply use the `kubectl +Kubernetes API. To manually create a service account, use the `kubectl create serviceaccount (NAME)` command. This creates a service account in the current namespace and an associated secret. @@ -420,12 +420,12 @@ users: refresh-token: q1bKLFOyUiosTfawzA93TzZIDzH2TNa2SMm0zEiPKTUwME6BkEo6Sql5yUWVBSWpKUGphaWpxSVAfekBOZbBhaEW+VlFUeVRGcluyVF5JT4+haZmPsluFoFu5XkpXk5BXq name: oidc ``` -Once your `id_token` expires, `kubectl` will attempt to refresh your `id_token` using your `refresh_token` and `client_secret` storing the new values for the `refresh_token` and `id_token` in your `.kube/config`. +Once your `id_token` expires, `kubectl` will attempt to refresh your `id_token` using your `refresh_token` and `client_secret` storing the new values for the `refresh_token` and `id_token` in your `.kube/config`. ##### Option 2 - Use the `--token` Option -The `kubectl` command lets you pass in a token using the `--token` option. Simply copy and paste the `id_token` into this option: +The `kubectl` command lets you pass in a token using the `--token` option. Copy and paste the `id_token` into this option: ```bash kubectl --token=eyJhbGciOiJSUzI1NiJ9.eyJpc3MiOiJodHRwczovL21sYi50cmVtb2xvLmxhbjo4MDQzL2F1dGgvaWRwL29pZGMiLCJhdWQiOiJrdWJlcm5ldGVzIiwiZXhwIjoxNDc0NTk2NjY5LCJqdGkiOiI2RDUzNXoxUEpFNjJOR3QxaWVyYm9RIiwiaWF0IjoxNDc0NTk2MzY5LCJuYmYiOjE0NzQ1OTYyNDksInN1YiI6Im13aW5kdSIsInVzZXJfcm9sZSI6WyJ1c2VycyIsIm5ldy1uYW1lc3BhY2Utdmlld2VyIl0sImVtYWlsIjoibXdpbmR1QG5vbW9yZWplZGkuY29tIn0.f2As579n9VNoaKzoF-dOQGmXkFKf1FMyNV0-va_B63jn-_n9LGSCca_6IVMP8pO-Zb4KvRqGyTP0r3HkHxYy5c81AnIh8ijarruczl-TK_yF5akjSTHFZD-0gRzlevBDiH8Q79NAr-ky0P4iIXS8lY9Vnjch5MF74Zx0c3alKJHJUnnpjIACByfF2SCaYzbWFMUNat-K1PaUk5-ujMBG7yYnr95xD-63n8CO8teGUAAEMx6zRjzfhnhbzX-ajwZLGwGUBT4WqjMs70-6a7_8gZmLZb2az1cZynkFRj2BaCkVT3A2RrjeEwZEtGXlMqKJ1_I2ulrOVsYx01_yD35-rw get nodes diff --git a/content/en/docs/reference/using-api/server-side-apply.md b/content/en/docs/reference/using-api/server-side-apply.md index 302ae94d8b..d91497f8f2 100644 --- a/content/en/docs/reference/using-api/server-side-apply.md +++ b/content/en/docs/reference/using-api/server-side-apply.md @@ -16,10 +16,10 @@ min-kubernetes-server-version: 1.16 ## Introduction -Server Side Apply helps users and controllers manage their resources via -declarative configurations. It allows them to create and/or modify their +Server Side Apply helps users and controllers manage their resources through +declarative configurations. Clients can create and modify their [objects](/docs/concepts/overview/working-with-objects/kubernetes-objects/) -declaratively, simply by sending their fully specified intent. +declaratively by sending their fully specified intent. A fully specified intent is a partial object that only includes the fields and values for which the user has an opinion. That intent either creates a new diff --git a/content/en/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm.md b/content/en/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm.md index 4d932e3e05..6516a18825 100644 --- a/content/en/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm.md +++ b/content/en/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm.md @@ -434,7 +434,7 @@ Now remove the node: kubectl delete node ``` -If you wish to start over simply run `kubeadm init` or `kubeadm join` with the +If you wish to start over, run `kubeadm init` or `kubeadm join` with the appropriate arguments. ### Clean up the control plane diff --git a/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md b/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md index 03ff264816..014df012c7 100644 --- a/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md +++ b/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md @@ -547,7 +547,7 @@ Your main source of help for troubleshooting your Kubernetes cluster should star 1. After launching `start.ps1`, flanneld is stuck in "Waiting for the Network to be created" - There are numerous reports of this [issue which are being investigated](https://github.com/coreos/flannel/issues/1066); most likely it is a timing issue for when the management IP of the flannel network is set. A workaround is to simply relaunch start.ps1 or relaunch it manually as follows: + There are numerous reports of this [issue](https://github.com/coreos/flannel/issues/1066); most likely it is a timing issue for when the management IP of the flannel network is set. A workaround is to relaunch start.ps1 or relaunch it manually as follows: ```powershell PS C:> [Environment]::SetEnvironmentVariable("NODE_NAME", "") diff --git a/content/en/docs/setup/production-environment/windows/user-guide-windows-containers.md b/content/en/docs/setup/production-environment/windows/user-guide-windows-containers.md index 6c9c05cc90..9133a2de1b 100644 --- a/content/en/docs/setup/production-environment/windows/user-guide-windows-containers.md +++ b/content/en/docs/setup/production-environment/windows/user-guide-windows-containers.md @@ -23,7 +23,7 @@ Windows applications constitute a large portion of the services and applications ## Before you begin * Create a Kubernetes cluster that includes a [master and a worker node running Windows Server](/docs/tasks/administer-cluster/kubeadm/adding-windows-nodes) -* It is important to note that creating and deploying services and workloads on Kubernetes behaves in much the same way for Linux and Windows containers. [Kubectl commands](/docs/reference/kubectl/overview/) to interface with the cluster are identical. The example in the section below is provided simply to jumpstart your experience with Windows containers. +* It is important to note that creating and deploying services and workloads on Kubernetes behaves in much the same way for Linux and Windows containers. [Kubectl commands](/docs/reference/kubectl/overview/) to interface with the cluster are identical. The example in the section below is provided to jumpstart your experience with Windows containers. ## Getting Started: Deploying a Windows container diff --git a/content/en/docs/tasks/access-application-cluster/access-cluster.md b/content/en/docs/tasks/access-application-cluster/access-cluster.md index 7f74320118..400a54ffb2 100644 --- a/content/en/docs/tasks/access-application-cluster/access-cluster.md +++ b/content/en/docs/tasks/access-application-cluster/access-cluster.md @@ -280,7 +280,7 @@ at `https://104.197.5.247/api/v1/namespaces/kube-system/services/elasticsearch-l #### Manually constructing apiserver proxy URLs -As mentioned above, you use the `kubectl cluster-info` command to retrieve the service's proxy URL. To create proxy URLs that include service endpoints, suffixes, and parameters, you simply append to the service's proxy URL: +As mentioned above, you use the `kubectl cluster-info` command to retrieve the service's proxy URL. To create proxy URLs that include service endpoints, suffixes, and parameters, you append to the service's proxy URL: `http://`*`kubernetes_master_address`*`/api/v1/namespaces/`*`namespace_name`*`/services/`*`service_name[:port_name]`*`/proxy` If you haven't specified a name for your port, you don't have to specify *port_name* in the URL. diff --git a/content/en/docs/tasks/administer-cluster/access-cluster-api.md b/content/en/docs/tasks/administer-cluster/access-cluster-api.md index ffe200b118..99fd3596b3 100644 --- a/content/en/docs/tasks/administer-cluster/access-cluster-api.md +++ b/content/en/docs/tasks/administer-cluster/access-cluster-api.md @@ -215,7 +215,7 @@ for i in ret.items: #### Java client -* To install the [Java Client](https://github.com/kubernetes-client/java), simply execute : +To install the [Java Client](https://github.com/kubernetes-client/java), run: ```shell # Clone java library diff --git a/content/en/docs/tasks/administer-cluster/access-cluster-services.md b/content/en/docs/tasks/administer-cluster/access-cluster-services.md index c318a3df35..f6ba4e4fc0 100644 --- a/content/en/docs/tasks/administer-cluster/access-cluster-services.md +++ b/content/en/docs/tasks/administer-cluster/access-cluster-services.md @@ -83,7 +83,7 @@ See [Access Clusters Using the Kubernetes API](/docs/tasks/administer-cluster/ac #### Manually constructing apiserver proxy URLs -As mentioned above, you use the `kubectl cluster-info` command to retrieve the service's proxy URL. To create proxy URLs that include service endpoints, suffixes, and parameters, you simply append to the service's proxy URL: +As mentioned above, you use the `kubectl cluster-info` command to retrieve the service's proxy URL. To create proxy URLs that include service endpoints, suffixes, and parameters, you append to the service's proxy URL: `http://`*`kubernetes_master_address`*`/api/v1/namespaces/`*`namespace_name`*`/services/`*`[https:]service_name[:port_name]`*`/proxy` If you haven't specified a name for your port, you don't have to specify *port_name* in the URL. diff --git a/content/en/docs/tasks/administer-cluster/change-default-storage-class.md b/content/en/docs/tasks/administer-cluster/change-default-storage-class.md index 9c08a2a4ad..0c7c3c3ca1 100644 --- a/content/en/docs/tasks/administer-cluster/change-default-storage-class.md +++ b/content/en/docs/tasks/administer-cluster/change-default-storage-class.md @@ -32,7 +32,7 @@ for example, it might provision storage that is too expensive. If this is the ca you can either change the default StorageClass or disable it completely to avoid dynamic provisioning of storage. -Simply deleting the default StorageClass may not work, as it may be re-created +Deleting the default StorageClass may not work, as it may be re-created automatically by the addon manager running in your cluster. Please consult the docs for your installation for details about addon manager and how to disable individual addons. diff --git a/content/en/docs/tasks/configure-pod-container/configure-service-account.md b/content/en/docs/tasks/configure-pod-container/configure-service-account.md index ca3d0b2966..d96a5c8270 100644 --- a/content/en/docs/tasks/configure-pod-container/configure-service-account.md +++ b/content/en/docs/tasks/configure-pod-container/configure-service-account.md @@ -23,16 +23,10 @@ authenticated by the apiserver as a particular User Account (currently this is usually `admin`, unless your cluster administrator has customized your cluster). Processes in containers inside pods can also contact the apiserver. When they do, they are authenticated as a particular Service Account (for example, `default`). - - - ## {{% heading "prerequisites" %}} - {{< include "task-tutorial-prereqs.md" >}} {{< version-check >}} - - ## Use the Default Service Account to access the API server. @@ -129,7 +123,7 @@ then you will see that a token has automatically been created and is referenced You may use authorization plugins to [set permissions on service accounts](/docs/reference/access-authn-authz/rbac/#service-account-permissions). -To use a non-default service account, simply set the `spec.serviceAccountName` +To use a non-default service account, set the `spec.serviceAccountName` field of a pod to the name of the service account you wish to use. The service account has to exist at the time the pod is created, or it will be rejected. diff --git a/content/en/docs/tasks/configure-pod-container/translate-compose-kubernetes.md b/content/en/docs/tasks/configure-pod-container/translate-compose-kubernetes.md index b780baf1f0..aa62ad33e5 100644 --- a/content/en/docs/tasks/configure-pod-container/translate-compose-kubernetes.md +++ b/content/en/docs/tasks/configure-pod-container/translate-compose-kubernetes.md @@ -12,16 +12,10 @@ What's Kompose? It's a conversion tool for all things compose (namely Docker Com More information can be found on the Kompose website at [http://kompose.io](http://kompose.io). - - - ## {{% heading "prerequisites" %}} - {{< include "task-tutorial-prereqs.md" >}} {{< version-check >}} - - ## Install Kompose @@ -49,7 +43,6 @@ sudo mv ./kompose /usr/local/bin/kompose Alternatively, you can download the [tarball](https://github.com/kubernetes/kompose/releases). - {{% /tab %}} {{% tab name="Build from source" %}} @@ -87,8 +80,8 @@ On macOS you can install latest release via [Homebrew](https://brew.sh): ```bash brew install kompose - ``` + {{% /tab %}} {{< /tabs >}} @@ -97,95 +90,117 @@ brew install kompose In just a few steps, we'll take you from Docker Compose to Kubernetes. All you need is an existing `docker-compose.yml` file. -1. Go to the directory containing your `docker-compose.yml` file. If you don't - have one, test using this one. +1. Go to the directory containing your `docker-compose.yml` file. If you don't have one, test using this one. - ```yaml - version: "2" + ```yaml + version: "2" - services: + services: - redis-master: - image: k8s.gcr.io/redis:e2e - ports: - - "6379" + redis-master: + image: k8s.gcr.io/redis:e2e + ports: + - "6379" - redis-slave: - image: gcr.io/google_samples/gb-redisslave:v3 - ports: - - "6379" - environment: - - GET_HOSTS_FROM=dns + redis-slave: + image: gcr.io/google_samples/gb-redisslave:v3 + ports: + - "6379" + environment: + - GET_HOSTS_FROM=dns - frontend: - image: gcr.io/google-samples/gb-frontend:v4 - ports: - - "80:80" - environment: - - GET_HOSTS_FROM=dns - labels: - kompose.service.type: LoadBalancer - ``` + frontend: + image: gcr.io/google-samples/gb-frontend:v4 + ports: + - "80:80" + environment: + - GET_HOSTS_FROM=dns + labels: + kompose.service.type: LoadBalancer + ``` -2. To convert the `docker-compose.yml` file to files that you can use with - `kubectl`, run `kompose convert` and then `kubectl apply -f `. +2. To convert the `docker-compose.yml` file to files that you can use with + `kubectl`, run `kompose convert` and then `kubectl apply -f `. - ```bash - $ kompose convert - INFO Kubernetes file "frontend-service.yaml" created - INFO Kubernetes file "redis-master-service.yaml" created - INFO Kubernetes file "redis-slave-service.yaml" created - INFO Kubernetes file "frontend-deployment.yaml" created - INFO Kubernetes file "redis-master-deployment.yaml" created - INFO Kubernetes file "redis-slave-deployment.yaml" created - ``` + ```bash + kompose convert + ``` - ```bash - $ kubectl apply -f frontend-service.yaml,redis-master-service.yaml,redis-slave-service.yaml,frontend-deployment.yaml,redis-master-deployment.yaml,redis-slave-deployment.yaml - service/frontend created - service/redis-master created - service/redis-slave created - deployment.apps/frontend created - deployment.apps/redis-master created - deployment.apps/redis-slave created - ``` + The output is similar to: - Your deployments are running in Kubernetes. + ```none + INFO Kubernetes file "frontend-service.yaml" created + INFO Kubernetes file "frontend-service.yaml" created + INFO Kubernetes file "frontend-service.yaml" created + INFO Kubernetes file "redis-master-service.yaml" created + INFO Kubernetes file "redis-master-service.yaml" created + INFO Kubernetes file "redis-master-service.yaml" created + INFO Kubernetes file "redis-slave-service.yaml" created + INFO Kubernetes file "redis-slave-service.yaml" created + INFO Kubernetes file "redis-slave-service.yaml" created + INFO Kubernetes file "frontend-deployment.yaml" created + INFO Kubernetes file "frontend-deployment.yaml" created + INFO Kubernetes file "frontend-deployment.yaml" created + INFO Kubernetes file "redis-master-deployment.yaml" created + INFO Kubernetes file "redis-master-deployment.yaml" created + INFO Kubernetes file "redis-master-deployment.yaml" created + INFO Kubernetes file "redis-slave-deployment.yaml" created + INFO Kubernetes file "redis-slave-deployment.yaml" created + INFO Kubernetes file "redis-slave-deployment.yaml" created + ``` -3. Access your application. + ```bash + kubectl apply -f frontend-service.yaml,redis-master-service.yaml,redis-slave-service.yaml,frontend-deployment.yaml, + ``` - If you're already using `minikube` for your development process: + The output is similar to: - ```bash - $ minikube service frontend - ``` + ```none + redis-master-deployment.yaml,redis-slave-deployment.yaml + service/frontend created + service/redis-master created + service/redis-slave created + deployment.apps/frontend created + deployment.apps/redis-master created + deployment.apps/redis-slave created + ``` - Otherwise, let's look up what IP your service is using! + Your deployments are running in Kubernetes. - ```sh - $ kubectl describe svc frontend - Name: frontend - Namespace: default - Labels: service=frontend - Selector: service=frontend - Type: LoadBalancer - IP: 10.0.0.183 - LoadBalancer Ingress: 192.0.2.89 - Port: 80 80/TCP - NodePort: 80 31144/TCP - Endpoints: 172.17.0.4:80 - Session Affinity: None - No events. +3. Access your application. - ``` + If you're already using `minikube` for your development process: - If you're using a cloud provider, your IP will be listed next to `LoadBalancer Ingress`. + ```bash + minikube service frontend + ``` - ```sh - $ curl http://192.0.2.89 - ``` + Otherwise, let's look up what IP your service is using! + ```sh + kubectl describe svc frontend + ``` + ```none + Name: frontend + Namespace: default + Labels: service=frontend + Selector: service=frontend + Type: LoadBalancer + IP: 10.0.0.183 + LoadBalancer Ingress: 192.0.2.89 + Port: 80 80/TCP + NodePort: 80 31144/TCP + Endpoints: 172.17.0.4:80 + Session Affinity: None + No events. + ``` + + If you're using a cloud provider, your IP will be listed next to `LoadBalancer Ingress`. + + ```sh + curl http://192.0.2.89 + ``` @@ -205,15 +220,17 @@ you need is an existing `docker-compose.yml` file. Kompose has support for two providers: OpenShift and Kubernetes. You can choose a targeted provider using global option `--provider`. If no provider is specified, Kubernetes is set by default. - ## `kompose convert` Kompose supports conversion of V1, V2, and V3 Docker Compose files into Kubernetes and OpenShift objects. -### Kubernetes +### Kubernetes `kompose convert` example -```sh -$ kompose --file docker-voting.yml convert +```shell +kompose --file docker-voting.yml convert +``` + +```none WARN Unsupported key networks - ignoring WARN Unsupported key build - ignoring INFO Kubernetes file "worker-svc.yaml" created @@ -226,16 +243,24 @@ INFO Kubernetes file "result-deployment.yaml" created INFO Kubernetes file "vote-deployment.yaml" created INFO Kubernetes file "worker-deployment.yaml" created INFO Kubernetes file "db-deployment.yaml" created +``` -$ ls +```shell +ls +``` + +```none db-deployment.yaml docker-compose.yml docker-gitlab.yml redis-deployment.yaml result-deployment.yaml vote-deployment.yaml worker-deployment.yaml db-svc.yaml docker-voting.yml redis-svc.yaml result-svc.yaml vote-svc.yaml worker-svc.yaml ``` You can also provide multiple docker-compose files at the same time: -```sh -$ kompose -f docker-compose.yml -f docker-guestbook.yml convert +```shell +kompose -f docker-compose.yml -f docker-guestbook.yml convert +``` + +```none INFO Kubernetes file "frontend-service.yaml" created INFO Kubernetes file "mlbparks-service.yaml" created INFO Kubernetes file "mongodb-service.yaml" created @@ -247,8 +272,13 @@ INFO Kubernetes file "mongodb-deployment.yaml" created INFO Kubernetes file "mongodb-claim0-persistentvolumeclaim.yaml" created INFO Kubernetes file "redis-master-deployment.yaml" created INFO Kubernetes file "redis-slave-deployment.yaml" created +``` -$ ls +```shell +ls +``` + +```none mlbparks-deployment.yaml mongodb-service.yaml redis-slave-service.jsonmlbparks-service.yaml frontend-deployment.yaml mongodb-claim0-persistentvolumeclaim.yaml redis-master-service.yaml frontend-service.yaml mongodb-deployment.yaml redis-slave-deployment.yaml @@ -257,10 +287,13 @@ redis-master-deployment.yaml When multiple docker-compose files are provided the configuration is merged. Any configuration that is common will be over ridden by subsequent file. -### OpenShift +### OpenShift `kompose convert` example ```sh -$ kompose --provider openshift --file docker-voting.yml convert +kompose --provider openshift --file docker-voting.yml convert +``` + +```none WARN [worker] Service cannot be created because of missing port. INFO OpenShift file "vote-service.yaml" created INFO OpenShift file "db-service.yaml" created @@ -281,7 +314,10 @@ INFO OpenShift file "result-imagestream.yaml" created It also supports creating buildconfig for build directive in a service. By default, it uses the remote repo for the current git branch as the source repo, and the current branch as the source branch for the build. You can specify a different source repo and branch using ``--build-repo`` and ``--build-branch`` options respectively. ```sh -$ kompose --provider openshift --file buildconfig/docker-compose.yml convert +kompose --provider openshift --file buildconfig/docker-compose.yml convert +``` + +```none WARN [foo] Service cannot be created because of missing port. INFO OpenShift Buildconfig using git@github.com:rtnpro/kompose.git::master as source. INFO OpenShift file "foo-deploymentconfig.yaml" created @@ -297,23 +333,31 @@ If you are manually pushing the OpenShift artifacts using ``oc create -f``, you Kompose supports a straightforward way to deploy your "composed" application to Kubernetes or OpenShift via `kompose up`. +### Kubernetes `kompose up` example -### Kubernetes -```sh -$ kompose --file ./examples/docker-guestbook.yml up +```shell +kompose --file ./examples/docker-guestbook.yml up +``` + +```none We are going to create Kubernetes deployments and services for your Dockerized application. If you need different kind of resources, use the 'kompose convert' and 'kubectl apply -f' commands instead. -INFO Successfully created service: redis-master -INFO Successfully created service: redis-slave -INFO Successfully created service: frontend +INFO Successfully created service: redis-master +INFO Successfully created service: redis-slave +INFO Successfully created service: frontend INFO Successfully created deployment: redis-master INFO Successfully created deployment: redis-slave -INFO Successfully created deployment: frontend +INFO Successfully created deployment: frontend Your application has been deployed to Kubernetes. You can run 'kubectl get deployment,svc,pods' for details. +``` -$ kubectl get deployment,svc,pods +```shell +kubectl get deployment,svc,pods +``` + +```none NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE deployment.extensions/frontend 1 1 1 1 4m deployment.extensions/redis-master 1 1 1 1 4m @@ -331,14 +375,19 @@ pod/redis-master-1432129712-63jn8 1/1 Running 0 4m pod/redis-slave-2504961300-nve7b 1/1 Running 0 4m ``` -**Note**: +{{< note >}} - You must have a running Kubernetes cluster with a pre-configured kubectl context. - Only deployments and services are generated and deployed to Kubernetes. If you need different kind of resources, use the `kompose convert` and `kubectl apply -f` commands instead. +{{< /note >}} -### OpenShift -```sh -$ kompose --file ./examples/docker-guestbook.yml --provider openshift up +### OpenShift `kompose up` example + +```shell +kompose --file ./examples/docker-guestbook.yml --provider openshift up +``` + +```none We are going to create OpenShift DeploymentConfigs and Services for your Dockerized application. If you need different kind of resources, use the 'kompose convert' and 'oc create -f' commands instead. @@ -353,8 +402,13 @@ INFO Successfully created deployment: redis-master INFO Successfully created ImageStream: redis-master Your application has been deployed to OpenShift. You can run 'oc get dc,svc,is' for details. +``` -$ oc get dc,svc,is +```shell +oc get dc,svc,is +``` + +```none NAME REVISION DESIRED CURRENT TRIGGERED BY dc/frontend 0 1 0 config,image(frontend:v4) dc/redis-master 0 1 0 config,image(redis-master:e2e) @@ -369,16 +423,16 @@ is/redis-master 172.30.12.200:5000/fff/redis-master is/redis-slave 172.30.12.200:5000/fff/redis-slave v1 ``` -**Note**: - -- You must have a running OpenShift cluster with a pre-configured `oc` context (`oc login`) +{{< note >}} +You must have a running OpenShift cluster with a pre-configured `oc` context (`oc login`). +{{< /note >}} ## `kompose down` -Once you have deployed "composed" application to Kubernetes, `$ kompose down` will help you to take the application out by deleting its deployments and services. If you need to remove other resources, use the 'kubectl' command. +Once you have deployed "composed" application to Kubernetes, `kompose down` will help you to take the application out by deleting its deployments and services. If you need to remove other resources, use the 'kubectl' command. -```sh -$ kompose --file docker-guestbook.yml down +```shell +kompose --file docker-guestbook.yml down INFO Successfully deleted service: redis-master INFO Successfully deleted deployment: redis-master INFO Successfully deleted service: redis-slave @@ -387,16 +441,16 @@ INFO Successfully deleted service: frontend INFO Successfully deleted deployment: frontend ``` -**Note**: - -- You must have a running Kubernetes cluster with a pre-configured kubectl context. +{{< note >}} +You must have a running Kubernetes cluster with a pre-configured `kubectl` context. +{{< /note >}} ## Build and Push Docker Images Kompose supports both building and pushing Docker images. When using the `build` key within your Docker Compose file, your image will: - - Automatically be built with Docker using the `image` key specified within your file - - Be pushed to the correct Docker repository using local credentials (located at `.docker/config`) +- Automatically be built with Docker using the `image` key specified within your file +- Be pushed to the correct Docker repository using local credentials (located at `.docker/config`) Using an [example Docker Compose file](https://raw.githubusercontent.com/kubernetes/kompose/master/examples/buildconfig/docker-compose.yml): @@ -412,7 +466,7 @@ services: Using `kompose up` with a `build` key: ```none -$ kompose up +kompose up INFO Build key detected. Attempting to build and push image 'docker.io/foo/bar' INFO Building image 'docker.io/foo/bar' from directory 'build' INFO Image 'docker.io/foo/bar' from directory 'build' built successfully @@ -432,10 +486,10 @@ In order to disable the functionality, or choose to use BuildConfig generation ( ```sh # Disable building/pushing Docker images -$ kompose up --build none +kompose up --build none # Generate Build Config artifacts for OpenShift -$ kompose up --provider openshift --build build-config +kompose up --provider openshift --build build-config ``` ## Alternative Conversions @@ -443,45 +497,54 @@ $ kompose up --provider openshift --build build-config The default `kompose` transformation will generate Kubernetes [Deployments](/docs/concepts/workloads/controllers/deployment/) and [Services](/docs/concepts/services-networking/service/), in yaml format. You have alternative option to generate json with `-j`. Also, you can alternatively generate [Replication Controllers](/docs/concepts/workloads/controllers/replicationcontroller/) objects, [Daemon Sets](/docs/concepts/workloads/controllers/daemonset/), or [Helm](https://github.com/helm/helm) charts. ```sh -$ kompose convert -j +kompose convert -j INFO Kubernetes file "redis-svc.json" created INFO Kubernetes file "web-svc.json" created INFO Kubernetes file "redis-deployment.json" created INFO Kubernetes file "web-deployment.json" created ``` + The `*-deployment.json` files contain the Deployment objects. ```sh -$ kompose convert --replication-controller +kompose convert --replication-controller INFO Kubernetes file "redis-svc.yaml" created INFO Kubernetes file "web-svc.yaml" created INFO Kubernetes file "redis-replicationcontroller.yaml" created INFO Kubernetes file "web-replicationcontroller.yaml" created ``` -The `*-replicationcontroller.yaml` files contain the Replication Controller objects. If you want to specify replicas (default is 1), use `--replicas` flag: `$ kompose convert --replication-controller --replicas 3` +The `*-replicationcontroller.yaml` files contain the Replication Controller objects. If you want to specify replicas (default is 1), use `--replicas` flag: `kompose convert --replication-controller --replicas 3` -```sh -$ kompose convert --daemon-set +```shell +kompose convert --daemon-set INFO Kubernetes file "redis-svc.yaml" created INFO Kubernetes file "web-svc.yaml" created INFO Kubernetes file "redis-daemonset.yaml" created INFO Kubernetes file "web-daemonset.yaml" created ``` -The `*-daemonset.yaml` files contain the Daemon Set objects +The `*-daemonset.yaml` files contain the DaemonSet objects -If you want to generate a Chart to be used with [Helm](https://github.com/kubernetes/helm) simply do: +If you want to generate a Chart to be used with [Helm](https://github.com/kubernetes/helm) run: -```sh -$ kompose convert -c +```shell +kompose convert -c +``` + +```none INFO Kubernetes file "web-svc.yaml" created INFO Kubernetes file "redis-svc.yaml" created INFO Kubernetes file "web-deployment.yaml" created INFO Kubernetes file "redis-deployment.yaml" created chart created in "./docker-compose/" +``` -$ tree docker-compose/ +```shell +tree docker-compose/ +``` + +```none docker-compose ├── Chart.yaml ├── README.md @@ -562,7 +625,7 @@ If you want to create normal pods without controllers you can use `restart` cons | `no` | Pod | `Never` | {{< note >}} -The controller object could be `deployment` or `replicationcontroller`, etc. +The controller object could be `deployment` or `replicationcontroller`. {{< /note >}} For example, the `pival` service will become pod down here. This container calculated value of `pi`. @@ -577,7 +640,7 @@ services: restart: "on-failure" ``` -### Warning about Deployment Config's +### Warning about Deployment Configurations If the Docker Compose file has a volume specified for a service, the Deployment (Kubernetes) or DeploymentConfig (OpenShift) strategy is changed to "Recreate" instead of "RollingUpdate" (default). This is done to avoid multiple instances of a service from accessing a volume at the same time. @@ -590,5 +653,3 @@ Please note that changing service name might break some `docker-compose` files. Kompose supports Docker Compose versions: 1, 2 and 3. We have limited support on versions 2.1 and 3.2 due to their experimental nature. A full list on compatibility between all three versions is listed in our [conversion document](https://github.com/kubernetes/kompose/blob/master/docs/conversion.md) including a list of all incompatible Docker Compose keys. - - diff --git a/content/en/docs/tasks/debug-application-cluster/debug-service.md b/content/en/docs/tasks/debug-application-cluster/debug-service.md index ba141135fa..9025375955 100644 --- a/content/en/docs/tasks/debug-application-cluster/debug-service.md +++ b/content/en/docs/tasks/debug-application-cluster/debug-service.md @@ -111,7 +111,7 @@ kubectl get pods -l app=hostnames \ 10.244.0.7 ``` -The example container used for this walk-through simply serves its own hostname +The example container used for this walk-through serves its own hostname via HTTP on port 9376, but if you are debugging your own app, you'll want to use whatever port number your Pods are listening on. diff --git a/content/en/docs/tasks/extend-kubectl/kubectl-plugins.md b/content/en/docs/tasks/extend-kubectl/kubectl-plugins.md index bec423043d..28fd615b45 100644 --- a/content/en/docs/tasks/extend-kubectl/kubectl-plugins.md +++ b/content/en/docs/tasks/extend-kubectl/kubectl-plugins.md @@ -12,20 +12,15 @@ content_type: task This guide demonstrates how to install and write extensions for [kubectl](/docs/reference/kubectl/kubectl/). By thinking of core `kubectl` commands as essential building blocks for interacting with a Kubernetes cluster, a cluster administrator can think of plugins as a means of utilizing these building blocks to create more complex behavior. Plugins extend `kubectl` with new sub-commands, allowing for new and custom features not included in the main distribution of `kubectl`. - - ## {{% heading "prerequisites" %}} - You need to have a working `kubectl` binary installed. - - ## Installing kubectl plugins -A plugin is nothing more than a standalone executable file, whose name begins with `kubectl-`. To install a plugin, simply move its executable file to anywhere on your `PATH`. +A plugin is a standalone executable file, whose name begins with `kubectl-`. To install a plugin, move its executable file to anywhere on your `PATH`. You can also discover and install kubectl plugins available in the open source using [Krew](https://krew.dev/). Krew is a plugin manager maintained by @@ -60,9 +55,9 @@ You can write a plugin in any programming language or script that allows you to There is no plugin installation or pre-loading required. Plugin executables receive the inherited environment from the `kubectl` binary. -A plugin determines which command path it wishes to implement based on its name. For -example, a plugin wanting to provide a new command `kubectl foo`, would simply be named -`kubectl-foo`, and live somewhere in your `PATH`. +A plugin determines which command path it wishes to implement based on its name. +For example, a plugin named `kubectl-foo` provides a command `kubectl foo`. You must +install the plugin executable somewhere in your `PATH`. ### Example plugin @@ -88,32 +83,34 @@ echo "I am a plugin named kubectl-foo" ### Using a plugin -To use the above plugin, simply make it executable: +To use a plugin, make the plugin executable: -``` +```shell sudo chmod +x ./kubectl-foo ``` and place it anywhere in your `PATH`: -``` +```shell sudo mv ./kubectl-foo /usr/local/bin ``` You may now invoke your plugin as a `kubectl` command: -``` +```shell kubectl foo ``` + ``` I am a plugin named kubectl-foo ``` All args and flags are passed as-is to the executable: -``` +```shell kubectl foo version ``` + ``` 1.0.0 ``` @@ -124,6 +121,7 @@ All environment variables are also passed as-is to the executable: export KUBECONFIG=~/.kube/config kubectl foo config ``` + ``` /home//.kube/config ``` @@ -131,6 +129,7 @@ kubectl foo config ```shell KUBECONFIG=/etc/kube/config kubectl foo config ``` + ``` /etc/kube/config ``` @@ -376,16 +375,11 @@ set up a build environment (if it needs compiling), and deploy the plugin. If you also make compiled packages available, or use Krew, that will make installs easier. - - ## {{% heading "whatsnext" %}} - * Check the Sample CLI Plugin repository for a [detailed example](https://github.com/kubernetes/sample-cli-plugin) of a plugin written in Go. In case of any questions, feel free to reach out to the [SIG CLI team](https://github.com/kubernetes/community/tree/master/sig-cli). * Read about [Krew](https://krew.dev/), a package manager for kubectl plugins. - - diff --git a/content/en/docs/tasks/job/parallel-processing-expansion.md b/content/en/docs/tasks/job/parallel-processing-expansion.md index e92fa9f5bb..8f5994929e 100644 --- a/content/en/docs/tasks/job/parallel-processing-expansion.md +++ b/content/en/docs/tasks/job/parallel-processing-expansion.md @@ -12,7 +12,7 @@ based on a common template. You can use this approach to process batches of work parallel. For this example there are only three items: _apple_, _banana_, and _cherry_. -The sample Jobs process each item simply by printing a string then pausing. +The sample Jobs process each item by printing a string then pausing. See [using Jobs in real workloads](#using-jobs-in-real-workloads) to learn about how this pattern fits more realistic use cases. diff --git a/content/en/docs/tasks/run-application/delete-stateful-set.md b/content/en/docs/tasks/run-application/delete-stateful-set.md index 57e54e6797..a70e018b85 100644 --- a/content/en/docs/tasks/run-application/delete-stateful-set.md +++ b/content/en/docs/tasks/run-application/delete-stateful-set.md @@ -66,7 +66,7 @@ Use caution when deleting a PVC, as it may lead to data loss. ### Complete deletion of a StatefulSet -To simply delete everything in a StatefulSet, including the associated pods, you can run a series of commands similar to the following: +To delete everything in a StatefulSet, including the associated pods, you can run a series of commands similar to the following: ```shell grace=$(kubectl get pods --template '{{.spec.terminationGracePeriodSeconds}}') diff --git a/content/en/docs/tasks/run-application/run-replicated-stateful-application.md b/content/en/docs/tasks/run-application/run-replicated-stateful-application.md index f1738ff53e..79d779f0df 100644 --- a/content/en/docs/tasks/run-application/run-replicated-stateful-application.md +++ b/content/en/docs/tasks/run-application/run-replicated-stateful-application.md @@ -171,7 +171,7 @@ properties. The script in the `init-mysql` container also applies either `primary.cnf` or `replica.cnf` from the ConfigMap by copying the contents into `conf.d`. Because the example topology consists of a single primary MySQL server and any number of -replicas, the script simply assigns ordinal `0` to be the primary server, and everyone +replicas, the script assigns ordinal `0` to be the primary server, and everyone else to be replicas. Combined with the StatefulSet controller's [deployment order guarantee](/docs/concepts/workloads/controllers/statefulset/#deployment-and-scaling-guarantees/), diff --git a/content/en/docs/tutorials/clusters/apparmor.md b/content/en/docs/tutorials/clusters/apparmor.md index 8ca9f30bad..54c8a0f44c 100644 --- a/content/en/docs/tutorials/clusters/apparmor.md +++ b/content/en/docs/tutorials/clusters/apparmor.md @@ -168,8 +168,7 @@ k8s-apparmor-example-deny-write (enforce) *This example assumes you have already set up a cluster with AppArmor support.* -First, we need to load the profile we want to use onto our nodes. The profile we'll use simply -denies all file writes: +First, we need to load the profile we want to use onto our nodes. This profile denies all file writes: ```shell #include diff --git a/content/en/docs/tutorials/stateful-application/basic-stateful-set.md b/content/en/docs/tutorials/stateful-application/basic-stateful-set.md index f43af0e15e..b8aaaabb5b 100644 --- a/content/en/docs/tutorials/stateful-application/basic-stateful-set.md +++ b/content/en/docs/tutorials/stateful-application/basic-stateful-set.md @@ -934,10 +934,10 @@ web-2 0/1 Terminating 0 3m When the `web` StatefulSet was recreated, it first relaunched `web-0`. Since `web-1` was already Running and Ready, when `web-0` transitioned to - Running and Ready, it simply adopted this Pod. Since you recreated the StatefulSet - with `replicas` equal to 2, once `web-0` had been recreated, and once - `web-1` had been determined to already be Running and Ready, `web-2` was - terminated. +Running and Ready, it adopted this Pod. Since you recreated the StatefulSet +with `replicas` equal to 2, once `web-0` had been recreated, and once +`web-1` had been determined to already be Running and Ready, `web-2` was +terminated. Let's take another look at the contents of the `index.html` file served by the Pods' webservers: @@ -945,6 +945,7 @@ Pods' webservers: ```shell for i in 0 1; do kubectl exec -i -t "web-$i" -- curl http://localhost/; done ``` + ``` web-0 web-1 @@ -970,15 +971,18 @@ In another terminal, delete the StatefulSet again. This time, omit the ```shell kubectl delete statefulset web ``` + ``` statefulset.apps "web" deleted ``` + Examine the output of the `kubectl get` command running in the first terminal, and wait for all of the Pods to transition to Terminating. ```shell kubectl get pods -w -l app=nginx ``` + ``` NAME READY STATUS RESTARTS AGE web-0 1/1 Running 0 11m @@ -1006,10 +1010,10 @@ the cascade does not delete the headless Service associated with the StatefulSet You must delete the `nginx` Service manually. {{< /note >}} - ```shell kubectl delete service nginx ``` + ``` service "nginx" deleted ``` @@ -1019,6 +1023,7 @@ Recreate the StatefulSet and headless Service one more time: ```shell kubectl apply -f web.yaml ``` + ``` service/nginx created statefulset.apps/web created @@ -1030,6 +1035,7 @@ the contents of their `index.html` files: ```shell for i in 0 1; do kubectl exec -i -t "web-$i" -- curl http://localhost/; done ``` + ``` web-0 web-1 @@ -1044,13 +1050,17 @@ Finally, delete the `nginx` Service... ```shell kubectl delete service nginx ``` + ``` service "nginx" deleted ``` + ...and the `web` StatefulSet: + ```shell kubectl delete statefulset web ``` + ``` statefulset "web" deleted ``` diff --git a/content/en/docs/tutorials/stateless-application/guestbook.md b/content/en/docs/tutorials/stateless-application/guestbook.md index 817526c6d1..b60afcf5f8 100644 --- a/content/en/docs/tutorials/stateless-application/guestbook.md +++ b/content/en/docs/tutorials/stateless-application/guestbook.md @@ -153,7 +153,7 @@ The `mongo` Services you applied is only accessible within the Kubernetes cluste If you want guests to be able to access your guestbook, you must configure the frontend Service to be externally visible, so a client can request the Service from outside the Kubernetes cluster. However a Kubernetes user you can use `kubectl port-forward` to access the service even though it uses a `ClusterIP`. {{< note >}} -Some cloud providers, like Google Compute Engine or Google Kubernetes Engine, support external load balancers. If your cloud provider supports load balancers and you want to use it, simply uncomment `type: LoadBalancer`. +Some cloud providers, like Google Compute Engine or Google Kubernetes Engine, support external load balancers. If your cloud provider supports load balancers and you want to use it, uncomment `type: LoadBalancer`. {{< /note >}} {{< codenew file="application/guestbook/frontend-service.yaml" >}} From 5ed2b712a2cebf473d37a23afa4d88510867fe35 Mon Sep 17 00:00:00 2001 From: Arhell Date: Mon, 8 Feb 2021 01:57:13 +0200 Subject: [PATCH 167/190] =?UTF-8?q?[zh]=20Tidy=20list=20of=20generators=20?= =?UTF-8?q?for=20=E2=80=9Ckubectl=20create=E2=80=9D?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../zh/docs/reference/kubectl/conventions.md | 68 +++++++++---------- 1 file changed, 34 insertions(+), 34 deletions(-) diff --git a/content/zh/docs/reference/kubectl/conventions.md b/content/zh/docs/reference/kubectl/conventions.md index de8232d75b..d8f372086e 100644 --- a/content/zh/docs/reference/kubectl/conventions.md +++ b/content/zh/docs/reference/kubectl/conventions.md @@ -85,42 +85,42 @@ See the Kubernetes v1.17 documentation for a [list](https://v1-17.docs.kubernete #### 生成器 你可以使用 kubectl 命令生成以下资源， `kubectl create --dry-run=client -o yaml`： -``` - clusterrole 创建 ClusterRole。 - clusterrolebinding 为特定的 ClusterRole 创建 ClusterRoleBinding。 - configmap 使用本地文件、目录或文本值创建 Configmap。 - cronjob 使用指定的名称创建 Cronjob。 - deployment 使用指定的名称创建 Deployment。 - job 使用指定的名称创建 Job。 - namespace 使用指定的名称创建名称空间。 - poddisruptionbudget 使用指定名称创建 Pod 干扰预算。 - priorityclass 使用指定的名称创建 Priorityclass。 - quota 使用指定的名称创建配额。 - role 使用单一规则创建角色。 - rolebinding 为特定角色或 ClusterRole 创建 RoleBinding。 - secret 使用指定的子命令创建 Secret。 - service 使用指定的子命令创建服务。 - serviceaccount 使用指定的名称创建服务帐户。 -``` + +* `clusterrole`: 创建 ClusterRole。 +* `clusterrolebinding`: 为特定的 ClusterRole 创建 ClusterRoleBinding。 +* `configmap`: 使用本地文件、目录或文本值创建 Configmap。 +* `cronjob`: 使用指定的名称创建 Cronjob。 +* `deployment`: 使用指定的名称创建 Deployment。 +* `job`: 使用指定的名称创建 Job。 +* `namespace`: 使用指定的名称创建名称空间。 +* `poddisruptionbudget`: 使用指定名称创建 Pod 干扰预算。 +* `priorityclass`: 使用指定的名称创建 Priorityclass。 +* `quota`: 使用指定的名称创建配额。 +* `role`: 使用单一规则创建角色。 +* `rolebinding`: 为特定角色或 ClusterRole 创建 RoleBinding。 +* `secret`: 使用指定的子命令创建 Secret。 +* `service`: 使用指定的子命令创建服务。 +* `serviceaccount`: 使用指定的名称创建服务帐户。 + ### `kubectl apply` From a0325b1a0174fac35a4ab29b7ccef5c77107917e Mon Sep 17 00:00:00 2001 From: childe Date: Mon, 8 Feb 2021 14:41:13 +0800 Subject: [PATCH 168/190] Update taint-and-toleration.md --- .../docs/concepts/scheduling-eviction/taint-and-toleration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/zh/docs/concepts/scheduling-eviction/taint-and-toleration.md b/content/zh/docs/concepts/scheduling-eviction/taint-and-toleration.md index c2631fb958..430f0d60b6 100644 --- a/content/zh/docs/concepts/scheduling-eviction/taint-and-toleration.md +++ b/content/zh/docs/concepts/scheduling-eviction/taint-and-toleration.md @@ -169,7 +169,7 @@ scheduled onto the node (if it is not yet running on the node). 则 Kubernetes 不会将 Pod 分配到该节点。 * 如果未被过滤的污点中不存在 effect 值为 `NoSchedule` 的污点， 但是存在 effect 值为 `PreferNoSchedule` 的污点， - 则 Kubernetes 会 *尝试* 将 Pod 分配到该节点。 + 则 Kubernetes 会 *尝试* 不将 Pod 分配到该节点。 * 如果未被过滤的污点中存在至少一个 effect 值为 `NoExecute` 的污点， 则 Kubernetes 不会将 Pod 分配到该节点（如果 Pod 还未在节点上运行）， 或者将 Pod 从该节点驱逐（如果 Pod 已经在节点上运行）。 From af40c89b7befc2ffd2ae1c16b8d38b305ce3fa54 Mon Sep 17 00:00:00 2001 From: Paran Lee Date: Sun, 7 Feb 2021 22:46:46 -0900 Subject: [PATCH 169/190] Update Kompose version from 1.21.0 to 1.22.0 --- .../configure-pod-container/translate-compose-kubernetes.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/content/en/docs/tasks/configure-pod-container/translate-compose-kubernetes.md b/content/en/docs/tasks/configure-pod-container/translate-compose-kubernetes.md index b780baf1f0..cc4d5c9e3c 100644 --- a/content/en/docs/tasks/configure-pod-container/translate-compose-kubernetes.md +++ b/content/en/docs/tasks/configure-pod-container/translate-compose-kubernetes.md @@ -35,13 +35,13 @@ Kompose is released via GitHub on a three-week cycle, you can see all current re ```sh # Linux -curl -L https://github.com/kubernetes/kompose/releases/download/v1.21.0/kompose-linux-amd64 -o kompose +curl -L https://github.com/kubernetes/kompose/releases/download/v1.22.0/kompose-linux-amd64 -o kompose # macOS -curl -L https://github.com/kubernetes/kompose/releases/download/v1.21.0/kompose-darwin-amd64 -o kompose +curl -L https://github.com/kubernetes/kompose/releases/download/v1.22.0/kompose-darwin-amd64 -o kompose # Windows -curl -L https://github.com/kubernetes/kompose/releases/download/v1.21.0/kompose-windows-amd64.exe -o kompose.exe +curl -L https://github.com/kubernetes/kompose/releases/download/v1.22.0/kompose-windows-amd64.exe -o kompose.exe chmod +x kompose sudo mv ./kompose /usr/local/bin/kompose From 892aedc0107d53274d10feb8fd6f1a03cf10a06f Mon Sep 17 00:00:00 2001 From: Luke Addison Date: Mon, 8 Feb 2021 10:52:15 +0000 Subject: [PATCH 170/190] Address review comments --- .../concepts/containers/container-lifecycle-hooks.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/content/en/docs/concepts/containers/container-lifecycle-hooks.md b/content/en/docs/concepts/containers/container-lifecycle-hooks.md index c2811fb118..b41c183bc2 100644 --- a/content/en/docs/concepts/containers/container-lifecycle-hooks.md +++ b/content/en/docs/concepts/containers/container-lifecycle-hooks.md @@ -37,11 +37,12 @@ No parameters are passed to the handler. `PreStop` This hook is called immediately before a container is terminated due to an API request or management -event such as a liveness/startup probe failure, preemption, resource contention and others. This -hook is not called if the container crashes or exits. The Pod's termination grace period countdown -begins before the `PreStop` hook is executed. Regardless of the outcome of the handler, the -container will eventually terminate within the Pod's termination grace period. No parameters are -passed to the handler. +event such as a liveness/startup probe failure, preemption, resource contention and others. A call +to the `PreStop` hook fails if the container is already in a terminated or completed state and the +hook must complete before the TERM signal to stop the container can be sent. The Pod's termination +grace period countdown begins before the `PreStop` hook is executed so regardless of the outcome of +the handler, the container will eventually terminate within the Pod's termination grace period. No +parameters are passed to the handler. A more detailed description of the termination behavior can be found in [Termination of Pods](/docs/concepts/workloads/pods/pod-lifecycle/#pod-termination). From cbf7a63a32483511c648cbff3224c4b7199db752 Mon Sep 17 00:00:00 2001 From: Luke Addison Date: Mon, 8 Feb 2021 10:53:36 +0000 Subject: [PATCH 171/190] Add comma --- .../en/docs/concepts/containers/container-lifecycle-hooks.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/concepts/containers/container-lifecycle-hooks.md b/content/en/docs/concepts/containers/container-lifecycle-hooks.md index b41c183bc2..96569f9518 100644 --- a/content/en/docs/concepts/containers/container-lifecycle-hooks.md +++ b/content/en/docs/concepts/containers/container-lifecycle-hooks.md @@ -40,7 +40,7 @@ This hook is called immediately before a container is terminated due to an API r event such as a liveness/startup probe failure, preemption, resource contention and others. A call to the `PreStop` hook fails if the container is already in a terminated or completed state and the hook must complete before the TERM signal to stop the container can be sent. The Pod's termination -grace period countdown begins before the `PreStop` hook is executed so regardless of the outcome of +grace period countdown begins before the `PreStop` hook is executed, so regardless of the outcome of the handler, the container will eventually terminate within the Pod's termination grace period. No parameters are passed to the handler. From 6b19266c3e69230d6bec6bc32b42aa31aef241cc Mon Sep 17 00:00:00 2001 From: Yao Ding Date: Mon, 8 Feb 2021 14:43:40 -0500 Subject: [PATCH 172/190] fix indentation --- content/en/docs/concepts/workloads/pods/ephemeral-containers.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/concepts/workloads/pods/ephemeral-containers.md b/content/en/docs/concepts/workloads/pods/ephemeral-containers.md index 49894937e3..c999e5263b 100644 --- a/content/en/docs/concepts/workloads/pods/ephemeral-containers.md +++ b/content/en/docs/concepts/workloads/pods/ephemeral-containers.md @@ -103,7 +103,7 @@ the ephemeral container to add as an `EphemeralContainers` list: "apiVersion": "v1", "kind": "EphemeralContainers", "metadata": { - "name": "example-pod" + "name": "example-pod" }, "ephemeralContainers": [{ "command": [ From da2d988e6744a7e8242b9c52fc206c7c2c997309 Mon Sep 17 00:00:00 2001 From: Arhell Date: Tue, 9 Feb 2021 00:58:25 +0200 Subject: [PATCH 173/190] [zh] update docker-cli-to-kubectl --- content/zh/docs/reference/kubectl/docker-cli-to-kubectl.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/zh/docs/reference/kubectl/docker-cli-to-kubectl.md b/content/zh/docs/reference/kubectl/docker-cli-to-kubectl.md index a89f43c32c..b53e6fcd77 100644 --- a/content/zh/docs/reference/kubectl/docker-cli-to-kubectl.md +++ b/content/zh/docs/reference/kubectl/docker-cli-to-kubectl.md @@ -62,7 +62,7 @@ kubectl create deployment --image=nginx nginx-app deployment.apps/nginx-app created ``` -``` +```shell # add env to nginx-app kubectl set env deployment/nginx-app DOMAIN=cluster ``` From 0ec95934a251f16cc0e06590a368307d1815ae57 Mon Sep 17 00:00:00 2001 From: Zhang Yong Date: Tue, 9 Feb 2021 11:05:57 +0800 Subject: [PATCH 174/190] Dual-stack docs correction --- content/en/docs/concepts/services-networking/dual-stack.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/concepts/services-networking/dual-stack.md b/content/en/docs/concepts/services-networking/dual-stack.md index d8b900847f..20cbcb5f33 100644 --- a/content/en/docs/concepts/services-networking/dual-stack.md +++ b/content/en/docs/concepts/services-networking/dual-stack.md @@ -163,7 +163,7 @@ status: loadBalancer: {} ``` -1. When dual-stack is enabled on a cluster, existing [headless Services](/docs/concepts/services-networking/service/#headless-services) with selectors are configured by the control plane to set `.spec.ipFamilyPolicy` to `SingleStack` and set `.spec.ipFamilies` to the address family of the first service cluster IP range (configured via the `--service-cluster-ip-range` flag to the kube-controller-manager) even though `.spec.ClusterIP` is set to `None`. +1. When dual-stack is enabled on a cluster, existing [headless Services](/docs/concepts/services-networking/service/#headless-services) with selectors are configured by the control plane to set `.spec.ipFamilyPolicy` to `SingleStack` and set `.spec.ipFamilies` to the address family of the first service cluster IP range (configured via the `--service-cluster-ip-range` flag to the kube-apiserver) even though `.spec.ClusterIP` is set to `None`. {{< codenew file="service/networking/dual-stack-default-svc.yaml" >}} From 188ccc220b8383fbb6fb73314e8a85b4199fa161 Mon Sep 17 00:00:00 2001 From: Weiping Cai Date: Thu, 28 Jan 2021 16:38:55 +0800 Subject: [PATCH 175/190] Clairly Resourcequota limit for PriorityClass Signed-off-by: Weiping Cai --- .../docs/concepts/policy/resource-quotas.md | 27 +++++++++++++------ .../policy/priority-class-resourcequota.yaml | 10 +++++++ 2 files changed, 29 insertions(+), 8 deletions(-) create mode 100644 content/en/examples/policy/priority-class-resourcequota.yaml diff --git a/content/en/docs/concepts/policy/resource-quotas.md b/content/en/docs/concepts/policy/resource-quotas.md index 0edb1be338..d5c94d99cf 100644 --- a/content/en/docs/concepts/policy/resource-quotas.md +++ b/content/en/docs/concepts/policy/resource-quotas.md @@ -610,17 +610,28 @@ plugins: values: ["cluster-services"] ``` -Now, "cluster-services" pods will be allowed in only those namespaces where a quota object with a matching `scopeSelector` is present. -For example: +Then, create a resource quota object in the `kube-system` namespace: -```yaml - scopeSelector: - matchExpressions: - - scopeName: PriorityClass - operator: In - values: ["cluster-services"] +{{< codenew file="policy/priority-class-resourcequota.yaml" >}} + +```shell +$ kubectl apply -f https://k8s.io/examples/policy/priority-class-resourcequota.yaml -n kube-system ``` +``` +resourcequota/pods-cluster-services created +``` + +In this case, a pod creation will be allowed if: + +1. the Pod's `priorityClassName` is not specified. +1. the Pod's `priorityClassName` is specified to a value other than `cluster-services`. +1. the Pod's `priorityClassName` is set to `cluster-services`, it is to be created + in the `kube-system` namespace, and it has passed the resource quota check. + +A Pod creation request is rejected if its `priorityClassName` is set to `cluster-services` +and it is to be created in a namespace other than `kube-system`. + ## {{% heading "whatsnext" %}} - See [ResourceQuota design doc](https://git.k8s.io/community/contributors/design-proposals/resource-management/admission_control_resource_quota.md) for more information. diff --git a/content/en/examples/policy/priority-class-resourcequota.yaml b/content/en/examples/policy/priority-class-resourcequota.yaml new file mode 100644 index 0000000000..7350d00c8f --- /dev/null +++ b/content/en/examples/policy/priority-class-resourcequota.yaml @@ -0,0 +1,10 @@ +apiVersion: v1 +kind: ResourceQuota +metadata: + name: pods-cluster-services +spec: + scopeSelector: + matchExpressions: + - operator : In + scopeName: PriorityClass + values: ["cluster-services"] \ No newline at end of file From ecbbb15a59311f287854ec6852e5a8f17a6913a1 Mon Sep 17 00:00:00 2001 From: luzg Date: Sun, 7 Feb 2021 11:30:58 +0800 Subject: [PATCH 176/190] [zh] translate Migrating telemetry and security agents from dockershim fix according to howieyuen and tengqm --- ...migrating-telemetry-and-security-agents.md | 157 ++++++++++++++++++ 1 file changed, 157 insertions(+) create mode 100644 content/zh/docs/tasks/administer-cluster/migrating-from-dockershim/migrating-telemetry-and-security-agents.md diff --git a/content/zh/docs/tasks/administer-cluster/migrating-from-dockershim/migrating-telemetry-and-security-agents.md b/content/zh/docs/tasks/administer-cluster/migrating-from-dockershim/migrating-telemetry-and-security-agents.md new file mode 100644 index 0000000000..b22c7d4b67 --- /dev/null +++ b/content/zh/docs/tasks/administer-cluster/migrating-from-dockershim/migrating-telemetry-and-security-agents.md @@ -0,0 +1,157 @@ +--- +title: 从 dockershim 迁移遥测和安全代理 +content_type: task +weight: 70 +--- + + + + + +在 Kubernetes 1.20 版本中，dockershim 被弃用。 +在博文[弃用 Dockershim 常见问题](/zh/blog/2020/12/02/dockershim-faq/)中， +你大概已经了解到，大多数应用并没有直接通过运行时来托管容器。 +但是，仍然有大量的遥测和安全代理依赖 docker 来收集容器元数据、日志和指标。 +本文汇总了一些信息和链接：信息用于阐述如何探查这些依赖，链接用于解释如何迁移这些代理去使用通用的工具或其他容器运行。 + + +## 遥测和安全代理 {#telemetry-and-security-agents} + + +为了让代理运行在 Kubernetes 集群中，我们有几种办法。 +代理既可以直接在节点上运行，也可以作为守护进程运行。 + + +### 为什么遥测代理依赖于 Docker？ {#why-do-telemetry-agents-relyon-docker} + + +因为历史原因，Kubernetes 建立在 Docker 之上。 +Kubernetes 管理网络和调度，Docker 则在具体的节点上定位并操作容器。 +所以，你可以从 Kubernetes 取得调度相关的元数据，比如 Pod 名称；从 Docker 取得容器状态信息。 +后来，人们开发了更多的运行时来管理容器。 +同时一些项目和 Kubernetes 特性也不断涌现，支持跨多个运行时收集容器状态信息。 + + +一些代理和 Docker 工具紧密绑定。此类代理可以这样运行命令，比如用 +[`docker ps`](https://docs.docker.com/engine/reference/commandline/ps/) +或 [`docker top`](https://docs.docker.com/engine/reference/commandline/top/) +这类命令来列出容器和进程，用 +[docker logs](https://docs.docker.com/engine/reference/commandline/logs/) +订阅 Docker 的日志。 +但随着 Docker 作为容器运行时被弃用，这些命令将不再工作。 + + +### 识别依赖于 Docker 的 DaemonSet {#identify-docker-dependency} + + +如果某 Pod 想调用运行在节点上的 `dockerd`，该 Pod 必须满足以下两个条件之一： + +- 将包含 Docker 守护进程特权套接字的文件系统挂载为一个{{< glossary_tooltip text="卷" term_id="volume" >}}；或 +- 直接以卷的形式挂载 Docker 守护进程特权套接字的特定路径。 + + +举例来说：在 COS 镜像中，Docker 通过 `/var/run/docker.sock` 开放其 Unix 域套接字。 +这意味着 Pod 的规约中需要包含 `hostPath` 卷以挂载 `/var/run/docker.sock`。 + + +下面是一个 shell 示例脚本，用于查找包含直接映射 Docker 套接字的挂载点的 Pod。 +你也可以删掉 grep `/var/run/docker.sock` 这一代码片段以查看其它挂载信息。 + +```bash +kubectl get pods --all-namespaces \ +-o=jsonpath='{range .items[*]}{"\n"}{.metadata.namespace}{":\t"}{.metadata.name}{":\t"}{range .spec.volumes[*]}{.hostPath.path}{", "}{end}{end}' \ +| sort \ +| grep '/var/run/docker.sock' +``` + + +{{< note >}} +对于 Pod 来说，访问宿主机上的 Docker 还有其他方式。 +例如，可以挂载 `/var/run` 的父目录而非其完整路径 +（就像[这个例子](https://gist.github.com/itaysk/7bc3e56d69c4d72a549286d98fd557dd)）。 +上述脚本只检测最常见的使用方式。 +{{< /note >}} + + +### 检测节点代理对 Docker 的依赖性 {#detecting-docker-dependency-from-node-agents} + + +在你的集群节点被定制、且在各个节点上均安装了额外的安全和遥测代理的场景下， +一定要和代理的供应商确认：该代理是否依赖于 Docker。 + + +### 遥测和安全代理的供应商 {#telemetry-and-security-agent-vendors} + + +我们通过 +[谷歌文档](https://docs.google.com/document/d/1ZFi4uKit63ga5sxEiZblfb-c23lFhvy6RXVPikS8wf0/edit#) +提供了为各类遥测和安全代理供应商准备的持续更新的迁移指导。 +请与供应商联系，获取从 dockershim 迁移的最新说明。 From 70ebc7f6ec73680507283873ef8f8094fcc77fc9 Mon Sep 17 00:00:00 2001 From: Yao Ding Date: Tue, 9 Feb 2021 08:11:02 -0500 Subject: [PATCH 177/190] Update ephemeral-containers.md --- content/en/docs/concepts/workloads/pods/ephemeral-containers.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/concepts/workloads/pods/ephemeral-containers.md b/content/en/docs/concepts/workloads/pods/ephemeral-containers.md index c999e5263b..011fa4adf4 100644 --- a/content/en/docs/concepts/workloads/pods/ephemeral-containers.md +++ b/content/en/docs/concepts/workloads/pods/ephemeral-containers.md @@ -103,7 +103,7 @@ the ephemeral container to add as an `EphemeralContainers` list: "apiVersion": "v1", "kind": "EphemeralContainers", "metadata": { - "name": "example-pod" + "name": "example-pod" }, "ephemeralContainers": [{ "command": [ From ee5b6e82471c5525f96cd9ba3ed1f60f40ba6f9b Mon Sep 17 00:00:00 2001 From: Jailton Lopes Date: Tue, 9 Feb 2021 13:20:06 -0300 Subject: [PATCH 178/190] Add content/pt/docs/tutorials/hello-minikube.md --- content/pt/docs/tutorials/hello-minikube.md | 258 ++++++++++++++++++++ 1 file changed, 258 insertions(+) create mode 100644 content/pt/docs/tutorials/hello-minikube.md diff --git a/content/pt/docs/tutorials/hello-minikube.md b/content/pt/docs/tutorials/hello-minikube.md new file mode 100644 index 0000000000..3f6238777b --- /dev/null +++ b/content/pt/docs/tutorials/hello-minikube.md @@ -0,0 +1,258 @@ +--- +title: Olá, Minikube! +content_type: tutorial +weight: 5 +menu: + main: + title: "Iniciar" + weight: 10 + post: > +

Pronto para meter a mão na massa? Vamos criar um cluster Kubernetes simples e executar uma aplicacão exemplo.

+card: + name: tutorials + weight: 10 +--- + + + +Este tutorial mostra como executar uma aplicação exemplo no Kubernetes utilizando o [Minikube](https://minikube.sigs.k8s.io) e o [Katacoda](https://www.katacoda.com). O Katacoda disponibiliza um ambiente Kubernetes gratuito e acessível via navegador. + +{{< note >}} +Você também consegue seguir os passos desse tutorial instalando o Minikube localmente. Para instruções de instalação, acesse: [iniciando com minikube](https://minikube.sigs.k8s.io/docs/start/). +{{< /note >}} + +## Objetivos + +* Instalar uma aplicação exemplo no minikube. +* Executar a aplicação. +* Visualizar os logs da aplicação. + +## Antes de você iniciar + +Este tutorial disponibiliza uma imagem de contêiner que utiliza o NGIX para retornar todas as requisições. + + + +## Criando um cluster do Minikube + +1. Clique no botão abaixo **para iniciar o terminal do Katacoda**. + + {{< kat-button >}} + +{{< note >}} +Se você instalou o Minikube localmente, execute: `minikube start`. +{{< /note >}} + +2. Abra o painel do Kubernetes em um navegador: + + ```shell + minikube dashboard + ``` + +3. Apenas no ambiente do Katacoda: Na parte superior do terminal, clique em **Preview Port 30000**. + +## Criando um Deployment + +Um [*Pod*](/docs/concepts/workloads/pods/) Kubernetes consiste em um ou mais contêineres agrupados para fins de administração e gerenciamento de rede. O Pod desse tutorial possui apenas um contêiner. Um [*Deployment*](/docs/concepts/workloads/controllers/deployment/) Kubernetes verifica a saúde do seu Pod e reinicia o contêiner do Pod caso o mesmo seja finalizado. Deployments são a maneira recomendada de gerenciar a criação e escalonamento dos Pods. + +1. Usando o comando `kubectl create` para criar um Deployment que gerencia um Pod. O Pod executa um contêiner baseado na imagem docker disponibilizada. + + ```shell + kubectl create deployment hello-node --image=k8s.gcr.io/echoserver:1.4 + ``` + +2. Visualizando o Deployment: + + ```shell + kubectl get deployments + ``` + + A saída será semelhante a: + + ``` + NAME READY UP-TO-DATE AVAILABLE AGE + hello-node 1/1 1 1 1m + ``` + +3. Visualizando o Pod: + + ```shell + kubectl get pods + ``` + + A saída será semelhante a: + + ``` + NAME READY STATUS RESTARTS AGE + hello-node-5f76cf6ccf-br9b5 1/1 Running 0 1m + ``` + +4. Visualizando os eventos do cluster: + + ```shell + kubectl get events + ``` + +5. Visualizando a configuração do `kubectl`: + + ```shell + kubectl config view + ``` + +{{< note >}} +Para mais informações sobre o comando `kubectl`, veja o [kubectl overview](/docs/reference/kubectl/overview/). +{{< /note >}} + +## Criando um serviço + +Por padrão, um Pod só é acessível utilizando o seu endereço IP interno no cluster Kubernetes. Para dispobiblilizar o contêiner `hello-node` fora da rede virtual do Kubernetes, você deve expor o Pod como um [*serviço*](/docs/concepts/services-networking/service/) Kubernetes. + +1. Expondo o Pod usando o comando `kubectl expose`: + + ```shell + kubectl expose deployment hello-node --type=LoadBalancer --port=8080 + ``` + + O parâmetro `--type=LoadBalancer` indica que você deseja expor o seu serviço fora do cluster Kubernetes. + + A aplicação dentro da imagem `k8s.gcr.io/echoserver` "escuta" apenas na porta TCP 8080. Se você usou + `kubectl expose` para expor uma porta diferente, os clientes não conseguirão se conectar a essa outra porta. + +2. Visualizando o serviço que você acabou de criar: + + ```shell + kubectl get services + ``` + + A saída será semelhante a: + + ``` + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE + hello-node LoadBalancer 10.108.144.78 8080:30369/TCP 21s + kubernetes ClusterIP 10.96.0.1 443/TCP 23m + ``` + + Em provedores de Cloud que fornecem serviços de balaceamento de carga para o Kubernetes, um IP externo seria provisionado para acessar o serviço. No Minikube, o tipo `LoadBalancer` torna o serviço acessível por meio do comando `minikube service`. + +3. Executar o comando a seguir: + + ```shell + minikube service hello-node + ``` + +4. (**Apenas no ambiente do Katacoda**) Clicar no sinal de mais e então clicar em **Select port to view on Host 1**. + +5. (**Apenas no ambiente do Katacoda**) Observe o número da porta com 5 dígitos exibido ao lado de `8080` na saída do serviço. Este número de porta é gerado aleatoriamente e pode ser diferente para você. Digite seu número na caixa de texto do número da porta e clique em **Display Port**. Usando o exemplo anterior, você digitaria `30369`. + +Isso abre uma janela do navegador, acessa o seu aplicativo e mostra o retorno da requisição. + +## Habilitando Complementos (addons) + +O Minikube inclue um conjunto integrado de {{< glossary_tooltip text="complementos" term_id="addons" >}} que podem ser habilitados, desabilitados e executados no ambiente Kubernetes local. + +1. Listando os complementos suportados atualmente: + + ```shell + minikube addons list + ``` + + A saída será semelhante a: + + ``` + addon-manager: enabled + dashboard: enabled + default-storageclass: enabled + efk: disabled + freshpod: disabled + gvisor: disabled + helm-tiller: disabled + ingress: disabled + ingress-dns: disabled + logviewer: disabled + metrics-server: disabled + nvidia-driver-installer: disabled + nvidia-gpu-device-plugin: disabled + registry: disabled + registry-creds: disabled + storage-provisioner: enabled + storage-provisioner-gluster: disabled + ``` + +2. Habilitando um complemento, por exemplo, `metrics-server`: + + ```shell + minikube addons enable metrics-server + ``` + + A saída será semelhante a: + + ``` + metrics-server was successfully enabled + ``` + +3. Visualizando os Pods e os Serviços que você acabou de criar: + + ```shell + kubectl get pod,svc -n kube-system + ``` + + A saída será semelhante a: + + ``` + NAME READY STATUS RESTARTS AGE + pod/coredns-5644d7b6d9-mh9ll 1/1 Running 0 34m + pod/coredns-5644d7b6d9-pqd2t 1/1 Running 0 34m + pod/metrics-server-67fb648c5 1/1 Running 0 26s + pod/etcd-minikube 1/1 Running 0 34m + pod/influxdb-grafana-b29w8 2/2 Running 0 26s + pod/kube-addon-manager-minikube 1/1 Running 0 34m + pod/kube-apiserver-minikube 1/1 Running 0 34m + pod/kube-controller-manager-minikube 1/1 Running 0 34m + pod/kube-proxy-rnlps 1/1 Running 0 34m + pod/kube-scheduler-minikube 1/1 Running 0 34m + pod/storage-provisioner 1/1 Running 0 34m + + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE + service/metrics-server ClusterIP 10.96.241.45 80/TCP 26s + service/kube-dns ClusterIP 10.96.0.10 53/UDP,53/TCP 34m + service/monitoring-grafana NodePort 10.99.24.54 80:30002/TCP 26s + service/monitoring-influxdb ClusterIP 10.111.169.94 8083/TCP,8086/TCP 26s + ``` + +4. Desabilitando o complemento `metrics-server`: + + ```shell + minikube addons disable metrics-server + ``` + + A saída será semelhante a: + + ``` + metrics-server was successfully disabled + ``` + +## Removendo os recursos do Minikube + +Agora você pode remover todos os recursos criados no seu cluster: + +```shell +kubectl delete service hello-node +kubectl delete deployment hello-node +``` +(**Opcional**) Pare a máquina virtual (VM) do Minikube: + +```shell +minikube stop +``` +(**Opcional**) Remova a VM do Minikube: + +```shell +minikube delete +``` + +## Próximos passos + +* Aprender mais sobre [Deployment objects](/docs/concepts/workloads/controllers/deployment/). +* Aprender mais sobre [Deploying applications](/docs/tasks/run-application/run-stateless-application-deployment/). +* Aprender mais sobre [Service objects](/docs/concepts/services-networking/service/). + From 52f35431d9c6d5e8bd16a09e8052632f432fc6ea Mon Sep 17 00:00:00 2001 From: Bin Chen Date: Wed, 10 Feb 2021 16:41:32 +1100 Subject: [PATCH 179/190] service-network: add some clarity to headless service --- content/en/docs/concepts/services-networking/service.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/concepts/services-networking/service.md b/content/en/docs/concepts/services-networking/service.md index 83b7850364..e8578ac642 100644 --- a/content/en/docs/concepts/services-networking/service.md +++ b/content/en/docs/concepts/services-networking/service.md @@ -458,7 +458,7 @@ selectors defined: For headless Services that define selectors, the endpoints controller creates `Endpoints` records in the API, and modifies the DNS configuration to return -records (addresses) that point directly to the `Pods` backing the `Service`. +A records (IP addresses) that point directly to the `Pods` backing the `Service`. ### Without selectors From ddf8d77d7ae44a79d607d0c18a1f550134910513 Mon Sep 17 00:00:00 2001 From: Jack Henschel Date: Wed, 10 Feb 2021 10:39:32 +0100 Subject: [PATCH 180/190] Fix column alignment in debug-service.md Previously, the columns showing one of kubectl's output were misaligned and thereby confusing. --- .../en/docs/tasks/debug-application-cluster/debug-service.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/tasks/debug-application-cluster/debug-service.md b/content/en/docs/tasks/debug-application-cluster/debug-service.md index 9025375955..3613e5b2cb 100644 --- a/content/en/docs/tasks/debug-application-cluster/debug-service.md +++ b/content/en/docs/tasks/debug-application-cluster/debug-service.md @@ -421,7 +421,7 @@ Earlier you saw that the Pods were running. You can re-check that: kubectl get pods -l app=hostnames ``` ```none -NAME READY STATUS RESTARTS AGE +NAME READY STATUS RESTARTS AGE hostnames-632524106-bbpiw 1/1 Running 0 1h hostnames-632524106-ly40y 1/1 Running 0 1h hostnames-632524106-tlaok 1/1 Running 0 1h From a1f5566dc0f13aeac7da8ca9a42eeeb6368aeb80 Mon Sep 17 00:00:00 2001 From: Naoki Oketani Date: Wed, 10 Feb 2021 21:33:19 +0900 Subject: [PATCH 181/190] remove command from output block --- .../run-application/run-stateless-application-deployment.md | 1 - 1 file changed, 1 deletion(-) diff --git a/content/en/docs/tasks/run-application/run-stateless-application-deployment.md b/content/en/docs/tasks/run-application/run-stateless-application-deployment.md index df604facf8..62bd984ddc 100644 --- a/content/en/docs/tasks/run-application/run-stateless-application-deployment.md +++ b/content/en/docs/tasks/run-application/run-stateless-application-deployment.md @@ -51,7 +51,6 @@ a Deployment that runs the nginx:1.14.2 Docker image: The output is similar to this: - user@computer:~/website$ kubectl describe deployment nginx-deployment Name: nginx-deployment Namespace: default CreationTimestamp: Tue, 30 Aug 2016 18:11:37 -0700 From 5d973fa5fd325a4c1d5f8822236f8f51c545f5a8 Mon Sep 17 00:00:00 2001 From: Jailton Lopes Date: Wed, 10 Feb 2021 10:13:46 -0300 Subject: [PATCH 182/190] Fix typos in file docs/tutorials/hello-minikube.md --- content/pt/docs/tutorials/hello-minikube.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/content/pt/docs/tutorials/hello-minikube.md b/content/pt/docs/tutorials/hello-minikube.md index 3f6238777b..e3f5ebbc2f 100644 --- a/content/pt/docs/tutorials/hello-minikube.md +++ b/content/pt/docs/tutorials/hello-minikube.md @@ -29,7 +29,7 @@ Você também consegue seguir os passos desse tutorial instalando o Minikube loc ## Antes de você iniciar -Este tutorial disponibiliza uma imagem de contêiner que utiliza o NGIX para retornar todas as requisições. +Este tutorial disponibiliza uma imagem de contêiner que utiliza o NGINX para retornar todas as requisições. @@ -132,7 +132,7 @@ Por padrão, um Pod só é acessível utilizando o seu endereço IP interno no c kubernetes ClusterIP 10.96.0.1 443/TCP 23m ``` - Em provedores de Cloud que fornecem serviços de balaceamento de carga para o Kubernetes, um IP externo seria provisionado para acessar o serviço. No Minikube, o tipo `LoadBalancer` torna o serviço acessível por meio do comando `minikube service`. + Em provedores de Cloud que fornecem serviços de balanceamento de carga para o Kubernetes, um IP externo seria provisionado para acessar o serviço. No Minikube, o tipo `LoadBalancer` torna o serviço acessível por meio do comando `minikube service`. 3. Executar o comando a seguir: @@ -148,7 +148,7 @@ Isso abre uma janela do navegador, acessa o seu aplicativo e mostra o retorno da ## Habilitando Complementos (addons) -O Minikube inclue um conjunto integrado de {{< glossary_tooltip text="complementos" term_id="addons" >}} que podem ser habilitados, desabilitados e executados no ambiente Kubernetes local. +O Minikube inclui um conjunto integrado de {{< glossary_tooltip text="complementos" term_id="addons" >}} que podem ser habilitados, desabilitados e executados no ambiente Kubernetes local. 1. Listando os complementos suportados atualmente: From dc4836b6263df9723961509f123c8cf373c4cdea Mon Sep 17 00:00:00 2001 From: eTimS Date: Wed, 10 Feb 2021 15:22:48 +0100 Subject: [PATCH 183/190] Minor - Systemd enable command to enable CRI-O at boot time Added "sudo systemctl enable crio enable" to configure systemd to enable CRI-O at boot time, as shown in the CRI-O install.md doc: https://github.com/cri-o/cri-o/blob/master/install.md#user-content-starting-cri-o --- .../en/docs/setup/production-environment/container-runtimes.md | 1 + 1 file changed, 1 insertion(+) diff --git a/content/en/docs/setup/production-environment/container-runtimes.md b/content/en/docs/setup/production-environment/container-runtimes.md index 59725188d8..f56fc4ede2 100644 --- a/content/en/docs/setup/production-environment/container-runtimes.md +++ b/content/en/docs/setup/production-environment/container-runtimes.md @@ -420,6 +420,7 @@ Start CRI-O: ```shell sudo systemctl daemon-reload +sudo systemctl enable crio sudo systemctl start crio ``` From f26aab795e55da19f6342b0b1c49e4fdcb5ce148 Mon Sep 17 00:00:00 2001 From: Jailton Lopes Date: Wed, 10 Feb 2021 17:13:31 -0300 Subject: [PATCH 184/190] Fix typo in file docs/tutorials/hello-minikube.md --- content/pt/docs/tutorials/hello-minikube.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/pt/docs/tutorials/hello-minikube.md b/content/pt/docs/tutorials/hello-minikube.md index e3f5ebbc2f..0db5d20ddc 100644 --- a/content/pt/docs/tutorials/hello-minikube.md +++ b/content/pt/docs/tutorials/hello-minikube.md @@ -7,7 +7,7 @@ menu: title: "Iniciar" weight: 10 post: > -

Pronto para meter a mão na massa? Vamos criar um cluster Kubernetes simples e executar uma aplicacão exemplo.

+

Pronto para meter a mão na massa? Vamos criar um cluster Kubernetes simples e executar uma aplicação exemplo.

card: name: tutorials weight: 10 From f4a96f4e7cd7143d81a14a2b3e79bbdc95c95f1a Mon Sep 17 00:00:00 2001 From: eTimS Date: Thu, 11 Feb 2021 00:27:23 +0100 Subject: [PATCH 185/190] Update content/en/docs/setup/production-environment/container-runtimes.md Co-authored-by: Tim Bannister --- .../en/docs/setup/production-environment/container-runtimes.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/content/en/docs/setup/production-environment/container-runtimes.md b/content/en/docs/setup/production-environment/container-runtimes.md index f56fc4ede2..5aa4ac894e 100644 --- a/content/en/docs/setup/production-environment/container-runtimes.md +++ b/content/en/docs/setup/production-environment/container-runtimes.md @@ -420,8 +420,7 @@ Start CRI-O: ```shell sudo systemctl daemon-reload -sudo systemctl enable crio -sudo systemctl start crio +sudo systemctl enable crio --now ``` Refer to the [CRI-O installation guide](https://github.com/cri-o/cri-o/blob/master/install.md) From 9a3347cd2d7c1019d04a99d84d2ae7e86c1875a2 Mon Sep 17 00:00:00 2001 From: Bin Chen Date: Thu, 11 Feb 2021 10:40:49 +1100 Subject: [PATCH 186/190] security: add container runtime class as an option to provider extra container security --- content/en/docs/concepts/security/overview.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/content/en/docs/concepts/security/overview.md b/content/en/docs/concepts/security/overview.md index fe9129c109..b23a07c79a 100644 --- a/content/en/docs/concepts/security/overview.md +++ b/content/en/docs/concepts/security/overview.md @@ -120,6 +120,7 @@ Area of Concern for Containers | Recommendation | Container Vulnerability Scanning and OS Dependency Security | As part of an image build step, you should scan your containers for known vulnerabilities. Image Signing and Enforcement | Sign container images to maintain a system of trust for the content of your containers. Disallow privileged users | When constructing containers, consult your documentation for how to create users inside of the containers that have the least level of operating system privilege necessary in order to carry out the goal of the container. +Use container runtime with stronger isolation | Select [container runtime classes](/docs/concepts/containers/runtime-class/) that provider stronger isolation ## Code @@ -152,3 +153,4 @@ Learn about related Kubernetes security topics: * [Data encryption in transit](/docs/tasks/tls/managing-tls-in-a-cluster/) for the control plane * [Data encryption at rest](/docs/tasks/administer-cluster/encrypt-data/) * [Secrets in Kubernetes](/docs/concepts/configuration/secret/) +* [Runtime class](/docs/concepts/containers/runtime-class) From a33e903e3a1f436d3ca7dc6ae75e9da5cf8d1d2c Mon Sep 17 00:00:00 2001 From: Jack Henschel Date: Wed, 10 Feb 2021 12:23:04 +0100 Subject: [PATCH 187/190] Clarify selection of default HPA scaling policy Without first mentioning where the number 40 is coming from, it was a bit unclear why 40 would be the threshold value for policy selection in this case. I hope this patch clarifies it. --- .../run-application/horizontal-pod-autoscale.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/content/en/docs/tasks/run-application/horizontal-pod-autoscale.md b/content/en/docs/tasks/run-application/horizontal-pod-autoscale.md index 56a48c8b30..763d9ab996 100644 --- a/content/en/docs/tasks/run-application/horizontal-pod-autoscale.md +++ b/content/en/docs/tasks/run-application/horizontal-pod-autoscale.md @@ -383,7 +383,12 @@ behavior: periodSeconds: 60 ``` -When the number of pods is more than 40 the second policy will be used for scaling down. +`periodSeconds` indicates the length of time in the past for which the policy must hold true. +The first policy _(Pods)_ allows at most 4 replicas to be scaled down in one minute. The second policy +_(Percent)_ allows at most 10% of the current replicas to be scaled down in one minute. + +Since by default the policy which allows the highest amount of change is selected, the second policy will +only be used when the number of pod replicas is more than 40. With 40 or less replicas, the first policy will be applied. For instance if there are 80 replicas and the target has to be scaled down to 10 replicas then during the first step 8 replicas will be reduced. In the next iteration when the number of replicas is 72, 10% of the pods is 7.2 but the number is rounded up to 8. On each loop of @@ -391,10 +396,6 @@ the autoscaler controller the number of pods to be change is re-calculated based of current replicas. When the number of replicas falls below 40 the first policy _(Pods)_ is applied and 4 replicas will be reduced at a time. -`periodSeconds` indicates the length of time in the past for which the policy must hold true. -The first policy allows at most 4 replicas to be scaled down in one minute. The second policy -allows at most 10% of the current replicas to be scaled down in one minute. - The policy selection can be changed by specifying the `selectPolicy` field for a scaling direction. By setting the value to `Min` which would select the policy which allows the smallest change in the replica count. Setting the value to `Disabled` completely disables @@ -441,7 +442,7 @@ behavior: periodSeconds: 15 selectPolicy: Max ``` -For scaling down the stabilization window is _300_ seconds(or the value of the +For scaling down the stabilization window is _300_ seconds (or the value of the `--horizontal-pod-autoscaler-downscale-stabilization` flag if provided). There is only a single policy for scaling down which allows a 100% of the currently running replicas to be removed which means the scaling target can be scaled down to the minimum allowed replicas. From e0e68de173d017c89e6c74c5423e7d3d4f9198e5 Mon Sep 17 00:00:00 2001 From: Naoki Oketani Date: Thu, 11 Feb 2021 18:29:01 +0900 Subject: [PATCH 188/190] style: separate commands from output --- .../run-single-instance-stateful-application.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/content/en/docs/tasks/run-application/run-single-instance-stateful-application.md b/content/en/docs/tasks/run-application/run-single-instance-stateful-application.md index 4c43948a21..bdc3b0c524 100644 --- a/content/en/docs/tasks/run-application/run-single-instance-stateful-application.md +++ b/content/en/docs/tasks/run-application/run-single-instance-stateful-application.md @@ -65,6 +65,8 @@ for a secure solution. kubectl describe deployment mysql + The output is similar to this: + Name: mysql Namespace: default CreationTimestamp: Tue, 01 Nov 2016 11:18:45 -0700 @@ -105,6 +107,8 @@ for a secure solution. kubectl get pods -l app=mysql + The output is similar to this: + NAME READY STATUS RESTARTS AGE mysql-63082529-2z3ki 1/1 Running 0 3m @@ -112,6 +116,8 @@ for a secure solution. kubectl describe pvc mysql-pv-claim + The output is similar to this: + Name: mysql-pv-claim Namespace: default StorageClass: From e13caad951ac48d665fa3b8cd8fa177c8d72e0f1 Mon Sep 17 00:00:00 2001 From: Naoki Oketani Date: Thu, 11 Feb 2021 20:04:59 +0900 Subject: [PATCH 189/190] fix an invalid location.hash --- .../run-application/run-replicated-stateful-application.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/tasks/run-application/run-replicated-stateful-application.md b/content/en/docs/tasks/run-application/run-replicated-stateful-application.md index 79d779f0df..36e5334f3d 100644 --- a/content/en/docs/tasks/run-application/run-replicated-stateful-application.md +++ b/content/en/docs/tasks/run-application/run-replicated-stateful-application.md @@ -174,7 +174,7 @@ Because the example topology consists of a single primary MySQL server and any n replicas, the script assigns ordinal `0` to be the primary server, and everyone else to be replicas. Combined with the StatefulSet controller's -[deployment order guarantee](/docs/concepts/workloads/controllers/statefulset/#deployment-and-scaling-guarantees/), +[deployment order guarantee](/docs/concepts/workloads/controllers/statefulset/#deployment-and-scaling-guarantees), this ensures the primary MySQL server is Ready before creating replicas, so they can begin replicating. From f141a3c4146a9177a3782bb0341d114e12dd3a60 Mon Sep 17 00:00:00 2001 From: Ricardo Castro Date: Thu, 11 Feb 2021 13:16:12 +0000 Subject: [PATCH 190/190] fix: add missing end --- .../2019-03-15-Kubernetes-setup-using-Ansible-and-Vagrant.md | 1 + 1 file changed, 1 insertion(+) diff --git a/content/en/blog/_posts/2019-03-15-Kubernetes-setup-using-Ansible-and-Vagrant.md b/content/en/blog/_posts/2019-03-15-Kubernetes-setup-using-Ansible-and-Vagrant.md index 8b31d1df0b..247748b6f0 100644 --- a/content/en/blog/_posts/2019-03-15-Kubernetes-setup-using-Ansible-and-Vagrant.md +++ b/content/en/blog/_posts/2019-03-15-Kubernetes-setup-using-Ansible-and-Vagrant.md @@ -66,6 +66,7 @@ Vagrant.configure("2") do |config| end end end +end ``` ### Step 2: Create an Ansible playbook for Kubernetes master.